From clp2 at rebertia.com Fri Jan 1 00:24:56 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Thu, 31 Dec 2009 21:24:56 -0800 Subject: Exception as the primary error handling mechanism? In-Reply-To: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <50697b2c0912312124n7f989568t13e5a1dd8d883e9c@mail.gmail.com> On Thu, Dec 31, 2009 at 8:47 PM, Peng Yu wrote: > I observe that python library primarily use exception for error > handling rather than use error code. > > In the article API Design Matters by Michi Henning > > Communications of the ACM > Vol. 52 No. 5, Pages 46-56 > 10.1145/1506409.1506424 > http://cacm.acm.org/magazines/2009/5/24646-api-design-matters/fulltext > > It says "Another popular design flaw?namely, throwing exceptions for > expected outcomes?also causes inefficiencies because catching and > handling exceptions is almost always slower than testing a return > value." > > My observation is contradicted to the above statement by Henning. If > my observation is wrong, please just ignore my question below. > > Otherwise, could some python expert explain to me why exception is > widely used for error handling in python? Is it because the efficiency > is not the primary goal of python? Correct; programmer efficiency is a more important goal for Python instead. Python is ~60-100x slower than C;[1] if someone is worried by the inefficiency caused by exceptions, then they're using completely the wrong language. Cheers, Chris -- http://blog.rebertia.com [1] http://shootout.alioth.debian.org/u64/which-programming-languages-are-fastest.php?gcc=on&python=on&calc=chart From benjamin.kaplan at case.edu Fri Jan 1 00:26:09 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Fri, 1 Jan 2010 00:26:09 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: On Thu, Dec 31, 2009 at 11:47 PM, Peng Yu wrote: > I observe that python library primarily use exception for error > handling rather than use error code. > > In the article API Design Matters by Michi Henning > > Communications of the ACM > Vol. 52 No. 5, Pages 46-56 > 10.1145/1506409.1506424 > http://cacm.acm.org/magazines/2009/5/24646-api-design-matters/fulltext > > It says "Another popular design flaw?namely, throwing exceptions for > expected outcomes?also causes inefficiencies because catching and > handling exceptions is almost always slower than testing a return > value." > > My observation is contradicted to the above statement by Henning. If > my observation is wrong, please just ignore my question below. > > Otherwise, could some python expert explain to me why exception is > widely used for error handling in python? Is it because the efficiency > is not the primary goal of python? > -- > http://mail.python.org/mailman/listinfo/python-list > Read the quote again "Another popular design flaw?namely, throwing exceptions *for expected outcomes*" In Python, throwing exceptions for expected outcomes is considered very bad form (well, except for StopIteration but that should almost never be handled directly by the programmer). To answer why people recommend using "Easier to Ask Forgiveness than Permission" as opposed to "Look Before You Leap" : Because of the way it's implemented, Python works quite differently from most languages. An attribute look-up is rather expensive because it's a hash table look-up at run time. Wrapping a small piece of code in a try block however, isn't (relatively) very expensive at all in Python. It's only catching the exception that's expensive, but if you're catching the exception, something has gone wrong anyway and performance isn't your biggest issue. From wolftracks at invalid.com Fri Jan 1 00:49:28 2010 From: wolftracks at invalid.com (W. eWatson) Date: Thu, 31 Dec 2009 21:49:28 -0800 Subject: mod, modulo and % under 2.4 and 2.5 In-Reply-To: <87iqbm1qwz.fsf@benfinney.id.au> References: <034d421c$0$1277$c3e8da3@news.astraweb.com> <87iqbm1qwz.fsf@benfinney.id.au> Message-ID: Ben Finney wrote: > "W. eWatson" writes: > >> Steven D'Aprano wrote: >>> NameError: name 'mod' is not defined > >> So where is it? Here are the choices. >> import sys, os, glob >> import string >> from numpy import * > > If you use ?from foo import *? you forfeit any way of saying where a > name in your code gets bound. > > Hence, don't do that. > Good idea! From yoavglazner at gmail.com Fri Jan 1 02:26:19 2010 From: yoavglazner at gmail.com (Glazner) Date: Thu, 31 Dec 2009 23:26:19 -0800 (PST) Subject: multiprocessing module References: <79854c42-b2af-4adb-8967-3dc5e4ac0d2a@l13g2000yqb.googlegroups.com> Message-ID: <57c4f9dd-7a01-439a-b538-d604df908345@k23g2000yqa.googlegroups.com> On Dec 15 2009, 10:56?am, makobu wrote: > I have a function that makes two subprocess.Popen() calls on a file. > > I have 8 cores. I need 8 instances of that function running in > parallel at any given time till all the files are worked on. > Can the multiprocessing module do this? If so, whats the best method? > > A technical overview of how the multiprocessing module actually works > would also be really helpful. > > regards, > mak. I guess you need the Map-Reduce pattern. It appears like multiprocessing.Pool will do the trick. def doSomething(f): pass pool = Pool(8)# 8 processes 1 per core files = ['pop.txt','looper.txt','foo.bar'] results = pool.map(doSomething,files) #this does all the job got it? From victorsubervi at gmail.com Fri Jan 1 02:47:02 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 1 Jan 2010 02:47:02 -0500 Subject: Not Incrementing In-Reply-To: <4B3D0DB3.3000101@mrabarnett.plus.com> References: <4dc0cfea0912310851u6f61a93frad30042c9bb1b96a@mail.gmail.com> <4B3CE70D.5020109@ieee.org> <4dc0cfea0912311137w45f00e69h3625d83d7166d12e@mail.gmail.com> <4B3D02FC.8090403@mrabarnett.plus.com> <4dc0cfea0912311206q23fc3f3dh7b783916543ff744@mail.gmail.com> <4B3D0DB3.3000101@mrabarnett.plus.com> Message-ID: <4dc0cfea0912312347i40a29719x1f6f79c398aa022f@mail.gmail.com> On Thu, Dec 31, 2009 at 3:46 PM, MRAB wrote: > Victor Subervi wrote: > >> On Thu, Dec 31, 2009 at 3:01 PM, MRAB > python at mrabarnett.plus.com>> wrote: >> >> Victor Subervi wrote: >> >> On Thu, Dec 31, 2009 at 1:01 PM, Dave Angel > > >> >> wrote: >> >> [snip] >> >> >> Incidentally, good naming would help to make the code easier to >> debug. This finalTrees and trees variables don't look to me >> like >> you intended them to be trees at all, but lists of >> tab-indented data. >> >> No kidding. Many people have commented on my poor choices of >> names. That's one of the things on my agenda to change (with >> some tools that I understand are out there) once I get this >> program finished and working for clients. >> >> Your clients will get it _with_ its faults? You'll fix it with some >> tools that you _understand_ are out there, ie you haven't looked >> for/found them yet? Yikes! :-) >> >> >> How many espressos do you drink a day? Do you mainline the stuff? Relax, >> dude. Rome wasn't made in a day. I'll get there. Chill out, dude. >> >> Programming is about attention to detail. It's not like poetry, with its > "poetic licence". Has a mistake in poetry ever caused a rocket to fail > (Ariane 5, flight 501)? :-) Heavens no! And thank goodness! Yes, as I like to say, "The devil is in the details...and well employed in software development!" LOL! beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Fri Jan 1 02:48:12 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 1 Jan 2010 02:48:12 -0500 Subject: Cookies In-Reply-To: <4dc0cfea0912310829n3028c740w167978dc3bc637ae@mail.gmail.com> References: <4dc0cfea0912291142u3712fb5axe8eee6acc781640e@mail.gmail.com> <4dc0cfea0912300727uf4743b1s72e7bdd0f044341b@mail.gmail.com> <4dc0cfea0912300826td1a0a8eo14c16584270f36ba@mail.gmail.com> <4dc0cfea0912300919k23a71fb8td4233d6c5653b346@mail.gmail.com> <4dc0cfea0912301936q88a4996t810fd88818a85e3e@mail.gmail.com> <4dc0cfea0912310829n3028c740w167978dc3bc637ae@mail.gmail.com> Message-ID: <4dc0cfea0912312348i35b90b37u3777ddba1f7aa710@mail.gmail.com> I'm curious. Are there other instances where code needs to be inserted into the header as in the print cookie to get it to be baked on the user's PC? TIA, beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at REMOVE-THIS-cybersource.com.au Fri Jan 1 03:26:16 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 01 Jan 2010 08:26:16 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <034d9ead$0$1277$c3e8da3@news.astraweb.com> On Thu, 31 Dec 2009 20:47:49 -0800, Peng Yu wrote: > I observe that python library primarily use exception for error handling > rather than use error code. > > In the article API Design Matters by Michi Henning > > Communications of the ACM > Vol. 52 No. 5, Pages 46-56 > 10.1145/1506409.1506424 > http://cacm.acm.org/magazines/2009/5/24646-api-design-matters/fulltext > > It says "Another popular design flaw?namely, throwing exceptions for > expected outcomes?also causes inefficiencies because catching and > handling exceptions is almost always slower than testing a return > value." This is very, very wrong. Firstly, notice that the author doesn't compare the same thing. He compares "catching AND HANDLING" the exception (emphasis added) with *only* testing a return value. Of course it is faster to test a value and do nothing, than it is to catch an exception and then handle the exception. That's an unfair comparison, and that alone shows that the author is biased against exceptions. But it's also wrong. If you call a function one million times, and catch an exception ONCE (because exceptions are rare) that is likely to be much, much faster than testing a return code one million times. Before you can talk about which strategy is faster, you need to understand your problem. When exceptions are rare (in CPython, about one in ten or rarer) then try...except is faster than testing each time. The exact cut-off depends on how expensive the test is, and how much work gets done before the exception is raised. Using exceptions is only slow if they are common. But the most important reason for preferring exceptions is that the alternatives are error-prone! Testing error codes is the anti-pattern, not catching exceptions. See, for example: http://c2.com/cgi/wiki?UseExceptionsInsteadOfErrorValues http://c2.com/cgi/wiki?ExceptionsAreOurFriends http://c2.com/cgi/wiki?AvoidExceptionsWheneverPossible Despite the title of that last page, it has many excellent arguments for why exceptions are better than the alternatives. (Be warned: the c2 wiki is filled with Java and C++ programmers who mistake the work-arounds for quirks of their language as general design principles. For example, because exceptions in Java are evcen more expensive and slow than in Python, you will find lots of Java coders saying "don't use exceptions" instead of "don't use exceptions IN JAVA".) There are many problems with using error codes: * They complicate your code. Instead of returning the result you care about, you have to return a status code and the return result you care about. Even worse is to have a single global variable to hold the status of the last function call! * Nobody can agree whether the status code means the function call failed, or the function call succeeded. * If the function call failed, what do you return as the result code? * You can't be sure that the caller will remember to check the status code. In fact, you can be sure that the caller WILL forget sometimes! (This is human nature.) This leads to the frequent problem that by the time a caller checks the status code, the original error has been lost and the program is working with garbage. * Even if you remember to check the status code, it complicates the code, makes it less readable, confuses the intent of the code, and often leads to the Arrow Anti-pattern: http://c2.com/cgi/wiki?ArrowAntiPattern That last argument is critical. Exceptions exist to make writing correct code easier to write, understand and maintain. Python uses special result codes in at least two places: str.find(s) returns -1 if s is not in the string re.match() returns None is the regular expression fails Both of these are error-prone. Consider a naive way of getting the fractional part of a float string: >>> s = "234.567" >>> print s[s.find('.')+1:] 567 But see: >>> s = "234" >>> print s[s.find('.')+1:] 234 You need something like: p = s.find('.') if p == -1: print '' else: print s[p+1:] Similarly, we cannot safely do this in Python: >>> re.match(r'\d+', '123abcd').group() '123' >>> re.match(r'\d+', 'abcd').group() Traceback (most recent call last): File "", line 1, in AttributeError: 'NoneType' object has no attribute 'group' You need to do this: mo = re.match(r'\d+', '123abcd') if mo is not None: # or just `if mo` will work mo.group() Exceptions are about making it easier to have correct code. They're also about making it easier to have readable code. Which is easier to read, easier to understand and easier to debug? x = function(1, 2, 3) if x != -1: y = function(x, 1, 2) if y != -1: z = function(y, x, 1) if z != -1: print "result is", z else: print "an error occurred" else: print "an error occurred" else: print "an error occurred" versus: try: x = function(1, 2, 3) y = function(x, 1, 2) print "result is", function(y, x, 1) except ValueError: print "an error occurred" In Python, setting up the try...except block is very fast, about as fast as a plain "pass" statement, but actually catching the exception is quite slow. So let's compare string.find (which returns an error result) and string.index (which raises an exception): >>> from timeit import Timer >>> setup = "source = 'abcd'*100 + 'e'" >>> min(Timer("p = source.index('e')", setup).repeat()) 1.1308379173278809 >>> min(Timer("p = source.find('e')", setup).repeat()) 1.2237567901611328 There's hardly any difference at all, and in fact index is slightly faster. But what about if there's an exceptional case? >>> min(Timer(""" ... try: ... p = source.index('z') ... except ValueError: ... pass ... """, setup).repeat()) 3.5699808597564697 >>> min(Timer(""" ... p = source.find('z') ... if p == -1: ... pass ... """, setup).repeat()) 1.7874350070953369 So in Python, catching the exception is slower, in this case about twice as slow. But remember that the "if p == -1" test is not free. It might be cheap, but it does take time. If you call find() enough times, and every single time you then test the result returned, that extra cost may be more expensive than catching a rare exception. The general rule in Python is: * if the exceptional event is rare (say, on average, less than about one time in ten) then use a try...except and catch the exception; * but if it is very common (more than one time in ten) then it is faster to do a test. > My observation is contradicted to the above statement by Henning. If my > observation is wrong, please just ignore my question below. > > Otherwise, could some python expert explain to me why exception is > widely used for error handling in python? Is it because the efficiency > is not the primary goal of python? Yes. Python's aim is to be fast *enough*, without necessarily being as fast as possible. Python aims to be readable, and to be easy to write correct, bug-free code. -- Steven From victorsubervi at gmail.com Fri Jan 1 03:30:57 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 1 Jan 2010 03:30:57 -0500 Subject: Not Incrementing In-Reply-To: <4dc0cfea0912312347i40a29719x1f6f79c398aa022f@mail.gmail.com> References: <4dc0cfea0912310851u6f61a93frad30042c9bb1b96a@mail.gmail.com> <4B3CE70D.5020109@ieee.org> <4dc0cfea0912311137w45f00e69h3625d83d7166d12e@mail.gmail.com> <4B3D02FC.8090403@mrabarnett.plus.com> <4dc0cfea0912311206q23fc3f3dh7b783916543ff744@mail.gmail.com> <4B3D0DB3.3000101@mrabarnett.plus.com> <4dc0cfea0912312347i40a29719x1f6f79c398aa022f@mail.gmail.com> Message-ID: <4dc0cfea1001010030g28bca209o4e578fa6ad83b79@mail.gmail.com> On Fri, Jan 1, 2010 at 2:47 AM, Victor Subervi wrote: > On Thu, Dec 31, 2009 at 3:46 PM, MRAB wrote: > >> Victor Subervi wrote: >> >>> On Thu, Dec 31, 2009 at 3:01 PM, MRAB >> python at mrabarnett.plus.com>> wrote: >>> >>> Victor Subervi wrote: >>> >>> On Thu, Dec 31, 2009 at 1:01 PM, Dave Angel >> >> >>> >> wrote: >>> >>> [snip] >>> >>> >>> Incidentally, good naming would help to make the code easier to >>> debug. This finalTrees and trees variables don't look to me >>> like >>> you intended them to be trees at all, but lists of >>> tab-indented data. >>> >>> No kidding. Many people have commented on my poor choices of >>> names. That's one of the things on my agenda to change (with >>> some tools that I understand are out there) once I get this >>> program finished and working for clients. >>> >>> Your clients will get it _with_ its faults? You'll fix it with some >>> tools that you _understand_ are out there, ie you haven't looked >>> for/found them yet? Yikes! :-) >>> >>> >>> How many espressos do you drink a day? Do you mainline the stuff? Relax, >>> dude. Rome wasn't made in a day. I'll get there. Chill out, dude. >>> >>> Programming is about attention to detail. It's not like poetry, with its >> "poetic licence". Has a mistake in poetry ever caused a rocket to fail >> (Ariane 5, flight 501)? :-) > > > Heavens no! And thank goodness! Yes, as I like to say, "The devil is in the > details...and well employed in software development!" LOL! > beno > PS Poetry is the art of writing what goes without saying. beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From aahz at pythoncraft.com Fri Jan 1 03:43:48 2010 From: aahz at pythoncraft.com (Aahz) Date: 1 Jan 2010 00:43:48 -0800 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: In article , Benjamin Kaplan wrote: > >In Python, throwing exceptions for expected outcomes is considered >very bad form [...] Who says that? I certainly don't. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From ijeiseditor at gmail.com Fri Jan 1 04:58:00 2010 From: ijeiseditor at gmail.com (ijeiseditor) Date: Fri, 1 Jan 2010 01:58:00 -0800 (PST) Subject: International Journal of Electronics, Information and Systems (IJEIS) Call for Paper Message-ID: <2fd974c7-973f-4285-8a30-038efe9757a8@k19g2000yqc.googlegroups.com> The International Journal of Electronics, Information and Systems (IJEIS) publish original papers on all subjects relevant to electronics, computer science, communication network, and information systems. The highest priority will be given to those contributions concerned with a discussion of the background of a practical problem, the establishment of an appropriate model, the determination of a solution, approximate or exact, analytical or numerical, and a discussion of the relevance of the results when applied to the real- life problem. Paper submissions are invited in the area of electronics and computer science, in particular the technological advances and research results in the fields of theoretical, experimental, and applied electronics, computer science and Information technology. Topics of interest include but are not limited to the following: Electronics: Analogue and digital circuit design, Microwave circuits and systems, Optoelectronic circuits, Semiconductor devices, Sensor technology, Transport in electronic materials, VLSI technology and device processing, Analogue Circuits and Signal Processing, Digital Circuits and Signal Processing, RF and Wireless Circuits & Systems, Biomedical Circuits & Systems, System Architectures and Applications, System Integration, SoC and Mixed-Signal disign, Design Automation of Electronics & Systems, Assembly and Packaging, Sensing and Sensor Networks, Photonic and Optoelectronic Circuits, Test and Reliability, Neural Network Circuits & Systems, Heterogeneous Structures, Advanced Technologies, Telecommunications and Multimedia, Electronic Materials, Electronic Devices, Integrated Electronic-Circuits, Optoelectronics &Quantum Electronics, Biomedical Engineering, Electronics Applications, Sensing Systems, Communication and Networks, Electronic Commerce? All topics related Electronics Computer Science: Parallel Processing and Distributed Computing, Foundations of High- performance Computing, Graph Theory and Analysis of Algorithms, Artificial Intelligences and Pattern/Image Recognitions, Neural Networks and Biomedical Simulations, Virtual Visions and Virtual Simulations, Data Mining, Web Image Mining and Applications, Data Base Management and Information Retrievals Adaptive Systems, Bifurcation, Biocybernetics, Bioinformatics, Bio-Inspired Networks, Blind Systems, Cellular Neural Networks, Chaos, Chaos Neural Networks, Circuit Simulation, Circuit Theory, Communication Systems, Complex Systems, Control Systems, Cryptosystems, Data Compression, Data Mining, Detection, Echo Canceller, Estimation, Evolutional Computation, Filter, Filter Bank, Fractal, Fuzzy Systems, Image Processing, Image Recognition, Internet Security, Medical Applications, Modeling, Optimization, Speech Processing, Speech Synthesis, Speech Recognition, Synchronization, Video Signal Processing, Watermarking, Wavelet Transform, All topics related Computer Science Communication Network: Ad-Hoc, Mobile, Wireless Networks and Mobile Computing, Quantum Computing, Coding, and Error Controls Agent Computing and Multi-Agents Systems, Defining Spectrum Rights and Open Spectrum Solutions, Quality of Services and Communication Protocols, Satellite and Optical Communication Systems, 3G/4G Network Evolutions, CDMA/GSM Communication Protocols, Mobile Computing for e- Commerce,Transmission/Switching/Distribution technologies,Communication Theory,Signal Processing for Communications,Wireless Communications,Wireless & Mobile Networking,Optical Networks and Systems,Next-Generation Networking and Internet,Communication QoS, Reliability and Modelling,Ad-hoc, Sensor and Mesh Networking,Multimedia Services, Communication Software and Services,Communication and Information System Security, System control, network/service management, Network and Internet protocols and standards,Client-server, distributed and Web-based communication systems, Broadband and multimedia systems and applications, Trials of advanced systems and services, Any topics related Communication Network Information and Systems: Cryptography and Foundation of Computer Security, Authentication/ Authorization Issues, IDS/Firewall, Anti-Spam mail, Anti-virus issues, Wireless Access Security, Network Security Applications, Biometric authentication and algorithms, Fingerprint /Hand/Biometrics Recognitions and Technologies, IC-card Security, OTP and Key Management Issues, E-commerce, Ubiquitous and RFID Applications, Metadata and Meta Modeling, XML and Data Management, Knowledge Management, Web Security and Privacy, Cyber Threats, Web Services and Web Engineering, Web Intelligence, Wireless Applications, Protocols and Standards, Network Systems, Proxies and Servers, Multimedia Applications using Web Services, Ontology and the Semantic Web, B2B, B2C and C2C, e-Business System Design and Development, e-Payment, Portal Strategies, Social Networks and Information Systems, Social and Legal Issues, Digital Ecology, e-Governance, e-Learning and Virtual Classrooms, e-Entertainment, e-Journalism, Any topics related Information systems International Journal of Electronics, Information and Systems (IJEIS) ISSN: 1884-7609 Website: https://sites.google.com/site/ijeishp2/ Manuscript submission to: ijeiseditor at gmail.com All submitted papers will be judged based on their quality by the technical committee and reviewers. Papers that describe research and experimentation are encouraged. All paper submissions will be handled electronically and detailed instructions on submission procedure are available on IJEIS web pages (https://sites.google.com/site/ ijeishp2/). Researchers and authors are invited to participate in the peer-review process of IJEIS papers if your research interest matches with the themes of Call for Papers. For other information, please contact IJEIS Managing Editor, (ijeiseditor at gmail.com) From martin at v.loewis.de Fri Jan 1 05:34:19 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Fri, 01 Jan 2010 11:34:19 +0100 Subject: Exception as the primary error handling mechanism? In-Reply-To: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <4B3DCFAB.3030909@v.loewis.de> Peng Yu wrote: > I observe that python library primarily use exception for error > handling rather than use error code. [...] > It says "Another popular design flaw?namely, throwing exceptions for > expected outcomes?also causes inefficiencies because catching and > handling exceptions is almost always slower than testing a return > value." > > My observation is contradicted to the above statement by Henning. If > my observation is wrong, please just ignore my question below. Your observation is not wrong, but, as Benjamin already explained, you are misinterpreting Michi Henning's statement. He doesn't condemn exception handling per se, but only for the handling of *expected* outcomes. He would consider using exceptions fine for *exceptional* output, and that is exactly the way they are used in the Python API. Notice that in cases where the failure may be expected, Python also offers variants that avoid the exception: - if you look into a dictionary, expecting that a key may not be there, a regular access, d[k], may give a KeyError. However, alternatively, you can use d.get(k, default) which raises no exception, and you can test "k in d" in advance. - if you open a file, not knowing whether it will be there, you get an IOError. However, you can use os.path.exists in advance to determine whether the file is present (and create it if it's not). So, in these cases, it is a choice of the user to determine whether the error case is exceptional or not. Regards, Martin From jgardner at jonathangardner.net Fri Jan 1 05:43:21 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Fri, 1 Jan 2010 02:43:21 -0800 (PST) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <2109bc4b-834c-45a1-8287-57936c60a7bf@s31g2000yqs.googlegroups.com> On Jan 1, 12:43?am, a... at pythoncraft.com (Aahz) wrote: > In article , > Benjamin Kaplan ? wrote: > >In Python, throwing exceptions for expected outcomes is considered > >very bad form [...] > > Who says that? ?I certainly don't. Agreed. int("asdf") is supposed to return what, exactly? Any language that tries to return an int is horribly broken. From jpeyret at gmail.com Fri Jan 1 06:25:13 2010 From: jpeyret at gmail.com (J Peyret) Date: Fri, 1 Jan 2010 03:25:13 -0800 (PST) Subject: twenty years ago Guido created Python References: Message-ID: On Dec 31 2009, 2:06?pm, Steve Howell wrote: > FYI: > > http://twitter.com/gvanrossum > > Python is a truly awesome programming language. ?Not only is Guido a > genius language designer, but he is also a great project leader. ?What > an accomplishment. ?Congratulations to everybody who has contributed > to Python in the last two decades! Notwithstanding all of the above, which are all true, having met Guido, I say he is a genuinely nice human being. Go BSD derivatives. From simon at brunningonline.net Fri Jan 1 06:36:32 2010 From: simon at brunningonline.net (Simon Brunning) Date: Fri, 1 Jan 2010 11:36:32 +0000 Subject: pywinauto to show the dialog , menu, etc In-Reply-To: References: Message-ID: <8c7f10c61001010336s19c6f016l3d7c83554a282ad9@mail.gmail.com> 2009/12/31 Hari : > Hi > I am using pywinauto to automate an custom program to startup and load > process , execute etc. But cannot determine menuselect. Is there a way or > tool which can run against the exe to show the menu, dialog box, list box > which are contained within it. Winspector might be worth a try: http://www.windows-spy.com/ -- Cheers, Simon B. From hackingkk at gmail.com Fri Jan 1 07:04:44 2010 From: hackingkk at gmail.com (Krishnakant) Date: Fri, 01 Jan 2010 17:34:44 +0530 Subject: twenty years ago Guido created Python In-Reply-To: References: Message-ID: <1262347484.3458.3.camel@krishna-laptop> On Fri, 2010-01-01 at 03:25 -0800, J Peyret wrote: > On Dec 31 2009, 2:06 pm, Steve Howell wrote: > > FYI: > > > > http://twitter.com/gvanrossum > > > > Python is a truly awesome programming language. Not only is Guido a > > genius language designer, but he is also a great project leader. What > > an accomplishment. Congratulations to everybody who has contributed > > to Python in the last two decades! > > Notwithstanding all of the above, which are all true, having met > Guido, I say he is a genuinely nice human being. Go BSD derivatives. Indeed, python is a great programming language. May be it was not marketted as much as languages like java or not as much as platforms like rails. But who cares? I must mention here that I am totally blind and the talking software (screen reader ) I use on GNU/Linux called Orca is also programmed using python. And I make web applications and off late started to use pylons ( again centered around python). I see that python is for 2 kind of programmers. one which are absolute beginners to programming itself who want to learn the scientific art of programming and the second is the extreme experts who need the power and performance nicely balanced. Great work! happy hfacking. Krishnakant. From icanbob at gmail.com Fri Jan 1 07:39:47 2010 From: icanbob at gmail.com (bobicanprogram) Date: Fri, 1 Jan 2010 04:39:47 -0800 (PST) Subject: adding python engine into c++ application References: <3451d586-11b6-4183-88a6-d353bac44d31@q2g2000yqd.googlegroups.com> Message-ID: <9f36654c-1b8a-47e8-a12d-4c86ebf5f257@e27g2000yqd.googlegroups.com> On Dec 29 2009, 6:25 am, griwes wrote: > Hello, I am going to write an application in C++, application which > should be easy to extend by scripts. I chose python for script > language in this application. I'd like to have own API for python. And > here the question arises: how do I implement a python engine within an > application written in C++? > > Sory for any mistakes, english isn't my native language ;) If you want to keep your C++ and Python as separate but interacting modules you might want to check out the SIMPL project's (http:// www.icanprogram.com/SIMPL) Python hooks. There is some tutorial level code here: http://www.icanprogram.com/06py/main.html bob From lie.1296 at gmail.com Fri Jan 1 08:03:32 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Sat, 02 Jan 2010 00:03:32 +1100 Subject: multivariable assignment In-Reply-To: References: Message-ID: <4b3df2a3$1@dnews.tpgi.com.au> On 1/1/2010 3:13 AM, davidj411 wrote: > I am not sure why this behavior is this way. > at beginning of script, i want to create a bunch of empty lists and > use each one for its own purpose. > however, updating one list seems to update the others. > >>>> a = b = c = [] >>>> a.append('1') >>>> a.append('1') >>>> a.append('1') >>>> c > ['1', '1', '1'] >>>> a > ['1', '1', '1'] >>>> b > ['1', '1', '1'] Every time people get confused because of how python object model works, I always refer them to this article: http://effbot.org/zone/call-by-object.htm From wolfram.hinderer at googlemail.com Fri Jan 1 08:19:02 2010 From: wolfram.hinderer at googlemail.com (Wolfram Hinderer) Date: Fri, 1 Jan 2010 05:19:02 -0800 (PST) Subject: Dangerous behavior of list(generator) References: <7p356kF750U1@mid.individual.net> <034bd6f1$0$1277$c3e8da3@news.astraweb.com> <034d412d$0$1277$c3e8da3@news.astraweb.com> Message-ID: <6efeed07-e4df-480c-8591-e4f6e6a2bb58@j24g2000yqa.googlegroups.com> On 1 Jan., 02:47, Steven D'Aprano wrote: > On Thu, 31 Dec 2009 11:34:39 -0800, Tom Machinski wrote: > > On Wed, Dec 30, 2009 at 4:01 PM, Steven D'Aprano > > wrote: > >> On Wed, 30 Dec 2009 15:18:11 -0800, Tom Machinski wrote: > >>> Bottom line, I'm going to have to remove this pattern from my code: > > >>> ? foo = (foo for foo in foos if foo.bar).next() > > >> I don't see why. What's wrong with it? Unless you embed it in a call to > >> list, or similar, it will explicitly raise StopIteration as expected. > > > Exactly; this seems innocuous, but if some caller of this code uses it > > in a list() constructor, a very subtle and dangerous bug is introduced - > > see OP. This is the entire point of this post. > > Then don't use it in a list() constructor. > > That's a glib answer, of course. A better answer is to point out that the > problem is not with the above expression, but with letting StopIteration > bubble up as an error exception instead of dealing with it immediately. > That's not what it's for, and you can't trust it not to be captured by > something. If StopIteration represents an error condition, you need to > deal with it immediately and convert it to an exception which isn't > likely to disappear. > > > In a large, non-trivial application, you simply cannot afford the > > assumption that no caller will ever do that. Even if you have perfect > > memory, some of your other developers or library users may not. > > You shouldn't put the responsibility of dealing with the StopIteration on > the caller, because StopIteraction is a signal not an error condition, > and you can't tell when that signal will disappear. The responsibility > lies on the writer of the function containing the line (that is, the > Original Poster of this thread). > > So you need something like this: > > def my_function(): > ? ? try: > ? ? ? ? foo = (foo for foo in foos if foo.bar).next() > ? ? except StopIteration: > ? ? ? ? handle_empty_foos() > ? ? else: > ? ? ? ? handle_at_least_one_foo() > > handle_empty_foos may be as simple as raising a new exception and letting > that bubble up to whatever layer of the application is expected to deal > with it. > > > As for what's wrong with the "if not any" solution, Benjamin Kaplan's > > post hits the nail on its head. This is a bioinformatics application, so > > the iterable "foos" tends to be very large, so saving half the runtime > > makes a big difference. > > Possibly you haven't seen my reply to Benjamin, so I'll paraphrase: > that's incorrect, because any() is lazy and will return as soon as it > hits a non-false item. Tom's point is that if not any(foo for foo in foos if foo.bar): foo = (foo for foo in foos if foo.bar).next() iterates twice over (the same first few elements of) foos, which should take about twice as long as iterating once. The lazyness of "any" does not seem to matter here. Of course, you're right that the iteration might or might not be the bottleneck. On the other hand, foos might not even be reiterable. > If the foo items are arbitrary objects which have an equal chance of > being considered true or false, then on average it will have to look at > half the list, By which definition of chance? :-) Wolfram From iwaki at iwakihidekazu.net Fri Jan 1 08:46:36 2010 From: iwaki at iwakihidekazu.net (Hidekazu IWAKI) Date: Fri, 01 Jan 2010 22:46:36 +0900 Subject: Where is "urllib2" module in windows python3.1.1? Message-ID: <231220100101134636iwaki@iwakihidekazu.net> Hi; I'd like to import "urllib2" in windows python 3.1.1, but I'm not able to do it. So, I researched the library directory; the result is following: .... .... 2009/06/07 19:11 61,749 unittest.py 2010/01/01 22:18 urllib 2007/12/06 09:48 6,318 uu.py 2008/11/30 20:40 21,534 uuid.py .... .... Although it has "urllib", it doesn't have "urllib2". I guess to download and install the module. But I can't find "urllib2" module for windows with google. I wonder if it would be possible for me to give the location? From duncan.booth at invalid.invalid Fri Jan 1 08:57:03 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 1 Jan 2010 13:57:03 GMT Subject: Where is "urllib2" module in windows python3.1.1? References: Message-ID: Hidekazu IWAKI wrote: > Hi; > I'd like to import "urllib2" in windows python 3.1.1, but I'm not able to > do it. Python 3 doesn't have a urllib2 module; use the urllib package instead. See http://www.python.org/dev/peps/pep-3108/#urllib-package From usenot at geekmail.INVALID Fri Jan 1 09:01:32 2010 From: usenot at geekmail.INVALID (Andreas Waldenburger) Date: Fri, 1 Jan 2010 15:01:32 +0100 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4B3DCFAB.3030909@v.loewis.de> Message-ID: <20100101150132.0ddb72d6@geekmail.INVALID> On Fri, 01 Jan 2010 11:34:19 +0100 "Martin v. Loewis" wrote: > Your observation is not wrong, but, as Benjamin already explained, > you are misinterpreting Michi Henning's statement. He doesn't condemn > exception handling per se, but only for the handling of *expected* > outcomes. He would consider using exceptions fine for *exceptional* > output, and that is exactly the way they are used in the Python API. May I point out at this point that "exceptional" does not mean "unexpected"? You catch exceptions, not unexpectations. An exception is rare, but not surprising. Case in point: StopIteration. To put it differently: When you write "catch DeadParrot", you certainly expect to get a DeadParrot once in a while -- why else would you get it in your head to try and catch it? An unexpected exception is the one that crashes your program. /W -- INVALID? DE! From iwaki at iwakihidekazu.net Fri Jan 1 09:19:57 2010 From: iwaki at iwakihidekazu.net (IWAKI, Hidekazu) Date: Fri, 1 Jan 2010 23:19:57 +0900 Subject: Where is "urllib2" module in windows python3.1.1? In-Reply-To: References: Message-ID: >Python 3 doesn't have a urllib2 module; use the urllib package instead. Thank you for your answer and the link. Oh, sorry. It was one of the changes from .2.x to 3.x. I didn't know. There are really important and a lot of changes. Thank you! On Fri, Jan 1, 2010 at 10:57 PM, Duncan Booth wrote: > Hidekazu IWAKI wrote: > > > Hi; > > I'd like to import "urllib2" in windows python 3.1.1, but I'm not able to > > do it. > > Python 3 doesn't have a urllib2 module; use the urllib package instead. > > See http://www.python.org/dev/peps/pep-3108/#urllib-package > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at REMOVE-THIS-cybersource.com.au Fri Jan 1 09:24:50 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 01 Jan 2010 14:24:50 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <2109bc4b-834c-45a1-8287-57936c60a7bf@s31g2000yqs.googlegroups.com> Message-ID: <034df2b6$0$1277$c3e8da3@news.astraweb.com> On Fri, 01 Jan 2010 02:43:21 -0800, Jonathan Gardner wrote: > On Jan 1, 12:43?am, a... at pythoncraft.com (Aahz) wrote: >> In article , >> Benjamin Kaplan ? wrote: >> >In Python, throwing exceptions for expected outcomes is considered >> >very bad form [...] >> >> Who says that? ?I certainly don't. > > Agreed. > > int("asdf") is supposed to return what, exactly? Any language that tries > to return an int is horribly broken. [sarcasm] No no, the right way to deal with that is have int("asdf") return some arbitrary bit pattern, and expect the user to check a global variable to see whether the function returned a valid result or not. That's much better than catching an exception! [/sarcasm] -- Steven From lie.1296 at gmail.com Fri Jan 1 09:35:14 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Sat, 02 Jan 2010 01:35:14 +1100 Subject: Where is "urllib2" module in windows python3.1.1? In-Reply-To: References: Message-ID: <4b3e0821$1@dnews.tpgi.com.au> On 1/2/2010 12:46 AM, Hidekazu IWAKI wrote: > Hi; > I'd like to import "urllib2" in windows python 3.1.1, but I'm not able to > do it. > So, I researched the library directory; the result is following: > > ..... > ..... > 2009/06/07 19:11 61,749 unittest.py > 2010/01/01 22:18 urllib > 2007/12/06 09:48 6,318 uu.py > 2008/11/30 20:40 21,534 uuid.py > ..... > ..... > > Although it has "urllib", it doesn't have "urllib2". I guess to download > and install the module. But I can't find "urllib2" module for windows > with google. > > I wonder if it would be possible for me to give the location? urllib2 is merged into urllib in python3. From steve at REMOVE-THIS-cybersource.com.au Fri Jan 1 09:42:29 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 01 Jan 2010 14:42:29 GMT Subject: Dangerous behavior of list(generator) References: <7p356kF750U1@mid.individual.net> <034bd6f1$0$1277$c3e8da3@news.astraweb.com> <034d412d$0$1277$c3e8da3@news.astraweb.com> <6efeed07-e4df-480c-8591-e4f6e6a2bb58@j24g2000yqa.googlegroups.com> Message-ID: <034df6d9$0$1277$c3e8da3@news.astraweb.com> On Fri, 01 Jan 2010 05:19:02 -0800, Wolfram Hinderer wrote: > On 1 Jan., 02:47, Steven D'Aprano cybersource.com.au> wrote: >> On Thu, 31 Dec 2009 11:34:39 -0800, Tom Machinski wrote: [...] >> > As for what's wrong with the "if not any" solution, Benjamin Kaplan's >> > post hits the nail on its head. This is a bioinformatics application, >> > so the iterable "foos" tends to be very large, so saving half the >> > runtime makes a big difference. >> >> Possibly you haven't seen my reply to Benjamin, so I'll paraphrase: >> that's incorrect, because any() is lazy and will return as soon as it >> hits a non-false item. > > Tom's point is that > if not any(foo for foo in foos if foo.bar): > foo = (foo for foo in foos if foo.bar).next() > iterates twice over (the same first few elements of) foos, which should > take about twice as long as iterating once. The lazyness of "any" does > not seem to matter here. That's no different from any "Look Before You Leap" idiom. If you do this: if key in dict: x = dict[key] you search the dict twice, once to see if the key is there, and the second time to fetch the value. Whether that is better or faster than the alternative: try: x = dict[key] except KeyError: pass depends on how often you expect the lookup to fail. In any case, I would claim that Tom's argument is a classic example of premature optimization: by his own admission: 'the iterable "foos" tends to be very large' which implies that whatever happens to the foos after this test, it will probably be very time consuming. If it takes (for the sake of the argument) 10 milliseconds to process the entire iterable, who cares whether it takes 0.01 or 0.02 ms to check that the iterable is valid? > Of course, you're right that the iteration might or might not be the > bottleneck. On the other hand, foos might not even be reiterable. If that's the case, then the existing solution potentially throws away the first value of foos every time the caller tests to see if it is empty. Dealing with non-reiterable iterators can be a nuisance. In such a case, it may be best to avoid Look Before You Leap altogether: empty = True for foo in foos: if foo.bar: empty = False process(foo) if empty: handle_error_condition() -- Steven From steve at REMOVE-THIS-cybersource.com.au Fri Jan 1 09:49:02 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 01 Jan 2010 14:49:02 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <034df862$0$1277$c3e8da3@news.astraweb.com> On Fri, 01 Jan 2010 00:26:09 -0500, Benjamin Kaplan wrote: > On Thu, Dec 31, 2009 at 11:47 PM, Peng Yu wrote: >> I observe that python library primarily use exception for error >> handling rather than use error code. >> >> In the article API Design Matters by Michi Henning >> >> Communications of the ACM >> Vol. 52 No. 5, Pages 46-56 >> 10.1145/1506409.1506424 >> http://cacm.acm.org/magazines/2009/5/24646-api-design-matters/fulltext >> >> It says "Another popular design flaw?namely, throwing exceptions for >> expected outcomes?also causes inefficiencies because catching and >> handling exceptions is almost always slower than testing a return >> value." >> >> My observation is contradicted to the above statement by Henning. If my >> observation is wrong, please just ignore my question below. >> >> Otherwise, could some python expert explain to me why exception is >> widely used for error handling in python? Is it because the efficiency >> is not the primary goal of python? >> -- >> http://mail.python.org/mailman/listinfo/python-list >> >> > Read the quote again "Another popular design flaw?namely, throwing > exceptions *for expected outcomes*" > In Python, throwing exceptions for expected outcomes is considered very > bad form (well, except for StopIteration but that should almost never be > handled directly by the programmer). Exceptions are *exceptional*, not "errors" or "unexpected". They are exceptional because they aren't the "normal" case, but that doesn't mean they are surprising or unexpected. Are you surprised that your "for x in range(1000)" loop comes to an end? Of course you are not -- it is completely expected, even though less than 1% of the iterations are the last loop. The end of the sequence is EXCEPTIONAL but not UNEXPECTED. If you program without expecting that keys can sometimes be missing from dictionaries (KeyError), or that you might sometimes have to deal with a list index that is out of range (IndexError), or that the denominator in a division might be zero (ZeroDivisionError), then you must be writing really buggy code. None of these things are unexpected, but they are all exceptional. The urllib2 module defines a HTTPError class, which does double-duty as both an exception and a valid HTTP response. If you're doing any HTTP programming, you better expect to deal with HTTP 301, 302 etc. codes, or at least trust that the library you use will transparently handle them for you. > To answer why people recommend using "Easier to Ask Forgiveness than > Permission" as opposed to "Look Before You Leap" : Because of the way > it's implemented, Python works quite differently from most languages. An > attribute look-up is rather expensive because it's a hash table look-up > at run time. Wrapping a small piece of code in a try block however, > isn't (relatively) very expensive at all in Python. It's not just relatively inexpensive, it's absolutely inexpensive: it costs about as much as a pass statement in CPython, which is pretty much as cheap as it gets. (If anyone can demonstrate a cheaper operation available from pure Python, I'd love to see it.) > It's only catching the exception that's expensive, True. > but if you're catching the exception, > something has gone wrong anyway and performance isn't your biggest > issue. The second try...except clause in the urllib2 module reads: try: kind = int(kind) except ValueError: pass In this case, the error is simply ignored. Nothing has gone wrong. Here's an example from my own code: I have an API where you pass a mapping (either a dict or a list of (key, value) tuples) to a function. So I do this: try: it = obj.iteritems() except AttributeError: it = obj for key, value in it: do_stuff() There's nothing wrong with catching exceptions. -- Steven From phlip2005 at gmail.com Fri Jan 1 10:10:51 2010 From: phlip2005 at gmail.com (Phlip) Date: Fri, 1 Jan 2010 07:10:51 -0800 (PST) Subject: change an exception's message and re-raise it References: <97c21727-ed6d-4be4-8c22-24b7beccd7bb@e27g2000yqd.googlegroups.com> <034d2f31$0$1277$c3e8da3@news.astraweb.com> Message-ID: On Dec 31 2009, 4:30?pm, Steven D'Aprano wrote: > ... ? ? 1/0 > ... except ZeroDivisionError, e: > ... ? ? e.args = e.args + ('fe', 'fi', 'fo', 'fum') > ... ? ? raise When I added print e.args it showed the old args. Maybe I was trying too hard - this is why I said e seemed locked or something. This started working: new_exception = self.format_fault(e.args[0]) e.args = (new_exception,) + (e.args[1:]) raise May I ask if args always has more than one entry? I need to bypass the silly "'tuple' object does not support item assignment" issue... From phlip2005 at gmail.com Fri Jan 1 10:21:13 2010 From: phlip2005 at gmail.com (Phlip) Date: Fri, 1 Jan 2010 07:21:13 -0800 (PST) Subject: change an exception's message and re-raise it References: <97c21727-ed6d-4be4-8c22-24b7beccd7bb@e27g2000yqd.googlegroups.com> <034d2f31$0$1277$c3e8da3@news.astraweb.com> Message-ID: <910aeda1-9852-46e1-960c-06cc9b1357dd@22g2000yqr.googlegroups.com> On Dec 31 2009, 4:30?pm, Steven D'Aprano wrote: > For the record you can get the exception type from type(e): > > raise type(e)("whatever you want") > > but that creates a new exception, not re-raising the old one. Except if a type constructs with some other number of arguments, apparently... From lie.1296 at gmail.com Fri Jan 1 10:24:41 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Sat, 02 Jan 2010 02:24:41 +1100 Subject: Exception as the primary error handling mechanism? In-Reply-To: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <4b3e13bb@dnews.tpgi.com.au> On 1/1/2010 3:47 PM, Peng Yu wrote: > I observe that python library primarily use exception for error > handling rather than use error code. > > In the article API Design Matters by Michi Henning > > Communications of the ACM > Vol. 52 No. 5, Pages 46-56 > 10.1145/1506409.1506424 > http://cacm.acm.org/magazines/2009/5/24646-api-design-matters/fulltext > > It says "Another popular design flaw?namely, throwing exceptions for > expected outcomes?also causes inefficiencies because catching and > handling exceptions is almost always slower than testing a return > value." > > My observation is contradicted to the above statement by Henning. If > my observation is wrong, please just ignore my question below. > > Otherwise, could some python expert explain to me why exception is > widely used for error handling in python? Simple, when an exception is thrown and I don't catch it, the exception terminates the program immediately and I got a traceback showing the point of failure. When I return error value and I don't check for it, I passed passed errors silently and gets a traceback forty-two lines later when trying to use the resources I failed to acquire forty-two lines prior. > Is it because the efficiency > is not the primary goal of python? Efficiency is not primary goal of python, but since python encourages EAFP (Easier to Ask Forgiveness than Permission); the design decisions chosen makes setting up a try-block much cheaper than a language designed over LBYL (Look Before You Leap) and return codes. From jkpeck at gmail.com Fri Jan 1 10:41:30 2010 From: jkpeck at gmail.com (JKPeck) Date: Fri, 1 Jan 2010 07:41:30 -0800 (PST) Subject: How to Suppress Interactive Assignment to "_" Message-ID: <910af7d0-8aa7-43fa-b2c9-32e12dc2fbc9@m16g2000yqc.googlegroups.com> The gettext module uses the convention of defining a function named "_" that maps text into its translation. This conflicts with the automatic interactive interpreter assignment of expressions to a variable with that same name. While if you are careful, you can avoid that assignment while debugging, and you can choose a different function for gettext, this conflict is a nuisance. I am looking for a way to suppress the expression assignment to _ or to change the name of the variable assigned to. Is this possible? Using Python 2.6. TIA, Jon Peck From benjamin.kaplan at case.edu Fri Jan 1 11:02:28 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Fri, 1 Jan 2010 11:02:28 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <034df862$0$1277$c3e8da3@news.astraweb.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <034df862$0$1277$c3e8da3@news.astraweb.com> Message-ID: On Fri, Jan 1, 2010 at 9:49 AM, Steven D'Aprano wrote: > > Exceptions are *exceptional*, not "errors" or "unexpected". They are > exceptional because they aren't the "normal" case, but that doesn't mean > they are surprising or unexpected. Are you surprised that your "for x in > range(1000)" loop comes to an end? Of course you are not -- it is > completely expected, even though less than 1% of the iterations are the > last loop. The end of the sequence is EXCEPTIONAL but not UNEXPECTED. > Sorry if my word choice was confusing- I was trying to point out that in Python, you don't test errors for your typical conditions, but for ones that you know still exist but don't plan on occurring often. From mwilson at the-wire.com Fri Jan 1 11:06:21 2010 From: mwilson at the-wire.com (Mel) Date: Fri, 01 Jan 2010 11:06:21 -0500 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <2109bc4b-834c-45a1-8287-57936c60a7bf@s31g2000yqs.googlegroups.com> <034df2b6$0$1277$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > On Fri, 01 Jan 2010 02:43:21 -0800, Jonathan Gardner wrote: > >> On Jan 1, 12:43 am, a... at pythoncraft.com (Aahz) wrote: >>> In article , >>> Benjamin Kaplan wrote: >>> >In Python, throwing exceptions for expected outcomes is considered >>> >very bad form [...] >>> >>> Who says that? I certainly don't. >> >> Agreed. >> >> int("asdf") is supposed to return what, exactly? Any language that tries >> to return an int is horribly broken. > > > [sarcasm] > No no, the right way to deal with that is have int("asdf") return some > arbitrary bit pattern, and expect the user to check a global variable to > see whether the function returned a valid result or not. That's much > better than catching an exception! > [/sarcasm] Or the other way around, as in C (I suspect the original ACM article assumed C.) Look at the legion of C library subroutines that return only 0 for good or -1 for bad, and do all their real work in side-effects (through pointers as function arguments.) Python is a big improvement: use the function return values for the payload, and push the out-of-band "omyghod" response into an Exception. Mel. From steve at REMOVE-THIS-cybersource.com.au Fri Jan 1 11:26:59 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 01 Jan 2010 16:26:59 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <034df862$0$1277$c3e8da3@news.astraweb.com> Message-ID: <034e0f57$0$1277$c3e8da3@news.astraweb.com> On Fri, 01 Jan 2010 11:02:28 -0500, Benjamin Kaplan wrote: > I was trying to point out that in > Python, you don't test errors for your typical conditions, but for ones > that you know still exist but don't plan on occurring often. I'm sorry, but that makes no sense to me at all. I don't understand what you are trying to say. You do understand that exceptions aren't just for errors? They are raised under specific circumstances. Whether that circumstance is an error or not is entirely up to the caller. try: n = mylist.index('fault') except ValueError: print "All is good, no fault detected" else: print "Your data contains a fault in position", n People get hung up on the idea that exceptions == errors, but they aren't. They *may* be errors, and that is one common use, but that depends entirely on the caller. -- Steven From victorsubervi at gmail.com Fri Jan 1 11:52:17 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 1 Jan 2010 11:52:17 -0500 Subject: MySQL Error Message-ID: <4dc0cfea1001010852u1101627avf2677af1553fd484@mail.gmail.com> Hi; I'm trying to avoid the mortal sin of blank excepts. I intentionally threw this error: Traceback (most recent call last): File "/var/www/html/angrynates.com/cart/createAssociations2.py", line 137, in ? createAssociations2() File "/var/www/html/angrynates.com/cart/createAssociations2.py", line 108, in createAssociations2 cursor.execute(sql) File "/usr/lib64/python2.4/site-packages/MySQLdb/cursors.py", line 163, in execute self.errorhandler(self, exc, value) File "/usr/lib64/python2.4/site-packages/MySQLdb/connections.py", line 35, in defaulterrorhandler raise errorclass, errorvalue ProgrammingError: (1146, "Table 'test.productsAssociations' doesn't exist") However, "ProgrammingError" is not an error. How do I discover the real error, so I can write the appropriate except statement? TIA, beno -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Fri Jan 1 12:03:21 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 01 Jan 2010 17:03:21 +0000 Subject: twenty years ago Guido created Python In-Reply-To: References: Message-ID: <4B3E2AD9.5040901@mrabarnett.plus.com> J Peyret wrote: > On Dec 31 2009, 2:06 pm, Steve Howell wrote: >> FYI: >> >> http://twitter.com/gvanrossum >> >> Python is a truly awesome programming language. Not only is Guido a >> genius language designer, but he is also a great project leader. What >> an accomplishment. Congratulations to everybody who has contributed >> to Python in the last two decades! > > Notwithstanding all of the above, which are all true, having met > Guido, I say he is a genuinely nice human being. > And anyone who says he isn't will be put into the comfy chair! :-) From carsten.haese at gmail.com Fri Jan 1 12:04:10 2010 From: carsten.haese at gmail.com (Carsten Haese) Date: Fri, 01 Jan 2010 12:04:10 -0500 Subject: MySQL Error In-Reply-To: <4dc0cfea1001010852u1101627avf2677af1553fd484@mail.gmail.com> References: <4dc0cfea1001010852u1101627avf2677af1553fd484@mail.gmail.com> Message-ID: Victor Subervi wrote: > However, "ProgrammingError" is not an error. How do I discover the real > error, so I can write the appropriate except statement? You're not making any sense. How did you determine that ProgrammingError is not an error or that it's not the "real error"? Show us the code you ran, the output you expected, and the output it produced instead. Blind guess: You're using "except ProgrammingError" when you should be using "except MySQLdb.ProgrammingError". If this guess is incorrect, see above. -- Carsten Haese http://informixdb.sourceforge.net From __peter__ at web.de Fri Jan 1 12:06:18 2010 From: __peter__ at web.de (Peter Otten) Date: Fri, 01 Jan 2010 18:06:18 +0100 Subject: How to Suppress Interactive Assignment to "_" References: <910af7d0-8aa7-43fa-b2c9-32e12dc2fbc9@m16g2000yqc.googlegroups.com> Message-ID: JKPeck wrote: > The gettext module uses the convention of defining a function named > "_" that maps text into its translation. > This conflicts with the automatic interactive interpreter assignment > of expressions to a variable with that same name. > > While if you are careful, you can avoid that assignment while > debugging, and you can choose a different function for gettext, this > conflict is a nuisance. I am looking for a way to suppress the > expression assignment to _ or to change the name of the variable > assigned to. Is this possible? Using Python 2.6. $ cat displayhook.py import sys import __builtin__ as builtin def displayhook(obj): if obj is not None: builtin._last_result = obj print repr(obj) sys.displayhook = displayhook $ python -i displayhook.py >>> 42 42 >>> _ Traceback (most recent call last): File "", line 1, in NameError: name '_' is not defined >>> _last_result 42 From python at mrabarnett.plus.com Fri Jan 1 12:10:02 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 01 Jan 2010 17:10:02 +0000 Subject: MySQL Error In-Reply-To: <4dc0cfea1001010852u1101627avf2677af1553fd484@mail.gmail.com> References: <4dc0cfea1001010852u1101627avf2677af1553fd484@mail.gmail.com> Message-ID: <4B3E2C6A.4070506@mrabarnett.plus.com> Victor Subervi wrote: > Hi; > I'm trying to avoid the mortal sin of blank excepts. I intentionally > threw this error: > > Traceback (most recent call last): > File "/var/www/html/angrynates.com/cart/createAssociations2.py > ", line 137, in ? > createAssociations2() > File "/var/www/html/angrynates.com/cart/createAssociations2.py > ", line 108, in > createAssociations2 > cursor.execute(sql) > File "/usr/lib64/python2.4/site-packages/MySQLdb/cursors.py", line > 163, in execute > self.errorhandler(self, exc, value) > File "/usr/lib64/python2.4/site-packages/MySQLdb/connections.py", line > 35, in defaulterrorhandler > raise errorclass, errorvalue > ProgrammingError: (1146, "Table 'test.productsAssociations' doesn't exist") > > However, "ProgrammingError" is not an error. How do I discover the real > error, so I can write the appropriate except statement? > What makes you think it isn't? As the traceback says, "Table 'test.productsAssociations' doesn't exist", and that's the cause. From martin at v.loewis.de Fri Jan 1 12:18:32 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Fri, 01 Jan 2010 18:18:32 +0100 Subject: Exception as the primary error handling mechanism? In-Reply-To: <034e0f57$0$1277$c3e8da3@news.astraweb.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <034df862$0$1277$c3e8da3@news.astraweb.com> <034e0f57$0$1277$c3e8da3@news.astraweb.com> Message-ID: > You do understand that exceptions aren't just for errors? They are raised > under specific circumstances. Whether that circumstance is an error or > not is entirely up to the caller. I think that's a fairly narrow definition of the word error, and probably also the source of confusion in this thread. ISTM that there is a long tradition of giving different meaning to the word "error" in computing. For example, the Unix man pages list various conditions as "errors" purely by their outcome, and completely ignoring on whether the caller would consider the result erroneous - ISTM that a system call reports an "error" iff it is "unsuccessful". By that (common) usage of "error", it is a condition determined by the callee, not the caller (i.e. callee could not successfully complete the operation). In that sense, it is indeed equivalent to Python's usage of exceptions, which are also determined by the callee, and typically also in cases where successful completion is not possible. Whether these cases are "exceptional" in the word sense (i.e. deviating from the norm) would have to be decided by the application, again (which would set the norm). Regards, Martin From victorsubervi at gmail.com Fri Jan 1 13:32:20 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 1 Jan 2010 13:32:20 -0500 Subject: MySQL Error In-Reply-To: <4B3E2C6A.4070506@mrabarnett.plus.com> References: <4dc0cfea1001010852u1101627avf2677af1553fd484@mail.gmail.com> <4B3E2C6A.4070506@mrabarnett.plus.com> Message-ID: <4dc0cfea1001011032r3020992bs295c19af74ad61d4@mail.gmail.com> On Fri, Jan 1, 2010 at 12:10 PM, MRAB wrote: > Victor Subervi wrote: > >> Hi; >> I'm trying to avoid the mortal sin of blank excepts. I intentionally threw >> this error: >> >> Traceback (most recent call last): >> File "/var/www/html/angrynates.com/cart/createAssociations2.py < >> http://angrynates.com/cart/createAssociations2.py>", line 137, in ? >> createAssociations2() >> File "/var/www/html/angrynates.com/cart/createAssociations2.py < >> http://angrynates.com/cart/createAssociations2.py>", line 108, in >> createAssociations2 >> >> cursor.execute(sql) >> File "/usr/lib64/python2.4/site-packages/MySQLdb/cursors.py", line 163, >> in execute >> self.errorhandler(self, exc, value) >> File "/usr/lib64/python2.4/site-packages/MySQLdb/connections.py", line >> 35, in defaulterrorhandler >> raise errorclass, errorvalue >> ProgrammingError: (1146, "Table 'test.productsAssociations' doesn't >> exist") >> >> However, "ProgrammingError" is not an error. How do I discover the real >> error, so I can write the appropriate except statement? >> >> What makes you think it isn't? > Because I've tried except ProgrammingError: pass before and *that* threw an error. So I didnt' try again. beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From showell30 at yahoo.com Fri Jan 1 13:52:30 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 1 Jan 2010 10:52:30 -0800 (PST) Subject: twenty years ago Guido created Python References: Message-ID: <0003342a-9738-476d-b83d-f99921e18649@26g2000yqo.googlegroups.com> On Jan 1, 9:03?am, MRAB wrote: > J Peyret wrote: > > On Dec 31 2009, 2:06 pm, Steve Howell wrote: > >> FYI: > > >>http://twitter.com/gvanrossum > > >> Python is a truly awesome programming language. ?Not only is Guido a > >> genius language designer, but he is also a great project leader. ?What > >> an accomplishment. ?Congratulations to everybody who has contributed > >> to Python in the last two decades! > > > Notwithstanding all of the above, which are all true, having met > > Guido, I say he is a genuinely nice human being. > > And anyone who says he isn't will be put into the comfy chair! :-) And poked with the soft cushions! From jkpeck at gmail.com Fri Jan 1 13:53:31 2010 From: jkpeck at gmail.com (JKPeck) Date: Fri, 1 Jan 2010 10:53:31 -0800 (PST) Subject: How to Suppress Interactive Assignment to "_" References: <910af7d0-8aa7-43fa-b2c9-32e12dc2fbc9@m16g2000yqc.googlegroups.com> Message-ID: On Jan 1, 10:06?am, Peter Otten <__pete... at web.de> wrote: > JKPeck wrote: > > The gettext module uses the convention of defining a function named > > "_" that maps text into its translation. > > This conflicts with the automatic interactive interpreter assignment > > of expressions to a variable with that same name. > > > While if you are careful, you can avoid that assignment while > > debugging, and you can choose a different function for gettext, this > > conflict is a nuisance. ?I am looking for a way to suppress the > > expression assignment to _ or to change the name of the variable > > assigned to. ?Is this possible? ?Using Python 2.6. > > $ cat displayhook.py > import sys > import __builtin__ as builtin > > def displayhook(obj): > ? ? if obj is not None: > ? ? ? ? builtin._last_result = obj > ? ? ? ? print repr(obj) > > sys.displayhook = displayhook > $ python -i displayhook.py>>> 42 > 42 > >>> _ > > Traceback (most recent call last): > ? File "", line 1, in > NameError: name '_' is not defined>>> _last_result > > 42 Thanks. It's just what I needed. -Jon Peck From benjamin.kaplan at case.edu Fri Jan 1 14:04:34 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Fri, 1 Jan 2010 14:04:34 -0500 Subject: MySQL Error In-Reply-To: <4dc0cfea1001011032r3020992bs295c19af74ad61d4@mail.gmail.com> References: <4dc0cfea1001010852u1101627avf2677af1553fd484@mail.gmail.com> <4B3E2C6A.4070506@mrabarnett.plus.com> <4dc0cfea1001011032r3020992bs295c19af74ad61d4@mail.gmail.com> Message-ID: On Fri, Jan 1, 2010 at 1:32 PM, Victor Subervi wrote: > On Fri, Jan 1, 2010 at 12:10 PM, MRAB wrote: >> >> Victor Subervi wrote: >>> >>> Hi; >>> I'm trying to avoid the mortal sin of blank excepts. I intentionally >>> threw this error: >>> >>> Traceback (most recent call last): >>> ?File "/var/www/html/angrynates.com/cart/createAssociations2.py >>> ", line 137, in ? >>> ? ?createAssociations2() >>> ?File "/var/www/html/angrynates.com/cart/createAssociations2.py >>> ", line 108, in >>> createAssociations2 >>> ? ?cursor.execute(sql) >>> ?File "/usr/lib64/python2.4/site-packages/MySQLdb/cursors.py", line 163, >>> in execute >>> ? ?self.errorhandler(self, exc, value) >>> ?File "/usr/lib64/python2.4/site-packages/MySQLdb/connections.py", line >>> 35, in defaulterrorhandler >>> ? ?raise errorclass, errorvalue >>> ProgrammingError: (1146, "Table 'test.productsAssociations' doesn't >>> exist") >>> >>> However, "ProgrammingError" is not an error. How do I discover the real >>> error, so I can write the appropriate except statement? >>> >> What makes you think it isn't? > > Because I've tried > except ProgrammingError: > ? pass > before and *that* threw an error. So I didnt' try again. > beno Just because it's not in the current namespace doesn't mean it's not an error. From PEP 249 (the db api v2) : ProgrammingError Exception raised for programming errors, e.g. table not found or already exists, syntax error in the SQL statement, wrong number of parameters specified, etc. It must be a subclass of DatabaseError. > > -- > http://mail.python.org/mailman/listinfo/python-list > > From python at mrabarnett.plus.com Fri Jan 1 15:24:09 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 01 Jan 2010 20:24:09 +0000 Subject: MySQL Error In-Reply-To: References: <4dc0cfea1001010852u1101627avf2677af1553fd484@mail.gmail.com> <4B3E2C6A.4070506@mrabarnett.plus.com> <4dc0cfea1001011032r3020992bs295c19af74ad61d4@mail.gmail.com> Message-ID: <4B3E59E9.9020401@mrabarnett.plus.com> Benjamin Kaplan wrote: > On Fri, Jan 1, 2010 at 1:32 PM, Victor Subervi wrote: >> On Fri, Jan 1, 2010 at 12:10 PM, MRAB wrote: >>> Victor Subervi wrote: >>>> Hi; >>>> I'm trying to avoid the mortal sin of blank excepts. I intentionally >>>> threw this error: >>>> >>>> Traceback (most recent call last): >>>> File "/var/www/html/angrynates.com/cart/createAssociations2.py >>>> ", line 137, in ? >>>> createAssociations2() >>>> File "/var/www/html/angrynates.com/cart/createAssociations2.py >>>> ", line 108, in >>>> createAssociations2 >>>> cursor.execute(sql) >>>> File "/usr/lib64/python2.4/site-packages/MySQLdb/cursors.py", line 163, >>>> in execute >>>> self.errorhandler(self, exc, value) >>>> File "/usr/lib64/python2.4/site-packages/MySQLdb/connections.py", line >>>> 35, in defaulterrorhandler >>>> raise errorclass, errorvalue >>>> ProgrammingError: (1146, "Table 'test.productsAssociations' doesn't >>>> exist") >>>> >>>> However, "ProgrammingError" is not an error. How do I discover the real >>>> error, so I can write the appropriate except statement? >>>> >>> What makes you think it isn't? >> Because I've tried >> except ProgrammingError: >> pass >> before and *that* threw an error. So I didnt' try again. >> beno > > Just because it's not in the current namespace doesn't mean it's not > an error. From PEP 249 (the db api v2) : > > ProgrammingError > > Exception raised for programming errors, e.g. table not > found or already exists, syntax error in the SQL > statement, wrong number of parameters specified, etc. It > must be a subclass of DatabaseError. > You shouldn't just that it threw an error and give up, you should look at what the error was, in this case a NameError, which means that it doesn't know that name. Why? Because it's defined in the MySQL module that you imported, but you didn't import the name ProgrammingError. From python at bdurham.com Fri Jan 1 16:24:36 2010 From: python at bdurham.com (python at bdurham.com) Date: Fri, 01 Jan 2010 16:24:36 -0500 Subject: Which version of MSVC?90.DLL's to distribute with Python 2.6 based Py2exe executables? In-Reply-To: References: <05bd2f84-4f09-4a76-8e17-5868450ad393@c3g2000yqd.googlegroups.com> <753bb7c9-14bf-44e3-9679-0693635f2cd9@j14g2000yqm.googlegroups.com> <573025f1-0e5d-4d82-a86b-36ea33f85cad@c3g2000yqd.googlegroups.com> Message-ID: <1262381076.11442.1352533759@webmail.messagingengine.com> Waldemar, Thank your for sharing your technique - works great with 32-bit Python 2.6.4. Has anyone tried this with a 64-bit version of Python? Malcolm From python at bdurham.com Fri Jan 1 16:54:12 2010 From: python at bdurham.com (python at bdurham.com) Date: Fri, 01 Jan 2010 16:54:12 -0500 Subject: Safe file I/O to shared file (or SQLite) from multi-threaded web server Message-ID: <1262382852.16237.1352535253@webmail.messagingengine.com> I'm looking for the best practice way for a multi-threaded python web server application to read/write to a shared file or a SQLite database. What do I need to do (if anything) to make sure my writes to a regular file on disk or to a SQLite database are atomic in nature when multiple clients post data to my application simultaneously? Do I need to use a Queue type data structure and then run a background thread that monitors my Queue for data which it (and it alone) removes and copies to the destination file or SQLite datatbase? Note: In my specific case, the web server will be based on CherryPy 3.1 but I think this type of question is relevant across other Python based web server frameworks as well. Thank you, Malcolm From drsalists at gmail.com Fri Jan 1 17:02:36 2010 From: drsalists at gmail.com (Dan Stromberg) Date: Fri, 01 Jan 2010 14:02:36 -0800 Subject: Significant whitespace Message-ID: <4B3E70FC.1010702@gmail.com> I put together a page about significant whitespace (and the lack thereof). You're invited to check it out: http://stromberg.dnsalias.org/~dstromberg/significant-whitespace.html From clp2 at rebertia.com Fri Jan 1 17:19:28 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Fri, 1 Jan 2010 14:19:28 -0800 Subject: Significant whitespace In-Reply-To: <4B3E70FC.1010702@gmail.com> References: <4B3E70FC.1010702@gmail.com> Message-ID: <50697b2c1001011419q4f055bc7jec668e890aae974c@mail.gmail.com> On Fri, Jan 1, 2010 at 2:02 PM, Dan Stromberg wrote: > I put together a page about significant whitespace (and the lack thereof). > > You're invited to check it out: > > http://stromberg.dnsalias.org/~dstromberg/significant-whitespace.html For those of us who weren't around during the heyday of FORTRAN, can anyone describe this apparently much-reviled significant whitespace feature that continues to make some programmers unjustly fearful about Python's use of indentation? Thanks in advance. Cheers, Chris -- http://blog.rebertia.com From deets at nospam.web.de Fri Jan 1 17:20:19 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Fri, 01 Jan 2010 23:20:19 +0100 Subject: Safe file I/O to shared file (or SQLite) from multi-threaded web server In-Reply-To: References: Message-ID: <7q7ap3F3u36s0U1@mid.uni-berlin.de> python at bdurham.com schrieb: > I'm looking for the best practice way for a multi-threaded python web > server application to read/write to a shared file or a SQLite database. > > What do I need to do (if anything) to make sure my writes to a regular > file on disk or to a SQLite database are atomic in nature when multiple > clients post data to my application simultaneously? > > Do I need to use a Queue type data structure and then run a background > thread that monitors my Queue for data which it (and it alone) removes > and copies to the destination file or SQLite datatbase? > > Note: In my specific case, the web server will be based on CherryPy 3.1 > but I think this type of question is relevant across other Python based > web server frameworks as well. AFAIK, sqlite ensures process-serialization via locking, and threads synchronize themselves as well. So you shouldn't need to worry at all. Diez From bluemangroupie at gmail.com Fri Jan 1 17:51:25 2010 From: bluemangroupie at gmail.com (dennis) Date: Fri, 1 Jan 2010 14:51:25 -0800 (PST) Subject: twenty years ago Guido created Python References: Message-ID: <91141e23-6691-4f44-a73f-efd4292d9ffa@h9g2000yqa.googlegroups.com> On Jan 1, 6:04?am, Krishnakant wrote: > On Fri, 2010-01-01 at 03:25 -0800, J Peyret wrote: > > On Dec 31 2009, 2:06 pm, Steve Howell wrote: > > > FYI: > > > >http://twitter.com/gvanrossum > > > > Python is a truly awesome programming language. ?Not only is Guido a > > > genius language designer, but he is also a great project leader. ?What > > > an accomplishment. ?Congratulations to everybody who has contributed > > > to Python in the last two decades! > > > Notwithstanding all of the above, which are all true, having met > > Guido, I say he is a genuinely nice human being. ?Go BSD derivatives. > > Indeed, > python is a great programming language. > May be it was not marketted as much as languages like java or not as > much as platforms like rails. > But who cares? ?I must mention here that I am totally blind and the > talking software (screen reader ) I use on GNU/Linux called Orca is also > programmed using python. > > And I make web applications and off late started to use pylons ( ?again > centered around python). > I see that python is for 2 kind of programmers. > one which are absolute beginners to programming itself who want to learn > the scientific art of programming and the second is the extreme experts > who need the power and performance nicely balanced. > > Great work! > > happy hfacking. > Krishnakant. That's incredible. From ktenney at gmail.com Fri Jan 1 17:55:45 2010 From: ktenney at gmail.com (Kent Tenney) Date: Fri, 1 Jan 2010 16:55:45 -0600 Subject: Trying to run a sudo command from script Message-ID: Howdy, A script running as a regular user sometimes wants to run sudo commands. It gets the password with getpass. pw = getpass.getpass() I've fiddled a bunch with stuff like proc = subprocess.Popen('sudo touch /etc/foo'.split(), stdin=subprocess.PIPE) proc.communicate(input=pw) getting assorted errors with all variations I try. Googling says use pexpect, but I'd prefer a stdlib solution. Any help appreciated. Thanks, Kent From deets at nospam.web.de Fri Jan 1 18:08:58 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Sat, 02 Jan 2010 00:08:58 +0100 Subject: Trying to run a sudo command from script In-Reply-To: References: Message-ID: <7q7dkaF3t8r0rU1@mid.uni-berlin.de> Kent Tenney schrieb: > Howdy, > > A script running as a regular user sometimes wants > to run sudo commands. > > It gets the password with getpass. > pw = getpass.getpass() > > I've fiddled a bunch with stuff like > proc = subprocess.Popen('sudo touch /etc/foo'.split(), stdin=subprocess.PIPE) > proc.communicate(input=pw) > > getting assorted errors with all variations I try. > > Googling says use pexpect, but I'd prefer a stdlib solution. pexpect is pure python, and it's needed. There is no easy way around the issue, so if you insist on not using pexpect, you re-invent the wheel and write the exact same code - just more error-prone, because of wheel-reinvention.... Diez From mydimle at gmail.com Fri Jan 1 18:27:57 2010 From: mydimle at gmail.com (myle) Date: Fri, 1 Jan 2010 15:27:57 -0800 (PST) Subject: Bare Excepts References: <4dc0cfea0912300209x15e5bc96nd6e6224d89df0638@mail.gmail.com> <4B3B4FE5.8070605@sequans.com> <4dc0cfea0912300509v7f9bb367jf33bb63736407ed@mail.gmail.com> <4dc0cfea0912300512h43a4fdb1oa585a36391de341e@mail.gmail.com> <6944e440-c797-439d-9bf9-b61a05a3e26c@k23g2000yqa.googlegroups.com> <878wck2ojd.fsf@benfinney.id.au> <4dc0cfea0912301927q16445c59r1fb8594ee0f04d24@mail.gmail.com> Message-ID: Why we should prefer ``if: ...'' over a ``try: ... except something: pass'' block? In http://wiki.python.org/moin/PythonSpeed/PerformanceTips#InitializingDictionaryElements it is stated that a try catch block is faster if more often no exception occurs. Happy new year, Dimitris Leventeas From roy at panix.com Fri Jan 1 18:29:24 2010 From: roy at panix.com (Roy Smith) Date: Fri, 01 Jan 2010 18:29:24 -0500 Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> Message-ID: In article , Chris Rebert wrote: > On Fri, Jan 1, 2010 at 2:02 PM, Dan Stromberg wrote: > > I put together a page about significant whitespace (and the lack thereof). > > > > You're invited to check it out: > > > > http://stromberg.dnsalias.org/~dstromberg/significant-whitespace.html > > For those of us who weren't around during the heyday of FORTRAN, can > anyone describe this apparently much-reviled significant whitespace > feature that continues to make some programmers unjustly fearful about > Python's use of indentation? I know it's bad form to cite yourself, but I think I said it pretty well (oh my) 12 years ago: http://www.python.org/doc/humor/#bad-habits To address your question more directly, here's a couple of ways Fortran treated whitespace which would surprise the current crop of Java/PHP/Python/Ruby programmers: 1) Line numbers (i.e. the things you could GOTO to) were in column 2-7 (column 1 was reserved for a comment indicator). This is not quite significant whitespace, it's more like significant indentation. 2) Whitespace was not required in many places. For example, the following two statements (this will only make sense in a fixed-width font) are identical: DO 10 I = 1, 10 DO10I=1,10 People make fun of Fortran these days, but it was a staggeringly important advance in software development compared to what came before (i.e. assembler). The move from assembler to Fortran was a far more significant advance than the move from, say, Perl to Python. From mensanator at aol.com Fri Jan 1 18:29:46 2010 From: mensanator at aol.com (Mensanator) Date: Fri, 1 Jan 2010 15:29:46 -0800 (PST) Subject: Significant whitespace References: Message-ID: <5b7a43b4-c53e-49d5-a1ef-ee19533d851c@o35g2000vbl.googlegroups.com> On Jan 1, 4:02?pm, Dan Stromberg wrote: > I put together a page about significant whitespace (and the lack thereof). The real problem is your use of proportional spaced fonts. > > You're invited to check it out: > > http://stromberg.dnsalias.org/~dstromberg/significant-whitespace.html From tartley at tartley.com Fri Jan 1 18:51:00 2010 From: tartley at tartley.com (Jonathan Hartley) Date: Fri, 1 Jan 2010 15:51:00 -0800 (PST) Subject: Which version of MSVC?90.DLL's to distribute with Python 2.6 based Py2exe executables? References: <05bd2f84-4f09-4a76-8e17-5868450ad393@c3g2000yqd.googlegroups.com> <753bb7c9-14bf-44e3-9679-0693635f2cd9@j14g2000yqm.googlegroups.com> <573025f1-0e5d-4d82-a86b-36ea33f85cad@c3g2000yqd.googlegroups.com> Message-ID: <87e07145-4d35-47ff-9203-5c021a841f06@j24g2000yqa.googlegroups.com> > the method involves editing python26.dll in order to remove > dependency references and then dropping msvcr90.dll in the same > directory as the py2exe produced executable. Clever idea Waldemar, thanks for that, but for the moment, using the dll as a win32 assembly (ie. with a manifest file, as described by others earlier in this thread) seems to work just as well. >> > You'll need to include Microsoft.VC90.CRT.manifest and msvcr90.dll. So, for the record, I have included these two files in a directory called 'Microsoft.VC90.CRT', which is in the same dir as my executable, and that makes everything work, even on my 'bare bones' windows XP virtual machine. The end result is a small game some friends and I put together for the PyWeek competition a few months ago. If anyone wants to see exactly how the resulting dll and manifest look, you can download the Windows binary under the 'featured downloads' from here: http://code.google.com/p/brokenspell/ I appreciate David Bolen's comment that an installer isn't that hard to put together: Acknowledged, but for the moment I still seem to be getting away without needing one. Best regards, Jonathan From steve at REMOVE-THIS-cybersource.com.au Fri Jan 1 20:05:51 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 02 Jan 2010 01:05:51 GMT Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> Message-ID: <034e88f2$0$1277$c3e8da3@news.astraweb.com> On Fri, 01 Jan 2010 14:19:28 -0800, Chris Rebert wrote: > For those of us who weren't around during the heyday of FORTRAN, can > anyone describe this apparently much-reviled significant whitespace > feature that continues to make some programmers unjustly fearful about > Python's use of indentation? I'm not a Fortran expert, but I understand that whitespace was optional within lines unless needed to make something unambiguous. In Python terms, imagine if we could write foriinrange(10): instead of the usual for i in range(10): Since the colon makes it unambiguous that it is some sort of block construct, and it starts with "for", it must be a for loop. Pretty horrible, yes? -- Steven From sjmachin at lexicon.net Fri Jan 1 20:27:03 2010 From: sjmachin at lexicon.net (John Machin) Date: Fri, 1 Jan 2010 17:27:03 -0800 (PST) Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> Message-ID: <3db95947-1e35-4bd1-bd4c-37df646f9ed5@v25g2000yqk.googlegroups.com> On Jan 2, 10:29?am, Roy Smith wrote: > > To address your question more directly, here's a couple of ways Fortran > treated whitespace which would surprise the current crop of > Java/PHP/Python/Ruby programmers: > > 1) Line numbers (i.e. the things you could GOTO to) were in column 2-7 > (column 1 was reserved for a comment indicator). ?This is not quite > significant whitespace, it's more like significant indentation. That would also surprise former FORTRAN programmers (who rarely referred to the language as "Fortran"). A comment was signified by a C in col 1. Otherwise cols 1-5 were used for statement labels (the things you could GOTO), col 6 for a statement continuation indicator, cols 7-72 for statement text, and cols 73-80 for card sequence numbers. From pengyu.ut at gmail.com Fri Jan 1 20:36:24 2010 From: pengyu.ut at gmail.com (Peng Yu) Date: Fri, 1 Jan 2010 19:36:24 -0600 Subject: Exception as the primary error handling mechanism? In-Reply-To: <50697b2c0912312124n7f989568t13e5a1dd8d883e9c@mail.gmail.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <50697b2c0912312124n7f989568t13e5a1dd8d883e9c@mail.gmail.com> Message-ID: <366c6f341001011736u3d5adb1ek9357af29ce060d7c@mail.gmail.com> On Thu, Dec 31, 2009 at 11:24 PM, Chris Rebert wrote: > On Thu, Dec 31, 2009 at 8:47 PM, Peng Yu wrote: >> I observe that python library primarily use exception for error >> handling rather than use error code. >> >> In the article API Design Matters by Michi Henning >> >> Communications of the ACM >> Vol. 52 No. 5, Pages 46-56 >> 10.1145/1506409.1506424 >> http://cacm.acm.org/magazines/2009/5/24646-api-design-matters/fulltext >> >> It says "Another popular design flaw?namely, throwing exceptions for >> expected outcomes?also causes inefficiencies because catching and >> handling exceptions is almost always slower than testing a return >> value." >> >> My observation is contradicted to the above statement by Henning. If >> my observation is wrong, please just ignore my question below. >> >> Otherwise, could some python expert explain to me why exception is >> widely used for error handling in python? Is it because the efficiency >> is not the primary goal of python? > > Correct; programmer efficiency is a more important goal for Python instead. > Python is ~60-100x slower than C;[1] if someone is worried by the > inefficiency caused by exceptions, then they're using completely the > wrong language. Could somebody let me know how the python calls and exceptions are dispatched? Is there a reference for it? From wolftracks at invalid.com Fri Jan 1 20:37:40 2010 From: wolftracks at invalid.com (W. eWatson) Date: Fri, 01 Jan 2010 17:37:40 -0800 Subject: Potential Conflicts by Installing Two Versions of Python (Windows)? Message-ID: I suspect that if one installs v2.4 and 2.5, or any two versions, that one will dominate, or there will be a conflict. I suppose it would not be possible to choose which one should be used. Comments? From wuwei23 at gmail.com Fri Jan 1 20:54:36 2010 From: wuwei23 at gmail.com (alex23) Date: Fri, 1 Jan 2010 17:54:36 -0800 (PST) Subject: Potential Conflicts by Installing Two Versions of Python (Windows)? References: Message-ID: On Jan 2, 11:37?am, "W. eWatson" wrote: > I suspect that if one installs v2.4 and 2.5, or any two versions, that > one will dominate, or there will be a conflict. ?I suppose it would not > be possible to choose which one should be used. Comments? I suspect that you're not the first person to ask this question. I suppose it would not be possible to search the group and find previous advice given on this subject. http://u.nu/4ape4 Is phrasing it as an implicit shortcoming of Python supposed to spur others to comment? Perhaps that's the problem you're having searching: try keywords that represent what you're looking for, not what you're naively expecting. From benjamin.kaplan at case.edu Fri Jan 1 20:57:13 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Fri, 1 Jan 2010 20:57:13 -0500 Subject: Potential Conflicts by Installing Two Versions of Python (Windows)? In-Reply-To: References: Message-ID: The convention (more used among Unix variants but I guess the same thing applies to Windows if you're setting the system path) is that running "python" from the command line will give you the most recently installed one. If you want to specify a version, it would be "python24" or "python25". Each version of Python is installed in it's own directory and maintains its own path so there aren't any problems with stdlib conflicts. As far as Windows file associations go, I suppose the most recent install would dominate though you could always reset the associations yourself. On Fri, Jan 1, 2010 at 8:37 PM, W. eWatson wrote: > I suspect that if one installs v2.4 and 2.5, or any two versions, that one > will dominate, or there will be a conflict. ?I suppose it would not be > possible to choose which one should be used. Comments? > -- > http://mail.python.org/mailman/listinfo/python-list > From benjamin.kaplan at case.edu Fri Jan 1 20:57:34 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Fri, 1 Jan 2010 20:57:34 -0500 Subject: Potential Conflicts by Installing Two Versions of Python (Windows)? In-Reply-To: References: Message-ID: On Fri, Jan 1, 2010 at 8:37 PM, W. eWatson wrote: > I suspect that if one installs v2.4 and 2.5, or any two versions, that one > will dominate, or there will be a conflict. ?I suppose it would not be > possible to choose which one should be used. Comments? > -- > http://mail.python.org/mailman/listinfo/python-list > The convention (more used among Unix variants but I guess the same thing applies to Windows if you're setting the system path) is that running "python" from the command line will give you the most recently installed one. If you want to specify a version, it would be "python24" or "python25". Each version of Python is installed in it's own directory and maintains its own path so there aren't any problems with stdlib conflicts. As far as Windows file associations go, I suppose the most recent install would dominate though you could always reset the associations yourself. From cousinstanley at gmail.com Fri Jan 1 21:01:04 2010 From: cousinstanley at gmail.com (Cousin Stanley) Date: Sat, 2 Jan 2010 02:01:04 +0000 (UTC) Subject: Dynamic text color References: Message-ID: > John Posner wrote .... > .... > I've posted a complete solution .... > > http://cl1p.net/jjp_dynamic_text_color/. John .... Thanks for posting your solution to Dave McCormick's query about colorizing text .... I was not familiar with the re.finditer method for searching strings or with using tags in Tkinter Text widgets to change text attributes .... Instead of binding the Text widget to events to call the colorizer on each key-stroke, I used a button callback .... http://cl1p.net/cs_static_text_color This provides a static one-shot colorization after all of the text is entered and should save a few cpu cycles although it's not as nifty as the dynamic process .... I used a dictionary instead of a list for position data and also added a { term : color } dictionary so that words other than color names can be hi-lighted .... -- Stanley C. Kitching Human Being Phoenix, Arizona From python at mrabarnett.plus.com Fri Jan 1 21:47:52 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 02 Jan 2010 02:47:52 +0000 Subject: Significant whitespace In-Reply-To: <4B3E70FC.1010702@gmail.com> References: <4B3E70FC.1010702@gmail.com> Message-ID: <4B3EB3D8.6030603@mrabarnett.plus.com> Dan Stromberg wrote: > > I put together a page about significant whitespace (and the lack thereof). > > You're invited to check it out: > > http://stromberg.dnsalias.org/~dstromberg/significant-whitespace.html > You might also want to mention that programmers tend to indent anyway for clarity. From roy at panix.com Fri Jan 1 22:56:16 2010 From: roy at panix.com (Roy Smith) Date: Fri, 01 Jan 2010 22:56:16 -0500 Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> <3db95947-1e35-4bd1-bd4c-37df646f9ed5@v25g2000yqk.googlegroups.com> Message-ID: In article <3db95947-1e35-4bd1-bd4c-37df646f9ed5 at v25g2000yqk.googlegroups.com>, John Machin wrote: > On Jan 2, 10:29?am, Roy Smith wrote: > > > > > To address your question more directly, here's a couple of ways Fortran > > treated whitespace which would surprise the current crop of > > Java/PHP/Python/Ruby programmers: > > > > 1) Line numbers (i.e. the things you could GOTO to) were in column 2-7 > > (column 1 was reserved for a comment indicator). ?This is not quite > > significant whitespace, it's more like significant indentation. > > That would also surprise former FORTRAN programmers (who rarely > referred to the language as "Fortran"). A comment was signified by a C > in col 1. Otherwise cols 1-5 were used for statement labels (the > things you could GOTO), col 6 for a statement continuation indicator, > cols 7-72 for statement text, and cols 73-80 for card sequence numbers. My apologies, you are correct. The long disused neural pathways have started to degrade. Trust me, I really did write FORTRAN. On punch cards even. From rogerb at rogerbinns.com Sat Jan 2 00:20:52 2010 From: rogerb at rogerbinns.com (Roger Binns) Date: Fri, 01 Jan 2010 21:20:52 -0800 Subject: Safe file I/O to shared file (or SQLite) from multi-threaded web server In-Reply-To: <7q7ap3F3u36s0U1@mid.uni-berlin.de> References: <7q7ap3F3u36s0U1@mid.uni-berlin.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Diez B. Roggisch wrote: > AFAIK, sqlite ensures process-serialization via locking, and threads > synchronize themselves as well. SQLite versions prior to 3.5 did not support using the same connection or cursors in different threads. (You needed to allocate, use, and close all in the same thread.) Since then SQLite objects can be used in any thread you want at any time. The SQLite error handling API is not threadsafe and requires a lock to be held otherwise you can get incorrect errors or worst case program crashes. The sqlite3/pysqlite code does not hold that lock (API introduced in SQLite 3.6.5) so you are only safe if you continue to only use objects in the same thread. If you use APSW then you can use any SQLite object at any time in any thread (it holds the lock amongst other things). > So you shouldn't need to worry at all. The main gotcha is that SQLite uses file locking and the default behaviour when unable to get a lock is to immediately return an error. SQLite does have an API to specify how long it should wait to acquire the lock (it keeps retrying until the time expires). sqlite3/pysqlite only lets you specify this maximum time when opening the connection and defaults to 5 seconds. On a busy server this may be too short so you'll end up getting busy errors. (Remember that writes require at least two disk syncs and that the default behaviour for Linux is to flush all outstanding writes not just for the file requested.) If you use APSW then you get default SQLite behaviour and two APIs - one lets you set/change the timeout period and the other lets you install your own busy handler which can do whatever it wants in order to prod things along. (Disclosure: I am the author of APSW.) Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAks+17QACgkQmOOfHg372QSiCwCgpr6fSOr6UcUUZqTDoFA4RBcK zb8An21zZCr30AQ7VGP/Q/CsQ3z+2EVs =55MC -----END PGP SIGNATURE----- From donn.ingle at gmail.com Sat Jan 2 00:56:15 2010 From: donn.ingle at gmail.com (Donn) Date: Sat, 2 Jan 2010 07:56:15 +0200 Subject: Significant whitespace In-Reply-To: <4B3E70FC.1010702@gmail.com> References: <4B3E70FC.1010702@gmail.com> Message-ID: <201001020756.15385.donn.ingle@gmail.com> On Saturday 02 January 2010 00:02:36 Dan Stromberg wrote: > I put together a page about significant whitespace (and the lack thereof). The only thing about Python's style that worries me is that it can't be compressed like javascript can*, and perhaps that will prevent it becoming a browser-side language one day. How I'd love to code PyQuery instead of JQuery! * Or am I wrong? \d From clp2 at rebertia.com Sat Jan 2 01:53:10 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Fri, 1 Jan 2010 22:53:10 -0800 Subject: Significant whitespace In-Reply-To: <201001020756.15385.donn.ingle@gmail.com> References: <4B3E70FC.1010702@gmail.com> <201001020756.15385.donn.ingle@gmail.com> Message-ID: <50697b2c1001012253l3f3a166fiab496e348b221f34@mail.gmail.com> On Fri, Jan 1, 2010 at 9:56 PM, Donn wrote: > On Saturday 02 January 2010 00:02:36 Dan Stromberg wrote: >> I put together a page about significant whitespace (and the lack thereof). > The only thing about Python's style that worries me is that it can't be > compressed like javascript can*, and perhaps that will prevent it becoming a > browser-side language one day. How I'd love to code PyQuery instead of JQuery! > > * Or am I wrong? Not to my knowledge, but CPython's bytecode format could be used to similar effect. However, the larger and much more daunting obstacle to client-side Python would be that it's not sandboxed or secured like JavaScript is, although there are some people working on it. Cheers, Chris -- http://blog.rebertia.com From aahz at pythoncraft.com Sat Jan 2 01:56:46 2010 From: aahz at pythoncraft.com (Aahz) Date: 1 Jan 2010 22:56:46 -0800 Subject: Raw string substitution problem References: <931375.44828.qm@web58902.mail.re1.yahoo.com> <7p0q09F6qiU1@mid.individual.net> <7p2juvFu8tU1@mid.individual.net> Message-ID: In article <7p2juvFu8tU1 at mid.individual.net>, Gregory Ewing wrote: >MRAB wrote: >> >> In simple cases you might be replacing with the same string every time, >> but other cases you might want the replacement to contain substrings >> captured by the regex. > >But you can give it a function that has access to the match object and >can produce whatever replacement string it wants. Assuming I remember correctly, the function capability came after the replacement capability. I think that breaking replacement would be a Bad Idea. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From aahz at pythoncraft.com Sat Jan 2 02:02:50 2010 From: aahz at pythoncraft.com (Aahz) Date: 1 Jan 2010 23:02:50 -0800 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4B3DCFAB.3030909@v.loewis.de> Message-ID: In article <4B3DCFAB.3030909 at v.loewis.de>, Martin v. Loewis wrote: > >Notice that in cases where the failure may be expected, Python >also offers variants that avoid the exception: >- if you look into a dictionary, expecting that a key may not > be there, a regular access, d[k], may give a KeyError. However, > alternatively, you can use d.get(k, default) which raises no > exception, and you can test "k in d" in advance. >- if you open a file, not knowing whether it will be there, > you get an IOError. However, you can use os.path.exists in > advance to determine whether the file is present (and create > it if it's not). But you *still* need to catch IOError: someone might delete the file after the test. Figuring out how to deal with race conditions is one of the main reasons Alex Martelli advocates EAFP over LBYL. Of course, in the real world, you often end up wanting to do it both ways. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From r.grimm at science-computing.de Sat Jan 2 02:18:38 2010 From: r.grimm at science-computing.de (Rainer Grimm) Date: Fri, 1 Jan 2010 23:18:38 -0800 (PST) Subject: multivariable assignment References: Message-ID: <347e6dcf-1437-4d1f-a563-c9d865fc9eb9@j4g2000yqe.googlegroups.com> On Dec 31 2009, 5:13?pm, davidj411 wrote: > I am not sure why this behavior is this way. > at beginning of script, i want to create a bunch of empty lists and > use each one for its own purpose. > however, updating one list seems to update the others. > > >>> a = b = c = [] > >>> a.append('1') > >>> a.append('1') > >>> a.append('1') > >>> c > ['1', '1', '1'] > >>> a > ['1', '1', '1'] > >>> b > > ['1', '1', '1'] a, b and c are the same objects. You can check the object identity by >>> a = b = c = [] >>> print id(a),id(b),id(c) 3083044140 3083044140 3083044140 >>> a is b is c True Greetings From tjreedy at udel.edu Sat Jan 2 02:22:37 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 02 Jan 2010 02:22:37 -0500 Subject: Potential Conflicts by Installing Two Versions of Python (Windows)? In-Reply-To: References: Message-ID: On 1/1/2010 8:57 PM, Benjamin Kaplan wrote: > As far as Windows file associations go, I suppose the most recent > install would dominate though you could always reset the associations > yourself. The Windows installer asks whether one wants the about-to-be-installed version to capture the associations or not. From apt.shansen at gmail.com Sat Jan 2 03:05:56 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Sat, 2 Jan 2010 00:05:56 -0800 Subject: Exception as the primary error handling mechanism? In-Reply-To: <366c6f341001011736u3d5adb1ek9357af29ce060d7c@mail.gmail.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <50697b2c0912312124n7f989568t13e5a1dd8d883e9c@mail.gmail.com> <366c6f341001011736u3d5adb1ek9357af29ce060d7c@mail.gmail.com> Message-ID: <7a9c25c21001020005l1e1feff5head73ee0d7789705@mail.gmail.com> On Fri, Jan 1, 2010 at 5:36 PM, Peng Yu wrote: > >> Otherwise, could some python expert explain to me why exception is > >> widely used for error handling in python? Is it because the efficiency > >> is not the primary goal of python? > > > > Correct; programmer efficiency is a more important goal for Python > instead. > > Python is ~60-100x slower than C;[1] if someone is worried by the > > inefficiency caused by exceptions, then they're using completely the > > wrong language. > > Could somebody let me know how the python calls and exceptions are > dispatched? Is there a reference for it? > I don't quite understand what you're asking here, but it sounds almost like you're looking at the question from an incorrect POV. "Exceptions" are a general sort of concept in computer science and various computer programming languages, but they are not at all equal from one language to another. The document you referenced was speaking to a particular implementation of the concept, and speaking to particular characteristics of that language's implementation. Even though its not talking just about say, C, C#, Java, or anything -- its speaking from a certain POV of a certain classes of languages. In Python, setting up an exception -- the 'try' clause -- costs virtually nothing. Its about equivalent to having a 'pass' statement in there. If you do a test for every iteration of some activity, you're incurring a non-negligable cost each time. If you're performing an action and "usually" (to varying definitions of 'usually'), it's going to succeed-- then that test will result in far more cost in time then using a try/except clause in Python. Because in the implementation of exceptions in Python, you only pay a more expensive cost /if/ that exception is thrown and handled. If its very likely that in a situation an exception would be thrown, then yes-- then you should probably test first... if that exception-catch is so expensive as to be important to your. In most cases, its not. In the vast majority of cases, this is premature optimization and often adds additional weight to the test as you have to protect against race conditions. (As an aside, in many cases using exceptions actually helps you in a wider problem of preventing race conditions. Its not a cure-all by any means, but it helps) If someone specifies a file, the chances are-- the file is there. Its cheaper for you to just try to open it then to test if its there first, because if you do the test? Then the likely circumstance (the file exists) is running code to test that case /every time/. Whereas if you just try to open it, in the likely circumstance -- it works. In the exceptional circumstance, it might cost more then if you had tested first... but that's an exceptional circumstance and therefore more rare. --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From xahlee at gmail.com Sat Jan 2 04:14:12 2010 From: xahlee at gmail.com (Xah Lee) Date: Sat, 2 Jan 2010 01:14:12 -0800 (PST) Subject: Xah's Edu Corner: Teach Ourself Programing In Ten Years? Message-ID: <1035ce5d-914e-42c0-8e9c-a349a6d55e55@v25g2000yqk.googlegroups.com> On Dec 25 2009, 12:44 am, r... at rpw3.org (Rob Warnock) wrote: > jos... at corporate-world.lisp.de wrote: > > +--------------- > | p... at informatimago.com (Pascal J. Bourguignon) wrote: > | > LOL Yeah right! Give gavino ten years of rest to let > | > his unconscious mind work on it! > | > | Norvig's 'Teach Yourself Programming in Ten Years' ( > |http://norvig.com/21-days.html > | ) gets a new meaning... > +--------------- See: ? The Condition of Industrial Programers http://xahlee.org/UnixResource_dir/writ/it_programers.html plain text version follows. ------------------------------ The Condition of Industrial Programers Xah Lee, 2006-05 Before i stepped into the computing industry, my first industrial programing experience was at Wolfram Research Inc as a intern in 1995. (Wolfram Research is famously known for their highly successful flagship product Mathematica) I thought, that the programers at Wolfram are the world's top mathematicians, gathered together to research and decide and write a extremely advanced technology. But i realized it is not so. Not at all. In fact, we might say it's just a bunch of Ph Ds (or equivalent experience). The people there, are not unlike average white-collar Joes. Each working individually. And, fights and bouts of arguments between co-workers are not uncommon. Sometimes downright ugly in emails. Almost nothing is as i naively imagined, as if some world's top mathematicians are gathered together there, daily to confer and solve the world's top problems as in some top secret government agency depicted in movies. Well, that was my introduction to the industry. The bulk of my surprise is due to my naiveness and inexperience of the industry, of any industry, as i was just a intern and this is my first experience seeing how the real world works. After Wolfram, after a couple of years i went into the web programing industry in 1998, using unix, Perl, Apache, Java, database technologies, in the center of world's software technology the Silicon Valley. My evaluation of industrial programers and how software are written is a precipitous fall from my observations at Wolfram. In the so-called Info Tech industry, the vast majority of programers are poorly qualified. I learned this from my colleagues, and in dealing with programers from other companies, service providers, data centers, sys admins, API gateways, and duties of field tutoring. I didn't think i have very qualified expertise in what i do, but the reality i realized is that most are far lesser than me, and that is the common situation. That they have no understanding of basic mathematics such as trigonometry or calculus. Most have no interest in math whatsoever, and would be hard pressed for them to explain what is a ?algorithm?. I have always thought, that programing X software of field Y usually means that the programers are thoroughly fluent in languages, protocols, tools of X, and also being a top expert in field of Y. But to my great surprise, the fact is that that is almost never the case. In fact, most of the time the programers simply just had to learn a language, protocol, software tool, right at the moment as he is trying to implement a software for a field he never had experience in. I myself had to do jobs half of the time i've never done before. Constantly I'm learning new languages, protocols, systems, tools, APIs, other rising practices and technologies, reading semi-written or delve into non-existent docs. It is the norm in the IT industry, that most products are really produces of learning experiences. Extremely hurried grasping of new technologies in competition with deadlines. There is in fact little actual learning going on, as there are immense pressure to simply ?get it to (demonstrably) work? and ship it. Thinking back, in fact the Wolfram people are the most knowledgeable and inquisitive people i've met as colleagues, by far. What prompted me to write this essay is after reading the essay Teach Yourself Programming in Ten Years by Peter Norvig, 2001, at http://www.norvig.com/21-days.html. In which, the LISP dignitary Peter Norvig derides the widely popular computing books in the name of Teaching Yourself X In (Fast) Days. Although i agree with his sentiment that a language or technology takes time to master and use well, that these books form somewhat of a damaging fad and subtly multiply ignorance, but he fails to address the main point, that is: the cause of the popularity of such books, and how to remedy the situation. When you work in the industry, and are given a responsibility of coding in some new language the company decided to use, or emerging protocol (such as voice-chat protocols or cellphone internet), or your engineering group adopted a new team coding/reviewing process, you are not going to tell you boss ?nah, i want to do a good job so i'll study the issue a few months before i contribute?. Chances are, you are going to run out and buy a copy of ?XYZ in 7 days?, and complete the job in a way satisfactorily to your company, as well feeling proud of your abilities in acquiring new material. To see this in a different context, suppose you need to pass a important Math XYZ exam or review in your career or get a certificate, but you don't remember your Math XYZ. You will likely, run out and get a ?Math XYZ for Dummies?. Chances are, the book will indeed help you, and you will pass your exam or interview, and actually have learned something about XYZ, but never looked at Math XYZ squarely again. These books are the bedrock of the industry. It is not because people are impatient, or that they wish to hurry, but rather, it is the condition of the IT industry, in the same way modern society drives people to live certain life styles. No amount of patience or proselytization can right this, except that we change the industry's practice of quickly churning out bug-ridden software products to beat competitors. Companies do that due to market forces, and the market forces is a result of how people and organizations actually choose to purchase software. In my opinion, a solution to this is by installing the concept of responsible licenses. Please see this essay Responsible Software Licensing and spread the word. Xah ? http://xahlee.org/ ? From sanneh27 at hotmail.com Sat Jan 2 04:52:45 2010 From: sanneh27 at hotmail.com (baboucarr sanneh) Date: Sat, 2 Jan 2010 09:52:45 +0000 Subject: HAppy New Year Message-ID: Hi guys, Jus want to wish you a happy new year to u all.. 1.Copy the content below and Paste it on a Notepad.2 . Use replace all (Ctrl + H)3. Type in ?6? in the ?find what? column 4. Type in _ (underscore) in the ?replace with? column 5. Click on replace all. 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666996666699669966999999996699666669966666669966666699669999999966996666996666666666666696666666666666666666666 666996666699669966999999996699666669966666666996666996669999999966996666996666666666666999666666666666666666666 666996666699669966996666666699666669966666666699669966669966669966996666996666666666669969966666666666666666666 666996696699669966999999996699999999966666666669999666669966669966996666996666666666699666996666666666666666666 666996999699669966999999996699999999966666666666996666669966669966996666996666666666999999999666666666666666666 666999969999669966666666996699666669966666666666996666669966669966996666996666666669999999999966666666666666666 666999666999669966999999996699666669966666666666996666669999999966996666996666666699666666666996666666666666666 666996666699669966999999996699666669966666666666996666669999999966999999996666666996666666666699666666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 669966666996666666669666666666999999996699999999669966666699666666669966666996699999999669966666996666666666666 669966666996666666699966666666999999996699999999666996666996666666669996666996699999999669966666996666666666666 669966666996666666996996666666996666996699666699666699669966666666669999666996699666666669966666996666666666666 669999999996666669966699666666999999996699999999666669999666666666669969966996699999999669966966996666666666666 669999999996666699999999966666999999996699999999666666996666666666669966996996699999999669969996996666666666666 669966666996666999999999996666999666666699666666666666996666666666669966699996699666666669999699996666666666666 669966666996669966666666699666996666666699666666666666996666666666669966669996699999999669996669996666666666666 669966666996699666666666669966996666666699666666666666996666666666669966666996699999999669966666996666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666996666669966999999996666666669666666666999999996666666699999996699999999666999666699999999666666666666666666 666699666699666999999996666666699966666666999999996666666699999999699666699666699666699666699666666666666666666 666669966996666996666666666666996996666666996666996666666666666699699966699666699666699966699666666666666666666 666666999966666999999996666669966699666666999999996666666666666996699696699666699666699696699666666666666666666 666666699666666999999996666699999999966666999999996666666666669966699669699666699666699669699666666666666666666 666666699666666996666666666999999999996666996996666666666666996666699666999666699666699666999666666666666666666 666666699666666999999996669966666666699666996699666666666699999999699666699666699666699666699666666666666666666 666666699666666999999996699666666666669966996666996666666699999999699999999666999966699999999666666666666666666 66666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 Regards, Baboucarr. _________________________________________________________________ Windows Live: Friends get your Flickr, Yelp, and Digg updates when they e-mail you. http://www.microsoft.com/middleeast/windows/windowslive/see-it-in-action/social-network-basics.aspx?ocid=PID23461::T:WLMTAGL:ON:WL:en-xm:SI_SB_3:092010 -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul at subsignal.org Sat Jan 2 05:13:51 2010 From: paul at subsignal.org (=?ISO-8859-1?Q?Paul_K=F6lle?=) Date: Sat, 02 Jan 2010 11:13:51 +0100 Subject: Trying to run a sudo command from script In-Reply-To: References: Message-ID: Am 01.01.2010 23:55, schrieb Kent Tenney: > Howdy, Hi Kent, > A script running as a regular user sometimes wants > to run sudo commands. > > It gets the password with getpass. > pw = getpass.getpass() > > I've fiddled a bunch with stuff like > proc = subprocess.Popen('sudo touch /etc/foo'.split(), stdin=subprocess.PIPE) > proc.communicate(input=pw) If you don't use shell=True you have to provide the full path to commands (and split command and parameters as you do). So eather of this works for me: p = Popen('/usr/bin/sudo /usr/bin/touch /tmp/foo.txt'.split(), stdin=PIPE, stdout=PIPE) p = Popen('/usr/bin/sudo /usr/bin/touch /tmp/foo2.txt', stdin=PIPE, stdout=PIPE, shell=True) The bad news is: It this gives me a password promt inside the interactive interpreter. Seems you can't catch stdout this way. hth Paul > > getting assorted errors with all variations I try. > > Googling says use pexpect, but I'd prefer a stdlib solution. > > Any help appreciated. > > Thanks, > Kent From steve at REMOVE-THIS-cybersource.com.au Sat Jan 2 05:42:17 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 02 Jan 2010 10:42:17 GMT Subject: Bare Excepts References: <4dc0cfea0912300209x15e5bc96nd6e6224d89df0638@mail.gmail.com> <4B3B4FE5.8070605@sequans.com> <4dc0cfea0912300509v7f9bb367jf33bb63736407ed@mail.gmail.com> <4dc0cfea0912300512h43a4fdb1oa585a36391de341e@mail.gmail.com> <6944e440-c797-439d-9bf9-b61a05a3e26c@k23g2000yqa.googlegroups.com> <878wck2ojd.fsf@benfinney.id.au> <4dc0cfea0912301927q16445c59r1fb8594ee0f04d24@mail.gmail.com> Message-ID: <034f1009$0$1277$c3e8da3@news.astraweb.com> On Fri, 01 Jan 2010 15:27:57 -0800, myle wrote: > Why we should prefer ``if: ...'' over a ``try: ... except something: > pass'' block? We shouldn't, not in general. Often, the if test is just as expensive as actually doing it. E.g.: if x in mylist: position = mylist.index(x) else: do_something() has to search the list twice, first to see if x is there, then search it again to return the index. That does double the work needed. A better way is: try: position = mylist.index(x) except ValueError: do_something() unless you expect that x is often missing: setting up the try block is much faster than the test, but catching the exception is much, much slower. So long as exceptions are rare, it is better to use the try version (also known as "Better to ask forgiveness than permission"). But if exceptions are common, it may be better to use the if version ("Look before you leap"). The try version is also better whenever there could be a race condition. For example, when opening a file, you might be tempted to do this: if os.path.exists(filename): f = open(filename, 'r') text = f.read() else: print "file missing" but that is dangerous in a multi-processing computer system (which nearly all computers are these days). The problem is that in the fraction of a second after you test that the file exists, some other process might come along and delete it and the open call will then fail. So to be safe, you have to do this: if os.path.exists(filename): try: f = open(filename, 'r') except IOError: print "file missing" else: text = f.read() else: print "file missing" But of course now the test is redundant, and all you really need is the try...except block: try: f = open(filename, 'r') except IOError: print "file missing" else: text = f.read() (Note: the above is still not entirely robust. For example, if you try to read a file that you don't have permission to read, it wrongly reports it is missing instead of a permissions error. To make this more robust, you need to inspect the exception and check the error code.) -- Steven From duncan.booth at invalid.invalid Sat Jan 2 05:42:39 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 2 Jan 2010 10:42:39 GMT Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> Message-ID: Donn wrote: > On Saturday 02 January 2010 00:02:36 Dan Stromberg wrote: >> I put together a page about significant whitespace (and the lack >> thereof). > The only thing about Python's style that worries me is that it can't > be compressed like javascript can*, and perhaps that will prevent it > becoming a browser-side language one day. How I'd love to code PyQuery > instead of JQuery! You can do simple white-space removal as easily in Python as in Javascript, you just need to maintain a minimal indentation and as you don't need Javascript's curly braces there probably isn't must difference especially once the code has been gzipped for transfer to the browser. You can already run Python client side in certain situations (e.g. Silverlight), but if you want to write Python for the browser portably today have a look at Pyjamas http://code.google.com/p/pyjamas/ . Google's Chrome also supports native code (currently just for x86 machines though they have said they intend to support ARM), so there should be some potential there to compile a Python interpreter that runs client-side in Google's sandbox: see http://plash.beasts.org/wiki/NativeClient for an early port. From victorsubervi at gmail.com Sat Jan 2 05:53:26 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 2 Jan 2010 05:53:26 -0500 Subject: Append Problem Message-ID: <4dc0cfea1001020253u6f2bf8a9i7f16c088d4ad78f7@mail.gmail.com> Hi; I have the following code snippet: print 'Original: ', catChains, '
' while i < MAXLEVEL: flag = 0 j = 0 while j < len(parents): for chain in catChains: if parents[j] == chain[len(chain)-1]: chain.append(children[j]) print '1: ', catChains, '
' catChains.append(chain) print '2: ', catChains, '
' flag = 1 j += 1 i += 1 if flag == 0: break print 'Final: ', catChains which prints this: Original: [['prodCat1'], ['prodCat2']] 1: [['prodCat1', 'prodCat3'], ['prodCat2']] 2: [['prodCat1', 'prodCat3'], ['prodCat2'], ['prodCat1', 'prodCat3']] 1: [['prodCat1', 'prodCat3', 'prodCat5'], ['prodCat2'], ['prodCat1', 'prodCat3', 'prodCat5']] 2: [['prodCat1', 'prodCat3', 'prodCat5'], ['prodCat2'], ['prodCat1', 'prodCat3', 'prodCat5'], ['prodCat1', 'prodCat3', 'prodCat5']] Final: [['prodCat1', 'prodCat3', 'prodCat5'], ['prodCat2'], ['prodCat1', 'prodCat3', 'prodCat5'], ['prodCat1', 'prodCat3', 'prodCat5']] Why is it that my append statement *deletes* elements of my tuple? The entire code follows: #! /usr/bin/python import string import cgitb; cgitb.enable() import MySQLdb import cgi import sys,os sys.path.append(os.getcwd()) from login import login import datetime, Cookie, random from particulars import title from templateFrame import top, bottom from particulars import myCookie import time import fpformat from sets import Set def makeNav(): print 'Content-type: text/html\r\n' print '\n' user, passwd, db, host = login() db = MySQLdb.connect(host, user, passwd, db) cursor= db.cursor() form = cgi.FieldStorage() store = form.getfirst('store') sql = 'select Category, Parent from categories%s;' % (store[0].upper() + store[1:]) cursor.execute(sql) children = [itm[0] for itm in cursor] parents = [itm[1] for itm in cursor] catChains = [] i = 0 while i < len(parents): if parents[i] is None: child = [] child.append(children[i]) catChains.append(child) i += 1 i = 0 MAXLEVEL = 10 print 'Original: ', catChains, '
' while i < MAXLEVEL: flag = 0 j = 0 while j < len(parents): for chain in catChains: if parents[j] == chain[len(chain)-1]: chain.append(children[j]) print '1: ', catChains, '
' catChains.append(chain) print '2: ', catChains, '
' flag = 1 j += 1 i += 1 if flag == 0: break print 'Final: ', catChains lastChain = [] lastChain.append('root') print "' makeNav() PS (Mainly, I believe, for Carsten): While programming is difficult for me, I am writing (and have pretty much finished) this (almost) fully automated shopping cart. I don't intend to write any other serious programming project. However, a shopping cart is vital to my Web design business. I will hereafter outsource my programming. But, as any good supervisor knows, one has to know how to supervise! If I can't read code, then I'm at the mercy of my programmer(s). If I can't afford to hire some top gun who can supervise, then it's my responsibility. Plus, if my programmer(s) quit, I need to step in and take over. I'm an ok businessman, and absolutely gifted in sales. And in poetry ;) beno -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From doomster at knuut.de Sat Jan 2 07:01:28 2010 From: doomster at knuut.de (Ulrich Eckhardt) Date: Sat, 02 Jan 2010 13:01:28 +0100 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <50697b2c0912312124n7f989568t13e5a1dd8d883e9c@mail.gmail.com> Message-ID: <7q8qspF3u95vnU1@mid.uni-berlin.de> Peng Yu wrote: > Could somebody let me know how the python calls and exceptions are > dispatched? Is there a reference for it? I'm not a Python expert, but I have read some parts of the implementation. Hopefully someone steps up if I misrepresent things here... In order to understand Python exception handling, take a look at various C function implementations. You will see that they commonly return a pointer to a Python object (PyObject*), even if it is a pointer to the 'None' singleton. So, a function in Python _always_ returns something, even if it is 'None'. If, at the C level, a function doesn't return anything (i.e. a C NULL pointer) that means that the function raised an exception. Checking this pointer is pretty easy, typically you check that, clean up and return NULL yourself. Further functions for manipulating the exception stack and declarations of exception types and singletons are found in pyerrors.h (in Python 2.5, at least). I mentioned an "exception stack" above, though I'm not 100% sure if that is the proper term. I think that exceptions can be stacked upon each other (e.g. an HTTPD throwing a high-level RequestError when it encounters a low- level IOError) and that that is also how the backtrace is implemented, but I'm not sure about that. Hopefully someone can confirm or correct me here and that it helped you. Cheers! Uli From steve at REMOVE-THIS-cybersource.com.au Sat Jan 2 07:03:28 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 02 Jan 2010 12:03:28 GMT Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> Message-ID: <034f2310$0$1277$c3e8da3@news.astraweb.com> On Sat, 02 Jan 2010 10:42:39 +0000, Duncan Booth wrote: > Donn wrote: > >> On Saturday 02 January 2010 00:02:36 Dan Stromberg wrote: >>> I put together a page about significant whitespace (and the lack >>> thereof). >> The only thing about Python's style that worries me is that it can't be >> compressed like javascript can*, and perhaps that will prevent it >> becoming a browser-side language one day. How I'd love to code PyQuery >> instead of JQuery! > > You can do simple white-space removal as easily in Python as in > Javascript, you just need to maintain a minimal indentation and as you > don't need Javascript's curly braces there probably isn't must > difference especially once the code has been gzipped for transfer to the > browser. I don't understand why you would bother removing whitespace, particularly if you're going to gzip the file anyway. I've just picked one of my modules at random: about 47K in size, of which about 15K is whitespace. Compressed to .bz, it is 12K, and if I strip the whitespace from it first and then compress it, it is 11K. So, even though whitespace (spaces and newlines) make up about 30% of the original source, once compressed they represent only about 8%. Given the extra complexity of stripping whitespace, I don't know that it's worthwhile. It's not like you're going to reduce the download time from three hours to one; it's more like 2 seconds to 1.5. -- Steven From deets at nospam.web.de Sat Jan 2 07:05:11 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Sat, 02 Jan 2010 13:05:11 +0100 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <50697b2c0912312124n7f989568t13e5a1dd8d883e9c@mail.gmail.com> Message-ID: <7q8r3nF3u6mmdU1@mid.uni-berlin.de> Peng Yu schrieb: > On Thu, Dec 31, 2009 at 11:24 PM, Chris Rebert wrote: >> On Thu, Dec 31, 2009 at 8:47 PM, Peng Yu wrote: >>> I observe that python library primarily use exception for error >>> handling rather than use error code. >>> >>> In the article API Design Matters by Michi Henning >>> >>> Communications of the ACM >>> Vol. 52 No. 5, Pages 46-56 >>> 10.1145/1506409.1506424 >>> http://cacm.acm.org/magazines/2009/5/24646-api-design-matters/fulltext >>> >>> It says "Another popular design flaw?namely, throwing exceptions for >>> expected outcomes?also causes inefficiencies because catching and >>> handling exceptions is almost always slower than testing a return >>> value." >>> >>> My observation is contradicted to the above statement by Henning. If >>> my observation is wrong, please just ignore my question below. >>> >>> Otherwise, could some python expert explain to me why exception is >>> widely used for error handling in python? Is it because the efficiency >>> is not the primary goal of python? >> Correct; programmer efficiency is a more important goal for Python instead. >> Python is ~60-100x slower than C;[1] if someone is worried by the >> inefficiency caused by exceptions, then they're using completely the >> wrong language. > > Could somebody let me know how the python calls and exceptions are > dispatched? Is there a reference for it? The source? http://python.org/ftp/python/2.6.4/Python-2.6.4.tgz These are really deep internals that - if they really concern you - need intensive studies, not casual reading of introductionary documents. IMHO you shouldn't worry, but then, there's a lot things you seem to care I wouldn't... :) Diez From vicente.soler at gmail.com Sat Jan 2 07:50:32 2010 From: vicente.soler at gmail.com (vsoler) Date: Sat, 2 Jan 2010 04:50:32 -0800 (PST) Subject: Endless loop Message-ID: <7e1289b1-4d6d-42e0-9325-ee4066f3e5b0@r5g2000yqb.googlegroups.com> hello, I'm learning Python and OOP, and I am confronted with a rather theoretical problem. If I run the following script: class stepper: def __getitem__(self, i): return self.data[i] X=stepper() X.data="Spam" for item in X: print item, ... what I get is S p a m which seems logical to me since the loop stops after the 4th character. However if I run class stepper: def __getitem__(self, i): return i X=stepper() X.data="Spam" for item in X: print item, ... what I get is an endless loop, starting at zero: 0 1 2 3 4 5 6 7 8 9 10 11 and so on. My question is: why does this second script not stop after printing number 3? what made the first one stop while the second one will not? Thanks for your advise Vicente Soler From martin at v.loewis.de Sat Jan 2 07:52:10 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Sat, 02 Jan 2010 13:52:10 +0100 Subject: Exception as the primary error handling mechanism? In-Reply-To: <7q8qspF3u95vnU1@mid.uni-berlin.de> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <50697b2c0912312124n7f989568t13e5a1dd8d883e9c@mail.gmail.com> <7q8qspF3u95vnU1@mid.uni-berlin.de> Message-ID: <4B3F417A.6080109@v.loewis.de> > I mentioned an "exception stack" above, though I'm not 100% sure if that is > the proper term. I think that exceptions can be stacked upon each other > (e.g. an HTTPD throwing a high-level RequestError when it encounters a low- > level IOError) and that that is also how the backtrace is implemented, but > I'm not sure about that. Not exactly. In this scenario, the IOError exception gets caught, its entire traceback discarded, and an entirely new exception RequestError gets raised (that has no connection to the original IOError anymore, unless the httpd code explicitly links the two). Instead, the traceback objects are created for a single exception. They are essentially the same as the call stack, just in reverse order (so that you get the "most recent call last" traceback output). Each traceback links to a frame object, and a next traceback object. Regards, Martin From steve at holdenweb.com Sat Jan 2 08:08:12 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 02 Jan 2010 08:08:12 -0500 Subject: Trying to run a sudo command from script In-Reply-To: References: Message-ID: Paul K?lle wrote: > Am 01.01.2010 23:55, schrieb Kent Tenney: >> Howdy, > Hi Kent, > >> A script running as a regular user sometimes wants >> to run sudo commands. >> >> It gets the password with getpass. >> pw = getpass.getpass() >> >> I've fiddled a bunch with stuff like >> proc = subprocess.Popen('sudo touch /etc/foo'.split(), >> stdin=subprocess.PIPE) >> proc.communicate(input=pw) > If you don't use shell=True you have to provide the full path to > commands (and split command and parameters as you do). So eather of this > works for me: > p = Popen('/usr/bin/sudo /usr/bin/touch /tmp/foo.txt'.split(), > stdin=PIPE, stdout=PIPE) > > p = Popen('/usr/bin/sudo /usr/bin/touch /tmp/foo2.txt', stdin=PIPE, > stdout=PIPE, shell=True) > > The bad news is: It this gives me a password promt inside the > interactive interpreter. Seems you can't catch stdout this way. > [please put your answers *after* the questions!] I don't think it so much that you can't catch stdout. Rather, sudo ensures it is talking to the user by explicitly reading the password from /dev/tty. Consequently there is no way to send the password value over the process's stdin, hence the need to use pexpect. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From taradov at gmail.com Sat Jan 2 08:18:13 2010 From: taradov at gmail.com (alexru) Date: Sat, 2 Jan 2010 05:18:13 -0800 (PST) Subject: Endless loop References: <7e1289b1-4d6d-42e0-9325-ee4066f3e5b0@r5g2000yqb.googlegroups.com> Message-ID: <0b4c8c2b-36e5-4962-bf69-7af9b175a27b@22g2000yqr.googlegroups.com> On Jan 2, 3:50?pm, vsoler wrote: > My question is: why does this second script not stop after printing > number 3? ?what made the first one stop while the second one will not? First one will raise IndexError when string is over, second one won't. From doomster at knuut.de Sat Jan 2 08:21:27 2010 From: doomster at knuut.de (Ulrich Eckhardt) Date: Sat, 02 Jan 2010 14:21:27 +0100 Subject: Endless loop References: <7e1289b1-4d6d-42e0-9325-ee4066f3e5b0@r5g2000yqb.googlegroups.com> Message-ID: <7q8virF3v0lcqU1@mid.uni-berlin.de> vsoler wrote: > class stepper: > def __getitem__(self, i): > return self.data[i] > > X=stepper() > X.data="Spam" > for item in X: > print item, > > ... what I get is S p a m which seems logical to me since the > loop stops after the 4th character. I think you're mistaking the cause and effect here, see below. > class stepper: > def __getitem__(self, i): > return i > > X=stepper() > X.data="Spam" > for item in X: > print item, > > ... what I get is an endless loop, starting at zero: 0 1 2 3 4 5 6 > 7 8 9 10 11 and so on. > > My question is: why does this second script not stop after printing > number 3? what made the first one stop while the second one will not? First thing to observe in the second case is that X's data field is not used anywhere. The field is set and then not used any further. In particular, it is not used to somehow get the length of the sequence to return. Now, "spam"[4] will raise an IndexError, thus terminating the iteration. The second implementation which only returns the index never will do that, so the iteration never stops. Uli From steve at holdenweb.com Sat Jan 2 08:24:33 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 02 Jan 2010 08:24:33 -0500 Subject: Append Problem In-Reply-To: <4dc0cfea1001020253u6f2bf8a9i7f16c088d4ad78f7@mail.gmail.com> References: <4dc0cfea1001020253u6f2bf8a9i7f16c088d4ad78f7@mail.gmail.com> Message-ID: Victor Subervi wrote: > Hi; > I have the following code snippet: > > print 'Original: ', catChains, '
' > while i < MAXLEVEL: > flag = 0 > j = 0 > while j < len(parents): > for chain in catChains: > if parents[j] == chain[len(chain)-1]: > chain.append(children[j]) > print '1: ', catChains, '
' > catChains.append(chain) > print '2: ', catChains, '
' > flag = 1 > j += 1 > i += 1 > if flag == 0: > break > print 'Final: ', catChains > > which prints this: > > Original: [['prodCat1'], ['prodCat2']] > 1: [['prodCat1', 'prodCat3'], ['prodCat2']] > 2: [['prodCat1', 'prodCat3'], ['prodCat2'], ['prodCat1', 'prodCat3']] > 1: [['prodCat1', 'prodCat3', 'prodCat5'], ['prodCat2'], ['prodCat1', > 'prodCat3', 'prodCat5']] > 2: [['prodCat1', 'prodCat3', 'prodCat5'], ['prodCat2'], ['prodCat1', > 'prodCat3', 'prodCat5'], ['prodCat1', 'prodCat3', 'prodCat5']] > Final: [['prodCat1', 'prodCat3', 'prodCat5'], ['prodCat2'], ['prodCat1', > 'prodCat3', 'prodCat5'], ['prodCat1', 'prodCat3', 'prodCat5']] > > Why is it that my append statement *deletes* elements of my tuple? The > entire code follows: > There isn't a tuple is sight there - you are dealing with lists, and you are modifying the very list you are iterating over. Try rewriting the code to create a new list from the old one (i.e. iterate over catChains and have your code append to an initially empty list called, for example, newCatChains, then finally throw the old list away with catChains = newCatChains or some such). [code snipped] > > PS (Mainly, I believe, for Carsten): While programming is difficult for > me, I am writing (and have pretty much finished) this (almost) fully > automated shopping cart. I don't intend to write any other serious > programming project. However, a shopping cart is vital to my Web design > business. I will hereafter outsource my programming. But, as any good > supervisor knows, one has to know how to supervise! If I can't read > code, then I'm at the mercy of my programmer(s). If I can't afford to > hire some top gun who can supervise, then it's my responsibility. Plus, > if my programmer(s) quit, I need to step in and take over. I'm an ok > businessman, and absolutely gifted in sales. And in poetry ;) > beno > Given that there are umpteen thousand hosting services who will let you create a web site with shopping cart included I have to question whether you are making the best use of your time here. Of course it's your time, and your decision, so please don't think I am dictating to you. While the learning you have performed in getting this system to work (for some rather questionable value of "work", I can't help feeling, but we'll overlook the horrors induced by your lack of programming experience) is a testament to your persistence (and this list's tolerance) I can't help feeling that it leaves you no better qualified to supervise programmers. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From vicente.soler at gmail.com Sat Jan 2 08:45:18 2010 From: vicente.soler at gmail.com (vsoler) Date: Sat, 2 Jan 2010 05:45:18 -0800 (PST) Subject: Endless loop References: <7e1289b1-4d6d-42e0-9325-ee4066f3e5b0@r5g2000yqb.googlegroups.com> <7q8virF3v0lcqU1@mid.uni-berlin.de> Message-ID: On 2 ene, 14:21, Ulrich Eckhardt wrote: > vsoler wrote: > > class stepper: > > ? ? def __getitem__(self, i): > > ? ? ? ? return self.data[i] > > > X=stepper() > > X.data="Spam" > > for item in X: > > ? ? print item, > > > ... what I get is ? ? S p a m ? ? which seems logical to me since the > > loop stops after the 4th character. > > I think you're mistaking the cause and effect here, see below. > > > class stepper: > > ? ? def __getitem__(self, i): > > ? ? ? ? return i > > > X=stepper() > > X.data="Spam" > > for item in X: > > ? ? print item, > > > ... what I get is an endless loop, starting at zero: ? ?0 1 2 3 4 5 6 > > 7 8 9 10 11 ?and so on. > > > My question is: why does this second script not stop after printing > > number 3? ?what made the first one stop while the second one will not? > > First thing to observe in the second case is that X's data field is not used > anywhere. The field is set and then not used any further. In particular, it > is not used to somehow get the length of the sequence to return. > > Now, "spam"[4] will raise an IndexError, thus terminating the iteration. The > second implementation which only returns the index never will do that, so > the iteration never stops. > > Uli It's clear, I now understand. Thank you From lie.1296 at gmail.com Sat Jan 2 09:10:51 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Sun, 03 Jan 2010 01:10:51 +1100 Subject: Bare Excepts In-Reply-To: <034f1009$0$1277$c3e8da3@news.astraweb.com> References: <4dc0cfea0912300209x15e5bc96nd6e6224d89df0638@mail.gmail.com> <4B3B4FE5.8070605@sequans.com> <4dc0cfea0912300509v7f9bb367jf33bb63736407ed@mail.gmail.com> <4dc0cfea0912300512h43a4fdb1oa585a36391de341e@mail.gmail.com> <6944e440-c797-439d-9bf9-b61a05a3e26c@k23g2000yqa.googlegroups.com> <878wck2ojd.fsf@benfinney.id.au> <4dc0cfea0912301927q16445c59r1fb8594ee0f04d24@mail.gmail.com> <034f1009$0$1277$c3e8da3@news.astraweb.com> Message-ID: <4b3f53ec$1@dnews.tpgi.com.au> On 1/2/2010 9:42 PM, Steven D'Aprano wrote: > On Fri, 01 Jan 2010 15:27:57 -0800, myle wrote: > >> Why we should prefer ``if: ...'' over a ``try: ... except something: >> pass'' block? > > We shouldn't, not in general. One exception (pun intended) is if the try-block have a side effect that is difficult to cleanup cleanly. From admin at daffitt.com Sat Jan 2 09:15:26 2010 From: admin at daffitt.com (David M Covey Sr.) Date: Sat, 2 Jan 2010 09:15:26 -0500 Subject: Windows 7 : any problems installing or running Python ? Message-ID: <000001ca8bb6$08391d70$18ab5850$@com> Hello Skippy, In response to your message "Windows 7 : any problems installing or running Python ?" I found posted at (http://mail.python.org/pipermail/python-list/2009-August/1215524.html), I've got to say that I can't seem to get any version of Python to work on my computer. I have a Toshiba Satellite laptop running Windows 7 Home Premium with an AMD Turion II Dual-Core processor. I've tried all of the versions listed at http://python.org/download with no success. They install fine but when I try to run the IDLE (Python GUI), it does nothing at all. Do you have any suggestions that might help me out here? I would really appreciate your input. Thank you, David M Covey Sr. admin at daffitt.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From pengyu.ut at gmail.com Sat Jan 2 10:04:24 2010 From: pengyu.ut at gmail.com (Peng Yu) Date: Sat, 2 Jan 2010 09:04:24 -0600 Subject: Exception as the primary error handling mechanism? In-Reply-To: <7q8r3nF3u6mmdU1@mid.uni-berlin.de> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <50697b2c0912312124n7f989568t13e5a1dd8d883e9c@mail.gmail.com> <7q8r3nF3u6mmdU1@mid.uni-berlin.de> Message-ID: <366c6f341001020704m6811a2d0yeca281dda62d4c3e@mail.gmail.com> On Sat, Jan 2, 2010 at 6:05 AM, Diez B. Roggisch wrote: > Peng Yu schrieb: >> >> On Thu, Dec 31, 2009 at 11:24 PM, Chris Rebert wrote: >>> >>> On Thu, Dec 31, 2009 at 8:47 PM, Peng Yu wrote: >>>> >>>> I observe that python library primarily use exception for error >>>> handling rather than use error code. >>>> >>>> In the article API Design Matters by Michi Henning >>>> >>>> Communications of the ACM >>>> Vol. 52 No. 5, Pages 46-56 >>>> 10.1145/1506409.1506424 >>>> http://cacm.acm.org/magazines/2009/5/24646-api-design-matters/fulltext >>>> >>>> It says "Another popular design flaw?namely, throwing exceptions for >>>> expected outcomes?also causes inefficiencies because catching and >>>> handling exceptions is almost always slower than testing a return >>>> value." >>>> >>>> My observation is contradicted to the above statement by Henning. If >>>> my observation is wrong, please just ignore my question below. >>>> >>>> Otherwise, could some python expert explain to me why exception is >>>> widely used for error handling in python? Is it because the efficiency >>>> is not the primary goal of python? >>> >>> Correct; programmer efficiency is a more important goal for Python >>> instead. >>> Python is ~60-100x slower than C;[1] if someone is worried by the >>> inefficiency caused by exceptions, then they're using completely the >>> wrong language. >> >> Could somebody let me know how the python calls and exceptions are >> dispatched? Is there a reference for it? > > The source? > > http://python.org/ftp/python/2.6.4/Python-2.6.4.tgz > > These are really deep internals that - if they really concern you - need > intensive studies, not casual reading of introductionary documents. IMHO you > shouldn't worry, but then, there's a lot things you seem to care I > wouldn't... :) For my own interest, I want understand the run time behavior of python and what details causes it much slower. Although people choose python for its programming efficiency, but sometimes the runtime still matters. This is an important aspect of the language. I'm wondering this is not even documented. Why everybody has to go to the source code to understand it? Are you sure that there is no document that describes how python is working internally (including exceptions)? From martin at v.loewis.de Sat Jan 2 10:17:53 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Sat, 02 Jan 2010 16:17:53 +0100 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <50697b2c0912312124n7f989568t13e5a1dd8d883e9c@mail.gmail.com> <7q8r3nF3u6mmdU1@mid.uni-berlin.de> Message-ID: <4B3F63A1.1060201@v.loewis.de> > For my own interest, I want understand the run time behavior of python > and what details causes it much slower. Although people choose python > for its programming efficiency, but sometimes the runtime still > matters. This is an important aspect of the language. I'm wondering > this is not even documented. Why everybody has to go to the source > code to understand it? There are two answers to this question: a) Because the source is the most precise and most complete way of documenting it. Any higher-level documentation would necessarily be incomplete. b) Because nobody has contributed documentation. The two causes correlate: because writing documentation of VM internals takes a lot of effort and is of questionable use, nobody has written any. > Are you sure that there is no document that describes how python is > working internally (including exceptions)? Such documents certainly exist, but not as part of the Python distribution. See http://wiki.python.org/moin/CPythonVmInternals for one such document. Regards, Martin From victorsubervi at gmail.com Sat Jan 2 10:26:15 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 2 Jan 2010 10:26:15 -0500 Subject: Append Problem In-Reply-To: References: <4dc0cfea1001020253u6f2bf8a9i7f16c088d4ad78f7@mail.gmail.com> Message-ID: <4dc0cfea1001020726v50be38d4s5096c53d7e4c4152@mail.gmail.com> There isn't a tuple is sight there - you are dealing with lists, and you > are modifying the very list you are iterating over. > > Try rewriting the code to create a new list from the old one (i.e. > iterate over catChains and have your code append to an initially empty > list called, for example, newCatChains, then finally throw the old list > away with > > catChains = newCatChains > > or some such). > Thanks! > > PS (Mainly, I believe, for Carsten): While programming is difficult for > > me, I am writing (and have pretty much finished) this (almost) fully > > automated shopping cart. I don't intend to write any other serious > > programming project. However, a shopping cart is vital to my Web design > > business. I will hereafter outsource my programming. But, as any good > > supervisor knows, one has to know how to supervise! If I can't read > > code, then I'm at the mercy of my programmer(s). If I can't afford to > > hire some top gun who can supervise, then it's my responsibility. Plus, > > if my programmer(s) quit, I need to step in and take over. I'm an ok > > businessman, and absolutely gifted in sales. And in poetry ;) > > beno > > > Given that there are umpteen thousand hosting services who will let you > create a web site with shopping cart included I have to question whether > you are making the best use of your time here. Of course it's your time, > and your decision, so please don't think I am dictating to you. > It doesn't give one the control upon which I insist. No. That's not an option. > > While the learning you have performed in getting this system to work > (for some rather questionable value of "work", I can't help feeling, but > we'll overlook the horrors induced by your lack of programming > experience) is a testament to your persistence (and this list's > tolerance) I can't help feeling that it leaves you no better qualified > to supervise programmers. > Ah, but you aren't a businessman, are you? You wouldn't know... beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at holdenweb.com Sat Jan 2 10:46:43 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 02 Jan 2010 10:46:43 -0500 Subject: Append Problem In-Reply-To: <4dc0cfea1001020726v50be38d4s5096c53d7e4c4152@mail.gmail.com> References: <4dc0cfea1001020253u6f2bf8a9i7f16c088d4ad78f7@mail.gmail.com> <4dc0cfea1001020726v50be38d4s5096c53d7e4c4152@mail.gmail.com> Message-ID: <4B3F6A63.2080509@holdenweb.com> Victor Subervi wrote: [...] > While the learning you have performed in getting this system to work > (for some rather questionable value of "work", I can't help feeling, but > we'll overlook the horrors induced by your lack of programming > experience) is a testament to your persistence (and this list's > tolerance) I can't help feeling that it leaves you no better qualified > to supervise programmers. > > > Ah, but you aren't a businessman, are you? You wouldn't know... > beno See the .sig. Of course I'm a businessman. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From nobody at nowhere.com Sat Jan 2 10:59:14 2010 From: nobody at nowhere.com (Nobody) Date: Sat, 02 Jan 2010 15:59:14 +0000 Subject: Potential Conflicts by Installing Two Versions of Python (Windows)? References: Message-ID: On Fri, 01 Jan 2010 17:37:40 -0800, W. eWatson wrote: > I suspect that if one installs v2.4 and 2.5, or any two versions, that > one will dominate, or there will be a conflict. I suppose it would not > be possible to choose which one should be used. Comments? The only inherent conflict is that you can only associate the .py extension with one version for any particular user. This only matters if you need to be able to run Python files as if they were executables, e.g. by double-clicking them in Explorer. If you invoke Python files by explicitly specifying the Python interpreter, there isn't a problem. From victorsubervi at gmail.com Sat Jan 2 10:59:47 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 2 Jan 2010 10:59:47 -0500 Subject: Append Problem In-Reply-To: <4B3F6A63.2080509@holdenweb.com> References: <4dc0cfea1001020253u6f2bf8a9i7f16c088d4ad78f7@mail.gmail.com> <4dc0cfea1001020726v50be38d4s5096c53d7e4c4152@mail.gmail.com> <4B3F6A63.2080509@holdenweb.com> Message-ID: <4dc0cfea1001020759x475a4dccu2312e6ef1eac80a2@mail.gmail.com> On Sat, Jan 2, 2010 at 10:46 AM, Steve Holden wrote: > > See the .sig. Of course I'm a businessman. > Nice site! In that case, respectfully I disagree! beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From vnewel at invalid.invalid Sat Jan 2 11:27:48 2010 From: vnewel at invalid.invalid (VanceE) Date: Sat, 2 Jan 2010 11:27:48 -0500 Subject: assert type([]) == type(()) Message-ID: Just curious. type([]) == type(()) is False as expected and assert type([]) == type(()) throws an AssertionError as expected. However the following is not an error for x in []: assert type(x) == type(()) I expected an AssertionError but get no errors at all. Any explaination? BTW I'm using Python2.5 and tried type(None) == type(()) which is False as expected. TIA From jrh at joshh.co.uk Sat Jan 2 11:40:14 2010 From: jrh at joshh.co.uk (Josh Holland) Date: Sat, 2 Jan 2010 16:40:14 +0000 (UTC) Subject: assert type([]) == type(()) References: Message-ID: On 2010-01-02, VanceE wrote: > for x in []: > assert type(x) == type(()) > > I expected an AssertionError but get no errors at all. > Any explaination? That loop never runs. It immediately raises a StopIteration and the body is never executed. cf. for x in []: print "In loop body" You won't see anything printed. OTOH, look at this one: for x in [[]]: # a list containing an empty list assert type(x) == type(()) That will raise the AssertionError as expected. -- Josh "dutchie" Holland http://joshh.co.uk http://twitter.com/jshholland http://identi.ca/jshholland From wojciech_mula at poczta.null.onet.pl.invalid Sat Jan 2 11:41:20 2010 From: wojciech_mula at poczta.null.onet.pl.invalid (Wojciech =?ISO-8859-2?Q?Mu=B3a?=) Date: Sat, 2 Jan 2010 17:41:20 +0100 Subject: assert type([]) == type(()) References: Message-ID: <20100102174120.050865c9.wojciech_mula@poczta.null.onet.pl.invalid> VanceE wrote: > for x in []: > assert type(x) == type(()) > > I expected an AssertionError but get no errors at all. > Any explaination? [] is an empty sequence, so your loop executes exactly 0 times. :) for x in [None]: assert ... w. From python.list at tim.thechases.com Sat Jan 2 11:45:16 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Sat, 02 Jan 2010 10:45:16 -0600 Subject: assert type([]) == type(()) In-Reply-To: References: Message-ID: <4B3F781C.4040602@tim.thechases.com> > However the following is not an error > > for x in []: > assert type(x) == type(()) > > I expected an AssertionError but get no errors at all. > Any explaination? number_of_times_through_the_loop = 0 for x in []: assert type(x) == type(()) number_of_times_through_the_loop += 1 print number_of_times_through_the_loop make sense? :) -tkc From zuo at chopin.edu.pl Sat Jan 2 11:51:58 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sat, 02 Jan 2010 17:51:58 +0100 Subject: assert type([]) == type(()) In-Reply-To: References: Message-ID: > However the following is not an error > > for x in []: > assert type(x) == type(()) Trying to iterate over an empty sequence or iterator causes 0 (zero) steps of iteration -- so above assert statement is never run. Cheers, *j -- Jan Kaliszewski (zuo) From aahz at pythoncraft.com Sat Jan 2 12:40:44 2010 From: aahz at pythoncraft.com (Aahz) Date: 2 Jan 2010 09:40:44 -0800 Subject: Bare Excepts References: <4dc0cfea0912300209x15e5bc96nd6e6224d89df0638@mail.gmail.com> Message-ID: In article <034f1009$0$1277$c3e8da3 at news.astraweb.com>, Steven D'Aprano wrote: > >The try version is also better whenever there could be a race condition. >For example, when opening a file, you might be tempted to do this: > >if os.path.exists(filename): > f = open(filename, 'r') > text = f.read() >else: > print "file missing" > >but that is dangerous in a multi-processing computer system (which nearly >all computers are these days). The problem is that in the fraction of a >second after you test that the file exists, some other process might come >along and delete it and the open call will then fail. So to be safe, you >have to do this: > >if os.path.exists(filename): > try: > f = open(filename, 'r') > except IOError: > print "file missing" > else: > text = f.read() >else: > print "file missing" > >But of course now the test is redundant, and all you really need is the >try...except block: > >try: > f = open(filename, 'r') >except IOError: > print "file missing" >else: > text = f.read() OTOH, if you want to do something different depending on whether the file exists, you need to use both approaches: if os.path.exists(fname): try: f = open(fname, 'rb') data = f.read() f.close() return data except IOError: logger.error("Can't read: %s", fname) return '' else: try: f = open(fname, 'wb') f.write(data) f.close() except IOError: logger.error("Can't write: %s", fname) return None (This is a somewhat stupid example strictly for illustration. A better and more-elaborate example would be something like trying to copy a file to fname and rename an existing file to '.bak' if it exists. The tricky part would be trying to rename the '.bak' to fname if the copy fails. And yes, that's exactly what some code I wrote a few days ago does.) -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From mike at clove.com Sat Jan 2 13:21:11 2010 From: mike at clove.com (Mike Howard) Date: Sat, 02 Jan 2010 11:21:11 -0700 Subject: Python Concurrency Workshop, Jan 14-15, 2010 In-Reply-To: <12C497C8-47D0-4DFA-B8FD-5FDB18D5A951@dabeaz.com> References: <12C497C8-47D0-4DFA-B8FD-5FDB18D5A951@dabeaz.com> Message-ID: <4B3F8E97.90508@clove.com> Hi Dave, Chicago in January? How about moving it to Denver - it's a nice town and I live close by. Mike David Beazley wrote: > Python Concurrency Workshop, v2.0 > January 14-15, 2010 > Chicago, Illinois > http://www.dabeaz.com/chicago/concurrent.html > > *** Last Two Weeks to Register *** > > Join David Beazley, author of the Python Essential Reference, for an > in-depth workshop on concurrent programming techniques and > idioms. This workshop, designed for more experienced Python > programmers, covers threads, synchronization, message passing, > multiprocessing, distributed computing, coroutines, asynchronous I/O > and other related topics with an eye towards writing programs that can > run on multiple CPU cores, clusters, or distributed systems. A major > theme of the workshop is to explore and understand different > programming techniques, their associated performance properties, and > other tradeoffs. You'll definitely walk away with new insight and a > better understanding of how different parts of Python work under the > covers. > > Workshop attendance is strictly limited to six people. More information, > including a detailed topic index, is available at: > > http://www.dabeaz.com/chicago/concurrent.html > > Hopefully I'll see you in a few weeks! > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: mike.vcf Type: text/x-vcard Size: 242 bytes Desc: not available URL: From mackrackit at gmail.com Sat Jan 2 13:47:24 2010 From: mackrackit at gmail.com (Dave McCormick) Date: Sat, 02 Jan 2010 11:47:24 -0700 Subject: Dynamic text color In-Reply-To: References: Message-ID: <4B3F94BC.4030904@gmail.com> WooHoo!!! I got it!!! Yup, I am sure it can be optimized but it works!!! John, I have to admit that I spent several hours working on this before I looked at your example, then I spent another several hours getting this far. Would never have gotten it with out you help. Thanks!!! Also reading the thread "Significant whitespace" helped. I had an indent problem in a FOR loop. This is a whole lot different than the MCU programming I do in ASM or BASIC, starting to make sense though. Thanks again for all the help and putting up with folks like me, Dave Here is what I can up with. ####### from Tkinter import * import re redList = "red dog".split() blueList = "blue ball".split() greenList = "green grass".split() def get_position(event): complete = Tbox.get("1.0", END) ####RED######## for word in redList: new_Rword(complete, word) def new_Rword(complete, word): Tbox.tag_remove(word, "1.0", END) for matchobj in re.finditer(word, complete): start,end = matchobj.span() Tbox.tag_add("red", "1.0 + %d chars" % start,"1.0 + %d chars" % end) Tbox.tag_config("red", foreground="red") ####BLUE####### for word in blueList: new_Bword(complete, word) def new_Bword(complete, word): Tbox.tag_remove(word, "1.0", END) for matchobj in re.finditer(word, complete): start,end = matchobj.span() Tbox.tag_add("blue", "1.0 + %d chars" % start,"1.0 + %d chars" % end) Tbox.tag_config("blue", foreground="blue") ####GREEN####### for word in greenList: new_Gword(complete, word) def new_Gword(complete, word): Tbox.tag_remove(word, "1.0", END) for matchobj in re.finditer(word, complete): start,end = matchobj.span() Tbox.tag_add("green", "1.0 + %d chars" % start,"1.0 + %d chars" % end) Tbox.tag_config("green", foreground="green") root = Tk() Tbox = Text(root, width=40, height=15, wrap=CHAR, font="Times 14 bold", bg="#dddddd") Tbox.pack() Tbox.bind("", get_position) Tbox.focus() root.mainloop() John Posner wrote: > On Thu, 31 Dec 2009 10:24:44 -0500, Dave McCormick > wrote: > >> John, >> >> Thank you for the tips. >> I was changing the line-column index to a FLOAT because the search >> would return the starting position (pos) of the string, then by >> making it a FLOAT and adding the string length I was able to get the >> end position. >> If "red" was on line 1 column 0.. >> Tbox.tag_add("red", pos, float(pos)+.03) >> = >> Tbox.tag_add("red", 1.0, 1.3) >> It was all I could come up with. > > Yup, Dave, I've dug this kind of hole for myself many times! > >> >> You have convinced me about the re.finditer for this, I think... >> Still in the prototyping mode: >> >> def get_position(event): >> pos = Tbox.get(1.0, END) >> match = [ matchobj.span() for matchobj in >> re.finditer("red", pos) ] >> print "match ",match #debug to shell > > Notes: > > * Variable "pos" should be "text" or "complete_text" or something > similar. > > * The first argument to the get() function must be a string: > > wrong ... complete_text = Tbox.get(1.0, END) > right ... complete_text = Tbox.get("1.0", END) > > But there's a more important problem. Is this function supposed to > handle *one* word to be colored red, or *all the words* to be colored > red? Here's what you want to do on each user keystroke: > > 1. Use get() to place the entire contents of the Text widget in a > variable, say "complete_text". > > 2. Use re.finditer() to generate START,END pairs for the > substrings to > be colored red. You might find it easier to create a list from the > iterator, though it isn't really necessary: > > start_end_pairs = list(re.finditer("red", complete_text)) > > 3. Loop over the START,END pairs in this list. In each loop, use > tag_add() to tag one of the substrings. > > [OOPS: I apologize if my suggestion in the previous post misled you. I > described the output of finditer() as "a list of (start,end) pairs for > an invocation of Text.tag_add()". I should have said "a list of > (start,end) pairs, *WHICH CAN BE LOOPED OVER, FOR A SERIES OF > INVOCATIONS* of Text.tag_add()".] > > Note that the 3 steps above only handle the color red. So you want to > place these steps in a function, then call the function for each color: > > insert_color_markup(text, "red") > insert_color_markup(text, "green") > insert_color_markup(text, "blue") > > For each call, pass in the contents of complete_text as the first > argument. > > So the function-call hierarchy would be: > > get_position() <--- invoked on each keystroke > insert_color_markup() <--- called once for each color > get() then finditer() then loop-driven calls to tag_add() > > I hope that makes sense to you! > > >> Gives all of START,END pairs just fine. It is the last hint about >> line-column indexes that I am have problems with. All of the >> documentation I can find about "text.tag_add()" uses line-column for >> coordinates. > > Lie Ryan has already pointed you to clarifying documentation. Be sure > to bookmark http://infohost.nmt.edu/tcc/help/pubs/tkinter/ in your Web > browser! > >> If I count characters from the beginning how do I know what line the >> text is on? Would you mind making your last hint a bit stronger... > > The point is that *you don't need to*. Handling the text as a simple > sequence of characters is much simpler than worrying about line/column > pairs. (If you were using a different shade of red on different lines, > you *would* need to worry about line/column pairs.) > > One more suggestion, which I forgot to make in the previous round. > Your code includes this: > > Tbox.grid(column=0, row=0, sticky=(N+W+E+S)) > root.grid_columnconfigure(0, weight=1) > root.grid_rowconfigure(0, weight=1) > > You're working too hard here. This is sufficient: > > Tbox.pack() > > Keep at it, Dave. I've posted a complete solution at > http://cl1p.net/jjp_dynamic_text_color/. But I suggest that you put in > a couple of more hours of coding before peeking at it. > > Best, > John From mwilson at the-wire.com Sat Jan 2 14:00:18 2010 From: mwilson at the-wire.com (Mel) Date: Sat, 02 Jan 2010 14:00:18 -0500 Subject: assert type([]) == type(()) References: Message-ID: VanceE wrote: > Just curious. > > type([]) == type(()) > is False as expected > > and > > assert type([]) == type(()) > throws an AssertionError as expected. > > However the following is not an error > > for x in []: > assert type(x) == type(()) > > I expected an AssertionError but get no errors at all. > Any explaination? Others have said what happens. Bit of a mind bender -- here's the smallest change to do what you expected: Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41) [GCC 4.3.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> for x in [],: ... assert type(x) == type (()) ... Traceback (most recent call last): File "", line 2, in AssertionError >>> Because of the comma, this creates a tuple containing the empty list, and iterates over the tuple. Mel. From vnewel at invalid.invalid Sat Jan 2 15:06:06 2010 From: vnewel at invalid.invalid (VanceE) Date: Sat, 2 Jan 2010 15:06:06 -0500 Subject: Thx (Was: assert type([]) == type(()) ) References: Message-ID: A big Thank You to all for the answer (and the hints). That sure explains a lot. Again, Thank you very much. Cheers, Vance From martin at v.loewis.de Sat Jan 2 15:17:59 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Sat, 02 Jan 2010 21:17:59 +0100 Subject: Dangerous behavior of list(generator) In-Reply-To: References: <7p356kF750U1@mid.individual.net> Message-ID: <4B3FA9F7.2080304@v.loewis.de> >> Bottom line, I'm going to have to remove this pattern from my code: >> >> foo = (foo for foo in foos if foo.bar).next() I recommend to rewrite this like so: def first(gen): try: return gen.next() except StopIteration: raise ValueError, "No first value" foo = first(foo for foo in foos if foo.bar) As others have said: don't let StopIteration appear unexpectedly; IOW, consume generators right away in a loop construct (where this first function is a loop construct as well). A different way of writing it would be def first(gen): for value in gen: return value raise ValueError, "empty collection" Regards, Martin From martin at v.loewis.de Sat Jan 2 15:27:54 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Sat, 02 Jan 2010 21:27:54 +0100 Subject: Dangerous behavior of list(generator) In-Reply-To: References: <20091213143521.2549.845539319.divmod.xquotient.731@localhost.localdomain> <4B2652AB.7080501@egenix.com> Message-ID: <4B3FAC4A.4090306@v.loewis.de> > I'm asking about why the behavior of a StopIteration exception being > handled from the `expression` of a generator expression to mean "stop > the loop" is accepted by "the devs" as acceptable. I may be late to this discussion, but the answer is "most definitely yes". *Any* exception leads to termination of the iterator, and StopIteration is no different: py> def stop(e): ... def f(): ... raise e ... return f ... py> g = (f() for f in (lambda:1,stop(StopIteration),lambda:2)) py> g.next py> g.next() 1 py> g.next() Traceback (most recent call last): File "", line 1, in File "", line 1, in File "", line 3, in f StopIteration py> g.next() Traceback (most recent call last): File "", line 1, in StopIteration py> g = (f() for f in (lambda:1,stop(ValueError),lambda:2)) py> g.next() 1 py> g.next() Traceback (most recent call last): File "", line 1, in File "", line 1, in File "", line 3, in f ValueError py> g.next() Traceback (most recent call last): File "", line 1, in StopIteration Regards, Martin From pydecker at gmail.com Sat Jan 2 16:04:03 2010 From: pydecker at gmail.com (Peter Decker) Date: Sat, 2 Jan 2010 16:04:03 -0500 Subject: whoops: create a splash window in python In-Reply-To: <4B3C374C.6010307@depauw.edu> References: <4b3c246d$1@dnews.tpgi.com.au> <4B3C3584.8070500@depauw.edu> <4B3C374C.6010307@depauw.edu> Message-ID: On Thu, Dec 31, 2009 at 12:31 AM, Ron Croonenberg wrote: > is there a way, in python, to create a splash window and when the program > has completed disappears by sending a msg to it? (I tried creating two gtk > windows but gtk_main doesn't seem to return unless it gets closed.) It's really simple to do this in Dabo (which uses wxPython under the hood, but is _so_ much more elegant!). When you create your app, add the following two parameters: app = dabo.dApp(showSplashScreen=True, splashImage="/path/to/splash.png") That's it! -- # p.d. From tjreedy at udel.edu Sat Jan 2 16:48:35 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 02 Jan 2010 16:48:35 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <366c6f341001020704m6811a2d0yeca281dda62d4c3e@mail.gmail.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <50697b2c0912312124n7f989568t13e5a1dd8d883e9c@mail.gmail.com> <7q8r3nF3u6mmdU1@mid.uni-berlin.de> <366c6f341001020704m6811a2d0yeca281dda62d4c3e@mail.gmail.com> Message-ID: On 1/2/2010 10:04 AM, Peng Yu wrote: > For my own interest, I want understand the run time behavior of python That depends on the implementation. > and what details causes it much slower. A language feature that slows all implementation is the dynamic name/slot binding and resolution. Any implementation can be made faster by restricting the dynamism (which makes the imlementaion one of a subset of Python). > Although people choose python > for its programming efficiency, but sometimes the runtime still > matters. There is no 'the' runtime. Whether or not there even *is* a runtime, as usually understoold, is a matter of the implementation. > This is an important aspect of the language. It is an aspect of each implementation, of which there are now more than one. Terry Jan Reedy From Dr.Alain.Picard at gmail.com Sat Jan 2 16:48:57 2010 From: Dr.Alain.Picard at gmail.com (Alain Picard) Date: Sun, 03 Jan 2010 08:48:57 +1100 Subject: Xah's Edu Corner: Teach Ourself Programing In Ten Years? References: <1035ce5d-914e-42c0-8e9c-a349a6d55e55@v25g2000yqk.googlegroups.com> Message-ID: <87my0w89fa.fsf@gmail.com> [Aplogies about the wild cross-post follow-up --- I guess the topic really is relevant to most programming communities.] Xah Lee writes: > To see this in a different context, suppose you need to pass a > important Math XYZ exam or review in your career or get a certificate, > but you don't remember your Math XYZ. There's a difference between cramming to remember XYZ, and never having understood (or even been exposed to!) XYZ previously. The former is reasonable; the latter bound to fail. > [SNIP] In my opinion, a solution to this is by installing > the concept of responsible licenses. Please see this essay Responsible > Software Licensing IOW, you think IT will have to become like engineering and "grow up". I.e. when your manager says "do XYZ in 7 days", you just laugh at them, and your risk of being fired are minimal, because all other certified engineers who could replace you also will laugh at him because your duty of care to your profession and certification is greater than the one you owe your bozo manager. I'm, personally, not holding my breath. If you want to change the world, you start by changing yourself. Cheers, --ap From john at castleamber.com Sat Jan 2 17:15:52 2010 From: john at castleamber.com (John Bokma) Date: Sat, 02 Jan 2010 16:15:52 -0600 Subject: Xah's Edu Corner: Teach Ourself Programing In Ten Years? References: <1035ce5d-914e-42c0-8e9c-a349a6d55e55@v25g2000yqk.googlegroups.com> <87my0w89fa.fsf@gmail.com> Message-ID: <87y6kgrw4n.fsf@castleamber.com> Alain Picard writes: > If you want to change the world, you start by changing yourself. Like for starters setting a follow-up to header, especially if you spam 4 groups. But Xah is Xah. -- John Bokma Read my blog: http://johnbokma.com/ Hire me (Perl/Python): http://castleamber.com/ From emile at fenx.com Sat Jan 2 17:19:17 2010 From: emile at fenx.com (Emile van Sebille) Date: Sat, 02 Jan 2010 14:19:17 -0800 Subject: Significant whitespace In-Reply-To: <034e88f2$0$1277$c3e8da3@news.astraweb.com> References: <4B3E70FC.1010702@gmail.com> <034e88f2$0$1277$c3e8da3@news.astraweb.com> Message-ID: On 1/1/2010 5:05 PM Steven D'Aprano said... > In Python terms, imagine if we could write > > foriinrange(10): > > instead of the usual > > for i in range(10): > > Since the colon makes it unambiguous that it is some sort of block > construct, and it starts with "for", it must be a for loop. Pretty > horrible, yes? Yes -- early on I got stung by something similar. My first programming job (1977?) was working in a basic dialect called Buisness Basic Level II (BBII). To delete a record in the file you used a remove statement. White space on the line was optional, allowing you to write lines like '100 remove(1,key=k$)' and '100 fori=1to10'. REM was how a remark statement began, which allowed statements like "100 REM comment". I was sent on-site to a customer running the prior version (BBI). I didn't realize the REMOVE command changed between versions from REKEY, and it was only after a more experienced programmer co-wroker pointed out that my REMOVE statement was being interpreted as a REM that I finally found the bug I'd created. Manuals and documentation, while often lacking, are as often underrated. Emile From emile at fenx.com Sat Jan 2 17:42:35 2010 From: emile at fenx.com (Emile van Sebille) Date: Sat, 02 Jan 2010 14:42:35 -0800 Subject: Xah's Edu Corner: Teach Ourself Programing In Ten Years? In-Reply-To: <1035ce5d-914e-42c0-8e9c-a349a6d55e55@v25g2000yqk.googlegroups.com> References: <1035ce5d-914e-42c0-8e9c-a349a6d55e55@v25g2000yqk.googlegroups.com> Message-ID: On 1/2/2010 1:14 AM Xah Lee said... > These books are the bedrock of the industry. It is not because people > are impatient, or that they wish to hurry, but rather, it is the > condition of the IT industry, in the same way modern society drives > people to live certain life styles. Turing complete. Once you've learned to program, learning dialects is a different study. I hated Word Perfect early on, but if I were faced with it today I expect that everything I know about word processing would kelp to deploy it effectively with minimal in-depth knowledge of it's peculiarities. Likewise, I don't work in perl, but I don't hesitate to deploy and modify perl projects to my needs, and mostly having only brushed up on the semantics that need to change. > No amount of patience or > proselytization can right this, except that we change the industry's > practice of quickly churning out bug-ridden software products to beat > competitors. Companies do that due to market forces, and the market > forces is a result of how people and organizations actually choose to > purchase software. In my opinion, a solution to this is by installing > the concept of responsible licenses. Please see this essay Responsible > Software Licensing and spread the word. Licensing programmers isn't likely to turn Microsoft around. Enforcement doesn't carry much weight with them. Let's work to strengthen that, then there's a functioning mechanism in place to work with. Not-holding-my-breath-ly y'rs, Emile From iwaki at iwakihidekazu.net Sat Jan 2 17:55:54 2010 From: iwaki at iwakihidekazu.net (Hidekazu IWAKI) Date: Sun, 03 Jan 2010 07:55:54 +0900 Subject: Where is "urllib2" module in windows python3.1.1? In-Reply-To: References: Message-ID: <243620100102225554iwaki@iwakihidekazu.net> Hi. Thank you for the answers. Oh, sorry. It was one of the changes from .2.x to 3.x. I didn't know. There are really important and a lot of changes. Thank you! At 01 Jan 2010 13:57:03 +0000 Duncan Booth wrote: > Hidekazu IWAKI wrote: > > > Hi; > > I'd like to import "urllib2" in windows python 3.1.1, but I'm not able to > > do it. > > Python 3 doesn't have a urllib2 module; use the urllib package instead. > > See http://www.python.org/dev/peps/pep-3108/#urllib-package > -- > http://mail.python.org/mailman/listinfo/python-list From bblais at bryant.edu Sat Jan 2 18:17:52 2010 From: bblais at bryant.edu (Brian Blais) Date: Sat, 02 Jan 2010 18:17:52 -0500 Subject: Python Concurrency Workshop, Jan 14-15, 2010 In-Reply-To: <4B3F8E97.90508@clove.com> References: <12C497C8-47D0-4DFA-B8FD-5FDB18D5A951@dabeaz.com> <4B3F8E97.90508@clove.com> Message-ID: <22DCF99A-C64E-46E1-8AA7-C8DF383153E2@bryant.edu> On Jan 2, 2010, at 13:21 , Mike Howard wrote: > Hi Dave, > > Chicago in January? > > How about moving it to Denver - it's a nice town and I live close by. > > Mike > > David Beazley wrote: >> Python Concurrency Workshop, v2.0 I think, in the spirit of the topic, they should hold it at both places at the same time. bb -- Brian Blais bblais at bryant.edu http://web.bryant.edu/~bblais http://bblais.blogspot.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From fpm at u.washington.edu Sat Jan 2 18:21:45 2010 From: fpm at u.washington.edu (cassiope) Date: Sat, 2 Jan 2010 15:21:45 -0800 (PST) Subject: IOError - cannot create file (linux daemon-invoked script) Message-ID: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> I have a daemon on a Linux system that supports a number of Windows clients. Among the functions is to send e-mails, which is sufficiently complicated that I fork() a separate process which gets setuid to a lesser user, and calls a python script which does the actual formatting and emailing (the daemon is written in C). I want to save a copy of the email in a particular directory which is accessible to the Windows clients via samba. The strange thing is that even with the right user-id, I cannot seem to write to the directory, getting an IOError exception. Changing the directory to world-writable fixes this. I can confirm the uid and gid for the script by having the script print these values just before trying to create/write the file. Becoming the same lesser user, I have no problem writing a file to the same directory. Is there anything that I can do to diagnose why this script is failing? For various reasons I don't want to make the directory world- writable. This is on a Debian "squeeze" system, with python 2.5. Thanks for any insights! From steve at holdenweb.com Sat Jan 2 18:46:22 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 02 Jan 2010 18:46:22 -0500 Subject: IOError - cannot create file (linux daemon-invoked script) In-Reply-To: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> References: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> Message-ID: cassiope wrote: > I have a daemon on a Linux system that supports a number of Windows > clients. Among the functions is to send e-mails, which is > sufficiently complicated that I fork() a separate process which gets > setuid to a lesser user, and calls a python script which does the > actual formatting and emailing (the daemon is written in C). I want > to save a copy of the email in a particular directory which is > accessible to the Windows clients via samba. > > The strange thing is that even with the right user-id, I cannot seem > to write to the directory, getting an IOError exception. Changing the > directory to world-writable fixes this. I can confirm the uid and gid > for the script by having the script print these values just before > trying to create/write the file. Becoming the same lesser user, I > have no problem writing a file to the same directory. > Have you looked at the IOError's errno attribute to find out exactly why the Python subprocess is unable to write to the directory? > Is there anything that I can do to diagnose why this script is > failing? For various reasons I don't want to make the directory world- > writable. > I'd concur on that decision. > This is on a Debian "squeeze" system, with python 2.5. > > Thanks for any insights! Take a closer look at the exception, that might stimulate a thought or two. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From llcawthorne at archlinux.us Sat Jan 2 18:47:37 2010 From: llcawthorne at archlinux.us (Lewis Cawthorne) Date: Sat, 02 Jan 2010 18:47:37 -0500 Subject: Windows 7 : any problems installing or running Python ? In-Reply-To: <000001ca8bb6$08391d70$18ab5850$@com> References: <000001ca8bb6$08391d70$18ab5850$@com> Message-ID: <4B3FDB19.5080405@archlinux.us> I'm not Skippy, but I do have my wife's Toshiba Satellite running Windows 7 Home Premium laying around. I figured that since I am fairly well set to duplicate all the information you gave in your problem, I would take a look. The bad news is that Python works great on it, so your problem needs more troubleshooting.. I have IDLE open right now with no issues. Python version: 3.1.1, Idle version: 3.1.1, Tk version: 8.5. Have you tried launching Python from a command line in the directory that you installed it to perhaps? Maybe it would spit out an error message (or the interpreter might just work and maybe you have a Tk problem... who knows). Lewis Cawthorne On 01/02/2010 09:15 AM, David M Covey Sr. wrote: > > Hello Skippy, > > In response to your message "Windows 7 : any problems installing or > running Python ?" I found posted at > (http://mail.python.org/pipermail/python-list/2009-August/1215524.html), > I've got to say that I can't seem to get any version of Python to work > on my computer. I have a Toshiba Satellite laptop running Windows 7 > Home Premium with an AMD Turion II Dual-Core processor. I've tried all > of the versions listed at http://python.org/download with no success. > They install fine but when I try to run the IDLE (Python GUI), it does > nothing at all. > > Do you have any suggestions that might help me out here? I would > really appreciate your input. > > Thank you, > > David M Covey Sr. > > admin at daffitt.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sysadmin at example.com Sat Jan 2 19:01:50 2010 From: sysadmin at example.com (Wanna-Be Sys Admin) Date: Sat, 02 Jan 2010 16:01:50 -0800 Subject: Xah's Edu Corner: Teach Ourself Programing In Ten Years? References: <1035ce5d-914e-42c0-8e9c-a349a6d55e55@v25g2000yqk.googlegroups.com> <87my0w89fa.fsf@gmail.com> Message-ID: Alain Picard wrote: > [Aplogies about the wild cross-post follow-up --- I guess the topic > [really > is?relevant?to?most?programming?communities.] No, it's not relevant. Xah Lee is a self spammer, in that he spams about himself, tries to get people hyped up about him and thinks he's impressing people (he's not). He continually cross posts to the language groups he thinks he knows about (often to Perl, where there's no mention of Perl, other than he's used it once or so). He knows very little about any of these languages, but likes to talk a whole lot, as in talking up about himself and how he thinks himself a genius. If you could please not cross post his self-spamming replies to the Python and Perl groups, at least, I'd appreciate it. Most people know about him and have filtered his posts by now. Thanks. -- Not really a wanna-be, but I don't know everything. From steve at REMOVE-THIS-cybersource.com.au Sat Jan 2 19:30:03 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 03 Jan 2010 00:30:03 GMT Subject: Bare Excepts References: <4dc0cfea0912300209x15e5bc96nd6e6224d89df0638@mail.gmail.com> <034f1009$0$1277$c3e8da3@news.astraweb.com> Message-ID: <034fd208$0$1277$c3e8da3@news.astraweb.com> On Sat, 02 Jan 2010 09:40:44 -0800, Aahz wrote: > OTOH, if you want to do something different depending on whether the > file exists, you need to use both approaches: > > if os.path.exists(fname): > try: > f = open(fname, 'rb') > data = f.read() > f.close() > return data > except IOError: > logger.error("Can't read: %s", fname) return '' > else: > try: > f = open(fname, 'wb') > f.write(data) > f.close() > except IOError: > logger.error("Can't write: %s", fname) > return None Unfortunately, this is still vulnerable to the same sort of race condition I spoke about. Even more unfortunately, I don't know that there is any fool-proof way of avoiding such race conditions in general. Particularly the problem of "open this file for writing only if it doesn't already exist". > (This is a somewhat stupid example strictly for illustration. A better > and more-elaborate example would be something like trying to copy a file > to fname and rename an existing file to '.bak' if it exists. The tricky > part would be trying to rename the '.bak' to fname if the copy fails. > And yes, that's exactly what some code I wrote a few days ago does.) Sounds interesting and useful. Would you care to share it with us, or to publish it as a recipe? -- Steven From steve at REMOVE-THIS-cybersource.com.au Sat Jan 2 19:30:17 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 03 Jan 2010 00:30:17 GMT Subject: Bare Excepts References: <4dc0cfea0912300209x15e5bc96nd6e6224d89df0638@mail.gmail.com> <4B3B4FE5.8070605@sequans.com> <4dc0cfea0912300509v7f9bb367jf33bb63736407ed@mail.gmail.com> <4dc0cfea0912300512h43a4fdb1oa585a36391de341e@mail.gmail.com> <6944e440-c797-439d-9bf9-b61a05a3e26c@k23g2000yqa.googlegroups.com> <878wck2ojd.fsf@benfinney.id.au> <4dc0cfea0912301927q16445c59r1fb8594ee0f04d24@mail.gmail.com> <034f1009$0$1277$c3e8da3@news.astraweb.com> <4b3f53ec$1@dnews.tpgi.com.au> Message-ID: <034fd216$0$1277$c3e8da3@news.astraweb.com> On Sun, 03 Jan 2010 01:10:51 +1100, Lie Ryan wrote: > On 1/2/2010 9:42 PM, Steven D'Aprano wrote: >> On Fri, 01 Jan 2010 15:27:57 -0800, myle wrote: >> >>> Why we should prefer ``if: ...'' over a ``try: ... except something: >>> pass'' block? >> >> We shouldn't, not in general. > > One exception (pun intended) is if the try-block have a side effect that > is difficult to cleanup cleanly. What if the "if ..." test has a side-effect? This is an argument against side-effects, not against using exceptions. But of course you are right, there are lots of good reasons for choosing to use an if test rather than a try block, or vice versa. I would argue that the try block is generally more Pythonic, and should be preferred, but that doesn't mean that there aren't reasons for sometimes choosing other strategies instead. -- Steven From jjposner at optimum.net Sat Jan 2 20:38:58 2010 From: jjposner at optimum.net (John Posner) Date: Sat, 02 Jan 2010 20:38:58 -0500 (EST) Subject: Dynamic text color Message-ID: <10564900.973857.1262482738487.JavaMail.jjposner@mail.srv.hcvlny.cv.net> On Sat, Jan 2, 2010 at 1:47 PM, Dave McCormick wrote: > WooHoo!!! > I got it!!! Yup, I am sure it can be optimized but it works!!! Hmmm ... it doesn't work for me ... ####RED######## > for word in redList: > new_Rword(complete, word) def new_Rword(complete, word): > Tbox.tag_remove(word, "1.0", END) > for matchobj in re.finditer(word, complete): > start,end = matchobj.span() Tbox.tag_add("red", > "1.0 + %d chars" % start,"1.0 + %d chars" % end) > Tbox.tag_config("red", foreground="red") How *could* this work, Dave, since you call function new_Rword() before you define it? I rearranged the statements in your program, and it now works for me: #--------------------------------------------- from Tkinter import * import re redList = "red dog".split() blueList = "blue ball".split() greenList = "green grass".split() def get_position(event): complete = Tbox.get("1.0", END) for word in redList: new_Rword(complete, word) for word in blueList: new_Bword(complete, word) for word in greenList: new_Gword(complete, word) def new_Rword(complete, word): Tbox.tag_remove(word, "1.0", END) for matchobj in re.finditer(word, complete): start,end = matchobj.span() Tbox.tag_add("red", "1.0 + %d chars" % start,"1.0 + %d chars" % end) def new_Bword(complete, word): Tbox.tag_remove(word, "1.0", END) for matchobj in re.finditer(word, complete): start,end = matchobj.span() Tbox.tag_add("blue", "1.0 + %d chars" % start,"1.0 + %d chars" % end) def new_Gword(complete, word): Tbox.tag_remove(word, "1.0", END) for matchobj in re.finditer(word, complete): start,end = matchobj.span() Tbox.tag_add("green", "1.0 + %d chars" % start,"1.0 + %d chars" % end) root = Tk() Tbox = Text(root, width=40, height=15, wrap=CHAR, font="Times 14 bold", bg="#dddddd") Tbox.tag_config("red", foreground="red") Tbox.tag_config("blue", foreground="blue") Tbox.tag_config("green", foreground="green") Tbox.pack() Tbox.bind("", get_position) Tbox.focus() root.mainloop() #--------------------------------------------- This version also has the advantage of defining each function just once, instead of multiple times on each keystroke! Still to-do: * rename get_position() to get_complete_text() * replace new_Rword(), new_Bword(), and new_Gword() with a single function that has an extra parameter, "color". Keep at it! -John From jjposner at optimum.net Sat Jan 2 20:52:33 2010 From: jjposner at optimum.net (John Posner) Date: Sat, 02 Jan 2010 20:52:33 -0500 (EST) Subject: Dynamic text color Message-ID: <21336368.974257.1262483554337.JavaMail.jjposner@mail.srv.hcvlny.cv.net> On Sat, Jan 2, 2010 at 1:47 PM, Dave McCormick wrote: > WooHoo!!! > I got it!!! Yup, I am sure it can be optimized but it works!!! Dave, please ignore a couple of my bogus complaints in the previous message: ... you call function new_Rword() before you define it ... this version also has the advantage of defining each function just once, instead of multiple times on each keystroke But I stand by my overall statement that the program didn't work for me, and that rearranging the lines produces a working program. -John From mensanator at aol.com Sat Jan 2 20:52:55 2010 From: mensanator at aol.com (Mensanator) Date: Sat, 2 Jan 2010 17:52:55 -0800 (PST) Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> <034e88f2$0$1277$c3e8da3@news.astraweb.com> Message-ID: <4c436757-cd85-4dea-ab5e-df00f826b9bd@r24g2000yqd.googlegroups.com> On Jan 2, 4:19?pm, Emile van Sebille wrote: > On 1/1/2010 5:05 PM Steven D'Aprano said... > > > In Python terms, imagine if we could write > > > ? ? ?foriinrange(10): > > > instead of the usual > > > ? ? ?for i in range(10): > > > Since the colon makes it unambiguous that it is some sort of block > > construct, and it starts with "for", it must be a for loop. Pretty > > horrible, yes? > > Yes -- early on I got stung by something similar. ?My first programming > job (1977?) was working in a basic dialect called Buisness Basic Level > II (BBII). ?To delete a record in the file you used a remove statement. > ? White space on the line was optional, allowing you to write lines like > '100 remove(1,key=k$)' and '100 fori=1to10'. ?REM was how a remark > statement began, which allowed statements like "100 REM comment". ?I was > sent on-site to a customer running the prior version (BBI). ?I didn't > realize the REMOVE command changed between versions from REKEY, and it > was only after a more experienced programmer co-wroker pointed out that > my REMOVE statement was being interpreted as a REM that I finally found > the bug I'd created. Better than the other way 'round, eh? > > Manuals and documentation, while often lacking, are as often underrated. > > Emile From robert.kern at gmail.com Sat Jan 2 21:27:09 2010 From: robert.kern at gmail.com (Robert Kern) Date: Sat, 02 Jan 2010 20:27:09 -0600 Subject: Python Concurrency Workshop, Jan 14-15, 2010 In-Reply-To: <22DCF99A-C64E-46E1-8AA7-C8DF383153E2@bryant.edu> References: <12C497C8-47D0-4DFA-B8FD-5FDB18D5A951@dabeaz.com> <4B3F8E97.90508@clove.com> <22DCF99A-C64E-46E1-8AA7-C8DF383153E2@bryant.edu> Message-ID: On 2010-01-02 17:17 , Brian Blais wrote: > On Jan 2, 2010, at 13:21 , Mike Howard wrote: > >> Hi Dave, >> >> Chicago in January? >> >> How about moving it to Denver - it's a nice town and I live close by. >> >> Mike >> >> David Beazley wrote: >>> Python Concurrency Workshop, v2.0 > > I think, in the spirit of the topic, they should hold it at both places > at the same time. .... Well played. +1 QOTW -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From davea at ieee.org Sat Jan 2 21:35:48 2010 From: davea at ieee.org (Dave Angel) Date: Sat, 02 Jan 2010 21:35:48 -0500 Subject: Bare Excepts In-Reply-To: <034fd208$0$1277$c3e8da3@news.astraweb.com> References: <4dc0cfea0912300209x15e5bc96nd6e6224d89df0638@mail.gmail.com> <034f1009$0$1277$c3e8da3@news.astraweb.com> <034fd208$0$1277$c3e8da3@news.astraweb.com> Message-ID: <4B400284.1060909@ieee.org> Steven D'Aprano wrote: > On Sat, 02 Jan 2010 09:40:44 -0800, Aahz wrote: > > >> OTOH, if you want to do something different depending on whether the >> file exists, you need to use both approaches: >> >> if os.path.exists(fname): >> try: >> f = open(fname, 'rb') >> data = f.read() >> f.close() >> return data >> except IOError: >> logger.error("Can't read: %s", fname) return '' >> else: >> try: >> f = open(fname, 'wb') >> f.write(data) >> f.close() >> except IOError: >> logger.error("Can't write: %s", fname) >> return None >> > > Unfortunately, this is still vulnerable to the same sort of race > condition I spoke about. > > Even more unfortunately, I don't know that there is any fool-proof way of > avoiding such race conditions in general. Particularly the problem of > "open this file for writing only if it doesn't already exist". > > In Windows, there is a way to do it. It's just not exposed to the Python built-in function open(). You use the CreateFile() function, with /dwCreationDisposition/ of CREATE_NEW. It's atomic, and fails politely if the file already exists. No idea if Unix has a similar functionality. DaveA From bthate at gmail.com Sat Jan 2 21:40:01 2010 From: bthate at gmail.com (Bart Thate) Date: Sat, 2 Jan 2010 18:40:01 -0800 (PST) Subject: CMNDBOT 0.1 released Message-ID: new in this release: * updated the repository to GZRBOT code * a outputcache and poller gadget is now available to support writing to waves (right now the poller polls every minute) * RSS plugin looks stable todo: * make gozernet work .. this lets GZRBOT bots communicate with each other by using json over xmpp * use this to implement wave <-> IRC relaying * port monitoring of bot output * port karma plugin * port quote plugin demo: http://cmndbot.appspot.com wave/xmpp: cmndbot at appspot.com about CMNDBOT: CMNDBOT is a port of GOZERBOT to the Google Application Engine. It supports wave, web and xmpp. It supports a plugin structure that lets you add commands or register callbacks for events. License is BSD From lists at cheimes.de Sat Jan 2 21:40:17 2010 From: lists at cheimes.de (Christian Heimes) Date: Sun, 03 Jan 2010 03:40:17 +0100 Subject: IOError - cannot create file (linux daemon-invoked script) In-Reply-To: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> References: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> Message-ID: cassiope wrote: > The strange thing is that even with the right user-id, I cannot seem > to write to the directory, getting an IOError exception. Changing the > directory to world-writable fixes this. I can confirm the uid and gid > for the script by having the script print these values just before > trying to create/write the file. Becoming the same lesser user, I > have no problem writing a file to the same directory. Are you able to write to the directory with the user id when you tried to create a file manually? How are you changing the uid and gid of your script? IIRC you have to set the effective user id with os.seteuid() and os.setegid(). Christian From n00m at narod.ru Sat Jan 2 22:17:38 2010 From: n00m at narod.ru (n00m) Date: Sat, 2 Jan 2010 19:17:38 -0800 (PST) Subject: Any Swisses here? Message-ID: <145e6acd-4260-495b-b562-da634fccfa1a@j19g2000yqk.googlegroups.com> Congrats! Your choice -- to ban building of muslim mosques -- is the only choice to save our civililazation. This yellow plague (incl. Chinese etc) must be eliminated from our Planet, They are very cunning critters, they can play on strings of compassion, but its riffraffs. What do you know about Russia? Who are yuo against Russia? Nothing. Dust. We are simply tired people. From zhushazang at yahoo.com.br Sat Jan 2 22:26:59 2010 From: zhushazang at yahoo.com.br (Zhu Sha Zang) Date: Sun, 03 Jan 2010 01:26:59 -0200 Subject: Any Swisses here? In-Reply-To: <145e6acd-4260-495b-b562-da634fccfa1a@j19g2000yqk.googlegroups.com> References: <145e6acd-4260-495b-b562-da634fccfa1a@j19g2000yqk.googlegroups.com> Message-ID: <4B400E83.1080305@yahoo.com.br> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Em 03-01-2010 01:17, n00m escreveu: > Congrats! > Your choice -- to ban building of muslim mosques -- is the only choice > to save our civililazation. > This yellow plague (incl. Chinese etc) must be eliminated from our > Planet, > They are very cunning critters, they can play on strings of > compassion, > but its riffraffs. What do you know about Russia? Who are yuo against > Russia? > Nothing. Dust. We are simply tired people. WTF? -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAktADoIACgkQ35zeJy7JhCgkOACfZG9y57J6WpmPRcxBLT9lHi4D AjYAn059bwVoN+D2/WZHwsGPVVP/I3wF =msDn -----END PGP SIGNATURE----- From steve at holdenweb.com Sat Jan 2 22:30:27 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 02 Jan 2010 22:30:27 -0500 Subject: Any Swisses here? In-Reply-To: <4B400E83.1080305@yahoo.com.br> References: <145e6acd-4260-495b-b562-da634fccfa1a@j19g2000yqk.googlegroups.com> <4B400E83.1080305@yahoo.com.br> Message-ID: Zhu Sha Zang wrote: >> [stuff and nonsense from a third party] > WTF? We do get the occasional bigot dropping in from time to time. Best to ignore them 'til they go away. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From n00m at narod.ru Sat Jan 2 22:38:30 2010 From: n00m at narod.ru (n00m) Date: Sat, 2 Jan 2010 19:38:30 -0800 (PST) Subject: Any Swisses here? References: <145e6acd-4260-495b-b562-da634fccfa1a@j19g2000yqk.googlegroups.com> <4B400E83.1080305@yahoo.com.br> Message-ID: On Jan 3, 5:30?am, Steve Holden wrote: > Zhu Sha Zang wrote: > >> [stuff and nonsense from a third party] > > WTF? > > We do get the occasional bigot dropping in from time to time. Best to > ignore them 'til they go away. > > regards > ?Steve > -- > Steve Holden ? ? ? ? ? +1 571 484 6266 ? +1 800 494 3119 > PyCon is coming! Atlanta, Feb 2010 ?http://us.pycon.org/ > Holden Web LLC ? ? ? ? ? ? ? ?http://www.holdenweb.com/ > UPCOMING EVENTS: ? ? ? ?http://holdenweb.eventbrite.com/ And who are YOU? What you achieved in the life? For me you is nothing I recall only Kipling, Dickens, Newton its your background. My backgroung is the best rockets From sisson.j at gmail.com Sat Jan 2 22:47:42 2010 From: sisson.j at gmail.com (J Sisson) Date: Sat, 2 Jan 2010 21:47:42 -0600 Subject: Any Swisses here? In-Reply-To: References: <145e6acd-4260-495b-b562-da634fccfa1a@j19g2000yqk.googlegroups.com> <4B400E83.1080305@yahoo.com.br> Message-ID: <4297a9021001021947y78b6fa4cy28f0ff53d59930a2@mail.gmail.com> On Sat, Jan 2, 2010 at 9:38 PM, n00m wrote: > What you achieved in the life? English mastery, for starters... -------------- next part -------------- An HTML attachment was scrubbed... URL: From cs at zip.com.au Sat Jan 2 23:02:38 2010 From: cs at zip.com.au (Cameron Simpson) Date: Sun, 3 Jan 2010 15:02:38 +1100 Subject: IOError - cannot create file (linux daemon-invoked script) In-Reply-To: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> References: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> Message-ID: <20100103040238.GA15992@cskk.homeip.net> On 02Jan2010 15:21, cassiope wrote: | [...] I want | to save a copy of the email in a particular directory which is | accessible to the Windows clients via samba. | | The strange thing is that even with the right user-id, I cannot seem | to write to the directory, getting an IOError exception. Changing the | directory to world-writable fixes this. I can confirm the uid and gid | for the script by having the script print these values just before | trying to create/write the file. Becoming the same lesser user, I | have no problem writing a file to the same directory. Can you show us: - the directory user and group ownership and permissions - the daemon's user and group values? You can also strace your daemon: strace -f -e trace=file your-daemon your-daemon-args... 2>strace.out and then examine the log for the precise UNIX-level failure. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Money won't buy happiness, but it will pay the salary of a large research staff to study the problem. - Bill Vaughan From nagle at animats.com Sun Jan 3 01:40:31 2010 From: nagle at animats.com (John Nagle) Date: Sat, 02 Jan 2010 22:40:31 -0800 Subject: Safe file I/O to shared file (or SQLite) from multi-threaded web server In-Reply-To: References: Message-ID: <4b403899$0$1646$742ec2ed@news.sonic.net> python at bdurham.com wrote: > I'm looking for the best practice way for a multi-threaded python web > server application to read/write to a shared file or a SQLite database. > > What do I need to do (if anything) to make sure my writes to a regular > file on disk or to a SQLite database are atomic in nature when multiple > clients post data to my application simultaneously? SQLite can do that correctly, but SQLite isn't intended for use as a database engine for a busy database being used by many concurrent operations. Especially if those operations involve updates. Any update in SQLite locks all tables involved for the duration of the operation. When SQLite hits a lock, it returns an error code, and the caller should retry after a delay. If this occurs frequently in your application, you've hit the limits of SQLite. Then it's time to move up to MySQL. If you have enough traffic that you need a multi-threaded web server, it's probably time to move up. John Nagle From mackrackit at gmail.com Sun Jan 3 03:57:11 2010 From: mackrackit at gmail.com (Dave McCormick) Date: Sun, 03 Jan 2010 01:57:11 -0700 Subject: Dynamic text color In-Reply-To: <21336368.974257.1262483554337.JavaMail.jjposner@mail.srv.hcvlny.cv.net> References: <21336368.974257.1262483554337.JavaMail.jjposner@mail.srv.hcvlny.cv.net> Message-ID: <4B405BE7.4060202@gmail.com> John, Interesting and confusing... I tested my code on Ubuntu 8.04, Win XP and 7, and WinMob before I called it "good". And of course your modifications work on all too. """ * rename get_position() to get_complete_text() * replace new_Rword(), new_Bword(), and new_Gword() with a single function that has an extra parameter, "color". """ The first easy, the second is going to take some thinking.... Keep at it I will. I am hooked!!! Being able to write once and run on all of the above platforms is amazing!!! Thanks again!!! Dave John Posner wrote: > On Sat, Jan 2, 2010 at 1:47 PM, Dave McCormick wrote: > >> WooHoo!!! >> I got it!!! Yup, I am sure it can be optimized but it works!!! > > Dave, please ignore a couple of my bogus complaints in the previous > message: > > ... you call function new_Rword() before you define it > > ... this version also has the advantage of defining each function > just once, instead of multiple times on each keystroke > > > But I stand by my overall statement that the program didn't work for > me, and that rearranging the lines produces a working program. > > -John From vicente.soler at gmail.com Sun Jan 3 06:27:46 2010 From: vicente.soler at gmail.com (vsoler) Date: Sun, 3 Jan 2010 03:27:46 -0800 (PST) Subject: What is the best data structure for a very simple spreadsheet? Message-ID: Hi, Not sure this is the best group to post, but I cannot think of any other. My application would contain a limited set of "cells" represented by the instances of a Cell class: class Cell: ... A1=Cell(7) A2=Cell(2*A1) A3=Cell(3*A1+A2) A4=Cell(A3*4) Of course, A1 = 7, A2 = 14, A3 = 35 and A4 = 140 Now, I somehow want to be able to show a dependency tree 1 level dependency trees A1: None A2: A1 A3: A1, A2 A4: A3 All levels dependency trees A1: None A2: A1 A3: A1, A2 A4: A3, A2, A1 Leaf + values dependency trees: A1: 7 A2: A1=7, 2 A3: 3, A1=7, 2 A4: 3, A1=7, 2, 4 What I'd like to know is: 1) what are, in your opinion, the basic elements of the Cell class? 2) Do I need a parser to evaluate the formulas like ?3*A1+A2?? Can you recommend one library that already contains one? 3) Do I need a tree data structure to represent my data? would the tree be an attribute of the class instance? I imagine a lot can be said on these questions. What I am looking for is some hints that help me get out of where I am now. Any help is highly appreciated. Vicente Soler From python at bdurham.com Sun Jan 3 07:09:52 2010 From: python at bdurham.com (python at bdurham.com) Date: Sun, 03 Jan 2010 07:09:52 -0500 Subject: Windows 7 : any problems installing or running Python ? In-Reply-To: <000001ca8bb6$08391d70$18ab5850$@com> References: <000001ca8bb6$08391d70$18ab5850$@com> Message-ID: <1262520592.29221.1352686197@webmail.messagingengine.com> David, Try disabling your firewall software momentarily to see if that makes a difference. IDLE uses a local port for inter-process communication - you may need to configure your firewall to allow IDLE's port usage. To test whether Python itself has been properly installed, open up a cmd prompt, change your current directory to the folder where you installed Python, create the short, 1 line script that follows, and then run it. If this works, then Python has been successfully installed on your workstation and the problem you're having is specific to IDLE. 1 line script (place in hello.py) print "Hello world" Command line to run above script: python hello.py You might also try installing an alternate Python build, eg. if you installed from python.org, then try one of the setups from ActiveState. Or vice-versa. We're running Python 2.6.4 on Windows XP (32-bit), Windows Vista (32 and 64-bit), and Windows 7 (32 and 64-bit) without problem. Good luck! Malcolm -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at REMOVE-THIS-cybersource.com.au Sun Jan 3 07:28:40 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 03 Jan 2010 12:28:40 GMT Subject: What is the best data structure for a very simple spreadsheet? References: Message-ID: <03507a71$0$1277$c3e8da3@news.astraweb.com> On Sun, 03 Jan 2010 03:27:46 -0800, vsoler wrote: > My application would contain a limited set of "cells" represented by the > instances of a Cell class: > > class Cell: > ... > > A1=Cell(7) > A2=Cell(2*A1) > A3=Cell(3*A1+A2) > A4=Cell(A3*4) > > Of course, A1 = 7, A2 = 14, A3 = 35 and A4 = 140 > > Now, I somehow want to be able to show a dependency tree > > 1 level dependency trees > A1: None > A2: A1 > A3: A1, A2 > A4: A3 > > All levels dependency trees > > A1: None > A2: A1 > A3: A1, A2 > A4: A3, A2, A1 > > Leaf + values dependency trees: > > A1: 7 > A2: A1=7, 2 > A3: 3, A1=7, 2 > A4: 3, A1=7, 2, 4 > > What I'd like to know is: > > 1) what are, in your opinion, the basic elements of the Cell class? def Cell(object): def __init__(self, payload): self.payload = payload def __str__(self): return str(self.payload) def __float__(self): return float(self.payload) def dependency(self): try: return self.payload.dependency() except AttributeError: return ['None'] Cells can contain one of three things: a number, a string, or a formula. The first two can be supported by providing a built-in Python object (float or str) as payload. You can support formulae two ways that I can think of: (1) Provide a formula as a string, with a leading '='. Then, whenever you want to evaluate such a cell, you fetch the string from the cell, parse it, generate an arithmetic expression, and calculate it. (2) Instead of parsing the formula on every single spreadsheet refresh, use a couple of helper classes: class Indirect(object): def __init__(self, ref, sheet=None): if sheet is None: self.sheet = default_sheet() else: self.sheet = sheet self.ref = ref def __str__(self): return str(self.sheet[self.ref]) def float(self): return float(self.sheet[self.ref]) def dependency(self): return [self.ref] class Formula(object): def __init__(self, x, y, op): self.x = x self.y = y self.op = op def eval(self): return self.op(float(x), float(y)) def dependency(self): return self.x.dependency(level) + self.y.dependency(level) Then do something like this: sheet = {} sheet['A1'] = Cell(7) sheet['A2'] = Cell(Formula(2, Indirect('A1'), operator.mul)) sheet['A3'] = Cell( Formula( Formula(3, Indirect('A1'), operator.mul), Indirect('A2'), operator.add )) sheet['A4'] = Cell(Formula(Indirect('A3'), 4, operator.mul)) Then you only need to parse each human-readable formula like '=2*A1' once. > 2) Do I need a parser to evaluate the formulas like ?3*A1+A2?? Yes. > Can you recommend one library that already contains one? Try PyParsing. > 3) Do I need a tree > data structure to represent my data? would the tree be an attribute of > the class instance? I suspect a dict will be faster. To get the dependencies of each cell: for key, value in sheet.items(): print key, value.dependency() Keep in mind I haven't tested ANY of this -- it is entirely stream of consciousness. I've never actually done this, so I have no idea whether it is a good approach or not, but it seems to me that it might be. -- Steven From vicente.soler at gmail.com Sun Jan 3 07:38:28 2010 From: vicente.soler at gmail.com (vsoler) Date: Sun, 3 Jan 2010 04:38:28 -0800 (PST) Subject: What is the best data structure for a very simple spreadsheet? References: <03507a71$0$1277$c3e8da3@news.astraweb.com> Message-ID: <339dfbd9-d6d9-4f53-b998-46092f85d260@m25g2000yqc.googlegroups.com> On Jan 3, 1:28?pm, Steven D'Aprano wrote: > On Sun, 03 Jan 2010 03:27:46 -0800, vsoler wrote: > > My application would contain a limited set of "cells" represented by the > > instances of a Cell class: > > > class Cell: > > ... > > > A1=Cell(7) > > A2=Cell(2*A1) > > A3=Cell(3*A1+A2) > > A4=Cell(A3*4) > > > Of course, A1 = 7, A2 = 14, A3 = 35 and A4 = 140 > > > Now, I somehow want to be able to show a dependency tree > > > 1 level dependency trees > > ? A1: None > > ? A2: A1 > > ? A3: A1, A2 > > ? A4: A3 > > > All levels dependency trees > > > ? A1: None > > ? A2: A1 > > ? A3: A1, A2 > > ? A4: A3, A2, A1 > > > Leaf + values dependency trees: > > > ? A1: 7 > > ? A2: A1=7, 2 > > ? A3: 3, A1=7, 2 > > ? A4: 3, A1=7, 2, 4 > > > What I'd like to know is: > > > 1) what are, in your opinion, the basic elements of the Cell class? > > def Cell(object): > ? ? def __init__(self, payload): > ? ? ? ? self.payload = payload > ? ? def __str__(self): > ? ? ? ? return str(self.payload) > ? ? def __float__(self): > ? ? ? ? return float(self.payload) > ? ? def dependency(self): > ? ? ? ? try: > ? ? ? ? ? ? return self.payload.dependency() > ? ? ? ? except AttributeError: > ? ? ? ? ? ? return ['None'] > > Cells can contain one of three things: a number, a string, or a formula. > The first two can be supported by providing a built-in Python object > (float or str) as payload. You can support formulae two ways that I can > think of: > > (1) Provide a formula as a string, with a leading '='. Then, whenever you > want to evaluate such a cell, you fetch the string from the cell, parse > it, generate an arithmetic expression, and calculate it. > > (2) Instead of parsing the formula on every single spreadsheet refresh, > use a couple of helper classes: > > class Indirect(object): > ? ? def __init__(self, ref, sheet=None): > ? ? ? ? if sheet is None: > ? ? ? ? ? ? self.sheet = default_sheet() > ? ? ? ? else: > ? ? ? ? ? ? self.sheet = sheet > ? ? ? ? self.ref = ref > ? ? def __str__(self): > ? ? ? ? return str(self.sheet[self.ref]) > ? ? def float(self): > ? ? ? ? return float(self.sheet[self.ref]) > ? ? def dependency(self): > ? ? ? ? return [self.ref] > > class Formula(object): > ? ? def __init__(self, x, y, op): > ? ? ? ? self.x = x > ? ? ? ? self.y = y > ? ? ? ? self.op = op > ? ? def eval(self): > ? ? ? ? return self.op(float(x), float(y)) > ? ? def dependency(self): > ? ? ? ? return self.x.dependency(level) + self.y.dependency(level) > > Then do something like this: > > sheet = {} > sheet['A1'] = Cell(7) > sheet['A2'] = Cell(Formula(2, Indirect('A1'), operator.mul)) > sheet['A3'] = Cell( > ? Formula( > ? ? Formula(3, Indirect('A1'), operator.mul), > ? ? Indirect('A2'), > ? ? operator.add > ? ? )) > sheet['A4'] = Cell(Formula(Indirect('A3'), 4, operator.mul)) > > Then you only need to parse each human-readable formula like '=2*A1' once. > > > 2) Do I need a parser to evaluate the formulas like ?3*A1+A2?? > > Yes. > > > Can you recommend one library that already contains one? > > Try PyParsing. > > > 3) Do I need a tree > > data structure to represent my data? would the tree be an attribute of > > the class instance? > > I suspect a dict will be faster. > > To get the dependencies of each cell: > > for key, value in sheet.items(): > ? ? print key, value.dependency() > > Keep in mind I haven't tested ANY of this -- it is entirely stream of > consciousness. I've never actually done this, so I have no idea whether > it is a good approach or not, but it seems to me that it might be. > > -- > Steven WOW!!! After lunch I am going to read your post thoroughly, but I can already see that you've put into it a lot of time and expertise. Thank you From ktenney at gmail.com Sun Jan 3 09:05:44 2010 From: ktenney at gmail.com (Kent Tenney) Date: Sun, 3 Jan 2010 08:05:44 -0600 Subject: Trying to run a sudo command from script In-Reply-To: <7q7dkaF3t8r0rU1@mid.uni-berlin.de> References: <7q7dkaF3t8r0rU1@mid.uni-berlin.de> Message-ID: On Fri, Jan 1, 2010 at 5:08 PM, Diez B. Roggisch wrote: > Kent Tenney schrieb: >> >> Howdy, >> >> A script running as a regular user sometimes wants >> to run sudo commands. >> >> It gets the password with getpass. >> pw = getpass.getpass() >> >> I've fiddled a bunch with stuff like >> proc = subprocess.Popen('sudo touch /etc/foo'.split(), >> stdin=subprocess.PIPE) >> proc.communicate(input=pw) >> >> getting assorted errors with all variations I try. >> >> Googling says use pexpect, but I'd prefer a stdlib solution. > > pexpect is pure python, and it's needed. There is no easy way around the > issue, so if you insist on not using pexpect, you re-invent the wheel and > write the exact same code - just more error-prone, because of > wheel-reinvention.... Indeed, the requirements of this are way more complex than I guessed. The following seems to work well, took some fiddling with EOF. def sudo(command, password=None, prompt="Enter password "): import pexpect if not password: import getpass password = getpass.getpass(prompt) command = "sudo " + command child = pexpect.spawn(command) child.expect(['ssword', pexpect.EOF]) child.sendline(password) child.expect(pexpect.EOF) # is this necessary? child.close() Thanks, Kent > > Diez > -- > http://mail.python.org/mailman/listinfo/python-list > From steve at holdenweb.com Sun Jan 3 09:31:57 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 03 Jan 2010 09:31:57 -0500 Subject: Safe file I/O to shared file (or SQLite) from multi-threaded web server In-Reply-To: <4b403899$0$1646$742ec2ed@news.sonic.net> References: <4b403899$0$1646$742ec2ed@news.sonic.net> Message-ID: <4B40AA5D.1090507@holdenweb.com> John Nagle wrote: > python at bdurham.com wrote: >> I'm looking for the best practice way for a multi-threaded python web >> server application to read/write to a shared file or a SQLite database. >> >> What do I need to do (if anything) to make sure my writes to a regular >> file on disk or to a SQLite database are atomic in nature when multiple >> clients post data to my application simultaneously? > > SQLite can do that correctly, but SQLite isn't intended for use > as a database engine for a busy database being used by many > concurrent operations. Especially if those operations involve > updates. Any update in SQLite locks all tables involved for the duration > of the operation. When SQLite hits a lock, it returns an error code, and > the caller should retry after a delay. If this occurs frequently in > your application, you've hit the limits of SQLite. Then it's > time to move up to MySQL. > Or PostgreSQL, which has superior SQL standards conformance and excellent high-volume data performance. > If you have enough traffic that you need a multi-threaded web server, > it's probably time to move up. > Yes, but not to MySQL, please. Particularly since there is a sword of Damocles hanging over its head while the Oracle takeover of Sun is pending. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Sun Jan 3 09:31:57 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 03 Jan 2010 09:31:57 -0500 Subject: Safe file I/O to shared file (or SQLite) from multi-threaded web server In-Reply-To: <4b403899$0$1646$742ec2ed@news.sonic.net> References: <4b403899$0$1646$742ec2ed@news.sonic.net> Message-ID: <4B40AA5D.1090507@holdenweb.com> John Nagle wrote: > python at bdurham.com wrote: >> I'm looking for the best practice way for a multi-threaded python web >> server application to read/write to a shared file or a SQLite database. >> >> What do I need to do (if anything) to make sure my writes to a regular >> file on disk or to a SQLite database are atomic in nature when multiple >> clients post data to my application simultaneously? > > SQLite can do that correctly, but SQLite isn't intended for use > as a database engine for a busy database being used by many > concurrent operations. Especially if those operations involve > updates. Any update in SQLite locks all tables involved for the duration > of the operation. When SQLite hits a lock, it returns an error code, and > the caller should retry after a delay. If this occurs frequently in > your application, you've hit the limits of SQLite. Then it's > time to move up to MySQL. > Or PostgreSQL, which has superior SQL standards conformance and excellent high-volume data performance. > If you have enough traffic that you need a multi-threaded web server, > it's probably time to move up. > Yes, but not to MySQL, please. Particularly since there is a sword of Damocles hanging over its head while the Oracle takeover of Sun is pending. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From lie.1296 at gmail.com Sun Jan 3 09:58:05 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Mon, 04 Jan 2010 01:58:05 +1100 Subject: What is the best data structure for a very simple spreadsheet? In-Reply-To: References: Message-ID: <4b40b07e@dnews.tpgi.com.au> On 1/3/2010 10:27 PM, vsoler wrote: > 1) what are, in your opinion, the basic elements of the Cell class? The "user-entered formula" and "effective value". A Cell containing a formula "abc" has a value of "abc"; a cell containing the formula "=1+5" has a value of "6". You could use the 'property' decorator for the "effective value" attribute. > 2) Do I need a parser to evaluate the formulas like ?3*A1+A2?? Can you > recommend one library that already contains one? Yes you do. PyParsing, regex, write your own parser; choose your own poison depending on the expected complexity of the formulaes. > 3) Do I need a tree data structure to represent my data? There are two main options for your "primary data structure": - table (2x2 list) -- if you expect most of your cells to be filled, this is simple, but uses lots of memory - sparse table (list/dicts of cells) -- if you have a large table that is mostly empty cells The Dependency Tree is the "side data structure"; a "Cell" is a "Node"; a Cell's "dependencies" is the Node's "children"; a Cell's "dependant" is the Node's "parent". Cells maintains a list of dependencies (or dynamically generates them when requested) by analyzing the "formula" attribute for references to other cells. PS: Strictly speaking, this "side data structure" is not a "Tree" as there are multiple root nodes. I think it's called "Directed Acyclic Graph". However, it's true that, given a cell as a root, the cell's dependencies is a proper tree. > would the > tree be an attribute of the class instance? you could use 'property' decorator and make the dependency tree a "computed attribute" of the cell; many people recommended against a 'property' that is heavy to compute though and calculating dependencies may be quite heavy depending on the complexity of the spreadsheet. > I imagine a lot can be said on these questions. What I am looking for > is some hints that help me get out of where I am now. > > Any help is highly appreciated. > > Vicente Soler From joncle at googlemail.com Sun Jan 3 10:25:49 2010 From: joncle at googlemail.com (Jon Clements) Date: Sun, 3 Jan 2010 07:25:49 -0800 (PST) Subject: What is the best data structure for a very simple spreadsheet? References: <4b40b07e@dnews.tpgi.com.au> Message-ID: <6aad5bae-e50a-425c-b9c4-bc0c2d595670@o28g2000yqh.googlegroups.com> On Jan 3, 2:58?pm, Lie Ryan wrote: > On 1/3/2010 10:27 PM, vsoler wrote: > > > 1) what are, in your opinion, the basic elements of the Cell class? > > The "user-entered formula" and "effective value". A Cell containing a > formula "abc" has a value of "abc"; a cell containing the formula "=1+5" > has a value of "6". You could use the 'property' decorator for the > "effective value" attribute. > > > 2) Do I need a parser to evaluate the formulas like 3*A1+A2 ? Can you > > recommend one library that already contains one? > > Yes you do. PyParsing, regex, write your own parser; choose your own > poison depending on the expected complexity of the formulaes. > > > 3) Do I need a tree data structure to represent my data? > > There are two main options for your "primary data structure": > - table (2x2 list) -- if you expect most of your cells to be filled, > this is simple, but uses lots of memory > - sparse table (list/dicts of cells) -- if you have a large table that > is mostly empty cells > > The Dependency Tree is the "side data structure"; a "Cell" is a "Node"; > a Cell's "dependencies" is the Node's "children"; a Cell's "dependant" > is the Node's "parent". Cells maintains a list of dependencies (or > dynamically generates them when requested) by analyzing the "formula" > attribute for references to other cells. > > PS: Strictly speaking, this "side data structure" is not a "Tree" as > there are multiple root nodes. I think it's called "Directed Acyclic > Graph". However, it's true that, given a cell as a root, the cell's > dependencies is a proper tree. > > > would the > > tree be an attribute of the class instance? > > you could use 'property' decorator and make the dependency tree a > "computed attribute" of the cell; many people recommended against a > 'property' that is heavy to compute though and calculating dependencies > may be quite heavy depending on the complexity of the spreadsheet. > > > I imagine a lot can be said on these questions. What I am looking for > > is some hints that help me get out of where I am now. > > > Any help is highly appreciated. > > > Vicente Soler > > As well as what Steven & Lie have mentioned, maybe you could get inspiration from http://pyspread.sourceforge.net/ Although I've not looked at it, my gut feeling is the author would've had to address these issues (take that as a disclaimer of some sort :) ) >From not having to had a need for this sort of thing, I'd probably go for a sparse representation as a graph. But again, depends why you're segmenting the "spreadsheet" from "need to tell dependencies". Jon. From laiwei.ustc at gmail.com Sun Jan 3 11:01:22 2010 From: laiwei.ustc at gmail.com (laiwei.ustc) Date: Mon, 4 Jan 2010 00:01:22 +0800 Subject: HAppy New Year References: Message-ID: <201001040001187704615@gmail.com> HI Baboucarr good work beautiful thanks 2010-01-03 laiwei.ustc ???? baboucarr sanneh ????? 2010-01-03 22:32:29 ???? python-list at python.org ??? ??? HAppy New Year Hi guys, Jus want to wish you a happy new year to u all.. 1.Copy the content below and Paste it on a Notepad. 2 . Use replace all (Ctrl + H) 3. Type in ?6? in the ?find what? column 4. Type in _ (underscore) in the ?replace with? column 5. Click on replace all. 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666996666699669966999999996699666669966666669966666699669999999966996666996666666666666696666666666666666666666 666996666699669966999999996699666669966666666996666996669999999966996666996666666666666999666666666666666666666 666996666699669966996666666699666669966666666699669966669966669966996666996666666666669969966666666666666666666 666996696699669966999999996699999999966666666669999666669966669966996666996666666666699666996666666666666666666 666996999699669966999999996699999999966666666666996666669966669966996666996666666666999999999666666666666666666 666999969999669966666666996699666669966666666666996666669966669966996666996666666669999999999966666666666666666 666999666999669966999999996699666669966666666666996666669999999966996666996666666699666666666996666666666666666 666996666699669966999999996699666669966666666666996666669999999966999999996666666996666666666699666666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 669966666996666666669666666666999999996699999999669966666699666666669966666996699999999669966666996666666666666 669966666996666666699966666666999999996699999999666996666996666666669996666996699999999669966666996666666666666 669966666996666666996996666666996666996699666699666699669966666666669999666996699666666669966666996666666666666 669999999996666669966699666666999999996699999999666669999666666666669969966996699999999669966966996666666666666 669999999996666699999999966666999999996699999999666666996666666666669966996996699999999669969996996666666666666 669966666996666999999999996666999666666699666666666666996666666666669966699996699666666669999699996666666666666 669966666996669966666666699666996666666699666666666666996666666666669966669996699999999669996669996666666666666 669966666996699666666666669966996666666699666666666666996666666666669966666996699999999669966666996666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666996666669966999999996666666669666666666999999996666666699999996699999999666999666699999999666666666666666666 666699666699666999999996666666699966666666999999996666666699999999699666699666699666699666699666666666666666666 666669966996666996666666666666996996666666996666996666666666666699699966699666699666699966699666666666666666666 666666999966666999999996666669966699666666999999996666666666666996699696699666699666699696699666666666666666666 666666699666666999999996666699999999966666999999996666666666669966699669699666699666699669699666666666666666666 666666699666666996666666666999999999996666996996666666666666996666699666999666699666699666999666666666666666666 666666699666666999999996669966666666699666996699666666666699999999699666699666699666699666699666666666666666666 666666699666666999999996699666666666669966996666996666666699999999699999999666999966699999999666666666666666666 66666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 Regards, Baboucarr. Windows Live: Friends get your Flickr, Yelp, and Digg updates when they e-mail you. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: laiwei.ustc at gmail.com.jpg Type: image/jpeg Size: 7941 bytes Desc: not available URL: From jackdied at gmail.com Sun Jan 3 11:07:07 2010 From: jackdied at gmail.com (Jack Diederich) Date: Sun, 3 Jan 2010 11:07:07 -0500 Subject: Any Swisses here? In-Reply-To: References: <145e6acd-4260-495b-b562-da634fccfa1a@j19g2000yqk.googlegroups.com> <4B400E83.1080305@yahoo.com.br> Message-ID: On Sat, Jan 2, 2010 at 10:38 PM, n00m wrote: > On Jan 3, 5:30?am, Steve Holden wrote: >> Zhu Sha Zang wrote: >> >> [stuff and nonsense from a third party] >> > WTF? >> >> We do get the occasional bigot dropping in from time to time. Best to >> ignore them 'til they go away. >> > And who are YOU? > What you achieved in the life? > For me you is nothing I recall only Kipling, Dickens, Newton > its your background. My backgroung is the best rockets Don't be so dismissive Steve. What we have here is clearly a struggling young AI trying to extrapolate the average content of an alt.group to the comp.lang community. The fact that it passed the gmail spam filters is a sign of progress. So hello new friend, and your backgroung is the best rockets! -Jack From cjns1989 at gmail.com Sun Jan 3 12:20:26 2010 From: cjns1989 at gmail.com (Chris Jones) Date: Sun, 03 Jan 2010 12:20:26 -0500 Subject: twenty years ago Guido created Python In-Reply-To: References: Message-ID: <20100103172026.GO3125@turki.gavron.org> On Thu, Dec 31, 2009 at 05:06:24PM EST, Steve Howell wrote: > FYI: > > http://twitter.com/gvanrossum > > Python is a truly awesome programming language. Not only is Guido a > genius language designer, but he is also a great project leader. What > an accomplishment. Congratulations to everybody who has contributed > to Python in the last two decades! Interesting to note that Guido's achievements prompt much less response, and get much less coverage, than Xah Lee's :-) I guess Guido's not one to care much about beauty contests anyway. HNY, CJ From python at mrabarnett.plus.com Sun Jan 3 14:13:27 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 03 Jan 2010 19:13:27 +0000 Subject: What is the best data structure for a very simple spreadsheet? In-Reply-To: References: Message-ID: <4B40EC57.7000401@mrabarnett.plus.com> vsoler wrote: > Hi, > > Not sure this is the best group to post, but I cannot think of any > other. > > My application would contain a limited set of "cells" represented by > the instances of a Cell class: > > class Cell: > ... > > A1=Cell(7) > A2=Cell(2*A1) > A3=Cell(3*A1+A2) > A4=Cell(A3*4) > > Of course, A1 = 7, A2 = 14, A3 = 35 and A4 = 140 > > Now, I somehow want to be able to show a dependency tree > > 1 level dependency trees > A1: None > A2: A1 > A3: A1, A2 > A4: A3 > > All levels dependency trees > > A1: None > A2: A1 > A3: A1, A2 > A4: A3, A2, A1 > > Leaf + values dependency trees: > > A1: 7 > A2: A1=7, 2 > A3: 3, A1=7, 2 > A4: 3, A1=7, 2, 4 > > What I'd like to know is: > > 1) what are, in your opinion, the basic elements of the Cell class? > 2) Do I need a parser to evaluate the formulas like ?3*A1+A2?? Can you > recommend one library that already contains one? > 3) Do I need a tree data structure to represent my data? would the > tree be an attribute of the class instance? > > I imagine a lot can be said on these questions. What I am looking for > is some hints that help me get out of where I am now. > > Any help is highly appreciated. > As well as considering the other replies, you don't necessarily need to parse the expressions yourself. If A1 is a cell then 2*A1 tries to call the __rmul__ method of A1 (and A1*2 tries to call the __mul__ method of A1), so you could have it return a formula object that will multiply and return the numeric value of self (A1) by 2 when its own numeric value is requested. This means that a cell could be initialised with either a number or a formula. From dotancohen at gmail.com Sun Jan 3 14:18:38 2010 From: dotancohen at gmail.com (Dotan Cohen) Date: Sun, 3 Jan 2010 21:18:38 +0200 Subject: HAppy New Year In-Reply-To: References: Message-ID: <880dece01001031118h4e6e8a7nac7f8a6b80ca8b5d@mail.gmail.com> 2010/1/2 baboucarr sanneh : > Hi guys, > > Jus want to wish you a happy new year to u all.. > > 1.Copy the content below and Paste it on a Notepad. > 2 . Use replace all (Ctrl + H) > 3. Type in ?6? in the ?find what? column > > 4.?? Type in _ (underscore) in the ?replace with? column > > 5.?? Click on replace all. > Wow, I just love Windows users. Sending this to the Python list? Paste "on a notepad"?!? The "Replace All" function in "a notepad" is Ctrl-H?!?? What does H stand for?!?!? Explicit instructions how to :%s/6/_/g in "a notepad"?!?!? Happy new year! -- Dotan Cohen http://what-is-what.com http://gibberish.co.il From steve at holdenweb.com Sun Jan 3 14:54:14 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 03 Jan 2010 14:54:14 -0500 Subject: HAppy New Year In-Reply-To: <880dece01001031118h4e6e8a7nac7f8a6b80ca8b5d@mail.gmail.com> References: <880dece01001031118h4e6e8a7nac7f8a6b80ca8b5d@mail.gmail.com> Message-ID: Dotan Cohen wrote: > 2010/1/2 baboucarr sanneh : >> Hi guys, >> >> Jus want to wish you a happy new year to u all.. >> >> 1.Copy the content below and Paste it on a Notepad. >> 2 . Use replace all (Ctrl + H) >> 3. Type in ?6? in the ?find what? column >> >> 4. Type in _ (underscore) in the ?replace with? column >> >> 5. Click on replace all. >> > > Wow, I just love Windows users. Sending this to the Python list? Paste > "on a notepad"?!? The "Replace All" function in "a notepad" is > Ctrl-H?!?? What does H stand for?!?!? Explicit instructions how to > :%s/6/_/g in "a notepad"?!?!? > > Happy new year! > What I want to know is why didn't he just write """\ ... original long string ... """.replace("6", "_") But watch that snarkiness ... I'm a Windows user too! regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From dotancohen at gmail.com Sun Jan 3 15:04:40 2010 From: dotancohen at gmail.com (Dotan Cohen) Date: Sun, 3 Jan 2010 22:04:40 +0200 Subject: HAppy New Year In-Reply-To: References: <880dece01001031118h4e6e8a7nac7f8a6b80ca8b5d@mail.gmail.com> Message-ID: <880dece01001031204v1f9f1552je08ca9e98970a732@mail.gmail.com> > What I want to know is why didn't he just write > > """\ > ... original long string ... > """.replace("6", "_") > Uf! Why didn't I think of that?! > But watch that snarkiness ... I'm a Windows user too! > That's fine, I know that fresh air has to get into the basement somehow ! -- Dotan Cohen http://what-is-what.com http://gibberish.co.il From dteslenko at gmail.com Sun Jan 3 15:07:17 2010 From: dteslenko at gmail.com (Dmitry Teslenko) Date: Sun, 3 Jan 2010 23:07:17 +0300 Subject: [gtk+thread] Why worker thread never wakes from time.sleep()? Message-ID: <91325fec1001031207s6bb8cf3am3b3d999cd94c1b99@mail.gmail.com> Hello! I have simple gui gtk app. It has worker thread that populates list with strings and gtk window with main loop which pops strings from this list and shows them in TreeView. Thread runs get_data_from_pcap to populate list with strings. Gtk app calls update_store() with gobject.timeout_add every second. If I comment time.sleep() in update_store(), worker thread never wakes up after his time.sleep(). Why? Here's runnable example: #!/usr/bin/python # -*- coding: utf-8 -*- import pygtk pygtk.require('2.0') import gtk import gobject import pcap import sys import threading import time CONSOLE_ENCODING = 'utf-8' if sys.platform == 'win32': CONSOLE_ENCODING = 'cp866' global_pcap_queue = [] global_pcap_lock = threading.Lock() global_pcap_stop_event = threading.Event() class CityGame: def __init__(self): window = gtk.Window(gtk.WINDOW_TOPLEVEL) window.connect('delete_event', self.delete_event) window.connect('destroy', self.destroy) store = gtk.ListStore(gobject.TYPE_STRING) view = gtk.TreeView(store) #name col = gtk.TreeViewColumn('Data') cell = gtk.CellRendererText() view.append_column(col) col.pack_start(cell, True) col.add_attribute(cell, 'text', 0) vb = gtk.VBox() vb.pack_start(view) window.add(vb) window.set_size_request(400, 300) window.show_all() self.__store = store def main(self): gobject.timeout_add(1000, self.update_store) gtk.main() def update_store(self): data = None global_pcap_lock.acquire() if len(global_pcap_queue): data = global_pcap_queue.pop(0) print 'Update' global_pcap_lock.release() time.sleep(0.01) if data: self.__store.append([data]) return True def delete_event(self, widget, event, data = None): dlg = gtk.MessageDialog(flags = gtk.DIALOG_MODAL, type = gtk.MESSAGE_QUESTION, buttons = gtk.BUTTONS_YES_NO, message_format = 'Are you sure you want to quit?') dlg.set_title('CityGame') result = dlg.run() dlg.destroy() return (result != gtk.RESPONSE_YES) def destroy(self, widget, data = None): gtk.main_quit() def main(args): cg = CityGame() cg.main() def get_data_from_pcap(): #while True: while not global_pcap_stop_event.isSet(): global_pcap_lock.acquire() global_pcap_queue.append(str(time.time())) print 'Thread' global_pcap_lock.release() time.sleep(0.01) return if __name__ == '__main__': global_pcap_stop_event.clear() pcap_thread = threading.Thread(target = get_data_from_pcap) pcap_thread.start() main(sys.argv[1:]) global_pcap_stop_event.set() pcap_thread.join() -- A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing in e-mail? From peloko45 at gmail.com Sun Jan 3 16:09:49 2010 From: peloko45 at gmail.com (Joan Miller) Date: Sun, 3 Jan 2010 13:09:49 -0800 (PST) Subject: Set variables based on dictionary Message-ID: <49caee31-b697-4aa2-8e83-8b43316a0d67@j5g2000yqm.googlegroups.com> How to set local variables based on dictionary contents? From ben+python at benfinney.id.au Sun Jan 3 16:19:02 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 04 Jan 2010 08:19:02 +1100 Subject: twenty years ago Guido created Python References: Message-ID: <871vi628ft.fsf@benfinney.id.au> Chris Jones writes: > Interesting to note that Guido's achievements prompt much less > response, and get much less coverage [?] The entirety of ?comp.lang.python? is an ongoing response to Guido van Rossum's achievements (and all the others that make Python great). Mere week- or month-long threads are paltry by comparison. -- \ ?If this is your first visit to the USSR, you are welcome to | `\ it.? ?hotel room, Moscow | _o__) | Ben Finney From massimodipierro71 at gmail.com Sun Jan 3 16:40:19 2010 From: massimodipierro71 at gmail.com (mdipierro) Date: Sun, 3 Jan 2010 13:40:19 -0800 (PST) Subject: What is the best data structure for a very simple spreadsheet? References: Message-ID: <82c5a9f1-ed36-40c4-83d6-2d2d33ee40bc@j24g2000yqa.googlegroups.com> Perhaps this can be useful: http://www.web2py.com/examples/spreadsheet The code is in a single file with not dependencies and it does not require web2py to run: http://code.google.com/p/web2py/source/browse/gluon/contrib/spreadsheet.py Here is a sample controller that shows you how to embed the spreadsheet in web page: http://code.google.com/p/web2py/source/browse/applications/examples/controllers/spreadsheet.py Massimo On Jan 3, 5:27?am, vsoler wrote: > Hi, > > Not sure this is the best group to post, but I cannot think of any > other. > > My application would contain a limited set of "cells" represented by > the instances of a Cell class: > > class Cell: > ... > > A1=Cell(7) > A2=Cell(2*A1) > A3=Cell(3*A1+A2) > A4=Cell(A3*4) > > Of course, A1 = 7, A2 = 14, A3 = 35 and A4 = 140 > > Now, I somehow want to be able to show a dependency tree > > 1 level dependency trees > ? A1: None > ? A2: A1 > ? A3: A1, A2 > ? A4: A3 > > All levels dependency trees > > ? A1: None > ? A2: A1 > ? A3: A1, A2 > ? A4: A3, A2, A1 > > Leaf + values dependency trees: > > ? A1: 7 > ? A2: A1=7, 2 > ? A3: 3, A1=7, 2 > ? A4: 3, A1=7, 2, 4 > > What I'd like to know is: > > 1) what are, in your opinion, the basic elements of the Cell class? > 2) Do I need a parser to evaluate the formulas like ?3*A1+A2?? Can you > recommend one library that already contains one? > 3) Do I need a tree data structure to represent my data? would the > tree be an attribute of the class instance? > > I imagine a lot can be said on these questions. What I am looking for > is some hints that help me get out of where I am now. > > Any help is highly appreciated. > > Vicente Soler From michi at triodia.com Sun Jan 3 16:44:29 2010 From: michi at triodia.com (Michi) Date: Sun, 3 Jan 2010 13:44:29 -0800 (PST) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> On Jan 1, 2:47?pm, Peng Yu wrote: > > In the article API Design Matters by Michi Henning > > Communications of the ACM > Vol. 52 No. 5, Pages 46-56 > 10.1145/1506409.1506424http://cacm.acm.org/magazines/2009/5/24646-api-design-matters/fulltext > > It says "Another popular design flaw?namely, throwing exceptions for > expected outcomes?also causes inefficiencies because catching and > handling exceptions is almost always slower than testing a return > value." > > My observation is contradicted to the above statement by Henning. If > my observation is wrong, please just ignore my question below. Seeing that quite a few people have put their own interpretation on what I wrote, I figured I'll post a clarification. The quoted sentence appears in a section of the article that deals with efficiency. I point out in that section that bad APIs often have a price not just in terms of usability and defect rate, but that they are often inefficient as well. (For example, wrapper APIs often require additional memory allocations and/or data copies.) Incorrect use of exceptions also incurs an efficiency penalty. In many language implementations, exception handling is expensive; significantly more expensive than testing a return value. Consider the following: int x; try { x = func(); } catch (SomeException) { doSomething(); return; } doSomethingElse(); Here is the alternative without exceptions. (func() returns SpecialValue instead of throwing.) int x; x = func(); if (x == SpecialValue) { doSomething(); return; } doSomethingElse(); In many language implementations, the second version is considerably faster, especially when the exception may be thrown from deep in the bowels of func(), possibly many frames down the call tree. If func() throws an exception for something that routinely occurs in the normal use of the API, the extra cost can be noticeable. Note that I am not advocating not to use exceptions. I *am* advocating to not throw exceptions for conditions that are not exceptional. The classic example of this are lookup functions that, for example, retrieve the value of an environment variable, do a table lookup, or similar. Many such APIs throw an exception when the lookup fails because the key isn't the table. However, very often, looking for something that isn't there is a common case, such as when looking for a value and, if the value isn't present already, adding it. Here is an example of this: KeyType k = ...; ValueType v; try { v = collection.lookup(k); } catch (NotFoundException) { collection.add(k, defaultValue); v = defaultValue; } doSomethingWithValue(v); The same code if collection doesn't throw when I look up something that isn't there: KeyType k = ...; ValueType v; v = collection.lookup(k); if (v == null) { collection.add(k, defaultValue); v = defaultValue; } doSomethingWithValue(v); The problem is that, if I do something like this in a loop, and the loop is performance-critical, the exception version can cause a significant penalty. As the API designer, when I make the choice between returning a special value to indicate some condition, or throwing an exception, I should consider the following questions: * Is the special condition such that, under most conceivable circumstances, the caller will treat the condition as an unexpected error? * Is it appropriate to force the caller to deal with the condition in a catch-handler? * If the caller fails to explicitly deal with the condition, is it appropriate to terminate the program? Only if the answer to these questions is "yes" is it appropriate to throw an exception. Note the third question, which is often forgotten. By throwing an exception, I not only force the caller to handle the exception with a catch-handler (as opposed to leaving the choice to the caller), I also force the caller to *always* handle the exception: if the caller wants to ignore the condition, he/she still has to write a catch-handler and failure to do so terminates the program. Apart from the potential performance penalty, throwing exceptions for expected outcomes is bad also because it forces a try-catch block on the caller. One example of this is the .NET socket API: if I do non- blocking I/O on a socket, I get an exception if no data is ready for reading (which is the common and expected case), and I get a zero return value if the connection was lost (which is the uncommon and unexpected case). In other words, the .NET API gets this completely the wrong way round. Code that needs to do non-blocking reads from a socket turns into a proper mess as a result because the outcome of a read() call is tri- state: * Data was available and returned: no exception * No data available: exception * Connection lost: no exception Because such code normally lives in a loop that decrements a byte count until the expected number of bytes have been read, the control flow because really awkward because the successful case must be dealt with in both the try block and the catch handler, and the error condition must be dealt with in the try block as well. If the API did what it should, namely, throw an exception when the connection is lost, and not throw when I do a read (whether data was ready or not), the code would be far simpler and far more maintainable. At no point did I ever advocate not to use exception handling. Exceptions are the correct mechanism to handle errors. However, what is considered an error is very much in the eye of the beholder. As the API creator, if I indicate errors with exceptions, I make a policy decision about what is an error and what is not. It behooves me to be conservative in that policy: I should throw exceptions only for conditions that are unlikely to arise during routine and normal use of the API. Cheers, Michi. From darcy at druid.net Sun Jan 3 16:45:00 2010 From: darcy at druid.net (D'Arcy J.M. Cain) Date: Sun, 3 Jan 2010 16:45:00 -0500 Subject: HAppy New Year In-Reply-To: <880dece01001031118h4e6e8a7nac7f8a6b80ca8b5d@mail.gmail.com> References: <880dece01001031118h4e6e8a7nac7f8a6b80ca8b5d@mail.gmail.com> Message-ID: <20100103164500.3277112c.darcy@druid.net> On Sun, 3 Jan 2010 21:18:38 +0200 Dotan Cohen wrote: > Wow, I just love Windows users. Sending this to the Python list? Paste I was going to ask him if his mommy and daddy knew that he was playing with the grown-ups' computer. -- D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner. From fpm at u.washington.edu Sun Jan 3 16:54:16 2010 From: fpm at u.washington.edu (cassiope) Date: Sun, 3 Jan 2010 13:54:16 -0800 (PST) Subject: IOError - cannot create file (linux daemon-invoked script) References: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> Message-ID: On Jan 2, 3:46?pm, Steve Holden wrote: > cassiope wrote: > > I have a daemon on a Linux system that supports a number of Windows > > clients. ?Among the functions is to send e-mails, which is > > sufficiently complicated that I fork() a separate process which gets > > setuid to a lesser user, and calls a python script which does the > > actual formatting and emailing (the daemon is written in C). ?I want > > to save a copy of the email in a particular directory which is > > accessible to the Windows clients via samba. > > > The strange thing is that even with the right user-id, I cannot seem > > to write to the directory, getting an IOError exception. ?Changing the > > directory to world-writable fixes this. ?I can confirm the uid and gid > > for the script by having the script print these values just before > > trying to create/write the file. ?Becoming the same lesser user, I > > have no problem writing a file to the same directory. > > Have you looked at the IOError's errno attribute to find out exactly why > the Python subprocess is unable to write to the directory? It's errno=13 ... "permission denied". > > Is there anything that I can do to diagnose why this script is > > failing? ?For various reasons I don't want to make the directory world- > > writable. > > I'd concur on that decision. > > > This is on a Debian "squeeze" system, with python 2.5. > > > Thanks for any insights! > > Take a closer look at the exception, that might stimulate a thought or two. > > regards > ?Steve > -- > Steve Holden ? ? ? ? ? +1 571 484 6266 ? +1 800 494 3119 > PyCon is coming! Atlanta, Feb 2010 ?http://us.pycon.org/ > Holden Web LLC ? ? ? ? ? ? ? ?http://www.holdenweb.com/ > UPCOMING EVENTS: ? ? ? ?http://holdenweb.eventbrite.com/ From fpm at u.washington.edu Sun Jan 3 16:56:24 2010 From: fpm at u.washington.edu (cassiope) Date: Sun, 3 Jan 2010 13:56:24 -0800 (PST) Subject: IOError - cannot create file (linux daemon-invoked script) References: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> Message-ID: <40b81356-f0c1-4249-a536-916356377afa@u41g2000yqe.googlegroups.com> On Jan 2, 6:40?pm, Christian Heimes wrote: > cassiope wrote: > > The strange thing is that even with the right user-id, I cannot seem > > to write to the directory, getting an IOError exception. ?Changing the > > directory to world-writable fixes this. ?I can confirm the uid and gid > > for the script by having the script print these values just before > > trying to create/write the file. ?Becoming the same lesser user, I > > have no problem writing a file to the same directory. > > Are you able to write to the directory with the user id when you tried > to create a file manually? Yes. Sorry that wasn't clear. > How are you changing the uid and gid of your script? IIRC you have to > set the effective user id with os.seteuid() and os.setegid(). I'm changing the uid and gid in the daemon (which runs with root permissions until the fork and uid/gid change). The uid and gid are confirmed by printing os.getuid() and os.getgid() in the script. > Christian From __peter__ at web.de Sun Jan 3 16:57:09 2010 From: __peter__ at web.de (Peter Otten) Date: Sun, 03 Jan 2010 22:57:09 +0100 Subject: Set variables based on dictionary References: <49caee31-b697-4aa2-8e83-8b43316a0d67@j5g2000yqm.googlegroups.com> Message-ID: Joan Miller wrote: > How to set local variables based on dictionary contents? >>> def f(**d): ... for k, v in d.iteritems(): exec "%s = v" % k ... return locals() ... >>> f(a=42, b="yadda") {'a': 42, 'k': 'b', 'b': 'yadda', 'd': {'a': 42, 'b': 'yadda'}, 'v': 'yadda'} >>> f(**{"print 'warning: this can execute arbitrary code'; a":42}) warning: this can execute arbitrary code {'a': 42, 'k': "print 'warning: this can execute arbitrary code'; a", 'd': {"print 'warning: this can execute arbitrary code'; a": 42}, 'v': 42} Peter From steve at holdenweb.com Sun Jan 3 17:13:32 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 03 Jan 2010 17:13:32 -0500 Subject: IOError - cannot create file (linux daemon-invoked script) In-Reply-To: <40b81356-f0c1-4249-a536-916356377afa@u41g2000yqe.googlegroups.com> References: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> <40b81356-f0c1-4249-a536-916356377afa@u41g2000yqe.googlegroups.com> Message-ID: cassiope wrote: > On Jan 2, 6:40 pm, Christian Heimes wrote: >> cassiope wrote: >>> The strange thing is that even with the right user-id, I cannot seem >>> to write to the directory, getting an IOError exception. Changing the >>> directory to world-writable fixes this. I can confirm the uid and gid >>> for the script by having the script print these values just before >>> trying to create/write the file. Becoming the same lesser user, I >>> have no problem writing a file to the same directory. >> Are you able to write to the directory with the user id when you tried >> to create a file manually? > > Yes. Sorry that wasn't clear. > >> How are you changing the uid and gid of your script? IIRC you have to >> set the effective user id with os.seteuid() and os.setegid(). > > I'm changing the uid and gid in the daemon (which runs with root > permissions > until the fork and uid/gid change). The uid and gid are confirmed by > printing os.getuid() and os.getgid() in the script. > And what do os.geteuid() and os.getegid() report? I suppose it's possible there's some bizarre difference between the effective and actual process parameters. IS the filestore a local file system, or an NFS mount? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From fpm at u.washington.edu Sun Jan 3 17:20:19 2010 From: fpm at u.washington.edu (cassiope) Date: Sun, 3 Jan 2010 14:20:19 -0800 (PST) Subject: IOError - cannot create file (linux daemon-invoked script) References: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> Message-ID: <8617a8f1-7567-4728-9568-9d03245df1fd@j24g2000yqa.googlegroups.com> On Jan 2, 8:02?pm, Cameron Simpson wrote: > On 02Jan2010 15:21, cassiope wrote: > | [...] ?I want > | to save a copy of the email in a particular directory which is > | accessible to the Windows clients via samba. > | > | The strange thing is that even with the right user-id, I cannot seem > | to write to the directory, getting an IOError exception. ?Changing the > | directory to world-writable fixes this. ?I can confirm the uid and gid > | for the script by having the script print these values just before > | trying to create/write the file. ?Becoming the same lesser user, I > | have no problem writing a file to the same directory. > > Can you show us: > ? - the directory user and group ownership and permissions > ? - the daemon's user and group values? Directory permissions: 774 Directory ownership: "lesser user", "special group" where /etc/group has "special group" members including the "lesser user", as well as those who are expected to use the daemon, but not root. Script ownership: "lesser user"; permissions 755 Daemon ownership: root; permissions: 755 (always started by root). The script also has to connect to a postgresql database for part of its work - that part works, > You can also strace your daemon: > > ? strace -f -e trace=file your-daemon your-daemon-args... 2>strace.out > > and then examine the log for the precise UNIX-level failure. > > Cheers, > -- > Cameron Simpson DoD#743http://www.cskk.ezoshosting.com/cs/ > > Money won't buy happiness, but it will pay the salary of a large research > staff to study the problem. - Bill Vaughan Thanks, Cameron (and Steve and Christian). My first shot with strace (it's been awhile since I've used that - I think your syntax may be a tiny bit off - but it's probably the tool I need to use. Will explore further... From cjns1989 at gmail.com Sun Jan 3 17:20:21 2010 From: cjns1989 at gmail.com (Chris Jones) Date: Sun, 03 Jan 2010 17:20:21 -0500 Subject: twenty years ago Guido created Python In-Reply-To: <871vi628ft.fsf@benfinney.id.au> References: <871vi628ft.fsf@benfinney.id.au> Message-ID: <20100103222021.GU3125@turki.gavron.org> On Sun, Jan 03, 2010 at 04:19:02PM EST, Ben Finney wrote: > Chris Jones writes: > > Interesting to note that Guido's achievements prompt much less > > response, and get much less coverage [?] > > The entirety of ?comp.lang.python? is an ongoing response to Guido van > Rossum's achievements (and all the others that make Python great). > Mere week- or month-long threads are paltry by comparison. That I understand. But my remark was not really about Guido.. but rather in a roundabout way, more of a poke at Xah Lee, hopefully avoiding the troll-feeding trap. Sorry you missed my point. Come to think of it, since I have nothing much to say about python, but nevertheless enjoy reading most of the material on this list, this was mostly an excuse to wish everyone here a Happy New Year. CJ From cs at zip.com.au Sun Jan 3 18:00:28 2010 From: cs at zip.com.au (Cameron Simpson) Date: Mon, 4 Jan 2010 10:00:28 +1100 Subject: IOError - cannot create file (linux daemon-invoked script) In-Reply-To: <8617a8f1-7567-4728-9568-9d03245df1fd@j24g2000yqa.googlegroups.com> References: <8617a8f1-7567-4728-9568-9d03245df1fd@j24g2000yqa.googlegroups.com> Message-ID: <20100103230028.GA11635@cskk.homeip.net> On 03Jan2010 14:20, cassiope wrote: | On Jan 2, 8:02?pm, Cameron Simpson wrote: | > Can you show us: | > ? - the directory user and group ownership and permissions | > ? - the daemon's user and group values? | | Directory permissions: 774 That's unusual - why the "4"? Directories with read but no search (1) are of limited use. (Not none - it's only unusual, not insane). | Directory ownership: "lesser user", "special group" where /etc/group | has "special group" members including the "lesser user", as well as | those who are expected to use the daemon, but not root. | Script ownership: "lesser user"; permissions 755 | Daemon ownership: root; permissions: 755 (always started by root). And the script/daemon _runs_ as the "lesser user"? If so, superficially the permissions look like they should work. -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ I couldn't think of anything else to do with it, so I put it on the web. From python at mrabarnett.plus.com Sun Jan 3 18:28:27 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 03 Jan 2010 23:28:27 +0000 Subject: Exception as the primary error handling mechanism? In-Reply-To: <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> Message-ID: <4B41281B.1080707@mrabarnett.plus.com> Michi wrote: > On Jan 1, 2:47 pm, Peng Yu wrote: >> In the article API Design Matters by Michi Henning >> >> Communications of the ACM >> Vol. 52 No. 5, Pages 46-56 >> 10.1145/1506409.1506424http://cacm.acm.org/magazines/2009/5/24646-api-design-matters/fulltext >> >> It says "Another popular design flaw?namely, throwing exceptions for >> expected outcomes?also causes inefficiencies because catching and >> handling exceptions is almost always slower than testing a return >> value." >> >> My observation is contradicted to the above statement by Henning. If >> my observation is wrong, please just ignore my question below. > > Seeing that quite a few people have put their own interpretation on > what I wrote, I figured I'll post a clarification. > > The quoted sentence appears in a section of the article that deals > with efficiency. I point out in that section that bad APIs often have > a price not just in terms of usability and defect rate, but that they > are often inefficient as well. (For example, wrapper APIs often > require additional memory allocations and/or data copies.) Incorrect > use of exceptions also incurs an efficiency penalty. > > In many language implementations, exception handling is expensive; > significantly more expensive than testing a return value. Consider the > following: > > int x; > try { > x = func(); > } catch (SomeException) { > doSomething(); > return; > } > doSomethingElse(); > > Here is the alternative without exceptions. (func() returns > SpecialValue instead of throwing.) > > int x; > x = func(); > if (x == SpecialValue) { > doSomething(); > return; > } > doSomethingElse(); > > In many language implementations, the second version is considerably > faster, especially when the exception may be thrown from deep in the > bowels of func(), possibly many frames down the call tree. > > If func() throws an exception for something that routinely occurs in > the normal use of the API, the extra cost can be noticeable. Note that > I am not advocating not to use exceptions. I *am* advocating to not > throw exceptions for conditions that are not exceptional. > > The classic example of this are lookup functions that, for example, > retrieve the value of an environment variable, do a table lookup, or > similar. Many such APIs throw an exception when the lookup fails > because the key isn't the table. However, very often, looking for > something that isn't there is a common case, such as when looking for > a value and, if the value isn't present already, adding it. Here is an > example of this: > > KeyType k = ...; > ValueType v; > > try { > v = collection.lookup(k); > } catch (NotFoundException) { > collection.add(k, defaultValue); > v = defaultValue; > } > doSomethingWithValue(v); > > The same code if collection doesn't throw when I look up something > that isn't there: > > KeyType k = ...; > ValueType v; > > v = collection.lookup(k); > if (v == null) { > collection.add(k, defaultValue); > v = defaultValue; > } > doSomethingWithValue(v); > > The problem is that, if I do something like this in a loop, and the > loop is performance-critical, the exception version can cause a > significant penalty. > In Python, of course, there's a method for this: setdefault. > As the API designer, when I make the choice between returning a > special value to indicate some condition, or throwing an exception, I > should consider the following questions: > > * Is the special condition such that, under most conceivable > circumstances, the caller will treat the condition as an unexpected > error? > > * Is it appropriate to force the caller to deal with the condition in > a catch-handler? > > * If the caller fails to explicitly deal with the condition, is it > appropriate to terminate the program? > > Only if the answer to these questions is "yes" is it appropriate to > throw an exception. Note the third question, which is often forgotten. > By throwing an exception, I not only force the caller to handle the > exception with a catch-handler (as opposed to leaving the choice to > the caller), I also force the caller to *always* handle the exception: > if the caller wants to ignore the condition, he/she still has to write > a catch-handler and failure to do so terminates the program. > > Apart from the potential performance penalty, throwing exceptions for > expected outcomes is bad also because it forces a try-catch block on > the caller. One example of this is the .NET socket API: if I do non- > blocking I/O on a socket, I get an exception if no data is ready for > reading (which is the common and expected case), and I get a zero > return value if the connection was lost (which is the uncommon and > unexpected case). > > In other words, the .NET API gets this completely the wrong way round. > Code that needs to do non-blocking reads from a socket turns into a > proper mess as a result because the outcome of a read() call is tri- > state: > > * Data was available and returned: no exception > > * No data available: exception > > * Connection lost: no exception > > Because such code normally lives in a loop that decrements a byte > count until the expected number of bytes have been read, the control > flow because really awkward because the successful case must be dealt > with in both the try block and the catch handler, and the error > condition must be dealt with in the try block as well. > > If the API did what it should, namely, throw an exception when the > connection is lost, and not throw when I do a read (whether data was > ready or not), the code would be far simpler and far more > maintainable. > > At no point did I ever advocate not to use exception handling. > Exceptions are the correct mechanism to handle errors. However, what > is considered an error is very much in the eye of the beholder. As the > API creator, if I indicate errors with exceptions, I make a policy > decision about what is an error and what is not. It behooves me to be > conservative in that policy: I should throw exceptions only for > conditions that are unlikely to arise during routine and normal use of > the API. > In another area, string slicing in C# uses the Substring method, where you provide the start position and number of characters. If the start index is out of bounds (it must be >= 0 and < length) or the string is too short, then it throws an exception. In practice I find Python's behaviour easier to use (and the code is shorter too!). C# also misses Python's trick (in Python 2.6 and above) of giving string instances a format method, instead making it a class method, so you need to write: string.format(format_string, ...) instead of Python's: format_string.format(...) On the other hand, C#'s equivalent of raw strings treat backslashes always as a normal character. I think it's the only feature of C#'s string handling that I prefer to Python's. From fpm at u.washington.edu Sun Jan 3 18:56:46 2010 From: fpm at u.washington.edu (cassiope) Date: Sun, 3 Jan 2010 15:56:46 -0800 (PST) Subject: IOError - cannot create file (linux daemon-invoked script) References: <8617a8f1-7567-4728-9568-9d03245df1fd@j24g2000yqa.googlegroups.com> Message-ID: On Jan 3, 3:00?pm, Cameron Simpson wrote: > On 03Jan2010 14:20, cassiope wrote: > | On Jan 2, 8:02?pm, Cameron Simpson wrote: > | > Can you show us: > | > ? - the directory user and group ownership and permissions > | > ? - the daemon's user and group values? > | > | Directory permissions: 774 > > That's unusual - why the "4"? Directories with read but no search (1) > are of limited use. (Not none - it's only unusual, not insane). > > | Directory ownership: "lesser user", "special group" where /etc/group > | has "special group" members including the "lesser user", as well as > | those who are expected to use the daemon, but not root. > | Script ownership: "lesser user"; permissions 755 > | Daemon ownership: root; permissions: 755 (always started by root). > > And the script/daemon _runs_ as the "lesser user"? > > If so, superficially the permissions look like they should work. > -- > Cameron Simpson DoD#743http://www.cskk.ezoshosting.com/cs/ > > I couldn't think of anything else to do with it, so I put it on the web. Strace confirms the uid and gid == "lesser user". Changing the directory permissions to 775 changes nothing. Clearly get EACCES error on the attempted file creation. The only other thing is that as part of the python interpreter call, I provide a "reduced environment", just UID,GID,TMP,PWD,USER, and HOME. Is anything else needed? Thanks again, Cameron! From cs at zip.com.au Sun Jan 3 19:36:39 2010 From: cs at zip.com.au (Cameron Simpson) Date: Mon, 4 Jan 2010 11:36:39 +1100 Subject: IOError - cannot create file (linux daemon-invoked script) In-Reply-To: References: Message-ID: <20100104003639.GA6459@cskk.homeip.net> On 03Jan2010 15:56, cassiope wrote: | Strace confirms the uid and gid == "lesser user". Changing the | directory | permissions to 775 changes nothing. Clearly get EACCES error on the | attempted | file creation. | | The only other thing is that as part of the python interpreter call, I | provide | a "reduced environment", just UID,GID,TMP,PWD,USER, and HOME. Is | anything | else needed? Should be irrelevant. Ok: does the file to be created already exist? If so, what are its permissions? If the file exists and isn't writable you may get this error. Also, did you eyeball the actual open() call to ensure the file pathname is correct, and doesn't use a bogus (non-existent) directory name? -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ "GOD IS MY SOURCE" - Bumper sticker, Chapel Hill, NC I'll have to remember that one for the next code review meeting. - ajvander at ingr.com (Alain van der Heide) From timr at probo.com Sun Jan 3 20:09:46 2010 From: timr at probo.com (Tim Roberts) Date: Sun, 03 Jan 2010 17:09:46 -0800 Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> Message-ID: Roy Smith wrote: > >2) Whitespace was not required in many places. For example, the following >two statements (this will only make sense in a fixed-width font) are >identical: > > DO 10 I = 1, 10 > DO10I=1,10 More than "not required", it was "not relevant". This led to one of the most infamous programming blunders in the early days of the space program, when one programmer accidentially typed a period instead of a comma resulting in the loss of a satellite: DO 10 I = 1. 10 What this actually does is store the floating point value "1.1" in a new variable called "DO10I". -- Tim Roberts, timr at probo.com Providenza & Boekelheide, Inc. From ldo at geek-central.gen.new_zealand Sun Jan 3 20:23:45 2010 From: ldo at geek-central.gen.new_zealand (Lawrence D'Oliveiro) Date: Mon, 04 Jan 2010 14:23:45 +1300 Subject: DST and datetime References: <874on82oan.fsf@benfinney.id.au> Message-ID: In message <874on82oan.fsf at benfinney.id.au>, Ben Finney wrote: > Or you could use the ready-made wheel maintained by others: > > tzinfo Objects > But that?s only an abstract base class, which means it doesn?t actually implement any reading of actual timezone info. > World timezone definitions, modern and historical > Shame that they maintain their own duplicate of the Olson database, instead of reading the original directly from /usr/share/zoneinfo . From ldo at geek-central.gen.new_zealand Sun Jan 3 20:31:11 2010 From: ldo at geek-central.gen.new_zealand (Lawrence D'Oliveiro) Date: Mon, 04 Jan 2010 14:31:11 +1300 Subject: Safe file I/O to shared file (or SQLite) from multi-threaded web server References: <4b403899$0$1646$742ec2ed@news.sonic.net> Message-ID: In message , Steve Holden wrote: > Yes, but not to MySQL, please. Particularly since there is a sword of > Damocles hanging over its head while the Oracle takeover of Sun is > pending. Ah, I see the FUDsters are crawling out of the woodwork here, as well. I?ve got news for you: MySQL is an open-source product. And you can?t kill Open Source. So go crawling back to your proprietary world, if that?s the only world you understand. From mensanator at aol.com Sun Jan 3 20:42:39 2010 From: mensanator at aol.com (Mensanator) Date: Sun, 3 Jan 2010 17:42:39 -0800 (PST) Subject: How do you configure IDLE on a Mac, seeing as there's no Configure Option... Message-ID: ...because there's no [Options] menu on the shell window? Or at least give me a clue to how to use Courier New font? For some inscrutable reason, depite the plethora of formatting tools, someone decided that proportional spaced fonts ought to be the default for IDLE. From steve at holdenweb.com Sun Jan 3 20:45:28 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 03 Jan 2010 20:45:28 -0500 Subject: Safe file I/O to shared file (or SQLite) from multi-threaded web server In-Reply-To: References: <4b403899$0$1646$742ec2ed@news.sonic.net> Message-ID: Lawrence D'Oliveiro wrote: > In message , Steve > Holden wrote: > >> Yes, but not to MySQL, please. Particularly since there is a sword of >> Damocles hanging over its head while the Oracle takeover of Sun is >> pending. > > Ah, I see the FUDsters are crawling out of the woodwork here, as well. I?ve > got news for you: MySQL is an open-source product. And you can?t kill Open > Source. So go crawling back to your proprietary world, if that?s the only > world you understand. I have no objection to you attempting to inform me about things I already understand, but I would appreciate at least some attempt on your part to maintain civility in your discourse. There is no need to be so obnoxious, or so ill-informed: I didn't get to chair the Python Software Foundation by "crawling [around in a] proprietary world", so kindly mind your manners. MySQL has always been technically inferior to other choices of open source database. The current state of affairs was entirely predictable, and appears to be more to do with Monty Widenius' wish to continue exploiting a brand that he sold toSun two years ago than it has to do with technical issues, as I pointed out yesterday. http://holdenweb.blogspot.com/2010/01/wht-save-mysql-now.html regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From news123 at free.fr Sun Jan 3 21:12:09 2010 From: news123 at free.fr (News123) Date: Mon, 04 Jan 2010 03:12:09 +0100 Subject: python xmlrpc client with ssl client certificates and standard modules Message-ID: <4b414e79$0$23321$426a34cc@news.free.fr> Hi, I was googling fot quite some time and was not really succesfull. I found one solution, which I will try soon. It is http://www.cs.technion.ac.il/~danken/xmlrpc-ssl.html (found in http://hamakor.org.il/pipermail/python-il/2008-February/000029.html ) This will probably work, but it requires the module M2Crypto. In order to avoid installing M2Crypto an all hosts that want to run the script I wondered, whether there is no other solution. I can do xmlrpc over ssl WITHOUT certificates with following code: import xmlrpclib server_url = 'https://myserver' server = xmlrpclib.Server(server_url); and I can perform a https get request WITH certificates with below snippet: import httplib conn = httplib.HTTPSConnection( HOSTNAME, key_file = KEYFILE, cert_file = CERTFILE ) conn.putrequest('GET', '/') conn.endheaders() response = conn.getresponse() print response.read() I'm just lost of how to 'combine' both. Thanks in advance for any suggestions / hints N From prakash.stack at gmail.com Sun Jan 3 21:45:24 2010 From: prakash.stack at gmail.com (prakash jp) Date: Mon, 4 Jan 2010 08:15:24 +0530 Subject: Which version of MSVC?90.DLL's to distribute with Python 2.6 based Py2exe executables? In-Reply-To: References: <05bd2f84-4f09-4a76-8e17-5868450ad393@c3g2000yqd.googlegroups.com> <753bb7c9-14bf-44e3-9679-0693635f2cd9@j14g2000yqm.googlegroups.com> <573025f1-0e5d-4d82-a86b-36ea33f85cad@c3g2000yqd.googlegroups.com> Message-ID: <805f59d51001031845k6a9b241bo9ed60df646b4f88e@mail.gmail.com> *#How to use setup.py file with py2exe:* ** python daniesetup.py py2exe --bundle 1 *#Also the data files have to taken care off in the options* list *#Here is a sample setup.py:* *#----------------------------------------------------* from distutils.core import setup import py2exe import sys # no arguments if len(sys.argv) == 1: sys.argv.append("py2exe") # creates a standalone .exe file, no zip files setup( options = {"py2exe": {"compressed": 1, "optimize": 2, "ascii": 1, "bundle_files": 1}}, zipfile = None, # replace test.py with your own code filename here ... console = [{"script": 'test.py'}] ) Regards Prakash -------------- next part -------------- An HTML attachment was scrubbed... URL: From roy at panix.com Sun Jan 3 21:59:02 2010 From: roy at panix.com (Roy Smith) Date: Sun, 03 Jan 2010 21:59:02 -0500 Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> Message-ID: In article , Tim Roberts wrote: > Roy Smith wrote: > > > >2) Whitespace was not required in many places. For example, the following > >two statements (this will only make sense in a fixed-width font) are > >identical: > > > > DO 10 I = 1, 10 > > DO10I=1,10 > > More than "not required", it was "not relevant". This led to one of the > most infamous programming blunders in the early days of the space program, > when one programmer accidentially typed a period instead of a comma > resulting in the loss of a satellite: > > DO 10 I = 1. 10 > > What this actually does is store the floating point value "1.1" in a new > variable called "DO10I". 1.1 or 1.01? There were some places where a blank was equivalent to a zero in a number. I don't remember if this was one of them. From n00m at narod.ru Sun Jan 3 22:05:13 2010 From: n00m at narod.ru (n00m) Date: Sun, 3 Jan 2010 19:05:13 -0800 (PST) Subject: twenty years ago Guido created Python References: <871vi628ft.fsf@benfinney.id.au> Message-ID: i'm sobered up Yes of course Guido and his lang is both superb things From ben+python at benfinney.id.au Sun Jan 3 22:18:18 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 04 Jan 2010 14:18:18 +1100 Subject: twenty years ago Guido created Python References: <871vi628ft.fsf@benfinney.id.au> Message-ID: <871vi6zhfp.fsf@benfinney.id.au> Chris Jones writes: > On Sun, Jan 03, 2010 at 04:19:02PM EST, Ben Finney wrote: > > The entirety of ?comp.lang.python? is an ongoing response to Guido > > van Rossum's achievements (and all the others that make Python > > great). Mere week- or month-long threads are paltry by comparison. > > That I understand. But my remark was not really about Guido.. but > rather in a roundabout way, more of a poke at [a troll], hopefully > avoiding the troll-feeding trap. You didn't avoid it, since you irrelevantly raised (twice, now) discussion of a troll in a thread originally unrelated to that person. > Sorry you missed my point. I didn't, since I attempted to bring it back on topic by expunging the off-topic part of your message. Sorry I failed to salvage the thread. -- \ ?If sharing a thing in no way diminishes it, it is not rightly | `\ owned if it is not shared.? ?Augustine of Hippo (354?430 CE) | _o__) | Ben Finney From n00m at narod.ru Sun Jan 3 22:23:47 2010 From: n00m at narod.ru (n00m) Date: Sun, 3 Jan 2010 19:23:47 -0800 (PST) Subject: Any Swisses here? References: <145e6acd-4260-495b-b562-da634fccfa1a@j19g2000yqk.googlegroups.com> <4B400E83.1080305@yahoo.com.br> <0e89f58c-a001-42c9-a851-6f0ce6570873@h9g2000yqa.googlegroups.com> Message-ID: What Imeant by what waht try to solve e.g. http://acm.sgu.ru/problem.php?contest=0&problem=482 99% people here are 0 in this. They know this they know that, but they can't nothing, they simply are sitting on this group and apraised themselves But you can nothing From steven at REMOVE.THIS.cybersource.com.au Sun Jan 3 22:30:42 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 04 Jan 2010 03:30:42 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> Message-ID: On Sun, 03 Jan 2010 13:44:29 -0800, Michi wrote: > The quoted sentence appears in a section of the article that deals with > efficiency. I point out in that section that bad APIs often have a price > not just in terms of usability and defect rate, but that they are often > inefficient as well. This is very true, but good APIs often trade-off increased usability and reduced defect rate against machine efficiency too. In fact, I would argue that this is a general design principle of programming languages: since correctness and programmer productivity are almost always more important than machine efficiency, the long-term trend across virtually all languages is to increase correctness and productivity even if doing so costs some extra CPU cycles. > (For example, wrapper APIs often require additional > memory allocations and/or data copies.) Incorrect use of exceptions also > incurs an efficiency penalty. And? *Correct* use of exceptions also incur a penalty. So does the use of functions. Does this imply that putting code in functions is a poor API? Certainly not. > In many language implementations, exception handling is expensive; > significantly more expensive than testing a return value. And in some it is less expensive. But no matter how much more expensive, there will always be a cut-off point where it is cheaper on average to suffer the cost of handling an exception than it is to make unnecessary tests. In Python, for dictionary key access, that cut-off is approximately at one failure per ten or twenty attempts. So unless you expect more than one in ten attempts to lead to a failure, testing first is actually a pessimation, not an optimization. > Consider the following: > > int x; > try { > x = func(); > } catch (SomeException) { > doSomething(); > return; > } > doSomethingElse(); > > Here is the alternative without exceptions. (func() returns SpecialValue > instead of throwing.) > > int x; > x = func(); > if (x == SpecialValue) { > doSomething(); > return; > } > doSomethingElse(); In some, limited, cases you might be able to use the magic return value strategy, but this invariably leads to lost programmer productivity, more complex code, lowered readability and usability, and more defects, because programmers will invariably neglect to test for the special value: int x; x = func(); doSomething(x); return; Or worse, they will write doSomething() so that it too needs to know about SpecialValue, and so do all the functions it calls. Instead of dealing with the failure in one place, you can end up having to deal with it in a dozen places. But even worse is common case that SpecialValue is a legal value when passed to doSomething, and you end up with the error propagating deep into the application before being found. Or even worse, it is never found at all, and the application simply does the wrong thing. > In many language implementations, the second version is considerably > faster, especially when the exception may be thrown from deep in the > bowels of func(), possibly many frames down the call tree. This is a classic example of premature optimization. Unless such inefficiency can be demonstrated to actually matter, then you do nobody any favours by preferring the API that leads to more defects on the basis of *assumed* efficiency. If your test for a special value is 100 times faster than handling the exception, and exceptions occur only one time in 1000, then using a strategy of testing for a special value is actually ten times slower on average than catching an exception. > If func() throws an exception for something that routinely occurs in the > normal use of the API, the extra cost can be noticeable. "Can be". But it also might not be noticeable at all. [...] > Here is an example of this: > > KeyType k = ...; > ValueType v; > > try { > v = collection.lookup(k); > } catch (NotFoundException) { > collection.add(k, defaultValue); > v = defaultValue; > } > doSomethingWithValue(v); > > The same code if collection doesn't throw when I look up something that > isn't there: > > KeyType k = ...; > ValueType v; > > v = collection.lookup(k); > if (v == null) { > collection.add(k, defaultValue); > v = defaultValue; > } > doSomethingWithValue(v); > > The problem is that, if I do something like this in a loop, and the loop > is performance-critical, the exception version can cause a significant > penalty. No, the real problems are: (1) The caller has to remember to check the return result for the magic value. Failure to do so leads to bugs, in some cases, serious and hard-to- find bugs. (2) If missing keys are rare enough, the cost of all those unnecessary tests will out-weigh the saving of avoiding catching the exception. "Rare enough" may still be very common: in the case of Python, the cross-over point is approximately 1 time in 15. (3) Your collection now cannot use the magic value as a legitimate value. This last one can be *very* problematic. In the early 1990s, I was programming using a callback API that could only return an integer. The standard way of indicating an error was to return -1. But what happens if -1 is a legitimate return value, e.g. for a maths function? The solution used was to have the function create a global variable holding a flag: result = function(args) if result == -1: if globalErrorState == -1: print "An error occurred" exit doSomething(result) That is simply horrible. > As the API designer, when I make the choice between returning a special > value to indicate some condition, or throwing an exception, I should > consider the following questions: > > * Is the special condition such that, under most conceivable > circumstances, the caller will treat the condition as an unexpected > error? Wrong. It doesn't matter whether it is an error or not. They are called EXCEPTIONS, not ERRORS. What matters is that it is an exceptional case. Whether that exceptional case is an error condition or not is dependent on the application. > * Is it appropriate to force the caller to deal with the condition in > a catch-handler? > > * If the caller fails to explicitly deal with the condition, is it > appropriate to terminate the program? > > Only if the answer to these questions is "yes" is it appropriate to > throw an exception. Note the third question, which is often forgotten. > By throwing an exception, I not only force the caller to handle the > exception with a catch-handler (as opposed to leaving the choice to the > caller), I also force the caller to *always* handle the exception: if > the caller wants to ignore the condition, he/she still has to write a > catch-handler and failure to do so terminates the program. That's a feature of exceptions, not a problem. > Apart from the potential performance penalty, throwing exceptions for > expected outcomes is bad also because it forces a try-catch block on the > caller. But it's okay to force a `if (result==MagicValue)` test instead? Look, the caller has to deal with exceptional cases (which may include error conditions) one way or the other. If you don't deal with them at all, your code will core dump, or behave incorrectly, or something. If the caller fails to deal with the exceptional case, it is better to cause an exception that terminates the application immediately than it is to allow the application to generate incorrect results. > One example of this is the .NET socket API: if I do non- > blocking I/O on a socket, I get an exception if no data is ready for > reading (which is the common and expected case), and I get a zero return > value if the connection was lost (which is the uncommon and unexpected > case). > > In other words, the .NET API gets this completely the wrong way round. Well we can agree on that! > If the API did what it should, namely, throw an exception when the > connection is lost, and not throw when I do a read (whether data was > ready or not), the code would be far simpler and far more maintainable. > > At no point did I ever advocate not to use exception handling. > Exceptions are the correct mechanism to handle errors. However, what is > considered an error is very much in the eye of the beholder. As the API > creator, if I indicate errors with exceptions, I make a policy decision > about what is an error and what is not. It behooves me to be > conservative in that policy: I should throw exceptions only for > conditions that are unlikely to arise during routine and normal use of > the API. But lost connections *are* routine and normal. Hopefully they are rare. -- Steven From drobinow at gmail.com Sun Jan 3 22:33:57 2010 From: drobinow at gmail.com (David Robinow) Date: Sun, 3 Jan 2010 22:33:57 -0500 Subject: Significant whitespace In-Reply-To: References: <4B3E70FC.1010702@gmail.com> Message-ID: <4eb0089f1001031933o5b70edeo4d8fd57784747113@mail.gmail.com> On Sun, Jan 3, 2010 at 8:09 PM, Tim Roberts wrote: > More than "not required", it was "not relevant". ?This led to one of the > most infamous programming blunders in the early days of the space program, > when one programmer accidentially typed a period instead of a comma > resulting in the loss of a satellite: Interesting story. Did you make it up? From roy at panix.com Sun Jan 3 22:36:44 2010 From: roy at panix.com (Roy Smith) Date: Sun, 03 Jan 2010 22:36:44 -0500 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> Message-ID: In article , Steven D'Aprano wrote: > This last one can be *very* problematic. In the early 1990s, I was > programming using a callback API that could only return an integer. The > standard way of indicating an error was to return -1. But what happens if > -1 is a legitimate return value, e.g. for a maths function? One of the truly nice features of Python is the universally distinguished value, None. From roy at panix.com Sun Jan 3 22:42:16 2010 From: roy at panix.com (Roy Smith) Date: Sun, 03 Jan 2010 22:42:16 -0500 Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> Message-ID: In article , David Robinow wrote: > On Sun, Jan 3, 2010 at 8:09 PM, Tim Roberts wrote: > > More than "not required", it was "not relevant". ?This led to one of the > > most infamous programming blunders in the early days of the space program, > > when one programmer accidentially typed a period instead of a comma > > resulting in the loss of a satellite: > Interesting story. Did you make it up? It's a fairly well known story. http://catless.ncl.ac.uk/Risks/5.64.html#subj4.2 From drobinow at gmail.com Sun Jan 3 23:42:44 2010 From: drobinow at gmail.com (David Robinow) Date: Sun, 3 Jan 2010 23:42:44 -0500 Subject: Significant whitespace In-Reply-To: References: <4B3E70FC.1010702@gmail.com> Message-ID: <4eb0089f1001032042v2aaabbdasf3405ffaf571545@mail.gmail.com> On Sun, Jan 3, 2010 at 10:42 PM, Roy Smith wrote: > In article , > ?David Robinow wrote: > >> On Sun, Jan 3, 2010 at 8:09 PM, Tim Roberts wrote: >> > More than "not required", it was "not relevant". ?This led to one of the >> > most infamous programming blunders in the early days of the space program, >> > when one programmer accidentially typed a period instead of a comma >> > resulting in the loss of a satellite: >> Interesting story. Did you make it up? > > It's a fairly well known story. > > http://catless.ncl.ac.uk/Risks/5.64.html#subj4.2 Sure. But the question is, "Who made it up?" http://en.wikiquote.org/wiki/Fortran Computer folklore has incorrectly attributed the loss of the Mariner 1 space probe to a syntax error in a Fortran program. For example, "Recall the first American space probe to Venus, reportedly lost because Fortran cannot recognize a missing comma in a DO statement?"[ From n00m at narod.ru Sun Jan 3 23:44:38 2010 From: n00m at narod.ru (n00m) Date: Sun, 3 Jan 2010 20:44:38 -0800 (PST) Subject: twenty years ago Guido created Python References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> Message-ID: <04dfc0f0-bee3-4df3-bc50-e8e9309b45d1@e37g2000yqn.googlegroups.com> Now tell me: what ahat this mobs of yellow, africans, asiats, are doing in Stockholm? Do you think your govs let me in just to vist England, Norway? Never. Its all conditionally. But the fact is. IS. You roll our world into abyss of stupidity From robertoedwins at gmail.com Mon Jan 4 00:50:06 2010 From: robertoedwins at gmail.com (rieh25) Date: Sun, 3 Jan 2010 21:50:06 -0800 (PST) Subject: Question about an application Message-ID: <27009118.post@talk.nabble.com> I am thinking of installing a python webserver I coded in every computer at my work. This would allow me to run specific tasks in them, like creating backups, installing things, etc. Is there another way to run programs in remote computers? Thanks for your opinions. -- View this message in context: http://old.nabble.com/Question-about-an-application-tp27009118p27009118.html Sent from the Python - python-list mailing list archive at Nabble.com. From rodrick.brown at gmail.com Mon Jan 4 00:54:22 2010 From: rodrick.brown at gmail.com (rodrick brown) Date: Mon, 4 Jan 2010 00:54:22 -0500 Subject: Question about an application In-Reply-To: <27009118.post@talk.nabble.com> References: <27009118.post@talk.nabble.com> Message-ID: <9A7F3B4C-5621-44CF-ABCA-D440E7ADA01E@gmail.com> Take a look at ssh Sent from my iPhone 3GS. On Jan 4, 2010, at 12:50 AM, rieh25 wrote: > > I am thinking of installing a python webserver I coded in every > computer at > my work. This would allow me to run specific tasks in them, like > creating > backups, installing things, etc. Is there another way to run > programs in > remote computers? Thanks for your opinions. > -- > View this message in context: http://old.nabble.com/Question-about-an-application-tp27009118p27009118.html > Sent from the Python - python-list mailing list archive at Nabble.com. > > -- > http://mail.python.org/mailman/listinfo/python-list From aioe.org at technicalbloke.com Mon Jan 4 01:11:35 2010 From: aioe.org at technicalbloke.com (r0g) Date: Mon, 04 Jan 2010 06:11:35 +0000 Subject: twenty years ago Guido created Python References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> Message-ID: n00m wrote: > Ok, let me evolve a bit my thought. > Guido is Eropean Culture. > Now say what was invented by Japanese, by Chinese? > Nothing. > Barbaric cultures, they are animals inborn animals. > Russian Sikorsky built choppers for Eisenhower. > Look at letter "S" on them -- its his name > What you know about his destiny? > You know knothing. Yuou don't know that only the > greatest Russian composer Rakhmaninov helped him, in America. > Literally -- by money, the composer gave him all his fortune ~$1500 > Be not this help -- no choppers in America Announcement: You can download the latest version of this cutting edge bot from here... http://www.illiteratexenophobicrantingracistsimulator.ru/usenet_spambot.py Plugins available here... http://www.illiteratexenophobicrantingracistsimulator.ru/terrible_spelling.py http://www.illiteratexenophobicrantingracistsimulator.ru/inconsequential_facts.py http://www.illiteratexenophobicrantingracistsimulator.ru/rabid_nationalism.py Happy new year everybody! Roger. From robertoedwins at gmail.com Mon Jan 4 01:20:38 2010 From: robertoedwins at gmail.com (rieh25) Date: Sun, 3 Jan 2010 22:20:38 -0800 (PST) Subject: Question about an application In-Reply-To: <9A7F3B4C-5621-44CF-ABCA-D440E7ADA01E@gmail.com> References: <27009118.post@talk.nabble.com> <9A7F3B4C-5621-44CF-ABCA-D440E7ADA01E@gmail.com> Message-ID: <27009252.post@talk.nabble.com> Thanks, I had forgotten about it. I'll investigate if there are ways to automate using it to run programs in several computers at the same time. Rodrick Brown wrote: > > Take a look at ssh > > Sent from my iPhone 3GS. > > On Jan 4, 2010, at 12:50 AM, rieh25 wrote: > >> >> I am thinking of installing a python webserver I coded in every >> computer at >> my work. This would allow me to run specific tasks in them, like >> creating >> backups, installing things, etc. Is there another way to run >> programs in >> remote computers? Thanks for your opinions. >> -- >> View this message in context: >> http://old.nabble.com/Question-about-an-application-tp27009118p27009118.html >> Sent from the Python - python-list mailing list archive at Nabble.com. >> >> -- >> http://mail.python.org/mailman/listinfo/python-list > -- > http://mail.python.org/mailman/listinfo/python-list > > -- View this message in context: http://old.nabble.com/Question-about-an-application-tp27009118p27009252.html Sent from the Python - python-list mailing list archive at Nabble.com. From cjns1989 at gmail.com Mon Jan 4 01:25:33 2010 From: cjns1989 at gmail.com (Chris Jones) Date: Mon, 04 Jan 2010 01:25:33 -0500 Subject: twenty years ago Guido created Python In-Reply-To: <871vi6zhfp.fsf@benfinney.id.au> References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> Message-ID: <20100104062533.GB3125@turki.gavron.org> On Sun, Jan 03, 2010 at 10:18:18PM EST, Ben Finney wrote: > Chris Jones writes: [..] > > Sorry you missed my point. > > I didn't, since I attempted to bring it back on topic by expunging the > off-topic part of your message. Er.. what's 'off-topic' about mentioning that a couple of trolls on this list often seem to attract far more interest than a genuine testimonial to Guido's achievement? > Sorry I failed to salvage the thread. Man with a Mission, hehe.. CJ From aioe.org at technicalbloke.com Mon Jan 4 01:54:05 2010 From: aioe.org at technicalbloke.com (r0g) Date: Mon, 04 Jan 2010 06:54:05 +0000 Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> Message-ID: David Robinow wrote: > On Sun, Jan 3, 2010 at 10:42 PM, Roy Smith wrote: >> In article , >> David Robinow wrote: >> >>> On Sun, Jan 3, 2010 at 8:09 PM, Tim Roberts wrote: >>>> More than "not required", it was "not relevant". This led to one of the >>>> most infamous programming blunders in the early days of the space program, >>>> when one programmer accidentially typed a period instead of a comma >>>> resulting in the loss of a satellite: >>> Interesting story. Did you make it up? >> It's a fairly well known story. >> >> http://catless.ncl.ac.uk/Risks/5.64.html#subj4.2 > Sure. But the question is, "Who made it up?" > http://en.wikiquote.org/wiki/Fortran > > Computer folklore has incorrectly attributed the loss of the Mariner 1 > space probe to a syntax error in a Fortran program. For example, > "Recall the first American space probe to Venus, reportedly lost > because Fortran cannot recognize a missing comma in a DO statement?"[ Wow you're totally right! It was a missing UNDERSCORE not a DOT being typed instead of a COMMA! We've been lied to ALL ALONG! What kind of sickos would do this? Let's find this Tim character, flay him alive and then go crown you the KING OF EVERYTHING for rescuing us from that deadly nugget of misinformation! Honestly I might have believed those dot/comma LIES if it wasn't for your noble selfless efforts to uncover the SHOCKING TRUTH. Imagine what might have happened! The character in question being an UNDERSCORE COMPLETELY CHANGES the meaning and point of the story. As for it being LEFT OUT rather than TRANSPOSED, well, anyone can see how that TURNS THE WHOLE STORY UPSIDE DOWN. Of course there are the naysayers who might claim absolute accuracy is no big deal in stories that are essentially parables but they do not understand the HARM that can come from not remembering really tiny, almost inconsequential details. For what it's worth I DO NOT consider you a tiresome pedant and I think your taunting, patronising style is completely appropriate and proportional to Tim's sickening crimes against fact. Roger. From steven at REMOVE.THIS.cybersource.com.au Mon Jan 4 01:55:01 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 04 Jan 2010 06:55:01 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> Message-ID: On Sun, 03 Jan 2010 22:36:44 -0500, Roy Smith wrote: > In article , > Steven D'Aprano wrote: > >> This last one can be *very* problematic. In the early 1990s, I was >> programming using a callback API that could only return an integer. The >> standard way of indicating an error was to return -1. But what happens >> if -1 is a legitimate return value, e.g. for a maths function? > > One of the truly nice features of Python is the universally > distinguished value, None. What happens if you need to return None as a legitimate value? Here's a good example: iterating over a list. Python generates an exception when you hit the end of the list. If instead, Python returned None when the index is out of bounds, you couldn't store None in a list without breaking code. So we produce a special sentinel object EndOfSequence. Now we can't do this: for obj in ["", 12, None, EndOfSequence, [], {}]: print dir(obj) # or some other useful operation The fundamental flaw of using magic values is that, there will always be some application where you want to use the magic value as a non-special value, and then you're screwed. This is why, for instance, it's difficult for C strings to contain a null byte, and there are problems with text files on DOS and CP/M (and Windows under some circumstances) that contain a ^Z byte. -- Steven From steven at REMOVE.THIS.cybersource.com.au Mon Jan 4 01:56:03 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 04 Jan 2010 06:56:03 GMT Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> Message-ID: On Sun, 03 Jan 2010 23:42:44 -0500, David Robinow wrote: > On Sun, Jan 3, 2010 at 10:42 PM, Roy Smith wrote: >> In article , >> ?David Robinow wrote: >> >>> On Sun, Jan 3, 2010 at 8:09 PM, Tim Roberts wrote: >>> > More than "not required", it was "not relevant". ?This led to one of >>> > the most infamous programming blunders in the early days of the >>> > space program, when one programmer accidentially typed a period >>> > instead of a comma resulting in the loss of a satellite: >>> Interesting story. Did you make it up? >> >> It's a fairly well known story. >> >> http://catless.ncl.ac.uk/Risks/5.64.html#subj4.2 > Sure. But the question is, "Who made it up?" > http://en.wikiquote.org/wiki/Fortran > > Computer folklore has incorrectly attributed the loss of the Mariner 1 > space probe to a syntax error in a Fortran program. For example, "Recall > the first American space probe to Venus, reportedly lost because Fortran > cannot recognize a missing comma in a DO statement?"[ What makes you believe it is incorrect? -- Steven From aioe.org at technicalbloke.com Mon Jan 4 02:05:23 2010 From: aioe.org at technicalbloke.com (r0g) Date: Mon, 04 Jan 2010 07:05:23 +0000 Subject: Question about an application References: <27009118.post@talk.nabble.com> <9A7F3B4C-5621-44CF-ABCA-D440E7ADA01E@gmail.com> Message-ID: rieh25 wrote: > Thanks, I had forgotten about it. I'll investigate if there are ways to > automate using it to run programs in several computers at the same time. > > > Rodrick Brown wrote: >> Take a look at ssh >> There are. Take a look at paramiko if you want to interface with SSH within python. Depending on what you need to do it may well be easier to use bash scripts / batch files or simply the CLI instead e.g. ssh account_name at 192.168.x.x 'ls /' > root_folder_listing.txt This line... starts ssh logs into account on machine 192.168.x.x executes ls pipes the data back to a file on your local machine Easy peasy :) Roger. From highcar at gmail.com Mon Jan 4 02:42:36 2010 From: highcar at gmail.com (elca) Date: Sun, 3 Jan 2010 23:42:36 -0800 (PST) Subject: python mechanize proxy support question Message-ID: <27009696.post@talk.nabble.com> Hello All Happy New Year! i have some question about python mechanize 's proxy support. im making some web client script, and i would like to insert proxy support function into my script. for example ,if i have such like following some script. how can i add proxy support into my mechanize script? i was look for some reference , but not so much good hint from google. params = urllib.urlencode({'id':id, 'passwd':pw}) rq = mechanize.Request("http://www.example.com" params) rs = mechanize.urlopen(rq) whenever i open this 'www.example.com' website , i would like open go through proxy. Thanks in advance! -- View this message in context: http://old.nabble.com/python-mechanize-proxy-support-question-tp27009696p27009696.html Sent from the Python - python-list mailing list archive at Nabble.com. From gagsl-py2 at yahoo.com.ar Mon Jan 4 02:51:40 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 04 Jan 2010 04:51:40 -0300 Subject: TypeError: can only concatenate list (not "tuple") to list Message-ID: This py> [1,2,3] + (4,5) Traceback (most recent call last): File "", line 1, in TypeError: can only concatenate list (not "tuple") to list -- Gabriel Genellina From clp2 at rebertia.com Mon Jan 4 02:58:54 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 3 Jan 2010 23:58:54 -0800 Subject: TypeError: can only concatenate list (not "tuple") to list In-Reply-To: References: Message-ID: <50697b2c1001032358j59a2f08epfb01e6ec73615dc7@mail.gmail.com> On Sun, Jan 3, 2010 at 11:51 PM, Gabriel Genellina wrote: > This > py> [1,2,3] + (4,5) > Traceback (most recent call last): > ?File "", line 1, in > TypeError: can only concatenate list (not "tuple") to list Given that tuples are sometimes used as a poor man's object (i.e. collection of data fields), whereas lists are not typically used that way, I'd say it's probably a good thing an explicit type conversion is required here. Cheers, Chris -- http://blog.rebertia.com From gagsl-py2 at yahoo.com.ar Mon Jan 4 02:59:02 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 04 Jan 2010 04:59:02 -0300 Subject: TypeError: can only concatenate list (not "tuple") to list Message-ID: Is there any reason for this error? Apart from "nobody cared to write the code" py> [1,2,3] + (4,5) Traceback (most recent call last): File "", line 1, in TypeError: can only concatenate list (not "tuple") to list In-place addition += does work: py> a = [1,2,3] py> a += (4,5) py> a [1, 2, 3, 4, 5] -- Gabriel Genellina From david at bibliolabs.com Mon Jan 4 03:20:44 2010 From: david at bibliolabs.com (David Williams) Date: Mon, 4 Jan 2010 02:20:44 -0600 (CST) Subject: TypeError: can only concatenate list (not "tuple") to list In-Reply-To: References: Message-ID: <1984.68.59.29.80.1262593244.squirrel@www.bibliobazaar.com> > Is there any reason for this error? Apart from "nobody cared to write the > code" > > py> [1,2,3] + (4,5) > Traceback (most recent call last): > File "", line 1, in > TypeError: can only concatenate list (not "tuple") to list > > In-place addition += does work: > > py> a = [1,2,3] > py> a += (4,5) > py> a > [1, 2, 3, 4, 5] > > -- > Gabriel Genellina > > -- > http://mail.python.org/mailman/listinfo/python-list > They are different types. From steven at REMOVE.THIS.cybersource.com.au Mon Jan 4 03:22:44 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 04 Jan 2010 08:22:44 GMT Subject: TypeError: can only concatenate list (not "tuple") to list References: Message-ID: On Mon, 04 Jan 2010 04:59:02 -0300, Gabriel Genellina wrote: > Is there any reason for this error? Apart from "nobody cared to write > the code" Yes, because such implicit conversions would be a bad idea. > py> [1,2,3] + (4,5) What result are you expecting? A list or a tuple? > Traceback (most recent call last): > File "", line 1, in > TypeError: can only concatenate list (not "tuple") to list Apart from the different error message, this is essentially the same error as this: >>> 2 + "2" Traceback (most recent call last): File "", line 1, in TypeError: unsupported operand type(s) for +: 'int' and 'str' > In-place addition += does work: > > py> a = [1,2,3] > py> a += (4,5) > py> a > [1, 2, 3, 4, 5] I call that an impressive gotcha. I believe that is because in-place addition of lists is implemented as functionally equivalent to the extend method: >>> a += "abc" # same as a.extend("abc") >>> a [1, 2, 3, 4, 5, 'a', 'b', 'c'] >>> a += {None: -1} >>> a [1, 2, 3, 4, 5, 'a', 'b', 'c', None] -- Steven From david at bibliolabs.com Mon Jan 4 03:24:56 2010 From: david at bibliolabs.com (David Williams) Date: Mon, 4 Jan 2010 02:24:56 -0600 (CST) Subject: TypeError: can only concatenate list (not "tuple") to list In-Reply-To: References: Message-ID: <1992.68.59.29.80.1262593496.squirrel@www.bibliobazaar.com> > Is there any reason for this error? Apart from "nobody cared to write the > code" > > py> [1,2,3] + (4,5) > Traceback (most recent call last): > File "", line 1, in > TypeError: can only concatenate list (not "tuple") to list > > In-place addition += does work: > > py> a = [1,2,3] > py> a += (4,5) > py> a > [1, 2, 3, 4, 5] > > -- > Gabriel Genellina > > -- > http://mail.python.org/mailman/listinfo/python-list > I guess to expand a bit more on what I said... What should the result be? A list or a tuple? The reason += works is because the end result is clear; a list. But it is ambiguous in the case of concatenation: did you want a tuple or a list? From gagsl-py2 at yahoo.com.ar Mon Jan 4 03:27:59 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 04 Jan 2010 05:27:59 -0300 Subject: TypeError: can only concatenate list (not "tuple") to list References: Message-ID: En Mon, 04 Jan 2010 04:58:54 -0300, Chris Rebert escribi?: > On Sun, Jan 3, 2010 at 11:51 PM, Gabriel Genellina > wrote: >> py> [1,2,3] + (4,5) >> Traceback (most recent call last): >> File "", line 1, in >> TypeError: can only concatenate list (not "tuple") to list Sorry, I inadvertedly posted an incomplete message. Note the last part: >> In-place addition += does work: >> >> py> a = [1,2,3] >> py> a += (4,5) >> py> a >> [1, 2, 3, 4, 5] > Given that tuples are sometimes used as a poor man's object (i.e. > collection of data fields), whereas lists are not typically used that > way, I'd say it's probably a good thing an explicit type conversion is > required here. In that case += should not be allowed either... -- Gabriel Genellina From sanneh27 at hotmail.com Mon Jan 4 03:30:04 2010 From: sanneh27 at hotmail.com (baboucarr sanneh) Date: Mon, 4 Jan 2010 08:30:04 +0000 Subject: HAppy New Year In-Reply-To: <880dece01001031204v1f9f1552je08ca9e98970a732@mail.gmail.com> References: , <880dece01001031118h4e6e8a7nac7f8a6b80ca8b5d@mail.gmail.com>, , <880dece01001031204v1f9f1552je08ca9e98970a732@mail.gmail.com> Message-ID: Oops ya rite.. Y didn't i think of that ?..neway thnx guys guess y we ave this mailing list to share ideas rite... Regards, Baboucarr. > Date: Sun, 3 Jan 2010 22:04:40 +0200 > Subject: Re: HAppy New Year > From: dotancohen at gmail.com > To: steve at holdenweb.com > CC: python-list at python.org > > > What I want to know is why didn't he just write > > > > """\ > > ... original long string ... > > """.replace("6", "_") > > > > Uf! Why didn't I think of that?! > > > > But watch that snarkiness ... I'm a Windows user too! > > > > That's fine, I know that fresh air has to get into the basement somehow ! > > -- > Dotan Cohen > > http://what-is-what.com > http://gibberish.co.il > -- > http://mail.python.org/mailman/listinfo/python-list _________________________________________________________________ Windows Live: Make it easier for your friends to see what you?re up to on Facebook. http://www.microsoft.com/middleeast/windows/windowslive/see-it-in-action/social-network-basics.aspx?ocid=PID23461::T:WLMTAGL:ON:WL:en-xm:SI_SB_2:092009 -------------- next part -------------- An HTML attachment was scrubbed... URL: From wuwei23 at gmail.com Mon Jan 4 03:34:14 2010 From: wuwei23 at gmail.com (alex23) Date: Mon, 4 Jan 2010 00:34:14 -0800 (PST) Subject: python mechanize proxy support question References: Message-ID: <4ca2b83a-3b82-4d20-85a3-b6304f25fb81@21g2000yqj.googlegroups.com> On Jan 4, 5:42?pm, elca wrote: > how can i add proxy support into my mechanize script? > i was look for some reference , but not so much good hint from google. There are examples on using proxies with mechanize on the module's home page: http://wwwsearch.sourceforge.net/mechanize/ From fetchinson at googlemail.com Mon Jan 4 03:50:27 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Mon, 4 Jan 2010 09:50:27 +0100 Subject: twenty years ago Guido created Python In-Reply-To: <871vi6zhfp.fsf@benfinney.id.au> References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> Message-ID: > Sorry I failed to salvage the thread. Go, quick, these threads are all around us! They are popping up everywhere! Go, go, quick! There is another troll thread right there! They are encircling us! They keep coming! At least one troll thread started on usenet just since I started typing! Quick, quick, do something! Do something! Save us! -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From gagsl-py2 at yahoo.com.ar Mon Jan 4 03:51:06 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 04 Jan 2010 05:51:06 -0300 Subject: How to Suppress Interactive Assignment to "_" References: <910af7d0-8aa7-43fa-b2c9-32e12dc2fbc9@m16g2000yqc.googlegroups.com> Message-ID: En Fri, 01 Jan 2010 15:53:31 -0300, JKPeck escribi?: > On Jan 1, 10:06 am, Peter Otten <__pete... at web.de> wrote: >> JKPeck wrote: >> > The gettext module uses the convention of defining a function named >> > "_" that maps text into its translation. >> > This conflicts with the automatic interactive interpreter assignment >> > of expressions to a variable with that same name. >> >> $ cat displayhook.py >> [...] > > Thanks. It's just what I needed. In case you didn't know, look at sitecustomize.py and the PYTHONSTARTUP [2] environment variable. They allow for the above code to be automatically executed. [1] http://docs.python.org/library/site.html#index-549 [2] http://docs.python.org/using/cmdline.html#envvar-PYTHONSTARTUP -- Gabriel Genellina From massimodipierro71 at gmail.com Mon Jan 4 03:59:39 2010 From: massimodipierro71 at gmail.com (mdipierro) Date: Mon, 4 Jan 2010 00:59:39 -0800 (PST) Subject: python xmlrpc client with ssl client certificates and standard modules References: <4b414e79$0$23321$426a34cc@news.free.fr> Message-ID: xmlrpc acts at the application layer and ssl at the transport layer so they can inter operate easily as long as you do not use the certificate to authenticate the client but only validate the server and encrypt data (which you can also do but it is more complicated) One option for you is to use web2py which include an xmlrpc server that uses a wsgi ssl enabled web server. Here is how: 1) Install web2py 2) Visit http://127.0.0.1:8000/admin and create a new application from the web based IDE 3) create your web service for example, in a controller default.py @service.xmlrpc def add(a,b): return int(a)+int(b) 4) Restart web2py with python web2py.py -a ADMIN_PASSWD -c SSL_CERTIFICATE -k SSL_PRIVATE_KEY -i 0.0.0.0 -p 443 5) You can now access the service from any Python program: >>> import xmlrpclib >>> server_url = 'https://myserver:443/yourapp/default/call/ xmlrpc' >>> server = xmlrpclib.Server(server_url) >>> print server.add(3,4) 7 Hope this helps. On Jan 3, 8:12?pm, News123 wrote: > Hi, > > I was googling fot quite some time and was not really succesfull. > > I found one solution, which I will try soon. > It ishttp://www.cs.technion.ac.il/~danken/xmlrpc-ssl.html > (found inhttp://hamakor.org.il/pipermail/python-il/2008-February/000029.html) > > This will probably work, but it requires the module M2Crypto. > > In order to avoid installing M2Crypto an all hosts that want to run the > script I wondered, whether there is no other solution. > > I can do xmlrpc over ssl WITHOUT certificates with following code: > > import xmlrpclib > server_url = 'https://myserver' > server = xmlrpclib.Server(server_url); > > and I can perform a https get request WITH certificates with below snippet: > > import httplib > conn = httplib.HTTPSConnection( > ? ? ? ? HOSTNAME, > ? ? ? ? key_file = KEYFILE, > ? ? ? ? cert_file = CERTFILE > ) > conn.putrequest('GET', '/') > conn.endheaders() > response = conn.getresponse() > print response.read() > > I'm just lost of how to 'combine' both. > > Thanks in advance for any suggestions / hints > > N From gagsl-py2 at yahoo.com.ar Mon Jan 4 04:27:48 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 04 Jan 2010 06:27:48 -0300 Subject: TypeError: can only concatenate list (not "tuple") to list References: <1992.68.59.29.80.1262593496.squirrel@www.bibliobazaar.com> Message-ID: En Mon, 04 Jan 2010 05:24:56 -0300, David Williams escribi?: >> py> [1,2,3] + (4,5) >> Traceback (most recent call last): >> File "", line 1, in >> TypeError: can only concatenate list (not "tuple") to list >> >> In-place addition += does work: >> >> py> a = [1,2,3] >> py> a += (4,5) >> py> a >> [1, 2, 3, 4, 5] > > I guess to expand a bit more on what I said... What should the result > be? > A list or a tuple? The reason += works is because the end result is > clear; a list. But it is ambiguous in the case of concatenation: did you > want a tuple or a list? Uhm... it seems "obvious" to me that [1,2,3] + (4,5) should be [1,2,3,4,5]. A list. That's what I would expect, although I cannot explain why is it *so* obvious to me. Given that 2 + 3.5, and 'abc' + u'def' both return an instance of their right operand's type, I should probably revise my preconceptions... -- Gabriel Genellina From news123 at free.fr Mon Jan 4 04:38:58 2010 From: news123 at free.fr (News123) Date: Mon, 04 Jan 2010 10:38:58 +0100 Subject: python xmlrpc client with ssl client certificates and standard modules In-Reply-To: References: <4b414e79$0$23321$426a34cc@news.free.fr> Message-ID: <4b41b733$0$13831$426a74cc@news.free.fr> Thanks for your answer. I'll look at web2py. However web2py seems to address the xmlrpc server (at least in your example). The xmlrpc server application exists alerady and requires a client certificate. The client example doesn't seem to be using a certificate. So I'll be reading a little into web2py. bye N mdipierro wrote: > xmlrpc acts at the application layer and ssl at the transport layer so > they can inter operate easily as long as you do not use the > certificate to authenticate the client but only validate the server > and encrypt data (which you can also do but it is more complicated) > > One option for you is to use web2py which include an xmlrpc server > that uses a wsgi ssl enabled web server. > > Here is how: > > 1) Install web2py > 2) Visit http://127.0.0.1:8000/admin and create a new application from > the web based IDE > 3) create your web service for example, in a controller default.py > > @service.xmlrpc > def add(a,b): return int(a)+int(b) > > 4) Restart web2py with > > python web2py.py -a ADMIN_PASSWD -c SSL_CERTIFICATE -k > SSL_PRIVATE_KEY -i 0.0.0.0 -p 443 > > 5) You can now access the service from any Python program: > > >>> import xmlrpclib > >>> server_url = 'https://myserver:443/yourapp/default/call/ > xmlrpc' > >>> server = xmlrpclib.Server(server_url) > >>> print server.add(3,4) > 7 > > Hope this helps. > > > > On Jan 3, 8:12 pm, News123 wrote: >> Hi, >> >> I was googling fot quite some time and was not really succesfull. >> >> I found one solution, which I will try soon. >> It ishttp://www.cs.technion.ac.il/~danken/xmlrpc-ssl.html >> (found inhttp://hamakor.org.il/pipermail/python-il/2008-February/000029.html) >> >> This will probably work, but it requires the module M2Crypto. >> >> In order to avoid installing M2Crypto an all hosts that want to run the >> script I wondered, whether there is no other solution. >> >> I can do xmlrpc over ssl WITHOUT certificates with following code: >> >> import xmlrpclib >> server_url = 'https://myserver' >> server = xmlrpclib.Server(server_url); >> >> and I can perform a https get request WITH certificates with below snippet: >> >> import httplib >> conn = httplib.HTTPSConnection( >> HOSTNAME, >> key_file = KEYFILE, >> cert_file = CERTFILE >> ) >> conn.putrequest('GET', '/') >> conn.endheaders() >> response = conn.getresponse() >> print response.read() >> >> I'm just lost of how to 'combine' both. >> >> Thanks in advance for any suggestions / hints >> >> N > From gagsl-py2 at yahoo.com.ar Mon Jan 4 04:39:45 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 04 Jan 2010 06:39:45 -0300 Subject: TypeError: can only concatenate list (not "tuple") to list References: Message-ID: En Mon, 04 Jan 2010 05:22:44 -0300, Steven D'Aprano escribi?: > On Mon, 04 Jan 2010 04:59:02 -0300, Gabriel Genellina wrote: > >> Is there any reason for this error? Apart from "nobody cared to write >> the code" > > Yes, because such implicit conversions would be a bad idea. I'm slowly convincing myself that it was actually a bad idea... >> In-place addition += does work: >> >> py> a = [1,2,3] >> py> a += (4,5) >> py> a >> [1, 2, 3, 4, 5] > > I call that an impressive gotcha. I believe that is because in-place > addition of lists is implemented as functionally equivalent to the extend > method: > >>>> a += "abc" # same as a.extend("abc") >>>> a > [1, 2, 3, 4, 5, 'a', 'b', 'c'] >>>> a += {None: -1} >>>> a > [1, 2, 3, 4, 5, 'a', 'b', 'c', None] So += and extend are completely permissive - they slurp whatever comes from iterating their right operand. Totally unexpected in some cases, as in your examples above... -- Gabriel Genellina From steven at REMOVE.THIS.cybersource.com.au Mon Jan 4 04:45:46 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 04 Jan 2010 09:45:46 GMT Subject: TypeError: can only concatenate list (not "tuple") to list References: <1992.68.59.29.80.1262593496.squirrel@www.bibliobazaar.com> Message-ID: On Mon, 04 Jan 2010 06:27:48 -0300, Gabriel Genellina wrote: > En Mon, 04 Jan 2010 05:24:56 -0300, David Williams > escribi?: > >>> py> [1,2,3] + (4,5) >>> Traceback (most recent call last): >>> File "", line 1, in >>> TypeError: can only concatenate list (not "tuple") to list >>> >>> In-place addition += does work: >>> >>> py> a = [1,2,3] >>> py> a += (4,5) >>> py> a >>> [1, 2, 3, 4, 5] >> >> I guess to expand a bit more on what I said... What should the result >> be? >> A list or a tuple? The reason += works is because the end result is >> clear; a list. But it is ambiguous in the case of concatenation: did >> you want a tuple or a list? > > Uhm... it seems "obvious" to me that [1,2,3] + (4,5) should be > [1,2,3,4,5]. A list. That's what I would expect, although I cannot > explain why is it *so* obvious to me. If you can't explain it, it's probably a bad idea. Why should lists be privileged over tuples? How does it work? Do tuples automatically turn into lists, or does the left hand value over-ride the right hand value? In other words, would you expect: "1" + 1 = "11" 1 + "1" = 2 or would you expect: "1" + 1 = "11" 1 + "1" = "11" > Given that 2 + 3.5, and 'abc' + u'def' both return an instance of their > right operand's type, I should probably revise my preconceptions... Yes, you should be a little more careful in your tests. >>> 2 + 3.5 5.5 >>> 3.5 + 2 5.5 Nothing to do with left versus right. -- Steven From ben+python at benfinney.id.au Mon Jan 4 04:48:37 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 04 Jan 2010 20:48:37 +1100 Subject: twenty years ago Guido created Python References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> Message-ID: <87oclaxksq.fsf@benfinney.id.au> Daniel Fetchinson writes: > Go, quick, these threads are all around us! They are popping up > everywhere! Go, go, quick! There is another troll thread right there! I wouldn't characterise this thread as a ?troll thread?. If you think it is, that's fine I suppose. I'll continue to try improving the signal to noise ratio (modulo responses like this, sadly) by trying to emit more signal and less noise. -- \ ?Never do anything against conscience even if the state demands | `\ it.? ?Albert Einstein | _o__) | Ben Finney From tim.wintle at teamrubber.com Mon Jan 4 05:43:06 2010 From: tim.wintle at teamrubber.com (Tim Wintle) Date: Mon, 04 Jan 2010 10:43:06 +0000 Subject: ANN: Pymazon 0.1beta released. In-Reply-To: <7f014ea60912291044i58f9490amddfc6cf24bdb2a18@mail.gmail.com> References: <7f014ea60912291044i58f9490amddfc6cf24bdb2a18@mail.gmail.com> Message-ID: <1262601786.8156.2.camel@localhost> On Tue, 2009-12-29 at 19:44 +0100, Chris Colbert wrote: > I'm happy to announce the first beta release of Pymazon: a Python > implemented alternative to the Amazon mp3 downloader. > > Pymazon was created specifically to alleviate the issues surrounding > the Linux version of the Amazon mp3 downloader (though it should run > just fine in Windows too). Thanks! I've been complaining to them for not providing 64-bit binaries (or source) for ages! (Although hats off to them for even providing the number of linux variants they do support) From larry at hastings.org Mon Jan 4 06:09:32 2010 From: larry at hastings.org (Larry Hastings) Date: Mon, 04 Jan 2010 03:09:32 -0800 Subject: Best practices for simultaneously installed versioned packages? Message-ID: <4B41CC6C.3020808@hastings.org> I'm writing a package for Python 3--let's call it "spacegoblin". I fear someday I may need multiple versions installed and available simultaneously, even within one version of Python. So I want to plan ahead for that possibility. What would be the best way to allow this? Right now I install the package with the version in the name, like "spacegoblin_1_0" and "spacegoblin_1_1". But I have an uneasy sense that I'm doing something stupid... mainly because I've never seen anyone do this before. A coworker said they did it like "spacegoblin.1_0" and "spacegoblin.1_1" at a previous employer. That seems like an improvement, though not the "yes that's obviously right" answer I'm holding out for. Your thoughts? Thanks, /larry/ p.s. Before you ask: no, I don't want to use "virtualenv" for this. -------------- next part -------------- An HTML attachment was scrubbed... URL: From highcar at gmail.com Mon Jan 4 06:25:29 2010 From: highcar at gmail.com (elca) Date: Mon, 4 Jan 2010 03:25:29 -0800 (PST) Subject: python mechanize proxy support question In-Reply-To: <4ca2b83a-3b82-4d20-85a3-b6304f25fb81@21g2000yqj.googlegroups.com> References: <27009696.post@talk.nabble.com> <4ca2b83a-3b82-4d20-85a3-b6304f25fb81@21g2000yqj.googlegroups.com> Message-ID: <27011661.post@talk.nabble.com> alex23 wrote: > > On Jan 4, 5:42?pm, elca wrote: >> how can i add proxy support into my mechanize script? >> i was look for some reference , but not so much good hint from google. > > There are examples on using proxies with mechanize on the module's > home page: > http://wwwsearch.sourceforge.net/mechanize/ > -- > http://mail.python.org/mailman/listinfo/python-list > > HI.. that is only support mechanize.browser module.. actually im looking mechanize.urlopen method. thanks -- View this message in context: http://old.nabble.com/python-mechanize-proxy-support-question-tp27009696p27011661.html Sent from the Python - python-list mailing list archive at Nabble.com. From steve at holdenweb.com Mon Jan 4 06:49:57 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 04 Jan 2010 06:49:57 -0500 Subject: twenty years ago Guido created Python In-Reply-To: <87oclaxksq.fsf@benfinney.id.au> References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> <87oclaxksq.fsf@benfinney.id.au> Message-ID: Ben Finney wrote: > Daniel Fetchinson writes: > >> Go, quick, these threads are all around us! They are popping up >> everywhere! Go, go, quick! There is another troll thread right there! > > I wouldn't characterise this thread as a ?troll thread?. If you think it > is, that's fine I suppose. > > I'll continue to try improving the signal to noise ratio (modulo > responses like this, sadly) by trying to emit more signal and less > noise. > And I, for one, offer you my thanks for your efforts. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Mon Jan 4 06:54:37 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 04 Jan 2010 06:54:37 -0500 Subject: Significant whitespace In-Reply-To: References: <4B3E70FC.1010702@gmail.com> Message-ID: r0g wrote: > David Robinow wrote: >> On Sun, Jan 3, 2010 at 10:42 PM, Roy Smith wrote: >>> In article , >>> David Robinow wrote: >>> >>>> On Sun, Jan 3, 2010 at 8:09 PM, Tim Roberts wrote: >>>>> More than "not required", it was "not relevant". This led to one of the >>>>> most infamous programming blunders in the early days of the space program, >>>>> when one programmer accidentially typed a period instead of a comma >>>>> resulting in the loss of a satellite: >>>> Interesting story. Did you make it up? >>> It's a fairly well known story. >>> >>> http://catless.ncl.ac.uk/Risks/5.64.html#subj4.2 >> Sure. But the question is, "Who made it up?" >> http://en.wikiquote.org/wiki/Fortran >> >> Computer folklore has incorrectly attributed the loss of the Mariner 1 >> space probe to a syntax error in a Fortran program. For example, >> "Recall the first American space probe to Venus, reportedly lost >> because Fortran cannot recognize a missing comma in a DO statement?"[ > > > > Wow you're totally right! It was a missing UNDERSCORE not a DOT being > typed instead of a COMMA! We've been lied to ALL ALONG! What kind of > sickos would do this? Let's find this Tim character, flay him alive and > then go crown you the KING OF EVERYTHING for rescuing us from that > deadly nugget of misinformation! > > Honestly I might have believed those dot/comma LIES if it wasn't for > your noble selfless efforts to uncover the SHOCKING TRUTH. Imagine what > might have happened! The character in question being an UNDERSCORE > COMPLETELY CHANGES the meaning and point of the story. As for it being > LEFT OUT rather than TRANSPOSED, well, anyone can see how that TURNS THE > WHOLE STORY UPSIDE DOWN. > > Of course there are the naysayers who might claim absolute accuracy is > no big deal in stories that are essentially parables but they do not > understand the HARM that can come from not remembering really tiny, > almost inconsequential details. For what it's worth I DO NOT consider > you a tiresome pedant and I think your taunting, patronising style is > completely appropriate and proportional to Tim's sickening crimes > against fact. > > Roger. Personally I think the overall tone of this group would have been marginally improved if you had summoned the self-restraint to simply not make this posting. Fortunately it's still an unmoderated group, so you can continue to be as hostile and snarky as you like. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Mon Jan 4 06:57:05 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 04 Jan 2010 06:57:05 -0500 Subject: Question about an application In-Reply-To: <27009252.post@talk.nabble.com> References: <27009118.post@talk.nabble.com> <9A7F3B4C-5621-44CF-ABCA-D440E7ADA01E@gmail.com> <27009252.post@talk.nabble.com> Message-ID: rieh25 wrote: [top-posting corrected] >> On Jan 4, 2010, at 12:50 AM, rieh25 wrote: >> >>> I am thinking of installing a python webserver I coded in every >>> computer at >>> my work. This would allow me to run specific tasks in them, like >>> creating >>> backups, installing things, etc. Is there another way to run >>> programs in >>> remote computers? Thanks for your opinions. > > Rodrick Brown wrote: >> Take a look at ssh >> >> Sent from my iPhone 3GS. > Thanks, I had forgotten about it. I'll investigate if there are ways to > automate using it to run programs in several computers at the same time. > You might also want to investigate Fabric, which was put together for such purposes. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From jeanmichel at sequans.com Mon Jan 4 08:36:25 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 04 Jan 2010 14:36:25 +0100 Subject: TypeError: can only concatenate list (not "tuple") to list In-Reply-To: References: <1992.68.59.29.80.1262593496.squirrel@www.bibliobazaar.com> Message-ID: <4B41EED9.4040506@sequans.com> Gabriel Genellina wrote: > En Mon, 04 Jan 2010 05:24:56 -0300, David Williams > escribi?: > >>> py> [1,2,3] + (4,5) >>> Traceback (most recent call last): >>> File "", line 1, in >>> TypeError: can only concatenate list (not "tuple") to list >>> >>> In-place addition += does work: >>> >>> py> a = [1,2,3] >>> py> a += (4,5) >>> py> a >>> [1, 2, 3, 4, 5] >> >> I guess to expand a bit more on what I said... What should the >> result be? >> A list or a tuple? The reason += works is because the end result is >> clear; a list. But it is ambiguous in the case of concatenation: did >> you >> want a tuple or a list? > > Uhm... it seems "obvious" to me that [1,2,3] + (4,5) should be > [1,2,3,4,5]. A list. That's what I would expect, although I cannot > explain why is it *so* obvious to me. > Given that 2 + 3.5, and 'abc' + u'def' both return an instance of > their right operand's type, I should probably revise my preconceptions... > Haa... The well known 'obviousness theorem' :o) by which everything that I say becomes true. Best theorem ever. I'm really surprised that s = [1,2,3] ; s += (4,5) works fine. Semantically, it is adding 2 operands of different type, this is not very smart. As variales have no predefined type and can change over statements, it would be unwise to assume that s += (4,5) should produce a list. JM From fetchinson at googlemail.com Mon Jan 4 08:51:42 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Mon, 4 Jan 2010 14:51:42 +0100 Subject: twenty years ago Guido created Python In-Reply-To: References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> <87oclaxksq.fsf@benfinney.id.au> Message-ID: >>> Go, quick, these threads are all around us! They are popping up >>> everywhere! Go, go, quick! There is another troll thread right there! >> >> I wouldn't characterise this thread as a ?troll thread?. If you think it >> is, that's fine I suppose. >> >> I'll continue to try improving the signal to noise ratio (modulo >> responses like this, sadly) by trying to emit more signal and less >> noise. >> > And I, for one, offer you my thanks for your efforts. Me too. But Ben will not get the Best Sense of Humor Award of 2010 for sure :) Although who knows, there is a full year ahead and things can change ........ Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From sccolbert at gmail.com Mon Jan 4 08:52:16 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Mon, 4 Jan 2010 14:52:16 +0100 Subject: ANN: Pymazon 0.1beta released. In-Reply-To: <1262601786.8156.2.camel@localhost> References: <7f014ea60912291044i58f9490amddfc6cf24bdb2a18@mail.gmail.com> <1262601786.8156.2.camel@localhost> Message-ID: <7f014ea61001040552x5c7ae3fbg844386245e2c1707@mail.gmail.com> On Mon, Jan 4, 2010 at 11:43 AM, Tim Wintle wrote: > On Tue, 2009-12-29 at 19:44 +0100, Chris Colbert wrote: > > I'm happy to announce the first beta release of Pymazon: a Python > > implemented alternative to the Amazon mp3 downloader. > > > > Pymazon was created specifically to alleviate the issues surrounding > > the Linux version of the Amazon mp3 downloader (though it should run > > just fine in Windows too). > > Thanks! > > I've been complaining to them for not providing 64-bit binaries (or > source) for ages! (Although hats off to them for even providing the > number of linux variants they do support) > Youre completely right, and my language on the google code site was a little harsh towards Amazon in that respect. I've fixed that now. Cheers! -------------- next part -------------- An HTML attachment was scrubbed... URL: From massimodipierro71 at gmail.com Mon Jan 4 09:05:32 2010 From: massimodipierro71 at gmail.com (mdipierro) Date: Mon, 4 Jan 2010 06:05:32 -0800 (PST) Subject: python xmlrpc client with ssl client certificates and standard modules References: <4b414e79$0$23321$426a34cc@news.free.fr> <4b41b733$0$13831$426a74cc@news.free.fr> Message-ID: If it is a client problem than web2py will be on help. If your server is written already you may be able to use it with the ssl cherrypy wsgi server (the one that web2py uses) and you do not need web2py at all. Massimo On Jan 4, 3:38?am, News123 wrote: > Thanks for your answer. > > I'll look at web2py. > > However web2py seems to address the xmlrpc server (at least in your > example). The xmlrpc server application exists alerady and requires a > client certificate. > > The client example doesn't seem to be using a certificate. > > So I'll be reading a little into web2py. > > bye > > N > > mdipierro wrote: > > xmlrpc acts at the application layer and ssl at the transport layer so > > they can inter operate easily as long as you do not use the > > certificate to authenticate the client but only validate the server > > and encrypt data (which you can also do but it is more complicated) > > > One option for you is to use web2py which include an xmlrpc server > > that uses a wsgi ssl enabled web server. > > > Here is how: > > > 1) Install web2py > > 2) Visithttp://127.0.0.1:8000/adminand create a new application from > > the web based IDE > > 3) create your web service for example, in a controller default.py > > > ? ? ?@service.xmlrpc > > ? ? ?def add(a,b): return int(a)+int(b) > > > 4) Restart web2py with > > > ? ? ?python web2py.py -a ADMIN_PASSWD -c SSL_CERTIFICATE -k > > SSL_PRIVATE_KEY -i 0.0.0.0 -p 443 > > > 5) You can now access the service from any Python program: > > > ? ? ?>>> import xmlrpclib > > ? ? ?>>> server_url = 'https://myserver:443/yourapp/default/call/ > > xmlrpc' > > ? ? ?>>> server = xmlrpclib.Server(server_url) > > ? ? ?>>> print server.add(3,4) > > ? ? ?7 > > > Hope this helps. > > > On Jan 3, 8:12 pm, News123 wrote: > >> Hi, > > >> I was googling fot quite some time and was not really succesfull. > > >> I found one solution, which I will try soon. > >> It ishttp://www.cs.technion.ac.il/~danken/xmlrpc-ssl.html > >> (found inhttp://hamakor.org.il/pipermail/python-il/2008-February/000029.html) > > >> This will probably work, but it requires the module M2Crypto. > > >> In order to avoid installing M2Crypto an all hosts that want to run the > >> script I wondered, whether there is no other solution. > > >> I can do xmlrpc over ssl WITHOUT certificates with following code: > > >> import xmlrpclib > >> server_url = 'https://myserver' > >> server = xmlrpclib.Server(server_url); > > >> and I can perform a https get request WITH certificates with below snippet: > > >> import httplib > >> conn = httplib.HTTPSConnection( > >> ? ? ? ? HOSTNAME, > >> ? ? ? ? key_file = KEYFILE, > >> ? ? ? ? cert_file = CERTFILE > >> ) > >> conn.putrequest('GET', '/') > >> conn.endheaders() > >> response = conn.getresponse() > >> print response.read() > > >> I'm just lost of how to 'combine' both. > > >> Thanks in advance for any suggestions / hints > > >> N > > From alfps at start.no Mon Jan 4 10:17:12 2010 From: alfps at start.no (Alf P. Steinbach) Date: Mon, 04 Jan 2010 16:17:12 +0100 Subject: How do you configure IDLE on a Mac, seeing as there's no Configure Option... In-Reply-To: References: Message-ID: * Mensanator: > ...because there's no [Options] menu on the shell window? > > Or at least give me a clue to how to use Courier New font? > > For some inscrutable reason, depite the plethora of formatting tools, > someone decided that proportional spaced fonts ought to be the > default for IDLE. Why not just use the Python interpreter, a reasonable programmer's editor and, if you need it, some debugger. The source code level debugging in IDLE is nothing to write home about. In Windows XP it is just silly: it marks the current line by ordinary text selection, which (depending on your config, but I think this is default) is invisible when you're looking at the debugger window for single stepping. You can see it by switching back to the source code window but that's annoying, not very practical. Invisible highlighting of the current line, he he... Cheers & hth., - Alf From nobody at nowhere.com Mon Jan 4 10:46:57 2010 From: nobody at nowhere.com (Nobody) Date: Mon, 04 Jan 2010 15:46:57 +0000 Subject: IOError - cannot create file (linux daemon-invoked script) References: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> <40b81356-f0c1-4249-a536-916356377afa@u41g2000yqe.googlegroups.com> Message-ID: On Sun, 03 Jan 2010 13:56:24 -0800, cassiope wrote: > I'm changing the uid and gid in the daemon (which runs with root > permissions > until the fork and uid/gid change). The uid and gid are confirmed by > printing os.getuid() and os.getgid() in the script. Those tell you the *real* UID/GID. Filesystem checks are performed using the *effective* UID/GID. You need to set them with seteuid/setegid and check them with geteuid/getegid. From benjamin.kaplan at case.edu Mon Jan 4 11:05:00 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Mon, 4 Jan 2010 11:05:00 -0500 Subject: How do you configure IDLE on a Mac, seeing as there's no Configure Option... In-Reply-To: References: Message-ID: On Sun, Jan 3, 2010 at 8:42 PM, Mensanator wrote: > ...because there's no [Options] menu on the shell window? > > Or at least give me a clue to how to use Courier New font? > > For some inscrutable reason, depite the plethora of formatting tools, > someone decided that proportional spaced fonts ought to be the > default for IDLE. > -- > http://mail.python.org/mailman/listinfo/python-list > Mac Applications don't have "options" menus. They have a "preferences" pane which is the second item in the application menu (the one that takes the name of the active program) and has a keyboard shortcut of Cmd-, Apple's user-interface guidelines are pretty specific, so you'll find that this works for all native OS X (i.e. not X11) applications. From briandenzer at gmail.com Mon Jan 4 11:09:56 2010 From: briandenzer at gmail.com (Brian D) Date: Mon, 4 Jan 2010 08:09:56 -0800 (PST) Subject: fsync() doesn't work as advertised? Message-ID: If I'm running a process in a loop that runs for a long time, I occasionally would like to look at a log to see how it's going. I know about the logging module, and may yet decide to use that. Still, I'm troubled by how fsync() doesn't seem to work as advertised: http://docs.python.org/library/os.html "If you?re starting with a Python file object f, first do f.flush(), and then do os.fsync(f.fileno())" Has anyone else experienced and overcome this issue? What I've seen is that flush() alone produces a complete log when the loop finishes. When I used fsync(), I lost all of the write entries except the first, along with odd error trap and the last entry. From benjamin.kaplan at case.edu Mon Jan 4 11:16:39 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Mon, 4 Jan 2010 11:16:39 -0500 Subject: Best practices for simultaneously installed versioned packages? In-Reply-To: <4B41CC6C.3020808@hastings.org> References: <4B41CC6C.3020808@hastings.org> Message-ID: On Mon, Jan 4, 2010 at 6:09 AM, Larry Hastings wrote: > > I'm writing a package for Python 3--let's call it "spacegoblin". I fear > someday I may need multiple versions installed and available simultaneously, > even within one version of Python. So I want to plan ahead for that > possibility. What would be the best way to allow this? Right now I install > the package with the version in the name, like "spacegoblin_1_0" and > "spacegoblin_1_1". But I have an uneasy sense that I'm doing something > stupid... mainly because I've never seen anyone do this before. > > A coworker said they did it like "spacegoblin.1_0" and "spacegoblin.1_1" at > a previous employer. That seems like an improvement, though not the "yes > that's obviously right" answer I'm holding out for. Your thoughts? > > Thanks, > > > /larry/ > > p.s. Before you ask: no, I don't want to use "virtualenv" for this. > wxpython installs a "wxversion" module which has functions like getInstalled(), ensureMinimal(version), and select(version). You can call wxversion.select before importing wx and it will make sure that the correct version is imported. You might want to look up what they did. From me4 at privacy.net Mon Jan 4 11:22:42 2010 From: me4 at privacy.net (=?iso-8859-1?Q?Jens_M=FCller?=) Date: Mon, 4 Jan 2010 17:22:42 +0100 Subject: Speeding up network access: threading? Message-ID: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> Hello, what would be best practise for speeding up a larger number of http-get requests done via urllib? Until now they are made in sequence, each request taking up to one second. The results must be merged into a list, while the original sequence needs not to be kept. I think speed could be improved by parallizing. One could use multiple threads. Are there any python best practises, or even existing modules, for creating and handling a task queue with a fixed number of concurrent threads? Thanks and regards! From solipsis at pitrou.net Mon Jan 4 11:29:45 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 4 Jan 2010 16:29:45 +0000 (UTC) Subject: fsync() doesn't work as advertised? References: Message-ID: Le Mon, 04 Jan 2010 08:09:56 -0800, Brian D a ?crit?: > > What I've seen is that flush() alone produces a complete log when the > loop finishes. When I used fsync(), I lost all of the write entries > except the first, along with odd error trap and the last entry. Perhaps you are writing to the file from several threads or processes at once? By the way, you shouldn't need fsync() if you merely want to look at the log files, because your OS will have an up-to-date view of the file contents anyway. fsync() is useful if you want to be sure the data has been written to the hard disk drive, rather than just kept in the operating system's filesystem cache. From steve at holdenweb.com Mon Jan 4 11:48:18 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 04 Jan 2010 11:48:18 -0500 Subject: How do you configure IDLE on a Mac, seeing as there's no Configure Option... In-Reply-To: References: Message-ID: Alf P. Steinbach wrote: > * Mensanator: >> ...because there's no [Options] menu on the shell window? >> >> Or at least give me a clue to how to use Courier New font? >> >> For some inscrutable reason, depite the plethora of formatting tools, >> someone decided that proportional spaced fonts ought to be the >> default for IDLE. Yeah, that was a smart move, wasn't it? > > Why not just use the Python interpreter, a reasonable programmer's > editor and, if you need it, some debugger. > > The source code level debugging in IDLE is nothing to write home about. > I quite agree. > In Windows XP it is just silly: it marks the current line by ordinary > text selection, which (depending on your config, but I think this is > default) is invisible when you're looking at the debugger window for > single stepping. You can see it by switching back to the source code > window but that's annoying, not very practical. Invisible highlighting > of the current line, he he... > I remember being extremely surprised to read that a Python developer I greatly admire uses IDLE as his main development interface. Frankly I have always found IDLE extremely non-intuitive, and I am constantly frustrated at having to explain about its shortcomings and oddities to newcomers in my "Introduction to Python" classes. Who actually maintains IDLE nowadays? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From jjposner at optimum.net Mon Jan 4 11:49:15 2010 From: jjposner at optimum.net (John Posner) Date: Mon, 04 Jan 2010 11:49:15 -0500 Subject: Dynamic text color References: Message-ID: On Fri, 01 Jan 2010 21:01:04 -0500, Cousin Stanley wrote: > > I was not familiar with the re.finditer method > for searching strings ... Stanley and Dave -- So far, we've just been using finditer() to perform standard-string searches (e.g. on the word "red"). Since Dave now wants to color multiple words the same color (e.g. the words in redList), we can use a single regular-expression search to locate *all* the words in a list. This eliminates the need to use a "for" loop to handle the list. Here's what I mean: >>> import re >>> s = "it is neither red nor crimson, but scarlet, you see" ########## individual searches >>> [matchobj.span() for matchobj in re.finditer("red", s)] [(14, 17)] >>> [matchobj.span() for matchobj in re.finditer("crimson", s)] [(22, 29)] >>> [matchobj.span() for matchobj in re.finditer("scarlet", s)] [(35, 42)] ########## one "swell foop" >>> redList = "red crimson scarlet".split() >>> redList_regexp = "|".join(redList) >>> redList_regexp 'red|crimson|scarlet' >>> [matchobj.span() for matchobj in re.finditer(redList_regexp, s)] [(14, 17), (22, 29), (35, 42)] -John From exarkun at twistedmatrix.com Mon Jan 4 11:52:28 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Mon, 04 Jan 2010 16:52:28 -0000 Subject: Speeding up network access: threading? In-Reply-To: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> References: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <20100104165228.15596.233351734.divmod.xquotient.816@localhost.localdomain> On 04:22 pm, me4 at privacy.net wrote: >Hello, > >what would be best practise for speeding up a larger number of http-get >requests done via urllib? Until now they are made in sequence, each >request taking up to one second. The results must be merged into a >list, while the original sequence needs not to be kept. > >I think speed could be improved by parallizing. One could use multiple >threads. >Are there any python best practises, or even existing modules, for >creating and handling a task queue with a fixed number of concurrent >threads? Using multiple threads is one approach. There are a few thread pool implementations lying about; one is part of Twisted, . Another approach is to use non-blocking or asynchronous I/O to make multiple requests without using multiple threads. Twisted can help you out with this, too. There's two async HTTP client APIs available. The older one: http://twistedmatrix.com/documents/current/api/twisted.web.client.getPage.html http://twistedmatrix.com/documents/current/api/twisted.web.client.HTTPClientFactory.html And the newer one, introduced in 9.0: http://twistedmatrix.com/documents/current/api/twisted.web.client.Agent.html Jean-Paul From fpm at u.washington.edu Mon Jan 4 12:16:52 2010 From: fpm at u.washington.edu (cassiope) Date: Mon, 4 Jan 2010 09:16:52 -0800 (PST) Subject: IOError - cannot create file (linux daemon-invoked script) References: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> <40b81356-f0c1-4249-a536-916356377afa@u41g2000yqe.googlegroups.com> Message-ID: On Jan 4, 7:46?am, Nobody wrote: > On Sun, 03 Jan 2010 13:56:24 -0800, cassiope wrote: > > I'm changing the uid and gid in the daemon (which runs with root > > permissions > > until the fork and uid/gid change). ?The uid and gid are confirmed by > > printing os.getuid() and os.getgid() in the script. > > Those tell you the *real* UID/GID. Filesystem checks are performed using > the *effective* UID/GID. > > You need to set them with seteuid/setegid and check them with > geteuid/getegid. To Cameron: the file doesn't (yet) exist; and it has the correct full path. To "Nobody" : hey, this seems interesting. First test, invoking seteuid() and setegid() didn't help - but strange thing was these calls didn't show up in the strace, so perhaps I wasn't testing what I thought I was. I'll have to check this later as people are now back to work and needing the daemon for its other functions... Thanks for your ideas! I'll be checking it more later... From tjreedy at udel.edu Mon Jan 4 12:17:46 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 04 Jan 2010 12:17:46 -0500 Subject: Speeding up network access: threading? In-Reply-To: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> References: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> Message-ID: On 1/4/2010 11:22 AM, Jens M?ller wrote: > Hello, > > what would be best practise for speeding up a larger number of http-get > requests done via urllib? Until now they are made in sequence, each > request taking up to one second. The results must be merged into a list, > while the original sequence needs not to be kept. > > I think speed could be improved by parallizing. One could use multiple > threads. > Are there any python best practises, or even existing modules, for > creating and handling a task queue with a fixed number of concurrent > threads? I believe code of this type has been published here in various threads. The fairly obvious thing to do is use a queue.queue for tasks and another for results and a pool of threads that read, fetch, and write. From louisrom1 at gmail.com Mon Jan 4 12:24:22 2010 From: louisrom1 at gmail.com (louisJ) Date: Mon, 4 Jan 2010 09:24:22 -0800 (PST) Subject: No module named _socket, on windows Message-ID: Hi I installed python 2.6 (from python.org) for windows XP, and then Pylab. When I type "import pylab" in a python shell it shows the error: ImportError: No module named _socket Any idea what can I do to make this work? thank you From victorsubervi at gmail.com Mon Jan 4 12:28:13 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Mon, 4 Jan 2010 13:28:13 -0400 Subject: Can't Add Variable Message-ID: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> Hi; I have this code snippet: sql '''create table if not exists %sCustomerData ( ID tinyint(8) unsigned primary key auto_increment, Email varchar(120) not null, PhoneNumber varchar(20) not null, BillingName varchar(80) not null, BillingAddress1 varchar(100) not null, BillingAddress2 varchar(100) null, BillingCity varchar(50) not null, BillingState varchar(2) not null, BillingPostalCode varchar(8) not null, ShippingName varchar(80) not null, ShippingAddress1 varchar(100) not null, ShippingAddress2 varchar(100) null, ShippingCity varchar(50) not null, ShippingState varchar(2) not null, ShippingPostalCode varchar(8) not null, TypeOfCard set("VISA", "MasterCard", "AmEx", "Discover", "PayPal") null, CreditCardNumber tinyint(8) unsigned null, CreditCardExpMonth tinyint(2) unsigned null, CreditCardExpYear tinyint(4) unsigned null );''' % store It throws this error: [Mon Jan 04 09:23:54 2010] [error] [client 66.82.9.61] File "/var/www/html/angrynates.com/cart/cart2.py", line 62, referer: http://angrynates.com/cart/cart.py [Mon Jan 04 09:23:54 2010] [error] [client 66.82.9.61] );''' % store, referer: http://angrynates.com/cart/cart.py [Mon Jan 04 09:23:54 2010] [error] [client 66.82.9.61] ^, referer: http://angrynates.com/cart/cart.py [Mon Jan 04 09:23:54 2010] [error] [client 66.82.9.61] SyntaxError: invalid syntax, referer: http://angrynates.com/cart/cart.py [Mon Jan 04 09:23:54 2010] [error] [client 66.82.9.61] Premature end of script headers: cart2.py, referer: http://angrynates.com/cart/cart.py Variable "store" is defined. How is my syntax invalid? Total code follows. TIA, beno #! /usr/bin/python import string import cgitb; cgitb.enable() import MySQLdb import cgi import sys,os sys.path.append(os.getcwd()) from login import login import datetime, Cookie, random from particulars import title, myCookie, ourOptions, whereToShip, paymentOptionsPickup, paymentOptionsFax, paymentGateways, VISA, masterCard, amEx, discover, payPal from templateFrame import top, bottom import time import fpformat from sets import Set ourURL = string.split(__file__, 'html/') ourURL = string.split(ourFile[1], '/')[0] def commitSale(): user, passwd, db, host = login() database = MySQLdb.connect(host, user, passwd, db) cursor= database.cursor() ourEmail = email() form = cgi.FieldStorage() which = form.getfirst('which') store = form.getfirst('store') patientID = form.getfirst('patientID') customerLoginEmail = form.getfirst('customerEmail') # This is for customers who lost their login info customerLoginFirstName = form.getfirst('customerLoginFirstName') customerLoginLastName = form.getfirst('customerLoginLastName') customerLoginData = [patientID, customerLoginEmail, customerLoginFirstName, customerLoginLastName] if (store == 'prescriptions') and ((customerLoginEmail is not None) or (customerLoginFirstName is not None)): myMail(which, store, '', ourURL, '', '', customerLoginData) elif which == 'order': form = cgi.FieldStorage() tmpTable = form.getfirst('tmpTable') howPay = form.getfirst('howPay') ccNumber = form.getfirst('creditCardNumber') ccFirstHalf = ccNumber[0:8] ccSecondHalf = ccNumber[8:] sql '''create table if not exists %sCustomerData ( ID tinyint(8) unsigned primary key auto_increment, Email varchar(120) not null, PhoneNumber varchar(20) not null, BillingName varchar(80) not null, BillingAddress1 varchar(100) not null, BillingAddress2 varchar(100) null, BillingCity varchar(50) not null, BillingState varchar(2) not null, BillingPostalCode varchar(8) not null, ShippingName varchar(80) not null, ShippingAddress1 varchar(100) not null, ShippingAddress2 varchar(100) null, ShippingCity varchar(50) not null, ShippingState varchar(2) not null, ShippingPostalCode varchar(8) not null, TypeOfCard set("VISA", "MasterCard", "AmEx", "Discover", "PayPal") null, CreditCardNumber tinyint(8) unsigned null, CreditCardExpMonth tinyint(2) unsigned null, CreditCardExpYear tinyint(4) unsigned null );''' % store print sql # cursor.execute(sql) sql = '''insert into %sCustomerData (Email, PhoneNumber, BillingName, BillingAddress1, BillingAddress2, BillingCity, BillingState, BillingPostalCode, ShippingName, ShippingAddress1, ShippingAddress2, ShippingCity, ShippingState, ShippingPostalCode, TypeOfCard, CreditCardNumber, CreditCardExpMonth, CreditCardExpYear) value ("%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s");' % (store, form.getfirst('customerEmail'), form.getfirst('customerPhoneNumber'), form.getfirst('customerBillingName'), form.getfirst('customerBillingAddr1'), form.getfirst('customerBillingAddr2'), form.getfirst('customerBillingCity'), form.getfirst('customerBillingState'), form.getfirst('customerBillingPostalCode'), form.getfirst('customerShippingName'), form.getfirst('shippingSameAsShipping'), form.getfirst('customerShippingAddr1'), form.getfirst('customerShippingAddr2'), form.getfirst('customerShippingCity'), form.getfirst('customerShippingState'), form.getfirst('customerShippingPostalCode'), form.getfirst('typeOfCard'), ccFirstHalf, form.getfirst('creditCartCVC')) print sql # cursor.execute(sql) db.commit() ccFirstHalf = None cursor.execute('select max(ID) from %sCustomerData;' % store) custID = cursor.fetchone()[0] myMail(which, store, BillingName, ourURL, tmpTable, ccSecondHalf) cursor.execute('describe %s;' % tmpTable) fields = [itm[0] for itm in cursor] cursor.execute('select * from %s;' % tmpTable) order = cursor.fetchall() allValues = [] for item in order: itemValues = [] i = 0 while i < len(fields): itemValues(item[i]) i += 1 allValues.append(itemValues) order = '' for itemValues in allValues: i = 0 for item in itemValues: order += '%s: %s\t' % (fields[i], item) i += 1 order += '\n' order += 'Total: %s\n' % form.getfirst('total') def cart2(): user, passwd, db, host = login() db = MySQLdb.connect(host, user, passwd, db) cursor= db.cursor() print '''Content-Type: text/html\r\n ''' head = '' form = cgi.FieldStorage() numberOfProducts = int(form.getfirst('numberOfProducts')) total = form.getfirst('total') store = form.getfirst('store') i = 0 html = '' deleteThese = [] editThis = 0 quantity = 0 tmpTable = form.getfirst('tmpTable') while i < numberOfProducts: id = int(form.getfirst('id%d' % i)) if form.getfirst('delete%d' % i) is not None: deleteThese.append(form.getfirst('delete%d' % i)) if form.getfirst('edit%d' % i) is not None: editThis = int(form.getfirst('edit%d' % i)) quantity = form.getfirst('quantity%d' % i) id = form.getfirst('quantityID%d' % i) i += 1 i = 0 while i < len(deleteThese): sql = 'delete from %s where ProdID=%s;' % (tmpTable, deleteThese[i]) html += sql cursor.execute(sql) head = "" % store i += 1 if editThis != 0: sql = 'update %s set Quantity=%s where ProdID=%s;' % (tmpTable, quantity, id) html += sql cursor.execute(sql) head = "" % (store, quantity) if head == '': paymentMethods = [] cardTypes = [] for paymentStore, paymentType in paymentOptionsPickup().iteritems(): if (paymentStore == store) and (paymentType == 'on'): paymentMethods.append('pickup') for paymentStore, paymentType in paymentOptionsFax().iteritems(): if (paymentStore == store) and (paymentType == 'on'): paymentMethods.append('fax') for cardStore, cardType in VISA().iteritems(): if (cardStore == store) and (cardType == 'on'): cardTypes.append('VISA') for cardStore, cardType in masterCard().iteritems(): if (cardStore == store) and (cardType == 'on'): cardTypes.append('MasterCard') for cardStore, cardType in amEx().iteritems(): if (cardStore == store) and (cardType == 'on'): cardTypes.append('American Express') for cardStore, cardType in discover().iteritems(): if (cardStore == store) and (cardType == 'on'): cardTypes.append('Discover') for cardStore, cardType in payPal().iteritems(): if (cardStore == store) and (cardType == 'on'): cardTypes.append('PayPal') print '' print "" % store print "" % total print "" print "" % tmpTable print '''
''' if store != 'prescriptions': print ''' ''' else: patientID = form.getfirst('patientID') sql = 'select FirstName, LastName from patientsPersonalData where ID=%s;' % patientID cursor.execute(sql) print "%s, please tell us how you would like to pay for this order?" % (cursor.fetchone()[0] + ' ' + cursor.fetchone()[1]) PRINT PAYMENT OPTION STUFF HERE print "If you would like to pay by credit card, please fill in the following:" print ''' PRINT CC INFO HERE
Billing Information
Name:
Address 1:
Address 2:
City:
State:
Postal Code
Email Address
Phone Number
Shipping Information Same as billing information?
Shipping Information
Name:
Address 1:
Address 2:
City:
State:
Postal Code
Credit Card Information
Type of Card:
Credit Card Number:
CVC Code:
Expiration Date: Month: Year:
Method of Payment
Pay by Credit Card: ''' for method in paymentMethods: print "   Pay by %s: " % (method[0].upper() + method[1:], method) print '''
''' commitSale() else: print head print '' print '' # print html print '' cart2() -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From goon12 at gmail.com Mon Jan 4 12:40:13 2010 From: goon12 at gmail.com (Joe Riopel) Date: Mon, 4 Jan 2010 12:40:13 -0500 Subject: Can't Add Variable In-Reply-To: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> References: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> Message-ID: <6a2ccd191001040940v69ffc4bav153971cd39febc01@mail.gmail.com> On Mon, Jan 4, 2010 at 12:28 PM, Victor Subervi wrote: > Hi; > I have this code snippet: > > ??? sql '''create table if not exists %sCustomerData ( I think you may have forgotten the "=" after sql. From apt.shansen at gmail.com Mon Jan 4 12:40:18 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Mon, 4 Jan 2010 09:40:18 -0800 Subject: Can't Add Variable In-Reply-To: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> References: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> Message-ID: <7a9c25c21001040940s6e1b09dcg38d4f02c0871c361@mail.gmail.com> On Mon, Jan 4, 2010 at 9:28 AM, Victor Subervi wrote: > Hi; > I have this code snippet: > > sql '''create table if not exists %sCustomerData ( > You left out the actual assignment operator. --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From clp2 at rebertia.com Mon Jan 4 12:42:26 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Mon, 4 Jan 2010 09:42:26 -0800 Subject: Can't Add Variable In-Reply-To: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> References: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> Message-ID: <50697b2c1001040942y4f20e7f8xac6a29e195d535c@mail.gmail.com> On Mon, Jan 4, 2010 at 9:28 AM, Victor Subervi wrote: > Hi; > I have this code snippet: > > ??? sql '''create table if not exists %sCustomerData ( You're missing an equal sign there to start with (i.e. sql = '''). Cheers, Chris -- http://blog.rebertia.com From clp2 at rebertia.com Mon Jan 4 12:47:06 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Mon, 4 Jan 2010 09:47:06 -0800 Subject: How to validate the __init__ parameters In-Reply-To: References: <803c4db7-3813-448a-b428-0ec11c57f098@t42g2000yqd.googlegroups.com> <00b98b9d$0$15623$c3e8da3@news.astraweb.com> Message-ID: <50697b2c1001040947w342466d6rdc0d1c72c22e7919@mail.gmail.com> On Mon, Jan 4, 2010 at 10:17 AM, Albert van der Horst wrote: > This triggers a question: I can see the traceback, but it > would be much more valuable, if I could see the arguments > passed to the functions. Is there a tool? print(locals()) #this actually gives the bindings for the entire local namespace #but is less work than writing something more precise by hand. It is amazing how handy the humble print() function/statement is when debugging. Cheers, Chris -- http://blog.rebertia.com From python at mrabarnett.plus.com Mon Jan 4 12:55:09 2010 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 04 Jan 2010 17:55:09 +0000 Subject: Can't Add Variable In-Reply-To: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> References: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> Message-ID: <4B422B7D.7010001@mrabarnett.plus.com> Victor Subervi wrote: > Hi; > I have this code snippet: > > sql '''create table if not exists %sCustomerData ( > ID tinyint(8) unsigned primary key auto_increment, > Email varchar(120) not null, > PhoneNumber varchar(20) not null, > BillingName varchar(80) not null, > BillingAddress1 varchar(100) not null, > BillingAddress2 varchar(100) null, > BillingCity varchar(50) not null, > BillingState varchar(2) not null, > BillingPostalCode varchar(8) not null, > ShippingName varchar(80) not null, > ShippingAddress1 varchar(100) not null, > ShippingAddress2 varchar(100) null, > ShippingCity varchar(50) not null, > ShippingState varchar(2) not null, > ShippingPostalCode varchar(8) not null, > TypeOfCard set("VISA", "MasterCard", "AmEx", "Discover", "PayPal") > null, > CreditCardNumber tinyint(8) unsigned null, > CreditCardExpMonth tinyint(2) unsigned null, > CreditCardExpYear tinyint(4) unsigned null > );''' % store > > It throws this error: > > [Mon Jan 04 09:23:54 2010] [error] [client 66.82.9.61] File > "/var/www/html/angrynates.com/cart/cart2.py > ", line 62, referer: > http://angrynates.com/cart/cart.py > [Mon Jan 04 09:23:54 2010] [error] [client 66.82.9.61] );''' % > store, referer: http://angrynates.com/cart/cart.py > [Mon Jan 04 09:23:54 2010] [error] [client 66.82.9.61] ^, > referer: http://angrynates.com/cart/cart.py > [Mon Jan 04 09:23:54 2010] [error] [client 66.82.9.61] SyntaxError: > invalid syntax, referer: http://angrynates.com/cart/cart.py > [Mon Jan 04 09:23:54 2010] [error] [client 66.82.9.61] Premature end of > script headers: cart2.py, referer: http://angrynates.com/cart/cart.py > > Variable "store" is defined. How is my syntax invalid? Total code follows. > If it was complaining that "store" wasn't defined, the exception would have been NameError. It's a SyntaxError because of the missing "=". From george.trojan at noaa.gov Mon Jan 4 12:58:31 2010 From: george.trojan at noaa.gov (George Trojan) Date: Mon, 04 Jan 2010 17:58:31 +0000 Subject: asyncore based port splitter code questions Message-ID: The following code is a attempt at port splitter: I want to forward data coming on tcp connection to several host/port addresses. It sort of works, but I am not happy with it. asyncore based code is supposed to be simple, but I need while loops and a lot of try/except clauses. Also, I had to add suspend/activate_channel methods in the Writer class that use variables with leading underscores. Otherwise the handle_write() method is called in a tight loop. I designed the code by looking at Python 2.3 source for asyncore and originally wanted to use add_channel() and del_channel() methods. However in Python 2.6 del_channel() closes the socket in addition to deleting it from the map. I do not want to have one connection per message, the traffic may be high and there are no message delimiters. The purpose of this exercise is to split incoming operational data so I can test a new version of software. Comments please - I have cognitive dissonance about the code, my little yellow rubber duck is of no help here. The code is run as: python2.6 afwdport.py 50002 localhost 50003 catbert 50001 where 50002 is the localhost incoming data port, (localhost, 50003) and (catbert, 50001) are destinations. George import asyncore, os, socket, sys, time TMOUT = 10 #---------------------------------------------------------------------- def log_msg(msg): print >> sys.stderr, '%s: %s' % (time.ctime(), msg) #---------------------------------------------------------------------- class Reader(asyncore.dispatcher): def __init__(self, sock, writers): asyncore.dispatcher.__init__(self, sock) self.writers = writers def handle_read(self): data = self.recv(1024) for writer in self.writers: writer.add_data(data) def handle_expt(self): self.handle_close() def handle_close(self): log_msg('closing reader connection') self.close() def writable(self): return False #---------------------------------------------------------------------- class Writer(asyncore.dispatcher): def __init__(self, address): asyncore.dispatcher.__init__(self) self.address = address self.data = '' self.mksocket() def suspend_channel(self, map=None): fd = self._fileno if map is None: map = self._map if fd in map: del map[fd] def activate_channel(self): if self._fileno not in self._map: self._map[self._fileno] = self def mksocket(self): self.create_socket(socket.AF_INET, socket.SOCK_STREAM) self.set_reuse_addr() self.connect(self.address) log_msg('connected to %s' % str(self.address)) def add_data(self, data): self.data += data self.activate_channel() def handle_write(self): while self.data: log_msg('sending data to %s' % str(self.address)) sent = self.send(self.data) self.data = self.data[sent:] self.suspend_channel() def handle_expt(self): err = self.socket.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR) log_msg(asyncore._strerror(err)) self.handle_close() def handle_close(self): log_msg('closing writer connection') self.close() # try to reconnect time.sleep(TMOUT) self.mksocket() def readable(self): return False #---------------------------------------------------------------------- class Dispatcher(asyncore.dispatcher): def __init__(self, port, destinations): asyncore.dispatcher.__init__(self) self.address = socket.gethostbyname(socket.gethostname()), port self.writers = [Writer(_) for _ in destinations] self.reader = None self.handle_connect() def handle_connect(self): self.create_socket(socket.AF_INET, socket.SOCK_STREAM) self.bind(self.address) self.listen(1) log_msg('listening on %s' % str(self.address)) def handle_accept(self): conn, addr = self.accept() log_msg('connection from %s' % str(addr)) # current read connection not closed for some reason if self.reader: self.reader.close() self.reader = Reader(conn, self.writers) def cleanup(self): try: if self.reader: self.reader.close() except socket.error, e: log_msg('error closing reader connection %s' % e) # writer might have unwatched connections for w in self.writers: try: w.close() except socket.error, e: log_msg('error closing writer connection %s' % e) #---------------------------------------------------------------------- def main(port, destinations): disp = None try: # asyncore.loop() exits when input connection closes while True: try: disp = Dispatcher(port, destinations) asyncore.loop(timeout=TMOUT, use_poll=True) except socket.error, (errno, e): if errno == 98: log_msg('sleeping %d s: %s', (30, e)) time.sleep(30) except BaseException, e: log_msg('terminating - uncaught exception: %s' % e) raise SystemExit finally: if disp: disp.cleanup() #---------------------------------------------------------------------- if __name__ == '__main__': nargs = len(sys.argv) try: assert nargs > 3 and nargs % 2 == 0 port = int(sys.argv[1]) destinations = [(sys.argv[n], int(sys.argv[n+1])) \ for n in range(2, nargs-1, 2)] main(port, destinations) except (AssertionError, ValueError), e: print 'Error: %s' % e print 'Usage: python %s local-port host port ...' % sys.argv[0] raise SystemExit(1) From victorsubervi at gmail.com Mon Jan 4 13:00:48 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Mon, 4 Jan 2010 14:00:48 -0400 Subject: Can't Add Variable In-Reply-To: <50697b2c1001040942y4f20e7f8xac6a29e195d535c@mail.gmail.com> References: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> <50697b2c1001040942y4f20e7f8xac6a29e195d535c@mail.gmail.com> Message-ID: <4dc0cfea1001041000rdf417a1g22d12917196a2192@mail.gmail.com> On Mon, Jan 4, 2010 at 1:42 PM, Chris Rebert wrote: > On Mon, Jan 4, 2010 at 9:28 AM, Victor Subervi > wrote: > > Hi; > > I have this code snippet: > > > > sql '''create table if not exists %sCustomerData ( > > You're missing an equal sign there to start with (i.e. sql = '''). > LOL. I think it's time to put a gun to my head ;/ Thanks. beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From briandenzer at gmail.com Mon Jan 4 13:09:16 2010 From: briandenzer at gmail.com (Brian D) Date: Mon, 4 Jan 2010 10:09:16 -0800 (PST) Subject: fsync() doesn't work as advertised? References: Message-ID: On Jan 4, 10:29?am, Antoine Pitrou wrote: > Le Mon, 04 Jan 2010 08:09:56 -0800, Brian D a ?crit?: > > > > > What I've seen is that flush() alone produces a complete log when the > > loop finishes. When I used fsync(), I lost all of the write entries > > except the first, along with odd error trap and the last entry. > > Perhaps you are writing to the file from several threads or processes at > once? > > By the way, you shouldn't need fsync() if you merely want to look at the > log files, because your OS will have an up-to-date view of the file > contents anyway. fsync() is useful if you want to be sure the data has > been written to the hard disk drive, rather than just kept in the > operating system's filesystem cache. Sure -- I hadn't considered how threads might affect the write process. That's a good lead to perhaps fixing the problem. Thanks for your help, Antoine. From albert at spenarnc.xs4all.nl Mon Jan 4 13:17:04 2010 From: albert at spenarnc.xs4all.nl (Albert van der Horst) Date: 04 Jan 2010 18:17:04 GMT Subject: How to validate the __init__ parameters References: <803c4db7-3813-448a-b428-0ec11c57f098@t42g2000yqd.googlegroups.com> <00b98b9d$0$15623$c3e8da3@news.astraweb.com> Message-ID: In article , Steve Holden wrote: > >What's the exact reason for requiring that a creator argument be of a >specific type? So operations on the instances don't go wrong? Well, why >not just admit that we don't have control over everything, and just *let >things go wrong* when the wrong type is passed? > >What will then happen? Why, an exception will be raised! > >You might argue that the user, seeing the traceback from the exception, >won't know what to make of it. In response to which I would assert (pun >intended) that neither would the traceback produced by the assertion >failure. > >Given that somebody who knows what they are talking about has to cast >their eye over a traceback to understand what's wrong with the program, >it's not much extra effort for that person realise that a method has >been called with an invalid argument type. I accept that there are >legitimate arguments in opposition to this point of view, but it has a >certain engineering efficiency (as long as we are prepared to work >inside frameworks like Django that will encapsulate any error tracebacks >so that you see all the information that you need, and the user gets >some other inexplicable error message like "A programmer probably >screwed up). I can agree to a large extent with this reasoning. However, let us consider an example. I just solved the euler problem 269, something with cutting a piece of paper at exact boundaries. At some point I have reasoned that at least one of 4 arguments must be greater than one, such that an infinite recursion is prevented. Now I greatly prefer to have an assert there and then, instead of the infinite recursion, if my reasoning is wrong. You know it is not the only point where I did a similar assumption, and the problem was complicated enough as it is (for me). This triggers a question: I can see the traceback, but it would be much more valuable, if I could see the arguments passed to the functions. Is there a tool? > >In other words, "be prepared to fix your program when it goes wrong" is >a valid alternative to trying to anticipate every single last thing that >might go wrong. This philosophy might not be appropriate for >extra-terrestrial exploration, but most Python programmers aren't doing >that. Funny that you make this distinction. There is less of a distinction than you may think. Even while in embedded programming we anticipate as much as possible. there is also the watch dog timer, that resets the whole system for things unforeseen, hardware failures included. You bet space crafts have those on board. > >regards > Steve Groetjes Albert -- -- Albert van der Horst, UTRECHT,THE NETHERLANDS Economic growth -- being exponential -- ultimately falters. albert at spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst From phlip2005 at gmail.com Mon Jan 4 13:53:43 2010 From: phlip2005 at gmail.com (Phlip) Date: Mon, 04 Jan 2010 10:53:43 -0800 Subject: How to validate the __init__ parameters In-Reply-To: References: <803c4db7-3813-448a-b428-0ec11c57f098@t42g2000yqd.googlegroups.com> <00b98b9d$0$15623$c3e8da3@news.astraweb.com> Message-ID: Steve Holden wrote: > What's the exact reason for requiring that a creator argument be of a > specific type? So operations on the instances don't go wrong? Well, why > not just admit that we don't have control over everything, and just *let > things go wrong* when the wrong type is passed? Because some interfaces are "public", meaning visible within their own little arena of modules, and some interfaces are "published", meaning visible to other packages. Encapsulation is hierarchical - this is just a "small things go inside big things" situation. The higher an interface's level, the more "published" it is. Published interfaces should fail as early and clearly as possible if they are going to fail. The obvious example here is an interface that fails and prints a message telling a newbie how to call the package the right way. But that newbie could be you! (And don't forget the wall-to-wall unit tests, too;) -- Phlip http://c2.com/cgi/wiki?MoreliaViridis From gagsl-py2 at yahoo.com.ar Mon Jan 4 14:19:21 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 04 Jan 2010 16:19:21 -0300 Subject: No module named _socket, on windows References: Message-ID: En Mon, 04 Jan 2010 14:24:22 -0300, louisJ escribi?: > I installed python 2.6 (from python.org) for windows XP, and then > Pylab. > When I type "import pylab" in a python shell it shows the error: > > ImportError: No module named _socket Open the Python command line, type the following lines, and tell us what happens: Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import socket >>> socket._socket -- Gabriel Genellina From gagsl-py2 at yahoo.com.ar Mon Jan 4 14:51:53 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 04 Jan 2010 16:51:53 -0300 Subject: How to validate the __init__ parameters References: <803c4db7-3813-448a-b428-0ec11c57f098@t42g2000yqd.googlegroups.com> <00b98b9d$0$15623$c3e8da3@news.astraweb.com> Message-ID: En Mon, 04 Jan 2010 15:17:04 -0300, Albert van der Horst escribi?: > This triggers a question: I can see the traceback, but it > would be much more valuable, if I could see the arguments > passed to the functions. Is there a tool? Yes, the cgitb module [1]. Despite its name it's a general purpose module. This code: import cgitb cgitb.enable(format="text") spam = [] def a(x, y): """This is function a""" z = x+y return b(z) def b(z, n=3): """This is function b""" return c(foo=z*n) def c(foo=0, bar=1): """This is function c""" baz = foo+bar spam.somenamethatdoesnotexist(foo+bar) a(10, 20) generates this error message: A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurr ed. d:\temp\test_traceback.py in () 19 baz = foo+bar 20 spam.somenamethatdoesnotexist(foo+bar) 21 22 23 a(10, 20) a = d:\temp\test_traceback.py in a(x=10, y=20) 7 """This is function a""" 8 z = x+y 9 return b(z) 10 11 global b = z = 30 d:\temp\test_traceback.py in b(z=30, n=3) 12 def b(z, n=3): 13 """This is function b""" 14 return c(foo=z*n) 15 16 global c = foo undefined z = 30 n = 3 d:\temp\test_traceback.py in c(foo=90, bar=1) 18 """This is function c""" 19 baz = foo+bar 20 spam.somenamethatdoesnotexist(foo+bar) 21 22 global spam = [] spam.somenamethatdoesnotexist undefined foo = 90 bar = 1 : 'list' object has no attribu te 'somenamethatdoesnotexist' __class__ = __dict__ = {} __doc__ = 'Attribute not found.' ... more exception attributes ... The above is a description of an error in a Python program. Her e is the original traceback: Traceback (most recent call last): File "d:\temp\test_traceback.py", line 23, in a(10, 20) File "d:\temp\test_traceback.py", line 9, in a return b(z) File "d:\temp\test_traceback.py", line 14, in b return c(foo=z*n) File "d:\temp\test_traceback.py", line 20, in c spam.somenamethatdoesnotexist(foo+bar) AttributeError: 'list' object has no attribute 'somenamethatdoes notexist' [1] http://docs.python.org/library/cgitb.html -- Gabriel Genellina From phlip2005 at gmail.com Mon Jan 4 15:01:46 2010 From: phlip2005 at gmail.com (Phlip) Date: Mon, 4 Jan 2010 12:01:46 -0800 (PST) Subject: Where's a DOM builder that uses the Builder Pattern to ... build DOMs? Message-ID: <04523152-a0ff-432c-a48f-2c653577e7b7@m25g2000yqc.googlegroups.com> Not Hyp: I hope I'm wrong, but seems that DOMBuilder, found among the various xml.dom packages, cannot build DOM like this: var html = DomBuilder.apply(); var form = html.FORM( html.DIV( html.INPUT({type : 'text', name : 'email'}), html.INPUT({type : 'text', name : 'password'}), html.INPUT({type : 'submit'}), ) ); Do anyone know any good DOM builder packages that do build DOM good like a DOM builder should? -- Phlip http://zeekland.zeroplayer.com/Uncle_Wiggilys_Travels/1 From nad at acm.org Mon Jan 4 15:25:01 2010 From: nad at acm.org (Ned Deily) Date: Mon, 04 Jan 2010 12:25:01 -0800 Subject: How do you configure IDLE on a Mac, seeing as there's no Configure Option... References: Message-ID: In article , Benjamin Kaplan wrote: > On Sun, Jan 3, 2010 at 8:42 PM, Mensanator wrote: > > ...because there's no [Options] menu on the shell window? > > > > Or at least give me a clue to how to use Courier New font? > > > > For some inscrutable reason, depite the plethora of formatting tools, > > someone decided that proportional spaced fonts ought to be the > > default for IDLE. > Mac Applications don't have "options" menus. They have a "preferences" > pane which is the second item in the application menu (the one that > takes the name of the active program) and has a keyboard shortcut of > Cmd-, > > Apple's user-interface guidelines are pretty specific, so you'll find > that this works for all native OS X (i.e. not X11) applications. But be aware that IDLE on OS X has had various problems with missing and/or duplicate menu items. IIRC, the current python.org OS X IDLEs (2.6.4 and 3.1.1) should be OK and you can modify the font via the Preferences menu (and currently changed preferences apply to all versions of IDLE). Note also that there are two different ways to launch the OS X IDLE. python.org installers put a double-clickable IDLE.app in /Applications/Python n.x. There is also a command-line "idlen.n" in /Library/Frameworks/Python.framework/Versions/n.n/bin which, optionally, has a symlink from /usr/local/bin. -- Ned Deily, nad at acm.org From nad at acm.org Mon Jan 4 15:31:10 2010 From: nad at acm.org (Ned Deily) Date: Mon, 04 Jan 2010 12:31:10 -0800 Subject: Best practices for simultaneously installed versioned packages? References: <4B41CC6C.3020808@hastings.org> Message-ID: In article , Benjamin Kaplan wrote: > On Mon, Jan 4, 2010 at 6:09 AM, Larry Hastings wrote: > > I'm writing a package for Python 3--let's call it "spacegoblin". I fear > > someday I may need multiple versions installed and available simultaneously, > > even within one version of Python. So I want to plan ahead for that > > possibility. What would be the best way to allow this? Right now I install > > the package with the version in the name, like "spacegoblin_1_0" and > > "spacegoblin_1_1". But I have an uneasy sense that I'm doing something > > stupid... mainly because I've never seen anyone do this before. > > > > A coworker said they did it like "spacegoblin.1_0" and "spacegoblin.1_1" at > > a previous employer. That seems like an improvement, though not the "yes > > that's obviously right" answer I'm holding out for. Your thoughts?> > wxpython installs a "wxversion" module which has functions like > getInstalled(), ensureMinimal(version), and select(version). You can > call wxversion.select before importing wx and it will make sure that > the correct version is imported. You might want to look up what they > did. Also, setuptools (and, its successor, distribute, which supports Python 3) allow the installation and management of multiple versions of a package within one python site-library instance. -- Ned Deily, nad at acm.org From victorsubervi at gmail.com Mon Jan 4 16:00:12 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Mon, 4 Jan 2010 17:00:12 -0400 Subject: Premature End Of Script Headers Message-ID: <4dc0cfea1001041300g57801a2fp1945a5ad628eea0f@mail.gmail.com> Hi; Here's my entire test script: #!/usr/bin/python def myMail(): print 'Content-type: text/html' print print ''' ''' Here's the error: [Mon Jan 04 12:59:15 2010] [error] [client 66.82.9.61] Premature end of script headers: mail.py, referer: http://angrynates.com/cart/cart2.py [root at 13gems simplemail]# ls -al total 60 drwxr-xr-x 2 beno beno 4096 Jan 4 12:58 . drwxrwxrwx 6 beno beno 4096 Jan 4 12:53 .. -rwxr-xr-x 1 beno beno 252 Jan 4 06:18 mailer_test.py -rwxr-xr-x 1 beno beno 807 Jan 4 06:18 mail_old.py -rwxr-xr-x 1 beno beno 2988 Jan 4 12:58 mail.py -rwxr-xr-x 1 beno beno 17918 Jan 4 06:18 simplemail.py -rwxr-xr-x 1 beno beno 16230 Jan 4 06:18 template.py -rwxr-xr-x 1 beno beno 263 Jan 4 06:18 testmail.py What gives? TIA, beno -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben+python at benfinney.id.au Mon Jan 4 16:06:08 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 05 Jan 2010 08:06:08 +1100 Subject: twenty years ago Guido created Python References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> <87oclaxksq.fsf@benfinney.id.au> Message-ID: <877hrxy3zz.fsf@benfinney.id.au> Daniel Fetchinson writes: > But Ben will not get the Best Sense of Humor Award of 2010 for sure :) Sometimes the failing of humour is not with the recipient. -- \ ?Are you pondering what I'm pondering, Pinky?? ?Sure, Brain, | `\ but how are we going to find chaps our size?? ?_Pinky and The | _o__) Brain_ | Ben Finney From navjotmusic at gmail.com Mon Jan 4 16:22:20 2010 From: navjotmusic at gmail.com (Nav) Date: Mon, 4 Jan 2010 13:22:20 -0800 (PST) Subject: Newbie help- Can multiple instances with multiple names automatically created. Message-ID: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> I have a class of let's say empty bottle which can have a mix of two items. I want to create let's say 30 of these objects which will have names based on the 2 attributes (apple juice, beer, grape juice, beer, etc) that I provide from a list. All the objects are a mix of (1 of three alcohols) and (1 of 10 juices), so I don't want to go through typing in the names of all the objects (which would be totally stupid). I get problems if I try such as I can't assign to literal etc. If I make a list of names using the attributes. then equate names to objects. the list gets populated by the objects and the names disappear. I want the ability to be able to call up any object by its name and manipulate it and yet not have to assign the name manually. How can this be done? From news123 at free.fr Mon Jan 4 16:22:40 2010 From: news123 at free.fr (News123) Date: Mon, 04 Jan 2010 22:22:40 +0100 Subject: python xmlrpc client with ssl client certificates and standard modules In-Reply-To: References: <4b414e79$0$23321$426a34cc@news.free.fr> <4b41b733$0$13831$426a74cc@news.free.fr> Message-ID: <4b425c20$0$27951$426a34cc@news.free.fr> Hi Massimo, I'm still a litle confused: My setup: server host: ------------ apache, php with an xmlrpc server interface. no python installed. multiple client hosts (linux / windows only default python installed) ----------------------------------------------------------------------- an existing python script performing an xmlrpc call to the server host. The current working (without certificates) code snippet is: import xmlrpclib server_url = 'https://myserver' server = xmlrpclib.Server(server_url); result = server.myfunction(args) The whole setup is working as long as no client certificates are imposed by the server. The whole setup is not working as soon as the server is configured to accept only a given set of SSL-client certificates. My question is how to change above four line code snippet, such, that a client certificate will be sent to the xmlrpc server asuming the variables CLIENT_KEY_FILE and CLIENT_CRT_FILE are defined and pointing to the client certificate files. I hope to have more time tomorrow to check out the option, that I found and the option, that you suggest. option 1: http://www.cs.technion.ac.il/~danken/xmlrpc-ssl.html ( with non standard py module M2Crypto ) option 2: or web2py, which at first (so far no second) glance seems more to be targeted at implementing server side application with ajax or the client side. thanks again and bye N mdipierro wrote: > If it is a client problem than web2py will be on help. > > If your server is written already you may be able to use it with the > ssl cherrypy wsgi server (the one that web2py uses) and you do not > need web2py at all. > > Massimo > > On Jan 4, 3:38 am, News123 wrote: >> Thanks for your answer. >> >> I'll look at web2py. >> >> However web2py seems to address the xmlrpc server (at least in your >> example). The xmlrpc server application exists alerady and requires a >> client certificate. >> >> The client example doesn't seem to be using a certificate. >> >> So I'll be reading a little into web2py. >> >> bye >> >> N >> >> mdipierro wrote: >>> xmlrpc acts at the application layer and ssl at the transport layer so >>> they can inter operate easily as long as you do not use the >>> certificate to authenticate the client but only validate the server >>> and encrypt data (which you can also do but it is more complicated) >>> One option for you is to use web2py which include an xmlrpc server >>> that uses a wsgi ssl enabled web server. >>> Here is how: >>> 1) Install web2py >>> 2) Visithttp://127.0.0.1:8000/adminand create a new application from >>> the web based IDE >>> 3) create your web service for example, in a controller default.py >>> @service.xmlrpc >>> def add(a,b): return int(a)+int(b) >>> 4) Restart web2py with >>> python web2py.py -a ADMIN_PASSWD -c SSL_CERTIFICATE -k >>> SSL_PRIVATE_KEY -i 0.0.0.0 -p 443 >>> 5) You can now access the service from any Python program: >>> >>> import xmlrpclib >>> >>> server_url = 'https://myserver:443/yourapp/default/call/ >>> xmlrpc' >>> >>> server = xmlrpclib.Server(server_url) >>> >>> print server.add(3,4) >>> 7 >>> Hope this helps. >>> On Jan 3, 8:12 pm, News123 wrote: >>>> Hi, >>>> I was googling fot quite some time and was not really succesfull. >>>> I found one solution, which I will try soon. >>>> It ishttp://www.cs.technion.ac.il/~danken/xmlrpc-ssl.html >>>> (found inhttp://hamakor.org.il/pipermail/python-il/2008-February/000029.html) >>>> This will probably work, but it requires the module M2Crypto. >>>> In order to avoid installing M2Crypto an all hosts that want to run the >>>> script I wondered, whether there is no other solution. >>>> I can do xmlrpc over ssl WITHOUT certificates with following code: >>>> import xmlrpclib >>>> server_url = 'https://myserver' >>>> server = xmlrpclib.Server(server_url); >>>> and I can perform a https get request WITH certificates with below snippet: >>>> import httplib >>>> conn = httplib.HTTPSConnection( >>>> HOSTNAME, >>>> key_file = KEYFILE, >>>> cert_file = CERTFILE >>>> ) >>>> conn.putrequest('GET', '/') >>>> conn.endheaders() >>>> response = conn.getresponse() >>>> print response.read() >>>> I'm just lost of how to 'combine' both. >>>> Thanks in advance for any suggestions / hints >>>> N >> > From shawn at milochik.com Mon Jan 4 16:32:22 2010 From: shawn at milochik.com (Shawn Milochik) Date: Mon, 4 Jan 2010 16:32:22 -0500 Subject: Newbie help- Can multiple instances with multiple names automatically created. In-Reply-To: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> Message-ID: <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> You could put them in a dictionary with the key being the name, instead of a list. Shawn From michi at triodia.com Mon Jan 4 16:34:34 2010 From: michi at triodia.com (Michi) Date: Mon, 4 Jan 2010 13:34:34 -0800 (PST) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> Message-ID: <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> On Jan 4, 1:30?pm, Steven D'Aprano wrote: > > This is very true, but good APIs often trade-off increased usability and > reduced defect rate against machine efficiency too. In fact, I would > argue that this is a general design principle of programming languages: > since correctness and programmer productivity are almost always more > important than machine efficiency, the long-term trend across virtually > all languages is to increase correctness and productivity even if doing > so costs some extra CPU cycles. Yes, I agree with that in general. Correctness and productivity are more important, as a rule, and should be given priority. > > (For example, wrapper APIs often require additional > > memory allocations and/or data copies.) Incorrect use of exceptions also > > incurs an efficiency penalty. > > And? *Correct* use of exceptions also incur a penalty. So does the use of > functions. Does this imply that putting code in functions is a poor API? > Certainly not. It does imply that incorrect use of exceptions incurs an unnecessary performance penalty, no more, no less, just as incorrect use of wrappers incurs an unnecessary performance penalty. > But no matter how much more expensive, there will always be a cut-off > point where it is cheaper on average to suffer the cost of handling an > exception than it is to make unnecessary tests. > > In Python, for dictionary key access, that cut-off is approximately at > one failure per ten or twenty attempts. So unless you expect more than > one in ten attempts to lead to a failure, testing first is actually a > pessimation, not an optimization. What this really comes down to is how frequently or infrequently a particular condition arises before that condition should be considered an exceptional condition rather than a normal one. It also relates to how the set of conditions partitions into "normal" conditions and "abnormal" conditions. The difficulty for the API designer is to make these choices correctly. > In some, limited, cases you might be able to use the magic return value > strategy, but this invariably leads to lost programmer productivity, more > complex code, lowered readability and usability, and more defects, > because programmers will invariably neglect to test for the special value: I disagree here, to the extent that, whether something is an error or not can very much depend on the circumstances in which the API is used. The collection case is a very typical example. Whether failing to locate a value in a collection is an error very much depends on what the collection is used for. In some cases, it's a hard error (because it might, for example, imply that internal program state has been corrupted); in other cases, not finding a value is perfectly normal. For the API designer, the problem is that an API that throws an exception when it should not sucks just as much as an API that doesn't throw an exception when it should. For general-purpose APIs, such as a collection API, as the designer, I usually cannot know. As I said elsewhere in the article, general-purpose APIs should be policy-free, and special-purpose APIs should be policy-rich. As the designer, the more I know about the circumstances in which the API will be used, the more fascist I can be in the design and bolt down the API more in terms of static and run-time safety. Wanting to ignore a return value from a function is perfectly normal and legitimate in many cases. However, if a function throws instead of returning a value, ignoring that value becomes more difficult for the caller and can extract a performance penalty that may be unacceptable to the caller. The problem really is that, at the time the API is designed, there often is no way to tell whether this will actually be the case; in turn, no matter whether I choose to throw an exception or return an error code, it will be wrong for some people some of the time. > This is a classic example of premature optimization. Unless such > inefficiency can be demonstrated to actually matter, then you do nobody > any favours by preferring the API that leads to more defects on the basis > of *assumed* efficiency. I agree with the concern about premature optimisation. However, I don't agree with a blanket statement that special return values always and unconditionally lead to more defects. Returning to the .NET non- blocking I/O example, the fact that the API throws an exception when it shouldn't very much complicates the code and introduces a lot of extra control logic that is much more likely to be wrong than a simple if-then-else statement. As I said, throwing an exception when none should be thrown can be just as harmful as the opposite case. > It doesn't matter whether it is an error or not. They are called > EXCEPTIONS, not ERRORS. What matters is that it is an exceptional case. > Whether that exceptional case is an error condition or not is dependent > on the application. Exactly. To me, that implies that making something an exception that, to the caller, shouldn't be is just as inconvenient as the other way around. > > ?* Is it appropriate to force the caller to deal with the condition in > > a catch-handler? > > > ?* If the caller fails to explicitly deal with the condition, is it > > appropriate to terminate the program? > > > Only if the answer to these questions is "yes" is it appropriate to > > throw an exception. Note the third question, which is often forgotten. > > By throwing an exception, I not only force the caller to handle the > > exception with a catch-handler (as opposed to leaving the choice to the > > caller), I also force the caller to *always* handle the exception: if > > the caller wants to ignore the condition, he/she still has to write a > > catch-handler and failure to do so terminates the program. > > That's a feature of exceptions, not a problem. Yes, and didn't say that it is a problem. However, making the wrong choice for the use of the feature is a problem, just as making the wrong choice for not using the feature is. > > Apart from the potential performance penalty, throwing exceptions for > > expected outcomes is bad also because it forces a try-catch block on the > > caller. > > But it's okay to force a `if (result==MagicValue)` test instead? Yes, in some cases it is. For example: int numBytes; int fd = open(...); while ((numBytes = read(fd, ?)) > 0) { // process data... } Would you prefer to see EOF indicated by an exception rather than a zero return value? I wouldn't. > Look, the caller has to deal with exceptional cases (which may include > error conditions) one way or the other. If you don't deal with them at > all, your code will core dump, or behave incorrectly, or something. If > the caller fails to deal with the exceptional case, it is better to cause > an exception that terminates the application immediately than it is to > allow the application to generate incorrect results. I agree that failing to deal with exceptional cases causes problems. I also agree that exceptions, in general, are better than error codes because they are less likely to go unnoticed. But, as I said, it really depends on the caller whether something should be an exception or not. The core problem isn't whether exceptions are good or bad in a particular case, but that most APIs make this an either-or choice. For example, if I had an API that allowed me to choose at run time whether an exception will be thrown for a particular condition, I could adapt that API to my needs, instead of being stuck with whatever the designer came up with. There are many ways this could be done. For example, I could have a find() operation on a collection that throws if a value isn't found, and I could have findNoThrow() if I want a sentinel value returned. Or, the API could offer a callback hook that decides at run time whether to throw or not. (There are many other possible ways to do this, such as setting the behaviour at construction time, or by having different collection types with different behaviours.) The point is that a more flexible API is likely to be more useful than one that sets a single exception policy for everyone. > > As the API > > creator, if I indicate errors with exceptions, I make a policy decision > > about what is an error and what is not. It behooves me to be > > conservative in that policy: I should throw exceptions only for > > conditions that are unlikely to arise during routine and normal use of > > the API. > > But lost connections *are* routine and normal. Hopefully they are rare. In the context of my example, they are not. The range of behaviours naturally falls into these categories: * No data ready * Data ready * EOF * Socket error The first three cases are the "normal" ones; they operate on the same program state and they are completely expected: while reading a message off the wire, the program will almost certainly encounter the first two conditions and, if there is no error, it will always encounter the EOF condition. The fourth case is the unexpected one, in the sense that this case will often not arise at all. That's not to say that lost connections aren't routine; they are. But, when a connection is lost, the program has to do different things and operate on different state than when the connection stays up. This strongly suggests that the first three conditions should be dealt with by return values and/or out parameters, and the fourth condition should be dealt with as an exception. Cheers, Michi. From louisrom1 at gmail.com Mon Jan 4 16:41:20 2010 From: louisrom1 at gmail.com (louisJ) Date: Mon, 4 Jan 2010 13:41:20 -0800 (PST) Subject: No module named _socket, on windows References: Message-ID: I don't know what happened or what I did in the meantime but it works now...no more errors. Gabriel, for information now I have: >>> import socket >>> socket._socket Thank you anyway. From clp2 at rebertia.com Mon Jan 4 16:54:41 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Mon, 4 Jan 2010 13:54:41 -0800 Subject: Newbie help- Can multiple instances with multiple names automatically created. In-Reply-To: <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> Message-ID: <50697b2c1001041354x588aee56xd984740d12d2e6d5@mail.gmail.com> On Mon, Jan 4, 2010 at 1:32 PM, Shawn Milochik wrote: > You could put them in a dictionary with the key being the name, instead of a list. To illustrate that for the OP: name2drink = {} for booze in liquors: for juice in juices: name = juice +" "+booze # or however you're naming them drink = Bottle(booze, juice) name2drink[name] = drink #example use favorite = name2drink["apple wine"] favorite.rating = 9/10 Cheers, Chris -- http://blog.rebertia.com From albert at spenarnc.xs4all.nl Mon Jan 4 17:00:22 2010 From: albert at spenarnc.xs4all.nl (Albert van der Horst) Date: 04 Jan 2010 22:00:22 GMT Subject: How to validate the __init__ parameters References: <803c4db7-3813-448a-b428-0ec11c57f098@t42g2000yqd.googlegroups.com> Message-ID: In article , Chris Rebert wrote: >On Mon, Jan 4, 2010 at 10:17 AM, Albert van der Horst > wrote: > >> This triggers a question: I can see the traceback, but it >> would be much more valuable, if I could see the arguments >> passed to the functions. Is there a tool? > >print(locals()) #this actually gives the bindings for the entire local namespace >#but is less work than writing something more precise by hand. > >It is amazing how handy the humble print() function/statement is >when debugging. It is also amazing how much output is generated in this way in a typical Euler program (that require seconds, if not minutes solid computer time on Giga Hz machines). I have to routinely clean out my 30 Gbyte home directory. Not so long ago generating so much debug output wasn't an option! What I hoped for is something along the line of what you can do with a coredump in a classical debugger. Especially if recursion runs dozens of levels deep, it is difficult to keep track. But locals() is nice function, thanks. > >Cheers, >Chris Groetjes Albert -- -- Albert van der Horst, UTRECHT,THE NETHERLANDS Economic growth -- being exponential -- ultimately falters. albert at spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst From richardbp at gmail.com Mon Jan 4 17:04:12 2010 From: richardbp at gmail.com (Richard) Date: Mon, 4 Jan 2010 14:04:12 -0800 (PST) Subject: Efficiently determine where documents differ Message-ID: Hello, I have been using the difflib library to find where 2 large HTML documents differ. The Differ().compare() method does this, but it is very slow - atleast 100x slower than the unix diff command. How can I efficiently determine where 2 documents differ in Python? (Ideally I am after the positions rather the actual text, which is what SequenceMatcher().get_opcodes() returns.) Richard From martin at v.loewis.de Mon Jan 4 17:12:43 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Mon, 04 Jan 2010 23:12:43 +0100 Subject: python xmlrpc client with ssl client certificates and standard modules In-Reply-To: <4b414e79$0$23321$426a34cc@news.free.fr> References: <4b414e79$0$23321$426a34cc@news.free.fr> Message-ID: <4B4267DB.3080409@v.loewis.de> > I can do xmlrpc over ssl WITHOUT certificates with following code: > > import xmlrpclib > server_url = 'https://myserver' > server = xmlrpclib.Server(server_url); > > > and I can perform a https get request WITH certificates with below snippet: > > import httplib > conn = httplib.HTTPSConnection( > HOSTNAME, > key_file = KEYFILE, > cert_file = CERTFILE > ) > conn.putrequest('GET', '/') > conn.endheaders() > response = conn.getresponse() > print response.read() > > > I'm just lost of how to 'combine' both. In this case, read through the source of xmlrpclib: a) SafeTransport receives x509 parameters from get_host_info b) get_host_info supports a case where host is a tuple host, x509 So, without testing: server = xmlrpclib.Server((server_url, {'key_file': KEYFILE, 'cert_file': CERTFILE})) Please do read the code before trying this out. HTH, Martin From n00m at narod.ru Mon Jan 4 17:20:24 2010 From: n00m at narod.ru (n00m) Date: Mon, 4 Jan 2010 14:20:24 -0800 (PST) Subject: twenty years ago Guido created Python References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> <87oclaxksq.fsf@benfinney.id.au> <877hrxy3zz.fsf@benfinney.id.au> Message-ID: <02d9bd25-5ae0-4de3-910d-dc36bec0fc11@m3g2000yqf.googlegroups.com> Ben, go away from here. With all your stupids sigs. Do you think are you original? You are a stupid animal. Guido, Tim Peters, Raymond Hettinger are geniuis. I don't know exactly Python mob. Maybe forgot someone. You is only a source of depspise for them You get your everymonth fucking 1000 e sitting here whom are you teaching? This crowds of animals? They and you you cant nothing From gtu2003 at alice.it Mon Jan 4 17:35:02 2010 From: gtu2003 at alice.it (wiso) Date: Mon, 04 Jan 2010 23:35:02 +0100 Subject: FileInput too slow Message-ID: <4b426d17$0$1113$4fafbaef@reader4.news.tin.it> I'm trying the fileinput module, and I like it, but I don't understand why it's so slow... look: from time import time from fileinput import FileInput file = ['r1_200907.log', 'r1_200908.log', 'r1_200909.log', 'r1_200910.log', 'r1_200911.log'] def f1(): n = 0 for f in file: print "new file: %s" % f ff = open(f) for line in ff: n += 1 ff.close() return n def f2(): f = FileInput(file) for line in f: if f.isfirstline(): print "new file: %s" % f.filename() return f.lineno() def f3(): # f2 simpler f = FileInput(file) for line in f: pass return f.lineno() t = time(); f1(); print time()-t # 1.0 t = time(); f2(); print time()-t # 7.0 !!! t = time(); f3(); print time()-t # 5.5 I'm using text files, there are 2563150 lines in total. From gagsl-py2 at yahoo.com.ar Mon Jan 4 17:46:13 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 04 Jan 2010 19:46:13 -0300 Subject: Efficiently determine where documents differ References: Message-ID: En Mon, 04 Jan 2010 19:04:12 -0300, Richard escribi?: > I have been using the difflib library to find where 2 large HTML > documents differ. The Differ().compare() method does this, but it is > very slow - atleast 100x slower than the unix diff command. Differ compares sequences of lines *and* lines as sequences of characters to provide intra-line differences. The diff command only processes lines. If you aren't interested in intra-line differences, use a SequenceMatcher instead. Or, invoke the diff command using subprocess.Popen + communicate. -- Gabriel Genellina From steve at holdenweb.com Mon Jan 4 17:48:41 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 04 Jan 2010 17:48:41 -0500 Subject: Premature End Of Script Headers In-Reply-To: <4dc0cfea1001041300g57801a2fp1945a5ad628eea0f@mail.gmail.com> References: <4dc0cfea1001041300g57801a2fp1945a5ad628eea0f@mail.gmail.com> Message-ID: Victor Subervi wrote: > Hi; > > Here's my entire test script: > > #!/usr/bin/python > > def myMail(): > print 'Content-type: text/html' > print > print ''' > > > > > > > ''' > > Here's the error: > > [Mon Jan 04 12:59:15 2010] [error] [client 66.82.9.61] Premature end of > script headers: mail.py, referer: http://angrynates.com/cart/cart2.py > > [root at 13gems simplemail]# ls -al > total 60 > drwxr-xr-x 2 beno beno 4096 Jan 4 12:58 . > drwxrwxrwx 6 beno beno 4096 Jan 4 12:53 .. > -rwxr-xr-x 1 beno beno 252 Jan 4 06:18 mailer_test.py > -rwxr-xr-x 1 beno beno 807 Jan 4 06:18 mail_old.py > -rwxr-xr-x 1 beno beno 2988 Jan 4 12:58 mail.py > -rwxr-xr-x 1 beno beno 17918 Jan 4 06:18 simplemail.py > -rwxr-xr-x 1 beno beno 16230 Jan 4 06:18 template.py > -rwxr-xr-x 1 beno beno 263 Jan 4 06:18 testmail.py > What is this directory listing supposed to tell us? > What gives? Perhaps, having defined that nice function, you shoudl call it to produce some output? The Apache server is complaining that there's nothing there! regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From zuo at chopin.edu.pl Mon Jan 4 17:56:57 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Mon, 04 Jan 2010 23:56:57 +0100 Subject: Newbie help- Can multiple instances with multiple names automatically created. In-Reply-To: <50697b2c1001041354x588aee56xd984740d12d2e6d5@mail.gmail.com> References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <50697b2c1001041354x588aee56xd984740d12d2e6d5@mail.gmail.com> Message-ID: 2010-01-04, 22:54:41 Chris Rebert wrote: > name2drink = {} > for booze in liquors: > for juice in juices: > name = juice +" "+booze # or however you're naming them > drink = Bottle(booze, juice) > name2drink[name] = drink @Nav: ...and if you really desire to have those objects in global (module's) namespace, you can do that: global_namespace = globals() for booze in liquors: for juice in juices: name = '{0}_with_{1}_juice'.format(booze, juice) drink = Bottle(booze, juice) global_namespace[name] = drink # then you have them in module's namespace print(wine_with_apple) wine_with_apple.rating = 0.9 Though in most cases it'd be not necessary. Cheers, *j PS. Another way to express practically the same: from itertools import product globals().update(('{0}_with_{1}_juice'.format(booze, juice), Bottle(booze, juice)) for booze, juice in product(liquors, juices)) -- Jan Kaliszewski (zuo) From navjotmusic at gmail.com Mon Jan 4 17:59:28 2010 From: navjotmusic at gmail.com (Nav) Date: Mon, 4 Jan 2010 14:59:28 -0800 (PST) Subject: Newbie help- Can multiple instances with multiple names automatically created. References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> Message-ID: <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> On Jan 4, 4:54?pm, Chris Rebert wrote: > On Mon, Jan 4, 2010 at 1:32 PM, Shawn Milochik wrote: > > You could put them in a dictionary with the key being the name, instead of a list. > > To illustrate that for the OP: > > name2drink = {} > for booze in liquors: > ? ? for juice in juices: > ? ? ? ? name = juice +" "+booze # or however you're naming them > ? ? ? ? drink = Bottle(booze, juice) > ? ? ? ? name2drink[name] = drink > > #example use > favorite = name2drink["apple wine"] > favorite.rating = 9/10 typing favorite = such and such is what I am trying to avoid. I want to be able to use the name 'apple_wine' as the variable which has the object apple wine but not have to do this manually as you did with favorite. From shawn at milochik.com Mon Jan 4 18:06:57 2010 From: shawn at milochik.com (Shawn Milochik) Date: Mon, 4 Jan 2010 18:06:57 -0500 Subject: Newbie help- Can multiple instances with multiple names automatically created. In-Reply-To: <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> Message-ID: <7545C537-E886-4254-8770-C636717703FB@milochik.com> On Jan 4, 2010, at 5:59 PM, Nav wrote: > On Jan 4, 4:54 pm, Chris Rebert wrote: >> On Mon, Jan 4, 2010 at 1:32 PM, Shawn Milochik wrote: >>> You could put them in a dictionary with the key being the name, instead of a list. >> >> To illustrate that for the OP: >> >> name2drink = {} >> for booze in liquors: >> for juice in juices: >> name = juice +" "+booze # or however you're naming them >> drink = Bottle(booze, juice) >> name2drink[name] = drink >> >> #example use >> favorite = name2drink["apple wine"] >> favorite.rating = 9/10 > > typing > favorite = such and such is what I am trying to avoid. > > I want to be able to use the name 'apple_wine' as the variable which > has the object apple wine but not have to do this manually as you did > with favorite. > > > > > Then don't assign a variable to favorite, and just use name2drink["apple wine"]. From steve at holdenweb.com Mon Jan 4 18:12:03 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 04 Jan 2010 18:12:03 -0500 Subject: Newbie help- Can multiple instances with multiple names automatically created. In-Reply-To: <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> Message-ID: Nav wrote: > On Jan 4, 4:54 pm, Chris Rebert wrote: >> On Mon, Jan 4, 2010 at 1:32 PM, Shawn Milochik wrote: >>> You could put them in a dictionary with the key being the name, instead of a list. >> To illustrate that for the OP: >> >> name2drink = {} >> for booze in liquors: >> for juice in juices: >> name = juice +" "+booze # or however you're naming them >> drink = Bottle(booze, juice) >> name2drink[name] = drink >> >> #example use >> favorite = name2drink["apple wine"] >> favorite.rating = 9/10 > > typing > favorite = such and such is what I am trying to avoid. > > I want to be able to use the name 'apple_wine' as the variable which > has the object apple wine but not have to do this manually as you did > with favorite. > Why? The example is trying to show you that it's much more sensible (i.e. better controlled, easier to manage, less likely to cause problems) if instead of looking up the names in the global namespace you instead looked them up in a dictionary. This question arises so frequently it should really be a FAQ, but the closest I can come is http://techblog.ironfroggy.com/2007/06/dynamic-hell.html which does at least exercise the necessary arguments. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From python at mrabarnett.plus.com Mon Jan 4 18:18:22 2010 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 04 Jan 2010 23:18:22 +0000 Subject: Exception as the primary error handling mechanism? In-Reply-To: <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> Message-ID: <4B42773E.7050806@mrabarnett.plus.com> Michi wrote: > On Jan 4, 1:30 pm, Steven D'Aprano > wrote: [snip] >>> * Is it appropriate to force the caller to deal with the condition in >>> a catch-handler? >>> * If the caller fails to explicitly deal with the condition, is it >>> appropriate to terminate the program? >>> Only if the answer to these questions is "yes" is it appropriate to >>> throw an exception. Note the third question, which is often forgotten. >>> By throwing an exception, I not only force the caller to handle the >>> exception with a catch-handler (as opposed to leaving the choice to the >>> caller), I also force the caller to *always* handle the exception: if >>> the caller wants to ignore the condition, he/she still has to write a >>> catch-handler and failure to do so terminates the program. >> That's a feature of exceptions, not a problem. > > Yes, and didn't say that it is a problem. However, making the wrong > choice for the use of the feature is a problem, just as making the > wrong choice for not using the feature is. > >>> Apart from the potential performance penalty, throwing exceptions for >>> expected outcomes is bad also because it forces a try-catch block on the >>> caller. >> But it's okay to force a `if (result==MagicValue)` test instead? > > Yes, in some cases it is. For example: > > int numBytes; > int fd = open(...); > while ((numBytes = read(fd, ?)) > 0) > { > // process data... > } > > Would you prefer to see EOF indicated by an exception rather than a > zero return value? I wouldn't. > I wouldn't consider zero to be a magic value in this case. Returning a negative number if an error occurred would be magic. A better comparison might be str.find vs str.index, the former returning a magic value -1. Which is used more often? >> Look, the caller has to deal with exceptional cases (which may include >> error conditions) one way or the other. If you don't deal with them at >> all, your code will core dump, or behave incorrectly, or something. If >> the caller fails to deal with the exceptional case, it is better to cause >> an exception that terminates the application immediately than it is to >> allow the application to generate incorrect results. > > I agree that failing to deal with exceptional cases causes problems. I > also agree that exceptions, in general, are better than error codes > because they are less likely to go unnoticed. But, as I said, it > really depends on the caller whether something should be an exception > or not. > > The core problem isn't whether exceptions are good or bad in a > particular case, but that most APIs make this an either-or choice. For > example, if I had an API that allowed me to choose at run time whether > an exception will be thrown for a particular condition, I could adapt > that API to my needs, instead of being stuck with whatever the > designer came up with. > > There are many ways this could be done. For example, I could have a > find() operation on a collection that throws if a value isn't found, > and I could have findNoThrow() if I want a sentinel value returned. > Or, the API could offer a callback hook that decides at run time > whether to throw or not. (There are many other possible ways to do > this, such as setting the behaviour at construction time, or by having > different collection types with different behaviours.) > Or find() could have an extra keyword argument, eg. string.find(substring, default=-1), although that should probably be string.index(substring, default=-1) as a replacement for string.find(substring). From navjotmusic at gmail.com Mon Jan 4 18:21:15 2010 From: navjotmusic at gmail.com (Nav) Date: Mon, 4 Jan 2010 15:21:15 -0800 (PST) Subject: Newbie help- Can multiple instances with multiple names automatically created. References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <50697b2c1001041354x588aee56xd984740d12d2e6d5@mail.gmail.com> Message-ID: Thanks Jan, You read my mind. That is exactly what I needed. Thanks for showing the product function from itertools as well. It seems easier to grasp than the nested loops, I had been using. I noticed chopin.edu.pl. Are you a musician? Nav From navjotmusic at gmail.com Mon Jan 4 18:33:25 2010 From: navjotmusic at gmail.com (Nav) Date: Mon, 4 Jan 2010 15:33:25 -0800 (PST) Subject: Newbie help- Can multiple instances with multiple names automatically created. References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> Message-ID: Thanks for pointing it out Steve. The blog post doesn't explain it very well. I understand the risk of exec or eval(input). but what are the risks of globalnamespace use and what are the benefits? From deets at nospam.web.de Mon Jan 4 18:35:28 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Tue, 05 Jan 2010 00:35:28 +0100 Subject: Safe file I/O to shared file (or SQLite) from multi-threaded web server In-Reply-To: References: <4b403899$0$1646$742ec2ed@news.sonic.net> Message-ID: <7qfca0FncdU1@mid.uni-berlin.de> Lawrence D'Oliveiro schrieb: > In message , Steve > Holden wrote: > >> Yes, but not to MySQL, please. Particularly since there is a sword of >> Damocles hanging over its head while the Oracle takeover of Sun is >> pending. > > Ah, I see the FUDsters are crawling out of the woodwork here, as well. I?ve > got news for you: MySQL is an open-source product. And you can?t kill Open > Source. So go crawling back to your proprietary world, if that?s the only > world you understand. Since when is suggesting Postgres a sign of a proprietary world crawler? And while I don't wish MySQL anything bad - open source *can* die, and will, if leadership changes for the worst - which can happen. And is certainly an immediate threat here. Diez From mensanator at aol.com Mon Jan 4 18:37:29 2010 From: mensanator at aol.com (Mensanator) Date: Mon, 4 Jan 2010 15:37:29 -0800 (PST) Subject: twenty years ago Guido created Python References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> <87oclaxksq.fsf@benfinney.id.au> <877hrxy3zz.fsf@benfinney.id.au> <02d9bd25-5ae0-4de3-910d-dc36bec0fc11@m3g2000yqf.googlegroups.com> Message-ID: On Jan 4, 4:20?pm, n00m wrote: > Ben, go away from here. With all your stupids sigs. > Do you think are you original? > You are a stupid animal. > Guido, Tim Peters, Raymond Hettinger are geniuis. > I don't know exactly Python mob. Maybe forgot someone. > You is only a source of depspise for them > You get your everymonth fucking 1000 e sitting here > whom are you teaching? This crowds of animals? > They and you you cant nothing Rants are so much funnier when the speaker can't speak English. From steve at REMOVE-THIS-cybersource.com.au Mon Jan 4 18:44:09 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 04 Jan 2010 23:44:09 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> Message-ID: <03526a3a$0$1336$c3e8da3@news.astraweb.com> On Mon, 04 Jan 2010 13:34:34 -0800, Michi wrote: > On Jan 4, 1:30?pm, Steven D'Aprano > wrote: >> >> This is very true, but good APIs often trade-off increased usability >> and reduced defect rate against machine efficiency too. In fact, I >> would argue that this is a general design principle of programming >> languages: since correctness and programmer productivity are almost >> always more important than machine efficiency, the long-term trend >> across virtually all languages is to increase correctness and >> productivity even if doing so costs some extra CPU cycles. > > Yes, I agree with that in general. Correctness and productivity are more > important, as a rule, and should be given priority. I'm glad we agree on that, but I wonder why you previously emphasised machine efficiency so much, and correctness almost not at all, in your previous post? >> > (For example, wrapper APIs often require additional memory >> > allocations and/or data copies.) Incorrect use of exceptions also >> > incurs an efficiency penalty. >> >> And? *Correct* use of exceptions also incur a penalty. So does the use >> of functions. Does this imply that putting code in functions is a poor >> API? Certainly not. > > It does imply that incorrect use of exceptions incurs an unnecessary > performance penalty, no more, no less, just as incorrect use of wrappers > incurs an unnecessary performance penalty. If all you're argument is that we shouldn't write crappy APIs, then I agree with you completely. The .NET example you gave previously is a good example of an API that is simply poor: using exceptions isn't a panacea that magically makes code better. So I can't disagree that using exceptions badly incurs an unnecessary performance penalty, but it also incurs an unnecessary penalty against correctness and programmer productivity. >> But no matter how much more expensive, there will always be a cut-off >> point where it is cheaper on average to suffer the cost of handling an >> exception than it is to make unnecessary tests. >> >> In Python, for dictionary key access, that cut-off is approximately at >> one failure per ten or twenty attempts. So unless you expect more than >> one in ten attempts to lead to a failure, testing first is actually a >> pessimation, not an optimization. > > What this really comes down to is how frequently or infrequently a > particular condition arises before that condition should be considered > an exceptional condition rather than a normal one. It also relates to > how the set of conditions partitions into "normal" conditions and > "abnormal" conditions. The difficulty for the API designer is to make > these choices correctly. The first case is impossible for the API designer to predict, although she may be able to make some educated estimates based on experience. For instance I know that when I search a string for a substring, "on average" I expect to find the substring present more often than not. I've put "on average" in scare-quotes because it's not a statistical average at all, but a human expectation -- a prejudice in fact. I *expect* to have searching succeed more often than fail, not because I actually know how many searches succeed and fail, but because I think of searching for an item to "naturally" find the item. But if I actually profiled my code in use on real data, who knows what ratio of success/failure I would find? In the second case, the decision of what counts as "ordinary" and what counts as "exceptional" should, in general, be rather obvious. (That's not to discount the possibility of unobvious cases, but that's probably a case that the function is too complex and tries to do too much.) Take the simplest description of what the function is supposed to do: (e.g. "find the offset of a substring in a source string"). That's the ordinary case, and should be returned. Is there anything else that the function may do? (E.g. fail to find the substring because it isn't there.) Then that's an exceptional case. (There may be other exceptional cases, which is another reason to prefer exceptions to magic return values. In general, it's much easier to deal with multiple exception types than it is to test for multiple magic return values. Consider a function that returns a pointer. You can return null to indicate an error. What if you want to distinguish between two different error states? What about ten error states?) I argue that as designers, we should default to raising an exception and only choose otherwise if there is a good reason not to. As we agreed earlier, exceptions (in general) are better for correctness and productivity, which in turn are (in general) more important than machine efficiency. The implication of this is that in general, we should prefer exceptions, and only avoid them when necessary. Your argument seems to be that we should avoid exceptions by default, and only use them if unavoidable. I think that is backwards. >> In some, limited, cases you might be able to use the magic return value >> strategy, but this invariably leads to lost programmer productivity, >> more complex code, lowered readability and usability, and more defects, >> because programmers will invariably neglect to test for the special >> value: > > I disagree here, to the extent that, whether something is an error or > not can very much depend on the circumstances in which the API is used. That's certainly true: a missing key (for example) may be an error, or a present key may be an error, or neither may be an error, just different branches of an algorithm. That's an application-specific decision. But I don't see how that relates to my claim that magic return values are less robust and usable than exceptions. Whether it is an error or not, it still needs to be handled. If the caller neglects to handle the special case, an exception-based strategy will almost certainly lead to the application halting (hopefully leading to a harmless bug report rather than the crash of a billion-dollar space probe), but a magic return value will very often lead to the application silently generating invalid results. [...] > Wanting to ignore a return value from a function is perfectly normal and > legitimate in many cases. I wouldn't say that's normal. If you don't care about the function's result, why are you calling it? For the side-effects? In languages that support procedures, such mutator functions should be written as procedures that don't return anything. For languages that don't, like Python, they should be written as de-facto procedures, always return None, and allow the user to pretend that nothing was returned. That is to say, ignoring the return value is acceptable as a work-around for the lack of true procedures. But even there, procedures necessarily operate by side-effect, and side-effects should be avoided as much as possible. So I would say, ideally, wanting to ignore the return value should be exceptionally rare. > However, if a function throws instead of > returning a value, ignoring that value becomes more difficult for the > caller and can extract a performance penalty that may be unacceptable to > the caller. There's that premature micro-optimization again. > The problem really is that, at the time the API is designed, > there often is no way to tell whether this will actually be the case; in > turn, no matter whether I choose to throw an exception or return an > error code, it will be wrong for some people some of the time. I've been wondering when you would reach the conclusion that an API should offer both forms. For example, Python offers both key-lookup that raises exceptions (dict[key]) and key-lookup that doesn't (dict.get(key)). The danger of this is that it complicates the API, leads to a more complex implementation, and may result in duplicated code (if the two functions have independent implementations). But if you don't duplicate the code, then the assumed performance benefit of magic return values over exceptions might very well be completely negated: def get(self, key): # This is not the real Python dict.get implementation! # This is merely an illustration of how it *could* be. try: return self[key] except KeyError: return None This just emphasises the importance of not optimising code by assumption. If you haven't *measured* the speed of a function you don't know whether it will be faster or slower than catching an exception. You will note that the above has nothing to do with the API, but is entirely an implementation decision. This to me demonstrates that the question of machine efficiency is irrelevant to API design. >> This is a classic example of premature optimization. Unless such >> inefficiency can be demonstrated to actually matter, then you do nobody >> any favours by preferring the API that leads to more defects on the >> basis of *assumed* efficiency. > > I agree with the concern about premature optimisation. However, I don't > agree with a blanket statement that special return values always and > unconditionally lead to more defects. I can't say that they *always* lead to more defects, since that also depends on the competence of the caller, but I will say that as a general principle, they should be *expected* to lead to more defects. > Returning to the .NET non- > blocking I/O example, the fact that the API throws an exception when it > shouldn't very much complicates the code and introduces a lot of extra > control logic that is much more likely to be wrong than a simple > if-then-else statement. As I said, throwing an exception when none > should be thrown can be just as harmful as the opposite case. In this case, it's worse than that -- they use a special return value when there should be an exception, and an exception when there should be an ordinary, non-special value (an empty string, if I recall correctly). >> It doesn't matter whether it is an error or not. They are called >> EXCEPTIONS, not ERRORS. What matters is that it is an exceptional case. >> Whether that exceptional case is an error condition or not is dependent >> on the application. > > Exactly. To me, that implies that making something an exception that, to > the caller, shouldn't be is just as inconvenient as the other way > around. Well, obviously I agree that you should only make things be an exception if they actually should be an exception. I don't quite see where the implication is -- I find myself in the curious position of agreeing with your conclusion while questioning your reasoning, as if you had said something like: All cats have four legs, therefore cats are mammals. >> > Apart from the potential performance penalty, throwing exceptions for >> > expected outcomes is bad also because it forces a try-catch block on >> > the caller. >> >> But it's okay to force a `if (result==MagicValue)` test instead? > > Yes, in some cases it is. For example: > > int numBytes; > int fd = open(...); > while ((numBytes = read(fd, ?)) > 0) { > // process data... > } > > Would you prefer to see EOF indicated by an exception rather than a zero > return value? I wouldn't. Why not? Assuming this is a blocking read, once you hit EOF you will never recover from it. Is this about the micro-optimisation again? Disc IO is almost certainly a thousand times slower than any exception you could catch here. In Python, we *do* use exceptions for file reads. An explicit read returns an empty string, and we might write: f = open(filename) while 1: block = f.read(buffersize) if not block: f.close() break process(block) This would arguably be easier to write and read, and demonstrates the intent of the while loop better: f = open(filename) try: while 1: process(f.read(buffersize)) except EOFError: f.close() (But the above doesn't work, because an explicit read doesn't raise an exception.) However, there's another idiom for reading a file which does use an exception: line-by-line reading. f = open(filename) for line in f: process(line) f.close() Because iterating over the file generates a StopIteration when EOF is reached, the for loop automatically breaks. If you wanted to handle that by hand, something like this should work (but is unnecessary, because Python already does it for you): f = open(filename) try: while 1: process(f.next()) except StopIteration: f.close() [...] > The core problem isn't whether exceptions are good or bad in a > particular case, but that most APIs make this an either-or choice. For > example, if I had an API that allowed me to choose at run time whether > an exception will be thrown for a particular condition, I could adapt > that API to my needs, instead of being stuck with whatever the designer > came up with. > > There are many ways this could be done. For example, I could have a > find() operation on a collection that throws if a value isn't found, and > I could have findNoThrow() if I want a sentinel value returned. Or, the > API could offer a callback hook that decides at run time whether to > throw or not. (There are many other possible ways to do this, such as > setting the behaviour at construction time, or by having different > collection types with different behaviours.) > > The point is that a more flexible API is likely to be more useful than > one that sets a single exception policy for everyone. This has costs of its own. The costs of developer education -- learning about, memorising, and deciding between such multiple APIs does not come for free. The costs of developing and maintaining the multiple functions. The risks of duplicated code in the implementation. The cost of writing documentation. A bloated API is not free of costs. >> > As the API >> > creator, if I indicate errors with exceptions, I make a policy >> > decision about what is an error and what is not. It behooves me to be >> > conservative in that policy: I should throw exceptions only for >> > conditions that are unlikely to arise during routine and normal use >> > of the API. >> >> But lost connections *are* routine and normal. Hopefully they are rare. > > In the context of my example, they are not. The range of behaviours > naturally falls into these categories: > > * No data ready > * Data ready > * EOF > * Socket error Right -- that fourth example is one of the NATURAL categories that any half-way decent developer needs to be aware of. When you say something isn't natural, and then immediately contradict yourself, that's a sign you need to think about what you really mean :) > The first three cases are the "normal" ones; they operate on the same > program state and they are completely expected: while reading a message > off the wire, the program will almost certainly encounter the first two > conditions and, if there is no error, it will always encounter the EOF > condition. I would call these the ordinary cases, as opposed to the exceptional cases. > The fourth case is the unexpected one, in the sense that this > case will often not arise at all. But it is still expected -- you have to expect that you might get a socket error, and code accordingly. > That's not to say that lost connections aren't routine; they are. Right -- we actually agree on this, we just disagree on the terminology. I believe that talking about "normal" and "errors" is misleading. Better is to talk about "ordinary" and "exceptional". > But, when a connection is lost, > the program has to do different things and operate on different state > than when the connection stays up. This strongly suggests that the first > three conditions should be dealt with by return values and/or out > parameters, and the fourth condition should be dealt with as an > exception. Agreed. -- Steven From keesvanschaik at gmail.com Mon Jan 4 18:50:16 2010 From: keesvanschaik at gmail.com (KvS) Date: Mon, 4 Jan 2010 15:50:16 -0800 (PST) Subject: Printing plain text with exact positioning on Windows Message-ID: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> Dear all, for a python program running on Win XP that is used to produce invoices I need to be able to have the printer print a page with just plain text, but positioned (almost) exactly at prescribed places (due to the use of 'pre formatted' invoice forms). In particular as close to the left and right edges of the page as possible. So far I've been using a combination of HTML & CSS, IE automation and fiddling in the registry to set left and right printing margins of IE to 0 and back to defaults after the printing is done. This worked ok, but recently some problem occured that I wasn't able to solve so far (seemingly random crashes, probably due to an update to IE 8, I've posted about this problem before: http://groups.google.com/group/comp.lang.python/browse_thread/thread/3922fdfa4db31ffe/ba8dc24c2b85252c?lnk=gst&q=KvS#ba8dc24c2b85252c but unfortunately no answers). So now I'm looking for other ways to get this printing job done. I know of Tim Goldens page about Windows printing: http://timgolden.me.uk/python/win32_how_do_i/print.html and have been googling a lot, but I'm still not convinced what the best way to go is. E.g. I can't get clear whether the combination reportlab & Acrobat Reader allows to print very close to the left and right edge of the paper (without a user having to change something in a Print dialog etc.). You would expect this is a relatively typical problem and others have ran into it before me. Any thoughts on this? Thanks in advance, Kees From aleksandar27 at BRISIOVOnet.hr Mon Jan 4 18:58:26 2010 From: aleksandar27 at BRISIOVOnet.hr (alejandro) Date: Tue, 5 Jan 2010 00:58:26 +0100 Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> Message-ID: I think the easyest way for printing text is with ReportLab. Just few lines of code and you have a nice pdf... From keesvanschaik at gmail.com Mon Jan 4 19:15:18 2010 From: keesvanschaik at gmail.com (KvS) Date: Mon, 4 Jan 2010 16:15:18 -0800 (PST) Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> Message-ID: <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> On Jan 4, 11:58?pm, "alejandro" wrote: > I think the easyest way for printing text is with ReportLab. Just few lines > of code and you have a nice pdf... Ok, thanks. In that scenario I would also need to be able to programatically adjust the printing margins in Acrobat reader, i.e. automate it to some extent. Is that also possible? From cs at zip.com.au Mon Jan 4 19:23:34 2010 From: cs at zip.com.au (Cameron Simpson) Date: Tue, 5 Jan 2010 11:23:34 +1100 Subject: IOError - cannot create file (linux daemon-invoked script) In-Reply-To: References: Message-ID: <20100105002334.GA10849@cskk.homeip.net> On 04Jan2010 09:16, cassiope wrote: | To Cameron: the file doesn't (yet) exist; and it has the correct full | path. Can you show us the strace output of the failing open() call? | To "Nobody" : hey, this seems interesting. First test, invoking | seteuid() | and setegid() didn't help - but strange thing was these calls didn't | show | up in the strace, so perhaps I wasn't testing what I thought I was. If you're using the "-e trace=file" option it won't. That constrains the output to file operations to make the log easier to read. Discard the -e option to get everything. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Rimmer: It will be happened; it shall be going to be happening; it will be was an event that could will have been taken place in the future. - Red Dwarf, _Future Echoes_ From aioe.org at technicalbloke.com Mon Jan 4 19:52:56 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 00:52:56 +0000 Subject: TypeError: can only concatenate list (not "tuple") to list References: Message-ID: Gabriel Genellina wrote: > En Mon, 04 Jan 2010 04:58:54 -0300, Chris Rebert > escribi?: >> On Sun, Jan 3, 2010 at 11:51 PM, Gabriel Genellina >> wrote: > >>> py> [1,2,3] + (4,5) >>> Traceback (most recent call last): >>> File "", line 1, in >>> TypeError: can only concatenate list (not "tuple") to list > > Sorry, I inadvertedly posted an incomplete message. Note the last part: > >>> In-place addition += does work: >>> >>> py> a = [1,2,3] >>> py> a += (4,5) >>> py> a >>> [1, 2, 3, 4, 5] > >> Given that tuples are sometimes used as a poor man's object (i.e. >> collection of data fields), whereas lists are not typically used that >> way, I'd say it's probably a good thing an explicit type conversion is >> required here. > > In that case += should not be allowed either... > I'd be strongly inclined to think the result would be the sequence on the left with the data from the second sequence appended to it. What's wrong with a little duck typing here eh? Roger. From aleksandar27 at BRISIOVOnet.hr Mon Jan 4 20:01:33 2010 From: aleksandar27 at BRISIOVOnet.hr (alejandro) Date: Tue, 5 Jan 2010 02:01:33 +0100 Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> Message-ID: Don't remember much.. but maybe you could play with canvas? I think i don't undertand(i am a croatian so english is not my mother language). If you have the need that the pdf and the printer setup have lets say the same paper size, sorry but can't help. If you want lets say different printer margins for different documents/pages just calculate them. A4 is 210x297mm and you have need for a margin of 10mm.. width = 210 - margin height = 297 - margin now for positioning use the variables instead the absolute positioning.. i think that reportlab has some methods that alow you to position text/photos from the left top corner instead the normal positioning. Don't know if I helped you in any way... Good luck! "KvS" wrote in message news:54eb4786-5db3-453d-971a-a4b359fc2de8 at f5g2000yqh.googlegroups.com... On Jan 4, 11:58 pm, "alejandro" wrote: > I think the easyest way for printing text is with ReportLab. Just few > lines > of code and you have a nice pdf... Ok, thanks. In that scenario I would also need to be able to programatically adjust the printing margins in Acrobat reader, i.e. automate it to some extent. Is that also possible? From gagsl-py2 at yahoo.com.ar Mon Jan 4 20:01:39 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 04 Jan 2010 22:01:39 -0300 Subject: FileInput too slow References: <4b426d17$0$1113$4fafbaef@reader4.news.tin.it> Message-ID: En Mon, 04 Jan 2010 19:35:02 -0300, wiso escribi?: > I'm trying the fileinput module, and I like it, but I don't understand > why > it's so slow... look: > > from time import time > from fileinput import FileInput > > file = ['r1_200907.log', 'r1_200908.log', 'r1_200909.log', > 'r1_200910.log', > 'r1_200911.log'] > > def f1(): > n = 0 > for f in file: > print "new file: %s" % f > ff = open(f) > for line in ff: > n += 1 > ff.close() > return n > > def f2(): > f = FileInput(file) > for line in f: > if f.isfirstline(): print "new file: %s" % f.filename() > return f.lineno() > > def f3(): # f2 simpler > f = FileInput(file) > for line in f: > pass > return f.lineno() Yes, the fileinput module is A LOT slower than normal file processing. You may use itertools.chain instead: def f4(): f = itertools.chain.from_iterable(open(fn) for fn in file) n = 0 for line in f: n += 1 return n I get similar timings as f1() above. Known major issues of this "poor man's" implementation: - no lineno/filelineno/isfirstline attributes - close() is implicit - only for reading; inplace and backup don't work -- Gabriel Genellina From aleksandar27 at BRISIOVOnet.hr Mon Jan 4 20:05:22 2010 From: aleksandar27 at BRISIOVOnet.hr (alejandro) Date: Tue, 5 Jan 2010 02:05:22 +0100 Subject: mechanize module problem Message-ID: When I inport it wia python command line it all looks fine but if i try to run it from eclipse it gives me an error that there is no package mechanize. Have anybody had this situation? win XP python26 From chris.gonnerman at newcenturycomputers.net Mon Jan 4 20:15:29 2010 From: chris.gonnerman at newcenturycomputers.net (Chris Gonnerman) Date: Mon, 04 Jan 2010 19:15:29 -0600 Subject: [Python] Printing plain text with exact positioning on Windows In-Reply-To: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> Message-ID: <4B4292B1.9030602@newcenturycomputers.net> KvS wrote: > So now I'm looking for other ways to get this printing job done. I > know of Tim Goldens page about Windows printing: > http://timgolden.me.uk/python/win32_how_do_i/print.html and have been > googling a lot, but I'm still not convinced what the best way to go > is. E.g. I can't get clear whether the combination reportlab & Acrobat > Reader allows to print very close to the left and right edge of the > paper (without a user having to change something in a Print dialog > etc.). > I have a page, and a module, for Windows printing which gives a lot of control. http://newcenturycomputers.net/projects/pythonicwindowsprinting.html But in your case, I think I'd consider ReportLab. Also, it seems to me that I've seen a simpler PDF generator module for Python. Dunno where, though. -- Chris. From aleksandar27 at BRISIOVOnet.hr Mon Jan 4 20:26:15 2010 From: aleksandar27 at BRISIOVOnet.hr (alejandro) Date: Tue, 5 Jan 2010 02:26:15 +0100 Subject: mechanize module problem References: Message-ID: resolved! don't know why or how but it is... just restared eclipse for the fourth time "alejandro" wrote in message news:hhu38i$h21$1 at ss408.t-com.hr... > When I inport it wia python command line it all looks fine but if i try to > run it from eclipse it gives me an error that there is no package > mechanize. Have anybody had this situation? > win XP > python26 > > From wuwei23 at gmail.com Mon Jan 4 20:34:17 2010 From: wuwei23 at gmail.com (alex23) Date: Mon, 4 Jan 2010 17:34:17 -0800 (PST) Subject: python mechanize proxy support question References: <27009696.post@talk.nabble.com> <4ca2b83a-3b82-4d20-85a3-b6304f25fb81@21g2000yqj.googlegroups.com> Message-ID: <60cd73ea-1bb6-4fc8-9992-27115cc8ff09@j4g2000yqe.googlegroups.com> On Jan 4, 9:25?pm, elca wrote: > that is only support mechanize.browser module.. actually > im looking mechanize.urlopen method. >From the docs: In these examples, the workings are hidden inside the mechanize.urlopen () function, which is an extension of urllib2.urlopen(). Redirects, proxies and cookies are handled automatically by this function (note that you may need a bit of configuration to get your proxies correctly set up: see urllib2 documentation). http://wwwsearch.sourceforge.net/mechanize/doc.html From aahz at pythoncraft.com Mon Jan 4 20:35:24 2010 From: aahz at pythoncraft.com (Aahz) Date: 4 Jan 2010 17:35:24 -0800 Subject: Design question about pretree classifier References: <7351dfbf-90a2-4303-b3d8-2377dcd2698e@o28g2000yqh.googlegroups.com> Message-ID: In article , Steve Holden wrote: >Julian wrote: >> >> But: >> >> - none classification: return an exception or None? I think None is >> better, hence its not an exception that there is no classification but >> a defined state. What do you think? >> - many classifications: what to do? retun a sequence of strings? raise >> an exception and implement another method wich returns than the >> classifications? what should I do here? > >Always return a list or tuple. For no classifications it should be >empty, for one classification it should have one element, ... , for N >classifications it should have N elements. Why not a set? If you're only going list classifications, a tuple or list would be fine, but if you're going to probe classifications, you should use something that isn't O(N). -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From aahz at pythoncraft.com Mon Jan 4 20:36:52 2010 From: aahz at pythoncraft.com (Aahz) Date: 4 Jan 2010 17:36:52 -0800 Subject: Using PyImport_ExtendInittab with package References: Message-ID: In article , Julien Danjou wrote: > >I'm trying to embed Python and therefore use PyImport_ExtendInittab() to >register modules. >My current problem is that, if it works well with a simple module >"hello", naming a module "hello.foobar" in the inittab struct does not >seems to work. >imp.find_module("hello.foobar") returns correctly that the module is >found, but import hello.foobar fails badly. > >Is . notation supported in inittab? Am I doing something wrong? This is probably wrong in some detail, but I think you need to import each level of a package separately. I.e. you need to import 'hello' before you can import 'hello.foobar'. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From wuwei23 at gmail.com Mon Jan 4 20:41:36 2010 From: wuwei23 at gmail.com (alex23) Date: Mon, 4 Jan 2010 17:41:36 -0800 (PST) Subject: Newbie help- Can multiple instances with multiple names automatically created. References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> Message-ID: <14370388-6f6e-451c-b8a6-4ce2f6ec32fc@l30g2000yqb.googlegroups.com> On Jan 5, 9:33?am, Nav wrote: > what are the risks of globalnamespace use You're unnecessarily tying your code to the implementation. > and what are the benefits? Absolutely none that using a dictionary doesn't also give you. From keesvanschaik at gmail.com Mon Jan 4 20:50:13 2010 From: keesvanschaik at gmail.com (KvS) Date: Mon, 4 Jan 2010 17:50:13 -0800 (PST) Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> Message-ID: <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> On Jan 5, 1:01?am, "alejandro" wrote: > Don't remember much.. but maybe you could play with canvas? > I think i don't undertand(i am a croatian so english is not my mother > language). If you have the need that the pdf and the printer setup have lets > say the same paper size, sorry but can't help. > If you want lets say different printer margins for different documents/pages > just calculate them. > A4 is 210x297mm and you have need for a margin of 10mm.. > width = 210 - margin > height = 297 - margin > now for positioning use the variables instead the absolute positioning.. > > i think that reportlab has some methods that alow you to position > text/photos from the left top corner instead the normal positioning. > > Don't know if I helped you in any way... > > Good luck! > > "KvS" wrote in message > > news:54eb4786-5db3-453d-971a-a4b359fc2de8 at f5g2000yqh.googlegroups.com... > On Jan 4, 11:58 pm, "alejandro" wrote: > > > I think the easyest way for printing text is with ReportLab. Just few > > lines > > of code and you have a nice pdf... > > Ok, thanks. In that scenario I would also need to be able to > programatically adjust the printing margins in Acrobat reader, i.e. > automate it to some extent. Is that also possible? I trust I can do that in reportlab. But then if you print the pdf, you would still get some margin between the edge of the paper and the text due to the printing settings of Acrobat Reader basically. I need the text to be as close as possible to the actual edge of the paper, so I need to somehow make those margins as small as possible as well... From aioe.org at technicalbloke.com Mon Jan 4 20:58:09 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 01:58:09 +0000 Subject: Off Topic ( was Re: Significant whitespace) References: <4B3E70FC.1010702@gmail.com> Message-ID: Steve Holden wrote: > r0g wrote: >> David Robinow wrote: >>> On Sun, Jan 3, 2010 at 10:42 PM, Roy Smith wrote: >>>> In article , >>>> David Robinow wrote: >>>> >>>>> On Sun, Jan 3, 2010 at 8:09 PM, Tim Roberts wrote: >>>>>> More than "not required", it was "not relevant". This led to one of the >>>>>> most infamous programming blunders in the early days of the space program, >>>>>> when one programmer accidentially typed a period instead of a comma >>>>>> resulting in the loss of a satellite: >>>>> Interesting story. Did you make it up? >>>> It's a fairly well known story. >>>> >>>> http://catless.ncl.ac.uk/Risks/5.64.html#subj4.2 >>> Sure. But the question is, "Who made it up?" >>> http://en.wikiquote.org/wiki/Fortran >> completely appropriate and proportional to Tim's sickening crimes >> against fact. >> >> Roger. > > Personally I think the overall tone of this group would have been > marginally improved if you had summoned the self-restraint to simply not > make this posting. > > Fortunately it's still an unmoderated group, so you can continue to be > as hostile and snarky as you like. > > regards > Steve Well I could say the same thing Steve: you could also not have posted and thereby spared the group even more off topic noise, I'd have quite happily left it there. Of course our emotional urges trump rationality almost every time so I understand your urge to address what has upset you, much as I hope you understand my subsequent urge to respond. I appreciate this is a professional programming forum and that my post was crass in the extreme but I get really fed up with the needless pedantry of computer geeks sometimes; not to mention the insufferably sanctimonious tone they adopted whilst engaging in it. It ranges from semi-conscious passive aggression to out and out dick swinging and I think the "overall tone of this group" would be "marginally improved" by doing away with it. There are perfectly good way's of putting things that don't require taunting and smugness. Contrast the clearly snarky personal challenge... "Interesting story. Did you make it up?" ...with the far more convivial correction... "Interestingly it was a missing underscore character ;) see http://example.com/rahrarhrah"... Both could satisfy the inner pedant's demands for attention, why choose the first? Personally I think the first respondent should have "summoned the self restraint" to just let it drop too, but they didn't and I feel they did the OP an injustice by that. My apologies for the noise everybody, I shall say no more on the subject. Cheers, Roger. From wuwei23 at gmail.com Mon Jan 4 21:14:17 2010 From: wuwei23 at gmail.com (alex23) Date: Mon, 4 Jan 2010 18:14:17 -0800 (PST) Subject: Can't Add Variable References: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> Message-ID: <5b5fd3f1-c5b5-4b65-b1bd-f7f64a9ba079@m25g2000yqc.googlegroups.com> On Jan 5, 3:40?am, Joe Riopel wrote: > On Mon, Jan 4, 2010 at 12:28 PM, Victor Subervi wrote: > > Hi; > > I have this code snippet: > > > ??? sql '''create table if not exists %sCustomerData ( > > I think you may have forgotten the "=" after sql. Jesus wept. How long is this list going to continue to finish Victor's project for him? These questions _clearly_ belong on python-tutor. From python at mrabarnett.plus.com Mon Jan 4 21:25:55 2010 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 05 Jan 2010 02:25:55 +0000 Subject: Can't Add Variable In-Reply-To: <5b5fd3f1-c5b5-4b65-b1bd-f7f64a9ba079@m25g2000yqc.googlegroups.com> References: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> <5b5fd3f1-c5b5-4b65-b1bd-f7f64a9ba079@m25g2000yqc.googlegroups.com> Message-ID: <4B42A333.7060003@mrabarnett.plus.com> alex23 wrote: > On Jan 5, 3:40 am, Joe Riopel wrote: >> On Mon, Jan 4, 2010 at 12:28 PM, Victor Subervi wrote: >>> Hi; >>> I have this code snippet: >>> sql '''create table if not exists %sCustomerData ( >> I think you may have forgotten the "=" after sql. > > Jesus wept. How long is this list going to continue to finish Victor's > project for him? > > These questions _clearly_ belong on python-tutor. Who is this "Jesus" person? You should remember that this is a predominantly Guido-worshipping list! :-) From aioe.org at technicalbloke.com Mon Jan 4 21:31:34 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 02:31:34 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> Message-ID: Michi wrote: > On Jan 4, 1:30 pm, Steven D'Aprano > wrote: >> In some, limited, cases you might be able to use the magic return value >> strategy, but this invariably leads to lost programmer productivity, more >> complex code, lowered readability and usability, and more defects, >> because programmers will invariably neglect to test for the special value: > > I disagree here, to the extent that, whether something is an error or > not can very much depend on the circumstances in which the API is > used. The collection case is a very typical example. Whether failing > to locate a value in a collection is an error very much depends on > what the collection is used for. In some cases, it's a hard error > (because it might, for example, imply that internal program state has > been corrupted); in other cases, not finding a value is perfectly > normal. A pattern I have used a few times is that of returning an explicit success/failure code alongside whatever the function normally returns. While subsequent programmers might not intuit the need to test for (implicit) "magic" return values they ought to notice if they start getting tuples back where they expected scalars... def foo(x) if x>0: return True, x*x else: return False, "Bad value of x in foo:",str(x) ok, value = foo(-1) if ok: print "foo of x is", value else: print "ERROR:", value Roger. From steven at REMOVE.THIS.cybersource.com.au Mon Jan 4 21:35:03 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 05 Jan 2010 02:35:03 GMT Subject: TypeError: can only concatenate list (not "tuple") to list References: Message-ID: On Tue, 05 Jan 2010 00:52:56 +0000, r0g wrote: > I'd be strongly inclined to think the result would be the sequence on > the left with the data from the second sequence appended to it. What's > wrong with a little duck typing here eh? That's not the existing behaviour. List concatenation doesn't mutate the left hand list, it creates a new list: >>> L = [1, 2, 3] >>> L2 = L + [4, 5, 6] >>> L [1, 2, 3] >>> L2 [1, 2, 3, 4, 5, 6] But if you insist on in-place modification, why do you prefer appending the right hand sequence to the left instead of prepending the left hand sequence to the right? -- Steven From aioe.org at technicalbloke.com Mon Jan 4 22:01:52 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 03:01:52 +0000 Subject: TypeError: can only concatenate list (not "tuple") to list References: Message-ID: Steven D'Aprano wrote: > On Tue, 05 Jan 2010 00:52:56 +0000, r0g wrote: > >> I'd be strongly inclined to think the result would be the sequence on >> the left with the data from the second sequence appended to it. What's >> wrong with a little duck typing here eh? OK, I hadn't read all the other responses when I posted, some of which make fair points why this wouldn't be wise. Fair enough. > > That's not the existing behaviour. List concatenation doesn't mutate the > left hand list, it creates a new list: > I would expect it to append. That's my prejudice though, as I do that far more often :/ > >>>> L = [1, 2, 3] >>>> L2 = L + [4, 5, 6] >>>> L > [1, 2, 3] >>>> L2 > [1, 2, 3, 4, 5, 6] > > > But if you insist on in-place modification, why do you prefer appending > the right hand sequence to the left instead of prepending the left hand > sequence to the right? > > > In-place seems more natural for a mutable type. I admit the left right thing is my prejudice though, western cultural bias I suppose. Its not entirely unprecedented though, the parser reads left to right and the leftmost terms take precedent in lazy logic evaluation. Still, the responses to this have convinced me the + operator shouldn't make assumptions, I'm more open to how += works though as it implies in-place and the left over right precedent quite nicely. Roger. From navjotmusic at gmail.com Mon Jan 4 22:12:53 2010 From: navjotmusic at gmail.com (Nav) Date: Mon, 4 Jan 2010 19:12:53 -0800 (PST) Subject: Newbie help- Can multiple instances with multiple names automatically created. References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> <14370388-6f6e-451c-b8a6-4ce2f6ec32fc@l30g2000yqb.googlegroups.com> Message-ID: <1df4f606-8e46-4ff1-9470-583e35612b25@r5g2000yqb.googlegroups.com> Okay, let me ask another question: When we create instances of objects by doing x = className () are we using globalnamespace? if yes then: if using globalnamespace is bad then why does every book or tutorial about python classes give the above style of assignment as an example? Second why do we use a dictionary to create something like this? I know how it works, but what is wrong with simply creating instances automatically? Once created the data for the instances is automatically saved in their own space? Why have a round about way using dictionaries? Is there an advantage or does it conflict with something else? Why not have for i in specialList: #I presume it would have to be special, because it can't be a problematic type i = className(whatever) > > > what are the risks of globalnamespace use > > You're unnecessarily tying your code to the implementation. > > > and what are the benefits? > > Absolutely none that using a dictionary doesn't also give you. From steven at REMOVE.THIS.cybersource.com.au Mon Jan 4 22:27:36 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 05 Jan 2010 03:27:36 GMT Subject: FileInput too slow References: <4b426d17$0$1113$4fafbaef@reader4.news.tin.it> Message-ID: On Mon, 04 Jan 2010 23:35:02 +0100, wiso wrote: > I'm trying the fileinput module, and I like it, but I don't understand > why it's so slow... Because it is written for convenience, not speed. From the source code: "Performance: this module is unfortunately one of the slower ways of processing large numbers of input lines." > look: > > from time import time > from fileinput import FileInput > > file = ['r1_200907.log', 'r1_200908.log', 'r1_200909.log', > 'r1_200910.log', 'r1_200911.log'] > > def f1(): > n = 0 > for f in file: > print "new file: %s" % f > ff = open(f) > for line in ff: > n += 1 > ff.close() > return n > > def f2(): > f = FileInput(file) > for line in f: > if f.isfirstline(): print "new file: %s" % f.filename() > return f.lineno() > > def f3(): # f2 simpler > f = FileInput(file) > for line in f: > pass > return f.lineno() > > > t = time(); f1(); print time()-t # 1.0 > t = time(); f2(); print time()-t # 7.0 !!! > t = time(); f3(); print time()-t # 5.5 > > I'm using text files, there are 2563150 lines in total. The extra second and a half in f2() is probably due to the time it takes to call f.isfirstline() 2563150 times. -- Steven From tjreedy at udel.edu Mon Jan 4 22:30:52 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 04 Jan 2010 22:30:52 -0500 Subject: FileInput too slow In-Reply-To: <4b426d17$0$1113$4fafbaef@reader4.news.tin.it> References: <4b426d17$0$1113$4fafbaef@reader4.news.tin.it> Message-ID: On 1/4/2010 5:35 PM, wiso wrote: > I'm trying the fileinput module, and I like it, but I don't understand why > it's so slow... look: > > from time import time > from fileinput import FileInput > > file = ['r1_200907.log', 'r1_200908.log', 'r1_200909.log', 'r1_200910.log', > 'r1_200911.log'] > > def f1(): > n = 0 > for f in file: > print "new file: %s" % f > ff = open(f) > for line in ff: > n += 1 > ff.close() > return n > > def f2(): > f = FileInput(file) > for line in f: > if f.isfirstline(): print "new file: %s" % f.filename() > return f.lineno() > > def f3(): # f2 simpler > f = FileInput(file) > for line in f: > pass > return f.lineno() > > > t = time(); f1(); print time()-t # 1.0 > t = time(); f2(); print time()-t # 7.0 !!! > t = time(); f3(); print time()-t # 5.5 > > > I'm using text files, there are 2563150 lines in total. 1. Timings should include platform and Python version. 2. fileinput executes a lot of Python code on top of the underlying file methods. Your n += 1 is inadequate as compensation. Fileinput does at least the following for each line : try: line = self._buffer[self._bufindex] except IndexError: pass else: self._bufindex += 1 self._lineno += 1 self._filelineno += 1 That is 5 attribute accesses, an indexing, and 3 additions 3. You are welcome to read the Python source in .../pythonxy/Lib/fileinput.py 4. Doc string for 3.1 version says "Performance: this module is unfortunately one of the slower ways of processing large numbers of input lines. Nevertheless, a significant speed-up has been obtained by using readlines(bufsize) instead of readline(). A new keyword argument, bufsize=N, is present on the input() function and the FileInput() class to override the default buffer size." If your version has bufsize, try something larger than the default of 8*1024, say 1024*1024. Terry Jan Reedy From larry at hastings.org Mon Jan 4 22:39:10 2010 From: larry at hastings.org (Larry Hastings) Date: Mon, 04 Jan 2010 19:39:10 -0800 Subject: Best practices for simultaneously installed versioned packages? In-Reply-To: References: <4B41CC6C.3020808@hastings.org> Message-ID: <4B42B45E.9070900@hastings.org> > In article > , > Benjamin Kaplan wrote: > >> wxpython installs a "wxversion" module which has functions like >> getInstalled(), ensureMinimal(version), and select(version). You can >> call wxversion.select before importing wx and it will make sure that >> the correct version is imported. You might want to look up what they >> did. >> I can imagine what they did: wxversion.select() explicitly imports the desired wxwindows module using __import__(), then assigns it to sys.modules["wxwindows"]. A good idea! Ned Deily wrote: > Also, setuptools (and, its successor, distribute, which supports Python > 3) allow the installation and management of multiple versions of a > package within one python site-library instance. > That sounds lovely too, but I can't figure out what facility setuptools / distribute permits that. Could you be more specific? A link to the documentation for this feature would be much appreciated. Thanks, /larry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven at REMOVE.THIS.cybersource.com.au Mon Jan 4 22:52:02 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 05 Jan 2010 03:52:02 GMT Subject: Newbie help- Can multiple instances with multiple names automatically created. References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> <14370388-6f6e-451c-b8a6-4ce2f6ec32fc@l30g2000yqb.googlegroups.com> <1df4f606-8e46-4ff1-9470-583e35612b25@r5g2000yqb.googlegroups.com> Message-ID: On Mon, 04 Jan 2010 19:12:53 -0800, Nav wrote: > Okay, let me ask another question: > > When we create instances of objects by doing > x = className() > are we using globalnamespace? That depends on whether you are doing x = className() inside a function (or class), or in the top level of the program. If I do this: x = 1234 def function(): y = 4567 then x is defined in the global namespace and y is defined in the namespace which is local to function(). > if yes then: > if using globalnamespace is bad then why does every book or tutorial > about python classes give the above style of assignment as an example? No, you're confused -- the problem isn't with using the global namespace. The problem is that you don't know what names you want to use ahead of time. You use assignment like: x = something() when you know the name x when you are writing the code. That way you can write x in the code, and all is good: x = something() print x.method() mydict = {x: -1} assert mydict.keys() == [x] Now, imagine that you didn't know what names you have to use. Say, for example, that you need a variable number of Somethings: a = Something() b = Something() c = Something() d = Something() # I never know when to stop... z = Something() # ... keep going? too far? who knows??? HELP! process(a) process(b) process(c) # when do I stop??? process(x) process(y) # ... That's the wrong way to deal with it. So instead you use a list: mylist = [] # define ONE NAME in the global namespace for i in range(some_number): mylist.append(Something()) # later... for x in mylist: # again, we use ONE name, `x` process(x) A list implicitly maps numbers (the position) to values. If you want to map strings (names) to values, use a dict. Here is an example. Notice we don't know how many players there are, or what their names are: print "Welcome to the game." print "Please enter the name of each player," print "or the word 'STOP' when there are no more players." players = {} i = 1 while True: # loop forever name = raw_input("Name of player %d? " % i) name = name.strip() # get rid of extra whitespace if name.upper() == 'STOP': # we're done break players[name] = NewPlayer() # much later... for name, player in players.items(): print "Welcome, player %s" % name play_game(player) The only downside is that dicts are unordered, so the order of the players is not the same as the order they were entered in. -- Steven From wuwei23 at gmail.com Mon Jan 4 22:56:51 2010 From: wuwei23 at gmail.com (alex23) Date: Mon, 4 Jan 2010 19:56:51 -0800 (PST) Subject: Newbie help- Can multiple instances with multiple names automatically created. References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> <14370388-6f6e-451c-b8a6-4ce2f6ec32fc@l30g2000yqb.googlegroups.com> <1df4f606-8e46-4ff1-9470-583e35612b25@r5g2000yqb.googlegroups.com> Message-ID: <09b79018-e226-4f7e-8bf6-78ada2ad3c77@22g2000yqr.googlegroups.com> On Jan 5, 1:12?pm, Nav wrote: > When we create instances of objects by doing > x = className () > are we using globalnamespace? Well, you're using a namespace, which namespaces its in would depend on the scope in which that assignment occurred. And there's not really a global namespace, only module ones. > if yes then: > ? if using globalnamespace is bad then why does every book or tutorial > about python classes give the above ?style of assignment as an > example? That's a basic assignment example. It's not a direct manipulation of globals(), like the solution given by Jan, which you seem to feel is the answer. And unless it happens outside of a nested scope like a class or function definition, it's not 'global'. > Second why do we use a dictionary to create something like this? I > know how it works, but what is wrong with simply creating instances > automatically? If I came across code like this in a joint project I was working on: def somefunc(*someparams): magical_instance_injection() for elem in magically_appearing_collection: .... We'd have serious words about the readability & maintainability of such code. If the values are important, make a clear set up that places them in a mapped collection, and make it obvious that the collection is how those values are used. Also, you've removed the set up from the place where the values are actually used, meaning I'd have to look at two sets of code to identify the values' names, as well as to confirm that 'some_random_label' _is_ actually one of the instances and not just a value you're using without instantiating. > Once created the data for the instances is > automatically saved in their own space? Why have a round about way > using dictionaries? Because using a dictionary removes the dependency your function has on the injection code, which in turns makes it more flexible for re-use. Let's assume you write your injection function, and call it within every function that uses those objects: def inject_values(): # here be global() manipulations def do_that_thang(): inject_values() ... What if you want to do_that_thang to a separate set of values? With this approach, you'd have to extend like so: def inject_values2(): ... def do_that_thang2(): inject_values2() ... You could, of course, have one inject_values function that accepts a flag and returns the appropriate set of values, but now you're having to modify that function every time the values change. With a dictionary, it's just so much simpler: values1 = dict( a = SomeObject(1), b = AnotherObject(1)... ) values2 = dict( a = SomeObject(2), b = AnotherObject(2)... ) def do_that_thang(values): c = values['a'].thanger(values['b']) ... It's clean, readable and far more easily extensible. It's also far more obvious what you're doing, which you'll really appreciate when you return to the code in a year's time :) From mensanator at aol.com Mon Jan 4 23:44:01 2010 From: mensanator at aol.com (Mensanator) Date: Mon, 4 Jan 2010 20:44:01 -0800 (PST) Subject: How do you configure IDLE on a Mac, seeing as there's no Configure Option... References: Message-ID: <54d9a742-d7fb-44b0-9cf6-ffa49ca5f2f4@u41g2000yqe.googlegroups.com> On Jan 4, 10:05?am, Benjamin Kaplan wrote: > On Sun, Jan 3, 2010 at 8:42 PM, Mensanator wrote: > > ...because there's no [Options] menu on the shell window? > > > Or at least give me a clue to how to use Courier New font? > > > For some inscrutable reason, depite the plethora of formatting tools, > > someone decided that proportional spaced fonts ought to be the > > default for IDLE. > > -- > >http://mail.python.org/mailman/listinfo/python-list > > Mac Applications don't have "options" menus. They have a "preferences" > pane which is the second item in the application menu Sometimes it's the second item, sometimes it's not > (the one that > takes the name of the active program) and has a keyboard shortcut of > Cmd-, > > Apple's user-interface guidelines are pretty specific, so you'll find > that this works for all native OS X (i.e. not X11) applications. Right, it doesn't apply to IDLE which uses X11 (there is a preferences window, but it doesn't have any font control, unlike stuff like Safari). I assume there must be a configuration file. Any idea what that file is called? From fpm at u.washington.edu Tue Jan 5 00:17:58 2010 From: fpm at u.washington.edu (cassiope) Date: Mon, 4 Jan 2010 21:17:58 -0800 (PST) Subject: IOError - cannot create file (linux daemon-invoked script) References: Message-ID: On Jan 4, 4:23?pm, Cameron Simpson wrote: > On 04Jan2010 09:16, cassiope wrote: > | To Cameron: the file doesn't (yet) exist; and it has the correct full > | path. > > Can you show us the strace output of the failing open() call? Ah...presumably you mean: [pid 1976] open("/var/tmp/share/lvrq-Robert.Smith", O_WRONLY| O_CREAT|O_TRUNC, 0666) = -1 EACCES (Permission denied) > | To "Nobody" : hey, this seems interesting. ?First test, invoking > | seteuid() > | and setegid() didn't help - but strange thing was these calls didn't > | show > | up in the strace, so perhaps I wasn't testing what I thought I was. > > If you're using the "-e trace=file" option it won't. That constrains the > output to file operations to make the log easier to read. Discard the -e > option to get everything. > > Cheers, > -- > Cameron Simpson DoD#743http://www.cskk.ezoshosting.com/cs/ Wasn't using the -e option. It turns out that the compiler was changing the code to use the linux functions setresgid() and setresuid(). That's why I didn't see it previously. If I only use the seteuid/setegid, it "works" - it is able to write the file. Unfortunately this may be partly due to its incompletely dropping priviledges - the file has root ownership, not "lesser user" ownership. Using setuid/setgid or setresuid/setresgid where just real OR both real and effective identities are set to "lesser user" - it still doesn't work - no file is written. Again, "lesser user" has no problem writing a file into this directory. I remain mystified :( Thanks for your valiant efforts! From mensanator at aol.com Tue Jan 5 00:24:55 2010 From: mensanator at aol.com (Mensanator) Date: Mon, 4 Jan 2010 21:24:55 -0800 (PST) Subject: How do you configure IDLE on a Mac, seeing as there's no Configure Option... References: Message-ID: <0d70cb54-3d77-4176-b621-e764ecf613b7@26g2000yqo.googlegroups.com> On Jan 4, 2:25?pm, Ned Deily wrote: > In article > , > ?Benjamin Kaplan wrote: > > > On Sun, Jan 3, 2010 at 8:42 PM, Mensanator wrote: > > > ...because there's no [Options] menu on the shell window? > > > > Or at least give me a clue to how to use Courier New font? > > > > For some inscrutable reason, depite the plethora of formatting tools, > > > someone decided that proportional spaced fonts ought to be the > > > default for IDLE. > > Mac Applications don't have "options" menus. They have a "preferences" > > pane which is the second item in the application menu (the one that > > takes the name of the active program) and has a keyboard shortcut of > > Cmd-, > > > Apple's user-interface guidelines are pretty specific, so you'll find > > that this works for all native OS X (i.e. not X11) applications. > > But be aware that IDLE on OS X has had various problems with missing > and/or duplicate menu items. ?IIRC, the current python.org OS X IDLEs > (2.6.4 and 3.1.1) should be OK and you can modify the font via the > Preferences menu No, I already checked. Both system preferences and X11 preferences. > (and currently changed preferences apply to all > versions of IDLE). ?Note also that there are two different ways to > launch the OS X IDLE. ?python.org installers put a double-clickable > IDLE.app in /Applications/Python n.x. Yeah, I've still got that one installed, but it has the broken distutils from python.org. >?There is also a command-line > "idlen.n" in /Library/Frameworks/Python.framework/Versions/n.n/bin > which, optionally, has a symlink from /usr/local/bin. AARRGGHH!!! I'm pulling my hair out! I haven't been using the IDLE from the broken python.org disk image. I assume I've been using the IDLE from macports. From the command prompt I've been typing "idle". This launches a "shell" window which appears to have an X11 parent application for which there are no "preferences" applicable to fonts. However, if I use the quick launcher from the python.org, I get a "shell" whose parent is named "IDLE"! And that one has a completely different preferences, one similar the the Windows Configure which allows me to set the font! Now, if I close this shell and start IDLE from the command line again, I still get a "shell" with an X11 parent, but, lo and behold, the font has changed to what I had previously set with the IDLE parent. Course, I can't import gmpy, cause the python.org version can't compile it, so I still have to use the macports install of 3.1, but that's ok, once I use the IDLE application to set the preferences, I can switch back to the X11 version and the preferences will follow. Wow! That was fun. Thanks for the help everyone. Don't ask me to explain what I just did, sometimes you just have to let art flow over you. > > -- > ?Ned Deily, > ?n... at acm.org From navjotmusic at gmail.com Tue Jan 5 00:27:12 2010 From: navjotmusic at gmail.com (Nav) Date: Mon, 4 Jan 2010 21:27:12 -0800 (PST) Subject: Newbie help- Can multiple instances with multiple names automatically created. References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> <14370388-6f6e-451c-b8a6-4ce2f6ec32fc@l30g2000yqb.googlegroups.com> <1df4f606-8e46-4ff1-9470-583e35612b25@r5g2000yqb.googlegroups.com> <09b79018-e226-4f7e-8bf6-78ada2ad3c77@22g2000yqr.googlegroups.com> Message-ID: <6f1cf0c7-eafc-472e-bfff-f86a8ae33658@r24g2000yqd.googlegroups.com> "> if yes then: > if using globalnamespace is bad then why does every book or tutorial > about python classes give the above style of assignment as an > example? That's a basic assignment example. It's not a direct manipulation of globals(), like the solution given by Jan, which you seem to feel is the answer. And unless it happens outside of a nested scope like a class or function definition, it's not 'global'. " Thanks for clearing that up Alex. The reason I liked Jan's solution at first glance was that it allowed me to manipulate the objects directly after I created the class. But I am using Shawn's advice now because it lets me abstract, but still allows me to directly manipulate the object if I need to. @ Steven.... "No, you're confused -- the problem isn't with using the global namespace. The problem is that you don't know what names you want to use ahead of time. " Actually I know what the names would be and how I want to use them. If you look at my example in the beginning you will notice that it creates unique objects with unique names, which anyone who looks at what is being mixed can easily figure out and of course the number and names of objects is directly dependent on the mixed elements. (although perhaps bottle is not the best analogy because the contents can change, which will not change once the object is created.) Having a stable and descriptive naming strategy was important, especially to use the unique object directly, and to not have to go searching where they might be located at any moment, e.g. if they are being moved about as part of other data structures. If anyone has any other thoughts regarding this, I would appreciate them. - Thanks everyone. From davea at ieee.org Tue Jan 5 00:28:58 2010 From: davea at ieee.org (Dave Angel) Date: Tue, 05 Jan 2010 00:28:58 -0500 Subject: Newbie help- Can multiple instances with multiple names automatically created. In-Reply-To: <1df4f606-8e46-4ff1-9470-583e35612b25@r5g2000yqb.googlegroups.com> References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> <14370388-6f6e-451c-b8a6-4ce2f6ec32fc@l30g2000yqb.googlegroups.com> <1df4f606-8e46-4ff1-9470-583e35612b25@r5g2000yqb.googlegroups.com> Message-ID: <4B42CE1A.2050801@ieee.org> (You top-posted. It's polite on most newsgroups, and specifically on this one to add your comments *following* the quoted earlier text) Nav wrote: > Okay, let me ask another question: > > When we create instances of objects by doing > x = className () > are we using globalnamespace? > > If that line appears at top-level, it uses the global namespace for the symbol x. If it's inside a class, or inside a function, then it uses some other namespace. The risk for globals is mainly that the namespace gets "polluted", or full of unrelated symbols, which sooner or later are going to collide. And this is made much worse if someone is creating such names indirectly, by monkeying with the non-portable constructs such as global(), or by doing from mymodule import * There are other risks, but this is a short message. As far as I know, there's very little performance cost for having lots of stuff in globals. It's just bad practice because it frequently leads to bugs. > if yes then: > if using globalnamespace is bad then why does every book or tutorial > about python classes give the above style of assignment as an > example? > > Beginner books keep things simple. Besides the same statement is great if it's in a function. And it's usually a small step for the beginner to start moving stuff from the top-level of the module into named functions. And eventually to functions that are small enough to be maintainable. > Second why do we use a dictionary to create something like this? I > know how it works, but what is wrong with simply creating instances > automatically? Once created the data for the instances is > automatically saved in their own space? Why have a round about way > using dictionaries? > Is there an advantage or does it conflict with something else? > > What? Automatically where? What's roundabout about dictionaries? > Why not have > for i in specialList: #I presume it would have to be special, because > it can't be a problematic type > i = className(whatever) > > > Once you bind a new value to i, it no longer has any relationship to the value which was in specialList. Perhaps you want something like: for i in xrange(len(specialList)): specialList[i] = className(...) which will replace the present values in specialList with instances of className. > > >>> what are the risks of globalnamespace use >>> >> You're unnecessarily tying your code to the implementation. >> >> >>> and what are the benefits? >>> >> Absolutely none that using a dictionary doesn't also give you. >> > > > From fpm at u.washington.edu Tue Jan 5 00:30:31 2010 From: fpm at u.washington.edu (cassiope) Date: Mon, 4 Jan 2010 21:30:31 -0800 (PST) Subject: IOError - cannot create file (linux daemon-invoked script) References: Message-ID: One more tidbit observed: my last note, that it works when using seteuid/setegid? Well - that only applies if the daemon is running under strace (!). It fails if started directly by root, or if the strace session has ended, leaving the main body of the daemon running in its normal headless manner. I wonder if running under "strace -f" - might setegid/seteuid be prevented from having their normal effect? Sigh. From mensanator at aol.com Tue Jan 5 00:32:13 2010 From: mensanator at aol.com (Mensanator) Date: Mon, 4 Jan 2010 21:32:13 -0800 (PST) Subject: How do you configure IDLE on a Mac, seeing as there's no Configure Option... References: <54d9a742-d7fb-44b0-9cf6-ffa49ca5f2f4@u41g2000yqe.googlegroups.com> Message-ID: <30ebdab2-4dc8-411e-a3e3-482f99cfd2ce@k17g2000yqh.googlegroups.com> On Jan 4, 10:44?pm, Mensanator wrote: > On Jan 4, 10:05?am, Benjamin Kaplan wrote: > > > On Sun, Jan 3, 2010 at 8:42 PM, Mensanator wrote: > > > ...because there's no [Options] menu on the shell window? > > > > Or at least give me a clue to how to use Courier New font? > > > > For some inscrutable reason, depite the plethora of formatting tools, > > > someone decided that proportional spaced fonts ought to be the > > > default for IDLE. > > > -- > > >http://mail.python.org/mailman/listinfo/python-list > > > Mac Applications don't have "options" menus. They have a "preferences" > > pane which is the second item in the application menu > > Sometimes it's the second item, sometimes it's not > > > (the one that > > takes the name of the active program) and has a keyboard shortcut of > > Cmd-, > > > Apple's user-interface guidelines are pretty specific, so you'll find > > that this works for all native OS X (i.e. not X11) applications. > > Right, it doesn't apply to IDLE which uses X11 Depending on how you launch it. If I type "idle" at a command prompt, I get a shell window whose parent is X11. Here you will get X11 preferences which can't be used to change fonts. If I click the IDLE icon, then I get a shell whose parent is IDLE, not X11 and here you'll find a preferences window similar to the OPTIONS/Configure in Windows. > (there is a preferences > window, > but it doesn't have any font control, unlike stuff like Safari). > > I assume there must be a configuration file. Any idea what that file > is called? I still don't know but apparently as long as I use the IDLE application instead of X11, I can set the font. Thanks for your time. From mensanator at aol.com Tue Jan 5 00:39:38 2010 From: mensanator at aol.com (Mensanator) Date: Mon, 4 Jan 2010 21:39:38 -0800 (PST) Subject: How do you configure IDLE on a Mac, seeing as there's no Configure Option... References: Message-ID: <12cf7e45-b0f7-4e61-88c0-6cf44ddf4e09@a15g2000yqm.googlegroups.com> On Jan 4, 9:17?am, "Alf P. Steinbach" wrote: > * Mensanator: > > > ...because there's no [Options] menu on the shell window? > > > Or at least give me a clue to how to use Courier New font? > > > For some inscrutable reason, depite the plethora of formatting tools, > > someone decided that proportional spaced fonts ought to be the > > default for IDLE. > > Why not just use the Python interpreter, a reasonable programmer's editor and, > if you need it, some debugger. I think I've solved my problem. I'll just say here that at least IDLE is cross-platform AND is part of the "batteries included". I'll be using it on both my new Mac and my desktop PCs running Windows. > > The source code level debugging in IDLE is nothing to write home about. I've got 20 years worth of programs and web pages written with Notepad, so don't let me hear any dissing of IDLE, ok? :-) > > In Windows XP it is just silly: it marks the current line by ordinary text > selection, which (depending on your config, but I think this is default) is > invisible when you're looking at the debugger window for single stepping. You > can see it by switching back to the source code window but that's annoying, not > very practical. Invisible highlighting of the current line, he he... > > Cheers & hth., > > - Alf From steven at REMOVE.THIS.cybersource.com.au Tue Jan 5 01:03:28 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 05 Jan 2010 06:03:28 GMT Subject: Newbie help- Can multiple instances with multiple names automatically created. References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> <14370388-6f6e-451c-b8a6-4ce2f6ec32fc@l30g2000yqb.googlegroups.com> <1df4f606-8e46-4ff1-9470-583e35612b25@r5g2000yqb.googlegroups.com> <09b79018-e226-4f7e-8bf6-78ada2ad3c77@22g2000yqr.googlegroups.com> <6f1cf0c7-eafc-472e-bfff-f86a8ae33658@r24g2000yqd.googlegroups.com> Message-ID: On Mon, 04 Jan 2010 21:27:12 -0800, Nav wrote: > @ Steven.... > "No, you're confused -- the problem isn't with using the global > namespace. > The problem is that you don't know what names you want to use ahead of > time. " > > Actually I know what the names would be and how I want to use them. You said earlier: "I have a class of let's say empty bottle which can have a mix of two items. I want to create let's say 30 of these objects which will have names based on the 2 attributes (apple juice, beer, grape juice, beer, etc) that I provide from a list." Your description is confusing to me. What on earth is an empty bottle which has a mix of two items in it? Surely that means it's not empty any more? But putting that aside: "All the objects are a mix of (1 of three alcohols) and (1 of 10 juices), so I don't want to go through typing in the names of all the objects (which would be totally stupid)." Right... so your problem isn't that you don't know what the variable names is, but there are too many to comfortably enumerate in the source code. The answer is, again, avoid named variables. Instead of (say): gin_apple_strawberry = Bottle('gin', 'apple') gin_apple_orange = Bottle('gin', 'orange') # etc. again you should use a list or dict: bottles = [] for alcohol in ('gin', 'beer', 'wine'): for fruit in ('apple', 'banana', 'blueberry', 'strawberry', 'orange', 'peach'): bottles.append(Bottle(alcohol, fruit)) for bottle in bottles: process(bottle) -- Steven From nad at acm.org Tue Jan 5 01:13:10 2010 From: nad at acm.org (Ned Deily) Date: Mon, 04 Jan 2010 22:13:10 -0800 Subject: Best practices for simultaneously installed versioned packages? References: <4B41CC6C.3020808@hastings.org> <4B42B45E.9070900@hastings.org> Message-ID: In article <4B42B45E.9070900 at hastings.org>, Larry Hastings wrote: > Ned Deily wrote: > > Also, setuptools (and, its successor, distribute, which supports Python > > 3) allow the installation and management of multiple versions of a > > package within one python site-library instance. > That sounds lovely too, but I can't figure out what facility setuptools > / distribute permits that. Could you be more specific? A link to the > documentation for this feature would be much appreciated. By default, easy_install from setuptools/distribute uses version information from a package's setup.py to qualify the package name and manipulate python's sys.path to ensure the most recent version is the one found by default. There are options to easy_install which control this. See in particular the easy_install "--multi-version" command-line option (the links here are to those for the Distribute fork of setuptools): http://packages.python.org/distribute/easy_install.html If you need to declare specific version dependencies for other packages, you can do so in setup.py. See the Declaring Dependencies section of the setuptools API documentation: http://packages.python.org/distribute/setuptools.html Much of the magic is handled by setuptools version of pkg_resources: http://packages.python.org/distribute/pkg_resources.html -- Ned Deily, nad at acm.org From nad at acm.org Tue Jan 5 01:32:47 2010 From: nad at acm.org (Ned Deily) Date: Mon, 04 Jan 2010 22:32:47 -0800 Subject: How do you configure IDLE on a Mac, seeing as there's no Configure Option... References: <0d70cb54-3d77-4176-b621-e764ecf613b7@26g2000yqo.googlegroups.com> Message-ID: In article <0d70cb54-3d77-4176-b621-e764ecf613b7 at 26g2000yqo.googlegroups.com>, Mensanator wrote: > I assume I've been using the IDLE from macports. From the command > prompt I've > been typing "idle". This launches a "shell" window which appears to > have an X11 > parent application for which there are no "preferences" applicable to > fonts. > > However, if I use the quick launcher from the python.org, I get a > "shell" whose > parent is named "IDLE"! And that one has a completely different > preferences, > one similar the the Windows Configure which allows me to set the font! > > Now, if I close this shell and start IDLE from the command line again, > I still > get a "shell" with an X11 parent, but, lo and behold, the font has > changed to > what I had previously set with the IDLE parent. > > Course, I can't import gmpy, cause the python.org version can't > compile it, so I > still have to use the macports install of 3.1, but that's ok, once I > use > the IDLE application to set the preferences, I can switch back to the > X11 version and the preferences will follow. The prefs follow because all versions of IDLE use the same (unversioned) directory for configuration files, ~/.idlerc/. In particular, the configuration file ~/.idlerc/config-main.cfg contains, among other things, any changes to the default font. So, if you're successful at changing it in one version of IDLE, it will likely affect all versions you have. Note the file is a simple ini format: [EditorWindow] font = monaco so you can edit it by hand. BTW, the python.org IDLEs and the Apple-supplied IDLEs use the system-supplied Aqua (aka Quartz) Tk not the X11 one that MacPorts builds by default. The MacPorts Tk port does have a "quartz" variant but that doesn't yet work in 64-bit mode. -- Ned Deily, nad at acm.org From lie.1296 at gmail.com Tue Jan 5 02:02:37 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Tue, 05 Jan 2010 18:02:37 +1100 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> Message-ID: <4b42e40c$1@dnews.tpgi.com.au> On 1/5/2010 1:31 PM, r0g wrote: > Michi wrote: >> On Jan 4, 1:30 pm, Steven D'Aprano >> wrote: > >>> In some, limited, cases you might be able to use the magic return value >>> strategy, but this invariably leads to lost programmer productivity, more >>> complex code, lowered readability and usability, and more defects, >>> because programmers will invariably neglect to test for the special value: >> >> I disagree here, to the extent that, whether something is an error or >> not can very much depend on the circumstances in which the API is >> used. The collection case is a very typical example. Whether failing >> to locate a value in a collection is an error very much depends on >> what the collection is used for. In some cases, it's a hard error >> (because it might, for example, imply that internal program state has >> been corrupted); in other cases, not finding a value is perfectly >> normal. > > > > A pattern I have used a few times is that of returning an explicit > success/failure code alongside whatever the function normally returns. > While subsequent programmers might not intuit the need to test for > (implicit) "magic" return values they ought to notice if they start > getting tuples back where they expected scalars... > > def foo(x) > if x>0: > return True, x*x > else: > return False, "Bad value of x in foo:",str(x) > > ok, value = foo(-1) > if ok: > print "foo of x is", value > else: > print "ERROR:", value Except that that is a reinvention of try-wheel: def foo(x): if x > 0: return x*x else: raise MathError("Bad value of x in foo: %s" % x) try: print foo(-1) except MathError, e: print "ERROR: System integrity is doubted" or rather; that is perhaps a good example of when to use 'assert'. If the domain of foo() is positive integers, calling -1 on foo is a bug in the caller, not foo(). I have been looking at Haskell recently and the way the pure functional language handled exceptions and I/O gives me a new distinct "insight" that exceptions can be thought of as a special return value that is implicitly wrapped and unwrapped up the call stack until it is explicitly handled. From stefan_ml at behnel.de Tue Jan 5 03:16:17 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 05 Jan 2010 09:16:17 +0100 Subject: Where's a DOM builder that uses the Builder Pattern to ... build DOMs? In-Reply-To: <04523152-a0ff-432c-a48f-2c653577e7b7@m25g2000yqc.googlegroups.com> References: <04523152-a0ff-432c-a48f-2c653577e7b7@m25g2000yqc.googlegroups.com> Message-ID: <4b42f551$0$7625$9b4e6d93@newsspool1.arcor-online.net> Phlip, 04.01.2010 21:01: > Not Hyp: > > I hope I'm wrong, but seems that DOMBuilder, found among the various > xml.dom packages, cannot build DOM like this: > > var html = DomBuilder.apply(); > > var form = html.FORM( > html.DIV( > html.INPUT({type : 'text', name : 'email'}), > html.INPUT({type : 'text', name : 'password'}), > html.INPUT({type : 'submit'}), > ) > ); > > Do anyone know any good DOM builder packages that do build DOM good > like a DOM builder should? You might be looking for something like this: http://codespeak.net/lxml/lxmlhtml.html#creating-html-with-the-e-factory Note that there are tons of ways to generate HTML with Python. A quick web search (or a quick read on PyPI or the Python Wiki) should get you started. Stefan From steven at REMOVE.THIS.cybersource.com.au Tue Jan 5 03:26:10 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 05 Jan 2010 08:26:10 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> Message-ID: On Tue, 05 Jan 2010 02:31:34 +0000, r0g wrote: > A pattern I have used a few times is that of returning an explicit > success/failure code alongside whatever the function normally returns. That doesn't work for languages that can only return a single result, e.g. C or Pascal. You can fake it by creating a struct that contains a flag and the result you want, but that means doubling the number of data types you deal with. > While subsequent programmers might not intuit the need to test for > (implicit) "magic" return values they ought to notice if they start > getting tuples back where they expected scalars... What if they're expecting tuples as the result? > def foo(x) > if x>0: > return True, x*x > else: > return False, "Bad value of x in foo:",str(x) > > ok, value = foo(-1) Oops, that gives: ValueError: too many values to unpack because you've returned three items instead of two. When an idiom is easy to get wrong, it's time to think hard about it. > if ok: > print "foo of x is", value > else: > print "ERROR:", value Whenever I come across a function that returns a flag and a result, I never know whether the flag comes first or second. Should I write: flag, result = foo(x) or result, flag = foo(x) I've seen APIs that do both. And I never know if the flag should be interpreted as a success or a failure. Should I write: ok, result = foo(x) if ok: process(result) else: fail() or err, result = foo(x) if err: fail() else: process(result) Again, I've seen APIs that do both. And if the flag indicates failure, what should go into result? An error code? An error message? That's impossible for statically-typed languages, unless they have variant records or the function normally returns a string. And even if you dismiss all those concerns, it still hurts readability by obfuscating the code. Consider somebody who wants to do this: result = foo(bar(x)) but instead has to do this: flag, result = bar(x) if flag: # I think this means success flag, result = foo(x) # oops, I meant result Again, it's error-prone and messy. Imagine writing: flag, a = sin(x) if flag: flag, b = sqrt(x) if flag: flag, c = cos(b) if flag: flag, d = exp(a + c) if flag: flag, e = log(x) if flag: # Finally, the result we want!!! flag, y = d/e if not flag: fail(y) else: fail(e) else: fail(d) else: fail(c) else: fail(b) else: fail(a) Compare that to the way with exceptions: y = exp(sin(x) + cos(sqrt(x)))/log(x) Which would you prefer? -- Steven From g.bogle at auckland.no.spam.ac.nz Tue Jan 5 03:36:59 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Tue, 05 Jan 2010 21:36:59 +1300 Subject: A null program - what is it doing? Message-ID: No doubt a dumb question from a noob: The following program (a cut down version of some test code) uses no CPU, and does not terminate: import sys from PyQt4.QtCore import * if __name__=="__main__": app = QCoreApplication(sys.argv) sys.exit(app.exec_()) What is the program doing? What is the correct way to terminate the execution? Thanks in advance for educating me. From news123 at free.fr Tue Jan 5 04:03:56 2010 From: news123 at free.fr (News123) Date: Tue, 05 Jan 2010 10:03:56 +0100 Subject: python xmlrpc client with ssl client certificates and standard modules In-Reply-To: <4B4267DB.3080409@v.loewis.de> References: <4b414e79$0$23321$426a34cc@news.free.fr> <4B4267DB.3080409@v.loewis.de> Message-ID: <4b43007c$0$11218$426a74cc@news.free.fr> Hi Martin, Thanks a lot for your reply. It helped me to find the correct solution. Unfortunaltely xmlrpclib.ServerProxy does not allow a host tuple, but just a uri. So the simplest solution, that I found is to create a custom transport import xmlrpclib class SafeTransportWithCert(xmlrpclib.SafeTransport): __cert_file = DFLT_CERTFILE __key_file = DFLT_KEYFILE def make_connection(self,host): host_with_cert = (host, { 'key_file' : self.__key_file, 'cert_file' : self.__cert_file } ) return \ xmlrpclib.SafeTransport.make_connection( self,host_with_cert) transport = SafeTransportWithCert() server = xmlrpclib.ServerProxy(server_url, transport = transport) rslt = server.mymethod(args) Perfect. Now the server can ensure, that only certified clients connect. My next task is how to find out at the client side, that the server certificate is a properly signed one. bye N Martin v. Loewis wrote: >> I can do xmlrpc over ssl WITHOUT certificates with following code: >> >> import xmlrpclib >> server_url = 'https://myserver' >> server = xmlrpclib.Server(server_url); >> >> >> and I can perform a https get request WITH certificates with below snippet: >> >> import httplib >> conn = httplib.HTTPSConnection( >> HOSTNAME, >> key_file = KEYFILE, >> cert_file = CERTFILE >> ) >> conn.putrequest('GET', '/') >> conn.endheaders() >> response = conn.getresponse() >> print response.read() >> >> >> I'm just lost of how to 'combine' both. > > In this case, read through the source of xmlrpclib: > > a) SafeTransport receives x509 parameters from get_host_info > b) get_host_info supports a case where host is a tuple host, x509 > > So, without testing: > > server = xmlrpclib.Server((server_url, {'key_file': KEYFILE, > 'cert_file': CERTFILE})) > > Please do read the code before trying this out. > > HTH, > Martin From aioe.org at technicalbloke.com Tue Jan 5 04:07:20 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 09:07:20 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> Message-ID: Lie Ryan wrote: > On 1/5/2010 1:31 PM, r0g wrote: >> Michi wrote: >>> On Jan 4, 1:30 pm, Steven D'Aprano >>> wrote: >> A pattern I have used a few times is that of returning an explicit >> success/failure code alongside whatever the function normally returns. >> While subsequent programmers might not intuit the need to test for >> (implicit) "magic" return values they ought to notice if they start >> getting tuples back where they expected scalars... >> >> def foo(x) >> if x>0: >> return True, x*x >> else: >> return False, "Bad value of x in foo:",str(x) >> >> ok, value = foo(-1) >> if ok: >> print "foo of x is", value >> else: >> print "ERROR:", value > > Except that that is a reinvention of try-wheel: > True, but there's more than one way to skin a cat! Mine's faster if you expect a high rate of failures (over 15%). > def foo(x): > if x > 0: > return x*x > else: > raise MathError("Bad value of x in foo: %s" % x) > > try: > print foo(-1) > except MathError, e: > print "ERROR: System integrity is doubted" > > or rather; that is perhaps a good example of when to use 'assert'. If > the domain of foo() is positive integers, calling -1 on foo is a bug in > the caller, not foo(). Maybe, although I recently learned on here that one can't rely on assert statements in production code, their intended use is to aid debugging and testing really. Besides, that was just a toy example. > > I have been looking at Haskell recently and the way the pure functional > language handled exceptions and I/O gives me a new distinct "insight" > that exceptions can be thought of as a special return value that is > implicitly wrapped and unwrapped up the call stack until it is > explicitly handled. Yes there's some very interesting paradigms coming out of functional programming but, unless you're a maths major, functional languages are a long way off being productivity tools! Elegant: yes, provable: maybe, practical for everyday coding: not by a long shot! Roger. From clp2 at rebertia.com Tue Jan 5 04:19:00 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Tue, 5 Jan 2010 01:19:00 -0800 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> Message-ID: <50697b2c1001050119g7ae3b41fme7b5f8824396a99d@mail.gmail.com> On Tue, Jan 5, 2010 at 1:07 AM, r0g wrote: > Lie Ryan wrote: >> I have been looking at Haskell recently and the way the pure functional >> language handled exceptions and I/O gives me a new distinct "insight" >> that exceptions can be thought of as a special return value that is >> implicitly wrapped and unwrapped up the call stack until it is >> explicitly handled. > > Yes there's some very interesting paradigms coming out of functional > programming but, unless you're a maths major, functional languages are a > long way off being productivity tools! Elegant: yes, provable: maybe, > practical for everyday coding: not by a long shot! Methinks the authors of Real World Haskell (excellent read btw) have a bone to pick with you. Cheers, Chris -- http://blog.rebertia.com From aleksandar27 at BRISIOVOnet.hr Tue Jan 5 04:21:29 2010 From: aleksandar27 at BRISIOVOnet.hr (alejandro) Date: Tue, 5 Jan 2010 10:21:29 +0100 Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> Message-ID: Did you mean borderless printing? Every printer needs his margins, some more some less. Some printers have the ability to do borderless printing but usualy they can do it only on special or photo paper. So you can adjust the pdf as you wish, even with no margins, and then try to find under printer options "borderless printing". That is why I didn't understand :-)) it is a printer thing not pdf! From ashish.vyas at motorola.com Tue Jan 5 04:38:50 2010 From: ashish.vyas at motorola.com (VYAS ASHISH M-NTB837) Date: Tue, 5 Jan 2010 17:38:50 +0800 Subject: lxml 2.2.4 on python3.1, Windows XP gives importerror Message-ID: <7C57DB58C81FB64CA979EC6C4DB73E7A03E7039F@ZMY16EXM67.ds.mot.com> Dear All I have Python 3.1 installed on Windows XP and Works nice. I downloaded lxml 2.2.4 (lxml-2.2.4.win32-py3.1.exe) from pypi. When I try: from lxml import etree I get: ImportError: DLL load failed: This application has failed to start because the application configuration is incorrect. Reinstalling the application may fix this problem. For information: 'import lxml' works fine. After reinstalling python3.1 also the error message is the same. Any help is appreciated! Regards, Ashish Vyas -------------- next part -------------- An HTML attachment was scrubbed... URL: From richardbp at gmail.com Tue Jan 5 04:43:12 2010 From: richardbp at gmail.com (Richard) Date: Tue, 5 Jan 2010 01:43:12 -0800 (PST) Subject: Efficiently determine where documents differ References: Message-ID: On Jan 5, 9:46?am, "Gabriel Genellina" wrote: > En Mon, 04 Jan 2010 19:04:12 -0300, Richard escribi?: > > > I have been using the difflib library to find where 2 large HTML > > documents differ. The Differ().compare() method does this, but it is > > very slow - atleast 100x slower than the unix diff command. > > Differ compares sequences of lines *and* lines as sequences of characters ? > to provide intra-line differences. The diff command only processes lines. > If you aren't interested in intra-line differences, use a SequenceMatcher ? > instead. Or, invoke the diff command using ? subprocess.Popen + ? > communicate. > > -- > Gabriel Genellina thank you very much Gabriel! Passing a list of the document lines makes the efficiency comparable to the diff command. Richard From aioe.org at technicalbloke.com Tue Jan 5 04:52:32 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 09:52:32 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> Message-ID: Steven D'Aprano wrote: > On Tue, 05 Jan 2010 02:31:34 +0000, r0g wrote: > >> A pattern I have used a few times is that of returning an explicit >> success/failure code alongside whatever the function normally returns. > > That doesn't work for languages that can only return a single result, > e.g. C or Pascal. You can fake it by creating a struct that contains a > flag and the result you want, but that means doubling the number of data > types you deal with. No, but that's why I try not to use languages where you can only return a single result, I always found that an arbitrary and annoying constraint to have. I leads to ugly practices like "magic" return values in C or explicitly packing things into hashtables like PHP, yuk! > > >> While subsequent programmers might not intuit the need to test for >> (implicit) "magic" return values they ought to notice if they start >> getting tuples back where they expected scalars... > > What if they're expecting tuples as the result? > > > >> def foo(x) >> if x>0: >> return True, x*x >> else: >> return False, "Bad value of x in foo:",str(x) >> >> ok, value = foo(-1) > > Oops, that gives: > > ValueError: too many values to unpack > > > because you've returned three items instead of two. When an idiom is easy > to get wrong, it's time to think hard about it. > That seems pretty clear to me, "too many values to unpack", either I've not given it enough variables to unpack the result into or I've returned too many things. That would take a couple of seconds to notice and fix. In fact I was trying to make the point that it would be quite noticable if a function returned more things than the programmer was expecting, this illustrates that quite well :) > > >> if ok: >> print "foo of x is", value >> else: >> print "ERROR:", value > > > Whenever I come across a function that returns a flag and a result, I > never know whether the flag comes first or second. Should I write: > Flag then result, isn't it obvious? The whole point of returning a flag AND a result is so you can test the flag so you know what to do with the result so that implies a natural order. Of course it doesn't matter technically which way you do it, make a convention and stick to it. If you get perpetually confused as to the order of parameters then you'd better avoid this kind of thing, can't say as I've ever had a problem with it though. > And I never know if the flag should be interpreted as a success or a > failure. Should I write: > > ok, result = foo(x) > if ok: process(result) > else: fail() Yes. That would be my strong preference anyway. Naturally you can do it the other way round if you like, as long as you document it properly in your API. As you say different APIs do it differently... Unix has a convention of returning 0 on no-error but unix has to encapsulate a lot in that "error code" which is a bit of an anachronism these days. I'd argue in favour of remaining positive and using names like ok or success, this is closer to the familiar paradigm of checking a result does not evaluate to false before using it... name = "" if name: print name > And if the flag indicates failure, what should go into result? An error > code? An error message? That's impossible for statically-typed languages, > unless they have variant records or the function normally returns a > string. Yeah, in my example it's an error message. Maybe I shouldn't have used the word "pattern" above though as it has overtones of "universally applicable" which it clearly isn't. > Again, it's error-prone and messy. Imagine writing: > > > flag, a = sin(x) > if flag: > flag, b = sqrt(x) > if flag: > Compare that to the way with exceptions: > > y = exp(sin(x) + cos(sqrt(x)))/log(x) > > > Which would you prefer? > LOL, straw man is straw! You know full well I'm not suggesting every function return a flag, that would be silly. There's no reason returning flag and a value shouldn't be quite readable and there may be times when it's preferable to raising an exception. I use exceptions a lot as they're often the right tool for the job and they seem pleasingly pythonic but from time to time they can be too slow or verbose, where's the sense in forcing yourself to use them then? Roger. From paul.nospam at rudin.co.uk Tue Jan 5 04:54:42 2010 From: paul.nospam at rudin.co.uk (Paul Rudin) Date: Tue, 05 Jan 2010 09:54:42 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> Message-ID: <87eim4j2ql.fsf@rudin.co.uk> r0g writes: > Steven D'Aprano wrote: >> On Tue, 05 Jan 2010 02:31:34 +0000, r0g wrote: >> >>> A pattern I have used a few times is that of returning an explicit >>> success/failure code alongside whatever the function normally returns. >> >> That doesn't work for languages that can only return a single result, >> e.g. C or Pascal. You can fake it by creating a struct that contains a >> flag and the result you want, but that means doubling the number of data >> types you deal with. > > > No, but that's why I try not to use languages where you can only return > a single result, I always found that an arbitrary and annoying > constraint to have. I leads to ugly practices like "magic" return values > in C or explicitly packing things into hashtables like PHP, yuk! Doesn't python just return a single result? (I know it can be a tuple and assignment statements will unpack a tuple for you.) From aioe.org at technicalbloke.com Tue Jan 5 04:58:56 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 09:58:56 +0000 Subject: A null program - what is it doing? References: Message-ID: Gib Bogle wrote: > No doubt a dumb question from a noob: > > The following program (a cut down version of some test code) uses no > CPU, and does not terminate: > > import sys > from PyQt4.QtCore import * > > if __name__=="__main__": > app = QCoreApplication(sys.argv) > sys.exit(app.exec_()) > > What is the program doing? What is the correct way to terminate the > execution? > > Thanks in advance for educating me. I've never used QT but other graphical toolkits I have used all start their own "main loop" which is a loop that cycles round receiving, queuing and dispatching "events". You probably need to call the QCoreApplication's quit method to break out of this e.g. app.exit() or something similar, have a look at some complete PyQt4 examples or google for PyQt4 mainloop. Roger. From almar.klein at gmail.com Tue Jan 5 05:18:33 2010 From: almar.klein at gmail.com (Almar Klein) Date: Tue, 5 Jan 2010 11:18:33 +0100 Subject: A null program - what is it doing? In-Reply-To: References: Message-ID: 2010/1/5 r0g > Gib Bogle wrote: > > No doubt a dumb question from a noob: > > > > The following program (a cut down version of some test code) uses no > > CPU, and does not terminate: > > > > import sys > > from PyQt4.QtCore import * > > > > if __name__=="__main__": > > app = QCoreApplication(sys.argv) > > sys.exit(app.exec_()) > > > > What is the program doing? What is the correct way to terminate the > > execution? > > > > Thanks in advance for educating me. > > > > I've never used QT but other graphical toolkits I have used all start > their own "main loop" which is a loop that cycles round receiving, > queuing and dispatching "events". You probably need to call the > QCoreApplication's quit method to break out of this e.g. > > app.exit() or something similar, have a look at some complete PyQt4 > examples or google for PyQt4 mainloop. > app.exec_() starts the QT mainloop. If you would have created a window or app before starting the loop, you app would now be responsive and wait for you to do things with it. When you would then close the app, the function returns the error code with which the app closed, and then subsequently the Python process exits with that same error code. But since no widgets were created before starting the mainloop, I don't think you can stop the process in any way other than killing it. By the way, QT has excellent documentation: http://doc.trolltech.com/4.4/qapplication.html#exec Cheers, Almar -------------- next part -------------- An HTML attachment was scrubbed... URL: From aioe.org at technicalbloke.com Tue Jan 5 05:22:23 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 10:22:23 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> Message-ID: Chris Rebert wrote: > > On Tue, Jan 5, 2010 at 1:07 AM, r0g wrote: >> Lie Ryan wrote: >>> I have been looking at Haskell recently and the way the pure functional >>> language handled exceptions and I/O gives me a new distinct "insight" >>> that exceptions can be thought of as a special return value that is >>> implicitly wrapped and unwrapped up the call stack until it is >>> explicitly handled. >> Yes there's some very interesting paradigms coming out of functional >> programming but, unless you're a maths major, functional languages are a >> long way off being productivity tools! Elegant: yes, provable: maybe, >> practical for everyday coding: not by a long shot! > > Methinks the authors of Real World Haskell (excellent read btw) have a > bone to pick with you. > > Cheers, > Chris > -- > http://blog.rebertia.com LOL, it seems things have come a long way since ML! I'm impressed how many useful libraries Haskell has, and that they've included IF-THEN-ELSE in the syntax! :) For all its advantages I still think you need to be fundamentally cleverer to write the same programs in a functional language than an old fashioned "English like" language. Maybe I'm just mistrusting of the new school though and you'll see me on comp.lang.haskell in a few years having to eat my own monads! Roger. From aioe.org at technicalbloke.com Tue Jan 5 05:31:03 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 10:31:03 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> Message-ID: Paul Rudin wrote: > r0g writes: > >> Steven D'Aprano wrote: >>> On Tue, 05 Jan 2010 02:31:34 +0000, r0g wrote: >>> >>>> A pattern I have used a few times is that of returning an explicit >>>> success/failure code alongside whatever the function normally returns. >>> That doesn't work for languages that can only return a single result, >>> e.g. C or Pascal. You can fake it by creating a struct that contains a >>> flag and the result you want, but that means doubling the number of data >>> types you deal with. >> >> No, but that's why I try not to use languages where you can only return >> a single result, I always found that an arbitrary and annoying >> constraint to have. I leads to ugly practices like "magic" return values >> in C or explicitly packing things into hashtables like PHP, yuk! > > Doesn't python just return a single result? (I know it can be a tuple and > assignment statements will unpack a tuple for you.) Yes, it returns a tuple if you return more than one value, it just has a lovely syntax for it. In static languages you'd need to manually create an new array or struct, pack your return vars into it and unpack them on the other side. That's something I'd be happy never to see again, sadly I have to write in PHP sometimes :( Roger. From coert at holmes.nl Tue Jan 5 06:19:00 2010 From: coert at holmes.nl (Coert Klaver (DT)) Date: Tue, 5 Jan 2010 12:19:00 +0100 Subject: ctypes: How to call unexported functions in a dll Message-ID: Hi, I am using ctypes in python 3 on a WXP machine Loading a dll and using its exported functions works fine. Now I want to use a function in the dll that is not exported. In C this can be done by just casting the address in the dll of that function to an apropriate function pointer and call it (you need to be sure about the address). Can a similar thing be done directly with ctypes? A work around I see is writing in wrapper dll in c that loads the main dll, exports a function that calls the unexported function in the main dll, but I don't find that an elegant solution. Thanks for any hint in the right direction. BR Coert From davea at ieee.org Tue Jan 5 06:59:59 2010 From: davea at ieee.org (Dave Angel) Date: Tue, 05 Jan 2010 06:59:59 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> Message-ID: <4B4329BF.6080704@ieee.org> r0g wrote: > > > Maybe, although I recently learned on here that one can't rely on assert > statements in production code, their intended use is to aid debugging > and testing really. > > Hopefully, what you learned is that you can't use assert() in production code to validate user data. It's fine to use it to validate program logic, because that shouldn't still need testing in production. DaveA From davea at ieee.org Tue Jan 5 07:12:40 2010 From: davea at ieee.org (Dave Angel) Date: Tue, 05 Jan 2010 07:12:40 -0500 Subject: A null program - what is it doing? In-Reply-To: References: Message-ID: <4B432CB8.8040500@ieee.org> r0g wrote: > Gib Bogle wrote: > >> No doubt a dumb question from a noob: >> >> The following program (a cut down version of some test code) uses no >> CPU, and does not terminate: >> >> import sys >> from PyQt4.QtCore import * >> >> if __name__=="__main__": >> app = QCoreApplication(sys.argv) >> sys.exit(app.exec_()) >> >> What is the program doing? What is the correct way to terminate the >> execution? >> >> Thanks in advance for educating me. >> > > > > I've never used QT but other graphical toolkits I have used all start > their own "main loop" which is a loop that cycles round receiving, > queuing and dispatching "events". You probably need to call the > QCoreApplication's quit method to break out of this e.g. > > app.exit() or something similar, have a look at some complete PyQt4 > examples or google for PyQt4 mainloop. > > > Roger. > > Likewise, I haven't used QT, but have used others. Gib: It looks like app.exec() is the mainloop for QT, and it won't terminate till it gets a terminate event. That might be the user clicking the X in the corner, or it might be a special keystroke like Alt-F4. Or it might be some other event for which your event handler makes an explicit call to terminate. Once one of these things happens, the app.exec() will return, and the sys.exit() will execute. DaveA From kk720801 at yahoo.com Tue Jan 5 07:33:52 2010 From: kk720801 at yahoo.com (Krzysztof Kobus) Date: Tue, 5 Jan 2010 04:33:52 -0800 (PST) Subject: embedded python on mac - linking problem Message-ID: <477262.24163.qm@web53508.mail.re2.yahoo.com> Hi, I have a problem with linking python module with my application on mac in order to make the module available in "embedded python". My python module is contained in j3kmodule.cxx file and module initialization function is exported in j3kmodule.h j3kmodule.h: ------------ PyMODINIT_FUNC PyInit_j3k(void); j3kmodule.cxx: -------------- PyMODINIT_FUNC PyInit_j3k(void) { PyObject *m = NULL; if ((m = PyModule_Create(&j3k_module)) == NULL) return NULL; return m; } Then in my application in KkPython.cxx file I have: KkPython.cxx: ------------- #include "j3kmodule.h" /* Add a builtin module, before Py_Initialize */ PyImport_AppendInittab("j3k", PyInit_j3k); I link my application with the module and get following linking error on mac although on open suse linux exactly the same procedure works fine: g++ -headerpad_max_install_names -o ../../bin/render.app/Contents/MacOS/render main.o KkPython.o -lkkbase -L/Users/kk/dev/J3K/trunk/j3ksrc/examples/render/../../lib/ -lQtSolutions_PropertyBrowser-2.5 -lpython3.1 /Users/kk/dev/J3K/trunk/j3ksrc/python/modules/build/lib.macosx-10.3-i386-3.1/j3k.so -framework OpenGL -framework AGL ld: warning in /Users/kk/dev/J3K/trunk/j3ksrc/python/modules/build/lib.macosx-10.3-i386-3.1/j3k.so, file is not of required architecture Undefined symbols: "_PyInit_j3k", referenced from: _PyInit_j3k$non_lazy_ptr in KkPython.o ld: symbol(s) not found collect2: ld returned 1 exit status make: *** [../../bin/render.app/Contents/MacOS/render] Error 1 I appreciate any hints, best greetings, Krzysztof Kobus From keesvanschaik at gmail.com Tue Jan 5 07:40:14 2010 From: keesvanschaik at gmail.com (KvS) Date: Tue, 5 Jan 2010 04:40:14 -0800 (PST) Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> Message-ID: <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> On Jan 5, 9:21?am, "alejandro" wrote: > Did you mean borderless printing? > Every printer needs his margins, some more some less. Some printers have the > ability to do borderless printing but usualy they can do it only on special > or photo paper. So you can adjust the pdf as you wish, even with no margins, > and then try to find under printer options "borderless printing". That is > why I didn't understand :-)) it is a printer thing not pdf! As much as possible "borderless", yes. Of course the printer will still apply some small margin, but that's ok. A margin of say <0.5 cm. is fine. So it's not a printer thing, I accept the (physical) limitations of the printer, but I want to avoid any extra margins due to software settings. So I need the create a pdf without/less as possible margins, and then I need Acrobat Reader to (silently) print it without any/less as possible extra margins added. I believe the first step is possible, but I am wondering if the second step is possible as well, i.e. can I adjust the options normally appearing in the Printing Dialog through Python? From ben+python at benfinney.id.au Tue Jan 5 07:52:46 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 05 Jan 2010 23:52:46 +1100 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> Message-ID: <87tyv0ww69.fsf@benfinney.id.au> r0g writes: > Paul Rudin wrote: > > Doesn't python just return a single result? (I know it can be a > > tuple and assignment statements will unpack a tuple for you.) > > Yes, it returns a tuple if you return more than one value, it just has > a lovely syntax for it. No, there is nothing inherent to the ?return? statement for dealing with multiple values. The ?return? statement *always* returns a single value: whatever value you specify as the argument to the statement (or the value ?None? if no argument is specified. If you specify a tuple ? and Python has a nice syntax for creating a literal tuple ? that's the single value the statement will use. -- \ ?There's no excuse to be bored. Sad, yes. Angry, yes. | `\ Depressed, yes. Crazy, yes. But there's no excuse for boredom, | _o__) ever.? ?Viggo Mortensen | Ben Finney From chris.gonnerman at newcenturycomputers.net Tue Jan 5 07:56:02 2010 From: chris.gonnerman at newcenturycomputers.net (Chris Gonnerman) Date: Tue, 05 Jan 2010 06:56:02 -0600 Subject: [Python] Re: Printing plain text with exact positioning on Windows In-Reply-To: <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> Message-ID: <4B4336E2.9090908@newcenturycomputers.net> KvS wrote: > ... can I adjust the options normally appearing in > the Printing Dialog through Python? > Yes, if you use my method or my module, as I gave in my previous post. If you use Adobe Reader to print, I'm not sure how to automate the print settings. From aioe.org at technicalbloke.com Tue Jan 5 08:06:20 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 13:06:20 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> Message-ID: Dave Angel wrote: > > > r0g wrote: >> >> >> Maybe, although I recently learned on here that one can't rely on assert >> statements in production code, their intended use is to aid debugging >> and testing really. >> >> > Hopefully, what you learned is that you can't use assert() in production > code to validate user data. It's fine to use it to validate program > logic, because that shouldn't still need testing in production. > > > > DaveA Well maybe I didn't quite get it then, could you explain a bit further? My understanding was that asserts aren't executed at all if python is started with the -O or -OO option, or run through an optimizer. If that's the case how can you expect it to validate anything at all in production? Do you mean for debugging in situ or something? Could you maybe give me an example scenario to illustrate your point? Cheers, Roger. From roy at panix.com Tue Jan 5 08:15:55 2010 From: roy at panix.com (Roy Smith) Date: Tue, 05 Jan 2010 08:15:55 -0500 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> Message-ID: In article , r0g wrote: > No, but that's why I try not to use languages where you can only return > a single result, I always found that an arbitrary and annoying > constraint to have. I leads to ugly practices like "magic" return values > in C or explicitly packing things into hashtables like PHP, yuk! Python only lets you return a single result, just like C or C++. The difference is that in Python it's trivial to build tuples on the fly and return them. About the closest you get to that in C++ is std::pair, and that's about 5 bucks short and a week late. From steve at REMOVE-THIS-cybersource.com.au Tue Jan 5 09:02:50 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 05 Jan 2010 14:02:50 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> Message-ID: <03533377$0$1336$c3e8da3@news.astraweb.com> On Tue, 05 Jan 2010 13:06:20 +0000, r0g wrote: > Dave Angel wrote: >> >> >> r0g wrote: >>> >>> >>> Maybe, although I recently learned on here that one can't rely on >>> assert >>> statements in production code, their intended use is to aid debugging >>> and testing really. >>> >>> >> Hopefully, what you learned is that you can't use assert() in >> production code to validate user data. It's fine to use it to validate >> program logic, because that shouldn't still need testing in production. >> >> >> >> DaveA > > > > Well maybe I didn't quite get it then, could you explain a bit further? > > My understanding was that asserts aren't executed at all if python is > started with the -O or -OO option, Correct. > or run through an optimizer. I don't know what you mean by that. > If > that's the case how can you expect it to validate anything at all in > production? The asserts still operate so long as you don't use the -O switch. > Do you mean for debugging in situ or something? Could you > maybe give me an example scenario to illustrate your point? There are at least two sorts of validation that you will generally need to perform: validating user data, and validating your program logic. You *always* need to validate user data (contents of user-editable config files, command line arguments, data files, text they type into fields, etc.) because you have no control over what they put into that. So you shouldn't use assert for validating user data except for quick-and-dirty scripts you intend to use once and throw away. Program logic, on the other hand, theoretically shouldn't need to be validated at all, because we, the programmers, are very clever and naturally never make mistakes. Since we never make mistakes, any logic validation we do is pointless and a waste of time, and therefore we should be able to optimise it away to save time. *cough* Since in reality we're not that clever and do make mistakes, we actually do want to do some such program validation, but with the option to optimise it away. Hence the assert statement. So, a totally made-up example: def function(x, y): if x < 0: raise ValueError("x must be zero or positive") if y > 0: raise ValueError("y must be zero or negative") z = x*y assert z < 0, "expected product of +ve and -ve number to be -ve" return 1.0/(z-1) This example cunningly demonstrates: (1) Using explicit test-and-raise for ensuring that user-supplied arguments are always validated; (2) Using an assertion to test your program logic; (3) That the assertion in fact will catch an error in the program logic, since if you pass x or y equal to zero, the assertion will fail. Any time you are tempted to write a comment saying "This can't happen, but we check for it just in case", that is a perfect candidate for an assertion. Since it can't happen, it doesn't matter if it goes away with the -O flag; but since we're imperfect, and we want to cover ourselves just in case it does happen, we perform the test when not optimized. >From my own code, I have a global constant: UNICODE_NUMERALS = u'\uff10\uff11\uff12\uff13\uff14\uff15\uff16\uff17 \uff18\uff19' And then to make sure I haven't missed any: assert len(UNICODE_NUMERALS) == 10 In another function, I validate a mapping {key:value} to ensure that all the values are unique: seen_values = set() for k,v in mapping.items(): if v in seen_values: raise ValueError('duplicate value %s' % k) seen_values.add(v) # If we get here without error, then the mapping contains no # duplicate values. assert len(seen_values) == len(mapping) The assertion acts as a double-check on my logic, not the data. If my logic is wrong (perhaps there is a way to get past the for-loop while there is a duplicate?) then the assertion will catch it. -- Steven From me4 at privacy.net Tue Jan 5 09:04:56 2010 From: me4 at privacy.net (=?iso-8859-1?Q?Jens_M=FCller?=) Date: Tue, 5 Jan 2010 15:04:56 +0100 Subject: Speeding up network access: threading? In-Reply-To: References: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <4b4365b2$0$6568$9b4e6d93@newsspool4.arcor-online.net> Hello, > The fairly obvious thing to do is use a queue.queue for tasks and another > for results and a pool of threads that read, fetch, and write. Thanks, indeed. Is a list thrad-safe or do I need to lock when adding the results of my worker threads to a list? The order of the elements in the list does not matter. Jens From starglider.dev at gmail.com Tue Jan 5 09:09:43 2010 From: starglider.dev at gmail.com (starglider develop) Date: Tue, 5 Jan 2010 14:09:43 +0000 Subject: Talking with ebay using easyBay Message-ID: Hi, I made an application to manage auctions with easyBay my problem is that easybay site is down and there is any other source of information in the net regarding the packadge. Any one has the docs? or know how to make a search? Thank you in advance. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eight32 at gmail.com Tue Jan 5 09:15:52 2010 From: eight32 at gmail.com (Stuart Murray-Smith) Date: Tue, 5 Jan 2010 16:15:52 +0200 Subject: Python books, literature etc Message-ID: Greetings list I can code in Python (strong beginner), and would like to read more books and/or online resources. Could someone please point out any good books, websites, tutorials etc to help me get to the next level. Your help && insight highly appreciated :) Stuart From darcy at druid.net Tue Jan 5 09:19:59 2010 From: darcy at druid.net (D'Arcy J.M. Cain) Date: Tue, 5 Jan 2010 09:19:59 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <03533377$0$1336$c3e8da3@news.astraweb.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> <03533377$0$1336$c3e8da3@news.astraweb.com> Message-ID: <20100105091959.ce6bf039.darcy@druid.net> On 05 Jan 2010 14:02:50 GMT Steven D'Aprano wrote: > shouldn't use assert for validating user data except for quick-and-dirty > scripts you intend to use once and throw away. A mythcial beast that has yet to be spotted in the wild. -- D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner. From aioe.org at technicalbloke.com Tue Jan 5 09:21:05 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 14:21:05 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> Message-ID: Ben Finney wrote: > r0g writes: > >> Paul Rudin wrote: >>> Doesn't python just return a single result? (I know it can be a >>> tuple and assignment statements will unpack a tuple for you.) >> Yes, it returns a tuple if you return more than one value, it just has >> a lovely syntax for it. > > No, there is nothing inherent to the ?return? statement for dealing with > multiple values. > > The ?return? statement *always* returns a single value: whatever value > you specify as the argument to the statement (or the value ?None? if no > argument is specified. If you specify a tuple ? and Python has a nice > syntax for creating a literal tuple ? that's the single value the > statement will use. > That's what I said Ben... >>> Doesn't python just return a single result? >> Yes, See how I agree that "The ?return? statement *always* returns a single value"? >> it returns a tuple if you return more than one value, it just has >> a lovely syntax for it. You're confusing literal and conceptual returns. You can tell there are two senses of return at play because the two returns have different possessives within the same sentence. Look... "Yes, IT returns a tuple" - Speaking literally about Python... return 1, 2, 3 # Python returned one value "YOU return more than one value" - Speaking conceptually from a programmers perspective. return 1, 2, 3 # I returned three values Both are valid in their own context. The second (conceptual) statement MIGHT be ambiguous if it weren't for the first literal one. Both cannot be literally true and the first one clearly IS a literal factual statement about Python so the second cannot also be interpreted as literal. So let's disgard it... >> it returns a tuple, it just has a lovely syntax for it. The first two "it"s are Python, the third could either be... (a) The construction of tuples or at a stretch... (b) The act of returning tuples i.e. some special syntax for returning tuples. You seem to think I meant (b) - I actually meant (a) Maybe I could have made that clearer but I don't think people want to read legalese and I think it takes a particular pedantic, nitpicking disposition to even spot such small ambiguities. Of course I'm now guilty of pedantry too :/ I might have let it slip had you not started your reply with the word "No", that just p***** me off. Having said that I find the mental image of you slamming your fist on the table and shouting it out loud whenever you read something you disagree with on usenet quite amusing! Roger. From n00m at narod.ru Tue Jan 5 09:22:10 2010 From: n00m at narod.ru (n00m) Date: Tue, 5 Jan 2010 06:22:10 -0800 (PST) Subject: twenty years ago Guido created Python References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> <87oclaxksq.fsf@benfinney.id.au> <877hrxy3zz.fsf@benfinney.id.au> <02d9bd25-5ae0-4de3-910d-dc36bec0fc11@m3g2000yqf.googlegroups.com> Message-ID: Stick your English into your ass From aioe.org at technicalbloke.com Tue Jan 5 09:48:46 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 14:48:46 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> <03533377$0$1336$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > On Tue, 05 Jan 2010 13:06:20 +0000, r0g wrote: >> Well maybe I didn't quite get it then, could you explain a bit further? >> >> My understanding was that asserts aren't executed at all if python is >> started with the -O or -OO option, > > Correct. > > >> or run through an optimizer. > > I don't know what you mean by that. I've never used them but I heard there are optimizers for python (psycho?). I assumed these would do everythin -O does and more, including losing the asserts. > >> If >> that's the case how can you expect it to validate anything at all in >> production? > > The asserts still operate so long as you don't use the -O switch. > >> Do you mean for debugging in situ or something? Could you >> maybe give me an example scenario to illustrate your point? > > > There are at least two sorts of validation that you will generally need > to perform: validating user data, and validating your program logic. > Cool, that's what I thought i.e. you can't rely on asserts being there so don't use them for anything critical but it's still a good idea to use them for logic/consistency checking in production code as, should you be running your production code unoptimised, it might catch something you'd otherwise miss. Thanks for responding is such detail :) Roger. From theller at python.net Tue Jan 5 10:02:59 2010 From: theller at python.net (Thomas Heller) Date: Tue, 05 Jan 2010 16:02:59 +0100 Subject: ctypes: How to call unexported functions in a dll In-Reply-To: References: Message-ID: <7qh2l3Fka8U1@mid.individual.net> Am 05.01.2010 12:19, schrieb Coert Klaver (DT): > Hi, > > I am using ctypes in python 3 on a WXP machine > > Loading a dll and using its exported functions works fine. > > Now I want to use a function in the dll that is not exported. > > In C this can be done by just casting the address in the dll of that > function to an apropriate function pointer and call it (you need to be > sure about the address). Can a similar thing be done directly with > ctypes? > > A work around I see is writing in wrapper dll in c that loads the main > dll, exports a function that calls the unexported function in the main > dll, but I don't find that an elegant solution. No need for a workaround. One solution is to first create a prototype for the function by calling WINFUNCTYPE or CFUNCTYPE, depending on the calling convention: stdcall or cdecl, then call the prototype with the address of the dll function which will return a Python callable. Demonstrated by the following script using GetProcAddress to get the address of the GetModuleHandleA win32 api function: >>> from ctypes import * >>> dll = windll.kernel32 >>> addr = dll.GetProcAddress(dll._handle, "GetModuleHandleA") >>> print hex(addr) 0x7c80b741 >>> proto = WINFUNCTYPE(c_int, c_char_p) >>> func = proto(addr) >>> func >>> func(None) 486539264 >>> hex(func(None)) '0x1d000000' >>> hex(func("python24.dll")) '0x1e000000' >>> hex(func("python.exe")) '0x1d000000' >>> Thomas From mackrackit at gmail.com Tue Jan 5 10:31:09 2010 From: mackrackit at gmail.com (Dave McCormick) Date: Tue, 05 Jan 2010 08:31:09 -0700 Subject: Dynamic text color In-Reply-To: References: Message-ID: <4B435B3D.3040405@gmail.com> John Posner wrote: > On Fri, 01 Jan 2010 21:01:04 -0500, Cousin Stanley > wrote: > > >> >> I was not familiar with the re.finditer method >> for searching strings ... > > Stanley and Dave -- > > So far, we've just been using finditer() to perform standard-string > searches (e.g. on the word "red"). Since Dave now wants to color > multiple words the same color (e.g. the words in redList), we can use > a single regular-expression search to locate *all* the words in a > list. This eliminates the need to use a "for" loop to handle the list. > Here's what I mean: > > >>> import re > >>> s = "it is neither red nor crimson, but scarlet, you see" > > ########## individual searches > > >>> [matchobj.span() for matchobj in re.finditer("red", s)] > [(14, 17)] > >>> [matchobj.span() for matchobj in re.finditer("crimson", s)] > [(22, 29)] > >>> [matchobj.span() for matchobj in re.finditer("scarlet", s)] > [(35, 42)] > > ########## one "swell foop" > > >>> redList = "red crimson scarlet".split() > >>> redList_regexp = "|".join(redList) > >>> redList_regexp > 'red|crimson|scarlet' > >>> [matchobj.span() for matchobj in re.finditer(redList_regexp, s)] > [(14, 17), (22, 29), (35, 42)] > > -John Thanks again John, This is fun!!! I made a "red.text" file to hold the "red words", they are separated by a space in the file. Still need to add the extra parameter "color" someplace. But this is what I have so far. ############## file = 'red.txt' file = open("red.txt","r") rList = file.readlines() file.close() redList = str(rList).split() blueList = "blue ball".split() greenList = "green grass".split() def get_complete_text(event): complete_text = Tbox.get("1.0", END) Tbox.tag_remove("red", "1.0", END) Tbox.tag_remove("blue", "1.0", END) Tbox.tag_remove("green", "1.0", END) ####RED######## redList_regexp = "|".join(redList) for matchobj in re.finditer(redList_regexp, complete_text): start,end = matchobj.span() Tbox.tag_add("red", "1.0 + %d chars" % start,"1.0 + %d chars" % end) Tbox.tag_config("red", foreground="red") ####BLUE####### blueList_regexp = "|".join(blueList) for matchobj in re.finditer(blueList_regexp, complete_text): start,end = matchobj.span() Tbox.tag_add("blue", "1.0 + %d chars" % start,"1.0 + %d chars" % end) Tbox.tag_config("blue", foreground="blue") ####GREEN####### greenList_regexp = "|".join(greenList) for matchobj in re.finditer(greenList_regexp, complete_text): start,end = matchobj.span() Tbox.tag_add("green", "1.0 + %d chars" % start,"1.0 + %d chars" % end) Tbox.tag_config("green", foreground="green") From davea at ieee.org Tue Jan 5 10:34:56 2010 From: davea at ieee.org (Dave Angel) Date: Tue, 05 Jan 2010 10:34:56 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> Message-ID: <4B435C20.6050105@ieee.org> r0g wrote: > Dave Angel wrote: > >> r0g wrote: >> >>> >>> >>> Maybe, although I recently learned on here that one can't rely on assert >>> statements in production code, their intended use is to aid debugging >>> and testing really. >>> >>> >>> >> Hopefully, what you learned is that you can't use assert() in production >> code to validate user data. It's fine to use it to validate program >> logic, because that shouldn't still need testing in production. >> >> >> >> DaveA >> > > > > Well maybe I didn't quite get it then, could you explain a bit further? > > My understanding was that asserts aren't executed at all if python is > started with the -O or -OO option, or run through an optimizer. If > that's the case how can you expect it to validate anything at all in > production? Do you mean for debugging in situ or something? Could you > maybe give me an example scenario to illustrate your point? > > Cheers, > > Roger. > > You understand the -O and -OO options fine. But the point is that you should not use assert() for anything that will be properly debugged before going to the user. You use if statements, and throw's to catch the error, and print to stderr, or GUI dialog boxes, or whatever mechanism you use to tell your user. But those errors are ones caused by his data, not by your buggy code. And the message tells him what's wrong with his data, not that you encountered a negative value for some low level function. I agree with Steve's pessimistic view of the state of most released software. But if you view a particular internal check as useful for production, then it should be coded in another mechanism, not in assert. Go ahead and write one, with a UI that's appropriate for your particular application. But it should do a lot more than assert does, including telling the user your contact information to call for support. def production_assert(expression, message): if not expression: dialog_box("Serious internal bug, call NNN-NNN-NNNN immediately", message) For an overly simplified example showing a user validation, and an assert : import sys def main(): try: text = raw_input("Enter your age, between 1 and 22 ") age = int(text) except ValueError, e: age = -1 if not 1 <= age <= 22: #not an assert print "Age must be between 1 and 22" print "Run program again" sys.exit(2) grade = calc_grade(age) print "Your grade is probably", grade table = [0, 0, 0, 0, 0, "K", "First", "2nd", 3] def calc_grade(age): """ calculate a probable grade value, given an i2nteger age between 1 and 22, inclusive """ assert(1 <= age <= len(table)) grade = table[age] #assume I have a fixed-length table for this return grade main() Note a few things. One I have a bug, in that the table isn't as big as the limit I'm checking for. With defensive coding, I'd have another assert for that, or even have the table size be available as a global constant (all uppers) so that everyone's in synch on the upper limit. But in any case, the test suite would be checking to make sure the code worked for 1, for 22, for a couple of values in between, and that a proper error response happened when a non-integer was entered, or one outside of the range. That all happens in separate code, not something in this file. And the test suite is run after every change to the sources, and certainly before release to production. Next, see the docstring. It establishes a precondition for the function. Since the function is called only by me (not the user), any preconditions can be checked with an assert. An assert without a supporting comment (or docstring) is almost worthless. And finally, notice that I check the user's input *before* passing it on to any uncontrolled code. So any asserts after that cannot fire, unless I have a bug which was not caught during testing. All opinions my own, of course. DaveA From benjamin.kaplan at case.edu Tue Jan 5 10:48:45 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Tue, 5 Jan 2010 10:48:45 -0500 Subject: embedded python on mac - linking problem In-Reply-To: <477262.24163.qm@web53508.mail.re2.yahoo.com> References: <477262.24163.qm@web53508.mail.re2.yahoo.com> Message-ID: On Tue, Jan 5, 2010 at 7:33 AM, Krzysztof Kobus wrote: > Hi, > > I have a problem with linking python module with my application on mac in order to make the module available in "embedded python". > > My python module is contained in ?j3kmodule.cxx file and module initialization function is exported in j3kmodule.h > > j3kmodule.h: > ------------ > PyMODINIT_FUNC PyInit_j3k(void); > > > j3kmodule.cxx: > -------------- > PyMODINIT_FUNC > PyInit_j3k(void) > ?{ > ?PyObject *m = NULL; > > ?if ((m = PyModule_Create(&j3k_module)) == NULL) > ? ?return NULL; > > ?return m; > ?} > > > Then in my application in KkPython.cxx file I have: > > KkPython.cxx: > ------------- > > #include "j3kmodule.h" > > /* Add a builtin module, before Py_Initialize */ > PyImport_AppendInittab("j3k", PyInit_j3k); > > > I link my application with the module and get following linking error on mac although on open suse linux exactly the same procedure works fine: > > g++ -headerpad_max_install_names -o ../../bin/render.app/Contents/MacOS/render main.o KkPython.o -lkkbase -L/Users/kk/dev/J3K/trunk/j3ksrc/examples/render/../../lib/ -lQtSolutions_PropertyBrowser-2.5 -lpython3.1 /Users/kk/dev/J3K/trunk/j3ksrc/python/modules/build/lib.macosx-10.3-i386-3.1/j3k.so -framework OpenGL -framework AGL > ld: warning in /Users/kk/dev/J3K/trunk/j3ksrc/python/modules/build/lib.macosx-10.3-i386-3.1/j3k.so, file is not of required architecture > Undefined symbols: > ?"_PyInit_j3k", referenced from: > ? ? ?_PyInit_j3k$non_lazy_ptr in KkPython.o > ld: symbol(s) not found > collect2: ld returned 1 exit status > make: *** [../../bin/render.app/Contents/MacOS/render] Error 1 > > > I appreciate any hints, > > best greetings, > > Krzysztof Kobus > Well, it seems that one of your files is a different architecture than the others. Based on the location, I'd say it's i386 while the rest of it would be PowerPC. You can cross-compile but you can't link an i386 library to a PowerPC library. From me4 at privacy.net Tue Jan 5 11:15:45 2010 From: me4 at privacy.net (=?iso-8859-1?Q?Jens_M=FCller?=) Date: Tue, 5 Jan 2010 17:15:45 +0100 Subject: Speeding up network access: threading? In-Reply-To: References: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <4b4365b2$1$6568$9b4e6d93@newsspool4.arcor-online.net> Hello, > The fairly obvious thing to do is use a queue.queue for tasks and another > for results and a pool of threads that read, fetch, and write. Thanks, indeed. Is a list thrad-safe or do I need to lock when adding the results of my worker threads to a list? The order of the elements in the list does not matter. Jens From keesvanschaik at gmail.com Tue Jan 5 11:22:53 2010 From: keesvanschaik at gmail.com (KvS) Date: Tue, 5 Jan 2010 08:22:53 -0800 (PST) Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> Message-ID: <38940c20-d714-4267-969a-4d9db1b4d5aa@j4g2000yqe.googlegroups.com> On Jan 5, 12:56?pm, Chris Gonnerman wrote: > KvS wrote: > > ... can I adjust the options normally appearing in > > the Printing Dialog through Python? > > Yes, if you use my method or my module, as I gave in my previous post. ? > If you use Adobe Reader to print, I'm not sure how to automate the print > settings. Thanks Chris, I'll go on and have a look. From keesvanschaik at gmail.com Tue Jan 5 11:34:20 2010 From: keesvanschaik at gmail.com (KvS) Date: Tue, 5 Jan 2010 08:34:20 -0800 (PST) Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> Message-ID: <263f9c76-17fb-4398-b65c-5b1b0007682b@j5g2000yqm.googlegroups.com> On Jan 5, 12:56?pm, Chris Gonnerman wrote: > KvS wrote: > > ... can I adjust the options normally appearing in > > the Printing Dialog through Python? > > Yes, if you use my method or my module, as I gave in my previous post. ? > If you use Adobe Reader to print, I'm not sure how to automate the print > settings. Ok, actually I quite like being able to print straightforward through your code, i.e. without any extra modules installed. I understand that sending text to the printer is in principle as simple as dc.TextOut(scale_factor * 72, -1 * scale_factor * 72, "Testing...") I didn't see you do anything with adjusting margins in the code. Does that mean that if I would e.g. do dc.TextOut(0, 0, "Testing...") the printout would appear in the upper left corner of the paper, as close to the edges as the printer is capable of? (Sorry, but I only have Ubuntu available at the moment, no Windows). From jjposner at optimum.net Tue Jan 5 11:35:22 2010 From: jjposner at optimum.net (John Posner) Date: Tue, 05 Jan 2010 11:35:22 -0500 Subject: A null program - what is it doing? In-Reply-To: References: Message-ID: <4B436A4A.1020005@optimum.net> > > No doubt a dumb question from a noob: > > The following program (a cut down version of some test code) uses no > CPU, and does not terminate: > > import sys > from PyQt4.QtCore import * > > if __name__=="__main__": > app = QCoreApplication(sys.argv) > sys.exit(app.exec_()) > > What is the program doing? What is the correct way to terminate the > execution? > Are you trying to understand QCoreApplication()? I can't help you there, since I've never used it. If you're just trying to get started with PyQt, use QApplication() instead: import sys from PyQt4.QtCore import * from PyQt4.QtGui import * if __name__=="__main__": app = QApplication(sys.argv) window = QLabel("I'm a PyQt window") window.show() sys.exit(app.exec_()) To terminate execution, just close the window by clicking the "X" in the window banner. HTH, John From phlip2005 at gmail.com Tue Jan 5 12:00:01 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 5 Jan 2010 09:00:01 -0800 (PST) Subject: Where's a DOM builder that uses the Builder Pattern to ... build DOMs? References: <04523152-a0ff-432c-a48f-2c653577e7b7@m25g2000yqc.googlegroups.com> <4b42f551$0$7625$9b4e6d93@newsspool1.arcor-online.net> Message-ID: <026df5db-bcef-424d-8cd5-56dacebf01f8@c3g2000yqd.googlegroups.com> On Jan 5, 12:16?am, Stefan Behnel wrote: > Note that there are tons of ways to generate HTML with Python. Forgot to note - I'm generating schematic XML, and I'm trying to find a way better than the Django template I started with! From python at mrabarnett.plus.com Tue Jan 5 12:02:29 2010 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 05 Jan 2010 17:02:29 +0000 Subject: Speeding up network access: threading? In-Reply-To: <4b4365b2$1$6568$9b4e6d93@newsspool4.arcor-online.net> References: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> <4b4365b2$1$6568$9b4e6d93@newsspool4.arcor-online.net> Message-ID: <4B4370A5.8060205@mrabarnett.plus.com> Jens M?ller wrote: > Hello, > >> The fairly obvious thing to do is use a queue.queue for tasks and another >> for results and a pool of threads that read, fetch, and write. > > Thanks, indeed. > > Is a list thrad-safe or do I need to lock when adding the results of my > worker threads to a list? The order of the elements in the list does not > matter. > Terry said "queue". not "list". Use the Queue class (it's thread-safe) in the "Queue" module (assuming you're using Python 2.x; in Python 3.x it's called the "queue" module). From lie.1296 at gmail.com Tue Jan 5 12:10:27 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Wed, 06 Jan 2010 04:10:27 +1100 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> <03533377$0$1336$c3e8da3@news.astraweb.com> Message-ID: <4b437282$1@dnews.tpgi.com.au> On 1/6/2010 1:48 AM, r0g wrote: > Steven D'Aprano wrote: >> On Tue, 05 Jan 2010 13:06:20 +0000, r0g wrote: >>> If >>> that's the case how can you expect it to validate anything at all in >>> production? >> >> The asserts still operate so long as you don't use the -O switch. >> >>> Do you mean for debugging in situ or something? Could you >>> maybe give me an example scenario to illustrate your point? >> >> >> There are at least two sorts of validation that you will generally need >> to perform: validating user data, and validating your program logic. >> > > > > Cool, that's what I thought i.e. you can't rely on asserts being there > so don't use them for anything critical but it's still a good idea to > use them for logic/consistency checking in production code as, should > you be running your production code unoptimised, it might catch > something you'd otherwise miss. Steven described the traditional approach to using assertions; another approach to when to use assertion is the one inspired by Design-by-Contract paradigm. DbC extends the traditional approach by focusing on writing a contract (instead of writing assertions) and generating assertions[1] to validate the contract. Just like assertions, these contracts are meant to be removed in production releases. In Design-by-Contract, only codes that interacts with the outer-world (e.g. getting user/file/network input, etc) need to do any sort of validations. Codes that doesn't interact directly with outside world only need to have a "contract" and simplified by *not* needing argument checking, since the function relies on the caller obeying the contract[2] and never calling it with an invalid input. DbC uses assertions[1] spuriously, unlike the traditional approach which is much more conservative when using assertions. [1] or explicit language support which is just syntax sugar for assertions [2] of course, on a debug release, the contract validation code will still be enforced to catch logic/consistency bugs that causes the violation From solipsis at pitrou.net Tue Jan 5 12:43:06 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 5 Jan 2010 17:43:06 +0000 (UTC) Subject: Speeding up network access: threading? References: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> <4b4365b2$0$6568$9b4e6d93@newsspool4.arcor-online.net> Message-ID: Le Tue, 05 Jan 2010 15:04:56 +0100, Jens M?ller a ?crit?: > > Is a list thrad-safe or do I need to lock when adding the results of my > worker threads to a list? The order of the elements in the list does not > matter. The built-in list type is thread-safe, but is doesn't provide the waiting features that queue.Queue provides. Regards Antoine. From steve at holdenweb.com Tue Jan 5 12:44:40 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 05 Jan 2010 12:44:40 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <20100105091959.ce6bf039.darcy@druid.net> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> <03533377$0$1336$c3e8da3@news.astraweb.com> <20100105091959.ce6bf039.darcy@druid.net> Message-ID: <4B437A88.1070604@holdenweb.com> D'Arcy J.M. Cain wrote: > On 05 Jan 2010 14:02:50 GMT > Steven D'Aprano wrote: >> shouldn't use assert for validating user data except for quick-and-dirty >> scripts you intend to use once and throw away. > > A mythcial beast that has yet to be spotted in the wild. > Not true (he wrote, picking nits). Such programs are written all the time. The fact that they invariably get used more often than intended doesn't negate the intentions of the author. ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Tue Jan 5 12:44:40 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 05 Jan 2010 12:44:40 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <20100105091959.ce6bf039.darcy@druid.net> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> <03533377$0$1336$c3e8da3@news.astraweb.com> <20100105091959.ce6bf039.darcy@druid.net> Message-ID: <4B437A88.1070604@holdenweb.com> D'Arcy J.M. Cain wrote: > On 05 Jan 2010 14:02:50 GMT > Steven D'Aprano wrote: >> shouldn't use assert for validating user data except for quick-and-dirty >> scripts you intend to use once and throw away. > > A mythcial beast that has yet to be spotted in the wild. > Not true (he wrote, picking nits). Such programs are written all the time. The fact that they invariably get used more often than intended doesn't negate the intentions of the author. ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From jjposner at optimum.net Tue Jan 5 12:53:01 2010 From: jjposner at optimum.net (John Posner) Date: Tue, 05 Jan 2010 12:53:01 -0500 Subject: Dynamic text color References: Message-ID: On Tue, 05 Jan 2010 10:31:09 -0500, Dave McCormick wrote: > ... But this is what I have so far. > ############## > file = 'red.txt' > file = open("red.txt","r") > rList = file.readlines() > file.close() > redList = str(rList).split() Dave, you're doing exactly the right thing: gradually expanding your program, to provide more functionality and to learn more about the available programming tools. It's also very good that you take care to close() the file after processing it. Now for the bad news ... 1. Don't use "file" as a variable name -- it's a built-in object type. (Some people don't like the fact that Python allows you to redefine such "reserved words".) 2. It's probably not the best idea to use a single variable (you use "file") to do double-duty: to hold the name of a file, and to hold the open-file object returned by the open() function. It's perfectly legal, but it hides information that might be useful when you're debugging a program. This is better: fname = 'red.txt' inpf = open(fname, "r") 3. It might be better to use read() rather than readlines() to process the "red.txt" file. It depends on what that file is supposed to contain. For example, if you expect "red.txt" to contain exactly one line, which has one or more words, you can process the open-file object like this: file_contents = inpf.read() redList = file_contents.split() ... or ... redList = inpf.read().split() It's certainly a mistake to use the expression "str(rList).split()". Using str() to convert the list "rList" into a string creates a mess that includes square-bracket characters. Did this actually work for you? Best, John From vicente.soler at gmail.com Tue Jan 5 13:12:00 2010 From: vicente.soler at gmail.com (vsoler) Date: Tue, 5 Jan 2010 10:12:00 -0800 (PST) Subject: parsing an Excel formula with the re module Message-ID: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> Hello, I am acessing an Excel file by means of Win 32 COM technology. For a given cell, I am able to read its formula. I want to make a map of how cells reference one another, how different sheets reference one another, how workbooks reference one another, etc. Hence, I need to parse Excel formulas. Can I do it by means only of re (regular expressions)? I know that for simple formulas such as "=3*A7+5" it is indeed possible. What about complex for formulas that include functions, sheet names and possibly other *.xls files? For example "=Book1!A5+8" should be parsed into ["=","Book1", "!", "A5","+","8"] Can anybody help? Any suggestions? Vicente Soler From keesvanschaik at gmail.com Tue Jan 5 13:19:09 2010 From: keesvanschaik at gmail.com (KvS) Date: Tue, 5 Jan 2010 10:19:09 -0800 (PST) Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> Message-ID: On Jan 5, 12:56?pm, Chris Gonnerman wrote: > KvS wrote: > > ... can I adjust the options normally appearing in > > the Printing Dialog through Python? > > Yes, if you use my method or my module, as I gave in my previous post. ? > If you use Adobe Reader to print, I'm not sure how to automate the print > settings. Sorry, one more. I completely forgot it's not exactly plain text, but occasionally also a limited number of non-ASCII characters (accents in names etc.). Would this be possible through your method? From mensanator at aol.com Tue Jan 5 13:24:39 2010 From: mensanator at aol.com (Mensanator) Date: Tue, 5 Jan 2010 10:24:39 -0800 (PST) Subject: it gets worse (was: How do you configure IDLE on a Mac...) References: <0d70cb54-3d77-4176-b621-e764ecf613b7@26g2000yqo.googlegroups.com> Message-ID: <6672dad2-26ba-458b-8075-21bac6506179@e37g2000yqn.googlegroups.com> On Jan 5, 12:32?am, Ned Deily wrote: > In article > <0d70cb54-3d77-4176-b621-e764ecf61... at 26g2000yqo.googlegroups.com>, > > > > > > ?Mensanator wrote: > > I assume I've been using the IDLE from macports. From the command > > prompt I've > > been typing "idle". This launches a "shell" window which appears to > > have an X11 > > parent application for which there are no "preferences" applicable to > > fonts. > > > However, if I use the quick launcher from the python.org, I get a > > "shell" whose > > parent is named "IDLE"! And that one has a completely different > > preferences, > > one similar the the Windows Configure which allows me to set the font! > > > Now, if I close this shell and start IDLE from the command line again, > > I still > > get a "shell" with an X11 parent, but, lo and behold, the font has > > changed to > > what I had previously set with the IDLE parent. > > > Course, I can't import gmpy, cause the python.org version can't > > compile it, so I > > still have to use the macports install of 3.1, but that's ok, once I > > use > > the IDLE application to set the preferences, I can switch back to the > > X11 version and the preferences will follow. > > The prefs follow because all versions of IDLE use the same (unversioned) > directory for configuration files, ~/.idlerc/. ?In particular, the > configuration file ~/.idlerc/config-main.cfg contains, among other > things, any changes to the default font. ?So, if you're successful at > changing it in one version of IDLE, it will likely affect all versions > you have. ?Note the file is a simple ini format: > > [EditorWindow] > font = monaco > > so you can edit it by hand. Good to know. But, as the subject says... > > BTW, the python.org IDLEs and the Apple-supplied IDLEs use the > system-supplied Aqua (aka Quartz) Tk not the X11 one that MacPorts > builds by default. ?The MacPorts Tk port does have a "quartz" variant > but that doesn't yet work in 64-bit mode. So, for all practical purposes, the macports install is broken also. IDLE simply does not work in an X11 window (you think someone would have noticed that). The missing preferences is just the beginning. Apparently NONE of the menu item shortcuts work. For example, the Cut, Copy, Paste shortcuts are given as Command-X, Command-C and Command-V. But that doesn't work in an X11 window, apperently only in an Aqua Tk (parent application appears as IDLE). Of course, I can do Control-X, Control-C and Control-V to do Cut, Copy and Paste. Don't know if this works for all shortcuts, but I suppose I could just pick them from the menu (and I can bang my head against the wall while I'm at it). What do you think, suppose I copy the gmpy built with the macports install over to the directory where the python.org version is? Would it import? If that'll work, I can switch back to using the python.org install and use it's version of IDLE. I certainly won't be needing distutils once I have a working version of gmpy. > > -- > ?Ned Deily, > ?n... at acm.org- Hide quoted text - > > - Show quoted text - From python at mrabarnett.plus.com Tue Jan 5 13:35:02 2010 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 05 Jan 2010 18:35:02 +0000 Subject: parsing an Excel formula with the re module In-Reply-To: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> Message-ID: <4B438656.7080601@mrabarnett.plus.com> vsoler wrote: > Hello, > > I am acessing an Excel file by means of Win 32 COM technology. > For a given cell, I am able to read its formula. I want to make a map > of how cells reference one another, how different sheets reference one > another, how workbooks reference one another, etc. > > Hence, I need to parse Excel formulas. Can I do it by means only of re > (regular expressions)? > > I know that for simple formulas such as "=3*A7+5" it is indeed > possible. What about complex for formulas that include functions, > sheet names and possibly other *.xls files? > > For example "=Book1!A5+8" should be parsed into ["=","Book1", "!", > "A5","+","8"] > > Can anybody help? Any suggestions? > Do you mean "how" or do you really mean "whether", ie, get a list of the other cells that are referred to by a certain cell, for example, "=3*A7+5" should give ["A7"] and "=Book1!A5+8" should give ["Book1!A5"]? From me4 at privacy.net Tue Jan 5 13:45:50 2010 From: me4 at privacy.net (=?iso-8859-1?Q?Jens_M=FCller?=) Date: Tue, 5 Jan 2010 19:45:50 +0100 Subject: Speeding up network access: threading? In-Reply-To: References: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> <4b4365b2$1$6568$9b4e6d93@newsspool4.arcor-online.net> Message-ID: <4b4388dd$0$7616$9b4e6d93@newsspool1.arcor-online.net> Hi and sorry for double posting - had mailer problems, > Terry said "queue". not "list". Use the Queue class (it's thread-safe) > in the "Queue" module (assuming you're using Python 2.x; in Python 3.x > it's called the "queue" module). Yes yes, I know. I use a queue to realize the thread pool queue, that works all right. But each worker thread calculates a result and needs to make it avaialable to the application in the main thread again. Therefore, it appends its result to a common list. This seems works as well, but I was thinking of possible conflict situations that maybe could happen when two threads append their results to that same result list at the same moment. Regards, Jens From nobody at nowhere.com Tue Jan 5 13:58:23 2010 From: nobody at nowhere.com (Nobody) Date: Tue, 05 Jan 2010 18:58:23 +0000 Subject: IOError - cannot create file (linux daemon-invoked script) References: Message-ID: On Mon, 04 Jan 2010 21:30:31 -0800, cassiope wrote: > One more tidbit observed: my last note, that it works when using > seteuid/setegid? > Well - that only applies if the daemon is running under strace (!). > It fails > if started directly by root, or if the strace session has ended, > leaving the > main body of the daemon running in its normal headless manner. > > I wonder if running under "strace -f" - might setegid/seteuid be > prevented from > having their normal effect? Possibly. The ptrace() syscall on which strace depends will fail if you try to trace a "privileged" process and you aren't root, so it's possible that a ptrace()d process will refuse to become privileged. Here, "privileged" includes a process which has changed any of its UIDs or GIDs (this prevents a normal user from tracing, killing, etc an otherwise privileged process which has switched to the user's UID for the time being). From phlip2005 at gmail.com Tue Jan 5 14:00:24 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 5 Jan 2010 11:00:24 -0800 (PST) Subject: twenty years ago Guido created Python References: Message-ID: <51564fb3-a367-4f11-b97f-5518deedbe48@a21g2000yqc.googlegroups.com> On Dec 31 2009, 2:06?pm, Steve Howell wrote: > Python is a truly awesome programming language. ?Not only is Guido a > genius language designer, but he is also a great project leader. ?What > an accomplishment. ?Congratulations to everybody who has contributed > to Python in the last two decades! The more languages you learn before getting to Smalltalk, the more awesome Smalltalk will be for you. -- Phlip From vicente.soler at gmail.com Tue Jan 5 14:00:43 2010 From: vicente.soler at gmail.com (vsoler) Date: Tue, 5 Jan 2010 11:00:43 -0800 (PST) Subject: parsing an Excel formula with the re module References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> Message-ID: <11cdf1c8-51e4-45ec-9ac4-1fd77c5e018a@e37g2000yqn.googlegroups.com> On 5 ene, 19:35, MRAB wrote: > vsoler wrote: > > Hello, > > > I am acessing an Excel file by means of Win 32 COM technology. > > For a given cell, I am able to read its formula. I want to make a map > > of how cells reference one another, how different sheets reference one > > another, how workbooks reference one another, etc. > > > Hence, I need to parse Excel formulas. Can I do it by means only of re > > (regular expressions)? > > > I know that for simple formulas such as "=3*A7+5" it is indeed > > possible. What about complex for formulas that include functions, > > sheet names and possibly other *.xls files? > > > For example ? ?"=Book1!A5+8" should be parsed into ["=","Book1", "!", > > "A5","+","8"] > > > Can anybody help? Any suggestions? > > Do you mean "how" or do you really mean "whether", ie, get a list of the > other cells that are referred to by a certain cell, for example, > "=3*A7+5" should give ["A7"] and "=Book1!A5+8" should give ["Book1!A5"]? I'd like to know how to do it, should it be possible. Vicente From jjposner at optimum.net Tue Jan 5 14:04:05 2010 From: jjposner at optimum.net (John Posner) Date: Tue, 05 Jan 2010 14:04:05 -0500 Subject: parsing an Excel formula with the re module References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> Message-ID: On Tue, 05 Jan 2010 13:12:00 -0500, vsoler wrote: > Hello, > > I am acessing an Excel file by means of Win 32 COM technology. > For a given cell, I am able to read its formula. I want to make a map > of how cells reference one another, how different sheets reference one > another, how workbooks reference one another, etc. > > Hence, I need to parse Excel formulas. Can I do it by means only of re > (regular expressions)? > > I know that for simple formulas such as "=3*A7+5" it is indeed > possible. What about complex for formulas that include functions, > sheet names and possibly other *.xls files? > > For example "=Book1!A5+8" should be parsed into ["=","Book1", "!", > "A5","+","8"] > > Can anybody help? Any suggestions? It seems like you want to recreate data structures that Excel, itself, must maintain in order to recalculate cells in the correct order. As long as you're using COM, you might be able to tap into those data structures. My 15-year-old (!) "Using Excel Visual Basic for Applications" book wasn't any help. :-( After a short Google session, I came up with one possible lead: http://www.decisionmodels.com/ Good luck! John From mensanator at aol.com Tue Jan 5 14:05:32 2010 From: mensanator at aol.com (Mensanator) Date: Tue, 5 Jan 2010 11:05:32 -0800 (PST) Subject: parsing an Excel formula with the re module References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> Message-ID: <5ef3575d-eb1d-4a15-90e0-87dd85cdc2e6@35g2000yqa.googlegroups.com> On Jan 5, 12:35?pm, MRAB wrote: > vsoler wrote: > > Hello, > > > I am acessing an Excel file by means of Win 32 COM technology. > > For a given cell, I am able to read its formula. I want to make a map > > of how cells reference one another, how different sheets reference one > > another, how workbooks reference one another, etc. > > > Hence, I need to parse Excel formulas. Can I do it by means only of re > > (regular expressions)? > > > I know that for simple formulas such as "=3*A7+5" it is indeed > > possible. What about complex for formulas that include functions, > > sheet names and possibly other *.xls files? > > > For example ? ?"=Book1!A5+8" should be parsed into ["=","Book1", "!", > > "A5","+","8"] > > > Can anybody help? Any suggestions? > > Do you mean "how" or do you really mean "whether", ie, get a list of the > other cells that are referred to by a certain cell, for example, > "=3*A7+5" should give ["A7"] and "=Book1!A5+8" should give ["Book1!A5] Ok, although "Book1" would be the default name of a workbook, with default worksheets labeled "Sheet1". "Sheet2", etc. If I had a worksheet named "Sheety" that wanted to reference a cell on "Sheetx" OF THE SAME WORKBOOK, it would be =Sheet2!A7. If the reference was to a completely different workbook (say Book1 with worksheets labeled "Sheet1", "Sheet2") then the cell might have =[Book1]Sheet1!A7. And don't forget the $'s! You may see =[Book1]Sheet1!$A$7. From neilc at norwich.edu Tue Jan 5 14:07:26 2010 From: neilc at norwich.edu (Neil Cerutti) Date: 5 Jan 2010 19:07:26 GMT Subject: Dynamic text color References: Message-ID: <7qhgveFe0fU1@mid.individual.net> On 2010-01-05, John Posner wrote: > 2. It's probably not the best idea to use a single variable > (you use "file") to do double-duty: to hold the name of a > file, and to hold the open-file object returned by the open() > function. It's perfectly legal, but it hides information that > might be useful when you're debugging a program. This is > better: > > fname = 'red.txt' > inpf = open(fname, "r") Alternatively: >>> infile = open("red.txt", "r") >>> infile.name 'red.txt' -- Neil Cerutti From nobody at nowhere.com Tue Jan 5 14:08:02 2010 From: nobody at nowhere.com (Nobody) Date: Tue, 05 Jan 2010 19:08:02 +0000 Subject: fsync() doesn't work as advertised? References: Message-ID: On Mon, 04 Jan 2010 08:09:56 -0800, Brian D wrote: > If I'm running a process in a loop that runs for a long time, I > occasionally would like to look at a log to see how it's going. > > I know about the logging module, and may yet decide to use that. > > Still, I'm troubled by how fsync() doesn't seem to work as advertised: > > http://docs.python.org/library/os.html > > "If you?re starting with a Python file object f, first do f.flush(), > and then do os.fsync(f.fileno())" The .flush() method (and the C fflush() function) causes the contents of application buffers to be sent to the OS, which basically copies the data into the OS-level buffers. fsync() causes the OS-level buffers to be written to the physical drive. File operations normally use the OS-level buffers; e.g. if one process write()s to a file and another process read()s it, the latter will see what the former has written regardless of whether the data has been written to the drive. The main reason for using fsync() is to prevent important data from being lost in the event of an unexpected reboot or power-cycle (an expected reboot via the "shutdown" or "halt" commands will flush all OS-level buffers to the drive first). Other than that, fsync() is almost invisible (I say "almost", as there are mechanisms to bypass the OS-level buffers, e.g. the O_DIRECT open() flag). From vicente.soler at gmail.com Tue Jan 5 14:09:09 2010 From: vicente.soler at gmail.com (vsoler) Date: Tue, 5 Jan 2010 11:09:09 -0800 (PST) Subject: What is the best data structure for a very simple spreadsheet? References: <82c5a9f1-ed36-40c4-83d6-2d2d33ee40bc@j24g2000yqa.googlegroups.com> Message-ID: <3b7d5cf5-2a2d-46a9-9f1b-490fac3da54b@v25g2000yqk.googlegroups.com> On 3 ene, 22:40, mdipierro wrote: > Perhaps this can be useful:http://www.web2py.com/examples/spreadsheet > > The code is in a single file with not dependencies and it does not > require web2py to run:http://code.google.com/p/web2py/source/browse/gluon/contrib/spreadshe... > > Here is a sample controller that shows you how to embed the > spreadsheet in web page:http://code.google.com/p/web2py/source/browse/applications/examples/c... > > Massimo > > On Jan 3, 5:27?am, vsoler wrote: > > > Hi, > > > Not sure this is the best group to post, but I cannot think of any > > other. > > > My application would contain a limited set of "cells" represented by > > the instances of a Cell class: > > > class Cell: > > ... > > > A1=Cell(7) > > A2=Cell(2*A1) > > A3=Cell(3*A1+A2) > > A4=Cell(A3*4) > > > Of course, A1 = 7, A2 = 14, A3 = 35 and A4 = 140 > > > Now, I somehow want to be able to show a dependency tree > > > 1 level dependency trees > > ? A1: None > > ? A2: A1 > > ? A3: A1, A2 > > ? A4: A3 > > > All levels dependency trees > > > ? A1: None > > ? A2: A1 > > ? A3: A1, A2 > > ? A4: A3, A2, A1 > > > Leaf + values dependency trees: > > > ? A1: 7 > > ? A2: A1=7, 2 > > ? A3: 3, A1=7, 2 > > ? A4: 3, A1=7, 2, 4 > > > What I'd like to know is: > > > 1) what are, in your opinion, the basic elements of the Cell class? > > 2) Do I need a parser to evaluate the formulas like ?3*A1+A2?? Can you > > recommend one library that already contains one? > > 3) Do I need a tree data structure to represent my data? would the > > tree be an attribute of the class instance? > > > I imagine a lot can be said on these questions. What I am looking for > > is some hints that help me get out of where I am now. > > > Any help is highly appreciated. > > > Vicente Soler > > There is something that I appreciate in this group, and it is the high degree of knowledge of all the participants that are helping with their answers. After studying your suggestions, I'll come back to you. Thank you very much. Vicente Soler From steve at holdenweb.com Tue Jan 5 14:11:05 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 05 Jan 2010 14:11:05 -0500 Subject: Speeding up network access: threading? In-Reply-To: <4b4388dd$0$7616$9b4e6d93@newsspool1.arcor-online.net> References: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> <4b4365b2$1$6568$9b4e6d93@newsspool4.arcor-online.net> <4b4388dd$0$7616$9b4e6d93@newsspool1.arcor-online.net> Message-ID: <4B438EC9.3010100@holdenweb.com> Jens M?ller wrote: > Hi and sorry for double posting - had mailer problems, > >> Terry said "queue". not "list". Use the Queue class (it's thread-safe) >> in the "Queue" module (assuming you're using Python 2.x; in Python 3.x >> it's called the "queue" module). > > Yes yes, I know. I use a queue to realize the thread pool queue, that > works all right. > > But each worker thread calculates a result and needs to make it > avaialable to the application in the main thread again. Therefore, it > appends its result to a common list. This seems works as well, but I was > thinking of possible conflict situations that maybe could happen when > two threads append their results to that same result list at the same > moment. > If you don't need to take anything off the list ever, just create a separate thread that reads items from an output Queue and appends them to the list. If you *do* take them off, then use a Queue. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Tue Jan 5 14:11:05 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 05 Jan 2010 14:11:05 -0500 Subject: Speeding up network access: threading? In-Reply-To: <4b4388dd$0$7616$9b4e6d93@newsspool1.arcor-online.net> References: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> <4b4365b2$1$6568$9b4e6d93@newsspool4.arcor-online.net> <4b4388dd$0$7616$9b4e6d93@newsspool1.arcor-online.net> Message-ID: <4B438EC9.3010100@holdenweb.com> Jens M?ller wrote: > Hi and sorry for double posting - had mailer problems, > >> Terry said "queue". not "list". Use the Queue class (it's thread-safe) >> in the "Queue" module (assuming you're using Python 2.x; in Python 3.x >> it's called the "queue" module). > > Yes yes, I know. I use a queue to realize the thread pool queue, that > works all right. > > But each worker thread calculates a result and needs to make it > avaialable to the application in the main thread again. Therefore, it > appends its result to a common list. This seems works as well, but I was > thinking of possible conflict situations that maybe could happen when > two threads append their results to that same result list at the same > moment. > If you don't need to take anything off the list ever, just create a separate thread that reads items from an output Queue and appends them to the list. If you *do* take them off, then use a Queue. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Tue Jan 5 14:15:02 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 05 Jan 2010 14:15:02 -0500 Subject: twenty years ago Guido created Python In-Reply-To: <51564fb3-a367-4f11-b97f-5518deedbe48@a21g2000yqc.googlegroups.com> References: <51564fb3-a367-4f11-b97f-5518deedbe48@a21g2000yqc.googlegroups.com> Message-ID: <4B438FB6.9050003@holdenweb.com> Phlip wrote: > On Dec 31 2009, 2:06 pm, Steve Howell wrote: > >> Python is a truly awesome programming language. Not only is Guido a >> genius language designer, but he is also a great project leader. What >> an accomplishment. Congratulations to everybody who has contributed >> to Python in the last two decades! > > The more languages you learn before getting to Smalltalk, the more > awesome Smalltalk will be for you. > After implementing SmallTalk I more or less gave up OO programming for ten years, resuming it only after I met Python. SmallTalk didn't seem that awesome to me. Though for its time it was an incredible system, its insistence on a SmallTalk-only VM environment seemed a little solipsistic. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From nobody at nowhere.com Tue Jan 5 14:16:32 2010 From: nobody at nowhere.com (Nobody) Date: Tue, 05 Jan 2010 19:16:32 +0000 Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> Message-ID: On Tue, 05 Jan 2010 04:40:14 -0800, KvS wrote: >> Did you mean borderless printing? >> Every printer needs his margins, some more some less. Some printers have the >> ability to do borderless printing but usualy they can do it only on special >> or photo paper. So you can adjust the pdf as you wish, even with no margins, >> and then try to find under printer options "borderless printing". That is >> why I didn't understand :-)) it is a printer thing not pdf! > > As much as possible "borderless", yes. Of course the printer will > still apply some small margin, but that's ok. A margin of say <0.5 cm. > is fine. So it's not a printer thing, I accept the (physical) > limitations of the printer, but I want to avoid any extra margins due > to software settings. "Hardcopy" document formats such as PostScript and PDF use positions relative to the edges of the page, not the margins. From python at mrabarnett.plus.com Tue Jan 5 14:20:09 2010 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 05 Jan 2010 19:20:09 +0000 Subject: parsing an Excel formula with the re module In-Reply-To: <11cdf1c8-51e4-45ec-9ac4-1fd77c5e018a@e37g2000yqn.googlegroups.com> References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> <11cdf1c8-51e4-45ec-9ac4-1fd77c5e018a@e37g2000yqn.googlegroups.com> Message-ID: <4B4390E9.1070900@mrabarnett.plus.com> vsoler wrote: > On 5 ene, 19:35, MRAB wrote: >> vsoler wrote: >>> Hello, >>> I am acessing an Excel file by means of Win 32 COM technology. >>> For a given cell, I am able to read its formula. I want to make a map >>> of how cells reference one another, how different sheets reference one >>> another, how workbooks reference one another, etc. >>> Hence, I need to parse Excel formulas. Can I do it by means only of re >>> (regular expressions)? >>> I know that for simple formulas such as "=3*A7+5" it is indeed >>> possible. What about complex for formulas that include functions, >>> sheet names and possibly other *.xls files? >>> For example "=Book1!A5+8" should be parsed into ["=","Book1", "!", >>> "A5","+","8"] >>> Can anybody help? Any suggestions? >> Do you mean "how" or do you really mean "whether", ie, get a list of the >> other cells that are referred to by a certain cell, for example, >> "=3*A7+5" should give ["A7"] and "=Book1!A5+8" should give ["Book1!A5"]? > > I'd like to know how to do it, should it be possible. > Something like this should work: references = re.findall(r"\b((?:\w+!)?[A-Za-z]+\d+)\b", formula) From vicente.soler at gmail.com Tue Jan 5 14:21:50 2010 From: vicente.soler at gmail.com (vsoler) Date: Tue, 5 Jan 2010 11:21:50 -0800 (PST) Subject: parsing an Excel formula with the re module References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> <5ef3575d-eb1d-4a15-90e0-87dd85cdc2e6@35g2000yqa.googlegroups.com> Message-ID: <6473e930-c5f4-423f-b78b-aae63f2aa794@k17g2000yqh.googlegroups.com> On 5 ene, 20:05, Mensanator wrote: > On Jan 5, 12:35?pm, MRAB wrote: > > > > > vsoler wrote: > > > Hello, > > > > I am acessing an Excel file by means of Win 32 COM technology. > > > For a given cell, I am able to read its formula. I want to make a map > > > of how cells reference one another, how different sheets reference one > > > another, how workbooks reference one another, etc. > > > > Hence, I need to parse Excel formulas. Can I do it by means only of re > > > (regular expressions)? > > > > I know that for simple formulas such as "=3*A7+5" it is indeed > > > possible. What about complex for formulas that include functions, > > > sheet names and possibly other *.xls files? > > > > For example ? ?"=Book1!A5+8" should be parsed into ["=","Book1", "!", > > > "A5","+","8"] > > > > Can anybody help? Any suggestions? > > > Do you mean "how" or do you really mean "whether", ie, get a list of the > > other cells that are referred to by a certain cell, for example, > > "=3*A7+5" should give ["A7"] and "=Book1!A5+8" should give ["Book1!A5] > > Ok, although "Book1" would be the default name of a workbook, with > default > worksheets labeled "Sheet1". "Sheet2", etc. > > If I had a worksheet named "Sheety" that wanted to reference a cell on > "Sheetx" > OF THE SAME WORKBOOK, it would be =Sheet2!A7. If the reference was to > a completely > different workbook (say Book1 with worksheets labeled "Sheet1", > "Sheet2") then > the cell might have =[Book1]Sheet1!A7. > > And don't forget the $'s! You may see =[Book1]Sheet1!$A$7. Yes, Mensanator, but... what re should I use? I'm looking for the re statement. No doubt you can help! Thank you. From aps.it2000 at gmail.com Tue Jan 5 14:31:47 2010 From: aps.it2000 at gmail.com (aung paing Soe) Date: Tue, 5 Jan 2010 11:31:47 -0800 Subject: Fwd: I would like to install Python on my 64 bit Win 7 In-Reply-To: References: Message-ID: ---------- Forwarded message ---------- From: aung paing Soe Date: Tue, Jan 5, 2010 at 11:27 AM Subject: I would like to install Python on my 64 bit Win 7 To: webmaster at python.org Hello , I would like to study about Python Programming . So I want to install Python . But my laptop is Window 7 64-bit home basic . So please give me a advice how to install Python in my 64 bit computer. I really want to study python programming . I am looking forward your reply. Thank you very much Yours, Beginner -------------- next part -------------- An HTML attachment was scrubbed... URL: From keesvanschaik at gmail.com Tue Jan 5 14:40:25 2010 From: keesvanschaik at gmail.com (KvS) Date: Tue, 5 Jan 2010 11:40:25 -0800 (PST) Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> Message-ID: <96ea39f2-1fb6-4b03-b206-d4c1445ea021@v25g2000yqk.googlegroups.com> On Jan 5, 7:16?pm, Nobody wrote: > On Tue, 05 Jan 2010 04:40:14 -0800, KvS wrote: > >> Did you mean borderless printing? > >> Every printer needs his margins, some more some less. Some printers have the > >> ability to do borderless printing but usualy they can do it only on special > >> or photo paper. So you can adjust the pdf as you wish, even with no margins, > >> and then try to find under printer options "borderless printing". That is > >> why I didn't understand :-)) it is a printer thing not pdf! > > > As much as possible "borderless", yes. Of course the printer will > > still apply some small margin, but that's ok. A margin of say <0.5 cm. > > is fine. So it's not a printer thing, I accept the (physical) > > limitations of the printer, but I want to avoid any extra margins due > > to software settings. > > "Hardcopy" document formats such as PostScript and PDF use positions > relative to the edges of the page, not the margins. Right. Still, Acrobat Reader by default scales the contents to fit on a page and creates some margins by doing so, no? So if my text is close to the left and right edges, as I want, it will get scaled and extra margins will occur. Avoiding this still requires me to be able to turn off this scaling in the printing preferences somehow programmatically, so it doesn't seem to make the problem easier? From python.list at tim.thechases.com Tue Jan 5 14:49:13 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Tue, 05 Jan 2010 13:49:13 -0600 Subject: parsing an Excel formula with the re module In-Reply-To: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> Message-ID: <4B4397B9.7000804@tim.thechases.com> vsoler wrote: > Hence, I need to parse Excel formulas. Can I do it by means only of re > (regular expressions)? > > I know that for simple formulas such as "=3*A7+5" it is indeed > possible. What about complex for formulas that include functions, > sheet names and possibly other *.xls files? Where things start getting ugly is when you have nested function calls, such as =if(Sum(A1:A25)>42,Min(B1:B25), if(Sum(C1:C25)>3.14, (Min(C1:C25)+3)*18,Max(B1:B25))) Regular expressions don't do well with nested parens (especially arbitrarily-nesting-depth such as are possible), so I'd suggest going for a full-blown parsing solution like pyparsing. If you have fair control over what can be contained in the formulas and you know they won't contain nested parens/functions, you might be able to formulate some sort of "kinda, sorta, maybe parses some forms of formulas" regexp. -tkc From vicente.soler at gmail.com Tue Jan 5 14:54:00 2010 From: vicente.soler at gmail.com (vsoler) Date: Tue, 5 Jan 2010 11:54:00 -0800 (PST) Subject: parsing an Excel formula with the re module References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> <5ef3575d-eb1d-4a15-90e0-87dd85cdc2e6@35g2000yqa.googlegroups.com> <6473e930-c5f4-423f-b78b-aae63f2aa794@k17g2000yqh.googlegroups.com> Message-ID: On 5 ene, 20:21, vsoler wrote: > On 5 ene, 20:05, Mensanator wrote: > > > > > On Jan 5, 12:35?pm, MRAB wrote: > > > > vsoler wrote: > > > > Hello, > > > > > I am acessing an Excel file by means of Win 32 COM technology. > > > > For a given cell, I am able to read its formula. I want to make a map > > > > of how cells reference one another, how different sheets reference one > > > > another, how workbooks reference one another, etc. > > > > > Hence, I need to parse Excel formulas. Can I do it by means only of re > > > > (regular expressions)? > > > > > I know that for simple formulas such as "=3*A7+5" it is indeed > > > > possible. What about complex for formulas that include functions, > > > > sheet names and possibly other *.xls files? > > > > > For example ? ?"=Book1!A5+8" should be parsed into ["=","Book1", "!", > > > > "A5","+","8"] > > > > > Can anybody help? Any suggestions? > > > > Do you mean "how" or do you really mean "whether", ie, get a list of the > > > other cells that are referred to by a certain cell, for example, > > > "=3*A7+5" should give ["A7"] and "=Book1!A5+8" should give ["Book1!A5] > > > Ok, although "Book1" would be the default name of a workbook, with > > default > > worksheets labeled "Sheet1". "Sheet2", etc. > > > If I had a worksheet named "Sheety" that wanted to reference a cell on > > "Sheetx" > > OF THE SAME WORKBOOK, it would be =Sheet2!A7. If the reference was to > > a completely > > different workbook (say Book1 with worksheets labeled "Sheet1", > > "Sheet2") then > > the cell might have =[Book1]Sheet1!A7. > > > And don't forget the $'s! You may see =[Book1]Sheet1!$A$7. > > Yes, Mensanator, but... ?what re should I use? I'm looking for the re > statement. No doubt you can help! > > Thank you. Let me give you an example: >>> import re >>> re.split("([^0-9])", "123+456*/") [?123?, ?+?, ?456?, ?*?, ??, ?/?, ??] I find it excellent that one single statement is able to do a lexical analysis of an expression! If the expression contains variables, such as A12 or B9, I can try another re expression. Which one should I use? And if my expression contains parenthesis? And the sin() function? Vicente Soler From phlip2005 at gmail.com Tue Jan 5 14:58:36 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 5 Jan 2010 11:58:36 -0800 (PST) Subject: please help shrink this each_with_index() implementation Message-ID: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> Hypo Nt: def each_with_index(seq): index = 0 result = [] for item in seq: result.append([item, index]) index += 1 return result My Pythonic sequencing skills are obviously feeble. Can anything think of a way to write that in fewer lines? -- Phlip http://c2.com/cgi/wiki?MoreliaViridis From mackrackit at gmail.com Tue Jan 5 15:03:13 2010 From: mackrackit at gmail.com (Dave McCormick) Date: Tue, 05 Jan 2010 13:03:13 -0700 Subject: Fwd: I would like to install Python on my 64 bit Win 7 In-Reply-To: References: Message-ID: <4B439B01.50407@gmail.com> An HTML attachment was scrubbed... URL: From mensanator at aol.com Tue Jan 5 15:05:06 2010 From: mensanator at aol.com (Mensanator) Date: Tue, 5 Jan 2010 12:05:06 -0800 (PST) Subject: twenty years ago Guido created Python References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> <87oclaxksq.fsf@benfinney.id.au> <877hrxy3zz.fsf@benfinney.id.au> <02d9bd25-5ae0-4de3-910d-dc36bec0fc11@m3g2000yqf.googlegroups.com> Message-ID: <29699581-f804-4ae0-9721-1d415ce1065b@h9g2000yqa.googlegroups.com> On Jan 5, 8:22?am, n00m wrote: > Stick your English into your ass Most people would say "up your ass". And use a period at the end of the sentence. Got any more funny insults? From mackrackit at gmail.com Tue Jan 5 15:08:04 2010 From: mackrackit at gmail.com (Dave McCormick) Date: Tue, 05 Jan 2010 13:08:04 -0700 Subject: Dynamic text color In-Reply-To: References: Message-ID: <4B439C24.9050205@gmail.com> John Posner wrote: > > Dave, you're doing exactly the right thing: gradually expanding your > program, to provide more functionality and to learn more about the > available programming tools. It's also very good that you take care to > close() the file after processing it. Now for the bad news ... Seems like there is always bad news :) > > 1. Don't use "file" as a variable name -- it's a built-in object type. > (Some people don't like the fact that Python allows you to redefine > such "reserved words".) > > 2. It's probably not the best idea to use a single variable (you use > "file") to do double-duty: to hold the name of a file, and to hold the > open-file object returned by the open() function. It's perfectly > legal, but it hides information that might be useful when you're > debugging a program. This is better: > > 3. It might be better to use read() rather than readlines() to process > the "red.txt" file. It depends on what that file is supposed to > contain. For example, if you expect "red.txt" to contain exactly one > line, which has one or more words, you can process the open-file > object like this: All noted and fixed. > > It's certainly a mistake to use the expression "str(rList).split()". > Using str() to convert the list "rList" into a string creates a mess > that includes square-bracket characters. Did this actually work for you? It sort of worked. With one color file it seemed fine but after I posted I added another color file and things fell apart. Now with the above fixes it works with three colors from three files. When the list are printed to the shell the list look like this: redList ['red', 'dog', 'apple', '#'] blueList ['blue', 'ball', 'berry'] greenList ['green', 'grass', 'do'] But another problem is noticed. It does not matter if the list is built in code or from a file. If dog is entered, "do" will be green with the "g" being red. Back to the drawing board..... Here is the complete code" ###### from Tkinter import * import re RFfile = 'red.txt' inpRF = open(RFfile,"r") rList = inpRF.read() inpRF.close() BFfile = 'blue.txt' inpBF = open(BFfile,"r") bList = inpBF.read() inpBF.close() GFfile = 'green.txt' inpGF = open(GFfile,"r") gList = inpGF.read() inpGF.close() def get_complete_text(event): complete_text = Tbox.get("1.0", END) redList = str(rList).split() blueList = str(bList).split() greenList = str(gList).split() print "redList",redList print "blueList",blueList print "greenList",greenList Tbox.tag_remove("red", "1.0", END) Tbox.tag_remove("blue", "1.0", END) Tbox.tag_remove("green", "1.0", END) ####RED######## redList_regexp = "|".join(redList) for matchobj in re.finditer(redList_regexp, complete_text): start,end = matchobj.span() Tbox.tag_add("red", "1.0 + %d chars" % start,"1.0 + %d chars" % end) Tbox.tag_config("red", foreground="red") ####BLUE####### blueList_regexp = "|".join(blueList) for matchobj in re.finditer(blueList_regexp, complete_text): start,end = matchobj.span() Tbox.tag_add("blue", "1.0 + %d chars" % start,"1.0 + %d chars" % end) Tbox.tag_config("blue", foreground="blue") ####GREEN####### greenList_regexp = "|".join(greenList) for matchobj in re.finditer(greenList_regexp, complete_text): start,end = matchobj.span() Tbox.tag_add("green", "1.0 + %d chars" % start,"1.0 + %d chars" % end) Tbox.tag_config("green", foreground="green") root = Tk() Tbox = Text(root, width=40, height=15, wrap=CHAR, font="Times 14 bold", bg="#dddddd") Tbox.pack() Tbox.bind("", get_complete_text) Tbox.focus() root.mainloop() #### Thanks again, Dave From benjamin.kaplan at case.edu Tue Jan 5 15:13:24 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Tue, 5 Jan 2010 15:13:24 -0500 Subject: Fwd: I would like to install Python on my 64 bit Win 7 In-Reply-To: <4B439B01.50407@gmail.com> References: <4B439B01.50407@gmail.com> Message-ID: On Tue, Jan 5, 2010 at 3:03 PM, Dave McCormick wrote: > > > aung paing Soe wrote: > > ---------- Forwarded message ---------- > From: aung paing Soe > Date: Tue, Jan 5, 2010 at 11:27 AM > Subject: I would like to install Python on my 64 bit Win 7 > To: webmaster at python.org > > > Hello , > ????????? I would like to study about Python Programming . So I want to > install Python . > But my laptop is Window 7 64-bit home basic? . > So please give me a advice how to install Python in my 64 bit computer. > I really want to study python programming . > I am looking forward your reply. > Thank you very much > > Yours, > Beginner > > I am using WIN7 64 bit also. > Go to python.org and look for > http://www.python.org/download/releases/2.5.4/ > python-2.5.4.amd64.msi > > Dave Unless you need a package that requires Python 2.5, it's a good idea to use Python 2.6 instead. 2.5 isn't getting any more bug fixes and 2.6 includes quite a few new features. http://python.org/download/releases/2.6.4/ > > -- > http://mail.python.org/mailman/listinfo/python-list > > From akean at clear.net.nz Tue Jan 5 15:17:11 2010 From: akean at clear.net.nz (akean) Date: Tue, 5 Jan 2010 12:17:11 -0800 (PST) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> Message-ID: On Jan 6, 8:58?am, Phlip wrote: > Hypo Nt: > > def each_with_index(seq): > ? ? index = 0 > ? ? result = [] > > ? ? for item in seq: > ? ? ? result.append([item, index]) > ? ? ? index += 1 > > ? ? return result > > My Pythonic sequencing skills are obviously feeble. Can anything think > of a way to write that in fewer lines? > > -- > ? Phlip > ?http://c2.com/cgi/wiki?MoreliaViridis y = [[seq[i],i] for i in range(len(seq))] gives the same result. The index is accessible without assigning it to another variable. -- Anita From nawijn at gmail.com Tue Jan 5 15:20:58 2010 From: nawijn at gmail.com (Marco Nawijn) Date: Tue, 5 Jan 2010 12:20:58 -0800 (PST) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> Message-ID: On Jan 5, 8:58?pm, Phlip wrote: > Hypo Nt: > > def each_with_index(seq): > ? ? index = 0 > ? ? result = [] > > ? ? for item in seq: > ? ? ? result.append([item, index]) > ? ? ? index += 1 > > ? ? return result > > My Pythonic sequencing skills are obviously feeble. Can anything think > of a way to write that in fewer lines? > > -- > ? Phlip > ?http://c2.com/cgi/wiki?MoreliaViridis You could use the build-in function enumerate inside a list comprehension. >>> seq = range(5) >>> [ (i,s) for i,s in enumerate(seq) ] [(0, 0), (1, 1), (2, 2), (3, 3), (4, 4)] This will reduce the function to a one-liner. Regards, Marco From carsten.haese at gmail.com Tue Jan 5 15:22:51 2010 From: carsten.haese at gmail.com (Carsten Haese) Date: Tue, 05 Jan 2010 15:22:51 -0500 Subject: please help shrink this each_with_index() implementation In-Reply-To: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> Message-ID: Phlip wrote: > Hypo Nt: > > def each_with_index(seq): > index = 0 > result = [] > > for item in seq: > result.append([item, index]) > index += 1 > > return result > > My Pythonic sequencing skills are obviously feeble. Can anything think > of a way to write that in fewer lines? Couldn't you just use the built-in enumerate() to replace the whole thing? -- Carsten Haese http://informixdb.sourceforge.net From jjposner at optimum.net Tue Jan 5 15:34:14 2010 From: jjposner at optimum.net (John Posner) Date: Tue, 05 Jan 2010 15:34:14 -0500 Subject: Dynamic text color References: Message-ID: On Tue, 05 Jan 2010 15:08:04 -0500, Dave McCormick wrote: >> >> It's certainly a mistake to use the expression "str(rList).split()". >> Using str() to convert the list "rList" into a string creates a mess >> that includes square-bracket characters. Did this actually work for you? > It sort of worked. With one color file it seemed fine but after I > posted I added another color file and things fell apart. > Now with the above fixes it works with three colors from three files. > When the list are printed to the shell the list look like this: > redList ['red', 'dog', 'apple', '#'] > blueList ['blue', 'ball', 'berry'] > greenList ['green', 'grass', 'do'] > > But another problem is noticed. It does not matter if the list is built > in code or from a file. > If dog is entered, "do" will be green with the "g" being red. Back to > the drawing board..... It sounds like the program is doing exactly what you TOLD it to do (which might not be what you WANT it to do): 1. In an earlier pass on the text, color the string "dog" red. 2. In a later pass, color the string "do" green. You need to decide what you WANT to happen if one word to be colored is a substring of another word to be colored differently. Or maybe you want to outlaw such situations. After making that decision, you can start to think about how to write the appropriate code. Best, John From phlip2005 at gmail.com Tue Jan 5 15:36:07 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 5 Jan 2010 12:36:07 -0800 (PST) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> Message-ID: > > My Pythonic sequencing skills are obviously feeble. Can anything think > > of a way to write that in fewer lines? Thanks, all! > Couldn't you just use the built-in enumerate() to replace the whole thing? Because that would involve, like, reading an entire Python book just to locate that method? GMAB I'm too busy writing high-end Django via TDD & BDD! C-: -- Phlip http://zeekland.zeroplayer.com/Pigleg_Too/1 From solipsis at pitrou.net Tue Jan 5 16:10:00 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 5 Jan 2010 21:10:00 +0000 (UTC) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> Message-ID: >> Couldn't you just use the built-in enumerate() to replace the whole >> thing? > > Because that would involve, like, reading an entire Python book just to > locate that method? Actually, no. It just involves reading one of the most important pages in the documentation, the page which describes the built-in functions: http://docs.python.org/library/functions.html Don't forget that the Python documentation is rich and structured. And good luck. From g.bogle at auckland.no.spam.ac.nz Tue Jan 5 16:16:01 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Wed, 06 Jan 2010 10:16:01 +1300 Subject: A null program - what is it doing? References: Message-ID: r0g wrote: > Gib Bogle wrote: >> No doubt a dumb question from a noob: >> >> The following program (a cut down version of some test code) uses no >> CPU, and does not terminate: >> >> import sys >> from PyQt4.QtCore import * >> >> if __name__=="__main__": >> app = QCoreApplication(sys.argv) >> sys.exit(app.exec_()) >> >> What is the program doing? What is the correct way to terminate the >> execution? >> >> Thanks in advance for educating me. > > > > I've never used QT but other graphical toolkits I have used all start > their own "main loop" which is a loop that cycles round receiving, > queuing and dispatching "events". You probably need to call the > QCoreApplication's quit method to break out of this e.g. > > app.exit() or something similar, have a look at some complete PyQt4 > examples or google for PyQt4 mainloop. > > > Roger. Thanks. I've realized that QCoreApplication (or QApplication) manages the event loop, and normally when it is executed you are showing a widget of some kind, closing which ends the application. Alt-F4 acts like Ctrl-C to terminate from the keyboard. From tn.pablo at gmail.com Tue Jan 5 16:28:01 2010 From: tn.pablo at gmail.com (Pablo Torres N.) Date: Tue, 5 Jan 2010 13:28:01 -0800 (PST) Subject: Commands for a breakpoint in .pdbrc Message-ID: Hi all, I'd like to save the commands for a breakpoint in a .pdbrc, something like: b 81 commands 1 pp foo.attr1 pp foo.attr2 end b 108 commands 2 pp bar.attr1 pp bar.attr2 end This would automate setting the environment for the debugging session. However, this does not work with 'python -m pdb script.py', because at the line 'commands 1', the pdb prompt starts and asks me for the commands for the first breakpoint, ignoring what I wrote in .pdbrc; further, it raises a NameError after I type 'end' at the pdb prompt, because of 'foo.attr1', 'foo.attr2' and even 'end'. The same happens for the rest of the breakpoints, so I end up with them set but not their commands. What would be the correct way to do this? Is it even possible? Thanks a lot, Pablo Torres N. From deets at nospam.web.de Tue Jan 5 16:39:05 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Tue, 05 Jan 2010 22:39:05 +0100 Subject: embedded python on mac - linking problem In-Reply-To: References: Message-ID: <7qhprpF3qeU1@mid.uni-berlin.de> Krzysztof Kobus schrieb: > Hi, > > I have a problem with linking python module with my application on mac in order to make the module available in "embedded python". > > My python module is contained in j3kmodule.cxx file and module initialization function is exported in j3kmodule.h > > j3kmodule.h: > ------------ > PyMODINIT_FUNC PyInit_j3k(void); > > > j3kmodule.cxx: > -------------- > PyMODINIT_FUNC > PyInit_j3k(void) > { > PyObject *m = NULL; > > if ((m = PyModule_Create(&j3k_module)) == NULL) > return NULL; > > return m; > } > > > Then in my application in KkPython.cxx file I have: > > KkPython.cxx: > ------------- > > #include "j3kmodule.h" > > /* Add a builtin module, before Py_Initialize */ > PyImport_AppendInittab("j3k", PyInit_j3k); > > > I link my application with the module and get following linking error on mac although on open suse linux exactly the same procedure works fine: > > g++ -headerpad_max_install_names -o ../../bin/render.app/Contents/MacOS/render main.o KkPython.o -lkkbase -L/Users/kk/dev/J3K/trunk/j3ksrc/examples/render/../../lib/ -lQtSolutions_PropertyBrowser-2.5 -lpython3.1 /Users/kk/dev/J3K/trunk/j3ksrc/python/modules/build/lib.macosx-10.3-i386-3.1/j3k.so -framework OpenGL -framework AGL > ld: warning in /Users/kk/dev/J3K/trunk/j3ksrc/python/modules/build/lib.macosx-10.3-i386-3.1/j3k.so, file is not of required architecture > Undefined symbols: > "_PyInit_j3k", referenced from: > _PyInit_j3k$non_lazy_ptr in KkPython.o > ld: symbol(s) not found > collect2: ld returned 1 exit status > make: *** [../../bin/render.app/Contents/MacOS/render] Error 1 > > > I appreciate any hints, The missing symbol looks like a C++-symbol - but Python is C. Do you maybe miss the extern "C" declaration. Diez From tosters at gmail.com Tue Jan 5 16:52:18 2010 From: tosters at gmail.com (t0ster) Date: Tue, 5 Jan 2010 13:52:18 -0800 (PST) Subject: Python multiprocessing: Permission denied Message-ID: <1c85295e-11ee-471c-bd2f-420e0f2fc3b6@j4g2000yqe.googlegroups.com> Hi guys, I'm getting an error when trying to execute python program that uses multiprocessing package: File "/usr/local/lib/python2.6/multiprocessing/__init__.py", line 178, in RLock return RLock() File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line 142, in __init__ SemLock.__init__(self, RECURSIVE_MUTEX, 1, 1) File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line 49, in __init__ sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue) OSError: [Errno 13] Permission denied It looks like the user don't have permission to access shared memory. When executing with root privileges it works fine. Is there any solution to run it as normal user(not root)? Python version 2.6.2 , OS is Linux 2.6.18 (CentOS release 5.4) and it's VPS machine. From mackrackit at gmail.com Tue Jan 5 16:54:44 2010 From: mackrackit at gmail.com (Dave McCormick) Date: Tue, 05 Jan 2010 14:54:44 -0700 Subject: Dynamic text color In-Reply-To: References: Message-ID: <4B43B524.9090805@gmail.com> John Posner wrote: > On Tue, 05 Jan 2010 15:08:04 -0500, Dave McCormick > wrote: > > It sounds like the program is doing exactly what you TOLD it to do > (which might not be what you WANT it to do): > > 1. In an earlier pass on the text, color the string "dog" red. > 2. In a later pass, color the string "do" green. > > You need to decide what you WANT to happen if one word to be colored > is a substring of another word to be colored differently. Or maybe you > want to outlaw such situations. After making that decision, you can > start to think about how to write the appropriate code. > > Best, > John Darn thing doing what I told it to do... Guess that means I did something right :) But it is not what I am wanting. I first thought to make it look for a space but that would not work when a single character like "#" is to be colored if there is a "string" of them. Or if all of the characters between quotes are to be colored. I always did like puzzles! Dave From nad at acm.org Tue Jan 5 17:03:45 2010 From: nad at acm.org (Ned Deily) Date: Tue, 05 Jan 2010 14:03:45 -0800 Subject: it gets worse (was: How do you configure IDLE on a Mac...) References: <0d70cb54-3d77-4176-b621-e764ecf613b7@26g2000yqo.googlegroups.com> <6672dad2-26ba-458b-8075-21bac6506179@e37g2000yqn.googlegroups.com> Message-ID: In article <6672dad2-26ba-458b-8075-21bac6506179 at e37g2000yqn.googlegroups.com>, Mensanator wrote: [...] > So, for all practical purposes, the macports install is broken also. > > IDLE simply does not work in an X11 window (you think someone would > have noticed that). The missing preferences is just the beginning. > Apparently NONE of the menu item shortcuts work. > > For example, the Cut, Copy, Paste shortcuts are given as Command-X, > Command-C and Command-V. But that doesn't work in an X11 window, > apperently only in an Aqua Tk (parent application appears as IDLE). > > Of course, I can do Control-X, Control-C and Control-V to do Cut, > Copy and Paste. Don't know if this works for all shortcuts, but > I suppose I could just pick them from the menu (and I can bang > my head against the wall while I'm at it). > > What do you think, suppose I copy the gmpy built with the macports > install over to the directory where the python.org version is? Would > it > import? If that'll work, I can switch back to using the python.org > install and use it's version of IDLE. I certainly won't be needing > distutils once I have a working version of gmpy. Let's go back to your original problem, which, if I understand correctly, was trying to get going with Python 3 and gmpy on OS X 10.6. (Sorry I was away over the holidays and didn't get a chance to respond to your original postings at the time.) I believe the problems you originally encountered with installing gmpy were all due to a couple of problems with building C extension modules on 10.6 when using the current 3.1.1 OS X python.org. Unfortunately, 3.1.1 was released before 10.6 was so there are a couple of important fixes that haven't yet been released for 3.1 (but are in the 2.6.4 installer which was released after 10.6 came out). Fortunately, though, there are simple workarounds for the problems. Keep in mind, though, that, at the moment, the python.org installers for OS X are 32-bit only; that will change in the future but if you do need a 64-bit Python 3 you'll need to stick to other solutions like MacPorts for the time being. First, make sure the gmp library you've installed has 32-bit support. If you installed it using MacPorts, check with the file command: $ file /opt/local/lib/libgmp.dylib /opt/local/lib/libgmp.dylib: Mach-O universal binary with 2 architectures /opt/local/lib/libgmp.dylib (for architecture i386): Mach-O dynamically linked shared library i386 /opt/local/lib/libgmp.dylib (for architecture x86_64): Mach-O 64-bit dynamically linked shared library x86_64 If it doesn't have an i386 variant, reinstall the gmp library from MacPorts: $ sudo port selfupdate # make sure MacPorts is up-to-date $ sudo port clean gmp $ sudo port install gmp +universal # install 32-/64-bit variants Second, you need to install the MacOSX10.4u SDK because the current python.org pythons are built with it. That SDK is included in the Snow Leopard Xcode installer package but it is not installed by default. There should be an Xcode.mpkg somewhere, perhaps on your hard disk if your system came with Snow Leopard factory-installed or perhaps on a restore DVD. If not, it's on the retail Snow Leopard DVD and can be downloaded from the Apple Developer site. After launching the Xcode installer, just select and install the "Mac OS 10.4 Support" package from the Custom Install menu. Third, you need to tell Distutils to use the older gcc-4.0 instead of the gcc-4.2 which is now the default on 10.6. $ cd /path/to/gmpy-1.11rc1 $ export CC=/usr/bin/gcc-4.0 $ /usr/local/bin/python3.1 setup.py install ... $ /usr/local/bin/python3.1 test3/gmpy_test.py Unit tests for gmpy 1.11 on Python 3.1.1 (r311:74543, Aug 24 2009, 18:44:04) [GCC 4.0.1 (Apple Inc. build 5493)] Testing gmpy 1.11 (GMP 4.3.1), default caching (100, 128) ... 1500 tests in 42 items. 1500 passed and 0 failed. -- Ned Deily, nad at acm.org From python at mrabarnett.plus.com Tue Jan 5 17:37:08 2010 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 05 Jan 2010 22:37:08 +0000 Subject: parsing an Excel formula with the re module In-Reply-To: <5ef3575d-eb1d-4a15-90e0-87dd85cdc2e6@35g2000yqa.googlegroups.com> References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> <5ef3575d-eb1d-4a15-90e0-87dd85cdc2e6@35g2000yqa.googlegroups.com> Message-ID: <4B43BF14.7000102@mrabarnett.plus.com> Mensanator wrote: > On Jan 5, 12:35 pm, MRAB wrote: >> vsoler wrote: >>> Hello, >>> I am acessing an Excel file by means of Win 32 COM technology. >>> For a given cell, I am able to read its formula. I want to make a map >>> of how cells reference one another, how different sheets reference one >>> another, how workbooks reference one another, etc. >>> Hence, I need to parse Excel formulas. Can I do it by means only of re >>> (regular expressions)? >>> I know that for simple formulas such as "=3*A7+5" it is indeed >>> possible. What about complex for formulas that include functions, >>> sheet names and possibly other *.xls files? >>> For example "=Book1!A5+8" should be parsed into ["=","Book1", "!", >>> "A5","+","8"] >>> Can anybody help? Any suggestions? >> Do you mean "how" or do you really mean "whether", ie, get a list of the >> other cells that are referred to by a certain cell, for example, >> "=3*A7+5" should give ["A7"] and "=Book1!A5+8" should give ["Book1!A5] > > Ok, although "Book1" would be the default name of a workbook, with > default > worksheets labeled "Sheet1". "Sheet2", etc. > > If I had a worksheet named "Sheety" that wanted to reference a cell on > "Sheetx" > OF THE SAME WORKBOOK, it would be =Sheet2!A7. If the reference was to > a completely > different workbook (say Book1 with worksheets labeled "Sheet1", > "Sheet2") then > the cell might have =[Book1]Sheet1!A7. > > And don't forget the $'s! You may see =[Book1]Sheet1!$A$7. I forgot about the dollars! In that case, the regex is: references = re.findall(r"\b((?:\w+!)?\$?[A-Za-z]+\$?\d+)\b", formula) From ben+python at benfinney.id.au Tue Jan 5 17:39:08 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 06 Jan 2010 09:39:08 +1100 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> Message-ID: <87pr5ow50z.fsf@benfinney.id.au> r0g writes: > Of course I'm now guilty of pedantry too :/ I might have let it slip > had you not started your reply with the word "No", that just p***** me > off. Well, if being told ?no? is going to piss you off, I think you're in for a rough time. > Having said that I find the mental image of you slamming your fist on > the table and shouting it out loud whenever you read something you > disagree with on usenet quite amusing! In return, I find myself quite amused that you would get such an image. To reduce the stress you describe above, you might want to read what is written, rather than inventing fanciful emotion where it wasn't in the message to begin with. -- \ ?One time I went to a drive-in in a cab. The movie cost me | `\ ninety-five dollars.? ?Steven Wright | _o__) | Ben Finney From phlip2005 at gmail.com Tue Jan 5 17:40:49 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 5 Jan 2010 14:40:49 -0800 (PST) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> Message-ID: <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> On Jan 5, 1:10?pm, Antoine Pitrou wrote: > http://docs.python.org/library/functions.html > > Don't forget that the Python documentation is rich and structured. > And good luck. Does it say how to convert a string containing either an integer representation, or something alphabetic, into an integer, or a zero, in like 1 method call? (No except: ?) Nothing personal, but I'm finding the super-hard stuff very facile & tractable, and the easy stuff absurdly hard around here... From steve at REMOVE-THIS-cybersource.com.au Tue Jan 5 18:20:49 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 05 Jan 2010 23:20:49 GMT Subject: Python multiprocessing: Permission denied References: <1c85295e-11ee-471c-bd2f-420e0f2fc3b6@j4g2000yqe.googlegroups.com> Message-ID: <0353b63c$0$1336$c3e8da3@news.astraweb.com> On Tue, 05 Jan 2010 13:52:18 -0800, t0ster wrote: > It looks like the user don't have permission to access shared memory. > When executing with root privileges it works fine. > > Is there any solution to run it as normal user(not root)? Then give the user permission to access shared memory. Why do you expect that Python would be able to over-ride the operating system's security? This problem is no different from saying "It looks like the user doesn't have permission to access this file". -- Steven From steve at REMOVE-THIS-cybersource.com.au Tue Jan 5 18:22:27 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 05 Jan 2010 23:22:27 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> Message-ID: <0353b69e$0$1336$c3e8da3@news.astraweb.com> On Wed, 06 Jan 2010 09:39:08 +1100, Ben Finney wrote: > r0g writes: > >> Of course I'm now guilty of pedantry too :/ I might have let it slip >> had you not started your reply with the word "No", that just p***** me >> off. > > Well, if being told ?no? is going to piss you off, I think you're in for > a rough time. Oh, you're in trouble now! If you think he gets upset at being told no, you should see how upset he gets at being told he's in for a rough time!!! *wink* -- Steven From steve at REMOVE-THIS-cybersource.com.au Tue Jan 5 18:26:26 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 05 Jan 2010 23:26:26 GMT Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> Message-ID: <0353b78e$0$1336$c3e8da3@news.astraweb.com> On Tue, 05 Jan 2010 14:40:49 -0800, Phlip wrote: > On Jan 5, 1:10?pm, Antoine Pitrou wrote: > >> http://docs.python.org/library/functions.html >> >> Don't forget that the Python documentation is rich and structured. And >> good luck. > > Does it say how to convert a string containing either an integer > representation, or something alphabetic, into an integer, or a zero, in > like 1 method call? (No except: ?) If you mean something like this: >>> int('153') 153 then yes it does. But if you mean something like this: >>> some_mysterious_function('one hundred and fifty-three') 153 then no, it doesn't. > Nothing personal, but I'm finding the super-hard stuff very facile & > tractable, and the easy stuff absurdly hard around here... Then perhaps you should work through the tutorial to learn the basics. -- Steven From phlip2005 at gmail.com Tue Jan 5 18:30:09 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 5 Jan 2010 15:30:09 -0800 (PST) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> Message-ID: > > Does it say how to convert a string containing either an integer > > representation, or something alphabetic, into an integer, or a zero, in > > like 1 method call? (No except: ?) > > If you mean something like this: > > >>> int('153') > > 153 The point: int('') or int('something') both throw an error. In general, this is hand-holding, but in specific I don't think the "rich and structured" documentation will cover how to beat a 0 out of it in less than 3 lines. So I will persist in my idiotic questions here! > Then perhaps you should work through the tutorial to learn the basics. They will tell me how to use except: (which is a good example why a program should not use exceptions for its normal control flow if at all possible). Please, please, please save your newbie admonitions for those who qualify! From skwslide at gmail.com Tue Jan 5 18:50:39 2010 From: skwslide at gmail.com (Steven K. Wong) Date: Tue, 5 Jan 2010 15:50:39 -0800 (PST) Subject: subprocess.Popen does not close pipe in an error case Message-ID: <07641018-9581-44af-bb20-8f9b96a44c4f@p8g2000yqb.googlegroups.com> Below, I have a Python script that launches 2 child programs, prog1 and prog2, with prog1's stdout connected to prog2's stdin via a pipe. (It's like executing "prog1 | prog2" in the shell.) If both child programs exit with 0, then the script runs to completion. But if prog2 exits with non-0, prog1 does not exit and the script hangs (i.e. prog1.poll() always returns None) -- unless I uncomment the 2 lines marked by XXX to close prog1.stdout. I was expecting that I don't have to explicitly close prog1.stdout, whether prog2 succeeds or fails. Is the current behavior a bug in the subprocess module or is it expected? Or am I doing something wrong? Thanks. import subprocess import time # prog1: a program that writes lots of data to the pipe cmd = ['zcat', '--force', 'a_large_file'] prog1 = subprocess.Popen(cmd, bufsize=-1, stdout=subprocess.PIPE) # prog2: a program that fails without reading much data from the pipe cmd = ['python', '-c', 'import time; time.sleep(10); asdf'] prog2 = subprocess.Popen(cmd, bufsize=-1, stdin=prog1.stdout, stdout=open('popen.out', 'w')) print 'waiting for a while' retCodeProg2 = prog2.wait() print 'prog2 returns', retCodeProg2 # XXX # if retCodeProg2 != 0: # prog1.stdout.close() while prog1.poll() is None: print 'sleep a bit' time.sleep(1) retCodeProg1 = prog1.poll() print 'prog1 returns', retCodeProg1 From phlip2005 at gmail.com Tue Jan 5 18:51:29 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 5 Jan 2010 15:51:29 -0800 (PST) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: Peng Yu wrote: > Otherwise, could some python expert explain to me why exception is > widely used for error handling in python? Is it because the efficiency > is not the primary goal of python? It's not about efficiency, it's about making assumptions for the programmer about what kind of rigor they need. Why can't int('nonnumeric') return None? Why can't a Django Record.objects.get(pk=-1) return a None? That's what it's for. (A related question - why can't I just go 'if record = method(): use (record)'. Why extra lines just to trap and assign the variable before using it?) There are workarounds that sometimes benefit the code. In the case of collections, like recordsets, you might be better off using for ... all (): Then your controlled block efficiently does not happen if it saw no records. "Efficiently" in terms of programmer complexity - the number and meaning of lines that a programmer must comprehend. And why can't Record.objects.get(pk='nonnumeric') return None? Because, of course, deep inside it calls int(). I can't simplify the calling code, and rely on garbage-in-None-out, because Python decided which simplifications I should avoid with self-righteous indignation. The Samurai Principle (return victorious, or not at all) is very useful, sometimes. But other times it just prematurely depletes your supply of Samurai... -- Phlip http://zeekland.zeroplayer.com/ From skwslide at gmail.com Tue Jan 5 18:53:15 2010 From: skwslide at gmail.com (Steven K. Wong) Date: Tue, 5 Jan 2010 15:53:15 -0800 (PST) Subject: subprocess.Popen does not close pipe in an error case References: <07641018-9581-44af-bb20-8f9b96a44c4f@p8g2000yqb.googlegroups.com> Message-ID: <15549117-c3f3-40ab-85de-c8fb85193d1d@f5g2000yqh.googlegroups.com> BTW, I'm using Python 2.6.2 on Linux. From haggardii at gmail.com Tue Jan 5 19:14:49 2010 From: haggardii at gmail.com (Matt Haggard) Date: Tue, 5 Jan 2010 16:14:49 -0800 (PST) Subject: unittest inconsistent Message-ID: Can anyone tell me why this test fails? http://pastebin.com/f20039b17 This is a minimal example of a much more complex thing I'm trying to do. I'm trying to hijack a function and inspect the args passed to it by another function. The reason the 'Tester' object has no attribute 'arg1' is because "self" still refers to the object made for testA. From mackrackit at gmail.com Tue Jan 5 19:26:38 2010 From: mackrackit at gmail.com (Mackrackit) Date: Tue, 5 Jan 2010 17:26:38 -0700 Subject: please help shrink this each_with_index() implementation In-Reply-To: References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> Message-ID: On Jan 5, 2010, at 4:30 PM, Phlip wrote: >> >> > The point: int('') or int('something') both throw an error. In > general, this is hand-holding, but in specific I don't think the "rich > and structured" documentation will cover how to beat a 0 out of it in > less than 3 lines. So I will persist in my idiotic questions here! >> What does an extra two or three lines of code matter? Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From aioe.org at technicalbloke.com Tue Jan 5 19:58:58 2010 From: aioe.org at technicalbloke.com (r0g) Date: Wed, 06 Jan 2010 00:58:58 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > On Wed, 06 Jan 2010 09:39:08 +1100, Ben Finney wrote: > >> r0g writes: >> >>> Of course I'm now guilty of pedantry too :/ I might have let it slip >>> had you not started your reply with the word "No", that just p***** me >>> off. >> Well, if being told ?no? is going to piss you off, I think you're in for >> a rough time. > > Oh, you're in trouble now! If you think he gets upset at being told no, > you should see how upset he gets at being told he's in for a rough time!!! > > *wink* > > > NO! It's a rude way to start a sentence don't you think? Just because you're correcting someone doesn't mean you have to be combative and try and make them feel small. Unless they've adopted a hostile or wilfully ignorant tone there's no reason to be so brusqe with people. You can be both nice AND terse you know. I can't imagine why I expect good manners on usenet though, AFAICT it's never been like that (well not since I got on it anyway). Roger. From clp2 at rebertia.com Tue Jan 5 20:01:02 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Tue, 5 Jan 2010 17:01:02 -0800 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <50697b2c1001051701v6b635deeh1a02426f10836d24@mail.gmail.com> On Tue, Jan 5, 2010 at 3:51 PM, Phlip wrote: > Peng Yu wrote: >> Otherwise, could some python expert explain to me why exception is >> widely used for error handling in python? Is it because the efficiency >> is not the primary goal of python? > > It's not about efficiency, it's about making assumptions for the > programmer about what kind of rigor they need. > > Why can't int('nonnumeric') return None? Errors should never pass silently. Unless explicitly silenced. -- The Zen of Python (http://www.python.org/dev/peps/pep-0020/) Better to throw an exception and ensure the case is specifically dealt with one way or another than to silently return an error flag result which may only delay the error until later in the program, making it harder to debug. Is it that much of a burden to write and use the small function that does what you want? def int_or_None(string): try: return int(string) except ValueError: return None Heck, you can even write it inline and dispense with the function if you want: try: foo = int(bar) except ValueError: foo = None Quibbling over a mere one more line of code (or writing one short function) seems a bit petty. > (A related question - why can't I just go 'if record = method(): ?use > (record)'. Why extra lines just to trap and assign the variable before > using it?) I believe that's disallowed so as to prevent the subtle bugs seen in C code which result from when someone makes a typo and omits the second "=" in their `if foo == bar():` test. Cheers, Chris -- http://blog.rebertia.com From aioe.org at technicalbloke.com Tue Jan 5 20:03:12 2010 From: aioe.org at technicalbloke.com (r0g) Date: Wed, 06 Jan 2010 01:03:12 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> <03533377$0$1336$c3e8da3@news.astraweb.com> <4b437282$1@dnews.tpgi.com.au> Message-ID: Lie Ryan wrote: > On 1/6/2010 1:48 AM, r0g wrote: >> Steven D'Aprano wrote: >>> On Tue, 05 Jan 2010 13:06:20 +0000, r0g wrote: >>>> If >>>> that's the case how can you expect it to validate anything at all in >>>> production? >>> >>> The asserts still operate so long as you don't use the -O switch. >>> > checking, since the function relies on the caller obeying the > contract[2] and never calling it with an invalid input. > > DbC uses assertions[1] spuriously, unlike the traditional approach which > is much more conservative when using assertions. > > [1] or explicit language support which is just syntax sugar for assertions > [2] of course, on a debug release, the contract validation code will > still be enforced to catch logic/consistency bugs that causes the violation Thanks for the responses Steven/Dave/Lie, that's some really insightful stuff :) Roger. From phlip2005 at gmail.com Tue Jan 5 20:45:58 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 5 Jan 2010 17:45:58 -0800 (PST) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> On Jan 5, 5:01?pm, Chris Rebert wrote: > > Why can't int('nonnumeric') return None? > > Errors should never pass silently. You are saying I, as the programmer, cannot decide what is an error and what is a pass-thru. The decision is made for me. (Yes yes I can write int_or_None(), etc...) Here's a super easy example: { 42: 'forty two' }.get(41, None) Because I can supply a default, I can decide what is an error and what is . Now the equivalent in a language that does not enjoy this false "Zen": { 42: 'forty two' }[41] # returns None { 42: 'forty two' }.fetch(41, None) # ibid { 42: 'forty two' }.fetch(41) # raises an exception The quicky validation is available if I _request_ it. > Quibbling over a mere one more line of code (or writing one short > function) seems a bit petty. Because that "Zen of Python" is an empty sophistry that forces me to add a "mere one more line of code" over and over again... > > (A related question - why can't I just go 'if record = method(): ?use > > (record)'. Why extra lines just to trap and assign the variable before > > using it?) > > I believe that's disallowed so as to prevent the subtle bugs seen in C > code which result from when someone makes a typo and omits the second > "=" in their `if foo == bar():` test. Don't prevent me from using a technique just because others had trouble with it. And if bar() == foo is the superior technique anyway, because the == happens in chronological and lexical order after the bar() call. From phlip2005 at gmail.com Tue Jan 5 20:50:00 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 5 Jan 2010 17:50:00 -0800 (PST) Subject: unittest inconsistent References: Message-ID: On Jan 5, 4:14?pm, Matt Haggard wrote: > Can anyone tell me why this test fails? > > http://pastebin.com/f20039b17 > > This is a minimal example of a much more complex thing I'm trying to > do. ?I'm trying to hijack a function and inspect the args passed to it > by another function. > > The reason the 'Tester' object has no attribute 'arg1' is because > "self" still refers to the object made for testA. I hope someone else can spot the low-level reason... ...but why aren't you using http://pypi.python.org/pypi/mock/ ? Look up its patch_object facility... From fetchinson at googlemail.com Tue Jan 5 20:50:30 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Tue, 5 Jan 2010 17:50:30 -0800 Subject: twenty years ago Guido created Python In-Reply-To: <4B438FB6.9050003@holdenweb.com> References: <51564fb3-a367-4f11-b97f-5518deedbe48@a21g2000yqc.googlegroups.com> <4B438FB6.9050003@holdenweb.com> Message-ID: >>> Python is a truly awesome programming language. Not only is Guido a >>> genius language designer, but he is also a great project leader. What >>> an accomplishment. Congratulations to everybody who has contributed >>> to Python in the last two decades! >> >> The more languages you learn before getting to Smalltalk, the more >> awesome Smalltalk will be for you. >> > After implementing SmallTalk I more or less gave up OO programming for > ten years, But why? What was so frightening about the OO model of SmallTalk? Cheers, Daniel > resuming it only after I met Python. SmallTalk didn't seem > that awesome to me. > > Though for its time it was an incredible system, its insistence on a > SmallTalk-only VM environment seemed a little solipsistic. -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From phlip2005 at gmail.com Tue Jan 5 20:56:48 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 5 Jan 2010 17:56:48 -0800 (PST) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <09b70061-c411-45a9-8546-c9113c127ced@p8g2000yqb.googlegroups.com> > Errors should never pass silently. > Unless explicitly silenced. > -- The Zen of Python (http://www.python.org/dev/peps/pep-0020/) "The person who says it cannot be done should never interrupt the person doing it" From andre.roberge at gmail.com Tue Jan 5 20:57:53 2010 From: andre.roberge at gmail.com (=?ISO-8859-1?Q?Andr=E9?=) Date: Tue, 5 Jan 2010 17:57:53 -0800 (PST) Subject: unittest inconsistent References: Message-ID: <3dede91a-ab3c-42c0-af45-adde220ededd@z2g2000vbf.googlegroups.com> On Jan 5, 8:14?pm, Matt Haggard wrote: > Can anyone tell me why this test fails? > > http://pastebin.com/f20039b17 > > This is a minimal example of a much more complex thing I'm trying to > do. ?I'm trying to hijack a function and inspect the args passed to it > by another function. > > The reason the 'Tester' object has no attribute 'arg1' is because > "self" still refers to the object made for testA. Quick answer: change faketest.py as follows: #-------------------------------------------------- # faketest.py #-------------------------------------------------- #from importme import render import importme def run(somearg): return importme.render(somearg) ========= A long answer, with explanation, will cost you twice as much ;-) (but will have to wait) Andr? From steve at holdenweb.com Tue Jan 5 21:06:01 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 05 Jan 2010 21:06:01 -0500 Subject: parsing an Excel formula with the re module In-Reply-To: <4B4397B9.7000804@tim.thechases.com> References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> <4B4397B9.7000804@tim.thechases.com> Message-ID: Tim Chase wrote: > vsoler wrote: >> Hence, I need to parse Excel formulas. Can I do it by means only of re >> (regular expressions)? >> >> I know that for simple formulas such as "=3*A7+5" it is indeed >> possible. What about complex for formulas that include functions, >> sheet names and possibly other *.xls files? > > Where things start getting ugly is when you have nested function calls, > such as > > =if(Sum(A1:A25)>42,Min(B1:B25), if(Sum(C1:C25)>3.14, > (Min(C1:C25)+3)*18,Max(B1:B25))) > > Regular expressions don't do well with nested parens (especially > arbitrarily-nesting-depth such as are possible), so I'd suggest going > for a full-blown parsing solution like pyparsing. > > If you have fair control over what can be contained in the formulas and > you know they won't contain nested parens/functions, you might be able > to formulate some sort of "kinda, sorta, maybe parses some forms of > formulas" regexp. > And don't forget about named ranges, which can reference cells without using anything but a plain identifier ... regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From sridharr at activestate.com Tue Jan 5 21:25:47 2010 From: sridharr at activestate.com (Sridhar Ratnakumar) Date: Tue, 05 Jan 2010 18:25:47 -0800 Subject: Fwd: I would like to install Python on my 64 bit Win 7 In-Reply-To: References: Message-ID: <4B43F4AB.6070308@activestate.com> On 1/5/2010 11:31 AM, aung paing Soe wrote: > > Hello , > I would like to study about Python Programming . So I want to > install Python . > But my laptop is Window 7 64-bit home basic . > So please give me a advice how to install Python in my 64 bit computer. > I really want to study python programming . > I am looking forward your reply. > Thank you very much Even though your laptop runs a 64-bit operating system, you do not have to install 64-bit Python build. You may also install a 32-bit build. (Some Python packages may not even build on 64-bit Python) You can install Python from one of the two places: 1. http://python.org/download/releases/2.6.4/ (AMD64 == 64-bit) 2. http://www.activestate.com/activepython/ (Includes PyWin32, a package manager and extra documentation and tutorials). -srid From solipsis at pitrou.net Tue Jan 5 21:28:08 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 6 Jan 2010 02:28:08 +0000 (UTC) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> Message-ID: > The point: int('') or int('something') both throw an error. In general, > this is hand-holding, but in specific I don't think the "rich and > structured" documentation will cover how to beat a 0 out of it in less > than 3 lines. Because it's a bad idea to do so and Python doesn't encourage such sloppy behaviour. If you like it, though, you might prefer PHP. By the way: error reporting is definitely *not* hand-holding. It is simply not good to let errors pass silently by default. Again, if you think the contrary, PHP is your friend ;-) From aahz at pythoncraft.com Tue Jan 5 21:38:11 2010 From: aahz at pythoncraft.com (Aahz) Date: 5 Jan 2010 18:38:11 -0800 Subject: Minor bug in multiprocessing? References: Message-ID: [p&e] In article , Frank Millman wrote: > >Is this worth reporting, if it has not been reported already? Defiitely report it. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From ethan at stoneleaf.us Tue Jan 5 21:41:40 2010 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 05 Jan 2010 18:41:40 -0800 Subject: Object Relational Mappers are evil (a meditation) In-Reply-To: <85hbr9ergw.fsf@agentultra.com> References: <06532563-3f2b-4cca-802f-34cb326e6d9f@v20g2000vbs.googlegroups.com> <16a44521-1f07-454a-ace7-9ba5d5db4b9c@y28g2000prd.googlegroups.com> <50f98a4c0910060810k3a4fa910vabc5bf51381d00f9@mail.gmail.com> <00ac7cf3$0$15654$c3e8da3@news.astraweb.com> <85hbrrxaf9.fsf@agentultra.com> <85d42eyag8.fsf@agentultra.com> <7osvd9F3r79lcU1@mid.individual.net> <858wd2xmi4.fsf@agentultra.com> <4b2c77f1$1@dnews.tpgi.com.au> <854onkxonm.fsf@agentultra.com> <00b98caf$0$15623$c3e8da3@news.astraweb.com> <85zl59d6s8.fsf@agentultra.com> <00bc18b1$0$15637$c3e8da3@news.astraweb.com> <85hbr9ergw.fsf@agentultra.com> Message-ID: <4B43F864.8080009@stoneleaf.us> J Kenneth King wrote: > > In many contexts I'm sure there is reason to use Perl instead of Python > just as there are situations where C is more appropriate than either. > > However, the mark of a poor programmer in my line of reasoning is one > who cannot recognize such distinctions. > > One must be aware of the benefits and short-comings of their tools. If > your tools influence the way you think then you are being ignorant of > this principle. And I would suggest that makes you a poor programmer. Perhaps "influence the way you think" is not the right way to phrase it... how about "be the tool" ;) We have all seen the struggles that newcomers to a language go through as they try (or don't try) to adjust their thinking to the tool at hand -- programming Java, BASIC, FORTRAN, or xyz in Python. Even now Phlip is raging against exceptions and the very Zen of Python. Converting FoxPro to Python is an interesting excercise for me -- version 6 at least doesn't have many of the cool things that Python does, and consequently thinking in Python while writing FoxPro (when I have to) is extremely frustrating; going the other way is a bit of a challenge also, although much more rewarding. For a more concrete example, take sail-boats and speed-boats: you get used to the speed boat, it's quick handling and sharp turns... then you take a sail boat out for a cruise -- if you don't adjust your thinking from speed to sail, you could very well end up in the rocks. To sum up: I agree with your "poor programmer" line of reasoning in the second paragraph above, but not with the "tools influencing the way we think" line of reasoning -- while I am cognizant of Python's shortcomings, I am very much enjoying the changes in my thinking the more I use it. ~Ethan~ From clp2 at rebertia.com Tue Jan 5 21:48:55 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Tue, 5 Jan 2010 18:48:55 -0800 Subject: Exception as the primary error handling mechanism? In-Reply-To: <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> Message-ID: <50697b2c1001051848ye0793eep953a890ca4973a20@mail.gmail.com> On Tue, Jan 5, 2010 at 5:45 PM, Phlip wrote: > On Jan 5, 5:01?pm, Chris Rebert wrote: >> > Why can't int('nonnumeric') return None? >> >> Errors should never pass silently. > > You are saying I, as the programmer, cannot decide what is an error > and what is a pass-thru. The decision is made for me. (Yes yes I can > write int_or_None(), etc...) No, you can certainly decide yourself. You merely need be explicit about it; you're leaving out the other tandem half of the couplet: "*Unless* explicitly silenced." Throwing exceptions is only the default because it tends to lead to more reliable code in that you're forced to deal with the error condition by either catching an exception or preventing one from being thrown (by, say, using a different method in the API that just returns a default value instead). As an aside, I would guess the built-in types don't have default value parameters for conversions partly because it'd be a bit less elegant to implement; since None is a commonly used default value, they'd have to use a different sentinel as said parameter's default value to indicate that the caller wanted an exception raised, and having a hierarchy of nil values in the core language detracts from the language's elegance. At any rate, if you like dict.get(), I don't see why, say, my_int('nonnumeric', None) should be such a problem. > Here's a super easy example: > > ?{ 42: 'forty two' }.get(41, None) > > Because I can supply a default, I can decide what is an error and what > is . Exactly, that's explicitly silencing the error; no one ever said an `except` clause was the only silencing mechanism. You're making your intention clear by using .get() and passing in a desired default. I agree this is a fine thing. > Now the equivalent in a language that does not enjoy this false "Zen": > > ?{ 42: 'forty two' }[41] ?# returns None > ?{ 42: 'forty two' }.fetch(41, None) ?# ibid > ?{ 42: 'forty two' }.fetch(41) ?# raises an exception > > The quicky validation is available if I _request_ it. Seems like rather "fast-and-loose" programming if you don't care about validation enough of the time to merit it being default. If your programming is indeed so fast-and-loose, I refer you to the recent comment about PHP (and other less "exception-happy" languages). Anyway, I do totally agree that you should, if feasible, be provided an easy way to designate a common error-handling strategy (for example, in this case, by using a default value via .fetch()). However, go too loose on error handling and exceptions and one ends up with something like JavaScript with its infamous `undefined` value which can make debugging a nightmare (whoever came up with the idea of JS's `undefined` should be slapped upside the head). Cheers, Chris -- http://blog.rebertia.com From mensanator at aol.com Tue Jan 5 21:51:22 2010 From: mensanator at aol.com (Mensanator) Date: Tue, 5 Jan 2010 18:51:22 -0800 (PST) Subject: it gets worse (was: How do you configure IDLE on a Mac...) References: <0d70cb54-3d77-4176-b621-e764ecf613b7@26g2000yqo.googlegroups.com> <6672dad2-26ba-458b-8075-21bac6506179@e37g2000yqn.googlegroups.com> Message-ID: On Jan 5, 4:03?pm, Ned Deily wrote: > In article > <6672dad2-26ba-458b-8075-21bac6506... at e37g2000yqn.googlegroups.com>,?Mensanator wrote: > > [...] > > > > > > > So, for all practical purposes, the macports install is broken also. > > > IDLE simply does not work in an X11 window (you think someone would > > have noticed that). The missing preferences is just the beginning. > > Apparently NONE of the menu item shortcuts work. > > > For example, the Cut, Copy, Paste shortcuts are given as Command-X, > > Command-C and Command-V. But that doesn't work in an X11 window, > > apperently only in an Aqua Tk (parent application appears as IDLE). > > > Of course, I can do Control-X, Control-C and Control-V to do Cut, > > Copy and Paste. Don't know if this works for all shortcuts, but > > I suppose I could just pick them from the menu (and I can bang > > my head against the wall while I'm at it). > > > What do you think, suppose I copy the gmpy built with the macports > > install over to the directory where the python.org version is? Would > > it > > import? If that'll work, I can switch back to using the python.org > > install and use it's version of IDLE. I certainly won't be needing > > distutils once I have a working version of gmpy. > > Let's go back to your original problem, which, if I understand > correctly, was trying to get going with Python 3 and gmpy on OS X 10.6. ? Right. > (Sorry I was away over the holidays and didn't get a chance to respond > to your original postings at the time.) ? Hey, no problem. I bought this dingus with the idea I would spend my the holiday time to get it to work. Boy, was I right. > I believe the problems you > originally encountered with installing gmpy were all due to a couple of > problems with building C extension modules on 10.6 when using the > current 3.1.1 OS X python.org. ? Yeah, I used the Mac disk image for Python 3.1. > Unfortunately, 3.1.1 was released before > 10.6 was so there are a couple of important fixes that haven't yet been > released for 3.1 (but are in the 2.6.4 installer which was released > after 10.6 came out). ?Fortunately, though, there are simple workarounds > for the problems. ?Keep in mind, though, that, at the moment, the > python.org installers for OS X are 32-bit only; I just checked, I was told to check sys.maxint but that doesn't exist, I assume it's now sys.maxsize. On the python.org disk image, that returns >>> hex(sys.maxsize) '0x7fffffff' looks like 32 bits. > that will change in the > future but if you do need a 64-bit Python 3 you'll need to stick to > other solutions like MacPorts for the time being. The macports install of 3.1 gives: >>> hex(sys.maxsize) '07x7fffffffffffffff' so the macports must be 64 bits. > > First, make sure the gmp library you've installed has 32-bit support. ? Uh, why would I want that? If it comes down to a choice between IDLE and 64 bits, I'll live without IDLE. > If you installed it using MacPorts, check with the file command: > > $ file /opt/local/lib/libgmp.dylib > /opt/local/lib/libgmp.dylib: Mach-O universal binary with 2 architectures > /opt/local/lib/libgmp.dylib (for architecture i386): ?Mach-O dynamically > linked shared library i386 > /opt/local/lib/libgmp.dylib (for architecture x86_64): ? Mach-O 64-bit > dynamically linked shared library x86_64 I get /opt/local/lib/libgmpdylib: Mach-0 64-bit dynamically linked shared library x86_64 > > If it doesn't have an i386 variant, reinstall the gmp library from > MacPorts: But I only need that if I want to run the 32 bit version of Python from python.org. If I'm willing to use the 64 bit version from macports, I don't care, right? > > $ sudo port selfupdate ? ?# make sure MacPorts is up-to-date > $ sudo port clean gmp > $ sudo port install gmp +universal ? ? # install 32-/64-bit variants > > Second, you need to install the MacOSX10.4u SDK because the current > python.org pythons are built with it. Then I'll just not use the download from python.org. Is there a way to uninstall that disk image? >?That SDK is included in the Snow > Leopard Xcode installer package but it is not installed by default. ? > There should be an Xcode.mpkg somewhere, perhaps on your hard disk if > your system came with Snow Leopard factory-installed or perhaps on a > restore DVD. ?If not, it's on the retail Snow Leopard DVD and can be > downloaded from the Apple Developer site. ?After launching the Xcode > installer, just select and install the "Mac OS 10.4 Support" package > from the Custom Install menu. > > Third, you need to tell Distutils to use the older gcc-4.0 instead of > the gcc-4.2 which is now the default on 10.6. > > $ cd /path/to/gmpy-1.11rc1 > $ export CC=/usr/bin/gcc-4.0 > $ /usr/local/bin/python3.1 setup.py install > ... > $ /usr/local/bin/python3.1 test3/gmpy_test.py > Unit tests for gmpy 1.11 > ? ? on Python 3.1.1 (r311:74543, Aug 24 2009, 18:44:04) > [GCC 4.0.1 (Apple Inc. build 5493)] > Testing gmpy 1.11 (GMP 4.3.1), default caching (100, 128) > ... > 1500 tests in 42 items. > 1500 passed and 0 failed. It would appear that all the above is unnecessary, as I have 64 bit Python from macports working with the 64 bit version of gmpy compiled by macports. Not only does it pass the unit test, but since it's 64 bits, I can reach beyond the "outrageous exponent error" that I've seen in 32 bit Windows versions. I'm told that gmp can't do more than 48 billion decimal digits, so I now hit that restriction before I reach the next level of "outrageous exponent". I just won't be able to properly run IDLE, but as I said, that's expendable. Anyway, thanks for pointing that out, didn't realize the python.org disk image was 32 bits. There was no point in downloading it at all. As I pointed out in the thread "the need for 64 bits", once I've seen Paris, there's no going back. > > -- > ?Ned Deily, > ?n... at acm.org From python at mrabarnett.plus.com Tue Jan 5 21:59:50 2010 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 06 Jan 2010 02:59:50 +0000 Subject: please help shrink this each_with_index() implementation In-Reply-To: References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> Message-ID: <4B43FCA6.8040101@mrabarnett.plus.com> Antoine Pitrou wrote: >> The point: int('') or int('something') both throw an error. In general, >> this is hand-holding, but in specific I don't think the "rich and >> structured" documentation will cover how to beat a 0 out of it in less >> than 3 lines. > > Because it's a bad idea to do so and Python doesn't encourage such sloppy > behaviour. If you like it, though, you might prefer PHP. > > By the way: error reporting is definitely *not* hand-holding. It is > simply not good to let errors pass silently by default. Again, if you > think the contrary, PHP is your friend ;-) > Someone wrote about a port of a Perl script to Python. The script would parse the output from a program it would call, but the Python script sometimes raised a ValueError when it tried to convert a certain field to a number. It turned out that sometimes the field would contain "ERROR" instead of a number. The Perl script had been silently 'converting' any such "ERROR" to 0! From wuwei23 at gmail.com Tue Jan 5 22:46:01 2010 From: wuwei23 at gmail.com (alex23) Date: Tue, 5 Jan 2010 19:46:01 -0800 (PST) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> Message-ID: <1715bea7-1ff1-4294-9dad-672873e14530@v25g2000yqk.googlegroups.com> Phlip wrote: > They will tell me how to use except: (which is a good example why a > program should not use exceptions for its normal control flow if at > all possible). Really? Magic functions that coerce and eat errors are a better coding technique than exceptions and explicit handling? What kool-aid have you been drinking? From adityashukla1983 at gmail.com Tue Jan 5 23:26:39 2010 From: adityashukla1983 at gmail.com (aditya shukla) Date: Tue, 5 Jan 2010 22:26:39 -0600 Subject: Do I have to use threads? Message-ID: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> Hello people, I have 5 directories corresponding 5 different urls .I want to download images from those urls and place them in the respective directories.I have to extract the contents and download them simultaneously.I can extract the contents and do then one by one. My questions is for doing it simultaneously do I have to use threads? Please point me in the right direction. Thanks Aditya -------------- next part -------------- An HTML attachment was scrubbed... URL: From philip at semanchuk.com Tue Jan 5 23:36:19 2010 From: philip at semanchuk.com (Philip Semanchuk) Date: Tue, 5 Jan 2010 23:36:19 -0500 Subject: Do I have to use threads? In-Reply-To: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> Message-ID: On Jan 5, 2010, at 11:26 PM, aditya shukla wrote: > Hello people, > > I have 5 directories corresponding 5 different urls .I want to > download > images from those urls and place them in the respective > directories.I have > to extract the contents and download them simultaneously.I can > extract the > contents and do then one by one. My questions is for doing it > simultaneously > do I have to use threads? No. You could spawn 5 copies of wget (or curl or a Python program that you've written). Whether or not that will perform better or be easier to code, debug and maintain depends on the other aspects of your program(s). bye Philip From steven at REMOVE.THIS.cybersource.com.au Tue Jan 5 23:48:30 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 06 Jan 2010 04:48:30 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: On Wed, 06 Jan 2010 00:58:58 +0000, r0g wrote: > Steven D'Aprano wrote: >> On Wed, 06 Jan 2010 09:39:08 +1100, Ben Finney wrote: >> >>> r0g writes: >>> >>>> Of course I'm now guilty of pedantry too :/ I might have let it slip >>>> had you not started your reply with the word "No", that just p***** >>>> me off. >>> Well, if being told ?no? is going to piss you off, I think you're in >>> for a rough time. >> >> Oh, you're in trouble now! If you think he gets upset at being told no, >> you should see how upset he gets at being told he's in for a rough >> time!!! >> >> *wink* >> >> >> >> > > NO! It's a rude way to start a sentence don't you think? No. I'm not Japanese, I don't feel any social prohibition at saying No in this context. I'm also happy to start a sentence with "You're wrong", and occasionally I give in to temptation to start it with "Are you on crack?". > Just because > you're correcting someone doesn't mean you have to be combative and try > and make them feel small. Unless they've adopted a hostile or wilfully > ignorant tone there's no reason to be so brusqe with people. You can be > both nice AND terse you know. Now I feel hurt that you're publicly rebuking me and making me feel as if I've done something wrong by trying to lighten the mood with a small joke... Nah, just kidding. -- Steven From steven at REMOVE.THIS.cybersource.com.au Tue Jan 5 23:49:52 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 06 Jan 2010 04:49:52 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: On Tue, 05 Jan 2010 15:51:29 -0800, Phlip wrote: > Why can't int('nonnumeric') return None? It could do that, but it shouldn't, because returning magic values instead of raising exceptions is usually a bad, bad idea. > (A related question - why can't I just go 'if record = method(): use > (record)'. Why extra lines just to trap and assign the variable before > using it?) Because that idiom is responsible for probably the most common error in C of all, at least one of the most common errors. Thank goodness Python forbids such a dangerous construct. -- Steven From steven at REMOVE.THIS.cybersource.com.au Tue Jan 5 23:52:12 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 06 Jan 2010 04:52:12 GMT Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> Message-ID: On Tue, 05 Jan 2010 15:30:09 -0800, Phlip wrote: >> > Does it say how to convert a string containing either an integer >> > representation, or something alphabetic, into an integer, or a zero, >> > in like 1 method call? (No except: ?) >> >> If you mean something like this: >> >> >>> int('153') >> >> 153 > > The point: int('') or int('something') both throw an error. Well, that's a bug. Obviously they should return 42. Or is that 23? I forget. But seriously... of course they do. What did you expect them to do? Any Python function will raise an exception if you pass invalid data to it. So the solutions are, don't pass invalid data, or wrap the function in a try block. If you don't want to use try, then make sure that your data is good. > In general, > this is hand-holding, but in specific I don't think the "rich and > structured" documentation will cover how to beat a 0 out of it in less > than 3 lines. So I will persist in my idiotic questions here! > >> Then perhaps you should work through the tutorial to learn the basics. > > They will tell me how to use except: (which is a good example why a > program should not use exceptions for its normal control flow if at all > possible). Huh? So because YOU don't know how to use except, programs shouldn't use exceptions for flow control? I reject that, and I will continue using exceptions for flow control when I think it is appropriate. > Please, please, please save your newbie admonitions for those who > qualify! Oh please, get off your high horse. You're asking newbie questions, no matter how many years of using Python you may or may not have, your experience is obviously low. Stick around and you might learn something, but if you bite every time somebody tries to teach you, you'll soon run out of people willing to help you. -- Steven From chris.gonnerman at newcenturycomputers.net Wed Jan 6 00:08:42 2010 From: chris.gonnerman at newcenturycomputers.net (Chris Gonnerman) Date: Tue, 05 Jan 2010 23:08:42 -0600 Subject: [Python] Re: Printing plain text with exact positioning on Windows In-Reply-To: <263f9c76-17fb-4398-b65c-5b1b0007682b@j5g2000yqm.googlegroups.com> References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> <263f9c76-17fb-4398-b65c-5b1b0007682b@j5g2000yqm.googlegroups.com> Message-ID: <4B441ADA.9090301@newcenturycomputers.net> KvS wrote: > Ok, actually I quite like being able to print straightforward through > your code, i.e. without any extra modules installed. I understand that > sending text to the printer is in principle as simple as > > dc.TextOut(scale_factor * 72, > -1 * scale_factor * 72, > "Testing...") > > I didn't see you do anything with adjusting margins in the code. Does > that mean that if I would e.g. do > > dc.TextOut(0, > 0, > "Testing...") > > the printout would appear in the upper left corner of the paper, as > close to the edges as the printer is capable of? (Sorry, but I only > have Ubuntu available at the moment, no Windows). > Actually, as I understand it (and I'll admit my understanding is a bit flawed in some areas), when you tell MSWinPrint.py to print at (0, 0), you are telling it to print at the actual paper edge; if that's outside the printable area (and it probably is for most printers), your printing will be cut off. Best thing to do is to try it. From chris.gonnerman at newcenturycomputers.net Wed Jan 6 00:10:04 2010 From: chris.gonnerman at newcenturycomputers.net (Chris Gonnerman) Date: Tue, 05 Jan 2010 23:10:04 -0600 Subject: [Python] Re: Printing plain text with exact positioning on Windows In-Reply-To: References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> Message-ID: <4B441B2C.9000702@newcenturycomputers.net> KvS wrote: > Sorry, one more. I completely forgot it's not exactly plain text, but > occasionally also a limited number of non-ASCII characters (accents in > names etc.). Would this be possible through your method? If Windows can print it, then MSWinPrint.py should be able to also. But I haven't tested that extensively. YMMV. From steven at REMOVE.THIS.cybersource.com.au Wed Jan 6 00:10:28 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 06 Jan 2010 05:10:28 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> Message-ID: On Tue, 05 Jan 2010 17:45:58 -0800, Phlip wrote: > On Jan 5, 5:01?pm, Chris Rebert wrote: > >> > Why can't int('nonnumeric') return None? >> >> Errors should never pass silently. > > You are saying I, as the programmer, cannot decide what is an error and > what is a pass-thru. The decision is made for me. Every function decides for you what is an error and what isn't. >>> max() Traceback (most recent call last): File "", line 1, in TypeError: max expected 1 arguments, got 0 But I wanted it to return (sys.maxint - 7). How DARE the creator of the language decide that it should be an error instead of returning the arbitrary result I choose! Not. > (Yes yes I can write int_or_None(), etc...) That's right. As a programmer, your job is to program. If a language doesn't provide a function you want, write your own using the primitives available to you. > Here's a super easy example: > > { 42: 'forty two' }.get(41, None) > > Because I can supply a default, I can decide what is an error and what > is . You can ALWAYS decide what is an error. d = {42: 'forty two'} try: d[41] except KeyError: print "All is good, no problems, dict does not contain 41" except: print "WARNING WARNING WARNING!!!" print "FATAL ERROR: dict contains 41!!!" sys.exit() You do so by programming. As a programmer, that's your job. > Now the equivalent in a language that does not enjoy this false "Zen": > > { 42: 'forty two' }[41] # returns None Suppose you have a dict d supplied from somewhere else. You don't know what's in it. You do this: d[41] and you get a result None. Does this mean that the dict looks like this? d = {} or like this? d = {41: None} > { 42: 'forty two' }.fetch(41, None) # ibid In Python, "fetch" is spelled "get". > { 42: 'forty two' }.fetch(41) # raises an exception In Python, that would be spelled {42: 'forty two'}[41] > The quicky validation is available if I _request_ it. When you write d[41] you are requesting it. If you don't want it, use get instead. >> Quibbling over a mere one more line of code (or writing one short >> function) seems a bit petty. > > Because that "Zen of Python" is an empty sophistry that forces me to add > a "mere one more line of code" over and over again... If you're writing that one line of code over and over again, that's a good sign that you're doing it wrong and should rethink your strategy. >> > (A related question - why can't I just go 'if record = method(): ?use >> > (record)'. Why extra lines just to trap and assign the variable >> > before using it?) >> >> I believe that's disallowed so as to prevent the subtle bugs seen in C >> code which result from when someone makes a typo and omits the second >> "=" in their `if foo == bar():` test. > > Don't prevent me from using a technique just because others had trouble > with it. Any language allows and prevents certain techniques, simply by the very nature of the language. Every language has it's own syntax: you can't write weakly-typed stack-based concatenative code (like Forth) in Java, or strongly-typed dynamic object-oriented Python code in Pascal. Every language forces the programmer to use some features and avoid others. > And if bar() == foo is the superior technique anyway, because the == > happens in chronological and lexical order after the bar() call. That makes no sense. -- Steven From rodrick.brown at gmail.com Wed Jan 6 00:40:39 2010 From: rodrick.brown at gmail.com (Rodrick Brown) Date: Wed, 6 Jan 2010 00:40:39 -0500 Subject: Do I have to use threads? In-Reply-To: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> Message-ID: On Tue, Jan 5, 2010 at 11:26 PM, aditya shukla wrote: > Hello people, > > I have 5 directories corresponding 5 different urls .I want to download > images from those urls and place them in the respective directories.I have > to extract the contents and download them simultaneously.I can extract the > contents and do then one by one. My questions is for doing it simultaneously > do I have to use threads? > > Please point me in the right direction. > > Threads in python are very easy to work with but not very efficient and for most cases slower than running multiple processes. Look at using multiple processes instead of going with threads performance will be much better. > Thanks > > Aditya > > -- > http://mail.python.org/mailman/listinfo/python-list > > -- [ Rodrick R. Brown ] http://www.rodrickbrown.com http://www.linkedin.com/in/rodrickbrown -------------- next part -------------- An HTML attachment was scrubbed... URL: From Brian.Mingus at Colorado.EDU Wed Jan 6 00:45:14 2010 From: Brian.Mingus at Colorado.EDU (Brian J Mingus) Date: Tue, 5 Jan 2010 22:45:14 -0700 Subject: Do I have to use threads? In-Reply-To: References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> Message-ID: <9839a05c1001052145j663f9f73nfac6b3cdeec411fe@mail.gmail.com> On Tue, Jan 5, 2010 at 9:36 PM, Philip Semanchuk wrote: > > On Jan 5, 2010, at 11:26 PM, aditya shukla wrote: > > Hello people, >> >> I have 5 directories corresponding 5 different urls .I want to download >> images from those urls and place them in the respective directories.I have >> to extract the contents and download them simultaneously.I can extract the >> contents and do then one by one. My questions is for doing it >> simultaneously >> do I have to use threads? >> > > No. You could spawn 5 copies of wget (or curl or a Python program that > you've written). Whether or not that will perform better or be easier to > code, debug and maintain depends on the other aspects of your program(s). > > bye > Philip Obviously, spawning 5 copies of wget is equivalent to starting 5 threads. The answer is 'yes'. -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank at chagford.com Wed Jan 6 00:57:08 2010 From: frank at chagford.com (Frank Millman) Date: Wed, 6 Jan 2010 07:57:08 +0200 Subject: Minor bug in multiprocessing? References: Message-ID: "Aahz" wrote: > Frank Millman wrote: >> >>Is this worth reporting, if it has not been reported already? > > Defiitely report it. Thanks, Aahz. I took the lack of responses to indicate that there was no reason *not* to report it, so I reported it on 24th December (issue 7571), and it was fixed on the same day (r77038). Frank From victorsubervi at gmail.com Wed Jan 6 01:21:47 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 6 Jan 2010 01:21:47 -0500 Subject: chown'ing by script Message-ID: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> Hi; I have a script that is called via the web. This script writes another script that is also called by the web, which in turn needs to have execution privileges. The problem is that the programmatically created file is owned by apache.apache and thus doesn't have execution privileges. I've tried os.chown(...) but this throws an OSError. I understand that chown'ing programmatically opens a big security hole. However, setting the gid to give apache execution privileges isn't any better. What do you suggest? TIA, beno -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From adityashukla1983 at gmail.com Wed Jan 6 01:21:58 2010 From: adityashukla1983 at gmail.com (aditya shukla) Date: Wed, 6 Jan 2010 00:21:58 -0600 Subject: Do I have to use threads? In-Reply-To: References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> Message-ID: <73045cca1001052221r18a17b3by28b650039d720fec@mail.gmail.com> Thanks.i will look into multiprocessing. Aditya -------------- next part -------------- An HTML attachment was scrubbed... URL: From cs at zip.com.au Wed Jan 6 01:29:10 2010 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 6 Jan 2010 17:29:10 +1100 Subject: chown'ing by script In-Reply-To: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> References: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> Message-ID: <20100106062910.GA18453@cskk.homeip.net> On 06Jan2010 01:21, Victor Subervi wrote: | I have a script that is called via the web. This script writes another | script that is also called by the web, which in turn needs to have execution | privileges. The problem is that the programmatically created file is owned | by apache.apache and thus doesn't have execution privileges. I've tried | os.chown(...) but this throws an OSError. I understand that chown'ing | programmatically opens a big security hole. However, setting the gid to give | apache execution privileges isn't any better. What do you suggest? Are you sure you don't want chmod? -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Winter is gods' way of telling us to polish. - Peter Harper From carsten.haese at gmail.com Wed Jan 6 01:41:57 2010 From: carsten.haese at gmail.com (Carsten Haese) Date: Wed, 06 Jan 2010 01:41:57 -0500 Subject: chown'ing by script In-Reply-To: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> References: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> Message-ID: Victor Subervi wrote: > Hi; > I have a script that is called via the web. This script writes another > script that is also called by the web, which in turn needs to have > execution privileges. The problem is that the programmatically created > file is owned by apache.apache and thus doesn't have execution > privileges. I've tried os.chown(...) but this throws an OSError. I > understand that chown'ing programmatically opens a big security hole. > However, setting the gid to give apache execution privileges isn't any > better. What do you suggest? I suggest you find a way to achieve whatever it is you are trying to achieve without having to execute programmatically created scripts. What is the underlying problem you're trying to solve with this approach? -Carsten From wuwei23 at gmail.com Wed Jan 6 01:48:59 2010 From: wuwei23 at gmail.com (alex23) Date: Tue, 5 Jan 2010 22:48:59 -0800 (PST) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> Message-ID: <51fbdc79-f093-497b-9a98-ce4636cb5610@j14g2000yqm.googlegroups.com> Steven D'Aprano wrote: > Stick around and you might learn something, > but if you bite every time somebody tries to teach you, you'll soon run > out of people willing to help you. The ongoing crowdsourced development by this group of "Victor Subervi's" project would seem to indicate that this isn't the case. Or that if it is, that it will be a good handful of months at the very soonest before it becomes true. From half.italian at gmail.com Wed Jan 6 01:50:10 2010 From: half.italian at gmail.com (Sean DiZazzo) Date: Tue, 5 Jan 2010 22:50:10 -0800 (PST) Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> <96ea39f2-1fb6-4b03-b206-d4c1445ea021@v25g2000yqk.googlegroups.com> Message-ID: On Jan 5, 11:40?am, KvS wrote: > On Jan 5, 7:16?pm, Nobody wrote: > > > > > On Tue, 05 Jan 2010 04:40:14 -0800, KvS wrote: > > >> Did you mean borderless printing? > > >> Every printer needs his margins, some more some less. Some printers have the > > >> ability to do borderless printing but usualy they can do it only on special > > >> or photo paper. So you can adjust the pdf as you wish, even with no margins, > > >> and then try to find under printer options "borderless printing". That is > > >> why I didn't understand :-)) it is a printer thing not pdf! > > > > As much as possible "borderless", yes. Of course the printer will > > > still apply some small margin, but that's ok. A margin of say <0.5 cm. > > > is fine. So it's not a printer thing, I accept the (physical) > > > limitations of the printer, but I want to avoid any extra margins due > > > to software settings. > > > "Hardcopy" document formats such as PostScript and PDF use positions > > relative to the edges of the page, not the margins. > > Right. Still, Acrobat Reader by default scales the contents to fit on > a page and creates some margins by doing so, no? So if my text is > close to the left and right edges, as I want, it will get scaled and > extra margins will occur. Avoiding this still requires me to be able > to turn off this scaling in the printing preferences somehow > programmatically, so it doesn't seem to make the problem easier? Maybe you could have the user print your data on a larger sheet of paper;one that is sure to include all of the data, and include crop marks on the printout. The user then cuts along the crop marks to leave a perfectly sized, marginless page. This is how printers do bleeds. From benjamin.kaplan at case.edu Wed Jan 6 01:54:17 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Wed, 6 Jan 2010 01:54:17 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> Message-ID: On Tue, Jan 5, 2010 at 8:45 PM, Phlip wrote: > > Here's a super easy example: > > ?{ 42: 'forty two' }.get(41, None) > > Because I can supply a default, I can decide what is an error and what > is . > > Now the equivalent in a language that does not enjoy this false "Zen": > > ?{ 42: 'forty two' }[41] ?# returns None > ?{ 42: 'forty two' }.fetch(41, None) ?# ibid > ?{ 42: 'forty two' }.fetch(41) ?# raises an exception Here's another super easy example. Should {42 : 'forty two'}[41] be treated the same as {41: None}[41] ? In cases where None is a valid result, you can't use it to signal failure. From victorsubervi at gmail.com Wed Jan 6 02:11:31 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 6 Jan 2010 02:11:31 -0500 Subject: TypeError Message-ID: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> Hi; I get this error: /var/www/html/angrynates.com/christians/cart/simplemail/mail.py 153 154 ''' 155 commitSale() 156 myMail() 157 print ''' commitSale = /var/www/html/angrynates.com/christians/cart/simplemail/mail.py in commitSale() 98 cursor.execute('select max(ID) from %sCustomerData;' % store) 99 custID = cursor.fetchone()[0] 100 customerData(store, tmpTable, custID, patientID) 101 102 def myMail(): global customerData = , global store = 'products', tmpTable = 'tem12627568064', custID = 1, global patientID = 'None' /var/www/html/angrynates.com/christians/cart/customerData.py in customerData(store='products', tmpTable='tem12627568064', custID=1, patientID='None') 39 40 """ 41 print """ 42 print '

%s Customer Data

' % (store[0].upper() + store[1:]) 43 cursor.execute('describe %sCustomerData' % store) store = 'products' ValueError: unsupported format character '(' (0x28) at index 54 args = ("unsupported format character '(' (0x28) at index 54",) Apparently that character is a "file separator", which I presume is an invisible character. I tried retyping the area in question, but with no avail (threw same error). Please advise. Complete code follows. TIA, beno #!/usr/bin/python import sys,os sys.path.append(os.getcwd()) import MySQLdb from login import login import re, string def customerData(store, tmpTable, custID, patientID): user, passwd, db, host = login() db = MySQLdb.connect(host, user, passwd, db) cursor = db.cursor() page= """#!/usr/bin/python import sys,os sys.path.append(os.getcwd()) import MySQLdb from login import login import re, string from particulars import ourStores, ourOptions import fpformat from sets import Set from processOrder import processOrder def %sCustomerData(): print 'Content-Type: text/html' print print ''' """ print """ print '

%s Customer Data

' % (store[0].upper() + store[1:]) cursor.execute('describe %sCustomerData' % store) cols = [itm[0] for itm in cursor] cursor.execute('select * from %sCustomerData where ID=%s' % (store, custID)) ourCustomerData = cursor.fetchone() ourColsAndCustomerDataDict = dict(zip(cols, ourCustomerData)) for col, data in ourColsAndCustomerDataDict.iteritems(): print '%s: %s
' % (col, data) print "

Customer Order

" print "\n " numberShippingFields, totalTotal, html = processOrder(patientID, store, tmpTable) print html print " " if store != 'prescriptions': numberCols = 12 + numberShippingFields else: numberCols = 8 + numberShippingFields print ' \n \n \n ' % (numberCols - 1, str(fpformat.fix(round(int(totalTotal * 100))/100,2))) print "
TOTAL$%s
" cursor.close() print ''' ''' %sCustomerData() """ % (store, store) os.chdir('..') try: os.delete('%s/%sCustomerData.py' % (os.getcwd(), store)) except AttributeError: pass file = '%sCustomerData.py' % store f = open(file, 'w') f.write(page) f.close() # os.chown('%s/%s' % (os.getcwd(), file), 500, 500) os.chmod('%s/%s' % (os.getcwd(), file), 0755) -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From wuwei23 at gmail.com Wed Jan 6 02:12:00 2010 From: wuwei23 at gmail.com (alex23) Date: Tue, 5 Jan 2010 23:12:00 -0800 (PST) Subject: chown'ing by script References: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> Message-ID: Carsten Haese wrote: > What is the underlying problem you're trying to solve with this > approach? To be paid for developing a web site shopping cart without actually having to learn Python. From victorsubervi at gmail.com Wed Jan 6 02:14:11 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 6 Jan 2010 02:14:11 -0500 Subject: chown'ing by script In-Reply-To: References: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> Message-ID: <4dc0cfea1001052314s77f23c84w3b331bfdaaf71d89@mail.gmail.com> On Wed, Jan 6, 2010 at 1:41 AM, Carsten Haese wrote: > Victor Subervi wrote: > > Hi; > > I have a script that is called via the web. This script writes another > > script that is also called by the web, which in turn needs to have > > execution privileges. The problem is that the programmatically created > > file is owned by apache.apache and thus doesn't have execution > > privileges. I've tried os.chown(...) but this throws an OSError. I > > understand that chown'ing programmatically opens a big security hole. > > However, setting the gid to give apache execution privileges isn't any > > better. What do you suggest? > > I suggest you find a way to achieve whatever it is you are trying to > achieve without having to execute programmatically created scripts. What > is the underlying problem you're trying to solve with this approach? > As I was writing to respond, I re-thought the problem out and now I don't need any more advice...at least on this issue ;) Thanks! beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From gherron at islandtraining.com Wed Jan 6 02:29:20 2010 From: gherron at islandtraining.com (Gary Herron) Date: Tue, 05 Jan 2010 23:29:20 -0800 Subject: Do I have to use threads? In-Reply-To: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> Message-ID: <4B443BD0.8020704@islandtraining.com> aditya shukla wrote: > Hello people, > > I have 5 directories corresponding 5 different urls .I want to > download images from those urls and place them in the respective > directories.I have to extract the contents and download them > simultaneously.I can extract the contents and do then one by one. My > questions is for doing it simultaneously do I have to use threads? > > Please point me in the right direction. > > > Thanks > > Aditya You've been given some bad advice here. First -- threads are lighter-weight than processes, so threads are probably *more* efficient. However, with only five thread/processes, the difference is probably not noticeable. (If the prejudice against threads comes from concerns over the GIL -- that also is a misplaced concern in this instance. Since you only have network connection, you will receive only one packet at a time, so only one thread will be active at a time. If the extraction process uses a significant enough amount of CPU time so that the extractions are all running at the same time *AND* if you are running on a machine with separate CPU/cores *AND* you would like the extractions to be running truly in parallel on those separate cores, *THEN*, and only then, will processes be more efficient than threads.) Second, running 5 wgets is equivalent to 5 processes not 5 threads. And third -- you don't have to use either threads *or* processes. There is another possibility which is much more light-weight: asynchronous I/O, available through the low level select module, or more usefully via the higher-level asyncore module. (Although the learning curve might trip you up, and some people find the programming model for asyncore hard to fathom, I find it more intuitive in this case than threads/processes.) In fact, the asyncore manual page has a ~20 line class which implements a web page retrieval. You could replace that example's single call to http_client with five calls, one for each of your ULRs. Then when you enter the last line (that is the asyncore.loop() call) the five will be downloading simultaneously. See http://docs.python.org/library/asyncore.html Gary Herron From ashish.vyas at motorola.com Wed Jan 6 03:51:46 2010 From: ashish.vyas at motorola.com (VYAS ASHISH M-NTB837) Date: Wed, 6 Jan 2010 16:51:46 +0800 Subject: lxml 2.2.4 on python3.1, Windows XP gives importerror In-Reply-To: <7C57DB58C81FB64CA979EC6C4DB73E7A03E7039F@ZMY16EXM67.ds.mot.com> Message-ID: <7C57DB58C81FB64CA979EC6C4DB73E7A03E7053F@ZMY16EXM67.ds.mot.com> Posting again as I did not get any response: Dear All I have Python 3.1 installed on Windows XP and Works nice. I downloaded lxml 2.2.4 (lxml-2.2.4.win32-py3.1.exe) from pypi. When I try: from lxml import etree I get: ImportError: DLL load failed: This application has failed to start because the application configuration is incorrect. Reinstalling the application may fix this problem. For information: 'import lxml' works fine. After reinstalling python3.1 also the error message is the same. Any help is appreciated! Regards, Ashish Vyas -------------- next part -------------- An HTML attachment was scrubbed... URL: From __peter__ at web.de Wed Jan 6 04:01:42 2010 From: __peter__ at web.de (Peter Otten) Date: Wed, 06 Jan 2010 10:01:42 +0100 Subject: unittest inconsistent References: <3dede91a-ab3c-42c0-af45-adde220ededd@z2g2000vbf.googlegroups.com> Message-ID: Andr? wrote: > On Jan 5, 8:14 pm, Matt Haggard wrote: >> Can anyone tell me why this test fails? >> >> http://pastebin.com/f20039b17 >> >> This is a minimal example of a much more complex thing I'm trying to >> do. I'm trying to hijack a function and inspect the args passed to it >> by another function. >> >> The reason the 'Tester' object has no attribute 'arg1' is because >> "self" still refers to the object made for testA. > > Quick answer: change faketest.py as follows: > > #-------------------------------------------------- > # faketest.py > #-------------------------------------------------- > > #from importme import render > import importme > > def run(somearg): > return importme.render(somearg) > > ========= > A long answer, with explanation, will cost you twice as much ;-) > (but will have to wait) > > Andr? Or you figure it out yourself staring at >>> import os >>> from os import rename >>> os.rename = 42 >>> rename >>> os.rename 42 from module import name binds the object referred to by module.name to the name variable in the current module. You can think of it as a shortcut for import module name = module.name del module When you later rebind import module module.name = something_else the reference in the current module isn't magically updated to point to something_else. Peter From ben+python at benfinney.id.au Wed Jan 6 04:15:33 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 06 Jan 2010 20:15:33 +1100 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: <87ljgbwq4q.fsf@benfinney.id.au> r0g writes: > NO! It's a rude way to start a sentence don't you think? Shouting is usually rude, yes. > Just because you're correcting someone doesn't mean you have to be > combative and try and make them feel small. Again, you're reading something that isn't there. I utterly deny the motives you're imputing. > I can't imagine why I expect good manners on usenet though, AFAICT > it's never been like that (well not since I got on it anyway). I'll reiterate that you can address this by paying more attention to what is actually written, and endeavouring to avoid finding emotions or motives that are likely not in the message. -- \ ?The greatest tragedy in mankind's entire history may be the | `\ hijacking of morality by religion.? ?Arthur C. Clarke, 1991 | _o__) | Ben Finney From victorsubervi at gmail.com Wed Jan 6 04:33:25 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 6 Jan 2010 04:33:25 -0500 Subject: chown'ing by script In-Reply-To: References: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> Message-ID: <4dc0cfea1001060133q35dd0e4dk4a905e6957ff86a2@mail.gmail.com> On Wed, Jan 6, 2010 at 2:12 AM, alex23 wrote: > Carsten Haese wrote: > > What is the underlying problem you're trying to solve with this > > approach? > > To be paid for developing a web site shopping cart without actually > having to learn Python. > LOL! I've written about 12,000 lines of code in this, BTW. Crawl back into your cave :) beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsagert at gmail.com Wed Jan 6 05:11:49 2010 From: bsagert at gmail.com (Bill) Date: Wed, 6 Jan 2010 02:11:49 -0800 (PST) Subject: Need help to pass self.count to other classes. Message-ID: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> After a year with Python 2.5 on my Windows box, I still have trouble understanding classes. Below, see the batch file and the configuration script for my Python interactive prompt. The widths of the secondary prompts increase when the self.count of SysPrompt1 exceeds 99. I am using a global variable "zxc" to share self.count, which is not Pythonic. How can I pass in self.count without a global? I did RTFM, aka Google, but to no avail. echo py.bat set PYTHONSTARTUP=c:\scripts\startup.py python ^Z # startup.py # inspired by: # http://www.doughellmann.com/PyMOTW/sys/interpreter.html import sys class SysPrompt1(object): def __init__(self): self.count = 0 def __str__(self): self.count += 1 global zxc zxc = self.count return '[%2d]> ' % self.count class SysPrompt2(object): def __str__(self): global zxc if zxc > 99: return '...... ' else: return '..... ' class DisplayHook(object): def __call__(self, value): if value is None: return global zxc if zxc > 99: print '[ out]', value, '\n' else: print '[out]', value, '\n' class ExceptHook(object): def __call__(self, type, value, trace): global zxc if zxc > 99: print '[ err]', value, '\n' else: print '[err]', value, '\n' sys.ps1 = SysPrompt1() sys.ps2 = SysPrompt2() sys.displayhook = DisplayHook() sys.excepthook = ExceptHook() From bieffe62 at gmail.com Wed Jan 6 05:43:06 2010 From: bieffe62 at gmail.com (Francesco Bochicchio) Date: Wed, 6 Jan 2010 02:43:06 -0800 (PST) Subject: Need help to pass self.count to other classes. References: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> Message-ID: <987471cd-2bf1-4eb7-af77-e52abae82f16@o28g2000yqh.googlegroups.com> On 6 Gen, 11:11, Bill wrote: > After a year with Python 2.5 on my Windows box, I still have trouble > understanding classes. > > Below, see the batch file and the configuration script for > my Python interactive prompt. > > The widths of the secondary prompts increase when ?the self.count of > SysPrompt1 exceeds 99. > > I am using a global variable "zxc" to share self.count, which is not > Pythonic. > > How can I pass in self.count without a global? > I did RTFM, aka Google, but to no avail. > > echo py.bat > set PYTHONSTARTUP=c:\scripts\startup.py > python > ^Z > > # startup.py > # inspired by: > #http://www.doughellmann.com/PyMOTW/sys/interpreter.html > > import sys > > class SysPrompt1(object): > ? ? def __init__(self): > ? ? ? ? self.count = 0 > ? ? def __str__(self): > ? ? ? ? self.count += 1 > ? ? ? ? global zxc > ? ? ? ? zxc = self.count > ? ? ? ? return '[%2d]> ' % self.count > > class SysPrompt2(object): > ? ? def __str__(self): > ? ? ? ? global zxc > ? ? ? ? if zxc > 99: return '...... ' > ? ? ? ? else: return '..... ' > > class DisplayHook(object): > ? ? def __call__(self, value): > ? ? ? ? if value is None: return > ? ? ? ? global zxc > ? ? ? ? if zxc > 99: print '[ out]', value, '\n' > ? ? ? ? else: print '[out]', value, '\n' > > class ExceptHook(object): > ? ? def __call__(self, type, value, trace): > ? ? ? ? global zxc > ? ? ? ? if zxc > 99: print '[ err]', value, '\n' > ? ? ? ? else: print '[err]', value, '\n' > > sys.ps1 = SysPrompt1() > sys.ps2 = SysPrompt2() > sys.displayhook = DisplayHook() > sys.excepthook = ExceptHook() First of all, you shouldn't do OOP if you don't feel it. Python, unlike Java and like C++, supports also procedural programming, so you can write your scripts without writing classes (but still using objects, since all in python is an object). If you have classes with no data and a single __call_ method, then they are no classes, they are functions (or methods) in disguise. So, one solution could be to use plain functions and use global as you already do. 'global' is pythonic if you are not doing OOP, although I don't like it. If you want to stick to OOP, then I suggest to have a make display_hook and except_hook methods of your class SysPrompt1; This way all your code can access to self.count without needing globals. As for your class SysPrompt2, I don't understand enough your code to know what you are trying to do. maybe make it a sunclass of SysPrompt1 ? HTH Ciao ---- FB From gtu2003 at alice.it Wed Jan 6 06:03:36 2010 From: gtu2003 at alice.it (wiso) Date: Wed, 06 Jan 2010 12:03:36 +0100 Subject: Convert month name to month number faster Message-ID: <4b446e09$0$1107$4fafbaef@reader4.news.tin.it> I'm optimizing the inner most loop of my script. I need to convert month name to month number. I'm using python 2.6 on linux x64. month_dict = {"Jan":1,"Feb":2,"Mar":3,"Apr":4, "May":5, "Jun":6, "Jul":7,"Aug":8,"Sep":9,"Oct":10,"Nov":11,"Dec":12} def to_dict(name): return month_dict[name] def to_if(name): if name == "Jan": return 1 elif name == "Feb": return 2 elif name == "Mar": return 3 elif name == "Apr": return 4 elif name == "May": return 5 elif name == "Jun": return 6 elif name == "Jul": return 7 elif name == "Aug": return 8 elif name == "Sep": return 9 elif name == "Oct": return 10 elif name == "Nov": return 11 elif name == "Dec": return 12 else: raise ValueError import random l = [random.choice(month_dict.keys()) for _ in range(1000000)] from time import time t = time(); xxx=map(to_dict,l); print time() - t # 0.5 t = time(); xxx=map(to_if,l); print time() - t # 1.0 is there a faster solution? Maybe something with str.translate? The problem is a little different because I don't read random data, but sorted data. For example: l = [x for x in ("Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec") for _ in range(1000)] # ["Jan","Jan", ..., "Feb", "Feb", ...] so maybe the to_if approach will be faster if I write the case in the best order. Look: l = ["Jan"] * 1000000 # to_if is in the best order for "Jan" t = time(); xxx=map(to_dict,l); print time() - t # 0.5 t = time(); xxx=map(to_if,l); print time() - t # 0.5 From ashish.vyas at motorola.com Wed Jan 6 06:14:04 2010 From: ashish.vyas at motorola.com (VYAS ASHISH M-NTB837) Date: Wed, 6 Jan 2010 19:14:04 +0800 Subject: Convert month name to month number faster In-Reply-To: <4b446e09$0$1107$4fafbaef@reader4.news.tin.it> Message-ID: <7C57DB58C81FB64CA979EC6C4DB73E7A03E70581@ZMY16EXM67.ds.mot.com> How about using list.index() and storing month names in a list? You may want to measure performance your self and conclude. Regards, Ashish Vyas -----Original Message----- From: python-list-bounces+ntb837=motorola.com at python.org [mailto:python-list-bounces+ntb837=motorola.com at python.org] On Behalf Of wiso Sent: Wednesday, January 06, 2010 4:34 PM To: python-list at python.org Subject: Convert month name to month number faster I'm optimizing the inner most loop of my script. I need to convert month name to month number. I'm using python 2.6 on linux x64. month_dict = {"Jan":1,"Feb":2,"Mar":3,"Apr":4, "May":5, "Jun":6, "Jul":7,"Aug":8,"Sep":9,"Oct":10,"Nov":11,"Dec":12} def to_dict(name): return month_dict[name] def to_if(name): if name == "Jan": return 1 elif name == "Feb": return 2 elif name == "Mar": return 3 elif name == "Apr": return 4 elif name == "May": return 5 elif name == "Jun": return 6 elif name == "Jul": return 7 elif name == "Aug": return 8 elif name == "Sep": return 9 elif name == "Oct": return 10 elif name == "Nov": return 11 elif name == "Dec": return 12 else: raise ValueError import random l = [random.choice(month_dict.keys()) for _ in range(1000000)] from time import time t = time(); xxx=map(to_dict,l); print time() - t # 0.5 t = time(); xxx=map(to_if,l); print time() - t # 1.0 is there a faster solution? Maybe something with str.translate? The problem is a little different because I don't read random data, but sorted data. For example: l = [x for x in ("Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec" ) for _ in range(1000)] # ["Jan","Jan", ..., "Feb", "Feb", ...] so maybe the to_if approach will be faster if I write the case in the best order. Look: l = ["Jan"] * 1000000 # to_if is in the best order for "Jan" t = time(); xxx=map(to_dict,l); print time() - t # 0.5 t = time(); xxx=map(to_if,l); print time() - t # 0.5 -- http://mail.python.org/mailman/listinfo/python-list From solipsis at pitrou.net Wed Jan 6 06:53:46 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 6 Jan 2010 11:53:46 +0000 (UTC) Subject: Convert month name to month number faster References: <4b446e09$0$1107$4fafbaef@reader4.news.tin.it> Message-ID: Le Wed, 06 Jan 2010 12:03:36 +0100, wiso a ?crit?: > from time import time > t = time(); xxx=map(to_dict,l); print time() - t # 0.5 t = time(); > xxx=map(to_if,l); print time() - t # 1.0 Don't define your own function just for attribute access. Instead just write: xxx = map(month_dict.__getitem__, l) From gerard.blais at gmail.com Wed Jan 6 06:54:17 2010 From: gerard.blais at gmail.com (Gerry) Date: Wed, 6 Jan 2010 03:54:17 -0800 (PST) Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> <96ea39f2-1fb6-4b03-b206-d4c1445ea021@v25g2000yqk.googlegroups.com> Message-ID: <4a4634ea-95b0-4b6e-ba35-bc12cd8e7cbe@21g2000vbh.googlegroups.com> If this is, by any chance, an HP printer, the printer may support PCL 5 (or a similar language). I've written PCL scripts (in Pascal, so a while ago) to precisely print points at the printer resolution (i.e., I picked which six-hundredth of an inch in height and width dimensions to print a dot. PCL 5 absolutely would let you put text anywhere you wanted, including the absolute limits on margins that the hardware supports. (PCL is printer control language). Gerry From kk720801 at yahoo.com Wed Jan 6 06:57:52 2010 From: kk720801 at yahoo.com (Krzysztof Kobus) Date: Wed, 6 Jan 2010 03:57:52 -0800 (PST) Subject: No subject Message-ID: <433599.23145.qm@web53502.mail.re2.yahoo.com> Hi, > Well, it seems that one of your files is a different architecture than > the others. Based on the location, I'd say it's i386 while the rest of > it would be PowerPC. You can cross-compile but you can't link an i386 > library to a PowerPC library. Thank you for the hint. I have checked with "file" command the format of the libraries on mac and got following result: one of my application's libraries: libkkbase.dylib: Mach-O dynamically linked shared library i386 my python module that has been built using standard python distutils: j3k.so: Mach-O bundle i386 I am not actually sure what's the difference between "bundle" and "dynamically linked shared library" on mac and if they can be linked together? Perhaps I should configure distutils on mac somehow specially to make the module ready for embedding? Thanks, KK On Tue, Jan 5, 2010 at 7:33 AM, Krzysztof Kobus wrote: > Hi, > > I have a problem with linking python module with my application on mac in order to make the module available in "embedded python". > > My python module is contained in j3kmodule.cxx file and module initialization function is exported in j3kmodule.h > > j3kmodule.h: > ------------ > PyMODINIT_FUNC PyInit_j3k(void); > > > j3kmodule.cxx: > -------------- > PyMODINIT_FUNC > PyInit_j3k(void) > { > PyObject *m = NULL; > > if ((m = PyModule_Create(&j3k_module)) == NULL) > return NULL; > > return m; > } > > > Then in my application in KkPython.cxx file I have: > > KkPython.cxx: > ------------- > > #include "j3kmodule.h" > > /* Add a builtin module, before Py_Initialize */ > PyImport_AppendInittab("j3k", PyInit_j3k); > > > I link my application with the module and get following linking error on mac although on open suse linux exactly the same procedure works fine: > > g++ -headerpad_max_install_names -o ../../bin/render.app/Contents/MacOS/render main.o KkPython.o -lkkbase -L/Users/kk/dev/J3K/trunk/j3ksrc/examples/render/../../lib/ -lQtSolutions_PropertyBrowser-2.5 -lpython3.1 /Users/kk/dev/J3K/trunk/j3ksrc/python/modules/build/lib.macosx-10.3-i386-3.1/j3k.so -framework OpenGL -framework AGL > ld: warning in /Users/kk/dev/J3K/trunk/j3ksrc/python/modules/build/lib.macosx-10.3-i386-3.1/j3k.so, file is not of required architecture > Undefined symbols: > "_PyInit_j3k", referenced from: > _PyInit_j3k$non_lazy_ptr in KkPython.o > ld: symbol(s) not found > collect2: ld returned 1 exit status > make: *** [../../bin/render.app/Contents/MacOS/render] Error 1 > > > I appreciate any hints, > > best greetings, > > Krzysztof Kobus > From sccolbert at gmail.com Wed Jan 6 06:57:58 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Wed, 6 Jan 2010 12:57:58 +0100 Subject: how to change when the logging module creates the log file? Message-ID: <7f014ea61001060357r62567020wb82aa92940aa5763@mail.gmail.com> I have an application the writes to a log file when specific exceptions are handled. However, if no exceptions are encountered, I don't want to create a log at all. The problem I am running into is that the stdlib logging module creates the log file immediately upon logger instantiation. Thus: >>> logger = logging.basicConifg('testlog.txt') already creates the file 'testlog.txt'. So, at program close, I am reading the size of the log file, and if it is empty I remove it with os.remove(). This works fine on Linux, but throws a permission denied exception on Windows. There has to be a better way to do this than using a hack like that. Is there a way to make the logging module hold-off on file creation until the first log is generated? I could do it by wrapping logger in a class, but that would remove the beauty of having any module import logging from the stdlib and being able to write to the log. Thanks for any pointers! Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: From wuwei23 at gmail.com Wed Jan 6 06:58:39 2010 From: wuwei23 at gmail.com (alex23) Date: Wed, 6 Jan 2010 03:58:39 -0800 (PST) Subject: Convert month name to month number faster References: <4b446e09$0$1107$4fafbaef@reader4.news.tin.it> Message-ID: <126bd455-52f2-4e92-8427-1982b4de925e@b2g2000yqi.googlegroups.com> On Jan 6, 9:03?pm, wiso wrote: > I'm optimizing the inner most loop of my script. I need to convert month > name to month number. I'm using python 2.6 on linux x64. > > month_dict = {"Jan":1,"Feb":2,"Mar":3,"Apr":4, "May":5, "Jun":6, > ? ? ? ? ? ?"Jul":7,"Aug":8,"Sep":9,"Oct":10,"Nov":11,"Dec":12} > > def to_dict(name): > ? return month_dict[name] Try replacing the to_dict function with: to_dict = month_dict.get That removes one extra function call per lookup. On my computer, this reduces the time for your test from 0.26 to 0.09. From kk720801 at yahoo.com Wed Jan 6 06:59:50 2010 From: kk720801 at yahoo.com (Krzysztof Kobus) Date: Wed, 6 Jan 2010 03:59:50 -0800 (PST) Subject: embedded python on mac - linking problem Message-ID: <921541.24413.qm@web53506.mail.re2.yahoo.com> Hi, > > Well, it seems that one of your files is a different architecture than > > the others. Based on the location, I'd say it's i386 while the rest of > > it would be PowerPC. You can cross-compile but you can't link an i386 > > library to a PowerPC library. Thank you for the hint. I have checked with "file" command the format of the libraries on mac and got following result: one of my application's libraries: libkkbase.dylib: Mach-O dynamically linked shared library i386 my python module that has been built using standard python distutils: j3k.so: Mach-O bundle i386 I am not actually sure what's the difference between "bundle" and "dynamically linked shared library" on mac and if they can be linked together? Perhaps I should configure distutils on mac somehow specially to make the module ready for embedding? Thanks, KK > On Tue, Jan 5, 2010 at 7:33 AM, Krzysztof Kobus wrote: > > Hi, > > > > > I have a problem with linking python module with my application on mac > in order to make the module available in "embedded python". > > > > My python module is contained in j3kmodule.cxx file and module initialization > function is exported in j3kmodule.h > > > > j3kmodule.h: > > ------------ > > PyMODINIT_FUNC PyInit_j3k(void); > > > > > > j3kmodule.cxx: > > -------------- > > PyMODINIT_FUNC > > PyInit_j3k(void) > > { > > PyObject *m = NULL; > > > > if ((m = PyModule_Create(&j3k_module)) == NULL) > > return NULL; > > > > return m; > > } > > > > > > Then in my application in KkPython.cxx file I have: > > > > KkPython.cxx: > > ------------- > > > > #include "j3kmodule.h" > > > > /* Add a builtin module, before Py_Initialize */ > > PyImport_AppendInittab("j3k", PyInit_j3k); > > > > > > > I link my application with the module and get following linking error > on mac although on open suse linux exactly the same procedure works > fine: > > > > g++ -headerpad_max_install_names -o > ../../bin/render.app/Contents/MacOS/render main.o KkPython.o -lkkbase > -L/Users/kk/dev/J3K/trunk/j3ksrc/examples/render/../../lib/ > -lQtSolutions_PropertyBrowser-2.5 -lpython3.1 > /Users/kk/dev/J3K/trunk/j3ksrc/python/modules/build/lib.macosx-10.3-i386-3.1/j3k.so > -framework OpenGL -framework AGL > > ld: warning in > /Users/kk/dev/J3K/trunk/j3ksrc/python/modules/build/lib.macosx-10.3-i386-3.1/j3k.so, > file is not of required architecture > > Undefined symbols: > > "_PyInit_j3k", referenced from: > > _PyInit_j3k$non_lazy_ptr in KkPython.o > > ld: symbol(s) not found > > collect2: ld returned 1 exit status > > make: *** [../../bin/render.app/Contents/MacOS/render] Error 1 > > > > > > I appreciate any hints, > > > > best greetings, > > > > Krzysztof Kobus > > From gtu2003 at alice.it Wed Jan 6 07:03:38 2010 From: gtu2003 at alice.it (wiso) Date: Wed, 06 Jan 2010 13:03:38 +0100 Subject: Convert month name to month number faster References: <4b446e09$0$1107$4fafbaef@reader4.news.tin.it> Message-ID: <4b447c1b$0$1115$4fafbaef@reader4.news.tin.it> Antoine Pitrou wrote: > Le Wed, 06 Jan 2010 12:03:36 +0100, wiso a ?crit : > > >> from time import time >> t = time(); xxx=map(to_dict,l); print time() - t # 0.5 t = time(); >> xxx=map(to_if,l); print time() - t # 1.0 > > Don't define your own function just for attribute access. Instead just > write: > > xxx = map(month_dict.__getitem__, l) t = time(); xxx=map(month_dict.__getitem__,l); print time() - t # 0.2 month_list = ("","Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec") t = time(); xxx=map(month_list.index,l); time() - t # 0.6 From peter.bienstman at gmail.com Wed Jan 6 07:26:02 2010 From: peter.bienstman at gmail.com (pbienst) Date: Wed, 6 Jan 2010 04:26:02 -0800 (PST) Subject: creating tar file and streaming it over HTTP? Message-ID: <5222d923-7aef-47cf-a0d6-f37f09a66593@35g2000yqa.googlegroups.com> I would like to bundle up a number of files in a tar file and send it over a HTTP connection, but I would like to do this without creating the tar file on disk first. I know I can get tarfile to output to a stream by doing something like tar_pipe = tarfile.open(mode="w|", fileobj=my_file_obj) However, I can't figure out which file object to use to send this over a HTTP connection. I tried conn = httplib.HTTPConnection(hostname, port) conn.putrequest("PUT", "/client/files") conn.endheaders() tar_pipe = tarfile.open(mode="w|", fileobj=conn.sock.makefile()) for filename in filenames: tar_pipe.add(filename) tar_pipe.close() conn.getresponse() but that does not send any data... From steve at REMOVE-THIS-cybersource.com.au Wed Jan 6 07:48:32 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 06 Jan 2010 12:48:32 GMT Subject: Convert month name to month number faster References: <4b446e09$0$1107$4fafbaef@reader4.news.tin.it> Message-ID: <03547388$0$1336$c3e8da3@news.astraweb.com> On Wed, 06 Jan 2010 12:03:36 +0100, wiso wrote: > I'm optimizing the inner most loop of my script. I need to convert month > name to month number. I'm using python 2.6 on linux x64. According to your own figures below, it takes less than a nanosecond per lookup, at worst, even using a remarkably inefficient technique. Are you trying to tell us that this is the bottleneck in your script? I'm sorry, I find that implausible. I think you're wasting your time trying to optimise something that doesn't need optimizing. Even if you halve the time, and deal with a million data points each time you run your script, you will only save half a second per run. I can see from the times you posted that you've spent at least an hour trying to optimise this. To make up for that one hour, you will need to run your script 7200 times, before you see *any* time savings at all. > month_dict = {"Jan":1,"Feb":2,"Mar":3,"Apr":4, "May":5, "Jun":6, > "Jul":7,"Aug":8,"Sep":9,"Oct":10,"Nov":11,"Dec":12} > > def to_dict(name): > return month_dict[name] This leads to a pointless function call. Just call month_dict[name] instead of calling a function that calls it. > def to_if(name): > if name == "Jan": return 1 > elif name == "Feb": return 2 > elif name == "Mar": return 3 > elif name == "Apr": return 4 > elif name == "May": return 5 > elif name == "Jun": return 6 > elif name == "Jul": return 7 > elif name == "Aug": return 8 > elif name == "Sep": return 9 > elif name == "Oct": return 10 > elif name == "Nov": return 11 > elif name == "Dec": return 12 > else: raise ValueError That is remarkably awful. > import random > l = [random.choice(month_dict.keys()) for _ in range(1000000)] > > from time import time > t = time(); xxx=map(to_dict,l); print time() - t # 0.5 > t = time(); xxx=map(to_if,l); print time() - t # 1.0 This is not a reliable way to do timings. You should use the timeit module. > is there a faster solution? Maybe something with str.translate? What makes you think str.translate is even remotely useful for this? -- Steven From athena.kurukshetra at gmail.com Wed Jan 6 08:08:20 2010 From: athena.kurukshetra at gmail.com (dhashrath govindarajan) Date: Wed, 6 Jan 2010 05:08:20 -0800 (PST) Subject: Online math-coding contest Message-ID: Hi , we gladly invite you to take part in Athena - the Online Math Coding Contest of Kurukshetra 2010 , the International Techo-Management Fest organised by College Of Engineering Guindy , India under the patronage of UNESCO . Here's your chance to lock horns against the best minds across the globe and showcase your algorithmic prowess and mathematical acumen !! Participants from 20+ countries already registered!! catch the action at www.athena.kurukshetra.org.in Exciting prizes to be won !! For queries please contact athena at kurukshetra.org.in Dates : Practice contest : 6th January 5.00 P.M Indian Standard Time Main contest : 8th January - 15th January awaiting your presence , Team Athena From briandenzer at gmail.com Wed Jan 6 08:10:56 2010 From: briandenzer at gmail.com (Brian D) Date: Wed, 6 Jan 2010 05:10:56 -0800 (PST) Subject: fsync() doesn't work as advertised? References: Message-ID: On Jan 5, 1:08?pm, Nobody wrote: > On Mon, 04 Jan 2010 08:09:56 -0800, Brian D wrote: > > If I'm running a process in a loop that runs for a long time, I > > occasionally would like to look at a log to see how it's going. > > > I know about the logging module, and may yet decide to use that. > > > Still, I'm troubled by howfsync() doesn't seem to work as advertised: > > >http://docs.python.org/library/os.html > > > "If you?re starting with a Python file object f, first do f.flush(), > > and then do os.fsync(f.fileno())" > > The .flush() method (and the C fflush() function) causes the > contents of application buffers to be sent to the OS, which basically > copies the data into the OS-level buffers. > > fsync() causes the OS-level buffers to be written to the physical drive. > > File operations normally use the OS-level buffers; e.g. if one process > write()s to a file and another process read()s it, the latter will see > what the former has written regardless of whether the data has been > written to the drive. > > The main reason for usingfsync() is to prevent important data from being > lost in the event of an unexpected reboot or power-cycle (an expected > reboot via the "shutdown" or "halt" commands will flush all OS-level > buffers to the drive first). Other than that,fsync() is almost invisible > (I say "almost", as there are mechanisms to bypass the OS-level buffers, > e.g. the O_DIRECT open() flag). An excellent explanation of the process. I've seen this in other programming environments, so I could visualize something to that effect, but couldn't have verbalized it. I certainly have a better idea what's happening. Thanks for the contribution. From bruno.42.desthuilliers at websiteburo.invalid Wed Jan 6 08:13:25 2010 From: bruno.42.desthuilliers at websiteburo.invalid (Bruno Desthuilliers) Date: Wed, 06 Jan 2010 14:13:25 +0100 Subject: Need help to pass self.count to other classes. In-Reply-To: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> References: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> Message-ID: <4b448c69$0$6291$426a74cc@news.free.fr> Bill a ?crit : > After a year with Python 2.5 on my Windows box, I still have trouble > understanding classes. > > Below, see the batch file and the configuration script for > my Python interactive prompt. > > The widths of the secondary prompts increase when the self.count of > SysPrompt1 exceeds 99. > > I am using a global variable "zxc" to share self.count, which is not > Pythonic. It's not "unpythonic" - but it's not good OO style neither !-) > How can I pass in self.count without a global? (snip) > sys.ps1 = SysPrompt1() > sys.ps2 = SysPrompt2() > sys.displayhook = DisplayHook() > sys.excepthook = ExceptHook() > hint #1: a bound method is also a callable. hint #2: an object can be an attribute of another object. Possible OO solution: class Counter(object): def __init__(self): self._count = 0 def inc(self): self._count += 1 value = property(fget=lambda s: s._count) class Prompt1(object): def __init__(self, counter): self._counter = counter def _str_(self): self._counter.inc() return '[%2d]> ' % self._counter.value class Prompt2(object): def __init__(self, counter): self._counter = counter def _str_(self): if self._counter.value > 99: # XXX magic number return '...... ' else: return '..... ' class MySysHookHandler(object): def __init__(self): self._counter = Counter() self.ps1 = Prompt1(self._counter) self.ps2 = Prompt2(self._counter) def displayhook(self, value): if value is None: return if self._counter.value > 99: # XXX magic number print '[ out]', value, '\n' else: print '[out]', value, '\n' def excepthook(self, type, value, trace): if self._counter.value > 99: # XXX magic number print '[ err]', value, '\n' else: print '[err]', value, '\n' handler = MySysHook() sys.ps1 = handler.ps1 sys.ps2 = handler.ps2 sys.displayhook = handler.displayhook sys.excepthook = handler.excepthook HTH From philip at semanchuk.com Wed Jan 6 08:24:54 2010 From: philip at semanchuk.com (Philip Semanchuk) Date: Wed, 6 Jan 2010 08:24:54 -0500 Subject: Do I have to use threads? In-Reply-To: <9839a05c1001052145j663f9f73nfac6b3cdeec411fe@mail.gmail.com> References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> <9839a05c1001052145j663f9f73nfac6b3cdeec411fe@mail.gmail.com> Message-ID: On Jan 6, 2010, at 12:45 AM, Brian J Mingus wrote: > On Tue, Jan 5, 2010 at 9:36 PM, Philip Semanchuk > wrote: > >> >> On Jan 5, 2010, at 11:26 PM, aditya shukla wrote: >> >> Hello people, >>> >>> I have 5 directories corresponding 5 different urls .I want to >>> download >>> images from those urls and place them in the respective >>> directories.I have >>> to extract the contents and download them simultaneously.I can >>> extract the >>> contents and do then one by one. My questions is for doing it >>> simultaneously >>> do I have to use threads? >>> >> >> No. You could spawn 5 copies of wget (or curl or a Python program >> that >> you've written). Whether or not that will perform better or be >> easier to >> code, debug and maintain depends on the other aspects of your >> program(s). >> >> bye >> Philip > > > Obviously, spawning 5 copies of wget is equivalent to starting 5 > threads. > The answer is 'yes'. ??? Process != thread From steve at holdenweb.com Wed Jan 6 08:27:27 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 08:27:27 -0500 Subject: creating tar file and streaming it over HTTP? In-Reply-To: <5222d923-7aef-47cf-a0d6-f37f09a66593@35g2000yqa.googlegroups.com> References: <5222d923-7aef-47cf-a0d6-f37f09a66593@35g2000yqa.googlegroups.com> Message-ID: pbienst wrote: > I would like to bundle up a number of files in a tar file and send it > over a HTTP connection, but I would like to do this without creating > the tar file on disk first. > > I know I can get tarfile to output to a stream by doing something like > > tar_pipe = tarfile.open(mode="w|", fileobj=my_file_obj) > > However, I can't figure out which file object to use to send this over > a HTTP connection. > > I tried > > conn = httplib.HTTPConnection(hostname, port) > conn.putrequest("PUT", "/client/files") > conn.endheaders() > tar_pipe = tarfile.open(mode="w|", fileobj=conn.sock.makefile()) > for filename in filenames: > tar_pipe.add(filename) > tar_pipe.close() > conn.getresponse() > > but that does not send any data... > I haven't used socket.makefile() in a blue age so this is untested, but I'm guessing you need a *write* file (remember, sockets have two directions). So try .... fileobj=conn.sock.makefile("w")... and let me know if that helps. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From Brian.Mingus at Colorado.EDU Wed Jan 6 08:52:27 2010 From: Brian.Mingus at Colorado.EDU (Brian J Mingus) Date: Wed, 6 Jan 2010 06:52:27 -0700 Subject: Do I have to use threads? In-Reply-To: References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> <9839a05c1001052145j663f9f73nfac6b3cdeec411fe@mail.gmail.com> Message-ID: <9839a05c1001060552g5a5acb67na5cebe50191a26f4@mail.gmail.com> On Wed, Jan 6, 2010 at 6:24 AM, Philip Semanchuk wrote: > > On Jan 6, 2010, at 12:45 AM, Brian J Mingus wrote: > > On Tue, Jan 5, 2010 at 9:36 PM, Philip Semanchuk > >wrote: >> >> >>> On Jan 5, 2010, at 11:26 PM, aditya shukla wrote: >>> >>> Hello people, >>> >>>> >>>> I have 5 directories corresponding 5 different urls .I want to download >>>> images from those urls and place them in the respective directories.I >>>> have >>>> to extract the contents and download them simultaneously.I can extract >>>> the >>>> contents and do then one by one. My questions is for doing it >>>> simultaneously >>>> do I have to use threads? >>>> >>>> >>> No. You could spawn 5 copies of wget (or curl or a Python program that >>> you've written). Whether or not that will perform better or be easier to >>> code, debug and maintain depends on the other aspects of your program(s). >>> >>> bye >>> Philip >>> >> >> >> Obviously, spawning 5 copies of wget is equivalent to starting 5 threads. >> The answer is 'yes'. >> > > ??? > > Process != thread Just like the other nitpicker it is up to you to explain why the differences, and not he similarities, are relevant to this problem. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at holdenweb.com Wed Jan 6 08:56:23 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 08:56:23 -0500 Subject: Need help to pass self.count to other classes. In-Reply-To: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> References: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> Message-ID: Bill wrote: > After a year with Python 2.5 on my Windows box, I still have trouble > understanding classes. > > Below, see the batch file and the configuration script for > my Python interactive prompt. > > The widths of the secondary prompts increase when the self.count of > SysPrompt1 exceeds 99. > > I am using a global variable "zxc" to share self.count, which is not > Pythonic. > > How can I pass in self.count without a global? > I did RTFM, aka Google, but to no avail. > > echo py.bat > set PYTHONSTARTUP=c:\scripts\startup.py > python > ^Z > > # startup.py > # inspired by: > # http://www.doughellmann.com/PyMOTW/sys/interpreter.html > > import sys > > class SysPrompt1(object): > def __init__(self): > self.count = 0 > def __str__(self): > self.count += 1 > global zxc > zxc = self.count > return '[%2d]> ' % self.count > > class SysPrompt2(object): > def __str__(self): > global zxc > if zxc > 99: return '...... ' > else: return '..... ' > > class DisplayHook(object): > def __call__(self, value): > if value is None: return > global zxc > if zxc > 99: print '[ out]', value, '\n' > else: print '[out]', value, '\n' > > class ExceptHook(object): > def __call__(self, type, value, trace): > global zxc > if zxc > 99: print '[ err]', value, '\n' > else: print '[err]', value, '\n' > > sys.ps1 = SysPrompt1() > sys.ps2 = SysPrompt2() > sys.displayhook = DisplayHook() > sys.excepthook = ExceptHook() > As Francesco points out, OO programming isn't *mandatory* in Python. However, if you used a single class instead of multiples then you could use an instance variable for zxc, and have each of the functions be bound methods (i.e. methods of the instance, not the class). This is untested code (some days I don't seem to write any other kind ...) but it should give you the flavor: class kbInterface(object): def __init__(self): self.zxc = 0 def prompt1(self): self.count += 1 return "[%d]> " def prompt2(self): l = len(str(self.count))+1 return "%s " % "."*l def dhook(self, value): print "[%d out]" % self.count def ehook(self, type, value, trace): print "[%d err]\n" % value kbi = kbInterface() sys.ps1 = kbi.prompt1 sys.ps2 = kbi.prompt2 sys.displayhook = kbi.dhook sys.excepthook = kbi.ehook Do you get the idea? Now the count is shared between all the methods, and it's available in the instance's namespace. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Wed Jan 6 08:58:13 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 08:58:13 -0500 Subject: chown'ing by script In-Reply-To: <4dc0cfea1001060133q35dd0e4dk4a905e6957ff86a2@mail.gmail.com> References: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> <4dc0cfea1001060133q35dd0e4dk4a905e6957ff86a2@mail.gmail.com> Message-ID: Victor Subervi wrote: > On Wed, Jan 6, 2010 at 2:12 AM, alex23 > wrote: > > Carsten Haese > wrote: > > What is the underlying problem you're trying to solve with this > > approach? > > To be paid for developing a web site shopping cart without actually > having to learn Python. > > > LOL! I've written about 12,000 lines of code in this, BTW. Crawl back > into your cave :) > beno > Which, I don't doubt, could have been 2,000 lines had you bothered to take the excellent advice you've been offered on learning Python. Still, "the better the advice the worse it's wasted" ... regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From exarkun at twistedmatrix.com Wed Jan 6 09:11:34 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Wed, 06 Jan 2010 14:11:34 -0000 Subject: Do I have to use threads? In-Reply-To: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> Message-ID: <20100106141134.2771.1806839751.divmod.xquotient.41@localhost.localdomain> On 04:26 am, adityashukla1983 at gmail.com wrote: >Hello people, > >I have 5 directories corresponding 5 different urls .I want to >download >images from those urls and place them in the respective directories.I >have >to extract the contents and download them simultaneously.I can extract >the >contents and do then one by one. My questions is for doing it >simultaneously >do I have to use threads? > >Please point me in the right direction. See Twisted, http://twistedmatrix.com/ in particular, Twisted Web's asynchronous HTTP client, http://twistedmatrix.com/documents/current/web/howto/client.html http://twistedmatrix.com/documents/current/api/twisted.web.client.html Jean-Paul From shawn at milochik.com Wed Jan 6 09:35:44 2010 From: shawn at milochik.com (Shawn Milochik) Date: Wed, 6 Jan 2010 09:35:44 -0500 Subject: Python books, literature etc In-Reply-To: References: Message-ID: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> Search Google. You'll find it all. Search this list's archives. This kind of thing has been discussed a thousand times. It also wouldn't hurt to brush up on this: http://catb.org/~esr/faqs/smart-questions.html From jfabiani at yolo.com Wed Jan 6 09:43:29 2010 From: jfabiani at yolo.com (jfabiani at yolo.com) Date: Wed, 06 Jan 2010 06:43:29 -0800 Subject: Convert month name to month number faster References: Message-ID: VYAS ASHISH M-NTB837 wrote: > > How about using list.index() and storing month names in a list? You may > want to measure performance your self and conclude. > > Regards, > Ashish Vyas > > -----Original Message----- > From: python-list-bounces+ntb837=motorola.com at python.org > [mailto:python-list-bounces+ntb837=motorola.com at python.org] On Behalf Of > wiso > Sent: Wednesday, January 06, 2010 4:34 PM > To: python-list at python.org > Subject: Convert month name to month number faster > > I'm optimizing the inner most loop of my script. I need to convert month > name to month number. I'm using python 2.6 on linux x64. > > > month_dict = {"Jan":1,"Feb":2,"Mar":3,"Apr":4, "May":5, "Jun":6, > "Jul":7,"Aug":8,"Sep":9,"Oct":10,"Nov":11,"Dec":12} > > def to_dict(name): > return month_dict[name] > > def to_if(name): > if name == "Jan": return 1 > elif name == "Feb": return 2 > elif name == "Mar": return 3 > elif name == "Apr": return 4 > elif name == "May": return 5 > elif name == "Jun": return 6 > elif name == "Jul": return 7 > elif name == "Aug": return 8 > elif name == "Sep": return 9 > elif name == "Oct": return 10 > elif name == "Nov": return 11 > elif name == "Dec": return 12 > else: raise ValueError > > import random > l = [random.choice(month_dict.keys()) for _ in range(1000000)] > > from time import time > t = time(); xxx=map(to_dict,l); print time() - t # 0.5 > t = time(); xxx=map(to_if,l); print time() - t # 1.0 > > > is there a faster solution? Maybe something with str.translate? > > The problem is a little different because I don't read random data, but > sorted data. For example: > > l = [x for x in > ("Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec" > ) > for _ in range(1000)] # ["Jan","Jan", ..., "Feb", "Feb", ...] > > so maybe the to_if approach will be faster if I write the case in the > best > order. Look: > > l = ["Jan"] * 1000000 # to_if is in the best order for "Jan" > t = time(); xxx=map(to_dict,l); print time() - t # 0.5 > t = time(); xxx=map(to_if,l); print time() - t # 0.5 > > what just using get(key[, default])? Return the value for key if key is in the dictionary, else default. If default is not given, it defaults to None, so that this method never raises a KeyError. From darcy at druid.net Wed Jan 6 09:46:33 2010 From: darcy at druid.net (D'Arcy J.M. Cain) Date: Wed, 6 Jan 2010 09:46:33 -0500 Subject: chown'ing by script In-Reply-To: References: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> <4dc0cfea1001060133q35dd0e4dk4a905e6957ff86a2@mail.gmail.com> Message-ID: <20100106094633.b820725b.darcy@druid.net> On Wed, 06 Jan 2010 08:58:13 -0500 Steve Holden wrote: > Victor Subervi wrote: [Usual nonsense removed] > Which, I don't doubt, could have been 2,000 lines had you bothered to Steve - any chance that you could stop replying to this idiot or at least do it privately. There's not much point to kill-filing someone if someone else is just going to repeat his posts. -- D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner. From kk720801 at yahoo.com Wed Jan 6 09:48:28 2010 From: kk720801 at yahoo.com (Krzysztof Kobus) Date: Wed, 6 Jan 2010 06:48:28 -0800 (PST) Subject: embedded python on mac - linking problem Message-ID: <782094.10621.qm@web53506.mail.re2.yahoo.com> Hi, >The missing symbol looks like a C++-symbol - but Python is C. Do you maybe miss the > extern "C" > > declaration. I have not specified extern "C" as I assume it is a part of PyMODINIT_FUNC define. At least documentation says so: "Note that PyMODINIT_FUNC declares the function as PyObject * return type, declares any special linkage declarations required by the platform, and for C++ declares the function as extern "C"." Regards, KK From martinskou2 at gmail.com Wed Jan 6 09:53:40 2010 From: martinskou2 at gmail.com (msj@infoserv.dk) Date: Wed, 6 Jan 2010 06:53:40 -0800 (PST) Subject: Introspection Message-ID: <0d782b7a-11fb-4933-84b7-29a3653ec847@p8g2000yqb.googlegroups.com> I'm looking for a way to make a list of string literals in a class. Example: class A: def method(self): print 'A','BC' >>> ExtractLiterals(A) ['A','BC'] Is this possible? Can anyone point me in the right direction? Thanks. /Martin From steve at holdenweb.com Wed Jan 6 10:07:59 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 10:07:59 -0500 Subject: chown'ing by script In-Reply-To: <20100106094633.b820725b.darcy@druid.net> References: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> <4dc0cfea1001060133q35dd0e4dk4a905e6957ff86a2@mail.gmail.com> <20100106094633.b820725b.darcy@druid.net> Message-ID: <4B44A74F.8090400@holdenweb.com> D'Arcy J.M. Cain wrote: > On Wed, 06 Jan 2010 08:58:13 -0500 > Steve Holden wrote: >> Victor Subervi wrote: > [Usual nonsense removed] > >> Which, I don't doubt, could have been 2,000 lines had you bothered to > > Steve - any chance that you could stop replying to this idiot or at > least do it privately. There's not much point to kill-filing someone > if someone else is just going to repeat his posts. > I think idiot's a *bit* strong, but I have to confess that even my tireless wish to evangelize Python has in Victor's case become a bit tired, so I will be happy to oblige. Victor, please refer to python-tutor for further (and more appropriate) assistance. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From phlip2005 at gmail.com Wed Jan 6 10:10:43 2010 From: phlip2005 at gmail.com (Phlip) Date: Wed, 6 Jan 2010 07:10:43 -0800 (PST) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: On Jan 5, 8:49?pm, Steven D'Aprano wrote: > > (A related question - why can't I just go 'if record = method(): ?use > > (record)'. Why extra lines just to trap and assign the variable before > > using it?) > > Because that idiom is responsible for probably the most common error in C > of all, at least one of the most common errors. Thank goodness Python > forbids such a dangerous construct. switching = for == is the "most common error in C"? I can't tell if you are joking. From phlip2005 at gmail.com Wed Jan 6 10:12:12 2010 From: phlip2005 at gmail.com (Phlip) Date: Wed, 6 Jan 2010 07:12:12 -0800 (PST) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> Message-ID: On Jan 5, 10:54?pm, Benjamin Kaplan wrote: > {41: None}[41] ? > > In cases where None is a valid result, you can't use it to signal failure. Asked and answered. You change the "sentinel" in .fetch to something else. But y'all keep on defending the language making your programming decisions for you! From dreadpiratejeff at gmail.com Wed Jan 6 10:17:29 2010 From: dreadpiratejeff at gmail.com (J) Date: Wed, 6 Jan 2010 10:17:29 -0500 Subject: Python books, literature etc In-Reply-To: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> Message-ID: <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> On Wed, Jan 6, 2010 at 09:35, Shawn Milochik wrote: > Search Google. You'll find it all. > > Search this list's archives. This kind of thing has been discussed a thousand times. > > It also wouldn't hurt to brush up on this: > http://catb.org/~esr/faqs/smart-questions.html Heh... I've seen that link mentioned many many times on some other lists I belong to :) But I agree. HOWEVER, that is a valid question. BUT, the answer is really up to the person asking it. For example, I too am a relative beginner with Python. Luckily I DO have at least an educational background in OOP, and at least a professional background in basic coding (some perl, BASH, etc). So I do tend to rely a lot on google when I run into a snag, THEN if that doesn't work, I come here with specific questions. FWIW, my Google searches always look something like "python " and 99% of the time, that gives me the answers I seek. That being said, however, I also am somewhat old school and prefer to have hard copy at hand too. Sometimes, it's just that much more satisfying to have a physical book handy to look things up in. Maybe it's a comfort thing, I don't know... So in that vein, while I can't suggest any specific books, I can say this... right now, I have a copy of O'Reilly's Learning Python that I use as a reference, as well as a much older copy of the Python 2.1 Bible that I picked up almost a decade ago now the first time I messed around with Python. My personal preference, while searching Google and asking on lists like this is quick and invaluable for the "real world" knowledge that is shared, is to also have A: something along the lines of the Learning Foo books from O'Reilly because they tend to have lengthy explanations that I usually can understand on my own, and B: some sort of Cookbook on the topic at hand. I haven't found a Python cookbook that I like enough to buy yet, but I'm still looking. Those come in handy for simple snippets and examples when doing specific tasks. So yeah, Google and this list are certainly invaluable, but some of us actually like having the heft of a lofty tome to peruse for the answers to the mysteries of the universe. Cheers Jeff -- Charles de Gaulle - "The better I get to know men, the more I find myself loving dogs." - http://www.brainyquote.com/quotes/authors/c/charles_de_gaulle.html From steve at holdenweb.com Wed Jan 6 10:21:34 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 10:21:34 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> Message-ID: <4B44AA7E.9070903@holdenweb.com> Phlip wrote: [...] > Don't prevent me from using a technique just because others had > trouble with it. > I presume you also campaign against anti-lock braking systems (or at least don't use cars which have them - after all, anyone who knows how to drive should be able to brake properly, right? And even if they don't, *you* can, so you shouldn't have to pay the extra. Of course, if someone else happens to hit you because they *couldn't* drive safely you will happily give up your life). > And if bar() == foo is the superior technique anyway, because the == > happens in chronological and lexical order after the bar() call. That's about the most specious piece of hogwash I've seen this week. Python doesn't make any guarantees about the evaluation order of =='s operands (unlike "and" and "or"). That's cargo cult programming right there. Readability is surely the only reason to decide how to write that "if", and I suspect most people would choose "if foo == bar()" for that reason, though the difference is slight. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Wed Jan 6 10:21:34 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 10:21:34 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> Message-ID: <4B44AA7E.9070903@holdenweb.com> Phlip wrote: [...] > Don't prevent me from using a technique just because others had > trouble with it. > I presume you also campaign against anti-lock braking systems (or at least don't use cars which have them - after all, anyone who knows how to drive should be able to brake properly, right? And even if they don't, *you* can, so you shouldn't have to pay the extra. Of course, if someone else happens to hit you because they *couldn't* drive safely you will happily give up your life). > And if bar() == foo is the superior technique anyway, because the == > happens in chronological and lexical order after the bar() call. That's about the most specious piece of hogwash I've seen this week. Python doesn't make any guarantees about the evaluation order of =='s operands (unlike "and" and "or"). That's cargo cult programming right there. Readability is surely the only reason to decide how to write that "if", and I suspect most people would choose "if foo == bar()" for that reason, though the difference is slight. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From invalid at invalid.invalid Wed Jan 6 10:34:54 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Wed, 6 Jan 2010 15:34:54 +0000 (UTC) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: On 2010-01-06, r0g wrote: > NO! It's a rude way to start a sentence don't you think? No. When somebody asks a yes/no question, answering yes or no seems quite polite to me. Following the yes/no answer with an explanation of the answer is always nice, and I've little doubt that's what happened. > Just because you're correcting someone doesn't mean you have > to be combative and try and make them feel small. Answering a yes/no question with "no" doesn't seem to me to be combative if the correct answer is indeed "no". But I've lost track of the post you found objectionable... -- Grant Edwards grante Yow! And then we could sit at on the hoods of cars at visi.com stop lights! From bruno.42.desthuilliers at websiteburo.invalid Wed Jan 6 10:41:36 2010 From: bruno.42.desthuilliers at websiteburo.invalid (Bruno Desthuilliers) Date: Wed, 06 Jan 2010 16:41:36 +0100 Subject: Need help to pass self.count to other classes. In-Reply-To: References: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> Message-ID: <4b44af22$0$13292$426a74cc@news.free.fr> Steve Holden a ?crit : (snip) > This is untested code indeed !-) > class kbInterface(object): > def __init__(self): > self.zxc = 0 > def prompt1(self): > self.count += 1 Ahem... (snip) From miki.tebeka at gmail.com Wed Jan 6 10:44:31 2010 From: miki.tebeka at gmail.com (Miki) Date: Wed, 6 Jan 2010 07:44:31 -0800 (PST) Subject: Introspection References: <0d782b7a-11fb-4933-84b7-29a3653ec847@p8g2000yqb.googlegroups.com> Message-ID: <92011e74-c6d3-4a97-a78e-20f062a97e40@r24g2000yqd.googlegroups.com> Hello Martin, > I'm looking for a way to make a list of string literals in a class. from inspect import getsourcelines from tokenize import generate_tokens, STRING, NUMBER def is_literal(t): return t[0] in (STRING, NUMBER) def get_lieterals(obj): lines, _ = getsourcelines(obj) readline = iter(lines).next return [t[1] for t in generate_tokens(readline) if is_literal(t)] if __name__ == "__main__": class A: def f(self): print "A", "B" print get_lieterals(A) HTH, -- Miki From steve at holdenweb.com Wed Jan 6 10:47:05 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 10:47:05 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <4B44B079.5090008@holdenweb.com> Phlip wrote: > On Jan 5, 8:49 pm, Steven D'Aprano > wrote: > >>> (A related question - why can't I just go 'if record = method(): use >>> (record)'. Why extra lines just to trap and assign the variable before >>> using it?) >> Because that idiom is responsible for probably the most common error in C >> of all, at least one of the most common errors. Thank goodness Python >> forbids such a dangerous construct. > > switching = for == is the "most common error in C"? > > I can't tell if you are joking. Well, there's also the "indenting code without surrounding it by braces" error, but y'all just keep defending the approach to programming that *you* think is best. Just don't expect others to agree. The FAQ clearly documents why Python is a statement-based, and not an expression-based, language. Me, I think Guido is a pretty good language designer, and while we've had our differences I think that Python is a *much* better language than C and most others. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From bruno.42.desthuilliers at websiteburo.invalid Wed Jan 6 10:50:29 2010 From: bruno.42.desthuilliers at websiteburo.invalid (Bruno Desthuilliers) Date: Wed, 06 Jan 2010 16:50:29 +0100 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <4b44b138$0$11237$426a74cc@news.free.fr> Phlip a ?crit : > On Jan 5, 8:49 pm, Steven D'Aprano > wrote: > >>> (A related question - why can't I just go 'if record = method(): use >>> (record)'. Why extra lines just to trap and assign the variable before >>> using it?) >> Because that idiom is responsible for probably the most common error in C >> of all, at least one of the most common errors. Thank goodness Python >> forbids such a dangerous construct. > > switching = for == is the "most common error in C"? > > I can't tell if you are joking. It's at least a _very_ common error in all languages that allow this construct. In C, it's common enough to gave birth to the "BestPractice" you described, ie swapping operand orders in equality test to have the compiler detect the problem - at least when one of the operand is a function call expression or constant (it obviously won't 'work' when both operands are variables). Anyway: in Python, assignment is not an expression, and this isn't going to change anytime soon. From jjposner at optimum.net Wed Jan 6 11:18:35 2010 From: jjposner at optimum.net (John Posner) Date: Wed, 06 Jan 2010 11:18:35 -0500 Subject: Dynamic text color References: Message-ID: On Tue, 05 Jan 2010 16:54:44 -0500, Dave McCormick wrote: > But it is not what I am wanting. I first thought to make it look for a > space but that would not work when a single character like "#" is to be > colored if there is a "string" of them. Or if all of the characters > between quotes are to be colored. Regular expressions are good at handling searches like: * all the characters between quotes * the two-character string "do", but only if it's a complete word -John From bsk16 at case.edu Wed Jan 6 11:24:39 2010 From: bsk16 at case.edu (Ben Kaplan) Date: Wed, 06 Jan 2010 11:24:39 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> Message-ID: <4B44B947.1040102@case.edu> On 1/6/10 10:12 AM, Phlip wrote: > On Jan 5, 10:54 pm, Benjamin Kaplan wrote: > > >> {41: None}[41] ? >> >> In cases where None is a valid result, you can't use it to signal failure. >> > Asked and answered. You change the "sentinel" in .fetch to something > else. > When did I specify a sentinel value? You're saying that if None is a valid value in the dict, you can't use slice syntax for it. > But y'all keep on defending the language making your programming > decisions for you! > Your example is the one where the language makes programming decisions for you. Your language is deciding that if I use slice syntax, I want the sentinel to be None. Python is deciding that unless I give it a sentinel, there is no sentinel. From python at mrabarnett.plus.com Wed Jan 6 11:29:04 2010 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 06 Jan 2010 16:29:04 +0000 Subject: TypeError In-Reply-To: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> References: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> Message-ID: <4B44BA50.1060203@mrabarnett.plus.com> Victor Subervi wrote: > Hi; > I get this error: > > /var/www/html/angrynates.com/christians/cart/simplemail/mail.py > > 153 > 154 ''' > 155 commitSale() > 156 myMail() > 157 print ''' > commitSale = > /var/www/html/angrynates.com/christians/cart/simplemail/mail.py > in commitSale() > 98 cursor.execute('select max(ID) from %sCustomerData;' % store) > 99 custID = cursor.fetchone()[0] > 100 customerData(store, tmpTable, custID, patientID) > 101 > 102 def myMail(): > global customerData = , global store = > 'products', tmpTable = 'tem12627568064', custID = 1, global patientID = > 'None' > /var/www/html/angrynates.com/christians/cart/customerData.py > in > customerData(store='products', tmpTable='tem12627568064', custID=1, > patientID='None') > 39 > 40 """ > 41 print """ > 42 print '

%s Customer Data

' % > (store[0].upper() + store[1:]) > 43 cursor.execute('describe %sCustomerData' % store) > store = 'products' > > ValueError: unsupported format character '(' (0x28) at index 54 > args = ("unsupported format character '(' (0x28) at index 54",) > > Apparently that character is a "file separator", which I presume is an > invisible character. I tried retyping the area in question, but with no > avail (threw same error). Please advise. Complete code follows. > Please provide the actual code (and, preferably, an easier-to-understand traceback, like what CPython does!). What you supplied isn't runnable. From fpm at u.washington.edu Wed Jan 6 11:29:19 2010 From: fpm at u.washington.edu (cassiope) Date: Wed, 6 Jan 2010 08:29:19 -0800 (PST) Subject: IOError - cannot create file (linux daemon-invoked script) References: Message-ID: <31a53b2f-45cc-45a2-8ae8-c7658e8a3949@j19g2000yqk.googlegroups.com> On Jan 5, 10:58?am, Nobody wrote: > On Mon, 04 Jan 2010 21:30:31 -0800, cassiope wrote: > > One more tidbit observed: my last note, that it works when using > > seteuid/setegid? > > Well - that only applies if the daemon is running under strace (!). > > It fails > > if started directly by root, or if the strace session has ended, > > leaving the > > main body of the daemon running in its normal headless manner. > > > I wonder if running under "strace -f" - might setegid/seteuid be > > prevented from > > having their normal effect? > > Possibly. The ptrace() syscall on which strace depends will fail if you > try to trace a "privileged" process and you aren't root, so it's possible > that a ptrace()d process will refuse to become privileged. > > Here, "privileged" includes a process which has changed any of its UIDs or > GIDs (this prevents a normal user from tracing, killing, etc an otherwise > privileged process which has switched to the user's UID for the time being). Thanks. I guess it's time to trim this beast to a simpler state to isolate what's going wrong. This may take awhile amidst everything else I have to be doing... From steve at REMOVE-THIS-cybersource.com.au Wed Jan 6 11:38:35 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 06 Jan 2010 16:38:35 GMT Subject: Introspection References: <0d782b7a-11fb-4933-84b7-29a3653ec847@p8g2000yqb.googlegroups.com> Message-ID: <0354a972$0$1336$c3e8da3@news.astraweb.com> On Wed, 06 Jan 2010 06:53:40 -0800, msj at infoserv.dk wrote: > I'm looking for a way to make a list of string literals in a class. > > Example: > > class A: > def method(self): > print 'A','BC' > >>>> ExtractLiterals(A) > ['A','BC'] > > Is this possible? Can anyone point me in the right direction? class A: def extract_literals(self): return "A BC".split() def method(self): print self.extract_literals() a = A() a.extract_literals() -- Steven From peter.bienstman at gmail.com Wed Jan 6 11:39:08 2010 From: peter.bienstman at gmail.com (pbienst) Date: Wed, 6 Jan 2010 08:39:08 -0800 (PST) Subject: creating tar file and streaming it over HTTP? References: <5222d923-7aef-47cf-a0d6-f37f09a66593@35g2000yqa.googlegroups.com> Message-ID: Thanks for the tip! It doesn't change anything, though, so I've debugged this a little bit further. The problem seems to be that the receiving end (wsgi server) does not see the end of the data: socket = environ["wsgi.input"] while True: sys.stderr.write("before") chunk = socket.read(4096) sys.stderr.write("after") if not chunk: sys.stderr.write("done") break sys.stderr.write(chunk) There is data from the tar file being printed, but in the end it hangs in the 'before' statement, and never gets to 'done'. I tried flushing the fileobj created by makefile(), but that doesn't seem to help. There is also no environ["CONTENT_LENGTH"] that I can use to detect the end of the stream. I'm probably missing something basic here... From victorsubervi at gmail.com Wed Jan 6 11:40:35 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 6 Jan 2010 12:40:35 -0400 Subject: TypeError In-Reply-To: <4B44BA50.1060203@mrabarnett.plus.com> References: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> <4B44BA50.1060203@mrabarnett.plus.com> Message-ID: <4dc0cfea1001060840r4f507f3bg1f71dfa97e903517@mail.gmail.com> On Wed, Jan 6, 2010 at 12:29 PM, MRAB wrote: > Victor Subervi wrote: > >> Hi; >> I get this error: >> >> /var/www/html/angrynates.com/christians/cart/simplemail/mail.py < >> http://angrynates.com/christians/cart/simplemail/mail.py> >> >> 153 >> 154 ''' >> 155 commitSale() >> 156 myMail() >> 157 print ''' >> commitSale = >> /var/www/html/angrynates.com/christians/cart/simplemail/mail.py < >> http://angrynates.com/christians/cart/simplemail/mail.py> in commitSale() >> >> 98 cursor.execute('select max(ID) from %sCustomerData;' % store) >> 99 custID = cursor.fetchone()[0] >> 100 customerData(store, tmpTable, custID, patientID) >> 101 >> 102 def myMail(): >> global customerData = , global store = 'products', >> tmpTable = 'tem12627568064', custID = 1, global patientID = 'None' >> /var/www/html/angrynates.com/christians/cart/customerData.py < >> http://angrynates.com/christians/cart/customerData.py> in >> customerData(store='products', tmpTable='tem12627568064', custID=1, >> patientID='None') >> >> 39 >> 40 """ >> 41 print """ >> 42 print '

%s Customer Data

' % >> (store[0].upper() + store[1:]) >> 43 cursor.execute('describe %sCustomerData' % store) >> store = 'products' >> >> ValueError: unsupported format character '(' (0x28) at index 54 >> args = ("unsupported format character '(' (0x28) at index 54",) >> >> Apparently that character is a "file separator", which I presume is an >> invisible character. I tried retyping the area in question, but with no >> avail (threw same error). Please advise. Complete code follows. >> >> Please provide the actual code (and, preferably, an easier-to-understand > traceback, like what CPython does!). What you supplied isn't runnable. > I ended up working around this problem another way. Thanks. beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From phlip2005 at gmail.com Wed Jan 6 11:41:29 2010 From: phlip2005 at gmail.com (Phlip) Date: Wed, 06 Jan 2010 08:41:29 -0800 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: Steve Holden wrote: > y'all just keep defending the approach to programming that > *you* think is best. Speak for yourself... From steve at holdenweb.com Wed Jan 6 11:42:21 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 11:42:21 -0500 Subject: Need help to pass self.count to other classes. In-Reply-To: <4b44af22$0$13292$426a74cc@news.free.fr> References: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> <4b44af22$0$13292$426a74cc@news.free.fr> Message-ID: Bruno Desthuilliers wrote: > Steve Holden a ?crit : > (snip) >> This is untested code > > indeed !-) > >> class kbInterface(object): >> def __init__(self): >> self.zxc = 0 >> def prompt1(self): >> self.count += 1 > > Ahem... > > (snip) > > Caveat emptor ... this code is worth what you paid for it :) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From sridharr at activestate.com Wed Jan 6 11:44:58 2010 From: sridharr at activestate.com (Sridhar Ratnakumar) Date: Wed, 06 Jan 2010 08:44:58 -0800 Subject: lxml 2.2.4 on python3.1, Windows XP gives importerror In-Reply-To: <7C57DB58C81FB64CA979EC6C4DB73E7A03E7039F@ZMY16EXM67.ds.mot.com> References: <7C57DB58C81FB64CA979EC6C4DB73E7A03E7039F@ZMY16EXM67.ds.mot.com> Message-ID: <4B44BE0A.7060508@activestate.com> On 1/5/2010 1:38 AM, VYAS ASHISH M-NTB837 wrote: > Dear All > I have Python 3.1 installed on Windows XP and Works nice. > I downloaded lxml 2.2.4 (lxml-2.2.4.win32-py3.1.exe) from pypi. > When I try: > from lxml import etree > I get: > ImportError: DLL load failed: This application has failed to start > because the application configuration is incorrect. Reinstalling the > application may fix this problem. Are you by any chance running 64-bit Python (whereas the lxml binary you have installed is 32-bit)? C:> python -c "import platform; print platform.architecture()" -srid From sccolbert at gmail.com Wed Jan 6 11:47:20 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Wed, 6 Jan 2010 17:47:20 +0100 Subject: how to change when the logging module creates the log file? In-Reply-To: <7f014ea61001060357r62567020wb82aa92940aa5763@mail.gmail.com> References: <7f014ea61001060357r62567020wb82aa92940aa5763@mail.gmail.com> Message-ID: <7f014ea61001060847j16fe8298xd1377a5caf4df0ad@mail.gmail.com> i was able to fix the exception by calling logging.shutdown() before the call to os.remove(). However, I still think there is probably a more elegant solution. On Wed, Jan 6, 2010 at 12:57 PM, Chris Colbert wrote: > I have an application the writes to a log file when specific exceptions are > handled. However, if no exceptions are encountered, I don't want to create a > log at all. > > The problem I am running into is that the stdlib logging module creates the > log file immediately upon logger instantiation. > > Thus: > >>> logger = logging.basicConifg('testlog.txt') > > already creates the file 'testlog.txt'. > > So, at program close, I am reading the size of the log file, and if it is > empty I remove it with os.remove(). This works fine on Linux, but throws a > permission denied exception on Windows. > > There has to be a better way to do this than using a hack like that. Is > there a way to make the logging module hold-off on file creation until the > first log is generated? I could do it by wrapping logger in a class, but > that would remove the beauty of having any module import logging from the > stdlib and being able to write to the log. > > Thanks for any pointers! > > Chris > -------------- next part -------------- An HTML attachment was scrubbed... URL: From james.b.looney at ulalaunch.com Wed Jan 6 11:48:44 2010 From: james.b.looney at ulalaunch.com (Looney, James B) Date: Wed, 6 Jan 2010 09:48:44 -0700 Subject: Python tk Listbox: -listvariable Message-ID: <8306B8561B5E1F4A97129B09E06D2473023C3F4F37@BROWN1.ad.ulalaunch.com> Yesterday, I searched all over trying to figure out how to properly use the "listvariable" argument with tk's Listbox class. Unfortunately, no amount of searching (online) could come up with anything more useful than telling me the variable needed to be a list, and nothing built-in exists. I finally came across a way of using it that I consider fairly simple, and wanted to toss it out there for everyone else. One post I saw mentioned using StringVar. That just didn't work well in my head, so instead I used its parent class: Variable. Apparently, the trick is to use tuples. If I use a list, string, etc - the Listbox seems to split the content based on spaces. Not the behavior I was looking for as I have spaces in the strings I wish to display. If anyone has better suggestions, I'd love to hear them. (I thought about a ListVar class, but haven't made it much beyond the thought). Regardless, I hope the following code helps others avoid the confusion I went through. Sample code: import Tkinter, tkSimpleDialog from Tkconstants import * class ListboxTest( tkSimpleDialog.Dialog ): ############################################################################# def __init__( self, master, tupleItems = () ): self.myVar = Tkinter.Variable() self.myVar.set( tupleItems ) tkSimpleDialog.Dialog.__init__( self, master, "Listbox testing" ) ############################################################################# def body( self, master ): lbox = Tkinter.Listbox( master, listvariable = self.myVar ) lbox.grid( row = 0, column = 0, sticky = N + W ) self.myVar.set( self.myVar.get() + tuple( [ "***** Final string being added *****" ] ) ) print type( self.myVar.get() ), self.myVar.get() if( "__main__" == __name__ ): tk = Tkinter.Tk() lt = ListboxTest( tk, tuple( [ "String 1 - with some spaces", "String 2 - with more spaces" ] ) ) -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Wed Jan 6 11:49:29 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 6 Jan 2010 12:49:29 -0400 Subject: getfirst and re Message-ID: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> Hi; I need to do something like the following: pat = re.compile('edit[0-9]*:[0-9]*') check = form.getfirst(pat) (to check things like 'edit0:1') How do I do this? TIA, beno -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From python.list at tim.thechases.com Wed Jan 6 12:27:50 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Wed, 06 Jan 2010 11:27:50 -0600 Subject: getfirst and re In-Reply-To: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> Message-ID: <4B44C816.2090201@tim.thechases.com> > I need to do something like the following: > > pat = re.compile('edit[0-9]*:[0-9]*') > check = form.getfirst(pat) > > (to check things like 'edit0:1') How do I do this? Well, you can do it either as check = pat.search(string_to_search) which is pretty plainly detailed in the help for the "re" module, both under the search() function and under the "8.2.2 Matching vs. Searching" section. Alternatively, if you plan to get the others too, you can use fi = pat.finditer(string_to_search) check = fi.next().group(0) # beware this may throw # a StopIteration if there are no more matches. which would usually be done inside a loop where the StopIteration does the Right Thing(tm). But if you're using it on HTML form text, regexps are usually the wrong tool, and you should be using an HTML parser (such as BeautifulSoup) that knows how to handle odd text and escapings better and more robustly than regexps will. -tkc From mail.to.daniel.platz at googlemail.com Wed Jan 6 12:28:45 2010 From: mail.to.daniel.platz at googlemail.com (Daniel Platz) Date: Wed, 6 Jan 2010 09:28:45 -0800 (PST) Subject: Pass multidimensional array (matrix) to c function using ctypes Message-ID: <63ac1a01-8491-4885-bae7-cb884abb582f@34g2000yqp.googlegroups.com> Hello, I would like to pass a two dimensional array to C function in a dll. I use ctypes to call the function. I compile the dll with visual studio 2008 express and my C source code looks like this. #include #ifdef __cplusplus extern "C" { // only need to export C interface if // used by C++ source code using namespace std; #endif __declspec(dllexport) int print(double** ptr, int ny, int nx) { int i, j; for(i=0; i Furthermore, I am wondering if there is a fast way to use a numpy 2D array instead or alternatively to cast the ctypes array into a numpy array. Has someone an idea to help me? Thank you very much Daniel From victorsubervi at gmail.com Wed Jan 6 12:31:41 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 6 Jan 2010 13:31:41 -0400 Subject: getfirst and re In-Reply-To: <4B44C816.2090201@tim.thechases.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> Message-ID: <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> On Wed, Jan 6, 2010 at 1:27 PM, Tim Chase wrote: > But if you're using it on HTML form text, regexps are usually the wrong > tool, and you should be using an HTML parser (such as BeautifulSoup) that > knows how to handle odd text and escapings better and more robustly than > regexps will. > I have an automatically generated HTML form from which I need to extract data to the script which this form calls (to which the information is sent). I believe BeautifulSoup is geared to scraping pages that exist permanently on the web. By the time BeautifulSoup was called, this page would be gone. Any other ideas? TIA beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From nobody at nowhere.com Wed Jan 6 12:56:24 2010 From: nobody at nowhere.com (Nobody) Date: Wed, 06 Jan 2010 17:56:24 +0000 Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> <96ea39f2-1fb6-4b03-b206-d4c1445ea021@v25g2000yqk.googlegroups.com> Message-ID: On Tue, 05 Jan 2010 11:40:25 -0800, KvS wrote: >> "Hardcopy" document formats such as PostScript and PDF use positions >> relative to the edges of the page, not the margins. > > Right. Still, Acrobat Reader by default scales the contents to fit on > a page and creates some margins by doing so, no? So if my text is > close to the left and right edges, as I want, it will get scaled and > extra margins will occur. Avoiding this still requires me to be able > to turn off this scaling in the printing preferences somehow > programmatically, so it doesn't seem to make the problem easier? If the document is the same size as the physical page, it will be transferred directly without any scaling or offset. The document will not be scaled to fit the printable area; if the document contains any marks which lie within the printer's margins, those marks won't appear on the printed page. PostScript and PDF documents don't have "margins". There might be an area around the edge of the page which doesn't contain any marks, but that's irrelevant; the area is still part of the page. From nobody at nowhere.com Wed Jan 6 12:59:40 2010 From: nobody at nowhere.com (Nobody) Date: Wed, 06 Jan 2010 17:59:40 +0000 Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> Message-ID: On Tue, 05 Jan 2010 12:20:58 -0800, Marco Nawijn wrote: > You could use the build-in function enumerate inside a list > comprehension. > >>>> seq = range(5) >>>> [ (i,s) for i,s in enumerate(seq) ] > [(0, 0), (1, 1), (2, 2), (3, 3), (4, 4)] Just use list(), i.e. "list(enumerate(seq))". From python.list at tim.thechases.com Wed Jan 6 12:59:52 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Wed, 06 Jan 2010 11:59:52 -0600 Subject: getfirst and re In-Reply-To: <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> Message-ID: <4B44CF98.2090702@tim.thechases.com> Victor Subervi wrote: > On Wed, Jan 6, 2010 at 1:27 PM, Tim Chase wrote: > >> But if you're using it on HTML form text, regexps are usually the wrong >> tool, and you should be using an HTML parser (such as BeautifulSoup) that >> knows how to handle odd text and escapings better and more robustly than >> regexps will > > I have an automatically generated HTML form from which I need to extract > data to the script which this form calls (to which the information is sent). > I believe BeautifulSoup is geared to scraping pages that exist permanently > on the web. By the time BeautifulSoup was called, this page would be gone. BeautifulSoup takes string data fed to it, and builds a structure that can be neatly navigated. That string data can come from a web page, from a disk, or even a serial port, a random-character-generator, or just from HTML that's built up in memory and never sees a network or a disk. It's worth reading its documentation[1] and trying its examples to get familiar with it. -tkc [1] http://www.crummy.com/software/BeautifulSoup/documentation.html From thimxx at gmail.com Wed Jan 6 13:00:25 2010 From: thimxx at gmail.com (Valentin de Pablo Fouce) Date: Wed, 6 Jan 2010 10:00:25 -0800 (PST) Subject: File transfer with python Message-ID: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> Hi there, I hope this is the rigth place, if not please, tell me which is the right dicussion place. I apologize in such case. Ok, I am trying to do a very quick application (is "home based" so is not a big deal...). My intention is to transfer files from one computer to another. I am using several OS (Linux and Windows basicly, but potentially in the future Mac too), and the computers won't be potentially in the same network (ok, the most times will... but there will be exceptions). My intention is to be able to transfer files from one computer to another in this environment. Looking (and surfing) at internet the only suggestion given is to use low level sockets for this file transfer. Is there another way to do it, is there any top level library that helps you to do that? That's a lot! From mudit.tuli at gmail.com Wed Jan 6 13:01:46 2010 From: mudit.tuli at gmail.com (mudit tuli) Date: Wed, 6 Jan 2010 23:31:46 +0530 Subject: 3 byte network ordered int, How To ? Message-ID: <3d4c0f161001061001ma900949j1fdea2789cff7367@mail.gmail.com> For a single byte, struct.pack(') For two bytes, struct.pack(') what if I want three bytes ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Wed Jan 6 13:04:41 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 6 Jan 2010 14:04:41 -0400 Subject: getfirst and re In-Reply-To: <4B44CF98.2090702@tim.thechases.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4B44CF98.2090702@tim.thechases.com> Message-ID: <4dc0cfea1001061004u63ba8c17xca03c682b2d7b7fc@mail.gmail.com> On Wed, Jan 6, 2010 at 1:59 PM, Tim Chase wrote: > Victor Subervi wrote: > >> On Wed, Jan 6, 2010 at 1:27 PM, Tim Chase > >wrote: >> >> But if you're using it on HTML form text, regexps are usually the wrong >>> tool, and you should be using an HTML parser (such as BeautifulSoup) that >>> knows how to handle odd text and escapings better and more robustly than >>> regexps will >>> >> >> I have an automatically generated HTML form from which I need to extract >> data to the script which this form calls (to which the information is >> sent). >> I believe BeautifulSoup is geared to scraping pages that exist permanently >> on the web. By the time BeautifulSoup was called, this page would be gone. >> > > BeautifulSoup takes string data fed to it, and builds a structure that can > be neatly navigated. That string data can come from a web page, from a > disk, or even a serial port, a random-character-generator, or just from HTML > that's built up in memory and never sees a network or a disk. It's worth > reading its documentation[1] and trying its examples to get familiar with > it. > k. Thanks. beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From pavlovevidence at gmail.com Wed Jan 6 13:08:21 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Wed, 6 Jan 2010 10:08:21 -0800 (PST) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> Message-ID: <0be1d648-07ac-4662-950f-7feb24ec713c@c3g2000yqd.googlegroups.com> On Jan 5, 2:40?pm, Phlip wrote: > On Jan 5, 1:10?pm, Antoine Pitrou wrote: > > >http://docs.python.org/library/functions.html > > > Don't forget that the Python documentation is rich and structured. > > And good luck. > > Does it say how to convert a string containing either an integer > representation, or something alphabetic, into an integer, or a zero, > in like 1 method call? No, as he said, it only describes the most important functions. Carl Banks From nobody at nowhere.com Wed Jan 6 13:10:36 2010 From: nobody at nowhere.com (Nobody) Date: Wed, 06 Jan 2010 18:10:36 +0000 Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> <1715bea7-1ff1-4294-9dad-672873e14530@v25g2000yqk.googlegroups.com> Message-ID: On Tue, 05 Jan 2010 19:46:01 -0800, alex23 wrote: >> They will tell me how to use except: (which is a good example why a >> program should not use exceptions for its normal control flow if at >> all possible). > > Really? Magic functions that coerce and eat errors are a better coding > technique than exceptions and explicit handling? > > What kool-aid have you been drinking? Maybe he's doing it for a living? Contract programming seems to work on the basis that the ultimate requirement is for the client to hand over the money. If, at that point, the program is still full of bugs, you get to charge extra for "upgrades". Writing robust software from the outset puts you at a competitive disadvantage to those who understand how the system works. From shawn at milochik.com Wed Jan 6 13:15:31 2010 From: shawn at milochik.com (Shawn Milochik) Date: Wed, 6 Jan 2010 13:15:31 -0500 Subject: File transfer with python In-Reply-To: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> References: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> Message-ID: <90C09BF4-48D9-4245-A967-5DFABE2A64BD@milochik.com> Have a look at Paramiko. It lets you do secure transfers easily (scp/sftp) http://www.lag.net/paramiko/ Shawn From lie.1296 at gmail.com Wed Jan 6 13:20:41 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Thu, 07 Jan 2010 05:20:41 +1100 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> Message-ID: <4b44d479$1@dnews.tpgi.com.au> On 1/7/2010 2:12 AM, Phlip wrote: > On Jan 5, 10:54 pm, Benjamin Kaplan wrote: > >> {41: None}[41] ? >> >> In cases where None is a valid result, you can't use it to signal failure.. > > Asked and answered. You change the "sentinel" in .fetch to something > else. I believe Ben Kaplan's point is that if dict slicing returns sentinel for missing keys, the slicing syntax would be unusable to differentiate {41: sentinel} and {}; if the default sentinel had been None, that would make it too easy to forget to reset the sentinel and wrote a buggy code. > But y'all keep on defending the language making your programming > decisions for you! When designing a language, there are two different ways to signal a missing key in a dict: 1) raise an exception or 2) return a sentinel. Both are useful for different purpose. One or the other must be the default behavior, and the other must give way. Python decided that the default behavior should be raising exception and sentinel have to use the dict.get() method. Simple and clear. The other possible behavior (i.e. slicing returns a sentinel while dict.get() raises an exception) is arguably just as simple and just as clear; but python doesn't do it that way. Why? Because it's part of what makes python Python[1]. > But y'all keep on defending the language making your programming > decisions for you! Indeed, the language makes decisions for you; why would you want to use a language that nags you for every possible different behavior? I want a reasonable default and I want a reasonably easy way to access the non-default behaviors. [1] the essence of which is emboldened as the Zen, which is the broad design guideline for both python's syntax and codes written in python. Specifically, "Errors should never pass silently" because missing key indicates a possible error and "Simple is better than complex" because returning sentinel for missing keys implies the need for an interface to change the sentinel[2]. [2] or become like Java, whose HashMap cannot reliably store null and doesn't allow you to specify the sentinel. From lie.1296 at gmail.com Wed Jan 6 13:23:04 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Thu, 07 Jan 2010 05:23:04 +1100 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <4b44d507$1@dnews.tpgi.com.au> On 1/7/2010 3:41 AM, Phlip wrote: > Steve Holden wrote: > >> y'all just keep defending the approach to programming that >> *you* think is best. > > Speak for yourself... Everyone speaks for themselves, is that a problem? From nobody at nowhere.com Wed Jan 6 13:30:03 2010 From: nobody at nowhere.com (Nobody) Date: Wed, 06 Jan 2010 18:30:03 +0000 Subject: subprocess.Popen does not close pipe in an error case References: <07641018-9581-44af-bb20-8f9b96a44c4f@p8g2000yqb.googlegroups.com> Message-ID: On Tue, 05 Jan 2010 15:50:39 -0800, Steven K. Wong wrote: > Below, I have a Python script that launches 2 child programs, prog1 > and prog2, with prog1's stdout connected to prog2's stdin via a pipe. > (It's like executing "prog1 | prog2" in the shell.) > > If both child programs exit with 0, then the script runs to > completion. But if prog2 exits with non-0, prog1 does not exit and the > script hangs (i.e. prog1.poll() always returns None) -- unless I > uncomment the 2 lines marked by XXX to close prog1.stdout. > > I was expecting that I don't have to explicitly close prog1.stdout, > whether prog2 succeeds or fails. Is the current behavior a bug in the > subprocess module or is it expected? Or am I doing something wrong? > > Thanks. > > import subprocess > import time > > # prog1: a program that writes lots of data to the pipe > cmd = ['zcat', '--force', 'a_large_file'] > prog1 = subprocess.Popen(cmd, bufsize=-1, stdout=subprocess.PIPE) > > # prog2: a program that fails without reading much data from the pipe > cmd = ['python', '-c', 'import time; time.sleep(10); asdf'] > prog2 = subprocess.Popen(cmd, bufsize=-1, stdin=prog1.stdout, > stdout=open('popen.out', 'w')) I think that you should close prog1.stdout here. Otherwise, there will be two readers on the pipe (the calling process and prog2). Even if one of them dies, there's always the possibility that the caller might eventually decide to read prog1.stdout itself. If you close it in the caller, when prog2 terminates there will be no readers, and prog1 will get SIGPIPE (or write() will fail with EPIPE if SIGPIPE is handled). From no.i.dont at want.mail.from.spammers.com Wed Jan 6 13:33:15 2010 From: no.i.dont at want.mail.from.spammers.com (Fencer) Date: Wed, 06 Jan 2010 19:33:15 +0100 Subject: suds problem Message-ID: <7qk3bcFnbjU1@mid.individual.net> Hello, I just started using suds to use web services. First I tried suds with a very simple web service I had written and was running myself. That worked fine. Then I tried to use the web services provided by KEGG: http://soap.genome.jp/KEGG.wsdl But I get a SAXParseException due to a supposed mis-matched tag when I try to consume that wsdl with suds. I checked the wsdl in oxygene and it claims it's valid. What is the problem here? My test program is below and it's very simple: from suds.client import Client url = 'http://soap.genome.jp/KEGG.wsdl' client = Client(url) print client - Fencer From mrabarnett at mrabarnett.plus.com Wed Jan 6 13:33:19 2010 From: mrabarnett at mrabarnett.plus.com (Matthew Barnett) Date: Wed, 06 Jan 2010 18:33:19 +0000 Subject: 3 byte network ordered int, How To ? In-Reply-To: <3d4c0f161001061001ma900949j1fdea2789cff7367@mail.gmail.com> References: <3d4c0f161001061001ma900949j1fdea2789cff7367@mail.gmail.com> Message-ID: <4B44D76F.3010200@mrabarnett.plus.com> mudit tuli wrote: > For a single byte, struct.pack(') > For two bytes, struct.pack(') > what if I want three bytes ? > Four bytes and then discard the most-significant byte: struct.pack(')[ : -1] From hjtoi-better-remove-before-reply at comcast.net Wed Jan 6 13:34:11 2010 From: hjtoi-better-remove-before-reply at comcast.net (Heikki Toivonen) Date: Wed, 06 Jan 2010 10:34:11 -0800 Subject: python xmlrpc client with ssl client certificates and standard modules References: <4b414e79$0$23321$426a34cc@news.free.fr> Message-ID: News123 wrote: > This will probably work, but it requires the module M2Crypto. > > In order to avoid installing M2Crypto an all hosts that want to run the > script I wondered, whether there is no other solution. > > I can do xmlrpc over ssl WITHOUT certificates with following code: [...] Please note that if you just use the stdlib it is not secure out of the box. With Python 2.6 and the ssl module you can make it so, but it requires some work on your part. See for example http://www.heikkitoivonen.net/blog/2008/10/14/ssl-in-python-26/ -- Heikki Toivonen - http://heikkitoivonen.net From lie.1296 at gmail.com Wed Jan 6 13:48:04 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Thu, 07 Jan 2010 05:48:04 +1100 Subject: File transfer with python In-Reply-To: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> References: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> Message-ID: <4b44dae4$1@dnews.tpgi.com.au> On 1/7/2010 5:00 AM, Valentin de Pablo Fouce wrote: > My intention is to be able to transfer files from one computer to > another in this environment. Do you have a USB flashdrive? > Looking (and surfing) at internet the only suggestion given is to use > low level sockets for this file transfer. Is there another way to do > it, is there any top level library that helps you to do that? Networked file system (e.g. Samba/SMB), File Transfer Protocol (FTP), whatever works for you. pysamba is available from sourceforge and there is ftplib in the standard library. For most purposes though, you could just setup a network shared directory and drag and drop files from the file manager. From dreadpiratejeff at gmail.com Wed Jan 6 14:01:29 2010 From: dreadpiratejeff at gmail.com (J) Date: Wed, 6 Jan 2010 14:01:29 -0500 Subject: Python books, literature etc In-Reply-To: <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> Message-ID: <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> A good point was brought up to me privately, and I agree completely, that the OP should re-state the request with a bit more specifics... Since the OP says he is at least familiar with Python, does he need info on beginner level books that are general purpose, or is he interested in resources that are more specific (e.g. geared toward web programming, mathematical analysis, data modeling, etc).... My suggestions were meant just as an example of what I use in the course of learning something at the basic and intermediate level, once it goes beyond that, it's useful to know WHAT you intend to do so you can find the right resources to go in that direction. Just want to make sure I am not misunderstood or anything :-) For what it's worth, I also tend to collect technical books for some reason... My wife is just barely tolerant of my bookshelf full of things on various computer topics, astronomy, photography, radio and antenna theory and so forth ;-) I just let her keep her shoe collection, and we have a quid pro quo. Cheers Jeff -- Mike Ditka - "If God had wanted man to play soccer, he wouldn't have given us arms." - http://www.brainyquote.com/quotes/authors/m/mike_ditka.html From no.i.dont at want.mail.from.spammers.com Wed Jan 6 14:02:58 2010 From: no.i.dont at want.mail.from.spammers.com (Fencer) Date: Wed, 06 Jan 2010 20:02:58 +0100 Subject: suds problem In-Reply-To: <7qk3bcFnbjU1@mid.individual.net> References: <7qk3bcFnbjU1@mid.individual.net> Message-ID: <7qk532F25jU1@mid.individual.net> On 2010-01-06 19:33, Fencer wrote: > Hello, I just started using suds to use web services. First I tried suds > with a very simple web service I had written and was running myself. > That worked fine. Then I tried to use the web services provided by KEGG: > http://soap.genome.jp/KEGG.wsdl > But I get a SAXParseException due to a supposed mis-matched tag when I > try to consume that wsdl with suds. I checked the wsdl in oxygene and it > claims it's valid. What is the problem here? My test program is below > and it's very simple: > from suds.client import Client > > url = 'http://soap.genome.jp/KEGG.wsdl' > > client = Client(url) > > print client > > - Fencer Same problem with reactome: http://www.reactome.org:8080/caBIOWebApp/services/caBIOService?wsdl I suppose I'm mis-using suds, maybe some setting that wasn't needed for my simple web service but is needed for kegg and reactome. - Fencer From carsten.haese at gmail.com Wed Jan 6 14:09:38 2010 From: carsten.haese at gmail.com (Carsten Haese) Date: Wed, 06 Jan 2010 14:09:38 -0500 Subject: getfirst and re In-Reply-To: <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> Message-ID: Victor Subervi wrote: > I have an automatically generated HTML form from which I need to extract > data to the script which this form calls (to which the information is > sent). Ideally, the script that receives the submitted fields should know how the form was generated, so it knows what fields to expect. Failing that, you should realize that getfirst() is not the only way to access the contents of a cgi.FieldStorage object. If you need to know the values of all fields whose names obey a certain pattern, I recommend you start with a list of all field names in the FieldStorage object and pick out the ones whose names obey that pattern. To get the list of field names, you should consider using the .keys() method of the FieldStorage object. -Carsten From victorsubervi at gmail.com Wed Jan 6 14:19:40 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 6 Jan 2010 15:19:40 -0400 Subject: getfirst and re In-Reply-To: References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> Message-ID: <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> On Wed, Jan 6, 2010 at 3:09 PM, Carsten Haese wrote: > Victor Subervi wrote: > > I have an automatically generated HTML form from which I need to extract > > data to the script which this form calls (to which the information is > > sent). > > Ideally, the script that receives the submitted fields should know how > the form was generated, so it knows what fields to expect. Failing that, > you should realize that getfirst() is not the only way to access the > contents of a cgi.FieldStorage object. > > If you need to know the values of all fields whose names obey a certain > pattern, I recommend you start with a list of all field names in the > FieldStorage object and pick out the ones whose names obey that pattern. > To get the list of field names, you should consider using the .keys() > method of the FieldStorage object. > Cool. Thanks! beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From wolftracks at invalid.com Wed Jan 6 14:25:40 2010 From: wolftracks at invalid.com (W. eWatson) Date: Wed, 06 Jan 2010 11:25:40 -0800 Subject: Astronomy--Programs to Compute Siderial Time? Message-ID: Is there a smallish Python library of basic astronomical functions? There are a number of large such libraries that are crammed with excessive functions not needed for common calculations. From jcb at iteris.com Wed Jan 6 14:26:41 2010 From: jcb at iteris.com (Metalone) Date: Wed, 6 Jan 2010 11:26:41 -0800 (PST) Subject: Need help with multiprocessing.manager and passing the manager a multiprocessing.Connection Message-ID: <350adac0-c6ba-4861-8757-e1717ad8e636@c34g2000yqn.googlegroups.com> The following code snippet is taken from the Python 2.6 multiprocessing documentation with a simple change and this change does not work. I would like to know how to make it work or something similar. I want to pass a Connection object to the MathsClass. I get the following error on Windows: Traceback (most recent call last): File "c:\apps\python26\lib\multiprocessing\managers.py", line 214, in serve_cl ient request = recv() TypeError: Required argument 'handle' (pos 1) not found from multiprocessing.managers import BaseManager from multiprocessing import Pipe # I added this class MathsClass(object): def set(self, conn): # I added the set() function self.conn = conn def add(self, x, y): return x + y def mul(self, x, y): return x * y class MyManager(BaseManager): pass MyManager.register('Maths', MathsClass) if __name__ == '__main__': parent_conn, child_conn = Pipe() # I added this manager = MyManager() manager.start() maths = manager.Maths() print maths.add(4, 3) # prints 7 print maths.mul(7, 8) # prints 56 maths.set(child_conn) # I added this, error is thrown here From skwslide at gmail.com Wed Jan 6 14:39:37 2010 From: skwslide at gmail.com (Steven K. Wong) Date: Wed, 6 Jan 2010 11:39:37 -0800 (PST) Subject: subprocess.Popen does not close pipe in an error case References: <07641018-9581-44af-bb20-8f9b96a44c4f@p8g2000yqb.googlegroups.com> Message-ID: On Jan 6, 10:30 am, Nobody wrote: > I think that you should close prog1.stdout here. Otherwise, there will > be two readers on the pipe (the calling process and prog2). Even if one of > them dies, there's always the possibility that the caller might eventually > decide to read prog1.stdout itself. If you close it in the caller, when > prog2 terminates there will be no readers, and prog1 will get SIGPIPE (or > write() will fail with EPIPE if SIGPIPE is handled). Thanks for raising a great point, that prog1.stdout is also readable by the calling process, not just by prog2. Therefore, I agree it makes sense to explicitly call prog1.stdout.close() in the given code (say right after the creation of prog2). Suppose now all the prog1.poll() calls/loop are replaced by a single prog1.wait(). Without the explicit prog1.stdout.close(), prog1.wait() will not return, so the calling process still hangs. Because calling prog1.wait() means that the calling process will naturally never read prog1.stdout, I would argue that prog1.wait() should close the pipe before actually waiting for prog1 to exit. Makes sense? From invalid at invalid.invalid Wed Jan 6 14:51:38 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Wed, 6 Jan 2010 19:51:38 +0000 (UTC) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4b44d507$1@dnews.tpgi.com.au> Message-ID: On 2010-01-06, Lie Ryan wrote: > On 1/7/2010 3:41 AM, Phlip wrote: >> Steve Holden wrote: >> >>> y'all just keep defending the approach to programming that >>> *you* think is best. >> >> Speak for yourself... > > Everyone speaks for themselves, [...] Except for the Lorax. He speaks for the trees. -- Grant Edwards grante Yow! The SAME WAVE keeps at coming in and COLLAPSING visi.com like a rayon MUU-MUU ... From van.lindberg at gmail.com Wed Jan 6 15:01:10 2010 From: van.lindberg at gmail.com (VanL) Date: Wed, 06 Jan 2010 14:01:10 -0600 Subject: The END (of PyCon early bird registration) is NEAR! Message-ID: Today is the last day of registration for PyCon 2010 at the early bird rate. Registration at the early bird rate is still good as long as it is January 6 somewhere in the world. Register now! - https://us.pycon.org/2010/register/ From phlip2005 at gmail.com Wed Jan 6 15:12:08 2010 From: phlip2005 at gmail.com (Phlip) Date: Wed, 06 Jan 2010 12:12:08 -0800 Subject: please help shrink this each_with_index() implementation In-Reply-To: References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> <1715bea7-1ff1-4294-9dad-672873e14530@v25g2000yqk.googlegroups.com> Message-ID: Nobody wrote: > On Tue, 05 Jan 2010 19:46:01 -0800, alex23 wrote: > >>> They will tell me how to use except: (which is a good example why a >>> program should not use exceptions for its normal control flow if at >>> all possible). >> Really? Magic functions that coerce and eat errors are a better coding >> technique than exceptions and explicit handling? >> >> What kool-aid have you been drinking? > > Maybe he's doing it for a living? > > Contract programming seems to work on the basis that the ultimate > requirement is for the client to hand over the money. If, at that point, > the program is still full of bugs, you get to charge extra for "upgrades". Uh, no, right now I'm working with the long-term goal of creating useful modules that sustain us equitably. > Writing robust software from the outset puts you at a competitive > disadvantage to those who understand how the system works. And I, not my language, should pick and chose how to be rigorous. The language should not make the decision for me. From yoavglazner at gmail.com Wed Jan 6 15:15:45 2010 From: yoavglazner at gmail.com (Glazner) Date: Wed, 6 Jan 2010 12:15:45 -0800 (PST) Subject: an't start a thread Pool from another thread Message-ID: <47ceb43c-a219-4c5d-ad45-6ee1a96094b6@v25g2000yqk.googlegroups.com> Hi all, I hope someone can help me with this issue I see that i can't start a thread Pool from another thread, why? running python 2.6.4 windowsXP >>> import multiprocessing.dummy as threads >>> def makePool(): threads.Pool(3) >>> makePool() >>> import thread >>> thread.start_new(makePool,()) Unhandled exception in thread started by 6960 From phlip2005 at gmail.com Wed Jan 6 15:39:36 2010 From: phlip2005 at gmail.com (Phlip) Date: Wed, 6 Jan 2010 12:39:36 -0800 (PST) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4b44d507$1@dnews.tpgi.com.au> Message-ID: <8f6af7ac-35f6-4e22-b69b-7c293694be26@j14g2000yqm.googlegroups.com> On Jan 6, 10:23?am, Lie Ryan wrote: > On 1/7/2010 3:41 AM, Phlip wrote: > > > Steve Holden wrote: > > >> y'all just keep defending the approach to programming that > >> *you* think is best. > > > Speak for yourself... > > Everyone speaks for themselves, is that a problem? Of course not. I was pointing out that Steve is telling me not to force my programming opinions on everyone... ...while defending Python enforcing programming opinions on everyone. And now, if everyone will excuse me, I have to get back to writing a unit-test-to-code ratio of 2:1. Have fun being rigorous, everyone! From adityashukla1983 at gmail.com Wed Jan 6 15:44:52 2010 From: adityashukla1983 at gmail.com (aditya shukla) Date: Wed, 6 Jan 2010 14:44:52 -0600 Subject: Mencoder and creating videos Message-ID: <73045cca1001061244y61cc020fj5ea2c633df98d03f@mail.gmail.com> Hello Guys, I have a multiprocessing script which downloads images from 5 urls to 5 directories(usinf multiprocess in python 2.6).The download is for 5 mins.My aim is to create a video for every minute for each directory and dump the images as the video is created. My question are , should i use *mencoder.exe * to create the videos? and how should i create the videos without interrupting the download. Thanks Aditya -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Wed Jan 6 15:49:37 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 06 Jan 2010 15:49:37 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <4b44d479$1@dnews.tpgi.com.au> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> <4b44d479$1@dnews.tpgi.com.au> Message-ID: On 1/6/2010 1:20 PM, Lie Ryan wrote: > Python decided that the default behavior should be raising exception and > sentinel have to use the dict.get() method. Simple and clear. The other > possible behavior (i.e. slicing returns a sentinel while dict.get() > raises an exception) is arguably just as simple and just as clear; but > python doesn't do it that way. Why? Because it's part of what makes > python Python[1]. The altermatives are not quite equivalent. The current way lets one specify the sentinel whereas the alternative does not. There is hardly any reason to specify the exception. tjr From steve at REMOVE-THIS-cybersource.com.au Wed Jan 6 16:14:14 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 06 Jan 2010 21:14:14 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4b44d507$1@dnews.tpgi.com.au> <8f6af7ac-35f6-4e22-b69b-7c293694be26@j14g2000yqm.googlegroups.com> Message-ID: <0354ea0c$0$1336$c3e8da3@news.astraweb.com> On Wed, 06 Jan 2010 12:39:36 -0800, Phlip wrote: > And now, if everyone will excuse me, I have to get back to writing a > unit-test-to-code ratio of 2:1. In my experience, that's about half as many unit-tests as needed for full code coverage for even a simple class. If you're trying to impress us, you have failed. > Have fun being rigorous, everyone! You say that as if writing correct code was a bad thing. -- Steven From jason.scheirer at gmail.com Wed Jan 6 16:17:49 2010 From: jason.scheirer at gmail.com (Jason Scheirer) Date: Wed, 6 Jan 2010 13:17:49 -0800 (PST) Subject: Introspection References: <0d782b7a-11fb-4933-84b7-29a3653ec847@p8g2000yqb.googlegroups.com> <0354a972$0$1336$c3e8da3@news.astraweb.com> Message-ID: <9af6ffd9-a450-4eef-90ed-430aeea9f77c@j24g2000yqa.googlegroups.com> On Jan 6, 8:38?am, Steven D'Aprano wrote: > On Wed, 06 Jan 2010 06:53:40 -0800, m... at infoserv.dk wrote: > > I'm looking for a way to make a list of string literals in a class. > > > Example: > > > class A: > > ? ?def method(self): > > ? ? ? ?print 'A','BC' > > >>>> ExtractLiterals(A) > > ['A','BC'] > > > Is this possible? Can anyone point me in the right direction? > > class A: > ? ? def extract_literals(self): > ? ? ? ? return "A BC".split() > ? ? def method(self): > ? ? ? ? print self.extract_literals() > > a = A() > a.extract_literals() > > -- > Steven Slightly more robust than Miki's solution insofar as it doesn't require the source to exist in a .py file: import types def extract_literals(klass): for attr in (getattr(klass, item) for item in dir(klass)): if isinstance(attr, types.MethodType): for literal in attr.im_func.func_code.co_consts: if isinstance(literal, basestring): yield literal class full_of_strings(object): def a(self): return "a", "b", "c" def b(self): "y", "z" print list(extract_literals(full_of_strings)) ['a', 'b', 'c', 'y', 'z'] print list(extract_literals(full_of_strings())) ['a', 'b', 'c', 'y', 'z'] Note that this is evil and should be avoided. From steve at REMOVE-THIS-cybersource.com.au Wed Jan 6 16:20:40 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 06 Jan 2010 21:20:40 GMT Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> <1715bea7-1ff1-4294-9dad-672873e14530@v25g2000yqk.googlegroups.com> Message-ID: <0354eb8e$0$1336$c3e8da3@news.astraweb.com> On Wed, 06 Jan 2010 12:12:08 -0800, Phlip wrote: > And I, not my language, should pick and chose how to be rigorous. The > language should not make the decision for me. All languages make that decision for you by making some thing possible and other things not. The language designer, not the programmer, decides what syntactic features to allow and disallow, what programming styles to include, and what functionality to provide as primitives. -- Steven From zabin.farishta at gmail.com Wed Jan 6 16:23:39 2010 From: zabin.farishta at gmail.com (Zabin) Date: Wed, 6 Jan 2010 13:23:39 -0800 (PST) Subject: QDoubleValidator Message-ID: Hey! I am new PyQt programmer and want to restrict users to allow only numeric values into a table and lineedit boxes. I found the QDoubleValidator class but am unsure as to how to implement it. (I am a little shaky on the concept of parent and how to define them). Any help would be much appreciated! From steve at holdenweb.com Wed Jan 6 16:33:54 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 16:33:54 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <8f6af7ac-35f6-4e22-b69b-7c293694be26@j14g2000yqm.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4b44d507$1@dnews.tpgi.com.au> <8f6af7ac-35f6-4e22-b69b-7c293694be26@j14g2000yqm.googlegroups.com> Message-ID: Phlip wrote: > On Jan 6, 10:23 am, Lie Ryan wrote: >> On 1/7/2010 3:41 AM, Phlip wrote: >> >>> Steve Holden wrote: >>>> y'all just keep defending the approach to programming that >>>> *you* think is best. >>> Speak for yourself... >> Everyone speaks for themselves, is that a problem? > > Of course not. I was pointing out that Steve is telling me not to > force my programming opinions on everyone... > > ...while defending Python enforcing programming opinions on everyone. > That's because Python is not a person, and Guido is a better language designer than both of us put together. No languagecan be all things to all programmers, and Python represents a set of pragmatic and useful choices. > And now, if everyone will excuse me, I have to get back to writing a > unit-test-to-code ratio of 2:1. Have fun being rigorous, everyone! Consider yourself excused ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From zuo at chopin.edu.pl Wed Jan 6 16:42:21 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Wed, 06 Jan 2010 22:42:21 +0100 Subject: File transfer with python In-Reply-To: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> References: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> Message-ID: Valentin de Pablo Fouce wrote: > Ok, I am trying to do a very quick application (is "home based" so is > not a big deal...). My intention is to transfer files from one > computer to another. > My intention is to be able to transfer files from one computer to > another in this environment. > > Looking (and surfing) at internet the only suggestion given is to use > low level sockets for this file transfer. Is there another way to do > it, is there any top level library that helps you to do that? Python standard library offers tools for HTTP communication (rather easy to use) -- see: http://docs.python.org/library/simplehttpserver.html http://docs.python.org/library/cgihttpserver.html http://docs.python.org/library/basehttpserver.html -- as well as: http://docs.python.org/library/urllib.html http://docs.python.org/library/urllib2.html Cheers, *j -- Jan Kaliszewski (zuo) From aioe.org at technicalbloke.com Wed Jan 6 16:52:52 2010 From: aioe.org at technicalbloke.com (r0g) Date: Wed, 06 Jan 2010 21:52:52 +0000 Subject: creating tar file and streaming it over HTTP? References: <5222d923-7aef-47cf-a0d6-f37f09a66593@35g2000yqa.googlegroups.com> Message-ID: pbienst wrote: > I would like to bundle up a number of files in a tar file and send it > over a HTTP connection, but I would like to do this without creating > the tar file on disk first. > Stringio lets you treat a strings as a files... http://docs.python.org/library/stringio.html Roger. From trzewiczek at trzewiczek.info Wed Jan 6 16:53:09 2010 From: trzewiczek at trzewiczek.info (trzewiczek at trzewiczek.info) Date: Wed, 06 Jan 2010 22:53:09 +0100 Subject: GUI for multiplatform multimedia project Message-ID: Hi everyone, I posted that question on a python-forum, but got answer, so I ask here. I'm working on an artistic project and I'm looking for the best cross-platform GUI solution. The problem is that it's gonna be a tool that will have to be double-click installable/runnable and pre-installation of any libraries for end-users is very much like an evil. It really has to be double-click tool My first thought was PyQt, because it's a real framework with a lot of stuff inside (including Phonon) and I know some cross-platform media software written in C++ QT (like VLC). But on the other hand I've heard that it's not that easy to make it "double-clicky" multi-platform. Is that true? Another thing that matters for me is ease of integration with libraries like OpenCV. I will be VERY thankful for any help. I'm SO tired googling the problem (it's like weeks now!!) Best from Poland, trzewiczek From emile at fenx.com Wed Jan 6 16:56:07 2010 From: emile at fenx.com (Emile van Sebille) Date: Wed, 06 Jan 2010 13:56:07 -0800 Subject: Mencoder and creating videos In-Reply-To: <73045cca1001061244y61cc020fj5ea2c633df98d03f@mail.gmail.com> References: <73045cca1001061244y61cc020fj5ea2c633df98d03f@mail.gmail.com> Message-ID: On 1/6/2010 12:44 PM aditya shukla said... > Hello Guys, > > I have a multiprocessing script which downloads images from 5 urls to 5 > directories(usinf multiprocess in python 2.6).The download is for 5 > mins.My aim is to create a video for every minute for each directory and > dump the images as the video is created. My question are , should i use > _*mencoder.exe*_ to create the videos? I did something similar but on linux and used mencoder -- I'm guessing you can do the same on windows. > and how should i create the > videos without interrupting the download. In my situation I'd set up an ftp server and had webcams sending images once a second there. I'd also set up a cron job (scheduled task on windows systems) to create the video and archive the images. Emile From emile at fenx.com Wed Jan 6 16:58:17 2010 From: emile at fenx.com (Emile van Sebille) Date: Wed, 06 Jan 2010 13:58:17 -0800 Subject: File transfer with python In-Reply-To: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> References: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> Message-ID: On 1/6/2010 10:00 AM Valentin de Pablo Fouce said... > Hi there, > > I hope this is the rigth place, if not please, tell me which is the > right dicussion place. I apologize in such case. > > Ok, I am trying to do a very quick application (is "home based" so is > not a big deal...). My intention is to transfer files from one > computer to another. > > I am using several OS (Linux and Windows basicly, but potentially in > the future Mac too), and the computers won't be potentially in the > same network (ok, the most times will... but there will be > exceptions). cygwin + scp or rsync are other options. Emile > > My intention is to be able to transfer files from one computer to > another in this environment. > > Looking (and surfing) at internet the only suggestion given is to use > low level sockets for this file transfer. Is there another way to do > it, is there any top level library that helps you to do that? > > That's a lot! From steve at holdenweb.com Wed Jan 6 17:03:56 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 17:03:56 -0500 Subject: creating tar file and streaming it over HTTP? In-Reply-To: References: <5222d923-7aef-47cf-a0d6-f37f09a66593@35g2000yqa.googlegroups.com> Message-ID: <4B4508CC.6080508@holdenweb.com> r0g wrote: > pbienst wrote: >> I would like to bundle up a number of files in a tar file and send it >> over a HTTP connection, but I would like to do this without creating >> the tar file on disk first. >> > > Stringio lets you treat a strings as a files... > > http://docs.python.org/library/stringio.html > > Roger. ... though that does mean that the whole tar file has to be created before it can be sent, I suspect. If it will comfortably fit into memory that shouldn't matter, of course. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Wed Jan 6 17:03:56 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 17:03:56 -0500 Subject: creating tar file and streaming it over HTTP? In-Reply-To: References: <5222d923-7aef-47cf-a0d6-f37f09a66593@35g2000yqa.googlegroups.com> Message-ID: <4B4508CC.6080508@holdenweb.com> r0g wrote: > pbienst wrote: >> I would like to bundle up a number of files in a tar file and send it >> over a HTTP connection, but I would like to do this without creating >> the tar file on disk first. >> > > Stringio lets you treat a strings as a files... > > http://docs.python.org/library/stringio.html > > Roger. ... though that does mean that the whole tar file has to be created before it can be sent, I suspect. If it will comfortably fit into memory that shouldn't matter, of course. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From michi at triodia.com Wed Jan 6 17:15:55 2010 From: michi at triodia.com (Michi) Date: Wed, 6 Jan 2010 14:15:55 -0800 (PST) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <03526a3a$0$1336$c3e8da3@news.astraweb.com> Message-ID: On Jan 5, 9:44?am, Steven D'Aprano wrote: > > I'm glad we agree on that, but I wonder why you previously emphasised > machine efficiency so much, and correctness almost not at all, in your > previous post? Uh? Because the original poster quoted one small paragraph out of a large article and that paragraph happened to deal with this particular (and minor) point of API design? > If all you're argument is that we shouldn't write crappy APIs, then I > agree with you completely. Well, yes: the article was precisely about that. And the point about exception efficiency was a minor side remark in that article. > Your argument seems to be > that we should avoid exceptions by default, and only use them if > unavoidable. I think that is backwards. I never made that argument. After 25 years as a software engineer, I well and truly have come to appreciate exceptions as a superior form of error handling. I simply stated that throwing an exception when none should be thrown is a pain and often inefficient on top of that. That's all, really. > I wouldn't say that's normal. If you don't care about the function's > result, why are you calling it? For the side-effects? printf returns a value that is almost always ignored. And, yes, a function such as printf is inevitable called for its side effects. We could argue that printf is misdesigned (I would): the return value is not useful, otherwise it would be used more. And if printf threw an exception when something didn't work, that would be appropriate because it fails so rarely that, if it does fail, I probably want to know. > > However, if a function throws instead of > > returning a value, ignoring that value becomes more difficult for the > > caller and can extract a performance penalty that may be unacceptable to > > the caller. > > There's that premature micro-optimization again. Let's be clear here: the entire discussion is about *inappropriate* use of exceptions. This isn't a premature optimisation. It's about deciding when an exception is appropriate and when not. If I throw an exception when I shouldn't, I make the API harder to use *and* less efficient. The real crime isn't the loss of efficiency though, it's the inappropriate exception. > I've been wondering when you would reach the conclusion that an API > should offer both forms. For example, Python offers both key-lookup that > raises exceptions (dict[key]) and key-lookup that doesn't (dict.get(key)). > > The danger of this is that it complicates the API, leads to a more > complex implementation, and may result in duplicated code (if the two > functions have independent implementations). Offering a choice in some form can be appropriate for some APIs. I'm not advocating it as a panacea, and I'm aware of the down-side in increased complexity, learning curve, etc. (BTW, the article discusses this issue in some detail.) > Well, obviously I agree that you should only make things be an exception > if they actually should be an exception. I don't quite see where the > implication is In the context of the original article, I argued that throwing exceptions that are inappropriate is one of the many things that API designers get wrong. To many people, that's stating the obvious. The number of APIs that still make exactly this mistake suggests that the point is worth making though. Anyway, some of the early posts implied that I was arguing against exception handling per-se because exceptions can be less efficient. I responded to correct that misconception. What the article really said is that throwing an exception when none should be thrown is bad API design, and inefficient to boot. I stand by that statement. Cheers, Michi. From aioe.org at technicalbloke.com Wed Jan 6 17:36:41 2010 From: aioe.org at technicalbloke.com (r0g) Date: Wed, 06 Jan 2010 22:36:41 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> <87ljgbwq4q.fsf@benfinney.id.au> Message-ID: Ben Finney wrote: > r0g writes: > >> NO! It's a rude way to start a sentence don't you think? > > Shouting is usually rude, yes. > >> Just because you're correcting someone doesn't mean you have to be >> combative and try and make them feel small. > > Again, you're reading something that isn't there. I utterly deny the > motives you're imputing. > >> I can't imagine why I expect good manners on usenet though, AFAICT >> it's never been like that (well not since I got on it anyway). > > I'll reiterate that you can address this by paying more attention to > what is actually written, and endeavouring to avoid finding emotions or > motives that are likely not in the message. > Both you and Steve are routinely harsh is all I'm saying, although he's worse than you. I'm sure you're both very good programmers who don't have the time to tread carefully around those who come in, often under prepared, seeking advice. Fair enough you don't tolerate fools lightly, I can respect that (within reason) and you contribute a lot to the group so you're perhaps owed a bit more slack than your average Joe but... There's a difference between that and repeatedly seeking out reasons to stick your boot in - and to my mind you seem to do that. Maybe it's just me, but have you considered that maybe it's not?... Most of the time when I see an extremely blunt, overly harsh response to a question on here I know if I glance up to the name field I'll find one of your names there (as I say, probably Steve more often that yourself). I'm sure you think I'm making this up but many passive aggressive don't realise how they're acting - can't see the wood for the trees and all that. Anyway I'm going to butt out now, I'm sure everyone is getting thoroughly bored of the OT noise. Roger From pdlemper at earthlink.net Wed Jan 6 17:40:09 2010 From: pdlemper at earthlink.net (pdlemper at earthlink.net) Date: Wed, 06 Jan 2010 16:40:09 -0600 Subject: Python interactive terminal in Ubuntu Linux : some keys fouled up Message-ID: Have recently moved from XP to Ubuntu Linux. Successfully installed Python 3.1.1 in the Ubuntu 9.04 release on my desktop. Problem is the python interactive terminal >>> . Many of the keys now do not work. eg pressing left-arrow yields ^[[D right-arrow ^[[C Home ^[OH Del ^[[3~ up-arrow ^[[A Frustrating as I use all these , esp up-arrow to repeat recent lines. Found the same thing on my sons MacBook. This is not mentioned in two recent Python books or one on Ubuntu. Nor could I found help on the www. Is there any work-around ? Should I just forget the python prompt >>> ? Thanks, Dave pdlemper at earthlink.net From aioe.org at technicalbloke.com Wed Jan 6 17:46:33 2010 From: aioe.org at technicalbloke.com (r0g) Date: Wed, 06 Jan 2010 22:46:33 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: Grant Edwards wrote: > On 2010-01-06, r0g wrote: > >> NO! It's a rude way to start a sentence don't you think? > > No. When somebody asks a yes/no question, answering yes or no > seems quite polite to me. Following the yes/no answer with an > explanation of the answer is always nice, and I've little doubt > that's what happened. > Well actually I hadn't asked a question and I hadn't been talking to him, he just butted in with it. Otherwise yes I agree completely. Roger. From zabin.farishta at gmail.com Wed Jan 6 17:47:37 2010 From: zabin.farishta at gmail.com (Zabin) Date: Wed, 6 Jan 2010 14:47:37 -0800 (PST) Subject: QDoubleValidator References: Message-ID: On Jan 7, 10:23?am, Zabin wrote: > Hey! > > I am new PyQt programmer and want to restrict users to allow only > numeric values into a table and lineedit boxes. I found the > QDoubleValidator class but am unsure as to how to implement it. (I am > a little shaky on the concept of parent and how to define them). Any > help would be much appreciated! I managed to get it going for the line edit- but am stuck with checking a cell in a table as cells do not have the set validator attribute- here's the code i had for the line edit: self.ui.Thickness_datum.setValidator(QtGui.QDoubleValidator(-999.0, 999.0, 2, self.ui.Thickness_datum)) for the table i thought it would be: self.ui.table_process.item(row,2).setValidator(QtGui.QDoubleValidator (-999.0, 999.0, 2, self.ui.table_process.item(row,2))) Any help would be appreciated From jcb at iteris.com Wed Jan 6 17:50:28 2010 From: jcb at iteris.com (Metalone) Date: Wed, 6 Jan 2010 14:50:28 -0800 (PST) Subject: Problem with multiprocessing managers Message-ID: <7f1f21c0-7679-4a8d-9980-715a25841619@a6g2000yqm.googlegroups.com> >From the documentation for Using a remote manager there is the following example code: from multiprocessing.managers import BaseManager import Queue queue = Queue.Queue() class QueueManager(BaseManager): pass QueueManager.register('get_queue', callable=lambda:queue) m = QueueManager(address=('', 50000), authkey='abracadabra') s = m.get_server() s.serve_forever() I don't know how to stop the server. The documentation states that to call shutdown(), start() must be used instead of server_forever(). If I use m.start() instead of m.get_server().server_forever() I receive a PicklingError. From aioe.org at technicalbloke.com Wed Jan 6 17:55:27 2010 From: aioe.org at technicalbloke.com (r0g) Date: Wed, 06 Jan 2010 22:55:27 +0000 Subject: File transfer with python References: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> Message-ID: Valentin de Pablo Fouce wrote: > Hi there, > > I hope this is the rigth place, if not please, tell me which is the > right dicussion place. I apologize in such case. > > Ok, I am trying to do a very quick application (is "home based" so is > not a big deal...). My intention is to transfer files from one > computer to another. > > I am using several OS (Linux and Windows basicly, but potentially in > the future Mac too), and the computers won't be potentially in the > same network (ok, the most times will... but there will be > exceptions). > > My intention is to be able to transfer files from one computer to > another in this environment. > > Looking (and surfing) at internet the only suggestion given is to use > low level sockets for this file transfer. Is there another way to do > it, is there any top level library that helps you to do that? > > That's a lot! Well there's several file transfer protocols you could use, TFTP, FTP, SFTP and you can do SCP and Rysnc over SSH. AFAIK there's python libs that handle all of those. Unless you need very high performance I'd suggest using SFTP (Secure file transfer protocol). Lookup the Paramiko library, it does this and other SSH based stuff. Roger. From aioe.org at technicalbloke.com Wed Jan 6 18:24:52 2010 From: aioe.org at technicalbloke.com (r0g) Date: Wed, 06 Jan 2010 23:24:52 +0000 Subject: TypeError References: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> Message-ID: MRAB wrote: > Victor Subervi wrote: >> Hi; >> I get this error: >> >> /var/www/html/angrynates.com/christians/cart/simplemail/mail.py >> >> 153 >> 154 ''' >> 155 commitSale() >> 156 myMail() >> 157 print ''' >> commitSale = >> /var/www/html/angrynates.com/christians/cart/simplemail/mail.py >> in >> commitSale() >> 98 cursor.execute('select max(ID) from %sCustomerData;' % store) >> 99 custID = cursor.fetchone()[0] >> 100 customerData(store, tmpTable, custID, patientID) >> 101 >> 102 def myMail(): >> global customerData = , global store = >> 'products', tmpTable = 'tem12627568064', custID = 1, global patientID >> = 'None' >> /var/www/html/angrynates.com/christians/cart/customerData.py >> in >> customerData(store='products', tmpTable='tem12627568064', custID=1, >> patientID='None') >> 39 >> 40 """ >> 41 print """ >> 42 print '

%s Customer Data

' % >> (store[0].upper() + store[1:]) >> 43 cursor.execute('describe %sCustomerData' % store) >> store = 'products' >> >> ValueError: unsupported format character '(' (0x28) at index 54 >> args = ("unsupported format character '(' (0x28) at index 54",) >> >> Apparently that character is a "file separator", which I presume is an >> invisible character. I tried retyping the area in question, but with >> no avail (threw same error). Please advise. Complete code follows. >> > Please provide the actual code (and, preferably, an easier-to-understand > traceback, like what CPython does!). What you supplied isn't runnable. Oh and, it's probably not a good idea to post things that identify the site you're working on, especially if you're fairly new to programming and you're doing a shopping cart! Hackers have been known to trawl google looking for mention of novice coders websites, giving them domain names, internal paths, table names etc might well help them hack you! Cheers & good luck, Roger. From aioe.org at technicalbloke.com Wed Jan 6 18:44:48 2010 From: aioe.org at technicalbloke.com (r0g) Date: Wed, 06 Jan 2010 23:44:48 +0000 Subject: Mencoder and creating videos References: <73045cca1001061244y61cc020fj5ea2c633df98d03f@mail.gmail.com> Message-ID: Emile van Sebille wrote: > On 1/6/2010 12:44 PM aditya shukla said... >> Hello Guys, >> >> I have a multiprocessing script which downloads images from 5 urls to 5 >> directories(usinf multiprocess in python 2.6).The download is for 5 >> mins.My aim is to create a video for every minute for each directory and >> dump the images as the video is created. My question are , should i use >> _*mencoder.exe*_ to create the videos? > Ffmpeg will do multiprocessor encoding too if the compression codec supports it. It will also compile stills into video as long as you are careful to number the file names sequentially. There's no "official" windows build of it but there's various 3rd party builds floating around the web, or you can build it yourself. Roger, From solipsis at pitrou.net Wed Jan 6 18:45:31 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 6 Jan 2010 23:45:31 +0000 (UTC) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> <1715bea7-1ff1-4294-9dad-672873e14530@v25g2000yqk.googlegroups.com> Message-ID: Le Wed, 06 Jan 2010 12:12:08 -0800, Phlip a ?crit?: > > And I, not my language, should pick and chose how to be rigorous. The > language should not make the decision for me. And that's why there is the "try: ... except: ..." construct. Your rant is getting tiring. From steven at REMOVE.THIS.cybersource.com.au Wed Jan 6 18:45:40 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 06 Jan 2010 23:45:40 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: On Wed, 06 Jan 2010 22:46:33 +0000, r0g wrote: > Grant Edwards wrote: >> On 2010-01-06, r0g wrote: >> >>> NO! It's a rude way to start a sentence don't you think? >> >> No. When somebody asks a yes/no question, answering yes or no seems >> quite polite to me. Following the yes/no answer with an explanation of >> the answer is always nice, and I've little doubt that's what happened. >> >> > Well actually I hadn't asked a question and I hadn't been talking to > him, he just butted in with it. Otherwise yes I agree completely. It's a public forum. You're talking to the whole world. -- Steven From martin at v.loewis.de Wed Jan 6 18:55:12 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Thu, 07 Jan 2010 00:55:12 +0100 Subject: python xmlrpc client with ssl client certificates and standard modules In-Reply-To: <4b43007c$0$11218$426a74cc@news.free.fr> References: <4b414e79$0$23321$426a34cc@news.free.fr> <4B4267DB.3080409@v.loewis.de> <4b43007c$0$11218$426a74cc@news.free.fr> Message-ID: <4B4522E0.4030206@v.loewis.de> > My next task is how to find out at the client side, that the server > certificate is a properly signed one. As Heikki says, you'll need Python 2.6 for that. You'll probably need to extend your transport implementation. Regards, Martin From ben+python at benfinney.id.au Wed Jan 6 18:55:58 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 07 Jan 2010 10:55:58 +1100 Subject: Assertions, challenges, and polite discourse (was: Exception as the primary error handling mechanism?) References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: <87hbqywzxt.fsf_-_@benfinney.id.au> Grant Edwards writes: > Answering a yes/no question with "no" doesn't seem to me to be > combative if the correct answer is indeed "no". But I've lost > track of the post you found objectionable... In fairness, the ?No? was in response, not to an explicit question, but to an assertion. Every assertion expressed, though, implies the question ?is this assertion true??. It was that question that was answered ?No? (followed by an explanation of why the assertion was not true). People sometimes get upset ? on an immediate, irrational level ? when their assertions are challenged. There's no denying that emotions entangle our discourse, and our interpretation of the discourse of others. That's not something I'd ever want to eradicate. I ask only that, rather than decrying that assertions be challenged per se, the challenge be assessed to see whether it's valid. -- \ ?Everything you read in newspapers is absolutely true, except | `\ for that rare story of which you happen to have first-hand | _o__) knowledge.? ?Erwin Knoll | Ben Finney From aioe.org at technicalbloke.com Wed Jan 6 18:58:21 2010 From: aioe.org at technicalbloke.com (r0g) Date: Wed, 06 Jan 2010 23:58:21 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > On Wed, 06 Jan 2010 22:46:33 +0000, r0g wrote: > >> Grant Edwards wrote: >>> On 2010-01-06, r0g wrote: >>> >>>> NO! It's a rude way to start a sentence don't you think? >>> No. When somebody asks a yes/no question, answering yes or no seems >>> quite polite to me. Following the yes/no answer with an explanation of >>> the answer is always nice, and I've little doubt that's what happened. >>> >>> >> Well actually I hadn't asked a question and I hadn't been talking to >> him, he just butted in with it. Otherwise yes I agree completely. > > > It's a public forum. You're talking to the whole world. > > > > See? Spoiling for an argument even now! I never said you weren't allowed to butt in, just that you did. Butting in is fine, that's half the point of public groups after all but it's also besides the point. I was merely explaining to Grant that I hadn't posed a yes/no question to anyone, let alone you. Roger. From casevh at gmail.com Wed Jan 6 19:09:08 2010 From: casevh at gmail.com (casevh) Date: Wed, 6 Jan 2010 16:09:08 -0800 (PST) Subject: Python interactive terminal in Ubuntu Linux : some keys fouled up References: Message-ID: On Jan 6, 2:40?pm, pdlem... at earthlink.net wrote: > Have recently moved from XP to Ubuntu Linux. > Successfully installed Python 3.1.1 in the Ubuntu 9.04 > release on my desktop. > Problem is the python interactive terminal ?>>> . > Many of the keys now do not work. > eg pressing left-arrow yields ?^[[D > ? ? ? ? ? ? ? ? ? ? right-arrow ? ? ? ? ^[[C > ? ? ? ? ? ? ? ? ? ? Home ? ? ? ? ? ? ? ? ?^[OH > ? ? ? ? ? ? ? ? ? ? Del ? ? ? ? ? ? ? ? ? ? ?^[[3~ > ? ? ? ? ? ? ? ? ? ? up-arrow ? ? ? ? ? ?^[[A ? ? ? ? ? ? ? ? ? > > Frustrating as I use all these , esp ?up-arrow to > repeat recent lines. ?Found the same thing on > my sons MacBook. > > This is not mentioned in two recent Python books > or one on Ubuntu. Nor could I found help on the www. > > Is there any work-around ?? ?Should I just forget > the python prompt >>> ? ? ? ? ? ? ? ? ? ? ? > > Thanks, ? ? ? ? ? ? Dave ? ? ? ?pdlem... at earthlink.net Assuming you compiled the source code, you will also need to install "libreadline5-dev" via Synaptic or apt-get. casevh From no.i.dont at want.mail.from.spammers.com Wed Jan 6 19:10:24 2010 From: no.i.dont at want.mail.from.spammers.com (Fencer) Date: Thu, 07 Jan 2010 01:10:24 +0100 Subject: suds problem In-Reply-To: <7qk532F25jU1@mid.individual.net> References: <7qk3bcFnbjU1@mid.individual.net> <7qk532F25jU1@mid.individual.net> Message-ID: <7qkn3hFe6nU1@mid.individual.net> On 2010-01-06 20:02, Fencer wrote: > On 2010-01-06 19:33, Fencer wrote: >> Hello, I just started using suds to use web services. First I tried suds >> with a very simple web service I had written and was running myself. >> That worked fine. Then I tried to use the web services provided by KEGG: >> http://soap.genome.jp/KEGG.wsdl >> But I get a SAXParseException due to a supposed mis-matched tag when I >> try to consume that wsdl with suds. I checked the wsdl in oxygene and it >> claims it's valid. What is the problem here? My test program is below >> and it's very simple: >> from suds.client import Client >> >> url = 'http://soap.genome.jp/KEGG.wsdl' >> >> client = Client(url) >> >> print client >> >> - Fencer > > Same problem with reactome: > http://www.reactome.org:8080/caBIOWebApp/services/caBIOService?wsdl > > I suppose I'm mis-using suds, maybe some setting that wasn't needed for > my simple web service but is needed for kegg and reactome. > > - Fencer Seems to be a known problem regarding suds. :( A work-around was suggested to remove some cache files but that didn't solve it for me. Appreciate other ideas! - Fencer From sjmachin at lexicon.net Wed Jan 6 19:14:41 2010 From: sjmachin at lexicon.net (John Machin) Date: Wed, 6 Jan 2010 16:14:41 -0800 (PST) Subject: TypeError References: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> Message-ID: On Jan 7, 3:29?am, MRAB wrote: > Victor Subervi wrote: > > ValueError: unsupported format character '(' (0x28) at index 54 > > ? ? ? args = ("unsupported format character '(' (0x28) at index 54",) > > > Apparently that character is a "file separator", which I presume is an > > invisible character. I tried retyping the area in question, but with no > > avail (threw same error). Please advise. Complete code follows. > OP is barking up the wrong tree. "file separator" has ordinal 28 DECIMAL. Correct tree contains '(' (left parenthesis, ordinal 0x28 (HEX)) as the error message says. From sjmachin at lexicon.net Wed Jan 6 19:17:57 2010 From: sjmachin at lexicon.net (John Machin) Date: Wed, 6 Jan 2010 16:17:57 -0800 (PST) Subject: 3 byte network ordered int, How To ? References: <3d4c0f161001061001ma900949j1fdea2789cff7367@mail.gmail.com> Message-ID: <02bc346a-81dc-4071-8914-2f7ba0dc2cb2@q4g2000yqm.googlegroups.com> On Jan 7, 5:33?am, Matthew Barnett wrote: > mudit tuli wrote: > > For a single byte, struct.pack(') > > For two bytes, struct.pack(') > > what if I want three bytes ? > > Four bytes and then discard the most-significant byte: > > struct.pack(')[ : -1] AARRGGHH! network ordering is BIGendian, struct.pack('<..... is LITTLEendian From wuwei23 at gmail.com Wed Jan 6 19:33:58 2010 From: wuwei23 at gmail.com (alex23) Date: Wed, 6 Jan 2010 16:33:58 -0800 (PST) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> <1715bea7-1ff1-4294-9dad-672873e14530@v25g2000yqk.googlegroups.com> Message-ID: Phlip wrote: > And I, not my language, should pick and chose how to be rigorous. The language > should not make the decision for me. Since you seem unwilling to put the minimal effort into producing the support code you'd need to work with Python the way you want, perhaps Perl might be more to your liking? From aioe.org at technicalbloke.com Wed Jan 6 19:35:42 2010 From: aioe.org at technicalbloke.com (r0g) Date: Thu, 07 Jan 2010 00:35:42 +0000 Subject: Assertions, challenges, and polite discourse References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> <87hbqywzxt.fsf_-_@benfinney.id.au> Message-ID: Ben Finney wrote: > In fairness, the ?No? was in response, not to an explicit question, but > to an assertion. > > Every assertion expressed, though, implies the question ?is this > assertion true??. It was that question that was answered ?No? (followed > by an explanation of why the assertion was not true). > That's a fair point I had never really considered. I'd wouldn't have been nearly as upset if you'd started your sentence with AssertionError :) > People sometimes get upset ? on an immediate, irrational level ? when > their assertions are challenged. There's no denying that emotions > entangle our discourse, and our interpretation of the discourse of > others. That's truer than most people appreciate, to the extent that it's a good idea to tread very lightly when correcting strangers if you want rational discourse to continue. Even small amounts of negativity commonly provoke large threat responses in people which in turn inhibit rational thinking... Have a watch of this Google Tech Talk if you have time, it's really quite enlightening... http://www.youtube.com/watch?v=XeJSXfXep4M > > That's not something I'd ever want to eradicate. I ask only that, rather > than decrying that assertions be challenged per se, the challenge be > assessed to see whether it's valid. > Well I think sometimes, for the sake of expediency and overall pleasantness, it's better to let the smaller things go: and if you just can't let them go then at least try and issue corrections in a friendly manner rather than a cold or pious one. As this thread is demonstrating endless checking of every detail of every assertion leads to very long execution times and on balance I'm not sure we benefit from these extra digits of precision. Roger. From zabin.farishta at gmail.com Wed Jan 6 19:37:05 2010 From: zabin.farishta at gmail.com (Zabin) Date: Wed, 6 Jan 2010 16:37:05 -0800 (PST) Subject: Validating cells of a table PyQt Message-ID: <2d6c6f77-30cc-4787-919a-f2781026dd66@a21g2000yqc.googlegroups.com> Hey! I am new PyQT programmer. I am trying to create a table in which cells only take in numeric data. I am unable to use setValidator on table cells. Looking around i found some material on the editor attribute but I dont know how to apply this. Any help will be appreciated Cheers! From wolftracks at invalid.com Wed Jan 6 19:40:31 2010 From: wolftracks at invalid.com (W. eWatson) Date: Wed, 06 Jan 2010 16:40:31 -0800 Subject: Astronomy--Programs to Compute Siderial Time? In-Reply-To: References: Message-ID: W. eWatson wrote: > Is there a smallish Python library of basic astronomical functions? > There are a number of large such libraries that are crammed with > excessive functions not needed for common calculations. It looks like I've entered a new era in my knowledge of Python. I found a module somewhat like I want, siderial.py. You can see an intro to it at . It appears that I can get the code for it through section 1.2, near the bottom. I scooped it siderial.py up, and placed it in a corresponding file of the same name and type via NotePad. However, there is a xml file below it. I know little about it. I thought maybe I could do the same, but Notepad didn't like some characters in it. As I understand Python doc files are useful. So how do I get this done, and where do I put the files? From wuwei23 at gmail.com Wed Jan 6 19:40:46 2010 From: wuwei23 at gmail.com (alex23) Date: Wed, 6 Jan 2010 16:40:46 -0800 (PST) Subject: File transfer with python References: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> Message-ID: <0856a3ed-40bf-4027-89e0-ffcb6d9e1d04@35g2000yqa.googlegroups.com> Valentin de Pablo Fouce wrote: > My intention is to be able to transfer files from one computer to > another in this environment. > > Looking (and surfing) at internet the only suggestion given is to use > low level sockets for this file transfer. Is there another way to do > it, is there any top level library that helps you to do that? If your computers are distributed across many networks, and they all have internet access, Richard Jones' GmailFS[1] might give you some ideas. It uses the Python bindings for FUSE to create a local filestore that's a wrapper around the Gmail API, stashing the actual data on Google's servers. It's out of date now, but it shouldn't be much of an effort to update to the latest Gmail API, or to replace with any other backend data store you like. Google App Engine[2] provides a hefty amount of free storage to developers, so that's another option. 1: http://richard.jones.name/google-hacks/gmail-filesystem/gmail-filesystem.html 2: http://code.google.com/appengine/ From pavlovevidence at gmail.com Wed Jan 6 19:52:03 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Wed, 6 Jan 2010 16:52:03 -0800 (PST) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> <1715bea7-1ff1-4294-9dad-672873e14530@v25g2000yqk.googlegroups.com> Message-ID: <58a5e5bf-ca89-4fd6-be57-c187975a7391@s19g2000vbm.googlegroups.com> On Jan 6, 12:12?pm, Phlip wrote: > Nobody wrote: > > Writing robust software from the outset puts you at a competitive > > disadvantage to those who understand how the system works. > > And I, not my language, should pick and chose how to be rigorous. The language > should not make the decision for me. Oh well, your language does. Deal with it, or pick another language. Carl Banks P.S. Actually, Python has ways to request no expection when it's actually useful, e.g. the get method of dicts, but not when it's ridiculous and stupid like int() returning zero on error. From ben+python at benfinney.id.au Wed Jan 6 19:53:24 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 07 Jan 2010 11:53:24 +1100 Subject: Assertions, challenges, and polite discourse References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> <87hbqywzxt.fsf_-_@benfinney.id.au> Message-ID: <873a2iwxa3.fsf@benfinney.id.au> r0g writes: > Ben Finney wrote: > > People sometimes get upset ? on an immediate, irrational level ? > > when their assertions are challenged. There's no denying that > > emotions entangle our discourse, and our interpretation of the > > discourse of others. > > That's truer than most people appreciate, to the extent that it's a > good idea to tread very lightly when correcting strangers if you want > rational discourse to continue. Even small amounts of negativity > commonly provoke large threat responses in people which in turn > inhibit rational thinking... I prefer the strategy of acknowledging and desensitising this irrational response, by making it obvious that every assertion expressed is inevitably an exposure of that assertion to challenge and criticism. Challenging assertions and criticising reasoning are both healthy and in insufficient supply, and I want them to be normal and routine. I try to act accordingly. -- \ ?Nothing is more sacred than the facts.? ?Sam Harris, _The End | `\ of Faith_, 2004 | _o__) | Ben Finney From sjmachin at lexicon.net Wed Jan 6 19:53:54 2010 From: sjmachin at lexicon.net (John Machin) Date: Wed, 6 Jan 2010 16:53:54 -0800 (PST) Subject: parsing an Excel formula with the re module References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> <5ef3575d-eb1d-4a15-90e0-87dd85cdc2e6@35g2000yqa.googlegroups.com> <6473e930-c5f4-423f-b78b-aae63f2aa794@k17g2000yqh.googlegroups.com> Message-ID: <8cff3674-f113-4b80-8284-56faa1bc53da@c3g2000yqd.googlegroups.com> On Jan 6, 6:54?am, vsoler wrote: > On 5 ene, 20:21, vsoler wrote: > > > > > On 5 ene, 20:05, Mensanator wrote: > > > > On Jan 5, 12:35?pm, MRAB wrote: > > > > > vsoler wrote: > > > > > Hello, > > > > > > I am acessing an Excel file by means of Win 32 COM technology. > > > > > For a given cell, I am able to read its formula. I want to make a map > > > > > of how cells reference one another, how different sheets reference one > > > > > another, how workbooks reference one another, etc. > > > > > > Hence, I need to parse Excel formulas. Can I do it by means only of re > > > > > (regular expressions)? > > > > > > I know that for simple formulas such as "=3*A7+5" it is indeed > > > > > possible. What about complex for formulas that include functions, > > > > > sheet names and possibly other *.xls files? > > > > > > For example ? ?"=Book1!A5+8" should be parsed into ["=","Book1", "!", > > > > > "A5","+","8"] > > > > > > Can anybody help? Any suggestions? > > > > > Do you mean "how" or do you really mean "whether", ie, get a list of the > > > > other cells that are referred to by a certain cell, for example, > > > > "=3*A7+5" should give ["A7"] and "=Book1!A5+8" should give ["Book1!A5] > > > > Ok, although "Book1" would be the default name of a workbook, with > > > default > > > worksheets labeled "Sheet1". "Sheet2", etc. > > > > If I had a worksheet named "Sheety" that wanted to reference a cell on > > > "Sheetx" > > > OF THE SAME WORKBOOK, it would be =Sheet2!A7. If the reference was to > > > a completely > > > different workbook (say Book1 with worksheets labeled "Sheet1", > > > "Sheet2") then > > > the cell might have =[Book1]Sheet1!A7. > > > > And don't forget the $'s! You may see =[Book1]Sheet1!$A$7. > > > Yes, Mensanator, but... ?what re should I use? I'm looking for the re > > statement. No doubt you can help! > > > Thank you. > > Let me give you an example: > > >>> import re > >>> re.split("([^0-9])", "123+456*/") > > [?123?, ?+?, ?456?, ?*?, ??, ?/?, ??] > > I find it excellent that one single statement is able to do a lexical > analysis of an expression! That is NOT lexical analysis. > > If the expression contains variables, such as A12 or B9, I can try > another re expression. Which one should I use? > > And if my expression contains parenthesis? ? And the sin() function? You need a proper lexical analysis, followed by a parser. What you are trying to do can NOT be accomplished in any generality with a single regex. The Excel formula syntax has several tricky bits. E.g. IIRC whether TAX09 is a (macro) name or a cell reference depends on what version of Excel you are targetting but if it appears like TAX09! A1:B2 then it's a sheet name. The xlwt package (of which I am the maintainer) has a lexer and parser for a largish subset of the syntax ... see http://pypi.python.org/pypi/xlwt From roy at panix.com Wed Jan 6 20:22:48 2010 From: roy at panix.com (Roy Smith) Date: Wed, 06 Jan 2010 20:22:48 -0500 Subject: Astronomy--Programs to Compute Siderial Time? References: Message-ID: In article , "W. eWatson" wrote: > Is there a smallish Python library of basic astronomical functions? > There are a number of large such libraries that are crammed with > excessive functions not needed for common calculations. FWIW, if you have any interest in this kind of stuff, you must read the classic book on the subject: http://www.amazon.com/Astronomical-Formulae-Calculators-Jean-Meeus/dp/094339 6220 This is not some textbook which takes graduate level physics to understand. It's a straight-forward primer for people who want to use calculators to compute phase of the moon and stuff like that. From sjmachin at lexicon.net Wed Jan 6 20:31:16 2010 From: sjmachin at lexicon.net (John Machin) Date: Wed, 6 Jan 2010 17:31:16 -0800 (PST) Subject: TypeError References: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> Message-ID: <4c29f4d1-a4d2-4877-91fd-09b346f1a5f2@k19g2000yqc.googlegroups.com> On Jan 7, 11:14?am, John Machin wrote: > On Jan 7, 3:29?am, MRAB wrote: > > > Victor Subervi wrote: > > > ValueError: unsupported format character '(' (0x28) at index 54 > > > ? ? ? args = ("unsupported format character '(' (0x28) at index 54",) > > > > Apparently that character is a "file separator", which I presume is an > > > invisible character. I tried retyping the area in question, but with no > > > avail (threw same error). Please advise. Complete code follows. > > OP is barking up the wrong tree. "file separator" has ordinal 28 > DECIMAL. Correct tree contains '(' (left parenthesis, ordinal 0x28 > (HEX)) as the error message says. It took a bit of mucking about to get an example of that error message (without reading the Python source code): |>>> anything = object() \|>>> "foo%(" % anything Traceback (most recent call last): File "", line 1, in TypeError: format requires a mapping |>>> "foo%(" % {} Traceback (most recent call last): File "", line 1, in ValueError: incomplete format key |>>> "foo%2(" % anything Traceback (most recent call last): File "", line 1, in ValueError: unsupported format character '(' (0x28) at index 5 FWIW, the OP's message subject is "TypeError" but the reported message contains ValueError ... possibly indicative of code that first builds a format string (incorrectly) and then uses it with error messages that can vary from run to run depending on exactly what was stuffed into the format string. I note that in the code shown there are examples of building an SQL query where the table name is concocted at runtime via the % operator ... key phrases: "bad database design" (one table per store!), "SQL injection attack" A proper traceback would be very nice ... at this stage it's not certain what was the line of source that triggers the exception. From steven at REMOVE.THIS.cybersource.com.au Wed Jan 6 20:31:38 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 07 Jan 2010 01:31:38 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: On Wed, 06 Jan 2010 23:58:21 +0000, r0g wrote: > Steven D'Aprano wrote: >> On Wed, 06 Jan 2010 22:46:33 +0000, r0g wrote: >> >>> Grant Edwards wrote: >>>> On 2010-01-06, r0g wrote: >>>> >>>>> NO! It's a rude way to start a sentence don't you think? >>>> No. When somebody asks a yes/no question, answering yes or no seems >>>> quite polite to me. Following the yes/no answer with an explanation >>>> of the answer is always nice, and I've little doubt that's what >>>> happened. >>>> >>>> >>> Well actually I hadn't asked a question and I hadn't been talking to >>> him, he just butted in with it. Otherwise yes I agree completely. >> >> >> It's a public forum. You're talking to the whole world. > > > See? Spoiling for an argument even now! I never said you weren't allowed > to butt in, just that you did. Butting in is fine, that's half the point > of public groups after all but it's also besides the point. I was merely > explaining to Grant that I hadn't posed a yes/no question to anyone, let > alone you. I quote from your very next post: [quote] Well I think sometimes, for the sake of expediency and overall pleasantness, it's better to let the smaller things go: and if you just can't let them go then at least try and issue corrections in a friendly manner rather than a cold or pious one. [end quote] Perhaps you should consider taking your own advice instead of lecturing us in an unpleasant, aggressive manner about how horrible we are for answering your questions with things you don't want to hear? BTW, you were actually talking about Ben butting in -- it was Ben's answer that started with "No" that triggered this series of complaints from you: [quote] I might have let it slip had you not started your reply with the word "No", that just p***** me off. [end quote] And the offending, dastardly comment from Ben? [r0g] Yes, it returns a tuple if you return more than one value, it just has a lovely syntax for it. [ben] No, there is nothing inherent to the ?return? statement for dealing with multiple values. The word "No" was clearly and obviously a response to the previous sentence that started "Yes". If this is all it takes to put you in a temper ("p***** me off") then I suggest you need to look at your own behaviour and stop blaming others. -- Steven From aioe.org at technicalbloke.com Wed Jan 6 20:35:33 2010 From: aioe.org at technicalbloke.com (r0g) Date: Thu, 07 Jan 2010 01:35:33 +0000 Subject: Assertions, challenges, and polite discourse References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> <87hbqywzxt.fsf_-_@benfinney.id.au> <873a2iwxa3.fsf@benfinney.id.au> Message-ID: Ben Finney wrote: > r0g writes: > >> Ben Finney wrote: >>> People sometimes get upset ? on an immediate, irrational level ? >>> when their assertions are challenged. There's no denying that >>> emotions entangle our discourse, and our interpretation of the >>> discourse of others. >> That's truer than most people appreciate, to the extent that it's a >> good idea to tread very lightly when correcting strangers if you want >> rational discourse to continue. Even small amounts of negativity >> commonly provoke large threat responses in people which in turn >> inhibit rational thinking... > > I prefer the strategy of acknowledging and desensitising this irrational > response, by making it obvious that every assertion expressed is > inevitably an exposure of that assertion to challenge and criticism. I see what you're saying but the "tough love" approach has been shown to be ineffective in many of the situations it has been tried in, it's something people instinctively think ought to work but rarely does in the modern world. Anyway, it's only an irrational response in the sense that it is likely to provoke irrationality in your counterpart which isn't a good outcome for either of you. To our brains it is a very natural and powerful response. That it's emotional doesn't necessarily make it irrational. Bear in mind that evolution is the distilled rationality of countless generations and to fight it is to engage in a very one sided battle. Seriously, watch that video I recommended, it's very interesting and delves into the neurology of this exact subject. > > Challenging assertions and criticising reasoning are both healthy and in > insufficient supply, and I want them to be normal and routine. I try to > act accordingly. > OK that's fair enough as a general principle but I disagree we are insufficiently supplied with either here on usenet. I reckon we have too much if anything. I do understand that a balance must be struck and the bar ought to err on the high side after, all this group isn't yahoo answers but neither is it a peer reviewed journal or a legal proceeding. And anyway, my main point was concerning the tone used when challenging (perceived) falsehoods rather than the rationale behind challenging (perceived) falsehoods. Naturally it's right to correct non-trivial technical falsehoods in the context of this group. Roger. From wolftracks at invalid.com Wed Jan 6 20:43:10 2010 From: wolftracks at invalid.com (W. eWatson) Date: Wed, 06 Jan 2010 17:43:10 -0800 Subject: Astronomy--Programs to Compute Siderial Time? In-Reply-To: References: Message-ID: Roy Smith wrote: > In article , > "W. eWatson" wrote: > >> Is there a smallish Python library of basic astronomical functions? >> There are a number of large such libraries that are crammed with >> excessive functions not needed for common calculations. > > FWIW, if you have any interest in this kind of stuff, you must read the > classic book on the subject: > > http://www.amazon.com/Astronomical-Formulae-Calculators-Jean-Meeus/dp/094339 > 6220 > > This is not some textbook which takes graduate level physics to understand. > It's a straight-forward primer for people who want to use calculators to > compute phase of the moon and stuff like that. Thanks, but I'm quite familiar with it. My copy is some 10-15 years old. I suspect he never wrote a newer edition for any computer language. I do have access to a C++ library, but I don't think that's going to do me much good with Python. My problem at the moment is finding either someone who has implemented it or another similar module. Note I posted just before you that I have found a siderial.py module, but am not familiar with how one installs them or the doc mechanism. From sjmachin at lexicon.net Wed Jan 6 20:49:41 2010 From: sjmachin at lexicon.net (John Machin) Date: Wed, 6 Jan 2010 17:49:41 -0800 (PST) Subject: Astronomy--Programs to Compute Siderial Time? References: Message-ID: On Jan 7, 11:40?am, "W. eWatson" wrote: > W. eWatson wrote: > > Is there a smallish Python library of basic astronomical functions? > > There are a number of large such libraries that are crammed with > > excessive functions not needed for common calculations. > > It looks like I've entered a new era in my knowledge of Python. Mild curiosity: this would be a wonderful outcome, but what makes it look so? > I found > a module somewhat like I want, siderial.py. You can see an intro to it > at . > It appears that I can get the code for it through section 1.2, near the > bottom. I scooped it siderial.py up, and placed it in a corresponding > file of the same name and type via NotePad. However, there is a xml file > below it. I know little about it. I thought maybe I could do the same, > but Notepad didn't like some characters in it. As I understand Python > doc files are useful. So how do I get this done, and where do I put the > files? The file you need is sidereal.py, not your twice-mentioned siderial.py (the existence of which on the referenced website is doubtful). What you have been reading is the "Internal maintenance specification" (large font, near the top of the page) for the module. The xml file is the source of the docs, not meant to be user-legible. A very tiny amount of googling "sidereal.py" (quotes included) leads to the user documentation at http://infohost.nmt.edu/tcc/help/lang/python/examples/sidereal/ Where do you put the files? Well, we're now down to only one file, sidereal.py, and you put it wherever you'd put any other module that you'd like to call ... if there's only going to be one caller, put it in the same directory as that caller's code. More generally, drop it in /Lib/site-packages From aioe.org at technicalbloke.com Wed Jan 6 20:54:25 2010 From: aioe.org at technicalbloke.com (r0g) Date: Thu, 07 Jan 2010 01:54:25 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > On Wed, 06 Jan 2010 23:58:21 +0000, r0g wrote: > >> Steven D'Aprano wrote: >>> On Wed, 06 Jan 2010 22:46:33 +0000, r0g wrote: >>> >>>> Grant Edwards wrote: >>>>> On 2010-01-06, r0g wrote: >> See? Spoiling for an argument even now! I never said you weren't allowed >> to butt in, just that you did. Butting in is fine, that's half the point >> of public groups after all but it's also besides the point. I was merely >> explaining to Grant that I hadn't posed a yes/no question to anyone, let >> alone you. > > I quote from your very next post: > > > [quote] > Well I think sometimes, for the sake of expediency and overall > pleasantness, it's better to let the smaller things go: and if you just > can't let them go then at least try and issue corrections in a friendly > manner rather than a cold or pious one. > [end quote] > > I did answer HIM in a friendly manner, to you I'm merely responding in kind. Anyway I got upset at Ben's comments for several reasons which I explained at length in that very post (and several subsequent ones) so I won't rehash them any further here. The "No" in question was merely the straw that broke the camels back and triggered my admittedly undignified outburst. I have just seen too many supercilious replies beginning "No, " in this forum and so yes, it p***** me off - I can happily admit it. Of course clearly you're too cool to ever get p***** off yourself, I detect no background seething at all here. Roger. From steve at holdenweb.com Wed Jan 6 21:38:23 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 21:38:23 -0500 Subject: TypeError In-Reply-To: <4c29f4d1-a4d2-4877-91fd-09b346f1a5f2@k19g2000yqc.googlegroups.com> References: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> <4c29f4d1-a4d2-4877-91fd-09b346f1a5f2@k19g2000yqc.googlegroups.com> Message-ID: John Machin wrote: [...] > I note that in the code shown there are examples of building an SQL > query where the table name is concocted at runtime via the % > operator ... key phrases: "bad database design" (one table per > store!), "SQL injection attack" > I'm not trying to defend the code overall, but most databases won't let you parameterize the table or column names, just the data values. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Wed Jan 6 21:43:29 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 21:43:29 -0500 Subject: TypeError In-Reply-To: References: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> <4c29f4d1-a4d2-4877-91fd-09b346f1a5f2@k19g2000yqc.googlegroups.com> Message-ID: Steve Holden wrote: > John Machin wrote: > [...] >> I note that in the code shown there are examples of building an SQL >> query where the table name is concocted at runtime via the % >> operator ... key phrases: "bad database design" (one table per >> store!), "SQL injection attack" >> > I'm not trying to defend the code overall, but most databases won't let > you parameterize the table or column names, just the data values. > And, apropos of nothing in particular, here's a completely gratuitous additional chance to tell me off again for spamming the list about a conference: http://holdenweb.blogspot.com/2010/01/register-for-pycon-or-kitten-gets-it.html regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From wuwei23 at gmail.com Wed Jan 6 21:58:29 2010 From: wuwei23 at gmail.com (alex23) Date: Wed, 6 Jan 2010 18:58:29 -0800 (PST) Subject: Assertions, challenges, and polite discourse References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> <87hbqywzxt.fsf_-_@benfinney.id.au> Message-ID: <487fa8b4-5e27-4e61-99a3-4c57370154e6@c3g2000yqd.googlegroups.com> r0g wrote: > Well I think sometimes, for the sake of expediency and overall > pleasantness, it's better to let the smaller things go: and if you just > can't let them go then at least try and issue corrections in a friendly > manner rather than a cold or pious one. The irony, it is too rich... From skwslide at gmail.com Wed Jan 6 22:05:40 2010 From: skwslide at gmail.com (Steven K. Wong) Date: Wed, 6 Jan 2010 19:05:40 -0800 (PST) Subject: subprocess.Popen does not close pipe in an error case References: <07641018-9581-44af-bb20-8f9b96a44c4f@p8g2000yqb.googlegroups.com> Message-ID: <40a2adab-941a-4844-9279-10667bf2ba5e@v25g2000yqk.googlegroups.com> Well, the example code at http://www.python.org/doc/2.6.2/library/subprocess.html#replacing-shell-pipeline has the same issue: output=`dmesg | grep hda` ==> p1 = Popen(["dmesg"], stdout=PIPE) p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE) output = p2.communicate()[0] After communicate() returns, if you wait for p1 to finish (by calling p1.poll() repeatedly or p1.wait()), you can hang if the conditions described in the original post are true, i.e. p1 wrote lots of data to the pipe and p2 failed without reading much data from the pipe. Perhaps the doc can be improved to remind folks to close p1.stdout if the calling process doesn't need it, unless wait() is changed to close it and p1.wait() is called. Am I making any sense here? From sjmachin at lexicon.net Wed Jan 6 22:28:49 2010 From: sjmachin at lexicon.net (John Machin) Date: Wed, 6 Jan 2010 19:28:49 -0800 (PST) Subject: TypeError References: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> <4c29f4d1-a4d2-4877-91fd-09b346f1a5f2@k19g2000yqc.googlegroups.com> Message-ID: On Jan 7, 1:38?pm, Steve Holden wrote: > John Machin wrote: > > [...]> I note that in the code shown there are examples of building an SQL > > query where the table name is concocted at runtime via the % > > operator ... key phrases: "bad database design" (one table per > > store!), "SQL injection attack" > > I'm not trying to defend the code overall, but most databases won't let > you parameterize the table or column names, just the data values. That's correct, and that's presumably why the OP is constructing whole SQL statements on the fly e.g. cursor.execute('select max(ID) from %sCustomerData;' % store) What is the reason for "but" in "but most databases won't ..."? What are you rebutting? Let me try again: One table per store is bad design. The implementation of that bad design may use: cursor.execute('select max(ID) from %sCustomerData;' % store) or (if available) cursor.execute('select max(ID) from ?CustomerData;', (store, )) but the implementation means is irrelevant. From jjposner at optimum.net Wed Jan 6 22:31:42 2010 From: jjposner at optimum.net (John Posner) Date: Wed, 06 Jan 2010 22:31:42 -0500 Subject: QDoubleValidator References: Message-ID: On Wed, 06 Jan 2010 17:47:37 -0500, Zabin wrote: > On Jan 7, 10:23 am, Zabin wrote: >> Hey! >> >> I am new PyQt programmer and want to restrict users to allow only >> numeric values into a table and lineedit boxes. I found the >> QDoubleValidator class but am unsure as to how to implement it. (I am >> a little shaky on the concept of parent and how to define them). Any >> help would be much appreciated! You'll probably get more help in these Qt-specific forums: http://lists.trolltech.com/mailman/listinfo/qt-interest http://www.riverbankcomputing.com/mailman/listinfo/pyqt Good luck, John From wolftracks at invalid.com Wed Jan 6 22:40:52 2010 From: wolftracks at invalid.com (W. eWatson) Date: Wed, 06 Jan 2010 19:40:52 -0800 Subject: Astronomy--Programs to Compute Siderial Time? In-Reply-To: References: Message-ID: John Machin wrote: > On Jan 7, 11:40 am, "W. eWatson" wrote: >> W. eWatson wrote: >>> Is there a smallish Python library of basic astronomical functions? >>> There are a number of large such libraries that are crammed with >>> excessive functions not needed for common calculations. >> It looks like I've entered a new era in my knowledge of Python. > > Mild curiosity: this would be a wonderful outcome, but what makes it > look so? I actually need to learn how to make a module that can be imported, which in the short interlude I have done. Also looked into docstrings and docs, which I now have a decent grasp of. Never really used either doc info or writing my own module before. Easy. > >> I found >> a module somewhat like I want, siderial.py. You can see an intro to it >> at . >> It appears that I can get the code for it through section 1.2, near the >> bottom. I scooped it siderial.py up, and placed it in a corresponding >> file of the same name and type via NotePad. However, there is a xml file >> below it. I know little about it. I thought maybe I could do the same, >> but Notepad didn't like some characters in it. As I understand Python >> doc files are useful. So how do I get this done, and where do I put the >> files? > > The file you need is sidereal.py, not your twice-mentioned siderial.py > (the existence of which on the referenced website is doubtful). How right you are. I misspelled it twice, and quickly found that out when I tried to use the [side][real] (easy mnemonic, two words) module. sidereal. > > What you have been reading is the "Internal maintenance > specification" (large font, near the top of the page) for the module. > The xml file is the source of the docs, not meant to be user-legible. What is it used for? Do I need it? > A very tiny amount of googling "sidereal.py" (quotes included) leads > to the user documentation at http://infohost.nmt.edu/tcc/help/lang/python/examples/sidereal/ Found that too as I cruised around. > > Where do you put the files? Well, we're now down to only one file, > sidereal.py, and you put it wherever you'd put any other module that > you'd like to call ... if there's only going to be one caller, put it > in the same directory as that caller's code. More generally, drop it > in /Lib/site-packages Again in my "learning about modules", discovered that too. I think I'm on my way. Thanks. From steve at holdenweb.com Wed Jan 6 22:51:38 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 22:51:38 -0500 Subject: TypeError In-Reply-To: References: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> <4c29f4d1-a4d2-4877-91fd-09b346f1a5f2@k19g2000yqc.googlegroups.com> Message-ID: <4B455A4A.2090702@holdenweb.com> John Machin wrote: > On Jan 7, 1:38 pm, Steve Holden wrote: >> John Machin wrote: >> >> [...]> I note that in the code shown there are examples of building an SQL >>> query where the table name is concocted at runtime via the % >>> operator ... key phrases: "bad database design" (one table per >>> store!), "SQL injection attack" >> I'm not trying to defend the code overall, but most databases won't let >> you parameterize the table or column names, just the data values. > > That's correct, and that's presumably why the OP is constructing whole > SQL statements on the fly e.g. > > cursor.execute('select max(ID) from %sCustomerData;' % store) > Well yes, but that is just a symptom of the real disease, which is that he has very little idea what he is doing. > What is the reason for "but" in "but most databases won't ..."? What > are you rebutting? > I was simply pointing out that the OP had chosen the only available way of generating variable table names. The bad database design has been discussed,if not ad infinitum then certainly ad nauseam. All advice has been ignored. > Let me try again: One table per store is bad design. The > implementation of that bad design may use: > > cursor.execute('select max(ID) from %sCustomerData;' % store) > or (if available) > cursor.execute('select max(ID) from ?CustomerData;', (store, )) > but the implementation means is irrelevant. [Do you know any database on which the latter technique will work? I realise I said "most", but I suspect I should have said "all" - at least I can't think of a counterexample now I have put myself on the spot]. I pointed this out to the OP some time ago. It won't make any difference. In the particular code you mention the tables didn't seem to be constructed from user input, thereby removing much of the danger of SQL injection exploits, but I could be wrong - reading it made me feel squeamish so I didn't analyze it thoroughly. The whole thing is a hodge-podge of Python and HTML produced by an individual who appears to feel it isn't necessary to understand either HTTP or HTML in order to produce dynamic web sites, whose grasp of Python itself is slight and whose response to constructive criticism is to defer acting on it until it is no longer going to be helpful. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From philip at semanchuk.com Wed Jan 6 22:53:01 2010 From: philip at semanchuk.com (Philip Semanchuk) Date: Wed, 6 Jan 2010 22:53:01 -0500 Subject: GUI for multiplatform multimedia project In-Reply-To: References: Message-ID: On Jan 6, 2010, at 4:53 PM, wrote: > Hi everyone, > > I posted that question on a python-forum, but got answer, so I ask > here. > > I'm working on an artistic project and I'm looking for the best > cross-platform GUI solution. The problem is that it's gonna be a > tool that > will have to be double-click installable/runnable and pre- > installation of > any libraries for end-users is very much like an evil. It really has > to be > double-click tool > > My first thought was PyQt, because it's a real framework with a lot of > stuff inside (including Phonon) and I know some cross-platform media > software written in C++ QT (like VLC). But on the other hand I've > heard > that it's not that easy to make it "double-clicky" multi-platform. > Is that > true? > > Another thing that matters for me is ease of integration with > libraries > like OpenCV. > > I will be VERY thankful for any help. I'm SO tired googling the > problem > (it's like weeks now!!) Cze?? trzewiczek, I'm not well qualified to answer your question but since no one else has I'll make a stab at it. I'm interested because I'll have to solve the same problem in some months time with a suite of wxPython-based apps that rely on a lot of libraries. When you say "cross platform" I assume you mean OS X, Linux and Windows. All three of those require a different technique to create a double-clickable app, so in a way you have three problems to solve, not just one. For instance, py2exe is a popular solution for building monolithic application blobs for Windows, while py2app does the same for the Mac. It might be true that PyQt isn't all that easy to cram into a "double- clicky" application blob. But I don't know that e.g. wxPython will be any easier. Bundling a Python GUI app is non-trivial. I think that's exactly why you've been able to spend weeks googling -- there's no easy, obvious, this-is-it, canonical solution. Different people use different techniques. So to get back to your original question, although I don't know if one GUI toolkit is easier to bundle than another, I suspect that none of them are easy. I would pick the GUI toolkit based on what suits your technical & licensing needs best and worry about distribution later. That's one man's opinion. I hope someone else with more practical experience in the matter can offer you some advice. Good luck Philip From aioe.org at technicalbloke.com Wed Jan 6 23:25:30 2010 From: aioe.org at technicalbloke.com (r0g) Date: Thu, 07 Jan 2010 04:25:30 +0000 Subject: GUI for multiplatform multimedia project References: Message-ID: Philip Semanchuk wrote: > > On Jan 6, 2010, at 4:53 PM, > wrote: > >> Hi everyone, >> >> I posted that question on a python-forum, but got answer, so I ask here. >> >> I'm working on an artistic project and I'm looking for the best >> cross-platform GUI solution. The problem is that it's gonna be a tool >> that >> will have to be double-click installable/runnable and pre-installation of >> any libraries for end-users is very much like an evil. It really has >> to be >> double-click tool >> >> My first thought was PyQt, because it's a real framework with a lot of >> stuff inside (including Phonon) and I know some cross-platform media >> software written in C++ QT (like VLC). But on the other hand I've heard >> that it's not that easy to make it "double-clicky" multi-platform. Is >> that >> true? >> >> Another thing that matters for me is ease of integration with libraries >> like OpenCV. >> >> I will be VERY thankful for any help. I'm SO tired googling the problem >> (it's like weeks now!!) > > Cze?? trzewiczek, > I'm not well qualified to answer your question but since no one else has > I'll make a stab at it. I'm interested because I'll have to solve the > same problem in some months time with a suite of wxPython-based apps > that rely on a lot of libraries. > Actually the one I wouldn't recommend for the OP is wxPython. It's very good and I use it almost exclusively but as the OP has an "artistic project" I'm not sure it would be ideal. wxPython works by leveraging the runtime platform's native toolkit so your apps are very portable and fit in nicely with the look and feel of whatever system they're on but... Sometimes that can make it hard to do very precise layouts that look the same across platforms. Sometimes it doesn't manifest and the times where it does wont be a problem for the vast majority of apps but if you require very granular control you may be better off picking one toolkit and sticking to it. I'd pick QT over GTK if I had to make a choice and also I agree with the last poster that packaging any of them (for windows at least) will require a fair degree of faff. Roger. From aioe.org at technicalbloke.com Wed Jan 6 23:38:41 2010 From: aioe.org at technicalbloke.com (r0g) Date: Thu, 07 Jan 2010 04:38:41 +0000 Subject: Assertions, challenges, and polite discourse References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> <87hbqywzxt.fsf_-_@benfinney.id.au> <487fa8b4-5e27-4e61-99a3-4c57370154e6@c3g2000yqd.googlegroups.com> Message-ID: alex23 wrote: > r0g wrote: >> Well I think sometimes, for the sake of expediency and overall >> pleasantness, it's better to let the smaller things go: and if you just >> can't let them go then at least try and issue corrections in a friendly >> manner rather than a cold or pious one. > > The irony, it is too rich... Hi Alex, I didn't think anyone else would still be reading this thread! The above was in the context of people who have done nothing to warrant coldness and piety. Sadly we seem to have gotten way beyond that. You are right though, in retrospect I should have let this slip, clearly no minds have been changed by not doing :/ Roger. From ashish.vyas at motorola.com Wed Jan 6 23:48:58 2010 From: ashish.vyas at motorola.com (VYAS ASHISH M-NTB837) Date: Thu, 7 Jan 2010 12:48:58 +0800 Subject: lxml 2.2.4 on python3.1, Windows XP gives importerror In-Reply-To: <4B44BE0A.7060508@activestate.com> Message-ID: <7C57DB58C81FB64CA979EC6C4DB73E7A03E70654@ZMY16EXM67.ds.mot.com> Processor is Intel Pentium 32 bit. import platform print (platform.architecture()) gives -> ('32bit', 'WindowsPE') Regards, Ashish Vyas -----Original Message----- From: Sridhar Ratnakumar [mailto:sridharr at activestate.com] Sent: Wednesday, January 06, 2010 10:15 PM To: VYAS ASHISH M-NTB837 Cc: python-list at python.org Subject: Re: lxml 2.2.4 on python3.1, Windows XP gives importerror On 1/5/2010 1:38 AM, VYAS ASHISH M-NTB837 wrote: > Dear All > I have Python 3.1 installed on Windows XP and Works nice. > I downloaded lxml 2.2.4 (lxml-2.2.4.win32-py3.1.exe) from pypi. > When I try: > from lxml import etree > I get: > ImportError: DLL load failed: This application has failed to start > because the application configuration is incorrect. Reinstalling the > application may fix this problem. Are you by any chance running 64-bit Python (whereas the lxml binary you have installed is 32-bit)? C:> python -c "import platform; print platform.architecture()" -srid From jkordani at intlogsys.com Thu Jan 7 00:07:31 2010 From: jkordani at intlogsys.com (Joshua Kordani) Date: Thu, 07 Jan 2010 00:07:31 -0500 Subject: from datetime.datetime import today not working. python2.6.4 on windows Message-ID: <4B456C13.7060005@intlogsys.com> Greetings all! So I'm reading through the manual and I get to the point where it talks about packages and how to import them. namely section 6.4 in the tutorial. I wont repeat the section here, but I want to understand whats going on in the following (as typed on my computer). Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import datetime >>> dir(datetime) ['MAXYEAR', 'MINYEAR', '__doc__', '__name__', '__package__', 'date', 'datetime', 'datetime_CAPI', 'time', 'timedelta', 'tzinfo'] >>> dir(datetime.datetime) ['__add__', '__class__', '__delattr__', '__doc__', '__eq__', '__format__', '__ge __', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', ' __ne__', '__new__', '__radd__', '__reduce__', '__reduce_ex__', '__repr__', '__rs ub__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', 'a stimezone', 'combine', 'ctime', 'date', 'day', 'dst', 'fromordinal', 'fromtimest amp', 'hour', 'isocalendar', 'isoformat', 'isoweekday', 'max', 'microsecond', 'm in', 'minute', 'month', 'now', 'replace', 'resolution', 'second', 'strftime', 's trptime', 'time', 'timetuple', 'timetz', 'today', 'toordinal', 'tzinfo', 'tzname ', 'utcfromtimestamp', 'utcnow', 'utcoffset', 'utctimetuple', 'weekday', 'year'] >>> from datetime.datetime import today Traceback (most recent call last): File "", line 1, in ImportError: No module named datetime >>> so dir on datetime shows symbols date, time, datetime,etc dir on datetime shows today, now, etc lets say for arguments sake that I want to just import the today function, according to the documentation, the line should be: from datetime.datetime import today. as you can see, that didn't work. why not? Josh From benjamin.kaplan at case.edu Thu Jan 7 00:39:50 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Thu, 7 Jan 2010 00:39:50 -0500 Subject: from datetime.datetime import today not working. python2.6.4 on windows In-Reply-To: <4B456C13.7060005@intlogsys.com> References: <4B456C13.7060005@intlogsys.com> Message-ID: On Thu, Jan 7, 2010 at 12:07 AM, Joshua Kordani wrote: > Greetings all! > > So I'm reading through the manual and I get to the point where it talks > about packages and how to import them. ?namely section 6.4 in the tutorial. > ?I wont repeat the section here, but I want to understand whats going on in > the following (as typed on my computer). > > Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)] > on > win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> import datetime >>>> dir(datetime) > ['MAXYEAR', 'MINYEAR', '__doc__', '__name__', '__package__', 'date', > 'datetime', > ?'datetime_CAPI', 'time', 'timedelta', 'tzinfo'] >>>> dir(datetime.datetime) > ['__add__', '__class__', '__delattr__', '__doc__', '__eq__', '__format__', > '__ge > __', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', > '__lt__', ' > __ne__', '__new__', '__radd__', '__reduce__', '__reduce_ex__', '__repr__', > '__rs > ub__', '__setattr__', '__sizeof__', '__str__', '__sub__', > '__subclasshook__', 'a > stimezone', 'combine', 'ctime', 'date', 'day', 'dst', 'fromordinal', > 'fromtimest > amp', 'hour', 'isocalendar', 'isoformat', 'isoweekday', 'max', > 'microsecond', 'm > in', 'minute', 'month', 'now', 'replace', 'resolution', 'second', > 'strftime', 's > trptime', 'time', 'timetuple', 'timetz', 'today', 'toordinal', 'tzinfo', > 'tzname > ', 'utcfromtimestamp', 'utcnow', 'utcoffset', 'utctimetuple', 'weekday', > 'year'] > >>>> from datetime.datetime import today > Traceback (most recent call last): > ?File "", line 1, in > ImportError: No module named datetime >>>> > > so dir on datetime shows symbols date, time, datetime,etc > dir on datetime shows today, now, etc > > lets say for arguments sake that I want to just import the today function, > according to the documentation, the line should be: > from datetime.datetime import today. > > as you can see, that didn't work. ?why not? > >>> import datetime >>> type(datetime.datetime) datetime.datetime is a type (so it's a class), not a module and today is an attribute of that type. You can't import from a class. From steve at holdenweb.com Thu Jan 7 01:12:21 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 07 Jan 2010 01:12:21 -0500 Subject: from datetime.datetime import today not working. python2.6.4 on windows In-Reply-To: <4B456C13.7060005@intlogsys.com> References: <4B456C13.7060005@intlogsys.com> Message-ID: Joshua Kordani wrote: > Greetings all! > > So I'm reading through the manual and I get to the point where it talks > about packages and how to import them. namely section 6.4 in the > tutorial. I wont repeat the section here, but I want to understand > whats going on in the following (as typed on my computer). > > Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit > (Intel)] on > win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> import datetime >>>> dir(datetime) > ['MAXYEAR', 'MINYEAR', '__doc__', '__name__', '__package__', 'date', > 'datetime', > 'datetime_CAPI', 'time', 'timedelta', 'tzinfo'] >>>> dir(datetime.datetime) > ['__add__', '__class__', '__delattr__', '__doc__', '__eq__', > '__format__', '__ge > __', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', > '__lt__', ' > __ne__', '__new__', '__radd__', '__reduce__', '__reduce_ex__', > '__repr__', '__rs > ub__', '__setattr__', '__sizeof__', '__str__', '__sub__', > '__subclasshook__', 'a > stimezone', 'combine', 'ctime', 'date', 'day', 'dst', 'fromordinal', > 'fromtimest > amp', 'hour', 'isocalendar', 'isoformat', 'isoweekday', 'max', > 'microsecond', 'm > in', 'minute', 'month', 'now', 'replace', 'resolution', 'second', > 'strftime', 's > trptime', 'time', 'timetuple', 'timetz', 'today', 'toordinal', 'tzinfo', > 'tzname > ', 'utcfromtimestamp', 'utcnow', 'utcoffset', 'utctimetuple', 'weekday', > 'year'] > >>>> from datetime.datetime import today > Traceback (most recent call last): > File "", line 1, in > ImportError: No module named datetime >>>> > > so dir on datetime shows symbols date, time, datetime,etc > dir on datetime shows today, now, etc > > lets say for arguments sake that I want to just import the today > function, according to the documentation, the line should be: > from datetime.datetime import today. > > as you can see, that didn't work. why not? > Because datetime is a module, but datetime.datetime is a class. You can import individual names from a module, but a class is a monolithic all-or-nothing chunk. There's the potential for confusion because Python also has "packages", which are like modules but more structured: you can import a module from a package: >>> from xml import etree or import the package module directly: >>> import xml.etree) You can also import a submodule from the module: >>> from xml.etree import cElementTree or import the submodule directly >>> import xml.etree.cElementTree You can even import a class from the submodule: >>> from xml.etree.cElementTree import Element but what you *can't* do is import the submodule's class directly: >>> import xml.etree.cElementTree.Element Traceback (most recent call last): File "", line 1, in ImportError: No module named Element However, what you did was import the datetime module (which is not a package, and does not therefore contains submodules), and reference the datetime class within that module (datetime.datetime), which is all OK. But datetime.datetime is a class, not a module, so you can't import anything from it. Neither can you import it directly: >>> import datetime.datetime Traceback (most recent call last): File "", line 1, in ImportError: No module named datetime (in the last line, the name "datetime" refers to the class, not themodule. Hope this helps. You may learn a bit more by actually looking at the source of the module, which probably lives in C:\Python26\Lib\datetime.py regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From gherron at islandtraining.com Thu Jan 7 01:21:56 2010 From: gherron at islandtraining.com (Gary Herron) Date: Wed, 06 Jan 2010 22:21:56 -0800 Subject: from datetime.datetime import today not working. python2.6.4 on windows In-Reply-To: <4B456C13.7060005@intlogsys.com> References: <4B456C13.7060005@intlogsys.com> Message-ID: <4B457D84.4050207@islandtraining.com> Joshua Kordani wrote: > Greetings all! > > So I'm reading through the manual and I get to the point where it > talks about packages and how to import them. namely section 6.4 in > the tutorial. I wont repeat the section here, but I want to > understand whats going on in the following (as typed on my computer). > > Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit > (Intel)] on > win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> import datetime > >>> dir(datetime) > ['MAXYEAR', 'MINYEAR', '__doc__', '__name__', '__package__', 'date', > 'datetime', > 'datetime_CAPI', 'time', 'timedelta', 'tzinfo'] > >>> dir(datetime.datetime) > ['__add__', '__class__', '__delattr__', '__doc__', '__eq__', > '__format__', '__ge > __', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', > '__lt__', ' > __ne__', '__new__', '__radd__', '__reduce__', '__reduce_ex__', > '__repr__', '__rs > ub__', '__setattr__', '__sizeof__', '__str__', '__sub__', > '__subclasshook__', 'a > stimezone', 'combine', 'ctime', 'date', 'day', 'dst', 'fromordinal', > 'fromtimest > amp', 'hour', 'isocalendar', 'isoformat', 'isoweekday', 'max', > 'microsecond', 'm > in', 'minute', 'month', 'now', 'replace', 'resolution', 'second', > 'strftime', 's > trptime', 'time', 'timetuple', 'timetz', 'today', 'toordinal', > 'tzinfo', 'tzname > ', 'utcfromtimestamp', 'utcnow', 'utcoffset', 'utctimetuple', > 'weekday', 'year'] > > >>> from datetime.datetime import today > Traceback (most recent call last): > File "", line 1, in > ImportError: No module named datetime > >>> > > so dir on datetime shows symbols date, time, datetime,etc > dir on datetime shows today, now, etc > > lets say for arguments sake that I want to just import the today > function, according to the documentation, the line should be: > from datetime.datetime import today. > > as you can see, that didn't work. why not? Just a little confusion on your part. datetime is a module, so you can import it (as you did) and objects from it datetime.datetime is a type (like a class is a type) not a module so you cannot import from it. So you can import datetime and you can from datetime import but that's all the further you can go with imports. Some of the confusion may be from having a class (well actually it's a type) with the same name as its module. This is now considered bad form. You might achieve the effect you were you were trying for like this: import datetime today = datetime.datetime.today or from datetime import datetime today = datetime.today Then you'll be able to call today() at any time. Gary Herron > > Josh From metolone+gmane at gmail.com Thu Jan 7 01:25:40 2010 From: metolone+gmane at gmail.com (Mark Tolonen) Date: Wed, 6 Jan 2010 22:25:40 -0800 Subject: Pass multidimensional array (matrix) to c function using ctypes References: <63ac1a01-8491-4885-bae7-cb884abb582f@34g2000yqp.googlegroups.com> Message-ID: "Daniel Platz" wrote in message news:63ac1a01-8491-4885-bae7-cb884abb582f at 34g2000yqp.googlegroups.com... > Hello, > > I would like to pass a two dimensional array to C function in a dll. I > use ctypes to call the function. > > I compile the dll with visual studio 2008 express and my C source code > looks like this. > > #include > #ifdef __cplusplus > extern "C" { // only need to export C interface if > // used by C++ source code > using namespace std; > #endif > > > __declspec(dllexport) int print(double** ptr, int ny, int nx) > { > int i, j; > for(i=0; i { > for(j=0; j { > printf("%.3f \t", *(*(ptr+i)+j)); > } > printf("\n"); > } > return 0; > } > > #ifdef __cplusplus > } > #endif > > As you can see the function expects a doube** variable. I tried to > call this function from Python with this code. > > import ctypes as ct > > matrix = ct.cdll.LoadLibrary('matrix.dll') > getattr(matrix, 'print_matrix') > > ptr_type = ct.c_double * 5 * 4 > ptr = ptr_type() > matrix.print_matrix(ptr, ct.c_int(4), ct.c_int(5)) > > I expected to see a matrix showing only zeros since I can address the > single entries in Python by ptr[i][j]. Instead the interpreter returns > with the error message > > WindowsError: exception: access violation reading 0x000000 > WARNING: Failure executing file: > > Furthermore, I am wondering if there is a fast way to use a numpy 2D > array instead or alternatively to cast the ctypes array into a numpy > array. > > Has someone an idea to help me? A two-dimentional array is not equivalent to a double**, so depending on what your requirements are there are two solutions. 1. To work with your original matrix.c code above, some Python code that works is: ------------------------------------- import ctypes as ct # I like C types in caps DOUBLE = ct.c_double PDOUBLE = ct.POINTER(DOUBLE) PPDOUBLE = ct.POINTER(PDOUBLE) INT = ct.c_int matrix = ct.cdll.LoadLibrary('matrix.dll') print_matrix = getattr(matrix,'print') # An array of doubles can be passed to a function that takes double*. DBL5ARR = DOUBLE * 5 # An array of double* can be passed to your function as double**. PDBL4ARR = PDOUBLE * 4 # Declare double* array. ptr = PDBL4ARR() for i in range(4): # fill out each pointer with an array of doubles. ptr[i] = DBL5ARR() for j in range(5): ptr[i][j] = i + j # just to initialize the actual doubles. print_matrix(ptr,4,5) ----------------------------------------------- 2. In C, multidimensional arrays are really just single dimentional arrays where the compiler does the math for you, so if you declare your matrix print function like this: ----- matrix2.c --------------------------- #include #ifdef __cplusplus extern "C" { // only need to export C interface if // used by C++ source code #endif __declspec(dllexport) int print(double* ptr, int ny, int nx) { int i, j; for(i=0; i <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: r0g wrote: > Steven D'Aprano wrote: >> On Wed, 06 Jan 2010 23:58:21 +0000, r0g wrote: >> >>> Steven D'Aprano wrote: >>>> On Wed, 06 Jan 2010 22:46:33 +0000, r0g wrote: >>>> >>>>> Grant Edwards wrote: >>>>>> On 2010-01-06, r0g wrote: >>> See? Spoiling for an argument even now! I never said you weren't allowed >>> to butt in, just that you did. Butting in is fine, that's half the point > detect no background seething at all here. > > Roger. > Actually guys I've just reread this thread from the start and I clearly did overreact, sorry about that. In fact it reminded me of this comic... http://xkcd.com/481/ Time to smoke a joint and get a half decent nights sleep I think! Roger. From QBX634 at motorola.com Thu Jan 7 02:16:32 2010 From: QBX634 at motorola.com (Mishra Gopal-QBX634) Date: Thu, 7 Jan 2010 15:16:32 +0800 Subject: How to reduce the memory size of python Message-ID: <6AFAAA2E71101642861E945A057CE7CD023A882B@ZMY16EXM70.ds.mot.com> Hi, When i write following pyhon program and execute it in linux machine, if __name__=='__main__': while True: pass When i check the VmRSS size, it shows 2956 KB in size. Is there any way to reduce the memory size taken by python. I am working in flash memory devices. Any suggession is highly helpfull to me. Thanks, Gopal -------------- next part -------------- An HTML attachment was scrubbed... URL: From trzewiczek at trzewiczek.info Thu Jan 7 02:19:45 2010 From: trzewiczek at trzewiczek.info (t r z e w i c z e k) Date: Thu, 07 Jan 2010 08:19:45 +0100 Subject: Validating cells of a table PyQt In-Reply-To: <2d6c6f77-30cc-4787-919a-f2781026dd66@a21g2000yqc.googlegroups.com> References: <2d6c6f77-30cc-4787-919a-f2781026dd66@a21g2000yqc.googlegroups.com> Message-ID: On Thu, 07 Jan 2010 01:37:05 +0100, Zabin wrote: > Hey! > > I am new PyQT programmer. I am trying to create a table in which cells > only take in numeric data. I am unable to use setValidator on table > cells. Looking around i found some material on the editor attribute > but I dont know how to apply this. Any help will be appreciated > > Cheers! Hi, I'm not an expert but could you write, why you can't use validators on the cells? There is this free ebook of first edition of "C++ GUI Programming with Qt 4" that has an excel like application as an example and I remember there was a part with validators that looked pretty nice. You can easly and legally download this book from various sites. Best from Poland, trzewiczek -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From trzewiczek at trzewiczek.info Thu Jan 7 02:22:53 2010 From: trzewiczek at trzewiczek.info (t r z e w i c z e k) Date: Thu, 07 Jan 2010 08:22:53 +0100 Subject: Python interactive terminal in Ubuntu Linux : some keys fouled up In-Reply-To: References: Message-ID: On Thu, 07 Jan 2010 01:09:08 +0100, casevh wrote: > On Jan 6, 2:40 pm, pdlem... at earthlink.net wrote: >> Have recently moved from XP to Ubuntu Linux. >> Successfully installed Python 3.1.1 in the Ubuntu 9.04 >> release on my desktop. >> Problem is the python interactive terminal >>> . >> Many of the keys now do not work. >> eg pressing left-arrow yields ^[[D >> right-arrow ^[[C >> Home ^[OH >> Del ^[[3~ >> up-arrow ^[[A >> Frustrating as I use all these , esp up-arrow to >> repeat recent lines. Found the same thing on >> my sons MacBook. >> >> This is not mentioned in two recent Python books >> or one on Ubuntu. Nor could I found help on the www. >> >> Is there any work-around ? Should I just forget >> the python prompt >>> ? >> Thanks, Dave pdlem... at earthlink.net > > Assuming you compiled the source code, you will also need to install > "libreadline5-dev" via Synaptic or apt-get. > > casevh Or you can use Idle instead of terminal. It works kind of different. sudo apt-get install idle-python3.1 will do the thing! cheers, trzewiczek -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From steve at holdenweb.com Thu Jan 7 02:29:50 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 07 Jan 2010 02:29:50 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: <4B458D6E.5020204@holdenweb.com> r0g wrote: > r0g wrote: >> Steven D'Aprano wrote: >>> On Wed, 06 Jan 2010 23:58:21 +0000, r0g wrote: >>> >>>> Steven D'Aprano wrote: >>>>> On Wed, 06 Jan 2010 22:46:33 +0000, r0g wrote: >>>>> >>>>>> Grant Edwards wrote: >>>>>>> On 2010-01-06, r0g wrote: >>>> See? Spoiling for an argument even now! I never said you weren't allowed >>>> to butt in, just that you did. Butting in is fine, that's half the point > >> detect no background seething at all here. >> >> Roger. >> > > > Actually guys I've just reread this thread from the start and I clearly > did overreact, sorry about that. In fact it reminded me of this comic... > > http://xkcd.com/481/ > > Time to smoke a joint and get a half decent nights sleep I think! > > Roger. Brilliant. It takes a real whole human being to make an admission like that (or even to bother to question their own behavior sufficiently to bother re-reading the thread). I think a lot more of you for the admission. Not that you really need care one way or the other what *I* think ... We all have off-days. About three years ago I went completely apeshit at someone in a fit of tiredness-induced pique, as was at one point amply evidenced by http://mail.python.org/pipermail/python-list/2007-August/thread.html#454510 Strangely the universe conspired to stamp on that thread, and it appears to have been lost to the python.org archives. A Google search for "Steve Holden" and "not a fucking computer", however, reveals that my shame has not been totally expunged. [If only I could rickroll those links ... :)] regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Thu Jan 7 02:49:39 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 07 Jan 2010 02:49:39 -0500 Subject: How to reduce the memory size of python In-Reply-To: <6AFAAA2E71101642861E945A057CE7CD023A882B@ZMY16EXM70.ds.mot.com> References: <6AFAAA2E71101642861E945A057CE7CD023A882B@ZMY16EXM70.ds.mot.com> Message-ID: Mishra Gopal-QBX634 wrote: > Hi, > > When i write following pyhon program and execute it in linux machine, > > if __name__=='__main__': > while True: > pass > > When i check the VmRSS size, it shows 2956 KB in size. > > Is there any way to reduce the memory size taken by python. > > I am working in flash memory devices. > > Any suggession is highly helpfull to me. > It would not be easy to reduce the size of the standard interpreter, but there are various implementations for embedded devices. You may get some help from http://wiki.python.org/moin/Tiny%20Python and http://wiki.python.org/moin/Tiny%20Python A company called Synapse has a working cut-down Python implementation that they embed in their 802.15 wireless mesh devices, which IIRC occupies less than 128K. There is a lot in the standard interpreter that you won't need - many embedded systems don't need floating point arithmetic, for example. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From jkordani at intlogsys.com Thu Jan 7 02:51:07 2010 From: jkordani at intlogsys.com (Joshua Kordani) Date: Thu, 07 Jan 2010 02:51:07 -0500 Subject: from datetime.datetime import today not working. python2.6.4 on windows In-Reply-To: <4B457D84.4050207@islandtraining.com> References: <4B456C13.7060005@intlogsys.com> <4B457D84.4050207@islandtraining.com> Message-ID: <4B45926B.1040409@intlogsys.com> Gary Herron wrote: > Joshua Kordani wrote: >> Greetings all! >> >> So I'm reading through the manual and I get to the point where it >> talks about packages and how to import them. namely section 6.4 in >> the tutorial. I wont repeat the section here, but I want to >> understand whats going on in the following (as typed on my computer). >> >> Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit >> (Intel)] on >> win32 >> Type "help", "copyright", "credits" or "license" for more information. >> >>> import datetime >> >>> dir(datetime) >> ['MAXYEAR', 'MINYEAR', '__doc__', '__name__', '__package__', 'date', >> 'datetime', >> 'datetime_CAPI', 'time', 'timedelta', 'tzinfo'] >> >>> dir(datetime.datetime) >> ['__add__', '__class__', '__delattr__', '__doc__', '__eq__', >> '__format__', '__ge >> __', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', >> '__lt__', ' >> __ne__', '__new__', '__radd__', '__reduce__', '__reduce_ex__', >> '__repr__', '__rs >> ub__', '__setattr__', '__sizeof__', '__str__', '__sub__', >> '__subclasshook__', 'a >> stimezone', 'combine', 'ctime', 'date', 'day', 'dst', 'fromordinal', >> 'fromtimest >> amp', 'hour', 'isocalendar', 'isoformat', 'isoweekday', 'max', >> 'microsecond', 'm >> in', 'minute', 'month', 'now', 'replace', 'resolution', 'second', >> 'strftime', 's >> trptime', 'time', 'timetuple', 'timetz', 'today', 'toordinal', >> 'tzinfo', 'tzname >> ', 'utcfromtimestamp', 'utcnow', 'utcoffset', 'utctimetuple', >> 'weekday', 'year'] >> >> >>> from datetime.datetime import today >> Traceback (most recent call last): >> File "", line 1, in >> ImportError: No module named datetime >> >>> >> >> so dir on datetime shows symbols date, time, datetime,etc >> dir on datetime shows today, now, etc >> >> lets say for arguments sake that I want to just import the today >> function, according to the documentation, the line should be: >> from datetime.datetime import today. >> >> as you can see, that didn't work. why not? > > Just a little confusion on your part. > > datetime is a module, so you can import it (as you did) and objects from it > > datetime.datetime is a type (like a class is a type) not a module so you > cannot import from it. > > > So you can > import datetime > and you can > from datetime import > but that's all the further you can go with imports. > > Some of the confusion may be from having a class (well actually it's a > type) with the same name as its module. This is now considered bad form. > > You might achieve the effect you were you were trying for like this: > import datetime > today = datetime.datetime.today > or > from datetime import datetime > today = datetime.today > Then you'll be able to call > today() > at any time. > > Gary Herron > > >> >> Josh > @ All thanks for the responses! That was the key bit I was missing. Josh From marco.salden at gmail.com Thu Jan 7 03:00:05 2010 From: marco.salden at gmail.com (Marco Salden) Date: Thu, 7 Jan 2010 00:00:05 -0800 (PST) Subject: Do I have to use threads? References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> Message-ID: <432961e5-627e-452d-93db-9562951551d2@e27g2000yqd.googlegroups.com> On Jan 6, 5:36?am, Philip Semanchuk wrote: > On Jan 5, 2010, at 11:26 PM, aditya shukla wrote: > > > Hello people, > > > I have 5 directories corresponding 5 ?different urls .I want to ? > > download > > images from those urls and place them in the respective ? > > directories.I have > > to extract the contents and download them simultaneously.I can ? > > extract the > > contents and do then one by one. My questions is for doing it ? > > simultaneously > > do I have to use threads? > > No. You could spawn 5 copies of wget (or curl or a Python program that ? > you've written). Whether or not that will perform better or be easier ? > to code, debug and maintain depends on the other aspects of your ? > program(s). > > bye > Philip Yep, the more easier and straightforward the approach, the better: threads are always (programmers')-error-prone by nature. But my question would be: does it REALLY need to be simultaneously: the CPU/OS only has more overhead doing this in parallel with processess. Measuring sequential processing and then trying to optimize (e.g. for user response or whatever) would be my prefered way to go. Less=More. regards, Marco From andrew.gillanders at uqconnect.edu.au Thu Jan 7 03:19:47 2010 From: andrew.gillanders at uqconnect.edu.au (Andrew Gillanders) Date: Thu, 7 Jan 2010 18:19:47 +1000 Subject: buffer interface problem Message-ID: <3C5B8086-5D3D-444B-B9D2-C9152EBF5DA3@uqconnect.edu.au> I have run into a problem running a Python script that is part of the TerraGear suite for building scenery for FlightGear. I am using Mac OS X 10.4, running Python (version 3.0.1) in a Unix terminal. The purpose of the script is to walk a directory tree, unzipping files, and passing the contents to an executable C programme. The problem occurs here: gzin = GzipFile(fname, 'rb') data = gzin.readline() min_x,min_y = map(atoi,data.split()[:2]) The input file, when uncompressed, is an ASCII file with a line with two numbers, then a line of four numbers, then many long lines of numbers. I can see what the last is trying to do: split the string into two words, convert them to integers, and assign them to min_x and min_y. At the third line, I get the message "expected an object with the buffer interface". Which object is it referring to? Have some functions been changed to pass buffer objects instead of strings? How can I fix the source code to make it run? Any help appreciated Andrew (Python newbie) From QBX634 at motorola.com Thu Jan 7 03:34:23 2010 From: QBX634 at motorola.com (Mishra Gopal-QBX634) Date: Thu, 7 Jan 2010 16:34:23 +0800 Subject: How to reduce the memory size of python In-Reply-To: References: <6AFAAA2E71101642861E945A057CE7CD023A882B@ZMY16EXM70.ds.mot.com> Message-ID: <6AFAAA2E71101642861E945A057CE7CD023A884D@ZMY16EXM70.ds.mot.com> Hi, I use twisted framework too to handle the xmlrpc request. It takes around 3-4MB of memory while importing itself. Is there any python coding standard I should follow to save the memory. Like import logging takes 1MB of memory. We only use on function getLogger by 'from logging import getLogger' But it still take the same 1 MB memory. Instead of loading whole logging module only load the getLogger function. I there any way to save the memory with taking care of small things in code.. Thanks, Gopal -----Original Message----- From: python-list-bounces+qbx634=motorola.com at python.org [mailto:python-list-bounces+qbx634=motorola.com at python.org] On Behalf Of Steve Holden Sent: Thursday, January 07, 2010 1:20 PM To: python-list at python.org Subject: Re: How to reduce the memory size of python Mishra Gopal-QBX634 wrote: > Hi, > > When i write following pyhon program and execute it in linux machine, > > if __name__=='__main__': > while True: > pass > > When i check the VmRSS size, it shows 2956 KB in size. > > Is there any way to reduce the memory size taken by python. > > I am working in flash memory devices. > > Any suggession is highly helpfull to me. > It would not be easy to reduce the size of the standard interpreter, but there are various implementations for embedded devices. You may get some help from http://wiki.python.org/moin/Tiny%20Python and http://wiki.python.org/moin/Tiny%20Python A company called Synapse has a working cut-down Python implementation that they embed in their 802.15 wireless mesh devices, which IIRC occupies less than 128K. There is a lot in the standard interpreter that you won't need - many embedded systems don't need floating point arithmetic, for example. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ -- http://mail.python.org/mailman/listinfo/python-list From rajat.dudeja at gmail.com Thu Jan 7 04:12:23 2010 From: rajat.dudeja at gmail.com (Rajat) Date: Thu, 7 Jan 2010 01:12:23 -0800 (PST) Subject: How to execute a script from another script and other script does not do busy wait. Message-ID: <76bff0b6-70cb-4960-9b4d-3af7a9a1e0e7@z41g2000yqz.googlegroups.com> I want to run a python script( aka script2) from another python script (aka script1). While script1 executes script2 it waits for script2 to complete and in doing so it also does some other useful work.(does not do a busy wait). My intention is to update a third party through script1 that script2 is going to take longer. Please suggest how should I go about implementing it. I'm currently executing it as: import main from script2 ret_code = main() return ret_code which surely is not going to achieve me what I intend. Thanks, Rajat. From clp2 at rebertia.com Thu Jan 7 04:13:50 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Thu, 7 Jan 2010 01:13:50 -0800 Subject: buffer interface problem In-Reply-To: <3C5B8086-5D3D-444B-B9D2-C9152EBF5DA3@uqconnect.edu.au> References: <3C5B8086-5D3D-444B-B9D2-C9152EBF5DA3@uqconnect.edu.au> Message-ID: <50697b2c1001070113i3a8ce2aay50dac69ef8a25a83@mail.gmail.com> On Thu, Jan 7, 2010 at 12:19 AM, Andrew Gillanders wrote: > I have run into a problem running a Python script that is part of the > TerraGear suite for building scenery for FlightGear. I am using Mac OS X > 10.4, running Python (version 3.0.1) in a Unix terminal. > > The purpose of the script is to walk a directory tree, unzipping files, and > passing the contents to an executable C program. The problem occurs here: > > ? ?gzin = GzipFile(fname, 'rb') > ? ?data = gzin.readline() > ? ?min_x,min_y = map(atoi,data.split()[:2]) > > The input file, when uncompressed, is an ASCII file with a line with two > numbers, then a line of four numbers, then many long lines of numbers. I can > see what the last is trying to do: split the string into two words, convert > them to integers, and assign them to min_x and min_y. > > At the third line, I get the message "expected an object with the buffer > interface". Which object is it referring to? The elements of the list produced by `data.split()[:2]`, which are either Unicode strings or bytestrings, neither of which are buffers. > Have some functions been > changed to pass buffer objects instead of strings? How can I fix the source > code to make it run? The error is being raised by the atoi() function (in the future, please post the full Traceback, not just the final error message). What module/library does your atoi() function come from (look for an `import` statement mentioning it)? The only functions by that name in the Python standard library both operate on strings, not buffers, and thus can't be the same one your code is using. In any case, replacing `atoi` with `int` in your code will likely solve the problem. The built-in int() function* can convert strings to integers. Cheers, Chris -- http://blog.rebertia.com *Not really a function, but close enough for newbie explanatory purposes. From ashish.vyas at motorola.com Thu Jan 7 04:21:20 2010 From: ashish.vyas at motorola.com (VYAS ASHISH M-NTB837) Date: Thu, 7 Jan 2010 17:21:20 +0800 Subject: How to execute a script from another script and other script does notdo busy wait. In-Reply-To: <76bff0b6-70cb-4960-9b4d-3af7a9a1e0e7@z41g2000yqz.googlegroups.com> Message-ID: <7C57DB58C81FB64CA979EC6C4DB73E7A03E7071F@ZMY16EXM67.ds.mot.com> Use threads Regards, Ashish Vyas -----Original Message----- From: python-list-bounces+ntb837=motorola.com at python.org [mailto:python-list-bounces+ntb837=motorola.com at python.org] On Behalf Of Rajat Sent: Thursday, January 07, 2010 2:42 PM To: python-list at python.org Subject: How to execute a script from another script and other script does notdo busy wait. I want to run a python script( aka script2) from another python script (aka script1). While script1 executes script2 it waits for script2 to complete and in doing so it also does some other useful work.(does not do a busy wait). My intention is to update a third party through script1 that script2 is going to take longer. From sjmachin at lexicon.net Thu Jan 7 04:43:59 2010 From: sjmachin at lexicon.net (John Machin) Date: Thu, 7 Jan 2010 01:43:59 -0800 (PST) Subject: Astronomy--Programs to Compute Siderial Time? References: Message-ID: On Jan 7, 2:40?pm, "W. eWatson" wrote: > John Machin wrote: > > > What you have been reading is the "Internal maintenance > > specification" (large font, near the top of the page) for the module. > > The xml file is the source of the docs, not meant to be user-legible. > > What is it used for? The maintainer of the module processes the xml file with some script or other to create the user-legible docs. > Do I need it? No. From eight32 at gmail.com Thu Jan 7 04:46:38 2010 From: eight32 at gmail.com (Stuart Murray-Smith) Date: Thu, 7 Jan 2010 11:46:38 +0200 Subject: Python books, literature etc In-Reply-To: <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: 2010/1/6 J : > A good point was brought up to me privately, and I agree completely, > that the OP should re-state the request with a bit more specifics... > > Since the OP says he is at least familiar with Python, does he need > info on beginner level books that are general purpose, or is he > interested in resources that are more specific (e.g. geared toward web > programming, mathematical analysis, data modeling, etc).... > > My suggestions were meant just as an example of what I use in the > course of learning something at the basic and intermediate level, once > it goes beyond that, it's useful to know WHAT you intend to do so you > can find the right resources to go in that direction. > > Just want to make sure I am not misunderstood or anything :-) > > For what it's worth, I also tend to collect technical books for some > reason... My wife is just barely tolerant of my bookshelf full of > things on various computer topics, astronomy, photography, radio and > antenna theory and so forth ;-) ?I just let her keep her shoe > collection, and we have a quid pro quo. Thanks J for your reply, much appreciated :) Oops, vague OP, my bad. Agreed, Google turns up myriad of topical books, and ESR's guide to smart questions [1] helps set the pace of list culture. I subscribe to various list servers, on one of them we gracefully accept that a question like my OP is looking for opinion on a matter, which I now understand would be a list's sub-culture? Anyways, to rephrase, could someone kindly mention any of their preferred Python books, websites, tutorials etc to help me get to an intermediate/advanced level? Something that would help me add functionality to Ubiquity, say. Have a great day! Stu@ [1] http://catb.org/~esr/faqs/smart-questions.html From gagsl-py2 at yahoo.com.ar Thu Jan 7 05:08:21 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Thu, 7 Jan 2010 02:08:21 -0800 (PST) Subject: Python books, literature etc References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: <304854.43355.qm@smtp139.mail.mud.yahoo.com> Stuart Murray-Smith wrote in news:aadebb9f1001070146n70f5be7bw2e515f9d4afed93e at mail.gmail.com: > Anyways, to rephrase, could someone kindly mention any of their > preferred Python books, websites, tutorials etc to help me get > to an intermediate/advanced level? Something that would help me > add functionality to Ubiquity, say. Have a look at the Getting Started section of the wiki: http://wiki.python.org/moin/ specially the PythonBooks section -- Gabriel Genellina From gagsl-py2 at yahoo.com.ar Thu Jan 7 05:08:21 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Thu, 7 Jan 2010 10:08:21 +0000 (UTC) Subject: Python books, literature etc References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: Stuart Murray-Smith wrote in news:aadebb9f1001070146n70f5be7bw2e515f9d4afed93e at mail.gmail.com: > Anyways, to rephrase, could someone kindly mention any of their > preferred Python books, websites, tutorials etc to help me get > to an intermediate/advanced level? Something that would help me > add functionality to Ubiquity, say. Have a look at the Getting Started section of the wiki: http://wiki.python.org/moin/ specially the PythonBooks section -- Gabriel Genellina From bhavik.patel at gmail.com Thu Jan 7 05:20:32 2010 From: bhavik.patel at gmail.com (Bhavik) Date: Thu, 7 Jan 2010 02:20:32 -0800 (PST) Subject: Call a DLL function with argument type as unsigned char * Message-ID: Hello, I am a newbie to the python language, and I need to call a DLL function from the python program. The DLL function has following prototype: unsigned int DLLFunction(unsigned char *, unsigned int); Now, I need to declare an array of 6 bytes in the python, and pass that array as the first argument to the DLL function. I tried some methods mentioned on the Internet, but it gives error like "can not convert argument 1". Can someone tell me how to do this? Thanks for the help. From bhavik.patel at gmail.com Thu Jan 7 05:28:33 2010 From: bhavik.patel at gmail.com (Bhavik) Date: Thu, 7 Jan 2010 02:28:33 -0800 (PST) Subject: Call a DLL function with argument type as unsigned char * References: Message-ID: Just to clarify, I am using Python 2.5.1 From gagsl-py2 at yahoo.com.ar Thu Jan 7 06:09:53 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Thu, 07 Jan 2010 08:09:53 -0300 Subject: creating tar file and streaming it over HTTP? References: <5222d923-7aef-47cf-a0d6-f37f09a66593@35g2000yqa.googlegroups.com> Message-ID: En Wed, 06 Jan 2010 13:39:08 -0300, pbienst escribi?: > The problem seems to be that the > receiving end (wsgi server) does not see the end of the data: > > socket = environ["wsgi.input"] > while True: > sys.stderr.write("before") > chunk = socket.read(4096) > sys.stderr.write("after") > if not chunk: > sys.stderr.write("done") > break > sys.stderr.write(chunk) > > There is data from the tar file being printed, but in the end it hangs > in the 'before' statement, and never gets to 'done'. I tried flushing > the fileobj created by makefile(), but that doesn't seem to help. > There is also no environ["CONTENT_LENGTH"] that I can use to detect > the end of the stream. > > I'm probably missing something basic here... Either send a "Connection: Close" header (and close the connection at the end), or a "Transfer-Encoding: chunked" header (see http://en.wikipedia.org/wiki/Chunked_transfer_encoding ) -- Gabriel Genellina From rajat.dudeja at gmail.com Thu Jan 7 06:19:19 2010 From: rajat.dudeja at gmail.com (Rajat) Date: Thu, 7 Jan 2010 03:19:19 -0800 (PST) Subject: How to execute a script from another script and other script does notdo busy wait. References: Message-ID: <016d2fad-a925-47f4-8b76-5602a8532651@r5g2000yqb.googlegroups.com> On Jan 7, 2:21?pm, "VYAS ASHISH M-NTB837" wrote: > Use threads > > Regards, > Ashish Vyas > > > > -----Original Message----- > From: python-list-bounces+ntb837=motorola.... at python.org > > [mailto:python-list-bounces+ntb837=motorola.... at python.org] On Behalf Of > Rajat > Sent: Thursday, January 07, 2010 2:42 PM > To: python-l... at python.org > Subject: How to execute a script from another script and other script > does notdo busy wait. > > I want to run a python script( aka script2) from another python script > (aka script1). While script1 executes script2 it waits for script2 to > complete and in doing so it also does some other useful work.(does not > do a busy wait). > > My intention is to update a third party through script1 that script2 is > going to take longer.- Hide quoted text - > > - Show quoted text - Thanks Ashish. I've single CPU machine. I've a feeling that the thread created, which would run script2, would eat up all of the CPU if I do not use sleep() in script2. That way, script1 would still be waiting for script2 to finish. Thus, my program is no way different from the sample program I posted earlier. Is there any other way out? From ashish.vyas at motorola.com Thu Jan 7 06:33:43 2010 From: ashish.vyas at motorola.com (VYAS ASHISH M-NTB837) Date: Thu, 7 Jan 2010 19:33:43 +0800 Subject: How to execute a script from another script and other script doesnotdo busy wait. In-Reply-To: <016d2fad-a925-47f4-8b76-5602a8532651@r5g2000yqb.googlegroups.com> Message-ID: <7C57DB58C81FB64CA979EC6C4DB73E7A03E70781@ZMY16EXM67.ds.mot.com> Did you try? Thanks Ashish. I've single CPU machine. I've a feeling that the thread created, which would run script2, would eat up all of the CPU if I do not use sleep() in script2. That way, script1 would still be waiting for script2 to finish. Thus, my program is no way different from the sample program I posted earlier. Is there any other way out? From ben+python at benfinney.id.au Thu Jan 7 06:38:58 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 07 Jan 2010 22:38:58 +1100 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: <87tyuyuotp.fsf@benfinney.id.au> Steve Holden writes: > Brilliant. It takes a real whole human being to make an admission like > that (or even to bother to question their own behavior sufficiently to > bother re-reading the thread). I think a lot more of you for the > admission. Seconded. -- \ ?bash awk grep perl sed, df du, du-du du-du, vi troff su fsck | `\ rm * halt LART LART LART!? ?The Swedish BOFH, | _o__) alt.sysadmin.recovery | Ben Finney From rschroev_nospam_ml at fastmail.fm Thu Jan 7 06:43:14 2010 From: rschroev_nospam_ml at fastmail.fm (Roel Schroeven) Date: Thu, 07 Jan 2010 12:43:14 +0100 Subject: Exception as the primary error handling mechanism? In-Reply-To: <4b44b138$0$11237$426a74cc@news.free.fr> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4b44b138$0$11237$426a74cc@news.free.fr> Message-ID: Bruno Desthuilliers schreef: > Phlip a ?crit : >> On Jan 5, 8:49 pm, Steven D'Aprano >> wrote: >> >>>> (A related question - why can't I just go 'if record = method(): use >>>> (record)'. Why extra lines just to trap and assign the variable before >>>> using it?) >>> Because that idiom is responsible for probably the most common error in C >>> of all, at least one of the most common errors. Thank goodness Python >>> forbids such a dangerous construct. >> switching = for == is the "most common error in C"? >> >> I can't tell if you are joking. > > It's at least a _very_ common error in all languages that allow this > construct. Maybe it is, maybe it's not. All I know is my own experience; in all the years I've been doing C and C++ (1998 - now) I've made that mistake only twice. And in both cases I found the mistake very rapidly. > In C, it's common enough to gave birth to the "BestPractice" > you described, ie swapping operand orders in equality test to have the > compiler detect the problem - at least when one of the operand is a > function call expression or constant (it obviously won't 'work' when > both operands are variables). I've never liked that practice, for the following reasons: - As you say, it doesn't work when both operands are variables. In my experience, in many cases both operands are variables. - I tend to think that not following that practice trains me to be careful in all cases, whereas I'm afraid that following the practice will make me careless, which is dangerous in all the cases where the practice won't protect me. > Anyway: in Python, assignment is not an expression, and this isn't going > to change anytime soon. To be fully clear: I'm not advocating to change the current behavior in Python, I'm just stating my experience in other languages. -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven From steve at holdenweb.com Thu Jan 7 06:48:12 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 07 Jan 2010 06:48:12 -0500 Subject: How to reduce the memory size of python In-Reply-To: <6AFAAA2E71101642861E945A057CE7CD023A884D@ZMY16EXM70.ds.mot.com> References: <6AFAAA2E71101642861E945A057CE7CD023A882B@ZMY16EXM70.ds.mot.com> <6AFAAA2E71101642861E945A057CE7CD023A884D@ZMY16EXM70.ds.mot.com> Message-ID: Mishra Gopal-QBX634 wrote: > > Hi, > > I use twisted framework too to handle the xmlrpc request. It takes > around 3-4MB of memory while importing itself. > Is there any python coding standard I should follow to save the memory. > > Like import logging takes 1MB of memory. > We only use on function getLogger by 'from logging import getLogger' > > But it still take the same 1 MB memory. > > Instead of loading whole logging module only load the getLogger > function. > > I there any way to save the memory with taking care of small things in > code.. > No. You are seeing the size of the interpreter. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Thu Jan 7 06:50:51 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 07 Jan 2010 06:50:51 -0500 Subject: Call a DLL function with argument type as unsigned char * In-Reply-To: References: Message-ID: Bhavik wrote: > Just to clarify, I am using Python 2.5.1 > Take a look at the ctypes module, which allows you to do such things (at the risk of segmentation faults and the like if you get your calls wrong). regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From kk720801 at yahoo.com Thu Jan 7 07:22:21 2010 From: kk720801 at yahoo.com (Krzysztof Kobus) Date: Thu, 7 Jan 2010 04:22:21 -0800 (PST) Subject: embedded python on mac - linking problem Message-ID: <537150.77892.qm@web53503.mail.re2.yahoo.com> Hi, I have been able to solve the problem finally: Initially I was trying (wrongly) to link distutils-made module with my application and that has failed. Solution was to (instead of linking the module) compile the source files making up the module and link corresponding objects as any other sources of the application. Thank you for your help, KK From andrew.gillanders at uqconnect.edu.au Thu Jan 7 07:47:40 2010 From: andrew.gillanders at uqconnect.edu.au (Andrew Gillanders) Date: Thu, 7 Jan 2010 22:47:40 +1000 Subject: buffer interface problem In-Reply-To: <50697b2c1001070113i3a8ce2aay50dac69ef8a25a83@mail.gmail.com> References: <3C5B8086-5D3D-444B-B9D2-C9152EBF5DA3@uqconnect.edu.au> <50697b2c1001070113i3a8ce2aay50dac69ef8a25a83@mail.gmail.com> Message-ID: <3AC7891B-5B65-4C5E-99D6-EDF1BB37B1A9@uqconnect.edu.au> Thanks Chris. The atoi function was coming from the locale library (from locale import atoi). I changed it to int and now it works. The next hurdle is this: gzin = GzipFile(fname, 'rb') data = gzin.readline() # min_x,min_y = map(atoi,data.split()[:2]) min_x,min_y = map(int,data.split()[:2]) data = gzin.readline() # span_x,step_x,span_y,step_y = map(atoi,data.split()[:4]) span_x,step_x,span_y,step_y = map(int,data.split()[:4]) data = gzin.read().split('\n') The last line is a problem, giving me this message: Type str doesn't support the buffer API (I am guessing a conflict between split and read?) Sorry, I am new to Python, so how do I get a Traceback? Thanks Andrew On 07/01/2010, at 7:13 PM, Chris Rebert wrote: > On Thu, Jan 7, 2010 at 12:19 AM, Andrew Gillanders > wrote: >> I have run into a problem running a Python script that is part of the >> TerraGear suite for building scenery for FlightGear. I am using >> Mac OS X >> 10.4, running Python (version 3.0.1) in a Unix terminal. >> >> The purpose of the script is to walk a directory tree, unzipping >> files, and >> passing the contents to an executable C program. The problem >> occurs here: >> >> gzin = GzipFile(fname, 'rb') >> data = gzin.readline() >> min_x,min_y = map(atoi,data.split()[:2]) >> >> The input file, when uncompressed, is an ASCII file with a line >> with two >> numbers, then a line of four numbers, then many long lines of >> numbers. I can >> see what the last is trying to do: split the string into two >> words, convert >> them to integers, and assign them to min_x and min_y. >> >> At the third line, I get the message "expected an object with the >> buffer >> interface". Which object is it referring to? > > The elements of the list produced by `data.split()[:2]`, which are > either Unicode strings or bytestrings, neither of which are buffers. > >> Have some functions been >> changed to pass buffer objects instead of strings? How can I fix >> the source >> code to make it run? > > The error is being raised by the atoi() function (in the future, > please post the full Traceback, not just the final error message). > What module/library does your atoi() function come from (look for an > `import` statement mentioning it)? > The only functions by that name in the Python standard library both > operate on strings, not buffers, and thus can't be the same one your > code is using. > > In any case, replacing `atoi` with `int` in your code will likely > solve the problem. The built-in int() function* can convert strings to > integers. > > Cheers, > Chris > -- > http://blog.rebertia.com > > *Not really a function, but close enough for newbie explanatory > purposes. From crebert at ucsd.edu Thu Jan 7 08:03:56 2010 From: crebert at ucsd.edu (Chris Rebert) Date: Thu, 7 Jan 2010 05:03:56 -0800 Subject: buffer interface problem In-Reply-To: <3AC7891B-5B65-4C5E-99D6-EDF1BB37B1A9@uqconnect.edu.au> References: <3C5B8086-5D3D-444B-B9D2-C9152EBF5DA3@uqconnect.edu.au> <50697b2c1001070113i3a8ce2aay50dac69ef8a25a83@mail.gmail.com> <3AC7891B-5B65-4C5E-99D6-EDF1BB37B1A9@uqconnect.edu.au> Message-ID: <50697b2c1001070503q7f8aa4fase7a6f6627ac784e0@mail.gmail.com> On Thu, Jan 7, 2010 at 4:47 AM, Andrew Gillanders wrote: > On 07/01/2010, at 7:13 PM, Chris Rebert wrote: >> On Thu, Jan 7, 2010 at 12:19 AM, Andrew Gillanders >> wrote: >>> >>> I have run into a problem running a Python script that is part of the >>> TerraGear suite for building scenery for FlightGear. I am using Mac OS X >>> 10.4, running Python (version 3.0.1) in a Unix terminal. >>> >>> The purpose of the script is to walk a directory tree, unzipping files, >>> and >>> passing the contents to an executable C program. The problem occurs here: >>> >>> ? gzin = GzipFile(fname, 'rb') >>> ? data = gzin.readline() >>> ? min_x,min_y = map(atoi,data.split()[:2]) >>> >>> The input file, when uncompressed, is an ASCII file with a line with two >>> numbers, then a line of four numbers, then many long lines of numbers. I >>> can >>> see what the last is trying to do: split the string into two words, >>> convert >>> them to integers, and assign them to min_x and min_y. >>> >>> At the third line, I get the message "expected an object with the buffer >>> interface". Which object is it referring to? >> >> The elements of the list produced by `data.split()[:2]`, which are >> either Unicode strings or bytestrings, neither of which are buffers. >> >>> Have some functions been >>> changed to pass buffer objects instead of strings? How can I fix the >>> source >>> code to make it run? >> >> The error is being raised by the atoi() function (in the future, >> please post the full Traceback, not just the final error message). >> What module/library does your atoi() function come from (look for an >> `import` statement mentioning it)? >> The only functions by that name in the Python standard library both >> operate on strings, not buffers, and thus can't be the same one your >> code is using. >> >> In any case, replacing `atoi` with `int` in your code will likely >> solve the problem. The built-in int() function* can convert strings to >> integers. > Thanks Chris. The atoi function was coming from the locale library (from > locale import atoi). I changed it to int and now it works. Hm, that's odd since it was one of the 2 functions in the std lib which the docs say operates on strings... > The next hurdle is this: > gzin = GzipFile(fname, 'rb') > > data = gzin.readline() > # min_x,min_y = map(atoi,data.split()[:2]) > min_x,min_y = map(int,data.split()[:2]) > > data = gzin.readline() > # span_x,step_x,span_y,step_y = map(atoi,data.split()[:4]) > span_x,step_x,span_y,step_y = map(int,data.split()[:4]) > > data = gzin.read().split('\n') > > The last line is a problem, giving me this message: Type str doesn't support > the buffer API (I am guessing a conflict between split and read?) Ah, looking at the 3.0 docs on buffers, I'd surmise gzin.read() returns bytes (http://docs.python.org/3.1/library/functions.html#bytes) rather than a string. You'll want to decode the bytes into characters first, and then you can operate on the resulting string normally. Try: data = gzin.read().decode('ascii').split('\n') > Sorry, I am new to Python, so how do I get a Traceback? You should get one by default. Are you running the script in some environment other than the command line? Here's what a traceback looks like: Traceback (most recent call last): File "foo", line 161, in main() File "foo.py", line 157, in main bot.run() File "foo.py", line 68, in bar self.baz("Enter number: ") File "foo.py", line 112, in baz choice = int(raw_input(prompt))-1 ValueError: invalid literal for int() with base 10: 'y' Cheers, Chris -- http://blog.rebertia.com From mail.to.daniel.platz at googlemail.com Thu Jan 7 08:23:42 2010 From: mail.to.daniel.platz at googlemail.com (Daniel Platz) Date: Thu, 7 Jan 2010 05:23:42 -0800 (PST) Subject: Pass multidimensional array (matrix) to c function using ctypes References: <63ac1a01-8491-4885-bae7-cb884abb582f@34g2000yqp.googlegroups.com> Message-ID: <8c514dac-ed22-4ea1-b117-d5b98f099df9@m16g2000yqc.googlegroups.com> Thanks a lot. This solves my problem and I understand now much better what is going on. Best regards, Daniel From stefan_ml at behnel.de Thu Jan 7 08:36:49 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 07 Jan 2010 14:36:49 +0100 Subject: Where's a DOM builder that uses the Builder Pattern to ... build DOMs? In-Reply-To: <026df5db-bcef-424d-8cd5-56dacebf01f8@c3g2000yqd.googlegroups.com> References: <04523152-a0ff-432c-a48f-2c653577e7b7@m25g2000yqc.googlegroups.com> <4b42f551$0$7625$9b4e6d93@newsspool1.arcor-online.net> <026df5db-bcef-424d-8cd5-56dacebf01f8@c3g2000yqd.googlegroups.com> Message-ID: <4b45e370$0$6736$9b4e6d93@newsspool2.arcor-online.net> Phlip, 05.01.2010 18:00: > On Jan 5, 12:16 am, Stefan Behnel wrote: > >> Note that there are tons of ways to generate HTML with Python. > > Forgot to note - I'm generating schematic XML, and I'm trying to find > a way better than the Django template I started with! Well, then note that there are tons of ways to generate XML with Python, including the one I pointed you to. Stefan From rschroev_nospam_ml at fastmail.fm Thu Jan 7 08:37:07 2010 From: rschroev_nospam_ml at fastmail.fm (Roel Schroeven) Date: Thu, 07 Jan 2010 14:37:07 +0100 Subject: please help shrink this each_with_index() implementation In-Reply-To: References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> <1715bea7-1ff1-4294-9dad-672873e14530@v25g2000yqk.googlegroups.com> Message-ID: <7sl1n.7163$BK3.5872@newsfe16.ams2> Phlip schreef: > Nobody wrote: >> Writing robust software from the outset puts you at a competitive >> disadvantage to those who understand how the system works. > > And I, not my language, should pick and chose how to be rigorous. The language > should not make the decision for me. You can always easily make your program less rigorous than the language, but the reverse is generally very difficult. So a rigorous language gives you the choice, why a non-rigorous language does not. -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven From eight32 at gmail.com Thu Jan 7 09:17:26 2010 From: eight32 at gmail.com (Stuart Murray-Smith) Date: Thu, 7 Jan 2010 16:17:26 +0200 Subject: Python books, literature etc In-Reply-To: <304854.43355.qm@smtp139.mail.mud.yahoo.com> References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> <304854.43355.qm@smtp139.mail.mud.yahoo.com> Message-ID: > Have a look at the Getting Started section of the wiki: > > http://wiki.python.org/moin/ > > specially the PythonBooks section Perfect! Exactly what I'm looking for :) Thanks Gabriel! From lie.1296 at gmail.com Thu Jan 7 09:37:08 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Fri, 08 Jan 2010 01:37:08 +1100 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4b44b138$0$11237$426a74cc@news.free.fr> Message-ID: <4b45f195$1@dnews.tpgi.com.au> On 1/7/2010 10:43 PM, Roel Schroeven wrote: > - I tend to think that not following that practice trains me to be > careful in all cases, whereas I'm afraid that following the practice > will make me careless, which is dangerous in all the cases where the > practice won't protect me. > That's a sign of a gotcha... a well-designed language makes you think about your problem at hand and less about the language's syntax. From mackrackit at gmail.com Thu Jan 7 09:56:35 2010 From: mackrackit at gmail.com (Dave McCormick) Date: Thu, 07 Jan 2010 07:56:35 -0700 Subject: Exception as the primary error handling mechanism? In-Reply-To: <4b45f195$1@dnews.tpgi.com.au> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4b44b138$0$11237$426a74cc@news.free.fr> <4b45f195$1@dnews.tpgi.com.au> Message-ID: <4B45F623.7060303@gmail.com> Lie Ryan wrote: > That's a sign of a gotcha... a well-designed language makes you think > about your problem at hand and less about the language's syntax. Not until you learn the language that is. From a Python newbee.... ;-) From cmpython at gmail.com Thu Jan 7 10:26:18 2010 From: cmpython at gmail.com (CM) Date: Thu, 7 Jan 2010 07:26:18 -0800 (PST) Subject: GUI for multiplatform multimedia project References: Message-ID: On Jan 6, 4:53?pm, wrote: > Hi everyone, > > I posted that question on a python-forum, but got answer, so I ask here. > > I'm working on an artistic project and I'm looking for the best > cross-platform GUI solution. The problem is that it's gonna be a tool that > will have to be double-click installable/runnable and pre-installation of > any libraries for end-users is very much like an evil. It really has to be > double-click tool > > My first thought was PyQt, because it's a real framework with a lot of > stuff inside (including Phonon) and I know some cross-platform media > software written in C++ QT (like VLC). But on the other hand I've heard > that it's not that easy to make it "double-clicky" multi-platform. Is that > true? > > Another thing that matters for me is ease of integration with libraries > like OpenCV. > > I will be VERY thankful for any help. I'm SO tired googling the problem > (it's like weeks now!!) > > Best from Poland, > trzewiczek I don't know this for sure, but I would be surprised if any of the widget toolkits gave you much more trouble than any other when making your app into a bundled executable. I have made wxPython apps into a Windows .exe file easily using GUI2Exe, which is an excellent GUI interface (written in wxPython by Andrea Gavana) to a number of the executable bundlers: py2exe, PyInstaller, py2app, cx_Freeze, bbFreeze. Some of these are for Windows, some for Mac, some for Linux. wxPython apparently works with OpenCV: http://opencv.willowgarage.com/wiki/wxpython While you're Googling you might want to be aware of any legal concerns with py2exe and distributing dll files (if there are any). From vmail at mycircuit.org Thu Jan 7 11:10:05 2010 From: vmail at mycircuit.org (Peter) Date: Thu, 07 Jan 2010 17:10:05 +0100 Subject: Recommended "new" way for config files Message-ID: <4B46075D.8080606@mycircuit.org> Hi There seems to be several strategies to enhance the old ini-style config files with real python code, for example: 1) the documentation tool sphinx uses a python file conf.py that is exefile(d) , but execfile is suppressed in Python 3 2) there is a module cfgparse on sourceforge that supports a hybrid style 3) modern tools like ipython seems to favor a new style based on python code config files but also support a hybrid style mixing .ini files and python code files. 4) I could use __import__ to import modules based on some command line options Is there a strategy that should be prefered for new projects ? thanks peter From grahn+nntp at snipabacken.se Thu Jan 7 11:18:32 2010 From: grahn+nntp at snipabacken.se (Jorgen Grahn) Date: 7 Jan 2010 16:18:32 GMT Subject: How to execute a script from another script and other script does not do busy wait. References: <76bff0b6-70cb-4960-9b4d-3af7a9a1e0e7@z41g2000yqz.googlegroups.com> Message-ID: On Thu, 2010-01-07, Rajat wrote: > I want to run a python script( aka script2) from another python script > (aka script1). While script1 executes script2 it waits for script2 to > complete and in doing so it also does some other useful work.(does not > do a busy wait). > > My intention is to update a third party through script1 that script2 > is going to take longer. I do not understand that sentence. What are you trying to do, more exactly? The best solution can be threads, os.popen, os.system or something different -- depending on the details of what you want to do. > Please suggest how should I go about implementing it. > > I'm currently executing it as: > > import main from script2 > ret_code = main() > return ret_code > > which surely is not going to achieve me what I intend. > > > Thanks, > Rajat. /Jorgen -- // Jorgen Grahn O o . From lie.1296 at gmail.com Thu Jan 7 11:30:45 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Fri, 08 Jan 2010 03:30:45 +1100 Subject: Recommended "new" way for config files In-Reply-To: References: Message-ID: <4b460c36$1@dnews.tpgi.com.au> On 1/8/2010 3:10 AM, Peter wrote: > Is there a strategy that should be prefered for new projects ? The answer is, it depends. From jeanmichel at sequans.com Thu Jan 7 11:32:32 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Thu, 07 Jan 2010 17:32:32 +0100 Subject: Recommended "new" way for config files In-Reply-To: <4B46075D.8080606@mycircuit.org> References: <4B46075D.8080606@mycircuit.org> Message-ID: <4B460CA0.20800@sequans.com> Peter wrote: > Hi > There seems to be several strategies to enhance the old ini-style > config files with real python code, for example: > > 1) the documentation tool sphinx uses a python file conf.py that is > exefile(d) , but execfile is suppressed in Python 3 > 2) there is a module cfgparse on sourceforge that supports a hybrid style > 3) modern tools like ipython seems to favor a new style based on > python code config files but also support a hybrid style mixing .ini > files and python code files. > 4) I could use __import__ to import modules based on some command line > options > > > Is there a strategy that should be prefered for new projects ? > > thanks > peter I would add the standard module ConfigParser http://docs.python.org/library/configparser.html to your list. I don't know exactly what you intend to do with point 4/, but I would exclude it if any other point may fit. Imports can become tricky when used out of the common way. Anyway, hacking the import statement for managing configuration files does not sound very appropriate. The .ini file is the simpliest solution, at least from the user point of view, no need to learn any python syntax. However, speeking for myself, I am using python coded configuration files, but: we all worship python in the team and thus are familiar with it. JM From grahn+nntp at snipabacken.se Thu Jan 7 11:32:58 2010 From: grahn+nntp at snipabacken.se (Jorgen Grahn) Date: 7 Jan 2010 16:32:58 GMT Subject: Do I have to use threads? References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> <432961e5-627e-452d-93db-9562951551d2@e27g2000yqd.googlegroups.com> Message-ID: On Thu, 2010-01-07, Marco Salden wrote: > On Jan 6, 5:36?am, Philip Semanchuk wrote: >> On Jan 5, 2010, at 11:26 PM, aditya shukla wrote: >> >> > Hello people, >> >> > I have 5 directories corresponding 5 ?different urls .I want to ? >> > download >> > images from those urls and place them in the respective ? >> > directories.I have >> > to extract the contents and download them simultaneously.I can ? >> > extract the >> > contents and do then one by one. My questions is for doing it ? >> > simultaneously >> > do I have to use threads? >> >> No. You could spawn 5 copies of wget (or curl or a Python program that ? >> you've written). Whether or not that will perform better or be easier ? >> to code, debug and maintain depends on the other aspects of your ? >> program(s). >> >> bye >> Philip > > Yep, the more easier and straightforward the approach, the better: > threads are always (programmers')-error-prone by nature. > But my question would be: does it REALLY need to be simultaneously: > the CPU/OS only has more overhead doing this in parallel with > processess. Measuring sequential processing and then trying to > optimize (e.g. for user response or whatever) would be my prefered way > to go. Less=More. Normally when you do HTTP in parallell over several TCP sockets, it has nothing to do with CPU overhead. You just don't want every GET to be delayed just because the server(s) are lazy responding to the first few ones; or you might want to read the text of a web page and the CSS before a few huge pictures have been downloaded. His "I have to [do them] simultaneously" makes me want to ask "Why?". If he's expecting *many* pictures, I doubt that the parallel download will buy him much. Reusing the same TCP socket for all of them is more likely to help, especially if the pictures aren't tiny. One long-lived TCP connection is much more efficient than dozens of short-lived ones. Personally, I'd popen() wget and let it do the job for me. /Jorgen -- // Jorgen Grahn O o . From robert.kern at gmail.com Thu Jan 7 11:38:06 2010 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 07 Jan 2010 10:38:06 -0600 Subject: Recommended "new" way for config files In-Reply-To: <4B46075D.8080606@mycircuit.org> References: <4B46075D.8080606@mycircuit.org> Message-ID: On 2010-01-07 10:10 AM, Peter wrote: > Hi > There seems to be several strategies to enhance the old ini-style config > files with real python code, for example: > > 1) the documentation tool sphinx uses a python file conf.py that is > exefile(d) , but execfile is suppressed in Python 3 Only because it is redundant, not because it is a discouraged approach. You can still read the file and exec() the resulting string. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From phlip2005 at gmail.com Thu Jan 7 11:44:25 2010 From: phlip2005 at gmail.com (Phlip) Date: Thu, 7 Jan 2010 08:44:25 -0800 (PST) Subject: Where's a DOM builder that uses the Builder Pattern to ... build DOMs? References: <04523152-a0ff-432c-a48f-2c653577e7b7@m25g2000yqc.googlegroups.com> <4b42f551$0$7625$9b4e6d93@newsspool1.arcor-online.net> <026df5db-bcef-424d-8cd5-56dacebf01f8@c3g2000yqd.googlegroups.com> <4b45e370$0$6736$9b4e6d93@newsspool2.arcor-online.net> Message-ID: On Jan 7, 5:36?am, Stefan Behnel wrote: > Well, then note that there are tons of ways to generate XML with Python, > including the one I pointed you to. from lxml.html import builder as E xml = E.foo() All I want is "", but I get "AttributeError: 'module' object has no attribute 'foo'". A peek at dir(E) shows it only has HTML tags, all hard coded. So how to get it to generate any random XML tag my clients think of? I will write this myself with __getattr__ etc, if I can't find it, because the permissive & expressive builder pattern I'm after would be very ... permissive & expressive. All I want is a library that reads my mind!!! Is that too much to ask??? (Unless if the library insists on throwing a NullMind exception, on principle...) -- Phlip http://twitter.com/Pen_Bird From grahn+nntp at snipabacken.se Thu Jan 7 12:22:04 2010 From: grahn+nntp at snipabacken.se (Jorgen Grahn) Date: 7 Jan 2010 17:22:04 GMT Subject: Python books, literature etc References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: On Thu, 2010-01-07, Stuart Murray-Smith wrote: ... > [...] ESR's guide to > smart questions [1] helps set the pace of list culture. It's good, if you can ignore the "These People Are Very Important Hacker Gods, Not Mere Mortals" subtext. ... > Anyways, to rephrase, could someone kindly mention any of their > preferred Python books, websites, tutorials etc to help me get to an > intermediate/advanced level? Something that would help me add > functionality to Ubiquity, say. I may be alone in this, but Alex Martelli's book ("Python in a nutshell"?) on Python 2.2 and a bit of 2.3, plus the official documentation, plus this group, is all I think I need. But I had a lot of Unix, C, C++ and Perl experience to help me. /Jorgen -- // Jorgen Grahn O o . From python at mrabarnett.plus.com Thu Jan 7 12:38:46 2010 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 07 Jan 2010 17:38:46 +0000 Subject: Do I have to use threads? In-Reply-To: References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> <432961e5-627e-452d-93db-9562951551d2@e27g2000yqd.googlegroups.com> Message-ID: <4B461C26.6030209@mrabarnett.plus.com> Jorgen Grahn wrote: > On Thu, 2010-01-07, Marco Salden wrote: >> On Jan 6, 5:36 am, Philip Semanchuk wrote: >>> On Jan 5, 2010, at 11:26 PM, aditya shukla wrote: >>> >>>> Hello people, >>>> I have 5 directories corresponding 5 different urls .I want to >>>> download >>>> images from those urls and place them in the respective >>>> directories.I have >>>> to extract the contents and download them simultaneously.I can >>>> extract the >>>> contents and do then one by one. My questions is for doing it >>>> simultaneously >>>> do I have to use threads? >>> No. You could spawn 5 copies of wget (or curl or a Python program that >>> you've written). Whether or not that will perform better or be easier >>> to code, debug and maintain depends on the other aspects of your >>> program(s). >>> >>> bye >>> Philip >> Yep, the more easier and straightforward the approach, the better: >> threads are always (programmers')-error-prone by nature. >> But my question would be: does it REALLY need to be simultaneously: >> the CPU/OS only has more overhead doing this in parallel with >> processess. Measuring sequential processing and then trying to >> optimize (e.g. for user response or whatever) would be my prefered way >> to go. Less=More. > > Normally when you do HTTP in parallell over several TCP sockets, it > has nothing to do with CPU overhead. You just don't want every GET to > be delayed just because the server(s) are lazy responding to the first > few ones; or you might want to read the text of a web page and the CSS > before a few huge pictures have been downloaded. > > His "I have to [do them] simultaneously" makes me want to ask "Why?". > > If he's expecting *many* pictures, I doubt that the parallel download > will buy him much. Reusing the same TCP socket for all of them is > more likely to help, especially if the pictures aren't tiny. One > long-lived TCP connection is much more efficient than dozens of > short-lived ones. > > Personally, I'd popen() wget and let it do the job for me. > From my own experience: I wanted to download a number of webpages. I noticed that there was a significant delay before it would reply, and an especially long delay for one of them, so I used a number of threads, each one reading a URL from a queue, performing the download, and then reading the next URL, until there were none left (actually, until it read the sentinel None, which it put back for the other threads). The result? Shorter total download time because it could be downloading one webpage while waiting for another to reply. (Of course, I had to make sure that I didn't have too many threads, because that might've put too many demands on the website, not a nice thing to do!) From philip at semanchuk.com Thu Jan 7 12:53:39 2010 From: philip at semanchuk.com (Philip Semanchuk) Date: Thu, 7 Jan 2010 12:53:39 -0500 Subject: Do I have to use threads? In-Reply-To: References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> <432961e5-627e-452d-93db-9562951551d2@e27g2000yqd.googlegroups.com> Message-ID: <0CFE9D52-F77E-4158-A461-AD1F56DD2B04@semanchuk.com> On Jan 7, 2010, at 11:32 AM, Jorgen Grahn wrote: > On Thu, 2010-01-07, Marco Salden wrote: >> On Jan 6, 5:36 am, Philip Semanchuk wrote: >>> On Jan 5, 2010, at 11:26 PM, aditya shukla wrote: >>> >>>> Hello people, >>> >>>> I have 5 directories corresponding 5 different urls .I want to >>>> download >>>> images from those urls and place them in the respective >>>> directories.I have >>>> to extract the contents and download them simultaneously.I can >>>> extract the >>>> contents and do then one by one. My questions is for doing it >>>> simultaneously >>>> do I have to use threads? >>> >>> No. You could spawn 5 copies of wget (or curl or a Python program >>> that >>> you've written). Whether or not that will perform better or be >>> easier >>> to code, debug and maintain depends on the other aspects of your >>> program(s). >>> >>> bye >>> Philip >> >> Yep, the more easier and straightforward the approach, the better: >> threads are always (programmers')-error-prone by nature. >> But my question would be: does it REALLY need to be simultaneously: >> the CPU/OS only has more overhead doing this in parallel with >> processess. Measuring sequential processing and then trying to >> optimize (e.g. for user response or whatever) would be my prefered >> way >> to go. Less=More. > > Normally when you do HTTP in parallell over several TCP sockets, it > has nothing to do with CPU overhead. You just don't want every GET to > be delayed just because the server(s) are lazy responding to the first > few ones; or you might want to read the text of a web page and the CSS > before a few huge pictures have been downloaded. > > His "I have to [do them] simultaneously" makes me want to ask "Why?". Exactly what I was thinking. He's surely doing something more complicated than his post suggests, and without that detail it's impossible to say whether threads, processes, asynch or voodoo is the best approach. bye P From sccolbert at gmail.com Thu Jan 7 13:13:39 2010 From: sccolbert at gmail.com (S. Chris Colbert) Date: Thu, 7 Jan 2010 19:13:39 +0100 Subject: ANN: Pymazon 0.1.0 released! Message-ID: <201001071913.39467.sccolbert@gmail.com> Hello, I'm happy to announce the first non-beta release of Pymazon: a python implemented downloader for the Amazon mp3 store. Improvements from the beta: - Running download status indicator - Various fixes for Windows - Some code cleanup Pymazon was created to be a simple and easy alternative for the Linux version of the Amazon downloader, and alleviate the pain of getting it to work with 64bit Linux. You can read about Pymazon at http://pymazon.googlecode.com You can download from googlecode or the cheeseshop: $ pip install pymazon or $ easy_install pymazon It also works on Windows. Dependencies: PyCrypto (it's in the ubuntu repos and the cheeseshop) PyQt4 >= 4.5 (optional, only needed for GUI) GPLv3 License Cheers! SCC From vmail at mycircuit.org Thu Jan 7 13:19:40 2010 From: vmail at mycircuit.org (Peter) Date: Thu, 07 Jan 2010 19:19:40 +0100 Subject: Recommended "new" way for config files In-Reply-To: <4B460CA0.20800@sequans.com> References: <4B46075D.8080606@mycircuit.org> <4B460CA0.20800@sequans.com> Message-ID: <4B4625BC.6090201@mycircuit.org> Thanks for your answer, let me be more precise: > I would add the standard module ConfigParser > http://docs.python.org/library/configparser.html to your list. of course, that was the implicit starting point of my request, when talking about .ini files. > I don't know exactly what you intend to do with point 4/, It would allow me to select different conf.py files with command line switches, like for example a -c option. > but I would exclude it if any other point may fit. Imports can become > tricky when used out of the common way. Anyway, hacking the import > statement for managing configuration files does not sound very > appropriate. > Would this be considered a hack ? #!/usr/bin/env python import sys # parse command line options here if option='standard': const = __import__('consts') else: const = __import__('alternative_consts') > The .ini file is the simpliest solution, at least from the user point > of view, no need to learn any python syntax. I am speaking from the point of view of a python programmer, and I find the .ini restrictions not necessarily simple, for example when dealing with structured data (I suppose it is trivial to specify a dictionnary or a list for the purpose of my request) For example, configuration files for the logging module get unwieldy when you specify several loggers , handlers, formatters etc, because you have to break down structured data ( objects ) to name,value pairs. > However, speeking for myself, I am using python coded configuration > files, but: we all worship python in the team and thus are familiar > with it. > so do I. > JM > > So what is the "worshipped" approach, when you need more than name=value pairs ? Peter From Lee at JamToday.com Thu Jan 7 13:22:57 2010 From: Lee at JamToday.com (Lee) Date: Thu, 07 Jan 2010 13:22:57 -0500 Subject: Dictionary used to build a Triple Store Message-ID: Definitely a newbie question, so please bear with me. I'm reading "Programming the Semantic Web" by Segaran, Evans, and Tayor. It's about the Semantic Web BUT it uses python to build a "toy" triple store claimed to have good performance in the "tens of thousands" of triples. Just in case anybody doesnt know what an RDF triple is (not that it matters for my question) think of it as an ordered 3 tuple representing a Subject, a Predicate, and an Object eg: (John, loves, Mary) (Mary, has-a, lamb) {theSky, has-color,blue} To build the triple store entirely in Python, the authors recommend using the Python hash. Three hashes actually (I get that. You want to have a hash with the major index being the Subject in one hash, the Predicate in another hash, or the Object for the third hash) He creates a class SimpleGraph which initializes itself by setting the three hashes names _spo, _pos, and _osp thus class SimpleGraph; def __init__(self); self._spo={}; self._pos=(); self._osp={}; So far so good. I get the convention with the double underbars for the initializer but Q1: Not the main question but while I'm here....I'm a little fuzzy on the convention about the use of the single underbar in the definition of the hashes. Id the idea to "underbar" all objects and methods that belong to the class? Why do that? But now the good stuff: Our authors define the hashes thus: (showing only one of the three hashes because they're all the same idea) self._pos = {predicate:{object:set( [subject] ) }} Q2: Wha? Two surprises ... 1) Why not {predicate:{object:subject}} i.e. pos[predicate][object]=subject....why the set( [object] ) construct? putting the object into a list and turning the list into a set to be the "value" part of a name:value pair. Why not just use the naked subject for the value? 2) Why not something like pos[predicate][object][subject] = 1 .....or any constant. The idea being to create the set of three indexes. If the triple exists in the hash, its "in" your tripple store. If not, then there's no such triple. From clp2 at rebertia.com Thu Jan 7 13:23:58 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Thu, 7 Jan 2010 10:23:58 -0800 Subject: Recommended "new" way for config files In-Reply-To: <4B4625BC.6090201@mycircuit.org> References: <4B46075D.8080606@mycircuit.org> <4B460CA0.20800@sequans.com> <4B4625BC.6090201@mycircuit.org> Message-ID: <50697b2c1001071023m28e9a242uef20c6e65414c371@mail.gmail.com> On Thu, Jan 7, 2010 at 10:19 AM, Peter wrote: >> The .ini file is the simpliest solution, at least from the user point of >> view, no need to learn any python syntax. > > I am speaking from the point of view of a python programmer, and I find the > .ini restrictions not necessarily simple, for example when dealing with > structured data (I suppose it is trivial to specify a dictionnary or a list > for the purpose of my request) For example, configuration files for the > logging module get unwieldy when you specify several loggers , handlers, > formatters etc, because you have to break down structured data ( objects ) > to name,value pairs. > So what is the "worshipped" approach, when you need more than name=value > pairs ? JSON is one option: http://docs.python.org/library/json.html Cheers, Chris -- http://blog.rebertia.com From tjreedy at udel.edu Thu Jan 7 13:31:48 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 07 Jan 2010 13:31:48 -0500 Subject: How to reduce the memory size of python In-Reply-To: <6AFAAA2E71101642861E945A057CE7CD023A884D@ZMY16EXM70.ds.mot.com> References: <6AFAAA2E71101642861E945A057CE7CD023A882B@ZMY16EXM70.ds.mot.com> <6AFAAA2E71101642861E945A057CE7CD023A884D@ZMY16EXM70.ds.mot.com> Message-ID: On 1/7/2010 3:34 AM, Mishra Gopal-QBX634 wrote: > > Like import logging takes 1MB of memory. > We only use on function getLogger by 'from logging import getLogger' > > But it still take the same 1 MB memory. > > Instead of loading whole logging module only load the getLogger > function. from x import y causes creation of module x and binding of the module to sys.modules'x]. It then binds name 'y' in the current namespace to the corresponding object in x. Functions in general need a reference to the module namespace to resolve module-level variables. To save anything, you must cut the function out of the module and verify that it works in isolation. But I presume 'getLogger' refers to other stuff in the logging module and would not work in isolation. Terry Jan Reedy From vmail at mycircuit.org Thu Jan 7 13:34:18 2010 From: vmail at mycircuit.org (Peter) Date: Thu, 07 Jan 2010 19:34:18 +0100 Subject: Python books, literature etc In-Reply-To: References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: <4B46292A.4020308@mycircuit.org> >> Anyways, to rephrase, could someone kindly mention any of their >> preferred Python books, websites, tutorials etc to help me get to an >> intermediate/advanced level? Something that would help me add >> functionality to Ubiquity, say. >> > I may be alone in this, but Alex Martelli's book ("Python in a > nutshell"?) on Python 2.2 and a bit of 2.3, plus the official > documentation, plus this group, is all I think I need. > But I had a lot of Unix, C, C++ and Perl experience to help me. > > /Jorgen > > I find Alex Martellis "Python Cookbook" excellent/invaluable and ( and also his Nutshell book mentioned above ) and depending on your application domain, I liked: 1) Hans Petter Langtangen: Python Scripting for Computational Science A truly excellent book, not only with respect to Python Scripting , but also on how to avoid paying license fees by using opensource tools as an engineer ( plotting, graphing, gui dev etc ). Very good , pratical introduction to Python with careful and non-trivial examples and exercises. 2) There is a book at Apress on using Python and matplotlib ( amongst other ) "Beginning Python Visualization" which is not as comprehensive as reference 1) but useful , especially for beginners who wants to visualize data from an engineers background 3) "Programming for the semantic web" Oreilly is a very pratical and interesting guide to things like OWL, triplestore, logic, reasoning, data mining and it is amongst the very few books on these topics I have seen that has working code examples 4) "Natural language priocessing with Python " Oreilly is also a pratical book with lots of working code if you are interested in data mining, text searching and natural language tasks. It is based on a rather large opensource library for natural language processing ( sorry forgot the exact name,but easy to find on the net) All these book make you feel warm and confortable if you have ever tried to do these things in Perl, C++ or Java Peter From sccolbert at gmail.com Thu Jan 7 13:39:42 2010 From: sccolbert at gmail.com (S. Chris Colbert) Date: Thu, 7 Jan 2010 19:39:42 +0100 Subject: Dictionary used to build a Triple Store In-Reply-To: References: Message-ID: <201001071939.42847.sccolbert@gmail.com> > Definitely a newbie question, so please bear with me. > > I'm reading "Programming the Semantic Web" by Segaran, Evans, and Tayor. > > It's about the Semantic Web BUT it uses python to build a "toy" triple > store claimed to have good performance in the "tens of thousands" of > triples. > > Just in case anybody doesnt know what an RDF triple is (not that it > matters for my question) think of it as an ordered 3 tuple representing > a Subject, a Predicate, and an Object eg: (John, loves, Mary) (Mary, > has-a, lamb) {theSky, has-color,blue} > > To build the triple store entirely in Python, the authors recommend > using the Python hash. Three hashes actually (I get that. You > want to have a hash with the major index being the Subject in one hash, > the Predicate in another hash, or the Object for the third hash) > > He creates a class SimpleGraph which initializes itself by setting the > three hashes names _spo, _pos, and _osp thus > > class SimpleGraph; > def __init__(self); > self._spo={}; > self._pos=(); > self._osp={}; > > So far so good. I get the convention with the double underbars for the > initializer but > > Q1: Not the main question but while I'm here....I'm a little fuzzy on > the convention about the use of the single underbar in the definition of > the hashes. Id the idea to "underbar" all objects and methods that > belong to the class? Why do that? > > But now the good stuff: > > Our authors define the hashes thus: (showing only one of the three > hashes because they're all the same idea) > > self._pos = {predicate:{object:set( [subject] ) }} > > Q2: Wha? Two surprises ... > 1) Why not {predicate:{object:subject}} i.e. > pos[predicate][object]=subject....why the set( [object] ) construct? > putting the object into a list and turning the list into a set to be the > "value" part of a name:value pair. Why not just use the naked subject > for the value? > because the argument has to be iterable. In [1]: set(1) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) /home/brucewayne/ in () TypeError: 'int' object is not iterable > 2) Why not something like pos[predicate][object][subject] = 1 > .....or any constant. The idea being to create the set of three indexes. > If the triple exists in the hash, its "in" your tripple store. If not, > then there's no such triple. > I can't really answer that, I imagine there is a better way to code what is trying to be accomplished. But I'm no Steven D'Aprano and I'm already a few beers in ;) From steve at holdenweb.com Thu Jan 7 13:54:05 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 07 Jan 2010 13:54:05 -0500 Subject: Dictionary used to build a Triple Store In-Reply-To: References: Message-ID: Lee wrote: > Definitely a newbie question, so please bear with me. > > I'm reading "Programming the Semantic Web" by Segaran, Evans, and Tayor. > > It's about the Semantic Web BUT it uses python to build a "toy" triple > store claimed to have good performance in the "tens of thousands" of > triples. > > Just in case anybody doesnt know what an RDF triple is (not that it > matters for my question) think of it as an ordered 3 tuple representing > a Subject, a Predicate, and an Object eg: (John, loves, Mary) (Mary, > has-a, lamb) {theSky, has-color,blue} > > To build the triple store entirely in Python, the authors recommend > using the Python hash. Three hashes actually (I get that. You > want to have a hash with the major index being the Subject in one hash, > the Predicate in another hash, or the Object for the third hash) > > He creates a class SimpleGraph which initializes itself by setting the > three hashes names _spo, _pos, and _osp thus > > class SimpleGraph; > def __init__(self); > self._spo={}; > self._pos=(); > self._osp={}; > > So far so good. I get the convention with the double underbars for the > initializer but > > Q1: Not the main question but while I'm here....I'm a little fuzzy on > the convention about the use of the single underbar in the definition of > the hashes. Id the idea to "underbar" all objects and methods that > belong to the class? Why do that? > > But now the good stuff: > > Our authors define the hashes thus: (showing only one of the three > hashes because they're all the same idea) > > self._pos = {predicate:{object:set( [subject] ) }} > > Q2: Wha? Two surprises ... > 1) Why not {predicate:{object:subject}} i.e. > pos[predicate][object]=subject....why the set( [object] ) construct? > putting the object into a list and turning the list into a set to be the > "value" part of a name:value pair. Why not just use the naked subject > for the value? > Because for a given predicate there can be many objects, and you need to be able to look up the subjects associated with the same object and predicate. (I am assuming this is initialization code: to add another subject with the same object to the predicate you would use self._pos[predicate][object].add(subject) > 2) Why not something like pos[predicate][object][subject] = 1 .....or > any constant. The idea being to create the set of three indexes. If the > triple exists in the hash, its "in" your tripple store. If not, then > there's no such triple. > Because it's less efficient. Since there will only ever be one unique occurrence of each (predicate, object, subject) triple using a dict would be unnecessarily wasteful. Containment checks for sets are just as fast as for dicts, asn you don't need to store all those references to 1. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From rschroev_nospam_ml at fastmail.fm Thu Jan 7 14:30:37 2010 From: rschroev_nospam_ml at fastmail.fm (Roel Schroeven) Date: Thu, 07 Jan 2010 20:30:37 +0100 Subject: Exception as the primary error handling mechanism? In-Reply-To: <4b45f195$1@dnews.tpgi.com.au> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4b44b138$0$11237$426a74cc@news.free.fr> <4b45f195$1@dnews.tpgi.com.au> Message-ID: Lie Ryan schreef: > On 1/7/2010 10:43 PM, Roel Schroeven wrote: >> - I tend to think that not following that practice trains me to be >> careful in all cases, whereas I'm afraid that following the practice >> will make me careless, which is dangerous in all the cases where the >> practice won't protect me. >> > > That's a sign of a gotcha... a well-designed language makes you think > about your problem at hand and less about the language's syntax. It's not a big deal to me, but you're right, it's a gotcha. I don't think there's a single language without gotcha's; one of the things I like about Python is that is has many less than the other languages I know. -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven From vardan.pogosyan at gmail.com Thu Jan 7 14:32:07 2010 From: vardan.pogosyan at gmail.com (h0uk) Date: Thu, 7 Jan 2010 11:32:07 -0800 (PST) Subject: PyQt QThreadPool error Message-ID: <7dc7a9f3-cff9-4911-bfde-6086db471b41@k17g2000yqh.googlegroups.com> Hello. I have the following code: #workers = {} QtCore.QThreadPool.globalInstance().setExpiryTimeout (300000) QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) for i in range(1, int(userscnt) + 1): work = wk.Worker(i) # connect signals work.mmShowCaptcha.connect(self.show_captcha_dlg) work.log.connect(self.handle_log) self.captcha_answer.connect(work.mmCaptchaAnswer) work.setAutoDelete(True) QtCore.QThreadPool.globalInstance().start(work) On last line of code ( QtCore.QThreadPool.globalInstance().start (work) ) i get an error: SystemError: error return without exception set What is wrong in my code??? Any advise??? Thanks From danmcleran at yahoo.com Thu Jan 7 14:48:45 2010 From: danmcleran at yahoo.com (danmcleran at yahoo.com) Date: Thu, 7 Jan 2010 11:48:45 -0800 (PST) Subject: How to execute a script from another script and other script does not do busy wait. References: <76bff0b6-70cb-4960-9b4d-3af7a9a1e0e7@z41g2000yqz.googlegroups.com> Message-ID: <342e58d5-ecd5-4cc5-ad4c-cd1ea169f9e0@c34g2000yqn.googlegroups.com> On Jan 7, 9:18?am, Jorgen Grahn wrote: > On Thu, 2010-01-07, Rajat wrote: > > I want to run a python script( aka script2) from another python script > > (aka script1). While script1 executes script2 it waits for script2 to > > complete and in doing so it also does some other useful work.(does not > > do a busy wait). > > > My intention is to update a third party through script1 that script2 > > is going to take longer. > > I do not understand that sentence. > What are you trying to do, more exactly? ?The best solution can be > threads, os.popen, os.system or something different -- depending on > the details of what you want to do. > > > Please suggest how should I go about implementing it. > > > I'm currently executing it as: > > > import main from script2 > > ret_code ?= main() > > return ret_code > > > which surely is not going to achieve me what I intend. > > > Thanks, > > Rajat. > > /Jorgen > > -- > ? // Jorgen Grahn \X/ ? ? snipabacken.se> ? O ?o ? . I personally use subprocess. Once you launch via subprocess you can wait or not. p = subprocess.Popen(...) p.wait() #or not. See subprocess docs. From deets at nospam.web.de Thu Jan 7 14:56:22 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Thu, 07 Jan 2010 20:56:22 +0100 Subject: GUI for multiplatform multimedia project In-Reply-To: References: Message-ID: <7qmsj8Fc9tU1@mid.uni-berlin.de> trzewiczek at trzewiczek.info schrieb: > Hi everyone, > > I posted that question on a python-forum, but got answer, so I ask here. > > I'm working on an artistic project and I'm looking for the best > cross-platform GUI solution. The problem is that it's gonna be a tool that > will have to be double-click installable/runnable and pre-installation of > any libraries for end-users is very much like an evil. It really has to be > double-click tool > > My first thought was PyQt, because it's a real framework with a lot of > stuff inside (including Phonon) and I know some cross-platform media > software written in C++ QT (like VLC). But on the other hand I've heard > that it's not that easy to make it "double-clicky" multi-platform. Is that > true? I don't know exactly what you mean with that, but I doubt it's easier with anything else. Another option might be pygame + simple OpenGL though, if you are planning on heavy use of Canvas-like things, that might be good enough & less heavyweight. > > Another thing that matters for me is ease of integration with libraries > like OpenCV. That has little todo with the toolkit. You need some image-converting-code, I've written such for Cocoa, to convert OpenCV's RGBA to OSX ARGB. But the last resort would be to save the images from OpenCV and read them using Qt (or whatever toolkit you use in th eend. Diez From deets at nospam.web.de Thu Jan 7 15:02:25 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Thu, 07 Jan 2010 21:02:25 +0100 Subject: PyQt QThreadPool error In-Reply-To: <7dc7a9f3-cff9-4911-bfde-6086db471b41@k17g2000yqh.googlegroups.com> References: <7dc7a9f3-cff9-4911-bfde-6086db471b41@k17g2000yqh.googlegroups.com> Message-ID: <7qmsujFc9tU2@mid.uni-berlin.de> h0uk schrieb: > Hello. > > I have the following code: > > #workers = {} > QtCore.QThreadPool.globalInstance().setExpiryTimeout > (300000) > QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) > for i in range(1, int(userscnt) + 1): > work = wk.Worker(i) > # connect signals > work.mmShowCaptcha.connect(self.show_captcha_dlg) > work.log.connect(self.handle_log) > self.captcha_answer.connect(work.mmCaptchaAnswer) > work.setAutoDelete(True) > QtCore.QThreadPool.globalInstance().start(work) > > > > On last line of code ( QtCore.QThreadPool.globalInstance().start > (work) ) i get an error: > > SystemError: error return without exception set > > What is wrong in my code??? Any advise??? The error is on C-level. AFAIK it occurs when a Python-C-function returns "NULL" without setting an exception. It's hard to say where it really occurs. I'd use a debug-build of PyQt, and possibly Python, and then investigate using gdb. Alternatively, what happens when you do some "dummy"-work that doesn't use signals and no other libraries? Diez From vmail at mycircuit.org Thu Jan 7 15:12:54 2010 From: vmail at mycircuit.org (Peter) Date: Thu, 07 Jan 2010 21:12:54 +0100 Subject: Recommended "new" way for config files In-Reply-To: <50697b2c1001071023m28e9a242uef20c6e65414c371@mail.gmail.com> References: <4B46075D.8080606@mycircuit.org> <4B460CA0.20800@sequans.com> <4B4625BC.6090201@mycircuit.org> <50697b2c1001071023m28e9a242uef20c6e65414c371@mail.gmail.com> Message-ID: <4B464046.6050505@mycircuit.org> > > >> So what is the "worshipped" approach, when you need more than name=value >> pairs ? >> > JSON is one option: http://docs.python.org/library/json.html > > > Thanks, didn't think about that, although most of the apps I know don't seem to use this approach for improved conf file handling ( ipython, pylons, etc ). To me , the ipython way ( hybrid: ipy_user_conf.py and *.ini files ) seems to be the most comprehensive way amongst the larger apps I know of, since it let you have a python coded file for what ever you might want to do during initialization and have additional .ini files, ,possibily several in different locations, for simpler options in name,value format. Has anybody experiences with other tools that use this approach ? Peter From zuo at chopin.edu.pl Thu Jan 7 15:18:27 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Thu, 07 Jan 2010 21:18:27 +0100 Subject: How to execute a script from another script and other script does notdo busy wait. In-Reply-To: <016d2fad-a925-47f4-8b76-5602a8532651@r5g2000yqb.googlegroups.com> References: <016d2fad-a925-47f4-8b76-5602a8532651@r5g2000yqb.googlegroups.com> Message-ID: Rajat wrote: > I've single CPU machine. I've a feeling that the thread created, which > would run script2, would eat up all of the CPU if I do not use sleep() > in script2. > That way, script1 would still be waiting for script2 to finish. Single CPU is not a problem for threads (in fact it's even better). It'll work. Try it. Another possibility is to run script2 in a separate process (e.g. using subprocess module). Cheers, *j From zuo at chopin.edu.pl Thu Jan 7 15:27:52 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Thu, 07 Jan 2010 21:27:52 +0100 Subject: File transfer with python In-Reply-To: References: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> Message-ID: Valentin de Pablo Fouce wrote: > On 6 ene, 22:42, "Jan Kaliszewski" wrote: >> Valentin de Pablo Fouce wrote: >> >> > Ok, I am trying to do a very quick application (is "home based" so is >> > not a big deal...). My intention is to transfer files from one >> > computer to another. >> > My intention is to be able to transfer files from one computer to >> > another in this environment. >> >> > Looking (and surfing) at internet the only suggestion given is to use >> > low level sockets for this file transfer. Is there another way to do >> > it, is there any top level library that helps you to do that? >> >> Python standard library offers tools for HTTP communication (rather >> easy to use) -- > Your solution looks quite nice...but one question, just by looking to > it I think I will need to create an HTTP server on a "file" server PC, > isn't it? Yes, in case of such communication protocols like HTTP always one side have a role of a serwer, and the other -- of a client. However, please note that files can be both downloaded *from* serwer and uploaded *to* serwer. You can also run *each* side both as a client *and* as a serwer (though it's rather unnecessary...). Cheers, *j From aahz at pythoncraft.com Thu Jan 7 15:42:56 2010 From: aahz at pythoncraft.com (Aahz) Date: 7 Jan 2010 12:42:56 -0800 Subject: Regex help needed! References: <19de1d6e-5ba9-42b5-9221-ed7246e39b4a@u36g2000prn.googlegroups.com> Message-ID: In article <19de1d6e-5ba9-42b5-9221-ed7246e39b4a at u36g2000prn.googlegroups.com>, Oltmans wrote: > >I've written this regex that's kind of working >re.findall("\w+\s*\W+amazon_(\d+)",str) > >but I was just wondering that there might be a better RegEx to do that >same thing. Can you kindly suggest a better/improved Regex. Thank you >in advance. 'Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems.' --Jamie Zawinski Take the advice other people gave you and use BeautifulSoup. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." --Red Adair From vardan.pogosyan at gmail.com Thu Jan 7 16:03:24 2010 From: vardan.pogosyan at gmail.com (h0uk) Date: Thu, 7 Jan 2010 13:03:24 -0800 (PST) Subject: PyQt QThreadPool error References: <7dc7a9f3-cff9-4911-bfde-6086db471b41@k17g2000yqh.googlegroups.com> <7qmsujFc9tU2@mid.uni-berlin.de> Message-ID: <7c781f4c-45c4-4e15-ac21-c4ca174a9a89@l30g2000yqb.googlegroups.com> On 8 ???, 01:02, "Diez B. Roggisch" wrote: > h0uk schrieb: > > > > > Hello. > > > I have the following code: > > > ? ? ? ? ? ? #workers = {} > > ? ? ? ? ? ? QtCore.QThreadPool.globalInstance().setExpiryTimeout > > (300000) > > ? ? ? ? ? ? QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) > > ? ? ? ? ? ? for i in range(1, int(userscnt) + 1): > > ? ? ? ? ? ? ? ? work = wk.Worker(i) > > ? ? ? ? ? ? ? ? # connect signals > > ? ? ? ? ? ? ? ? work.mmShowCaptcha.connect(self.show_captcha_dlg) > > ? ? ? ? ? ? ? ? work.log.connect(self.handle_log) > > ? ? ? ? ? ? ? ? self.captcha_answer.connect(work.mmCaptchaAnswer) > > ? ? ? ? ? ? ? ? work.setAutoDelete(True) > > ? ? ? ? ? ? ? ? QtCore.QThreadPool.globalInstance().start(work) > > > On last line of code ( QtCore.QThreadPool.globalInstance().start > > (work) ) i get an error: > > > SystemError: error return without exception set > > > What is wrong in my code??? Any advise??? > > The error is on C-level. AFAIK it occurs when a Python-C-function > returns "NULL" without setting an exception. > > It's hard to say where it really occurs. I'd use a debug-build of PyQt, > and possibly Python, and then investigate using gdb. > > Alternatively, what happens when you do some "dummy"-work that doesn't > use signals and no other libraries? > > Diez About some "dummy" code: #!/usr/bin/env python # -*- coding: utf-8 -*- import sys import os import time from PyQt4 import QtCore, QtGui class Job(QtCore.QRunnable): def __init__(self, name): QtCore.QRunnable.__init__(self) self._name = name def run(self): time.sleep(10) print self._name def autoDelete(self): return self._auto def setAutoDelete(self, auto): self._auto = auto if __name__ == "__main__": app = QtGui.QApplication(sys.argv) QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) j = Job("Job-1") j.setAutoDelete(True) QtCore.QThreadPool.globalInstance().start(j) Even this code not work. On the last line of code ( QtCore.QThreadPool.globalInstance().start(j) ) i get the error: >> An unhandled win32 exception occured in python.exe From mailist at sokol-web.de Thu Jan 7 16:19:38 2010 From: mailist at sokol-web.de (Kamill Sokol) Date: Thu, 7 Jan 2010 13:19:38 -0800 (PST) Subject: File transfer with python References: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> Message-ID: <3171c09a-a72b-4485-9b96-31b8b6f9e032@c34g2000yqn.googlegroups.com> On Jan 6, 7:00?pm, Valentin de Pablo Fouce wrote: > Hi there, > > My intention is to transfer files from one > computer to another. > Hi Valentin, take a look at dropbox! Clients are available for mac, linux and windows. Up and running in just 2 minutes. http://dropbox.com Regards Kamill From deets at nospam.web.de Thu Jan 7 16:25:33 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Thu, 07 Jan 2010 22:25:33 +0100 Subject: PyQt QThreadPool error In-Reply-To: <7c781f4c-45c4-4e15-ac21-c4ca174a9a89@l30g2000yqb.googlegroups.com> References: <7dc7a9f3-cff9-4911-bfde-6086db471b41@k17g2000yqh.googlegroups.com> <7qmsujFc9tU2@mid.uni-berlin.de> <7c781f4c-45c4-4e15-ac21-c4ca174a9a89@l30g2000yqb.googlegroups.com> Message-ID: <7qn1qdFcqvU1@mid.uni-berlin.de> h0uk schrieb: > On 8 ???, 01:02, "Diez B. Roggisch" wrote: >> h0uk schrieb: >> >> >> >>> Hello. >>> I have the following code: >>> #workers = {} >>> QtCore.QThreadPool.globalInstance().setExpiryTimeout >>> (300000) >>> QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) >>> for i in range(1, int(userscnt) + 1): >>> work = wk.Worker(i) >>> # connect signals >>> work.mmShowCaptcha.connect(self.show_captcha_dlg) >>> work.log.connect(self.handle_log) >>> self.captcha_answer.connect(work.mmCaptchaAnswer) >>> work.setAutoDelete(True) >>> QtCore.QThreadPool.globalInstance().start(work) >>> On last line of code ( QtCore.QThreadPool.globalInstance().start >>> (work) ) i get an error: >>> SystemError: error return without exception set >>> What is wrong in my code??? Any advise??? >> The error is on C-level. AFAIK it occurs when a Python-C-function >> returns "NULL" without setting an exception. >> >> It's hard to say where it really occurs. I'd use a debug-build of PyQt, >> and possibly Python, and then investigate using gdb. >> >> Alternatively, what happens when you do some "dummy"-work that doesn't >> use signals and no other libraries? >> >> Diez > > About some "dummy" code: > > #!/usr/bin/env python > # -*- coding: utf-8 -*- > > import sys > import os > import time > > from PyQt4 import QtCore, QtGui > > class Job(QtCore.QRunnable): > def __init__(self, name): > QtCore.QRunnable.__init__(self) > self._name = name > > def run(self): > time.sleep(10) > print self._name > > def autoDelete(self): > return self._auto > > def setAutoDelete(self, auto): > self._auto = auto > > > > if __name__ == "__main__": > > app = QtGui.QApplication(sys.argv) > > QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) > > j = Job("Job-1") > j.setAutoDelete(True) > QtCore.QThreadPool.globalInstance().start(j) > > > Even this code not work. On the last line of code > ( QtCore.QThreadPool.globalInstance().start(j) ) i get the error: > >>> An unhandled win32 exception occured in python.exe Hm. I suggest you take this to the PyQt mailinglist. Phil Thompson is very responsive. Diez From vardan.pogosyan at gmail.com Thu Jan 7 16:33:24 2010 From: vardan.pogosyan at gmail.com (h0uk) Date: Thu, 7 Jan 2010 13:33:24 -0800 (PST) Subject: PyQt QThreadPool error References: <7dc7a9f3-cff9-4911-bfde-6086db471b41@k17g2000yqh.googlegroups.com> <7qmsujFc9tU2@mid.uni-berlin.de> <7c781f4c-45c4-4e15-ac21-c4ca174a9a89@l30g2000yqb.googlegroups.com> <7qn1qdFcqvU1@mid.uni-berlin.de> Message-ID: <343c51a8-e352-4a9c-bb4d-bdc2b82ecc47@j14g2000yqm.googlegroups.com> On 8 ???, 02:25, "Diez B. Roggisch" wrote: > h0uk schrieb: > > > > > On 8 ???, 01:02, "Diez B. Roggisch" wrote: > >> h0uk schrieb: > > >>> Hello. > >>> I have the following code: > >>> ? ? ? ? ? ? #workers = {} > >>> ? ? ? ? ? ? QtCore.QThreadPool.globalInstance().setExpiryTimeout > >>> (300000) > >>> ? ? ? ? ? ? QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) > >>> ? ? ? ? ? ? for i in range(1, int(userscnt) + 1): > >>> ? ? ? ? ? ? ? ? work = wk.Worker(i) > >>> ? ? ? ? ? ? ? ? # connect signals > >>> ? ? ? ? ? ? ? ? work.mmShowCaptcha.connect(self.show_captcha_dlg) > >>> ? ? ? ? ? ? ? ? work.log.connect(self.handle_log) > >>> ? ? ? ? ? ? ? ? self.captcha_answer.connect(work.mmCaptchaAnswer) > >>> ? ? ? ? ? ? ? ? work.setAutoDelete(True) > >>> ? ? ? ? ? ? ? ? QtCore.QThreadPool.globalInstance().start(work) > >>> On last line of code ( QtCore.QThreadPool.globalInstance().start > >>> (work) ) i get an error: > >>> SystemError: error return without exception set > >>> What is wrong in my code??? Any advise??? > >> The error is on C-level. AFAIK it occurs when a Python-C-function > >> returns "NULL" without setting an exception. > > >> It's hard to say where it really occurs. I'd use a debug-build of PyQt, > >> and possibly Python, and then investigate using gdb. > > >> Alternatively, what happens when you do some "dummy"-work that doesn't > >> use signals and no other libraries? > > >> Diez > > > About some "dummy" code: > > > #!/usr/bin/env python > > # -*- coding: utf-8 -*- > > > import sys > > import os > > import time > > > from PyQt4 import QtCore, QtGui > > > class Job(QtCore.QRunnable): > > ? ?def __init__(self, name): > > ? ? ? ? ? ?QtCore.QRunnable.__init__(self) > > ? ? ? ? ? ?self._name = name > > > ? ?def run(self): > > ? ? ? ? ? ?time.sleep(10) > > ? ? ? ? ? ?print self._name > > > ? ?def autoDelete(self): > > ? ? ? ? ? ?return self._auto > > > ? ?def setAutoDelete(self, auto): > > ? ? ? ? ? ?self._auto = auto > > > if __name__ == "__main__": > > > ? ?app = QtGui.QApplication(sys.argv) > > > ? ?QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) > > > ? ?j = Job("Job-1") > > ? ?j.setAutoDelete(True) > > ? ?QtCore.QThreadPool.globalInstance().start(j) > > > Even this code not work. On the last line of code > > ( QtCore.QThreadPool.globalInstance().start(j) ) i get the error: > > >>> An unhandled win32 exception occured in python.exe > > Hm. I suggest you take this to the PyQt mailinglist. Phil Thompson is > very responsive. > > Diez Thanks you, Diez. Thanks for your time. I already wrote to PyQt mailing list, but yet no get answer. I will be waiting and try to write Phil Thompson. From phil at riverbankcomputing.com Thu Jan 7 17:02:56 2010 From: phil at riverbankcomputing.com (Phil Thompson) Date: Thu, 07 Jan 2010 22:02:56 +0000 Subject: PyQt QThreadPool error In-Reply-To: <7c781f4c-45c4-4e15-ac21-c4ca174a9a89@l30g2000yqb.googlegroups.com> References: <7dc7a9f3-cff9-4911-bfde-6086db471b41@k17g2000yqh.googlegroups.com> <7qmsujFc9tU2@mid.uni-berlin.de> <7c781f4c-45c4-4e15-ac21-c4ca174a9a89@l30g2000yqb.googlegroups.com> Message-ID: <30264a260cf2f1faafd319e080de1522@localhost> On Thu, 7 Jan 2010 13:03:24 -0800 (PST), h0uk wrote: > On 8 ???, 01:02, "Diez B. Roggisch" wrote: >> h0uk schrieb: >> >> >> >> > Hello. >> >> > I have the following code: >> >> > ? ? ? ? ? ? #workers = {} >> > ? ? ? ? ? ? QtCore.QThreadPool.globalInstance().setExpiryTimeout >> > (300000) >> > ? ? ? ? ? ? >> > QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) >> > ? ? ? ? ? ? for i in range(1, int(userscnt) + 1): >> > ? ? ? ? ? ? ? ? work = wk.Worker(i) >> > ? ? ? ? ? ? ? ? # connect signals >> > ? ? ? ? ? ? ? ? >> > work.mmShowCaptcha.connect(self.show_captcha_dlg) >> > ? ? ? ? ? ? ? ? work.log.connect(self.handle_log) >> > ? ? ? ? ? ? ? ? >> > self.captcha_answer.connect(work.mmCaptchaAnswer) >> > ? ? ? ? ? ? ? ? work.setAutoDelete(True) >> > ? ? ? ? ? ? ? ? QtCore.QThreadPool.globalInstance().start(work) >> >> > On last line of code ( QtCore.QThreadPool.globalInstance().start >> > (work) ) i get an error: >> >> > SystemError: error return without exception set >> >> > What is wrong in my code??? Any advise??? >> >> The error is on C-level. AFAIK it occurs when a Python-C-function >> returns "NULL" without setting an exception. >> >> It's hard to say where it really occurs. I'd use a debug-build of PyQt, >> and possibly Python, and then investigate using gdb. >> >> Alternatively, what happens when you do some "dummy"-work that doesn't >> use signals and no other libraries? >> >> Diez > > About some "dummy" code: > > #!/usr/bin/env python > # -*- coding: utf-8 -*- > > import sys > import os > import time > > from PyQt4 import QtCore, QtGui > > class Job(QtCore.QRunnable): > def __init__(self, name): > QtCore.QRunnable.__init__(self) > self._name = name > > def run(self): > time.sleep(10) > print self._name > > def autoDelete(self): > return self._auto > > def setAutoDelete(self, auto): > self._auto = auto > > > > if __name__ == "__main__": > > app = QtGui.QApplication(sys.argv) > > QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) > > j = Job("Job-1") > j.setAutoDelete(True) > QtCore.QThreadPool.globalInstance().start(j) > > > Even this code not work. On the last line of code > ( QtCore.QThreadPool.globalInstance().start(j) ) i get the error: > >>> An unhandled win32 exception occured in python.exe You aren't letting the thread run before exiting the program. Try adding... app.exec_() ...after you call start(). Also, I'm not sure what you are trying to achieve with your implementations of autoDelete() and setAutoDelete(). Phil From klaussfreire at gmail.com Thu Jan 7 17:18:34 2010 From: klaussfreire at gmail.com (Klauss) Date: Thu, 7 Jan 2010 14:18:34 -0800 (PST) Subject: a huge shared read-only data in parallel accesses -- How? multithreading? multiprocessing? References: <6fdd9088-7418-488a-9868-abe6446cb388@r5g2000yqb.googlegroups.com> <33b66d1e-d674-4008-88f6-45823aa74485@s3g2000yqs.googlegroups.com> Message-ID: On Dec 31 2009, 6:36?pm, garyrob wrote: > One thing I'm not clear on regarding Klauss' patch. He says it's > applicable where the data is primarily non-numeric. In trying to > understand why that would be the case, I'm thinking that the increased > per-object memory overhead for reference-counting would outweigh the > space gains from the shared memory. > > Klauss's test code stores a large number of dictionaries which each > contain just 3 items. The stored items are strings, but short ones... > it looks like they take up less space than double floats(?). > > So my understanding is that the point is that the overhead for the > dictionaries is big enough that the patch is very helpful even though > the stored items are small. And that the patch would be less and less > effective as the number of items stored in each dictionary became > greater and greater, until eventually the patch might do more use more > space for reference counting than it saved by shared memory. Not really. The real difference is that numbers (ints and floats) are allocated out of small contiguous pools. So even if a great percentage of those objects would remain read-only, there's probably holes in those pools left by the irregular access pattern during initialization, and those holes would be written to eventually as the pool gets used. In essence, those pools aren't read-only for other reasons than reference counting. Dictionaries, tuples and lists (and many other types) don't exhibit that behavior. From g.bogle at auckland.no.spam.ac.nz Thu Jan 7 17:30:53 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Fri, 08 Jan 2010 11:30:53 +1300 Subject: Threading change, 2.5.4 -> 2.6.1 Message-ID: The code below runs with Python 2.5.4, but gives the following error messages with Python 2.6.1. What needs to be done to make it work? Thanks. C:\Summer09\Tutorials>python url_queue.pyw Traceback (most recent call last): File "url_queue.pyw", line 3, in import threading File "C:\Summer09\Tutorials\threading.py", line 9, in class ProcessingThread(threading.Thread, QtCore.QObject): AttributeError: 'module' object has no attribute 'Thread' url_queue.py ------------ #!/usr/bin/env python import Queue import threading import urllib2 import time hosts = ["http://yahoo.com", "http://google.com", "http://amazon.com", "http://ibm.com", "http://apple.com"] queue = Queue.Queue() class ThreadUrl(threading.Thread): #"""Threaded Url Grab""" def __init__(self, queue,i): threading.Thread.__init__(self) self.queue = queue self.num = i print "Thread: ",self.num def run(self): while True: #grabs host from queue host = self.queue.get() print "num, host: ",self.num,host #grabs urls of hosts and prints first 1024 bytes of page url = urllib2.urlopen(host) print url.read(1024) #signals to queue job is done self.queue.task_done() start = time.time() def main(): #spawn a pool of threads, and pass them queue instance for i in range(5): t = ThreadUrl(queue,i) t.setDaemon(True) t.start() #populate queue with data for host in hosts: queue.put(host) #wait on the queue until everything has been processed queue.join() main() print "Elapsed Time: %s" % (time.time() - start) From darkrho at gmail.com Thu Jan 7 17:38:57 2010 From: darkrho at gmail.com (Rolando Espinoza La Fuente) Date: Thu, 7 Jan 2010 18:38:57 -0400 Subject: Regex help needed! In-Reply-To: References: <19de1d6e-5ba9-42b5-9221-ed7246e39b4a@u36g2000prn.googlegroups.com> Message-ID: <4eca3f41001071438vb9656ffk13d93255e333466a@mail.gmail.com> # http://gist.github.com/271661 import lxml.html import re src = """ lksjdfls
kdjff lsdfs
sdjfls
sdfsd
welcome
hello, my age is 86 years old and I was born in 1945. Do you know that PI is roughly 3.1443534534534534534 """ regex = re.compile('amazon_(\d+)') doc = lxml.html.document_fromstring(src) for div in doc.xpath('//div[starts-with(@id, "amazon_")]'): match = regex.match(div.get('id')) if match: print match.groups()[0] On Thu, Jan 7, 2010 at 4:42 PM, Aahz wrote: > In article <19de1d6e-5ba9-42b5-9221-ed7246e39b4a at u36g2000prn.googlegroups.com>, > Oltmans ? wrote: >> >>I've written this regex that's kind of working >>re.findall("\w+\s*\W+amazon_(\d+)",str) >> >>but I was just wondering that there might be a better RegEx to do that >>same thing. Can you kindly suggest a better/improved Regex. Thank you >>in advance. > > 'Some people, when confronted with a problem, think "I know, I'll use > regular expressions." ?Now they have two problems.' > --Jamie Zawinski > > Take the advice other people gave you and use BeautifulSoup. > -- > Aahz (aahz at pythoncraft.com) ? ? ? ? ? <*> ? ? ? ? http://www.pythoncraft.com/ > > "If you think it's expensive to hire a professional to do the job, wait > until you hire an amateur." ?--Red Adair > -- > http://mail.python.org/mailman/listinfo/python-list > -- Rolando Espinoza La fuente www.rolandoespinoza.info From MLMDev at Comcast.net Thu Jan 7 17:47:13 2010 From: MLMDev at Comcast.net (Mitchell L Model) Date: Thu, 7 Jan 2010 17:47:13 -0500 Subject: One function calling another defined in the same file being exec'd Message-ID: <67CD7A9A-2446-4CE3-AAF6-8F737FA5A68D@Comcast.net> [Python 3.1] I thought I thoroughly understood eval, exec, globals, and locals, but I encountered something bewildering today. I have some short files I want to exec. (Users of my application write them, and the application gives them a command that opens a file dialog box and execs the chosen file. Users are expected to be able to write simple Python scripts, including function definitions. Neither security nor errors are relevant for the purposes of this discussion, though I do deal with them in my actual code.) Here is a short piece of code to exec a file and report its result. (The file being exec'd must assign 'result'.) def dofile(filename): ldict = {'result': None} with open(filename) as file: exec(file.read(), globals(), ldict) print('Result for {}: {}'.format(filename, ldict['result'])) First I call dofile() on a file containing the following: ################################ def fn(arg): return sum(range(arg)) result = fn(5) ################################ The results are as expected. Next I call dofile() on a slightly more complex file, in which one function calls another function defined earlier in the same file. ################################ def fn1(val): return sum(range(val)) def fn2(arg): return fn1(arg) result = fn2(5) ################################ This produces a surprise: NameError: global name 'fn1' is not defined [1] How is it that fn2 can be called from the top-level of the script but fn1 cannot be called from fn2? [2] Is this correct behavior or is there something wrong with Python here? [3] How should I write a file to be exec'd that defines several functions that call each other, as in the trivial fn1-fn2 example above? From anon at blank.com Thu Jan 7 17:55:40 2010 From: anon at blank.com (anon) Date: Thu, 07 Jan 2010 22:55:40 GMT Subject: Threading change, 2.5.4 -> 2.6.1 In-Reply-To: References: Message-ID: Gib Bogle wrote: > The code below runs with Python 2.5.4, but gives the following error > messages with Python 2.6.1. What needs to be done to make it work? > Thanks. > > C:\Summer09\Tutorials>python url_queue.pyw > Traceback (most recent call last): > File "url_queue.pyw", line 3, in > import threading > File "C:\Summer09\Tutorials\threading.py", line 9, in > class ProcessingThread(threading.Thread, QtCore.QObject): > AttributeError: 'module' object has no attribute 'Thread' > Nothing to do with the version of python You have a file called threading.py in C:\Summer09\Tutorials (the same folder as url_queue.pyw). The line 'import threading' is finding this module before python's threading module. Rename this file and you should be fine. From Lee at JamToday.com Thu Jan 7 17:58:37 2010 From: Lee at JamToday.com (Lee) Date: Thu, 07 Jan 2010 17:58:37 -0500 Subject: Dictionary used to build a Triple Store In-Reply-To: References: Message-ID: Lee wrote: > Definitely a newbie question, so please bear with me. > > I'm reading "Programming the Semantic Web" by Segaran, Evans, and Tayor. > > It's about the Semantic Web BUT it uses python to build a "toy" triple > store claimed to have good performance in the "tens of thousands" of > triples. > > Just in case anybody doesnt know what an RDF triple is (not that it > matters for my question) think of it as an ordered 3 tuple representing > a Subject, a Predicate, and an Object eg: (John, loves, Mary) (Mary, > has-a, lamb) {theSky, has-color,blue} > > To build the triple store entirely in Python, the authors recommend > using the Python hash. Three hashes actually (I get that. You > want to have a hash with the major index being the Subject in one hash, > the Predicate in another hash, or the Object for the third hash) > > He creates a class SimpleGraph which initializes itself by setting the > three hashes names _spo, _pos, and _osp thus > > class SimpleGraph; > def __init__(self); > self._spo={}; > self._pos=(); > self._osp={}; > > So far so good. I get the convention with the double underbars for the > initializer but > > Q1: Not the main question but while I'm here....I'm a little fuzzy on > the convention about the use of the single underbar in the definition of > the hashes. Id the idea to "underbar" all objects and methods that > belong to the class? Why do that? > > But now the good stuff: > > Our authors define the hashes thus: (showing only one of the three > hashes because they're all the same idea) > > self._pos = {predicate:{object:set( [subject] ) }} > > Q2: Wha? Two surprises ... > 1) Why not {predicate:{object:subject}} i.e. > pos[predicate][object]=subject....why the set( [object] ) construct? > putting the object into a list and turning the list into a set to be the > "value" part of a name:value pair. Why not just use the naked subject > for the value? > > 2) Why not something like pos[predicate][object][subject] = 1 .....or > any constant. The idea being to create the set of three indexes. If the > triple exists in the hash, its "in" your tripple store. If not, then > there's no such triple. > > OK, Thanx. That cleares things up. I had forgotton that once you have a "Cell" in a 2D matrix to represent the first two components of the 3 tuple, you then want multiple values IN the cell for the possibly multi valued third component. From python at mrabarnett.plus.com Thu Jan 7 18:01:52 2010 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 07 Jan 2010 23:01:52 +0000 Subject: Threading change, 2.5.4 -> 2.6.1 In-Reply-To: References: Message-ID: <4B4667E0.8070101@mrabarnett.plus.com> Gib Bogle wrote: > The code below runs with Python 2.5.4, but gives the following error > messages with Python 2.6.1. What needs to be done to make it work? > Thanks. > > C:\Summer09\Tutorials>python url_queue.pyw > Traceback (most recent call last): > File "url_queue.pyw", line 3, in > import threading > File "C:\Summer09\Tutorials\threading.py", line 9, in > class ProcessingThread(threading.Thread, QtCore.QObject): > AttributeError: 'module' object has no attribute 'Thread' > [snip] url_queue.pyw is trying to import Python's "threading" module, but it's finding the threading.py script in your "Tutorials" folder first. Renaming your tutorial script to something other than "threading.py". From vardan.pogosyan at gmail.com Thu Jan 7 18:07:10 2010 From: vardan.pogosyan at gmail.com (h0uk) Date: Thu, 7 Jan 2010 15:07:10 -0800 (PST) Subject: PyQt QThreadPool error References: <7dc7a9f3-cff9-4911-bfde-6086db471b41@k17g2000yqh.googlegroups.com> <7qmsujFc9tU2@mid.uni-berlin.de> <7c781f4c-45c4-4e15-ac21-c4ca174a9a89@l30g2000yqb.googlegroups.com> Message-ID: On 8 ???, 03:02, Phil Thompson wrote: > On Thu, 7 Jan 2010 13:03:24 -0800 (PST), h0uk > wrote: > > > On 8 ???, 01:02, "Diez B. Roggisch" wrote: > >> h0uk schrieb: > > >> > Hello. > > >> > I have the following code: > > >> > ? ? ? ? ? ? #workers = {} > >> > ? ? ? ? ? ? QtCore.QThreadPool.globalInstance().setExpiryTimeout > >> > (300000) > >> > ? ? ? ? ? ? > >> > QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) > >> > ? ? ? ? ? ? for i in range(1, int(userscnt) + 1): > >> > ? ? ? ? ? ? ? ? work = wk.Worker(i) > >> > ? ? ? ? ? ? ? ? # connect signals > >> > ? ? ? ? ? ? ? ? > >> > work.mmShowCaptcha.connect(self.show_captcha_dlg) > >> > ? ? ? ? ? ? ? ? work.log.connect(self.handle_log) > >> > ? ? ? ? ? ? ? ? > >> > self.captcha_answer.connect(work.mmCaptchaAnswer) > >> > ? ? ? ? ? ? ? ? work.setAutoDelete(True) > >> > ? ? ? ? ? ? ? ? > > QtCore.QThreadPool.globalInstance().start(work) > > > > > > >> > On last line of code ( QtCore.QThreadPool.globalInstance().start > >> > (work) ) i get an error: > > >> > SystemError: error return without exception set > > >> > What is wrong in my code??? Any advise??? > > >> The error is on C-level. AFAIK it occurs when a Python-C-function > >> returns "NULL" without setting an exception. > > >> It's hard to say where it really occurs. I'd use a debug-build of PyQt, > >> and possibly Python, and then investigate using gdb. > > >> Alternatively, what happens when you do some "dummy"-work that doesn't > >> use signals and no other libraries? > > >> Diez > > > About some "dummy" code: > > > #!/usr/bin/env python > > # -*- coding: utf-8 -*- > > > import sys > > import os > > import time > > > from PyQt4 import QtCore, QtGui > > > class Job(QtCore.QRunnable): > > ? ?def __init__(self, name): > > ? ? ? ? ? ?QtCore.QRunnable.__init__(self) > > ? ? ? ? ? ?self._name = name > > > ? ?def run(self): > > ? ? ? ? ? ?time.sleep(10) > > ? ? ? ? ? ?print self._name > > > ? ?def autoDelete(self): > > ? ? ? ? ? ?return self._auto > > > ? ?def setAutoDelete(self, auto): > > ? ? ? ? ? ?self._auto = auto > > > if __name__ == "__main__": > > > ? ?app = QtGui.QApplication(sys.argv) > > > ? ?QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) > > > ? ?j = Job("Job-1") > > ? ?j.setAutoDelete(True) > > ? ?QtCore.QThreadPool.globalInstance().start(j) > > > Even this code not work. On the last line of code > > ( QtCore.QThreadPool.globalInstance().start(j) ) i get the error: > > >>> An unhandled win32 exception occured in python.exe > > You aren't letting the thread run before exiting the program. Try adding... > > ? ? app.exec_() > > ...after you call start(). > > Also, I'm not sure what you are trying to achieve with your implementations > of autoDelete() and setAutoDelete(). > > Phil Phil you right about app.exec_(). But situation is sligthly different. I want to have more than one Job. I add these Jobs into QThreadPool trough cycle. And I also want these Jobs to run sequentially. The following code illustrate what I mean: #!/usr/bin/env python # -*- coding: utf-8 -*- import sys import os import time from PyQt4 import QtCore, QtGui class Job(QtCore.QRunnable): def __init__(self, name): QtCore.QRunnable.__init__(self) self._name = name def run(self): time.sleep(3) print self._name if __name__ == "__main__": app = QtGui.QApplication(sys.argv) QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) for i in range(5): j = Job("Job-" + str(i)) j.setAutoDelete(True) QtCore.QThreadPool.globalInstance().start(j, i) app.exec_() After 5 cycle I get the same error: An unhandled win32 exception occured in python.exe. How I can do it?? To run my Jobs sequentially??? Vardan. From vmail at mycircuit.org Thu Jan 7 18:13:16 2010 From: vmail at mycircuit.org (Peter) Date: Fri, 08 Jan 2010 00:13:16 +0100 Subject: restructuredText editor ? Message-ID: <4B466A8C.2020609@mycircuit.org> What editor do people out there use to edit .rst files for sphinx-python documentation ? Peter From mailtome200420032002 at gmail.com Thu Jan 7 18:23:48 2010 From: mailtome200420032002 at gmail.com (aj) Date: Thu, 7 Jan 2010 15:23:48 -0800 (PST) Subject: Accessing python from a network share in windows 7 Message-ID: I access python from a network share. This works fine on XP but on windows 7 it throws the following error: Python 2.6.1 (r261:67517, Dec 4 2008, 16:51:00) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import random Traceback (most recent call last): File "", line 1, in File "t:\win32\python-2.6.1\lib\random.py", line 871, in _inst = Random() File "t:\win32\python-2.6.1\lib\random.py", line 96, in __init__ self.seed(x) File "t:\win32\python-2.6.1\lib\random.py", line 110, in seed a = long(_hexlify(_urandom(16)), 16) WindowsError: [Error 127] The specified procedure could not be found Is there some security policy that I need to enable/disable to use python from a network on windows 7? From MLMDev at Comcast.net Thu Jan 7 18:34:57 2010 From: MLMDev at Comcast.net (Mitchell L Model) Date: Thu, 7 Jan 2010 18:34:57 -0500 Subject: One function calling another defined in the same file being exec'd Message-ID: I forgot to offer one answer for question [3] in what I just posted: I can define all the secondary functions inside one main one and just call the main one. That provides a separate local scope within the main function, with the secondary functions defined inside it when (each time) the main function is called. Not too bad, but will freak out my users and it doesn't seem as if it should be necessary to resort to this. From anon at blank.com Thu Jan 7 18:39:45 2010 From: anon at blank.com (anon) Date: Thu, 07 Jan 2010 23:39:45 GMT Subject: One function calling another defined in the same file being exec'd In-Reply-To: References: Message-ID: <5hu1n.23389$Ym4.557@text.news.virginmedia.com> Rather than exec the files, why not import them? I can get both your examples to work using the 'imp' module. http://docs.python.org/3.1/library/imp.html#module-imp I used python 2.6.4. Note that 3.1 also has 'importlib' module. import imp # the name of the python file written by a user name = 'test1' fp, pathname, description = imp.find_module(name) test1 = imp.load_module(name, fp, pathname, description) print test1.result # remember to close file (see docs) fp.close() From python at mrabarnett.plus.com Thu Jan 7 18:51:50 2010 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 07 Jan 2010 23:51:50 +0000 Subject: Accessing python from a network share in windows 7 In-Reply-To: References: Message-ID: <4B467396.3080009@mrabarnett.plus.com> aj wrote: > I access python from a network share. This works fine on XP but on > windows 7 it throws the following error: > > Python 2.6.1 (r261:67517, Dec 4 2008, 16:51:00) [MSC v.1500 32 bit > (Intel)] on > win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> import random > Traceback (most recent call last): > File "", line 1, in > File "t:\win32\python-2.6.1\lib\random.py", line 871, in > _inst = Random() > File "t:\win32\python-2.6.1\lib\random.py", line 96, in __init__ > self.seed(x) > File "t:\win32\python-2.6.1\lib\random.py", line 110, in seed > a = long(_hexlify(_urandom(16)), 16) > WindowsError: [Error 127] The specified procedure could not be found > > Is there some security policy that I need to enable/disable to use > python from a network on windows 7? Is it a problem with the share or with Windows 7? Does it work with Windows 7 when running a local copy of Python? From mailtome200420032002 at gmail.com Thu Jan 7 19:01:05 2010 From: mailtome200420032002 at gmail.com (aj) Date: Thu, 7 Jan 2010 16:01:05 -0800 (PST) Subject: Accessing python from a network share in windows 7 References: Message-ID: <08ac1875-8747-4225-bf78-21972e959386@k9g2000vbl.googlegroups.com> On Jan 7, 3:51?pm, MRAB wrote: > aj wrote: > > I access python from a network share. This works fine on XP but on > > windows 7 it throws the following error: > > > Python 2.6.1 (r261:67517, Dec ?4 2008, 16:51:00) [MSC v.1500 32 bit > > (Intel)] on > > win32 > > Type "help", "copyright", "credits" or "license" for more information. > >>>> import random > > Traceback (most recent call last): > > ? File "", line 1, in > > ? File "t:\win32\python-2.6.1\lib\random.py", line 871, in > > ? ? _inst = Random() > > ? File "t:\win32\python-2.6.1\lib\random.py", line 96, in __init__ > > ? ? self.seed(x) > > ? File "t:\win32\python-2.6.1\lib\random.py", line 110, in seed > > ? ? a = long(_hexlify(_urandom(16)), 16) > > WindowsError: [Error 127] The specified procedure could not be found > > > Is there some security policy that I need to enable/disable to use > > python from a network on windows 7? > > Is it a problem with the share or with Windows 7? Does it work with > Windows 7 when running a local copy of Python? It works without any issue on win7 if I copy python to my local drive. Also, accessing python from the same network share works fine on win XP. So I am suspecting some security policy of win7 that is causing problem while accessing it over a network share. From suresh.amritapuri at gmail.com Thu Jan 7 19:14:45 2010 From: suresh.amritapuri at gmail.com (suresh.amritapuri) Date: Thu, 7 Jan 2010 16:14:45 -0800 (PST) Subject: PIL show() not working for 2nd pic Message-ID: Hi I am using PIL for image processing in ubuntu 9.04. When i give two im.show() commands for two different images, the second image is not displayed (eye of gnome is the display program). It says no such file or directory. Any ideas? thanks suresh From g.bogle at auckland.no.spam.ac.nz Thu Jan 7 20:14:59 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Fri, 08 Jan 2010 14:14:59 +1300 Subject: Threading change, 2.5.4 -> 2.6.1 References: Message-ID: MRAB wrote: > Gib Bogle wrote: >> The code below runs with Python 2.5.4, but gives the following error >> messages with Python 2.6.1. What needs to be done to make it work? >> Thanks. >> >> C:\Summer09\Tutorials>python url_queue.pyw >> Traceback (most recent call last): >> File "url_queue.pyw", line 3, in >> import threading >> File "C:\Summer09\Tutorials\threading.py", line 9, in >> class ProcessingThread(threading.Thread, QtCore.QObject): >> AttributeError: 'module' object has no attribute 'Thread' >> > [snip] > url_queue.pyw is trying to import Python's "threading" module, but it's > finding the threading.py script in your "Tutorials" folder first. > > Renaming your tutorial script to something other than "threading.py". Thanks very much to you both. It's actually not in my folder, it is (presumably) in my student's folder (I'm the one using 2.5.4, she has 2.6.1 and thought this could be the problem.) From no.i.dont at want.mail.from.spammers.com Thu Jan 7 20:21:01 2010 From: no.i.dont at want.mail.from.spammers.com (Fencer) Date: Fri, 08 Jan 2010 02:21:01 +0100 Subject: How do I access what's in this module? Message-ID: <7qnfk0Fr6kU1@mid.individual.net> Hello, look at this lxml documentation page: http://codespeak.net/lxml/api/index.html How do I access the functions and variables listed? I tried from lxml.etree import ElementTree and the import itself seems to pass without complaint by the python interpreter but I can't seem to access anything in ElementTree, not the functions or variables. What is the proper way to import that module? For example: >>> from lxml.etree import ElementTree >>> ElementTree.dump(None) Traceback (most recent call last): File "", line 1, in Also, can I access those items that begin with an underscore if I get the import sorted? - Fencer From apt.shansen at gmail.com Thu Jan 7 20:39:30 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Thu, 7 Jan 2010 17:39:30 -0800 Subject: Where's a DOM builder that uses the Builder Pattern to ... build DOMs? In-Reply-To: References: <04523152-a0ff-432c-a48f-2c653577e7b7@m25g2000yqc.googlegroups.com> <4b42f551$0$7625$9b4e6d93@newsspool1.arcor-online.net> <026df5db-bcef-424d-8cd5-56dacebf01f8@c3g2000yqd.googlegroups.com> <4b45e370$0$6736$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <7a9c25c21001071739t5e2f2689vaf9452ec67784964@mail.gmail.com> On Thu, Jan 7, 2010 at 8:44 AM, Phlip wrote: > On Jan 7, 5:36 am, Stefan Behnel wrote: > > > Well, then note that there are tons of ways to generate XML with Python, > > including the one I pointed you to. > > from lxml.html import builder as E > xml = E.foo() > > All I want is "", but I get "AttributeError: 'module' object has > no attribute 'foo'". > > A peek at dir(E) shows it only has HTML tags, all hard coded. > > So how to get it to generate any random XML tag my clients think of? > If you want to generate random XML, don't use the HTML sub-module of lxml. Its a specific sub-set of functionality for HTML only documents. >>> from lxml.builder import ElementMaker >>> from lxml import etree >>> E = ElementMaker() >>> html = E.html( ... E.form( ... E.input(type="text", name="email"), ... E.input(type="text", name="blah") ... ) ... ) >>> etree.tostring(html) '
' HTH, --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at REMOVE-THIS-cybersource.com.au Thu Jan 7 20:57:53 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 08 Jan 2010 01:57:53 GMT Subject: One function calling another defined in the same file being exec'd References: Message-ID: <03567e01$0$1336$c3e8da3@news.astraweb.com> On Thu, 07 Jan 2010 17:47:13 -0500, Mitchell L Model wrote: > Next I call dofile() on a slightly more complex file, in which one > function calls another function defined earlier in the same file. > > ################################ > def fn1(val): > return sum(range(val)) > > def fn2(arg): > return fn1(arg) > > result = fn2(5) > ################################ > > This produces a surprise: > > NameError: global name 'fn1' is not defined > > [1] How is it that fn2 can be called from the top-level of the script > but fn1 cannot be called from fn2? This might help you to see what's going on. Define your own cut-down version of the global namespace, and a local namespace, and a string to execute: myglobals = {'__builtins__': None, 'globals': globals, 'locals': locals, 'print': print} mylocals = {'result': None} s = """def f(): print("Globals inside f:", globals()) print("Locals inside f:", locals()) print("Globals at the top level:", globals()) print("Locals at the top level:", locals()) f() """ exec(s, myglobals, mylocals) And this is what you should see: Globals at the top level: {'__builtins__': None, 'print': , 'globals': , 'locals': } Locals at the top level: {'result': None, 'f': } Globals inside f: {'__builtins__': None, 'print': , 'globals': , 'locals': } Locals inside f: {} Does that clarify what's going on? > [2] Is this correct behavior or is there something wrong with Python > here? This certainly surprised me too. I don't know if it is correct or not, but it goes back to at least Python 2.5. > [3] How should I write a file to be exec'd that defines several > functions that call each other, as in the trivial fn1-fn2 example above? My preference would be to say, don't use exec, just import the module. Put responsibility on the user to ensure that they set a global "result", and then just do this: mod = __import__('user_supplied_file_name') result = mod.result But if that's unworkable for you, then try simulating the namespace setup at the top level of a module. The thing to remember is that in the top level of a module: >>> globals() is locals() True so let's simulate that: myglobals = {'result': None} # You probably also want __builtins__ s = """def f(): return g() + 1 def g(): return 2 result = f() """ exec(s, myglobals, myglobals) myglobals['result'] This works for me. -- Steven From sjmachin at lexicon.net Thu Jan 7 22:40:26 2010 From: sjmachin at lexicon.net (John Machin) Date: Thu, 7 Jan 2010 19:40:26 -0800 (PST) Subject: How do I access what's in this module? References: <7qnfk0Fr6kU1@mid.individual.net> Message-ID: <29a775c4-223f-4e74-b739-12d633e5739e@a21g2000yqc.googlegroups.com> On Jan 8, 12:21?pm, Fencer wrote: > Hello, look at this lxml documentation page:http://codespeak.net/lxml/api/index.html That's for getting details about an object once you know what object you need to use to do what. In the meantime, consider reading the tutorial and executing some of the examples: http://codespeak.net/lxml/tutorial.html > How do I access the functions and variables listed? > > I tried from lxml.etree import ElementTree and the import itself seems > to pass without complaint by the python interpreter but I can't seem to > access anything in ElementTree, not the functions or variables. What is > the proper way to import that module? > > For example: > ?>>> from lxml.etree import ElementTree > ?>>> ElementTree.dump(None) > Traceback (most recent call last): > ? ?File "", line 1, in lxml.etree is a module. ElementTree is effectively a class. The error message that you omitted to show us might have given you a clue. To save keystrokes you may like to try from lxml import etree as ET and thereafter refer to the module as "ET" | >>> from lxml import etree as ET | >>> type(ET) | | >>> type(ET.ElementTree) | | >>> help(ET.ElementTree) | Help on built-in function ElementTree in module lxml.etree: | | ElementTree(...) | ElementTree(element=None, file=None, parser=None) | | ElementTree wrapper class. > Also, can I access those items that begin with an underscore if I get > the import sorted? Using pommy slang like "sorted" in an IT context has the potential to confuse your transatlantic correspondents :-) Can access? Yes. Should access? The usual Python convention is that an object whose name begins with an underscore should be accessed only via a documented interface (or, at your own risk, if you think you know what you are doing). HTH, John From WaterLin at ymail.invalid Thu Jan 7 22:44:48 2010 From: WaterLin at ymail.invalid (Water Lin) Date: Fri, 08 Jan 2010 11:44:48 +0800 Subject: Ask how to use HTMLParser Message-ID: <87vdfdtg3z.fsf@ymail.invalid> I am a new guy to use Python, but I want to parse a html page now. I tried to use HTMLParse. Here is my sample code: ---------------------- from HTMLParser import HTMLParser from urllib2 import urlopen class MyParser(HTMLParser): title = "" is_title = "" def __init__(self, url): HTMLParser.__init__(self) req = urlopen(url) self.feed(req.read()) def handle_starttag(self, tag, attrs): if tag == 'div' and attrs[0][1] == 'articleTitle': print "Found link => %s" % attrs[0][1] self.is_title = 1 def handle_data(self, data): if self.is_title: print "here" self.title = data print self.title self.is_title = 0 ----------------------- For the tag -------
open article title
------- I use my code to parse it. I can locate the div tag but I don't know how to get the text for the tag which is "open article title" in my example. How can I get the html content? What's wrong in my handle_data function? Thanks Water Lin -- Water Lin's notes and pencils: http://en.waterlin.org Email: WaterLin at ymail.com From no.i.dont at want.mail.from.spammers.com Thu Jan 7 22:45:19 2010 From: no.i.dont at want.mail.from.spammers.com (Fencer) Date: Fri, 08 Jan 2010 04:45:19 +0100 Subject: How do I access what's in this module? In-Reply-To: <29a775c4-223f-4e74-b739-12d633e5739e@a21g2000yqc.googlegroups.com> References: <7qnfk0Fr6kU1@mid.individual.net> <29a775c4-223f-4e74-b739-12d633e5739e@a21g2000yqc.googlegroups.com> Message-ID: <7qno2iF1b7U1@mid.individual.net> On 2010-01-08 04:40, John Machin wrote: >> >> For example: >> >>> from lxml.etree import ElementTree >> >>> ElementTree.dump(None) >> Traceback (most recent call last): >> File "", line 1, in > > lxml.etree is a module. ElementTree is effectively a class. The error > message that you omitted to show us might have given you a clue. But I did show the error message? It's just above what you just wrote. I try to include all relevant information in my posts. > > Using pommy slang like "sorted" in an IT context has the potential to > confuse your transatlantic correspondents :-) Ah, of course! :-) > > Can access? Yes. Should access? The usual Python convention is that an > object whose name begins with an underscore should be accessed only > via a documented interface (or, at your own risk, if you think you > know what you are doing). It turns out I no longer want to access anything in there but I thank you for your information nontheless. > > HTH, > John - Fencer From sjmachin at lexicon.net Thu Jan 7 23:01:00 2010 From: sjmachin at lexicon.net (John Machin) Date: Thu, 7 Jan 2010 20:01:00 -0800 (PST) Subject: How do I access what's in this module? References: <7qnfk0Fr6kU1@mid.individual.net> <29a775c4-223f-4e74-b739-12d633e5739e@a21g2000yqc.googlegroups.com> <7qno2iF1b7U1@mid.individual.net> Message-ID: On Jan 8, 2:45?pm, Fencer wrote: > On 2010-01-08 04:40, John Machin wrote: > > > > >> For example: > >> ? >>> ?from lxml.etree import ElementTree > >> ? >>> ?ElementTree.dump(None) > >> Traceback (most recent call last): > >> ? ? File "", line 1, in > > > lxml.etree is a module. ElementTree is effectively a class. The error > > message that you omitted to show us might have given you a clue. > > But I did show the error message? It's just above what you just wrote. I > try to include all relevant information in my posts. Traceback (most recent call last): File "", line 1, in Also, can I access those items ... Error message should appear after line starting with "File". Above excerpt taken from google groups; identical to what shows in http://news.gmane.org/gmane.comp.python.general ... what are you looking at? With Windows XP and Python 2.5.4 I get: Traceback (most recent call last): File "", line 1, in AttributeError: 'builtin_function_or_method' object has no attribute 'dump' > It turns out I no longer want to access anything in there but I thank > you for your information nontheless. You're welcome -- the advice on _methods is portable :-) From vardan.pogosyan at gmail.com Thu Jan 7 23:17:17 2010 From: vardan.pogosyan at gmail.com (h0uk) Date: Thu, 7 Jan 2010 20:17:17 -0800 (PST) Subject: Ask how to use HTMLParser References: <87vdfdtg3z.fsf@ymail.invalid> Message-ID: <395c651a-65f5-4ceb-a220-58eabc067c71@21g2000yqj.googlegroups.com> On 8 ???, 08:44, Water Lin wrote: > I am a new guy to use Python, but I want to parse a html page now. I > tried to use HTMLParse. Here is my sample code: > ---------------------- > from HTMLParser import HTMLParser > from urllib2 import urlopen > > class MyParser(HTMLParser): > ? ? title = "" > ? ? is_title = "" > ? ? def __init__(self, url): > ? ? ? ? HTMLParser.__init__(self) > ? ? ? ? req = urlopen(url) > ? ? ? ? self.feed(req.read()) > > ? ? def handle_starttag(self, tag, attrs): > ? ? ? ? if tag == 'div' and attrs[0][1] == 'articleTitle': > ? ? ? ? ? ? print "Found link => %s" % attrs[0][1] > ? ? ? ? ? ? self.is_title = 1 > > ? ? def handle_data(self, data): > ? ? ? ? if self.is_title: > ? ? ? ? ? ? print "here" > ? ? ? ? ? ? self.title = data > ? ? ? ? ? ? print self.title > ? ? ? ? ? ? self.is_title = 0 > ----------------------- > > For the tag > ------- >
open article title
> ------- > > I use my code to parse it. I can locate the div tag but I don't know how > to get the text for the tag which is "open article title" in my example. > > How can I get the html content? What's wrong in my handle_data function? > > Thanks > > Water Lin > > -- > Water Lin's notes and pencils:http://en.waterlin.org > Email: Water... at ymail.com Hi. Have you get errors or anything else??? What is wrong?? Vardan. From vardan.pogosyan at gmail.com Thu Jan 7 23:42:58 2010 From: vardan.pogosyan at gmail.com (h0uk) Date: Thu, 7 Jan 2010 20:42:58 -0800 (PST) Subject: Ask how to use HTMLParser References: <87vdfdtg3z.fsf@ymail.invalid> Message-ID: On 8 ???, 08:44, Water Lin wrote: > I am a new guy to use Python, but I want to parse a html page now. I > tried to use HTMLParse. Here is my sample code: > ---------------------- > from HTMLParser import HTMLParser > from urllib2 import urlopen > > class MyParser(HTMLParser): > ? ? title = "" > ? ? is_title = "" > ? ? def __init__(self, url): > ? ? ? ? HTMLParser.__init__(self) > ? ? ? ? req = urlopen(url) > ? ? ? ? self.feed(req.read()) > > ? ? def handle_starttag(self, tag, attrs): > ? ? ? ? if tag == 'div' and attrs[0][1] == 'articleTitle': > ? ? ? ? ? ? print "Found link => %s" % attrs[0][1] > ? ? ? ? ? ? self.is_title = 1 > > ? ? def handle_data(self, data): > ? ? ? ? if self.is_title: > ? ? ? ? ? ? print "here" > ? ? ? ? ? ? self.title = data > ? ? ? ? ? ? print self.title > ? ? ? ? ? ? self.is_title = 0 > ----------------------- > > For the tag > ------- >
open article title
> ------- > > I use my code to parse it. I can locate the div tag but I don't know how > to get the text for the tag which is "open article title" in my example. > > How can I get the html content? What's wrong in my handle_data function? > > Thanks > > Water Lin > > -- > Water Lin's notes and pencils:http://en.waterlin.org > Email: Water... at ymail.com I want to say your code works well From steve at REMOVE-THIS-cybersource.com.au Thu Jan 7 23:56:18 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 08 Jan 2010 04:56:18 GMT Subject: Need help to pass self.count to other classes. References: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> Message-ID: <0356a7d1$0$1336$c3e8da3@news.astraweb.com> On Wed, 06 Jan 2010 08:56:23 -0500, Steve Holden wrote: > This is untested code (some days I don't seem to write any other kind > ...) but it should give you the flavor: > > class kbInterface(object): > def __init__(self): > self.zxc = 0 > def prompt1(self): > self.count += 1 > return "[%d]> " > def prompt2(self): > l = len(str(self.count))+1 > return "%s " % "."*l > def dhook(self, value): > print "[%d out]" % self.count > def ehook(self, type, value, trace): > print "[%d err]\n" % value > > kbi = kbInterface() > sys.ps1 = kbi.prompt1 > sys.ps2 = kbi.prompt2 > sys.displayhook = kbi.dhook > sys.excepthook = kbi.ehook Unfortunately this won't do what you expect, because sys.ps1 and ps2 should be either strings, or objects with a __str__ method. They aren't called to generate the prompt. (After fixing the typo with self.count vs self.zxc) >>> kbi = kbInterface() >>> sys.ps1 = kbi.prompt1 >print "Hello" Hello > -- Steven From pete at shinners.org Fri Jan 8 01:19:44 2010 From: pete at shinners.org (peteshinners) Date: Thu, 7 Jan 2010 22:19:44 -0800 (PST) Subject: Clarifications on compiling for Windows Message-ID: <1f0b409b-f943-4b1a-a806-87b834337367@p8g2000yqb.googlegroups.com> My presentation for Pycon is coming together, but I need to make sure my information about compiling Python and Python extensions for Windows is correct. I'm really only experienced with this on the Linux side of things. First of all, is the Windows FAQ fairly up to date? Should people be referring to section 6 if they are going to build an application with an embedded Python interpreter? http://www.python.org/doc/faq/windows/#how-can-i-embed-python-into-a-windows-application If I understand correctly, compiled extensions for Python on Windows should match the compiler that was used to build the interpreter itself? Is there a list somewhere that shows which version of msvc was used to compile the recent Python binaries? Thank you for feedback. I definitely want to make sure I have this correct before telling anybody else? From WaterLin at ymail.invalid Fri Jan 8 01:44:16 2010 From: WaterLin at ymail.invalid (Water Lin) Date: Fri, 08 Jan 2010 14:44:16 +0800 Subject: Ask how to use HTMLParser References: <87vdfdtg3z.fsf@ymail.invalid> Message-ID: <87k4vtrt8f.fsf@ymail.invalid> h0uk writes: > On 8 ???, 08:44, Water Lin wrote: >> I am a new guy to use Python, but I want to parse a html page now. I >> tried to use HTMLParse. Here is my sample code: >> ---------------------- >> from HTMLParser import HTMLParser >> from urllib2 import urlopen >> >> class MyParser(HTMLParser): >> ? ? title = "" >> ? ? is_title = "" >> ? ? def __init__(self, url): >> ? ? ? ? HTMLParser.__init__(self) >> ? ? ? ? req = urlopen(url) >> ? ? ? ? self.feed(req.read()) >> >> ? ? def handle_starttag(self, tag, attrs): >> ? ? ? ? if tag == 'div' and attrs[0][1] == 'articleTitle': >> ? ? ? ? ? ? print "Found link => %s" % attrs[0][1] >> ? ? ? ? ? ? self.is_title = 1 >> >> ? ? def handle_data(self, data): >> ? ? ? ? if self.is_title: >> ? ? ? ? ? ? print "here" >> ? ? ? ? ? ? self.title = data >> ? ? ? ? ? ? print self.title >> ? ? ? ? ? ? self.is_title = 0 >> ----------------------- >> >> For the tag >> ------- >>
open article title
>> ------- >> >> I use my code to parse it. I can locate the div tag but I don't know how >> to get the text for the tag which is "open article title" in my example. >> >> How can I get the html content? What's wrong in my handle_data function? >> >> Thanks >> >> Water Lin >> >> -- >> Water Lin's notes and pencils:http://en.waterlin.org >> Email: Water... at ymail.com > > I want to say your code works well But in handle_data I can't print self.title. I don't why I can't set the self.title in handle_data. Thanks Water Lin -- Water Lin's notes and pencils: http://en.waterlin.org Email: WaterLin at ymail.com From mensanator at aol.com Fri Jan 8 01:51:26 2010 From: mensanator at aol.com (Mensanator) Date: Thu, 7 Jan 2010 22:51:26 -0800 (PST) Subject: Clarifications on compiling for Windows References: <1f0b409b-f943-4b1a-a806-87b834337367@p8g2000yqb.googlegroups.com> Message-ID: <6170f903-4e83-4716-9b1d-cc40bba31a65@j5g2000yqm.googlegroups.com> On Jan 8, 12:19?am, peteshinners wrote: > My presentation for Pycon is coming together, but I need to make sure > my information about compiling Python and Python extensions for > Windows is correct. I'm really only experienced with this on the Linux > side of things. > > First of all, is the Windows FAQ fairly up to date? Should people be > referring to section 6 if they are going to build an application with > an embedded Python interpreter?http://www.python.org/doc/faq/windows/#how-can-i-embed-python-into-a-... > > If I understand correctly, compiled extensions for Python on Windows > should match the compiler that was used to build the interpreter > itself? Is there a list somewhere that shows which version of msvc was > used to compile the recent Python binaries? > > Thank you for feedback. I definitely want to make sure I have this > correct before telling anybody else? You aren't going to try it? From casevh at gmail.com Fri Jan 8 02:55:32 2010 From: casevh at gmail.com (casevh) Date: Thu, 7 Jan 2010 23:55:32 -0800 (PST) Subject: Caching objects in a C extension Message-ID: <1bd0de37-1a32-44f4-91fa-d3126e3aaeda@u7g2000yqm.googlegroups.com> I'm working with a C extension that needs to rapidly create and delete objects. I came up with an approach to cache objects that are being deleted and resurrect them instead of creating new objects. It appears to work well but I'm afraid I may be missing something (besides heeding the warning in the documentation that _Py_NewReference is for internal interpreter use only). Below is a simplified version of the approach I'm using: MyType_dealloc(MyTypeObject *self) { if(I_want_to_save_MyType(self)) { // Save the object pointer in a cache save_it(self); } else { PyObject_Del(self); } } MyType_new(void) { MyTypeObject *self; if(there_is_an_object_in_the_cache) { self = get_object_from_cache; _Py_NewReference((PyObject*)self); } else { if(!(self = PyObjectNew(MyTypeObject, &MyType)) return NULL; initialize_the_new_object(self); } return self; } The objects referenced in the cache have a reference count of 0 and I don't increment the reference count until I need to resurrect the object. Could these objects be clobbered by the garbage collector? Would it be safer to create the new reference before stuffing the object into the cache (even though it will look like there is a memory leak when running under a debug build)? Thanks in advance for any comments, casevh From vardan.pogosyan at gmail.com Fri Jan 8 03:24:00 2010 From: vardan.pogosyan at gmail.com (h0uk) Date: Fri, 8 Jan 2010 00:24:00 -0800 (PST) Subject: Ask how to use HTMLParser References: <87vdfdtg3z.fsf@ymail.invalid> <87k4vtrt8f.fsf@ymail.invalid> Message-ID: On 8 ???, 11:44, Water Lin wrote: > h0uk writes: > > On 8 ???, 08:44, Water Lin wrote: > >> I am a new guy to use Python, but I want to parse a html page now. I > >> tried to use HTMLParse. Here is my sample code: > >> ---------------------- > >> from HTMLParser import HTMLParser > >> from urllib2 import urlopen > > >> class MyParser(HTMLParser): > >> ? ? title = "" > >> ? ? is_title = "" > >> ? ? def __init__(self, url): > >> ? ? ? ? HTMLParser.__init__(self) > >> ? ? ? ? req = urlopen(url) > >> ? ? ? ? self.feed(req.read()) > > >> ? ? def handle_starttag(self, tag, attrs): > >> ? ? ? ? if tag == 'div' and attrs[0][1] == 'articleTitle': > >> ? ? ? ? ? ? print "Found link => %s" % attrs[0][1] > >> ? ? ? ? ? ? self.is_title = 1 > > >> ? ? def handle_data(self, data): > >> ? ? ? ? if self.is_title: > >> ? ? ? ? ? ? print "here" > >> ? ? ? ? ? ? self.title = data > >> ? ? ? ? ? ? print self.title > >> ? ? ? ? ? ? self.is_title = 0 > >> ----------------------- > > >> For the tag > >> ------- > >>
open article title
> >> ------- > > >> I use my code to parse it. I can locate the div tag but I don't know how > >> to get the text for the tag which is "open article title" in my example. > > >> How can I get the html content? What's wrong in my handle_data function? > > >> Thanks > > >> Water Lin > > >> -- > >> Water Lin's notes and pencils:http://en.waterlin.org > >> Email: Water... at ymail.com > > > I want to say your code works well > > But in handle_data I can't print self.title. I don't why I can't set the > self.title in handle_data. > > Thanks > > Water Lin > > -- > Water Lin's notes and pencils:http://en.waterlin.org > Email: Water... at ymail.com I have tested your code as : #!/usr/bin/env python # -*- conding: utf-8 -*- from HTMLParser import HTMLParser class MyParser(HTMLParser): title = "" is_title = "" def __init__(self, data): HTMLParser.__init__(self) self.feed(data) def handle_starttag(self, tag, attrs): if tag == 'div' and attrs[0][1] == 'articleTitle': print "Found link => %s" % attrs[0][1] self.is_title = 1 def handle_data(self, data): if self.is_title: print "here" self.title = data print self.title self.is_title = 0 if __name__ == "__main__": m = MyParser("""
open article title
Ask how to use HTMLParser
Parametrs
""") All stuff printed and handled fine. Also, the 'print self.title' statement works fine. Try run my code. Vardan. From nobody at nowhere.org Fri Jan 8 03:39:50 2010 From: nobody at nowhere.org (Georg) Date: Fri, 8 Jan 2010 09:39:50 +0100 Subject: C Structure rebuild with ctypes References: <7p6ksnFkg1U1@mid.individual.net> Message-ID: <7qo9avFivmU1@mid.individual.net> Hi Mark, > Are you passing in these values, or are they being returned? To me the > depth of the pointer references implies numVars, varNames, and varTypes > are out parameters. I'll assume that for now. If they are in/out > parameters let me know. If these parameters were in parameters. What would I have to do different? Best regards Georg From victorsubervi at gmail.com Fri Jan 8 04:23:00 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 8 Jan 2010 04:23:00 -0500 Subject: getfirst and re In-Reply-To: <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> Message-ID: <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> On Wed, Jan 6, 2010 at 2:19 PM, Victor Subervi wrote: > On Wed, Jan 6, 2010 at 3:09 PM, Carsten Haese wrote: > >> Victor Subervi wrote: >> > I have an automatically generated HTML form from which I need to extract >> > data to the script which this form calls (to which the information is >> > sent). >> >> Ideally, the script that receives the submitted fields should know how >> the form was generated, so it knows what fields to expect. Failing that, >> you should realize that getfirst() is not the only way to access the >> contents of a cgi.FieldStorage object. >> >> If you need to know the values of all fields whose names obey a certain >> pattern, I recommend you start with a list of all field names in the >> FieldStorage object and pick out the ones whose names obey that pattern. >> To get the list of field names, you should consider using the .keys() >> method of the FieldStorage object. >> > > Code snippet: def cgiFieldStorageToDict(fieldStorage): params = {} for key in fieldStorage.keys(): params[key] = cgi.FieldStorage[key].value return params def display(): top() dict = cgiFieldStorageToDict(cgi.FieldStorage()) print dict Error: /var/www/html/angrynates.com/christians/cart/display.py 163 print 'All products together usually cost: $%f

' % str(fpformat.fix(round(int(allPrices * 100))/100, 2)) 164 cursor.close() 165 bottom() 166 167 display() display = /var/www/html/angrynates.com/christians/cart/display.py in display() 120 def display(): 121 top() 122 dict = cgiFieldStorageToDict(cgi.FieldStorage()) 123 print dict 124 if store == 'prescriptions': builtin dict = , global cgiFieldStorageToDict = , global cgi = , cgi.FieldStorage = /var/www/html/angrynates.com/christians/cart/display.py in cgiFieldStorageToDict(fieldStorage=FieldStorage(None, None, [MiniFieldStorage('stor...products'), MiniFieldStorage('cat', 'prodCat1')])) 115 params = {} 116 for key in fieldStorage.keys(): 117 params[key] = cgi.FieldStorage[key].value 118 return params 119 params = {}, key = 'store', global cgi = , cgi.FieldStorage = , ].value undefined TypeError: unsubscriptable object args = ('unsubscriptable object',) What do? TIA, beno #! /usr/bin/python import cgitb; cgitb.enable() import cgi import MySQLdb import sys,os sys.path.append(os.getcwd()) from login import login from template import top, bottom from sets import Set import fpformat form = cgi.FieldStorage() store = form.getfirst('store') cat = form.getfirst('cat', '') user, passwd, db, host = login() db = MySQLdb.connect(host, user, passwd, db) cursor = db.cursor() def displayProducts(patientID=''): try: # These are stores with categories where ordering by price is important sql = 'select ID from %s where Category="%s" order by Price desc;' % (store, cat) cursor.execute(sql) except: # Stores, like prescriptions, where ordering by price is not important sql = 'select ID from %s;' % (store) cursor.execute(sql) ids = [itm[0] for itm in cursor] cursor.execute('describe %s;' % store) colFields, colFieldValues = [itm[0] for itm in cursor], [itm[1] for itm in cursor] i = 0 if len(ids) > 0: for id in ids: # print '\n' print '
' print "" % store print "" % id j = 0 for col in colFields: sql = 'select %s from %s where ID="%s";' % (col, store, str(id)) cursor.execute(sql) colValue = cursor.fetchone() if col == 'SKU': print "" % colValue[0] print '%s: %s
\n' % (col, colValue[0]) elif col == 'Category': print "" % colValue[0] print '%s: %s
\n' % (col, colValue[0]) elif col == 'OutOfStock': if colValue[0] == '1': # This product is out of stock outOfStockFlag = 'yes' else: outOfStockFlag = 'no' elif col[:3] != 'pic': notSet = 1 if isinstance(colValue[0], (str, int, float, long, complex, unicode, list, buffer, xrange, tuple)): pass else: try: html = "%s:
" print html except: pass if notSet == 1: if len(col) > 49: colX = col[:50] + '...' else: colX = col print '%s: %s
\n' % (colX, colValue[0]) elif col == 'pic1': try: if (colValue[0] != None): # if (colValue[0] != None) and (len(colValue[0] > 0)): print '
\n' % (store, col[3:], id, store, col[3:], id, store, col[3:], id) except TypeError: raise except: raise # i += 1 # try: # content = colValues[0][x].tostring() # pic = "tmp" + str(i) + ".jpg" # try: # os.remove(pic) # except: # pass # img = open(pic, "w") # img.write(content) # print '

' % pic # img.close() # except: # pass j += 1 if store != 'prescriptions': if outOfStockFlag == 'yes': print 'This item is currently out of stock.' else: print "" else: print "" % patientID print "" print '


' print '\n' print '\n' def cgiFieldStorageToDict(fieldStorage): params = {} for key in fieldStorage.keys(): params[key] = cgi.FieldStorage[key].value return params def display(): top() dict = cgiFieldStorageToDict(cgi.FieldStorage()) print dict if store == 'prescriptions': password = form.getfirst('password') email = form.getfirst('email') sql = 'select ID, FirstName, LastName from %s where PW="%s" and Email="%s";' % ('patientsPersonalData', password, email) try: cursor.execute(sql) patientID = [itm[0] for itm in cursor][0] print "

Welcome, %s %s!

" % ([itm[1] for itm in cursor][0], [itm[2] for itm in cursor][0]) displayProducts(patientID) except: print "We're sorry. The email address and password you entered do not correspond with what is recorded in our database. Please click the "back" button and try again.

" else: displayProducts() cursor.execute('show tables like "%sPackages";' % store) if cursor.fetchone() is not None: sql = 'select ID from categories%s where Category="%s";' % (store[0].upper() + store[1:], cat) cursor.execute(sql) categoryID = cursor.fetchone()[0] sql = 'select ID from %sPackages p join %sCategoriesPackages c where c.CategoryID=%s;' % (store, store, categoryID) cursor.execute(sql) packageIDs = [itm[0] for itm in cursor] for pid in packageIDs: print 'XXX' sql = 'select ProductID from %sProductsPackages where PackageID=%s;' % (store, pid) cursor.execute(sql) productIDs = [itm[0] for itm in cursor] sql = 'select Name, Price from %sPackages where ID=%s;' % (store, pid) name, price = cursor.fetchone() print 'Package Name: %s
' % name print 'Price: %s
' % price allPrices = 0.00 for pid in productIDs: sql = 'select Name, Price from %s;' % store # print sql cursor.execute(sql) pName, pPrice = cursor.fetchone() pPrice = float(pPrice) allPrices += pPrice print 'Product Name: %s
' % pName print 'All products together usually cost: $%f

' % str(fpformat.fix(round(int(allPrices * 100))/100, 2)) cursor.close() bottom() display() -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank at chagford.com Fri Jan 8 04:33:42 2010 From: frank at chagford.com (Frank Millman) Date: Fri, 8 Jan 2010 11:33:42 +0200 Subject: multiprocessing and remote objects Message-ID: Hi all I am experimenting with the multiprocessing module, to get a feel of what is possible and what is not. I have got a situation that does not behave as expected. I can work around it, but I would like to find out the underlying reason, to get a more solid understanding. I am trying to create 'remote objects' that can be shared between processes. I can create and return an instance of a class, with no problem. Now I want to add a method to the class which, when called remotely, creates and returns an instance of another class. I cannot get this to work. On the server side I have - tables = {} # a cache of table instances, keyed on name class Table(object): def __init__(self, name): self.name = name def get_table(self, tablename): # if table with this name does not exist, # create it and store it in cache if tablename not in tables: tables[tablename] = Table(tablename) # retrieve table instance from cache and return it return tables[tablename] class MyManager(BaseManager): pass MyManager.register('get_table', get_table, method_to_typeid={'get_table': 'Table') MyManager.register('Table', Table) if __name__ == '__main__': manager = MyManager(address=('127.0.0.1', 50000)) server = manager.get_server() server.serve_forever() On the client side I have - class MyManager(BaseManager): pass MyManager.register('get_table') MyManager.register('Table') if __name__ == '__main__': manager = MyManager(address=('127.0.0.1', 50000)) manager.connect() cust = manager.get_table('Customers') So far so good. Then I extend the server program as follows - class Table(object): def __init__(self, name): self.name = name + self.columns = {} + def add_column(self, colname): + if colname not in self.columns: + self.columns[colname] = Column(colname) + return self.columns[colname] + class Column(object): + def __init__(self, colname): + self.colname = colname MyManager.register('get_table', get_table, method_to_typeid={'get_table': 'Table') - MyManager.register('Table', Table) + MyManager.register('Table', Table, + method_to_typeid={'add_column': 'Column') + MyManager.register('Column', Column) and I extend the client program as follows - MyManager.register('get_table') MyManager.register('Table') + MyManager.register('Column') cust = manager.get_table('Customers') + acno = cust.add_column('Acno') The server appears to create and return the column ok, but on the client side I get the following traceback - Traceback (most recent call last): File "F:\junk\multiprocess\mp13b.py", line 29, in acno = cust.add_column('Acno') File "", line 2, in add_column File "C:\Python26\lib\multiprocessing\managers.py", line 726, in _callmethod kind, result = conn.recv() AttributeError: 'module' object has no attribute 'Column' Does anyone know if it is possible to do what I am attempting, and what the correct approach is? BTW, version is Python 2.6.2. Thanks Frank Millman From taradov at gmail.com Fri Jan 8 05:25:18 2010 From: taradov at gmail.com (alexru) Date: Fri, 8 Jan 2010 02:25:18 -0800 (PST) Subject: Standardized interpreter speed evaluation tool Message-ID: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> Is there any standardized interpreter speed evaluation tool? Say I made few changes in interpreter code and want to know if those changes made python any better, which test should I use? From davea at ieee.org Fri Jan 8 05:34:08 2010 From: davea at ieee.org (Dave Angel) Date: Fri, 08 Jan 2010 05:34:08 -0500 Subject: Ask how to use HTMLParser In-Reply-To: <87k4vtrt8f.fsf@ymail.invalid> References: <87vdfdtg3z.fsf@ymail.invalid> <87k4vtrt8f.fsf@ymail.invalid> Message-ID: <4B470A20.8060303@ieee.org> Water Lin wrote: > h0uk writes: > > >> On 8 ???, 08:44, Water Lin wrote: >> >>> I am a new guy to use Python, but I want to parse a html page now. I >>> tried to use HTMLParse. Here is my sample code: >>> ---------------------- >>> from HTMLParser import HTMLParser >>> from urllib2 import urlopen >>> >>> class MyParser(HTMLParser): >>> title = "" >>> is_title = "" >>> def __init__(self, url): >>> HTMLParser.__init__(self) >>> req = urlopen(url) >>> self.feed(req.read()) >>> >>> def handle_starttag(self, tag, attrs): >>> if tag == 'div' and attrs[0][1] == 'articleTitle': >>> print "Found link => %s" % attrs[0][1] >>> self.is_title = 1 >>> >>> def handle_data(self, data): >>> if self.is_title: >>> print "here" >>> self.title = data >>> print self.title >>> self.is_title = 0 >>> ----------------------- >>> >>> For the tag >>> ------- >>>
open article title
>>> ------- >>> >>> I use my code to parse it. I can locate the div tag but I don't know how >>> to get the text for the tag which is "open article title" in my example. >>> >>> How can I get the html content? What's wrong in my handle_data function? >>> >>> Thanks >>> >>> Water Lin >>> >>> -- >>> Water Lin's notes and pencils:http://en.waterlin.org >>> Email: Water... at ymail.com >>> >> I want to say your code works well >> > > But in handle_data I can't print self.title. I don't why I can't set the > self.title in handle_data. > > Thanks > > Water Lin > > I don't know HTMLParser, but I see a possible confusion point in your class definition. You have both class-attributes and instance-attributes of the same names (title and is_title). So if you have more than one instance of MyParser, then they won't see each other's changes. Normally, I'd move the initialization of such attributes into the __init__() method, so the behavior is clear. When an instance-attribute has the same name as a class-attribute, the instance-attribute takes precedence, and "hides" the class-attribute, for further processing in that same instance. So effectively, the class-attribute acts as a default value. From no.i.dont at want.mail.from.spammers.com Fri Jan 8 05:42:04 2010 From: no.i.dont at want.mail.from.spammers.com (Fencer) Date: Fri, 08 Jan 2010 11:42:04 +0100 Subject: How do I access what's in this module? In-Reply-To: References: <7qnfk0Fr6kU1@mid.individual.net> <29a775c4-223f-4e74-b739-12d633e5739e@a21g2000yqc.googlegroups.com> <7qno2iF1b7U1@mid.individual.net> Message-ID: <7qogftFu1aU1@mid.individual.net> On 2010-01-08 05:01, John Machin wrote: > > Error message should appear after line starting with "File". Above > excerpt taken from google groups; identical to what shows in > http://news.gmane.org/gmane.comp.python.general ... what are you > looking at? > > With Windows XP and Python 2.5.4 I get: > > Traceback (most recent call last): > File "", line 1, in > AttributeError: 'builtin_function_or_method' object has no attribute > 'dump' I'm sorry, I managed to leave out that last line by mistake! My bad. I didn't spot that in my first reply to you because I was under the impression that you hadn't seen the tiniest part of traceback. As you neatly pointed out earlier, it's easy to become confused when communicating. :) > >> It turns out I no longer want to access anything in there but I thank >> you for your information nontheless. > > You're welcome -- the advice on _methods is portable :-) I will look more closely at what other advice you write, I must confess I didn't actually do that because, as I mentioned, I no longer had any interest in accessing the module and I was busy (and still am) with another problem. Thanks John! - Fencer From deets at nospam.web.de Fri Jan 8 05:55:35 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Fri, 08 Jan 2010 11:55:35 +0100 Subject: Standardized interpreter speed evaluation tool In-Reply-To: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> References: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> Message-ID: <7qoh97F2mjU1@mid.uni-berlin.de> alexru schrieb: > Is there any standardized interpreter speed evaluation tool? Say I > made few changes in interpreter code and want to know if those changes > made python any better, which test should I use? Pybench I guess. Diez From deets at nospam.web.de Fri Jan 8 05:59:32 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Fri, 08 Jan 2010 11:59:32 +0100 Subject: Caching objects in a C extension In-Reply-To: <1bd0de37-1a32-44f4-91fa-d3126e3aaeda@u7g2000yqm.googlegroups.com> References: <1bd0de37-1a32-44f4-91fa-d3126e3aaeda@u7g2000yqm.googlegroups.com> Message-ID: <7qohgkF2mjU2@mid.uni-berlin.de> casevh schrieb: > I'm working with a C extension that needs to rapidly create and delete > objects. I came up with an approach to cache objects that are being > deleted and resurrect them instead of creating new objects. It appears > to work well but I'm afraid I may be missing something (besides > heeding the warning in the documentation that _Py_NewReference is for > internal interpreter use only). > > Below is a simplified version of the approach I'm using: > > MyType_dealloc(MyTypeObject *self) > { > if(I_want_to_save_MyType(self)) { > // Save the object pointer in a cache > save_it(self); > } else { > PyObject_Del(self); > } > } > > MyType_new(void) > { > MyTypeObject *self; > if(there_is_an_object_in_the_cache) { > self = get_object_from_cache; > _Py_NewReference((PyObject*)self); > } else { > if(!(self = PyObjectNew(MyTypeObject, &MyType)) > return NULL; > initialize_the_new_object(self); > } > return self; > } > > The objects referenced in the cache have a reference count of 0 and I > don't increment the reference count until I need to resurrect the > object. Could these objects be clobbered by the garbage collector? > Would it be safer to create the new reference before stuffing the > object into the cache (even though it will look like there is a memory > leak when running under a debug build)? Deep out of my guts I'd say keeping a reference, and using you own LRU-scheme would be the safest without residing to use dark magic. Diez From davea at ieee.org Fri Jan 8 06:11:49 2010 From: davea at ieee.org (Dave Angel) Date: Fri, 08 Jan 2010 06:11:49 -0500 Subject: Standardized interpreter speed evaluation tool In-Reply-To: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> References: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> Message-ID: <4B4712F5.70205@ieee.org> alexru wrote: > Is there any standardized interpreter speed evaluation tool? Say I > made few changes in interpreter code and want to know if those changes > made python any better, which test should I use? > > Not trying to be a smart-aleck, but the test you use should reflect your definition of the phrase "any better." For example, suppose you decided that you could speed things up by pre-calculating a few dozen megabytes of data, and including that in the python.dll. This would change the memory footprint of Python, and possibly the interpreter's startup/load time, not just the runtime of whatever loop you are testing. But if I assume you know all that and just want to do timings. There are at least two stdlib functions that can help: import time time.time() and time.clock() will give you a wall-clock floating point number, and you can subtract two of these within the same program to see how long something takes. This approach ignores interpreter startup, and does nothing to compensate for other processes running on the system. But it can be very useful, and easy to run. The resolution on each function varies by OS, so you may have to experiment to see which one gives the most precision. import time start = time.time() dotest() print "Elapsed time", time.time() - start timeit module can be used within code for timing, or it may be used to load and run a test from the command line. In the latter version, it includes the startup time for the interpreter, which might be important. It also can execute the desired code repeatedly, so you can get some form of averaging, or amortizing. *** python -m timeit .... Note that due to system caching, doing multiple runs consecutively may give different results than ones that are separated by other programs running. And of course when you recompile and link., the system buffers will contain an unusual set of data. So there are ways (which I don't recall) of flushing the system buffers to let programs start on equal footing. DaveA From phil at riverbankcomputing.com Fri Jan 8 06:27:38 2010 From: phil at riverbankcomputing.com (Phil Thompson) Date: Fri, 08 Jan 2010 11:27:38 +0000 Subject: PyQt QThreadPool error In-Reply-To: References: <7dc7a9f3-cff9-4911-bfde-6086db471b41@k17g2000yqh.googlegroups.com> <7qmsujFc9tU2@mid.uni-berlin.de> <7c781f4c-45c4-4e15-ac21-c4ca174a9a89@l30g2000yqb.googlegroups.com> Message-ID: <4fe15ddf1fa181a42d42dae8e99a5997@localhost> On Thu, 7 Jan 2010 15:07:10 -0800 (PST), h0uk wrote: ... > Phil you right about app.exec_(). But situation is sligthly different. > > I want to have more than one Job. I add these Jobs into QThreadPool > trough cycle. And I also want these Jobs to run sequentially. > > The following code illustrate what I mean: > > #!/usr/bin/env python > # -*- coding: utf-8 -*- > > import sys > import os > import time > > from PyQt4 import QtCore, QtGui > > class Job(QtCore.QRunnable): > def __init__(self, name): > QtCore.QRunnable.__init__(self) > self._name = name > > def run(self): > time.sleep(3) > print self._name > > > if __name__ == "__main__": > > app = QtGui.QApplication(sys.argv) > > QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) > > for i in range(5): > j = Job("Job-" + str(i)) > j.setAutoDelete(True) > QtCore.QThreadPool.globalInstance().start(j, i) > app.exec_() > > After 5 cycle I get the same error: An unhandled win32 exception > occured in python.exe. > > How I can do it?? To run my Jobs sequentially??? It's a PyQt bug. The workaround is not to use setAutoDelete() and instead keep an explicit reference to your Job instances - for example in a list of jobs. Phil From tiago.b.almeida at gmail.com Fri Jan 8 06:39:06 2010 From: tiago.b.almeida at gmail.com (tiago almeida) Date: Fri, 8 Jan 2010 11:39:06 +0000 Subject: Transforming a List of elements into a List of lists of elements Message-ID: Hello all, I'd like to ask how you'd implement a function *f* that transforms a list of elements into a list of lists of elements by grouping contiguous elements together. Examples: a=[1,2,3,4,5] print( f(a, 2) ) # -> [ [1, 2], [3, 4], [5] ] print( f(a, 3) ) # -> [ [1, 2, 3], [4, 5] ] print( f(a, 1) ) # -> [ [1], [2], [3], [4], [5] ] I have done a small function that does this using a while loop but what I'm looking for is a more *pythonic* way to do this. Maybe some function in some standard Module does this and I don't know? Ps: I don't post the function here because I don't have it with me. Thanks a lot in advance! Tiago Almeida tiago.b.almeida at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From zuo at chopin.edu.pl Fri Jan 8 07:16:34 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Fri, 08 Jan 2010 13:16:34 +0100 Subject: Transforming a List of elements into a List of lists of elements In-Reply-To: References: Message-ID: 08-01-2010 tiago almeida wrote: > Hello all, > > I'd like to ask how you'd implement a function *f* that transforms a > list of elements into a list of lists of elements by grouping > contiguous elements together. > Examples: > > a=[1,2,3,4,5] > print( f(a, 2) ) # -> [ [1, 2], [3, 4], [5] ] > print( f(a, 3) ) # -> [ [1, 2, 3], [4, 5] ] > print( f(a, 1) ) # -> [ [1], [2], [3], [4], [5] ] > > > I have done a small function that does this using a while loop but > what I'm looking for is a more *pythonic* way to do this. Maybe > some function in some standard Module does this and I don't know? [In Python 2.x] >>> s = [1,2,3,4,5] >>> zip(*[iter(s)]*3) # zip truncates result [(1, 2, 3)] >>> zip(*[iter(s)]*2) [(1, 2), (3, 4)] >>> zip(*[iter(s)]*3) # or simpler: zip(s) :-) [(1,), (2,), (3,), (4,), (5,)] >>> map(None, *[iter(s)]*2) # map fills result with None [(1, 2), (3, 4), (5, None)] >>> map(None, *[iter(s)]*3) [(1, 2, 3), (4, 5, None)] Instead of map, you can use izip_longest from itertools module: >>> list(itertools.izip_longest(*[iter(s)]*3)) [(1, 2, 3), (4, 5, None)] >>> list(itertools.izip_longest(*[iter(s)]*2)) [(1, 2), (3, 4), (5, None)] See: http://docs.python.org/library/functions.html#zip http://docs.python.org/library/functions.html#map http://docs.python.org/library/itertools.html#itertools.izip_longest Cheers, *j -- Jan Kaliszewski (zuo) From jeanmichel at sequans.com Fri Jan 8 07:39:05 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Fri, 08 Jan 2010 13:39:05 +0100 Subject: Transforming a List of elements into a List of lists of elements In-Reply-To: References: Message-ID: <4B472769.5000201@sequans.com> tiago almeida wrote: > Hello all, > > I'd like to ask how you'd implement a function /f/ that transforms a > list of elements into a list of lists of elements by grouping > contiguous elements together. > Examples: > > a=[1,2,3,4,5] > print( f(a, 2) ) # -> [ [1, 2], [3, 4], [5] ] > print( f(a, 3) ) # -> [ [1, 2, 3], [4, 5] ] > print( f(a, 1) ) # -> [ [1], [2], [3], [4], [5] ] > > > I have done a small function that does this using a while loop but > what I'm looking for is a more /pythonic/ way to do this. Maybe some > function in some standard Module does this and I don't know? > > Ps: I don't post the function here because I don't have it with me. > > Thanks a lot in advance! > Tiago Almeida > tiago.b.almeida at gmail.com > using list comprehention (http://docs.python.org/tutorial/datastructures.html) # note: poor naming def func(l, slice): r = range(0, len(l), slice) result = [l[sl:sl+slice] for sl in r] print result >>> a=[1,2,3,4,5] >>> func(a,1) ; func(a,2) ; func(a,3) [[1], [2], [3], [4], [5]] [[1, 2], [3, 4], [5]] [[1, 2, 3], [4, 5]] I would add that I personally find list comprehension hard to read, so I'm not sure it is that pythonic. Your old school while loop could be less concise/elegant, but perfectly readable. Jean-Michel From zuo at chopin.edu.pl Fri Jan 8 07:39:45 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Fri, 08 Jan 2010 13:39:45 +0100 Subject: Transforming a List of elements into a List of lists of elements In-Reply-To: References: Message-ID: PS. Sorry, I wrote: > >>> zip(*[iter(s)]*3) # or simpler: zip(s) :-) But should be > >>> zip(*[iter(s)]*1) # or simpler: zip(s) :-) *j From phd at phd.pp.ru Fri Jan 8 07:43:36 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Fri, 8 Jan 2010 15:43:36 +0300 Subject: SQLObject 0.11.3 Message-ID: <20100108124336.GB10206@phd.pp.ru> Hello! I'm pleased to announce version 0.11.3, a minor bugfix release of 0.11 branch of SQLObject. What is SQLObject ================= SQLObject is an object-relational mapper. Your database tables are described as classes, and rows are instances of those classes. SQLObject is meant to be easy to use and quick to get started with. SQLObject supports a number of backends: MySQL, PostgreSQL, SQLite, Firebird, Sybase, MSSQL and MaxDB (also known as SAPDB). Where is SQLObject ================== Site: http://sqlobject.org Development: http://sqlobject.org/devel/ Mailing list: https://lists.sourceforge.net/mailman/listinfo/sqlobject-discuss Archives: http://news.gmane.org/gmane.comp.python.sqlobject Download: http://cheeseshop.python.org/pypi/SQLObject/0.11.3 News and changes: http://sqlobject.org/News.html What's New ========== News since 0.11.2 ----------------- * The cache culling algorithm was enhanced to eliminate memory leaks by removing references to dead objects; tested on a website that runs around 4 million requests a day. * Fixed a bug in col.py and dbconnection.py - if dbEncoding is None suppose it's 'ascii'. * Fixed a bug in FirebirdConnection. For a more complete list, please see the news: http://sqlobject.org/News.html Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From phd at phd.pp.ru Fri Jan 8 07:45:21 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Fri, 8 Jan 2010 15:45:21 +0300 Subject: SQLObject 0.12.1 Message-ID: <20100108124521.GF10206@phd.pp.ru> Hello! I'm pleased to announce version 0.12.1, a bugfix release of branch 0.12 of SQLObject. What is SQLObject ================= SQLObject is an object-relational mapper. Your database tables are described as classes, and rows are instances of those classes. SQLObject is meant to be easy to use and quick to get started with. SQLObject supports a number of backends: MySQL, PostgreSQL, SQLite, Firebird, Sybase, MSSQL and MaxDB (also known as SAPDB). Where is SQLObject ================== Site: http://sqlobject.org Development: http://sqlobject.org/devel/ Mailing list: https://lists.sourceforge.net/mailman/listinfo/sqlobject-discuss Archives: http://news.gmane.org/gmane.comp.python.sqlobject Download: http://cheeseshop.python.org/pypi/SQLObject/0.12.1 News and changes: http://sqlobject.org/News.html What's New ========== News since 0.12.0 ----------------- * The cache culling algorithm was enhanced to eliminate memory leaks by removing references to dead objects; tested on a website that runs around 4 million requests a day. * Fixed a bug in col.py and dbconnection.py - if dbEncoding is None suppose it's 'ascii'. * Fixed three bugs in PostgresConnection. * Fixed a bug in FirebirdConnection. For a more complete list, please see the news: http://sqlobject.org/News.html Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From Erman.CELEN at 3ds.com Fri Jan 8 08:48:00 2010 From: Erman.CELEN at 3ds.com (CELEN Erman) Date: Fri, 8 Jan 2010 08:48:00 -0500 Subject: C Module's '1.#INF' changes to 'inf' at Python Message-ID: Hi All, My problem is that I've noticed a strange behavior in Python while handling FPEs on Windows after switching compilers (msvc8 to msvc9) and I am trying to find out how Python handles INF values to figure out where the problem might be. The problem appeared when we noticed that behavior of Numeric (older version of NumPy) library has changed when dealing with Floating-Point Exceptions. We are building our own slightly modified version of Python in-house (including Numeric/NumPy) and when we switched from the msvc8 to msvc9 compiler, Numeric started to return '-inf' as a result of 'Numeric.log10(0.0)' instead of rasing an OverflowError. I know that Numeric is using umath instead of the math library and since math.log10(0.0) is still raising an Error (which is ValueError in 2.6 and OverflowError in 2.4, but still an error) I thought that it might have something to do with how umath or Numeric handles the input or return values of log functions. Before hunting blindly I wanted to see how Python interprets INF results. I built a simple module with only one function that causes an FPE with log10(0.0) using C's math library and prints the results with printf. Then I imported that module from python, called the function and printed the result in python too to see if there are any differences between two values (which I found out that there are) Here is the that part of the code from my C Python module: static PyObject *badNum(PyObject *self, PyObject *args) { int sts = 0; //Not used, just dummy double a = 0.0; double c = 0.0; if (!PyArg_ParseTuple(args, "i", &sts)) //Dummy check, no use return NULL; disableFPEs(); c = log10(a); //Since FPEs disabled, should return '-1.#INF' enableFPEs(); printf("C-Val: %f\n", c); return Py_BuildValue("d", c); } After I build the module and imported it into Python, I called the function and printed the return value in Python. The output was different in two versions of Python: Output1 (Module was built with msvc8 and run by python24) C-Val: -1.#INF Py-Val: -1.#INF Output2 (Module was built with msvc9 and run by python26) C-Val: -1.#INF Py-Val: -inf I now know that compiler change didn't cause any change in return value of log10(0.0) at C level (both compilations resulted in '-1.#INF') so I am thinking that somehow at somewhere in Python core the value is translated into '-inf' and that might be the reason why Numeric/NumPy is missing the problem and having issues raising OverflowError like it's supposed to. I started debugging Python.exe (with my module loaded) in Visual Studio. While tracing what's happening after the line "return Py_BuildValue("d", c);" I went down to the "floatobject.c " file (which I thought is highly relevant to floating-point value conversions) and in particular "format_float()" function but since I am not that familiar with Python internals, I am having trouble pinpointing the issue and the code where the "conversion" is happening and I can't even be sure if I am looking at the right place. I would appreciate if someone could help me out or at least could point me to the right direction so that I can figure out the rest. Thanks, Best Regards, Ali Erman CELEN Platform Specialists / Porting Dassault Syst?mes I www.3ds.com This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged. If you are not one of the named recipients or have received this email in error, (i) you should not read, disclose, or copy it, (ii) please notify sender of your receipt by reply email and delete this email and all attachments, (iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.For other languages, go to http://www.3ds.com/terms/email-disclaimer. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ashish.vyas at motorola.com Fri Jan 8 08:49:30 2010 From: ashish.vyas at motorola.com (VYAS ASHISH M-NTB837) Date: Fri, 8 Jan 2010 21:49:30 +0800 Subject: dtd validation on python 3.1 on Windows. Message-ID: <7C57DB58C81FB64CA979EC6C4DB73E7A03E709A1@ZMY16EXM67.ds.mot.com> What is the best way to validate xmls against my dtd in python? I dont see minodom doing this. Sax has dtd validation, but does not complain on dtd violations. (it does access the dtd, if it does not find dtd it complains me.) I am using python 3.1 on Win32 machine. I was planning to try lxml2.2.4 but it shows DLL importerror. I posted the problem on this forum a few days back, but there is no reply on that. Any help?! Regards Ashish -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at holdenweb.com Fri Jan 8 08:58:50 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 08 Jan 2010 08:58:50 -0500 Subject: Need help to pass self.count to other classes. In-Reply-To: <0356a7d1$0$1336$c3e8da3@news.astraweb.com> References: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> <0356a7d1$0$1336$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: [... points out my misapprehension ...] > >>>> kbi = kbInterface() >>>> sys.ps1 = kbi.prompt1 > 0xb7cbd52c>>print "Hello" > Hello > 0xb7cbd52c>> > Right, this is expert mode ... regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From grahn+nntp at snipabacken.se Fri Jan 8 09:05:52 2010 From: grahn+nntp at snipabacken.se (Jorgen Grahn) Date: 8 Jan 2010 14:05:52 GMT Subject: How to execute a script from another script and other script does not do busy wait. References: <76bff0b6-70cb-4960-9b4d-3af7a9a1e0e7@z41g2000yqz.googlegroups.com> <342e58d5-ecd5-4cc5-ad4c-cd1ea169f9e0@c34g2000yqn.googlegroups.com> Message-ID: On Thu, 2010-01-07, danmcleran at yahoo.com wrote: > On Jan 7, 9:18?am, Jorgen Grahn wrote: >> On Thu, 2010-01-07, Rajat wrote: >> > I want to run a python script( aka script2) from another python script >> > (aka script1). While script1 executes script2 it waits for script2 to >> > complete and in doing so it also does some other useful work.(does not >> > do a busy wait). >> >> > My intention is to update a third party through script1 that script2 >> > is going to take longer. >> >> I do not understand that sentence. >> What are you trying to do, more exactly? ?The best solution can be >> threads, os.popen, os.system or something different -- depending on >> the details of what you want to do. ... > I personally use subprocess. Once you launch via subprocess you can > wait or not. > > p = subprocess.Popen(...) > p.wait() #or not. > > See subprocess docs. Yes, that was included in "or something different" above. I have never used it myself, since I've needed to be compatible with Python < 2.4. Still, we need to know what he tries to do. /Jorgen -- // Jorgen Grahn O o . From carsten.haese at gmail.com Fri Jan 8 09:11:05 2010 From: carsten.haese at gmail.com (Carsten Haese) Date: Fri, 08 Jan 2010 09:11:05 -0500 Subject: getfirst and re In-Reply-To: <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> Message-ID: Victor Subervi wrote: > Code snippet: > [...] > > Error: > [...] > What do? After eliminating the pieces of your post that have been copied or quoted from elsewhere, I am left with a total of five words of your own to ask this question, which seems to be approximately the same amount of effort you have put into trying to figure out what's causing the error. If you put in a little bit more effort than that, you might figure out yourself what's causing the error, which should then logically lead you to how to fix the error. Good luck. -Carsten From grahn+nntp at snipabacken.se Fri Jan 8 09:21:38 2010 From: grahn+nntp at snipabacken.se (Jorgen Grahn) Date: 8 Jan 2010 14:21:38 GMT Subject: Do I have to use threads? References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> Message-ID: On Wed, 2010-01-06, Gary Herron wrote: > aditya shukla wrote: >> Hello people, >> >> I have 5 directories corresponding 5 different urls .I want to >> download images from those urls and place them in the respective >> directories.I have to extract the contents and download them >> simultaneously.I can extract the contents and do then one by one. My >> questions is for doing it simultaneously do I have to use threads? >> >> Please point me in the right direction. >> >> >> Thanks >> >> Aditya > > You've been given some bad advice here. > > First -- threads are lighter-weight than processes, so threads are > probably *more* efficient. However, with only five thread/processes, > the difference is probably not noticeable. (If the prejudice against > threads comes from concerns over the GIL -- that also is a misplaced > concern in this instance. Since you only have network connection, you > will receive only one packet at a time, so only one thread will be > active at a time. If the extraction process uses a significant enough > amount of CPU time I wonder what that "extraction" would be, by the way. Unless you ask for compression of the HTTP data, the images come as-is on the TCP stream. > so that the extractions are all running at the same > time *AND* if you are running on a machine with separate CPU/cores *AND* > you would like the extractions to be running truly in parallel on those > separate cores, *THEN*, and only then, will processes be more efficient > than threads.) I can't remember what the bad advice was, but here processes versus threads clearly doesn't matter performance-wise. I generally recommend processes, because how they work is well-known and they're not as vulnerable to weird synchronization bugs as threads. > Second, running 5 wgets is equivalent to 5 processes not 5 threads. > > And third -- you don't have to use either threads *or* processes. There > is another possibility which is much more light-weight: asynchronous > I/O, available through the low level select module, or more usefully > via the higher-level asyncore module. Yeah, that would be my first choice too for a problem which isn't clearly CPU-bound. Or my second choice -- the first would be calling on a utility like wget(1). /Jorgen -- // Jorgen Grahn O o . From victorsubervi at gmail.com Fri Jan 8 09:22:35 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 8 Jan 2010 10:22:35 -0400 Subject: getfirst and re In-Reply-To: References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> Message-ID: <4dc0cfea1001080622i39e8e92dpcbfce1cec472c2f8@mail.gmail.com> On Fri, Jan 8, 2010 at 10:11 AM, Carsten Haese wrote: > Victor Subervi wrote: > > Code snippet: > > [...] > > > > Error: > > [...] > > What do? > > After eliminating the pieces of your post that have been copied or > quoted from elsewhere, I am left with a total of five words of your own > to ask this question, which seems to be approximately the same amount of > effort you have put into trying to figure out what's causing the error. > First I get scolded for not including enough information. Now I get scolded for including too much. Seems I can't win. I *am* putting effort into understanding this. I'm sorry I'm not as sharp as you are in these matters. params = {}, key = 'store', global cgi = , cgi.FieldStorage = , ].value undefined TypeError: unsubscriptable object args = ('unsubscriptable object',) Why is the value undefined? What is an unsubscriptable object? TIA, beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From aioe.org at technicalbloke.com Fri Jan 8 09:31:21 2010 From: aioe.org at technicalbloke.com (r0g) Date: Fri, 08 Jan 2010 14:31:21 +0000 Subject: Do I have to use threads? References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> <432961e5-627e-452d-93db-9562951551d2@e27g2000yqd.googlegroups.com> Message-ID: Marco Salden wrote: > On Jan 6, 5:36 am, Philip Semanchuk wrote: >> On Jan 5, 2010, at 11:26 PM, aditya shukla wrote: >> >>> Hello people, >>> I have 5 directories corresponding 5 different urls .I want to >>> download >>> images from those urls and place them in the respective >>> directories.I have >>> to extract the contents and download them simultaneously.I can >>> extract the >>> contents and do then one by one. My questions is for doing it >>> simultaneously >>> do I have to use threads? >> No. You could spawn 5 copies of wget (or curl or a Python program that >> you've written). Whether or not that will perform better or be easier >> to code, debug and maintain depends on the other aspects of your >> program(s). >> >> bye >> Philip > > Yep, the more easier and straightforward the approach, the better: > threads are always (programmers')-error-prone by nature. > But my question would be: does it REALLY need to be simultaneously: > the CPU/OS only has more overhead doing this in parallel with > processess. Measuring sequential processing and then trying to > optimize (e.g. for user response or whatever) would be my prefered way > to go. Less=More. > > regards, > Marco Threads aren't as hard a some people make out although it does depend on the problem. If your processes are effectively independent then threads are probably the right solution. You can turn any function into a thread quite easily, I posted a function for this a while back... http://groups.google.com/group/comp.lang.python/msg/3361a897db3834b4?dmode=source Also it's often a good idea to build in a flag that switches your app from multi threaded to single threaded as it's easier to debug the latter. Roger. From mccolgst at gmail.com Fri Jan 8 09:32:42 2010 From: mccolgst at gmail.com (McColgst) Date: Fri, 8 Jan 2010 06:32:42 -0800 (PST) Subject: PIL show() not working for 2nd pic References: Message-ID: Do you get any errors or warnings? Could we see the code you ran to get this problem? Thanks Sean From steve at holdenweb.com Fri Jan 8 09:34:42 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 08 Jan 2010 09:34:42 -0500 Subject: getfirst and re In-Reply-To: <4dc0cfea1001080622i39e8e92dpcbfce1cec472c2f8@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4dc0cfea1001080622i39e8e92dpcbfce1cec472c2f8@mail.gmail.com> Message-ID: Victor Subervi wrote: > On Fri, Jan 8, 2010 at 10:11 AM, Carsten Haese > wrote: > > Victor Subervi wrote: > > Code snippet: > > [...] > > > > Error: > > [...] > > What do? > > After eliminating the pieces of your post that have been copied or > quoted from elsewhere, I am left with a total of five words of your own > to ask this question, which seems to be approximately the same amount of > effort you have put into trying to figure out what's causing the error. > > > First I get scolded for not including enough information. Now I get > scolded for including too much. Seems I can't win. I *am* putting effort > into understanding this. I'm sorry I'm not as sharp as you are in these > matters. > > params = {}, key = 'store', global cgi = '/usr/lib64/python2.4/cgi.pyc' >>, cgi.FieldStorage = , ].value undefined > > TypeError: unsubscriptable object > args = ('unsubscriptable object',) > > Why is the value undefined? What is an unsubscriptable object? > TIA, > beno > What is "why"? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From grahn+nntp at snipabacken.se Fri Jan 8 09:37:07 2010 From: grahn+nntp at snipabacken.se (Jorgen Grahn) Date: 8 Jan 2010 14:37:07 GMT Subject: Scripting (was Re: Python books, literature etc) References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: On Thu, 2010-01-07, Peter wrote: > [...] depending on your > application domain, I liked: > > 1) Hans Petter Langtangen: Python Scripting for Computational Science > A truly excellent book, not only with respect to Python Scripting , but > also on how to avoid paying license fees by using opensource tools as > an engineer ( plotting, graphing, gui dev etc ). Very good , pratical > introduction to Python with careful and non-trivial examples and exercises. Sounds good. Regarding the book's title: is it just me, or are Python programmers in general put off when people call it "scripting"? I won't attempt a strict definition of the term "scripting language", but it seems like non-programmers use it to mean "less scary than what you might think of as programming", while programmers interpret it as "not useful as a general-purpose language". /Jorgen -- // Jorgen Grahn O o . From steve at holdenweb.com Fri Jan 8 09:40:27 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 08 Jan 2010 09:40:27 -0500 Subject: getfirst and re In-Reply-To: <4dc0cfea1001080622i39e8e92dpcbfce1cec472c2f8@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4dc0cfea1001080622i39e8e92dpcbfce1cec472c2f8@mail.gmail.com> Message-ID: Victor Subervi wrote: [...] > TypeError: unsubscriptable object > args = ('unsubscriptable object',) > > Why is the value undefined? What is an unsubscriptable object? > TIA, > beno > Sorry, that wasn't very helpful. Here's some more advice: Google is you friend. Try Googling for components of the error traceback. It's unlikely you are the first person to experience this problem. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From gagsl-py2 at yahoo.com.ar Fri Jan 8 09:43:33 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 08 Jan 2010 11:43:33 -0300 Subject: One function calling another defined in the same file being exec'd References: <67CD7A9A-2446-4CE3-AAF6-8F737FA5A68D@Comcast.net> Message-ID: En Thu, 07 Jan 2010 19:47:13 -0300, Mitchell L Model escribi?: > def dofile(filename): > ldict = {'result': None} > with open(filename) as file: > exec(file.read(), globals(), ldict) > print('Result for {}: {}'.format(filename, ldict['result'])) > > Next I call dofile() on a slightly more complex file, in which one > function > calls another function defined earlier in the same file. > > ################################ > def fn1(val): > return sum(range(val)) > > def fn2(arg): > return fn1(arg) > > result = fn2(5) > ################################ > > This produces a surprise: > > NameError: global name 'fn1' is not defined Ok - short answer or long answer? Short answer: Emulate how modules work. Make globals() same as locals(). (BTW, are you sure you want the file to run with the *same* globals as the caller? It sees the dofile() function and everything you have defined/imported there...). Simply use: exec(..., ldict, ldict) > [1] How is it that fn2 can be called from the top-level of the script > but fn1 > cannot be called from fn2? Long answer: First, add these lines before result=fn2(5): print("globals=", globals().keys()) print("locals=", locals().keys()) import dis dis.dis(fn2) and you'll get: globals()= dict_keys(['dofile', '__builtins__', '__file__', '__package__', '__name__', '__doc__']) locals()= dict_keys(['result', 'fn1', 'fn2']) So fn1 and fn2 are defined in the *local* namespace (as always happens in Python, unless you use the global statement). Now look at the code of fn2: 6 0 LOAD_GLOBAL 0 (fn1) 3 LOAD_FAST 0 (arg) 6 CALL_FUNCTION 1 9 RETURN_VALUE Again, the compiler knows that fn1 is not local to fn2, so it must be global (because there is no other enclosing scope) and emits a LOAD_GLOBAL instruction. But when the code is executed, 'fn1' is not in the global scope... Solution: make 'fn1' exist in the global scope. Since assignments (implied by the def statement) are always in the local scope, the only alternative is to make both scopes (global and local) the very same one. This shows that the identity "globals() is locals()" is essential for the module system to work. > [2] Is this correct behavior or is there something wrong with Python > here? It's perfectly logical once you get it... :) > [3] How should I write a file to be exec'd that defines several > functions that > call each other, as in the trivial fn1-fn2 example above? Use the same namespace for both locals and globals: exec(file.read(), ldict, ldict) -- Gabriel Genellina From grahn+nntp at snipabacken.se Fri Jan 8 09:52:32 2010 From: grahn+nntp at snipabacken.se (Jorgen Grahn) Date: 8 Jan 2010 14:52:32 GMT Subject: Recommended "new" way for config files References: <4B46075D.8080606@mycircuit.org> Message-ID: On Thu, 2010-01-07, Jean-Michel Pichavant wrote: > Peter wrote: >> Hi >> There seems to be several strategies to enhance the old ini-style >> config files with real python code, for example: ... >> Is there a strategy that should be prefered for new projects ? ... > The .ini file is the simpliest solution, at least from the user point of > view, no need to learn any python syntax. Yeah. Use whatever your users expect, and deal with it. The language you've happened to implement your stuff in should normally be irrelevant to the users. I wouldn't use .ini-style, but that's because I'm a Unix guy and they remind me of brief and painful experiments with Windows 3.1. Just remember to include support for commented-out lines. /Jorgen -- // Jorgen Grahn O o . From carsten.haese at gmail.com Fri Jan 8 09:55:17 2010 From: carsten.haese at gmail.com (Carsten Haese) Date: Fri, 08 Jan 2010 09:55:17 -0500 Subject: getfirst and re In-Reply-To: <4dc0cfea1001080622i39e8e92dpcbfce1cec472c2f8@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4dc0cfea1001080622i39e8e92dpcbfce1cec472c2f8@mail.gmail.com> Message-ID: Victor Subervi wrote: > First I get scolded for not including enough information. Now I get > scolded for including too much. Seems I can't win. I *am* putting effort > into understanding this. I'm sorry I'm not as sharp as you are in these > matters. I didn't scold you for including too much information. I scolded you for not demonstrating any effort on your part to try to figure out the problem. Maybe you did put effort into figuring it out, but you didn't show us any of it. You simply threw your entire code out there and asked an open-ended question that gave us no insight into your thought process. > params = {}, key = 'store', global cgi = '/usr/lib64/python2.4/cgi.pyc' >>, cgi.FieldStorage = , ].value undefined > > TypeError: unsubscriptable object > args = ('unsubscriptable object',) > > Why is the value undefined? What is an unsubscriptable object? At least those are specific questions. "value undefined" means that the cgitb traceback is unable to show a meaningful value for the object known as <>. Maybe you should ask yourself why that is and why you're operating on an object that doesn't have a meaningful value. An unsubscriptable object is an object that can't be subscripted. In other words, it's an object <> for which the expression foo[bar] is invalid and results in exactly the kind of exception you're struggling with right now. -Carsten From aioe.org at technicalbloke.com Fri Jan 8 09:57:20 2010 From: aioe.org at technicalbloke.com (r0g) Date: Fri, 08 Jan 2010 14:57:20 +0000 Subject: Recommended "new" way for config files References: <4B46075D.8080606@mycircuit.org> <4B460CA0.20800@sequans.com> <4B4625BC.6090201@mycircuit.org> Message-ID: Chris Rebert wrote: > On Thu, Jan 7, 2010 at 10:19 AM, Peter wrote: > >>> The .ini file is the simpliest solution, at least from the user point of >>> view, no need to learn any python syntax. >> I am speaking from the point of view of a python programmer, and I find the >> .ini restrictions not necessarily simple, for example when dealing with >> structured data (I suppose it is trivial to specify a dictionnary or a list >> for the purpose of my request) For example, configuration files for the >> logging module get unwieldy when you specify several loggers , handlers, >> formatters etc, because you have to break down structured data ( objects ) >> to name,value pairs. > >> So what is the "worshipped" approach, when you need more than name=value >> pairs ? > > JSON is one option: http://docs.python.org/library/json.html > > Cheers, > Chris > -- > http://blog.rebertia.com Yes, JSON is rapidly becoming a standard for stuff like this as it's widely portable and less bulky than XML. It's the native format for couchdb too which is nice if you want to replicate and share the contents of your documents. Roger. From dreadpiratejeff at gmail.com Fri Jan 8 09:57:56 2010 From: dreadpiratejeff at gmail.com (J) Date: Fri, 8 Jan 2010 09:57:56 -0500 Subject: Scripting (was Re: Python books, literature etc) In-Reply-To: References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: <36dec4ff1001080657m3c7a0de7u46cc576f72a701df@mail.gmail.com> On Fri, Jan 8, 2010 at 09:37, Jorgen Grahn wrote: > Regarding the book's title: is it just me, or are Python programmers > in general put off when people call it "scripting"? > > I won't attempt a strict definition of the term "scripting language", > but it seems like non-programmers use it to mean "less scary than what > you might think of as programming", while programmers interpret it as > "not useful as a general-purpose language". I dunno... I consider it programming when I'm writing bash scripts. Same with running python scripts. My personal take on it, so YMMV, is that scripting is just a synonym for programming an interpreted language, as opposed to programming (common parlance) which is usually meant writing code for a compiled language (C, C++, VB, etc...) Then again, I also tend to use scripting, coding and programming interchangeably too. And sometimes scripting = just writing a quick and dirty program to do a small task, programming = writing something much larger for long term use. Either way, I'm not offended by any of those terms as they all involve programming, regardless of whether or not someone actually calls it programming. For another analogy, what do they call Chinese food in China? Food. Cheers Jeff -- Ted Turner - "Sports is like a war without the killing." - http://www.brainyquote.com/quotes/authors/t/ted_turner.html From aioe.org at technicalbloke.com Fri Jan 8 10:09:07 2010 From: aioe.org at technicalbloke.com (r0g) Date: Fri, 08 Jan 2010 15:09:07 +0000 Subject: Accessing python from a network share in windows 7 References: <08ac1875-8747-4225-bf78-21972e959386@k9g2000vbl.googlegroups.com> Message-ID: aj wrote: > On Jan 7, 3:51 pm, MRAB wrote: >> aj wrote: > It works without any issue on win7 if I copy python to my local drive. > Also, accessing python from the same network share works fine on win > XP. So I am suspecting some security policy of win7 that is causing > problem while accessing it over a network share. You might also want to ask on Microsoft Answers... http://social.answers.microsoft.com/Forums/en-US/categories Good luck, Roger From robert.kern at gmail.com Fri Jan 8 10:36:44 2010 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 08 Jan 2010 09:36:44 -0600 Subject: C Module's '1.#INF' changes to 'inf' at Python In-Reply-To: References: Message-ID: On 2010-01-08 07:48 AM, CELEN Erman wrote: > Hi All, > > My problem is that I?ve noticed a strange behavior in Python while > handling FPEs on Windows after switching compilers (msvc8 to msvc9) and > I am trying to find out how Python handles INF values to figure out > where the problem might be. > > The problem appeared when we noticed that behavior of Numeric (older > version of NumPy) library has changed when dealing with Floating-Point > Exceptions. We are building our own slightly modified version of Python > in-house (including Numeric/NumPy) and when we switched from the msvc8 > to msvc9 compiler, Numeric started to return ?-inf? as a result of > ?Numeric.log10(0.0)? instead of rasing an OverflowError. I know that > Numeric is using umath instead of the math library and since > math.log10(0.0) is still raising an Error (which is ValueError in 2.6 > and OverflowError in 2.4, but still an error) I thought that it might > have something to do with how umath or Numeric handles the input or > return values of log functions. Numeric.log10() will check to see if the errno was set to ERANGE. It does not check if a floating point exception flag was set, which is tricky to do across platforms. The newer numpy can do it because we've finally managed to implement all of that platform-specific code, but the earlier Numeric does not. Presumably, the msvc8 C runtime's implementation of log10() sets errno and the msvc9 runtime's version does not. > Before hunting blindly I wanted to see how Python interprets INF > results. I built a simple module with only one function that causes an > FPE with log10(0.0) using C?s math library and prints the results with > printf. Then I imported that module from python, called the function and > printed the result in python too to see if there are any differences > between two values (which I found out that there are) Python 2.6 changed the string representations of the special floating point values inf and nan. Previously, the string representation was left up to the C runtime, so they differed between platforms. Python 2.6 normalized the string representation across all platforms. The underlying values are the same. What version of Python are you using? -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From mail.to.daniel.platz at googlemail.com Fri Jan 8 10:37:49 2010 From: mail.to.daniel.platz at googlemail.com (Daniel Platz) Date: Fri, 8 Jan 2010 07:37:49 -0800 (PST) Subject: Manipulating pointers in C using ctypes Message-ID: <08bda34e-9bee-44b1-b2de-80d647151786@a15g2000yqm.googlegroups.com> Hello! I have to ask a newbie question about manipulating pointers in C using ctypes. I have a C dll with two functions. The first one creates a pointer and returns it to python. The second one takes a pointer as an argument shows its address and the value at which it is pointing. This I have implemented using the following code ----------- -------- pointers.c ---------------------------- #include #ifdef __cplusplus extern "C" { // only need to export C interface if // used by C++ source code using namespace std; #endif __declspec(dllexport) void* create() { double number = 2.2; double* ptr = &number; printf("Pointer address \t %p \n", ptr); printf("Value at pointer \t %f \n", ptr[0]); return (void*) ptr; } __declspec(dllexport) int show(double* ptr) { printf("Pointer address \t %p \n", ptr); printf("Pointer value \t %f\n", *ptr); *ptr = 2.4; printf("New pointer value \t %f\n", *ptr); return 0; } #ifdef __cplusplus } #endif ------------------------------------------------------------------------ Please note that in the second function, the show function, I want to manipulate the value at which the pointer points. Now, I call this function from python with the following script. --------------------------- pointers.py -------------------------- import ctypes as ct # Load dll pointers = ct.cdll.LoadLibrary('pointers.dll') getattr(pointers, 'create') getattr(pointers, 'show') pointers.create.restype = ct.c_void_p # Create pointer in C ptr = pointers.create() # Show pointer address and value print 'Adress returned to python ' +hex(ptr) pointers.show(ct.c_void_p(ptr)) ------------------------------------------------------------------- Calling this script gives me the following output: Pointer address 0021E508 Value at pointer 2.200000 Adress returned to python 0x21e508 Pointer address 0021E508 Pointer value 0.000000 (2.20000 expected) New pointer value 2.400000 (2.40000 expected) But the script returns also an error. WindowsError: exception: access violation reading 0x40033333 WARNING: Failure executing file: Another thing that I find strange is that the return value of pointers.create is actually an integer instead of an ct.c_void_p object. Moreover, I also tried to directly manipulate the address of the pointer given as an argument to pointers.show. But when it returns to python the pointer points still at the same address as before the function call. Can someone help me with this problem? I would be very glad about an answer. With kind regards, Daniel From chander at otg-nc.com Fri Jan 8 10:39:44 2010 From: chander at otg-nc.com (Chander Ganesan) Date: Fri, 08 Jan 2010 10:39:44 -0500 Subject: Advanced Python Programming - March 8-12, 2010 Message-ID: <4B4751C0.6070705@otg-nc.com> Looking to make the most of Python's advanced features? Then this is the class for you! The Open Technology Group's Advanced Python Programming course focuses on topics of interest to experienced Python programmers, and teaches you how to leverage advanced features and functionality of Python, including: - Python objects, generators, and decorators - Writing and using regular expressions - Using sockets and networking - Improving performance using threads and the multiprocess module - Signal handling - Cooperative multitasking & networking with Twisted - Unit testing with the unittest module - XML document processing - and much more! This 5 day course is taught in a hands-on, instructor-led setting with classes limited to no more than 12 students. Hands on exercises allow students to put what they learn into practice immediately, and receive immediate instructor feedback. All-inclusive pricing includes round-trip airfare, hotel accommodation, and local shuttle services for just $2,595 (the course by itself is only $2,295). For more course details and to enroll, visit our web site at http://www.otg-nc.com/advanced-python-training , reply to this message, or contact us at 877-258-8987. Space is limited, so enroll today! -- Chander Ganesan Open Technology Group, Inc. One Copley Parkway, Suite 210 Morrisville, NC 27560 919-463-0999/877-258-8987 http://www.otg-nc.com From python at mrabarnett.plus.com Fri Jan 8 10:39:55 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 08 Jan 2010 15:39:55 +0000 Subject: getfirst and re In-Reply-To: <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> Message-ID: <4B4751CB.2050907@mrabarnett.plus.com> Victor Subervi wrote: [snip] > > Code snippet: > > def cgiFieldStorageToDict(fieldStorage): > params = {} > for key in fieldStorage.keys(): > params[key] = cgi.FieldStorage[key].value ^^^^^^^^^^^^^^^^^^^^^ This is your problem. > return params > [snip] From deets at nospam.web.de Fri Jan 8 11:24:12 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Fri, 08 Jan 2010 17:24:12 +0100 Subject: restructuredText editor ? In-Reply-To: References: Message-ID: <7qp4hcFsg3U1@mid.uni-berlin.de> Peter schrieb: > What editor do people out there use to edit .rst files for sphinx-python > documentation ? Emacs has a RST-mode (which unfortunately has some runtime-issues, the longer the text & the edit, the slower it gets), and together with flymake & rst2xml, you get nice error-notifications. Diez From python at mrabarnett.plus.com Fri Jan 8 11:24:51 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 08 Jan 2010 16:24:51 +0000 Subject: Scripting (was Re: Python books, literature etc) In-Reply-To: References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: <4B475C53.2040807@mrabarnett.plus.com> Jorgen Grahn wrote: > On Thu, 2010-01-07, Peter wrote: > >> [...] depending on your >> application domain, I liked: >> >> 1) Hans Petter Langtangen: Python Scripting for Computational Science >> A truly excellent book, not only with respect to Python Scripting , but >> also on how to avoid paying license fees by using opensource tools as >> an engineer ( plotting, graphing, gui dev etc ). Very good , pratical >> introduction to Python with careful and non-trivial examples and exercises. > > Sounds good. > > Regarding the book's title: is it just me, or are Python programmers > in general put off when people call it "scripting"? > > I won't attempt a strict definition of the term "scripting language", > but it seems like non-programmers use it to mean "less scary than what > you might think of as programming", while programmers interpret it as > "not useful as a general-purpose language". > I'd probably say that in "scripting", convenience is more important than speed. You don't need to create a project, just put the code into a file and then run it. From casevh at gmail.com Fri Jan 8 11:39:17 2010 From: casevh at gmail.com (casevh) Date: Fri, 8 Jan 2010 08:39:17 -0800 (PST) Subject: Caching objects in a C extension References: <1bd0de37-1a32-44f4-91fa-d3126e3aaeda@u7g2000yqm.googlegroups.com> <7qohgkF2mjU2@mid.uni-berlin.de> Message-ID: On Jan 8, 2:59?am, "Diez B. Roggisch" wrote: > casevh schrieb: > > > > > > > I'm working with a C extension that needs to rapidly create and delete > > objects. I came up with an approach to cache objects that are being > > deleted and resurrect them instead of creating new objects. It appears > > to work well but I'm afraid I may be missing something (besides > > heeding the warning in the documentation that _Py_NewReference is for > > internal interpreter use only). > > > Below is a simplified version of the approach I'm using: > > > MyType_dealloc(MyTypeObject *self) > > { > > ? ? if(I_want_to_save_MyType(self)) { > > ? ? ? ? // Save the object pointer in a cache > > ? ? ? ? save_it(self); > > ? ? } else { > > ? ? ? ? PyObject_Del(self); > > ? ? } > > } > > > MyType_new(void) > > { > > ? ? MyTypeObject *self; > > ? ? if(there_is_an_object_in_the_cache) { > > ? ? ? ? self = get_object_from_cache; > > ? ? ? ? _Py_NewReference((PyObject*)self); > > ? ? } else { > > ? ? ? ? if(!(self = PyObjectNew(MyTypeObject, &MyType)) > > ? ? ? ? ? ? return NULL; > > ? ? ? ? initialize_the_new_object(self); > > ? ? } > > ? ? return self; > > } > > > The objects referenced in the cache have a reference count of 0 and I > > don't increment the reference count until I need to resurrect the > > object. Could these objects be clobbered by the garbage collector? > > Would it be safer to create the new reference before stuffing the > > object into the cache (even though it will look like there is a memory > > leak when running under a debug build)? > > Deep out of my guts I'd say keeping a reference, and using you own > LRU-scheme would be the safest without residing to use dark magic. > > Diez- Hide quoted text - > > - Show quoted text - Thanks for the reply. I realized that I missed one detail. The objects are created by the extension but are deleted by Python. I don't know that an object is no longer needed until its tp_dealloc is called. At that point, its reference count is 0. casevh From tanix at mongo.net Fri Jan 8 11:50:49 2010 From: tanix at mongo.net (tanix) Date: Fri, 08 Jan 2010 16:50:49 GMT Subject: Python Goldmine has been updated: http://preciseinfo.org/Convert/index_Convert_Python.html Message-ID: Python Goldmine collection contains the extensive collection of articles going back several years. It includes thousands of code examples and expert discussions on all major topics. The information is organized by relevant topics, covered by the corresponding chapters. The information was filtered with sophisticated filters and vast majority of artilces with little relevance have been filtered out. If you have any specific requests for some new chapters to be added and it is of interest to others, please post your requests on this thread. If anyone feels he has above average level of competence, or can reccommend someone who posts on this group, you may request to be included in the expert chapters. The Python Goldmine is at: http://preciseinfo.org/Convert/index_Convert_Python.html -- Programmer's Goldmine collections: http://preciseinfo.org Tens of thousands of code examples and expert discussions on C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, organized by major topics of language, tools, methods, techniques. From solipsis at pitrou.net Fri Jan 8 11:56:33 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 8 Jan 2010 16:56:33 +0000 (UTC) Subject: Caching objects in a C extension References: <1bd0de37-1a32-44f4-91fa-d3126e3aaeda@u7g2000yqm.googlegroups.com> <7qohgkF2mjU2@mid.uni-berlin.de> Message-ID: Le Fri, 08 Jan 2010 08:39:17 -0800, casevh a ?crit?: > > Thanks for the reply. I realized that I missed one detail. The objects > are created by the extension but are deleted by Python. I don't know > that an object is no longer needed until its tp_dealloc is called. At > that point, its reference count is 0. tuple objects and others already have such a caching scheme, so you could download the Python source and look at e.g. Objects/tupleobject.c to see how it's done. Regards Antoine. From MLMDev at Comcast.net Fri Jan 8 12:02:30 2010 From: MLMDev at Comcast.net (Mitchell L Model) Date: Fri, 8 Jan 2010 12:02:30 -0500 Subject: One function calling another defined in the same file being exec'd In-Reply-To: References: Message-ID: On Jan 7, 2010, at 10:45 PM, Steven D'Aprano wrote an extensive answer to my questions about one function calling another in the same file being exec'd. His suggestion about printing out locals() and globals() in the various possible places provided the clues to explain what was going on. I would like to summarize what I have learned from this, because although I have known all the relevant pieces for many years I never put them together in a way that explains the odd behavior I observed. Statements that bind new names -- assignment, def, and class -- do so in the local scope. While exec'ing a file the local scope is determined by the arguments passed to exec; in my case, I passed an explicit local scope. It was particularly obtuse of me not to notice the effects of this because I was intentionally using it so that an assignment to 'result' in the exec'd script would enable the exec'ing code to retrieve the value of result. However, although the purity of Python with respect to the binding actions of def and class statements is wonderful and powerful, it is very difficult cognitively to view a def on a page and think "aha! that's just like an assignment of a newly created function to a name", even though that is precisely the documented behavior of def. So mentally I was making an incorrect distinction between what was getting bound locally and what was getting bound globally in the exec'd script. Moreover, the normal behavior of imported code, in which any function in the module can refer to any other function in the module, seduced me into this inappropriate distinction. To my eye I was just defining and using function definitions the way they are in modules. There is a key difference between module import and exec: as Steven pointed out, inside a module locals() is globals(). On further reflection, I will add that what appears to be happening is that during import both the global and local dictionaries are set to a copy of the globals() from the importing scope and that copy becomes the value of the module's __dict__ once import has completed successfully. Top-level statements bind names in locals(), as always, but because locals() and globals() are the same dictionary, they are also binding them in globals(), so that every function defined in the module uses the modified copy of globals -- the value of the module's __dict__ -- as its globals() when it executes. Because exec leaves locals() and globals() distinct, functions defined at the top level of a string being exec'd don't see other assignments and definitions that are also in the string. Another misleading detail is that top-level expressions in the exec can use other top-level names (assigned, def'd, etc.), which they will find in the exec string's local scope, but function bodies do not see the string's local scope. The problem I encountered arises because the function definitions need to access each other through the global scope, not the local scope. In fact, the problem would arise if one of the functions tried to call itself recursively, since its own name would not be in the global scope. So we have a combination of two distinctions: the different ways module import and exec use globals and locals and the difference between top-level statements finding other top-level names in locals but functions looking for them in globals. Sorry for the long post. These distinctions go deep into the semantics of Python namespaces, which though they are lean, pure, and beautiful, have some consequences that can be surprising -- more so the more familiar you are with other languages that do things differently. Oh, and as far as using import instead of exec for my scripts, I don't think that's appropriate, if only because I don't want my application's namespace polluted by what could be many of these pseudo- modules users might load during a session. (Yes, I could remove the name once the import is finished, but importing solely for side- effects rather than to use the imported module is offensive. Well, I would be using one module name -- result -- but that doesn't seem to justify the complexities of setting up the import and accessing the module when exec does in principle just what I need.) Finally, once all of this is really understood, there is a simple way to change an exec string's def's to bind globally instead of locally: simply begin the exec with a global declaration for any function called by one of the others. In my example, adding a "global fn1" at the beginning of the file fixes it so exec works. ################################ global fn1 # enable fn1 to be called from fn2! def fn1(val): return sum(range(val)) def fn2(arg): return fn1(arg) result = fn2(5) ################################ From vardan.pogosyan at gmail.com Fri Jan 8 12:11:10 2010 From: vardan.pogosyan at gmail.com (h0uk) Date: Fri, 8 Jan 2010 09:11:10 -0800 (PST) Subject: PyQt QThreadPool error References: <7dc7a9f3-cff9-4911-bfde-6086db471b41@k17g2000yqh.googlegroups.com> <7qmsujFc9tU2@mid.uni-berlin.de> <7c781f4c-45c4-4e15-ac21-c4ca174a9a89@l30g2000yqb.googlegroups.com> Message-ID: On 8 ???, 16:27, Phil Thompson wrote: > On Thu, 7 Jan 2010 15:07:10 -0800 (PST), h0uk > wrote: > > ... > > > > > Phil you right about app.exec_(). But situation is sligthly different. > > > I want to have more than one Job. I add these Jobs into QThreadPool > > trough cycle. And I also want these Jobs to run ?sequentially. > > > The following code illustrate what I mean: > > > #!/usr/bin/env python > > # -*- coding: utf-8 -*- > > > import sys > > import os > > import time > > > from PyQt4 import QtCore, QtGui > > > class Job(QtCore.QRunnable): > > ? ?def __init__(self, name): > > ? ? ? ? ? ?QtCore.QRunnable.__init__(self) > > ? ? ? ? ? ?self._name = name > > > ? ?def run(self): > > ? ? ? ? ? ?time.sleep(3) > > ? ? ? ? ? ?print self._name > > > if __name__ == "__main__": > > > ? ?app = QtGui.QApplication(sys.argv) > > > ? ?QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) > > > ? ?for i in range(5): > > ? ? ? ? ? ?j = Job("Job-" + str(i)) > > ? ? ? ? ? ?j.setAutoDelete(True) > > ? ? ? ? ? ?QtCore.QThreadPool.globalInstance().start(j, i) > > ? ?app.exec_() > > > After 5 cycle I get the same error: ?An unhandled win32 exception > > occured in python.exe. > > > How I can do it?? To run my Jobs sequentially??? > > It's a PyQt bug. The workaround is not to use setAutoDelete() and instead > keep an explicit reference to your Job instances - for example in a list of > jobs. > > Phil Thanks, Phil. From MLMDev at Comcast.net Fri Jan 8 12:13:34 2010 From: MLMDev at Comcast.net (Mitchell L Model) Date: Fri, 8 Jan 2010 12:13:34 -0500 Subject: One function calling another defined in the same file being exec'd In-Reply-To: References: Message-ID: <41676BD4-F23E-4F02-8845-3E3327644859@Comcast.net> On Jan 8, 2010, at 9:55 AM, "Gabriel Genellina" wrote: > > Ok - short answer or long answer? > > Short answer: Emulate how modules work. Make globals() same as > locals(). (BTW, are you sure you want the file to run with the > *same* globals as the caller? It sees the dofile() function and > everything you have defined/imported there...). Simply use: > exec(..., ldict, ldict) > >> [1] How is it that fn2 can be called from the top-level of the >> script but fn1 >> cannot be called from fn2? > > Long answer: First, add these lines before result=fn2(5): > > print("globals=", globals().keys()) > print("locals=", locals().keys()) > import dis > dis.dis(fn2) > > and you'll get: > > globals()= dict_keys(['dofile', '__builtins__', '__file__', > '__package__', '__name__', '__doc__']) > locals()= dict_keys(['result', 'fn1', 'fn2']) > > So fn1 and fn2 are defined in the *local* namespace (as always > happens in Python, unless you use the global statement). Now look at > the code of fn2: > > 6 0 LOAD_GLOBAL 0 (fn1) > 3 LOAD_FAST 0 (arg) > 6 CALL_FUNCTION 1 > 9 RETURN_VALUE > > Again, the compiler knows that fn1 is not local to fn2, so it must > be global (because there is no other enclosing scope) and emits a > LOAD_GLOBAL instruction. But when the code is executed, 'fn1' is not > in the global scope... > > Solution: make 'fn1' exist in the global scope. Since assignments > (implied by the def statement) are always in the local scope, the > only alternative is to make both scopes (global and local) the very > same one. This is very helpful additional information and clarification! Thanks. > > This shows that the identity "globals() is locals()" is essential > for the module system to work. Yes, though I doubt more than a few Python programmers would guess that identity. > >> [2] Is this correct behavior or is there something wrong with >> Python here? > > It's perfectly logical once you get it... :) I think I'm convinced. > >> [3] How should I write a file to be exec'd that defines several >> functions that >> call each other, as in the trivial fn1-fn2 example above? > > Use the same namespace for both locals and globals: > exec(file.read(), ldict, ldict) > I was going to say that this wouldn't work because the script couldn't use any built-in names, but the way exec works if the value passed for the globals argument doesn't contain an entry for '__builtins__' it adds one. I would have a further problem in that there are some names I want users to be able to use in their scripts, in particular classes that have been imported into the scope of the code doing the exec, but come to think of it I don't want to expose the entire globals() anyway. The solution is do use the same dictionary for both globals and locals, as you suggest, to emulate the behavior of module import, and explicitly add to it the names I want to make available (and since they are primarily classes, there are relatively few of those, as opposed to an API of hundreds of functions). Thanks for the help. From deets at nospam.web.de Fri Jan 8 12:19:21 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Fri, 08 Jan 2010 18:19:21 +0100 Subject: Caching objects in a C extension In-Reply-To: References: <1bd0de37-1a32-44f4-91fa-d3126e3aaeda@u7g2000yqm.googlegroups.com> <7qohgkF2mjU2@mid.uni-berlin.de> Message-ID: <7qp7opFglrU1@mid.uni-berlin.de> casevh schrieb: > On Jan 8, 2:59 am, "Diez B. Roggisch" wrote: >> casevh schrieb: >> >> >> >> >> >>> I'm working with a C extension that needs to rapidly create and delete >>> objects. I came up with an approach to cache objects that are being >>> deleted and resurrect them instead of creating new objects. It appears >>> to work well but I'm afraid I may be missing something (besides >>> heeding the warning in the documentation that _Py_NewReference is for >>> internal interpreter use only). >>> Below is a simplified version of the approach I'm using: >>> MyType_dealloc(MyTypeObject *self) >>> { >>> if(I_want_to_save_MyType(self)) { >>> // Save the object pointer in a cache >>> save_it(self); >>> } else { >>> PyObject_Del(self); >>> } >>> } >>> MyType_new(void) >>> { >>> MyTypeObject *self; >>> if(there_is_an_object_in_the_cache) { >>> self = get_object_from_cache; >>> _Py_NewReference((PyObject*)self); >>> } else { >>> if(!(self = PyObjectNew(MyTypeObject, &MyType)) >>> return NULL; >>> initialize_the_new_object(self); >>> } >>> return self; >>> } >>> The objects referenced in the cache have a reference count of 0 and I >>> don't increment the reference count until I need to resurrect the >>> object. Could these objects be clobbered by the garbage collector? >>> Would it be safer to create the new reference before stuffing the >>> object into the cache (even though it will look like there is a memory >>> leak when running under a debug build)? >> Deep out of my guts I'd say keeping a reference, and using you own >> LRU-scheme would be the safest without residing to use dark magic. >> >> Diez- Hide quoted text - >> >> - Show quoted text - > > Thanks for the reply. I realized that I missed one detail. The objects > are created by the extension but are deleted by Python. I don't know > that an object is no longer needed until its tp_dealloc is called. At > that point, its reference count is 0. I don't fully understand. Whoever creates these objects, you get a reference to them at some point. Then you increment (through the destined Macros) the ref-count. All objects in your pool with refcount 1 are canditates for removal. All you need to do is to keep a kind of timestamp together with them, since when they are released. If that's to old, fully release them. Diez From steve at holdenweb.com Fri Jan 8 12:26:31 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 08 Jan 2010 12:26:31 -0500 Subject: getfirst and re In-Reply-To: <4B4751CB.2050907@mrabarnett.plus.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4B4751CB.2050907@mrabarnett.plus.com> Message-ID: MRAB wrote: > Victor Subervi wrote: > [snip] >> >> Code snippet: >> >> def cgiFieldStorageToDict(fieldStorage): ^^^^^^^^^^^^ Further hint ... >> params = {} >> for key in fieldStorage.keys(): >> params[key] = cgi.FieldStorage[key].value > ^^^^^^^^^^^^^^^^^^^^^ > This is your problem. > >> return params >> > [snip] > regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From casevh at gmail.com Fri Jan 8 12:50:05 2010 From: casevh at gmail.com (casevh) Date: Fri, 8 Jan 2010 09:50:05 -0800 (PST) Subject: Caching objects in a C extension References: <1bd0de37-1a32-44f4-91fa-d3126e3aaeda@u7g2000yqm.googlegroups.com> <7qohgkF2mjU2@mid.uni-berlin.de> <7qp7opFglrU1@mid.uni-berlin.de> Message-ID: <4ff69845-b7c6-4b0c-a0c5-c4154551098c@m25g2000yqc.googlegroups.com> On Jan 8, 9:19?am, "Diez B. Roggisch" wrote: > casevh schrieb: > > > > > > > On Jan 8, 2:59 am, "Diez B. Roggisch" wrote: > >> casevh schrieb: > > >>> I'm working with a C extension that needs to rapidly create and delete > >>> objects. I came up with an approach to cache objects that are being > >>> deleted and resurrect them instead of creating new objects. It appears > >>> to work well but I'm afraid I may be missing something (besides > >>> heeding the warning in the documentation that _Py_NewReference is for > >>> internal interpreter use only). > >>> Below is a simplified version of the approach I'm using: > >>> MyType_dealloc(MyTypeObject *self) > >>> { > >>> ? ? if(I_want_to_save_MyType(self)) { > >>> ? ? ? ? // Save the object pointer in a cache > >>> ? ? ? ? save_it(self); > >>> ? ? } else { > >>> ? ? ? ? PyObject_Del(self); > >>> ? ? } > >>> } > >>> MyType_new(void) > >>> { > >>> ? ? MyTypeObject *self; > >>> ? ? if(there_is_an_object_in_the_cache) { > >>> ? ? ? ? self = get_object_from_cache; > >>> ? ? ? ? _Py_NewReference((PyObject*)self); > >>> ? ? } else { > >>> ? ? ? ? if(!(self = PyObjectNew(MyTypeObject, &MyType)) > >>> ? ? ? ? ? ? return NULL; > >>> ? ? ? ? initialize_the_new_object(self); > >>> ? ? } > >>> ? ? return self; > >>> } > >>> The objects referenced in the cache have a reference count of 0 and I > >>> don't increment the reference count until I need to resurrect the > >>> object. Could these objects be clobbered by the garbage collector? > >>> Would it be safer to create the new reference before stuffing the > >>> object into the cache (even though it will look like there is a memory > >>> leak when running under a debug build)? > >> Deep out of my guts I'd say keeping a reference, and using you own > >> LRU-scheme would be the safest without residing to use dark magic. > > >> Diez- Hide quoted text - > > >> - Show quoted text - > > > Thanks for the reply. I realized that I missed one detail. The objects > > are created by the extension but are deleted by Python. I don't know > > that an object is no longer needed until its tp_dealloc is called. At > > that point, its reference count is 0. > > I don't fully understand. Whoever creates these objects, you get a > reference to them at some point. Then you increment (through the > destined Macros) the ref-count. > > All objects in your pool with refcount 1 are canditates for removal. All > you need to do is to keep a kind of timestamp together with them, since > when they are released. If that's to old, fully release them. > > Diez- Hide quoted text - > > - Show quoted text - These are numeric objects created by gmpy. I'm trying to minimize the overhead for using mpz with small numbers. Objects are created and deleted very often by the interpreter as expressions are evaluated. I don't keep ownership of the objects. casevh From gtu2003 at alice.it Fri Jan 8 12:53:45 2010 From: gtu2003 at alice.it (wiso) Date: Fri, 08 Jan 2010 18:53:45 +0100 Subject: monitor reading file with thread Message-ID: <4b47712a$0$1115$4fafbaef@reader4.news.tin.it> I'm reading and processing a huge file, so during the execution I want to now the state of the processing: how many lines are already processed, and so on. The first approach is: f = open(filename) n = 0 for l in f: if n % 1000 = 0: print "Reading %d lines" %n do_something(l) but I want something diffent. I want to monitor the running of the computing every n seconds. It's the first time I use threading, and I think mine is not the best solution: import threading import time import Queue class Reader(): def __init__(self,filename): self.filename = filename self.lineno = 0 def __iter__(self): f = open(self.filename) for line in f: self.lineno += 1 time.sleep(0.01) # slow down yield line f.close() class Monitor(threading.Thread): def __init__(self,reader,stop_queue,interval=2): threading.Thread.__init__(self) self.interval = interval self.reader = reader self.stop_queue = stop_queue def run(self): while True: try: if self.stop_queue.get(timeout=self.interval) == "stop": break except Queue.Empty: pass print "MONITOR: ", reader.lineno reader = Reader("r1_200910.log") q = Queue.Queue() monitor = Monitor(reader,q) monitor.start() for line in reader: pass # do_somethinghard(line) q.put("stop") monitor.join() It't works, but I don't like how I'm stopping the thread. In general the Monitor class can be more complex, for example a progress bar. Using python 2.6 From marlowequart at hotmail.com Fri Jan 8 12:59:15 2010 From: marlowequart at hotmail.com (marlowe) Date: Fri, 8 Jan 2010 09:59:15 -0800 (PST) Subject: table from csv file Message-ID: <59653a09-c3f7-464f-a36f-475f0d759e46@e27g2000yqd.googlegroups.com> I am trying to create a table in python from a csv file where I input which columns I would like to see, and the table only shows those columns. I have attached an example of the csv file i am using, and some of the code I have written. I am having trouble converting variables between lists, dictionaries and tuples. Is there a name for what I am attempting to do? any help to get me on the right track with this is appreciated. test.csv Date Open High Low Close Volume Adj Close 12/14/09 110.01 110.7 109.5 110.24 16316000 110.24 12/11/09 110.6 110.82 108.72 109.32 28983100 109.32 12/10/09 110.66 111.03 110.04 110.82 20491700 110.82 12/9/09 111.6 112.48 109.38 110.84 37104900 110.84 12/8/09 112.76 113.06 110.21 110.95 37630800 110.95 12/7/09 111.51 114.22 111.44 113.11 42480500 113.11 12/4/09 117.15 117.26 112.41 113.75 79182600 113.75 12/3/09 118.57 119.54 118.03 118.7 28802100 118.7 12/2/09 118.8 119.27 118.3 119.18 30994600 119.18 12/1/09 117.3 117.93 116.78 117.38 27641000 117.38 11/30/09 114.48 115.89 114.27 115.64 16402300 115.64 11/27/09 113.08 115.81 113.02 115.06 21234400 115.06 11/25/09 115.69 116.88 115.53 116.62 24553300 116.62 11/24/09 114.73 114.81 113.97 114.73 22599700 114.73 11/23/09 114.67 115.12 113.99 114.29 24422700 114.29 11/20/09 111.74 112.94 111.54 112.94 17302500 112.94 11/19/09 111.85 112.4 110.76 112.3 21239800 112.3 11/18/09 112.69 113.09 111.8 112.25 22320600 112.25 11/17/09 111.09 111.99 110.9 111.97 19732900 111.97 11/16/09 110.7 112.16 110.65 111.63 25002300 111.63 11/13/09 108.32 109.8 108.14 109.74 17246000 109.74 11/12/09 109.16 109.56 108.12 108.21 17848300 108.21 11/11/09 109.49 109.71 109 109.6 17654100 109.6 11/10/09 108.03 108.78 107.7 108.39 15973300 108.39 11/9/09 108.69 108.75 107.91 108.19 18444800 108.19 11/6/09 107.38 108.04 107.06 107.43 14789000 107.43 11/5/09 106.81 107.2 106.6 106.98 10189000 106.98 11/4/09 107.11 107.68 106.43 107.1 27125500 107.1 table.py; import csv (D, O, H, L, C, V, A) = (11, 'open', 'high', 'low', 'close', 66, 77) d = {'high':H, 'low':L, 'close':C, 'open':O} spacing = '%-*s' w=raw_input('what do you want to see? (use commas to seperate values) ') y=w.lower() x=y.replace(' ','') print x p = x.split(',') #this takes string, converts to list print p num = len(p) format = num*spacing width = 12*num wi = 12 secwidth = width - wi bb = [] i=0 while i < num: #creates new list with variables ll = d[p[i]] bb.insert(i,ll) i+=1 print bb i = 0 while i < num: bb.insert(i*2, 12) #this works on list i+=1 print bb i = 0 while i < num: p.insert(i*2, 12) #this works on list i+=1 q = tuple (p) #takes list, converts to tuple reader = csv.reader(open('/prog/test.csv', "rb")) rownum = 0 for row in reader: if rownum == 0: print '=' * width print format % (q) print '-' * width print '' else: D, O, H, L, C, V, A = row [:7] o = tuple (bb) print format % (o) rownum += 1 From __peter__ at web.de Fri Jan 8 13:10:37 2010 From: __peter__ at web.de (Peter Otten) Date: Fri, 08 Jan 2010 19:10:37 +0100 Subject: Need help to pass self.count to other classes. References: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> <0356a7d1$0$1336$c3e8da3@news.astraweb.com> Message-ID: Steve Holden wrote: > Steven D'Aprano wrote: > [... points out my misapprehension ...] >> >>>>> kbi = kbInterface() >>>>> sys.ps1 = kbi.prompt1 >> > 0xb7cbd52c>>print "Hello" >> Hello >> > 0xb7cbd52c>> >> > Right, this is expert mode ... Here's a way to turn expert mode into something less advanced: >>> import sys >>> def expert_mode(): ... return "for newbies ;) " ... >>> sys.ps1 = expert_mode class Str: ... def __init__(self, f): self.f = f ... def __str__(self): return self.f() ... sys.ps1 = Str(expert_mode) for newbies ;) Peter From mackrackit at gmail.com Fri Jan 8 13:37:41 2010 From: mackrackit at gmail.com (Dave McCormick) Date: Fri, 8 Jan 2010 11:37:41 -0700 Subject: Dynamic text color In-Reply-To: References: Message-ID: <2a36ceaf1001081037n5ed5701ft9104c1396f36977c@mail.gmail.com> On Wed, Jan 6, 2010 at 9:18 AM, John Posner wrote: > On Tue, 05 Jan 2010 16:54:44 -0500, Dave McCormick > wrote: > > But it is not what I am wanting. I first thought to make it look for a >> space but that would not work when a single character like "#" is to be >> colored if there is a "string" of them. Or if all of the characters between >> quotes are to be colored. >> > > Regular expressions are good at handling searches like: > > * all the characters between quotes > * the two-character string "do", but only if it's a complete word > > -John > > -- > http://mail.python.org/mailman/listinfo/python-list > I need another hint... Been doing some reading and playing and it looks like r'\bxxx\b' is what I need. But I can not figure out how to pass a variable between \b___\b If the word in question is between the "\b \b" and in the list then it works like I want it to. The below does not work. greenList_regexp = "|".join(greenList) for matchobj in re.finditer(r'\bgreenList_regexp\b', complete_text): start,end = matchobj.span() Thanks, Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Fri Jan 8 13:43:03 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 8 Jan 2010 14:43:03 -0400 Subject: getfirst and re In-Reply-To: References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4B4751CB.2050907@mrabarnett.plus.com> Message-ID: <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden wrote: > MRAB wrote: > > Victor Subervi wrote: > > [snip] > >> > >> Code snippet: > >> > >> def cgiFieldStorageToDict(fieldStorage): > ^^^^^^^^^^^^ > Further hint ... > > >> params = {} > >> for key in fieldStorage.keys(): > >> params[key] = cgi.FieldStorage[key].value > > ^^^^^^^^^^^^^^^^^^^^^ > > This is your problem. > The problem is that I don't understand this code that I exactly copied from a Web page tutorial. Can you folks point me to tutorials where I can learn to comprehend this code? Specifically, the line in question. How is it that one can code "params[key]" (what does that mean?) and the other side, what does that mean -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinay_sajip at yahoo.co.uk Fri Jan 8 13:50:21 2010 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Fri, 8 Jan 2010 10:50:21 -0800 (PST) Subject: Recommended "new" way for config files References: <4B46075D.8080606@mycircuit.org> <4B460CA0.20800@sequans.com> <4B4625BC.6090201@mycircuit.org> <50697b2c1001071023m28e9a242uef20c6e65414c371@mail.gmail.com> Message-ID: <70ca1982-6f43-41d2-84eb-c1e226463bff@35g2000yqa.googlegroups.com> On Jan 7, 8:12?pm, Peter wrote: > > > > >> So what is the "worshipped" approach, when you need more than name=value > >> pairs ? > > > JSON is one option:http://docs.python.org/library/json.html > > Thanks, didn't think about that, although most of the apps I know don't > seem to use this approach for improved conf file handling ( ipython, > pylons, etc ). > > To me , the ipython way ( hybrid: ipy_user_conf.py and *.ini files ) > seems to be the most comprehensive way amongst the larger apps I know > of, since it let you have a python coded file for what ever you might > want to do during initialization and have additional .ini files, > ,possibily several in different locations, for simpler options in > name,value format. > > Has anybody experiences with other tools that use this approach ? > > Peter I don't know if you will find it relevant to your needs, but PEP 391 suggests an alternative method of configuring logging using dicts. The use of dicts as the basic format (rather than Python code) allows for some end-user flexibility and interoperability with other systems: e.g. JSON, YAML and Python can all produce Python dicts. Regards, Vinay Sajip From jeanmichel at sequans.com Fri Jan 8 13:52:10 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Fri, 08 Jan 2010 19:52:10 +0100 Subject: getfirst and re In-Reply-To: <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4B4751CB.2050907@mrabarnett.plus.com> <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> Message-ID: <4B477EDA.4000105@sequans.com> Victor Subervi wrote: > On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden > wrote: > > MRAB wrote: > > Victor Subervi wrote: > > [snip] > >> > >> Code snippet: > >> > >> def cgiFieldStorageToDict(fieldStorage): > ^^^^^^^^^^^^ > Further hint ... > > >> params = {} > >> for key in fieldStorage.keys(): > >> params[key] = cgi.FieldStorage[key].value > > ^^^^^^^^^^^^^^^^^^^^^ > > This is your problem. > > > The problem is that I don't understand this code that I exactly copied > from a Web page tutorial. Can you folks point me to tutorials where I > can learn to comprehend this code? Specifically, the line in question. > How is it that one can code "params[key]" (what does that mean?) and > the other side, what does that mean I think you are gathering more fans Victor :) http://docs.python.org/tutorial/datastructures.html#dictionaries JM From joncle at googlemail.com Fri Jan 8 13:55:55 2010 From: joncle at googlemail.com (Jon Clements) Date: Fri, 8 Jan 2010 10:55:55 -0800 (PST) Subject: table from csv file References: <59653a09-c3f7-464f-a36f-475f0d759e46@e27g2000yqd.googlegroups.com> Message-ID: <77b18520-c2c4-4e21-9147-1164d81b9f25@l30g2000yqb.googlegroups.com> On Jan 8, 5:59?pm, marlowe wrote: > I am trying to create a table in python from a csv file where I input > which columns I would like to see, and the table only shows those > columns. I have attached an example of the csv file i am using, and > some of the code I have written. I am having trouble converting > variables between lists, dictionaries and tuples. Is there a name for > what I am attempting to do? any help to get me on the right track with > this is appreciated. > > test.csv > Date ? ?Open ? ? High ? ?Low ? ?Close ? Volume ?Adj Close > 12/14/09 ? ? ? ?110.01 ?110.7 ? 109.5 ? 110.24 ?16316000 ? ? ? ?110.24 > 12/11/09 ? ? ? ?110.6 ? 110.82 ?108.72 ?109.32 ?28983100 ? ? ? ?109.32 > 12/10/09 ? ? ? ?110.66 ?111.03 ?110.04 ?110.82 ?20491700 ? ? ? ?110.82 > 12/9/09 111.6 ? 112.48 ?109.38 ?110.84 ?37104900 ? ? ? ?110.84 > 12/8/09 112.76 ?113.06 ?110.21 ?110.95 ?37630800 ? ? ? ?110.95 > 12/7/09 111.51 ?114.22 ?111.44 ?113.11 ?42480500 ? ? ? ?113.11 > 12/4/09 117.15 ?117.26 ?112.41 ?113.75 ?79182600 ? ? ? ?113.75 > 12/3/09 118.57 ?119.54 ?118.03 ?118.7 ? 28802100 ? ? ? ?118.7 > 12/2/09 118.8 ? 119.27 ?118.3 ? 119.18 ?30994600 ? ? ? ?119.18 > 12/1/09 117.3 ? 117.93 ?116.78 ?117.38 ?27641000 ? ? ? ?117.38 > 11/30/09 ? ? ? ?114.48 ?115.89 ?114.27 ?115.64 ?16402300 ? ? ? ?115.64 > 11/27/09 ? ? ? ?113.08 ?115.81 ?113.02 ?115.06 ?21234400 ? ? ? ?115.06 > 11/25/09 ? ? ? ?115.69 ?116.88 ?115.53 ?116.62 ?24553300 ? ? ? ?116.62 > 11/24/09 ? ? ? ?114.73 ?114.81 ?113.97 ?114.73 ?22599700 ? ? ? ?114.73 > 11/23/09 ? ? ? ?114.67 ?115.12 ?113.99 ?114.29 ?24422700 ? ? ? ?114.29 > 11/20/09 ? ? ? ?111.74 ?112.94 ?111.54 ?112.94 ?17302500 ? ? ? ?112.94 > 11/19/09 ? ? ? ?111.85 ?112.4 ? 110.76 ?112.3 ? 21239800 ? ? ? ?112.3 > 11/18/09 ? ? ? ?112.69 ?113.09 ?111.8 ? 112.25 ?22320600 ? ? ? ?112.25 > 11/17/09 ? ? ? ?111.09 ?111.99 ?110.9 ? 111.97 ?19732900 ? ? ? ?111.97 > 11/16/09 ? ? ? ?110.7 ? 112.16 ?110.65 ?111.63 ?25002300 ? ? ? ?111.63 > 11/13/09 ? ? ? ?108.32 ?109.8 ? 108.14 ?109.74 ?17246000 ? ? ? ?109.74 > 11/12/09 ? ? ? ?109.16 ?109.56 ?108.12 ?108.21 ?17848300 ? ? ? ?108.21 > 11/11/09 ? ? ? ?109.49 ?109.71 ?109 ? ? 109.6 ? 17654100 ? ? ? ?109.6 > 11/10/09 ? ? ? ?108.03 ?108.78 ?107.7 ? 108.39 ?15973300 ? ? ? ?108.39 > 11/9/09 108.69 ?108.75 ?107.91 ?108.19 ?18444800 ? ? ? ?108.19 > 11/6/09 107.38 ?108.04 ?107.06 ?107.43 ?14789000 ? ? ? ?107.43 > 11/5/09 106.81 ?107.2 ? 106.6 ? 106.98 ?10189000 ? ? ? ?106.98 > 11/4/09 107.11 ?107.68 ?106.43 ?107.1 ? 27125500 ? ? ? ?107.1 > > table.py; > > import csv > > (D, O, H, L, C, V, A) = (11, 'open', 'high', 'low', 'close', 66, 77) > > d = {'high':H, 'low':L, 'close':C, 'open':O} > > spacing = '%-*s' > > w=raw_input('what do you want to see? (use commas to seperate values) > ') > y=w.lower() > x=y.replace(' ','') > print x > > p = x.split(',') ? ? ? ? ? ? ? ? ? ?#this takes string, converts to > list > print p > num = len(p) > format = num*spacing > width = 12*num > wi = 12 > secwidth = width - wi > > bb = [] > i=0 > while i < num: ? ? ? ? ? ? ? ? ? ? ?#creates new list with variables > ? ? ll = d[p[i]] > ? ? bb.insert(i,ll) > ? ? i+=1 > print bb > i = 0 > while i < num: > ? ? bb.insert(i*2, 12) ? ? ? ? ? ? ? #this works on list > ? ? i+=1 > print bb > > i = 0 > while i < num: > ? ? p.insert(i*2, 12) ? ? ? ? ? ? ? #this works on list > ? ? i+=1 > > q = tuple (p) ? ? ? ? ? ? ? ? ? ? ? #takes list, converts to tuple > > reader = csv.reader(open('/prog/test.csv', "rb")) > > rownum = 0 > for row in reader: > ? ? if rownum == 0: > ? ? ? ? print '=' * width > ? ? ? ? print format % (q) > ? ? ? ? print '-' * width > ? ? ? ? print '' > > ? ? else: > ? ? ? ? D, O, H, L, C, V, A = row [:7] > ? ? ? ? o = tuple (bb) > ? ? ? ? print format % (o) > > ? ? rownum += 1 This might be a useful starting point (I'm guessing this is what you're after...) Let's assume your 'CSV' file is tab separated as it's certainly not comma separated :) import csv csvin = csv.reader(open('test.csv'), delimiter='\t') header = dict( (val.strip(),idx) for idx, val in enumerate(next (csvin)) ) We can use header as a column name->column index lookup eg header ['Open'] == 1 from operator import itemgetter wanted = ['Open', 'Close'] # Although you'll want to use raw_input and split on ',' getcols = itemgetter(*[header[col] for col in wanted]) getcols is a helper function that'll return a tuple of the columns in the requested order... for row in csvin: print getcols(row) Loop over the rest of the file and output the required columns. hth Jon. From victorsubervi at gmail.com Fri Jan 8 14:02:41 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 8 Jan 2010 15:02:41 -0400 Subject: getfirst and re In-Reply-To: <4B477EDA.4000105@sequans.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4B4751CB.2050907@mrabarnett.plus.com> <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> <4B477EDA.4000105@sequans.com> Message-ID: <4dc0cfea1001081102h15a16dbm7e9da66aa0e322@mail.gmail.com> On Fri, Jan 8, 2010 at 2:52 PM, Jean-Michel Pichavant < jeanmichel at sequans.com> wrote: > Victor Subervi wrote: > > On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden > steve at holdenweb.com>> wrote: >> >> MRAB wrote: >> > Victor Subervi wrote: >> > [snip] >> >> >> >> Code snippet: >> >> >> >> def cgiFieldStorageToDict(fieldStorage): >> ^^^^^^^^^^^^ >> Further hint ... >> >> >> params = {} >> >> for key in fieldStorage.keys(): >> >> params[key] = cgi.FieldStorage[key].value >> > ^^^^^^^^^^^^^^^^^^^^^ >> > This is your problem. >> >> >> The problem is that I don't understand this code that I exactly copied >> from a Web page tutorial. Can you folks point me to tutorials where I can >> learn to comprehend this code? Specifically, the line in question. How is it >> that one can code "params[key]" (what does that mean?) and the other side, >> what does that mean >> > I think you are gathering more fans Victor :) > Fans? You aught to hear me sing. I didn't think I'd gain fans here LOL! > > http://docs.python.org/tutorial/datastructures.html#dictionaries > Thanks, beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From jason.scheirer at gmail.com Fri Jan 8 14:02:50 2010 From: jason.scheirer at gmail.com (Jason Scheirer) Date: Fri, 8 Jan 2010 11:02:50 -0800 (PST) Subject: Manipulating pointers in C using ctypes References: <08bda34e-9bee-44b1-b2de-80d647151786@a15g2000yqm.googlegroups.com> Message-ID: <035f8cc8-c19c-4c91-8bd0-db9d74efe2ef@q4g2000yqm.googlegroups.com> On Jan 8, 7:37?am, Daniel Platz wrote: > Hello! > > I have to ask a newbie question about manipulating pointers in C using > ctypes. I have a C dll with two functions. The first one creates a > pointer and returns it to python. The second one takes a pointer as an > argument shows its address and the value at which it is pointing. This > I have implemented using the following code > > ----------- -------- pointers.c ---------------------------- > #include > #ifdef __cplusplus > extern "C" { ?// only need to export C interface if > ? ? ? ? ? ? ? // used by C++ source code > using namespace std; > #endif > > __declspec(dllexport) void* create() > { > ? ? ? ? double number = 2.2; > ? ? ? ? double* ptr = &number; > ? ? ? ? printf("Pointer address \t %p \n", ptr); > ? ? ? ? printf("Value at pointer \t %f \n", ptr[0]); > ? ? ? ? return (void*) ptr; > > } > > __declspec(dllexport) int show(double* ptr) > { > ? ? ? ? printf("Pointer address \t %p \n", ptr); > ? ? ? ? printf("Pointer value \t %f\n", *ptr); > ? ? ? ? *ptr = 2.4; > ? ? ? ? printf("New pointer value \t %f\n", *ptr); > ? ? ? ? return 0; > > } > > #ifdef __cplusplus} > > #endif > ------------------------------------------------------------------------ > > Please note that in the second function, the show function, I want to > manipulate the value at which the pointer points. > Now, I call this function from python with the following script. > > --------------------------- pointers.py -------------------------- > import ctypes as ct > > # Load dll > pointers = ct.cdll.LoadLibrary('pointers.dll') > getattr(pointers, 'create') > getattr(pointers, 'show') > pointers.create.restype = ct.c_void_p > > # Create pointer in C > ptr = pointers.create() > > # Show pointer address and value > print 'Adress returned to python ' +hex(ptr) > pointers.show(ct.c_void_p(ptr)) > ------------------------------------------------------------------- > > Calling this script gives me the following output: > > Pointer address ? ? ? ? ?0021E508 > Value at pointer ? ? ? ? 2.200000 > Adress returned to python 0x21e508 > Pointer address ? ? ? ? ?0021E508 > Pointer value ? ?0.000000 ? ?(2.20000 expected) > New pointer value ? ? ? ?2.400000 (2.40000 expected) > > But the script returns also an error. > > WindowsError: exception: access violation reading 0x40033333 > WARNING: Failure executing file: > > Another thing that I find strange is that the return value of > pointers.create is actually an integer instead of an ct.c_void_p > object. > > Moreover, I also tried to directly manipulate the address of the > pointer given as an argument to pointers.show. But when it returns to > python the pointer points still at the same address as before the > function call. > > Can someone help me with this problem? I would be very glad about an > answer. > > With kind regards, > > Daniel try this: __declspec(dllexport) void* create() { double* ptr = new double; *ptr = 2.2; printf("Pointer address \t %p \n", ptr); printf("Value at pointer \t %f \n", ptr[0]); return (void*) ptr; } Basically once you leave create() the address being used for number is no longer valid. The other implication for this is you are going to need a delete() function to free that memory you allocated for the double as well. From victorsubervi at gmail.com Fri Jan 8 14:09:04 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 8 Jan 2010 15:09:04 -0400 Subject: getfirst and re In-Reply-To: <4B477EDA.4000105@sequans.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4B4751CB.2050907@mrabarnett.plus.com> <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> <4B477EDA.4000105@sequans.com> Message-ID: <4dc0cfea1001081109n384ed968sf955e02c8971cf7@mail.gmail.com> On Fri, Jan 8, 2010 at 2:52 PM, Jean-Michel Pichavant < jeanmichel at sequans.com> wrote: > Victor Subervi wrote: > > On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden > steve at holdenweb.com>> wrote: >> >> MRAB wrote: >> > Victor Subervi wrote: >> > [snip] >> >> >> >> Code snippet: >> >> >> >> def cgiFieldStorageToDict(fieldStorage): >> ^^^^^^^^^^^^ >> Further hint ... >> >> >> params = {} >> >> for key in fieldStorage.keys(): >> >> params[key] = cgi.FieldStorage[key].value >> > ^^^^^^^^^^^^^^^^^^^^^ >> > This is your problem. >> >> >> The problem is that I don't understand this code that I exactly copied >> from a Web page tutorial. Can you folks point me to tutorials where I can >> learn to comprehend this code? Specifically, the line in question. How is it >> that one can code "params[key]" (what does that mean?) and the other side, >> what does that mean >> > I think you are gathering more fans Victor :) > > http://docs.python.org/tutorial/datastructures.html#dictionaries > This still isn't telling me what I need to know. Perhaps I'm missing the point, as in the recent case with putting the "print cookie" statement in the header. I am assuming (translation: making an a$$ out of you and me) that "params[key] automatically assigns the fieldStorage key to the newly created params key in the dict of the same, and assigning the value from the cgi.FieldStorage of that key to the params value. Is that correct? TIA, beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From adityashukla1983 at gmail.com Fri Jan 8 14:14:07 2010 From: adityashukla1983 at gmail.com (aditya shukla) Date: Fri, 8 Jan 2010 13:14:07 -0600 Subject: Mencoder not working from a python script Message-ID: <73045cca1001081114p1be22328k538a4512d7bc8cc0@mail.gmail.com> Hello guys, I am trying to create a python script which uses mencoder to create videos from a set of images.When I am using the command(windows 7) i am able to generate the video properly.But when used within a python script i am ggetting error.Below is the code and error.any help is appreciated. import *subprocess*,*sys*,os def *createVideo*(): mencoder = *"C:\\MPlayer-p4-svn-30075\\mencoder.exe"* path=*"mf://\"C:\\videos\\test\\*.jpg\"" * output=*"\"C:\\videos\\test\\xyz.avi\""* commande = *"%s %s -mf w=800:h=600:fps=25:type=jpg -ovc copy -oac copy -o %s"* os.system(commande % (mencoder, path,output)) if __name__==*"__main__"*: createVideo() --------------------------------error----------------- MEncoder Sherpya-SVN-r30075-4.2.5 (C) 2000-2009 MPlayer Team success: format: 16 data: 0x0 - 0x0 MF file format detected. [mf] search expr: C:\cameras\Perkins Road\*.jpg [mf] number of files: 301 (1204) VIDEO: [IJPG] 800x600 24bpp 25.000 fps 0.0 kbps ( 0.0 kbyte/s) [V] filefmt:16 fourcc:0x47504A49 size:800x600 fps:25.000 ftime:=0.0400 videocodec: framecopy (800x600 24bpp fourcc=47504a49) Writing index... Writing header... ODML: Aspect information not (yet?) available or unspecified, not writing vprp header. Video stream: nan kbit/s (-2147483648 B/s) size: 0 bytes 0.000 secs 0 frames Thanks Aditya -------------- next part -------------- An HTML attachment was scrubbed... URL: From fetchinson at googlemail.com Fri Jan 8 14:14:51 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Fri, 8 Jan 2010 20:14:51 +0100 Subject: lightweight encryption of text file Message-ID: I have a plain text file which I would like to protect in a very simple minded, yet for my purposes sufficient, way. I'd like to encrypt/convert it into a binary file in such a way that possession of a password allows anyone to convert it back into the original text file while not possessing the password one would only see the following with the standard linux utility 'file': [fetchinson at fetch ~]$ file encrypted.data encrypted.data: data and the effort required to convert the file back to the original text file without the password would be equivalent to guessing the password. I'm fully aware of the security implications of this loose specification, but for my purposes this would be a good solution. What would be the simplest way to achieve this using preferably stock python without 3rd party modules? If a not too complex 3rd party module made it really simple that would be acceptable too. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From victorsubervi at gmail.com Fri Jan 8 14:15:51 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 8 Jan 2010 15:15:51 -0400 Subject: getfirst and re In-Reply-To: <4dc0cfea1001081109n384ed968sf955e02c8971cf7@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4B4751CB.2050907@mrabarnett.plus.com> <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> <4B477EDA.4000105@sequans.com> <4dc0cfea1001081109n384ed968sf955e02c8971cf7@mail.gmail.com> Message-ID: <4dc0cfea1001081115kffe51cdpb3b31be683822db9@mail.gmail.com> On Fri, Jan 8, 2010 at 3:09 PM, Victor Subervi wrote: > > > On Fri, Jan 8, 2010 at 2:52 PM, Jean-Michel Pichavant < > jeanmichel at sequans.com> wrote: > >> Victor Subervi wrote: >> >> On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden >> steve at holdenweb.com>> wrote: >>> >>> MRAB wrote: >>> > Victor Subervi wrote: >>> > [snip] >>> >> >>> >> Code snippet: >>> >> >>> >> def cgiFieldStorageToDict(fieldStorage): >>> ^^^^^^^^^^^^ >>> Further hint ... >>> >>> >> params = {} >>> >> for key in fieldStorage.keys(): >>> >> params[key] = cgi.FieldStorage[key].value >>> > ^^^^^^^^^^^^^^^^^^^^^ >>> > This is your problem. >>> >>> >>> The problem is that I don't understand this code that I exactly copied >>> from a Web page tutorial. Can you folks point me to tutorials where I can >>> learn to comprehend this code? Specifically, the line in question. How is it >>> that one can code "params[key]" (what does that mean?) and the other side, >>> what does that mean >>> >> I think you are gathering more fans Victor :) >> >> http://docs.python.org/tutorial/datastructures.html#dictionaries >> > > This still isn't telling me what I need to know. Perhaps I'm missing the > point, as in the recent case with putting the "print cookie" statement in > the header. I am assuming (translation: making an a$$ out of you and me) > that "params[key] automatically assigns the fieldStorage key to the newly > created params key in the dict of the same, and assigning the value from the > cgi.FieldStorage of that key to the params value. Is that correct? > TIA, > beno > I may have answered my own question. I have this: def cgiFieldStorageToDict(fieldStorage): params = {} for key in fieldStorage.keys(): params[key] = fieldStorage[key].value return params dict = cgiFieldStorageToDict(cgi.FieldStorage()) print dict which gave me this: {'store': 'products', 'cat': 'prodCat1'} which looks about right. I would have written the code like this: keys = [] values = [] for key, value in fieldStorage.iteritems(): keys.append(key) values.append(value) params = dict(zip(keys, values)) which obviously isn't as elegant. But that's what I knew. Learned another trick, I guess ;) Thanks all. beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Fri Jan 8 14:21:42 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 08 Jan 2010 19:21:42 +0000 Subject: getfirst and re In-Reply-To: <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4B4751CB.2050907@mrabarnett.plus.com> <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> Message-ID: <4B4785C6.4010801@mrabarnett.plus.com> Victor Subervi wrote: > On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden > wrote: > > MRAB wrote: > > Victor Subervi wrote: > > [snip] > >> > >> Code snippet: > >> > >> def cgiFieldStorageToDict(fieldStorage): > ^^^^^^^^^^^^ > Further hint ... > > >> params = {} > >> for key in fieldStorage.keys(): > >> params[key] = cgi.FieldStorage[key].value > > ^^^^^^^^^^^^^^^^^^^^^ > > This is your problem. > > > The problem is that I don't understand this code that I exactly copied > from a Web page tutorial. Can you folks point me to tutorials where I > can learn to comprehend this code? Specifically, the line in question. > How is it that one can code "params[key]" (what does that mean?) and the > other side, what does that mean > If you got it from: Recipe 81547: Using a simple dictionary for CGI parameters http://code.activestate.com/recipes/81547/ then no, it wasn't "exactly copied". If you'd copy-and-pasted it then it would've been correct, as well as much quicker to do... From martinskou2 at gmail.com Fri Jan 8 14:26:25 2010 From: martinskou2 at gmail.com (msj@infoserv.dk) Date: Fri, 8 Jan 2010 11:26:25 -0800 (PST) Subject: Introspection References: <0d782b7a-11fb-4933-84b7-29a3653ec847@p8g2000yqb.googlegroups.com> <0354a972$0$1336$c3e8da3@news.astraweb.com> <9af6ffd9-a450-4eef-90ed-430aeea9f77c@j24g2000yqa.googlegroups.com> Message-ID: <1a190037-2326-4bdf-97fe-d0364e64d3e8@r5g2000yqb.googlegroups.com> Thanks Miki and Jason. I knew it could be done :-) From vmail at mycircuit.org Fri Jan 8 14:27:16 2010 From: vmail at mycircuit.org (Peter) Date: Fri, 08 Jan 2010 20:27:16 +0100 Subject: Scripting (was Re: Python books, literature etc) In-Reply-To: References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: <4B478714.9040104@mycircuit.org> > Sounds good. > > Regarding the book's title: is it just me, or are Python programmers > in general put off when people call it "scripting"? > > I won't attempt a strict definition of the term "scripting language", > but it seems like non-programmers use it to mean "less scary than what > you might think of as programming", while programmers interpret it as > "not useful as a general-purpose language". > > It took me a while to take "scripting" seriously. I grew up with Pascal and Eiffel and I found it difficult to appreciate dynamic typing and scripting. The author Langtangen is explaining in detail why he considers scripting useful, in particular he provides an automatic test suite to run different language versions ( perl, python, c, c++) of the same program to compare performance. The results are amazing, in that some of the examples run faster than the C++ version. I find Python extremly useful as a general purpose language ( its clearly now my prefered one ) and I find it equally useful to develop toy apps in C++, Haskell and Lisp, just to better appreciate the idea of "general purpose". For me, it has turned out that the point is not "scripting versus not scripting" or "static versus dynamic typing" but having automatic unittests or not having automatic unittests. My most important module is "nose" for running unittests the easy way. Peter From python at mrabarnett.plus.com Fri Jan 8 14:28:57 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 08 Jan 2010 19:28:57 +0000 Subject: Dynamic text color In-Reply-To: <2a36ceaf1001081037n5ed5701ft9104c1396f36977c@mail.gmail.com> References: <2a36ceaf1001081037n5ed5701ft9104c1396f36977c@mail.gmail.com> Message-ID: <4B478779.2080908@mrabarnett.plus.com> Dave McCormick wrote: > > > On Wed, Jan 6, 2010 at 9:18 AM, John Posner > wrote: > > On Tue, 05 Jan 2010 16:54:44 -0500, Dave McCormick > > wrote: > > But it is not what I am wanting. I first thought to make it look > for a space but that would not work when a single character like > "#" is to be colored if there is a "string" of them. Or if all > of the characters between quotes are to be colored. > > > Regular expressions are good at handling searches like: > > * all the characters between quotes > * the two-character string "do", but only if it's a complete word > > -John > > -- > http://mail.python.org/mailman/listinfo/python-list > > I need another hint... > > Been doing some reading and playing and it looks like > r'\bxxx\b' > is what I need. But I can not figure out how to pass a variable between > \b___\b > If the word in question is between the "\b \b" and in the list then it > works like I want it to. > The below does not work. > > greenList_regexp = "|".join(greenList) > for matchobj in re.finditer(r'\bgreenList_regexp\b', complete_text): > start,end = matchobj.span() > The regex r'\bgreenList_regexp\b' will match the string 'greenList_regexp' if it's a whole word. What you mean is "any of these words, provided that they're whole words". You'll need to group the alternatives within "(?:...)", like this: r'\b(?:' + greenList_regexp + ')\b' From vmail at mycircuit.org Fri Jan 8 14:29:35 2010 From: vmail at mycircuit.org (Peter) Date: Fri, 08 Jan 2010 20:29:35 +0100 Subject: Recommended "new" way for config files In-Reply-To: References: <4B46075D.8080606@mycircuit.org> <4B460CA0.20800@sequans.com> <4B4625BC.6090201@mycircuit.org> Message-ID: <4B47879F.7020709@mycircuit.org> On 01/08/2010 03:57 PM, r0g wrote: > Chris Rebert wrote: > >> On Thu, Jan 7, 2010 at 10:19 AM, Peter wrote: >> >> >>>> The .ini file is the simpliest solution, at least from the user point of >>>> view, no need to learn any python syntax. >>>> >>> I am speaking from the point of view of a python programmer, and I find the >>> .ini restrictions not necessarily simple, for example when dealing with >>> structured data (I suppose it is trivial to specify a dictionnary or a list >>> for the purpose of my request) For example, configuration files for the >>> logging module get unwieldy when you specify several loggers , handlers, >>> formatters etc, because you have to break down structured data ( objects ) >>> to name,value pairs. >>> >> >> >>> So what is the "worshipped" approach, when you need more than name=value >>> pairs ? >>> >> JSON is one option: http://docs.python.org/library/json.html >> >> Cheers, >> Chris >> -- >> http://blog.rebertia.com >> > > Yes, JSON is rapidly becoming a standard for stuff like this as it's > widely portable and less bulky than XML. It's the native format for > couchdb too which is nice if you want to replicate and share the > contents of your documents. > > Roger. > Excellent, thanks , have to check that out. Any experience with yaml ? Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Fri Jan 8 14:32:34 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 8 Jan 2010 15:32:34 -0400 Subject: Another Screwy Problem Message-ID: <4dc0cfea1001081132p191fe045p72a79744054fc520@mail.gmail.com> Hi; I have this line of code: sql = 'select Name, Price from %sPackages where ID=%s;' % (store, pid) which prints to this: select Name, Price from productsPackages where ID=1; which when I enter it into the MySQL interpreter gives me this: mysql> select Name, Price from productsPackages where ID=1; +------+--------+ | Name | Price | +------+--------+ | pkg | 123.45 | +------+--------+ 1 row in set (0.00 sec) exactly what I expect. However, in my script for some reason it returns this: ((1,),) Why would it do that? I guess there's some foolish thing I did in my code somewhere, but I'll be darned if I know where. Here's the whole script: #! /usr/bin/python import cgitb; cgitb.enable() import cgi import MySQLdb import sys,os sys.path.append(os.getcwd()) from login import login from template import top, bottom from sets import Set import fpformat form = cgi.FieldStorage() store = form.getfirst('store') cat = form.getfirst('cat', '') user, passwd, db, host = login() db = MySQLdb.connect(host, user, passwd, db) cursor = db.cursor() def displayProducts(patientID=''): try: # These are stores with categories where ordering by price is important sql = 'select ID from %s where Category="%s" order by Price desc;' % (store, cat) cursor.execute(sql) except: # Stores, like prescriptions, where ordering by price is not important sql = 'select ID from %s;' % (store) cursor.execute(sql) ids = [itm[0] for itm in cursor] cursor.execute('describe %s;' % store) colFields, colFieldValues = [itm[0] for itm in cursor], [itm[1] for itm in cursor] i = 0 if len(ids) > 0: for id in ids: # print '\n' print '
' print "" % store print "" % id j = 0 for col in colFields: sql = 'select %s from %s where ID="%s";' % (col, store, str(id)) cursor.execute(sql) colValue = cursor.fetchone() if col == 'SKU': print "" % colValue[0] print '%s: %s
\n' % (col, colValue[0]) elif col == 'Category': print "" % colValue[0] print '%s: %s
\n' % (col, colValue[0]) elif col == 'OutOfStock': if colValue[0] == '1': # This product is out of stock outOfStockFlag = 'yes' else: outOfStockFlag = 'no' elif col[:3] != 'pic': notSet = 1 if isinstance(colValue[0], (str, int, float, long, complex, unicode, list, buffer, xrange, tuple)): pass else: try: html = "%s:
" print html except: pass if notSet == 1: if len(col) > 49: colX = col[:50] + '...' else: colX = col print '%s: %s
\n' % (colX, colValue[0]) elif col == 'pic1': try: if (colValue[0] != None): # if (colValue[0] != None) and (len(colValue[0] > 0)): print '
\n' % (store, col[3:], id, store, col[3:], id, store, col[3:], id) except TypeError: raise except: raise # i += 1 # try: # content = colValues[0][x].tostring() # pic = "tmp" + str(i) + ".jpg" # try: # os.remove(pic) # except: # pass # img = open(pic, "w") # img.write(content) # print '

' % pic # img.close() # except: # pass j += 1 if store != 'prescriptions': if outOfStockFlag == 'yes': print 'This item is currently out of stock.' else: print "" else: print "" % patientID print "" print '


' print '\n' print '\n' def cgiFieldStorageToDict(fieldStorage): params = {} for key in fieldStorage.keys(): params[key] = fieldStorage[key].value return params def display(): top() dict = cgiFieldStorageToDict(cgi.FieldStorage()) print dict if store == 'prescriptions': password = form.getfirst('password') email = form.getfirst('email') sql = 'select ID, FirstName, LastName from %s where PW="%s" and Email="%s";' % ('patientsPersonalData', password, email) try: cursor.execute(sql) patientID = [itm[0] for itm in cursor][0] print "

Welcome, %s %s!

" % ([itm[1] for itm in cursor][0], [itm[2] for itm in cursor][0]) displayProducts(patientID) except: print "We're sorry. The email address and password you entered do not correspond with what is recorded in our database. Please click the "back" button and try again.

" else: displayProducts() cursor.execute('show tables like "%sPackages";' % store) if cursor.fetchone() is not None: sql = 'select ID from categories%s where Category="%s";' % (store[0].upper() + store[1:], cat) cursor.execute(sql) categoryID = cursor.fetchone()[0] sql = 'select p.ID from %sPackages p join %sCategoriesPackages c where c.CategoryID=%s;' % (store, store, categoryID) cursor.execute(sql) packageIDs = [itm[0] for itm in cursor] for pid in packageIDs: sql = 'select ProductID from %sProductsPackages where PackageID=%s;' % (store, pid) cursor.execute(sql) productIDs = [itm[0] for itm in cursor] sql = 'select Name, Price from %sPackages where ID=%s;' % (store, pid) # print sql print cursor.fetchall() name = [itm[0] for itm in cursor] price = [itm[1] for itm in cursor] print 'Package Name: %s
' % name print 'Price: %s
' % price allPrices = 0.00 for pid in productIDs: sql = 'select Name, Price from %s;' % store # print sql cursor.execute(sql) pName, pPrice = cursor.fetchone() pPrice = float(pPrice) allPrices += pPrice print 'Product Name: %s
' % pName print 'All products together usually cost: $%f

' % str(fpformat.fix(round(int(allPrices * 100))/100, 2)) cursor.close() bottom() display() TIA, beno -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Fri Jan 8 14:36:31 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 8 Jan 2010 15:36:31 -0400 Subject: getfirst and re In-Reply-To: <4B4785C6.4010801@mrabarnett.plus.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4B4751CB.2050907@mrabarnett.plus.com> <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> <4B4785C6.4010801@mrabarnett.plus.com> Message-ID: <4dc0cfea1001081136l3ea55531g5aafbe3a9125332a@mail.gmail.com> On Fri, Jan 8, 2010 at 3:21 PM, MRAB wrote: > Victor Subervi wrote: > > On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden > steve at holdenweb.com>> wrote: >> >> MRAB wrote: >> > Victor Subervi wrote: >> > [snip] >> >> >> >> Code snippet: >> >> >> >> def cgiFieldStorageToDict(fieldStorage): >> ^^^^^^^^^^^^ >> Further hint ... >> >> >> params = {} >> >> for key in fieldStorage.keys(): >> >> params[key] = cgi.FieldStorage[key].value >> > ^^^^^^^^^^^^^^^^^^^^^ >> > This is your problem. >> >> >> The problem is that I don't understand this code that I exactly copied >> from a Web page tutorial. Can you folks point me to tutorials where I can >> learn to comprehend this code? Specifically, the line in question. How is it >> that one can code "params[key]" (what does that mean?) and the other side, >> what does that mean >> >> If you got it from: > > Recipe 81547: Using a simple dictionary for CGI parameters > http://code.activestate.com/recipes/81547/ > > then no, it wasn't "exactly copied". If you'd copy-and-pasted it then it > would've been correct, as well as much quicker to do... I'm afraid to say you're right again :-} beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Fri Jan 8 14:37:49 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 08 Jan 2010 19:37:49 +0000 Subject: getfirst and re In-Reply-To: <4dc0cfea1001081115kffe51cdpb3b31be683822db9@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4B4751CB.2050907@mrabarnett.plus.com> <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> <4B477EDA.4000105@sequans.com> <4dc0cfea1001081109n384ed968sf955e02c8971cf7@mail.gmail.com> <4dc0cfea1001081115kffe51cdpb3b31be683822db9@mail.gmail.com> Message-ID: <4B47898D.1050807@mrabarnett.plus.com> Victor Subervi wrote: > On Fri, Jan 8, 2010 at 3:09 PM, Victor Subervi > wrote: > > > > On Fri, Jan 8, 2010 at 2:52 PM, Jean-Michel Pichavant > > wrote: > > Victor Subervi wrote: > > On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden > > >> > wrote: > > MRAB wrote: > > Victor Subervi wrote: > > [snip] > >> > >> Code snippet: > >> > >> def cgiFieldStorageToDict(fieldStorage): > ^^^^^^^^^^^^ > Further hint ... > > >> params = {} > >> for key in fieldStorage.keys(): > >> params[key] = cgi.FieldStorage[key].value > > ^^^^^^^^^^^^^^^^^^^^^ > > This is your problem. > > > The problem is that I don't understand this code that I > exactly copied from a Web page tutorial. Can you folks point > me to tutorials where I can learn to comprehend this code? > Specifically, the line in question. How is it that one can > code "params[key]" (what does that mean?) and the other > side, what does that mean > > I think you are gathering more fans Victor :) > > http://docs.python.org/tutorial/datastructures.html#dictionaries > > > This still isn't telling me what I need to know. Perhaps I'm missing > the point, as in the recent case with putting the "print cookie" > statement in the header. I am assuming (translation: making an a$$ > out of you and me) that "params[key] automatically assigns the > fieldStorage key to the newly created params key in the dict of the > same, and assigning the value from the cgi.FieldStorage of that key > to the params value. Is that correct? > TIA, > beno > > > I may have answered my own question. I have this: > > def cgiFieldStorageToDict(fieldStorage): > params = {} > for key in fieldStorage.keys(): > params[key] = fieldStorage[key].value > return params > Which is what Recipe 81547 actually says! > dict = cgiFieldStorageToDict(cgi.FieldStorage()) > print dict > > which gave me this: > > {'store': 'products', 'cat': 'prodCat1'} > > which looks about right. I would have written the code like this: > > keys = [] > values = [] > for key, value in fieldStorage.iteritems(): > keys.append(key) > values.append(value) > params = dict(zip(keys, values)) > > which obviously isn't as elegant. But that's what I knew. Learned > another trick, I guess ;) > If that works then so should this: params = dict(fieldStorage.iteritems()) From robert.kern at gmail.com Fri Jan 8 14:51:47 2010 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 08 Jan 2010 13:51:47 -0600 Subject: lightweight encryption of text file In-Reply-To: References: Message-ID: On 2010-01-08 13:14 PM, Daniel Fetchinson wrote: > I have a plain text file which I would like to protect in a very > simple minded, yet for my purposes sufficient, way. I'd like to > encrypt/convert it into a binary file in such a way that possession of > a password allows anyone to convert it back into the original text > file while not possessing the password one would only see the > following with the standard linux utility 'file': > > [fetchinson at fetch ~]$ file encrypted.data > encrypted.data: data > > and the effort required to convert the file back to the original text > file without the password would be equivalent to guessing the > password. > > I'm fully aware of the security implications of this loose > specification, but for my purposes this would be a good solution. > > What would be the simplest way to achieve this using preferably stock > python without 3rd party modules? If a not too complex 3rd party > module made it really simple that would be acceptable too. Paul Rubin's p3.py algorithm is probably the most straightforward way to meet these requirements. It's not a standard crypto algorithm by any means, but Paul knows his stuff and has devised it with these deployment restrictions in mind. http://www.nightsong.com/phr/crypto/p3.py -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From mackrackit at gmail.com Fri Jan 8 14:55:26 2010 From: mackrackit at gmail.com (Dave McCormick) Date: Fri, 8 Jan 2010 12:55:26 -0700 Subject: Dynamic text color In-Reply-To: <4B478779.2080908@mrabarnett.plus.com> References: <2a36ceaf1001081037n5ed5701ft9104c1396f36977c@mail.gmail.com> <4B478779.2080908@mrabarnett.plus.com> Message-ID: <2a36ceaf1001081155q72f771ebwf42fd00bf9d6e805@mail.gmail.com> On Fri, Jan 8, 2010 at 12:28 PM, MRAB wrote: > Dave McCormick wrote: > >> >> >> On Wed, Jan 6, 2010 at 9:18 AM, John Posner > jjposner at optimum.net>> wrote: >> >> On Tue, 05 Jan 2010 16:54:44 -0500, Dave McCormick >> > wrote: >> >> But it is not what I am wanting. I first thought to make it look >> for a space but that would not work when a single character like >> "#" is to be colored if there is a "string" of them. Or if all >> of the characters between quotes are to be colored. >> >> >> Regular expressions are good at handling searches like: >> >> * all the characters between quotes >> * the two-character string "do", but only if it's a complete word >> >> -John >> >> -- http://mail.python.org/mailman/listinfo/python-list >> >> I need another hint... >> >> Been doing some reading and playing and it looks like >> r'\bxxx\b' >> is what I need. But I can not figure out how to pass a variable between >> \b___\b >> If the word in question is between the "\b \b" and in the list then it >> works like I want it to. >> The below does not work. >> >> greenList_regexp = "|".join(greenList) >> for matchobj in re.finditer(r'\bgreenList_regexp\b', complete_text): >> start,end = matchobj.span() >> >> The regex r'\bgreenList_regexp\b' will match the string > 'greenList_regexp' if it's a whole word. > > What you mean is "any of these words, provided that they're whole > words". You'll need to group the alternatives within "(?:...)", like > this: > > r'\b(?:' + greenList_regexp + ')\b' > > > Thanks but that did not work either. Maybe I have something else wrong too? complete_text = Tbox.get("1.0", END) greenList = "green grass".split() greenList_regexp = "|".join(greenList) for matchobj in re.finditer(r'\b(?:' + greenList_regexp + ')\b', complete_text): start,end = matchobj.span() Tbox.tag_add("green", "1.0 + %d chars" % start,"1.0 + %d chars" % end) Tbox.tag_config("green", foreground="green") The words "green" and "grass" do not turn green. Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Fri Jan 8 15:01:55 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 8 Jan 2010 16:01:55 -0400 Subject: getfirst and re In-Reply-To: <4B47898D.1050807@mrabarnett.plus.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4B4751CB.2050907@mrabarnett.plus.com> <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> <4B477EDA.4000105@sequans.com> <4dc0cfea1001081109n384ed968sf955e02c8971cf7@mail.gmail.com> <4dc0cfea1001081115kffe51cdpb3b31be683822db9@mail.gmail.com> <4B47898D.1050807@mrabarnett.plus.com> Message-ID: <4dc0cfea1001081201q2ec3d6f5t8b85cd15479053f2@mail.gmail.com> On Fri, Jan 8, 2010 at 3:37 PM, MRAB wrote: > Victor Subervi wrote: > >> On Fri, Jan 8, 2010 at 3:09 PM, Victor Subervi > victorsubervi at gmail.com>> wrote: >> >> >> >> On Fri, Jan 8, 2010 at 2:52 PM, Jean-Michel Pichavant >> > wrote: >> >> Victor Subervi wrote: >> >> On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden >> >> >> >> >> wrote: >> >> MRAB wrote: >> > Victor Subervi wrote: >> > [snip] >> >> >> >> Code snippet: >> >> >> >> def cgiFieldStorageToDict(fieldStorage): >> ^^^^^^^^^^^^ >> Further hint ... >> >> >> params = {} >> >> for key in fieldStorage.keys(): >> >> params[key] = cgi.FieldStorage[key].value >> > ^^^^^^^^^^^^^^^^^^^^^ >> > This is your problem. >> >> >> The problem is that I don't understand this code that I >> exactly copied from a Web page tutorial. Can you folks point >> me to tutorials where I can learn to comprehend this code? >> Specifically, the line in question. How is it that one can >> code "params[key]" (what does that mean?) and the other >> side, what does that mean >> >> I think you are gathering more fans Victor :) >> >> http://docs.python.org/tutorial/datastructures.html#dictionaries >> >> >> This still isn't telling me what I need to know. Perhaps I'm missing >> the point, as in the recent case with putting the "print cookie" >> statement in the header. I am assuming (translation: making an a$$ >> out of you and me) that "params[key] automatically assigns the >> fieldStorage key to the newly created params key in the dict of the >> same, and assigning the value from the cgi.FieldStorage of that key >> to the params value. Is that correct? >> TIA, >> beno >> >> >> I may have answered my own question. I have this: >> >> def cgiFieldStorageToDict(fieldStorage): >> params = {} >> for key in fieldStorage.keys(): >> params[key] = fieldStorage[key].value >> return params >> >> Which is what Recipe 81547 actually says! > > > dict = cgiFieldStorageToDict(cgi.FieldStorage()) >> print dict >> >> which gave me this: >> >> {'store': 'products', 'cat': 'prodCat1'} >> >> which looks about right. I would have written the code like this: >> >> keys = [] >> values = [] >> for key, value in fieldStorage.iteritems(): >> keys.append(key) >> values.append(value) >> params = dict(zip(keys, values)) >> >> which obviously isn't as elegant. But that's what I knew. Learned another >> trick, I guess ;) >> >> If that works then so should this: > > params = dict(fieldStorage.iteritems()) Yeah. Thanks! beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From gnewsg at gmail.com Fri Jan 8 15:14:02 2010 From: gnewsg at gmail.com (Giampaolo Rodola') Date: Fri, 8 Jan 2010 12:14:02 -0800 (PST) Subject: asyncore based port splitter code questions References: Message-ID: On 4 Gen, 18:58, George Trojan wrote: > asyncore based code is supposed to be simple, > but I need while loops and a lot of try/except clauses. First of all: you DON'T have to use while loops or anything which is blocking where by "blocking" I mean anything like time.sleep(). asyncore, just like Twisted, is an asynchrounous abstraction on top of select(). Anything you call must return *immediately* otherwise the whole thing will hang aka "stop serving any connected client or server". > I designed the code by looking at Python 2.3 > source for asyncore and originally wanted to use add_channel() and > del_channel() methods. However in Python 2.6 del_channel() closes the > socket in addition to deleting it from the map. Don't look at the 2.3 source. Use asyncore of Python 2.6 which is far more improved, bug-fixed and also *different*, where by that I mean that it might actually behaves differently. If you are forced to use Python 2.3 my advice is to get a copy of Python's asyncore.py and asynchat.py and include them in your code. Secondly, to temporarily "sleep" your connections *don't* remove anything from your map. The correct way of doing things here is to override readable() and writable() methods and make them return False as long as you want your connection to hang. Now I'm going to comment some parts of your code. > class Reader(asyncore.dispatcher): > ? ? ?def __init__(self, sock, writers): > ? ? ? ? ?asyncore.dispatcher.__init__(self, sock) > ? ? ? ? ?self.writers = writers > > ? ? ?def handle_read(self): > ? ? ? ? ?data = self.recv(1024) > ? ? ? ? ?for writer in self.writers: > ? ? ? ? ? ? ?writer.add_data(data) [...] > ? ? ?def handle_write(self): > ? ? ? ? ?while self.data: > ? ? ? ? ? ? ?log_msg('sending data to %s' % str(self.address)) > ? ? ? ? ? ? ?sent = self.send(self.data) > ? ? ? ? ? ? ?self.data = self.data[sent:] > ? ? ? ? ?self.suspend_channel() By looking at how you are appending data you want to send in a buffer, it looks like you might want to use asynchat.async_chat rather than asyncore.dispatcher. async_chat.push() and async_chat.push_with_producer() methods already take care of buffers logic and make sure that all the data gets sent to the other peer without going lost. Actually there's no reason to use asyncore.dispatcher class directly except for creating a socket which listens on an interface and then passes the connection to another class inheriting from asynchat.async_chat which will actually handle that session. So my recommendation is to use asynchat.async_chat whenever possible. > ? ? ?def suspend_channel(self, map=None): > ? ? ? ? ?fd = self._fileno > ? ? ? ? ?if map is None: > ? ? ? ? ? ? ?map = self._map > ? ? ? ? ?if fd in map: > ? ? ? ? ? ? ?del map[fd] > As I said this is unecessary. Override readable() and writable() methods instead. > ? ? ?def handle_close(self): > ? ? ? ? ?log_msg('closing writer connection') > ? ? ? ? ?self.close() > ? ? ? ? ?# try to reconnect > ? ? ? ? ?time.sleep(TMOUT) > ? ? ? ? ?self.mksocket() You really don't want to use time.sleep() there. It blocks everything. > ? ? ? ? ?while True: > ? ? ? ? ? ? ?try: > ? ? ? ? ? ? ? ? ?disp = Dispatcher(port, destinations) > ? ? ? ? ? ? ? ? asyncore.loop(timeout=TMOUT, use_poll=True) > ? ? ? ? ? ? ?except socket.error, (errno, e): > ? ? ? ? ? ? ? ? ?if errno == 98: > ? ? ? ? ? ? ? ? ? ? ?log_msg('sleeping %d s: %s', (30, e)) > ? ? ? ? ? ? ? ? ? ? ?time.sleep(30) Same as above. As a final note I would recommend to take a look at pyftpdlib code which uses asyncore/asynchat as part of its core: http://code.google.com/p/pyftpdlib It can be of some help to figure out how things should be done. --- Giampaolo http://code.google.com/p/pyftpdlib/ From suresh.amritapuri at gmail.com Fri Jan 8 15:17:03 2010 From: suresh.amritapuri at gmail.com (suresh.amritapuri) Date: Fri, 8 Jan 2010 12:17:03 -0800 (PST) Subject: PIL show() not working for 2nd pic References: Message-ID: <5d0f9d9b-b0fb-47bb-a125-d7305a01e7fd@j5g2000yqm.googlegroups.com> On Jan 8, 6:32?am, McColgst wrote: > Do you get any errors or warnings? > Could we see the code you ran to get this problem? > > Thanks > Sean I used to get no such file or directory showing some files in /tmp directory. But today I am getting a different type of message, which appeared yesterday also. (eog:8368): Gtk-CRITICAL **: gtk_tree_model_get_iter: assertion `path- >depth > 0' failed (eog:8368): Gtk-CRITICAL **: gtk_list_store_get_value: assertion `VALID_ITER (iter, list_store)' failed (eog:8368): GLib-GObject-WARNING **: /build/buildd/glib2.0-2.20.1/ gobject/gtype.c:3940: type id `0' is invalid (eog:8368): GLib-GObject-WARNING **: can't peek value table for type `' which is not currently referenced Segmentation fault -----------code---------------------- #!/usr/bin/python print "Aum Amriteshwaryai Namaha" import Image imagePath = "/home/suresh/EE241/book_images_3ed/ch03/" im34 = Image.open(imagePath + "breast_digital_Xray.tif") im35 = Image.open(imagePath + "DFT_no_log.tif") im35.show() def neg(x): return 255-1-x import math def logtr(x): y = math.log(1+x,10) print y return y*100 im34x = im34.point(neg) im34x.show() im35x = im35.point(logtr) im35x.show() ---------------------------------------end of code Any ideas? Thanks suresh From dreadpiratejeff at gmail.com Fri Jan 8 15:31:35 2010 From: dreadpiratejeff at gmail.com (J) Date: Fri, 8 Jan 2010 15:31:35 -0500 Subject: table from csv file In-Reply-To: <77b18520-c2c4-4e21-9147-1164d81b9f25@l30g2000yqb.googlegroups.com> References: <59653a09-c3f7-464f-a36f-475f0d759e46@e27g2000yqd.googlegroups.com> <77b18520-c2c4-4e21-9147-1164d81b9f25@l30g2000yqb.googlegroups.com> Message-ID: <36dec4ff1001081231k62029267tb8df7ff16ad60ddb@mail.gmail.com> On Fri, Jan 8, 2010 at 13:55, Jon Clements wrote: > On Jan 8, 5:59?pm, marlowe wrote: >> I am trying to create a table in python from a csv file where I input >> which columns I would like to see, and the table only shows those >> columns. I have attached an example of the csv file i am using, and >> some of the code I have written. I am having trouble converting >> variables between lists, dictionaries and tuples. Is there a name for >> what I am attempting to do? any help to get me on the right track with >> this is appreciated. >> >> test.csv I had to edit that and comma delimit it, because cut and paste gave me random numbers/types of whitespace... [code snipped] > This might be a useful starting point (I'm guessing this is what > you're after...) > > Let's assume your 'CSV' file is tab separated as it's certainly not > comma separated :) > > import csv > csvin = csv.reader(open('test.csv'), delimiter='\t') > header = dict( (val.strip(),idx) for idx, val in enumerate(next > (csvin)) ) > > We can use header as a column name->column index lookup eg header > ['Open'] == 1 > > from operator import itemgetter > wanted = ['Open', 'Close'] # Although you'll want to use raw_input and > split on ',' > getcols = itemgetter(*[header[col] for col in wanted]) > > getcols is a helper function that'll return a tuple of the columns in > the requested order... > > for row in csvin: > ? ?print getcols(row) > > Loop over the rest of the file and output the required columns. As someone who knows just enough to be dangerous... what about this: import csv reader = open('C:/test.txt','rb') data = csv.DictReader(reader,restval='000',restkey='Misc') print "Options are: Date, Open, Close, High, Low, Volume, Adj Close" column = raw_input('What do you want? (comma delimited)?') choice = column.split(',') f = 12 print ''.join([s.center(f) for s in choice]) for row in data: print ''.join([(row.get(s)).center(f) for s in choice]) -- Mike Ditka - "If God had wanted man to play soccer, he wouldn't have given us arms." - http://www.brainyquote.com/quotes/authors/m/mike_ditka.html From carsten.haese at gmail.com Fri Jan 8 16:06:55 2010 From: carsten.haese at gmail.com (Carsten Haese) Date: Fri, 08 Jan 2010 16:06:55 -0500 Subject: Another Screwy Problem In-Reply-To: <4dc0cfea1001081132p191fe045p72a79744054fc520@mail.gmail.com> References: <4dc0cfea1001081132p191fe045p72a79744054fc520@mail.gmail.com> Message-ID: Victor Subervi wrote: > Hi; > I have this line of code: > sql = 'select Name, Price from %sPackages where ID=%s;' % (store, pid) > which prints to this: > select Name, Price from productsPackages where ID=1; > which when I enter it into the MySQL interpreter gives me this: > mysql> select Name, Price from productsPackages where ID=1; > +------+--------+ > | Name | Price | > +------+--------+ > | pkg | 123.45 | > +------+--------+ > 1 row in set (0.00 sec) > > exactly what I expect. However, in my script for some reason it returns > this: > ((1,),) The only logical explanation is that this is not the output from executing the above-mentioned query. Maybe you should ask yourself what it actually is. -Carsten From rsomerville at sjgeophysics.com Fri Jan 8 16:12:30 2010 From: rsomerville at sjgeophysics.com (Robert Somerville) Date: Fri, 08 Jan 2010 13:12:30 -0800 Subject: 24 bit signed integer binary conversion help needed Message-ID: <4B479FBE.7040206@sjgeophysics.com> hi; I am trying to read 24bit signed WAV format (little endian) data from a WAV file and convert it to 32 bit little endian integer format ... can anybody please tell me how to do the conversion from 24 bit to 32 bit with a snippet of Python code ??? Thanks so much Robert Somerville From hansmu at xs4all.nl Fri Jan 8 16:13:50 2010 From: hansmu at xs4all.nl (Hans Mulder) Date: Fri, 08 Jan 2010 22:13:50 +0100 Subject: Threading change, 2.5.4 -> 2.6.1 In-Reply-To: References: Message-ID: <4b47a083$0$22937$e4fe514c@news.xs4all.nl> anon wrote: > Gib Bogle wrote: >> The code below runs with Python 2.5.4, but gives the following error >> messages with Python 2.6.1. What needs to be done to make it work? >> Thanks. >> >> C:\Summer09\Tutorials>python url_queue.pyw >> Traceback (most recent call last): >> File "url_queue.pyw", line 3, in >> import threading >> File "C:\Summer09\Tutorials\threading.py", line 9, in >> class ProcessingThread(threading.Thread, QtCore.QObject): >> AttributeError: 'module' object has no attribute 'Thread' >> > > Nothing to do with the version of python > > You have a file called threading.py in C:\Summer09\Tutorials (the same > folder as url_queue.pyw). The line 'import threading' is finding this > module before python's threading module. > > Rename this file and you should be fine. Keep in mind that if there's a threading.pyc in C:\Summer09\Tutorials, you have to rename (or delete) that one as well. Hope this helps, -- HansM From invalid at invalid.invalid Fri Jan 8 16:24:36 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 8 Jan 2010 21:24:36 +0000 (UTC) Subject: 24 bit signed integer binary conversion help needed References: Message-ID: On 2010-01-08, Robert Somerville wrote: > I am trying to read 24bit signed WAV format (little endian) data from a > WAV file and convert it to 32 bit little endian integer format ... can > anybody please tell me how to do the conversion from 24 bit to 32 bit > with a snippet of Python code ??? def sext24(d): if ord(d[2]) & 0x80: return d+'\xff' else: return d+'\x00' -- Grant Edwards grante Yow! Am I having fun yet? at visi.com From jjposner at optimum.net Fri Jan 8 16:27:02 2010 From: jjposner at optimum.net (John Posner) Date: Fri, 08 Jan 2010 16:27:02 -0500 Subject: Dynamic text color References: <2a36ceaf1001081037n5ed5701ft9104c1396f36977c@mail.gmail.com> Message-ID: On Fri, 08 Jan 2010 14:28:57 -0500, MRAB wrote: > Dave McCormick wrote: >> On Wed, Jan 6, 2010 at 9:18 AM, John Posner > > wrote: >> On Tue, 05 Jan 2010 16:54:44 -0500, Dave McCormick >> > wrote: >> But it is not what I am wanting. I first thought to make it >> look >> for a space but that would not work when a single character like >> "#" is to be colored if there is a "string" of them. Or if all >> of the characters between quotes are to be colored. >> Regular expressions are good at handling searches like: >> * all the characters between quotes >> * the two-character string "do", but only if it's a complete word >> -John >> -- http://mail.python.org/mailman/listinfo/python-list >> I need another hint... >> Been doing some reading and playing and it looks like >> r'\bxxx\b' >> is what I need. But I can not figure out how to pass a variable between >> \b___\b >> If the word in question is between the "\b \b" and in the list then it >> works like I want it to. >> The below does not work. >> greenList_regexp = "|".join(greenList) >> for matchobj in re.finditer(r'\bgreenList_regexp\b', complete_text): >> start,end = matchobj.span() >> > The regex r'\bgreenList_regexp\b' will match the string > 'greenList_regexp' if it's a whole word. > > What you mean is "any of these words, provided that they're whole > words". You'll need to group the alternatives within "(?:...)", like > this: > > r'\b(?:' + greenList_regexp + ')\b' Oops, MRAB, you forgot to make the last literal a RAW string -- it should be r')\b' Dave, we're already into some pretty heavy regular-expression work, huh?. Here's another approach -- not nearly as elegant as MRAB's: Given this list: greenList = ['green', 'grass', 'grump'] ... you currently are using join() to construct this regexp search string: 'green|grass|grump' ... but you've decided that you really want this similar regexp search string: r'\bgreen\b|\bgrass\b|\bgrump\b' You can achieve this by transforming each item on the list, then invoking join() on the transformed list to create the search string. Here are a couple of ways to transform the list: * List comprehension: whole_word_greenList = [ r'\b' + word + r'\b' for word in greenList] * map() and a user-defined function: def xform_to_wholeword_searchstring(word): return r'\b' + word + r'\b' whole_word_greenList = map(xform_to_wholeword_searchstring, greenList) HTH, John From invalid at invalid.invalid Fri Jan 8 16:33:03 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 8 Jan 2010 21:33:03 +0000 (UTC) Subject: 24 bit signed integer binary conversion help needed References: Message-ID: On 2010-01-08, Grant Edwards wrote: > On 2010-01-08, Robert Somerville wrote: > >> I am trying to read 24bit signed WAV format (little endian) data from a >> WAV file and convert it to 32 bit little endian integer format ... can >> anybody please tell me how to do the conversion from 24 bit to 32 bit >> with a snippet of Python code ??? > > def sext24(d): > if ord(d[2]) & 0x80: > return d+'\xff' > else: > return d+'\x00' I guess I assumed you knew how to read 3 bytes of data from a file: f = open('datafile','rb') d = f.read(3) -- Grant Edwards grante Yow! I need to discuss at BUY-BACK PROVISIONS visi.com with at least six studio SLEAZEBALLS!! From irmen-NOSPAM- at xs4all.nl Fri Jan 8 16:34:13 2010 From: irmen-NOSPAM- at xs4all.nl (Irmen de Jong) Date: Fri, 08 Jan 2010 22:34:13 +0100 Subject: 24 bit signed integer binary conversion help needed In-Reply-To: References: Message-ID: <4b47a483$0$22903$e4fe514c@news.xs4all.nl> On 8-1-2010 22:12, Robert Somerville wrote: > hi; > I am trying to read 24bit signed WAV format (little endian) data from a > WAV file and convert it to 32 bit little endian integer format ... can > anybody please tell me how to do the conversion from 24 bit to 32 bit > with a snippet of Python code ??? > > Thanks so much > Robert Somerville Are you using the standard wave module? I guess that will produce a string of 3-byte audio frames with readframes(). Won't it work to chop this up in individual 3-byte frames, then appending a '\0' char to every frame, and then writing the frames to the target wave ? -irmen From invalid at invalid.invalid Fri Jan 8 16:37:26 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 8 Jan 2010 21:37:26 +0000 (UTC) Subject: 24 bit signed integer binary conversion help needed References: <4b47a483$0$22903$e4fe514c@news.xs4all.nl> Message-ID: On 2010-01-08, Irmen de Jong wrote: > On 8-1-2010 22:12, Robert Somerville wrote: >> hi; >> I am trying to read 24bit signed WAV format (little endian) data from a >> WAV file and convert it to 32 bit little endian integer format ... can >> anybody please tell me how to do the conversion from 24 bit to 32 bit >> with a snippet of Python code ??? > > Are you using the standard wave module? I guess that will > produce a string of 3-byte audio frames with readframes(). > > Won't it work to chop this up in individual 3-byte frames, > then appending a '\0' char to every frame, and then writing > the frames to the target wave ? Not if the 3-byte values are signed two's compliment values. -- Grant Edwards grante Yow! I feel like I am at sharing a ``CORN-DOG'' visi.com with NIKITA KHRUSCHEV ... From fetchinson at googlemail.com Fri Jan 8 16:39:39 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Fri, 8 Jan 2010 22:39:39 +0100 Subject: lightweight encryption of text file In-Reply-To: References: Message-ID: >> I have a plain text file which I would like to protect in a very >> simple minded, yet for my purposes sufficient, way. I'd like to >> encrypt/convert it into a binary file in such a way that possession of >> a password allows anyone to convert it back into the original text >> file while not possessing the password one would only see the >> following with the standard linux utility 'file': >> >> [fetchinson at fetch ~]$ file encrypted.data >> encrypted.data: data >> >> and the effort required to convert the file back to the original text >> file without the password would be equivalent to guessing the >> password. >> >> I'm fully aware of the security implications of this loose >> specification, but for my purposes this would be a good solution. >> >> What would be the simplest way to achieve this using preferably stock >> python without 3rd party modules? If a not too complex 3rd party >> module made it really simple that would be acceptable too. > > Paul Rubin's p3.py algorithm is probably the most straightforward way to > meet > these requirements. It's not a standard crypto algorithm by any means, but > Paul > knows his stuff and has devised it with these deployment restrictions in > mind. > > http://www.nightsong.com/phr/crypto/p3.py Thanks a lot, currently I'm having trouble using this code on python 2.6 but probably some small tweaking will fix it. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From suresh.amritapuri at gmail.com Fri Jan 8 16:43:45 2010 From: suresh.amritapuri at gmail.com (suresh.amritapuri) Date: Fri, 8 Jan 2010 13:43:45 -0800 (PST) Subject: PIL how to display multiple images side by side Message-ID: <6d9fef57-7fdc-46ec-9b53-7aa824e785e9@s31g2000yqs.googlegroups.com> Hi, In PIL, how to display multiple images in say m rows and n colums when I have m*n images. suresh From jcd at sdf.lonestar.org Fri Jan 8 16:44:55 2010 From: jcd at sdf.lonestar.org (J. Clifford Dyer) Date: Fri, 8 Jan 2010 16:44:55 -0500 Subject: Another Screwy Problem In-Reply-To: References: <4dc0cfea1001081132p191fe045p72a79744054fc520@mail.gmail.com> Message-ID: <20100108214455.GA12839@sdf.lonestar.org> Victor Subervi wrote: > Hi; > I have this line of code: > sql = 'select Name, Price from %sPackages where ID=%s;' % (store, pid) > which prints to this: > select Name, Price from productsPackages where ID=1; > which when I enter it into the MySQL interpreter gives me this: > mysql> select Name, Price from productsPackages where ID=1; > +------+--------+ > | Name | Price | > +------+--------+ > | pkg | 123.45 | > +------+--------+ > 1 row in set (0.00 sec) > > exactly what I expect. However, in my script for some reason it returns > this: > ((1,),) First, never use string formatting to pass parameters to your database. Read the MySQLdb documentation (or sqlite, or psycopg2) documentation for reasons why, and how to do it right. Second, in the same documentation, look up anything having the word "fetch" in it. That should show you how to get the data you want Third, please be more specific in your questions. You say "In my script for some reason *it* returns this: ((1,),)," but you don't show us what "it" is. How did you get that from your script? So far we've got a variable called sql with your query in it. How do I get the same wrong result you got? I don't know. I could pass it to a function that looks like this: def botch_sql_query(sql): return ((1,),) I could furthermore "fix" it, so that it looks right, by doing this: def fixed_sql_query(sql): return(('pkg', 123.45),) But that probably doesn't help. Give us enough code to be clear, but not so much as to be overwhelming. There's an essay called "How to ask smart question" by Eric Raymond that should help. Cheers, Cliff From fetchinson at googlemail.com Fri Jan 8 16:47:26 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Fri, 8 Jan 2010 22:47:26 +0100 Subject: lightweight encryption of text file In-Reply-To: References: Message-ID: >>> I have a plain text file which I would like to protect in a very >>> simple minded, yet for my purposes sufficient, way. I'd like to >>> encrypt/convert it into a binary file in such a way that possession of >>> a password allows anyone to convert it back into the original text >>> file while not possessing the password one would only see the >>> following with the standard linux utility 'file': >>> >>> [fetchinson at fetch ~]$ file encrypted.data >>> encrypted.data: data >>> >>> and the effort required to convert the file back to the original text >>> file without the password would be equivalent to guessing the >>> password. >>> >>> I'm fully aware of the security implications of this loose >>> specification, but for my purposes this would be a good solution. >>> >>> What would be the simplest way to achieve this using preferably stock >>> python without 3rd party modules? If a not too complex 3rd party >>> module made it really simple that would be acceptable too. >> >> Paul Rubin's p3.py algorithm is probably the most straightforward way to >> meet >> these requirements. It's not a standard crypto algorithm by any means, >> but >> Paul >> knows his stuff and has devised it with these deployment restrictions in >> mind. >> >> http://www.nightsong.com/phr/crypto/p3.py > > Thanks a lot, currently I'm having trouble using this code on python > 2.6 but probably some small tweaking will fix it. Actually, it also doesn't work with python 2.5 and currently I don't have access to anything older. array.array raises a ValueError: string length not a multiple of item size Does anyone recall a change to array.array? The full traceback is Traceback (most recent call last): File "p3.py", line 163, in _test() File "p3.py", line 143, in _test c1 = e(plain,key) File "p3.py", line 69, in p3_encrypt xkey = _expand_key(k_enc, n+4) File "p3.py", line 41, in _expand_key return array ('L', j) ValueError: string length not a multiple of item size Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From irmen-NOSPAM- at xs4all.nl Fri Jan 8 16:55:44 2010 From: irmen-NOSPAM- at xs4all.nl (Irmen de Jong) Date: Fri, 08 Jan 2010 22:55:44 +0100 Subject: 24 bit signed integer binary conversion help needed In-Reply-To: References: <4b47a483$0$22903$e4fe514c@news.xs4all.nl> Message-ID: <4b47a98e$0$22903$e4fe514c@news.xs4all.nl> On 8-1-2010 22:37, Grant Edwards wrote: > On 2010-01-08, Irmen de Jong wrote: >> Are you using the standard wave module? I guess that will >> produce a string of 3-byte audio frames with readframes(). >> >> Won't it work to chop this up in individual 3-byte frames, >> then appending a '\0' char to every frame, and then writing >> the frames to the target wave ? > > Not if the 3-byte values are signed two's compliment values. > Meh. The wave module is pretty useless for format conversion then until it grows some utility methods. -irmen From jcd at sdf.lonestar.org Fri Jan 8 16:57:35 2010 From: jcd at sdf.lonestar.org (J. Clifford Dyer) Date: Fri, 8 Jan 2010 16:57:35 -0500 Subject: Another Screwy Problem In-Reply-To: <4dc0cfea1001081132p191fe045p72a79744054fc520@mail.gmail.com> References: <4dc0cfea1001081132p191fe045p72a79744054fc520@mail.gmail.com> Message-ID: <20100108215735.GA22400@sdf.lonestar.org> On Fri, Jan 08, 2010 at 03:32:34PM -0400, Victor Subervi wrote regarding Another Screwy Problem: > Date: Fri, 8 Jan 2010 15:32:34 -0400 > From: Victor Subervi > To: python-list > Subject: Another Screwy Problem > > Hi; > I have this line of code: > sql = 'select Name, Price from %sPackages where ID=%s;' % (store, pid) > which prints to this: > select Name, Price from productsPackages where ID=1; > which when I enter it into the MySQL interpreter gives me this: > mysql> select Name, Price from productsPackages where ID=1; > +------+--------+ > | Name | Price | > +------+--------+ > | pkg | 123.45 | > +------+--------+ > 1 row in set (0.00 sec) > exactly what I expect. However, in my script for some reason it returns > this: > ((1,),) > Why would it do that? I guess there's some foolish thing I did in my > code somewhere, but I'll be darned if I know where. Here's the whole > script: > #! /usr/bin/python > import cgitb; cgitb.enable() > import cgi > import MySQLdb > import sys,os > sys.path.append(os.getcwd()) > from login import login > from template import top, bottom > from sets import Set > import fpformat > form = cgi.FieldStorage() > store = form.getfirst('store') > cat = form.getfirst('cat', '') > user, passwd, db, host = login() > db = MySQLdb.connect(host, user, passwd, db) > cursor = db.cursor() > def displayProducts(patientID=''): > try: # These are stores with categories where ordering by price is > important > sql = 'select ID from %s where Category="%s" order by Price desc;' > % (store, cat) > cursor.execute(sql) > except: # Stores, like prescriptions, where ordering by price is not > important > sql = 'select ID from %s;' % (store) > cursor.execute(sql) > ids = [itm[0] for itm in cursor] > cursor.execute('describe %s;' % store) > colFields, colFieldValues = [itm[0] for itm in cursor], [itm[1] for > itm in cursor] > i = 0 > if len(ids) > 0: > for id in ids: > # print '\n' > print '
' > print "" % store > print "" % id > j = 0 > for col in colFields: > sql = 'select %s from %s where ID="%s";' % (col, store, > str(id)) > cursor.execute(sql) > colValue = cursor.fetchone() > if col == 'SKU': > print "" % > colValue[0] > print '%s: %s
\n' % (col, colValue[0]) > elif col == 'Category': > print "" % > colValue[0] > print '%s: %s
\n' % (col, colValue[0]) > elif col == 'OutOfStock': > if colValue[0] == '1': # This product is out of stock > outOfStockFlag = 'yes' > else: > outOfStockFlag = 'no' > elif col[:3] != 'pic': > notSet = 1 > if isinstance(colValue[0], (str, int, float, long, complex, > unicode, list, buffer, xrange, tuple)): > pass > else: > try: > html = "%s:
" > print html > except: > pass > if notSet == 1: > if len(col) > 49: > colX = col[:50] + '...' > else: > colX = col > print '%s: %s
\n' % (colX, colValue[0]) > elif col == 'pic1': > try: > if (colValue[0] != None): > # if (colValue[0] != None) and (len(colValue[0] > 0)): > print ' class="highslide" href="getpic.py?store=%s&pic=%s&id=%s" > onclick="return hs.expand(this)"> src="getpic.py?store=%s&pic=%s&id=%s" width="100" height="80" alt="" > align="left" border="0" title="Click to enlarge" style="border: > 0px">
\n' % (store, col[3:], id, store, col[3:], > id, store, col[3:], id) > except TypeError: > raise > except: > raise > # i += 1 > # try: > # content = colValues[0][x].tostring() > # pic = "tmp" + str(i) + ".jpg" > # try: > # os.remove(pic) > # except: > # pass > # img = open(pic, "w") > # img.write(content) > # print '

' % pic > # img.close() > # except: > # pass > j += 1 > if store != 'prescriptions': > if outOfStockFlag == 'yes': > print 'This item is currently out of > stock.' > else: > print "" > else: > print "" % > patientID > print "" > print '


' > print '\n' > print '\n' > def cgiFieldStorageToDict(fieldStorage): > params = {} > for key in fieldStorage.keys(): > params[key] = fieldStorage[key].value > return params > def display(): > top() > dict = cgiFieldStorageToDict(cgi.FieldStorage()) > print dict > if store == 'prescriptions': > password = form.getfirst('password') > email = form.getfirst('email') > sql = 'select ID, FirstName, LastName from %s where PW="%s" and > Email="%s";' % ('patientsPersonalData', password, email) > try: > cursor.execute(sql) > patientID = [itm[0] for itm in cursor][0] > print "

Welcome, %s %s!

" % ([itm[1] for itm in > cursor][0], [itm[2] for itm in cursor][0]) > displayProducts(patientID) > except: > print "We're sorry. The email address and password you entered do > not correspond with what is recorded in our database. Please click the > "back" button and try again.

" > else: > displayProducts() > cursor.execute('show tables like "%sPackages";' % store) > if cursor.fetchone() is not None: > sql = 'select ID from categories%s where Category="%s";' % > (store[0].upper() + store[1:], cat) > cursor.execute(sql) > categoryID = cursor.fetchone()[0] > sql = 'select p.ID from %sPackages p join %sCategoriesPackages c > where c.CategoryID=%s;' % (store, store, categoryID) > cursor.execute(sql) > packageIDs = [itm[0] for itm in cursor] > for pid in packageIDs: > sql = 'select ProductID from %sProductsPackages where > PackageID=%s;' % (store, pid) > cursor.execute(sql) > productIDs = [itm[0] for itm in cursor] > sql = 'select Name, Price from %sPackages where ID=%s;' % > (store, pid) > # print sql > print cursor.fetchall() > name = [itm[0] for itm in cursor] > price = [itm[1] for itm in cursor] > print 'Package Name: %s
' % name > print 'Price: %s
' % price > allPrices = 0.00 > for pid in productIDs: > sql = 'select Name, Price from %s;' % store > # print sql > cursor.execute(sql) > pName, pPrice = cursor.fetchone() > pPrice = float(pPrice) > allPrices += pPrice > print 'Product Name: %s
' % pName > print 'All products together usually cost: $%f

' % > str(fpformat.fix(round(int(allPrices * 100))/100, 2)) > cursor.close() > bottom() > display() > TIA, > beno > -- > The Logos has come to bear > [1]http://logos.13gems.com/ > > References > > 1. http://logos.13gems.com/ > -- > http://mail.python.org/mailman/listinfo/python-list Sigh. Ignore my last message. I was reading the part that Caersten quoted, not the original message. Well, ignore the part where I say you didn't show us your code. You obviously did. However, you showed us so much code that it's hard to find your problem. Before you ask questions, try to trim your code down to just enough to demonstrate the problem. In this case: >>> db = MySQLdb.connect(host, user, passwd, db) >>> cursor = db.cursor() >>> sql = 'select Name, Price from %sPackages where ID=%s;' % (store, pid) >>> print cursor.fetchall() ((1,),) Would have been sufficient. Then you could ask, "why does cursor.fetchall() return ((1,),) instead of the expected results. (The answer is that you never executed your query.) From irmen-NOSPAM- at xs4all.nl Fri Jan 8 17:02:00 2010 From: irmen-NOSPAM- at xs4all.nl (Irmen de Jong) Date: Fri, 08 Jan 2010 23:02:00 +0100 Subject: lightweight encryption of text file In-Reply-To: References: Message-ID: <4b47ab07$0$22919$e4fe514c@news.xs4all.nl> On 8-1-2010 22:39, Daniel Fetchinson wrote: >> >> http://www.nightsong.com/phr/crypto/p3.py > > Thanks a lot, currently I'm having trouble using this code on python > 2.6 but probably some small tweaking will fix it. If you keep having issues with this module, maybe you can try this: http://www.freenet.org.nz/ezPyCrypto/ It provides a very easy compact api on top of PyCrypto. So you'll need to install that as well to be able to use this. -irmen From robert.kern at gmail.com Fri Jan 8 17:09:27 2010 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 08 Jan 2010 16:09:27 -0600 Subject: lightweight encryption of text file In-Reply-To: References: Message-ID: On 2010-01-08 15:47 PM, Daniel Fetchinson wrote: >>>> I have a plain text file which I would like to protect in a very >>>> simple minded, yet for my purposes sufficient, way. I'd like to >>>> encrypt/convert it into a binary file in such a way that possession of >>>> a password allows anyone to convert it back into the original text >>>> file while not possessing the password one would only see the >>>> following with the standard linux utility 'file': >>>> >>>> [fetchinson at fetch ~]$ file encrypted.data >>>> encrypted.data: data >>>> >>>> and the effort required to convert the file back to the original text >>>> file without the password would be equivalent to guessing the >>>> password. >>>> >>>> I'm fully aware of the security implications of this loose >>>> specification, but for my purposes this would be a good solution. >>>> >>>> What would be the simplest way to achieve this using preferably stock >>>> python without 3rd party modules? If a not too complex 3rd party >>>> module made it really simple that would be acceptable too. >>> >>> Paul Rubin's p3.py algorithm is probably the most straightforward way to >>> meet >>> these requirements. It's not a standard crypto algorithm by any means, >>> but >>> Paul >>> knows his stuff and has devised it with these deployment restrictions in >>> mind. >>> >>> http://www.nightsong.com/phr/crypto/p3.py >> >> Thanks a lot, currently I'm having trouble using this code on python >> 2.6 but probably some small tweaking will fix it. > > Actually, it also doesn't work with python 2.5 and currently I don't > have access to anything older. array.array raises a > > ValueError: string length not a multiple of item size > > Does anyone recall a change to array.array? > > The full traceback is > > Traceback (most recent call last): > File "p3.py", line 163, in > _test() > File "p3.py", line 143, in _test > c1 = e(plain,key) > File "p3.py", line 69, in p3_encrypt > xkey = _expand_key(k_enc, n+4) > File "p3.py", line 41, in _expand_key > return array ('L', j) > ValueError: string length not a multiple of item size Are you on a 64-bit platform? Unfortunately, array's integer typecodes are platform-specific, but p3.py requires a 32-bit integer and was written on a 32-bit platform. It's reasonably straightforward to fix. Put this bit of (untested) code at the top of the file and replace occurrences of 'L' with uint32: # Find the typecode of a 32-bit unsigned integer. for typecode in 'IL': if len(array(typecode, [0]).tostring()) == 4: uint32 = typecode break else: raise RuntimeError("Neither 'I' nor 'L' are unsigned 32-bit integers.") -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From rsomerville at sjgeophysics.com Fri Jan 8 17:11:22 2010 From: rsomerville at sjgeophysics.com (Robert Somerville) Date: Fri, 08 Jan 2010 14:11:22 -0800 Subject: 24 bit signed integer binary conversion help needed Message-ID: <4B47AD8A.3040406@sjgeophysics.com> An HTML attachment was scrubbed... URL: From mackrackit at gmail.com Fri Jan 8 17:21:07 2010 From: mackrackit at gmail.com (Dave McCormick) Date: Fri, 08 Jan 2010 15:21:07 -0700 Subject: Dynamic text color In-Reply-To: References: <2a36ceaf1001081037n5ed5701ft9104c1396f36977c@mail.gmail.com> Message-ID: <4B47AFD3.8040507@gmail.com> John Posner wrote: > On Fri, 08 Jan 2010 14:28:57 -0500, MRAB > wrote: > >> The regex r'\bgreenList_regexp\b' will match the string >> 'greenList_regexp' if it's a whole word. >> >> What you mean is "any of these words, provided that they're whole >> words". You'll need to group the alternatives within "(?:...)", like >> this: >> >> r'\b(?:' + greenList_regexp + ')\b' > > Oops, MRAB, you forgot to make the last literal a RAW string -- it > should be r')\b' > > Dave, we're already into some pretty heavy regular-expression work, > huh?. Here's another approach -- not nearly as elegant as MRAB's: > > Given this list: > > greenList = ['green', 'grass', 'grump'] > > ... you currently are using join() to construct this regexp search > string: > > 'green|grass|grump' > > ... but you've decided that you really want this similar regexp search > string: > > r'\bgreen\b|\bgrass\b|\bgrump\b' > > You can achieve this by transforming each item on the list, then > invoking join() on the transformed list to create the search string. > Here are a couple of ways to transform the list: > > * List comprehension: > > whole_word_greenList = [ r'\b' + word + r'\b' for word in greenList] > > * map() and a user-defined function: > > def xform_to_wholeword_searchstring(word): > return r'\b' + word + r'\b' > > whole_word_greenList = map(xform_to_wholeword_searchstring, greenList) > > > HTH, > John John, That second "r" appears to do the trick. Yea, pretty heavy into it. I read someplace that regular-expressions were tricky, but I did not expect this :) Now to start working this into the rest of my app and study your second approach. Thanks again for the help!!! Dave From sccolbert at gmail.com Fri Jan 8 17:21:16 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Fri, 8 Jan 2010 23:21:16 +0100 Subject: lightweight encryption of text file In-Reply-To: <4b47ab07$0$22919$e4fe514c@news.xs4all.nl> References: <4b47ab07$0$22919$e4fe514c@news.xs4all.nl> Message-ID: <7f014ea61001081421w42e95ccob56f16ac1ead0fa0@mail.gmail.com> PyCrypto is already pretty easy to use by itself. I dont know why you want a wrapper on top of it. On Fri, Jan 8, 2010 at 11:02 PM, Irmen de Jong wrote: > On 8-1-2010 22:39, Daniel Fetchinson wrote: > >> >>> http://www.nightsong.com/phr/crypto/p3.py >>> >> >> Thanks a lot, currently I'm having trouble using this code on python >> 2.6 but probably some small tweaking will fix it. >> > > If you keep having issues with this module, maybe you can try this: > > http://www.freenet.org.nz/ezPyCrypto/ > > It provides a very easy compact api on top of PyCrypto. So you'll need to > install that as well to be able to use this. > > -irmen > > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsomerville at sjgeophysics.com Fri Jan 8 17:25:38 2010 From: rsomerville at sjgeophysics.com (Robert Somerville) Date: Fri, 08 Jan 2010 14:25:38 -0800 Subject: 24 bit signed integer binary conversion help needed Message-ID: <4B47B0E2.3040500@sjgeophysics.com> thanks Grant, your sext24 function does the trick (this Python newbie thanks you a lot ...) so i loop on d=wave.readframes(1) call dd=sext24(d) on the frame read , then ddd=struct.unpack("<%ul" % 1 ,dd) unpack(dd) the output of sext24 into a 32bit integer (seems to do the trick correctly for signed 24 bit integers ....) -thanks , bob From ben+python at benfinney.id.au Fri Jan 8 17:44:31 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 09 Jan 2010 09:44:31 +1100 Subject: Scripting References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: <87pr5kushc.fsf@benfinney.id.au> Jorgen Grahn writes: > Regarding the book's title: is it just me, or are Python programmers > in general put off when people call it "scripting"? The term ?script? for a Python program is part of the official terminology, so those who would be put off by that term have to at least contend with that fact. I refer to such things as ?programs?, whatever language they're written in, since ?script? does seem to have derogatory connotations. I don't get particularly off-put either way. -- \ ?He was the mildest-mannered man / That ever scuttled ship or | `\ cut a throat.? ??Lord? George Gordon Noel Byron, _Don Juan_ | _o__) | Ben Finney From ben+python at benfinney.id.au Fri Jan 8 17:54:30 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 09 Jan 2010 09:54:30 +1100 Subject: Recommended "new" way for config files References: <4B46075D.8080606@mycircuit.org> <4B460CA0.20800@sequans.com> <4B4625BC.6090201@mycircuit.org> Message-ID: <87iqbcus0p.fsf@benfinney.id.au> Chris Rebert writes: > JSON is one option: http://docs.python.org/library/json.html YAML is another contender. Compared to JSON, it is yet to gain as much mind-share, but even more human-friendly and no less expressive. Here are some discussions of YAML that can help you evaluate it: -- \ ?The Things to do are: the things that need doing, that you see | `\ need to be done, and that no one else seems to see need to be | _o__) done.? ?Richard Buckminster Fuller, 1970-02-16 | Ben Finney From tjreedy at udel.edu Fri Jan 8 18:48:43 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 08 Jan 2010 18:48:43 -0500 Subject: Caching objects in a C extension In-Reply-To: <4ff69845-b7c6-4b0c-a0c5-c4154551098c@m25g2000yqc.googlegroups.com> References: <1bd0de37-1a32-44f4-91fa-d3126e3aaeda@u7g2000yqm.googlegroups.com> <7qohgkF2mjU2@mid.uni-berlin.de> <7qp7opFglrU1@mid.uni-berlin.de> <4ff69845-b7c6-4b0c-a0c5-c4154551098c@m25g2000yqc.googlegroups.com> Message-ID: On 1/8/2010 12:50 PM, casevh wrote: > These are numeric objects created by gmpy. I'm trying to minimize the > overhead for using mpz with small numbers. Objects are created and > deleted very often by the interpreter as expressions are evaluated. I > don't keep ownership of the objects. CPython creates a hidden list of small ints on startup for the same reason. Currently about -10 to 256. int.__new__ checks that a requested int object would have a value in this range (if so, reuse) -- or not (create new object). From tjreedy at udel.edu Fri Jan 8 18:52:21 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 08 Jan 2010 18:52:21 -0500 Subject: Standardized interpreter speed evaluation tool In-Reply-To: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> References: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> Message-ID: On 1/8/2010 5:25 AM, alexru wrote: > Is there any standardized interpreter speed evaluation tool? Say I > made few changes in interpreter code and want to know if those changes > made python any better, which test should I use? The Unladen Swallow (sp?) project at code.google.com/????, which looks to change and speedup CPython, has a suite of benchmarks that are are perhaps better than PyBench. From george.trojan at noaa.gov Fri Jan 8 18:58:51 2010 From: george.trojan at noaa.gov (George Trojan) Date: Fri, 08 Jan 2010 23:58:51 +0000 Subject: asyncore based port splitter code questions In-Reply-To: References: Message-ID: Thanks for your help. Some comments below. George Giampaolo Rodola' wrote: > On 4 Gen, 18:58, George Trojan wrote: > > > Secondly, to temporarily "sleep" your connections *don't* remove > anything from your map. > The correct way of doing things here is to override readable() and > writable() methods and make them return False as long as you want your > connection to hang. > Good advice. > Now I'm going to comment some parts of your code. > >> class Reader(asyncore.dispatcher): >> def __init__(self, sock, writers): >> asyncore.dispatcher.__init__(self, sock) >> self.writers = writers >> >> def handle_read(self): >> data = self.recv(1024) >> for writer in self.writers: >> writer.add_data(data) > [...] >> def handle_write(self): >> while self.data: >> log_msg('sending data to %s' % str(self.address)) >> sent = self.send(self.data) >> self.data = self.data[sent:] >> self.suspend_channel() > > > By looking at how you are appending data you want to send in a buffer, > it looks like you might want to use asynchat.async_chat rather than > asyncore.dispatcher. > async_chat.push() and async_chat.push_with_producer() methods already > take care of buffers logic and make sure that all the data gets sent > to the other peer without going lost. > > Actually there's no reason to use asyncore.dispatcher class directly > except for creating a socket which listens on an interface and then > passes the connection to another class inheriting from > asynchat.async_chat which will actually handle that session. My understanding is that asynchat is used for bi-directional connection, I don't see how it applies to my case (forwarding data). However I rewrote the Writer class following some of asynchat code. > > So my recommendation is to use asynchat.async_chat whenever possible. > You really don't want to use time.sleep() there. > It blocks everything. > >> while True: >> try: >> disp = Dispatcher(port, destinations) >> asyncore.loop(timeout=TMOUT, use_poll=True) >> except socket.error, (errno, e): >> if errno == 98: >> log_msg('sleeping %d s: %s', (30, e)) >> time.sleep(30) > > Same as above. > I wanted to reconnect after the os cleans up half-closed sockets. Otherwise the program exits immediately with a message: terminating - uncaught exception: [Errno 98] Address already in use > > As a final note I would recommend to take a look at pyftpdlib code > which uses asyncore/asynchat as part of its core: > http://code.google.com/p/pyftpdlib > It can be of some help to figure out how things should be done. > Thanks for good example to study. > > > --- Giampaolo > http://code.google.com/p/pyftpdlib/ From clp2 at rebertia.com Fri Jan 8 19:02:46 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Fri, 8 Jan 2010 16:02:46 -0800 Subject: Standardized interpreter speed evaluation tool In-Reply-To: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> References: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> Message-ID: <50697b2c1001081602m7614b94esc4cd9241fafbdf49@mail.gmail.com> On Fri, Jan 8, 2010 at 2:25 AM, alexru wrote: > Is there any standardized interpreter speed evaluation tool? Say I > made few changes in interpreter code and want to know if those changes > made python any better, which test should I use? Although apparently undocumented, test.pystone is some sort of interpreter benchmark. Cheers, Chris -- http://blog.rebertia.com From tjreedy at udel.edu Fri Jan 8 19:35:39 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 08 Jan 2010 19:35:39 -0500 Subject: One function calling another defined in the same file being exec'd In-Reply-To: References: Message-ID: On 1/8/2010 12:02 PM, Mitchell L Model wrote: > On further reflection, I will add that > what appears to be happening is that during import both the global and > local dictionaries are set to a copy of the globals() from the importing > scope and that copy becomes the value of the module's __dict__ once > import has completed successfully. I have no idea why you think that. The module dict starts empty except for __name__, __file__, and perhaps a couple of other 'hidden' items. It is not a copy and has nothing to do with importing scopes. > and that copy becomes the value of the module's __dict__ once > import has completed successfully. That new dict becomes .... . > Because exec leaves locals() and globals() distinct, Not necessarily. In 3.x, at least, exec(s) executes s in the current scope. If this is top level, where locals is globals, then same should be true within exec. d = {} exec(s, d) In 3.x, at least, d will also be used as locals. exec(s, d, d) Again, globals and locals are not distinct. It would seem that in 3.x, the only way for exec to have distinct globals and locals is to call exec(s) where they are distinct or to pass distince globals and locals. Some of the issues of this thread are discussed in Language Reference 4.1, Naming and Binding. I suppose it could be clearer that it is, but the addition of nonlocal scope complicated things. Terry Jan Reedy From steve at REMOVE-THIS-cybersource.com.au Fri Jan 8 20:50:59 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 09 Jan 2010 01:50:59 GMT Subject: lightweight encryption of text file References: Message-ID: <0357cddd$0$1336$c3e8da3@news.astraweb.com> On Fri, 08 Jan 2010 20:14:51 +0100, Daniel Fetchinson wrote: > I have a plain text file which I would like to protect in a very simple > minded, yet for my purposes sufficient, way. I'd like to encrypt/convert > it into a binary file in such a way that possession of a password allows > anyone to convert it back into the original text file while not > possessing the password one would only see the following with the > standard linux utility 'file': > > [fetchinson at fetch ~]$ file encrypted.data > encrypted.data: data If that is your sole requirement, then the addition of a single non-text byte (say, a null) anywhere in the file would be sufficient to have file identify it as data. You say "encrypt/convert" -- does this mean that you don't care if people can read the text in a hex editor, so long as file identifies it as data? Would something like a binary Vigenere Cipher be sufficient? # Untested def encrypt(plaintext, password): cipher = [] for i, c in enumerate(plaintext): shift = password[i % len(password)] shift = ord(shift) cipher.append((ord(c) + shift) % 256) return ''.join([chr(n) for n in cipher]) def decrypt(ciphertext, password): plain = [] for i, c in enumerate(ciphertext): shift = password[i % len(password)] shift = ord(shift) plain.append((256 + ord(c) - shift) % 256) return ''.join([chr(n) for n in plain]) (How times have changed... once upon a time, the Vigenere Cipher was considered the gold-standard unbreakable encryption technology. Now it merely qualifies as obfuscation.) Is it acceptable if there is a chance (small, possibly vanishingly small) that file will identify it as text? As far as I know, even the heavyweight "serious" encryption algorithms don't *guarantee* that the cipher text will include non-text bytes. > and the effort required to convert the file back to the original text > file without the password would be equivalent to guessing the password. If you seriously mean that, then "lightweight encryption" won't do the job, because it is vulnerable to frequency analysis, which is easier than guessing the password. You need proper, heavy-weight encryption. > I'm fully aware of the security implications of this loose > specification, but for my purposes this would be a good solution. Can you explain what your objection to real encryption is? Are you concerned about the complex API? The memory requirements and processing power required? (Neither of which are particularly high for small text files on a modern PC, but perhaps you have to encrypt tens of millions of huge files on an underpowered device...) > What would be the simplest way to achieve this using preferably stock > python without 3rd party modules? If a not too complex 3rd party module > made it really simple that would be acceptable too. The problem is that, as I see it, you've assumed a solution rather than state what your requirements are. I'm *guessing* that you are more concerned of having to learn to use a complex API, rather than actually *requiring* a lightweight encryption algorithm. If that's the case, then something serious like blowfish or similar would be perfectly acceptable to you, so long as the API was simple. (On the other hand, perhaps vulnerability to frequency analysis is a feature, not a bug, in your use-case. If you forget the password, you have a chance of recovering the text.) -- Steven From diesch at spamfence.net Fri Jan 8 21:29:41 2010 From: diesch at spamfence.net (Florian Diesch) Date: Sat, 09 Jan 2010 03:29:41 +0100 Subject: Scripting (was Re: Python books, literature etc) References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: Jorgen Grahn writes: > Regarding the book's title: is it just me, or are Python programmers > in general put off when people call it "scripting"? > > I won't attempt a strict definition of the term "scripting language", > but it seems like non-programmers use it to mean "less scary than what > you might think of as programming", while programmers interpret it as > "not useful as a general-purpose language". For me "scripting" means something like "task automation within a given program or environment", in contrast to "wring a stand-alone program". Florian -- From diesch at spamfence.net Fri Jan 8 21:32:05 2010 From: diesch at spamfence.net (Florian Diesch) Date: Sat, 09 Jan 2010 03:32:05 +0100 Subject: restructuredText editor ? References: Message-ID: <5e6l17-28k.ln1@mid.florian-diesch.de> Peter writes: > What editor do people out there use to edit .rst files for > sphinx-python documentation ? Emacs with ReST mode and YASnippet Florian -- From half.italian at gmail.com Fri Jan 8 22:43:04 2010 From: half.italian at gmail.com (Sean DiZazzo) Date: Fri, 8 Jan 2010 19:43:04 -0800 (PST) Subject: PIL how to display multiple images side by side References: <6d9fef57-7fdc-46ec-9b53-7aa824e785e9@s31g2000yqs.googlegroups.com> Message-ID: <6057f3b6-00f9-4fd2-b82f-65844e30d96f@v25g2000yqk.googlegroups.com> On Jan 8, 1:43?pm, "suresh.amritapuri" wrote: > Hi, > > In PIL, how to display multiple images in say m rows and n colums when > I have m*n images. > > suresh Sounds like a good project to learn PIL with. ~Sean From tosters at gmail.com Fri Jan 8 22:56:48 2010 From: tosters at gmail.com (t0ster) Date: Fri, 8 Jan 2010 19:56:48 -0800 (PST) Subject: Python multiprocessing: Permission denied References: <1c85295e-11ee-471c-bd2f-420e0f2fc3b6@j4g2000yqe.googlegroups.com> <0353b63c$0$1336$c3e8da3@news.astraweb.com> Message-ID: <87ebeabb-7336-4acd-9196-cd6d180587c9@e27g2000yqd.googlegroups.com> On Jan 6, 1:20?am, Steven D'Aprano wrote: > On Tue, 05 Jan 2010 13:52:18 -0800,t0sterwrote: > > It looks like the user don't have permission to access shared memory. > > When executing with root privileges it works fine. > > > Is there any solution to run it as normal user(not root)? > > Then give the user permission to access shared memory. > > Why do you expect that Python would be able to over-ride the operating > system's security? This problem is no different from saying "It looks > like the user doesn't have permission to access this file". > > -- > Steven Yes, your are right. The problem was solved by chmoding /dev/shm From casevh at gmail.com Fri Jan 8 23:33:45 2010 From: casevh at gmail.com (casevh) Date: Fri, 8 Jan 2010 20:33:45 -0800 (PST) Subject: Caching objects in a C extension References: <1bd0de37-1a32-44f4-91fa-d3126e3aaeda@u7g2000yqm.googlegroups.com> <7qohgkF2mjU2@mid.uni-berlin.de> Message-ID: On Jan 8, 8:56?am, Antoine Pitrou wrote: > Le Fri, 08 Jan 2010 08:39:17 -0800, casevh a ?crit?: > > > > > Thanks for the reply. I realized that I missed one detail. The objects > > are created by the extension but are deleted by Python. I don't know > > that an object is no longer needed until its tp_dealloc is called. At > > that point, its reference count is 0. > > tuple objects and others already have such a caching scheme, so you could > download the Python source and look at e.g. Objects/tupleobject.c to see > how it's done. > > Regards > > Antoine. Thanks. That's exactly the information I was looking for. casevh From metolone+gmane at gmail.com Sat Jan 9 00:09:05 2010 From: metolone+gmane at gmail.com (Mark Tolonen) Date: Fri, 8 Jan 2010 21:09:05 -0800 Subject: Manipulating pointers in C using ctypes References: <08bda34e-9bee-44b1-b2de-80d647151786@a15g2000yqm.googlegroups.com> Message-ID: "Daniel Platz" wrote in message news:08bda34e-9bee-44b1-b2de-80d647151786 at a15g2000yqm.googlegroups.com... > Hello! > > I have to ask a newbie question about manipulating pointers in C using > ctypes. I have a C dll with two functions. The first one creates a > pointer and returns it to python. The second one takes a pointer as an > argument shows its address and the value at which it is pointing. This > I have implemented using the following code > > ----------- -------- pointers.c ---------------------------- > #include > #ifdef __cplusplus > extern "C" { // only need to export C interface if > // used by C++ source code > using namespace std; > #endif > > __declspec(dllexport) void* create() > { > double number = 2.2; > double* ptr = &number; > printf("Pointer address \t %p \n", ptr); > printf("Value at pointer \t %f \n", ptr[0]); > return (void*) ptr; > } > > __declspec(dllexport) int show(double* ptr) > { > printf("Pointer address \t %p \n", ptr); > printf("Pointer value \t %f\n", *ptr); > *ptr = 2.4; > printf("New pointer value \t %f\n", *ptr); > return 0; > } > > #ifdef __cplusplus > } > #endif > ------------------------------------------------------------------------ > > Please note that in the second function, the show function, I want to > manipulate the value at which the pointer points. > Now, I call this function from python with the following script. > > --------------------------- pointers.py -------------------------- > import ctypes as ct > > # Load dll > pointers = ct.cdll.LoadLibrary('pointers.dll') > getattr(pointers, 'create') > getattr(pointers, 'show') > pointers.create.restype = ct.c_void_p > > # Create pointer in C > ptr = pointers.create() > > # Show pointer address and value > print 'Adress returned to python ' +hex(ptr) > pointers.show(ct.c_void_p(ptr)) > ------------------------------------------------------------------- > > Calling this script gives me the following output: > > Pointer address 0021E508 > Value at pointer 2.200000 > Adress returned to python 0x21e508 > Pointer address 0021E508 > Pointer value 0.000000 (2.20000 expected) > New pointer value 2.400000 (2.40000 expected) > > But the script returns also an error. > > WindowsError: exception: access violation reading 0x40033333 > WARNING: Failure executing file: > > Another thing that I find strange is that the return value of > pointers.create is actually an integer instead of an ct.c_void_p > object. > > Moreover, I also tried to directly manipulate the address of the > pointer given as an argument to pointers.show. But when it returns to > python the pointer points still at the same address as before the > function call. > > Can someone help me with this problem? I would be very glad about an > answer. As Jason observed, your create() function creates a number on the local stack, which becomes invalid once the function returns. If you don't want to worry about freeing the object created in Jason's solution, you can create the double in python yourself and let create initialize it: -------------- pointers.py ---------------------- import ctypes as ct pointers = ct.CDLL('pointers.dll') pointers.create.restype = None pointers.show.restype = None dbl = ct.c_double() print ct.byref(dbl),dbl ptr = pointers.create(ct.byref(dbl)) # Show pointer address and value print 'Value returned to python: %f' % dbl.value pointers.show(ct.byref(dbl)) print 'Value returned to python: %f' % dbl.value -------------- pointers.c ------------------------ #include __declspec(dllexport) void create(double* ptr) { *ptr = 2.2; printf("Pointer address %p\n", ptr); printf("Value at pointer %f\n", *ptr); } __declspec(dllexport) void show(double* ptr) { printf("Pointer address %p\n", ptr); printf("Pointer value %f\n", *ptr); *ptr = 2.4; printf("New pointer value %f\n", *ptr); } -------------- OUTPUT ------------------------- c_double(0.0) Pointer address 00A05DC8 Value at pointer 2.200000 Value returned to python: 2.200000 Pointer address 00A05DC8 Pointer value 2.200000 New pointer value 2.400000 Value returned to python: 2.400000 ----------------------------------------------------- HTH, Mark From metolone+gmane at gmail.com Sat Jan 9 00:28:21 2010 From: metolone+gmane at gmail.com (Mark Tolonen) Date: Fri, 8 Jan 2010 21:28:21 -0800 Subject: C Structure rebuild with ctypes References: <7p6ksnFkg1U1@mid.individual.net> <7qo9avFivmU1@mid.individual.net> Message-ID: "Georg" wrote in message news:7qo9avFivmU1 at mid.individual.net... > Hi Mark, > >> Are you passing in these values, or are they being returned? To me the >> depth of the pointer references implies numVars, varNames, and varTypes >> are out parameters. I'll assume that for now. If they are in/out >> parameters let me know. > > If these parameters were in parameters. What would I have to do different? If you are passing in the values, you can remove a level of * referencing, since the parameters aren't being modified. Note the syntax to create an array: (type * length)(initializers...): --------------- func.c -------------------- #include __declspec(dllexport) void func(int numVars, char **varNames, int *varTypes) { int i; printf("numVars = %d\n",numVars); for(i = 0; i < numVars; i++) printf("%d: %s\n",varTypes[i],varNames[i]); } --------------- func.py ------------------ import ctypes as c # int func (int numVars, char **varNames, int *varTypes) INT = c.c_int PINT = c.POINTER(INT) PCHAR = c.c_char_p PPCHAR = c.POINTER(PCHAR) func = c.CDLL('func').func func.restype = None func.argtypes = [INT,PPCHAR,PINT] numVars = 3 varNames = (PCHAR * numVars)('abc','def','ghi') varTypes = (INT * numVars)(1,2,3) func(numVars,varNames,varTypes) --------------- OUTPUT --------------- numVars = 3 1: abc 2: def 3: ghi -------------------------------------------- -Mark From lie.1296 at gmail.com Sat Jan 9 01:14:31 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Sat, 09 Jan 2010 17:14:31 +1100 Subject: Need help to pass self.count to other classes. In-Reply-To: <0356a7d1$0$1336$c3e8da3@news.astraweb.com> References: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> <0356a7d1$0$1336$c3e8da3@news.astraweb.com> Message-ID: <4b481ec8$1@dnews.tpgi.com.au> On 1/8/2010 3:56 PM, Steven D'Aprano wrote: > Unfortunately this won't do what you expect, because sys.ps1 and ps2 > should be either strings, or objects with a __str__ method. They aren't > called to generate the prompt. but their __str__ does get called to generate the prompt. import sys class kbInterfaceHelper(object): def __init__(self, str_func): self.str_func = str_func def __str__(self): return self.str_func() class kbInterface(object): def __init__(self): self.count = 0 def prompt1(self): self.count += 1 return "[%d]> " % self.count def prompt2(self): l = len(str(self.count))+1 return "%s " % "."*l def dhook(self, value): print "[%d out]" % self.count def ehook(self, type, value, trace): print "[%d err]\n" % value kbi = kbInterface() sys.ps1 = kbInterfaceHelper(kbi.prompt1) sys.ps2 = kbInterfaceHelper(kbi.prompt2) sys.displayhook = kbi.dhook sys.excepthook = kbi.ehook From parul.pandey85 at gmail.com Sat Jan 9 03:30:06 2010 From: parul.pandey85 at gmail.com (pp) Date: Sat, 9 Jan 2010 00:30:06 -0800 (PST) Subject: Append to an Excel file Message-ID: Hi All, How do I add a line to an existing file. This should append to the existing data in the excel file, which was saved previously. Thanks, PP From jason.scheirer at gmail.com Sat Jan 9 03:47:21 2010 From: jason.scheirer at gmail.com (Jason Scheirer) Date: Sat, 9 Jan 2010 00:47:21 -0800 (PST) Subject: Append to an Excel file References: Message-ID: <313cb2b5-4100-4d4d-bb38-dbfe3ca71e17@j5g2000yqm.googlegroups.com> On Jan 9, 12:30?am, pp wrote: > Hi All, > > How do I add a line to an existing file. This should append to the > existing data in the excel file, which was saved previously. > > Thanks, > PP http://pypi.python.org/pypi/xlwt From jason.scheirer at gmail.com Sat Jan 9 03:56:06 2010 From: jason.scheirer at gmail.com (Jason Scheirer) Date: Sat, 9 Jan 2010 00:56:06 -0800 (PST) Subject: Clarifications on compiling for Windows References: <1f0b409b-f943-4b1a-a806-87b834337367@p8g2000yqb.googlegroups.com> <6170f903-4e83-4716-9b1d-cc40bba31a65@j5g2000yqm.googlegroups.com> Message-ID: <22445936-8c2b-4d43-af3e-ef103c8e9a93@o28g2000yqh.googlegroups.com> On Jan 7, 10:51?pm, Mensanator wrote: > On Jan 8, 12:19?am, peteshinners wrote: > > > > > > > > > My presentation for Pycon is coming together, but I need to make sure > > my information about compiling Python and Python extensions for > > Windows is correct. I'm really only experienced with this on the Linux > > side of things. > > > First of all, is the Windows FAQ fairly up to date? Should people be > > referring to section 6 if they are going to build an application with > > an embedded Python interpreter?http://www.python.org/doc/faq/windows/#how-can-i-embed-python-into-a-... > > > If I understand correctly, compiled extensions for Python on Windows > > should match the compiler that was used to build the interpreter > > itself? Is there a list somewhere that shows which version of msvc was > > used to compile the recent Python binaries? > > > Thank you for feedback. I definitely want to make sure I have this > > correct before telling anybody else? > > You aren't going to try it? At a high level: YES. Avoid FILE* and you are golden on Windows. Works like a charm. MSVC 2008 works for 2.6 with the least effort (I strongly recommend having it installed as that's what the build uses). If you have VS2008 you will have no problem whatsoever with setup.py install, even with C extensions. I'd like to verify the same with earlier versions of VS but I can't. MinGW works, too, but with slightly more effort: there are some command line arguments you have to issue setup.py to know how to use/where the MinGW compiler is. From parul.pandey85 at gmail.com Sat Jan 9 03:58:29 2010 From: parul.pandey85 at gmail.com (pp) Date: Sat, 9 Jan 2010 00:58:29 -0800 (PST) Subject: Append to an Excel file References: <313cb2b5-4100-4d4d-bb38-dbfe3ca71e17@j5g2000yqm.googlegroups.com> Message-ID: <74a95a2f-90b7-48e6-9b69-c7250b979978@l30g2000yqb.googlegroups.com> On Jan 9, 1:47?am, Jason Scheirer wrote: > On Jan 9, 12:30?am, pp wrote: > > > Hi All, > > > How do I add a line to an existing file. This should append to the > > existing data in the excel file, which was saved previously. > > > Thanks, > > PP > > http://pypi.python.org/pypi/xlwt Hi Jason and all, Thanks I have seen this.. my question is there a way to append to a excel file which has been closed. Any specific modes which can be added to the sheet so that it adds a line to the data which was return in some earlier running of the program. Thanks. From kzagradskiy at gmail.com Sat Jan 9 04:07:39 2010 From: kzagradskiy at gmail.com (kzagradskiy) Date: Sat, 9 Jan 2010 01:07:39 -0800 (PST) Subject: Link to module Stack Message-ID: Link to module Stack: http://groups.google.com/group/comp.lang.python/browse_thread/thread/e6a0668bb2be9a8e/64cb44a120baeca2?lnk=gst&q=stack+module#64cb44a120baeca2 Here's the stack module for py4th. nick ------------------------------- #!/usr/Util/bin/python # # @(#)stack.py 1.1 # # stack.py # generic stack class. class Stack: def __init__(self): self.__heap = [] def push (self, word): self.__heap.append (word) def pop (self): if len(self.__heap) == 0: raise InnerInterpreterError, "stack underflow" result = self.__heap[-1] del self.__heap[-1] return result def __repr__(self): return `self.__heap` def __str__(self): return `self.__heap` def flush (self): self.__heap = [] From dickinsm at gmail.com Sat Jan 9 04:22:29 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 9 Jan 2010 01:22:29 -0800 (PST) Subject: C Module's '1.#INF' changes to 'inf' at Python References: Message-ID: On Jan 8, 3:36?pm, Robert Kern wrote: > On 2010-01-08 07:48 AM, CELEN Erman wrote: > > My problem is that I?ve noticed a strange behavior in Python while > > handling FPEs on Windows after switching compilers (msvc8 to msvc9) and > > I am trying to find out how Python handles INF values to figure out > > where the problem might be. > > [...] > > Python 2.6 changed the string representations of the special floating point > values inf and nan. Previously, the string representation was left up to the C > runtime, so they differed between platforms. Python 2.6 normalized the string > representation across all platforms. The underlying values are the same. What > version of Python are you using? In addition to this, for good or ill Python 2.6 also standardized exceptional behaviour for the math module functions: log10(0.0) and sqrt(-1) used to produce different results across implementations, but now both should consistently produce ValueError regardless of the platform. This is achieved by dealing directly with special cases in the input before delegating to the relevant libm function; this of course has an associated performance cost. There are some notes on the (intended) current behaviour at the top of the Modules/ mathmodule.c file: http://svn.python.org/view/*checkout*/python/branches/release26-maint/Modules/mathmodule.c -- Mark From steve at REMOVE-THIS-cybersource.com.au Sat Jan 9 04:25:08 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 09 Jan 2010 09:25:08 GMT Subject: Link to module Stack References: Message-ID: <0358384b$0$1336$c3e8da3@news.astraweb.com> On Sat, 09 Jan 2010 01:07:39 -0800, kzagradskiy wrote: > class Stack: > def __init__(self): > self.__heap = [] A "heap" has a technical meaning in programming. To describe the internals of a stack as "heap" will be disconcerting and confusing to anyone who knows about stacks and heaps. > def push (self, word): > self.__heap.append (word) > def pop (self): > if len(self.__heap) == 0: > raise InnerInterpreterError, "stack underflow" "InnerInterpreterError" is the most inappropriate exception name I've ever seen. It has nothing to do with the interpreter, it's a stack error. > result = self.__heap[-1] > del self.__heap[-1] That is better written as result = self.__heap.pop(). -- Steven From anthra.norell at bluewin.ch Sat Jan 9 04:52:06 2010 From: anthra.norell at bluewin.ch (Anthra Norell) Date: Sat, 09 Jan 2010 10:52:06 +0100 Subject: lightweight encryption of text file In-Reply-To: References: Message-ID: <4B4851C6.1050009@bluewin.ch> Daniel Fetchinson wrote: > I have a plain text file which I would like to protect in a very > simple minded, yet for my purposes sufficient, way. I'd like to > encrypt/convert it into a binary file in such a way that possession of > a password allows anyone to convert it back into the original text > file while not possessing the password one would only see the > following with the standard linux utility 'file': > > [fetchinson at fetch ~]$ file encrypted.data > encrypted.data: data > > and the effort required to convert the file back to the original text > file without the password would be equivalent to guessing the > password. > > I'm fully aware of the security implications of this loose > specification, but for my purposes this would be a good solution. > > What would be the simplest way to achieve this using preferably stock > python without 3rd party modules? If a not too complex 3rd part > module made it really simple that would be acceptable too. Daniel, Here's what looks like another thread veering off into package-ology, leaving a stumped OP behind. "Don't use a random generator for encryption purposes!" warns the manual, of which fact I was reminded in no uncertain terms on this forum a few years ago when I proposed the following little routine in response to a post very similar to yours. One critic challenged me to encode my credit card data and post it. Which I did. Upon which another critic conjured up the horror vision of gigahertzes hacking my pathetic little effort to pieces as I was reading his message. Of the well-meaning kind, he urged me to put an immediate stop to this foolishness. I didn't. No unplanned expenditures ensued. Or to quote ... I forget who: Fools and innovators are people who don't care much about what one is not supposed to do. So, take or leave what follows for what it is worth or not worth, I am confident it works and would serve your purpose, which, as I understand, is not to write a text book on cryptology. Regards Frederic ############################## import random def crypt_string (string, key, floor = 0, size_of_set = 255): # key is a number. The sign of that number controls which way the process # goes. If the number is positive, the result is an encryption. A negative # number orders a decryption. if key == 0: return string # No processing import random MAX_CHUNK_SIZE = 32 MIN_CHUNK_SIZE = 16 def process_sequence (sequence): s = '' for c in sequence: r = random.randint (0, size_of_set - 1) s += chr (((ord (c) - floor + r * sign) % size_of_set) + floor) return s def shuffle_sequence (sequence): random.shuffle (sequence) return sequence sign = (key > 0) * 2 - 1 random.seed (key * sign) s = '' if sign > 0: # forward i = 0 while i < len (string): random_size_of_chunk = random.randint (MIN_CHUNK_SIZE, MAX_CHUNK_SIZE) clear_chunk_shuffled = shuffle_sequence (list (string [i:i+random_size_of_chunk])) code_chunk_shuffled = process_sequence (clear_chunk_shuffled) s += code_chunk_shuffled i += len (code_chunk_shuffled) else: # backward i = 0 while i < len (string): random_size_of_chunk = random.randint (MIN_CHUNK_SIZE, MAX_CHUNK_SIZE) code_chunk_shuffled = list (string [i:i+random_size_of_chunk]) real_size_of_chunk = len (code_chunk_shuffled) unshuffling_template = shuffle_sequence (range (real_size_of_chunk)) # 1. same ... clear_chunk_shuffled = process_sequence (code_chunk_shuffled) # 2. ... order clear_chunk = real_size_of_chunk * [None] for ii in range (real_size_of_chunk): clear_chunk [unshuffling_template[ii]] = clear_chunk_shuffled [ii] s += ''.join (clear_chunk) i += real_size_of_chunk return s def _crypt_file (in_file, out_file, key): BUFFER_SIZE = 1024 while 1: s = in_file.read (BUFFER_SIZE) if s == '': break out_file.write (crypt_string (s, key)) def crypt_file (in_file_name, out_file_name, key): in_file = open (in_file_name, 'rb') out_file = open (out_file_name, 'wb') _crypt_file (in_file, out_file, key) out_file.close () From parul.pandey85 at gmail.com Sat Jan 9 05:24:16 2010 From: parul.pandey85 at gmail.com (pp) Date: Sat, 9 Jan 2010 02:24:16 -0800 (PST) Subject: Porblem with xlutils/xlrd/xlwt Message-ID: <8e9245ec-05cb-48ed-ae32-f9fb0c46509f@j5g2000yqm.googlegroups.com> Whenever i run the code below I get the following error: AttributeError: 'Book' object has no attribute 'on_demand' WARNING: Failure executing file: Why is it so?? from xlrd import open_workbook from xlwt import easyxf from xlutils.copy import copy rb = open_workbook('source.xls',formatting_info=True) rs = rb.sheet_by_index(0) wb = copy(rb) ws = wb.get_sheet(0) plain = easyxf('') for i,cell in enumerate(rs.col(2)): if not i: continue ws.write(i,2,cell.value,plain) for i,cell in enumerate(rs.col(4)): if not i: continue ws.write(i,4,cell.value-1000) wb.save('output.xls') From martin at v.loewis.de Sat Jan 9 05:24:40 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Sat, 09 Jan 2010 11:24:40 +0100 Subject: Clarifications on compiling for Windows In-Reply-To: <1f0b409b-f943-4b1a-a806-87b834337367@p8g2000yqb.googlegroups.com> References: <1f0b409b-f943-4b1a-a806-87b834337367@p8g2000yqb.googlegroups.com> Message-ID: <4B485968.7020605@v.loewis.de> > First of all, is the Windows FAQ fairly up to date? Fairly, yes. > Should people be > referring to section 6 if they are going to build an application with > an embedded Python interpreter? I think that's very selective in its view of problems - why would I be using SWIG, for example? (yet there are three issues dedicated to SWIG in this section) pythonNN.dll is not in \windows\system, but in system32 or winsxs. pythonNN.lib is *not* a static library, but (as the text then notes) an import library. So even if you link with pythonNN.lib, you *still* need pythonNN.dll at run-time (what is discussed as a drawback of dynamic linking). Of course, it might be possible to build a static library out of Python (which then still might be called pythonNN.lib). IMO, it should be possible to link Python into the executable (at the expense of not supporting dynamic loading of extension modules anymore). > If I understand correctly, compiled extensions for Python on Windows > should match the compiler that was used to build the interpreter > itself? Is there a list somewhere that shows which version of msvc was > used to compile the recent Python binaries? See PCbuild/readme.txt of the respective Python release. 2.3: VC 6 2.4, 2.5: VC 7.1 / VS .NET 2003 2.6, 3.1: VC 9 / VS 2008 Regards, Martin From fetchinson at googlemail.com Sat Jan 9 05:26:38 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Sat, 9 Jan 2010 11:26:38 +0100 Subject: lightweight encryption of text file In-Reply-To: References: Message-ID: >>>>> I have a plain text file which I would like to protect in a very >>>>> simple minded, yet for my purposes sufficient, way. I'd like to >>>>> encrypt/convert it into a binary file in such a way that possession of >>>>> a password allows anyone to convert it back into the original text >>>>> file while not possessing the password one would only see the >>>>> following with the standard linux utility 'file': >>>>> >>>>> [fetchinson at fetch ~]$ file encrypted.data >>>>> encrypted.data: data >>>>> >>>>> and the effort required to convert the file back to the original text >>>>> file without the password would be equivalent to guessing the >>>>> password. >>>>> >>>>> I'm fully aware of the security implications of this loose >>>>> specification, but for my purposes this would be a good solution. >>>>> >>>>> What would be the simplest way to achieve this using preferably stock >>>>> python without 3rd party modules? If a not too complex 3rd party >>>>> module made it really simple that would be acceptable too. >>>> >>>> Paul Rubin's p3.py algorithm is probably the most straightforward way to >>>> meet >>>> these requirements. It's not a standard crypto algorithm by any means, >>>> but >>>> Paul >>>> knows his stuff and has devised it with these deployment restrictions in >>>> mind. >>>> >>>> http://www.nightsong.com/phr/crypto/p3.py >>> >>> Thanks a lot, currently I'm having trouble using this code on python >>> 2.6 but probably some small tweaking will fix it. >> >> Actually, it also doesn't work with python 2.5 and currently I don't >> have access to anything older. array.array raises a >> >> ValueError: string length not a multiple of item size >> >> Does anyone recall a change to array.array? >> >> The full traceback is >> >> Traceback (most recent call last): >> File "p3.py", line 163, in >> _test() >> File "p3.py", line 143, in _test >> c1 = e(plain,key) >> File "p3.py", line 69, in p3_encrypt >> xkey = _expand_key(k_enc, n+4) >> File "p3.py", line 41, in _expand_key >> return array ('L', j) >> ValueError: string length not a multiple of item size > > Are you on a 64-bit platform? Unfortunately, array's integer typecodes are > platform-specific, but p3.py requires a 32-bit integer and was written on a > 32-bit platform. It's reasonably straightforward to fix. Put this bit of > (untested) code at the top of the file and replace occurrences of 'L' with > uint32: > > # Find the typecode of a 32-bit unsigned integer. > for typecode in 'IL': > if len(array(typecode, [0]).tostring()) == 4: > uint32 = typecode > break > else: > raise RuntimeError("Neither 'I' nor 'L' are unsigned 32-bit integers.") Thanks! That was exactly the problem, I'm on a 64 bit machine and your fix seems to work indeed. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From joncle at googlemail.com Sat Jan 9 05:26:48 2010 From: joncle at googlemail.com (Jon Clements) Date: Sat, 9 Jan 2010 02:26:48 -0800 (PST) Subject: table from csv file References: <59653a09-c3f7-464f-a36f-475f0d759e46@e27g2000yqd.googlegroups.com> <77b18520-c2c4-4e21-9147-1164d81b9f25@l30g2000yqb.googlegroups.com> Message-ID: <8e1d52da-f6e8-4600-8afa-3cc9be7bee80@a21g2000yqc.googlegroups.com> On Jan 8, 8:31?pm, J wrote: > On Fri, Jan 8, 2010 at 13:55, Jon Clements wrote: > > On Jan 8, 5:59?pm, marlowe wrote: > >> I am trying to create a table in python from a csv file where I input > >> which columns I would like to see, and the table only shows those > >> columns. I have attached an example of the csv file i am using, and > >> some of the code I have written. I am having trouble converting > >> variables between lists, dictionaries and tuples. Is there a name for > >> what I am attempting to do? any help to get me on the right track with > >> this is appreciated. > > >> test.csv > > I had to edit that and comma delimit it, because cut and paste gave me > random numbers/types of whitespace... > > [code snipped] > > > > > This might be a useful starting point (I'm guessing this is what > > you're after...) > > > Let's assume your 'CSV' file is tab separated as it's certainly not > > comma separated :) > > > import csv > > csvin = csv.reader(open('test.csv'), delimiter='\t') > > header = dict( (val.strip(),idx) for idx, val in enumerate(next > > (csvin)) ) > > > We can use header as a column name->column index lookup eg header > > ['Open'] == 1 > > > from operator import itemgetter > > wanted = ['Open', 'Close'] # Although you'll want to use raw_input and > > split on ',' > > getcols = itemgetter(*[header[col] for col in wanted]) > > > getcols is a helper function that'll return a tuple of the columns in > > the requested order... > > > for row in csvin: > > ? ?print getcols(row) > > > Loop over the rest of the file and output the required columns. > > As someone who knows just enough to be dangerous... what about this: > > import csv > > reader = open('C:/test.txt','rb') > data = csv.DictReader(reader,restval='000',restkey='Misc') [snip] DictReader works, but what use to bug me was the fact you couldn't then output the cols in the 'correct' order afterwards, so you had to store the header row anyway to re-order the rows... (although admittedly this doesn't affect the OP's question). However, I see that 2.6+ offers .fieldnames on DictReader objects. Cheers, Jon. From joncle at googlemail.com Sat Jan 9 05:42:43 2010 From: joncle at googlemail.com (Jon Clements) Date: Sat, 9 Jan 2010 02:42:43 -0800 (PST) Subject: Porblem with xlutils/xlrd/xlwt References: <8e9245ec-05cb-48ed-ae32-f9fb0c46509f@j5g2000yqm.googlegroups.com> Message-ID: <972fdcfc-73a6-4e87-9752-3bc10da0dcad@s3g2000yqs.googlegroups.com> On Jan 9, 10:24?am, pp wrote: > Whenever i run the code below I get the following error: > > AttributeError: 'Book' object has no attribute 'on_demand' > WARNING: Failure executing file: > > Why is it so?? > > from xlrd import open_workbook > from xlwt import easyxf > from xlutils.copy import copy > rb = ?open_workbook('source.xls',formatting_info=True) > rs = ?rb.sheet_by_index(0) > wb = ?copy(rb) > ws = ?wb.get_sheet(0) > plain = easyxf('') > for i,cell in enumerate(rs.col(2)): > ? ? ?if not i: > ? ? ? ? ?continue > ? ? ?ws.write(i,2,cell.value,plain) > for i,cell in enumerate(rs.col(4)): > ? ? ?if not i: > ? ? ? ? ?continue > ? ? ?ws.write(i,4,cell.value-1000) > wb.save('output.xls') I suspect your version of xlrd is not up to date (although I thought on_demand was ages ago!). Make sure all the tools are the latest versions from http://www.python-excel.org There's also a dedicated Google Group for the xl* products listed on that page. hth Jon. From parul.pandey85 at gmail.com Sat Jan 9 05:44:28 2010 From: parul.pandey85 at gmail.com (pp) Date: Sat, 9 Jan 2010 02:44:28 -0800 (PST) Subject: Porblem with xlutils/xlrd/xlwt References: <8e9245ec-05cb-48ed-ae32-f9fb0c46509f@j5g2000yqm.googlegroups.com> <972fdcfc-73a6-4e87-9752-3bc10da0dcad@s3g2000yqs.googlegroups.com> Message-ID: <43a1e323-8ccd-48b7-a6a4-e540aca2b1cd@35g2000yqa.googlegroups.com> On Jan 9, 3:42?am, Jon Clements wrote: > On Jan 9, 10:24?am, pp wrote: > > > > > Whenever i run the code below I get the following error: > > > AttributeError: 'Book' object has no attribute 'on_demand' > > WARNING: Failure executing file: > > > Why is it so?? > > > from xlrd import open_workbook > > from xlwt import easyxf > > from xlutils.copy import copy > > rb = ?open_workbook('source.xls',formatting_info=True) > > rs = ?rb.sheet_by_index(0) > > wb = ?copy(rb) > > ws = ?wb.get_sheet(0) > > plain = easyxf('') > > for i,cell in enumerate(rs.col(2)): > > ? ? ?if not i: > > ? ? ? ? ?continue > > ? ? ?ws.write(i,2,cell.value,plain) > > for i,cell in enumerate(rs.col(4)): > > ? ? ?if not i: > > ? ? ? ? ?continue > > ? ? ?ws.write(i,4,cell.value-1000) > > wb.save('output.xls') > > I suspect your version of xlrd is not up to date (although I thought > on_demand was ages ago!). > Make sure all the tools are the latest versions fromhttp://www.python-excel.org > > There's also a dedicated Google Group for the xl* products listed on > that page. > > hth > Jon. yeah all my versions are latest from http://www.python-excel.org . just checked!! what could be the problem? From fetchinson at googlemail.com Sat Jan 9 05:45:14 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Sat, 9 Jan 2010 11:45:14 +0100 Subject: lightweight encryption of text file In-Reply-To: <0357cddd$0$1336$c3e8da3@news.astraweb.com> References: <0357cddd$0$1336$c3e8da3@news.astraweb.com> Message-ID: On 1/9/10, Steven D'Aprano wrote: > On Fri, 08 Jan 2010 20:14:51 +0100, Daniel Fetchinson wrote: > >> I have a plain text file which I would like to protect in a very simple >> minded, yet for my purposes sufficient, way. I'd like to encrypt/convert >> it into a binary file in such a way that possession of a password allows >> anyone to convert it back into the original text file while not >> possessing the password one would only see the following with the >> standard linux utility 'file': >> >> [fetchinson at fetch ~]$ file encrypted.data >> encrypted.data: data > > If that is your sole requirement, No, that was not my sole requirement, I also wrote: """ and the effort required to convert the file back to the original text file without the password would be equivalent to guessing the password. """ > then the addition of a single non-text > byte (say, a null) anywhere in the file would be sufficient to have file > identify it as data. Yes, but this would not satisfy the other requirement quoted above. One could read the file without an effort that is equivalent to guessing a random password. > You say "encrypt/convert" -- does this mean that you > don't care if people can read the text in a hex editor, so long as file > identifies it as data? I do care. See the quote above :) > Would something like a binary Vigenere Cipher be sufficient? > > > # Untested > def encrypt(plaintext, password): > cipher = [] > for i, c in enumerate(plaintext): > shift = password[i % len(password)] > shift = ord(shift) > cipher.append((ord(c) + shift) % 256) > return ''.join([chr(n) for n in cipher]) > > def decrypt(ciphertext, password): > plain = [] > for i, c in enumerate(ciphertext): > shift = password[i % len(password)] > shift = ord(shift) > plain.append((256 + ord(c) - shift) % 256) > return ''.join([chr(n) for n in plain]) Thanks, this looks simple enough and probably sufficient for my purposes! I'll see if I'll use this or Paul Rubin's p3.py. > Is it acceptable if there is a chance (small, possibly vanishingly small) > that file will identify it as text? As far as I know, even the > heavyweight "serious" encryption algorithms don't *guarantee* that the > cipher text will include non-text bytes. Hmmm, that's a good point, but actually it doesn't matter if 'file' identifies it as text with a very small probability. >> and the effort required to convert the file back to the original text >> file without the password would be equivalent to guessing the password. > > If you seriously mean that, then "lightweight encryption" won't do the > job, because it is vulnerable to frequency analysis, which is easier than > guessing the password. You need proper, heavy-weight encryption. Well, probably you are right and I should have been more clear. What typically people call obfuscation is sufficient for me, as long as the obfuscation involves a password, something that your solution seems to do. >> I'm fully aware of the security implications of this loose >> specification, but for my purposes this would be a good solution. > > Can you explain what your objection to real encryption is? Not much really, I simply don't want to overkill, that's all. First and foremost I wouldn't want to introduce a dependency on a 3rd party module and also wouldn't want to read documentation of a complex API when all I need are two functions: encrypt( text, password) and decrypt( text, password ). I really like your solution because that's all it does. > The problem is that, as I see it, you've assumed a solution rather than > state what your requirements are. I'm *guessing* that you are more > concerned of having to learn to use a complex API, Well, that's sort of true about learning a complex API :) But it's also true that I'm not storing anything really valuable in the file but still wouldn't want to leave it lying around in plain text. In case I lose the laptop with the file I seriously doubt anybody who finds it will go through each and every file and try to find what's in it, even though they look like data files and there is no hint what so ever that any one of them contains encrypted info. If they see a text file, well, that can give them ideas, so let's encrypt a little bit. So basically that's the story, granted, it's not a full specification or anything like that, it's a description of a vague situation but that's really all I have :) Cheers, Daniel From fetchinson at googlemail.com Sat Jan 9 05:49:38 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Sat, 9 Jan 2010 11:49:38 +0100 Subject: lightweight encryption of text file In-Reply-To: <4B4851C6.1050009@bluewin.ch> References: <4B4851C6.1050009@bluewin.ch> Message-ID: > > I have a plain text file which I would like to protect in a very > > simple minded, yet for my purposes sufficient, way. I'd like to > > encrypt/convert it into a binary file in such a way that possession of > > a password allows anyone to convert it back into the original text > > file while not possessing the password one would only see the > > following with the standard linux utility 'file': > > > > [fetchinson at fetch ~]$ file encrypted.data > > encrypted.data: data > > > > and the effort required to convert the file back to the original text > > file without the password would be equivalent to guessing the > > password. > > > > I'm fully aware of the security implications of this loose > > specification, but for my purposes this would be a good solution. > > > > What would be the simplest way to achieve this using preferably stock > > python without 3rd party modules? If a not too complex 3rd part > > module made it really simple that would be acceptable too. > > > > > Daniel, > > Here's what looks like another thread veering off into package-ology, > leaving a stumped OP behind. > > "Don't use a random generator for encryption purposes!" warns the > manual, of which fact I was reminded in no uncertain terms on this forum > a few years ago when I proposed the following little routine in response > to a post very similar to yours. One critic challenged me to encode my > credit card data and post it. Which I did. Upon which another critic > conjured up the horror vision of gigahertzes hacking my pathetic little > effort to pieces as I was reading his message. Of the well-meaning kind, > he urged me to put an immediate stop to this foolishness. I didn't. > > No unplanned expenditures ensued. > > Or to quote ... I forget who: Fools and innovators are people who don't > care much about what one is not supposed to do. > > So, take or leave what follows for what it is worth or not worth, I am > confident it works and would serve your purpose, which, as I understand, > is not to write a text book on cryptology. > > Regards > > Frederic > > > ############################## > > > import random > > > def crypt_string (string, key, floor = 0, size_of_set = 255): > > # key is a number. The sign of that number controls which way the > process > # goes. If the number is positive, the result is an encryption. A > negative > # number orders a decryption. > > if key == 0: return string # No processing > > import random > > MAX_CHUNK_SIZE = 32 > MIN_CHUNK_SIZE = 16 > > def process_sequence (sequence): > s = '' > for c in sequence: > r = random.randint (0, size_of_set - 1) > s += chr (((ord (c) - floor + r * sign) % size_of_set) + floor) > return s > > def shuffle_sequence (sequence): > random.shuffle (sequence) > return sequence > > sign = (key > 0) * 2 - 1 > random.seed (key * sign) > > s = '' > > if sign > 0: # forward > > i = 0 > while i < len (string): > random_size_of_chunk = random.randint (MIN_CHUNK_SIZE, > MAX_CHUNK_SIZE) > clear_chunk_shuffled = shuffle_sequence (list (string > [i:i+random_size_of_chunk])) > code_chunk_shuffled = process_sequence (clear_chunk_shuffled) > s += code_chunk_shuffled > i += len (code_chunk_shuffled) > > else: # backward > > i = 0 > while i < len (string): > random_size_of_chunk = random.randint (MIN_CHUNK_SIZE, > MAX_CHUNK_SIZE) > code_chunk_shuffled = list (string [i:i+random_size_of_chunk]) > real_size_of_chunk = len (code_chunk_shuffled) > unshuffling_template = shuffle_sequence (range > (real_size_of_chunk)) # 1. same ... > clear_chunk_shuffled = process_sequence > (code_chunk_shuffled) # 2. ... order > clear_chunk = real_size_of_chunk * [None] > for ii in range (real_size_of_chunk): > clear_chunk [unshuffling_template[ii]] = > clear_chunk_shuffled [ii] > s += ''.join (clear_chunk) > i += real_size_of_chunk > > return s > > > > def _crypt_file (in_file, out_file, key): > > BUFFER_SIZE = 1024 > while 1: > s = in_file.read (BUFFER_SIZE) > if s == '': break > out_file.write (crypt_string (s, key)) > > > def crypt_file (in_file_name, out_file_name, key): > > in_file = open (in_file_name, 'rb') > out_file = open (out_file_name, 'wb') > _crypt_file (in_file, out_file, key) > out_file.close () Thanks a lot! Your description is a good fit for my purposes, indeed I don't plan on writing a text book on encryption :) Also, I don't plan on encrypting credit card numbers either, all I need is that a file doesn't look obviously full of ascii characters but something that is generic data. And since it will not be an entire system, with lots of files of this type, only we are talking about a single file, there is no incentive to decipher my algorithm. I'll take a look at your code, thanks a lot, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From joncle at googlemail.com Sat Jan 9 05:52:56 2010 From: joncle at googlemail.com (Jon Clements) Date: Sat, 9 Jan 2010 02:52:56 -0800 (PST) Subject: Porblem with xlutils/xlrd/xlwt References: <8e9245ec-05cb-48ed-ae32-f9fb0c46509f@j5g2000yqm.googlegroups.com> <972fdcfc-73a6-4e87-9752-3bc10da0dcad@s3g2000yqs.googlegroups.com> <43a1e323-8ccd-48b7-a6a4-e540aca2b1cd@35g2000yqa.googlegroups.com> Message-ID: <5ac38519-9ad4-4a89-9b3f-ccd56fb18721@m3g2000yqf.googlegroups.com> On Jan 9, 10:44?am, pp wrote: > On Jan 9, 3:42?am, Jon Clements wrote: > > > > > On Jan 9, 10:24?am, pp wrote: > > > > Whenever i run the code below I get the following error: > > > > AttributeError: 'Book' object has no attribute 'on_demand' > > > WARNING: Failure executing file: > > > > Why is it so?? > > > > from xlrd import open_workbook > > > from xlwt import easyxf > > > from xlutils.copy import copy > > > rb = ?open_workbook('source.xls',formatting_info=True) > > > rs = ?rb.sheet_by_index(0) > > > wb = ?copy(rb) > > > ws = ?wb.get_sheet(0) > > > plain = easyxf('') > > > for i,cell in enumerate(rs.col(2)): > > > ? ? ?if not i: > > > ? ? ? ? ?continue > > > ? ? ?ws.write(i,2,cell.value,plain) > > > for i,cell in enumerate(rs.col(4)): > > > ? ? ?if not i: > > > ? ? ? ? ?continue > > > ? ? ?ws.write(i,4,cell.value-1000) > > > wb.save('output.xls') > > > I suspect your version of xlrd is not up to date (although I thought > > on_demand was ages ago!). > > Make sure all the tools are the latest versions fromhttp://www.python-excel.org > > > There's also a dedicated Google Group for the xl* products listed on > > that page. > > > hth > > Jon. > > yeah all my versions are latest fromhttp://www.python-excel.org. > just checked!! > what could be the problem? Does rb = xlrd.open_workbook('somesheet.xls', on_demand=True) work by itself? From parul.pandey85 at gmail.com Sat Jan 9 05:56:03 2010 From: parul.pandey85 at gmail.com (pp) Date: Sat, 9 Jan 2010 02:56:03 -0800 (PST) Subject: Porblem with xlutils/xlrd/xlwt References: <8e9245ec-05cb-48ed-ae32-f9fb0c46509f@j5g2000yqm.googlegroups.com> <972fdcfc-73a6-4e87-9752-3bc10da0dcad@s3g2000yqs.googlegroups.com> <43a1e323-8ccd-48b7-a6a4-e540aca2b1cd@35g2000yqa.googlegroups.com> <5ac38519-9ad4-4a89-9b3f-ccd56fb18721@m3g2000yqf.googlegroups.com> Message-ID: On Jan 9, 3:52?am, Jon Clements wrote: > On Jan 9, 10:44?am, pp wrote: > > > > > On Jan 9, 3:42?am, Jon Clements wrote: > > > > On Jan 9, 10:24?am, pp wrote: > > > > > Whenever i run the code below I get the following error: > > > > > AttributeError: 'Book' object has no attribute 'on_demand' > > > > WARNING: Failure executing file: > > > > > Why is it so?? > > > > > from xlrd import open_workbook > > > > from xlwt import easyxf > > > > from xlutils.copy import copy > > > > rb = ?open_workbook('source.xls',formatting_info=True) > > > > rs = ?rb.sheet_by_index(0) > > > > wb = ?copy(rb) > > > > ws = ?wb.get_sheet(0) > > > > plain = easyxf('') > > > > for i,cell in enumerate(rs.col(2)): > > > > ? ? ?if not i: > > > > ? ? ? ? ?continue > > > > ? ? ?ws.write(i,2,cell.value,plain) > > > > for i,cell in enumerate(rs.col(4)): > > > > ? ? ?if not i: > > > > ? ? ? ? ?continue > > > > ? ? ?ws.write(i,4,cell.value-1000) > > > > wb.save('output.xls') > > > > I suspect your version of xlrd is not up to date (although I thought > > > on_demand was ages ago!). > > > Make sure all the tools are the latest versions fromhttp://www.python-excel.org > > > > There's also a dedicated Google Group for the xl* products listed on > > > that page. > > > > hth > > > Jon. > > > yeah all my versions are latest fromhttp://www.python-excel.org. > > just checked!! > > what could be the problem? > > Does rb = xlrd.open_workbook('somesheet.xls', on_demand=True) work by > itself? Yes it does. The problem is with line: wb = copy(rb) here I am getting the error: AttributeError: 'Book' object has no attribute 'on_demand' Thanks .. From davea at ieee.org Sat Jan 9 05:56:36 2010 From: davea at ieee.org (Dave Angel) Date: Sat, 09 Jan 2010 05:56:36 -0500 Subject: Link to module Stack In-Reply-To: <0358384b$0$1336$c3e8da3@news.astraweb.com> References: <0358384b$0$1336$c3e8da3@news.astraweb.com> Message-ID: <4B4860E4.2090705@ieee.org> Steven D'Aprano wrote: > On Sat, 09 Jan 2010 01:07:39 -0800, kzagradskiy wrote: > > >> class Stack: >> def __init__(self): >> self.__heap = [] >> > > A "heap" has a technical meaning in programming. To describe the > internals of a stack as "heap" will be disconcerting and confusing to > anyone who knows about stacks and heaps. > > > >> def push (self, word): >> self.__heap.append (word) >> def pop (self): >> if len(self.__heap) == 0: >> raise InnerInterpreterError, "stack underflow" >> > > "InnerInterpreterError" is the most inappropriate exception name I've > ever seen. It has nothing to do with the interpreter, it's a stack error. > > It has everything to do with the (Forth) interpreter. Exceptions can readily be named according to their application -- it's not always about Python. Anyway, Forth has an inner-interpreter and an outer-interpreter, and the name will make sense to a Forth programmer. >> result = self.__heap[-1] >> del self.__heap[-1] >> > > That is better written as result = self.__heap.pop(). > > > or even better, without the extra local var: def pop (self): if len(self.__heap) == 0: raise InnerInterpreterError, "stack underflow" return self.__heap.pop(1) P.S. - I'm puzzled why the OP even put this message here. There's no question posted with it. DaveA From davea at ieee.org Sat Jan 9 06:05:05 2010 From: davea at ieee.org (Dave Angel) Date: Sat, 09 Jan 2010 06:05:05 -0500 Subject: lightweight encryption of text file In-Reply-To: <4B4851C6.1050009@bluewin.ch> References: <4B4851C6.1050009@bluewin.ch> Message-ID: <4B4862E1.6080406@ieee.org> Anthra Norell wrote: >
Daniel > Fetchinson wrote: > > I have a plain text file which I would like to protect in a very > > simple minded, yet for my purposes sufficient, way. I'd like to > > encrypt/convert it into a binary file in such a way that possession of > > a password allows anyone to convert it back into the original text > > file while not possessing the password one would only see the > > following with the standard linux utility 'file': > > > > [fetchinson at fetch ~]$ file encrypted.data > > encrypted.data: data > > > > and the effort required to convert the file back to the original text > > file without the password would be equivalent to guessing the > > password. > > > > I'm fully aware of the security implications of this loose > > specification, but for my purposes this would be a good solution. > > > > What would be the simplest way to achieve this using preferably stock > > python without 3rd party modules? If a not too complex 3rd part > > module made it really simple that would be acceptable too. > > > > > Daniel, > > Here's what looks like another thread veering off into package-ology, > leaving a stumped OP behind. > > "Don't use a random generator for encryption purposes!" warns the > manual, of which fact I was reminded in no uncertain terms on this > forum a few years ago when I proposed the following little routine in > response to a post very similar to yours. One critic challenged me to > encode my credit card data and post it. Which I did. Upon which > another critic conjured up the horror vision of gigahertzes hacking my > pathetic little effort to pieces as I was reading his message. Of the > well-meaning kind, he urged me to put an immediate stop to this > foolishness. I didn't. > > No unplanned expenditures ensued. > > Or to quote ... I forget who: Fools and innovators are people who > don't care much about what one is not supposed to do. > > So, take or leave what follows for what it is worth or not worth, I am > confident it works and would serve your purpose, which, as I > understand, is not to write a text book on cryptology. > > Regards > > Frederic > > The problem I'd have with this approach (not studied in detail), is that there's no reason that next year's Python must use the same random number generator, or the same shuffle algorithm. So in order to assure that "encrypted" archives will be recoverable, one should store with them the CPython implementation, in source form, just in case that's needed to reconstruct things. DaveA From rdmoores at gmail.com Sat Jan 9 06:31:49 2010 From: rdmoores at gmail.com (Richard D. Moores) Date: Sat, 9 Jan 2010 03:31:49 -0800 Subject: How to get many places of pi from Machin's Equation? Message-ID: Machin's Equation is 4 arctan (1/5) - arctan(1/239) = pi/4 Using Python 3.1 and the math module: >>> from math import atan, pi >>> pi 3.141592653589793 >>> (4*atan(.2) - atan(1/239))*4 3.1415926535897936 >>> (4*atan(.2) - atan(1/239))*4 == pi False >>> abs((4*atan(.2) - atan(1/239))*4) - pi < .000000000000000001 False >>> abs((4*atan(.2) - atan(1/239))*4) - pi < .0000000000000001 False >>> abs((4*atan(.2) - atan(1/239))*4) - pi < .000000000000001 True >>> Is there a way in Python 3.1 to calculate pi to greater accuracy using Machin's Equation? Even to an arbitrary number of places? Thanks, Dick Moores From duncan.booth at invalid.invalid Sat Jan 9 06:40:42 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 9 Jan 2010 11:40:42 GMT Subject: Link to module Stack References: <0358384b$0$1336$c3e8da3@news.astraweb.com> Message-ID: Dave Angel wrote: > or even better, without the extra local var: > > def pop (self): > if len(self.__heap) == 0: > raise InnerInterpreterError, "stack underflow" > return self.__heap.pop(1) pop(1)? Anyway if would be simpler and almost certainly faster to not bother checking before the pop: def pop(self): try: return self.__heap.pop() except IndexError: raise InnerInterpreterError, "stack underflow" and if performance mattered the OP might even consider pre-binding the pop method in __init__: self.__pop = self.__heap.pop but that's probably premature optimisation. > P.S. - I'm puzzled why the OP even put this message here. There's no > question posted with it. Me too. It's a repost of something from 2004. Bizarre. From as at sci.fi Sat Jan 9 06:55:30 2010 From: as at sci.fi (Anssi Saari) Date: Sat, 09 Jan 2010 13:55:30 +0200 Subject: Accessing python from a network share in windows 7 References: Message-ID: aj writes: > I access python from a network share. This works fine on XP but on > windows 7 it throws the following error: > > Python 2.6.1 (r261:67517, Dec 4 2008, 16:51:00) [MSC v.1500 32 bit > (Intel)] on > win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> import random > Traceback (most recent call last): > File "", line 1, in > File "t:\win32\python-2.6.1\lib\random.py", line 871, in > _inst = Random() > File "t:\win32\python-2.6.1\lib\random.py", line 96, in __init__ > self.seed(x) > File "t:\win32\python-2.6.1\lib\random.py", line 110, in seed > a = long(_hexlify(_urandom(16)), 16) > WindowsError: [Error 127] The specified procedure could not be found > > Is there some security policy that I need to enable/disable to use > python from a network on windows 7? Well, there was just a complaint about this sort of thing on a local newsgroup here. Specifically, someone was trying to execute a Windows program from a share and every time Windows 7 pops up a warning window saying that the program is maybe from the evil interwebby and are you really sure you actually want to run it. So probably that's the command line version of same. Solution is apparently specifying your server to be in the local intranet, in IE's security settings. Apparently there is a non-working autodetection for what is a local intranet, so specifying the server IP address by hand in the advanced settigns is the working solution. From steve at holdenweb.com Sat Jan 9 07:50:48 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 09 Jan 2010 07:50:48 -0500 Subject: Standardized interpreter speed evaluation tool In-Reply-To: <50697b2c1001081602m7614b94esc4cd9241fafbdf49@mail.gmail.com> References: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> <50697b2c1001081602m7614b94esc4cd9241fafbdf49@mail.gmail.com> Message-ID: <4B487BA8.8070108@holdenweb.com> Chris Rebert wrote: > On Fri, Jan 8, 2010 at 2:25 AM, alexru wrote: >> Is there any standardized interpreter speed evaluation tool? Say I >> made few changes in interpreter code and want to know if those changes >> made python any better, which test should I use? > > Although apparently undocumented, test.pystone is some sort of > interpreter benchmark. > It's undocumented because it's not considered a representative benchmark.Sure, you can use it to get *some* idea of relative performance, but a single program is a very poor tool for such a complex topic a comparing implementations of a dynamic language. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Sat Jan 9 07:50:48 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 09 Jan 2010 07:50:48 -0500 Subject: Standardized interpreter speed evaluation tool In-Reply-To: <50697b2c1001081602m7614b94esc4cd9241fafbdf49@mail.gmail.com> References: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> <50697b2c1001081602m7614b94esc4cd9241fafbdf49@mail.gmail.com> Message-ID: <4B487BA8.8070108@holdenweb.com> Chris Rebert wrote: > On Fri, Jan 8, 2010 at 2:25 AM, alexru wrote: >> Is there any standardized interpreter speed evaluation tool? Say I >> made few changes in interpreter code and want to know if those changes >> made python any better, which test should I use? > > Although apparently undocumented, test.pystone is some sort of > interpreter benchmark. > It's undocumented because it's not considered a representative benchmark.Sure, you can use it to get *some* idea of relative performance, but a single program is a very poor tool for such a complex topic a comparing implementations of a dynamic language. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From arnodel at googlemail.com Sat Jan 9 07:54:11 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sat, 09 Jan 2010 12:54:11 +0000 Subject: lightweight encryption of text file References: Message-ID: Daniel Fetchinson writes: > I have a plain text file which I would like to protect in a very > simple minded, yet for my purposes sufficient, way. I'd like to > encrypt/convert it into a binary file in such a way that possession of > a password allows anyone to convert it back into the original text > file while not possessing the password one would only see the > following with the standard linux utility 'file': > > [fetchinson at fetch ~]$ file encrypted.data > encrypted.data: data [...] This is probably not what you want, but it is very simple and doesn't import any module:) I am not qualified to say how easy it is to discover the message without the password. def str2int(txt): return reduce(lambda n, c: n*255 + ord(c), txt, 0) def int2str(n): chars = [] while n: n, o = divmod(n, 255) chars.append(chr(o)) return ''.join(reversed(chars)) def encrypt(txt, pwd): return int2str(str2int(txt)*str2int(pwd)) def decrypt(txt, pwd): return int2str(str2int(txt)/str2int(pwd)) def test(txt, pwd): encrypted_txt = encrypt(txt, pwd) decrypted_txt = decrypt(encrypted_txt, pwd) print "text:%r" % txt print "encrypted:%r" % encrypted_txt print "decrypted:%r" % decrypted_txt >>> test("This encryption scheme is definitely unbreakable.", "secret") text:'This encryption scheme is definitely unbreakable.' encrypted:'&2\xa5\xd4\x17i+E\x01k\xfa\x94\xf80\xa8\x8f\xea.w\x128\xf1\xd9\x0f9\xf2t\xc9\r`\x90%\xd6\xf3~\x1f\x00%u&\x8a\xe4\xe0\xa7\xb8\xb0ec)S>\xcb\xf2>\xec' decrypted:'This encryption scheme is definitely unbreakable.' -- Arnaud From steve at holdenweb.com Sat Jan 9 07:59:01 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 09 Jan 2010 07:59:01 -0500 Subject: Append to an Excel file In-Reply-To: <74a95a2f-90b7-48e6-9b69-c7250b979978@l30g2000yqb.googlegroups.com> References: <313cb2b5-4100-4d4d-bb38-dbfe3ca71e17@j5g2000yqm.googlegroups.com> <74a95a2f-90b7-48e6-9b69-c7250b979978@l30g2000yqb.googlegroups.com> Message-ID: pp wrote: > On Jan 9, 1:47 am, Jason Scheirer wrote: >> On Jan 9, 12:30 am, pp wrote: >> >>> Hi All, >>> How do I add a line to an existing file. This should append to the >>> existing data in the excel file, which was saved previously. >>> Thanks, >>> PP >> http://pypi.python.org/pypi/xlwt > > Hi Jason and all, > > Thanks > > I have seen this.. my question is there a way to append to a excel > file which has been closed. Any specific modes which can be added to > the sheet so that it adds a line to the data which was return in some > earlier running of the program. > If you are talking about an XLS file and not a CSV then it's a highly structured object, and you can't just "stick bits on the end" with any expectation that Excel will know what to do with the new data. The most likely outcomes would be Excel either complaining the file format is invalid or ignoring the extra data. If it's a CSV file, then f = open(file, "a") should do it. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From peter.bienstman at gmail.com Sat Jan 9 08:19:00 2010 From: peter.bienstman at gmail.com (pbienst) Date: Sat, 9 Jan 2010 05:19:00 -0800 (PST) Subject: creating tar file and streaming it over HTTP? References: <5222d923-7aef-47cf-a0d6-f37f09a66593@35g2000yqa.googlegroups.com> Message-ID: <0995c098-8976-46ae-98f0-977fe5e09a39@22g2000yqr.googlegroups.com> OK, thanks to the feedback from everyone I got the PUT from a client to the WSGI server working. I'm now trying to go the other way around: use a tar stream in one of the functions in the WSGI server in order to send files to the client. Problem is that the WSGI specs expects an iterator as return value for streaming, whereas TarFile needs to write to a file obj. Is there any way I can get these two to work together? Peter From victorsubervi at gmail.com Sat Jan 9 08:23:06 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 9 Jan 2010 08:23:06 -0500 Subject: Something More Elegant Message-ID: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> Hi; The following code works fine. I would like you to suggest something more simple and elegant: sql = 'select p.ID from %sPackages p join %sCategoriesPackages c where c.CategoryID=%s;' % (store, store, categoryID) cursor.execute(sql) tmp = [itm[0] for itm in cursor] packageIDs = [] for t in tmp: if t not in packageIDs: packageIDs.append(t) TIA, beno -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From iurisilvio at gmail.com Sat Jan 9 08:30:17 2010 From: iurisilvio at gmail.com (Iuri) Date: Sat, 9 Jan 2010 11:30:17 -0200 Subject: Something More Elegant In-Reply-To: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> Message-ID: <789aac5a1001090530w7d5d2039h137402c0f9e0f339@mail.gmail.com> Your code select some ids from database and list distinct ids in packageIDs. You can use SELECT DISTINCT in your SQL statement. On Sat, Jan 9, 2010 at 11:23 AM, Victor Subervi wrote: > Hi; > The following code works fine. I would like you to suggest something more > simple and elegant: > > sql = 'select p.ID from %sPackages p join %sCategoriesPackages c > where c.CategoryID=%s;' % (store, store, categoryID) > cursor.execute(sql) > tmp = [itm[0] for itm in cursor] > packageIDs = [] > for t in tmp: > if t not in packageIDs: > packageIDs.append(t) > > TIA, > beno > > -- > The Logos has come to bear > http://logos.13gems.com/ > > -- > http://mail.python.org/mailman/listinfo/python-list > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From python.list at tim.thechases.com Sat Jan 9 08:39:11 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Sat, 09 Jan 2010 07:39:11 -0600 Subject: Something More Elegant In-Reply-To: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> Message-ID: <4B4886FF.2030503@tim.thechases.com> Victor Subervi wrote: > Hi; > The following code works fine. I would like you to suggest something more > simple and elegant: > > sql = 'select p.ID from %sPackages p join %sCategoriesPackages c where > c.CategoryID=%s;' % (store, store, categoryID) > cursor.execute(sql) > tmp = [itm[0] for itm in cursor] > packageIDs = [] > for t in tmp: > if t not in packageIDs: > packageIDs.append(t) You mean like sql = "select distinct p.ID from ..." % (...) # ^^^^^^^^ cursor.execute(sql) package_ids = [row[0] for row in cursor.fetchall()] It would also help if you didn't pass the categoryID as a string-formatted value, but as a proper parameter, something like sql = "... where c.categoryid=?" % (store, store) cursor.execute(sql, (category_id,)) This helps prevent SQL-injection attacks (assuming you have full control over the value of "store"...otherwise, as you've been advised, if the remote user has control over the value in "store", you're asking to be exploited). You'd have to check the place-holder character for your particular back-end: >>> import as db >>> print db.paramstyle should tell you whether to use "?", "%s", or some other notation. From victorsubervi at gmail.com Sat Jan 9 09:01:25 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 9 Jan 2010 09:01:25 -0500 Subject: Something More Elegant In-Reply-To: <4B4886FF.2030503@tim.thechases.com> References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> Message-ID: <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> On Sat, Jan 9, 2010 at 8:39 AM, Tim Chase wrote: > Victor Subervi wrote: > >> Hi; >> The following code works fine. I would like you to suggest something more >> simple and elegant: >> >> sql = 'select p.ID from %sPackages p join %sCategoriesPackages c >> where >> c.CategoryID=%s;' % (store, store, categoryID) >> cursor.execute(sql) >> tmp = [itm[0] for itm in cursor] >> packageIDs = [] >> for t in tmp: >> if t not in packageIDs: >> packageIDs.append(t) >> > > You mean like > > sql = "select distinct p.ID from ..." % (...) > Oh, that's good! > # ^^^^^^^^ > cursor.execute(sql) > package_ids = [row[0] for row in cursor.fetchall()] > > It would also help if you didn't pass the categoryID as a string-formatted > value, but as a proper parameter, something like > > sql = "... where c.categoryid=?" % (store, store) > cursor.execute(sql, (category_id,)) > I now have the following: sql = 'select distinct p.ID from %sPackages p join %sCategoriesPackages c where c.CategoryID=?;' % (store, store) cursor.execute(sql, (categoryID,)) packageIDs = [itm[0] for itm in cursor] It threw this error: /var/www/html/angrynates.com/christians/cart/display.py 141 print '\n' 142 cursor.close() 143 bottom() 144 145 display() display = /var/www/html/angrynates.com/christians/cart/display.py in display() 109 categoryID = cursor.fetchone()[0] 110 sql = 'select distinct p.ID from %sPackages p join %sCategoriesPackages c where c.CategoryID=?;' % (store, store) 111 cursor.execute(sql, (categoryID,)) 112 packageIDs = [itm[0] for itm in cursor] 113 for pid in packageIDs: global cursor = , cursor.execute = >, sql = 'select distinct p.ID from productsPackages p join productsCategoriesPackages c where c.CategoryID=?;', categoryID = 1L /usr/lib64/python2.4/site-packages/MySQLdb/cursors.py in execute(self=, query='select distinct p.ID from productsPackages p join productsCategoriesPackages c where c.CategoryID=?;', args=(1L,)) 146 query = query.encode(charset) 147 if args is not None: 148 query = query % db.literal(args) 149 try: 150 r = self._query(query) query = 'select distinct p.ID from productsPackages p join productsCategoriesPackages c where c.CategoryID=?;', db = , db.literal = >, args = (1L,) TypeError: not all arguments converted during string formatting args = ('not all arguments converted during string formatting',) > This helps prevent SQL-injection attacks (assuming you have full control > over the value of "store"...otherwise, as you've been advised, if the remote > user has control over the value in "store", you're asking to be exploited). > They have control over it. I pass it in the url. Please advise. > You'd have to check the place-holder character for your particular > back-end: > > >>> import as db > >>> print db.paramstyle > > Printed "format". What's that mean? I use MySQLdb TIA, beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcd at sdf.lonestar.org Sat Jan 9 09:07:02 2010 From: jcd at sdf.lonestar.org (J. Cliff Dyer) Date: Sat, 09 Jan 2010 09:07:02 -0500 Subject: Another Screwy Problem In-Reply-To: <4dc0cfea1001090459g618727b9j6ef90fadda6b4464@mail.gmail.com> References: <4dc0cfea1001081132p191fe045p72a79744054fc520@mail.gmail.com> <20100108214455.GA12839@sdf.lonestar.org> <4dc0cfea1001090459g618727b9j6ef90fadda6b4464@mail.gmail.com> Message-ID: <1263046022.2685.2.camel@webb> On Sat, 2010-01-09 at 07:59 -0500, Victor Subervi wrote: > On Fri, Jan 8, 2010 at 4:44 PM, J. Clifford Dyer > wrote: > Victor Subervi wrote: > > Hi; > > I have this line of code: > > sql = 'select Name, Price from %sPackages where ID=%s;' % > (store, pid) > > which prints to this: > > select Name, Price from productsPackages where ID=1; > > which when I enter it into the MySQL interpreter gives me > this: > > mysql> select Name, Price from productsPackages where ID=1; > > +------+--------+ > > | Name | Price | > > +------+--------+ > > | pkg | 123.45 | > > +------+--------+ > > 1 row in set (0.00 sec) > > > > exactly what I expect. However, in my script for some reason > it returns > > this: > > ((1,),) > > > > First, got your other email. I thought I had executed the statement. > Oops. Works fine now. Sorry. > > First, never use string formatting to pass parameters to your > database. Read the MySQLdb documentation (or sqlite, or > psycopg2) documentation for reasons why, and how to do it > right. > > The only thing I found, which collaborates with something someone else > taught me on this list about entering binary data, is that one must > pass the parameters in the execute statement. Is that what you mean? > If so, I find that for all purposes thus far other than binary data, > the way I've been doing it seems to work just fine. I would prefer to > keep doing it that way, because I find putting a print statement > between the sql= line and the execute statement gives me a good > opportunity to review the sql statement and catch errors. Is this not > good practice? > > > Thanks. > beno This is a horrendous practice. You leave yourself vulnerable not only to attacks, but to simple absent-mindedness as well. Using parameters in your execute statement will handle all necessary quoting for you, which eliminates the possibility of a bad query sneaking in. For more information, as I mentioned, look up SQL injection. Also, read this: http://xkcd.com/327/ Cheers, Cliff From sjmachin at lexicon.net Sat Jan 9 09:12:36 2010 From: sjmachin at lexicon.net (John Machin) Date: Sat, 9 Jan 2010 06:12:36 -0800 (PST) Subject: How to get many places of pi from Machin's Equation? References: Message-ID: <60828494-e7aa-4a99-9c33-7d3a525afe4b@22g2000yqr.googlegroups.com> On Jan 9, 10:31?pm, "Richard D. Moores" wrote: > Machin's Equation is > > 4 arctan (1/5) - arctan(1/239) = pi/4 > > Using Python 3.1 and the math module: > > > > >>> from math import atan, pi > >>> pi > 3.141592653589793 > >>> (4*atan(.2) - atan(1/239))*4 > 3.1415926535897936 > >>> (4*atan(.2) - atan(1/239))*4 == pi > False > >>> abs((4*atan(.2) - atan(1/239))*4) - pi < .000000000000000001 > False > >>> abs((4*atan(.2) - atan(1/239))*4) - pi < .0000000000000001 > False > >>> abs((4*atan(.2) - atan(1/239))*4) - pi < .000000000000001 > True > > Is there a way in Python 3.1 to calculate pi to greater accuracy using > Machin's Equation? Even to an arbitrary number of places? Considering that my namesake calculated pi to 100 decimal places with the computational equipment available in 1706 (i.e. not much), I'd bet you London to a brick that Python (any version from 0.1 onwards) could be used to simulate his calculations to any reasonable number of places. So my answers to your questions are yes and yes. Suggestion: search_the_fantastic_web("machin pi python") From victorsubervi at gmail.com Sat Jan 9 09:14:47 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 9 Jan 2010 09:14:47 -0500 Subject: Another Screwy Problem In-Reply-To: <1263046022.2685.2.camel@webb> References: <4dc0cfea1001081132p191fe045p72a79744054fc520@mail.gmail.com> <20100108214455.GA12839@sdf.lonestar.org> <4dc0cfea1001090459g618727b9j6ef90fadda6b4464@mail.gmail.com> <1263046022.2685.2.camel@webb> Message-ID: <4dc0cfea1001090614g27ec483cv21b718f8a1776c1b@mail.gmail.com> On Sat, Jan 9, 2010 at 9:07 AM, J. Cliff Dyer wrote: > On Sat, 2010-01-09 at 07:59 -0500, Victor Subervi wrote: > > On Fri, Jan 8, 2010 at 4:44 PM, J. Clifford Dyer > > wrote: > > Victor Subervi wrote: > > > Hi; > > > I have this line of code: > > > sql = 'select Name, Price from %sPackages where ID=%s;' % > > (store, pid) > > > which prints to this: > > > select Name, Price from productsPackages where ID=1; > > > which when I enter it into the MySQL interpreter gives me > > this: > > > mysql> select Name, Price from productsPackages where ID=1; > > > +------+--------+ > > > | Name | Price | > > > +------+--------+ > > > | pkg | 123.45 | > > > +------+--------+ > > > 1 row in set (0.00 sec) > > > > > > exactly what I expect. However, in my script for some reason > > it returns > > > this: > > > ((1,),) > > > > > > > > First, got your other email. I thought I had executed the statement. > > Oops. Works fine now. Sorry. > > > > First, never use string formatting to pass parameters to your > > database. Read the MySQLdb documentation (or sqlite, or > > psycopg2) documentation for reasons why, and how to do it > > right. > > > > The only thing I found, which collaborates with something someone else > > taught me on this list about entering binary data, is that one must > > pass the parameters in the execute statement. Is that what you mean? > > If so, I find that for all purposes thus far other than binary data, > > the way I've been doing it seems to work just fine. I would prefer to > > keep doing it that way, because I find putting a print statement > > between the sql= line and the execute statement gives me a good > > opportunity to review the sql statement and catch errors. Is this not > > good practice? > > > > > > Thanks. > > beno > > This is a horrendous practice. You leave yourself vulnerable not only > to attacks, but to simple absent-mindedness as well. Using parameters > in your execute statement will handle all necessary quoting for you, > which eliminates the possibility of a bad query sneaking in. For more > information, as I mentioned, look up SQL injection. Also, read this: > http://xkcd.com/327/ > > Thanks :) beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at holdenweb.com Sat Jan 9 09:35:38 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 09 Jan 2010 09:35:38 -0500 Subject: Something More Elegant In-Reply-To: <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> Message-ID: Victor Subervi wrote: > On Sat, Jan 9, 2010 at 8:39 AM, Tim Chase > wrote: > > Victor Subervi wrote: > > Hi; > The following code works fine. I would like you to suggest > something more > simple and elegant: > > sql = 'select p.ID from %sPackages p join > %sCategoriesPackages c where > c.CategoryID=%s;' % (store, store, categoryID) > cursor.execute(sql) > tmp = [itm[0] for itm in cursor] > packageIDs = [] > for t in tmp: > if t not in packageIDs: > packageIDs.append(t) > > > You mean like > > sql = "select distinct p.ID from ..." % (...) > > > Oh, that's good! > > > # ^^^^^^^^ > cursor.execute(sql) > package_ids = [row[0] for row in cursor.fetchall()] > > It would also help if you didn't pass the categoryID as a > string-formatted value, but as a proper parameter, something like > > sql = "... where c.categoryid=?" % (store, store) > cursor.execute(sql, (category_id,)) > > > I now have the following: > > sql = 'select distinct p.ID from %sPackages p join > %sCategoriesPackages c where c.CategoryID=?;' % (store, store) > cursor.execute(sql, (categoryID,)) > packageIDs = [itm[0] for itm in cursor] > > It threw this error: > > /var/www/html/angrynates.com/christians/cart/display.py > > 141 print '\n' > 142 cursor.close() > 143 bottom() > 144 > 145 display() > display = > /var/www/html/angrynates.com/christians/cart/display.py > in display() > 109 categoryID = cursor.fetchone()[0] > 110 sql = 'select distinct p.ID from %sPackages p join > %sCategoriesPackages c where c.CategoryID=?;' % (store, store) > 111 cursor.execute(sql, (categoryID,)) > 112 packageIDs = [itm[0] for itm in cursor] > 113 for pid in packageIDs: > global cursor = , cursor.execute = method Cursor.execute of >, sql = 'select > distinct p.ID from productsPackages p join productsCategoriesPackages c > where c.CategoryID=?;', categoryID = 1L > /usr/lib64/python2.4/site-packages/MySQLdb/cursors.py in > execute(self=, query='select distinct > p.ID from productsPackages p join productsCategoriesPackages c where > c.CategoryID=?;', args=(1L,)) > 146 query = query.encode(charset) > 147 if args is not None: > 148 query = query % db.literal(args) > 149 try: > 150 r = self._query(query) > query = 'select distinct p.ID from productsPackages p join > productsCategoriesPackages c where c.CategoryID=?;', db = 0x2b79db9dc470 to Connection>, db.literal = Connection.literal of <_mysql.connection open to 'localhost' at > 142be8b0>>, args = (1L,) > > TypeError: not all arguments converted during string formatting > args = ('not all arguments converted during string formatting',) > > > This helps prevent SQL-injection attacks (assuming you have full > control over the value of "store"...otherwise, as you've been > advised, if the remote user has control over the value in "store", > you're asking to be exploited). > > > They have control over it. I pass it in the url. Please advise. > > > You'd have to check the place-holder character for your particular > back-end: > > >>> import as db > >>> print db.paramstyle > > Printed "format". What's that mean? I use MySQLdb > TIA, > beno > Given that you actually started this thread by asking a good question that showed you had done some independent work, I'll bite. The problem is something that was discussed in one of your other numerous threads by John Machin and me. The issue is the parameterization of (i.e. sticking variable bits into) SQL queries. When you write curs.execute("some sql query with %s and %s in it", (data1, data2)) the second argument to execute is supposed to contain data values. This allows the SQL engine to do the preparatory work for a query once, and then use the same "prepared query" then next time it's executed. The preparation involves scanning the SQL query to make sure the syntax is correct, validating the table and column names, and developing a "query execution plan" that is a sequence of internal operations the database performs to get you the answer you want. (SQL, unlike Python, is a "declarative" language - rather than telling it what to do you describe the results you want to see and the engine works out how to provide it). Of course, if different tables are used for different queries then there is no hope that the same execution plan can be used for them. For this reason most database processors (and this certainly includes the one you are using) don't allow you to parameterize anything in SQL statement other than data values. So curs.execute("select field1 from tableA where id=%s", (3, )) is OK, but curs.execute("select field1 from tableA where %s=3", ("id", )) is definitely not. And curs.execute("select field1 from %stable where id=3", (storename, )) is right out. This goes back to the issue of your database design. You have chosen to represent each store with its own set of tables, where an experienced database designer would instead have used just one set of tables, each table having a "store" column that would allow you to use parameterized queries to select the relevant data. There are many other sound reasons for making such a design choice, which primarily boil down to operational and management efficiency and simplicity. But we are now in the realm of theory as far as you are concerned, since you have already stated several times that you aren't interested in correcting your design until after you have got the current mess into production. So good luck with that. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From gagsl-py2 at yahoo.com.ar Sat Jan 9 09:42:16 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Sat, 09 Jan 2010 11:42:16 -0300 Subject: How to get many places of pi from Machin's Equation? References: Message-ID: En Sat, 09 Jan 2010 08:31:49 -0300, Richard D. Moores escribi?: > Is there a way in Python 3.1 to calculate pi to greater accuracy using > Machin's Equation? Even to an arbitrary number of places? You may be interested in Demo/scripts/pi.py in the source distribution. It can generate pi with infinite precision, limited by available memory only. And this thread from last month: http://groups.google.com/group/comp.lang.python/t/e37bb8c59f2e5582/ -- Gabriel Genellina From gagsl-py2 at yahoo.com.ar Sat Jan 9 10:01:47 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Sat, 09 Jan 2010 12:01:47 -0300 Subject: Something More Elegant References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> Message-ID: En Sat, 09 Jan 2010 11:01:25 -0300, Victor Subervi escribi?: > On Sat, Jan 9, 2010 at 8:39 AM, Tim Chase > wrote: > >> It would also help if you didn't pass the categoryID as a >> string-formatted >> value, but as a proper parameter, something like >> >> sql = "... where c.categoryid=?" % (store, store) >> cursor.execute(sql, (category_id,)) >> > > I now have the following: > > sql = 'select distinct p.ID from %sPackages p join > %sCategoriesPackages c where c.CategoryID=?;' % (store, store) > cursor.execute(sql, (categoryID,)) > packageIDs = [itm[0] for itm in cursor] > > It threw this error: > > TypeError: not all arguments converted during string formatting > args = ('not all arguments converted during string formatting',) > >> You'd have to check the place-holder character for your particular >> back-end: >> >> >>> import as db >> >>> print db.paramstyle >> >> Printed "format". What's that mean? I use MySQLdb That means, MySQLdb uses %s as a placeholder for parameter substitution -- same as Python when doing string interpolation. Unfortunately this will confuse things. In your code above, the ? near the end should become %s -- but you don't want THAT %s to be interpreted by Python at that time, instead it must remain as a literal %s until the cursor.execute line. You have to escape the % by doubling it: %%s sql = 'select distinct p.ID from %sPackages p join %sCategoriesPackages c where c.CategoryID=%%s;' % (store, store) cursor.execute(sql, (categoryID,)) packageIDs = [itm[0] for itm in cursor] -- Gabriel Genellina From vmail at mycircuit.org Sat Jan 9 10:09:32 2010 From: vmail at mycircuit.org (Peter) Date: Sat, 09 Jan 2010 16:09:32 +0100 Subject: restructuredText editor ? In-Reply-To: <5e6l17-28k.ln1@mid.florian-diesch.de> References: <5e6l17-28k.ln1@mid.florian-diesch.de> Message-ID: <4B489C2C.5030409@mycircuit.org> On 01/09/2010 03:32 AM, Florian Diesch wrote: > Peter writes: > > >> What editor do people out there use to edit .rst files for >> sphinx-python documentation ? >> > Emacs with ReST mode and YASnippet > > > Florian > Great, works very well and thanks for mentionning YASnippets ( useful for many types of documents ) Peter From 3lvss0809 at gmail.com Sat Jan 9 10:12:18 2010 From: 3lvss0809 at gmail.com (3lvss0809 at gmail.com) Date: Sat, 9 Jan 2010 07:12:18 -0800 (PST) Subject: Microsoft Office Word and Python (Win XP) Message-ID: Hi. Im very new with python. I have got some answer on my issue to use interop or COM ''plugins'' to access MS Word through python but i don't even know what those two ''plugins'' are so I cannot use them. What I want to do is the following: I need the script that moves (only moves, not change or delete!) entire (100% of the text) text from one .doc file to another. But its not so easy as it sounds. The target .doc file is not the only one but can be many of them. All the target .doc files are always in the same folder (same path) but all of them don't have the same name. The .doc file FROM where I want to move entire text is only one, always in the same folder (same path) and always with the same file name. Names of the target are only similar but as I have said before, not the same. Here is the point of whole script: Target .doc files have the names: HD1.doc HD2.doc HD3.doc HD4.doc and so on What I would like to have is moved the entire (but really all of the text, must be 100% all) text into the .doc file with the highest ( ! ) number. The target .doc files will always start with ''HD'' and always be similar to above examples. It is possible that the doc file (target file) is only one, so only HD1.doc. Therefore ''1'' is the maximum number and the text is moved into this file. Sometimes the target file is empty but usually won't be. If it won't be then the text should be moved to the end of the text, into first new line (no empty lines inbetween). So for example in the target file which has the maximum number in its name is the following text: a b c In the file from which I want to move the text is: d This means I need in the target file this: a b c d Could someone tell me please how to do this? Thank you. From 3lvss0809 at gmail.com Sat Jan 9 10:12:18 2010 From: 3lvss0809 at gmail.com (3lvss0809 at gmail.com) Date: Sat, 9 Jan 2010 07:12:18 -0800 (PST) Subject: Microsoft Office Word and Python (Win XP) Message-ID: Hi. Im very new with python. I have got some answer on my issue to use interop or COM ''plugins'' to access MS Word through python but i don't even know what those two ''plugins'' are so I cannot use them. What I want to do is the following: I need the script that moves (only moves, not change or delete!) entire (100% of the text) text from one .doc file to another. But its not so easy as it sounds. The target .doc file is not the only one but can be many of them. All the target .doc files are always in the same folder (same path) but all of them don't have the same name. The .doc file FROM where I want to move entire text is only one, always in the same folder (same path) and always with the same file name. Names of the target are only similar but as I have said before, not the same. Here is the point of whole script: Target .doc files have the names: HD1.doc HD2.doc HD3.doc HD4.doc and so on What I would like to have is moved the entire (but really all of the text, must be 100% all) text into the .doc file with the highest ( ! ) number. The target .doc files will always start with ''HD'' and always be similar to above examples. It is possible that the doc file (target file) is only one, so only HD1.doc. Therefore ''1'' is the maximum number and the text is moved into this file. Sometimes the target file is empty but usually won't be. If it won't be then the text should be moved to the end of the text, into first new line (no empty lines inbetween). So for example in the target file which has the maximum number in its name is the following text: a b c In the file from which I want to move the text is: d This means I need in the target file this: a b c d Could someone tell me please how to do this? Thank you. From iurisilvio at gmail.com Sat Jan 9 10:14:39 2010 From: iurisilvio at gmail.com (Iuri) Date: Sat, 9 Jan 2010 13:14:39 -0200 Subject: Something More Elegant In-Reply-To: References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> Message-ID: <789aac5a1001090714q1453984cp8754d27fd5be6102@mail.gmail.com> And you should use cursor.fetchall() instead of cursor in list comprehension: packageIDs = [itm[0] for itm in cursor.fetchall()] On Sat, Jan 9, 2010 at 1:01 PM, Gabriel Genellina wrote: > En Sat, 09 Jan 2010 11:01:25 -0300, Victor Subervi < > victorsubervi at gmail.com> escribi?: > >> On Sat, Jan 9, 2010 at 8:39 AM, Tim Chase > >wrote: >> >> It would also help if you didn't pass the categoryID as a >>> string-formatted >>> value, but as a proper parameter, something like >>> >>> sql = "... where c.categoryid=?" % (store, store) >>> cursor.execute(sql, (category_id,)) >>> >>> >> I now have the following: >> >> sql = 'select distinct p.ID from %sPackages p join >> %sCategoriesPackages c where c.CategoryID=?;' % (store, store) >> cursor.execute(sql, (categoryID,)) >> packageIDs = [itm[0] for itm in cursor] >> >> It threw this error: >> >> TypeError: not all arguments converted during string formatting >> args = ('not all arguments converted during string formatting',) >> >> You'd have to check the place-holder character for your particular >>> back-end: >>> >>> >>> import as db >>> >>> print db.paramstyle >>> >>> Printed "format". What's that mean? I use MySQLdb >>> >> > That means, MySQLdb uses %s as a placeholder for parameter substitution -- > same as Python when doing string interpolation. Unfortunately this will > confuse things. In your code above, the ? near the end should become %s -- > but you don't want THAT %s to be interpreted by Python at that time, instead > it must remain as a literal %s until the cursor.execute line. You have to > escape the % by doubling it: %%s > > sql = 'select distinct p.ID from %sPackages p join > %sCategoriesPackages c where c.CategoryID=%%s;' % (store, store) > > cursor.execute(sql, (categoryID,)) > packageIDs = [itm[0] for itm in cursor] > > -- > Gabriel Genellina > > > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Sat Jan 9 10:15:48 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 9 Jan 2010 10:15:48 -0500 Subject: Something More Elegant In-Reply-To: References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> Message-ID: <4dc0cfea1001090715md97518tf618309eda1deabb@mail.gmail.com> On Sat, Jan 9, 2010 at 9:35 AM, Steve Holden wrote: > But we are now in the realm of theory as far as you are concerned, since > you have already stated several times that you aren't interested in > correcting your design until after you have got the current mess into > production. So good luck with that. > And if you were in my shoes, I'm sure you'd do the same thing. Well, it *is* working now :)) And I am interested in cleaning this up. I should probably start with the matter of databases, since that's something I won't be able to easily change once clients actually start entering data. Please share with me any further concepts or questions to get me thinking how to redesign the databases. TIA, beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From sjmachin at lexicon.net Sat Jan 9 10:23:31 2010 From: sjmachin at lexicon.net (John Machin) Date: Sat, 9 Jan 2010 07:23:31 -0800 (PST) Subject: Porblem with xlutils/xlrd/xlwt References: <8e9245ec-05cb-48ed-ae32-f9fb0c46509f@j5g2000yqm.googlegroups.com> <972fdcfc-73a6-4e87-9752-3bc10da0dcad@s3g2000yqs.googlegroups.com> <43a1e323-8ccd-48b7-a6a4-e540aca2b1cd@35g2000yqa.googlegroups.com> <5ac38519-9ad4-4a89-9b3f-ccd56fb18721@m3g2000yqf.googlegroups.com> Message-ID: On Jan 9, 9:56?pm, pp wrote: > On Jan 9, 3:52?am, Jon Clements wrote: > > > > > On Jan 9, 10:44?am, pp wrote: > > > > On Jan 9, 3:42?am, Jon Clements wrote: > > > > > On Jan 9, 10:24?am, pp wrote: > > > yeah all my versions are latest fromhttp://www.python-excel.org. > > > just checked!! How did you check? > > > what could be the problem? > > > Does rb = xlrd.open_workbook('somesheet.xls', on_demand=True) work by > > itself? > > Yes it does. The problem is with line: wb = ?copy(rb) > here I am getting the error: AttributeError: 'Book' object has no > attribute 'on_demand' Please replace the first 4 lines of your script by these 6 lines: import xlrd assert xlrd.__VERSION__ == "0.7.1" from xlwt import easyxf from xlutils.copy import copy rb = xlrd.open_workbook( 'source.xls',formatting_info=True, on_demand=False) and run it again. Please copy all the output and paste it into your response. From victorsubervi at gmail.com Sat Jan 9 10:28:31 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 9 Jan 2010 10:28:31 -0500 Subject: Something More Elegant In-Reply-To: <789aac5a1001090714q1453984cp8754d27fd5be6102@mail.gmail.com> References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> <789aac5a1001090714q1453984cp8754d27fd5be6102@mail.gmail.com> Message-ID: <4dc0cfea1001090728g7997b969pcf5a59070dc83591@mail.gmail.com> On Sat, Jan 9, 2010 at 10:14 AM, Iuri wrote: > And you should use cursor.fetchall() instead of cursor in list > comprehension: > > packageIDs = [itm[0] for itm in cursor.fetchall()] > Now, someone else on this list told me the other. Can you explain the difference? TIA, beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From dickinsm at gmail.com Sat Jan 9 10:31:18 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 9 Jan 2010 07:31:18 -0800 (PST) Subject: How to get many places of pi from Machin's Equation? References: Message-ID: <3a3b9062-e01c-4115-a0e9-3b1b271e6c12@v25g2000yqk.googlegroups.com> On Jan 9, 11:31?am, "Richard D. Moores" wrote: > Is there a way in Python 3.1 to calculate pi to greater accuracy using > Machin's Equation? Even to an arbitrary number of places? There's no arbitrary-precision version of atan included with Python. You could write your own (e.g., based on argument reduction + Taylor series) for use with the decimal module, or you could use one of the various 3rd party arbitrary-precision arithmetic packages that do provide atan. Mark From mccolgst at gmail.com Sat Jan 9 10:37:48 2010 From: mccolgst at gmail.com (McColgst) Date: Sat, 9 Jan 2010 07:37:48 -0800 (PST) Subject: Scripting (was Re: Python books, literature etc) References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: <741365c6-d955-40ec-bbe9-104c8c524534@k19g2000yqc.googlegroups.com> Just to kind of get back on topic: Before buying a book or making a terribly large investment, OP should consider the fact that Python 3 is out and gaining some popularity. From dickinsm at gmail.com Sat Jan 9 10:57:05 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 9 Jan 2010 07:57:05 -0800 (PST) Subject: How to get many places of pi from Machin's Equation? References: Message-ID: On Jan 9, 11:31?am, "Richard D. Moores" wrote: > Machin's Equation is > > 4 arctan (1/5) - arctan(1/239) = pi/4 > [...] > > Is there a way in Python 3.1 to calculate pi to greater accuracy using > Machin's Equation? Even to an arbitrary number of places? Here's some crude code (no error bounds, possibility of infinite loops, ...) that computes pi to 1000 places using Machin's formula and the decimal module. The last few digits will be bogus, and should be ignored. from decimal import Decimal, getcontext def atan(x): # reductions reductions = 0 while 100*abs(x) > 1: reductions += 1 x /= 1 + (1+x*x).sqrt() # Taylor series sum = 0 xpow = x x2 = x*x k = 1 while True: term = xpow/k oldsum = sum sum += term if sum == oldsum: break k += 2 xpow *= -x2 return sum * 2**reductions getcontext().prec = 1000 one = Decimal(1) print(16*atan(one/5) - 4*atan(one/239)) From victorsubervi at gmail.com Sat Jan 9 11:11:28 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 9 Jan 2010 11:11:28 -0500 Subject: Easy Q Message-ID: <4dc0cfea1001090811y198462a8s39976a0622ef6c59@mail.gmail.com> Hi; I have a string.join statement on a variable that comes from a cgi.FieldStorage().getlist. The variable may be a list or a single value. I need to treat it differently depending on which it is. How can I distinguish it? len(var) will obviously give me the length of the string if it's a string and the length of the list if it's a list. TIA, beno -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at REMOVE-THIS-cybersource.com.au Sat Jan 9 11:19:30 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 09 Jan 2010 16:19:30 GMT Subject: Link to module Stack References: <0358384b$0$1336$c3e8da3@news.astraweb.com> Message-ID: <03589968$0$1336$c3e8da3@news.astraweb.com> On Sat, 09 Jan 2010 05:56:36 -0500, Dave Angel wrote: >> "InnerInterpreterError" is the most inappropriate exception name I've >> ever seen. It has nothing to do with the interpreter, it's a stack >> error. >> >> > It has everything to do with the (Forth) interpreter. Exceptions can > readily be named according to their application -- it's not always about > Python. Anyway, Forth has an inner-interpreter and an > outer-interpreter, and the name will make sense to a Forth programmer. Pardon me, but I *am* a Forth programmer. Or was, it's been many years, and I'm rusty. I guess this is a difference of terminology: what you're calling an inner interpreter and an outer interpreter I know of as the Forth engine and the (text) interpreter. Gforth refers to them as such, so did Leo Brodie's Forth books, and the (ancient) Macintosh Forth compiler "Mach 2". But in any case... a stack is an general-purpose data structure, and the error message shouldn't be coupled so tightly to one use. That would be like this (made-up) example: >>> 1/0 Traceback (most recent call last): File "", line 1, in GraphicsApplicationError: too few pixels to calculate average Ridiculous, yes? Yes, Forth uses a stack (technically two, a parameter stack and a return stack, and some implementations include a third, floating point, stack). Virtually all languages use stacks in their implementation, and Python byte-code is also stack-based. >>> result = self.__heap[-1] >>> del self.__heap[-1] >>> >>> >> That is better written as result = self.__heap.pop(). >> >> >> > or even better, without the extra local var: > > def pop (self): > if len(self.__heap) == 0: > raise InnerInterpreterError, "stack underflow" > return self.__heap.pop(1) pop(1)? I don't think so. >>> L = list('abcdef') >>> L.pop(1) 'b' >>> L ['a', 'c', 'd', 'e', 'f'] You probably meant pop(-1), but that's unnecessary because pop defaults to popping from the end of the list. > P.S. - I'm puzzled why the OP even put this message here. There's no > question posted with it. Me too. -- Steven From mail.to.daniel.platz at googlemail.com Sat Jan 9 11:30:07 2010 From: mail.to.daniel.platz at googlemail.com (Daniel Platz) Date: Sat, 9 Jan 2010 08:30:07 -0800 (PST) Subject: Manipulating pointers in C using ctypes References: <08bda34e-9bee-44b1-b2de-80d647151786@a15g2000yqm.googlegroups.com> Message-ID: <78689d52-b729-4bc4-8c6b-34f415c6c869@u41g2000yqe.googlegroups.com> Thanks for valuable answers. Both solutions work and I understand my mistake. Best regards, Daniel From rdmoores at gmail.com Sat Jan 9 11:32:48 2010 From: rdmoores at gmail.com (Richard D. Moores) Date: Sat, 9 Jan 2010 08:32:48 -0800 Subject: How to get many places of pi from Machin's Equation? In-Reply-To: References: Message-ID: On Sat, Jan 9, 2010 at 07:57, Mark Dickinson wrote: > On Jan 9, 11:31?am, "Richard D. Moores" wrote: >> Machin's Equation is >> >> 4 arctan (1/5) - arctan(1/239) = pi/4 >> [...] >> >> Is there a way in Python 3.1 to calculate pi to greater accuracy using >> Machin's Equation? Even to an arbitrary number of places? > > Here's some crude code (no error bounds, ?possibility of infinite > loops, ...) that computes pi to 1000 places using Machin's formula and > the decimal module. ?The last few digits will be bogus, and should be > ignored. > > from decimal import Decimal, getcontext > > def atan(x): > ? ?# reductions > ? ?reductions = 0 > ? ?while 100*abs(x) > 1: > ? ? ? ?reductions += 1 > ? ? ? ?x /= 1 + (1+x*x).sqrt() > > ? ?# Taylor series > ? ?sum = 0 > ? ?xpow = x > ? ?x2 = x*x > ? ?k = 1 > ? ?while True: > ? ? ? ?term = xpow/k > ? ? ? ?oldsum = sum > ? ? ? ?sum += term > ? ? ? ?if sum == oldsum: > ? ? ? ? ? ?break > ? ? ? ?k += 2 > ? ? ? ?xpow *= -x2 > > ? ?return sum * 2**reductions > > getcontext().prec = 1000 > one = Decimal(1) > print(16*atan(one/5) - 4*atan(one/239)) > -- > http://mail.python.org/mailman/listinfo/python-list > Great! Done in Python 3 with arctan and the decimal module. And the first 997 digits were accurate. Just what I was after. I don't believe the Chudnovsky algorithm has been mentioned. It isn't what I wanted, but it is amazing. () Thanks, everyone! Dick Moores From dreadpiratejeff at gmail.com Sat Jan 9 11:34:29 2010 From: dreadpiratejeff at gmail.com (J) Date: Sat, 9 Jan 2010 11:34:29 -0500 Subject: table from csv file In-Reply-To: <8e1d52da-f6e8-4600-8afa-3cc9be7bee80@a21g2000yqc.googlegroups.com> References: <59653a09-c3f7-464f-a36f-475f0d759e46@e27g2000yqd.googlegroups.com> <77b18520-c2c4-4e21-9147-1164d81b9f25@l30g2000yqb.googlegroups.com> <8e1d52da-f6e8-4600-8afa-3cc9be7bee80@a21g2000yqc.googlegroups.com> Message-ID: <36dec4ff1001090834q756aa826u66486a8535484861@mail.gmail.com> On Sat, Jan 9, 2010 at 05:26, Jon Clements wrote: >> reader = open('C:/test.txt','rb') >> data = csv.DictReader(reader,restval='000',restkey='Misc') > > [snip] > > DictReader works, but what use to bug me was the fact you couldn't > then output the cols in the 'correct' order afterwards, so you had > to store the header row anyway to re-order the rows... > (although admittedly this doesn't affect the OP's question). > > However, I see that 2.6+ offers .fieldnames on DictReader objects. At a guess, and I stipulate that because I am just learning about all this, I'd say that's because DictReader returns a dictionary for each row retrieved... It DOES take the first row and make those the dict keys, but after that, it just returns a Dict for each rows and dictionaries seem to be randomly ordered. It may not be that great if you're doing something that needs order, but all he's doing is pulling arbitrary columns from a file and printing as a table, so that works great in this case... It was quite interesting messing around with that yesterday. I saw the question, and it looked interesting, so I went and learned and applied ;-) I'm rather smugly satisfied with myself ... Cheers, Jeff -- Samuel Goldwyn - "I'm willing to admit that I may not always be right, but I am never wrong." - http://www.brainyquote.com/quotes/authors/s/samuel_goldwyn.html From python at mrabarnett.plus.com Sat Jan 9 11:48:45 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 09 Jan 2010 16:48:45 +0000 Subject: Easy Q In-Reply-To: <4dc0cfea1001090811y198462a8s39976a0622ef6c59@mail.gmail.com> References: <4dc0cfea1001090811y198462a8s39976a0622ef6c59@mail.gmail.com> Message-ID: <4B48B36D.3000402@mrabarnett.plus.com> Victor Subervi wrote: > Hi; > I have a string.join statement on a variable that comes from a > cgi.FieldStorage().getlist. The variable may be a list or a single > value. I need to treat it differently depending on which it is. How can > I distinguish it? len(var) will obviously give me the length of the > string if it's a string and the length of the list if it's a list. > 1. string.join isn't a statement, it's a function, and why are you using it anyway? Strings have a .join method: >>> ", ".join(["first", "second", "third"]) 'first, second, third' 2. Are you sure cgi.FieldStorage().getlist sometimes returns a single value instead of a list? If that's the case then it's a very odd name for the method! From gherron at islandtraining.com Sat Jan 9 11:56:12 2010 From: gherron at islandtraining.com (Gary Herron) Date: Sat, 09 Jan 2010 08:56:12 -0800 Subject: Easy Q In-Reply-To: <4dc0cfea1001090811y198462a8s39976a0622ef6c59@mail.gmail.com> References: <4dc0cfea1001090811y198462a8s39976a0622ef6c59@mail.gmail.com> Message-ID: <4B48B52C.3020409@islandtraining.com> Victor Subervi wrote: > Hi; > I have a string.join statement on a variable that comes from a > cgi.FieldStorage().getlist. The variable may be a list or a single > value. I need to treat it differently depending on which it is. How > can I distinguish it? len(var) will obviously give me the length of > the string if it's a string and the length of the list if it's a list. > TIA, > beno Like this: if isinstance(var, list): ... join ... else: ... ??? ... Gary Herron From chambon.pascal at gmail.com Sat Jan 9 12:25:06 2010 From: chambon.pascal at gmail.com (Pakal) Date: Sat, 9 Jan 2010 09:25:06 -0800 (PST) Subject: Direct use of bytearray buffers with ctypes ? Message-ID: <9694d6f5-5189-4371-a933-f7c5da5b2d30@e27g2000yqd.googlegroups.com> Hello I'm having big trouble wrapping the win32 ReadFile() function with ctypes. I wanted to allow the user to give a bytearray to this function, so that the writable buffer of this bytearray is directly used to receive the data from the file ; thus, no temporary copy in a separate ctype buffer would be required. However, I've found no way to provide ReadFile with a pointer to the inner buffer of the bytearray object. I get misc. TypeErrors and ValueErrors all the time, by trying to use the from_buffer() method or other ctypes functions. Is that actually posisble to expose with ctypes the internals of a python object to the system, or is that unsupported at the moment ? From benjamin at python.org Sat Jan 9 12:29:33 2010 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 9 Jan 2010 11:29:33 -0600 Subject: [RELEASED] Python 2.7 alpha 2 Message-ID: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> On behalf of the Python development team, I'm gleeful to announce the second alpha release of Python 2.7. Python 2.7 is scheduled to be the last major version in the 2.x series. It includes many features that were first released in Python 3.1. The faster io module, the new nested with statement syntax, improved float repr, and the memoryview object have been backported from 3.1. Other features include an ordered dictionary implementation, unittests improvements, and support for ttk Tile in Tkinter. For a more extensive list of changes in 2.7, see http://doc.python.org/dev/whatsnew/2.7.html or Misc/NEWS in the Python distribution. To download Python 2.7 visit: http://www.python.org/download/releases/2.7/ Please note that this is a development release, intended as a preview of new features for the community, and is thus not suitable for production use. The 2.7 documentation can be found at: http://docs.python.org/2.7 Please consider trying Python 2.7 with your code and reporting any bugs you may notice to: http://bugs.python.org Have fun! -- Benjamin Peterson 2.7 Release Manager benjamin at python.org (on behalf of the entire python-dev team and 2.7's contributors) From nawijn at gmail.com Sat Jan 9 12:58:48 2010 From: nawijn at gmail.com (Marco Nawijn) Date: Sat, 9 Jan 2010 09:58:48 -0800 (PST) Subject: Microsoft Office Word and Python (Win XP) References: Message-ID: <0c53a831-98f8-4e17-8aff-75759d3789ed@m25g2000yqc.googlegroups.com> On Jan 9, 4:12?pm, "3lvss0... at gmail.com" <3lvss0... at gmail.com> wrote: > Hi. > Im very new with python. I have got some answer on my issue to use > interop or COM ''plugins'' to access MS Word through python but i > don't even know what those two ''plugins'' are so I cannot use them. > What I want to do is the following: > > I need the script that moves (only moves, not change or delete!) > entire (100% of the text) text from one .doc file to another. But its > not so easy as it sounds. The target .doc file is not the only one but > can be many of them. All the target .doc files are always in the same > folder (same path) but all of them don't have the same name. The .doc > file FROM where I want to move entire text is only one, always in the > same folder (same path) and always with the same file name. > Names of the target are only similar but as I have said before, not > the same. Here is the point of whole script: > Target .doc files have the names: > HD1.doc > HD2.doc > HD3.doc > HD4.doc > and so on > > What I would like to have is moved the entire (but really all of the > text, must be 100% all) text into the .doc file with the highest ( ! ) > number. The target .doc files will always start with ''HD'' and always > be similar to above examples. > It is possible that the doc file (target file) is only one, so only > HD1.doc. Therefore ''1'' is the maximum number and the text is moved > into this file. > Sometimes the target file is empty but usually won't be. If it won't > be then the text should be moved to the end of the text, into first > new line (no empty lines inbetween). > So for example in the target file which has the maximum number in its > name is the following text: > > a > b > c > > In the file from which I want to move the text is: > > d > > This means I need in the target file this: > > a > b > c > d > > Could someone tell me please how to do this? > > Thank you. Hi, I will try to head you in the right direction with the Python/MS.Word link. First of all, you need to install the win32 extension. See http://sourceforge.net/projects/pywin32/ Once you have this installed you can instantiate a MS.Word application like this (code untested): >>> from win32com.client import Dispatch >>> app = Dispatch("Word.Application") >>> app.Visible = True The code so-far is more or less equivalent to opening Word without opening a document (normally Word will start with an empty document). To open a document do something like the following. >>> doc = app.Documents.Open("c:\\example.doc") Further builtin Python modules that could be helpfull are: glob -> for searching files matching a pattern os, os.path -> for path related functionality like stripping directory names from a complete path Take a look at the online documentation for more information http://docs.python.org/modindex.html Good luck and let us know the result. Marco From tjreedy at udel.edu Sat Jan 9 13:00:07 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 09 Jan 2010 13:00:07 -0500 Subject: Microsoft Office Word and Python (Win XP) In-Reply-To: References: Message-ID: On 1/9/2010 10:12 AM, 3lvss0809 at gmail.com wrote: > I need the script that moves (only moves, not change or delete!) > entire (100% of the text) text from one .doc file to another. If you want to copy files without modification, use the OS copy command. You can use the subprocess module to do that from Python. From davea at ieee.org Sat Jan 9 13:02:22 2010 From: davea at ieee.org (Dave Angel) Date: Sat, 09 Jan 2010 13:02:22 -0500 Subject: Easy Q In-Reply-To: <4dc0cfea1001090811y198462a8s39976a0622ef6c59@mail.gmail.com> References: <4dc0cfea1001090811y198462a8s39976a0622ef6c59@mail.gmail.com> Message-ID: <4B48C4AE.2060100@ieee.org> Victor Subervi wrote: > Hi; > I have a string.join statement on a variable that comes from a > cgi.FieldStorage().getlist. The variable may be a list or a single value. I > need to treat it differently depending on which it is. How can I distinguish > it? len(var) will obviously give me the length of the string if it's a > string and the length of the list if it's a list. > TIA, > beno > > If you have to do an explicit type check of your variable, use an if statement, and the isinstance() built-in function. Note that you can check for multiple types, like list and tuple, in the same function call. DaveA From Dave Sat Jan 9 13:06:53 2010 From: Dave (Dave) Date: Sat, 09 Jan 2010 12:06:53 -0600 Subject: interactive terminal in Ubuntu Linux : libreadline5-dev works only in Python 2.6 not 3.1 Message-ID: On Jan 6 I inquired how to "fix" the 3.1.1 interactive terminal in Ubuntu Linux. Left arrow yields ^[[D , etc. casevh helped by suggesting "libreadline5-dev" be installed. Did so with Synaptic Package Manager. The behavior of the Python 3.3.1 terminal is unchanged but the 2.6.2 terminal is corrected. Is there any way to fix the 3.1.1 terminal command line ? Thanks, Dave WB3DWE pdlemper at earthlink.net From victorsubervi at gmail.com Sat Jan 9 13:18:26 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 9 Jan 2010 13:18:26 -0500 Subject: Easy Q In-Reply-To: <4B48B52C.3020409@islandtraining.com> References: <4dc0cfea1001090811y198462a8s39976a0622ef6c59@mail.gmail.com> <4B48B52C.3020409@islandtraining.com> Message-ID: <4dc0cfea1001091018t726f283g381f815723e20f94@mail.gmail.com> On Sat, Jan 9, 2010 at 11:56 AM, Gary Herron wrote: > Victor Subervi wrote: > >> Hi; >> I have a string.join statement on a variable that comes from a >> cgi.FieldStorage().getlist. The variable may be a list or a single value. I >> need to treat it differently depending on which it is. How can I distinguish >> it? len(var) will obviously give me the length of the string if it's a >> string and the length of the list if it's a list. >> TIA, >> beno >> > > Like this: > > if isinstance(var, list): > ... join ... > else: > ... ??? ... > Right.. Thanks! beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Sat Jan 9 13:22:10 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 9 Jan 2010 13:22:10 -0500 Subject: Something More Elegant In-Reply-To: References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> <789aac5a1001090714q1453984cp8754d27fd5be6102@mail.gmail.com> <4dc0cfea1001090728g7997b969pcf5a59070dc83591@mail.gmail.com> Message-ID: <4dc0cfea1001091022n57f901c7gead57d0d9b31ebec@mail.gmail.com> On Sat, Jan 9, 2010 at 1:00 PM, Dennis Lee Bieber wrote: > On Sat, 9 Jan 2010 10:28:31 -0500, Victor Subervi > declaimed the following in > gmane.comp.python.general: > > > On Sat, Jan 9, 2010 at 10:14 AM, Iuri wrote: > > > > > And you should use cursor.fetchall() instead of cursor in list > > > comprehension: > > > > > > packageIDs = [itm[0] for itm in cursor.fetchall()] > > > > > > > Now, someone else on this list told me the other. Can you explain the > > difference? > > Since a (one-liner) list comprehension is going to result in > processing all the data, it may be faster to fetch all the data at the > start. Depending upon the implementation of the database API, iterating > over a cursor object to retrieve one record at a time may result in a > hit on the database engine (some engines may hold the data in the server > and only transmit it record by record unless explicitly asked for all of > it -- and if the database server is on a different machine that could > make for a lot of slow network traffic). > > Also, you need to take the database locking system into account -- > until you not only read all the data, but commit the transaction (even a > read-only transaction), the engine may lock any other user from > accessing those records (and some engines may lock the entire table, not > just records). > > Iterating over a cursor may be useful if: 1) you have a really > massive database and your query is not filtering the data to manageable > levels (better done by looping over the query itself using limit and > offset features to control how many records are returned in a batch); 2) > it is a single user/single access database where locking data won't have > an impact; 3) you need to use the data in each record to update or > otherwise perform some other query on the database (though unless this > is done within the same transaction you might still have locking > issues). > > Wow! Thanks. I'll digest this over the weekend. beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From fetchinson at googlemail.com Sat Jan 9 14:01:33 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Sat, 9 Jan 2010 20:01:33 +0100 Subject: lightweight encryption of text file In-Reply-To: References: Message-ID: >> I have a plain text file which I would like to protect in a very >> simple minded, yet for my purposes sufficient, way. I'd like to >> encrypt/convert it into a binary file in such a way that possession of >> a password allows anyone to convert it back into the original text >> file while not possessing the password one would only see the >> following with the standard linux utility 'file': >> >> [fetchinson at fetch ~]$ file encrypted.data >> encrypted.data: data > [...] > > This is probably not what you want, but it is very simple and doesn't > import any module:) I am not qualified to say how easy it is to discover > the message without the password. > > def str2int(txt): > return reduce(lambda n, c: n*255 + ord(c), txt, 0) > > def int2str(n): > chars = [] > while n: > n, o = divmod(n, 255) > chars.append(chr(o)) > return ''.join(reversed(chars)) > > def encrypt(txt, pwd): > return int2str(str2int(txt)*str2int(pwd)) > > def decrypt(txt, pwd): > return int2str(str2int(txt)/str2int(pwd)) > > def test(txt, pwd): > encrypted_txt = encrypt(txt, pwd) > decrypted_txt = decrypt(encrypted_txt, pwd) > print "text:%r" % txt > print "encrypted:%r" % encrypted_txt > print "decrypted:%r" % decrypted_txt > > >>>> test("This encryption scheme is definitely unbreakable.", "secret") > text:'This encryption scheme is definitely unbreakable.' > encrypted:'&2\xa5\xd4\x17i+E\x01k\xfa\x94\xf80\xa8\x8f\xea.w\x128\xf1\xd9\x0f9\xf2t\xc9\r`\x90%\xd6\xf3~\x1f\x00%u&\x8a\xe4\xe0\xa7\xb8\xb0ec)S>\xcb\xf2>\xec' > decrypted:'This encryption scheme is definitely unbreakable.' Thanks, this looks pretty simple too, I will go with either Steven's or with your solution. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From 3lvss0809 at gmail.com Sat Jan 9 14:18:12 2010 From: 3lvss0809 at gmail.com (3lvss0809 at gmail.com) Date: Sat, 9 Jan 2010 11:18:12 -0800 (PST) Subject: Microsoft Office Word and Python (Win XP) References: Message-ID: Marco Nawijn: I have had installed pywin32 already. The three lines that you mentoined don't do this, also what did you mean with "doc = app.Documents.Open("c:\\example.doc")". Which document should I open with this line? It shouldn't be opened anything. I was asking about the script as automated process. If you know how could i do this? Terry Reedy: I have never mentoined copying files but moving the whole text from, always the same (same name, same path), .doc file with. However copying (=moving) text to correct .doc file would be good yes. I know command prompt and its copy function but this way it wouldn't work because I would have to define the target file - the file INTO which I want to move the text. But I will never know the file name (target file). The only thing I know is: - the file is one of .doc files that start with "HD" - whole name of those .doc file is always HDX.doc where X is a number and I need to move the text into the file with maximum X (the most high number) - all the HD files will be always in the same path (same folder) but I would like to use the path inside the code (it might be obvious that I have to) because on PC and laptop, I have two different usernames and since HD files are located inside Documents And Settings, I have to use two copies of the script - one for PC, one for laptop. Dennis Lee Bieber: Im not familiar with python, also Im not programmer. Thats why Im not able to do so when people tell me "do this then use XYZ function which will give you ZYX from what you can do that and you will get result". Im still willing to learn but there are thousands of python tutorials and the one for exsactly this topic probably doesn't exsist. The .doc extension is required, so I cannot use .txt because I need the HD files in .doc. From bartc at freeuk.com Sat Jan 9 14:19:33 2010 From: bartc at freeuk.com (bartc) Date: Sat, 09 Jan 2010 19:19:33 GMT Subject: Scripting (was Re: Python books, literature etc) In-Reply-To: References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: <9F42n.24112$Ym4.2976@text.news.virginmedia.com> "Peter" wrote in message news:mailman.661.1262978839.28905.python-list at python.org... > >> Sounds good. >> >> Regarding the book's title: is it just me, or are Python programmers >> in general put off when people call it "scripting"? >> >> I won't attempt a strict definition of the term "scripting language", >> but it seems like non-programmers use it to mean "less scary than what >> you might think of as programming", while programmers interpret it as >> "not useful as a general-purpose language". >> >> > It took me a while to take "scripting" seriously. I grew up with Pascal > and Eiffel and I found it difficult to appreciate dynamic typing and > scripting. The author Langtangen is explaining in detail why he considers > scripting useful, in particular he provides an automatic test suite to run > different language versions ( perl, python, c, c++) of the same program to > compare performance. The results are amazing, in that some of the examples > run faster than the C++ version. I think if you can get Python to run fast (compared to compiled languages), then that's scripting (ie. just using it to sequence lots of built-in functions and operations). If it runs a lot slower than those other languages, then you're probably doing some programming. And with programs where the runtime is not significant, it could be either... -- Bartc From cousinstanley at gmail.com Sat Jan 9 14:52:52 2010 From: cousinstanley at gmail.com (Cousin Stanley) Date: Sat, 9 Jan 2010 19:52:52 +0000 (UTC) Subject: PIL show() not working for 2nd pic References: Message-ID: > I am using PIL for image processing in ubuntu 9.04. When i give two > im.show() commands for two different images, the second image is not > displayed (eye of gnome is the display program). It says no such file > or directory. Any ideas? Suresh .... I also had problems with show() when using eye of gnome as the image viewer with your code but no problems using the display viewer from the imagemagick package .... im.show( command = 'eog' ) # problems im.show( command = 'display' ) # no problems # ---------------------------------------------------------- #!/usr/bin/python ''' NewsGroup .... comp.lang.python Subject ...... PIL show() not working for 2nd pic Date ......... 2010-01-07 Post_By ...... suresh.amritapuri Edit_By ...... Stanley C. Kitching ''' import math import Image list_source = [ 'image/beach.tif' , 'image/colors.tif' ] list_target = [ ] def neg( x ) : return 255 - 1 - x def logtr( x ) : y = math.log( 1 + x , 10 ) print y return y * 100 for this_image in list_source : im_source = Image.open( this_image ) im_neg = im_source.point( neg ) im_logtr = im_source.point( logtr ) list_target.append( im_source ) list_target.append( im_neg ) list_target.append( im_logtr ) for this_image in list_target : this_image.show( command = 'display' ) # ---------------------------------------------------------- -- Stanley C. Kitching Human Being Phoenix, Arizona From zuo at chopin.edu.pl Sat Jan 9 14:53:42 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sat, 09 Jan 2010 20:53:42 +0100 Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2 In-Reply-To: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> Message-ID: Hello, I have a question: are class decorator planned to be backported from 3.x? All the best, *j -- Jan Kaliszewski (zuo) From apt.shansen at gmail.com Sat Jan 9 15:00:27 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Sat, 9 Jan 2010 12:00:27 -0800 Subject: Something More Elegant In-Reply-To: <4dc0cfea1001090715md97518tf618309eda1deabb@mail.gmail.com> References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> <4dc0cfea1001090715md97518tf618309eda1deabb@mail.gmail.com> Message-ID: <7a9c25c21001091200r10c3c7b2jbc16040efa9f7b34@mail.gmail.com> On Sat, Jan 9, 2010 at 7:15 AM, Victor Subervi wrote: > On Sat, Jan 9, 2010 at 9:35 AM, Steve Holden wrote: > >> But we are now in the realm of theory as far as you are concerned, since >> you have already stated several times that you aren't interested in >> correcting your design until after you have got the current mess into >> production. So good luck with that. >> > > And if you were in my shoes, I'm sure you'd do the same thing. ... Really, no, he wouldn't :) We're not all just hobbyists here who only do work for random open source projects. A lot of us are professionals who actually do have clients, actually do have deadlines, actually do have an understanding for production requirements. Getting something into production as soon as possible is certainly an important goal in commercial work. But it is not the only goal. Proper database design is very important because if you don't do it, you'll actually end up usually wasting *more* time and effort then if you just bite the bullet and fix it now. Proper database design -- in particular in your case, not having multiple tables with various names that even need "%sThis" or "%sThat", and using parameterized queries to access those tables, is really important. It will save you time, it will save you effort, and it'll save you money-- because /not/ doing it is among other things, a major security risk. Getting code out fast and now is a noble goal in commercial projects, getting code out which is by design prone to attacks by hackers is negligence. Well, it *is* working now :)) And I am interested in cleaning this up. I > should probably start with the matter of databases, since that's something I > won't be able to easily change once clients actually start entering data. > Please share with me any further concepts or questions to get me thinking > how to redesign the databases. > Your first goal needs to be in the layout of your tables, yes. Instead of having multiple "Packages" tables that vary, have a single Packages table, with an additional column that determines what kind of package it is (this information used to be in the name of the table). This may seem like everything's jumbled together, but it works. Put an index on that column if you need to: don't worry about if that one table might have thousands or tens of thousands of records. Databases are -designed- to handle that, and handle it well and faster. Do the same for any other table which has various names-- CategoriesPackages seems to be another one. Segment the data by adding columns and indexes when needed, and not by breaking it out into different tables: if two tables have approximately the same columns, they belong in one table, with a column to simply differentiate between the two. These steps are taking your database towards the ultimate goal of normalization-- a laudable goal, but I won't go into that in detail. It takes a book. The next thing to do is to re-do your SQL queries. You should never do string interpolation, e.g: SQL="SELECT x FROM y WHERE z = %s" % (arg,) cur.execute(SQL) If you have done the first step, your tables always have a set names so you never need to interpolate to do the queries-- and then at this point, under no circumstances ever, ever-- consider this a law that you will get fined for violation-- use string interpolation to generate your queries. Instead, do: cur.execute("SELECT x FROM y WHERE z = %s", (arg,)) That looks really similar, but is lightyears away. Its very unfortunate that some database drivers use 'format' as the paramstyle because it confuses issues, but the two effects are very different. In one, Python is just munging together and creating a new string. In the other, the database driver is doing a few things. Its analyzing the query, its storing it (often caching it, which speeds up further executions of that query), etc, etc, and then finally its seeing that you are passing arguments into it, and it is -safely- binding those arguments into the expression; this prevents SQL Injection attacks. You can use interpolation and prevent injection if you -meticulously- check -every- string that comes from the user, and -never- trust it (even if that string was written out to a hidden and legitimate users have no way to alter, because illegitimate users will alter it anyways). Or you can use parameterized queries and just avoid it, while getting plenty of other benefits as well. At work, we had a third-party package that we re-sold as part of our offering, and glancing over its source, I noticed something. It did, in essence to check login: cur.execute("SELECT user_id FROM usertable WHERE username = '%s' AND password = '%s' % (username, password)) I blinked, and emailed them to point out the problem. I suggested they log in as: Username = dummyuser Password = '; DROP usertable You see, when using interpolation, the string that got sent to the database was: SELECT user_id FROM usertable WHERE username = 'dummyuser' AND password = ''; DROP usertable And thus, from the login screen of this app-- I destroyed their environment. Its sort of important that you not put code out into production which is susceptible to such things. Your clients will not at all appreciate it if and when some hacker discovers it and destroys their site, loosing them money. But even beyond that, there are many other benefits to just doing this right. You want to, believe me. Now, before you put anything into production. It might take a day or two longer, but its worth it. Finally, go finish getting rid of the bare excepts, I saw one recently :) Just rip them all out, every one. All at once. Then fix any errors that come along with specific excepts or pre-tests if appropriate. ;) --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From no.email at nospam.invalid Sat Jan 9 15:03:16 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 09 Jan 2010 12:03:16 -0800 Subject: lightweight encryption of text file References: Message-ID: <7xr5pzoxkr.fsf@ruckus.brouhaha.com> Daniel Fetchinson writes: > I have a plain text file which I would like to protect in a very > simple minded, yet for my purposes sufficient, way. For encrypting strings, use this module: http://nightsong.com/phr/crypto/p3.py Obviously this is limited to strings that fit in memory, which might be a problem with large files. Some day I might get around to adding a streaming interface to it. The "file" command will not recognize the ciphertext as encrypted data. It will just say "data". If you want to be more serious, use pgp or gpg with the -c option (password-based encryption). I think "file" does recognize the pgp file format as encrypted data (RFC 2440). From no.email at nospam.invalid Sat Jan 9 15:04:40 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 09 Jan 2010 12:04:40 -0800 Subject: lightweight encryption of text file References: Message-ID: <7xmy0noxif.fsf@ruckus.brouhaha.com> Daniel Fetchinson writes: >> http://www.nightsong.com/phr/crypto/p3.py > > Thanks a lot, currently I'm having trouble using this code on python > 2.6 but probably some small tweaking will fix it. Yikes, this is the first I've heard of such a problem. I will look into it. Thanks. (Also thanks to Robert for the recommendation). From no.email at nospam.invalid Sat Jan 9 15:18:33 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 09 Jan 2010 12:18:33 -0800 Subject: lightweight encryption of text file References: Message-ID: <7xiqbbowva.fsf@ruckus.brouhaha.com> Robert Kern writes: > Are you on a 64-bit platform? Unfortunately, array's integer typecodes > are platform-specific, but p3.py requires a 32-bit integer ... Oh yes, thanks, I never did get around to dealing with 64 bit platforms. I also notice that some of the unit test functions use print statements, which won't compile under Python 3.0. I will try to release a version in the next few days with fixes for both of these issues. From steve at holdenweb.com Sat Jan 9 15:31:18 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 09 Jan 2010 15:31:18 -0500 Subject: Link to module Stack In-Reply-To: <03589968$0$1336$c3e8da3@news.astraweb.com> References: <0358384b$0$1336$c3e8da3@news.astraweb.com> <03589968$0$1336$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > On Sat, 09 Jan 2010 05:56:36 -0500, Dave Angel wrote: > >>> "InnerInterpreterError" is the most inappropriate exception name I've >>> ever seen. It has nothing to do with the interpreter, it's a stack >>> error. >>> >>> >> It has everything to do with the (Forth) interpreter. Exceptions can >> readily be named according to their application -- it's not always about >> Python. Anyway, Forth has an inner-interpreter and an >> outer-interpreter, and the name will make sense to a Forth programmer. > > Pardon me, but I *am* a Forth programmer. Or was, it's been many years, > and I'm rusty. I guess this is a difference of terminology: what you're > calling an inner interpreter and an outer interpreter I know of as the > Forth engine and the (text) interpreter. Gforth refers to them as such, > so did Leo Brodie's Forth books, and the (ancient) Macintosh Forth > compiler "Mach 2". > > But in any case... a stack is an general-purpose data structure, and the > error message shouldn't be coupled so tightly to one use. That would be > like this (made-up) example: > >>>> 1/0 > Traceback (most recent call last): > File "", line 1, in > GraphicsApplicationError: too few pixels to calculate average > > > Ridiculous, yes? > > > Yes, Forth uses a stack (technically two, a parameter stack and a return > stack, and some implementations include a third, floating point, stack). > Virtually all languages use stacks in their implementation, and Python > byte-code is also stack-based. > > >>>> result = self.__heap[-1] >>>> del self.__heap[-1] >>>> >>>> >>> That is better written as result = self.__heap.pop(). >>> >>> >>> >> or even better, without the extra local var: >> >> def pop (self): >> if len(self.__heap) == 0: Since self.__heap is a list, the canonical Python for the above test would, of course, be the much simpler if not self.__heap regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From casevh at gmail.com Sat Jan 9 16:27:07 2010 From: casevh at gmail.com (casevh) Date: Sat, 9 Jan 2010 13:27:07 -0800 (PST) Subject: interactive terminal in Ubuntu Linux : libreadline5-dev works only in Python 2.6 not 3.1 References: Message-ID: <23c7de94-44ba-4567-b31f-8a5bb656e9e9@m3g2000yqf.googlegroups.com> On Jan 9, 10:06?am, Dave WB3DWE wrote: > On Jan 6 I inquired how to "fix" the 3.1.1 interactive terminal > in Ubuntu Linux. ? Left arrow yields ^[[D , etc. > > casevh helped by suggesting "libreadline5-dev" be installed. > Did so with Synaptic Package Manager. > The behavior of the Python 3.3.1 terminal is unchanged but > the 2.6.2 terminal is corrected. > > Is there any way to fix the 3.1.1 terminal command line ? > > Thanks, ? ? Dave WB3DWE ? ? ?pdlem... at earthlink.net Did you recompile Python 3.1.1 after installing libreadline5-dev? (From the Python 3.1.1 directory. Your options to configure may vary.) make distclean ./configure --prefix=/usr/local --with-computed-gotos --with-wide- unicode make make altinstall casevh From nawijn at gmail.com Sat Jan 9 16:30:48 2010 From: nawijn at gmail.com (Marco Nawijn) Date: Sat, 9 Jan 2010 13:30:48 -0800 (PST) Subject: Microsoft Office Word and Python (Win XP) References: Message-ID: On Jan 9, 8:18?pm, "3lvss0... at gmail.com" <3lvss0... at gmail.com> wrote: > Marco Nawijn: I have had installed pywin32 already. The three lines > that you mentoined don't do this I checked at my own computer and it works fine. > also what did you mean with "doc = > app.Documents.Open("c:\\example.doc")". Which document should I open > with this line? This was just meant as an example on how to open a Word document from within python. This would be the basis for copying/appending from your source document to your target document (e.g. HD10.doc). > It shouldn't be opened anything. I was asking about > the script as automated process. If you know how could i do this? Well it is a python script. So you can run it as an automated process. You should just set app.Visible=False so the Word user interface component is not shown. > > Terry Reedy: I have never mentoined copying files but moving the whole > text from, always the same (same name, same path), .doc file with. > However copying (=moving) text to correct .doc file would be good yes. > I know command prompt and its copy function but this way it wouldn't > work because I would have to define the target file - the file INTO > which I want to move the text. But I will never know the file name > (target file). The only thing I know is: > - the file is one of .doc files that start with "HD" > - whole name of those .doc file is always HDX.doc where X is a number > and I need to move the text into the file with maximum X (the most > high number) > - all the HD files will be always in the same path (same folder) but I > would like to use the path inside the code (it might be obvious that I > have to) because on PC and laptop, I have two different usernames and > since HD files are located inside Documents And Settings, I have to > use two copies of the script - one for PC, one for laptop. > > Dennis Lee Bieber: Im not familiar with python, also Im not > programmer. Thats why Im not able to do so when people tell me "do > this then use XYZ function which will give you ZYX from what you can > do that and you will get result". Im still willing to learn but there > are thousands of python tutorials and the one for exsactly this topic > probably doesn't exsist. The .doc extension is required, so I cannot > use .txt because I need the HD files in .doc. From jackdied at gmail.com Sat Jan 9 16:34:28 2010 From: jackdied at gmail.com (Jack Diederich) Date: Sat, 9 Jan 2010 16:34:28 -0500 Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2 In-Reply-To: References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> Message-ID: On Sat, Jan 9, 2010 at 2:53 PM, Jan Kaliszewski wrote: > Hello, > > I have a question: are class decorator planned to be backported from 3.x? > Eh? Class decorators have been in the 2.x series since 2.6. If you want to know more about class decorators check out this talk from PyCon 2009 http://pycon.blip.tv/file/1949345 by a very handsome and well beloved man, and come to pycon 2010 where there will be a couple more talks on the subject by slightly more homely but none the less competent speakers. -Jack From zuo at chopin.edu.pl Sat Jan 9 16:44:44 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sat, 09 Jan 2010 22:44:44 +0100 Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2 In-Reply-To: References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> Message-ID: 09-01-2010 o 22:34:28 Jack Diederich wrote: > On Sat, Jan 9, 2010 at 2:53 PM, Jan Kaliszewski >> I have a question: are class decorator planned to be backported from >> 3.x? >> > Eh? Class decorators have been in the 2.x series since 2.6. Oops, I overlooked the fact :) Thank you! *j -- Jan Kaliszewski (zuo) From niels.ellegaard at gmail.com Sat Jan 9 17:33:28 2010 From: niels.ellegaard at gmail.com (Niels L. Ellegaard) Date: Sat, 09 Jan 2010 23:33:28 +0100 Subject: Append to an Excel file References: <313cb2b5-4100-4d4d-bb38-dbfe3ca71e17@j5g2000yqm.googlegroups.com> <74a95a2f-90b7-48e6-9b69-c7250b979978@l30g2000yqb.googlegroups.com> Message-ID: <87tyuu7vt3.fsf@langager.dbnet.dk> pp writes: > On Jan 9, 1:47?am, Jason Scheirer wrote: >> On Jan 9, 12:30?am, pp wrote: >> >> > Hi All, >> >> > How do I add a line to an existing file. This should append to the >> > existing data in the excel file, which was saved previously. >> >> > Thanks, >> > PP >> >> http://pypi.python.org/pypi/xlwt > > Hi Jason and all, > > Thanks > > I have seen this.. my question is there a way to append to a excel > file which has been closed. Any specific modes which can be added to > the sheet so that it adds a line to the data which was return in some > earlier running of the program. I may be wrong, but I think that you have to do the following 1) Use xlrd to read the file. This creates an xlrd.Book 2) Use xlutils to transform the xlrd.Book into a xlwt.WorkBook 3) Edit the xlwt.WorkBook 4) Save the xlwt.WorkBook https://secure.simplistix.co.uk/svn/xlutils/trunk/xlutils/docs/copy.txt Niels From MLMDev at Comcast.net Sat Jan 9 17:43:34 2010 From: MLMDev at Comcast.net (Mitchell L Model) Date: Sat, 9 Jan 2010 17:43:34 -0500 Subject: Python-list Digest, Vol 76, Issue 97 In-Reply-To: References: Message-ID: <18FFC0A5-87C2-4E2F-B12D-BF269EF21477@Comcast.net> On Jan 8, 2010, at 7:35:39 PM EST, Terry Reedy wrote: > > On 1/8/2010 12:02 PM, Mitchell L Model wrote: > > >> On further reflection, I will add that >> what appears to be happening is that during import both the global >> and >> local dictionaries are set to a copy of the globals() from the >> importing >> scope and that copy becomes the value of the module's __dict__ once >> import has completed successfully. > > I have no idea why you think that. The module dict starts empty > except for __name__, __file__, and perhaps a couple of other > 'hidden' items. It is not a copy and has nothing to do with > importing scopes. Why I think -- or, rather, thought -- that was because of some defective experiments I ran. It was purely a delusion. Thank you for correcting it. > > > and that copy becomes the value of the module's __dict__ once > > import has completed successfully. > > That new dict becomes .... . > >> Because exec leaves locals() and globals() distinct, > > Not necessarily. > > In 3.x, at least, > exec(s) > executes s in the current scope. If this is top level, where locals > is globals, then same should be true within exec. Yes. To simplify some of my ramblings and incorporate the points you and others have made, and to once again acknowledge Python's elegance, an important observation which I bet even a lot of serious Python programs don't realize (or at least not consciously) is that: globals() is locals() in the following contexts: the interpreter top level the top level of a module (though as you point out, starts out as a very bare dictionary during import) a string being exec'd when the call to exec includes no dictionary argument(s) one dictionary argument the same dictionary as both the second and third arguments The identity does not hold for: a string being exec'd when a different dictionary is provided as the second and third arguments to exec inside anything that creates a scope: a function definition, class definition, etc. Did I get all that right? Are there any other contexts that should be included in these? > > d = {} > exec(s, d) > > In 3.x, at least, d will also be used as locals. Yes, talking about 3.x. > > exec(s, d, d) > > Again, globals and locals are not distinct. > > It would seem that in 3.x, the only way for exec to have distinct > globals and locals is to call exec(s) where they are distinct or to > pass distince globals and locals. Apparently so. To clarify "where they are distinct", that would mean from a context in which they were already distinct, which is not the case if exec is called from the top level, but is the case if called from within, say, a function, as my code does. > > Some of the issues of this thread are discussed in Language > Reference 4.1, Naming and Binding. I suppose it could be clearer > that it is, but the addition of nonlocal scope complicated things. I pretty much have that section memorized and reread it at least monthly. It's part of what I meant by starting my original comments by saying that I thought I understood all of this. Thank you (and others) for helping clarify exactly what's going on. As with so many things in Python, it is not always easy to keep one's preconceptions, delusions, and experiences with other languages out of the way of its simplicity, even if one is a very experienced and knowledgeable Python programmer. --- Mitchell -------------- next part -------------- An HTML attachment was scrubbed... URL: From pdlemper at earthlink.net Sat Jan 9 18:10:20 2010 From: pdlemper at earthlink.net (pdlemper at earthlink.net) Date: Sat, 09 Jan 2010 17:10:20 -0600 Subject: interactive terminal in Ubuntu Linux : libreadline5-dev works only in Python 2.6 not 3.1 References: <23c7de94-44ba-4567-b31f-8a5bb656e9e9@m3g2000yqf.googlegroups.com> Message-ID: On Sat, 9 Jan 2010 13:27:07 -0800 (PST), casevh wrote: > >Did you recompile Python 3.1.1 after installing libreadline5-dev? > >(From the Python 3.1.1 directory. Your options to configure may vary.) > >make distclean >./configure --prefix=/usr/local --with-computed-gotos --with-wide- >unicode >make >make altinstall > >casevh Thanks so much for your help . . . but I'm going backwards, probably due to my inadequate knowledge of Linux. Ran the above vebatim, except had to do sudo make install . Python recompiled and the system appears to be in same path/dir as before : /home/dave/python31/Python-3.1.1 Called up interactive prompt >>> with $ python3 , as before. Problems : 1. prompt keys remain fouled up as before. 2. will not import random module, either from prompt or from any of my prewritten modules. Get ImportError /usr/local/lib/Python3.1/lib-dynload/_collections.so : undefined symbol : PyUnicode UCS4_FromString Some other modules will import : math , sys, os 3. Some of my pre-existing modules will not run : have not checked them all, but apparently those with random. 4. Attempted to read my modules with gedit. Pulls them up but refuses to save : could not save file /usr/local/lib/python3.1/dlmodules/Ackermann.py You do not have permission necessary to save file. This is same path my modules were in before the recompile and the dir I made for them. Had no trouble saving them previously. If no fix available, I'll reinstall from the Python-3.1.1 that I extracted from the tarball , and go back to XP for a while : ( Dave WB3DWE pdlemper at earthlink.net From davea at ieee.org Sat Jan 9 19:17:08 2010 From: davea at ieee.org (Dave Angel) Date: Sat, 09 Jan 2010 19:17:08 -0500 Subject: Link to module Stack In-Reply-To: <03589968$0$1336$c3e8da3@news.astraweb.com> References: <0358384b$0$1336$c3e8da3@news.astraweb.com> <03589968$0$1336$c3e8da3@news.astraweb.com> Message-ID: <4B491C84.2000809@ieee.org> Steven D'Aprano wrote: > On Sat, 09 Jan 2010 05:56:36 -0500, Dave Angel wrote: > > >>> "InnerInterpreterError" is the most inappropriate exception name I've >>> ever seen. It has nothing to do with the interpreter, it's a stack >>> error. >>> >>> >>> >> It has everything to do with the (Forth) interpreter. Exceptions can >> readily be named according to their application -- it's not always about >> Python. Anyway, Forth has an inner-interpreter and an >> outer-interpreter, and the name will make sense to a Forth programmer. >> > > Pardon me, but I *am* a Forth programmer. Or was, it's been many years, > and I'm rusty. I guess this is a difference of terminology: what you're > calling an inner interpreter and an outer interpreter I know of as the > Forth engine and the (text) interpreter. Gforth refers to them as such, > so did Leo Brodie's Forth books, and the (ancient) Macintosh Forth > compiler "Mach 2". > > I'm pretty sure FIG-Forth called them an inner interpreter and outer interpreter, but I don't remember other sources. FIG-Forth was my first Forth system, gotten on an 8" diskette. The inner interpreter was LOADSW, JMP AX, I believe, as it was an indirected threaded interpreter implementation. > > >> or even better, without the extra local var: >> >> def pop (self): >> if len(self.__heap) == 0: >> raise InnerInterpreterError, "stack underflow" >> return self.__heap.pop(1) >> > > pop(1)? I don't think so. > > > That was a typo; I meant pop(). And of course others have improved on my remarks anyway. DaveA From 3lvss0809 at gmail.com Sat Jan 9 19:23:37 2010 From: 3lvss0809 at gmail.com (3lvss0809 at gmail.com) Date: Sat, 9 Jan 2010 16:23:37 -0800 (PST) Subject: Microsoft Office Word and Python (Win XP) References: Message-ID: <4dd1834d-b03e-4f75-a6f6-117d6f522ed9@a15g2000yqm.googlegroups.com> Marco did you also make the HD files to it worked for you? Because I cannot even imagine how only three lines would do everything - find correct folder (path) of the files, find maximum number, move the entire text,... In this 3 lines is not stated that we are talking about the files that start with "HD" so I wonder how it worked for you. Nothing was moved on my PC. I have done some research about the line "app = Dispatch("Word.Application")" and saw this: http://www.programmingforums.org/post105986.html The script described here is different comparing to what I want to do but I might be able to use some ideas - of course, since Im not a programmer, I need to study the functions before. On the link the user is trying to replace parts (predefinited or not - i don't know) of the .doc files - this is not moving/copying entire text but the topic is still about writing something into .doc through python. Unfortunatelly for me, his script doesn't need to search for correct .doc file. So I kept searching and came to this: http://www.daniweb.com/forums/thread129924.html This task is very close to what I want but still different. If we try to compare; "I am a text" (his task) = already exsisting (if any) text in my HDX file where X is the highest number (my task) "Hello" (his task) = whole text inside .doc with always the same name on the same location (my task) then thats it. I believe just those two differences exsist. From casevh at gmail.com Sat Jan 9 19:48:52 2010 From: casevh at gmail.com (casevh) Date: Sat, 9 Jan 2010 16:48:52 -0800 (PST) Subject: interactive terminal in Ubuntu Linux : libreadline5-dev works only in Python 2.6 not 3.1 References: <23c7de94-44ba-4567-b31f-8a5bb656e9e9@m3g2000yqf.googlegroups.com> Message-ID: <15a78d94-34d5-4bef-8344-6dc8843de4e5@k19g2000yqc.googlegroups.com> On Jan 9, 3:10?pm, pdlem... at earthlink.net wrote: > On Sat, 9 Jan 2010 13:27:07 -0800 (PST), casevh > wrote: > >Did you recompile Python 3.1.1 after installing libreadline5-dev? > > >(From the Python 3.1.1 directory. Your options to configure may vary.) > > >make distclean > >./configure --prefix=/usr/local --with-computed-gotos --with-wide- > >unicode > >make > >make altinstall > > >casevh > > Thanks so much for your help . . . but I'm going backwards, probably > due to my inadequate knowledge of Linux. > > Ran the above vebatim, except had to do ? ? ?sudo make install ? ?. > Python recompiled and the system appears to be in same path/dir as > before : ?/home/dave/python31/Python-3.1.1 > Called up interactive prompt ?>>> ?with ?$ python3 , as before. > It looks like you now have two copies of Python 3.1.1 installed. That's probably a side-effect of my instructions. I'll got through each instruction individually. It looks like your Python source code is in "/home/dave/python31/ Python-3.1.1". The commnad "make distclean" should remove the results of the prior configuration so to won't need to extract the source code again. The command "./configure" accepts several options that control how the source code is configured. To see all the available options, use the command "./configure --help". The first option - "--prefix=/usr/local" - identifies the location where Python 3.1.1 will be installed. The "/ usr/local" directory is a common location for user-compiled programs. If no location is specified, many applications assume "/usr/local". The option - "--with-computed-gotos" - is just a compiler options that produces a slightly faster interpreter. The final option - "--with- wide-unicode" - identifies the proper Unicode format to use. (I looks like this option may have been split via line wrapping on my earlier email. I think it is causing the UCS4 error.) The command "make" compiles Python 3.1.1. The command "sudo make altinstall" installs Python 3.1.1 under the "/ usr/local" directory. The option "altinstall" installs Python 3.1.1 and leaves its name as "python3.1". It should be located in "/usr/ local/bin/python3.1". Your own programs (Ackermann.py) should be kept somewhere in your home directory, for example "/home/dave/code". Typing "python3.1" will look in the current directory first, and then search the path where it should find "/usr/local/bin/python3.1". What will you need to do fix this? 1) Try the commands again. Make sure all the "./configure" options are on one line. Make sure to do "sudo make altinstall". (Don't use "sudo make install"; it will give your version of Python the name "python" and that can cause confusion on your system.) 2) Move your applications to another directory. 3) Try running "python3.1" while you are in that directory. If this doesn't work, report back on the error messages you receive. Don't give up; we can fix this. casevh > Problems : > ? ? 1. prompt keys remain fouled up as before. > ? ? 2. will not import ?random ?module, either from prompt or > ? ? ? ? ? ? from any of my prewritten modules. ?Get ImportError > ? ? ? ? ? ?/usr/local/lib/Python3.1/lib-dynload/_collections.so : > ? ? ? ? ? ? undefined symbol : PyUnicode UCS4_FromString > ? ? ? ? ? ? ? ? Some other modules will import : math , sys, os > ? ? 3. Some of my pre-existing modules will not run : have > ? ? ? ? ? not checked them all, but apparently those with random. > ? ? 4. Attempted to read my modules with gedit. ?Pulls them > ? ? ? ? ? up but refuses to save : ? ? ? ? ?could not save file > ? ? ? ? ? /usr/local/lib/python3.1/dlmodules/Ackermann.py > ? ? ? ? ? ? ? ?You do not have permission necessary to save file. > ? ? ? ? ? This is same path my modules were in before the > ? ? ? ? ? recompile and the dir I made for them. Had no trouble > ? ? ? ? ? saving them previously. > > If no fix available, I'll reinstall from the Python-3.1.1 that > I extracted from the tarball , and go back to XP for a while ?: ( > > Dave WB3DWE ? ? ? pdlem... at earthlink.net From peloko45 at gmail.com Sat Jan 9 20:01:54 2010 From: peloko45 at gmail.com (Joan Miller) Date: Sat, 9 Jan 2010 17:01:54 -0800 (PST) Subject: Prepend to logging message Message-ID: <5aa06ab5-61fd-46a8-9cac-7f3a2a5d382b@d20g2000yqh.googlegroups.com> How to prepend anything to a logging message? Is possible to do it from the dictionary object (ExtraLog) or is there is that override process() [1]? ------------------ class ExtraLog(object): def __getitem__(self, name): if name == 'foo': result = 'testing' return result def __iter__(self): keys = ['foo',] keys.extend(self.__dict__.keys()) return iter(keys) logger = logging.LoggerAdapter(logging.getLogger('foo'), ExtraLog()) ------------------ [1] http://docs.python.org/library/logging.html#logging.LoggerAdapter From monaghand.david at gmail.com Sat Jan 9 20:09:46 2010 From: monaghand.david at gmail.com (David Monaghan) Date: Sun, 10 Jan 2010 01:09:46 +0000 Subject: Microsoft Office Word and Python (Win XP) References: Message-ID: On Sat, 9 Jan 2010 11:18:12 -0800 (PST), "3lvss0809 at gmail.com" <3lvss0809 at gmail.com> wrote: >Dennis Lee Bieber: Im not familiar with python, also Im not >programmer. What you want to do isn't complicated, but it isn't simple either, unless you're familiar with VBA/VBS. I approach these problems by first getting the VBA code by recording a macro within Word. I then convert it to VB Script, which is a learning process in itself. Converting that script to Python com is another learning process and then putting the whole thing together with file finding and saving is another job. When you've done that, you won't feel able to say you're not a programmer - and you should feel familiar with Python, too. DaveM From ishwor.gurung at gmail.com Sat Jan 9 22:27:10 2010 From: ishwor.gurung at gmail.com (Ishwor Gurung) Date: Sun, 10 Jan 2010 14:27:10 +1100 Subject: Prepend to logging message In-Reply-To: <5aa06ab5-61fd-46a8-9cac-7f3a2a5d382b@d20g2000yqh.googlegroups.com> References: <5aa06ab5-61fd-46a8-9cac-7f3a2a5d382b@d20g2000yqh.googlegroups.com> Message-ID: <34534aed1001091927i8e9fe6dsd38cc46657cdfe2c@mail.gmail.com> Joan, 2010/1/10 Joan Miller : > How to prepend anything to a logging message? Is possible to do it > from the dictionary object (ExtraLog) or is there is that override > process() [1]? > > ------------------ > class ExtraLog(object): > > ? ?def __getitem__(self, name): > ? ? ? ?if name == 'foo': > ? ? ? ? ? ?result = 'testing' > ? ? ? ?return result > > ? ?def __iter__(self): > ? ? ? ?keys = ['foo',] > ? ? ? ?keys.extend(self.__dict__.keys()) > ? ? ? ?return iter(keys) > > logger = logging.LoggerAdapter(logging.getLogger('foo'), ExtraLog()) > ------------------ Yep. Just subclass LoggerAdapter and override process(..) Read this: http://docs.python.org/library/logging.html#adding-contextual-information-to-your-logging-output -- Regards Ishwor Gurung Key id:0xa98db35e Key fingerprint:FBEF 0D69 6DE1 C72B A5A8 35FE 5A9B F3BB 4E5E 17B5 From lie.1296 at gmail.com Sun Jan 10 00:24:01 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Sun, 10 Jan 2010 16:24:01 +1100 Subject: PIL how to display multiple images side by side In-Reply-To: <6d9fef57-7fdc-46ec-9b53-7aa824e785e9@s31g2000yqs.googlegroups.com> References: <6d9fef57-7fdc-46ec-9b53-7aa824e785e9@s31g2000yqs.googlegroups.com> Message-ID: <4b496473$1@dnews.tpgi.com.au> On 1/9/2010 8:43 AM, suresh.amritapuri wrote: > Hi, > > In PIL, how to display multiple images in say m rows and n colums when > I have m*n images. > > suresh Tkinter has PhotoImage widget and PIL has support for this widget: http://www.pythonware.com/library/pil/handbook/imagetk.htm From alfps at start.no Sun Jan 10 00:51:58 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 10 Jan 2010 06:51:58 +0100 Subject: PIL how to display multiple images side by side In-Reply-To: <4b496473$1@dnews.tpgi.com.au> References: <6d9fef57-7fdc-46ec-9b53-7aa824e785e9@s31g2000yqs.googlegroups.com> <4b496473$1@dnews.tpgi.com.au> Message-ID: * Lie Ryan: > On 1/9/2010 8:43 AM, suresh.amritapuri wrote: >> Hi, >> >> In PIL, how to display multiple images in say m rows and n colums when >> I have m*n images. >> >> suresh > > Tkinter has PhotoImage widget and PIL has support for this widget: > http://www.pythonware.com/library/pil/handbook/imagetk.htm Maybe I've misunderstood something (in that case glad to learn!), but I believe PhotoImage is not a widget, and that a PhotoImage has to be presented in e.g. a Label widget or some other widget that's able to display images. Cheers, - Alf From nobody at nowhere.com Sun Jan 10 02:51:02 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 10 Jan 2010 07:51:02 +0000 Subject: subprocess.Popen does not close pipe in an error case References: <07641018-9581-44af-bb20-8f9b96a44c4f@p8g2000yqb.googlegroups.com> Message-ID: On Wed, 06 Jan 2010 11:39:37 -0800, Steven K. Wong wrote: > Suppose now all the prog1.poll() calls/loop are replaced by a single > prog1.wait(). Without the explicit prog1.stdout.close(), prog1.wait() > will not return, so the calling process still hangs. Because calling > prog1.wait() means that the calling process will naturally never read > prog1.stdout, I would argue that prog1.wait() should close the pipe > before actually waiting for prog1 to exit. Makes sense? prog1.stdout might be being read by a different thread. From nobody at nowhere.com Sun Jan 10 02:54:02 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 10 Jan 2010 07:54:02 +0000 Subject: subprocess.Popen does not close pipe in an error case References: <07641018-9581-44af-bb20-8f9b96a44c4f@p8g2000yqb.googlegroups.com> <40a2adab-941a-4844-9279-10667bf2ba5e@v25g2000yqk.googlegroups.com> Message-ID: On Wed, 06 Jan 2010 19:05:40 -0800, Steven K. Wong wrote: > Well, the example code at > http://www.python.org/ ... /subprocess.html#replacing-shell-pipeline > has the same issue: > Perhaps the doc can be improved to remind folks to close p1.stdout if > the calling process doesn't need it, unless wait() is changed to close > it and p1.wait() is called. > > Am I making any sense here? The docs should include the p1.stdout.close(). It isn't needed in the typical case, where p2 runs until EOF on stdin, but (as you have noticed) it matters if p2 terminates prematurely. From nobody at nowhere.com Sun Jan 10 03:16:28 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 10 Jan 2010 08:16:28 +0000 Subject: Ask how to use HTMLParser References: <87vdfdtg3z.fsf@ymail.invalid> Message-ID: On Fri, 08 Jan 2010 11:44:48 +0800, Water Lin wrote: > I am a new guy to use Python, but I want to parse a html page now. I > tried to use HTMLParse. Here is my sample code: > ---------------------- > from HTMLParser import HTMLParser Note that HTMLParser only tokenises HTML; it doesn't actually *parse* it. You just get a stream of tag, text, entity, text, tag, ..., not a parse tree. In particular, if an element has its start and/or end tags omitted, you won't get any notification about the start and/or end of the element; you have to figure that out yourself from the fact that you're getting a tag which wouldn't be allowed outside or inside the element. E.g. if the document has omitted

tags, if you get a

tag when you are (or *thought* that you were) already within a paragraph, you can infer the omitted

tag. If you want an actual parser, look at BeautifulSoup. This also does a good job of handling invalid HTML (which seems to be far more common than genuine HTML). From nobody at nowhere.com Sun Jan 10 03:33:42 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 10 Jan 2010 08:33:42 +0000 Subject: lightweight encryption of text file References: Message-ID: On Fri, 08 Jan 2010 20:14:51 +0100, Daniel Fetchinson wrote: > I have a plain text file which I would like to protect in a very > simple minded, yet for my purposes sufficient, way. I'd like to > encrypt/convert it into a binary file in such a way that possession of > a password allows anyone to convert it back into the original text > file while not possessing the password one would only see the > following with the standard linux utility 'file': > What would be the simplest way to achieve this using preferably stock > python without 3rd party modules? If a not too complex 3rd party > module made it really simple that would be acceptable too. RC4 (aka ArcFour) is quite trivial to implement, and better than inventing your own cipher or using a Vignere: import itertools class arcfour: def __init__(self, key): self.s = range(256) self.schedule(map(ord, key)) self.pad = self.prng() def swap(self, i, j): self.s[i], self.s[j] = self.s[j], self.s[i] def schedule(self, key): j = 0 for i, c in zip(xrange(256), itertools.cycle(key)): j = (j + self.s[i] + c) % 256 self.swap(i, j) def prng(self): i = j = 0 while True: i = (i + 1) % 256 j = (j + self.s[i]) % 256 self.swap(i, j) yield self.s[(self.s[i] + self.s[j]) % 256] def crypt(self, string): chars = (chr(c ^ r) for c, r in zip(map(ord, string), self.pad)) return ''.join(chars) I suggest that you don't use the password itself as the key, unless you're sure that a low-entropy string won't be used. Instead, create an SHA hash (see the sha and hashlib modules) of the password and use that. From fetchinson at googlemail.com Sun Jan 10 03:59:31 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Sun, 10 Jan 2010 09:59:31 +0100 Subject: lightweight encryption of text file In-Reply-To: References: Message-ID: >> I have a plain text file which I would like to protect in a very >> simple minded, yet for my purposes sufficient, way. I'd like to >> encrypt/convert it into a binary file in such a way that possession of >> a password allows anyone to convert it back into the original text >> file while not possessing the password one would only see the >> following with the standard linux utility 'file': > >> What would be the simplest way to achieve this using preferably stock >> python without 3rd party modules? If a not too complex 3rd party >> module made it really simple that would be acceptable too. > > RC4 (aka ArcFour) is quite trivial to implement, and better than inventing > your own cipher or using a Vignere: > > import itertools > > class arcfour: > def __init__(self, key): > self.s = range(256) > self.schedule(map(ord, key)) > self.pad = self.prng() > > def swap(self, i, j): > self.s[i], self.s[j] = self.s[j], self.s[i] > > def schedule(self, key): > j = 0 > for i, c in zip(xrange(256), itertools.cycle(key)): > j = (j + self.s[i] + c) % 256 > self.swap(i, j) > > def prng(self): > i = j = 0 > while True: > i = (i + 1) % 256 > j = (j + self.s[i]) % 256 > self.swap(i, j) > yield self.s[(self.s[i] + self.s[j]) % 256] > > def crypt(self, string): > chars = (chr(c ^ r) for c, r in zip(map(ord, string), self.pad)) > return ''.join(chars) > > I suggest that you don't use the password itself as the key, unless you're > sure that a low-entropy string won't be used. Instead, create an SHA hash > (see the sha and hashlib modules) of the password and use that. Thanks, this looks very simple too, but where is the decryption code? Wikipedia seems to suggest that encryption and decryption are both the same but running crypt on the output of crypt doesn't give back the original string. So probably I'm misunderstanding something. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From peter at www.pjb.com.au Sun Jan 10 04:15:54 2010 From: peter at www.pjb.com.au (Peter Billam) Date: 10 Jan 2010 09:15:54 GMT Subject: Computer Music Toolkit (CMT) ? Message-ID: Greetings. Is there a way to get at the Computer Music Toolkit (CMT) http://www.ladspa.org/cmt/ functionality from Python (Python3 in my case) ? Googling is confusing because of www.cmt.com and cmt-graph and openscientist.lal.in2p3.fr/v9/cmt.html and pyAMISecure_and_cmt and so on... Regards, Peter -- Peter Billam www.pjb.com.au www.pjb.com.au/comp/contact.html From chambon.pascal at gmail.com Sun Jan 10 04:42:23 2010 From: chambon.pascal at gmail.com (Pakal) Date: Sun, 10 Jan 2010 01:42:23 -0800 (PST) Subject: Direct use of bytearray buffers with ctypes ? References: <9694d6f5-5189-4371-a933-f7c5da5b2d30@e27g2000yqd.googlegroups.com> Message-ID: <18fdc1e3-416b-445e-af44-80851c194e0f@s31g2000yqs.googlegroups.com> Oki, it seems I've found. To directly use a bytearray buffer from ctypes, you must first create a compatible ctypes type (I.E, a char array of same size), and only then instanciate this new type with newtype.from_buffer (bytearray_object). The little danger is : you must NOT change the size of bytearray as long as the ctypes array targets its inner buffer, else of cousre memory reallocations occurs, and your ctypes array points to invalid memory (awaiting a pretty segfault). But modifying chars or characters ranges from the buffer, as well via the bytearray object than via the ctypes array, is fine. I don't think problems can occur if both sides are accessed simultaneously, even though ctypes operations may release the GIL while they execute... the only race condition is the access of a memory segment, isn't it ? No crash shall occur with this... IDLE 2.6.4 >>> import ctypes >>> a = bytearray(3) >>> mytype = ctypes.c_char * 3 >>> h = mytype.from_buffer(a) >>> h <__main__.c_char_Array_3 object at 0x02B06350> >>> h[:] = "abc" >>> a bytearray(b'abc') >>> h[0] = "k" >>> a bytearray(b'kbc') From stefan_ml at behnel.de Sun Jan 10 04:47:48 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 10 Jan 2010 10:47:48 +0100 Subject: Computer Music Toolkit (CMT) ? In-Reply-To: References: Message-ID: <4b49a244$0$6726$9b4e6d93@newsspool2.arcor-online.net> Peter Billam, 10.01.2010 10:15: > Greetings. Is there a way to get at the Computer Music Toolkit (CMT) > http://www.ladspa.org/cmt/ > functionality from Python (Python3 in my case) ? > > Googling is confusing because of www.cmt.com and cmt-graph and > openscientist.lal.in2p3.fr/v9/cmt.html and pyAMISecure_and_cmt > and so on... Just a note on the last paragraph: searching for acronyms is usually simplified by adding the spelled-out name to the search keywords. It's highly unlikely that someone would write a Python tool for a library or application without putting the complete name of the thing it's made for on the project homepage. Stefan From peloko45 at gmail.com Sun Jan 10 04:47:52 2010 From: peloko45 at gmail.com (Joan Miller) Date: Sun, 10 Jan 2010 01:47:52 -0800 (PST) Subject: Prepend to logging message References: <5aa06ab5-61fd-46a8-9cac-7f3a2a5d382b@d20g2000yqh.googlegroups.com> Message-ID: <40d0e6b8-15be-4e3b-aca2-26e07d7fc8cf@v25g2000yqk.googlegroups.com> On 10 ene, 03:27, Ishwor Gurung wrote: > Joan, > > 2010/1/10 Joan Miller : > > > > > How to prepend anything to a logging message? Is possible to do it > > from the dictionary object (ExtraLog) or is there is that override > > process() [1]? > > > ------------------ > > class ExtraLog(object): > > > ? ?def __getitem__(self, name): > > ? ? ? ?if name == 'foo': > > ? ? ? ? ? ?result = 'testing' > > ? ? ? ?return result > > > ? ?def __iter__(self): > > ? ? ? ?keys = ['foo',] > > ? ? ? ?keys.extend(self.__dict__.keys()) > > ? ? ? ?return iter(keys) > > > logger = logging.LoggerAdapter(logging.getLogger('foo'), ExtraLog()) > > ------------------ > > Yep. Just subclass LoggerAdapter and override process(..) > Read this:http://docs.python.org/library/logging.html#adding-contextual-informa... > -- > Regards > Ishwor Gurung > Key id:0xa98db35e > Key fingerprint:FBEF 0D69 6DE1 C72B A5A8 ?35FE 5A9B F3BB 4E5E 17B5 Any example to override process() ? From parul.pandey85 at gmail.com Sun Jan 10 04:51:41 2010 From: parul.pandey85 at gmail.com (pp) Date: Sun, 10 Jan 2010 01:51:41 -0800 (PST) Subject: Porblem with xlutils/xlrd/xlwt References: <8e9245ec-05cb-48ed-ae32-f9fb0c46509f@j5g2000yqm.googlegroups.com> <972fdcfc-73a6-4e87-9752-3bc10da0dcad@s3g2000yqs.googlegroups.com> <43a1e323-8ccd-48b7-a6a4-e540aca2b1cd@35g2000yqa.googlegroups.com> <5ac38519-9ad4-4a89-9b3f-ccd56fb18721@m3g2000yqf.googlegroups.com> Message-ID: <610f640f-b03b-4aaa-93fe-a3fdf1381801@j4g2000yqe.googlegroups.com> On Jan 9, 8:23?am, John Machin wrote: > On Jan 9, 9:56?pm, pp wrote: > > > On Jan 9, 3:52?am, Jon Clements wrote: > > > > On Jan 9, 10:44?am, pp wrote: > > > > > On Jan 9, 3:42?am, Jon Clements wrote: > > > > > > On Jan 9, 10:24?am, pp wrote: > > > > yeah all my versions are latest fromhttp://www.python-excel.org. > > > > just checked!! > > How did you check? > > > > > what could be the problem? > > > > Does rb = xlrd.open_workbook('somesheet.xls', on_demand=True) work by > > > itself? > > > Yes it does. The problem is with line: wb = ?copy(rb) > > here I am getting the error: AttributeError: 'Book' object has no > > attribute 'on_demand' > > Please replace the first 4 lines of your script by these 6 lines: > > import xlrd > assert xlrd.__VERSION__ == "0.7.1" > from xlwt import easyxf > from xlutils.copy import copy > rb = xlrd.open_workbook( > ? ? 'source.xls',formatting_info=True, on_demand=False) > > and run it again. Please copy all the output and paste it into your > response. This time when I ran the code sent by you I got the following results:I am using ipython for running the code. AssertionError Traceback (most recent call last) /home/parul/CODES/copy_1.py in () 1 ----> 2 import xlrd 3 assert xlrd.__VERSION__ == "0.7.1" 4 from xlwt import easyxf 5 from xlutils.copy import copy 6 rb = xlrd.open_workbook('source.xls',formatting_info=True, on_demand=False) AssertionError: WARNING: Failure executing file: I used www.python-excel.org to get xlrd and xlwt .. so they are latest versions. From __peter__ at web.de Sun Jan 10 05:26:55 2010 From: __peter__ at web.de (Peter Otten) Date: Sun, 10 Jan 2010 11:26:55 +0100 Subject: Prepend to logging message References: <5aa06ab5-61fd-46a8-9cac-7f3a2a5d382b@d20g2000yqh.googlegroups.com> Message-ID: Joan Miller wrote: > How to prepend anything to a logging message? Is possible to do it > from the dictionary object (ExtraLog) or is there is that override > process() [1]? > > ------------------ > class ExtraLog(object): > > def __getitem__(self, name): > if name == 'foo': > result = 'testing' > return result > > def __iter__(self): > keys = ['foo',] > keys.extend(self.__dict__.keys()) > return iter(keys) format = "foo=%(foo)s " + logging.BASIC_FORMAT logging.basicConfig(format=format) logger = logging.LoggerAdapter(logging.getLogger('foo'), ExtraLog()) logger.error("yadda") Is that what you want? Peter From dikkie at nospam.org Sun Jan 10 05:59:03 2010 From: dikkie at nospam.org (Dikkie Dik) Date: Sun, 10 Jan 2010 11:59:03 +0100 Subject: (non-python) upgrading a MySQL database In-Reply-To: References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> <4dc0cfea1001090715md97518tf618309eda1deabb@mail.gmail.com> Message-ID: <4b49b2f7$0$8105$bf4948fe@news.tele2.nl> >> ... And I am interested in cleaning this up. I should probably >> start with the matter of databases, since that's something I won't be able >> to easily change once clients actually start entering data. Please share >> with me any further concepts or questions to get me thinking how to redesign >> the databases. I maintain my (MySQL) databases in an agile fashion. That means I start small, and let my databases evolve along with my applications. So the schema changes a lot during development. I take into account that the databases already contain data and that it must be preserved and migrated. I wrote a howto on my upgrading method and it can be found here: http://www.howtoforge.org/node/4833 Hope this is of any help. Good luck, Dikkie From sjmachin at lexicon.net Sun Jan 10 05:59:48 2010 From: sjmachin at lexicon.net (John Machin) Date: Sun, 10 Jan 2010 02:59:48 -0800 (PST) Subject: Porblem with xlutils/xlrd/xlwt References: <8e9245ec-05cb-48ed-ae32-f9fb0c46509f@j5g2000yqm.googlegroups.com> <972fdcfc-73a6-4e87-9752-3bc10da0dcad@s3g2000yqs.googlegroups.com> <43a1e323-8ccd-48b7-a6a4-e540aca2b1cd@35g2000yqa.googlegroups.com> <5ac38519-9ad4-4a89-9b3f-ccd56fb18721@m3g2000yqf.googlegroups.com> <610f640f-b03b-4aaa-93fe-a3fdf1381801@j4g2000yqe.googlegroups.com> Message-ID: <8bff475c-212a-4d15-85dd-a9e4fb103d35@26g2000yqo.googlegroups.com> On Jan 10, 8:51?pm, pp wrote: > On Jan 9, 8:23?am, John Machin wrote: > > > > > On Jan 9, 9:56?pm, pp wrote: > > > > On Jan 9, 3:52?am, Jon Clements wrote: > > > > > On Jan 9, 10:44?am, pp wrote: > > > > > > On Jan 9, 3:42?am, Jon Clements wrote: > > > > > > > On Jan 9, 10:24?am, pp wrote: > > > > > yeah all my versions are latest fromhttp://www.python-excel.org. > > > > > just checked!! > > > How did you check? You didn't answer this question. > > > > > > what could be the problem? > > > > > Does rb = xlrd.open_workbook('somesheet.xls', on_demand=True) work by > > > > itself? > > > > Yes it does. The problem is with line: wb = ?copy(rb) > > > here I am getting the error: AttributeError: 'Book' object has no > > > attribute 'on_demand' > > > Please replace the first 4 lines of your script by these 6 lines: > > > import xlrd > > assert xlrd.__VERSION__ == "0.7.1" > > from xlwt import easyxf > > from xlutils.copy import copy > > rb = xlrd.open_workbook( > > ? ? 'source.xls',formatting_info=True, on_demand=False) > > > and run it again. Please copy all the output and paste it into your > > response. > > This time when I ran the code sent by you I got the following > results:I am using ipython for running the code. > > AssertionError ? ? ? ? ? ? ? ? ? ? ? ? ? ?Traceback (most recent call > last) > > /home/parul/CODES/copy_1.py in () > ? ? ? 1 > ----> 2 import xlrd > ? ? ? 3 assert xlrd.__VERSION__ == "0.7.1" > ? ? ? 4 from xlwt import easyxf > ? ? ? 5 from xlutils.copy import copy > ? ? ? 6 rb = xlrd.open_workbook('source.xls',formatting_info=True, > on_demand=False) > > AssertionError: > WARNING: Failure executing file: > Your traceback appears to show an AssertionError from an import statement. We could do without an extra layer of noise in the channel; please consider giving ipython the flick (for debug purposes, at least) and use Python to run your script from the shell prompt. Change the second line to read: print xlrd.__VERSION__ > I used www.python-excel.org to get xlrd and xlwt .. so they are latest > versions. Let's concentrate on xlrd. I presume that means that you clicked on the xlrd Download link which took you to http://pypi.python.org/pypi/xlrd from which you can download the latest version of the package. That page has "xlrd 0.7.1" in a relatively large font at the top. You would have been presented with options to download one of these xlrd-0.7.1.tar.gz xlrd-0.7.1.win32.exe xlrd-0.7.1.zip (each uploaded on 2009-06-01). Which one did you download, and then what did you do with it? Or perhaps you ignored those and read further down to "Download link" which took you to an out-of-date page but you didn't notice the "0.6.1" in large bold type at the top nor the "Page last updated on 11 June 2007" at the bottom nor the "0.6.1" in the name of the file that you downloaded ... sorry about that; I've smacked the webmaster about the chops :-) Cheers, John From 3lvss0809 at gmail.com Sun Jan 10 06:00:29 2010 From: 3lvss0809 at gmail.com (3lvss0809 at gmail.com) Date: Sun, 10 Jan 2010 03:00:29 -0800 (PST) Subject: Microsoft Office Word and Python (Win XP) References: Message-ID: ? From peloko45 at gmail.com Sun Jan 10 06:23:37 2010 From: peloko45 at gmail.com (Joan Miller) Date: Sun, 10 Jan 2010 03:23:37 -0800 (PST) Subject: Prepend to logging message References: <5aa06ab5-61fd-46a8-9cac-7f3a2a5d382b@d20g2000yqh.googlegroups.com> Message-ID: On 10 ene, 10:26, Peter Otten <__pete... at web.de> wrote: > Joan Miller wrote: > > How to prepend anything to a logging message? Is possible to do it > > from the dictionary object (ExtraLog) or is there is that override > > process() [1]? > > > ------------------ > > class ExtraLog(object): > > > ? ? def __getitem__(self, name): > > ? ? ? ? if name == 'foo': > > ? ? ? ? ? ? result = 'testing' > > ? ? ? ? return result > > > ? ? def __iter__(self): > > ? ? ? ? keys = ['foo',] > > ? ? ? ? keys.extend(self.__dict__.keys()) > > ? ? ? ? return iter(keys) > > format = "foo=%(foo)s " + logging.BASIC_FORMAT > logging.basicConfig(format=format) > logger = logging.LoggerAdapter(logging.getLogger('foo'), ExtraLog()) > logger.error("yadda") > > Is that what you want? > > Peter I want that a message can be modified before of being logged. i.e. for "yadda" I would that were preppend 2 spaces. (And I want not manage that in the format to manage the indentation of all text) From matt at bubblegen.co.uk Sun Jan 10 06:30:24 2010 From: matt at bubblegen.co.uk (Matthew Lear) Date: Sun, 10 Jan 2010 11:30:24 +0000 Subject: getopt not raising exception Message-ID: <4B49BA50.8050207@bubblegen.co.uk> Hello, I'm having problems getting getopt to function correctly. Basically, no exception is being raised if no argument is passed to the code snippet below. I've read the Python documentation and tried example code from various sources which should cause an exception, only they don't. I've also tried executing the code on different machines too but to no avail. I'm sure I'm obviously doing something wrong but can't determine what. Any help would be much appreciated indeed. import sys, getopt try: opts, args = getopt.getopt(sys.argv, "h:", ["help"]) except getopt.GetoptError: print "error" sys.exit(2) If no args are passed when the script is run there is no exception raised. Why? Surely the "h:" means that this option must be passed? Thanks, -- Matt From mnordhoff at mattnordhoff.com Sun Jan 10 06:57:43 2010 From: mnordhoff at mattnordhoff.com (Matt Nordhoff) Date: Sun, 10 Jan 2010 11:57:43 +0000 Subject: getopt not raising exception In-Reply-To: <4B49BA50.8050207@bubblegen.co.uk> References: <4B49BA50.8050207@bubblegen.co.uk> Message-ID: <4B49C0B7.2000702@mattnordhoff.com> Matthew Lear wrote: > Hello, > > I'm having problems getting getopt to function correctly. Basically, no > exception is being raised if no argument is passed to the code snippet > below. I've read the Python documentation and tried example code from > various sources which should cause an exception, only they don't. I've > also tried executing the code on different machines too but to no avail. > I'm sure I'm obviously doing something wrong but can't determine what. > Any help would be much appreciated indeed. > > import sys, getopt > > try: > opts, args = getopt.getopt(sys.argv, "h:", ["help"]) > except getopt.GetoptError: > print "error" > sys.exit(2) > > If no args are passed when the script is run there is no exception > raised. Why? Surely the "h:" means that this option must be passed? > > Thanks, > -- Matt That's not what ":" means. It means that, *if* the option is passed, it must be followed by an additional argument, e.g.: foo.py -h something If you require that -h be passed, it's not much of an option! You should use a regular argument and check len(args) or somesuch. Or, if you must keep it an "option", do something equivalent for that. BTW: Checked out optparse? It's bigger and sexier! -- Matt Nordhoff From mail at hellmutweber.de Sun Jan 10 07:26:00 2010 From: mail at hellmutweber.de (Hellmut Weber) Date: Sun, 10 Jan 2010 13:26:00 +0100 Subject: integer and string compare, is that correct? Message-ID: <4B49C758.9050204@hellmutweber.de> Hi, being a causal python user (who likes the language quite a lot) it took me a while to realize the following: leo at sylvester py_count $ python Python 2.6.3 (r263:75183, Oct 26 2009, 12:34:23) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> max = '5' >>> n = 5 >>> n >= max False >>> n + max Traceback (most recent call last): File "", line 1, in TypeError: unsupported operand type(s) for +: 'int' and 'str' >>> Section 5.9 Comparison describes this. Can someone give me examples of use cases TIA Hellmut -- Dr. Hellmut Weber mail at hellmutweber.de Degenfeldstra?e 2 tel +49-89-3081172 D-80803 M?nchen-Schwabing mobil +49-172-8450321 please: No DOCs, no PPTs. why: tinyurl.com/cbgq From __peter__ at web.de Sun Jan 10 07:36:06 2010 From: __peter__ at web.de (Peter Otten) Date: Sun, 10 Jan 2010 13:36:06 +0100 Subject: Prepend to logging message References: <5aa06ab5-61fd-46a8-9cac-7f3a2a5d382b@d20g2000yqh.googlegroups.com> Message-ID: Joan Miller wrote: > On 10 ene, 10:26, Peter Otten <__pete... at web.de> wrote: >> Joan Miller wrote: >> > How to prepend anything to a logging message? Is possible to do it >> > from the dictionary object (ExtraLog) or is there is that override >> > process() [1]? >> >> > ------------------ >> > class ExtraLog(object): >> >> > def __getitem__(self, name): >> > if name == 'foo': >> > result = 'testing' >> > return result >> >> > def __iter__(self): >> > keys = ['foo',] >> > keys.extend(self.__dict__.keys()) >> > return iter(keys) >> >> format = "foo=%(foo)s " + logging.BASIC_FORMAT >> logging.basicConfig(format=format) >> logger = logging.LoggerAdapter(logging.getLogger('foo'), ExtraLog()) >> logger.error("yadda") >> >> Is that what you want? >> >> Peter > > I want that a message can be modified before of being logged. i.e. for > "yadda" I would that were preppend 2 spaces. (And I want not manage > that in the format to manage the indentation of all text) Following Ishwor's advice: import logging class MyLoggerAdapter(logging.LoggerAdapter): def process(self, msg, kwargs): return "message->" + msg.upper(), kwargs logging.basicConfig() logger = MyLoggerAdapter(logging.getLogger('foo'), {}) logger.error("yadda") Peter From clp2 at rebertia.com Sun Jan 10 07:46:03 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 10 Jan 2010 04:46:03 -0800 Subject: integer and string compare, is that correct? In-Reply-To: <4B49C758.9050204@hellmutweber.de> References: <4B49C758.9050204@hellmutweber.de> Message-ID: <50697b2c1001100446u41d9f20dt37c3dba252c9d460@mail.gmail.com> On Sun, Jan 10, 2010 at 4:26 AM, Hellmut Weber wrote: > Hi, > being a causal python user (who likes the language quite a lot) > it took me a while to realize the following: > > > leo at sylvester py_count $ python > Python 2.6.3 (r263:75183, Oct 26 2009, 12:34:23) > [GCC 4.4.1] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> max = '5' >>>> n = 5 >>>> n >= max > False >>>> n + max > Traceback (most recent call last): > ?File "", line 1, in > TypeError: unsupported operand type(s) for +: 'int' and 'str' >>>> > > > Section 5.9 Comparison describes this. > > Can someone give me examples of use cases The behavior of disparate types being comparable is deprecated and has been removed in Python 3.0+; don't rely upon it. (The ordering used is arbitrary but consistent) IIRC, the feature existed in prior versions to make lists containing mixed types sortable; this was found to be not all that useful and to hide what are quite arguably errors. Cheers, Chris -- http://blog.rebertia.com From clp2 at rebertia.com Sun Jan 10 07:48:19 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 10 Jan 2010 04:48:19 -0800 Subject: integer and string compare, is that correct? In-Reply-To: <50697b2c1001100446u41d9f20dt37c3dba252c9d460@mail.gmail.com> References: <4B49C758.9050204@hellmutweber.de> <50697b2c1001100446u41d9f20dt37c3dba252c9d460@mail.gmail.com> Message-ID: <50697b2c1001100448j57cfe8bewe1e4f8521c02e40d@mail.gmail.com> On Sun, Jan 10, 2010 at 4:46 AM, Chris Rebert wrote: > The behavior of disparate types being comparable is deprecated and has > been removed in Python 3.0+; don't rely upon it. Clarification: Equality testing between disparate types still works unaltered however. By "comparable", I meant >, <, >=, <= comparisons. Cheers, Chris From peloko45 at gmail.com Sun Jan 10 08:10:57 2010 From: peloko45 at gmail.com (Joan Miller) Date: Sun, 10 Jan 2010 05:10:57 -0800 (PST) Subject: Prepend to logging message References: <5aa06ab5-61fd-46a8-9cac-7f3a2a5d382b@d20g2000yqh.googlegroups.com> Message-ID: <06b9dbb3-dbbf-4929-a3a1-9a50893a8eb8@k19g2000yqc.googlegroups.com> On 10 ene, 12:36, Peter Otten <__pete... at web.de> wrote: > Joan Miller wrote: > > On 10 ene, 10:26, Peter Otten <__pete... at web.de> wrote: > >> Joan Miller wrote: > >> > How to prepend anything to a logging message? Is possible to do it > >> > from the dictionary object (ExtraLog) or is there is that override > >> > process() [1]? > > >> > ------------------ > >> > class ExtraLog(object): > > >> > def __getitem__(self, name): > >> > if name == 'foo': > >> > result = 'testing' > >> > return result > > >> > def __iter__(self): > >> > keys = ['foo',] > >> > keys.extend(self.__dict__.keys()) > >> > return iter(keys) > > >> format = "foo=%(foo)s " + logging.BASIC_FORMAT > >> logging.basicConfig(format=format) > >> logger = logging.LoggerAdapter(logging.getLogger('foo'), ExtraLog()) > >> logger.error("yadda") > > >> Is that what you want? > > >> Peter > > > I want that a message can be modified before of being logged. i.e. for > > "yadda" I would that were preppend 2 spaces. (And I want not manage > > that in the format to manage the indentation of all text) > > Following Ishwor's advice: > > import logging > > class MyLoggerAdapter(logging.LoggerAdapter): > ? ? def process(self, msg, kwargs): > ? ? ? ? return "message->" + msg.upper(), kwargs > > logging.basicConfig() > logger = MyLoggerAdapter(logging.getLogger('foo'), {}) > logger.error("yadda") > > Peter Thanks! I had to see the code to override it correctly [1] -------------------- class LoggerAdapter_(logging.LoggerAdapter): def __init__(self, logger, extra): self.logger = logger self.extra = extra def process(self, msg, kwargs): kwargs["extra"] = self.extra return "message->" + msg.upper(), kwargs -------------------- [1] http://bitbucket.org/mirror/python-trunk/src/tip/Lib/logging/__init__.py#cl-1264 From __peter__ at web.de Sun Jan 10 08:30:31 2010 From: __peter__ at web.de (Peter Otten) Date: Sun, 10 Jan 2010 14:30:31 +0100 Subject: integer and string compare, is that correct? References: Message-ID: Hellmut Weber wrote: > being a causal python user (who likes the language quite a lot) > it took me a while to realize the following: > > > leo at sylvester py_count $ python > Python 2.6.3 (r263:75183, Oct 26 2009, 12:34:23) > [GCC 4.4.1] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> max = '5' > >>> n = 5 > >>> n >= max > False > Section 5.9 Comparison describes this. > > Can someone give me examples of use cases The use cases for an order that works across types like int and str are weak to non-existent. Implementing it was considered a mistake and has been fixed in Python 3: Python 3.1.1+ (r311:74480, Nov 2 2009, 15:45:00) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 5 > "5" Traceback (most recent call last): File "", line 1, in TypeError: unorderable types: int() > str() Checking for equality is still possible: >>> 5 == "5" False Peter From drncode1 at live.com Sun Jan 10 08:35:36 2010 From: drncode1 at live.com (flow) Date: 11 Jan 2010 00:35:36 +1100 Subject: "Advanced" Python programming book? Message-ID: <4b49d7a8$1@dnews.tpgi.com.au> I've just finished reading a sort of beginner Python book, and I know quite a bit now but I'm looking for a book that can teach me advanced aspects of Python - code optimisation, threading, etc. Any recommendations? Cheers. From peloko45 at gmail.com Sun Jan 10 08:49:35 2010 From: peloko45 at gmail.com (Joan Miller) Date: Sun, 10 Jan 2010 05:49:35 -0800 (PST) Subject: Prepend to logging message References: <5aa06ab5-61fd-46a8-9cac-7f3a2a5d382b@d20g2000yqh.googlegroups.com> <06b9dbb3-dbbf-4929-a3a1-9a50893a8eb8@k19g2000yqc.googlegroups.com> Message-ID: <808f3af7-4652-4ad3-8c3d-d2c25465846f@r24g2000yqd.googlegroups.com> On 10 ene, 13:10, Joan Miller wrote: > On 10 ene, 12:36, Peter Otten <__pete... at web.de> wrote: > > > > > Joan Miller wrote: > > > On 10 ene, 10:26, Peter Otten <__pete... at web.de> wrote: > > >> Joan Miller wrote: > > >> > How to prepend anything to a logging message? Is possible to do it > > >> > from the dictionary object (ExtraLog) or is there is that override > > >> > process() [1]? > > > >> > ------------------ > > >> > class ExtraLog(object): > > > >> > def __getitem__(self, name): > > >> > if name == 'foo': > > >> > result = 'testing' > > >> > return result > > > >> > def __iter__(self): > > >> > keys = ['foo',] > > >> > keys.extend(self.__dict__.keys()) > > >> > return iter(keys) > > > >> format = "foo=%(foo)s " + logging.BASIC_FORMAT > > >> logging.basicConfig(format=format) > > >> logger = logging.LoggerAdapter(logging.getLogger('foo'), ExtraLog()) > > >> logger.error("yadda") > > > >> Is that what you want? > > > >> Peter > > > > I want that a message can be modified before of being logged. i.e. for > > > "yadda" I would that were preppend 2 spaces. (And I want not manage > > > that in the format to manage the indentation of all text) > > > Following Ishwor's advice: > > > import logging > > > class MyLoggerAdapter(logging.LoggerAdapter): > > ? ? def process(self, msg, kwargs): > > ? ? ? ? return "message->" + msg.upper(), kwargs > > > logging.basicConfig() > > logger = MyLoggerAdapter(logging.getLogger('foo'), {}) > > logger.error("yadda") > > > Peter > > Thanks! > > I had to see the code to override it correctly [1] > > -------------------- > class LoggerAdapter_(logging.LoggerAdapter): > > ? ? def __init__(self, logger, extra): > ? ? ? ? self.logger = logger > ? ? ? ? self.extra = extra > > ? ? def process(self, msg, kwargs): > ? ? ? ? kwargs["extra"] = self.extra > ? ? ? ? return "message->" + msg.upper(), kwargs > -------------------- > > [1]http://bitbucket.org/mirror/python-trunk/src/tip/Lib/logging/__init__... Sorry! It isn't necessary to copy __init__(). From ryniek90 at gmail.com Sun Jan 10 10:19:41 2010 From: ryniek90 at gmail.com (Ryniek90) Date: Sun, 10 Jan 2010 16:19:41 +0100 Subject: "Advanced" Python programming book? In-Reply-To: References: Message-ID: <4B49F00D.6000704@gmail.com> > I've just finished reading a sort of beginner Python book, and I know > quite a bit now but I'm looking for a book that can teach me advanced > aspects of Python - code optimisation, threading, etc. > > Any recommendations? > > Cheers. Check those link: http://www.amazon.com/Beginning-Python-Visualization-Transformation-Professionals/dp/1430218436/ref=sr_1_11?ie=UTF8&s=books&qid=1263136036&sr=1-11 http://www.amazon.com/Gray-Hat-Python-Programming-Engineers/dp/1593271921/ref=sr_1_12?ie=UTF8&s=books&qid=1263136036&sr=1-12 http://www.amazon.com/Python-Cookbook-Alex-Martelli/dp/0596007973/ref=sr_1_4?ie=UTF8&s=books&qid=1263136036&sr=1-4 http://www.amazon.com/High-Performance-Python-Anthony-Lewis/dp/0596522088/ref=sr_1_14?ie=UTF8&s=books&qid=1263136082&sr=1-14 http://www.amazon.com/Natural-Language-Processing-Python-Steve/dp/0596516495/ref=sr_1_16?ie=UTF8&s=books&qid=1263136082&sr=1-16 http://www.amazon.com/Python-Scripting-Computational-Science-Engineering/dp/3540739157/ref=sr_1_17?ie=UTF8&s=books&qid=1263136082&sr=1-17 http://www.amazon.com/Expert-Python-Programming-practices-distributing/dp/184719494X/ref=sr_1_20?ie=UTF8&s=books&qid=1263136082&sr=1-20 http://www.djangobook.com/ Similar topics you can find in Google. Check python technologies, like: PyCuda, SciPy, NumPy, PyGame, Django, Pylons, Zope, TurboGears, Twisted, PyGTK, PyQt and so on. Check this link: http://www.python.org/about/apps/ and this http://wiki.python.org/moin/ Cheers :) From steve at REMOVE-THIS-cybersource.com.au Sun Jan 10 10:30:12 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 10 Jan 2010 15:30:12 GMT Subject: lightweight encryption of text file References: Message-ID: <0359df54$0$1336$c3e8da3@news.astraweb.com> On Sun, 10 Jan 2010 09:59:31 +0100, Daniel Fetchinson wrote: > Thanks, this looks very simple too, but where is the decryption code? > Wikipedia seems to suggest that encryption and decryption are both the > same but running crypt on the output of crypt doesn't give back the > original string. So probably I'm misunderstanding something. Yes, the nature of a stream cipher :) What you're probably doing is what I did, before I had my Aha!!! moment: >>> arc = arcfour("password") >>> plaintext = "attack at dawn" >>> ciphertext = arc.crypt(plaintext) >>> print plaintext; print ciphertext.encode("hex").upper() attack at dawn 6371736C6E7C3F495C185629210B >>> x = arc.crypt(ciphertext) >>> assert x == plaintext Traceback (most recent call last): File "", line 1, in AssertionError >>> x '\x16\xf7\xf1\xcc\xda\xb5\xe0\xbf\x0b\x13 bF\x8f' So what's going on? Consider: Because Arcfour uses xor for the encryption step, decryption is exactly the same. So you only need one method to do both. But because Arcfour stores state, calling it twice in a row doesn't give the same result: >>> arc.crypt("hello").encode("hex").upper() 'CAA48DE953' >>> arc.crypt("hello").encode("hex").upper() '03405412CA' Arcfour is a stream cipher. When you call it twice on two different strings, that is logically equivalent to calling it once on a single long string made up of concatenating the two strings together. Each time you encrypt a single character, the internal state ("self.s") changes. To undo the change, you need the same state. The easiest way to do this is by creating a new instance: >>> encrypter = arcfour("password") >>> decrypter = arcfour("password") >>> plaintext = "attack at dawn" >>> ciphertext = encrypter.crypt(plaintext) >>> assert decrypter.crypt(ciphertext) == plaintext >>> So long as the two instances stay in lock-step (every time you use up a byte from the keystream in one, you do the same in the other) you can use one to decrypt the output of the other. It doesn't even really matter which one you use: >>> x = decrypter.crypt("Nobody expects the Spanish Inquisition!!!") >>> encrypter.crypt(x) 'Nobody expects the Spanish Inquisition!!!' In summary: use the arcfour class to create a stream. If you are just encrypting, or just decrypting, you can use one stream, or as many streams as you like, using different keys. But to do both, you need two streams, initiated with the same key, and kept in lockstep. The advantage of a stream cipher is that you can encrypt a text without needing all the text at once, and then decrypt it the same way: >>> output = [] >>> output.append(encrypt.crypt("abcdefghi")) >>> output.append(encrypt.crypt("jklmno")) >>> output.append(encrypt.crypt("p")) >>> output.append(encrypt.crypt("qrstuvwxyz")) >>> output = ''.join(output) >>> >>> plain = [] >>> plain.append(decrypt.crypt(output[0:20])) >>> plain.append(decrypt.crypt(output[20:24])) >>> plain.append(decrypt.crypt(output[24:])) >>> ''.join(plain) 'abcdefghijklmnopqrstuvwxyz' -- Steven From contact at pythonxy.com Sun Jan 10 10:50:20 2010 From: contact at pythonxy.com (Pierre Raybaut) Date: Sun, 10 Jan 2010 16:50:20 +0100 Subject: [ANN] Spyder v1.0.2 released Message-ID: <4B49F73C.9060402@pythonxy.com> Hi all, I'm pleased to announce here that Spyder version 1.0.2 has been released: http://packages.python.org/spyder Previously known as Pydee, Spyder (Scientific PYthon Development EnviRonment) is a free open-source Python development environment providing MATLAB-like features in a simple and light-weighted software, available for Windows XP/Vista/7, GNU/Linux and MacOS X: * advanced code editing features (code analysis, ...) * interactive console with MATLAB-like workpace (with GUI-based list, dictionary, tuple, text and array editors -- screenshots: http://packages.python.org/spyder/console.html#the-workspace) and integrated matplotlib figures * external console to open an interpreter or run a script in a separate process (with a global variable explorer providing the same features as the interactive console's workspace) * code analysis with pyflakes and pylint * search in files features * documentation viewer: automatically retrieves docstrings or source code of the function/class called in the interactive/external console * integrated file/directories explorer * MATLAB-like path management ...and more! Spyder is part of spyderlib, a Python module based on PyQt4 and QScintilla2 which provides powerful console-related PyQt4 widgets. Spyder v1.0.2 is a bugfix release: * External console: subprocess python calls were using the external console's sitecustomize.py (instead of system sitecustomize.py) * Added workaround for PyQt4 v4.6+ major bug with matplotlib * Added option to customize the way matplotlib figures are embedded (docked or floating window) * Matplotlib's "Option" dialog box is now supporting subplots * Array editor now supports complex arrays * Editor: replaced "Run selection or current line" option by "Run selection or current block" (without selection, this feature is similar to MATLAB's cell mode) * ...and a lot of minor bugfixes. - Pierre From nobody at nowhere.com Sun Jan 10 11:20:45 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 10 Jan 2010 16:20:45 +0000 Subject: lightweight encryption of text file References: Message-ID: On Sun, 10 Jan 2010 09:59:31 +0100, Daniel Fetchinson wrote: > Thanks, this looks very simple too, but where is the decryption code? > Wikipedia seems to suggest that encryption and decryption are both the > same but running crypt on the output of crypt doesn't give back the > original string. So probably I'm misunderstanding something. The encryption is stateful (it wouldn't be very good if it wasn't), so you need to create and initialise a new arcfour instance for decryption; you can't re-use the existing instance. From nobody at nowhere.com Sun Jan 10 11:24:53 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 10 Jan 2010 16:24:53 +0000 Subject: lightweight encryption of text file References: <0359df54$0$1336$c3e8da3@news.astraweb.com> Message-ID: On Sun, 10 Jan 2010 15:30:12 +0000, Steven D'Aprano wrote: >> Thanks, this looks very simple too, but where is the decryption code? >> Wikipedia seems to suggest that encryption and decryption are both the >> same but running crypt on the output of crypt doesn't give back the >> original string. So probably I'm misunderstanding something. > > Yes, the nature of a stream cipher :) It isn't limited to stream ciphers. You would have the same issue for a block cipher with chaining (i.e. *not* ECB mode). From nobody at nowhere.com Sun Jan 10 11:34:46 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 10 Jan 2010 16:34:46 +0000 Subject: integer and string compare, is that correct? References: Message-ID: Hellmut Weber wrote: >> being a causal python user (who likes the language quite a lot) >> it took me a while to realize the following: >> >>> max = '5' >> >>> n = 5 >> >>> n >= max >> False > >> Section 5.9 Comparison describes this. >> >> Can someone give me examples of use cases Peter Otten wrote: > The use cases for an order that works across types like int and str are weak > to non-existent. Implementing it was considered a mistake and has been fixed > in Python 3: >>>> 5 > "5" > Traceback (most recent call last): > File "", line 1, in > TypeError: unorderable types: int() > str() If you actually need to perform comparisons across types, you can rely upon the fact that tuple comparisons are non-strict and use e.g.: > a = 5 > b = '5' > (type(a).__name__, a) < (type(b).__name__, b) True > (type(a).__name__, a) > (type(b).__name__, b) False The second elements will only be compared if the first elements are equal (i.e. the values have the same type). From tomslists at netp.org Sun Jan 10 11:44:21 2010 From: tomslists at netp.org (Tom Pacheco) Date: Sun, 10 Jan 2010 11:44:21 -0500 Subject: "Advanced" Python programming book? In-Reply-To: <4B49F00D.6000704@gmail.com> References: <4B49F00D.6000704@gmail.com> Message-ID: <4B4A03E5.2000205@netp.org> Ryniek90 wrote: >> I've just finished reading a sort of beginner Python book, and I know >> quite a bit now but I'm looking for a book that can teach me advanced >> aspects of Python - code optimisation, threading, etc. >> >> Any recommendations? >> >> Cheers. >> > Check those link: > http://www.amazon.com/Beginning-Python-Visualization-Transformation-Professionals/dp/1430218436/ref=sr_1_11?ie=UTF8&s=books&qid=1263136036&sr=1-11 > > http://www.amazon.com/Gray-Hat-Python-Programming-Engineers/dp/1593271921/ref=sr_1_12?ie=UTF8&s=books&qid=1263136036&sr=1-12 > > http://www.amazon.com/Python-Cookbook-Alex-Martelli/dp/0596007973/ref=sr_1_4?ie=UTF8&s=books&qid=1263136036&sr=1-4 > > http://www.amazon.com/High-Performance-Python-Anthony-Lewis/dp/0596522088/ref=sr_1_14?ie=UTF8&s=books&qid=1263136082&sr=1-14 > > http://www.amazon.com/Natural-Language-Processing-Python-Steve/dp/0596516495/ref=sr_1_16?ie=UTF8&s=books&qid=1263136082&sr=1-16 > > http://www.amazon.com/Python-Scripting-Computational-Science-Engineering/dp/3540739157/ref=sr_1_17?ie=UTF8&s=books&qid=1263136082&sr=1-17 > > http://www.amazon.com/Expert-Python-Programming-practices-distributing/dp/184719494X/ref=sr_1_20?ie=UTF8&s=books&qid=1263136082&sr=1-20 > > http://www.djangobook.com/ > > > Similar topics you can find in Google. > Check python technologies, like: PyCuda, SciPy, NumPy, PyGame, Django, > Pylons, Zope, TurboGears, Twisted, PyGTK, PyQt and so on. > > Check this link: http://www.python.org/about/apps/ and this > http://wiki.python.org/moin/ > > Cheers :) > some good books.. i might add this to the list http://www.amazon.com/Python-Essential-Reference-David-Beazley/dp/0672329786/ref=pd_sim_b_2 From fetchinson at googlemail.com Sun Jan 10 11:48:02 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Sun, 10 Jan 2010 17:48:02 +0100 Subject: lightweight encryption of text file In-Reply-To: <0359df54$0$1336$c3e8da3@news.astraweb.com> References: <0359df54$0$1336$c3e8da3@news.astraweb.com> Message-ID: >> Thanks, this looks very simple too, but where is the decryption code? >> Wikipedia seems to suggest that encryption and decryption are both the >> same but running crypt on the output of crypt doesn't give back the >> original string. So probably I'm misunderstanding something. > > Yes, the nature of a stream cipher :) > > What you're probably doing is what I did, before I had my Aha!!! moment: > > >>>> arc = arcfour("password") >>>> plaintext = "attack at dawn" >>>> ciphertext = arc.crypt(plaintext) >>>> print plaintext; print ciphertext.encode("hex").upper() > attack at dawn > 6371736C6E7C3F495C185629210B >>>> x = arc.crypt(ciphertext) >>>> assert x == plaintext > Traceback (most recent call last): > File "", line 1, in > AssertionError >>>> x > '\x16\xf7\xf1\xcc\xda\xb5\xe0\xbf\x0b\x13 bF\x8f' > > > So what's going on? Consider: > > Because Arcfour uses xor for the encryption step, decryption is exactly > the same. So you only need one method to do both. > > But because Arcfour stores state, calling it twice in a row doesn't give > the same result: > >>>> arc.crypt("hello").encode("hex").upper() > 'CAA48DE953' >>>> arc.crypt("hello").encode("hex").upper() > '03405412CA' > > > Arcfour is a stream cipher. When you call it twice on two different > strings, that is logically equivalent to calling it once on a single long > string made up of concatenating the two strings together. Each time you > encrypt a single character, the internal state ("self.s") changes. To > undo the change, you need the same state. The easiest way to do this is > by creating a new instance: > > >>>> encrypter = arcfour("password") >>>> decrypter = arcfour("password") >>>> plaintext = "attack at dawn" >>>> ciphertext = encrypter.crypt(plaintext) >>>> assert decrypter.crypt(ciphertext) == plaintext >>>> > > > So long as the two instances stay in lock-step (every time you use up a > byte from the keystream in one, you do the same in the other) you can use > one to decrypt the output of the other. It doesn't even really matter > which one you use: > >>>> x = decrypter.crypt("Nobody expects the Spanish Inquisition!!!") >>>> encrypter.crypt(x) > 'Nobody expects the Spanish Inquisition!!!' > > > In summary: use the arcfour class to create a stream. If you are just > encrypting, or just decrypting, you can use one stream, or as many > streams as you like, using different keys. But to do both, you need two > streams, initiated with the same key, and kept in lockstep. > > The advantage of a stream cipher is that you can encrypt a text without > needing all the text at once, and then decrypt it the same way: > >>>> output = [] >>>> output.append(encrypt.crypt("abcdefghi")) >>>> output.append(encrypt.crypt("jklmno")) >>>> output.append(encrypt.crypt("p")) >>>> output.append(encrypt.crypt("qrstuvwxyz")) >>>> output = ''.join(output) >>>> >>>> plain = [] >>>> plain.append(decrypt.crypt(output[0:20])) >>>> plain.append(decrypt.crypt(output[20:24])) >>>> plain.append(decrypt.crypt(output[24:])) >>>> ''.join(plain) > 'abcdefghijklmnopqrstuvwxyz' Thanks Steven, this was very helpful! Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From no.email at nospam.invalid Sun Jan 10 11:54:51 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 10 Jan 2010 08:54:51 -0800 Subject: lightweight encryption of text file References: Message-ID: <7xfx6dhpd0.fsf@ruckus.brouhaha.com> Nobody writes: > RC4 (aka ArcFour) is quite trivial to implement, and better than inventing > your own cipher or using a Vignere: ... That's a cute implementation, but it has no authentication and doesn't include any randomness, which means if you use the same key for two inputs, there is a security failure (xor'ing the two ciphertexts reveals the xor of the plaintexts). It also looks rather slow. I don't make any guarantees about p3.py, but it has been reviewed by several experts and appears to be reasonably sound for the type of casual use being discussed here, and it is tuned for speed (given the implementation constraints). For more demanding purposes, you should use a more serious library like one of the OpenSSL wrappers. From victorsubervi at gmail.com Sun Jan 10 12:07:30 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sun, 10 Jan 2010 12:07:30 -0500 Subject: Academic Question Message-ID: <4dc0cfea1001100907t6bc032cftea2e9e258e7c1937@mail.gmail.com> Hi; The following code that works: #! /usr/bin/python import string import cgitb; cgitb.enable() import cgi import MySQLdb import sys,os from sets import Set import fpformat cwd = os.getcwd() sys.path.append(cwd) from login import login from particulars import ourOptions form = cgi.FieldStorage() store = form.getfirst('store') cat = form.getfirst('cat') id = form.getfirst('id') patientID = form.getfirst('patientID') try: browser = form.getfirst('browser', 'all') except: browser = headers() os.chdir('%s/..' % cwd) sys.path.append(os.getcwd()) from templateFrame import top, bottom os.chdir(cwd) Why doesn't it work if I move the bottom imports to the top? The form values get lost, even though I chdir to cwd. TIA, beno -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Sun Jan 10 12:17:50 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sun, 10 Jan 2010 12:17:50 -0500 Subject: Something More Elegant In-Reply-To: <7a9c25c21001091200r10c3c7b2jbc16040efa9f7b34@mail.gmail.com> References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> <4dc0cfea1001090715md97518tf618309eda1deabb@mail.gmail.com> <7a9c25c21001091200r10c3c7b2jbc16040efa9f7b34@mail.gmail.com> Message-ID: <4dc0cfea1001100917v1021e052y104642295303d7b6@mail.gmail.com> On Sat, Jan 9, 2010 at 3:00 PM, Stephen Hansen wrote: > On Sat, Jan 9, 2010 at 7:15 AM, Victor Subervi wrote: > >> On Sat, Jan 9, 2010 at 9:35 AM, Steve Holden wrote: >> >>> But we are now in the realm of theory as far as you are concerned, since >>> you have already stated several times that you aren't interested in >>> correcting your design until after you have got the current mess into >>> production. So good luck with that. >>> >> >> And if you were in my shoes, I'm sure you'd do the same thing. > > > ... Really, no, he wouldn't :) > Let's not argue an academic point. > We're not all just hobbyists here who only do work for random open source > projects. A lot of us are professionals who actually do have clients, > actually do have deadlines, actually do have an understanding for production > requirements. Getting something into production as soon as possible is > certainly an important goal in commercial work. But it is not the only goal. > Proper database design is very important because if you don't do it, you'll > actually end up usually wasting *more* time and effort then if you just bite > the bullet and fix it now. > Clearly. That's why when it was pointed out to me, I jumped on it. > > Proper database design -- in particular in your case, not having multiple > tables with various names that even need "%sThis" or "%sThat", and using > parameterized queries to access those tables, is really important. It will > save you time, it will save you effort, and it'll save you money-- because > /not/ doing it is among other things, a major security risk. Getting code > out fast and now is a noble goal in commercial projects, getting code out > which is by design prone to attacks by hackers is negligence. > You're preaching to the choir ;) Read the above. The only exception to this is putting all store stuff in the same table. The columns of each store table are different and automatically created through what I believe are called "mixins". > > Well, it *is* working now :)) And I am interested in cleaning this up. I >> should probably start with the matter of databases, since that's something I >> won't be able to easily change once clients actually start entering data. >> Please share with me any further concepts or questions to get me thinking >> how to redesign the databases. >> > > The next thing to do is to re-do your SQL queries. You should never do > string interpolation, e.g: > > SQL="SELECT x FROM y WHERE z = %s" % (arg,) > cur.execute(SQL) > > If you have done the first step, your tables always have a set names so you > never need to interpolate to do the queries-- and then at this point, under > no circumstances ever, ever-- consider this a law that you will get fined > for violation-- use string interpolation to generate your queries. > > Instead, do: > > cur.execute("SELECT x FROM y WHERE z = %s", (arg,)) > > That looks really similar, but is lightyears away. Its very unfortunate > that some database drivers use 'format' as the paramstyle because it > confuses issues, but the two effects are very different. In one, Python is > just munging together and creating a new string. In the other, the database > driver is doing a few things. Its analyzing the query, its storing it (often > caching it, which speeds up further executions of that query), etc, etc, and > then finally its seeing that you are passing arguments into it, and it is > -safely- binding those arguments into the expression; this prevents SQL > Injection attacks. You can use interpolation and prevent injection if you > -meticulously- check -every- string that comes from the user, and -never- > trust it (even if that string was written out to a hidden and > legitimate users have no way to alter, because illegitimate users will alter > it anyways). Or you can use parameterized queries and just avoid it, while > getting plenty of other benefits as well. > > > At work, we had a third-party package that we re-sold as part of our > offering, and glancing over its source, I noticed something. It did, in > essence to check login: > > cur.execute("SELECT user_id FROM usertable WHERE username = '%s' AND > password = '%s' % (username, password)) > > I blinked, and emailed them to point out the problem. I suggested they log > in as: > > Username = dummyuser > Password = '; DROP usertable > > You see, when using interpolation, the string that got sent to the database > was: > > SELECT user_id FROM usertable WHERE username = 'dummyuser' AND password > = ''; DROP usertable > > And thus, from the login screen of this app-- I destroyed their > environment. > > Its sort of important that you not put code out into production which is > susceptible to such things. Your clients will not at all appreciate it if > and when some hacker discovers it and destroys their site, loosing them > money. But even beyond that, there are many other benefits to just doing > this right. You want to, believe me. Now, before you put anything into > production. It might take a day or two longer, but its worth it. > > Finally, go finish getting rid of the bare excepts, I saw one recently :) > Just rip them all out, every one. All at once. Then fix any errors that come > along with specific excepts or pre-tests if appropriate. ;) > Frankly, none of this will take too long (except normalization, of course). And I'm on every one of them. Thank you very much! beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From wolftracks at invalid.com Sun Jan 10 12:28:15 2010 From: wolftracks at invalid.com (W. eWatson) Date: Sun, 10 Jan 2010 09:28:15 -0800 Subject: Fractional Hours from datetime? Message-ID: Maybe there's a more elegant way to do this. I want to express the result of datetime.datetime.now() in fractional hours. Here's one way. dt=datetime.datetime.now() xtup = dt.timetuple() h = xtup[3]+xtup[4]/60.0+xtup[5]/3600.00+xtup[6]/10**6 # now is in fractions of an hour From __peter__ at web.de Sun Jan 10 12:37:27 2010 From: __peter__ at web.de (Peter Otten) Date: Sun, 10 Jan 2010 18:37:27 +0100 Subject: integer and string compare, is that correct? References: Message-ID: Somebody wrote: > If you actually need to perform comparisons across types, you can rely > upon the fact that tuple comparisons are non-strict and use e.g.: > > > a = 5 > > b = '5' > > (type(a).__name__, a) < (type(b).__name__, b) > True > > (type(a).__name__, a) > (type(b).__name__, b) > False > > The second elements will only be compared if the first elements are equal > (i.e. the values have the same type). The same type *name*. To play it safe you'll have to compare the id, too: >>> items = [] >>> for i in range(2): ... class A: pass ... items.append(A()) ... >>> sorted(items, key=lambda a: (type(a).__name__, a)) Traceback (most recent call last): File "", line 1, in TypeError: unorderable types: A() < A() >>> sorted(items, key=lambda a: (type(a).__name__, id(type(a)), a)) [<__main__.A object at 0x14dfbd0>, <__main__.A object at 0x14dfc50>] Peter From aahz at pythoncraft.com Sun Jan 10 12:48:02 2010 From: aahz at pythoncraft.com (Aahz) Date: 10 Jan 2010 09:48:02 -0800 Subject: problem with cheetah References: <0fd84b05-cf12-485b-a14e-608e47679b32@s20g2000yqd.googlegroups.com> Message-ID: In article <0fd84b05-cf12-485b-a14e-608e47679b32 at s20g2000yqd.googlegroups.com>, mlowicki wrote: > >Hi!, i get such error when I try to install cheetah: Probably better to ask on the Cheetah list: https://lists.sourceforge.net/lists/listinfo/cheetahtemplate-discuss -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." --Red Adair From peloko45 at gmail.com Sun Jan 10 14:04:08 2010 From: peloko45 at gmail.com (Joan Miller) Date: Sun, 10 Jan 2010 11:04:08 -0800 (PST) Subject: Milliseconds in logging format Message-ID: How the logging '%(asctime)s' [1] specifier to gets the millisecond portion of the time if there is not a directive to get it from the time module [2] ? "The date format string follows the requirements of strftime()" [1] http://docs.python.org/library/logging.html#basic-example [2] http://docs.python.org/library/time.html#time.strftime From python at mrabarnett.plus.com Sun Jan 10 14:09:17 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 10 Jan 2010 19:09:17 +0000 Subject: Academic Question In-Reply-To: <4dc0cfea1001100907t6bc032cftea2e9e258e7c1937@mail.gmail.com> References: <4dc0cfea1001100907t6bc032cftea2e9e258e7c1937@mail.gmail.com> Message-ID: <4B4A25DD.4040703@mrabarnett.plus.com> Victor Subervi wrote: > Hi; > The following code that works: > > #! /usr/bin/python > > import string > import cgitb; cgitb.enable() > import cgi > import MySQLdb > import sys,os > from sets import Set > import fpformat > cwd = os.getcwd() > sys.path.append(cwd) > from login import login > from particulars import ourOptions > > form = cgi.FieldStorage() > store = form.getfirst('store') > cat = form.getfirst('cat') > id = form.getfirst('id') > patientID = form.getfirst('patientID') > > try: > browser = form.getfirst('browser', 'all') > except: > browser = headers() > A bare except, and if an exception _does_ occur, they'll be a NameError because 'headers' isn't defined. > os.chdir('%s/..' % cwd) > sys.path.append(os.getcwd()) > from templateFrame import top, bottom > os.chdir(cwd) > > Why doesn't it work if I move the bottom imports to the top? The form > values get lost, even though I chdir to cwd. > I try to avoid changing the directory. From awilliam at opengroupware.us Sun Jan 10 14:45:24 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Sun, 10 Jan 2010 14:45:24 -0500 Subject: "Bad file descriptor" in HTTPServer using Multiprocessing. Message-ID: <1263152724.13227.5.camel@linux-m3mt> I have a Python multiprocessing application where a master process starts server sub-processes and communicates with them via Pipes; that works very well. But one of the subprocesses, in turn, starts a collection of HTTPServer 'workers' (almost exactly as demonstrated in the docs). This works perfectly.... so long as the subprocess that starts the HTTPServer workers is the *first* process started by the master process. Otherwise the HTTPServer's serve_forever immediately abends with: "(9, 'Bad file descriptor')" I'm confused why the order of starting the processes matter, especially given that the HTTPServer's are start in a subprocess of a subprocess. The master doesn't do much of anything between starting each subprocess (which it does in a loop; identify all the subprocesses to start, and start them.). subprocess -------------------- ...... self._httpd = HTTPServer((HTTP_HOST, HTTP_PORT), HTTPRequestHandler) print 'HTTPServer created.' for i in range(10): p = multiprocessing.Process(target=serve_forever, args=(self._httpd, i, self)) self._workers.append(p) p.start() ..... def serve_forever(server, i, control): print 'coils.http starting HTTP worker #{0}'.format(i) try: server.serve_forever() except KeyboardInterrupt: pass except Exception, e: control.log.exception(e) print 'coils.http worker #{0} abended with exception.'.format(i) print e return -- OpenGroupware developer: awilliam at whitemice.org OpenGroupare & Cyrus IMAPd documenation @ From debatem1 at gmail.com Sun Jan 10 14:59:52 2010 From: debatem1 at gmail.com (geremy condra) Date: Sun, 10 Jan 2010 14:59:52 -0500 Subject: lightweight encryption of text file In-Reply-To: References: Message-ID: Not sure why in the world you would homebrew something like this- a small dependency isn't that bad, and aes can be pretty simple to use. Might as well go for the industrial strength approach. Geremy Condra From irmen-NOSPAM- at xs4all.nl Sun Jan 10 15:05:06 2010 From: irmen-NOSPAM- at xs4all.nl (Irmen de Jong) Date: Sun, 10 Jan 2010 21:05:06 +0100 Subject: Milliseconds in logging format In-Reply-To: References: Message-ID: <4b4a32a6$0$22940$e4fe514c@news.xs4all.nl> On 10-1-2010 20:04, Joan Miller wrote: > How the logging '%(asctime)s' [1] specifier to gets the millisecond > portion of the time if there is not a directive to get it from the > time module [2] ? > > > "The date format string follows the requirements of strftime()" > [1] http://docs.python.org/library/logging.html#basic-example > > [2] http://docs.python.org/library/time.html#time.strftime It appends them to the string that strftime() returned. If you set a custom datefmt, it will stop doing this and you'll have to use %(msecs) in your log format to get them back. See the source, Lib/logging/__init__.py around line 400. (Python 2.6.4) -irmen From bryanvick at gmail.com Sun Jan 10 15:21:40 2010 From: bryanvick at gmail.com (Bryan) Date: Sun, 10 Jan 2010 12:21:40 -0800 (PST) Subject: How to use list type generated by SWIG? Message-ID: I am writing a small script to manage my ipod. I am using the python bindings for libgpod. I have never used swig, or used python to program against a c/c++ library. I can get the library to find my ipod, and parse its DB format, but I'm not sure how to interact with the types that some of the functions return, specifically a Swig Glist (gtk's doubly-linked list) import gpod db = gpod.itdb_parse('/media/ipod', None) print db.tracks >>> Glist is not iterable or subscriptable. It contains a 'next' function which always seems to return None after printing "swig/python detected a memory leak of type 'GList *', no destructor found." to the console. How can I interact with this list? From no.email at nospam.invalid Sun Jan 10 15:26:05 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 10 Jan 2010 12:26:05 -0800 Subject: lightweight encryption of text file References: Message-ID: <7xiqb9yaea.fsf@ruckus.brouhaha.com> geremy condra writes: > Not sure why in the world you would homebrew something like this- a > small dependency isn't that bad, and aes can be pretty simple to use. > Might as well go for the industrial strength approach. In my experience, 1) small dependencies ARE that bad, since they mean you have to develop and test on every platform that you want your code to run on; 2) using a serious library requires quite a bit of knowledge and decision-making which not everyone is equipped to do. "AES" is not so simple to use unless you know what you're doing in terms of modes, nonces, etc. Having supported this kind of package in a commercial setting in the past, IMO, for the sort of (common) application in question, it's best to keep things as simple as possible and supply a single interface that provides encryption, authentication, and random initialization all in one call. The cost is a little bit of ciphertext bloat, but it prevents all kinds of security failures frequently overlooked by novices. I'd like it a lot if the Python stdlib could include a serious cryptography module. That was rejected for regulatory reasons several years ago, but maybe things are changing enough that the issue can be revisited sometime. From 3lvss0809 at gmail.com Sun Jan 10 15:27:58 2010 From: 3lvss0809 at gmail.com (3lvss0809 at gmail.com) Date: Sun, 10 Jan 2010 12:27:58 -0800 (PST) Subject: Microsoft Office Word and Python (Win XP) References: Message-ID: <86b15d62-0c8f-4341-980b-0ec99f15f1f4@21g2000yqj.googlegroups.com> no idea :-( From 3lvss0809 at gmail.com Sun Jan 10 15:30:33 2010 From: 3lvss0809 at gmail.com (3lvss0809 at gmail.com) Date: Sun, 10 Jan 2010 12:30:33 -0800 (PST) Subject: Microsoft Office Word and Python (Win XP) References: Message-ID: so does anyone know how I could do this? From matt at bubblegen.co.uk Sun Jan 10 16:39:49 2010 From: matt at bubblegen.co.uk (Matthew Lear) Date: Sun, 10 Jan 2010 21:39:49 +0000 Subject: getopt not raising exception In-Reply-To: <4B49C0B7.2000702@mattnordhoff.com> References: <4B49BA50.8050207@bubblegen.co.uk> <4B49C0B7.2000702@mattnordhoff.com> Message-ID: <4B4A4925.9000805@bubblegen.co.uk> Matt Nordhoff wrote: > BTW: Checked out optparse? It's bigger and sexier! Thanks for the tip. Works a treat. -- Matt From emmanuel.surleau at gmail.com Sun Jan 10 17:08:28 2010 From: emmanuel.surleau at gmail.com (Emmanuel Surleau) Date: Sun, 10 Jan 2010 23:08:28 +0100 Subject: getopt not raising exception In-Reply-To: <4B4A4925.9000805@bubblegen.co.uk> References: <4B49BA50.8050207@bubblegen.co.uk> <4B49C0B7.2000702@mattnordhoff.com> <4B4A4925.9000805@bubblegen.co.uk> Message-ID: <201001102308.28179.emmanuel.surleau@gmail.com> > Matt Nordhoff wrote: > > BTW: Checked out optparse? It's bigger and sexier! If you're doing things with positional arguments, you should consider using argparse (http://argparse.googlecode.com/svn/tags/r101/doc/index.html). It's like optparse, but (much) better. Cheers, Emm From stef.mientki at gmail.com Sun Jan 10 18:48:22 2010 From: stef.mientki at gmail.com (Stef Mientki) Date: Mon, 11 Jan 2010 00:48:22 +0100 Subject: class viewer ? Message-ID: <4B4A6746.9020405@gmail.com> hello, I'd like to have a class viewer, something different from pydoc, and I wonder if someone has made something similar. from the given class, it's ancestors and it's derived classes, I'ld like to get the following information in a tree like structure: - the file were the class is definied - the attributes, split in inherited / created / overriden - the methodes, split in inherited / created / overriden - the files were instances of the class are created - and probably I forget a few any suggestions ? thanks, Stef Mientki From zabin.farishta at gmail.com Sun Jan 10 19:10:01 2010 From: zabin.farishta at gmail.com (Zabin) Date: Sun, 10 Jan 2010 16:10:01 -0800 (PST) Subject: setTextAlignment function Qtablewidget PyQt Message-ID: Hey! I am trying to align contents of some table cells but find the code below working for some fields and not for others. I am stuck as to why this is happening. Help will be gretly appreciated! setTextAlignment(QtCore.Qt.AlignVCenter) Cheers Zabin From mail at hellmutweber.de Sun Jan 10 19:44:30 2010 From: mail at hellmutweber.de (Hellmut Weber) Date: Mon, 11 Jan 2010 01:44:30 +0100 Subject: integer and string compare, is that correct? In-Reply-To: References: Message-ID: <4B4A746E.4010109@hellmutweber.de> Hi, thanks to all who answered. I'm using Python 2.6.5 on my machine and consulted the corresponding documentation. I do appreciate the modified definition of python 3, that seems much more reasonable. Thanks for indicating. Greetings from Munich in Winter Hellmut Am 10.01.2010 17:34, schrieb Nobody: > Hellmut Weber wrote: > >>> being a causal python user (who likes the language quite a lot) >>> it took me a while to realize the following: > >>> >>> max = '5' >>> >>> n = 5 >>> >>> n>= max >>> False >> >>> Section 5.9 Comparison describes this. >>> >>> Can someone give me examples of use cases > > Peter Otten wrote: > >> The use cases for an order that works across types like int and str are weak >> to non-existent. Implementing it was considered a mistake and has been fixed >> in Python 3: > >>>>> 5> "5" >> Traceback (most recent call last): >> File "", line 1, in >> TypeError: unorderable types: int()> str() > > If you actually need to perform comparisons across types, you can rely > upon the fact that tuple comparisons are non-strict and use e.g.: > > > a = 5 > > b = '5' > > (type(a).__name__, a)< (type(b).__name__, b) > True > > (type(a).__name__, a)> (type(b).__name__, b) > False > > The second elements will only be compared if the first elements are equal > (i.e. the values have the same type). > -- Dr. Hellmut Weber mail at hellmutweber.de Degenfeldstra?e 2 tel +49-89-3081172 D-80803 M?nchen-Schwabing mobil +49-172-8450321 please: No DOCs, no PPTs. why: tinyurl.com/cbgq From MLMDev at Comcast.net Sun Jan 10 19:45:59 2010 From: MLMDev at Comcast.net (Mitchell L Model) Date: Sun, 10 Jan 2010 19:45:59 -0500 Subject: sys.stdout vs. sys.stderr Message-ID: <6C60CA7B-E01C-449F-B956-E9E58A0ADD34@Comcast.net> In Python 3.1 is there any difference in the buffering behavior of the initial sys.stdout and sys.stderr streams? They are both line_buffered and stdout doesn't seem to use a larger-grain buffering, so they seem to be identical with respect to buffering. Were they different at some earlier point in Python's evolution? From zabin.farishta at gmail.com Sun Jan 10 20:07:10 2010 From: zabin.farishta at gmail.com (Zabin) Date: Sun, 10 Jan 2010 17:07:10 -0800 (PST) Subject: setTextAlignment function Qtablewidget PyQt References: Message-ID: <10cb3536-abb7-4962-b1b9-0a4b07b26c5b@35g2000yqa.googlegroups.com> On Jan 11, 1:10?pm, Zabin wrote: > Hey! > > I am trying to align contents of some table cells but find the code > below working for some fields and not for others. I am stuck as to why > this is happening. Help will be gretly appreciated! > > setTextAlignment(QtCore.Qt.AlignVCenter) > > Cheers > Zabin Figured it out! It was just the '\n' character being read with the line from the text file that messed up the formatting. just needed to use the line.strip() function before passing it to the table as a table widget item. From tjreedy at udel.edu Sun Jan 10 20:43:15 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 10 Jan 2010 20:43:15 -0500 Subject: Computer Music Toolkit (CMT) ? In-Reply-To: References: Message-ID: On 1/10/2010 4:15 AM, Peter Billam wrote: > Greetings. Is there a way to get at the Computer Music Toolkit (CMT) > http://www.ladspa.org/cmt/ > functionality from Python (Python3 in my case) ? You can access compiled C shared libraries most easily via the ctypes module. Searching Python CMT "Computer Music Toolkit" (following Stefan's suggestion) returns a few hits that might be useful. Searching PyPI for CMT does not turn up any relevant packages, so if you do develop a ctypes wrapping, consider contributing it. Terry Jan Reedy From tjreedy at udel.edu Sun Jan 10 21:36:37 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 10 Jan 2010 21:36:37 -0500 Subject: Python Goldmine has been updated: http://preciseinfo.org/Convert/index_Convert_Python.html In-Reply-To: References: Message-ID: On 1/8/2010 11:50 AM, tanix wrote: > Python Goldmine collection contains the extensive collection of articles > going back several years. It includes thousands of code > examples and expert discussions on all major topics. > > The information is organized by relevant topics, covered > by the corresponding chapters. > > The information was filtered with sophisticated filters and vast > majority of artilces with little relevance have been filtered out. > > If you have any specific requests for some new chapters to be added > and it is of interest to others, please post your requests on this > thread. > > If anyone feels he has above average level of competence, or can > reccommend someone who posts on this group, you may request to be > included in the expert chapters. > > The Python Goldmine is at: > > http://preciseinfo.org/Convert/index_Convert_Python.html > > -- > Programmer's Goldmine collections: > > http://preciseinfo.org > > Tens of thousands of code examples and expert discussions on > C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, > organized by major topics of language, tools, methods, techniques. This site pops up spam windowns. One was blocked, one managed to bypass the popup blocker. Tnis is not friendly behaviour. Some categories have 100s of entries. Better, I think, to use a search engine such as Google with more specific search terms and a snippet of context for each result. tjr From steve at holdenweb.com Sun Jan 10 22:00:22 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 10 Jan 2010 22:00:22 -0500 Subject: Python Goldmine has been updated: http://preciseinfo.org/Convert/index_Convert_Python.html In-Reply-To: References: Message-ID: <4B4A9446.7020009@holdenweb.com> Terry Reedy wrote: > On 1/8/2010 11:50 AM, tanix wrote: >> Python Goldmine collection contains the extensive collection of articles >> going back several years. It includes thousands of code >> examples and expert discussions on all major topics. >> >> The information is organized by relevant topics, covered >> by the corresponding chapters. >> >> The information was filtered with sophisticated filters and vast >> majority of artilces with little relevance have been filtered out. >> >> If you have any specific requests for some new chapters to be added >> and it is of interest to others, please post your requests on this >> thread. >> >> If anyone feels he has above average level of competence, or can >> reccommend someone who posts on this group, you may request to be >> included in the expert chapters. >> >> The Python Goldmine is at: >> >> http://preciseinfo.org/Convert/index_Convert_Python.html >> >> -- >> Programmer's Goldmine collections: >> >> http://preciseinfo.org >> >> Tens of thousands of code examples and expert discussions on >> C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, >> organized by major topics of language, tools, methods, techniques. > > This site pops up spam windowns. One was blocked, one managed to bypass > the popup blocker. Tnis is not friendly behaviour. > > Some categories have 100s of entries. Better, I think, to use a search > engine such as Google with more specific search terms and a snippet of > context for each result. > Because I habitually run the NoScript extension to Firefox the popups didn't appear, but there didn't seem to be any original content on this site. Google continues to be your friend. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Sun Jan 10 22:34:51 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 10 Jan 2010 22:34:51 -0500 Subject: Python Goldmine has been updated: http://preciseinfo.org/Convert/index_Convert_Python.html In-Reply-To: <4B4A9446.7020009@holdenweb.com> References: <4B4A9446.7020009@holdenweb.com> Message-ID: Steve Holden wrote: [...] > Because I habitually run the NoScript extension to Firefox the popups > didn't appear, but there didn't seem to be any original content on this > site. Google continues to be your friend. > And dammit, why didn't I think to strip the links out instead of creating yet one more link to it? Sorry ... regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From nobody at nowhere.com Sun Jan 10 22:36:17 2010 From: nobody at nowhere.com (Nobody) Date: Mon, 11 Jan 2010 03:36:17 +0000 Subject: lightweight encryption of text file References: <7xfx6dhpd0.fsf@ruckus.brouhaha.com> Message-ID: On Sun, 10 Jan 2010 08:54:51 -0800, Paul Rubin wrote: > Nobody writes: >> RC4 (aka ArcFour) is quite trivial to implement, and better than inventing >> your own cipher or using a Vignere: ... > > That's a cute implementation, but it has no authentication and doesn't > include any randomness, which means if you use the same key for two > inputs, there is a security failure (xor'ing the two ciphertexts reveals > the xor of the plaintexts). Right. RC4 is a cipher, not a cryptosystem. But, yeah, the OP needs to be aware of the difference (and probably isn't, yet). So to take that a step further ... The key passed to arcfour.schedule() shouldn't be re-used. If you need to encrypt multiple files, use a different key for each. If you want to encrypt multiple files with the same "password", generate a unique key by hashing a combination of the password and a random salt (e.g. from /dev/random), and prepend the salt to the beginning of the stream. To decrypt, extract the salt from the stream to generate the key. If you need to verify the data, append a hash of the ciphertext (a hash of the plaintext would allow an attacker to confirm a guessed plaintext or to confirm that two files contain the same plaintext). Stream ciphers are vulnerable to replacement attacks: (p1 xor r) xor (p1 xor p2) == (p2 xor r) So if you can guess any part of the plaintext p1, you can replace it with alternative plaintext p2 without needing to decrypt/encrypt or knowing anything about the pad r. Also, if this is for something important, I'd be concerned about how to protect the key. That's hard enough to do in C, let alone in Python. > It also looks rather slow. Any kind of bulk binary data processing in pure Python is slow. The code was written mainly for simplicity, e.g. using generators means that you don't have to deal with buffer sizes. Replacing " % 256" with " & 255" might be worthwhile. > I don't make > any guarantees about p3.py, but it has been reviewed by several experts > and appears to be reasonably sound for the type of casual use being > discussed here, and it is tuned for speed (given the implementation > constraints). For more demanding purposes, you should use a more > serious library like one of the OpenSSL wrappers. The OP specifically wanted to avoid third-party libraries. From nobody at nowhere.com Sun Jan 10 22:44:26 2010 From: nobody at nowhere.com (Nobody) Date: Mon, 11 Jan 2010 03:44:26 +0000 Subject: lightweight encryption of text file References: <7xiqb9yaea.fsf@ruckus.brouhaha.com> Message-ID: On Sun, 10 Jan 2010 12:26:05 -0800, Paul Rubin wrote: > I'd like it a lot if the Python stdlib could include a serious > cryptography module. And I'd like a truckload of gold ;) Right now, even asking for HTTPS support is too much to ask. Heck, even asking for the fake HTTPS support to be identified as such is too much, apparently. From steve at holdenweb.com Sun Jan 10 22:52:24 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 10 Jan 2010 22:52:24 -0500 Subject: lightweight encryption of text file In-Reply-To: References: <7xiqb9yaea.fsf@ruckus.brouhaha.com> Message-ID: Nobody wrote: > On Sun, 10 Jan 2010 12:26:05 -0800, Paul Rubin wrote: > >> I'd like it a lot if the Python stdlib could include a serious >> cryptography module. > > And I'd like a truckload of gold ;) > > Right now, even asking for HTTPS support is too much to ask. Heck, > even asking for the fake HTTPS support to be identified as such is too > much, apparently. > No, Paul, nobody will complain if you *ask* ... A question I've been asking myself quite a lot recently is how the PSF could, were funding to be available, direct the development of Python in specific directions, and what those directions should be. Unfortunately there are probably as many answers as Python programmers in terms of the priorities to be adopted. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From austyn at gmail.com Sun Jan 10 23:04:18 2010 From: austyn at gmail.com (Austyn) Date: Sun, 10 Jan 2010 20:04:18 -0800 (PST) Subject: Fractional Hours from datetime? References: Message-ID: <3ca2bf74-744d-421e-a78e-38dbeac2cda1@m3g2000yqf.googlegroups.com> How about: import time arizona_utc_offset = -7.00 h = (time.time() / 3600 + arizona_utc_offset) % 24 dt.timetuple()[6] is the day of the week; struct tm_time doesn't include a sub-second field. On Jan 10, 10:28?am, "W. eWatson" wrote: > Maybe there's a more elegant way to do this. I want to express the > result of datetime.datetime.now() in fractional hours. > > Here's one way. > > dt=datetime.datetime.now() > xtup = dt.timetuple() > h = xtup[3]+xtup[4]/60.0+xtup[5]/3600.00+xtup[6]/10**6 > # ?now is in fractions of an hour From Dave Sun Jan 10 23:16:03 2010 From: Dave (Dave) Date: Sun, 10 Jan 2010 22:16:03 -0600 Subject: interactive terminal in Ubuntu Linux : libreadline5-dev works only in Python 2.6 not 3.1 References: <23c7de94-44ba-4567-b31f-8a5bb656e9e9@m3g2000yqf.googlegroups.com> <15a78d94-34d5-4bef-8344-6dc8843de4e5@k19g2000yqc.googlegroups.com> Message-ID: On Sat, 9 Jan 2010 16:48:52 -0800 (PST), casevh wrote: >On Jan 9, 3:10?pm, pdlem... at earthlink.net wrote: >> On Sat, 9 Jan 2010 13:27:07 -0800 (PST), casevh >> wrote: > >1) Try the commands again. Make sure all the "./configure" options are >on one line. Make sure to do "sudo make altinstall". (Don't use "sudo >make install"; it will give your version of Python the name "python" >and that can cause confusion on your system.) > >2) Move your applications to another directory. > >3) Try running "python3.1" while you are in that directory. > >If this doesn't work, report back on the error messages you receive. > Thanks casevh for your time & patience. The ./configure . . . was a "continuous" line , although it ran over to next line even on 132 char wide terminal because of names of system & dir. I was careful with the spaces and hyphens. In my reply the "make install" was a typo , I did run make altinstall. Moved all my code to pycode dir on my home directory. Removed all files from /usr/local/lib/python3.1/dlmodules and removed that dir. Twice ran the recompile : make distclean ./configure --prefix=/usr/local --with-computed-gotos --with-wide-unicode <^ one space> make sudo make altinstall After each reinstall had same problems : cannot import random or any code using it, although random.py is in Lib : Traceback File "", line 1, in File "random.py", line 46, in import collections as _collections File "collections.py", line 9 in from _collections import deque, default dict ImportError: /usr/local/lib/python3.1/lib-dynload/collections.so: undefined symbol: PyUnicodeUCS4_FromString After second reinstall today I also found that a module importing "time" would not run. Likewise could not import time at >>> . Same error, ending : undefined symbol: PyUnicode UCS4_FromString And my original problem still there : fouled up keys in interactive terminal. Seems minor now ; ) Should I try to remove everything and reopen the tarball ? Dave WB3DWE, central Texas From no.email at nospam.invalid Sun Jan 10 23:22:50 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 10 Jan 2010 20:22:50 -0800 Subject: lightweight encryption of text file References: <7xiqb9yaea.fsf@ruckus.brouhaha.com> Message-ID: <7xmy0li82t.fsf@ruckus.brouhaha.com> Steve Holden writes: >> Right now, even asking for HTTPS support is too much to ask. Heck, >> even asking for the fake HTTPS support to be identified as such is too >> much, apparently. >> > No, Paul, nobody will complain if you *ask* ... Er, that wasn't me... > A question I've been asking myself quite a lot recently is how the PSF > could, were funding to be available, direct the development of Python in > specific directions,... Crypto in the stdlib is not a matter of funding or (technical) priorities. It's a policy issue; the maintainers were (at least as of a few years ago) concerned about legal restrictions on crypto in some jurisdictions causing problems with distributing Python if it included crypto. I know that other systems (Java, Mozilla, etc) include crypto but they may have had to jump through some hoops for that purpose. From austyn at gmail.com Sun Jan 10 23:23:19 2010 From: austyn at gmail.com (Austyn) Date: Sun, 10 Jan 2010 20:23:19 -0800 (PST) Subject: Fractional Hours from datetime? References: <3ca2bf74-744d-421e-a78e-38dbeac2cda1@m3g2000yqf.googlegroups.com> Message-ID: <05ca0e15-b55e-4ede-81fe-c8f4459e3932@r5g2000yqb.googlegroups.com> Here's an improvement in case you want your code to work outside of Arizona: from time import time, timezone h = ((time() - timezone) / 3600) % 24 On Jan 10, 9:04?pm, Austyn wrote: > How about: > > import time > arizona_utc_offset = -7.00 > h = (time.time() / 3600 + arizona_utc_offset) % 24 > > dt.timetuple()[6] is the day of the week; struct tm_time doesn't > include a sub-second field. > > On Jan 10, 10:28?am, "W. eWatson" wrote: > > > > > Maybe there's a more elegant way to do this. I want to express the > > result of datetime.datetime.now() in fractional hours. > > > Here's one way. > > > dt=datetime.datetime.now() > > xtup = dt.timetuple() > > h = xtup[3]+xtup[4]/60.0+xtup[5]/3600.00+xtup[6]/10**6 > > # ?now is in fractions of an hour From no.email at nospam.invalid Sun Jan 10 23:27:55 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 10 Jan 2010 20:27:55 -0800 Subject: lightweight encryption of text file References: <7xfx6dhpd0.fsf@ruckus.brouhaha.com> Message-ID: <7xiqb9i7uc.fsf@ruckus.brouhaha.com> Nobody writes: > But, yeah, the OP needs to be aware of the difference (and probably isn't, > yet). So to take that a step further ... > The key passed to arcfour.schedule() shouldn't be re-used.... > If you need to verify the data, append a hash of the ciphertext ... > If you want to encrypt multiple files with the same "password", > generate a unique key by hashing a combination of the password and a > random salt (e.g. from /dev/random)... Yep, a whole lot of stuff that's easy to get wrong if it's left to the user (for example, you mean "a MAC of the ciphertext", not "a hash of the ciphertext"), and a lot of extra work even if the user gets it right. It's simplest for the library to provide a single interface that does all of it. >> It also looks rather slow. > > Any kind of bulk binary data processing in pure Python is slow. Well, slow is a relative term, but p3.py is about 5x faster than the fastest pure-Python rc4 implementation that I compared it to. Its heavy lifting is done by the SHA and array modules, that are written in C. From aahz at pythoncraft.com Sun Jan 10 23:53:42 2010 From: aahz at pythoncraft.com (Aahz) Date: 10 Jan 2010 20:53:42 -0800 Subject: py2exe "for loop" hangs in compiled program References: Message-ID: In article , p_tierchen <1JoS at sms.at> wrote: > >the application is an interface to a sqlite database and stores image >metadata (such as year, event, photographer, people on image etc.). i >use pyqt4 for the interface and developed this application on a linux >platform (python 2.5.4). friends of mine liked what i have done an want >it for their windows computers. so i setup a test system (win XP and >python 2.6.?). copying the code to this system and starting it: works >as intended. however they complain python whats this, this is sooo >complicated etc. so i took refuge in using py2exe, planing to give them >a zip compressed archive ... testing this option i found out that on >the surface all looks nice (GUI and functions etc) however: extending >to "larger" datasets (>~50 images) some of the for loops stop after >some time. https://lists.sourceforge.net/lists/listinfo/py2exe-users -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." --Red Adair From aahz at pythoncraft.com Sun Jan 10 23:54:38 2010 From: aahz at pythoncraft.com (Aahz) Date: 10 Jan 2010 20:54:38 -0800 Subject: How to validate the __init__ parameters References: <803c4db7-3813-448a-b428-0ec11c57f098@t42g2000yqd.googlegroups.com> Message-ID: In article , Jean-Michel Pichavant wrote: > >class A: > def __init__(self, foo = None, bar = None): > if len(foo) > 5: > raise ValueError('foo cannot exceed 5 characters') Bad Idea -- what happens when foo is None? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." --Red Adair From steve at holdenweb.com Sun Jan 10 23:57:41 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 10 Jan 2010 23:57:41 -0500 Subject: lightweight encryption of text file In-Reply-To: <7xmy0li82t.fsf@ruckus.brouhaha.com> References: <7xiqb9yaea.fsf@ruckus.brouhaha.com> <7xmy0li82t.fsf@ruckus.brouhaha.com> Message-ID: Paul Rubin wrote: > Steve Holden writes: >>> Right now, even asking for HTTPS support is too much to ask. Heck, >>> even asking for the fake HTTPS support to be identified as such is too >>> much, apparently. >>> >> No, Paul, nobody will complain if you *ask* ... > > Er, that wasn't me... > Oh sorry, no more it was. >> A question I've been asking myself quite a lot recently is how the PSF >> could, were funding to be available, direct the development of Python in >> specific directions,... > > Crypto in the stdlib is not a matter of funding or (technical) > priorities. It's a policy issue; the maintainers were (at least as of a > few years ago) concerned about legal restrictions on crypto in some > jurisdictions causing problems with distributing Python if it included > crypto. I know that other systems (Java, Mozilla, etc) include crypto > but they may have had to jump through some hoops for that purpose. > There are administrative hoops to jump through still in the US to export cryptographic code. Further, if the standard distribution were to include it then the requirements of the US government do still come into play, and it would be a pain to have to have people downloading the software certify (for example) that they weren't intending to re-export it to a "prohibited" country. We can get around some of these issues by retaining the hosting on mainland Europe rather than in the USA, but these issues do demand careful study which apparently nobody really has time for at present. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From pavlovevidence at gmail.com Mon Jan 11 00:21:14 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Sun, 10 Jan 2010 21:21:14 -0800 (PST) Subject: lightweight encryption of text file References: Message-ID: <7a2fbce3-d4cf-437b-9c0c-40f975189353@s31g2000yqs.googlegroups.com> On Jan 8, 11:14?am, Daniel Fetchinson wrote: > I have a plain text file which I would like to protect in a very > simple minded, yet for my purposes sufficient, way. I'd like to > encrypt/convert it into a binary file in such a way that possession of > a password allows anyone to convert it back into the original text > file while not possessing the password one would only see the > following with the standard linux utility 'file': > > [fetchinson at fetch ~]$ file encrypted.data > encrypted.data: data > > and the effort required to convert the file back to the original text > file without the password would be equivalent to guessing the > password. gpg -c simpletextfile.txt -o simpletextfile.gpg But I guess you can't depend on users to have gpg installed so you have to roll out some unvetted Python tool. Carl Banks From debatem1 at gmail.com Mon Jan 11 00:31:23 2010 From: debatem1 at gmail.com (geremy condra) Date: Mon, 11 Jan 2010 00:31:23 -0500 Subject: lightweight encryption of text file In-Reply-To: <7xiqb9yaea.fsf@ruckus.brouhaha.com> References: <7xiqb9yaea.fsf@ruckus.brouhaha.com> Message-ID: On Sun, Jan 10, 2010 at 3:26 PM, Paul Rubin wrote: > geremy condra writes: >> Not sure why in the world you would homebrew something like this- a >> small dependency isn't that bad, and aes can be pretty simple to use. >> Might as well go for the industrial strength approach. > > In my experience, 1) small dependencies ARE that bad, since they mean > you have to develop and test on every platform that you want your code > to run on; And having no dependencies frees you from the burden of testing where your software will be deployed? I don't think so. > 2) using a serious library requires quite a bit of knowledge > and decision-making which not everyone is equipped to do. Homebrewing is not a good solution to the problem of being ignorant of modern cryptography. >?"AES" is not so simple to use unless you know what you're doing in > terms of modes, nonces, etc. Seems pretty simple to me- use AES 192, don't use ECB mode, and use your library of choice's key strengthening utilities. Even blatantly ignoring that advice would still probably give you better results than homebrewing though, so I don't really see the issue here. > Having supported this kind of package in a commercial > setting in the past, IMO, for the sort of (common) application in > question, it's best to keep things as simple as possible and supply a > single interface that provides encryption, authentication, and random > initialization all in one call. ?The cost is a little bit of ciphertext > bloat, but it prevents all kinds of security failures frequently > overlooked by novices. > > I'd like it a lot if the Python stdlib could include a serious > cryptography module. ?That was rejected for regulatory reasons several > years ago, but maybe things are changing enough that the issue can be > revisited sometime. I agree. I inquired about it not too long ago on python-ideas; little serious discussion ensued. Geremy Condra From steve at holdenweb.com Mon Jan 11 00:54:04 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 11 Jan 2010 00:54:04 -0500 Subject: lightweight encryption of text file In-Reply-To: <7a2fbce3-d4cf-437b-9c0c-40f975189353@s31g2000yqs.googlegroups.com> References: <7a2fbce3-d4cf-437b-9c0c-40f975189353@s31g2000yqs.googlegroups.com> Message-ID: Carl Banks wrote: > On Jan 8, 11:14 am, Daniel Fetchinson > wrote: >> I have a plain text file which I would like to protect in a very >> simple minded, yet for my purposes sufficient, way. I'd like to >> encrypt/convert it into a binary file in such a way that possession of >> a password allows anyone to convert it back into the original text >> file while not possessing the password one would only see the >> following with the standard linux utility 'file': >> >> [fetchinson at fetch ~]$ file encrypted.data >> encrypted.data: data >> >> and the effort required to convert the file back to the original text >> file without the password would be equivalent to guessing the >> password. > > > gpg -c simpletextfile.txt -o simpletextfile.gpg > > But I guess you can't depend on users to have gpg installed so you > have to roll out some unvetted Python tool. > > The OP's statement of requirements would be pretty much satisfied by the "crypt" utility. He can even run it under Cygwin on Windows if necessary. Cryptographic sophistication (or even cryptographic security) was not requested (and would not be provided anyway by most of the suggested solutions). If any real protection is required then an ad-hoc program is definitely not the way to provide it. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From casevh at gmail.com Mon Jan 11 01:08:20 2010 From: casevh at gmail.com (casevh) Date: Sun, 10 Jan 2010 22:08:20 -0800 (PST) Subject: interactive terminal in Ubuntu Linux : libreadline5-dev works only in Python 2.6 not 3.1 References: <23c7de94-44ba-4567-b31f-8a5bb656e9e9@m3g2000yqf.googlegroups.com> <15a78d94-34d5-4bef-8344-6dc8843de4e5@k19g2000yqc.googlegroups.com> Message-ID: <43ee50ae-7a0e-4c93-b844-8bb6fb1ea07a@u41g2000yqe.googlegroups.com> On Jan 10, 8:16?pm, Dave WB3DWE wrote: > On Sat, 9 Jan 2010 16:48:52 -0800 (PST), casevh > wrote: > > >On Jan 9, 3:10?pm, pdlem... at earthlink.net wrote: > >> On Sat, 9 Jan 2010 13:27:07 -0800 (PST), casevh > >> wrote: > > >1) Try the commands again. Make sure all the "./configure" options are > >on one line. Make sure to do "sudo make altinstall". (Don't use "sudo > >make install"; it will give your version of Python the name "python" > >and that can cause confusion on your system.) > > >2) Move your applications to another directory. > > >3) Try running "python3.1" while you are in that directory. > > >If this doesn't work, report back on the error messages you receive. > > Thanks casevh for your time & patience. > The ./configure . . . ?was a "continuous" line , although it ran over to > next line even on 132 char wide terminal because of names of system & > dir. ?I was careful with the spaces and hyphens. > In my reply the "make install" was a typo , I did run ?make altinstall. > > Moved all my code to pycode dir on my home directory. ?Removed all > files from /usr/local/lib/python3.1/dlmodules and removed that dir. > > Twice ran the recompile : > ? ? make distclean > ? ? ./configure --prefix=/usr/local --with-computed-gotos > ? ? ? ? ? ? --with-wide-unicode > ? ? ? ? <^ one space> > ? ? make > ? ? sudo make altinstall > After each reinstall had same problems : ?cannot import ?random ?or > any code using it, although random.py is in Lib : > ? ? Traceback > ? ? ? ? File "", line 1, in > ? ? ? ? File "random.py", line 46, in > ? ? ? ? ? ? import collections as _collections > ? ? ? ? File "collections.py", line 9 in > ? ? ? ? ? ? from _collections import deque, default dict > ? ? ImportError: /usr/local/lib/python3.1/lib-dynload/collections.so: > ? ? ? ? undefined symbol: PyUnicodeUCS4_FromString > > After second reinstall today I also found that a module importing > ? ? "time" ?would not run. ?Likewise could not import ?time ?at ?>>> . > Same error, ending : undefined symbol: PyUnicode UCS4_FromString > > And my original problem still there : fouled up keys in interactive > terminal. Seems minor now ; ) ? Should I try to remove everything > and reopen the tarball ? > Dave WB3DWE, ?central Texas Are you sure you are using the new version of python3.1 (the one located in /usr/local/bin/)? What is the result of "which python3.1"? What happens if you run "/usr/local/bin/python3.1"? casevh From sebastian.langer at gmx.de Mon Jan 11 01:21:54 2010 From: sebastian.langer at gmx.de (Sebastian) Date: Sun, 10 Jan 2010 22:21:54 -0800 (PST) Subject: how to duplicate array entries Message-ID: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> Hi there, I have an array x=[1,2,3] Is there an operator which I can use to get the result [1,1,1,2,2,2,3,3,3] ? I tried x*3, which resulted in [1,2,3,1,2,3,1,2,3] I also tried [[b,b,b] for b in x] which led to [[1,2,3],[1,2,3], [1,2,3]], but this isn't what I want either. Cheers, Sebastian From sebastian.langer at gmx.de Mon Jan 11 01:24:07 2010 From: sebastian.langer at gmx.de (Sebastian) Date: Sun, 10 Jan 2010 22:24:07 -0800 (PST) Subject: how to duplicate array entries References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> Message-ID: <2f4fa997-b044-426c-8600-007e51c25c32@h9g2000yqa.googlegroups.com> On Jan 11, 4:21 pm, Sebastian wrote: > I also tried [[b,b,b] for b in x] which led to [[1,2,3],[1,2,3], > [1,2,3]] Sorry, I have to correct myself. The quoted line above resulted in [[1,1,1],[2,2,2],[3,3,3]] of course! Cheers, Sebastian From clp2 at rebertia.com Mon Jan 11 01:30:04 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 10 Jan 2010 22:30:04 -0800 Subject: how to duplicate array entries In-Reply-To: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> Message-ID: <50697b2c1001102230l4d547e5cxded09801daa0261a@mail.gmail.com> On Sun, Jan 10, 2010 at 10:21 PM, Sebastian wrote: > Hi there, > > I have an array ?x=[1,2,3] > > Is there an operator which I can use to get the result > [1,1,1,2,2,2,3,3,3] ? > > I tried x*3, which resulted in [1,2,3,1,2,3,1,2,3] > I also tried [[b,b,b] for b in x] which led to [[1,2,3],[1,2,3], > [1,2,3]], but this isn't what I want either. from itertools import chain, repeat n = 3 stretched = list(chain(*[repeat(item, n) for item in x])) Cheers, Chris -- http://blog.rebertia.com From paul.nospam at rudin.co.uk Mon Jan 11 02:07:07 2010 From: paul.nospam at rudin.co.uk (Paul Rudin) Date: Mon, 11 Jan 2010 07:07:07 +0000 Subject: how to duplicate array entries References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> Message-ID: <87ocl1ce78.fsf@rudin.co.uk> Sebastian writes: > Hi there, > > I have an array x=[1,2,3] In python such an object is called a "list". (In cpython it's implemented as an automatically resizable array.) > > Is there an operator which I can use to get the result > [1,1,1,2,2,2,3,3,3] ? There's no operator that will give you that directly - but there are plenty of one-liners that will yield that list. e.g: >>> list(itertools.chain(*([x]*3 for x in [1,2,3]))) [1, 1, 1, 2, 2, 2, 3, 3, 3] From danb_83 at yahoo.com Mon Jan 11 02:13:55 2010 From: danb_83 at yahoo.com (Dan Bishop) Date: Sun, 10 Jan 2010 23:13:55 -0800 (PST) Subject: integer and string compare, is that correct? References: Message-ID: <2f718029-1682-47f6-a2c3-6e40b51bd71a@v25g2000yqk.googlegroups.com> On Jan 10, 10:34?am, Nobody wrote: > Hellmut Weber wrote: > >> being a causal python user (who likes the language quite a lot) > >> it took me a while to realize the following: > >> ?>>> max = '5' > >> ?>>> n = 5 > >> ?>>> n >= max > >> False > > >> Section 5.9 Comparison describes this. > > >> Can someone give me examples of use cases > Peter Otten wrote: > > The use cases for an order that works across types like int and str are weak > > to non-existent. Implementing it was considered a mistake and has been fixed > > in Python 3: > >>>> 5 > "5" > > Traceback (most recent call last): > > ? File "", line 1, in > > TypeError: unorderable types: int() > str() > > If you actually need to perform comparisons across types, you can rely > upon the fact that tuple comparisons are non-strict and use e.g.: > > ? ? ? ? > a = 5 > ? ? ? ? > b = '5' > ? ? ? ? > (type(a).__name__, a) < (type(b).__name__, b) > ? ? ? ? True > ? ? ? ? > (type(a).__name__, a) > (type(b).__name__, b) > ? ? ? ? False > > The second elements will only be compared if the first elements are equal > (i.e. the values have the same type). But this method gives you 3.0 < 2 because 'float' < 'int'. Probably not what you want. From gherron at islandtraining.com Mon Jan 11 02:20:52 2010 From: gherron at islandtraining.com (Gary Herron) Date: Sun, 10 Jan 2010 23:20:52 -0800 Subject: how to duplicate array entries In-Reply-To: <87ocl1ce78.fsf@rudin.co.uk> References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <87ocl1ce78.fsf@rudin.co.uk> Message-ID: <4B4AD154.1020501@islandtraining.com> Paul Rudin wrote: > Sebastian writes: > > >> Hi there, >> >> I have an array x=[1,2,3] >> > > In python such an object is called a "list". > > (In cpython it's implemented as an automatically resizable array.) > > >> Is there an operator which I can use to get the result >> [1,1,1,2,2,2,3,3,3] ? >> > > There's no operator that will give you that directly - but there are > plenty of one-liners that will yield that list. > e.g: > > >>>> list(itertools.chain(*([x]*3 for x in [1,2,3]))) >>>> > [1, 1, 1, 2, 2, 2, 3, 3, 3] > List comprehension also works nicely for this problem, and may be clearer to some. >>> x = [1,2,3] >>> print [i for i in x for k in range(3)] [1, 1, 1, 2, 2, 2, 3, 3, 3] Gary Herron From steven at REMOVE.THIS.cybersource.com.au Mon Jan 11 02:34:04 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 11 Jan 2010 07:34:04 GMT Subject: how to duplicate array entries References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> Message-ID: On Sun, 10 Jan 2010 22:21:54 -0800, Sebastian wrote: > Hi there, > > I have an array x=[1,2,3] You have a list. Python has an array type, but you have to "import array" to use it. > Is there an operator which I can use to get the result > [1,1,1,2,2,2,3,3,3] ? Not an operator, but you can do it easily with a function. Here's the simple version: >>> def duplicate(items, count): ... L = [] ... for item in items: ... L.extend([item]*count) ... return L ... >>> duplicate([1,2,3], 3) [1, 1, 1, 2, 2, 2, 3, 3, 3] Here's a version which is short and simple enough to use in-line, but will be slow for large lists: >>> x = [1,2,3] >>> count = 3 >>> sum([[item]*count for item in x], []) [1, 1, 1, 2, 2, 2, 3, 3, 3] Finally, here's a nasty hack that you should never, ever, ever use for any reason except to win a bet: >>> [locals()['_[1]'].extend([item]*(count-1)) or item for item in x] [1, 1, 1, 2, 2, 2, 3, 3, 3] -- Steven From marco.salden at gmail.com Mon Jan 11 02:47:34 2010 From: marco.salden at gmail.com (Marco Salden) Date: Sun, 10 Jan 2010 23:47:34 -0800 (PST) Subject: "Advanced" Python programming book? References: <4b49d7a8$1@dnews.tpgi.com.au> Message-ID: <5db2ed03-2669-4b74-ab29-47692001ff34@b2g2000yqi.googlegroups.com> On Jan 10, 2:35?pm, flow wrote: > I've just finished reading a sort of beginner Python book, and I know > quite a bit now but I'm looking for a book that can teach me advanced > aspects of Python - code optimisation, threading, etc. > > Any recommendations? > > Cheers. I like to add this one, which is nice to read just after reading the "starters", covers all kinds of topics, and written by someone who knows Python (I think at least). http://oreilly.com/catalog/9780596009250/ HTH, Marco From ben+python at benfinney.id.au Mon Jan 11 02:48:03 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 11 Jan 2010 18:48:03 +1100 Subject: class viewer ? References: Message-ID: <87aawlt74c.fsf@benfinney.id.au> Stef Mientki writes: > from the given class, it's ancestors and it's derived classes, > I'ld like to get the following information in a tree like structure: > > - the file were the class is definied > - the attributes, split in inherited / created / overriden > - the methodes, split in inherited / created / overriden > - the files were instances of the class are created > - and probably I forget a few > > any suggestions ? The ?Emacs Code Browser? was seemingly designed for languages like C++ and Java, but it serves well enough for Python also. I find it annoying to use, since it seems to assume mouse navigation and has rather impenetrable (for me) documentation. But it does seem to do much of what you're asking for, and a big plus is that it's not specific to Python. -- \ ?The most common way people give up their power is by thinking | `\ they don't have any.? ?Alice Walker | _o__) | Ben Finney From marco.salden at gmail.com Mon Jan 11 02:50:21 2010 From: marco.salden at gmail.com (Marco Salden) Date: Sun, 10 Jan 2010 23:50:21 -0800 (PST) Subject: integer and string compare, is that correct? References: Message-ID: <7b3e40df-4b6a-41dc-95ff-f3b7ca14b6ac@b2g2000yqi.googlegroups.com> On Jan 10, 1:26?pm, Hellmut Weber wrote: > Hi, > being a causal python user (who likes the language quite a lot) > it took me a while to realize the following: > > leo at sylvester py_count $ python > Python 2.6.3 (r263:75183, Oct 26 2009, 12:34:23) > [GCC 4.4.1] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > ?>>> max = '5' > ?>>> n = 5 > ?>>> n >= max > False > ?>>> n + max > Traceback (most recent call last): > ? ?File "", line 1, in > TypeError: unsupported operand type(s) for +: 'int' and 'str' > ?>>> > > Section 5.9 Comparison describes this. > > Can someone give me examples of use cases > > TIA > > Hellmut > > -- > Dr. Hellmut Weber ? ? ? ? m... at hellmutweber.de > Degenfeldstra?e 2 ? ? ? ? tel ? +49-89-3081172 > D-80803 M?nchen-Schwabing mobil +49-172-8450321 > please: No DOCs, no PPTs. why: tinyurl.com/cbgq I would say you want to compare semantically an integer value with an integer value so why not: IDLE 1.1.3 >>> max = '5' >>> n = 5 >>> n==(int(max)) True >>> ? (in Python 2.4...) Regards, Marco From alfps at start.no Mon Jan 11 02:56:36 2010 From: alfps at start.no (Alf P. Steinbach) Date: Mon, 11 Jan 2010 08:56:36 +0100 Subject: how to duplicate array entries In-Reply-To: <87ocl1ce78.fsf@rudin.co.uk> References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <87ocl1ce78.fsf@rudin.co.uk> Message-ID: * Paul Rudin: > Sebastian writes: > >> I have an array x=[1,2,3] > > In python such an object is called a "list". > > (In cpython it's implemented as an automatically resizable array.) I don't think the OP's terminology needs correction. A Python "list" is an array functionality-wise. If one isn't observant of that fact then one ends up with O(n^2) time for the simplest things. Using the term "array" accentuates and clarifies this most important aspect. Using the misleading term "list", even if that's the type's name in Python, hides this most important aspect, and so is not, IMHO, a Good Idea except where it really matters that it's a 'list' array as opposed to, say, a 'tuple' array. >> Is there an operator which I can use to get the result >> [1,1,1,2,2,2,3,3,3] ? > > There's no operator that will give you that directly - but there are > plenty of one-liners that will yield that list. > e.g: > >>>> list(itertools.chain(*([x]*3 for x in [1,2,3]))) > [1, 1, 1, 2, 2, 2, 3, 3, 3] And I think it's worth noting that, for the general case, this notation is also hiding a gross inefficiency, first constructing sub-arrays and then copying them and joining them. It doesn't even buy clarity. So, just >>> def repeat_items_in( s, n ): ... a = [] ... for item in s: ... for i in range( n ): ... a.append( item ) ... return a ... >>> repeat_items_in( [1, 2, 3], 3 ) [1, 1, 1, 2, 2, 2, 3, 3, 3] >>> _ And if one absolutely feels like trading some efficiency and clarity for some more functional-programming expression like thing (I don't understand why people desire that!), just replace the 'append' line with a 'yield' and then write list( repeat_items_in( [1, 2, 3], 3 ) ) Re the thing I don't understand: it's the same in C++, people using hours on figuring out how to do something very simple in an ungrokkable indirect and "compiled" way using template metaprogramming stuff, when they could just write a simple 'for' loop and be done with in, say, 3 seconds, and much clearer too! Cheers, - Alf From steven at REMOVE.THIS.cybersource.com.au Mon Jan 11 03:26:05 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 11 Jan 2010 08:26:05 GMT Subject: how to duplicate array entries References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <87ocl1ce78.fsf@rudin.co.uk> Message-ID: On Mon, 11 Jan 2010 08:56:36 +0100, Alf P. Steinbach wrote: > * Paul Rudin: >> Sebastian writes: >> >>> I have an array x=[1,2,3] >> >> In python such an object is called a "list". >> >> (In cpython it's implemented as an automatically resizable array.) > > I don't think the OP's terminology needs correction. > > A Python "list" is an array functionality-wise. > > If one isn't observant of that fact then one ends up with O(n^2) time > for the simplest things. Well that's certainly not true. Some operations may be O(N**2), but others are not: list.append() is amortized O(N) and for individual appends, may be can be as fast as O(1). > Using the term "array" accentuates and clarifies this most important > aspect. But Python lists are designed to behave as lists. Just because CPython implements them using arrays doesn't make them arrays. Other Python implementations might use other implementations... If the creator of CLPython is out there, perhaps might like to comment on whether he uses Lisp linked-lists for the Python list type? > Using the misleading term "list", even if that's the type's name in > Python, hides this most important aspect, and so is not, IMHO, a Good > Idea except where it really matters that it's a 'list' array as opposed > to, say, a 'tuple' array. Or an "array" array. >>> from array import array >>> array >>> Is there an operator which I can use to get the result >>> [1,1,1,2,2,2,3,3,3] ? >> >> There's no operator that will give you that directly - but there are >> plenty of one-liners that will yield that list. e.g: >> >>>>> list(itertools.chain(*([x]*3 for x in [1,2,3]))) >> [1, 1, 1, 2, 2, 2, 3, 3, 3] > > And I think it's worth noting that, for the general case, this notation > is also hiding a gross inefficiency, first constructing sub-arrays and > then copying them and joining them. I wouldn't call that a gross inefficiency -- that's a gross exaggeration unless count is very large, and even then, possibly not that large. Only one such sub-array (sub-list) exists at any time. (Unless I am grossly misinformed.) > It doesn't even buy clarity. Not if you're unused to the functional, iterator-based idioms, no. But if you are, it does. > And if one absolutely feels like trading some efficiency and clarity for > some more functional-programming expression like thing (I don't > understand why people desire that!), I don't understand why you assume that functional forms are necessarily less efficient and clear than non-functional. Which is easier to read? >>> print sum([1,2,3]) 6 versus >>> total = 0 >>> for i in [1, 2, 3]: ... total += i ... >>> print total 6 [...] > Re the thing I don't understand: it's the same in C++, people using > hours on figuring out how to do something very simple in an ungrokkable > indirect and "compiled" way using template metaprogramming stuff, when > they could just write a simple 'for' loop and be done with in, say, 3 > seconds, and much clearer too! Amen to that brother! It's the obsession with one-liners and the desire for a single built-in command to do every imaginable task. -- Steven From sebastian.langer at gmx.de Mon Jan 11 03:39:10 2010 From: sebastian.langer at gmx.de (Sebastian) Date: Mon, 11 Jan 2010 00:39:10 -0800 (PST) Subject: how to duplicate array entries References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <87ocl1ce78.fsf@rudin.co.uk> Message-ID: Thank you for your answers! I actually implemented it using for loops before I posted here, but I was curious if there is a more elegant solution (judging from the post, Alf will probably say, that for loops are already elegant). Sebastian From frank at chagford.com Mon Jan 11 03:51:13 2010 From: frank at chagford.com (Frank Millman) Date: Mon, 11 Jan 2010 10:51:13 +0200 Subject: TypeError: __name__ must be set to a string object Message-ID: Hi all This problem is similar to one I posted recently regarding the multiprocessing module and unicode. However, although this one manifests itself while using the multiprocessing module, is caused by Python itself (2.6.2). At the top of my program I have 'from __future__ import unicode_literals'. The relevant lines from my program read - from multiprocessing.managers import BaseManager class MyManager(BaseManager): pass MyManager.register('my_function', my_function) Inside the multiprocessing module, the following lines are executed - @classmethod def register(cls, typeid, ...) [...] def temp(...): [...] temp.__name__ = typeid At this point, Python raises the exception 'TypeError: __name__ must be set to a string object'. I can fix it by changing my last line to - MyManager.register(str('my_function'), my_function) Is there any reason why __name__ cannot be a unicode object in Python 2.x? If so, there is probably little chance of this being changed, so it is probably not worth reporting. Any thoughts? Frank Millman From cloudgiant at gmail.com Mon Jan 11 03:56:46 2010 From: cloudgiant at gmail.com (Munir) Date: Mon, 11 Jan 2010 00:56:46 -0800 (PST) Subject: how to duplicate array entries References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> Message-ID: <14437829-e3ef-41ac-b533-bc7003c1434b@j19g2000yqk.googlegroups.com> > I have an array ?x=[1,2,3] > > Is there an operator which I can use to get the result > [1,1,1,2,2,2,3,3,3] ? > > I tried x*3, which resulted in [1,2,3,1,2,3,1,2,3] Have you tried: y = x*3 y.sort() Munir From __peter__ at web.de Mon Jan 11 03:57:30 2010 From: __peter__ at web.de (Peter Otten) Date: Mon, 11 Jan 2010 09:57:30 +0100 Subject: how to duplicate array entries References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <87ocl1ce78.fsf@rudin.co.uk> Message-ID: Alf P. Steinbach wrote: > Re the thing I don't understand: it's the same in C++, people using hours > on figuring out how to do something very simple in an ungrokkable indirect > and "compiled" way using template metaprogramming stuff, when they could > just write a simple 'for' loop and be done with in, say, 3 seconds, and > much clearer too! Most of that stuff doesn't end in code meant to do anything important. It's more like gymnastics that helps you keep your mind in shape. Or so I would hope. >>> items = [1, 2, 3] >>> result = 3*len(items)*[None] >>> result[::3] = result[1::3] = result[2::3] = items >>> result [1, 1, 1, 2, 2, 2, 3, 3, 3] >>> from itertools import * >>> list(chain.from_iterable(starmap(repeat, izip(items, repeat(3))))) [1, 1, 1, 2, 2, 2, 3, 3, 3] ;) Peter From alfps at start.no Mon Jan 11 04:03:04 2010 From: alfps at start.no (Alf P. Steinbach) Date: Mon, 11 Jan 2010 10:03:04 +0100 Subject: how to duplicate array entries In-Reply-To: References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <87ocl1ce78.fsf@rudin.co.uk> Message-ID: * Steven D'Aprano: > On Mon, 11 Jan 2010 08:56:36 +0100, Alf P. Steinbach wrote: > >> * Paul Rudin: >>> Sebastian writes: >>> >>>> I have an array x=[1,2,3] >>> In python such an object is called a "list". >>> >>> (In cpython it's implemented as an automatically resizable array.) >> I don't think the OP's terminology needs correction. >> >> A Python "list" is an array functionality-wise. >> >> If one isn't observant of that fact then one ends up with O(n^2) time >> for the simplest things. > > Well that's certainly not true. Some operations may be O(N**2), but > others are not: list.append() is amortized O(N) and for individual > appends, may be can be as fast as O(1). The second sentence may or may not be true. I don't know of any fundamental 'list' operations that have quadratic time. Is there? The first sentence is just baffling -- what on Earth is the "that" that you think is not true? OK, I can guess (correct me if I'm guessing wrong, please): you think I'm talking about elementary operations. I'm not. I'm talking about algorithmic complexity for loops doing e.g. insertions. >> Using the term "array" accentuates and clarifies this most important >> aspect. > > But Python lists are designed to behave as lists. No, I'm sorry, they're not. A Python 'list' has de facto constant time indexing, or "random access". A linked list -- what the informal "list" means in programming -- does not have constant time indexing. A linked list has constant time insertion. A Python 'list' has de facto linear time insertion (except when used as cursor gap array, of course, or e.g. implementing a linked list on top, such things). So in short, a Python 'list' has all the properties of arrays, and none of the properties of linked lists. > Just because CPython > implements them using arrays doesn't make them arrays. Other Python > implementations might use other implementations... No, I'm sorry, not without screwing up existing Python programs. Indexing is assumed as constant time in every CPython program. That is, in your own words, but here correct, that's "certainly not true". ;-) No (sensible) programming language allows a language implementation to change the running time of common loops from linear to quadratic. It would be decidedly un-pythonic. ;-) > If the creator of CLPython is out there, perhaps might like to comment on > whether he uses Lisp linked-lists for the Python list type? If he does then you're talking about a different language than the one that CPython implements: constant time indexing is very different from linear time. It doesn't matter if some bananas are called oranges. They don't turn into oranges no matter what they're called. >> Using the misleading term "list", even if that's the type's name in >> Python, hides this most important aspect, and so is not, IMHO, a Good >> Idea except where it really matters that it's a 'list' array as opposed >> to, say, a 'tuple' array. > > Or an "array" array. For example, yes. These different kinds of arrays have different restrictions: can't be used as dictionary key, can't be modified, has fixed item type. And when talking about such characteristics the type name can be relevant. >>>> from array import array >>>> array > > > > >>>> Is there an operator which I can use to get the result >>>> [1,1,1,2,2,2,3,3,3] ? >>> There's no operator that will give you that directly - but there are >>> plenty of one-liners that will yield that list. e.g: >>> >>>>>> list(itertools.chain(*([x]*3 for x in [1,2,3]))) >>> [1, 1, 1, 2, 2, 2, 3, 3, 3] >> And I think it's worth noting that, for the general case, this notation >> is also hiding a gross inefficiency, first constructing sub-arrays and >> then copying them and joining them. > > I wouldn't call that a gross inefficiency -- that's a gross exaggeration > unless count is very large, and even then, possibly not that large. Only > one such sub-array (sub-list) exists at any time. (Unless I am grossly > misinformed.) I'm sorry but to the best of my knowledge you're misinformed. Unless there's some pretty advanced lazy evaluation involved the * operator has to collect the subarrays into an array formal argument for the 'chain' routine. And at that point they all exist at the same time. >>> def knurre( *poff ): ... print( type( poff ) ) ... print( poff ) ... >>> a = [1, 2, 3] >>> knurre( *(3*[x] for x in a) ) ([1, 1, 1], [2, 2, 2], [3, 3, 3]) >>> _ >> It doesn't even buy clarity. > > Not if you're unused to the functional, iterator-based idioms, no. > > But if you are, it does. He he -- see above, with 99.x certainty you *misunderstood* the code. That's *not* clear code. That's, hereby (almost) proven :-), code that makes even experienced programmers misunderstand what's going on! >> And if one absolutely feels like trading some efficiency and clarity for >> some more functional-programming expression like thing (I don't >> understand why people desire that!), > > I don't understand why you assume that functional forms are necessarily > less efficient and clear than non-functional. Which is easier to read? > >>>> print sum([1,2,3]) > 6 > > versus > >>>> total = 0 >>>> for i in [1, 2, 3]: > ... total += i > ... >>>> print total > 6 No, here I very much agree with you: the first is bestest. :-) I like abstraction. But not for its own sake: there has to be some increase in clarity, some details that the abstraction removes from consideration, instead of introducing additional details for consideration! > [...] >> Re the thing I don't understand: it's the same in C++, people using >> hours on figuring out how to do something very simple in an ungrokkable >> indirect and "compiled" way using template metaprogramming stuff, when >> they could just write a simple 'for' loop and be done with in, say, 3 >> seconds, and much clearer too! > > Amen to that brother! > > It's the obsession with one-liners and the desire for a single built-in > command to do every imaginable task. He he... :-) Cheers, - Alf From drncode1 at live.com Mon Jan 11 04:06:38 2010 From: drncode1 at live.com (flow) Date: 11 Jan 2010 20:06:38 +1100 Subject: "Advanced" Python programming book? References: <4b49d7a8$1@dnews.tpgi.com.au> Message-ID: <4b4aea1e$1@dnews.tpgi.com.au> Cheers guys - I'll check the books out :) Thanks very much. From martin at v.loewis.de Mon Jan 11 04:09:36 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Mon, 11 Jan 2010 10:09:36 +0100 Subject: sys.stdout vs. sys.stderr In-Reply-To: References: Message-ID: > In Python 3.1 is there any difference in the buffering behavior of the > initial sys.stdout and sys.stderr streams? No. > Were they different at some earlier point in Python's evolution? That depends on the operating system. These used to be whatever the C library set up as stdout and stderr. Typically, they were buffered in the same way. Regards, Martin From clp2 at rebertia.com Mon Jan 11 04:21:40 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Mon, 11 Jan 2010 01:21:40 -0800 Subject: how to duplicate array entries In-Reply-To: References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <87ocl1ce78.fsf@rudin.co.uk> Message-ID: <50697b2c1001110121ua673e5ex446d5deaecad8043@mail.gmail.com> On Mon, Jan 11, 2010 at 1:03 AM, Alf P. Steinbach wrote: > * Steven D'Aprano: >> >> On Mon, 11 Jan 2010 08:56:36 +0100, Alf P. Steinbach wrote: >> >>> * Paul Rudin: >>>> >>>> Sebastian writes: >>>> >>>>> I have an array ?x=[1,2,3] >>>> >>>> In python such an object is called a "list". >>>> >>>> (In cpython it's implemented as an automatically resizable array.) >>> >>> I don't think the OP's terminology needs correction. >>> >>> A Python "list" is an array functionality-wise. >>> >>> If one isn't observant of that fact then one ends up with O(n^2) time >>> for the simplest things. >> >> Well that's certainly not true. Some operations may be O(N**2), but others >> are not: list.append() is amortized O(N) and for individual appends, may be >> can be as fast as O(1). > > The second sentence may or may not be true. I don't know of any fundamental > 'list' operations that have quadratic time. Is there? > > The first sentence is just baffling ?-- ?what on Earth is the "that" that > you think is not true? > > OK, I can guess (correct me if I'm guessing wrong, please): you think I'm > talking about elementary operations. I'm not. I'm talking about algorithmic > complexity for loops doing e.g. insertions. > > >>> Using the term "array" accentuates and clarifies this most important >>> aspect. >> >> But Python lists are designed to behave as lists. > > No, I'm sorry, they're not. > > A Python 'list' has de facto constant time indexing, or "random access". > > A linked list ?-- ?what the informal "list" means in programming Eh, it's a bit context-dependent. The abstract data type definition is a superset that includes both linked lists and dynamic arrays. FWIW, Java likewise uses "list" in its ADT sense. Cheers, Chris -- http://blog.rebertia.com From no.email at nospam.invalid Mon Jan 11 04:56:36 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 11 Jan 2010 01:56:36 -0800 Subject: lightweight encryption of text file References: <7xiqb9yaea.fsf@ruckus.brouhaha.com> Message-ID: <7xy6k53qy3.fsf@ruckus.brouhaha.com> geremy condra writes: > And having no dependencies frees you from the burden of testing > where your software will be deployed? I don't think so. If you just use the stdlib and are a bit careful about OS dependent features, your code can run pretty much everywhere. More to the point, if (say) you develop a pure Python app on Linux and a Windows user reports a problem, you can probably straighten it out without having to actually use a Windows development system. If you need C extensions you need Windows compilers. >> 2) using a serious library requires quite a bit of knowledge >> and decision-making which not everyone is equipped to do. > > Homebrewing is not a good solution to the problem of being > ignorant of modern cryptography. Not sure what you're getting at. If you're referring to p3.py as a homebrew algorithm designed by someone ignorant, I don't think that is accurate. I've been a fulltime crypto developer, and p3.py was reviewed by several experts on sci.crypt who all felt that its design is sound. I don't claim it's suitable for high-value data (stick with something standards-conformant for that) but it was designed as a replacement for the now-defunct rotor module, and is just about certainly better in every regard. Its only cryptographic assumption is that SHA1(key+X) is a pseudorandom function for fixed length X. That is not a certified characteristic of SHA1, but the HMAC standard (RFC 2104) is based on the same assumption (see Krawczyk's paper cited in the RFC). I'd go as far as to say that it is just about certainly better than RC4, which has well-known distinguishers of quite low complexity. >>?"AES" is not so simple to use unless you know what you're doing in >> terms of modes, nonces, etc. > > Seems pretty simple to me- use AES 192, don't use ECB mode, and > use your library of choice's key strengthening utilities. That does not address any questions of authentication, ciphertext malleability, IV generation, etc. p3.py handles all of that with no fuss imposed on the user. Really, p3.py was written because the same basic desire (simple, pure-Python encryption) kept coming up over and over in my own code and others', and it really seems to address the constraints about as well as anything I've been able to think of. From alfps at start.no Mon Jan 11 05:20:49 2010 From: alfps at start.no (Alf P. Steinbach) Date: Mon, 11 Jan 2010 11:20:49 +0100 Subject: how to duplicate array entries In-Reply-To: References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <87ocl1ce78.fsf@rudin.co.uk> Message-ID: * Chris Rebert: > On Mon, Jan 11, 2010 at 1:03 AM, Alf P. Steinbach wrote: >> * Steven D'Aprano: >>> On Mon, 11 Jan 2010 08:56:36 +0100, Alf P. Steinbach wrote: >>> >>>> * Paul Rudin: >>>>> Sebastian writes: >>>>> >>>>>> I have an array x=[1,2,3] >>>>> In python such an object is called a "list". >>>>> >>>>> (In cpython it's implemented as an automatically resizable array.) >>>> I don't think the OP's terminology needs correction. >>>> >>>> A Python "list" is an array functionality-wise. >>>> >>>> If one isn't observant of that fact then one ends up with O(n^2) time >>>> for the simplest things. >>> Well that's certainly not true. Some operations may be O(N**2), but others >>> are not: list.append() is amortized O(N) and for individual appends, may be >>> can be as fast as O(1). >> The second sentence may or may not be true. I don't know of any fundamental >> 'list' operations that have quadratic time. Is there? >> >> The first sentence is just baffling -- what on Earth is the "that" that >> you think is not true? >> >> OK, I can guess (correct me if I'm guessing wrong, please): you think I'm >> talking about elementary operations. I'm not. I'm talking about algorithmic >> complexity for loops doing e.g. insertions. >> >> >>>> Using the term "array" accentuates and clarifies this most important >>>> aspect. >>> But Python lists are designed to behave as lists. >> No, I'm sorry, they're not. >> >> A Python 'list' has de facto constant time indexing, or "random access". >> >> A linked list -- what the informal "list" means in programming > > Eh, it's a bit context-dependent. The abstract data type definition is > a superset that includes both linked lists and dynamic arrays. Assuming you're talking about some abstract type definition that's in some PEP somewhere (there's no abstract data type in the language specification, it's all informal) then that's a deficiency of the specification, since the type is designed around indexing operations. Perhaps the designers thought it would be "obvious", that no-one could mistake it for other than what it is? Anyway, that doesn't make it context-dependent: if true, it only makes it poorly specified. > FWIW, Java likewise uses "list" in its ADT sense. I'm sorry, I'm unfamiliar with that Java terminology (but then, reportedly, many Java programmers think that Java has "pass by reference", so nothing coming from that direction will surprise me very much!). The Java language specification has a section about arrays, none about lists AFAICS. Do you have a reference? Cheers & hth., - Alf From clp2 at rebertia.com Mon Jan 11 05:31:41 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Mon, 11 Jan 2010 02:31:41 -0800 Subject: how to duplicate array entries In-Reply-To: References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <87ocl1ce78.fsf@rudin.co.uk> Message-ID: <50697b2c1001110231l2ee64e3cub10ad46dbc5d6e2c@mail.gmail.com> On Mon, Jan 11, 2010 at 2:20 AM, Alf P. Steinbach wrote: > * Chris Rebert: >> On Mon, Jan 11, 2010 at 1:03 AM, Alf P. Steinbach wrote: >>> * Steven D'Aprano: >>>> On Mon, 11 Jan 2010 08:56:36 +0100, Alf P. Steinbach wrote: >>>>> * Paul Rudin: >>>>>> Sebastian writes: >>>>> Using the term "array" accentuates and clarifies this most important >>>>> aspect. >>>> >>>> But Python lists are designed to behave as lists. >>> >>> No, I'm sorry, they're not. >>> >>> A Python 'list' has de facto constant time indexing, or "random access". >>> >>> A linked list ?-- ?what the informal "list" means in programming >> >> Eh, it's a bit context-dependent. The abstract data type definition is >> a superset that includes both linked lists and dynamic arrays. > > Assuming you're talking about some abstract type definition that's in some > PEP somewhere No, I mean the computer science definition/term: http://en.wikipedia.org/wiki/List_(computer_science) >> FWIW, Java likewise uses "list" in its ADT sense. > > I'm sorry, I'm unfamiliar with that Java terminology (but then, reportedly, > many Java programmers think that Java has "pass by reference", so nothing > coming from that direction will surprise me very much!). The Java language > specification has a section about arrays, none about lists AFAICS. Do you > have a reference? http://java.sun.com/j2se/1.4.2/docs/api/java/util/List.html Cheers, Chris -- http://blog.rebertia.com From fetchinson at googlemail.com Mon Jan 11 05:46:07 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Mon, 11 Jan 2010 11:46:07 +0100 Subject: easy_install fails on python-magic Message-ID: I'm not sure whose fault this is, the author of python-magic or easy_install (I'm guessing easy_install) but in any case easy_install python-magic (as root) fails with Searching for python-magic Reading http://pypi.python.org/simple/python-magic/ Reading http://hupp.org/adam/hg/python-magic No local packages or download links found for python-magic Best match: None Traceback (most recent call last): File "/usr/bin/easy_install", line 8, in load_entry_point('setuptools==0.6c11', 'console_scripts', 'easy_install')() File "/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py", line 1712, in main File "/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py", line 1700, in with_ei_usage File "/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py", line 1716, in File "/usr/lib64/python2.6/distutils/core.py", line 152, in setup dist.run_commands() File "/usr/lib64/python2.6/distutils/dist.py", line 975, in run_commands self.run_command(cmd) File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command cmd_obj.run() File "/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py", line 211, in run File "/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py", line 434, in easy_install File "/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/package_index.py", line 475, in fetch_distribution AttributeError: 'NoneType' object has no attribute 'clone' As far as I can see the problem is that no version of python-magic has been supplied for python 2.6 which is the version I'm using, only 2.4 and 2.5. Nevertheless easy_install should I think fail with a more user friendly message, and definitely not with a long traceback. It should I think just inform the user that no version has been found, good bye. Or something like that. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From alfps at start.no Mon Jan 11 06:12:56 2010 From: alfps at start.no (Alf P. Steinbach) Date: Mon, 11 Jan 2010 12:12:56 +0100 Subject: how to duplicate array entries In-Reply-To: References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <87ocl1ce78.fsf@rudin.co.uk> Message-ID: * Chris Rebert: > On Mon, Jan 11, 2010 at 2:20 AM, Alf P. Steinbach wrote: >> * Chris Rebert: >>> On Mon, Jan 11, 2010 at 1:03 AM, Alf P. Steinbach wrote: >>>> * Steven D'Aprano: >>>>> On Mon, 11 Jan 2010 08:56:36 +0100, Alf P. Steinbach wrote: >>>>>> * Paul Rudin: >>>>>>> Sebastian writes: > >>>>>> Using the term "array" accentuates and clarifies this most important >>>>>> aspect. >>>>> But Python lists are designed to behave as lists. >>>> No, I'm sorry, they're not. >>>> >>>> A Python 'list' has de facto constant time indexing, or "random access". >>>> >>>> A linked list -- what the informal "list" means in programming >>> Eh, it's a bit context-dependent. The abstract data type definition is >>> a superset that includes both linked lists and dynamic arrays. >> Assuming you're talking about some abstract type definition that's in some >> PEP somewhere > > No, I mean the computer science definition/term: > http://en.wikipedia.org/wiki/List_(computer_science) Note that the default meaning is a list with the characteristics of a linked list. The abstract data type specified in that article is a bit more restricted than the usual meaning of list -- as the article notes, the ADT it presents is equivalent to an abstract stack, and it's essentially the Lisp view of lists, not only just linked list but a restricted view of linked lists. To understand it you have to keep in mind that such an ADT is a simplification, for the purpose of specifying logical functionality and nothing else. The algorithmic efficiency is simply not specified, but is implied. Unfortunately, as noted there, the article doesn't cite any references or sources... Here's one: http://www.itl.nist.gov/div897/sqg/dads/HTML/list.html >>> FWIW, Java likewise uses "list" in its ADT sense. >> I'm sorry, I'm unfamiliar with that Java terminology (but then, reportedly, >> many Java programmers think that Java has "pass by reference", so nothing >> coming from that direction will surprise me very much!). The Java language >> specification has a section about arrays, none about lists AFAICS. Do you >> have a reference? > > http://java.sun.com/j2se/1.4.2/docs/api/java/util/List.html Thanks. I didn't know that. It's a very dirty hodgepodge interface with *optional* methods that throw exceptions if not implemented and apparently no constraints on algorithmic complexity for various methods. As such, it's a very good example why 'list' should not be conflated with 'array'. It leads to such monstrosities where neither correctness nor any kind of efficiency is guaranteed :-) Cheers, & thanks, - Alf PS: Please do not mail me copies of your replies to the group. A mail copy means that I may have to answer your article twice, as in this case. From mal at egenix.com Mon Jan 11 07:12:19 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 11 Jan 2010 13:12:19 +0100 Subject: Fractional Hours from datetime? In-Reply-To: References: Message-ID: <4B4B15A3.7000305@egenix.com> W. eWatson wrote: > Maybe there's a more elegant way to do this. I want to express the > result of datetime.datetime.now() in fractional hours. > > Here's one way. > > dt=datetime.datetime.now() > xtup = dt.timetuple() > h = xtup[3]+xtup[4]/60.0+xtup[5]/3600.00+xtup[6]/10**6 > # now is in fractions of an hour Here's how you'd do that with mxDateTime: >>> from mx.DateTime import now >>> now().abstime / 3600.0 13.17341068830755 .abstime gives you the time in fractional seconds. http://www.egenix.com/products/python/mxBase/mxDateTime/ -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From john_re at fastmail.us Mon Jan 11 07:29:36 2010 From: john_re at fastmail.us (giovanni_re) Date: Mon, 11 Jan 2010 04:29:36 -0800 Subject: TONIGHT Join 5-6P Mon 11th - 1st Evening Meeting test IRC & VOIP online Python at BerkeleyTIP-Global - for forwarding Message-ID: <1263212976.2403.1354001243@webmail.messagingengine.com> You're invited to the first test of the Global Python bimonthly evening meetings at BerkeleyTIP-Global. :) Join in tonight, Monday Jan 11, 5-6P Pacific, 8-9P Eastern, = Tues Jan 12 1A-2A UTC. http://sites.google.com/site/berkeleytip/schedule On #berkeleytip on irc.freenode.net, & on voip - whatever is working - try btip server first. http://sites.google.com/site/berkeleytip/remote-attendance This will be an online only meeting - no in person meeting at UCB. Hot topics: Community Leadership Summit review of interesting sessions, Spring 2010 efforts for UCB & all UC's & all college activities, Upcoming KDE conference end of next week, for 1 week, in Los Angeles. What do _you_ want to discuss? == Some people have asked for an evening meeting, because: a) they can't make weekend meetings, b) they want more BTIP-Global. ;) So, this will be a test, everyone invited, to see if we can make this work. == BerkeleyTIP-Global is the Global All Free SW HW & Culture meeting online via VOIP. http://sites.google.com/site/berkeleytip/ Join the global mailing list, say "hi", & what you're interested in. :) http://groups.google.com/group/BerkTIPGlobal For Forwarding: You are invited to forward this announcement wherever it might be appreciated. From martin.hellwig at dcuktec.org Mon Jan 11 08:18:17 2010 From: martin.hellwig at dcuktec.org (Martin P. Hellwig) Date: Mon, 11 Jan 2010 13:18:17 +0000 Subject: Fractional Hours from datetime? In-Reply-To: References: Message-ID: W. eWatson wrote: > Maybe there's a more elegant way to do this. I want to express the > result of datetime.datetime.now() in fractional hours. > > Here's one way. > > dt=datetime.datetime.now() > xtup = dt.timetuple() > h = xtup[3]+xtup[4]/60.0+xtup[5]/3600.00+xtup[6]/10**6 > # now is in fractions of an hour Here is another (though personally I don't find this more elegant than yours, perhaps a bit more readable): >>> now = datetime.datetime.now() >>> fractional_hour = int(now.strftime('%H')) + int(now.strftime('%M')) / 60.0 -- MPH http://blog.dcuktec.com 'If consumed, best digested with added seasoning to own preference.' From martin.hellwig at dcuktec.org Mon Jan 11 08:25:22 2010 From: martin.hellwig at dcuktec.org (Martin P. Hellwig) Date: Mon, 11 Jan 2010 13:25:22 +0000 Subject: Fractional Hours from datetime? In-Reply-To: References: Message-ID: Martin P. Hellwig wrote: > W. eWatson wrote: >> Maybe there's a more elegant way to do this. I want to express the >> result of datetime.datetime.now() in fractional hours. >> >> Here's one way. >> >> dt=datetime.datetime.now() >> xtup = dt.timetuple() >> h = xtup[3]+xtup[4]/60.0+xtup[5]/3600.00+xtup[6]/10**6 >> # now is in fractions of an hour > > Here is another (though personally I don't find this more elegant than > yours, perhaps a bit more readable): > > >>> now = datetime.datetime.now() > >>> fractional_hour = int(now.strftime('%H')) + int(now.strftime('%M')) > / 60.0 > Actually my version is overcomplicated: >>> now = datetime.datetime.now() >>> fractional_hour = now.hour + now.minute / 60.0 -- MPH http://blog.dcuktec.com 'If consumed, best digested with added seasoning to own preference.' From jeanmichel at sequans.com Mon Jan 11 08:52:51 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 11 Jan 2010 14:52:51 +0100 Subject: How to validate the __init__ parameters In-Reply-To: References: <803c4db7-3813-448a-b428-0ec11c57f098@t42g2000yqd.googlegroups.com> Message-ID: <4B4B2D33.5020109@sequans.com> Aahz wrote: > In article , > Jean-Michel Pichavant wrote: > >> class A: >> def __init__(self, foo = None, bar = None): >> if len(foo) > 5: >> raise ValueError('foo cannot exceed 5 characters') >> > > Bad Idea -- what happens when foo is None? > You're right. That perfectly illustrates how the simplest solution is the often most valuable one: it is much more easy to find bugs, when there is any. JM From victorsubervi at gmail.com Mon Jan 11 08:55:21 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Mon, 11 Jan 2010 09:55:21 -0400 Subject: Academic Question In-Reply-To: <4B4A25DD.4040703@mrabarnett.plus.com> References: <4dc0cfea1001100907t6bc032cftea2e9e258e7c1937@mail.gmail.com> <4B4A25DD.4040703@mrabarnett.plus.com> Message-ID: <4dc0cfea1001110555q3730f9f2s1bb41522f7f16ff8@mail.gmail.com> On Sun, Jan 10, 2010 at 3:09 PM, MRAB wrote: > browser = form.getfirst('browser', 'all') >> except: >> browser = headers() >> >> try: > A bare except, and if an exception _does_ occur, they'll be a NameError > because 'headers' isn't defined. > > Oh, not the large halibut again! (I will be cleaning them up ;) > > > os.chdir('%s/..' % cwd) >> sys.path.append(os.getcwd()) >> from templateFrame import top, bottom >> os.chdir(cwd) >> >> Why doesn't it work if I move the bottom imports to the top? The form >> values get lost, even though I chdir to cwd. >> >> I try to avoid changing the directory. > It's either that or copying them all over to the other dir, which is even worse, since I don't want to maintain identical scripts. Thanks, beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From joaopcf at gmail.com Mon Jan 11 09:00:15 2010 From: joaopcf at gmail.com (=?ISO-8859-1?Q?Jo=E3o?=) Date: Mon, 11 Jan 2010 06:00:15 -0800 (PST) Subject: force URLencoding script Message-ID: <6bc42c49-e26c-4f7b-9538-dd2e319accfb@26g2000yqo.googlegroups.com> Hi. I'm trying to figure out how to force URLencoding in my Python 2.4.3 environment receiving data an input argument but I'm really at a loss here. What am I doing wrong? #!/usr/bin/env python import sys from urllib import urlencode, urlopen from urllib2 import Request import urlparse destination = sys.argv[1] msg = sys.argv[2] #Will I have problems with this one if the input is multiline? # the browser identifies itself using the User-Agent header # after creating the Request object, it's possible to pass in a dictionary of headers user_agent = 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)' # force the request to be identified as IE 5.5.... headers = { ?User-Agent? : user_agent } # force no proxy authentication = 'UID=22541&PW=gdyb21LQTcIANtvYMT7QVQ==&' # force Unicode display format message = u'M=%s&' % msg dest_number = 'N=%s' % destination data = authentication + message + dest_number url = 'http://10.112.28.221:38080/GwHTTPin/sendtext' print 'Encoded URL:', url #get full URL adding ? to it, followed by the encoded values #full_url = url + '?' url_values #should I force url_values = urllib.urlencode(data) instead? full_url = urllib2.Request(url, data, headers) response = urllib2.urlopen(full_url) #.urlopen works transparently with proxies which do not require authentication processed = urllib.open(full_url) From jeanmichel at sequans.com Mon Jan 11 09:00:40 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 11 Jan 2010 15:00:40 +0100 Subject: Easy Q In-Reply-To: <4dc0cfea1001091018t726f283g381f815723e20f94@mail.gmail.com> References: <4dc0cfea1001090811y198462a8s39976a0622ef6c59@mail.gmail.com> <4B48B52C.3020409@islandtraining.com> <4dc0cfea1001091018t726f283g381f815723e20f94@mail.gmail.com> Message-ID: <4B4B2F08.4010800@sequans.com> Victor Subervi wrote: > On Sat, Jan 9, 2010 at 11:56 AM, Gary Herron > > wrote: > > Victor Subervi wrote: > > Hi; > I have a string.join statement on a variable that comes from a > cgi.FieldStorage().getlist. The variable may be a list or a > single value. I need to treat it differently depending on > which it is. How can I distinguish it? len(var) will obviously > give me the length of the string if it's a string and the > length of the list if it's a list. > TIA, > beno > > > Like this: > > if isinstance(var, list): > ... join ... > else: > ... ??? ... > > > Right.. Thanks! > beno You should definitely check again MRAB's answer. Having getlist returning something else than a list, especially a non iterable single item *is* suspicious. Usually, properly designed functions return consistent types over calls. JM From victorsubervi at gmail.com Mon Jan 11 09:20:33 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Mon, 11 Jan 2010 10:20:33 -0400 Subject: Easy Q In-Reply-To: <4B4B2F08.4010800@sequans.com> References: <4dc0cfea1001090811y198462a8s39976a0622ef6c59@mail.gmail.com> <4B48B52C.3020409@islandtraining.com> <4dc0cfea1001091018t726f283g381f815723e20f94@mail.gmail.com> <4B4B2F08.4010800@sequans.com> Message-ID: <4dc0cfea1001110620we2dd8eey582a3fb6c7784c75@mail.gmail.com> On Mon, Jan 11, 2010 at 10:00 AM, Jean-Michel Pichavant < jeanmichel at sequans.com> wrote: > Victor Subervi wrote: > > On Sat, Jan 9, 2010 at 11:56 AM, Gary Herron > gherron at islandtraining.com>> wrote: >> >> Victor Subervi wrote: >> >> Hi; >> I have a string.join statement on a variable that comes from a >> cgi.FieldStorage().getlist. The variable may be a list or a >> single value. I need to treat it differently depending on >> which it is. How can I distinguish it? len(var) will obviously >> give me the length of the string if it's a string and the >> length of the list if it's a list. >> TIA, >> beno >> >> >> Like this: >> >> if isinstance(var, list): >> ... join ... >> else: >> ... ??? ... >> >> >> Right.. Thanks! >> beno >> > You should definitely check again MRAB's answer. > Having getlist returning something else than a list, especially a non > iterable single item *is* suspicious. > Usually, properly designed functions return consistent types over calls. > Obviously. Thanks, beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From michele.petrazzo at REMOVE_me_unipex.it Mon Jan 11 09:34:37 2010 From: michele.petrazzo at REMOVE_me_unipex.it (Michele Petrazzo) Date: Mon, 11 Jan 2010 15:34:37 +0100 Subject: Computer Music Toolkit (CMT) ? In-Reply-To: References: Message-ID: Terry Reedy wrote: > On 1/10/2010 4:15 AM, Peter Billam wrote: >> Greetings. Is there a way to get at the Computer Music Toolkit (CMT) >> http://www.ladspa.org/cmt/ >> functionality from Python (Python3 in my case) ? > > You can access compiled C shared libraries most easily via the ctypes > module. > > so if you do develop a ctypes wrapping, consider contributing it. Since I'm interested to do it for a my new project, if the OP are interested, please contact me directly. Michele From jhcepas at gmail.com Mon Jan 11 10:16:02 2010 From: jhcepas at gmail.com (Jaime Huerta Cepas) Date: Mon, 11 Jan 2010 16:16:02 +0100 Subject: does anyone know a way to package a "static" or "portable" version of a python module? Message-ID: does anyone know a way to package a "static" version of a python module? This is, the same that pyinstaller does with single scripts, but with a complete module directory. I'm specially interested in packaging all pyqt4, sip and qt4 dependencies within the module itself. thanks, Jaime. -------------- next part -------------- An HTML attachment was scrubbed... URL: From davea at ieee.org Mon Jan 11 10:26:40 2010 From: davea at ieee.org (Dave Angel) Date: Mon, 11 Jan 2010 10:26:40 -0500 Subject: Academic Question In-Reply-To: <4dc0cfea1001110555q3730f9f2s1bb41522f7f16ff8@mail.gmail.com> References: <4dc0cfea1001100907t6bc032cftea2e9e258e7c1937@mail.gmail.com> <4B4A25DD.4040703@mrabarnett.plus.com> <4dc0cfea1001110555q3730f9f2s1bb41522f7f16ff8@mail.gmail.com> Message-ID: <4B4B4330.9010508@ieee.org> Victor Subervi wrote: > On Sun, Jan 10, 2010 at 3:09 PM, MRAB wrote: > > >> browser = form.getfirst('browser', 'all') >> >>> except: >>> browser = headers() >>> >>> try: >>> >> A bare except, and if an exception _does_ occur, they'll be a NameError >> because 'headers' isn't defined. >> >> >> > > > Oh, not the large halibut again! (I will be cleaning them up ;) > > >> os.chdir('%s/..' % cwd) >> >>> sys.path.append(os.getcwd()) >>> from templateFrame import top, bottom >>> os.chdir(cwd) >>> >>> Why doesn't it work if I move the bottom imports to the top? The form >>> values get lost, even though I chdir to cwd. >>> >>> I try to avoid changing the directory. >>> > > It's either that or copying them all over to the other dir, which is even > worse, since I don't want to maintain identical scripts. > Thanks, > beno > > You miss the point. Rather than doing chdir to change the current directory, in order to use getcwd in your sys.path.append, calculate the appropriate directory yourself, and use it directly, without changing the current directory. Brute force would be something like (untested): sys.path.append(os.path.join(os.getcwd(), "..")) You could get trickier by stripping off the last node of the directory path, but it shouldn't be necessary. Incidentally, I'd tend to use os.path.dirname( __main__.file ) rather than os.getcwd(). That way it'd work even if current directory was changed by something else. In other words (untested): sys.path.append(os.path.join(os.path.dirname(__file__), "..")) this will add the parent directory of the current module to the os.path. HTH DaveA From narkewoody at gmail.com Mon Jan 11 10:27:03 2010 From: narkewoody at gmail.com (Steven Woody) Date: Mon, 11 Jan 2010 23:27:03 +0800 Subject: pyserial: Unexpected Local Echo Message-ID: Hi, I am using pyserial. But I always get the local echo after I write some characters onto serial port and I find no way to disable this behavior. When I say 'local echo', I mean the next read operation will get characters that was just write to the same port. I run my program on cygwin (pyserial was also built on the system from source code) and the serial port i am using is a USB adapter that simulates a port (COM4 on my XP) because my laptop don't have a real serial port. But I checked my COM4 settings, there is no any think like 'local echo'. Thanks in advance. -- Life is the only flaw in an otherwise perfect nonexistence -- Schopenhauer narke public key at http://subkeys.pgp.net:11371 (narkewoody at gmail.com) From victorsubervi at gmail.com Mon Jan 11 10:31:59 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Mon, 11 Jan 2010 11:31:59 -0400 Subject: Academic Question In-Reply-To: <4B4B4330.9010508@ieee.org> References: <4dc0cfea1001100907t6bc032cftea2e9e258e7c1937@mail.gmail.com> <4B4A25DD.4040703@mrabarnett.plus.com> <4dc0cfea1001110555q3730f9f2s1bb41522f7f16ff8@mail.gmail.com> <4B4B4330.9010508@ieee.org> Message-ID: <4dc0cfea1001110731r57cb1bb9s423f50e9c80c1743@mail.gmail.com> On Mon, Jan 11, 2010 at 11:26 AM, Dave Angel wrote: > Victor Subervi wrote: > >> On Sun, Jan 10, 2010 at 3:09 PM, MRAB wrote: >> >> >> >>> browser = form.getfirst('browser', 'all') >>> >>> >>>> except: >>>> browser = headers() >>>> >>>> try: >>>> >>>> >>> A bare except, and if an exception _does_ occur, they'll be a NameError >>> because 'headers' isn't defined. >>> >>> >>> >>> >> >> >> Oh, not the large halibut again! (I will be cleaning them up ;) >> >> >> >>> os.chdir('%s/..' % cwd) >>> >>> >>>> sys.path.append(os.getcwd()) >>>> from templateFrame import top, bottom >>>> os.chdir(cwd) >>>> >>>> Why doesn't it work if I move the bottom imports to the top? The form >>>> values get lost, even though I chdir to cwd. >>>> >>>> I try to avoid changing the directory. >>>> >>>> >>> >> It's either that or copying them all over to the other dir, which is even >> worse, since I don't want to maintain identical scripts. >> Thanks, >> beno >> >> >> > You miss the point. Rather than doing chdir to change the current > directory, in order to use getcwd in your sys.path.append, calculate the > appropriate directory yourself, and use it directly, without changing the > current directory. Brute force would be something like (untested): > > sys.path.append(os.path.join(os.getcwd(), "..")) > > You could get trickier by stripping off the last node of the directory > path, but it shouldn't be necessary. > > Incidentally, I'd tend to use os.path.dirname( __main__.file ) rather > than os.getcwd(). That way it'd work even if current directory was changed > by something else. In other words (untested): > > sys.path.append(os.path.join(os.path.dirname(__file__), "..")) > > this will add the parent directory of the current module to the os.path. > Well put. Thanks! beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From jlconlin at gmail.com Mon Jan 11 10:35:41 2010 From: jlconlin at gmail.com (Jeremy) Date: Mon, 11 Jan 2010 07:35:41 -0800 (PST) Subject: Different number of matches from re.findall and re.split Message-ID: <2f8b08af-65fd-4ff9-b90b-db3f689aca01@u7g2000yqm.googlegroups.com> Hello all, I am using re.split to separate some text into logical structures. The trouble is that re.split doesn't find everything while re.findall does; i.e.: > found = re.findall('^ 1', line, re.MULTILINE) > len(found) 6439 > tables = re.split('^ 1', line, re.MULTILINE) > len(tables) > 1 Can someone explain why these two commands are giving different results? I thought I should have the same number of matches (or maybe different by 1, but not 6000!) Thanks, Jeremy From steve at holdenweb.com Mon Jan 11 10:43:54 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 11 Jan 2010 10:43:54 -0500 Subject: pyserial: Unexpected Local Echo In-Reply-To: References: Message-ID: Steven Woody wrote: > Hi, > > I am using pyserial. But I always get the local echo after I write > some characters onto serial port and I find no way to disable this > behavior. When I say 'local echo', I mean the next read operation will > get characters that was just write to the same port. > > I run my program on cygwin (pyserial was also built on the system from > source code) and the serial port i am using is a USB adapter that > simulates a port (COM4 on my XP) because my laptop don't have a real > serial port. But I checked my COM4 settings, there is no any think > like 'local echo'. > > > Thanks in advance. > It sounds to me like the device you are connecting to implements echoing. Have you tried connecting a terminal emulator to it? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From iainking at gmail.com Mon Jan 11 10:44:45 2010 From: iainking at gmail.com (Iain King) Date: Mon, 11 Jan 2010 07:44:45 -0800 (PST) Subject: Different number of matches from re.findall and re.split References: <2f8b08af-65fd-4ff9-b90b-db3f689aca01@u7g2000yqm.googlegroups.com> Message-ID: <42ac849d-fecd-4489-8c9c-4ac4dc939898@m3g2000yqf.googlegroups.com> On Jan 11, 3:35?pm, Jeremy wrote: > Hello all, > > I am using re.split to separate some text into logical structures. > The trouble is that re.split doesn't find everything while re.findall > does; i.e.: > > > > > found = re.findall('^ 1', line, re.MULTILINE) > > len(found) > ? ?6439 > > tables = re.split('^ 1', line, re.MULTILINE) > > len(tables) > > 1 > > Can someone explain why these two commands are giving different > results? ?I thought I should have the same number of matches (or maybe > different by 1, but not 6000!) > > Thanks, > Jeremy re.split doesn't take re.MULTILINE as a flag: it doesn't take any flags. It does take a maxsplit parameter, which you are passing the value of re.MULTILINE (which happens to be 8 in my implementation). Since your pattern is looking for line starts, and your first line presumably has more splits than the maxsplits you are specifying, your re.split never finds more than 1. >>> a 'split(pattern, string, maxsplit=0)\n Split the source string by the occurren ces of the pattern,\n returning a list containing the resulting substrings.\n ' >>> re.split(" ", a, re.MULTILINE) ['split(pattern,', 'string,', 'maxsplit=0)\n', '', '', '', 'Split', 'the', 'sour ce string by the occurrences of the pattern,\n returning a list containing th e resulting substrings.\n'] >>> re.split(" ", a) ['split(pattern,', 'string,', 'maxsplit=0)\n', '', '', '', 'Split', 'the', 'sour ce', 'string', 'by', 'the', 'occurrences', 'of', 'the', 'pattern,\n', '', '', '' , 'returning', 'a', 'list', 'containing', 'the', 'resulting', 'substrings.\n'] Iain From jlconlin at gmail.com Mon Jan 11 10:49:28 2010 From: jlconlin at gmail.com (Jeremy) Date: Mon, 11 Jan 2010 07:49:28 -0800 (PST) Subject: Different number of matches from re.findall and re.split References: <2f8b08af-65fd-4ff9-b90b-db3f689aca01@u7g2000yqm.googlegroups.com> <42ac849d-fecd-4489-8c9c-4ac4dc939898@m3g2000yqf.googlegroups.com> Message-ID: <4a5ebe07-aac7-40e6-ad1d-12f3d5316703@j19g2000yqk.googlegroups.com> On Jan 11, 8:44?am, Iain King wrote: > On Jan 11, 3:35?pm, Jeremy wrote: > > > > > > > Hello all, > > > I am using re.split to separate some text into logical structures. > > The trouble is that re.split doesn't find everything while re.findall > > does; i.e.: > > > > found = re.findall('^ 1', line, re.MULTILINE) > > > len(found) > > ? ?6439 > > > tables = re.split('^ 1', line, re.MULTILINE) > > > len(tables) > > > 1 > > > Can someone explain why these two commands are giving different > > results? ?I thought I should have the same number of matches (or maybe > > different by 1, but not 6000!) > > > Thanks, > > Jeremy > > re.split doesn't take re.MULTILINE as a flag: it doesn't take any > flags. It does take a maxsplit parameter, which you are passing the > value of re.MULTILINE (which happens to be 8 in my implementation). > Since your pattern is looking for line starts, and your first line > presumably has more splits than the maxsplits you are specifying, your > re.split never finds more than 1. Yep. Thanks for pointing that out. I guess I just assumed that re.split was similar to re.search/match/findall in what it accepted as function parameters. I guess I'll have to use a \n instead of a ^ for split. Thanks, Jeremy From howe.steven at gmail.com Mon Jan 11 10:52:26 2010 From: howe.steven at gmail.com (Steven Howe) Date: Mon, 11 Jan 2010 07:52:26 -0800 Subject: how to duplicate array entries In-Reply-To: <14437829-e3ef-41ac-b533-bc7003c1434b@j19g2000yqk.googlegroups.com> References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <14437829-e3ef-41ac-b533-bc7003c1434b@j19g2000yqk.googlegroups.com> Message-ID: <4B4B493A.8050406@gmail.com> try --------------------------------------- #!/usr/bin/env python from types import ListType, IntType def array_expander( ar=None, ex=None ): if type( ex ) != IntType: return [] if ex <= 0: return [] if type( ar ) != ListType: return [] # working code starts here # res = [] for t in ar: for x in range( ex ): res.append( t ) return res # function ends # res = array_expander( [1,11,3,5], 4 ) print res --------------------------------------- [1, 1, 1, 1, 11, 11, 11, 11, 3, 3, 3, 3, 5, 5, 5, 5] -- *Kiri-kin-tha's* First Law of Metaphysics is /"Nothing unreal exists."/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From kw at codebykevin.com Mon Jan 11 10:54:57 2010 From: kw at codebykevin.com (Kevin Walzer) Date: Mon, 11 Jan 2010 10:54:57 -0500 Subject: Procedural API inside class--scoping questions Message-ID: I'm trying to make use of a Python library, aemreceive, that provides a procedural API. (aemreceive is a library for Python on the Mac that allows the application to receive and respond to Apple Events.) My Python apps basically run in a single fooApp class, and everything runs inside the class. To launch the app (my apps are Tkinter-based), I use something like this: if __name__== '__main__': app = fooApp(None) app.mainloop() In keeping with aemreceive's procedural API, I've added a runCommand function inside the app class to provide some basic functionality in response to Apple Events it will set some objects/variables, then display the output in the app. In my code it looks like this: def runCommand(string): self.searchterm=string self.servertree.selection_set('Default') self.getInfo() When I test this command, I get an error from Python: "self" is not defined. I think I understand what is going here. All of the other functions in the fooApp class take "self" as a parameter, i.e. they are components of the class. runCommand is not, so therefore "self" is undefined. My question is, how can I get the values from the class (self.searchterm, et.al) inside the runCommand function? Thanks, Kevin -- Kevin Walzer Code by Kevin http://www.codebykevin.com From arnodel at googlemail.com Mon Jan 11 10:56:19 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 11 Jan 2010 07:56:19 -0800 (PST) Subject: Different number of matches from re.findall and re.split References: <2f8b08af-65fd-4ff9-b90b-db3f689aca01@u7g2000yqm.googlegroups.com> Message-ID: <085d5292-00af-4452-9f9c-da881f580b88@a21g2000yqc.googlegroups.com> On 11 Jan, 15:35, Jeremy wrote: > Hello all, > > I am using re.split to separate some text into logical structures. > The trouble is that re.split doesn't find everything while re.findall > does; i.e.: > > > > > found = re.findall('^ 1', line, re.MULTILINE) > > len(found) > ? ?6439 > > tables = re.split('^ 1', line, re.MULTILINE) > > len(tables) > > 1 > > Can someone explain why these two commands are giving different > results? ?I thought I should have the same number of matches (or maybe > different by 1, but not 6000!) > > Thanks, > Jeremy When in doubt, the documentation is a good place to start :) http://docs.python.org/library/re.html#re.split re.split(pattern, string[, maxsplit=0]) http://docs.python.org/library/re.html#re.findall re.findall(pattern, string[, flags]) Notice that re.split's optional third argument is not for passing flags. HTH -- Arnaud From arnodel at googlemail.com Mon Jan 11 11:01:16 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 11 Jan 2010 08:01:16 -0800 (PST) Subject: Procedural API inside class--scoping questions References: Message-ID: On 11 Jan, 15:54, Kevin Walzer wrote: > I'm trying to make use of a Python library, aemreceive, that provides a > procedural API. (aemreceive is a library for Python on the Mac that > allows the application to receive and respond to Apple Events.) > > My Python apps basically run in a single fooApp class, and everything > runs inside the class. To launch the app (my apps are Tkinter-based), I > use something like this: > > if __name__== '__main__': > ? ? ?app = fooApp(None) > ? ? ?app.mainloop() > > In keeping with aemreceive's procedural API, I've added a runCommand > function inside the app class to provide some basic functionality in > response to Apple Events it will set some objects/variables, then > display the output in the app. In my code it looks like this: > > ? ? ?def runCommand(string): > ? ? ? ? ?self.searchterm=string > ? ? ? ? ?self.servertree.selection_set('Default') > ? ? ? ? ?self.getInfo() > > When I test this command, I get an error from Python: "self" is not > defined. > Is runcommand is method of your class? In that case you should define it as: def runCommand(self, string): ... HTH -- Arnaud From __peter__ at web.de Mon Jan 11 11:03:42 2010 From: __peter__ at web.de (Peter Otten) Date: Mon, 11 Jan 2010 17:03:42 +0100 Subject: Different number of matches from re.findall and re.split References: <2f8b08af-65fd-4ff9-b90b-db3f689aca01@u7g2000yqm.googlegroups.com> <42ac849d-fecd-4489-8c9c-4ac4dc939898@m3g2000yqf.googlegroups.com> <4a5ebe07-aac7-40e6-ad1d-12f3d5316703@j19g2000yqk.googlegroups.com> Message-ID: Jeremy wrote: > On Jan 11, 8:44 am, Iain King wrote: >> On Jan 11, 3:35 pm, Jeremy wrote: >> >> >> >> >> >> > Hello all, >> >> > I am using re.split to separate some text into logical structures. >> > The trouble is that re.split doesn't find everything while re.findall >> > does; i.e.: >> >> > > found = re.findall('^ 1', line, re.MULTILINE) >> > > len(found) >> > 6439 >> > > tables = re.split('^ 1', line, re.MULTILINE) >> > > len(tables) >> > > 1 >> >> > Can someone explain why these two commands are giving different >> > results? I thought I should have the same number of matches (or maybe >> > different by 1, but not 6000!) >> >> > Thanks, >> > Jeremy >> >> re.split doesn't take re.MULTILINE as a flag: it doesn't take any >> flags. It does take a maxsplit parameter, which you are passing the >> value of re.MULTILINE (which happens to be 8 in my implementation). >> Since your pattern is looking for line starts, and your first line >> presumably has more splits than the maxsplits you are specifying, your >> re.split never finds more than 1. > > Yep. Thanks for pointing that out. I guess I just assumed that > re.split was similar to re.search/match/findall in what it accepted as > function parameters. I guess I'll have to use a \n instead of a ^ for > split. You can precompile the pattern and then invoke the split() method: >>> re.compile("^X", re.MULTILINE).split("""X alpha ... beta ... X gamma ... delta X ... X ... zeta ... """) ['', ' alpha\nbeta\n', ' gamma\ndelta X\n', '\nzeta\n'] Peter From python at mrabarnett.plus.com Mon Jan 11 11:04:07 2010 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 11 Jan 2010 16:04:07 +0000 Subject: Different number of matches from re.findall and re.split In-Reply-To: <4a5ebe07-aac7-40e6-ad1d-12f3d5316703@j19g2000yqk.googlegroups.com> References: <2f8b08af-65fd-4ff9-b90b-db3f689aca01@u7g2000yqm.googlegroups.com> <42ac849d-fecd-4489-8c9c-4ac4dc939898@m3g2000yqf.googlegroups.com> <4a5ebe07-aac7-40e6-ad1d-12f3d5316703@j19g2000yqk.googlegroups.com> Message-ID: <4B4B4BF7.6040903@mrabarnett.plus.com> Jeremy wrote: > On Jan 11, 8:44 am, Iain King wrote: >> On Jan 11, 3:35 pm, Jeremy wrote: >> >> >> >> >> >>> Hello all, >>> I am using re.split to separate some text into logical structures. >>> The trouble is that re.split doesn't find everything while re.findall >>> does; i.e.: >>>> found = re.findall('^ 1', line, re.MULTILINE) >>>> len(found) >>> 6439 >>>> tables = re.split('^ 1', line, re.MULTILINE) >>>> len(tables) >>>> 1 >>> Can someone explain why these two commands are giving different >>> results? I thought I should have the same number of matches (or maybe >>> different by 1, but not 6000!) >>> Thanks, >>> Jeremy >> re.split doesn't take re.MULTILINE as a flag: it doesn't take any >> flags. It does take a maxsplit parameter, which you are passing the >> value of re.MULTILINE (which happens to be 8 in my implementation). >> Since your pattern is looking for line starts, and your first line >> presumably has more splits than the maxsplits you are specifying, your >> re.split never finds more than 1. > > Yep. Thanks for pointing that out. I guess I just assumed that > re.split was similar to re.search/match/findall in what it accepted as > function parameters. I guess I'll have to use a \n instead of a ^ for > split. > You could use the .split method of a pattern object instead: tables = re.compile('^ 1', re.MULTILINE).split(line) From ethanwinograd at gmail.com Mon Jan 11 11:04:08 2010 From: ethanwinograd at gmail.com (ethan) Date: Mon, 11 Jan 2010 08:04:08 -0800 (PST) Subject: Python POS/cash register projects? Message-ID: <599fd6e0-5d0e-4ed4-b5df-8ae8e237e8bb@w12g2000vbj.googlegroups.com> Anybody have any experience with creating a basic POS register system in Python? Any existing projects out there you are aware of? This would be a GUI app, standalone with some basic export and print functions. I see this as a great opportunity to deepen my Python experience but dont want to reinvent the wheel completely.. doesn't look like there is a lot out there.. From duncan.booth at invalid.invalid Mon Jan 11 11:28:54 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 11 Jan 2010 16:28:54 GMT Subject: Different number of matches from re.findall and re.split References: <4a5ebe07-aac7-40e6-ad1d-12f3d5316703@j19g2000yqk.googlegroups.com> Message-ID: MRAB wrote: >> Yep. Thanks for pointing that out. I guess I just assumed that >> re.split was similar to re.search/match/findall in what it accepted as >> function parameters. I guess I'll have to use a \n instead of a ^ for >> split. >> > You could use the .split method of a pattern object instead: > > tables = re.compile('^ 1', re.MULTILINE).split(line) or you might include the flag in the regular expression literal: '(?m)^ 1' -- Duncan Booth http://kupuguy.blogspot.com From robert.kern at gmail.com Mon Jan 11 11:44:10 2010 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 11 Jan 2010 10:44:10 -0600 Subject: lightweight encryption of text file In-Reply-To: <4B4851C6.1050009@bluewin.ch> References: <4B4851C6.1050009@bluewin.ch> Message-ID: On 2010-01-09 03:52 AM, Anthra Norell wrote: > Daniel Fetchinson wrote: > > I have a plain text file which I would like to protect in a very > > simple minded, yet for my purposes sufficient, way. I'd like to > > encrypt/convert it into a binary file in such a way that possession of > > a password allows anyone to convert it back into the original text > > file while not possessing the password one would only see the > > following with the standard linux utility 'file': > > > > [fetchinson at fetch ~]$ file encrypted.data > > encrypted.data: data > > > > and the effort required to convert the file back to the original text > > file without the password would be equivalent to guessing the > > password. > > > > I'm fully aware of the security implications of this loose > > specification, but for my purposes this would be a good solution. > > > > What would be the simplest way to achieve this using preferably stock > > python without 3rd party modules? If a not too complex 3rd part > > module made it really simple that would be acceptable too. > > Daniel, > > Here's what looks like another thread veering off into package-ology, > leaving a stumped OP behind. > > "Don't use a random generator for encryption purposes!" warns the > manual, of which fact I was reminded in no uncertain terms on this forum > a few years ago when I proposed the following little routine in response > to a post very similar to yours. One critic challenged me to encode my > credit card data and post it. Which I did. Actually, you just "encrypted" your credit card number and challenged comp.lang.python to crack it. No one challenged you to do anything of the sort. Fortunately, the ever-watchful eye of Google was upon us that day: http://groups.google.com/group/comp.lang.python/browse_thread/thread/5fb9ffada975bae9?pli=1 > Upon which another critic > conjured up the horror vision of gigahertzes hacking my pathetic little > effort to pieces as I was reading his message. Of the well-meaning kind, > he urged me to put an immediate stop to this foolishness. I didn't. > > No unplanned expenditures ensued. That's because comp.lang.python is not full of thieves, not because your algorithm is worth a damn. p3.py imposes no more overhead than this, but it has some real security properties. To quote Paul Rubin from that previous thread: """ Since good encryption schemes that don't have significant performance penalties are widely available, why mess with a crap scheme EVER? Why use a solution that "might or might not be adequate" when you can use one that's definitely ok? """ -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From wolftracks at invalid.com Mon Jan 11 11:44:26 2010 From: wolftracks at invalid.com (W. eWatson) Date: Mon, 11 Jan 2010 08:44:26 -0800 Subject: Fractional Hours from datetime? In-Reply-To: <05ca0e15-b55e-4ede-81fe-c8f4459e3932@r5g2000yqb.googlegroups.com> References: <3ca2bf74-744d-421e-a78e-38dbeac2cda1@m3g2000yqf.googlegroups.com> <05ca0e15-b55e-4ede-81fe-c8f4459e3932@r5g2000yqb.googlegroups.com> Message-ID: Austyn wrote: > Here's an improvement in case you want your code to work outside of > Arizona: > > from time import time, timezone > h = ((time() - timezone) / 3600) % 24 > > On Jan 10, 9:04 pm, Austyn wrote: >> How about: >> >> import time >> arizona_utc_offset = -7.00 >> h = (time.time() / 3600 + arizona_utc_offset) % 24 >> >> dt.timetuple()[6] is the day of the week; struct tm_time doesn't >> include a sub-second field. >> >> On Jan 10, 10:28 am, "W. eWatson" wrote: >> >> >> >>> Maybe there's a more elegant way to do this. I want to express the >>> result of datetime.datetime.now() in fractional hours. >>> Here's one way. >>> dt=datetime.datetime.now() >>> xtup = dt.timetuple() >>> h = xtup[3]+xtup[4]/60.0+xtup[5]/3600.00+xtup[6]/10**6 >>> # now is in fractions of an hour > There seems to be some controversy about this and other matters of datetime. From bruno.42.desthuilliers at websiteburo.invalid Mon Jan 11 12:05:43 2010 From: bruno.42.desthuilliers at websiteburo.invalid (Bruno Desthuilliers) Date: Mon, 11 Jan 2010 18:05:43 +0100 Subject: Procedural API inside class--scoping questions In-Reply-To: References: Message-ID: <4b4b5a65$0$32713$426a74cc@news.free.fr> Kevin Walzer a ?crit : > I'm trying to make use of a Python library, aemreceive, that provides a > procedural API. (aemreceive is a library for Python on the Mac that > allows the application to receive and respond to Apple Events.) > > My Python apps basically run in a single fooApp class, and everything > runs inside the class. To launch the app (my apps are Tkinter-based), I > use something like this: > > if __name__== '__main__': > app = fooApp(None) > app.mainloop() > > In keeping with aemreceive's procedural API, I've added a runCommand > function inside the app class to provide some basic functionality in > response to Apple Events it will set some objects/variables, then > display the output in the app. In my code it looks like this: > > def runCommand(string): > self.searchterm=string > self.servertree.selection_set('Default') > self.getInfo() > > When I test this command, I get an error from Python: "self" is not > defined. Indeed. > I think I understand what is going here. All of the other functions in > the fooApp class take "self" as a parameter, i.e. they are components of > the class. It actually works the other way round : it's because they are "components" (we prefer to say "attributes") of the class that they take the current instance as first param. FWIW, the following syntaxes are functionnaly equivalent: obj = MyClass() obj.some_method() MyClass.some_method(obj) # if some_method is not inherited: MyClass.__dict__['some_method'](obj) Anyway: > runCommand is not, so therefore "self" is undefined. My > question is, how can I get the values from the class (self.searchterm, s/class/instance/ here. > et.al) inside the runCommand function? The usual way - adds the relevant param: def runCommand(self, string): self.searchterm=string self.servertree.selection_set('Default') self.getInfo() The problem is that you don't provide much information about how this function is actually called. From mrkafk at gmail.com Mon Jan 11 12:29:49 2010 From: mrkafk at gmail.com (mk) Date: Mon, 11 Jan 2010 18:29:49 +0100 Subject: Interesting (?) problem Message-ID: Hello everyone, I have two lists of IP addresses: hostips = [ 'a', 'b', 'c', 'd', 'e' ] thread_results = [ 'd', 'b', 'c' ] I need to sort thread_results in the same order as hostips. (Obviously, hostips can contain any valid ip addresses as strings, they are sorted alphabetically here just for sake of example.) Since explicit is better than implicit, I will clarify: thread_results is obviously result of threads communicating with IPs from hostips, and that can finish at various times, thus returning ips into thread_results in any order. Sorting would be trivial to do if thread_results were not a subset of hostips (obviously, for some IPs communication can fail which excludes them from the result). One approach I can see is constructing hostips_limited list that would contain only ips that are in thread_results but that would preserve order of hostips: hostips_limited = [] for h in hostips: if h in thread_results: hostips_limited.append(h) ..and then doing sorting thread_results. But maybe there is more elegant / faster approach? Incidentally, it *seems* that list comprehension preserves order: hostips_limited = [ h for h in hostips if h in thread_results ] Empirically speaking it seems to work (I tested it on real ips), but please correct me if that's wrong. Regards, mk From jeanmichel at sequans.com Mon Jan 11 12:48:35 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 11 Jan 2010 18:48:35 +0100 Subject: Interesting (?) problem In-Reply-To: References: Message-ID: <4B4B6473.5050309@sequans.com> mk wrote: > > Incidentally, it *seems* that list comprehension preserves order: > > hostips_limited = [ h for h in hostips if h in thread_results ] > > Empirically speaking it seems to work (I tested it on real ips), but > please correct me if that's wrong. > > > > Regards, > mk > Sounds good to me. List are *ordered* items ; it does not suprise me that list comprehension consistently keep the order. JM From mrkafk at gmail.com Mon Jan 11 12:57:24 2010 From: mrkafk at gmail.com (mk) Date: Mon, 11 Jan 2010 18:57:24 +0100 Subject: Interesting (?) problem In-Reply-To: References: Message-ID: mk wrote: > Hello everyone, > > I have two lists of IP addresses: > > hostips = [ 'a', 'b', 'c', 'd', 'e' ] > > thread_results = [ 'd', 'b', 'c' ] > > I need to sort thread_results in the same order as hostips. P.S. One clarification: those lists are actually more complicated (thread_result is a list of tuples (ip, thread)), which is why I need thread_results sorted in order of hostips (instead of just constructing [ h for h in hostips if h in thread_results ] and be done with it). From arnodel at googlemail.com Mon Jan 11 13:17:38 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 11 Jan 2010 18:17:38 +0000 Subject: Interesting (?) problem References: Message-ID: mk writes: > Hello everyone, > > I have two lists of IP addresses: > > hostips = [ 'a', 'b', 'c', 'd', 'e' ] > > thread_results = [ 'd', 'b', 'c' ] > > I need to sort thread_results in the same order as hostips. [...solution:] > hostips_limited = [] > for h in hostips: > if h in thread_results: > hostips_limited.append(h) > > ..and then doing sorting thread_results. What do you mean by that last sentence? [... or:] > Incidentally, it *seems* that list comprehension preserves order: > > hostips_limited = [ h for h in hostips if h in thread_results ] That's ok, but why not keep thread_results as a set instead of a list if the ordering in that container is not significant but you are testing membership a lot? -- Arnaud From Erman.CELEN at 3ds.com Mon Jan 11 13:27:07 2010 From: Erman.CELEN at 3ds.com (CELEN Erman) Date: Mon, 11 Jan 2010 13:27:07 -0500 Subject: C Module's '1.#INF' changes to 'inf' at Python In-Reply-To: References: Message-ID: > Numeric.log10() will check to see if the errno was set to ERANGE. It does not > check if a floating point exception flag was set, which is tricky to do across > platforms. The newer numpy can do it because we've finally managed to implement > all of that platform-specific code, but the earlier Numeric does not. > Presumably, the msvc8 C runtime's implementation of log10() sets errno and the > msvc9 runtime's version does not. It doesn't seem like C part is changed. I confirmed that the behavior of log10(0.0) in C's standard math.h library didn't change between compilers (msvc8 and msvc9 both sets the errno to 34(ERANGE)). Now I'm thinking that this setting errno problem is happening somewhere in Numeric's log10 wrappers. As I see, Numeric's "PyUFunc_GenericFunction" checks errno value to see if the called function has set it and if it is non-zero, it calls math_error which raises the exception. The problem is that now errno is not being set but I can't see why since I can't step into that redirected function call ("*(double *)op = ((DoubleUnaryFunc *)func)(*(double *)ip1)" where function value is "0x01c4ede0 log10"). I am wondering which functions are actually being called when I call log10(0.0). Could you (or anybody) point me where this errno is supposed to be set in Numeric or umath when I call log10(0.0) so that I can take a look at why this is not being the case. (I also noticed that this behavior is same under standard NumPy 1.4 with standard Python 2.6 on Windows. If you call numpy.log10(0.0) you will get an "-inf" and no exceptions will be raised. Which is not the case with Python's standard math.log10(0.0) which will raise a ValueError) Best Regards, Ali Erman CELEN Platform Specialists / Porting This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged. If you are not one of the named recipients or have received this email in error, (i) you should not read, disclose, or copy it, (ii) please notify sender of your receipt by reply email and delete this email and all attachments, (iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.For other languages, go to http://www.3ds.com/terms/email-disclaimer. From __peter__ at web.de Mon Jan 11 13:34:51 2010 From: __peter__ at web.de (Peter Otten) Date: Mon, 11 Jan 2010 19:34:51 +0100 Subject: Interesting (?) problem References: Message-ID: mk wrote: > mk wrote: >> Hello everyone, >> >> I have two lists of IP addresses: >> >> hostips = [ 'a', 'b', 'c', 'd', 'e' ] >> >> thread_results = [ 'd', 'b', 'c' ] >> >> I need to sort thread_results in the same order as hostips. > > P.S. One clarification: those lists are actually more complicated > (thread_result is a list of tuples (ip, thread)), which is why I need > thread_results sorted in order of hostips (instead of just constructing > [ h for h in hostips if h in thread_results ] and be done with it). Make it [(host, thread) for host, thread in thread_results if host in hostips] then. However, if you care to explain what you're intending to do with the resulting list I'm sure someone will come up with a more efficient approach based on sets or dicts. Peter From scott.freemire at gmail.com Mon Jan 11 13:35:04 2010 From: scott.freemire at gmail.com (Scott) Date: Mon, 11 Jan 2010 10:35:04 -0800 (PST) Subject: Fundamental Function Question (beginner) Message-ID: <8d7ef0f0-2a0f-4a43-aced-8f4755406249@e27g2000yqd.googlegroups.com> When creating a function is there any difference between putting everything under the "def" or not? Here I created a function called CscoPortNum to convert the network port number field in a Cisco syslog string from a an ascii name back into its numeric form if required. Does it matter at all that I created the translation dictionary first and then started the def? # def CscoPortNum(RulL) # Accept a single ACL Rule as a List split into individual words and # return Port number. Convert from Cisco syslog Port name if required portfpth = "\\progra~1\\syslogd\\ACL_Logs\\Port-Translations.txt" # Create dictionary of portnames to portnumbers portD = {} for prtnmS in open(portfpth): prtnmS = prtnmS.rstrip() spltprtL = prtnmS.split(" ") portD[spltprtL[2]] = [spltprtL[1]] def CscoPortNum(RulL): if "eq" in RulL: # Is the Port listed? if RulL[RulL.index("eq")+1][0].isdigit(): # Is it numeric? # if re.search("\d", RulL[RulL.index("eq")+1][0]): # Is it numeric? portnum = RulL[RulL.index("eq")+1] # If numeric, use as is. else: # If named, look up numeric translation portnum = portD[RulL[RulL.index("eq")+1]] portnum = str(portnum).strip("[]'") else: portnum = "noeq" return portnum From jeanmichel at sequans.com Mon Jan 11 13:36:05 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 11 Jan 2010 19:36:05 +0100 Subject: Interesting (?) problem In-Reply-To: References: Message-ID: <4B4B6F95.1020704@sequans.com> mk wrote: > mk wrote: >> Hello everyone, >> >> I have two lists of IP addresses: >> >> hostips = [ 'a', 'b', 'c', 'd', 'e' ] >> >> thread_results = [ 'd', 'b', 'c' ] >> >> I need to sort thread_results in the same order as hostips. > > P.S. One clarification: those lists are actually more complicated > (thread_result is a list of tuples (ip, thread)), which is why I need > thread_results sorted in order of hostips (instead of just > constructing [ h for h in hostips if h in thread_results ] and be done > with it). > > > > Could be easily done by changing the thread_results structure. You seems to want to lookup using ips: thread_results = {'ip1':['t1','t2','t3'], 'ip3':['t4','t8']} # tx are threads results = [] for ip in hostips: results += [(ip, thread) for thread in thread_results.get(ip, [])] >>> print results Out[10]: [('ip1', 't1'), ('ip1', 't2'), ('ip1', 't3'), ('ip3', 't4'), ('ip3', 't8')] In a general manner, if you end up with duplicated informations inside your structures (like ips in you thread_results structure) that means you'll need sooner or later to do additional loop work to factorize the data. JM From nobody at nowhere.com Mon Jan 11 13:39:25 2010 From: nobody at nowhere.com (Nobody) Date: Mon, 11 Jan 2010 18:39:25 +0000 Subject: integer and string compare, is that correct? References: <2f718029-1682-47f6-a2c3-6e40b51bd71a@v25g2000yqk.googlegroups.com> Message-ID: On Sun, 10 Jan 2010 23:13:55 -0800, Dan Bishop wrote: >> If you actually need to perform comparisons across types, you can rely >> upon the fact that tuple comparisons are non-strict and use e.g.: >> >> ? ? ? ? > a = 5 >> ? ? ? ? > b = '5' >> ? ? ? ? > (type(a).__name__, a) < (type(b).__name__, b) >> ? ? ? ? True >> ? ? ? ? > (type(a).__name__, a) > (type(b).__name__, b) >> ? ? ? ? False >> >> The second elements will only be compared if the first elements are equal >> (i.e. the values have the same type). > > But this method gives you 3.0 < 2 because 'float' < 'int'. Probably > not what you want. If you're comparing instances of entirely arbitrary types, what you probably want (and the only thing you're going to get) is an entirely arbitrary ordering. The main case where such a comparison makes sense is for algorithms which require a total ordering (e.g. tree-like structures), and those won't care if 3<2 so long as the axioms for a total ordering hold. From no.email at nospam.invalid Mon Jan 11 13:42:32 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 11 Jan 2010 10:42:32 -0800 Subject: Interesting (?) problem References: Message-ID: <7xaawkpjon.fsf@ruckus.brouhaha.com> mk writes: > I have two lists of IP addresses: > > hostips = [ 'a', 'b', 'c', 'd', 'e' ] > > thread_results = [ 'd', 'b', 'c' ] > > I need to sort thread_results in the same order as hostips. Assuming each address in hostips appears just once: from itertools import izip,count d = dict(izip(hostips, count())) sorted_results = sorted(thread_results, key=d.get) From nobody at nowhere.com Mon Jan 11 13:47:31 2010 From: nobody at nowhere.com (Nobody) Date: Mon, 11 Jan 2010 18:47:31 +0000 Subject: sys.stdout vs. sys.stderr References: Message-ID: On Mon, 11 Jan 2010 10:09:36 +0100, Martin v. Loewis wrote: >> In Python 3.1 is there any difference in the buffering behavior of the >> initial sys.stdout and sys.stderr streams? > > No. > >> Were they different at some earlier point in Python's evolution? > > That depends on the operating system. These used to be whatever the > C library set up as stdout and stderr. Typically, they were buffered > in the same way. On Unix, stdout will be line buffered if it is associated with a tty and fully buffered otherwise, while stderr is always unbuffered. On Windows, stdout and stderr are unbuffered if they refer to a character device, fully buffered otherwise (Windows doesn't have line buffering; setvbuf(_IOLBF) is equivalent to setvbuf(_IOFBF)). ANSI C says: As initially opened, the standard error stream is not fully buffered; the standard input and standard output streams are fully buffered if and only if the stream can be determined not to refer to an interactive device. From robert.kern at gmail.com Mon Jan 11 13:52:53 2010 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 11 Jan 2010 12:52:53 -0600 Subject: C Module's '1.#INF' changes to 'inf' at Python In-Reply-To: References: Message-ID: On 2010-01-11 12:27 PM, CELEN Erman wrote: >> Numeric.log10() will check to see if the errno was set to ERANGE. It does not >> check if a floating point exception flag was set, which is tricky to do across >> platforms. The newer numpy can do it because we've finally managed to implement >> all of that platform-specific code, but the earlier Numeric does not. >> Presumably, the msvc8 C runtime's implementation of log10() sets errno and the >> msvc9 runtime's version does not. > > It doesn't seem like C part is changed. I confirmed that the behavior of log10(0.0) in C's standard math.h library didn't change between compilers (msvc8 and msvc9 both sets the errno to 34(ERANGE)). Now I'm thinking that this setting errno problem is happening somewhere in Numeric's log10 wrappers. > > As I see, Numeric's "PyUFunc_GenericFunction" checks errno value to see if the called function has set it and if it is non-zero, it calls math_error which raises the exception. The problem is that now errno is not being set but I can't see why since I can't step into that redirected function call ("*(double *)op = ((DoubleUnaryFunc *)func)(*(double *)ip1)" where function value is "0x01c4ede0 log10"). This is the math library's log10() function. > I am wondering which functions are actually being called when I call log10(0.0). Could you (or anybody) point me where this errno is supposed to be set in Numeric or umath when I call log10(0.0) so that I can take a look at why this is not being the case. errno gets set to 0 before PyUFunc_GenericFunction calls the underlying log10() function. Other than that, Numeric does not set errno. > (I also noticed that this behavior is same under standard NumPy 1.4 with standard Python 2.6 on Windows. If you call numpy.log10(0.0) you will get an "-inf" and no exceptions will be raised. Which is not the case with Python's standard math.log10(0.0) which will raise a ValueError) Correct. This is numpy's intended behavior. See numpy.seterr() to enable exceptions if you want them. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From python at mrabarnett.plus.com Mon Jan 11 13:57:09 2010 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 11 Jan 2010 18:57:09 +0000 Subject: Fundamental Function Question (beginner) In-Reply-To: <8d7ef0f0-2a0f-4a43-aced-8f4755406249@e27g2000yqd.googlegroups.com> References: <8d7ef0f0-2a0f-4a43-aced-8f4755406249@e27g2000yqd.googlegroups.com> Message-ID: <4B4B7485.7050409@mrabarnett.plus.com> Scott wrote: > When creating a function is there any difference between putting > everything under the "def" or not? > > Here I created a function called CscoPortNum to convert the network > port number field in a Cisco syslog string from a an ascii name back > into its numeric form if required. Does it matter at all that I > created the translation dictionary first and then started the def? > > # def CscoPortNum(RulL) > # Accept a single ACL Rule as a List split into individual words and > # return Port number. Convert from Cisco syslog Port name if required > portfpth = "\\progra~1\\syslogd\\ACL_Logs\\Port-Translations.txt" > # Create dictionary of portnames to portnumbers > portD = {} > for prtnmS in open(portfpth): > prtnmS = prtnmS.rstrip() > spltprtL = prtnmS.split(" ") > portD[spltprtL[2]] = [spltprtL[1]] > def CscoPortNum(RulL): > if "eq" in RulL: # Is the Port listed? > if RulL[RulL.index("eq")+1][0].isdigit(): # Is it numeric? > # if re.search("\d", RulL[RulL.index("eq")+1][0]): # Is it > numeric? > portnum = RulL[RulL.index("eq")+1] # If numeric, use > as is. > else: > # If named, look up numeric translation > portnum = portD[RulL[RulL.index("eq")+1]] > portnum = str(portnum).strip("[]'") > else: portnum = "noeq" > return portnum There's nothing wrong with building dicts or other lookup tables outside a function in order to avoid re-creating them every time the function is called. From nobody at nowhere.com Mon Jan 11 13:59:01 2010 From: nobody at nowhere.com (Nobody) Date: Mon, 11 Jan 2010 18:59:01 +0000 Subject: pyserial: Unexpected Local Echo References: Message-ID: On Mon, 11 Jan 2010 23:27:03 +0800, Steven Woody wrote: > I am using pyserial. But I always get the local echo after I write > some characters onto serial port and I find no way to disable this > behavior. When I say 'local echo', I mean the next read operation will > get characters that was just write to the same port. That explains the "echo" part. What makes you think that it's local? From aioe.org at technicalbloke.com Mon Jan 11 14:08:49 2010 From: aioe.org at technicalbloke.com (r0g) Date: Mon, 11 Jan 2010 19:08:49 +0000 Subject: Fundamental Function Question (beginner) References: <8d7ef0f0-2a0f-4a43-aced-8f4755406249@e27g2000yqd.googlegroups.com> Message-ID: Scott wrote: > When creating a function is there any difference between putting > everything under the "def" or not? > > Here I created a function called CscoPortNum to convert the network > port number field in a Cisco syslog string from a an ascii name back > into its numeric form if required. Does it matter at all that I > created the translation dictionary first and then started the def? > > # def CscoPortNum(RulL) > # Accept a single ACL Rule as a List split into individual words and > # return Port number. Convert from Cisco syslog Port name if required > portfpth = "\\progra~1\\syslogd\\ACL_Logs\\Port-Translations.txt" > # Create dictionary of portnames to portnumbers > portD = {} > for prtnmS in open(portfpth): > prtnmS = prtnmS.rstrip() > spltprtL = prtnmS.split(" ") > portD[spltprtL[2]] = [spltprtL[1]] > def CscoPortNum(RulL): > if "eq" in RulL: # Is the Port listed? > if RulL[RulL.index("eq")+1][0].isdigit(): # Is it numeric? > # if re.search("\d", RulL[RulL.index("eq")+1][0]): # Is it > numeric? > portnum = RulL[RulL.index("eq")+1] # If numeric, use > as is. > else: > # If named, look up numeric translation > portnum = portD[RulL[RulL.index("eq")+1]] > portnum = str(portnum).strip("[]'") > else: portnum = "noeq" > return portnum In this snippet no, you're not calling the function in the preceding code so there's no problem. You can intersperse functions with the rest of your code however you like, they just wont be visible to the preceding code, but it's better to stick them all at the top of your script. Even better when you have more than a handful is to bundle functions into separate py files and then import that file e.g. ---contents of foobar.py----- def foo(): print "foo" def bar: print "bar" ---contents of your main script----- import foobar print foo(),bar() Roger. From nobody at nowhere.com Mon Jan 11 14:10:30 2010 From: nobody at nowhere.com (Nobody) Date: Mon, 11 Jan 2010 19:10:30 +0000 Subject: Interesting (?) problem References: Message-ID: On Mon, 11 Jan 2010 18:57:24 +0100, mk wrote: >> I have two lists of IP addresses: >> >> hostips = [ 'a', 'b', 'c', 'd', 'e' ] >> >> thread_results = [ 'd', 'b', 'c' ] >> >> I need to sort thread_results in the same order as hostips. > > P.S. One clarification: those lists are actually more complicated > (thread_result is a list of tuples (ip, thread)), which is why I need > thread_results sorted in order of hostips (instead of just constructing > [ h for h in hostips if h in thread_results ] and be done with it). 1. thread_results_dict = dict([(v[0], v) for v in thread_results]) [thread_results_dict[h] for h in hostips if h in thread_results_dict] 2. hostips_dict = dict([(ip, idx) for idx, ip in enumerate(hostips)]) sorted(thread_results, key = lambda r: hostips_dict[r[0]]) From steve at holdenweb.com Mon Jan 11 14:11:11 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 11 Jan 2010 14:11:11 -0500 Subject: Different number of matches from re.findall and re.split In-Reply-To: <4a5ebe07-aac7-40e6-ad1d-12f3d5316703@j19g2000yqk.googlegroups.com> References: <2f8b08af-65fd-4ff9-b90b-db3f689aca01@u7g2000yqm.googlegroups.com> <42ac849d-fecd-4489-8c9c-4ac4dc939898@m3g2000yqf.googlegroups.com> <4a5ebe07-aac7-40e6-ad1d-12f3d5316703@j19g2000yqk.googlegroups.com> Message-ID: Jeremy wrote: > On Jan 11, 8:44 am, Iain King wrote: >> On Jan 11, 3:35 pm, Jeremy wrote: >> >> >> >> >> >>> Hello all, >>> I am using re.split to separate some text into logical structures. >>> The trouble is that re.split doesn't find everything while re.findall >>> does; i.e.: >>>> found = re.findall('^ 1', line, re.MULTILINE) >>>> len(found) >>> 6439 >>>> tables = re.split('^ 1', line, re.MULTILINE) >>>> len(tables) >>>> 1 >>> Can someone explain why these two commands are giving different >>> results? I thought I should have the same number of matches (or maybe >>> different by 1, but not 6000!) >>> Thanks, >>> Jeremy >> re.split doesn't take re.MULTILINE as a flag: it doesn't take any >> flags. It does take a maxsplit parameter, which you are passing the >> value of re.MULTILINE (which happens to be 8 in my implementation). >> Since your pattern is looking for line starts, and your first line >> presumably has more splits than the maxsplits you are specifying, your >> re.split never finds more than 1. > > Yep. Thanks for pointing that out. I guess I just assumed that > re.split was similar to re.search/match/findall in what it accepted as > function parameters. I guess I'll have to use a \n instead of a ^ for > split. > > Thanks, > Jeremy Remember you can specify flags inside the pattern itself. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From arnodel at googlemail.com Mon Jan 11 14:12:10 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 11 Jan 2010 19:12:10 +0000 Subject: integer and string compare, is that correct? References: <2f718029-1682-47f6-a2c3-6e40b51bd71a@v25g2000yqk.googlegroups.com> Message-ID: Nobody writes: > On Sun, 10 Jan 2010 23:13:55 -0800, Dan Bishop wrote: > >>> If you actually need to perform comparisons across types, you can rely >>> upon the fact that tuple comparisons are non-strict and use e.g.: >>> >>> ? ? ? ? > a = 5 >>> ? ? ? ? > b = '5' >>> ? ? ? ? > (type(a).__name__, a) < (type(b).__name__, b) >>> ? ? ? ? True >>> ? ? ? ? > (type(a).__name__, a) > (type(b).__name__, b) >>> ? ? ? ? False >>> >>> The second elements will only be compared if the first elements are equal >>> (i.e. the values have the same type). >> >> But this method gives you 3.0 < 2 because 'float' < 'int'. Probably >> not what you want. > > If you're comparing instances of entirely arbitrary types, what > you probably want (and the only thing you're going to get) is an > entirely arbitrary ordering. > > The main case where such a comparison makes sense is for algorithms which > require a total ordering (e.g. tree-like structures), and those won't care > if 3<2 so long as the axioms for a total ordering hold. It won't work for several reasons. Offhand I can think of two: 1. lists, tuples: >>> [1] < ['a'] Traceback (most recent call last): File "", line 1, in TypeError: unorderable types: int() < str() 2. Partially ordered or unordered builtin types: >>> 1j < 1+1j Traceback (most recent call last): File "", line 1, in TypeError: no ordering relation is defined for complex numbers -- Arnaud From mfmdevine at gmail.com Mon Jan 11 14:13:01 2010 From: mfmdevine at gmail.com (amadain) Date: Mon, 11 Jan 2010 11:13:01 -0800 (PST) Subject: xml.sax parsing elements with the same name Message-ID: <51ad9cee-6b7b-42c8-8f9e-44c0f8b96bc7@j19g2000yqk.googlegroups.com> I have an event log with 100s of thousands of entries with logs of the form: I am using xml.sax to parse the event log. The trouble with the file above is when I parse for result value I get the last result value (Blocked from above). I want to get the result value triggered (the second in the event). my code is as follows: def startElement(self, name, attrs): if name == 'event': self.eventTime = attrs.get('eventTimestamp',"") self.eventUniqueId = attrs.get('uniqueId', "") if name == 'result': self.resultValue = attrs.get('value',"") return def endElement(self, name): if name=="event": result= eval(self.filter) if result: ... How do I get the result value I require when events have the same names like above? From aioe.org at technicalbloke.com Mon Jan 11 14:16:36 2010 From: aioe.org at technicalbloke.com (r0g) Date: Mon, 11 Jan 2010 19:16:36 +0000 Subject: Fundamental Function Question (beginner) References: <8d7ef0f0-2a0f-4a43-aced-8f4755406249@e27g2000yqd.googlegroups.com> Message-ID: r0g wrote: > Scott wrote: >> When creating a function is there any difference between putting >> everything under the "def" or not? >> >> Here I created a function called CscoPortNum to convert the network >> port number field in a Cisco syslog string from a an ascii name back >> into its numeric form if required. Does it matter at all that I >> created the translation dictionary first and then started the def? >> >> # def CscoPortNum(RulL) >> # Accept a single ACL Rule as a List split into individual words and >> # return Port number. Convert from Cisco syslog Port name if required >> portfpth = "\\progra~1\\syslogd\\ACL_Logs\\Port-Translations.txt" >> # Create dictionary of portnames to portnumbers >> portD = {} >> for prtnmS in open(portfpth): >> prtnmS = prtnmS.rstrip() >> spltprtL = prtnmS.split(" ") >> portD[spltprtL[2]] = [spltprtL[1]] >> def CscoPortNum(RulL): >> if "eq" in RulL: # Is the Port listed? >> if RulL[RulL.index("eq")+1][0].isdigit(): # Is it numeric? >> # if re.search("\d", RulL[RulL.index("eq")+1][0]): # Is it >> numeric? >> portnum = RulL[RulL.index("eq")+1] # If numeric, use >> as is. >> else: >> # If named, look up numeric translation >> portnum = portD[RulL[RulL.index("eq")+1]] >> portnum = str(portnum).strip("[]'") >> else: portnum = "noeq" >> return portnum > > > In this snippet no, you're not calling the function in the preceding > code so there's no problem. You can intersperse functions with the rest > of your code however you like, they just wont be visible to the > preceding code, but it's better to stick them all at the top of your > script. Even better when you have more than a handful is to bundle > functions into separate py files and then import that file e.g. > > ---contents of foobar.py----- > def foo(): > print "foo" > def bar: > print "bar" > > ---contents of your main script----- > import foobar > print foo(),bar() > > Roger. Whoops, that should have been... > ---contents of your main script----- > import foobar > print foobar.foo(),foobar.bar() Roger. > From jlconlin at gmail.com Mon Jan 11 14:19:03 2010 From: jlconlin at gmail.com (Jeremy) Date: Mon, 11 Jan 2010 11:19:03 -0800 (PST) Subject: Different number of matches from re.findall and re.split References: <4a5ebe07-aac7-40e6-ad1d-12f3d5316703@j19g2000yqk.googlegroups.com> Message-ID: On Jan 11, 9:28?am, Duncan Booth wrote: > MRAB wrote: > >> Yep. ?Thanks for pointing that out. ?I guess I just assumed that > >> re.split was similar to re.search/match/findall in what it accepted as > >> function parameters. ?I guess I'll have to use a \n instead of a ^ for > >> split. > > > You could use the .split method of a pattern object instead: > > > ? ? ?tables = re.compile('^ 1', re.MULTILINE).split(line) > > or you might include the flag in the regular expression literal: '(?m)^ 1' Another great solution. This is what I will do. Thanks, Jeremy From jlconlin at gmail.com Mon Jan 11 14:20:34 2010 From: jlconlin at gmail.com (Jeremy) Date: Mon, 11 Jan 2010 11:20:34 -0800 (PST) Subject: What is built-in method sub Message-ID: <477f47ea-7cca-4ba3-b225-e16562bdae5e@21g2000yqj.googlegroups.com> I just profiled one of my Python scripts and discovered that >99% of the time was spent in {built-in method sub} What is this function and is there a way to optimize it? Thanks, Jeremy From john at castleamber.com Mon Jan 11 14:26:30 2010 From: john at castleamber.com (John Bokma) Date: Mon, 11 Jan 2010 13:26:30 -0600 Subject: xml.sax parsing elements with the same name References: <51ad9cee-6b7b-42c8-8f9e-44c0f8b96bc7@j19g2000yqk.googlegroups.com> Message-ID: <87k4vo5tp5.fsf@castleamber.com> amadain writes: > I have an event log with 100s of thousands of entries with logs of the > form: > > uniqueId="1261124569.35725_PFS_1_1340035961"> > > > > > > > > > > > > > > > > I am using xml.sax to parse the event log. The trouble with the file > above is when I parse for result value I get the last result value > (Blocked from above). I want to get the result value triggered (the > second in the event). > > my code is as follows: > > def startElement(self, name, attrs): > if name == 'event': > self.eventTime = attrs.get('eventTimestamp',"") > self.eventUniqueId = attrs.get('uniqueId', "") > if name == 'result': > self.resultValue = attrs.get('value',"") > return > > def endElement(self, name): > if name=="event": > result= eval(self.filter) > if result: > ... > > How do I get the result value I require when events have the same > names like above? You have to keep track if you're inside a filters section, and keep track of the filter elements (first, second, etc.) assuming you want the result value of the first filter. -- John Bokma Read my blog: http://johnbokma.com/ Hire me (Perl/Python): http://castleamber.com/ From phlip2005 at gmail.com Mon Jan 11 14:50:14 2010 From: phlip2005 at gmail.com (Phlip) Date: Mon, 11 Jan 2010 11:50:14 -0800 Subject: Fundamental Function Question (beginner) In-Reply-To: References: <8d7ef0f0-2a0f-4a43-aced-8f4755406249@e27g2000yqd.googlegroups.com> Message-ID: MRAB wrote: > Scott wrote: >> for prtnmS in open(portfpth): >> prtnmS = prtnmS.rstrip() > There's nothing wrong with building dicts or other lookup tables outside > a function in order to avoid re-creating them every time the function is > called. However, please consider writing complete, pronounceable names for variables. This looks like Fortran! From tanix at mongo.net Mon Jan 11 14:50:35 2010 From: tanix at mongo.net (tanix) Date: Mon, 11 Jan 2010 19:50:35 GMT Subject: Python Goldmine has been updated: http://preciseinfo.org/Convert/index_Convert_Python.html References: Message-ID: In article , Terry Reedy wrote: >On 1/8/2010 11:50 AM, tanix wrote: >> Python Goldmine collection contains the extensive collection of articles >> going back several years. It includes thousands of code >> examples and expert discussions on all major topics. >> >> The information is organized by relevant topics, covered >> by the corresponding chapters. >> >> The information was filtered with sophisticated filters and vast >> majority of artilces with little relevance have been filtered out. >> >> If you have any specific requests for some new chapters to be added >> and it is of interest to others, please post your requests on this >> thread. >> >> If anyone feels he has above average level of competence, or can >> reccommend someone who posts on this group, you may request to be >> included in the expert chapters. >> >> The Python Goldmine is at: >> >> http://preciseinfo.org/Convert/index_Convert_Python.html >> >> -- >> Programmer's Goldmine collections: >> >> http://preciseinfo.org >> >> Tens of thousands of code examples and expert discussions on >> C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, >> organized by major topics of language, tools, methods, techniques. > >This site pops up spam windowns. One was blocked, one managed to bypass >the popup blocker. Tnis is not friendly behaviour. I am sorry. But this is a known issue. This is one of counter vendors doing these popups. They were contacted about this but they refuse to deal with it. These popups are totally inappropriate. These counters will be removed with the next major site update. >Some categories have 100s of entries. Better, I think, to use a search >engine such as Google with more specific search terms and a snippet of >context for each result. Well, I can not tell you what is better for you. You have to decide on your own. What I can tell you is this: when you do Google search, you are not going to get the most appropriate results. Because their filtering is orders of magnitude less precise. There is currenly work going on to do internal site search that will further increase precision. But even as it stands right now, you are getting the code examples on related issues with > 90% certainty of articles to be on topic. Beyond that, do as you wish. >tjr -- Programmer's Goldmine collections: http://preciseinfo.org Tens of thousands of code examples and expert discussions on C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, organized by major topics of language, tools, methods, techniques. From tanix at mongo.net Mon Jan 11 14:52:02 2010 From: tanix at mongo.net (tanix) Date: Mon, 11 Jan 2010 19:52:02 GMT Subject: Python Goldmine has been updated: http://preciseinfo.org/Convert/index_Convert_Python.html References: Message-ID: In article , Steve Holden wrote: >Terry Reedy wrote: >> On 1/8/2010 11:50 AM, tanix wrote: >>> Python Goldmine collection contains the extensive collection of articles >>> going back several years. It includes thousands of code >>> examples and expert discussions on all major topics. >>> >>> The information is organized by relevant topics, covered >>> by the corresponding chapters. >>> >>> The information was filtered with sophisticated filters and vast >>> majority of artilces with little relevance have been filtered out. >>> >>> If you have any specific requests for some new chapters to be added >>> and it is of interest to others, please post your requests on this >>> thread. >>> >>> If anyone feels he has above average level of competence, or can >>> reccommend someone who posts on this group, you may request to be >>> included in the expert chapters. >>> >>> The Python Goldmine is at: >>> >>> http://preciseinfo.org/Convert/index_Convert_Python.html >>> >>> -- >>> Programmer's Goldmine collections: >>> >>> http://preciseinfo.org >>> >>> Tens of thousands of code examples and expert discussions on >>> C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, >>> organized by major topics of language, tools, methods, techniques. >> >> This site pops up spam windowns. One was blocked, one managed to bypass >> the popup blocker. Tnis is not friendly behaviour. >> >> Some categories have 100s of entries. Better, I think, to use a search >> engine such as Google with more specific search terms and a snippet of >> context for each result. >> >Because I habitually run the NoScript extension to Firefox the popups >didn't appear, but there didn't seem to be any original content on this >site. The site contains the selected articles from this group, filtered with high precision filters. > Google continues to be your friend. >regards > Steve -- Programmer's Goldmine collections: http://preciseinfo.org Tens of thousands of code examples and expert discussions on C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, organized by major topics of language, tools, methods, techniques. From tanix at mongo.net Mon Jan 11 14:54:22 2010 From: tanix at mongo.net (tanix) Date: Mon, 11 Jan 2010 19:54:22 GMT Subject: Python Goldmine has been updated: http://preciseinfo.org/Convert/index_Convert_Python.html References: <4B4A9446.7020009@holdenweb.com> Message-ID: In article , Steve Holden wrote: >Steve Holden wrote: >[...] >> Because I habitually run the NoScript extension to Firefox the popups >> didn't appear, but there didn't seem to be any original content on this >> site. Google continues to be your friend. >> >And dammit, why didn't I think to strip the links out instead of >creating yet one more link to it? Sorry ... There isn't a single ad on any of these sites beyond those nasty popups displayed by 3rd party counter vendor. Sorry, but this is the way it is right now. The simpliest thing is just to close the popup window without even looking at it because it is pretty much guaranteed to be totally off the wall add no one would be interested in seeing anyway. >regards > Steve -- Programmer's Goldmine collections: http://preciseinfo.org Tens of thousands of code examples and expert discussions on C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, organized by major topics of language, tools, methods, techniques. From pavlovevidence at gmail.com Mon Jan 11 14:54:47 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Mon, 11 Jan 2010 11:54:47 -0800 (PST) Subject: What is built-in method sub References: <477f47ea-7cca-4ba3-b225-e16562bdae5e@21g2000yqj.googlegroups.com> Message-ID: <4bfd1268-8bca-4813-be16-709cbff7bf8c@g31g2000vbr.googlegroups.com> On Jan 11, 11:20?am, Jeremy wrote: > I just profiled one of my Python scripts and discovered that >99% of > the time was spent in > > {built-in method sub} > > What is this function and is there a way to optimize it? I'm guessing this is re.sub (or, more likely, a method sub of an internal object that is called by re.sub). If all your script does is to make a bunch of regexp substitutions, then spending 99% of the time in this function might be reasonable. Optimize your regexps to improve performance. (We can help you if you care to share any.) If my guess is wrong, you'll have to be more specific about what your sctipt does, and maybe share the profile printout or something. Carl Banks From mrabarnett at mrabarnett.plus.com Mon Jan 11 14:56:57 2010 From: mrabarnett at mrabarnett.plus.com (Matthew Barnett) Date: Mon, 11 Jan 2010 19:56:57 +0000 Subject: What is built-in method sub In-Reply-To: <477f47ea-7cca-4ba3-b225-e16562bdae5e@21g2000yqj.googlegroups.com> References: <477f47ea-7cca-4ba3-b225-e16562bdae5e@21g2000yqj.googlegroups.com> Message-ID: <4B4B8289.6000901@mrabarnett.plus.com> Jeremy wrote: > I just profiled one of my Python scripts and discovered that >99% of > the time was spent in > > {built-in method sub} > > What is this function and is there a way to optimize it? > > Thanks, > Jeremy From python at mrabarnett.plus.com Mon Jan 11 14:58:29 2010 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 11 Jan 2010 19:58:29 +0000 Subject: What is built-in method sub In-Reply-To: <477f47ea-7cca-4ba3-b225-e16562bdae5e@21g2000yqj.googlegroups.com> References: <477f47ea-7cca-4ba3-b225-e16562bdae5e@21g2000yqj.googlegroups.com> Message-ID: <4B4B82E5.30807@mrabarnett.plus.com> Jeremy wrote: > I just profiled one of my Python scripts and discovered that >99% of > the time was spent in > > {built-in method sub} > > What is this function and is there a way to optimize it? > I think it's the subtraction operator. The only way to optimise it is to reduce the number of subtractions that you do! From philip at semanchuk.com Mon Jan 11 15:00:44 2010 From: philip at semanchuk.com (Philip Semanchuk) Date: Mon, 11 Jan 2010 15:00:44 -0500 Subject: Fundamental Function Question (beginner) In-Reply-To: References: <8d7ef0f0-2a0f-4a43-aced-8f4755406249@e27g2000yqd.googlegroups.com> Message-ID: <3168D89D-0A24-4732-9A53-5E692A28412D@semanchuk.com> On Jan 11, 2010, at 2:50 PM, Phlip wrote: > MRAB wrote: > >> Scott wrote: > >>> for prtnmS in open(portfpth): >>> prtnmS = prtnmS.rstrip() > >> There's nothing wrong with building dicts or other lookup tables >> outside >> a function in order to avoid re-creating them every time the >> function is >> called. > > However, please consider writing complete, pronounceable names for > variables. This looks like Fortran! Wht doYu mn? I thnk hisCde is ez2rd! From jlconlin at gmail.com Mon Jan 11 15:02:06 2010 From: jlconlin at gmail.com (Jeremy) Date: Mon, 11 Jan 2010 12:02:06 -0800 (PST) Subject: What is built-in method sub References: <477f47ea-7cca-4ba3-b225-e16562bdae5e@21g2000yqj.googlegroups.com> <4bfd1268-8bca-4813-be16-709cbff7bf8c@g31g2000vbr.googlegroups.com> Message-ID: <4777eb9a-0aa4-4e5f-8d61-225852691856@26g2000yqo.googlegroups.com> On Jan 11, 12:54?pm, Carl Banks wrote: > On Jan 11, 11:20?am, Jeremy wrote: > > > I just profiled one of my Python scripts and discovered that >99% of > > the time was spent in > > > {built-in method sub} > > > What is this function and is there a way to optimize it? > > I'm guessing this is re.sub (or, more likely, a method sub of an > internal object that is called by re.sub). > > If all your script does is to make a bunch of regexp substitutions, > then spending 99% of the time in this function might be reasonable. > Optimize your regexps to improve performance. ?(We can help you if you > care to share any.) > > If my guess is wrong, you'll have to be more specific about what your > sctipt does, and maybe share the profile printout or something. > > Carl Banks Your guess is correct. I had forgotten that I was using that function. I am using the re.sub command to remove trailing whitespace from lines in a text file. The commands I use are copied below. If you have any suggestions on how they could be improved, I would love to know. Thanks, Jeremy lines = self._outfile.readlines() self._outfile.close() line = string.join(lines) if self.removeWS: # Remove trailing white space on each line trailingPattern = '(\S*)\ +?\n' line = re.sub(trailingPattern, '\\1\n', line) From invalid at invalid.invalid Mon Jan 11 15:02:37 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Mon, 11 Jan 2010 20:02:37 +0000 (UTC) Subject: pyserial: Unexpected Local Echo References: Message-ID: On 2010-01-11, Steven Woody wrote: > I am using pyserial. But I always get the local echo after I > write some characters onto serial port I really doubt you're getting a local echo. Is the data coming out the serial port? Do you get the echo if you disconnect the serial cable? > and I find no way to disable this behavior. When I say 'local > echo', I mean the next read operation will get characters that > was just write to the same port. The device to which you're connected is echoing them. There's also a chance that your rxd line is floating and there's enough crosstalk in the cable to "echo" the data, but I'll bet money it's not being done locally (in the serial driver or port). > I run my program on cygwin (pyserial was also built on the > system from source code) and the serial port i am using is a > USB adapter that simulates a port (COM4 on my XP) because my > laptop don't have a real serial port. But I checked my COM4 > settings, there is no any think like 'local echo'. -- Grant Edwards grante Yow! Will it improve my at CASH FLOW? visi.com From mfmdevine at gmail.com Mon Jan 11 15:08:13 2010 From: mfmdevine at gmail.com (amadain) Date: Mon, 11 Jan 2010 12:08:13 -0800 (PST) Subject: xml.sax parsing elements with the same name References: <51ad9cee-6b7b-42c8-8f9e-44c0f8b96bc7@j19g2000yqk.googlegroups.com> <87k4vo5tp5.fsf@castleamber.com> Message-ID: <74552e4f-f60f-482f-9830-98c104e938a7@m16g2000yqc.googlegroups.com> On Jan 11, 7:26?pm, John Bokma wrote: > amadain writes: > > I have an event log with 100s of thousands of entries with logs of the > > form: > > > > uniqueId="1261124569.35725_PFS_1_1340035961"> > > ? ? > > ? ? ? > > ? ? ? ? ? > > ? ? ? ? ? ? ? > > ? ? ? ? ? ? ? ? ? ? > > ? ? ? ? ? ? ? > > ? ? ? ? ? > > ? ? ? ? ? > > ? ? ? ? ? ? ? > > ? ? ? ? ? ? ? ? ? ? > > ? ? ? ? ? ? ? > > ? ? ? ? ? > > ? ? ? > > > > > I am using xml.sax to parse the event log. The trouble with the file > > above is when I parse for result value I get the last result value > > (Blocked from above). I want to get the result value triggered (the > > second in the event). > > > my code is as follows: > > > ? ? def startElement(self, name, attrs): > > ? ? ? ? if name == 'event': > > ? ? ? ? ? ? self.eventTime = attrs.get('eventTimestamp',"") > > ? ? ? ? ? ? self.eventUniqueId = attrs.get('uniqueId', "") > > ? ? ? ? if name == 'result': > > ? ? ? ? ? ? self.resultValue = attrs.get('value',"") > > ? ? ? ? return > > > ? ? def endElement(self, name): > > ? ? ? ? if name=="event": > > ? ? ? ? ? ? result= eval(self.filter) > > ? ? ? ? ? ? if result: > > ? ? ? ? ? ?... > > > How do I get the result value I require when events have the same > > names like above? > > You have to keep track if you're inside a filters section, and keep > track of the filter elements (first, second, etc.) assuming you want the > result value of the first filter. > > -- > John Bokma > > Read my blog:http://johnbokma.com/ > Hire me (Perl/Python):http://castleamber.com/ how do I keep track? The first result value is outside a filters section and the rest are. Do you mean something like: def startElement(self, name, attrs): if name == 'event': self.eventTime = attrs.get('eventTimestamp',"") self.eventUniqueId = attrs.get('uniqueId', "") if name == 'result': self.resultValue = attrs.get('value',"") if name == filters: if name == 'result': self.resultValueF = attrs.get('value',"") return A From anthra.norell at bluewin.ch Mon Jan 11 15:09:43 2010 From: anthra.norell at bluewin.ch (Anthra Norell) Date: Mon, 11 Jan 2010 21:09:43 +0100 Subject: lightweight encryption of text file In-Reply-To: References: <4B4851C6.1050009@bluewin.ch> Message-ID: <4B4B8587.9080601@bluewin.ch> Robert Kern wrote: > On 2010-01-09 03:52 AM, Anthra Norell wrote: >> Daniel Fetchinson wrote: >> > I have a plain text file which I would like to protect in a very >> > simple minded, yet for my purposes sufficient, way. I'd like to >> > encrypt/convert it into a binary file in such a way that >> possession of >> > a password allows anyone to convert it back into the original text >> > file while not possessing the password one would only see the >> > following with the standard linux utility 'file': >> > >> > [fetchinson at fetch ~]$ file encrypted.data >> > encrypted.data: data >> > >> > and the effort required to convert the file back to the original text >> > file without the password would be equivalent to guessing the >> > password. >> > >> > I'm fully aware of the security implications of this loose >> > specification, but for my purposes this would be a good solution. >> > >> > What would be the simplest way to achieve this using preferably stock >> > python without 3rd party modules? If a not too complex 3rd part >> > module made it really simple that would be acceptable too. >> >> Daniel, >> >> Here's what looks like another thread veering off into package-ology, >> leaving a stumped OP behind. >> >> "Don't use a random generator for encryption purposes!" warns the >> manual, of which fact I was reminded in no uncertain terms on this forum >> a few years ago when I proposed the following little routine in response >> to a post very similar to yours. One critic challenged me to encode my >> credit card data and post it. Which I did. > > Actually, you just "encrypted" your credit card number and challenged > comp.lang.python to crack it. No one challenged you to do anything of > the sort. Fortunately, the ever-watchful eye of Google was upon us > that day: > > http://groups.google.com/group/comp.lang.python/browse_thread/thread/5fb9ffada975bae9?pli=1 My dear Robert. Thank you for the clarification. You are right. The thread recorded by Google doesn't mention the credit card detail. I remember it distinctly, though. I also remember that it wasn't my idea. And I recall being urged by another, well-mannered, member of this group to call it off right away. He wrote--I pretty much quote: "...there must be a number of machines out there grinding away at your code right now!" >> Upon which another critic >> conjured up the horror vision of gigahertzes hacking my pathetic little >> effort to pieces as I was reading his message. Of the well-meaning kind, >> he urged me to put an immediate stop to this foolishness. I didn't. >> >> No unplanned expenditures ensued. > > That's because comp.lang.python is not full of thieves, not because > your algorithm is worth a damn. You're right about the thieves. You have a point about my algorithm, although you might express it in a fashion that lives up to its merits. My algorithm would not resist a brute-force attack that iterates through all possible keys and analyzes the outcome for non-randomness. I knew that then and so I posted a second-level encryption, that is, an encryption of an encryption. Thus the brute-force attack wouldn't find anything non-random. By not disclosing the detail I may have breached some formal rule of the craft. If I had disclosed it, I have my doubts that the processing power available at a sensible cost would have done the job. The keys I said I used were long integers. There are roughly 4.3 billion of them. Cycling through all of them would surely take a considerable number of hours on a PC. If each one of the 4.3 billion top-level decryptions required another bottom-level run many hours long, the chances of cracking the code before dying of old age are very, very, very remote. I may be wrong about that. If you know better, your knowledge would serve the community a lot better than your ill temper. > > p3.py imposes no more overhead than this, but it has some real > security properties. To quote Paul Rubin from that previous thread: > > """ > Since good encryption schemes that don't have significant performance > penalties are widely available, why mess with a crap scheme EVER? Why > use a solution that "might or might not be adequate" when you can use > one that's definitely ok? > """ > Excellent point! Why EVER make anything yourself when you can buy it? Another excellent point! Cheers Frederic From deets at nospam.web.de Mon Jan 11 15:15:25 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Mon, 11 Jan 2010 21:15:25 +0100 Subject: What is built-in method sub In-Reply-To: <4777eb9a-0aa4-4e5f-8d61-225852691856@26g2000yqo.googlegroups.com> References: <477f47ea-7cca-4ba3-b225-e16562bdae5e@21g2000yqj.googlegroups.com> <4bfd1268-8bca-4813-be16-709cbff7bf8c@g31g2000vbr.googlegroups.com> <4777eb9a-0aa4-4e5f-8d61-225852691856@26g2000yqo.googlegroups.com> Message-ID: <7r1f6tFcrfU1@mid.uni-berlin.de> Jeremy schrieb: > On Jan 11, 12:54 pm, Carl Banks wrote: >> On Jan 11, 11:20 am, Jeremy wrote: >> >>> I just profiled one of my Python scripts and discovered that >99% of >>> the time was spent in >>> {built-in method sub} >>> What is this function and is there a way to optimize it? >> I'm guessing this is re.sub (or, more likely, a method sub of an >> internal object that is called by re.sub). >> >> If all your script does is to make a bunch of regexp substitutions, >> then spending 99% of the time in this function might be reasonable. >> Optimize your regexps to improve performance. (We can help you if you >> care to share any.) >> >> If my guess is wrong, you'll have to be more specific about what your >> sctipt does, and maybe share the profile printout or something. >> >> Carl Banks > > Your guess is correct. I had forgotten that I was using that > function. > > I am using the re.sub command to remove trailing whitespace from lines > in a text file. The commands I use are copied below. If you have any > suggestions on how they could be improved, I would love to know. > > Thanks, > Jeremy > > lines = self._outfile.readlines() > self._outfile.close() > > line = string.join(lines) > > if self.removeWS: > # Remove trailing white space on each line > trailingPattern = '(\S*)\ +?\n' > line = re.sub(trailingPattern, '\\1\n', line) line = line.rstrip()? Diez From suresh.amritapuri at gmail.com Mon Jan 11 15:19:03 2010 From: suresh.amritapuri at gmail.com (suresh.amritapuri) Date: Mon, 11 Jan 2010 12:19:03 -0800 (PST) Subject: PIL how to display multiple images side by side References: <6d9fef57-7fdc-46ec-9b53-7aa824e785e9@s31g2000yqs.googlegroups.com> <4b496473$1@dnews.tpgi.com.au> Message-ID: <34dd6abe-d9e5-4a9f-b396-256c19541b69@b2g2000yqi.googlegroups.com> On Jan 9, 9:51?pm, "Alf P. Steinbach" wrote: > * Lie Ryan: > > > On 1/9/2010 8:43 AM, suresh.amritapuri wrote: > >> Hi, > > >> In PIL, how to display multiple images in say m rows and n colums when > >> I have m*n images. > > >> suresh > > > Tkinter has PhotoImage widget and PIL has support for this widget: > >http://www.pythonware.com/library/pil/handbook/imagetk.htm > > Maybe I've misunderstood something (in that case glad to learn!), but I believe > PhotoImage is not a widget, and that a PhotoImage has to be presented in e.g. a > Label widget or some other widget that's able to display images. > > Cheers, > > - Alf Hi, Let me paste the code I have adapted from somewhere. I dont get a clue on how to go further so that all the images would stay in their respective positions. thanks suresh def button_click_exit_mainloop (event): event.widget.quit() # this will cause mainloop to unblock. root = Tkinter.Tk() root.bind("    
> b) If there's already a PYTHONPATH, Edit it, adding a semi-colon and > the full path of folder Module to the end. > > 5) Put your module folders into the folder Module. > > 6) (Here's a really arcane bit.) Into each module folder, put a file > named __init__.py. It will be executed when you load the module. It > can be empty, but it has to be there or else the module folder will be > ignored. Cheers & hth., - Alf Notes: [1] Windows XP and I believe also Vista places a limit of about 8 KiB on the length of any environment variable. From wuwei23 at gmail.com Sun Jan 17 23:15:27 2010 From: wuwei23 at gmail.com (alex23) Date: Sun, 17 Jan 2010 20:15:27 -0800 (PST) Subject: The answer References: Message-ID: <8fc5ec25-03ba-4347-9521-cd3a73864474@a15g2000yqm.googlegroups.com> On Jan 18, 12:30?pm, Jive Dadson wrote: > These instructions are for MS Windows. > > 1) Create your modules folder. Let's say it's named "Modules." ?The > documentation calls it a "package." > > 2) In an explorer window or on the desktop, right click on My Computer, > and select Properties. > > 3) Select the Advanced tab, and click on Environment Variables near the > bottom. > > 4) Look for an environment variable named PYTHONPATH. > > ? ? a) If you do not find one, create one using the New button(s). I > don't know if it has to be in User Variables or System Variables. ?To > save time experimenting, I just put one in both. For the value, put the > full path of the folder Modules. > > ? ? b) If there's already a PYTHONPATH, ?Edit it, adding a semi-colon > and the full path of folder Module to the end. > > 5) Put your module folders into the folder Module. > > 6) (Here's a really arcane bit.) Into each module folder, put a file > named __init__.py. ?It will be executed when you load the module. ?It > can be empty, but it has to be there or else the module folder will be > ignored. Actually, if you're using Python 2.6+/3.x, you can effectively skip steps 1-5, as these versions now support user site-packages. Rather than create a Module folder and modify your PYTHONPATH, add (if it doesn't exist already) the following folder: %APPDATA%/Python/Python26/site-packages Modules can sit directly in the folder, or within packages. For more details: http://www.python.org/dev/peps/pep-0370/ From wuwei23 at gmail.com Sun Jan 17 23:40:41 2010 From: wuwei23 at gmail.com (alex23) Date: Sun, 17 Jan 2010 20:40:41 -0800 (PST) Subject: IDLE Namespace Toolbox? Windows. References: Message-ID: <8cacd724-4654-4ef8-9407-f501b55b8986@a15g2000yqm.googlegroups.com> On Jan 18, 8:22?am, "W. eWatson" wrote: > I've found it a bit aggravating that using dir and > help, for example, that the output just rolls on off the screen and I > have to play around with the shell scroll bars to find what I'm looking > for. A few simple changes to mydir should change that, but I would hope > or think maybe there are even more tools to generally help. If you haven't already, you really should check out IPython: http://ipython.scipy.org It's an enhanced interactive shell packed full of convenience features. For example, output is by default paged, so you'll get the behaviour you want from help() and dir() straight away. However, you'll probably end up using IPython's help instead: ? will not only display the docstring, it'll provide metadata about the object, such as its base class, the file it was defined in and even the namespace it exists in. %page will pretty print the object and run it through the pager. %timeit is an _exceptionally_ handy wrapper around the timeit module. %bg runs in a separate, background thread There's a directory stack, macros, code in history can be edited, profiling & debugging, functions can be called without parenthesis (nice if you use IPython as a shell replacement), and you can easily capture the results of a command line call to a variable. But yes, along with all that, it pages object printing :) From notontheweb at noisp.com Sun Jan 17 23:49:41 2010 From: notontheweb at noisp.com (Jive Dadson) Date: Sun, 17 Jan 2010 20:49:41 -0800 Subject: The answer In-Reply-To: References: <4OP4n.99891$IU1.53386@en-nntp-04.dc1.easynews.com> Message-ID: alex23 wrote: > > Actually, if you're using Python 2.6+/3.x, you can effectively skip > steps 1-5, as these versions now support user site-packages. > > Rather than create a Module folder and modify your PYTHONPATH, add (if > it doesn't exist already) the following folder: > %APPDATA%/Python/Python26/site-packages > > Modules can sit directly in the folder, or within packages. > > For more details: http://www.python.org/dev/peps/pep-0370/ That requires a directory whose name embeds the Python version number, which is the evil from which I flee, or rather sought to flee. Imagine if all your C++ code had to go into directories that were named for some specific C++ compiler. It's just WRONG. It's a maintenance nightmare to have a bunch of different source files that all have to be updated whenever you fix a bug or add a feature. From wuwei23 at gmail.com Mon Jan 18 00:00:16 2010 From: wuwei23 at gmail.com (alex23) Date: Sun, 17 Jan 2010 21:00:16 -0800 (PST) Subject: The answer References: <4OP4n.99891$IU1.53386@en-nntp-04.dc1.easynews.com> Message-ID: <70363743-37ed-4c0d-b0c5-a70e75ddd7f9@e16g2000yqc.googlegroups.com> Jive Dadson wrote: > That requires a directory whose name embeds the Python version number, > which is the evil from which I flee, or rather sought to flee. ?Imagine > if all your C++ code had to go into directories that were named for some > specific C++ compiler. ?It's just WRONG. ?It's a maintenance nightmare > to have a bunch of different source files that all have to be updated > whenever you fix a bug or add a feature. With the PEP 370 approach, you can just designate a folder the core one and symlink all other versions to it. Or better yet, use version control to push updates to each folder. With all of the versions sharing one folder, as you'd prefer, it would be a lot more difficult to actually achieve version-level granularity if and when you need it. It's not WRONG, it's just providing a level of control that you don't need at this point. From afriere at yahoo.co.uk Mon Jan 18 00:30:07 2010 From: afriere at yahoo.co.uk (Asun Friere) Date: Sun, 17 Jan 2010 21:30:07 -0800 (PST) Subject: enhancing 'list' References: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> Message-ID: <88c73449-222f-4646-8fc3-4307e4bffcb4@u41g2000yqe.googlegroups.com> On Jan 18, 9:37?am, samwyse wrote: > Consider this a wish list. ?I know I'm unlikely to get any of these in > time for for my birthday, but still I felt the need to toss it out and > see what happens. > > Lately, I've slinging around a lot of lists, and there are some simple > things I'd like to do that just aren't there. > If memory serves me correctly, it has been possible to subclass 'built- in' types since Py2.2 or thereabouts. From steven at REMOVE.THIS.cybersource.com.au Mon Jan 18 00:34:59 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 18 Jan 2010 05:34:59 GMT Subject: Inheriting methods but over-riding docstrings References: <0361ea2d$0$1336$c3e8da3@news.astraweb.com> Message-ID: On Mon, 18 Jan 2010 01:45:17 +0100, Alf P. Steinbach wrote: > Steven: on a personal note, earlier when I saw you (I think it was you) > using the "Norwegian Parrot" example I thought it referred to me because > that was the only sense I could make of it, it followed right after some > discussion we had. Thus my impression of you or or responses in this > group was colored by a false interpretation. But, checking, which is > often a good idea!, and *which I should have done then*, as far as I can > see the term was first used in this group in April 2001, http://groups.google.com/group/comp.lang.python/browse_thread/ thread/12a125ceddd401e2/c021547a1dc14a41>. > > It's still a mystery to me what it refers to, though... :-) It refers to the famous Monty Python "Dead Parrot Sketch", involving a Norwegian Blue parrot that definitely isn't dead but merely pining for the fjords. http://www.mtholyoke.edu/~ebarnes/python/dead-parrot.htm http://en.wikipedia.org/wiki/Dead_Parrot No doubt you'll be able to find it on Youtube. A little-known FAQ is that Python is named for Monty Python, not the snake, and that traditionally metasyntactic variables such as foo, bar etc. are frequently named after Monty Python sketches. E.g. I will frequently reference the Cheeseshop sketch, the Spanish Inquisition, Ethel the Aardvark, Spam (the lunch meat, not the email), and similar. E.g. instead of foo, bar, baz, we frequently use spam, ham, eggs. http://www.python.org/doc/faq/general/#why-is-it-called-python -- Steven From notontheweb at noisp.com Mon Jan 18 00:44:44 2010 From: notontheweb at noisp.com (Jive Dadson) Date: Sun, 17 Jan 2010 21:44:44 -0800 Subject: Arrrrgh! Another module broken In-Reply-To: References: Message-ID: Matt Newville wrote: > On Jan 17, 7:25 pm, Jive Dadson wrote: >> I just found another module that broke when I went to 2.6. > Gnuplot. Apparently one of its routines has a parameter >> named "with." That used to be okay, and now it's not. > > This was fixed in version 1.8 of Gnuplot.py > >> Once I get everything to work under 2.6, I am using it >> forever or until new releases no longer break working >> code, whichever comes first. > > Hey, good luck with that forever plan. > > --Matt I have source code that uses the "with" kwarg. What do I replace it with? From clp2 at rebertia.com Mon Jan 18 00:55:36 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 17 Jan 2010 21:55:36 -0800 Subject: Arrrrgh! Another module broken In-Reply-To: References: Message-ID: <50697b2c1001172155s2e9e251ep185143be280d7fb2@mail.gmail.com> On Sun, Jan 17, 2010 at 9:44 PM, Jive Dadson wrote: > Matt Newville wrote: >> >> On Jan 17, 7:25 pm, Jive Dadson wrote: >>> >>> I just found another module that broke when I went to 2.6. > Gnuplot. >>> ?Apparently one of its routines has a parameter >>> named "with." ?That used to be okay, and now it's not. >> >> This was fixed in version 1.8 of Gnuplot.py >> >>> Once I get everything to work under 2.6, I am using it >>> forever or until new releases no longer break working >>> code, whichever comes first. >> >> Hey, good luck with that forever plan. > > I have source code that uses the "with" kwarg. What do I replace it with? Rename the argument to something else that's not a Python keyword. When keyword conflicts occur, appending an underscore (i.e. with_) is a popular option, as is using a nonstandard spelling (e.g. wif) Cheers, Chris -- http://blog.rebertia.com From aioe.org at technicalbloke.com Mon Jan 18 00:57:18 2010 From: aioe.org at technicalbloke.com (r0g) Date: Mon, 18 Jan 2010 05:57:18 +0000 Subject: Arrrrgh! Another module broken References: Message-ID: Jive Dadson wrote: > Matt Newville wrote: >> On Jan 17, 7:25 pm, Jive Dadson wrote: >>> I just found another module that broke when I went to 2.6. > >>> Gnuplot. Apparently one of its routines has a parameter >>> named "with." That used to be okay, and now it's not. >> >> This was fixed in version 1.8 of Gnuplot.py >> >>> Once I get everything to work under 2.6, I am using it >>> forever or until new releases no longer break working >>> code, whichever comes first. >> >> Hey, good luck with that forever plan. >> >> --Matt > > I have source code that uses the "with" kwarg. What do I replace it with? Good old fashioned open() and close() Roger. From steve at holdenweb.com Mon Jan 18 01:09:41 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 18 Jan 2010 01:09:41 -0500 Subject: pyserial: Unexpected Local Echo In-Reply-To: References: <4b50c6bf$0$1676$742ec2ed@news.sonic.net> Message-ID: Steven Woody wrote: > 2010/1/16 John Nagle : >> Grant Edwards wrote: >>> On 2010-01-11, Steven Woody wrote: >>> >>>> I am using pyserial. But I always get the local echo after I >>>> write some characters onto serial port >>> I really doubt you're getting a local echo. Is the data coming >>> out the serial port? Do you get the echo if you disconnect the >>> serial cable? >>> >>>> and I find no way to disable this behavior. When I say 'local >>>> echo', I mean the next read operation will get characters that >>>> was just write to the same port. >>> The device to which you're connected is echoing them. There's >>> also a chance that your rxd line is floating and there's enough >>> crosstalk in the cable to "echo" the data, but I'll bet money >>> it's not being done locally (in the serial driver or port). >>> >>>> I run my program on cygwin (pyserial was also built on the >>>> system from source code) and the serial port i am using is a >>>> USB adapter that simulates a port (COM4 on my XP) because my >>>> laptop don't have a real serial port. But I checked my COM4 >>>> settings, there is no any think like 'local echo'. >> You're using what? Some version of Python built on Cygwin >> running on a Windows XP system? What if you just run a stock >> Python built for Windows on Windows XP? Or run Linux? That >> half-and-half environment may not work right. pyserial has >> special cases in it for Windows and Linux, and it's not >> clear what it will do on Cygwin. >> >> That said, if you're getting echo from output back to input, >> I'd look at the USB to serial device. I've used devices with >> the Silicon Laboratories CP2102 part, and they work fine. >> ("http://www.aetherltd.com/connectingusb.html") >> >> Do you have something plugged into the serial port? If >> so, what? >> >> John Nagle >> -- >> http://mail.python.org/mailman/listinfo/python-list >> > > Now I tried run the same test code using pure Windows python + > pyserial, the result is same. > > Actually, the USB device is a optic head, that read data from an > electric energy meter: > > PC USB Port -> Optic Head -> Meter > > I get the echo even when the meter itself is disconnected, that mean > the echo is not generated by firmware inside the meter. So, if the > echo is also not generated from my PC, that much be from the optic > head itself. Is it possible? > > Regards, > woody > Of course this echo is good, as it allows you to verify that your transmission has been received ... somewhere ... regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From lie.1296 at gmail.com Mon Jan 18 01:14:12 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Mon, 18 Jan 2010 17:14:12 +1100 Subject: The answer In-Reply-To: References: Message-ID: <4b53fc49$1@dnews.tpgi.com.au> On 01/18/10 13:30, Jive Dadson wrote: > Okay, with your help I've figured it out. Instructions are below, but > read the caveat by Ben Fenny in this thread. All this stuff is good for > one default version of Python only. The PYTHONPATH described below, for > example, cannot specify a version number. Yes, that's a pain in the > butt, but there's no way around it. If you switch versions, you may > have to delete all the .pyc files that will show up in the module > folders. Python ought to check them to see if they are valid, but I do > not know if it does so. Err... "The answer" to... what? From dieter at handshake.de Mon Jan 18 01:25:58 2010 From: dieter at handshake.de (Dieter Maurer) Date: 18 Jan 2010 07:25:58 +0100 Subject: setattr() oddness In-Reply-To: <4b517acc$1@dnews.tpgi.com.au> References: <3eba07d0-fc14-4f51-a777-1c47cc866387@j14g2000yqm.googlegroups.com> <409dce36-691c-4c9f-ae69-aba358377b6a@j4g2000yqe.googlegroups.com> <4b517acc$1@dnews.tpgi.com.au> Message-ID: Lie Ryan writes on Sat, 16 Jan 2010 19:37:29 +1100: > On 01/16/10 10:10, Sean DiZazzo wrote: > > Interesting. I can understand the "would take time" argument, but I > > don't see any legitimate use case for an attribute only accessible via > > getattr(). Well, at least not a pythonic use case. > > mostly for people (ab)using attributes instead of dictionary. Here is one use case: A query application. Queries are described by complex query objects. For efficiency reasons, query results should be cached. For this, it is not unnatural to use query objects as cache keys. Then, query objects must not get changed in an uncontrolled way. I use "__setattr__" to control access to the objects. Dieter From steven at REMOVE.THIS.cybersource.com.au Mon Jan 18 01:47:59 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 18 Jan 2010 06:47:59 GMT Subject: setattr() oddness References: <3eba07d0-fc14-4f51-a777-1c47cc866387@j14g2000yqm.googlegroups.com> <409dce36-691c-4c9f-ae69-aba358377b6a@j4g2000yqe.googlegroups.com> <4b517acc$1@dnews.tpgi.com.au> Message-ID: On Mon, 18 Jan 2010 07:25:58 +0100, Dieter Maurer wrote: > Lie Ryan writes on Sat, 16 Jan 2010 19:37:29 +1100: >> On 01/16/10 10:10, Sean DiZazzo wrote: >> > Interesting. I can understand the "would take time" argument, but I >> > don't see any legitimate use case for an attribute only accessible >> > via getattr(). Well, at least not a pythonic use case. >> >> mostly for people (ab)using attributes instead of dictionary. > > Here is one use case: > > A query application. Queries are described by complex query objects. > For efficiency reasons, query results should be cached. For this, it is > not unnatural to use query objects as cache keys. Then, query objects > must not get changed in an uncontrolled way. I use "__setattr__" to > control access to the objects. (1) Wouldn't it be more natural to store these query keys in a list or dictionary rather than as attributes on an object? e.g. instead of: cache.__setattr__('complex query object', value) use: cache['complex query object'] = value (2) How does __setattr__ let you control access to the object? If a user wants to modify the cache, and they know the complex query object, what's stopping them from using __setattr__ too? -- Steven From nagle at animats.com Mon Jan 18 01:56:47 2010 From: nagle at animats.com (John Nagle) Date: Sun, 17 Jan 2010 22:56:47 -0800 Subject: Problems with collision response In-Reply-To: References: Message-ID: <4b5402b9$0$1589$742ec2ed@news.sonic.net> Joabos wrote: > I'm doing a project, and I need to insert collision detection and response on > it. Here's the code. What am I doing wrong? Where to begin... 1. Most of your code will never be executed because it is inside multiline triple-quoted strings. 2. Your approach to collision response probably won't handle multiple simultaneous collisions. 3. You're expecting others to debug your code. John Nagle From notontheweb at noisp.com Mon Jan 18 02:08:02 2010 From: notontheweb at noisp.com (Jive Dadson) Date: Sun, 17 Jan 2010 23:08:02 -0800 Subject: More version woes Message-ID: This has to do with Komodo. I cannot use Python 2.4, because numpy is broken on my machine for that release for reasons unknown. I want to use 2.6 anyway. But when I use Python 2.6 and Komodo 3.5, it runs slow as death. I think it might have something to do with the warning I'm getting. Does anyone know what this is about, and what to do about it? Or can anyone tell me why Komodo 3.5 is slow horribly slow with Python 2.6? C:\Program Files\ActiveState Komodo 3.5\lib\support\dbgp\pythonlib\dbgp\client.py:813: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 self.message = message C:\Program Files\ActiveState Komodo 3.5\lib\support\dbgp\pythonlib\dbgp\client.py:815: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 return self._cmd_error % (self.name,self.tid,self.errid,self.message) The offending bit of source... class CommandError(Exception): """Breakpoint class a simple exception that knows how to serialize itself to xml. """ _cmd_error = '' def __init__(self, name, tid, errid, message): self.name = name self.tid = tid self.errid = errid self.message = message # THIS IS WHAT IT DOESN'T LIKE def __str__(self): return self._cmd_error % (self.name,self.tid,self.errid,self.message) From notontheweb at noisp.com Mon Jan 18 02:11:38 2010 From: notontheweb at noisp.com (Jive Dadson) Date: Sun, 17 Jan 2010 23:11:38 -0800 Subject: Arrrrgh! Another module broken In-Reply-To: References: Message-ID: Jive Dadson wrote: > Matt Newville wrote: >> On Jan 17, 7:25 pm, Jive Dadson wrote: >>> I just found another module that broke when I went to 2.6. > >>> Gnuplot. Apparently one of its routines has a parameter >>> named "with." That used to be okay, and now it's not. >> >> This was fixed in version 1.8 of Gnuplot.py >> >>> Once I get everything to work under 2.6, I am using it >>> forever or until new releases no longer break working >>> code, whichever comes first. >> >> Hey, good luck with that forever plan. >> >> --Matt > > I have source code that uses the "with" kwarg. What do I replace it with? I found the answer. The new kwarg in Gnuplot is "with_". From notontheweb at noisp.com Mon Jan 18 02:15:28 2010 From: notontheweb at noisp.com (Jive Dadson) Date: Sun, 17 Jan 2010 23:15:28 -0800 Subject: More version woes In-Reply-To: References: Message-ID: Sorry. That deprecation warning has nothing to do with the slowness. It does torque my jaw, however. Komodo costs money, and Python 2.6 broke it. @#^&!!! (Again.) So, the new question is, does anyone know how to make Komodo 3.5 run at speed with Python 2.6? Or perhaps better yet, can someone suggest a good IDE that doesn't cost much? From tjreedy at udel.edu Mon Jan 18 02:56:16 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 18 Jan 2010 02:56:16 -0500 Subject: enhancing 'list' In-Reply-To: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> References: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> Message-ID: On 1/17/2010 5:37 PM, samwyse wrote: > Consider this a wish list. I know I'm unlikely to get any of these in > time for for my birthday, but still I felt the need to toss it out and > see what happens. > > Lately, I've slinging around a lot of lists, and there are some simple > things I'd like to do that just aren't there. > > s.count(x[, cmp[, key]]) > - return number of i?s for which s[i] == x. 'cmp' specifies a custom > comparison function of two arguments, as in '.sort'. 'key' specifies > a custom key extraction function of one argument. > s.index(x[, i[, j[, cmp[, key]]]]) > - return smallest k such that s[k] == x and i<= k< j. 'cmp' and > 'key' are as above. > s.rindex(x[, i[, j[, cmp[, key]]]]) > - return largest k such that s[k] == x and i<= k< j. 'cmp' and > 'key' are as above. > > There are two overlapping proposals here. One is to add the .rindex > method, which strings already have. The other is to extend the > optional arguments of .sort to all other methods that test for item > equality. > > One last thing, the Python 2.6.2 spec says .count and .index only > apply to mutable sequence types. I see no reason why they > (and .rindex) couldn't also apply to immutable sequences (tuples, in > particular). In 3.x, tuple does have those methods, even though the doc is not clear (unless fixed by now). From steven at REMOVE.THIS.cybersource.com.au Mon Jan 18 03:02:36 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 18 Jan 2010 08:02:36 GMT Subject: More version woes References: Message-ID: On Sun, 17 Jan 2010 23:15:28 -0800, Jive Dadson wrote: > Sorry. That deprecation warning has nothing to do with the slowness. It > does torque my jaw, however. Komodo costs money, and Python 2.6 broke > it. @#^&!!! (Again.) > > So, the new question is, does anyone know how to make Komodo 3.5 run at > speed with Python 2.6? Or perhaps better yet, can someone suggest a > good IDE that doesn't cost much? Perhaps you should be asking the Komodo forums? Or since you paid for it, perhaps you should be asking customer support? I suspect that they'll probably tell you that since the latest version of Komodo is 5.2, and you're using 3.5, you should stop using a version that is over four years old and almost certainly not supported. -- Steven From notontheweb at noisp.com Mon Jan 18 03:08:35 2010 From: notontheweb at noisp.com (Jive Dadson) Date: Mon, 18 Jan 2010 00:08:35 -0800 Subject: More version woes In-Reply-To: References: Message-ID: <7GU4n.73060$oC1.63466@en-nntp-01.dc1.easynews.com> Steven D'Aprano wrote: > On Sun, 17 Jan 2010 23:15:28 -0800, Jive Dadson wrote: > >> Sorry. That deprecation warning has nothing to do with the slowness. It >> does torque my jaw, however. Komodo costs money, and Python 2.6 broke >> it. @#^&!!! (Again.) >> >> So, the new question is, does anyone know how to make Komodo 3.5 run at >> speed with Python 2.6? Or perhaps better yet, can someone suggest a >> good IDE that doesn't cost much? > > Perhaps you should be asking the Komodo forums? > > Or since you paid for it, perhaps you should be asking customer support? > > I suspect that they'll probably tell you that since the latest version of > Komodo is 5.2, and you're using 3.5, you should stop using a version that > is over four years old and almost certainly not supported. > > Yep. They want $295, and I cannot justify that for personal use. I'll be looking for a cheaper one. From steven at REMOVE.THIS.cybersource.com.au Mon Jan 18 03:59:23 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 18 Jan 2010 08:59:23 GMT Subject: More version woes References: <7GU4n.73060$oC1.63466@en-nntp-01.dc1.easynews.com> Message-ID: On Mon, 18 Jan 2010 00:08:35 -0800, Jive Dadson wrote: >> I suspect that they'll probably tell you that since the latest version >> of Komodo is 5.2, and you're using 3.5, you should stop using a version >> that is over four years old and almost certainly not supported. >> >> >> > Yep. They want $295, and I cannot justify that for personal use. I'll > be looking for a cheaper one. This is why I use a text editor and the interactive console :) -- Steven From __peter__ at web.de Mon Jan 18 04:06:42 2010 From: __peter__ at web.de (Peter Otten) Date: Mon, 18 Jan 2010 10:06:42 +0100 Subject: enhancing 'list' References: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> Message-ID: samwyse wrote: > Lately, I've slinging around a lot of lists, and there are some simple > things I'd like to do that just aren't there. > > s.count(x[, cmp[, key]]) > - return number of i?s for which s[i] == x. 'cmp' specifies a custom > comparison function of two arguments, as in '.sort'. 'key' specifies > a custom key extraction function of one argument. What's your use case exactly? If I were to enhance count/index/rindex I would go for the simpler >>> missing = object() >>> class List(list): ... def count(self, value=missing, predicate=missing): ... if value is missing: ... if predicate is missing: ... raise TypeError ... return sum(1 for item in self if predicate(item)) ... else: ... if predicate is not missing: ... raise TypeError ... return list.count(self, value) ... >>> items = List(range(10)) >>> items.count(7) 1 >>> items.count(predicate=lambda item: item%3) 6 which nicely covers all applications I can imagine. Peter From deets at nospam.web.de Mon Jan 18 04:09:27 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Mon, 18 Jan 2010 10:09:27 +0100 Subject: More version woes In-Reply-To: <7GU4n.73060$oC1.63466@en-nntp-01.dc1.easynews.com> References: <7GU4n.73060$oC1.63466@en-nntp-01.dc1.easynews.com> Message-ID: <7rimq8Fcp7U1@mid.uni-berlin.de> > Yep. They want $295, and I cannot justify that for personal use. I'll be > looking for a cheaper one. There are bazillion discussions in this NG about IDEs and editors. GIYF. Diez From bettini at dsi.unifi.it Mon Jan 18 04:13:04 2010 From: bettini at dsi.unifi.it (Lorenzo Bettini) Date: Mon, 18 Jan 2010 10:13:04 +0100 Subject: LAST CALL FOR PAPERS: TOOLS EUROPE 2010 Message-ID: <4b542623$2@news.x-privat.org> ========================================================================== LAST CALL FOR PAPERS (Deadline: January 22, 2010) TOOLS EUROPE 2010 48th International Conference Objects, Models, Components, Patterns Co-located with *** International Conference on Model Transformation (ICMT 2010) *** *** International Conference on Software Composition (SC 2010) *** *** International Conference on Tests and Proofs (TAP 2010) *** M?laga - Spain, 28 June - 02 July 2010 http://malaga2010.lcc.uma.es/ ========================================================================== TOOLS EUROPE is devoted to the combination of technologies that have emerged as a result of object technology becoming "mainstream". Like its predecessors, TOOLS EUROPE combines an emphasis on quality with a strong practical focus. Started in 1989, TOOLS conferences, held in Europe, the USA, Australia, China and Eastern Europe, have played a major role in the development of object technology; many of seminal concepts were first presented at TOOLS. After an interruption of four years, the conference was revived in 2007 to reflect the maturing of the field and the new challenges ahead and has become a yearly event. Contributions are solicited on all aspects of object technology and related fields, in particular model-based development, component-based development, and patterns (design, analysis and other applications); more generally, any contribution addressing topics in advanced software technology fall within the scope of TOOLS. Reflecting the practical emphasis of TOOLS, contributions showcasing applications along with a sound conceptual contribution are particularly welcome. Topics include: * Object technology, including programming techniques, languages, tools * Testing of object-oriented systems * Patterns, pattern languages, tool support for patterns * Distributed and concurrent object systems * Real-time object-oriented programming and design * Experience reports, including efforts at standardisation * Applications to safety- and security-related software * Component-based programming, modelling, tools * Aspects and aspect-oriented programming and modelling * Frameworks for component-based development * Trusted and reliable components * Model-driven development and Model-Driven Architecture * Domain specific languages and language design * Tools and frameworks for supporting model-driven development * Language implementation techniques, compilers, run-time systems * Practical applications of program verification and analysis * Open source solutions & Reproduction studies All contributions will be subject to a rigorous selection process by the international Program Committee, with a stress on originality, practicality and overall quality. The proceedings will be published in Springer LNBIP. For detailed submission information see the conference page. Important Dates: Papers submission deadline: January 22, 2010 Acceptance notification: March 24, 2010 Camera-ready final copy: April 5, 2010 Conference: June 28 -- July 02, 2010 Conference Chair: Bertrand Meyer, ETH Z?rich and Eiffel Software Program Chair: Jan Vitek, Purdue University Publicity Chair: Osmar Santos, University of York Program Committee: Uwe Assman, University of Dresden, Germany Elisa Baniassad, Chinese University of Hong Kong, Hong Kong Alexandre Bergel, University of Chile, Chile Lorenzo Bettini, University of Torino, Italy Judith Bishop, Microsoft Research, USA William Cook, University of Texas Austin, USA Sophia Drossopolou, Imperial College London, UK Catherine Dubois, ENSIIE, France St?phane Ducasse, INRIA Lille, France Manuel Fahndrich, Microsoft Research, USA Harald Gall, University of Zurich, Switzerland Benoit Garbinato, University of Lausanne, Switzerland Angelo Gargantini, University of Bergamo, Italy Jeff Gray, University of Alabama Birmingham, USA Kathryn Gray, University of Cambridge, UK Thomas Gschwind, IBM Research, Switzerland Matthias Hauswith, University of Lugano, Switzerland Nigel Horspool, University of Victoria, Canada Tomas Kalibera, Charles University, Czech Republic Gerti Kappel, Vienna University of Technology, Austria Doug Lea, State University of New York Oswego, USA Shane Markstrum, Brucknell University, USA Peter M?ller, ETH Zurich, Switzerland Oscar Nierstrasz, University of Bern, Switzerland James Noble, Victoria University of Wellington, New Zealand Nate Nystrom, University of Texas Arlington, USA Manuel Oriol, University of York, UK Jonathan Ostroff, York University, Canada Richard Paige, University of York, UK Shaz Qadeer, Microsoft Research, USA Awais Rashid, Lancaster University, UK Vivek Sarkar, Rice University, USA Doug Schmidt, Vanderbilt University, USA Manuel Serrano, INRIA Sophia Antipolis, France Peter Thiemann, University of Freiburg, Germany Dave Thomas, Bedarra Research Labs, Canada Laurence Tratt, Bournemouth University, UK Mandana Vaziri, IBM Research, USA Tian Zhao, University of Wisconsin-Milwaukee, USA From gabriele.modena at gmail.com Mon Jan 18 04:33:44 2010 From: gabriele.modena at gmail.com (Gabriele Modena) Date: Mon, 18 Jan 2010 10:33:44 +0100 Subject: ctypes: nested structures and pointers Message-ID: <1fe1d5d61001180133gb27a01dsb7434087909aff0@mail.gmail.com> Hi all, I am trying to learn ctypes and I am facing some problems In wrapping two nested structs. <--- begin C code ----> struct dev_callbacks; // Prototype the callback struct typedef struct { const struct dev_callbacks* pdc; char acName[DEVICE_NAME_LENGTH]; chip_type ct; dev_spec ds; bool bActive; bool bCrc; bool bPar; uint8_t ui8TxBits; } dev_info; struct dev_callbacks { const char* acDriver; dev_info* (*connect)(const uint32_t uiIndex); bool (*transceive)(const dev_spec ds, const byte_t* pbtTx, const uint32_t uiTxLen, byte_t* pbtRx, uint32_t* puiRxLen); void (*disconnect)(dev_info* pdi); }; <---- End C code ----> I have two problems with this code: 1. what is the correct (pythonic) way to capture the prototype definition of dev_callbacks and the relation between that structure and dev_info? 2. is it correct to wrap "connect", "transceive" and "disconnect" in separate structures and reference them within pyDEV_CALLBACKS? The (stub) code I wrote for now looks like this: class pyDEV_CALLBACKS(Structure): _fields_ = [("acDriver", c_char_p), ("connect", c_void_p), ("transceive", c_bool), ("disconnect", c_void_p) ] class pyDEV_INFO(Structure): _fields_ = [ ("pdc", POINTER(pyDEV_CALLBACKS)), ("ct", c_ubyte), ("ds", c_void_p), ("acName", c_char * 256), ("bActive", c_bool), ("bCrc", c_bool), ("bPar", c_bool), ("ui8TxBits", c_ubyte) ] Passing the data structures to wrapper functions seem to work (the data is initialized), but the result is not what expected (I presume the problems are related to memory alignment due to wrong declarations), -- Gabriele From anandvaidya.ml at gmail.com Mon Jan 18 04:58:42 2010 From: anandvaidya.ml at gmail.com (Anand Vaidya) Date: Mon, 18 Jan 2010 01:58:42 -0800 (PST) Subject: Generic Python Benchmark suite? Message-ID: Is there a generic python benchmark suite in active development? I am looking forward to comparing some code on various python implementations (primarily CPython 2.x, CPython 3.x, UnladenSwallow, Psyco). I am happy with something that gives me a relative number eg: ULS is 30% faster than CPy2.x etc I found pybench which is probably not maintained actively. What do you suggest? PS: I think a benchmark should cover file / network, database I/O, data structures (dict, list etc), object creation/manipulation, numbers, measure looping inefficiencies, effects of caching (memcache etc) at the minimum From utente at esempio.net Mon Jan 18 05:15:37 2010 From: utente at esempio.net (superpollo) Date: Mon, 18 Jan 2010 11:15:37 +0100 Subject: substitution Message-ID: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> hi. what is the most pythonic way to substitute substrings? eg: i want to apply: foo --> bar baz --> quux quuux --> foo so that: fooxxxbazyyyquuux --> barxxxquuxyyyfoo bye From utente at esempio.net Mon Jan 18 05:21:54 2010 From: utente at esempio.net (superpollo) Date: Mon, 18 Jan 2010 11:21:54 +0100 Subject: substitution In-Reply-To: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> Message-ID: <4b543642$0$1109$4fafbaef@reader4.news.tin.it> superpollo ha scritto: > hi. > > what is the most pythonic way to substitute substrings? > > eg: i want to apply: > > foo --> bar > baz --> quux > quuux --> foo > > so that: > > fooxxxbazyyyquuux --> barxxxquuxyyyfoo > > bye i explain better: say the subs are: quuux --> foo foo --> bar baz --> quux then i cannot apply the subs in sequence (say, .replace() in a loop), otherwise: fooxxxbazyyyquuux --> fooxxxbazyyyfoo --> barxxxbazyyybar --> barxxxquuxyyybar not as intended... From stefan_ml at behnel.de Mon Jan 18 05:30:16 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 18 Jan 2010 11:30:16 +0100 Subject: Generic Python Benchmark suite? In-Reply-To: References: Message-ID: <4b543838$0$7618$9b4e6d93@newsspool1.arcor-online.net> Anand Vaidya, 18.01.2010 10:58: > Is there a generic python benchmark suite in active development? > [...] > PS: I think a benchmark should cover file / network, database I/O, > data structures (dict, list etc), object creation/manipulation, > numbers, measure looping inefficiencies, effects of caching (memcache > etc) at the minimum That doesn't sound generic at all. Maybe you should prefer an application based benchmark instead. Stefan From __peter__ at web.de Mon Jan 18 05:45:37 2010 From: __peter__ at web.de (Peter Otten) Date: Mon, 18 Jan 2010 11:45:37 +0100 Subject: substitution References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> Message-ID: superpollo wrote: > superpollo ha scritto: >> hi. >> >> what is the most pythonic way to substitute substrings? >> >> eg: i want to apply: >> >> foo --> bar >> baz --> quux >> quuux --> foo >> >> so that: >> >> fooxxxbazyyyquuux --> barxxxquuxyyyfoo >> >> bye > > i explain better: > > say the subs are: > > quuux --> foo > foo --> bar > baz --> quux > > then i cannot apply the subs in sequence (say, .replace() in a loop), > otherwise: > > fooxxxbazyyyquuux --> fooxxxbazyyyfoo --> barxxxbazyyybar --> > barxxxquuxyyybar > > not as intended... If you want to avoid regular expressions: def replace_many(s, pairs): if len(pairs): a, b = pairs[0] rest = pairs[1:] return b.join(replace_many(t, rest) for t in s.split(a)) else: return s assert replace_many("abc", ["ab", "bc", "ca"]) == "bca" assert (replace_many("fooxxxbazyyyquuux", [("quuux", "foo"), ("foo", "bar"), ("baz", "quux")]) == "barxxxquuxyyyfoo") Not tested. Peter From gklein at xs4all.nl Mon Jan 18 05:52:36 2010 From: gklein at xs4all.nl (Gertjan Klein) Date: Mon, 18 Jan 2010 11:52:36 +0100 Subject: maintain 2 versions of python on my computer References: <88886fe9-31ee-4ecc-9109-b645ceb46260@c34g2000yqn.googlegroups.com> <7a1f55ee-4c12-4f56-989a-abcd1c441365@j4g2000yqe.googlegroups.com> Message-ID: <16f8l51ui0eiq7p7lldoha772oslo5tqs6@4ax.com> Duncan Booth wrote: > @(C:\Python26\Python -x %~f0 %* || pause) && goto:EOF > import sys > print sys.version > # raise RuntimeError # uncomment to trigger the 'pause' That is nice! This should probably be in the documentation, here: http://docs.python.org/using/cmdline.html#cmdoption-x ... instead of "This is intended for a DOS specific hack only.". Gertjan. From phlip2005 at gmail.com Mon Jan 18 06:03:26 2010 From: phlip2005 at gmail.com (Phlip) Date: Mon, 18 Jan 2010 03:03:26 -0800 (PST) Subject: Is python not good enough? References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> Message-ID: <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> On Jan 12, 7:09?am, ikuta liu wrote: > Go language try to merge low level, hight level and browser language. Go uses := for assignment. This means, to appease the self-righteous indignation of the math professor who would claim = should mean "equality"... ...you gotta type a shift and 2 characters for a very common operator. Pass! -- Phlip From davea at ieee.org Mon Jan 18 06:03:52 2010 From: davea at ieee.org (Dave Angel) Date: Mon, 18 Jan 2010 06:03:52 -0500 Subject: The answer In-Reply-To: References: <4OP4n.99891$IU1.53386@en-nntp-04.dc1.easynews.com> Message-ID: <4B544018.7090508@ieee.org> Jive Dadson wrote: >
alex23 > wrote: > > > > Actually, if you're using Python 2.6+/3.x, you can effectively skip > > steps 1-5, as these versions now support user site-packages. > > > > Rather than create a Module folder and modify your PYTHONPATH, add (if > > it doesn't exist already) the following folder: > > %APPDATA%/Python/Python26/site-packages > > > > Modules can sit directly in the folder, or within packages. > > > > For more details: http://www.python.org/dev/peps/pep-0370/ > > That requires a directory whose name embeds the Python version number, > which is the evil from which I flee, or rather sought to flee. > Imagine if all your C++ code had to go into directories that were > named for some specific C++ compiler. It's just WRONG. It's a > maintenance nightmare to have a bunch of different source files that > all have to be updated whenever you fix a bug or add a feature. > > As others have pointed out, you need a "deploy" script, which in your case would copy the files from source control to the appropriate production folder. That's analogous to the compile, link and deploy steps of C++. And if you want to be even more analogous, copy just the .pyc files, after building them. Certainly you have lots more files in your version control system which are not intended to be copied to the "Modules" folder, such as the test suite. DaveA From steve at REMOVE-THIS-cybersource.com.au Mon Jan 18 06:10:39 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 18 Jan 2010 11:10:39 GMT Subject: substitution References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> Message-ID: <00ddc6b1$0$15570$c3e8da3@news.astraweb.com> On Mon, 18 Jan 2010 11:15:37 +0100, superpollo wrote: > hi. > > what is the most pythonic way to substitute substrings? > > eg: i want to apply: > > foo --> bar > baz --> quux > quuux --> foo > > so that: > > fooxxxbazyyyquuux --> barxxxquuxyyyfoo For simple cases, just use replace: >>> s = 'fooxxxbazyyyquuux' >>> s = s.replace('foo', 'bar') >>> s = s.replace('baz', 'quux') >>> s = s.replace('quuux', 'foo') >>> s == 'barxxxquuxyyyfoo' True In complicated cases, such as if there are conflicts or overlaps between strings, you may need to use a regular expression, or even parse the string yourself. The problem is that "substitute multiple strings" is not well defined, because in general it depends on the order you perform them. You can define a function to do the replacements in one order, but for another use you might need a different order. E.g.: replacing "a" -> "X" and "aa" -> "Y", if you have the string "aaa" what result do you expect? You could get "XXX", "XY" or "YX". None of these are wrong, it depends on which you prefer. -- Steven From python.list at tim.thechases.com Mon Jan 18 06:19:48 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Mon, 18 Jan 2010 05:19:48 -0600 Subject: Is python not good enough? In-Reply-To: <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> Message-ID: <4B5443D4.4020408@tim.thechases.com> Phlip wrote: > On Jan 12, 7:09 am, ikuta liu wrote: >> Go language try to merge low level, hight level and browser language. > > Go uses := for assignment. > > This means, to appease the self-righteous indignation of the math > professor who would claim = should mean "equality"... > > ...you gotta type a shift and 2 characters for a very common operator. > > Pass! Pass?! no...Pascal! :-) -tkc From samwyse at gmail.com Mon Jan 18 06:22:59 2010 From: samwyse at gmail.com (samwyse) Date: Mon, 18 Jan 2010 03:22:59 -0800 (PST) Subject: enhancing 'list' References: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> Message-ID: <375ab6da-58c2-4975-bc79-22b2219a3216@r24g2000yqd.googlegroups.com> On Jan 18, 1:56?am, Terry Reedy wrote: > On 1/17/2010 5:37 PM, samwyse wrote: > > > > > > > Consider this a wish list. ?I know I'm unlikely to get any of these in > > time for for my birthday, but still I felt the need to toss it out and > > see what happens. > > > Lately, I've slinging around a lot of lists, and there are some simple > > things I'd like to do that just aren't there. > > > s.count(x[, cmp[, key]]) > > - return number of i?s for which s[i] == x. ?'cmp' specifies a custom > > comparison function of two arguments, as in '.sort'. ?'key' specifies > > a custom key extraction function of one argument. > > s.index(x[, i[, j[, cmp[, key]]]]) > > - return smallest k such that s[k] == x and i<= k< ?j. ?'cmp' and > > 'key' are as above. > > s.rindex(x[, i[, j[, cmp[, key]]]]) > > - return largest k such that s[k] == x and i<= k< ?j. ?'cmp' and > > 'key' are as above. > > > There are two overlapping proposals here. ?One is to add the .rindex > > method, which strings already have. ?The other is to extend the > > optional arguments of .sort to all other methods that test for item > > equality. > > > One last thing, the Python 2.6.2 spec says .count and .index only > > apply to mutable sequence types. ?I see no reason why they > > (and .rindex) couldn't also apply to immutable sequences (tuples, in > > particular). > > In 3.x, tuple does have those methods, even though the doc is not clear > (unless fixed by now). That's good to hear. Perhaps I should have tried them directyly, but my 3.1 docs still echo the 2.x docs, which only show them for immutable sequences. From samwyse at gmail.com Mon Jan 18 06:59:07 2010 From: samwyse at gmail.com (samwyse) Date: Mon, 18 Jan 2010 03:59:07 -0800 (PST) Subject: enhancing 'list' References: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> Message-ID: On Jan 18, 3:06?am, Peter Otten <__pete... at web.de> wrote: > samwyse wrote: > > Lately, I've slinging around a lot of lists, and there are some simple > > things I'd like to do that just aren't there. > > > s.count(x[, cmp[, key]]) > > - return number of i?s for which s[i] == x. ?'cmp' specifies a custom > > comparison function of two arguments, as in '.sort'. ?'key' specifies > > a custom key extraction function of one argument. > > What's your use case exactly? If I were to enhance count/index/rindex I > would go for the simpler > > >>> missing = object() ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? > >>> class List(list): ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? > > ... ? ? def count(self, value=missing, predicate=missing): ? ? ? ? ? ? > ... ? ? ? ? ? ? if value is missing: > ... ? ? ? ? ? ? ? ? ? ? if predicate is missing: > ... ? ? ? ? ? ? ? ? ? ? ? ? ? ? raise TypeError > ... ? ? ? ? ? ? ? ? ? ? return sum(1 for item in self if predicate(item)) > ... ? ? ? ? ? ? else: > ... ? ? ? ? ? ? ? ? ? ? if predicate is not missing: > ... ? ? ? ? ? ? ? ? ? ? ? ? ? ? raise TypeError > ... ? ? ? ? ? ? ? ? ? ? return list.count(self, value) > ...>>> items = List(range(10)) > >>> items.count(7) > 1 > >>> items.count(predicate=lambda item: item%3) > > 6 > > which nicely covers all applications I can imagine. > > Peter That is a good idea. However, I was looking more at the simplicity of building of ideas that are already present in .sort. And this implementation is pretty simple as well. >>> class List(list): import __builtin__ def count(self, value, cmp=__builtin__.cmp): return sum(1 for item in self if not cmp(item, value)) >>> items = List(range(10)) >>> items.count(7) 1 >>> items.count(3, lambda a, b: not a%b) # My way 6 >>> items.count(Ellipsis, lambda a, b: not a%3) # Your way 6 As a side note, wouldn't it be nice if '...' could be used in more places than just slices? IMHO, a useful idiom would be to use it to signify "irrelevant" or "don't care", as opposed to 'None' which (in my mind, at least) signifies "missing" or "unknown". From samwyse at gmail.com Mon Jan 18 06:59:53 2010 From: samwyse at gmail.com (samwyse) Date: Mon, 18 Jan 2010 03:59:53 -0800 (PST) Subject: enhancing 'list' References: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> <88c73449-222f-4646-8fc3-4307e4bffcb4@u41g2000yqe.googlegroups.com> Message-ID: On Jan 17, 11:30?pm, Asun Friere wrote: > On Jan 18, 9:37?am, samwyse wrote: > > > Consider this a wish list. ?I know I'm unlikely to get any of these in > > time for for my birthday, but still I felt the need to toss it out and > > see what happens. > > > Lately, I've slinging around a lot of lists, and there are some simple > > things I'd like to do that just aren't there. > > If memory serves me correctly, it has been possible to subclass 'built- > in' types since Py2.2 or thereabouts. True, but I've had bad experiences doing that. See, for example, http://groups.google.com/group/comp.lang.python/browse_thread/thread/10cfe2affc265ac where I tried to subclass 'int'. More importantly, subclassing means that people have to keep re-inventing the same methods. Having a single implementation would save time, not to mention the speed advantages of implementing them in the hosting language (C, Java, .NET, etc). From __peter__ at web.de Mon Jan 18 07:20:19 2010 From: __peter__ at web.de (Peter Otten) Date: Mon, 18 Jan 2010 13:20:19 +0100 Subject: enhancing 'list' References: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> Message-ID: samwyse wrote: > On Jan 18, 3:06 am, Peter Otten <__pete... at web.de> wrote: >> samwyse wrote: >> > Lately, I've slinging around a lot of lists, and there are some simple >> > things I'd like to do that just aren't there. >> >> > s.count(x[, cmp[, key]]) >> > - return number of i?s for which s[i] == x. 'cmp' specifies a custom >> > comparison function of two arguments, as in '.sort'. 'key' specifies >> > a custom key extraction function of one argument. >> >> What's your use case exactly? If I were to enhance count/index/rindex I >> would go for the simpler >> >> >>> missing = object() >> >>> class List(list): >> >> ... def count(self, value=missing, predicate=missing): >> ... if value is missing: >> ... if predicate is missing: >> ... raise TypeError >> ... return sum(1 for item in self if predicate(item)) >> ... else: >> ... if predicate is not missing: >> ... raise TypeError >> ... return list.count(self, value) >> ...>>> items = List(range(10)) >> >>> items.count(7) >> 1 >> >>> items.count(predicate=lambda item: item%3) >> >> 6 >> >> which nicely covers all applications I can imagine. >> >> Peter > > That is a good idea. However, I was looking more at the simplicity of > building of ideas that are already present in .sort. And this > implementation is pretty simple as well. Note that the cmp() builtin and the cmp parameter for list.sort() are gone in Python 3. >>>> class List(list): > import __builtin__ > def count(self, value, cmp=__builtin__.cmp): > return sum(1 for item in self if not cmp(item, value)) > > >>>> items = List(range(10)) >>>> items.count(7) > 1 >>>> items.count(3, lambda a, b: not a%b) # My way > 6 >>>> items.count(Ellipsis, lambda a, b: not a%3) # Your way > 6 > > As a side note, wouldn't it be nice if '...' could be used in more > places than just slices? IMHO, a useful idiom would be to use it to > signify "irrelevant" or "don't care", as opposed to 'None' which (in > my mind, at least) signifies "missing" or "unknown". That is a pretty subtle distinction... I prefer keyword arguments, but in Python 3 you can use the ellipsis literal freely: >>> ... == ... True >>> [..., 42, ...].count(...) 2 Peter From samwyse at gmail.com Mon Jan 18 07:28:51 2010 From: samwyse at gmail.com (samwyse) Date: Mon, 18 Jan 2010 04:28:51 -0800 (PST) Subject: The answer References: Message-ID: <460a00bf-e87d-443b-b631-eaaa42ed4b4c@r24g2000yqd.googlegroups.com> On Jan 17, 8:30?pm, Jive Dadson wrote: > Okay, with your help I've figured it out. ?Instructions are below, but > read the caveat by Ben Fenny in this thread. ?All this stuff is good for > one default version of Python only. ?The PYTHONPATH described below, for > example, cannot specify a version number. ?Yes, that's a pain in the > butt, but there's no way around it. ?If you switch versions, you may > have to delete all the .pyc files that will show up in the module > folders. ?Python ought to check them to see if they are valid, but I do > not know if it does so. > > These instructions are for MS Windows. > > 1) Create your modules folder. Let's say it's named "Modules." ?The > documentation calls it a "package." > > 2) In an explorer window or on the desktop, right click on My Computer, > and select Properties. > > 3) Select the Advanced tab, and click on Environment Variables near the > bottom. > > 4) Look for an environment variable named PYTHONPATH. > > ? ? a) If you do not find one, create one using the New button(s). I > don't know if it has to be in User Variables or System Variables. ?To > save time experimenting, I just put one in both. For the value, put the > full path of the folder Modules. > > ? ? b) If there's already a PYTHONPATH, ?Edit it, adding a semi-colon > and the full path of folder Module to the end. > > 5) Put your module folders into the folder Module. > > 6) (Here's a really arcane bit.) Into each module folder, put a file > named __init__.py. ?It will be executed when you load the module. ?It > can be empty, but it has to be there or else the module folder will be > ignored. In your original thread, you never quite said why you can't use site- packages and .pth files. Are you not allowed to modify your local installation? If you are writing something for distribution to others, then site-packages and .pth files are the best way to go, since they don't assume any particular operating system. If you can't (or won't) use them, then just create Module as a sub-directory of wherever your program lives, since that directory is always prepended to PYTHONPATH. If you need to use the same module from multiple directories, most modern operating systems support symbolic links; if you're using Windows, well, here's a nickel kid, get yourself a better computer (http://farm1.static.flickr.com/ 89/240711122_f9888e5a3b_o.jpg). I don't think that __init__.py is very arcane, since it is described in detail in the documentation. It's also a great place to use the standard site.addsitedir() function, which is another platform independent way to manipulate Python's search path. From iainking at gmail.com Mon Jan 18 07:41:41 2010 From: iainking at gmail.com (Iain King) Date: Mon, 18 Jan 2010 04:41:41 -0800 (PST) Subject: substitution References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> Message-ID: <924f9bbd-ae54-4984-b6c7-a8950b5d4000@c29g2000yqd.googlegroups.com> On Jan 18, 10:21?am, superpollo wrote: > superpollo ha scritto: > > > hi. > > > what is the most pythonic way to substitute substrings? > > > eg: i want to apply: > > > foo --> bar > > baz --> quux > > quuux --> foo > > > so that: > > > fooxxxbazyyyquuux --> barxxxquuxyyyfoo > > > bye > > i explain better: > > say the subs are: > > quuux --> foo > foo --> bar > baz --> quux > > then i cannot apply the subs in sequence (say, .replace() in a loop), > otherwise: > > fooxxxbazyyyquuux --> fooxxxbazyyyfoo --> barxxxbazyyybar --> > barxxxquuxyyybar > > not as intended... Not sure if it's the most pythonic, but I'd probably do it like this: def token_replace(string, subs): subs = dict(subs) tokens = {} for i, sub in enumerate(subs): tokens[sub] = i tokens[i] = sub current = [string] for sub in subs: new = [] for piece in current: if type(piece) == str: chunks = piece.split(sub) new.append(chunks[0]) for chunk in chunks[1:]: new.append(tokens[sub]) new.append(chunk) else: new.append(piece) current = new output = [] for piece in current: if type(piece) == str: output.append(piece) else: output.append(subs[tokens[piece]]) return ''.join(output) >>> token_replace("fooxxxbazyyyquuux", [("quuux", "foo"), ("foo", "bar"), ("baz", "quux")]) 'barxxxquuxyyyfoo' I'm sure someone could whittle that down to a handful of list comps... Iain From anthra.norell at bluewin.ch Mon Jan 18 07:43:49 2010 From: anthra.norell at bluewin.ch (Anthra Norell) Date: Mon, 18 Jan 2010 13:43:49 +0100 Subject: substitution In-Reply-To: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> Message-ID: <4B545785.8090003@bluewin.ch> superpollo wrote: > hi. > > what is the most pythonic way to substitute substrings? > > eg: i want to apply: > > foo --> bar > baz --> quux > quuux --> foo > > so that: > > fooxxxbazyyyquuux --> barxxxquuxyyyfoo > > bye Try the code below the dotted line. It does any number of substitutions and handles overlaps correctly (long over short) Your case: >>> substitutions = (('foo', 'bar'), ('baz', 'quux'), ('quuux', 'foo')) # Sequence of doublets >>> T = Translator (substitutions) # Compile substitutions -> translator >>> s = 'fooxxxbazyyyquuux' # Your source string >>> d = 'barxxxquuxyyyfoo' # Your destination string >>> print T (s) barxxxquuxyyyfoo >>> print T (s) == d True Frederic ------------------------------------------------------------- class Translator: r""" Will translate any number of targets, handling them correctly if some overlap. Making Translator T = Translator (definitions, [eat = 1]) 'definitions' is a sequence of pairs: ((target, substitute),(t2, s2), ...) 'eat' says whether untargeted sections pass (translator) or are skipped (extractor). Makes a translator by default (eat = False) T.eat is an instance attribute that can be changed at any time. Definitions example: (('a','A'),('b','B'),('ab','ab'),('abc','xyz') # ('ab','ab') see Tricks. ('\x0c', 'page break'), ('\r\n','\n'), (' ','\t')) Order doesn't matter. Running translation = T (source) Tricks Deletion: ('target', '') Exception: (('\n',''), ('\n\n','\n\n')) # Eat LF except paragraph breaks. Exception: (('\n', '\r\n'), ('\r\n',\r\n')) # Unix to DOS, would leave DOS unchanged Translation cascade: # Rejoin text lines per paragraph Unix or DOS, inserting inter-word space if missing Mark_LF = Translator ((('\n','+LF+'),('\r\n','+LF+'),('\r\n\r\n','\r\n\r\n'),('\n\n','\n\n'))) # Pick positively identifiable mark for Unix and DOS end of lines Single_Space_Mark = Translator (((' +LF+', ' '),('+LF+', ' '),('-+LF+', ''))) no_lf_text = Single_Space_Mark (Mark_LF (text)) Translation cascade: # Nesting calls reptiles = T_latin_english (T_german_latin (reptilien)) Limitations 1. The number of substitutions and the maximum size of input depends on the respective capabilities of the Python re module. 2. Regular expressions will not work as such. Author: Frederic Rentsch (anthra.norell at bluewin.ch). """ def __init__ (self, definitions, eat = 0): ''' definitions: a sequence of pairs of strings. ((target, substitute), (t, s), ...) eat: False (0) means translate: unaffected data passes unaltered. True (1) means extract: unaffected data doesn't pass (gets eaten). Extraction filters typically require substitutes to end with some separator, else they fuse together. (E.g. ' ', '\t' or '\n') 'eat' is an attribute that can be switched anytime. ''' self.eat = eat self.compile_sequence_of_pairs (definitions) def compile_sequence_of_pairs (self, definitions): ''' Argument 'definitions' is a sequence of pairs: (('target 1', 'substitute 1'), ('t2', 's2'), ...) Order doesn't matter. ''' import re self.definitions = definitions targets, substitutes = zip (*definitions) re_targets = [re.escape (item) for item in targets] re_targets.sort (reverse = True) self.targets_set = set (targets) self.table = dict (definitions) regex_string = '|'.join (re_targets) self.regex = re.compile (regex_string, re.DOTALL) def __call__ (self, s): hits = self.regex.findall (s) nohits = self.regex.split (s) valid_hits = set (hits) & self.targets_set # Ignore targets with illegal re modifiers. if valid_hits: substitutes = [self.table [item] for item in hits if item in valid_hits] + [] # Make lengths equal for zip to work right if self.eat: return ''.join (substitutes) else: zipped = zip (nohits, substitutes) return ''.join (list (reduce (lambda a, b: a + b, [zipped][0]))) + nohits [-1] else: if self.eat: return '' else: return s From samwyse at gmail.com Mon Jan 18 07:49:06 2010 From: samwyse at gmail.com (samwyse) Date: Mon, 18 Jan 2010 04:49:06 -0800 (PST) Subject: enhancing 'list' References: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> Message-ID: <86ae75ac-f174-4e90-b1a3-301004967d1b@34g2000yqp.googlegroups.com> On Jan 18, 6:20?am, Peter Otten <__pete... at web.de> wrote: > Note that the cmp() builtin and the cmp parameter for list.sort() are gone > in Python 3. I've got Python 3 installed, and am using it for most new development. In this case case, however, I'm writing for the Google App Engine, which is stuck at 2.5. :( (Curiously, no matter how I order my PATH, the wrong version seems to appear first more than half the time! I'm seriously considering renaming all my Python 3 code to use a .py3 file extension.) > samwyse wrote: > > As a side note, wouldn't it be nice if '...' could be used in more > > places than just slices? ?IMHO, a useful idiom would be to use it to > > signify "irrelevant" or "don't care", as opposed to 'None' which (in > > my mind, at least) signifies "missing" or "unknown". > > That is a pretty subtle distinction... Hey, I'm a pretty subtle person... > I prefer keyword arguments, but in Python 3 you can use the ellipsis literal > freely: > > >>> ... == ... > True > >>> [..., 42, ...].count(...) > > 2 > > Peter I must have either missed or forgotten about that. Thanks! From stefan_ml at behnel.de Mon Jan 18 07:50:48 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 18 Jan 2010 13:50:48 +0100 Subject: enhancing 'list' In-Reply-To: <86ae75ac-f174-4e90-b1a3-301004967d1b@34g2000yqp.googlegroups.com> References: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> <86ae75ac-f174-4e90-b1a3-301004967d1b@34g2000yqp.googlegroups.com> Message-ID: <4b545928$0$7620$9b4e6d93@newsspool1.arcor-online.net> samwyse, 18.01.2010 13:49: > Curiously, no matter how I > order my PATH, the wrong version seems to appear first more than half > the time! I'm seriously considering renaming all my Python 3 code to > use a .py3 file extension. You should be able to start the interpreter as "python3.1" to be sure. Stefan From kaklis at gmail.com Mon Jan 18 07:52:41 2010 From: kaklis at gmail.com (kaklis at gmail.com) Date: Mon, 18 Jan 2010 04:52:41 -0800 (PST) Subject: Parse a log file Message-ID: <92293df0-7d2e-453a-b5bc-d4a7eb905fcf@l19g2000yqb.googlegroups.com> Hello to all! I want to parse a log file with the following format for example: TIMESTAMPE Operation FileName Bytes 12/Jan/2010:16:04:59 +0200 EXISTS sample3.3gp 37151 12/Jan/2010:16:04:59 +0200 EXISTS sample3.3gp 37151 12/Jan/2010:16:04:59 +0200 EXISTS sample3.3gp 37151 12/Jan/2010:16:04:59 +0200 EXISTS sample3.3gp 37151 12/Jan/2010:16:04:59 +0200 EXISTS sample3.3gp 37151 12/Jan/2010:16:05:05 +0200 DELETE sample3.3gp 37151 How can i count the operations for a month(e.g total of 40 Operations, 30 exists, 10 delete?) Any tips? Thanks in advance Antonis From cournape at gmail.com Mon Jan 18 07:56:31 2010 From: cournape at gmail.com (David Cournapeau) Date: Mon, 18 Jan 2010 21:56:31 +0900 Subject: Is python not good enough? In-Reply-To: <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> Message-ID: <5b8d13221001180456w3fab510bl10b69474476cbf62@mail.gmail.com> On Mon, Jan 18, 2010 at 8:03 PM, Phlip wrote: > This means, to appease the self-righteous indignation of the math > professor who would claim = should mean "equality"... Much more likely, this is part of the stated goal of making go very easy to analyse (to build tools and so that go is very fast to compile), as stated in its FAQ. cheers, David From cjwilliams43 at gmail.com Mon Jan 18 08:01:28 2010 From: cjwilliams43 at gmail.com (Colin W.) Date: Mon, 18 Jan 2010 08:01:28 -0500 Subject: r"string" vs R"string In-Reply-To: <00dd21f5$0$15570$c3e8da3@news.astraweb.com> References: <00dd21f5$0$15570$c3e8da3@news.astraweb.com> Message-ID: On 17-Jan-10 18:27 PM, Steven D'Aprano wrote: > On Sun, 17 Jan 2010 11:13:48 -0500, Roy Smith wrote: > >> In article, >> "Colin W." wrote: >> >>> On 17-Jan-10 02:16 AM, Terry Reedy wrote: >>>> On 1/17/2010 1:55 AM, Brendan Miller wrote: >>>>> Is there any difference whatsoever between a raw string beginning >>>>> with the captical R or one with the lower case r e.g. r"string" vs >>>>> R"string"? >>>> >>>> No. Nor is there and difference between the strings created with raw >>>> literals and cooked literals. >>>> >>>> >>> "cooked" literal ?? >> >> I've never seen it referred to this way in the Python literature, but >> "cooked" is a well-known term meaning, "not raw". The usage goes back >> decades. > > I think the use of "cooked" meaning "not raw" goes back a little bit more > than decades. The verb "to cook" dates from the late 14th century, and > the adjective form "cooked" would follow soon after. The use of "cooked" > to mean manipulated (as in "cooking the books") comes from the 1630s. > Extending it to strings (as in raw versus cooked strings) is an obvious > extension. > Yes, I should have cottoned on, but perhaps "... between the strings, raw and other." would have conveyed the idea. Colin W. From samwyse at gmail.com Mon Jan 18 08:05:01 2010 From: samwyse at gmail.com (samwyse) Date: Mon, 18 Jan 2010 05:05:01 -0800 (PST) Subject: Parse a log file References: <92293df0-7d2e-453a-b5bc-d4a7eb905fcf@l19g2000yqb.googlegroups.com> Message-ID: <419bf009-da09-4b89-8fe0-96f854d9f9b4@j14g2000yqm.googlegroups.com> On Jan 18, 6:52?am, "kak... at gmail.com" wrote: > Hello to all! > I want to parse a log file with the following format for > example: > ? ? ? ? ? ? ? TIMESTAMPE ? ? ? ? ? ?Operation ? ? FileName > Bytes > 12/Jan/2010:16:04:59 +0200 ? EXISTS ? ? ? sample3.3gp ? 37151 > 12/Jan/2010:16:04:59 +0200 ?EXISTS ? ? ? ?sample3.3gp ? 37151 > 12/Jan/2010:16:04:59 +0200 ?EXISTS ? ? ? ?sample3.3gp ? 37151 > 12/Jan/2010:16:04:59 +0200 ?EXISTS ? ? ? ?sample3.3gp ? 37151 > 12/Jan/2010:16:04:59 +0200 ?EXISTS ? ? ? ?sample3.3gp ? 37151 > 12/Jan/2010:16:05:05 +0200 ?DELETE ? ? ?sample3.3gp ? 37151 > > How can i count the operations for a month(e.g total of 40 Operations, > 30 exists, 10 delete?) > Any tips? > > Thanks in advance > Antonis time.strptime(string[, format]) Parse a string representing a time according to a format. The return value is a struct_time as returned by gmtime() or localtime(). The format parameter uses the same directives as those used by strftime (); it defaults to "%a %b %d %H:%M:%S %Y" which matches the formatting returned by ctime(). If string cannot be parsed according to format, or if it has excess data after parsing, ValueError is raised. The default values used to fill in any missing data when more accurate values cannot be inferred are (1900, 1, 1, 0, 0, 0, 0, 1, -1). >>> import time >>> ts='12/Jan/2010:16:04:59 +0200' >>> time.strptime(ts[:-6], '%d/%b/%Y:%H:%M:%S') time.struct_time(tm_year=2010, tm_mon=1, tm_mday=12, tm_hour=16, tm_min=4, tm_sec=59, tm_wday=1, tm_yday=12, tm_isdst=-1) I leave the conversion of the last six characters (the time zone offset) as an exercise for the student. :) From iainking at gmail.com Mon Jan 18 08:22:37 2010 From: iainking at gmail.com (Iain King) Date: Mon, 18 Jan 2010 05:22:37 -0800 (PST) Subject: substitution References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> <924f9bbd-ae54-4984-b6c7-a8950b5d4000@c29g2000yqd.googlegroups.com> Message-ID: <0fa11055-66c4-47ef-9e9c-01cad10d998e@a6g2000yqm.googlegroups.com> On Jan 18, 12:41?pm, Iain King wrote: > On Jan 18, 10:21?am, superpollo wrote: > > > > > superpollo ha scritto: > > > > hi. > > > > what is the most pythonic way to substitute substrings? > > > > eg: i want to apply: > > > > foo --> bar > > > baz --> quux > > > quuux --> foo > > > > so that: > > > > fooxxxbazyyyquuux --> barxxxquuxyyyfoo > > > > bye > > > i explain better: > > > say the subs are: > > > quuux --> foo > > foo --> bar > > baz --> quux > > > then i cannot apply the subs in sequence (say, .replace() in a loop), > > otherwise: > > > fooxxxbazyyyquuux --> fooxxxbazyyyfoo --> barxxxbazyyybar --> > > barxxxquuxyyybar > > > not as intended... > > Not sure if it's the most pythonic, but I'd probably do it like this: > > def token_replace(string, subs): > ? ? ? ? subs = dict(subs) > ? ? ? ? tokens = {} > ? ? ? ? for i, sub in enumerate(subs): > ? ? ? ? ? ? ? ? tokens[sub] = i > ? ? ? ? ? ? ? ? tokens[i] = sub > ? ? ? ? current = [string] > ? ? ? ? for sub in subs: > ? ? ? ? ? ? ? ? new = [] > ? ? ? ? ? ? ? ? for piece in current: > ? ? ? ? ? ? ? ? ? ? ? ? if type(piece) == str: > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? chunks = piece.split(sub) > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? new.append(chunks[0]) > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? for chunk in chunks[1:]: > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? new.append(tokens[sub]) > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? new.append(chunk) > ? ? ? ? ? ? ? ? ? ? ? ? else: > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? new.append(piece) > ? ? ? ? ? ? ? ? current = new > ? ? ? ? output = [] > ? ? ? ? for piece in current: > ? ? ? ? ? ? ? ? if type(piece) == str: > ? ? ? ? ? ? ? ? ? ? ? ? output.append(piece) > ? ? ? ? ? ? ? ? else: > ? ? ? ? ? ? ? ? ? ? ? ? output.append(subs[tokens[piece]]) > ? ? ? ? return ''.join(output) > > >>> token_replace("fooxxxbazyyyquuux", [("quuux", "foo"), ("foo", "bar"), ("baz", "quux")]) > > 'barxxxquuxyyyfoo' > > I'm sure someone could whittle that down to a handful of list comps... > Iain Slightly better (lets you have overlapping search strings, used in the order they are fed in): def token_replace(string, subs): tokens = {} if type(subs) == dict: for i, sub in enumerate(subs): tokens[sub] = i tokens[i] = subs[sub] else: s = [] for i, (k,v) in enumerate(subs): tokens[k] = i tokens[i] = v s.append(k) subs = s current = [string] for sub in subs: new = [] for piece in current: if type(piece) == str: chunks = piece.split(sub) new.append(chunks[0]) for chunk in chunks[1:]: new.append(tokens[sub]) new.append(chunk) else: new.append(piece) current = new output = [] for piece in current: if type(piece) == str: output.append(piece) else: output.append(tokens[piece]) return ''.join(output) From __peter__ at web.de Mon Jan 18 08:31:50 2010 From: __peter__ at web.de (Peter Otten) Date: Mon, 18 Jan 2010 14:31:50 +0100 Subject: substitution References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> <924f9bbd-ae54-4984-b6c7-a8950b5d4000@c29g2000yqd.googlegroups.com> Message-ID: Iain King wrote: > Not sure if it's the most pythonic, but I'd probably do it like this: > > def token_replace(string, subs): > subs = dict(subs) > tokens = {} > for i, sub in enumerate(subs): > tokens[sub] = i > tokens[i] = sub > current = [string] > for sub in subs: > new = [] > for piece in current: > if type(piece) == str: > chunks = piece.split(sub) > new.append(chunks[0]) > for chunk in chunks[1:]: > new.append(tokens[sub]) > new.append(chunk) > else: > new.append(piece) > current = new > output = [] > for piece in current: > if type(piece) == str: > output.append(piece) > else: > output.append(subs[tokens[piece]]) > return ''.join(output) > > >>> token_replace("fooxxxbazyyyquuux", [("quuux", "foo"), ("foo", "bar"), ("baz", "quux")]) > 'barxxxquuxyyyfoo' > > I'm sure someone could whittle that down to a handful of list comps... I tried, but failed: def join(chunks, separator): chunks = iter(chunks) yield next(chunks) for chunk in chunks: yield separator yield chunk def token_replace(string, subs): tokens = {} current = [string] for i, (find, replace) in enumerate(subs): tokens[i] = replace new = [] for piece in current: if piece in tokens: new.append(piece) else: new.extend(join(piece.split(find), i)) current = new return ''.join(tokens.get(piece, piece) for piece in current) You could replace the inner loop with sum(..., []), but that would be really ugly. Peter From utente at esempio.net Mon Jan 18 08:43:46 2010 From: utente at esempio.net (superpollo) Date: Mon, 18 Jan 2010 14:43:46 +0100 Subject: substitution In-Reply-To: References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> <924f9bbd-ae54-4984-b6c7-a8950b5d4000@c29g2000yqd.googlegroups.com> Message-ID: <4b546593$0$1114$4fafbaef@reader4.news.tin.it> it looked simpler when i posted, but i realize that the problem is non trivial. thanks to everybody. i guess that the algorithm would be easier if it was known in advance that the string to substitute must have some specific property, say: 1) they all must start with "XYZ" 2) they all have the same length N (e.g. 5) like this: qweXYZ12asdXYZ1345XYZ ---> qweIWAS12asdIWAS1345XYZ bye From anh.hai.trinh at gmail.com Mon Jan 18 08:59:59 2010 From: anh.hai.trinh at gmail.com (Anh Hai Trinh) Date: Mon, 18 Jan 2010 05:59:59 -0800 (PST) Subject: Is python not good enough? References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> Message-ID: <37f8195f-3f02-4a13-b677-6b5d5e1589e8@22g2000yqr.googlegroups.com> On Jan 18, 6:03?pm, Phlip wrote: > On Jan 12, 7:09?am, ikuta liu wrote: > > > Go language try to merge low level, hight level and browser language. > > Go uses := for assignment. Except that it doesn't. := is a declaration. s := "foo" is short for var s string = "foo" Cheers, ----aht From jeanmichel at sequans.com Mon Jan 18 09:10:37 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 18 Jan 2010 15:10:37 +0100 Subject: Changing var names In-Reply-To: <4dc0cfea1001151203w2d1c9520t692d2367a1882ee3@mail.gmail.com> References: <4dc0cfea1001150927i48b6c386w53f4487cd894d078@mail.gmail.com> <1263582938.3142.12.camel@linux-m3mt> <4dc0cfea1001151203w2d1c9520t692d2367a1882ee3@mail.gmail.com> Message-ID: <4B546BDD.1030602@sequans.com> Victor Subervi wrote: > On Fri, Jan 15, 2010 at 3:15 PM, Adam Tauno Williams > > wrote: > > On Fri, 2010-01-15 at 13:27 -0400, Victor Subervi wrote: > > Hi; > > Well it took me *less than a day* to fix the following problems: > > -- bare excepts (accidentally left a couple I think) > > -- sql injection attacks > > -- recreating tables to make them more reasonable > > Now, I believe someone once mentioned that there is some s/w > tool out > > there for changing the names of my variables more easily than going > > through every script one-by-one. If so, please share with me again > > what it is. > > This process is called 'refactoring' [a good term to Google], and > every > decent IDE provides some support [if it doesn't, it isn't a "decent" > IDE] > > > Thanks. I'll work with Eric, I guess, once I get my Mac. > beno Here are 2 well known IDEs, that help with refactoring (and many other things). http://netbeans.org/ http://www.eclipse.org/ Both supports python. I personnaly will stick to vim :o) JM From gagsl-py2 at yahoo.com.ar Mon Jan 18 09:11:46 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 18 Jan 2010 11:11:46 -0300 Subject: Inheriting methods but over-riding docstrings References: <0361ea2d$0$1336$c3e8da3@news.astraweb.com> Message-ID: En Sun, 17 Jan 2010 23:23:45 -0300, Steve Holden escribi?: > Gabriel Genellina wrote: >> Methods don't have docstrings; functions do. So one has to "clone" the >> function to set a new docstring. >> > On behalf of all methods I would like to say "We demand the right to > docstrings". > >>>> print abs.__doc__ > abs(number) -> number > > Return the absolute value of the argument. >>>> > > Perhaps I am misunderstanding "don't have docstrings". This code: Sorry, I wasn't clear at all. What I wanted to say is that docstrings are actually stored as *function* attributes; methods inherit the docstring from the function they are built on (as a read-only attribute). py> class Base(object): ... def method(self): ... "my docstring" ... py> Base.method.__doc__ 'my docstring' py> Base.method.__doc__ = "another docstring" Traceback (most recent call last): File "", line 1, in AttributeError: attribute '__doc__' of 'instancemethod' objects is not writable py> Base.method.im_func.__doc__ 'my docstring' py> Base.method.im_func.__doc__ = "another docstring" py> Base.method.__doc__ 'another docstring' So, if you want Base.foo and Derived.foo to share the same behavior but have separate docstrings (what Steven asked for), you have to create separate functions for both. That's what my already posted code does. > So, are the demands of the methods reasonable, or do they already have > what they are so loudly demanding? Well, they could demand the right to have a docstring of their own. Since 3.x eliminated a whole category of methods (unbound methods are gone), their ability to press for demands was greatly reduced. I don't think they could get such right granted in the near future... -- Gabriel Genellina From adi at digitaltrowel.com Mon Jan 18 09:17:56 2010 From: adi at digitaltrowel.com (Adi Eyal) Date: Mon, 18 Jan 2010 16:17:56 +0200 Subject: substitution Message-ID: > From:?superpollo > To: > Date:?Mon, 18 Jan 2010 11:15:37 +0100 > Subject:?substitution > hi. > > what is the most pythonic way to substitute substrings? > > eg: i want to apply: > > foo --> bar > baz --> quux > quuux --> foo > > so that: > > fooxxxbazyyyquuux --> barxxxquuxyyyfoo > > bye Using regular expressions the answer is short (and sweet) mapping = { "foo" : "bar", "baz" : "quux", "quuux" : "foo" } pattern = "(%s)" % "|".join(mapping.keys()) repl = lambda x : mapping.get(x.group(1), x.group(1)) s = "fooxxxbazyyyquuux" re.subn(pattern, repl, s) From iainking at gmail.com Mon Jan 18 09:23:44 2010 From: iainking at gmail.com (Iain King) Date: Mon, 18 Jan 2010 06:23:44 -0800 (PST) Subject: substitution References: Message-ID: <93831c02-6e5a-48f7-a698-3b3f1ef6e5b6@v25g2000yqk.googlegroups.com> On Jan 18, 2:17?pm, Adi Eyal wrote: > > From:?superpollo > > To: > > Date:?Mon, 18 Jan 2010 11:15:37 +0100 > > Subject:?substitution > > hi. > > > what is the most pythonic way to substitute substrings? > > > eg: i want to apply: > > > foo --> bar > > baz --> quux > > quuux --> foo > > > so that: > > > fooxxxbazyyyquuux --> barxxxquuxyyyfoo > > > bye > > Using regular expressions the answer is short (and sweet) > > mapping = { > ? ? ? ? "foo" : "bar", > ? ? ? ? "baz" : "quux", > ? ? ? ? "quuux" : "foo" > > } > > pattern = "(%s)" % "|".join(mapping.keys()) > repl = lambda x : mapping.get(x.group(1), x.group(1)) > s = "fooxxxbazyyyquuux" > re.subn(pattern, repl, s) Winner! :) Iain From ted at sjksdjk.it Mon Jan 18 09:32:36 2010 From: ted at sjksdjk.it (ted) Date: Mon, 18 Jan 2010 15:32:36 +0100 Subject: python gui ide under linux..like visual studio ;) ? Message-ID: <4b5470ce$0$1103$4fafbaef@reader4.news.tin.it> Hi at all... Can someone please give me some advice, about a good IDE with control GUI under Linux ? Actually i know QT Creator by Nokia which i can use with Python (but i don't know how). And, a good library for access to database (mysql, sql server, oracle) ? Thank you very much ! bye From oktakaweb at gmail.com Mon Jan 18 09:40:47 2010 From: oktakaweb at gmail.com (Oktaka Com) Date: Mon, 18 Jan 2010 06:40:47 -0800 (PST) Subject: multithreading, performance, again... References: Message-ID: <4826604d-85f3-48b9-863e-42401d29bf6a@l19g2000yqb.googlegroups.com> On 30 Aral?k 2009, 17:44, mk wrote: > Hello everyone, > > I have figured out (sort of) how to do profiling of multithreaded > programs with cProfile, it goes something like this: > > #!/usr/local/bin/python > > import cProfile > import threading > > class TestProf(threading.Thread): > ? ? ?def __init__(self, ip): > > ? ? ? ? ?threading.Thread.__init__(self) > ? ? ? ? ?self.ip = ip > > ? ? ?def run(self): > ? ? ? ? ?prof = cProfile.Profile() > ? ? ? ? ?retval = prof.runcall(self.runmethod) > ? ? ? ? ?prof.dump_stats('tprof' + self.ip) > > ? ? ?def runmethod(self): > ? ? ? ? ?pass > > tp = TestProf('10.0.10.10') > > tp.start() > tp.join() > > The problem is, now that I've done profiling in the actual program > (profiled version here:http://python.domeny.com/cssh_profiled.py) with > 9 threads and added up stats (using pstats.Stats.add()), the times I get > are trivial: > > ?>>> p.strip_dirs().sort_stats('cumulative').print_stats(10) > Wed Dec 30 16:23:59 2009 ? ?csshprof9.156.44.113 > Wed Dec 30 16:23:59 2009 ? ?csshprof9.156.46.243 > Wed Dec 30 16:23:59 2009 ? ?csshprof9.156.46.89 > Wed Dec 30 16:24:00 2009 ? ?csshprof9.156.47.125 > Wed Dec 30 16:24:00 2009 ? ?csshprof9.156.47.17 > Wed Dec 30 16:24:00 2009 ? ?csshprof9.156.47.29 > Wed Dec 30 16:24:01 2009 ? ?csshprof9.167.41.241 > Wed Dec 30 16:24:02 2009 ? ?csshprof9.168.119.15 > Wed Dec 30 16:24:02 2009 ? ?csshprof9.168.119.218 > > ? ? ? ? ? 39123 function calls (38988 primitive calls) in 6.004 CPU seconds > > ? ? Ordered by: cumulative time > ? ? List reduced from 224 to 10 due to restriction <10> > > ? ? ncalls ?tottime ?percall ?cumtime ?percall filename:lineno(function) > ? ? ? ? ?9 ? ?0.000 ? ?0.000 ? ?6.004 ? ?0.667 cssh.py:696(runmethod) > ? ? ? ?100 ? ?0.004 ? ?0.000 ? ?5.467 ? ?0.055 threading.py:389(wait) > ? ? ? ? 82 ? ?0.025 ? ?0.000 ? ?5.460 ? ?0.067 threading.py:228(wait) > ? ? ? ?400 ? ?5.400 ? ?0.013 ? ?5.400 ? ?0.013 {time.sleep} > ? ? ? ? ?9 ? ?0.000 ? ?0.000 ? ?5.263 ? ?0.585 cssh.py:452(ssh_connect) > ? ? ? ? ?9 ? ?0.003 ? ?0.000 ? ?5.262 ? ?0.585 client.py:226(connect) > ? ? ? ? ?9 ? ?0.001 ? ?0.000 ? ?2.804 ? ?0.312 > transport.py:394(start_client) > ? ? ? ? ?9 ? ?0.005 ? ?0.001 ? ?2.254 ? ?0.250 client.py:391(_auth) > ? ? ? ? 18 ? ?0.001 ? ?0.000 ? ?2.115 ? ?0.117 > transport.py:1169(auth_publickey) > ? ? ? ? 18 ? ?0.001 ? ?0.000 ? ?2.030 ? ?0.113 > auth_handler.py:156(wait_for_response) > > > > It's not burning CPU time in the main thread (profiling with cProfile > indicated smth similar to the above), it's not burning it in the > individual worker threads - so where the heck it is burning this CPU > time? bc 'top' shows heavy CPU load during most of the time of the > program run. > > help... > > regards, > mk See http://code.google.com/p/yappi/ if you want to profile multithreaded python app. From mwilson at the-wire.com Mon Jan 18 09:44:50 2010 From: mwilson at the-wire.com (Mel) Date: Mon, 18 Jan 2010 09:44:50 -0500 Subject: substitution References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> Message-ID: superpollo wrote: > hi. > > what is the most pythonic way to substitute substrings? > > eg: i want to apply: > > foo --> bar > baz --> quux > quuux --> foo > > so that: > > fooxxxbazyyyquuux --> barxxxquuxyyyfoo This is simple -- maybe a bit brutal -- and if the strings get long it will be slow: def replaced (input, replacements): output = [] while input: for target in replacements: if input.startswith (target): output.append (replacements [target]) input = input [len (target):] break else: output.append (input[0]) input = input [1:] return ''.join (output) original = 'fooxxxbazyyyquux' replacements = {'quux':'foo', 'foo':'bar', 'baz':'quux'} print original, '-->', replaced (original, replacements) Mel. From bartc at freeuk.com Mon Jan 18 10:03:57 2010 From: bartc at freeuk.com (bartc) Date: Mon, 18 Jan 2010 15:03:57 GMT Subject: basic Class in Python In-Reply-To: References: <86a6a8c8-49fe-4a3d-81a0-5a44a179c7d6@o28g2000yqh.googlegroups.com> Message-ID: "Wolfgang Rohdewald" wrote in message news:mailman.1056.1263771299.28905.python-list at python.org... > On Monday 18 January 2010, BarryJOgorman wrote: >> TypeError: object._new_() takes no parameters > >> def _init_(self, name, job=None, pay=0): > > __init__ needs two underscores left and right Any particular reason why two, and not one (or three)? In some fonts it's difficult to tell how many as they run together. -- Bartc From jeanmichel at sequans.com Mon Jan 18 10:07:56 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 18 Jan 2010 16:07:56 +0100 Subject: Proper display of XMLRPCserver exceptions Message-ID: <4B54794C.6020207@sequans.com> To all using xmlrpclib, I had trouble getting a proper display of my exceptions occuring on the server. Basically, xmlrpclib only display the exception itself without any traceback, on the client side. Something like :2"> Ok then there's a key error, but how the hell am I supposed to know where it occured ? So I wrote this: from SimpleXMLRPCServer import SimpleXMLRPCServer server = SimpleXMLRPCServer(('localhost', 8000), logRequests=False, allow_none=True) old_dispatch = server._dispatch def _new_dispatch(method, params): try: return old_dispatch(method,params) except Exception, exc: import traceback traceback.print_exc() raise server._dispatch = _new_dispatch Pretty dirty, isn't it, accessing the private attributes (too lazy to subclass) ! I guess there's an obvious way to do this with xmlrpclib that I just missed. Anyone knows ? JM From nobody at nowhere.com Mon Jan 18 10:09:31 2010 From: nobody at nowhere.com (Nobody) Date: Mon, 18 Jan 2010 15:09:31 +0000 Subject: substitution References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> Message-ID: On Mon, 18 Jan 2010 11:21:54 +0100, superpollo wrote: >> what is the most pythonic way to substitute substrings? > say the subs are: > > quuux --> foo > foo --> bar > baz --> quux > > then i cannot apply the subs in sequence (say, .replace() in a loop), > otherwise: > > fooxxxbazyyyquuux --> fooxxxbazyyyfoo --> barxxxbazyyybar --> > barxxxquuxyyybar > > not as intended... Are there any characters which are guaranteed never to occur in the string? If so: s = s.replace('quuux', '@1').replace('foo', '@2').replace('baz', '@3') s = s.replace('@1', 'foo').replace('@2', 'bar').replace('@3', 'quux') E.g.: def replace(subs, s): for i, (src, dst) in enumerate(subs): s = s.replace(src, '@%06d' % i) for i, (src, dst) in enumerate(subs): s = s.replace('@%06d' % i, dst) return s Not the most efficient solution (and problematic if there aren't any "unused" characters), but somewhat shorter than the other solutions. From R.Brodie at rl.ac.uk Mon Jan 18 10:21:36 2010 From: R.Brodie at rl.ac.uk (Richard Brodie) Date: Mon, 18 Jan 2010 15:21:36 -0000 Subject: basic Class in Python References: <86a6a8c8-49fe-4a3d-81a0-5a44a179c7d6@o28g2000yqh.googlegroups.com> Message-ID: "bartc" wrote in message news:xL_4n.28001$Ym4.5658 at text.news.virginmedia.com... > Any particular reason why two, and not one (or three)? In some fonts it's difficult to > tell how many as they run together. It follows the C convention for reserved identifers. From phlip2005 at gmail.com Mon Jan 18 10:37:36 2010 From: phlip2005 at gmail.com (Phlip) Date: Mon, 18 Jan 2010 07:37:36 -0800 (PST) Subject: Is python not good enough? References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> <37f8195f-3f02-4a13-b677-6b5d5e1589e8@22g2000yqr.googlegroups.com> Message-ID: <1e692369-cee9-4e63-8dce-33d70bf96368@p24g2000yqm.googlegroups.com> On Jan 18, 5:59?am, Anh Hai Trinh wrote: > > Go uses := for assignment. > > Except that it doesn't. := is a declaration. Ah, and that's why Go is easy for cheap parsers to rip. Tx all! I was formerly too mortified to proceed - now I'm back in the Go camp. They fixed the hideous redundancy of Java without the ill-defined scope issues of Python & Ruby, and without the tacky little 'var' of JavaScript! From dotancohen at gmail.com Mon Jan 18 10:42:38 2010 From: dotancohen at gmail.com (Dotan Cohen) Date: Mon, 18 Jan 2010 17:42:38 +0200 Subject: Generic Python Benchmark suite? In-Reply-To: References: Message-ID: <880dece01001180742h62f2febi37e44f282b583d50@mail.gmail.com> > What do you suggest? > $ man time -- Dotan Cohen http://what-is-what.com http://gibberish.co.il From utente at esempio.net Mon Jan 18 10:43:25 2010 From: utente at esempio.net (superpollo) Date: Mon, 18 Jan 2010 16:43:25 +0100 Subject: not and is not problem Message-ID: <4b54819e$0$1106$4fafbaef@reader4.news.tin.it> hi: #!/usr/bin/env python data = "seq=123" name , value = data.split("=") print name print value if not name == "seq": print "DOES NOT PRINT OF COURSE..." if name is not "seq": print "WTF! WHY DOES IT PRINT?" help please. bye From showell30 at yahoo.com Mon Jan 18 10:46:00 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 18 Jan 2010 07:46:00 -0800 (PST) Subject: ANN: shpaml 0.94b Message-ID: <8b8967a0-5fe3-471f-84fc-9f5d31a3c990@j19g2000yqk.googlegroups.com> Hi everybody, I would like to announce the latest version of shpaml, which is an indentation-based markup language similar to haml, but different. Shpaml allows you to author HTML-like content with an indentation- based syntax that eliminates the need to write and read close tags and angle brackets. It also has haml-like syntax for specifying divs, classes, and ids, which also makes your documents easier to read, particularly when you are also using CSS and jQuery-like tools. Finally, it is very transparent with respect to HTML semantics, as well as letting most HTML syntax pass through as needed. It is probably best understood by simply looking at a few examples: http://shpaml.webfactional.com/examples There is also a tutorial, and you can try out your own examples there, so need to download the software if you just want to poke around a bit: http://shpaml.webfactional.com/tutorial/1 If you do decide to download it, it is a pretty small module at 225 lines or so: http://shpaml.webfactional.com/source_code The project is about two months old now, and it is being actively maintained. We have a mailing list, users submitting patches, all that good stuff. I am labelling the current version as "beta," but it has been well tested on over a thousand lines of markup. Cheers, Steve From arnodel at googlemail.com Mon Jan 18 10:55:10 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 18 Jan 2010 07:55:10 -0800 (PST) Subject: not and is not problem References: <4b54819e$0$1106$4fafbaef@reader4.news.tin.it> Message-ID: <3e8d242a-7965-46c8-ab51-f61124aacc1a@j19g2000yqk.googlegroups.com> On 18 Jan, 15:43, superpollo wrote: > hi: > > #!/usr/bin/env python > data = "seq=123" > name , value = data.split("=") > print name > print value > if not name == "seq": > ? ? ?print "DOES NOT PRINT OF COURSE..." > if name is not "seq": > ? ? ?print "WTF! WHY DOES IT PRINT?" > > help please. > > bye is and == are different operators. * A == B means A.__eq__(B) * A is B means that A and B are the same object, living at the same physical location in the computer's memory. HTH -- Arnaud From apt.shansen at gmail.com Mon Jan 18 10:56:47 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Mon, 18 Jan 2010 07:56:47 -0800 Subject: not and is not problem In-Reply-To: <4b54819e$0$1106$4fafbaef@reader4.news.tin.it> References: <4b54819e$0$1106$4fafbaef@reader4.news.tin.it> Message-ID: <7a9c25c21001180756i13061fd5he51d9b0e6f3f38a5@mail.gmail.com> On Mon, Jan 18, 2010 at 7:43 AM, superpollo wrote: > #!/usr/bin/env python > data = "seq=123" > name , value = data.split("=") > print name > print value > if not name == "seq": > print "DOES NOT PRINT OF COURSE..." > if name is not "seq": > print "WTF! WHY DOES IT PRINT?" > Because name really is not "seq"; "seq" is an entirely new and different string. The "is" operator tests for object identity, == tests for equality. "name is name" will always return True, everything else will always return False. For example, you can do: other = name if name is other: print "OK" And it'll work, because those are the exact same object. You only use 'is' and 'is not' when you really want to ask, 'Is this the precisely same object?' Generally, that's only when you're testing if something is or is not None. Sometimes is or is not True/False, but usually 'if x' is preferred. --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthra.norell at bluewin.ch Mon Jan 18 10:59:57 2010 From: anthra.norell at bluewin.ch (Anthra Norell) Date: Mon, 18 Jan 2010 16:59:57 +0100 Subject: substitution In-Reply-To: <4B545785.8090003@bluewin.ch> References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4B545785.8090003@bluewin.ch> Message-ID: <4B54857D.407@bluewin.ch> Anthra Norell wrote: > superpollo wrote: >> hi. >> >> what is the most pythonic way to substitute substrings? >> >> eg: i want to apply: >> >> foo --> bar >> baz --> quux >> quuux --> foo >> >> so that: >> >> fooxxxbazyyyquuux --> barxxxquuxyyyfoo >> >> bye > So it goes. The more it matters, the sillier the misatakes. The method __init__ () was incomplete and __call__ () was missing, Sorry abount that. Here the whole thing again: class Translator: r""" Will translate any number of targets, handling them correctly if some overlap. Making Translator T = Translator (definitions, [eat = 1]) 'definitions' is a sequence of pairs: ((target, substitute),(t2, s2), ...) 'eat' says whether untargeted sections pass (translator) or are skipped (extractor). Makes a translator by default (eat = False) T.eat is an instance attribute that can be changed at any time. Definitions example: (('a','A'),('b','B'),('ab','ab'),('abc','xyz') # ('ab','ab') see Tricks. ('\x0c', 'page break'), ('\r\n','\n'), (' ','\t')) Order doesn't matter. Running translation = T (source) Tricks Deletion: ('target', '') Exception: (('\n',''), ('\n\n','\n\n')) # Eat LF except paragraph breaks. Exception: (('\n', '\r\n'), ('\r\n',\r\n')) # Unix to DOS, would leave DOS unchanged Translation cascade: # Rejoin text lines per paragraph Unix or DOS, inserting inter-word space if missing Mark_LF = Translator ((('\n','+LF+'),('\r\n','+LF+'),('\r\n\r\n','\r\n\r\n'),('\n\n','\n\n'))) # Pick positively identifiable mark for Unix and DOS end of lines Single_Space_Mark = Translator (((' +LF+', ' '),('+LF+', ' '),('-+LF+', ''))) no_lf_text = Single_Space_Mark (Mark_LF (text)) Translation cascade: # Nesting calls reptiles = T_latin_english (T_german_latin (reptilien)) Limitations 1. The number of substitutions and the maximum size of input depends on the respective capabilities of the Python re module. 2. Regular expressions will not work as such. Author: Frederic Rentsch (anthra.norell at bluewin.ch). """ def __init__ (self, definitions, eat = 0): ''' definitions: a sequence of pairs of strings. ((target, substitute), (t, s), ...) eat: False (0) means translate: unaffected data passes unaltered. True (1) means extract: unaffected data doesn't pass (gets eaten). Extraction filters typically require substitutes to end with some separator, else they fuse together. (E.g. ' ', '\t' or '\n') 'eat' is an attribute that can be switched anytime. ''' self.eat = eat self.compile_sequence_of_pairs (definitions) def compile_sequence_of_pairs (self, definitions): ''' Argument 'definitions' is a sequence of pairs: (('target 1', 'substitute 1'), ('t2', 's2'), ...) Order doesn't matter. ''' import re self.definitions = definitions targets, substitutes = zip (*definitions) re_targets = [re.escape (item) for item in targets] re_targets.sort (reverse = True) self.targets_set = set (targets) self.table = dict (definitions) regex_string = '|'.join (re_targets) self.regex = re.compile (regex_string, re.DOTALL) def __call__ (self, s): hits = self.regex.findall (s) nohits = self.regex.split (s) valid_hits = set (hits) & self.targets_set # Ignore targets with illegal re modifiers. if valid_hits: substitutes = [self.table [item] for item in hits if item in valid_hits] + [] # Make lengths equal for zip to work right if self.eat: return ''.join (substitutes) else: zipped = zip (nohits, substitutes) return ''.join (list (reduce (lambda a, b: a + b, [zipped][0]))) + nohits [-1] else: if self.eat: return '' else: return s From darcy at druid.net Mon Jan 18 11:03:31 2010 From: darcy at druid.net (D'Arcy J.M. Cain) Date: Mon, 18 Jan 2010 11:03:31 -0500 Subject: not and is not problem In-Reply-To: <4b54819e$0$1106$4fafbaef@reader4.news.tin.it> References: <4b54819e$0$1106$4fafbaef@reader4.news.tin.it> Message-ID: <20100118110331.49199fe8.darcy@druid.net> On Mon, 18 Jan 2010 16:43:25 +0100 superpollo wrote: > hi: > if name is not "seq": > print "WTF! WHY DOES IT PRINT?" Perhaps they aren't the same object. Some values are guaranteed to be the same (e.g. True and False) but not all. Try "!=" instead of "is not" in that experssion. -- D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner. From wkfung.eric at gmail.com Mon Jan 18 11:07:41 2010 From: wkfung.eric at gmail.com (Kit) Date: Mon, 18 Jan 2010 08:07:41 -0800 (PST) Subject: What is a list compression in Python? Message-ID: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> Hello Everyone, I am not sure if I have posted this question in a correct board. Can anyone please teach me: What is a list compression in Python? Would you mind give me some list compression examples? Thanks & really appreciate that. Kit From dreadpiratejeff at gmail.com Mon Jan 18 11:11:35 2010 From: dreadpiratejeff at gmail.com (J) Date: Mon, 18 Jan 2010 11:11:35 -0500 Subject: not and is not problem In-Reply-To: <4b54819e$0$1106$4fafbaef@reader4.news.tin.it> References: <4b54819e$0$1106$4fafbaef@reader4.news.tin.it> Message-ID: <36dec4ff1001180811h6a36c6ccm8f11626dd9ba3c55@mail.gmail.com> On Mon, Jan 18, 2010 at 10:43, superpollo wrote: > hi: > > #!/usr/bin/env python > data = "seq=123" > name , value = data.split("=") > print name > print value > if not name == "seq": > ? ?print "DOES NOT PRINT OF COURSE..." > if name is not "seq": > ? ?print "WTF! WHY DOES IT PRINT?" is will return True if two variables point to the same object, == if the objects referred to by the variables are equal. >> if not name == "seq": says if the object that the variable name points to is NOT the same as the string "seq" then do the following. Since the object that "name" points to contains the string "seq" and the string "seq" is identical in value to the "seq" in your comparison, the result is TRUE (b = a) and your if statment only proceeds if the comparison result is FALSE. >> if name is not "seq" is and is not relate to pointing to an object, not the object's contents. Example: >> name = "foo" >> name1 = name >> print name foo >> print name1 foo >> name is name1 True >> name1 is name True >> name = "bar" >> print name bar >> print name1 foo >> name is name1 False >> name is not name1 True Or even better: >> name = "foo" >> name1 = name >> id(name) 11875840 >> id(name1) 11875840 >> name = "bar" >> id(name) 11875744 >> id(name1) 11875840 >> id("foo") 11875840 >> id("bar") 11875744 shows the location for each object in relation to the name pointing to that object... -- Joan Crawford - "I, Joan Crawford, I believe in the dollar. Everything I earn, I spend." - http://www.brainyquote.com/quotes/authors/j/joan_crawford.html From utente at esempio.net Mon Jan 18 11:15:01 2010 From: utente at esempio.net (superpollo) Date: Mon, 18 Jan 2010 17:15:01 +0100 Subject: What is a list compression in Python? In-Reply-To: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> Message-ID: <4b548906$0$1113$4fafbaef@reader4.news.tin.it> Kit ha scritto: > Hello Everyone, I am not sure if I have posted this question in a > correct board. Can anyone please teach me: > > What is a list compression in Python? > > Would you mind give me some list compression examples? > > > Thanks & really appreciate that. > Kit i think that's compreHENsion... http://www.python.org/doc/2.5.4/tut/node7.html#SECTION007140000000000000000 From duncan.booth at invalid.invalid Mon Jan 18 11:20:46 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 18 Jan 2010 16:20:46 GMT Subject: substitution References: Message-ID: Adi Eyal wrote: > Using regular expressions the answer is short (and sweet) > > mapping = { > "foo" : "bar", > "baz" : "quux", > "quuux" : "foo" > } > > pattern = "(%s)" % "|".join(mapping.keys()) > repl = lambda x : mapping.get(x.group(1), x.group(1)) > s = "fooxxxbazyyyquuux" > re.subn(pattern, repl, s) I'd use a def as being IMHO clearer than the lambda but YMMV. More importantly I'd also escape the keys in case they contain any characters special to regular expressions: >>> mapping = { "foo" : "bar", "baz" : "quux", "quuux" : "foo" } >>> import re >>> pattern = "(%s)" % "|".join(re.escape(k) for k in mapping) >>> def repl(x): return mapping[x.group(1)] >>> s = "fooxxxbazyyyquuux" >>> re.subn(pattern, repl, s) ('barxxxquuxyyyfoo', 3) >>> re.sub(pattern, repl, s) 'barxxxquuxyyyfoo' >>> -- Duncan Booth http://kupuguy.blogspot.com From steve at REMOVE-THIS-cybersource.com.au Mon Jan 18 11:24:25 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 18 Jan 2010 16:24:25 GMT Subject: not and is not problem References: <4b54819e$0$1106$4fafbaef@reader4.news.tin.it> Message-ID: <00de103b$0$15570$c3e8da3@news.astraweb.com> On Mon, 18 Jan 2010 16:43:25 +0100, superpollo wrote: > hi: > > #!/usr/bin/env python > data = "seq=123" > name , value = data.split("=") > print name > print value > if not name == "seq": > print "DOES NOT PRINT OF COURSE..." > if name is not "seq": > print "WTF! WHY DOES IT PRINT?" `is` is not an alternative spelling for `==`. `is` tests for object identity, not equality. Unless you care about object identity, always use equals. >>> 'seq 1' is 'seq 1' # the SAME object is used twice True >>> s = "seq 1" # TWO objects are used >>> t = "seq 1" >>> s is t False Beware: Python caches strings that look like identifiers. This will include "seq". So, purely as an implementation-specific detail, Python will sometimes re-use the same string object, and sometimes not. Do not use `is` when you are testing for equality. By the way, instead of not name == "seq" you should write name != "seq" -- Steven From simon at brunningonline.net Mon Jan 18 11:24:54 2010 From: simon at brunningonline.net (Simon Brunning) Date: Mon, 18 Jan 2010 16:24:54 +0000 Subject: What is a list compression in Python? In-Reply-To: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> Message-ID: <8c7f10c61001180824i5cdd66d0xb8a4dc93422bb925@mail.gmail.com> 2010/1/18 Kit : > Hello Everyone, I am not sure if I have posted this question in a > correct board. Can anyone please teach me: > > What is a list compression in Python? Perhaps you mean a list comprehension? If so, see . -- Cheers, Simon B. From apt.shansen at gmail.com Mon Jan 18 11:25:24 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Mon, 18 Jan 2010 08:25:24 -0800 Subject: What is a list compression in Python? In-Reply-To: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> Message-ID: <7a9c25c21001180825ka187892s324f0887e3bee8b@mail.gmail.com> On Mon, Jan 18, 2010 at 8:07 AM, Kit wrote: > Hello Everyone, I am not sure if I have posted this question in a > correct board. Can anyone please teach me: > > What is a list compression in Python? > > Would you mind give me some list compression examples? > Do you mean list *comprehension*? If so, its a special syntax for list construct which can be used for shorter, clearer code (provided one doesn't abuse it, at which point it becomes quite obtuse). Its never required: nothing you do with a list comprehension you couldn't do with a standard loop. For example: evens = [] for n in range(100): if n %2 == 0: evens.append(n) verses: my_list = [n for n in range(100) if n % 2 == 0] The basic syntax is: [ for in ] The 'if' part at the end is optional. The syntax is converted into a for loop that builds a list, and then returns it to you. So, basically it becomes: temp = [] for in : temp.append() Except the 'temp' variable doesn't really have a name and is returned from the comprehension, where you can give it a name. Sometimes you need an 'if' clause, sometimes you don't. Sometimes your 'expression to be appended' is very simple, other times you mutate it. For example, an easy way to convert a list of numbers into a list of strings is: [str(x) for x in range(10)] HTH, --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthra.norell at bluewin.ch Mon Jan 18 11:26:45 2010 From: anthra.norell at bluewin.ch (Anthra Norell) Date: Mon, 18 Jan 2010 17:26:45 +0100 Subject: substitution In-Reply-To: <4B545785.8090003@bluewin.ch> References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4B545785.8090003@bluewin.ch> Message-ID: <4B548BC5.4010902@bluewin.ch> superpollo wrote: >> hi. >> >> what is the most pythonic way to substitute substrings? >> >> eg: i want to apply: >> >> foo --> bar >> baz --> quux >> quuux --> foo >> >> so that: >> >> fooxxxbazyyyquuux --> barxxxquuxyyyfoo >> >> bye > > Third attempt. Clearly something doesn't work right. My code gets clipped on the way up. I have to send it as an attachment. Here's again what it does: >>> substitutions = (('foo', 'bar'), ('baz', 'quux'), ('quuux', 'foo')) # Sequence of doublets >>> T = Translator (substitutions) # Compile substitutions -> translator >>> s = 'fooxxxbazyyyquuux' # Your source string >>> d = 'barxxxquuxyyyfoo' # Your destination string >>> print T (s) barxxxquuxyyyfoo >>> print T (s) == d True Code attached Regards Frederic -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: translator.py URL: From steve at REMOVE-THIS-cybersource.com.au Mon Jan 18 11:26:48 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 18 Jan 2010 16:26:48 GMT Subject: substitution References: <93831c02-6e5a-48f7-a698-3b3f1ef6e5b6@v25g2000yqk.googlegroups.com> Message-ID: <00de10c9$0$15570$c3e8da3@news.astraweb.com> On Mon, 18 Jan 2010 06:23:44 -0800, Iain King wrote: > On Jan 18, 2:17?pm, Adi Eyal wrote: [...] >> Using regular expressions the answer is short (and sweet) >> >> mapping = { >> ? ? ? ? "foo" : "bar", >> ? ? ? ? "baz" : "quux", >> ? ? ? ? "quuux" : "foo" >> >> } >> >> pattern = "(%s)" % "|".join(mapping.keys()) >> repl = lambda x : mapping.get(x.group(1), x.group(1)) >> s = "fooxxxbazyyyquuux" >> re.subn(pattern, repl, s) > > Winner! :) What are the rules for being declared "Winner"? For the simple case given, calling s.replace three times is much faster: more than twice as fast. But a bigger problem is that the above "winner" may not work correctly if there are conflicts between the target strings (e.g. 'a'->'X', 'aa'->'Y'). The problem is that the result you get depends on the order of the searches, BUT as given, that order is non-deterministic. dict.keys() returns in an arbitrary order, which means the caller can't specify the order except by accident. For example: >>> repl = lambda x : m[x.group(1)] >>> m = {'aa': 'Y', 'a': 'X'} >>> pattern = "(%s)" % "|".join(m.keys()) >>> subn(pattern, repl, 'aaa') # expecting 'YX' ('XXX', 3) The result that you get using this method will be consistent but arbitrary and unpredictable. For those who care, here's my timing code: from timeit import Timer setup = """ mapping = {"foo" : "bar", "baz" : "quux", "quuux" : "foo"} pattern = "(%s)" % "|".join(mapping.keys()) repl = lambda x : mapping.get(x.group(1), x.group(1)) repl = lambda x : mapping[x.group(1)] s = "fooxxxbazyyyquuux" from re import subn """ t1 = Timer("subn(pattern, repl, s)", setup) t2 = Timer( "s.replace('foo', 'bar').replace('baz', 'quux').replace('quuux', 'foo')", "s = 'fooxxxbazyyyquuux'") And the results on my PC: >>> min(t1.repeat(number=100000)) 1.1273870468139648 >>> min(t2.repeat(number=100000)) 0.49491715431213379 -- Steven From steve at REMOVE-THIS-cybersource.com.au Mon Jan 18 11:30:22 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 18 Jan 2010 16:30:22 GMT Subject: What is a list compression in Python? References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> Message-ID: <00de11a0$0$15570$c3e8da3@news.astraweb.com> On Mon, 18 Jan 2010 08:07:41 -0800, Kit wrote: > Hello Everyone, I am not sure if I have posted this question in a > correct board. Can anyone please teach me: > > What is a list compression in Python? Google "python list comprehension". If Google is broken for you, try Yahoo, or any other search engine. > Would you mind give me some list compression examples? Instead of this: L = [] for x in range(10): L.append(x**2) you can write: L = [x**2 for x in range(10)] Instead of this example: L = [] for x in range(10): if x % 2 == 0: L.append(x**2) you can write: L = [x**2 for x in range(10) if x % 2 == 0] -- Steven From holger at merlinux.eu Mon Jan 18 11:36:52 2010 From: holger at merlinux.eu (holger krekel) Date: Mon, 18 Jan 2010 17:36:52 +0100 Subject: py.test-1.2.0: junitxml, standalone test scripts, pluginization Message-ID: <20100118163652.GB31033@trillke.net> Hi all, i just released some bits related to automated testing with Python: py-1.2.0: py.test core which grew junitxml, standalone-script generation pytest-xdist-1.0: separately installable dist-testing & looponfailing plugin pytest-figleaf-1.0: separately installable figleaf-coverage testing plugin See below or at this URL for the announcement: http://pylib.org/announce/release-1.2.0.html If you didn't experience much speed-up or previously had problems with distributed testing i recommend you try to install "pytest-xdist" now and see if it works better. For me it speeds up some tests runs by 500% on a 4 CPU machine due to its better internal model and several fixes. (It's five times because several tests depend on IO and don't block CPU meanwhile). Another tip: if you use "pip" (best with a virtualenv) you can do e.g.: pip install pytest-xdist pip uninstall pytest-xdist to conveniently activate/deactivate plugins for py.test. easy_install works ok as well but has no uninstall, yet remains the only option for installing with Python3 at the moment, though. You need to use the fine 'distribute' project's easy_install for the latter. cheers & have fun, holger py.test/pylib 1.2.0: junitxml, standalone test scripts, pluginization -------------------------------------------------------------------------------- py.test is an advanced automated testing tool working with Python2, Python3 and Jython versions on all major operating systems. It has a simple plugin architecture and can run many existing common Python test suites without modification. It offers some unique features not found in other testing tools. See http://pytest.org for more info. py.test 1.2.0 brings many bug fixes and interesting new abilities: * --junitxml=path will create an XML file for use with CI processing * --genscript=path creates a standalone py.test-equivalent test-script * --ignore=path prevents collection of anything below that path * --confcutdir=path only lookup conftest.py test configs below that path * a 'pytest_report_header' hook to add info to the terminal report header * a 'pytestconfig' function argument gives direct access to option values * 'pytest_generate_tests' can now be put into a class as well * on CPython py.test additionally installs as "py.test-VERSION", on Jython as py.test-jython and on PyPy as py.test-pypy-XYZ Apart from many bug fixes 1.2.0 also has better pluginization: Distributed testing and looponfailing testing now live in the separately installable 'pytest-xdist' plugin. The same is true for 'pytest-figleaf' for doing coverage reporting. Those two plugins can serve well now as blue prints for doing your own. thanks to all who helped and gave feedback, have fun, holger krekel, January 2010 Changes between 1.2.0 and 1.1.1 ===================================== - moved dist/looponfailing from py.test core into a new separately released pytest-xdist plugin. - new junitxml plugin: --junitxml=path will generate a junit style xml file which is processable e.g. by the Hudson CI system. - new option: --genscript=path will generate a standalone py.test script which will not need any libraries installed. thanks to Ralf Schmitt. - new option: --ignore will prevent specified path from collection. Can be specified multiple times. - new option: --confcutdir=dir will make py.test only consider conftest files that are relative to the specified dir. - new funcarg: "pytestconfig" is the pytest config object for access to command line args and can now be easily used in a test. - install 'py.test' and `py.which` with a ``-$VERSION`` suffix to disambiguate between Python3, python2.X, Jython and PyPy installed versions. - new "pytestconfig" funcarg allows access to test config object - new "pytest_report_header" hook can return additional lines to be displayed at the header of a test run. - (experimental) allow "py.test path::name1::name2::..." for pointing to a test within a test collection directly. This might eventually evolve as a full substitute to "-k" specifications. - streamlined plugin loading: order is now as documented in customize.html: setuptools, ENV, commandline, conftest. also setuptools entry point names are turned to canonical namees ("pytest_*") - automatically skip tests that need 'capfd' but have no os.dup - allow pytest_generate_tests to be defined in classes as well - deprecate usage of 'disabled' attribute in favour of pytestmark - deprecate definition of Directory, Module, Class and Function nodes in conftest.py files. Use pytest collect hooks instead. - collection/item node specific runtest/collect hooks are only called exactly on matching conftest.py files, i.e. ones which are exactly below the filesystem path of an item - change: the first pytest_collect_directory hook to return something will now prevent further hooks to be called. - change: figleaf plugin now requires --figleaf to run. Also change its long command line options to be a bit shorter (see py.test -h). - change: pytest doctest plugin is now enabled by default and has a new option --doctest-glob to set a pattern for file matches. - change: remove internal py._* helper vars, only keep py._pydir - robustify capturing to survive if custom pytest_runtest_setup code failed and prevented the capturing setup code from running. - make py.test.* helpers provided by default plugins visible early - works transparently both for pydoc and for interactive sessions which will regularly see e.g. py.test.mark and py.test.importorskip. - simplify internal plugin manager machinery - simplify internal collection tree by introducing a RootCollector node - fix assert reinterpreation that sees a call containing "keyword=..." - fix issue66: invoke pytest_sessionstart and pytest_sessionfinish hooks on slaves during dist-testing, report module/session teardown hooks correctly. - fix issue65: properly handle dist-testing if no execnet/py lib installed remotely. - skip some install-tests if no execnet is available - fix docs, fix internal bin/ script generation -- ----- End forwarded message ----- -- From invalid at invalid.invalid Mon Jan 18 11:37:22 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Mon, 18 Jan 2010 16:37:22 +0000 (UTC) Subject: Arrrrgh! Another module broken References: Message-ID: On 2010-01-18, Jive Dadson wrote: > I just found another module that broke when I went to 2.6. Gnuplot. > Apparently one of its routines has a parameter named "with." That used > to be okay, and now it's not. I remember seeing depreicated warnings about that _years_ ago, and I would have sworn it's been fixed for at least a couple years. -- Grant From steve at REMOVE-THIS-cybersource.com.au Mon Jan 18 11:37:29 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 18 Jan 2010 16:37:29 GMT Subject: Is python not good enough? References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> Message-ID: <00de134b$0$15570$c3e8da3@news.astraweb.com> On Mon, 18 Jan 2010 03:03:26 -0800, Phlip wrote: > On Jan 12, 7:09?am, ikuta liu wrote: > >> Go language try to merge low level, hight level and browser language. > > Go uses := for assignment. > > This means, to appease the self-righteous indignation of the math > professor who would claim = should mean "equality"... > > ...you gotta type a shift and 2 characters for a very common operator. I doubt it has anything to do with "the math professor". Any maths professor will tell you that, in mathematics, = is used for both assignment and equality, since in maths they are the same thing. And besides, equality testing is no less common than assignment. To appease the "self-righteous indignation of the C coders", we have to type == instead of = for a very common operator. No matter what convention you use, you're going to upset some group of people. Seriously, I programmed in Pascal for many years, and typing := for assignment is not a burden. -- Steven From steve at REMOVE-THIS-cybersource.com.au Mon Jan 18 11:39:40 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 18 Jan 2010 16:39:40 GMT Subject: Is python not good enough? References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> <37f8195f-3f02-4a13-b677-6b5d5e1589e8@22g2000yqr.googlegroups.com> <1e692369-cee9-4e63-8dce-33d70bf96368@p24g2000yqm.googlegroups.com> Message-ID: <00de13ce$0$15570$c3e8da3@news.astraweb.com> On Mon, 18 Jan 2010 07:37:36 -0800, Phlip wrote: > They fixed the hideous redundancy of Java without the ill-defined scope > issues of Python Which ill-defined scope issues are you referring to? -- Steven From jonathan.slenders at gmail.com Mon Jan 18 11:44:41 2010 From: jonathan.slenders at gmail.com (Jonathan S) Date: Mon, 18 Jan 2010 08:44:41 -0800 (PST) Subject: Python decorator syntax limitations Message-ID: Hi all, The following is what I want to do, but this results in a syntax error: @news_page('template.html').lookup(News, 'news_id', 'news') def view(request, group, news): pass What does work is the equivalent old way of doing decorating: def view(request, group, news): pass view = news_page('template.html').lookup(News, 'news_id', 'news') (view) Any suggestions? I have my reasons for doing this, (news_page is a class, and __call__ is used to wrap the template.) I'm sure this is a limitation in the syntax, but would parenthesis somewhere help? Thanks! Jonathan From iainking at gmail.com Mon Jan 18 11:46:45 2010 From: iainking at gmail.com (Iain King) Date: Mon, 18 Jan 2010 08:46:45 -0800 (PST) Subject: substitution References: <93831c02-6e5a-48f7-a698-3b3f1ef6e5b6@v25g2000yqk.googlegroups.com> <00de10c9$0$15570$c3e8da3@news.astraweb.com> Message-ID: <2fbd0038-7e19-4d22-862a-9e60b6c04aa0@22g2000yqr.googlegroups.com> On Jan 18, 4:26?pm, Steven D'Aprano wrote: > On Mon, 18 Jan 2010 06:23:44 -0800, Iain King wrote: > > On Jan 18, 2:17?pm, Adi Eyal wrote: > [...] > >> Using regular expressions the answer is short (and sweet) > > >> mapping = { > >> ? ? ? ? "foo" : "bar", > >> ? ? ? ? "baz" : "quux", > >> ? ? ? ? "quuux" : "foo" > > >> } > > >> pattern = "(%s)" % "|".join(mapping.keys()) > >> repl = lambda x : mapping.get(x.group(1), x.group(1)) > >> s = "fooxxxbazyyyquuux" > >> re.subn(pattern, repl, s) > > > Winner! :) > > What are the rules for being declared "Winner"? For the simple case > given, calling s.replace three times is much faster: more than twice as > fast. > > But a bigger problem is that the above "winner" may not work correctly if > there are conflicts between the target strings (e.g. 'a'->'X', > 'aa'->'Y'). The problem is that the result you get depends on the order > of the searches, BUT as given, that order is non-deterministic. > dict.keys() returns in an arbitrary order, which means the caller can't > specify the order except by accident. For example: > > >>> repl = lambda x : m[x.group(1)] > >>> m = {'aa': 'Y', 'a': 'X'} > >>> pattern = "(%s)" % "|".join(m.keys()) > >>> subn(pattern, repl, 'aaa') ?# expecting 'YX' > > ('XXX', 3) > > The result that you get using this method will be consistent but > arbitrary and unpredictable. > > For those who care, here's my timing code: > > from timeit import Timer > > setup = """ > mapping = {"foo" : "bar", "baz" : "quux", "quuux" : "foo"} > pattern = "(%s)" % "|".join(mapping.keys()) > repl = lambda x : mapping.get(x.group(1), x.group(1)) > repl = lambda x : mapping[x.group(1)] > s = "fooxxxbazyyyquuux" > from re import subn > """ > > t1 = Timer("subn(pattern, repl, s)", setup) > t2 = Timer( > "s.replace('foo', 'bar').replace('baz', 'quux').replace('quuux', 'foo')", > "s = 'fooxxxbazyyyquuux'") > > And the results on my PC: > > >>> min(t1.repeat(number=100000)) > 1.1273870468139648 > >>> min(t2.repeat(number=100000)) > > 0.49491715431213379 > > -- > Steven Adi elicited that response from me because his solution was vastly more succinct than everything else that had appeared up til that point while still meeting the OP's requirements. The OP never cared about overlap between 2 'find' strings, just between the 'find' and 'replace' strings (though I did take it into account in my second post for the sake of completeness). His code could have been a little cleaner, I'd have trimmed it to: mapping = {"foo": "bar", "baz": "quux", "quuux": "foo"} pattern = "(%s)" % "|".join(mapping) repl = lambda x : mapping[x.group(1)] s = "fooxxxbazyyyquuux" re.subn(pattern, repl, s) but apart from that was very pythonic: explicit, succinct, and all the heavy work is done by the module (i.e. in compiled c code in the majority case of CPython). It can be 'upgraded' to cover the find- find overlap if you really want (I *believe* regexps will match the leftmost option in a group first): subs = [("foo", "bar"), ("baz", "quux"), ("quuux", "foo")] pattern = "(%s)" % "|".join((x[0] for x in subs)) mapping = dict(subs) repl = lambda x : mapping[x.group(1)] s = "fooxxxbazyyyquuux" re.subn(pattern, repl, s) Anyway, there's no prize for winning, but by all means: if you think someone else's code and not a variation on this should win for most pythonic, then make your nomination :) Iain From showell30 at yahoo.com Mon Jan 18 12:00:44 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 18 Jan 2010 09:00:44 -0800 (PST) Subject: Python decorator syntax limitations References: Message-ID: On Jan 18, 8:44?am, Jonathan S wrote: > Hi all, > The following is what I want to do, but this results in a syntax > error: > > @news_page('template.html').lookup(News, 'news_id', 'news') > def view(request, group, news): > ? ? pass > > What does work is the equivalent old way of doing decorating: > > def view(request, group, news): > ? ? pass > view = news_page('template.html').lookup(News, 'news_id', 'news') > (view) > > Any suggestions? I have my reasons for doing this, (news_page is a > class, and __call__ is used to wrap the template.) > I'm sure this is a limitation in the syntax, but would parenthesis > somewhere help? You might want to consider doing something like this: def lookup_view(page_class, template, class_, id_, label): def decorator(func): return page_class(template).lookup(class_, id_, label) (func) return decorator @lookup_view(news_page, 'template.html', News, 'news_id', 'news') def view(...): pass From jeanmichel at sequans.com Mon Jan 18 12:05:37 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 18 Jan 2010 18:05:37 +0100 Subject: not and is not problem In-Reply-To: <00de103b$0$15570$c3e8da3@news.astraweb.com> References: <4b54819e$0$1106$4fafbaef@reader4.news.tin.it> <00de103b$0$15570$c3e8da3@news.astraweb.com> Message-ID: <4B5494E1.4080205@sequans.com> Steven D'Aprano wrote: > On Mon, 18 Jan 2010 16:43:25 +0100, superpollo wrote: > > >> hi: >> >> #!/usr/bin/env python >> data = "seq=123" >> name , value = data.split("=") >> print name >> print value >> if not name == "seq": >> print "DOES NOT PRINT OF COURSE..." >> if name is not "seq": >> print "WTF! WHY DOES IT PRINT?" >> > [snip] > Beware: Python caches strings that look like identifiers. This will > include "seq". So, purely as an implementation-specific detail, Python > will sometimes re-use the same string object, and sometimes not. > This is the most important point to understand. Some people use equality where in fact they should test for identity, because of this 'cache' effect Steven described that makes you feel that the 'is' operator is not reliable. >>>'seq' is 'seq' True >>>name = 'seq' >>>name is 'seq' True >>>name, _ = 'seq _'.split() >>>name is 'seq' False In your case, what you want is equality, because you don't care if name and 'seq' are the same object in memory, there's a good chance they are not in fact (see last example). A simple example to illustrate the difference: >>>class Foo: >>> def __eq__(self, any): >>> """Foo equals everything." >>> return True >>>f = Foo() >>>f == None True >>>f is None False f equals None, but f is not None. JM From lie.1296 at gmail.com Mon Jan 18 12:25:11 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Tue, 19 Jan 2010 04:25:11 +1100 Subject: Python decorator syntax limitations In-Reply-To: References: Message-ID: <4b54998b$1@dnews.tpgi.com.au> On 01/19/10 03:44, Jonathan S wrote: > Any suggestions? I have my reasons for doing this, (news_page is a > class, and __call__ is used to wrap the template.) > I'm sure this is a limitation in the syntax, but would parenthesis > somewhere help? The restriction[1] is put in there since Guido has a "gut feeling" that allowing arbitrary expression in decorator syntax severely harms readability. http://mail.python.org/pipermail/python-dev/2004-August/046711.html A workaround is to put an assignment above it: nplookup = news_page('template.html').lookup @nplookup(News, 'news_id', 'news') def view(request, group, news): pass If you are sure you can put up a convincing argument for lifting this restriction, and you are willing to put some time arguing, you are welcome to start a thread in the python-dev mailing list. Be sure to read about previous discussions, as repeating arguments wouldn't change anything. [1] there is a subtle difference between "restriction" and "limitation" From gabriele.modena at gmail.com Mon Jan 18 12:25:20 2010 From: gabriele.modena at gmail.com (Gabriele Modena) Date: Mon, 18 Jan 2010 18:25:20 +0100 Subject: ctypes: nested structures and pointers In-Reply-To: <1fe1d5d61001180133gb27a01dsb7434087909aff0@mail.gmail.com> References: <1fe1d5d61001180133gb27a01dsb7434087909aff0@mail.gmail.com> Message-ID: <1fe1d5d61001180925gcad560ep1bdc82866c19bd29@mail.gmail.com> On Mon, Jan 18, 2010 at 10:33 AM, Gabriele Modena wrote: > ?1. what is the correct (pythonic) way to capture the prototype > definition of dev_callbacks and the relation between that structure > and dev_info? > > ?2. is it correct to wrap "connect", "transceive" and "disconnect" in > separate structures and reference them within ?pyDEV_CALLBACKS? Solved. The answer was in the doc: http://docs.python.org/library/ctypes.html#structured-data-types (_fields_ & _pack_). From solipsis at pitrou.net Mon Jan 18 12:42:53 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 18 Jan 2010 17:42:53 +0000 (UTC) Subject: Generic Python Benchmark suite? References: <4b543838$0$7618$9b4e6d93@newsspool1.arcor-online.net> Message-ID: Le Mon, 18 Jan 2010 11:30:16 +0100, Stefan Behnel a ?crit?: > Anand Vaidya, 18.01.2010 10:58: >> Is there a generic python benchmark suite in active development? [...] >> PS: I think a benchmark should cover file / network, database I/O, >> data structures (dict, list etc), object creation/manipulation, >> numbers, measure looping inefficiencies, effects of caching (memcache >> etc) at the minimum > > That doesn't sound generic at all. Maybe you should prefer an > application based benchmark instead. Actually, it sounds much more generic that most of the benchmarks we usually rely on :-) From __peter__ at web.de Mon Jan 18 12:46:08 2010 From: __peter__ at web.de (Peter Otten) Date: Mon, 18 Jan 2010 18:46:08 +0100 Subject: Python decorator syntax limitations References: Message-ID: Jonathan S wrote: > Hi all, > The following is what I want to do, but this results in a syntax > error: > > > @news_page('template.html').lookup(News, 'news_id', 'news') > def view(request, group, news): > pass > > > What does work is the equivalent old way of doing decorating: > > > def view(request, group, news): > pass > view = news_page('template.html').lookup(News, 'news_id', 'news') > (view) > > > > Any suggestions? I have my reasons for doing this, (news_page is a > class, and __call__ is used to wrap the template.) > I'm sure this is a limitation in the syntax, but would parenthesis > somewhere help? @apply(lambda:news_page('template.html').lookup(News, 'news_id', 'news')) def view(request, group, news): pass But I'd go with a dedicated helper function. Peter From solipsis at pitrou.net Mon Jan 18 12:47:40 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 18 Jan 2010 17:47:40 +0000 (UTC) Subject: Generic Python Benchmark suite? References: Message-ID: Le Mon, 18 Jan 2010 01:58:42 -0800, Anand Vaidya a ?crit?: > Is there a generic python benchmark suite in active development? I am > looking forward to comparing some code on various python implementations > (primarily CPython 2.x, CPython 3.x, UnladenSwallow, Psyco). > > I am happy with something that gives me a relative number eg: ULS is 30% > faster than CPy2.x etc > > I found pybench which is probably not maintained actively. pybench is not abandoned. However, it only gets an addition now and then. More importantly, it's a set of low-level microbenchmarks designed to stress the execution cost of certain primitives. It won't give you the answer to any high-level questions. In the SVN sandbox (*) you'll find a couple of other benchmarks: - stringbench, stressing string operations - iobench, stressing common file I/O operations - ccbench, trying to measure interpreter efficiency in the face of multi- threaded workloads (*) http://svn.python.org/view/sandbox/trunk/ Regards Antoine. From jeanmichel at sequans.com Mon Jan 18 12:54:06 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 18 Jan 2010 18:54:06 +0100 Subject: using super In-Reply-To: References: <0361ea2d$0$1336$c3e8da3@news.astraweb.com> Message-ID: <4B54A03E.6010702@sequans.com> >>> class SubClass(Base): >>> colour = "Red" >>> def parrot(self): >>> """docstring for Subclass""" >>> return super(Subclass, self).parrot() I'm not a big fan of super, but I'm still wondering if >>> return super(self.__class__, self).parrot() would have made it. What if Subclass has more than one base class ? JM From solipsis at pitrou.net Mon Jan 18 12:55:41 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 18 Jan 2010 17:55:41 +0000 (UTC) Subject: python gui ide under linux..like visual studio ;) ? References: <4b5470ce$0$1103$4fafbaef@reader4.news.tin.it> Message-ID: Le Mon, 18 Jan 2010 15:32:36 +0100, ted a ?crit?: > > And, a good library for access to database (mysql, sql server, oracle) ? If you want something high-level: http://www.sqlalchemy.org/ You won't regret it :) From duncan.booth at invalid.invalid Mon Jan 18 13:11:14 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 18 Jan 2010 18:11:14 GMT Subject: using super References: Message-ID: Jean-Michel Pichavant wrote: > >>>> class SubClass(Base): >>>> colour = "Red" >>>> def parrot(self): >>>> """docstring for Subclass""" >>>> return super(Subclass, self).parrot() > I'm not a big fan of super, but I'm still wondering if > > >>> return super(self.__class__, self).parrot() > > would have made it. No, it wouldn't. > What if Subclass has more than one base class ? super() will work in that case provided you specify the current class. It never works correctly if you specify the class of self. Consider another level of subclasses and it may be clearer: >>> class Base(object): def parrot(self): print "Base.parrot" >>> class SubClass(Base): def parrot(self): print "SubClass.parrot" return super(SubClass, self).parrot() >>> class SubSubClass(SubClass): def parrot(self): print "SubSubClass.parrot" return super(SubSubClass, self).parrot() >>> SubSubClass().parrot() SubSubClass.parrot SubClass.parrot Base.parrot >>> class SubClass(Base): def parrot(self): print "SubClass.parrot" return super(self.__class__, self).parrot() >>> class SubSubClass(SubClass): def parrot(self): print "SubSubClass.parrot" return super(self.__class__, self).parrot() >>> SubSubClass().parrot() SubClass.parrot SubClass.parrot SubClass.parrot SubClass.parrot SubClass.parrot SubClass.parrot SubClass.parrot SubClass.parrot SubClass.parrot ... (you're in an infinite loop now) ... From python at mrabarnett.plus.com Mon Jan 18 14:00:09 2010 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 18 Jan 2010 19:00:09 +0000 Subject: Is python not good enough? In-Reply-To: <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> Message-ID: <4B54AFB9.6000905@mrabarnett.plus.com> Phlip wrote: > On Jan 12, 7:09 am, ikuta liu wrote: > >> Go language try to merge low level, hight level and browser language. > > Go uses := for assignment. > > This means, to appease the self-righteous indignation of the math > professor who would claim = should mean "equality"... > > ...you gotta type a shift and 2 characters for a very common operator. > > Pass! > If I were going to list what I didn't like about Go, that wouldn't be one of them! From python at mrabarnett.plus.com Mon Jan 18 14:04:08 2010 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 18 Jan 2010 19:04:08 +0000 Subject: basic Class in Python In-Reply-To: References: <86a6a8c8-49fe-4a3d-81a0-5a44a179c7d6@o28g2000yqh.googlegroups.com> Message-ID: <4B54B0A8.9040706@mrabarnett.plus.com> bartc wrote: > > "Wolfgang Rohdewald" wrote in message > news:mailman.1056.1263771299.28905.python-list at python.org... >> On Monday 18 January 2010, BarryJOgorman wrote: >>> TypeError: object._new_() takes no parameters >> >>> def _init_(self, name, job=None, pay=0): >> >> __init__ needs two underscores left and right > > Any particular reason why two, and not one (or three)? In some fonts > it's difficult to tell how many as they run together. > I believe it was borrowed from C, which has __FILE__ and __LINE__ (CPython is written in C). With hindsight, of course, single underscores would've been sufficient... From aahz at pythoncraft.com Mon Jan 18 14:15:48 2010 From: aahz at pythoncraft.com (Aahz) Date: 18 Jan 2010 11:15:48 -0800 Subject: Python decorator syntax limitations References: <4b54998b$1@dnews.tpgi.com.au> Message-ID: In article <4b54998b$1 at dnews.tpgi.com.au>, Lie Ryan wrote: > >If you are sure you can put up a convincing argument for lifting this >restriction, and you are willing to put some time arguing, you are >welcome to start a thread in the python-dev mailing list. Be sure to >read about previous discussions, as repeating arguments wouldn't change >anything. While I haven't been reading python-dev in recent months, I'm pretty sure the moratorium makes the only appropriate place for this python-ideas, and even there people are not likely to invest much time: http://www.python.org/dev/peps/pep-3003/ -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." --Red Adair From adi at digitaltrowel.com Mon Jan 18 14:24:24 2010 From: adi at digitaltrowel.com (Adi Eyal) Date: Mon, 18 Jan 2010 21:24:24 +0200 Subject: substitution Message-ID: > From:?Steven D'Aprano > To:?python-list at python.org > Date:?18 Jan 2010 16:26:48 GMT > Subject:?Re: substitution > On Mon, 18 Jan 2010 06:23:44 -0800, Iain King wrote: > >> On Jan 18, 2:17?pm, Adi Eyal wrote: > [...] >>> Using regular expressions the answer is short (and sweet) >>> >>> mapping = { >>> ? ? ? ? "foo" : "bar", >>> ? ? ? ? "baz" : "quux", >>> ? ? ? ? "quuux" : "foo" >>> >>> } >>> >>> pattern = "(%s)" % "|".join(mapping.keys()) >>> repl = lambda x : mapping.get(x.group(1), x.group(1)) >>> s = "fooxxxbazyyyquuux" >>> re.subn(pattern, repl, s) >> >> Winner! :) > > What are the rules for being declared "Winner"? For the simple case > given, calling s.replace three times is much faster: more than twice as > fast. :) - The solution above is short and easy to read and digest. It is also easy to maintain - you simply need to add additional entries to the mapping dict if you need to add additional strings. Calling replace 3 times in a row doesn't work correctly e.g. foo => bar bar => baz foo.replace("foo", "bar").replace("bar", "baz") > > But a bigger problem is that the above "winner" may not work correctly if > there are conflicts between the target strings (e.g. 'a'->'X', > 'aa'->'Y'). The problem is that the result you get depends on the order > of the searches, BUT as given, that order is non-deterministic. > dict.keys() returns in an arbitrary order, which means the caller can't > specify the order except by accident. For example: > >>>> repl = lambda x : m[x.group(1)] >>>> m = {'aa': 'Y', 'a': 'X'} >>>> pattern = "(%s)" % "|".join(m.keys()) >>>> subn(pattern, repl, 'aaa') ?# expecting 'YX' > ('XXX', 3) > > The result that you get using this method will be consistent but > arbitrary and unpredictable. > That's a simple fix import re mapping = { "foo" : "bar", "baz" : "quux", "quuux" : "foo" } keys = [(len(key), key) for key in mapping.keys()] keys.sort(reverse=True) keys = [key for (_, key) in keys] pattern = "(%s)" % "|".join(keys) repl = lambda x : mapping[x.group(1)] s = "fooxxxbazyyyquuux" re.subn(pattern, repl, s) With regards to timing - I suspect that as the input string grows, the ratio of the regexp solution to the string solution will grow. If the map grows, the regexp solution seems faster (see below). Python's regular expression state machine is very inefficient with lots of disjunctions since it involves a lot of backtracking - this can be hand optimised for a much greater speed improvement. N = 1000 alphabet = "abcdefghijklmnopqrstuvwxyz" make_token = lambda : "".join(sample(alphabet, 5)) mapping = dict([(make_token(), make_token()) for i in range(N)]) keys = [(len(key), key) for key in mapping.keys()] keys.sort(reverse=True) keys = [key for (_, key) in keys] pattern = "(%s)" % "|".join(keys) replace_strs = ["replace('%s', '%s')" % (key, mapping[key]) for key in keys] command = "s." + ".".join(replace_strs) setup = """ import re regexp = re.compile("%s") repl = lambda x : mapping[x.group(1)] s = "fooxxxbazyyyquuux" """ % pattern t1 = Timer("regexp.subn(repl, s)", setup) t2 = Timer(command, "s = 'fooxxxbazyyyquuux'") res1 = sum(t1.repeat(number=1000)) res2 = sum(t2.repeat(number=1000)) print res1 / res2 on my machine this comes to >>> 0.316323109737 From arnodel at googlemail.com Mon Jan 18 14:31:39 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 18 Jan 2010 19:31:39 +0000 Subject: Python decorator syntax limitations References: <4b54998b$1@dnews.tpgi.com.au> Message-ID: aahz at pythoncraft.com (Aahz) writes: > In article <4b54998b$1 at dnews.tpgi.com.au>, > Lie Ryan wrote: >> >>If you are sure you can put up a convincing argument for lifting this >>restriction, and you are willing to put some time arguing, you are >>welcome to start a thread in the python-dev mailing list. Be sure to >>read about previous discussions, as repeating arguments wouldn't change >>anything. > > While I haven't been reading python-dev in recent months, I'm pretty sure > the moratorium makes the only appropriate place for this python-ideas, > and even there people are not likely to invest much time: > > http://www.python.org/dev/peps/pep-3003/ In actual fact there is a recent discussion of this on python-ideas: http://groups.google.co.uk/group/python-ideas/browse_thread/thread/1eebf486969c39a1/ -- Arnaud From aahz at pythoncraft.com Mon Jan 18 14:45:59 2010 From: aahz at pythoncraft.com (Aahz) Date: 18 Jan 2010 11:45:59 -0800 Subject: lightweight encryption of text file References: <0357cddd$0$1336$c3e8da3@news.astraweb.com> Message-ID: In article , Daniel Fetchinson wrote: > >Well, that's sort of true about learning a complex API :) But it's >also true that I'm not storing anything really valuable in the file >but still wouldn't want to leave it lying around in plain text. In >case I lose the laptop with the file I seriously doubt anybody who >finds it will go through each and every file and try to find what's in >it, even though they look like data files and there is no hint what so >ever that any one of them contains encrypted info. If they see a text >file, well, that can give them ideas, so let's encrypt a little bit. One reason I like OSX is that it's dirt-simple to encrypt my home dir. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." --Red Adair From jeanmichel at sequans.com Mon Jan 18 14:50:45 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 18 Jan 2010 20:50:45 +0100 Subject: using super In-Reply-To: References: Message-ID: <4B54BB95.1090407@sequans.com> Duncan Booth wrote: > Jean-Michel Pichavant wrote: > > >>>>> class SubClass(Base): >>>>> colour = "Red" >>>>> def parrot(self): >>>>> """docstring for Subclass""" >>>>> return super(Subclass, self).parrot() >>>>> >> I'm not a big fan of super, but I'm still wondering if >> >> >>>>> return super(self.__class__, self).parrot() >>>>> >> would have made it. >> > > No, it wouldn't. > > >> What if Subclass has more than one base class ? >> > > super() will work in that case provided you specify the current class. It > never works correctly if you specify the class of self. > > Consider another level of subclasses and it may be clearer: > > >>>> class Base(object): >>>> > def parrot(self): > print "Base.parrot" > > > >>>> class SubClass(Base): >>>> > def parrot(self): > print "SubClass.parrot" > return super(SubClass, self).parrot() > > > >>>> class SubSubClass(SubClass): >>>> > def parrot(self): > print "SubSubClass.parrot" > return super(SubSubClass, self).parrot() > > > >>>> SubSubClass().parrot() >>>> > SubSubClass.parrot > SubClass.parrot > Base.parrot > > >>>> class SubClass(Base): >>>> > def parrot(self): > print "SubClass.parrot" > return super(self.__class__, self).parrot() > > > >>>> class SubSubClass(SubClass): >>>> > def parrot(self): > print "SubSubClass.parrot" > return super(self.__class__, self).parrot() > > >>>> SubSubClass().parrot() >>>> > SubClass.parrot > SubClass.parrot > SubClass.parrot > SubClass.parrot > SubClass.parrot > SubClass.parrot > SubClass.parrot > SubClass.parrot > SubClass.parrot > ... (you're in an infinite loop now) ... > I see. Then is there a reason why >>> return super(Subclass, self).parrot() would be prefered over the "classic" >>> return Base.parrot(self) ? Or is it just a matter of preference ? JM From vicente.soler at gmail.com Mon Jan 18 15:02:51 2010 From: vicente.soler at gmail.com (vsoler) Date: Mon, 18 Jan 2010 12:02:51 -0800 (PST) Subject: Unable to install numpy Message-ID: <3a772f49-aee6-4450-b60e-8f05daec136a@21g2000yqj.googlegroups.com> Hi all, I just download Numpy, and tried to install it using "numpy-1.4.0- win32-superpack-python2.6.exe" I get an error: "Python version 2.6 required, which was not found in the Registry" However, I am using Python 2.6 every day. I'm running Windows 7. What can I do? From robert.kern at gmail.com Mon Jan 18 15:08:41 2010 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 18 Jan 2010 14:08:41 -0600 Subject: Unable to install numpy In-Reply-To: <3a772f49-aee6-4450-b60e-8f05daec136a@21g2000yqj.googlegroups.com> References: <3a772f49-aee6-4450-b60e-8f05daec136a@21g2000yqj.googlegroups.com> Message-ID: On 2010-01-18 14:02 PM, vsoler wrote: > Hi all, > > I just download Numpy, and tried to install it using "numpy-1.4.0- > win32-superpack-python2.6.exe" > > I get an error: "Python version 2.6 required, which was not found in > the Registry" > > However, I am using Python 2.6 every day. I'm running Windows 7. > > What can I do? Please ask numpy installation questions on the numpy mailing list. http://www.scipy.org/Mailing_Lists -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From arnodel at googlemail.com Mon Jan 18 15:19:40 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 18 Jan 2010 20:19:40 +0000 Subject: using super References: Message-ID: Jean-Michel Pichavant writes: [...] > Then is there a reason why >>>> return super(Subclass, self).parrot() > would be prefered over the "classic" >>>> return Base.parrot(self) > ? > > Or is it just a matter of preference ? Using super() calls the next method in the class's Method Resolution Order (mro - to see it on class A, use A.mro()). This guarantees that ancestor methods won't be called twice when the inheritance graph is not a tree. Here is a (minimal?) example. >>> class A(object): ... def foo(self): print 'A' ... >>> class B(A): ... def foo(self): ... super(B, self).foo() ... print 'B' ... >>> class C(A): ... def foo(self): ... super(C, self).foo() ... print 'C' ... >>> class D(B, C): ... def foo(self): ... super(D, self).foo() ... print 'D' ... >>> d = D() >>> d.foo() A C B D >>> D.mro() [, , , , ] The reason why super() may be necessary is that if an object is an instance of say class C, its method resolution order above class C is not known at compile time. HTH -- Arnaud From joaopcf at gmail.com Mon Jan 18 15:50:01 2010 From: joaopcf at gmail.com (=?ISO-8859-1?Q?Jo=E3o?=) Date: Mon, 18 Jan 2010 12:50:01 -0800 (PST) Subject: force URLencoding script References: <6bc42c49-e26c-4f7b-9538-dd2e319accfb@26g2000yqo.googlegroups.com> <9bc83518-b641-47a9-84ef-c68c9f825248@26g2000yqo.googlegroups.com> <4c68f05e-1fde-4e39-bb8a-803aa32de201@c34g2000yqn.googlegroups.com> Message-ID: <3877ca5f-2412-425b-bc98-2a36f6480346@m26g2000yqb.googlegroups.com> On Jan 15, 4:46?pm, r0g wrote: > Jo?o wrote: > > On Jan 15, 2:38 pm, r0g wrote: > >> Jo?o wrote: > >>> On Jan 14, 5:58 pm, r0g wrote: > >>>> Jo?o wrote: > >>>>> On Jan 12, 10:07 pm, r0g wrote: > >>>>>> Jo?o wrote: > >>>>> for the following data, > >>>>> authentication = "UID=somestring&" > >>>>> message = 'PROBLEM severity High: OperatorX Plat1(locationY) global > >>>>> Succ. : 94.470000%' > >>>>> dest_number = 'XXXXXXXXXXX' > >>>>> url_values = urlencode({'M':message}) > >>>>> enc_data = authentication + url_values + dest_number > >>>>> I'm getting null for > >>>>> full_url = Request(url, enc_data, headers) > >>>>> and thus, > >>>>> response = urlopen(full_url).read() > >>>>> returns, > >>>>> TypeError: > >>>>> ) > >>>> Are you sure it's returning a null and not just some other unexpected > >>>> type? > >>>> I think your problem may be that you are passing a urllib2 class to > >>>> urllib(1)'s urlopen. Try using urllib2's urlopen instead e.g. > >>>> import urllib2 > >>>> request_object = urllib2.Request('http://www.example.com') > >>>> response = urllib2.urlopen(request_object) > >>>> the_page = response.read() > >>>> Roger. > >>> Thanks Roger. > >>> I think it's a null because i did a print(full_url) right after the > >>> Request > >>> I tried > >>> request_object = urllib2.Request('http://www.example.com') > >>> print(request_object) > >>> but when printing I get: > >> Hi Jo?o, > > >> That's exactly what you want, an object that is an instance of the > >> Request class. That object doesn't do anything by itself, you still need > >> to a) Connect to the server and request that URL and b) Read the data > >> from the server. > > >> a) To connect to the web server and initialize the request you need to > >> call urllib2.urlopen() with the Request object you just created and > >> assign the result to a name e.g. > > >>>> response = urllib2.urlopen(request_object) > >> That will give you an object (response) that you can call the .read() > >> method of to get the web page data. > > >>>> the_page = response.read() > >> If that doesn't make sense or seem to work for you then please try > >> reading the following website from top to bottom before taking any > >> further steps... > > >>http://www.voidspace.org.uk/python/articles/urllib2.shtml > > >>> I've read about Python 2.4 not playing well with proxies even with no > >>> proxy activated. > >>> Any sugestion? > >> I doubt any language can play well with proxies if there are none so I > >> doubt it's a factor ;) > > >> Good luck, > > >> Roger. > > > lol. > > I've expressed myself poorly, > > I meant I read about some issues when getting the Request + urlopen > > working when there's a proxy involved (like in my case) > > even when activating a no_proxy configuration, something like, > > > proxy_support = urllib.ProxyHandler({}) > > opener = urllib.build_opener(proxy_support) > > urllib.install_opener(opener) > > > But I don't know how to use it :( > > That is how you use it IIRC, this installs the proxy handler into urllib > and subsequent objects you subclass from urllib will use the custom handler. > > From what I can tell, you should be using urllib2 though, not urllib. > > Lets take a step back. You had the following line... > > request_object = urllib2.Request('http://www.example.com') > > ...You printed it and it showed that you had created a Request object > right. Now what happens when you type... > > response = urllib2.urlopen(request_object) > print response > > ? > > Roger. Thanks for the patience Roger. Your explanation opened my eyes. I finally got it to work, and it turned out I didn't have to include any custom proxy handler to avoid our proxy. It ended on such a small and simple block of code after getting the pieces together.. #!/usr/bin/env python import sys from urllib2 import Request, urlopen from urllib import urlencode authentication = "UID=124675&PW=gdberishyb8LcIANtvT89QVQ==&" url = 'http://10.112.28.221:38080/GwHTTPin/sendtext' encoded_data = urlencode({'M':sys.argv[2], 'N':sys.argv[1]}) # Was having problem with this one, shouldn't have tried to pass the authentication as an urlencode parameter because it was breaking the password string! sent_data = authentication + encoded_data full_url = Request(url,sent_data) response = urlopen(full_url).read() From joaopcf at gmail.com Mon Jan 18 15:50:22 2010 From: joaopcf at gmail.com (=?ISO-8859-1?Q?Jo=E3o?=) Date: Mon, 18 Jan 2010 12:50:22 -0800 (PST) Subject: force URLencoding script References: <6bc42c49-e26c-4f7b-9538-dd2e319accfb@26g2000yqo.googlegroups.com> <9bc83518-b641-47a9-84ef-c68c9f825248@26g2000yqo.googlegroups.com> <4c68f05e-1fde-4e39-bb8a-803aa32de201@c34g2000yqn.googlegroups.com> Message-ID: <9ff54fda-9b55-4f9c-b1bc-48dd2ef66017@c29g2000yqd.googlegroups.com> On Jan 15, 4:46?pm, r0g wrote: > Jo?o wrote: > > On Jan 15, 2:38 pm, r0g wrote: > >> Jo?o wrote: > >>> On Jan 14, 5:58 pm, r0g wrote: > >>>> Jo?o wrote: > >>>>> On Jan 12, 10:07 pm, r0g wrote: > >>>>>> Jo?o wrote: > >>>>> for the following data, > >>>>> authentication = "UID=somestring&" > >>>>> message = 'PROBLEM severity High: OperatorX Plat1(locationY) global > >>>>> Succ. : 94.470000%' > >>>>> dest_number = 'XXXXXXXXXXX' > >>>>> url_values = urlencode({'M':message}) > >>>>> enc_data = authentication + url_values + dest_number > >>>>> I'm getting null for > >>>>> full_url = Request(url, enc_data, headers) > >>>>> and thus, > >>>>> response = urlopen(full_url).read() > >>>>> returns, > >>>>> TypeError: > >>>>> ) > >>>> Are you sure it's returning a null and not just some other unexpected > >>>> type? > >>>> I think your problem may be that you are passing a urllib2 class to > >>>> urllib(1)'s urlopen. Try using urllib2's urlopen instead e.g. > >>>> import urllib2 > >>>> request_object = urllib2.Request('http://www.example.com') > >>>> response = urllib2.urlopen(request_object) > >>>> the_page = response.read() > >>>> Roger. > >>> Thanks Roger. > >>> I think it's a null because i did a print(full_url) right after the > >>> Request > >>> I tried > >>> request_object = urllib2.Request('http://www.example.com') > >>> print(request_object) > >>> but when printing I get: > >> Hi Jo?o, > > >> That's exactly what you want, an object that is an instance of the > >> Request class. That object doesn't do anything by itself, you still need > >> to a) Connect to the server and request that URL and b) Read the data > >> from the server. > > >> a) To connect to the web server and initialize the request you need to > >> call urllib2.urlopen() with the Request object you just created and > >> assign the result to a name e.g. > > >>>> response = urllib2.urlopen(request_object) > >> That will give you an object (response) that you can call the .read() > >> method of to get the web page data. > > >>>> the_page = response.read() > >> If that doesn't make sense or seem to work for you then please try > >> reading the following website from top to bottom before taking any > >> further steps... > > >>http://www.voidspace.org.uk/python/articles/urllib2.shtml > > >>> I've read about Python 2.4 not playing well with proxies even with no > >>> proxy activated. > >>> Any sugestion? > >> I doubt any language can play well with proxies if there are none so I > >> doubt it's a factor ;) > > >> Good luck, > > >> Roger. > > > lol. > > I've expressed myself poorly, > > I meant I read about some issues when getting the Request + urlopen > > working when there's a proxy involved (like in my case) > > even when activating a no_proxy configuration, something like, > > > proxy_support = urllib.ProxyHandler({}) > > opener = urllib.build_opener(proxy_support) > > urllib.install_opener(opener) > > > But I don't know how to use it :( > > That is how you use it IIRC, this installs the proxy handler into urllib > and subsequent objects you subclass from urllib will use the custom handler. > > From what I can tell, you should be using urllib2 though, not urllib. > > Lets take a step back. You had the following line... > > request_object = urllib2.Request('http://www.example.com') > > ...You printed it and it showed that you had created a Request object > right. Now what happens when you type... > > response = urllib2.urlopen(request_object) > print response > > ? > > Roger. Thanks for the patience Roger. Your explanation opened my eyes. I finally got it to work, and it turned out I didn't have to include any custom proxy handler to avoid our proxy. It ended on such a small and simple block of code after getting the pieces together.. #!/usr/bin/env python import sys from urllib2 import Request, urlopen from urllib import urlencode authentication = "UID=124675&PW=gdberishyb8LcIANtvT89QVQ==&" url = 'http://10.112.28.221:38080/GwHTTPin/sendtext' encoded_data = urlencode({'M':sys.argv[2], 'N':sys.argv[1]}) # Was having problem with this one, shouldn't have tried to pass the authentication as an urlencode parameter because it was breaking the password string! sent_data = authentication + encoded_data full_url = Request(url,sent_data) response = urlopen(full_url).read() From jonathan.slenders at gmail.com Mon Jan 18 15:57:03 2010 From: jonathan.slenders at gmail.com (Jonathan S) Date: Mon, 18 Jan 2010 12:57:03 -0800 (PST) Subject: Python decorator syntax limitations References: <4b54998b$1@dnews.tpgi.com.au> Message-ID: <28e5220e-82f8-44d0-bede-a82fd3fbfd95@s31g2000yqs.googlegroups.com> Thanks a lot, all of you! This was really helpful. (or at least give me the inspiration I needed to finish it.) I'm sure this is a use case where most other options are less readable than the chain of methods in the decorator. In this use case, I had a lot of Django views to which access permissions had to be attached. Chaining the permissions was in my opinion the easiest way to go but appeared not to work because Guido had a gut feeling about that. Using multiple decorators in the usual way (like every following wraps the previous) does not work, because these decorators need to access the same authentication class. (little hard to explain, but take from me that I had a case where it didn't work.) Placing all possible options in the constructor method of the decorator like Steve Howell proposed would work but is ugly. This is what I made. I'll upgrade my news_page decorator to work in the following way. (Applying the news_page decorator to 'view', will turn it into a class, but make it callable buy wrapping the actual view in __call__. So we can still apply the other methods to the view. @news_page('template.html') def view(request, group, news): ... pass view.lookup(News, 'news_id', 'news') view.require_administrator() That is still rather ugly, because the 'lookup' option is appended behind the view. I kept playing with the code and came to the following: @require_administrator @do_lookup(News, 'news_id', 'news) @news_page('template.html') def view(request, group, news): ... pass Where 'do_lookup' and 'require_administrator' passes the method, but sets some parameters. So, 'do_lookup' can access class members of news_page. It would look like: def do_lookup(*lookup_options): def set_options(news_page_view): news_page_view.lookup(*lookup_options) return news_page_view return set_options def require_administrator(news_page_view): news_page_view.require_administrator() return news_page_view Maybe, I'll join the discussion later on, when I have time to read all the conversations and write down good arguments. But, honestly, I'm satisfied with the last result. Have a nice day! From kyosohma at gmail.com Mon Jan 18 15:59:20 2010 From: kyosohma at gmail.com (Mike Driscoll) Date: Mon, 18 Jan 2010 12:59:20 -0800 (PST) Subject: python gui ide under linux..like visual studio ;) ? References: <4b5470ce$0$1103$4fafbaef@reader4.news.tin.it> Message-ID: <3f3650e2-1c9d-43e1-b759-6c3d939de7bb@h9g2000yqa.googlegroups.com> On Jan 18, 8:32?am, ted wrote: > Hi at all... > Can someone please give me some advice, about a good IDE with control > GUI under Linux ? > > Actually i know QT Creator by Nokia which i can use with Python (but i > don't know how). > > And, a good library for access to database (mysql, sql server, oracle) ? > > Thank you very much ! > > bye Check out Dabo for the database stuff: http://dabodev.com/ There is nothing like Visual Studio. The closest I've found are things like the QT Creator and wxGlade / Boa Constructor / wxFormBuilder. I know I've heard of another one that was commercial, but I can't recall the name at the moment. ------------------- Mike Driscoll Blog: http://blog.pythonlibrary.org PyCon 2010 Atlanta Feb 19-21 http://us.pycon.org/ From timur at freescale.com Mon Jan 18 16:00:26 2010 From: timur at freescale.com (Timur Tabi) Date: Mon, 18 Jan 2010 15:00:26 -0600 Subject: I really need webbrowser.open('file://') to open a web browser In-Reply-To: <126148f0-fdcf-442d-b537-fd43b7bfb6b5@f12g2000yqn.googlegroups.com> References: <126148f0-fdcf-442d-b537-fd43b7bfb6b5@f12g2000yqn.googlegroups.com> Message-ID: On Sat, Jan 16, 2010 at 3:43 PM, Paul Boddie wrote: > Generally, the desktop-specific tools should know that a browser is > the appropriate application for an HTML file, and testing with both > xdg-open, gnome-open and "kfmclient openURL" seems to open browsers on > HTML files (using file:///...) for me (using KDE, Kubuntu 8.04). Of > course, this depends on the settings in use on your desktop, but it > should be noted that using "kfmclient exec" could have the effect you > describe. I'm using Gnome, and I have HTML files associated with Firefox. However, my default web browser is Seamonkey, and when I do webbrowser.open('http://...'), it opens that URL in Seamonkey, not Firefox. So if there is some Gnome association between an .html file and a text editor, I don't know where it is defined. > Not that I'm aware of. Sadly, standardisation of applications and > services - having a command which can open a particular class of > application (such as "e-mail reader", "Web browser") - seems to be > absent from the free desktop arena, although I do recall there being a > preferred applications dialogue in KDE, at least. I would be sympathetic to this problem if the API were called desktop.open(...). But it's called webbrowser.open(), so it has to be certain that a web browser is being at all times. IMHO, any other behavior is a bug. -- Timur Tabi Linux kernel developer at Freescale From mailbox394 at gmail.com Mon Jan 18 16:29:59 2010 From: mailbox394 at gmail.com (Gregory) Date: Mon, 18 Jan 2010 13:29:59 -0800 (PST) Subject: form mailer for info + files Message-ID: <3d2e2746-8e80-46fa-9e70-958955142fdc@36g2000yqu.googlegroups.com> There is code all over on how to create a form mailer. I need an example that will show how to upload two files and send them along with form field info. Is it required to overwrite the python cgi.fieldstorage to do this? Working with Python 2.4.3. From wayne.dads.bell at gmail.com Mon Jan 18 16:39:14 2010 From: wayne.dads.bell at gmail.com (dads) Date: Mon, 18 Jan 2010 13:39:14 -0800 (PST) Subject: searching and storing large quantities of xml! References: <4b52bdb8$0$6733$9b4e6d93@newsspool2.arcor-online.net> Message-ID: Thanks all, took your advice and have been playing all weekend which has been great fun. ElementTree is awesome. I created a script that organises the xml as they're in year blocks and I didn't realise the required xml is mixed up with other xml. Plus the volumes are much greater than I realised, I checked as back at work and it was something like 600,000 files in a year, just over a gig for each year. I'm going to add zipping up of the files and getting the required info and putting it in a db this week hopefully. It's been completely overhauled, originally I used modified date now it gets the date from the parsed xml, safer that way. The code is below but word of caution, it's hobbyist code so it'll probably make your eyes bleed =), thanks again: There was one thing that I forgot about - when ElementTree fails to parse due to an element not being closed why doesn't it close the file like object. As later on I would raise 'WindowsError: [Error 32] ...file being used by other process' when using shutil.move(). I got round this by using a 'try except' block. from __future__ import print_function import xml.etree.cElementTree as ET import calendar import zipfile import os.path import shutil import zlib import os class Xmlorg(object): def __init__(self): self.cwd = os.getcwd() self.year = os.path.basename(self.cwd) def _mkMonthAndDaysDirs(self): ''' creates dirs for every month and day of a of specidifed year. Works for leap years as well. (specified)year/(year)month/day ...2010/201001/01 ...2010/201001/02 ...2010/201001/03 ''' def addZero(n): if len(str(n)) < 2: return '0' + str(n) else: return str(n) dim = [ calendar.monthrange(year,month)[1] for year in \ [int(self.year)] for month in range(1,13) ] count = 1 for n in dim: month = addZero(count) count += 1 ym = os.path.join(self.cwd, self.year + month) os.mkdir(ym) for x in range(1,n+1): x = addZero(x) os.mkdir(os.path.join(ym, x)) def ParseAndOrg(self): '''requires dir and zip struct: .../(year)/(year).zip - example .../2008/2008.zip ''' def movef(fp1,fp2): '''moves files with exception handling''' try: shutil.move(fp1,fp2) except IOError, e: print(e) except WindowsError, e: print(e) self._mkMonthAndDaysDirs() os.mkdir(os.path.join(self.cwd, 'otherFileType')) # dir struct .../(year)/(year).zip - ex. .../2008/2008.zip zf = zipfile.ZipFile(os.path.join(self.cwd, self.year + '.zip')) zf.extractall() ld = os.listdir(self.cwd) for i in ld: if os.path.isfile(i) and i.endswith('.xml'): try: tree = ET.parse(i) except: print('%s np' % i) #not parsed root = tree.getroot() if root.findtext('Summary/FileType') == 'Order': date = root.findtext('OrderHeader/OrderDate')[:10] #dd/mm/yyyy dc = date.split('/') fp1 = os.path.join(self.cwd, i) fp2 = os.path.join(self.cwd, dc[2] + dc[1], dc[0]) movef(fp1,fp2) else: fp1 = os.path.join(self.cwd, i) fp2 = os.path.join(self.cwd, 'otherFileType') movef(fp1,fp2) if __name__ == '__main__': os.chdir('c:/sv_zip_test/2010/') #remove xo = Xmlorg() xo.ParseAndOrg() From tjreedy at udel.edu Mon Jan 18 16:42:19 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 18 Jan 2010 16:42:19 -0500 Subject: Generic Python Benchmark suite? In-Reply-To: References: Message-ID: On 1/18/2010 4:58 AM, Anand Vaidya wrote: > Is there a generic python benchmark suite in active development? I am > looking forward to comparing some code on various python > implementations (primarily CPython 2.x, CPython 3.x, UnladenSwallow, > Psyco). You might find this interesting if you have not seen it before: http://code.google.com/p/unladen-swallow/wiki/Benchmarks From python.list at tim.thechases.com Mon Jan 18 16:56:40 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Mon, 18 Jan 2010 15:56:40 -0600 Subject: Parse a log file In-Reply-To: <92293df0-7d2e-453a-b5bc-d4a7eb905fcf@l19g2000yqb.googlegroups.com> References: <92293df0-7d2e-453a-b5bc-d4a7eb905fcf@l19g2000yqb.googlegroups.com> Message-ID: <4B54D918.8040103@tim.thechases.com> kaklis at gmail.com wrote: > I want to parse a log file with the following format for > example: > TIMESTAMPE Operation FileName > Bytes > 12/Jan/2010:16:04:59 +0200 EXISTS sample3.3gp 37151 > 12/Jan/2010:16:04:59 +0200 EXISTS sample3.3gp 37151 > 12/Jan/2010:16:04:59 +0200 EXISTS sample3.3gp 37151 > 12/Jan/2010:16:04:59 +0200 EXISTS sample3.3gp 37151 > 12/Jan/2010:16:04:59 +0200 EXISTS sample3.3gp 37151 > 12/Jan/2010:16:05:05 +0200 DELETE sample3.3gp 37151 > > How can i count the operations for a month(e.g total of 40 Operations, > 30 exists, 10 delete?) It can be done pretty easily with a regexp to parse the relevant bits: import re r = re.compile(r'\d+/([^/]+)/(\d+)\S+\s+\S+\s+(\w+)') stats = {} for line in file('log.txt'): m = r.match(line) if m: stats[m.groups()] = stats.get(m.groups(), 0) + 1 print stats This prints out {('Jan', '2010', 'EXISTS'): 5, ('Jan', '2010', 'DELETE'): 1} With the resulting data structure, you can manipulate it to do coarser-grained aggregates such as the total operations, or remap month-name abbreviations into integers so they could be sorted for output. -tkc From showell30 at yahoo.com Mon Jan 18 17:00:17 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 18 Jan 2010 14:00:17 -0800 (PST) Subject: inner methods and recursion Message-ID: Hi, I have a style/design question relating to recursion and inner methods. I wrote some code recently that implements a recursive algorithm that takes several parameters from the original caller. Once the algorithm starts running and recursing, those parameters remain the same, so I define an inner method called "recurse" that essentially curries those parameters. In particular, the recurse method can get passed to a callback method that the caller supplies, so that the caller can wrap the recursive step with their own logic. The only thing I don't like about the technique that I'm using is that it seems needlessly repetitive to define mostly the same parameter list in two different places. The repetition seems mostly harmless, but it seems like some kind of a smell that I'm overlooking a simpler way to write the code. I just can't put my finger on what it is. def indent_lines(lines, branch_method, leaf_method, pass_syntax, flush_left_syntax, flush_left_empty_line, indentation_method, get_block, ): def recurse(lines): return indent_lines( lines, branch_method, leaf_method, pass_syntax, flush_left_syntax, flush_left_empty_line, indentation_method, get_block, ) output = [] while lines: if lines[0].strip() == '': lines.pop(0) output.append('') else: prefix, i = get_block(lines, indentation_method) if i == 1: line = lines.pop(0)[len(prefix):] if line == pass_syntax: pass elif line.startswith(flush_left_syntax): output.append(line[len(flush_left_syntax):]) elif line.startswith(flush_left_empty_line): output.append('') else: output.append(prefix + leaf_method(line)) else: block = lines[:i] lines = lines[i:] output += branch_method(prefix, block, recurse) return output Does anybody have any inspiration here? I vaguely recall a discussion some time back about having some kind of @recursive decorator, but I can't seem to find the thread. From aahz at pythoncraft.com Mon Jan 18 17:11:37 2010 From: aahz at pythoncraft.com (Aahz) Date: 18 Jan 2010 14:11:37 -0800 Subject: monitor reading file with thread References: <4b47712a$0$1115$4fafbaef@reader4.news.tin.it> Message-ID: In article <4b47712a$0$1115$4fafbaef at reader4.news.tin.it>, wiso wrote: > >class Reader(): > def __init__(self,filename): > self.filename = filename > self.lineno = 0 > def __iter__(self): > f = open(self.filename) > for line in f: > self.lineno += 1 > time.sleep(0.01) # slow down > yield line > f.close() There's no reason for the sleep. >class Monitor(threading.Thread): > def __init__(self,reader,stop_queue,interval=2): > threading.Thread.__init__(self) > self.interval = interval > self.reader = reader > self.stop_queue = stop_queue > def run(self): > while True: > try: > if self.stop_queue.get(timeout=self.interval) == "stop": > break > except Queue.Empty: > pass > print "MONITOR: ", reader.lineno I'd make this simpler, you only need a queue when you're transferring data between threads: class Monitor(threading.Thread): def __init__(self, reader, interval=2): threading.Thread.__init__(self) self.interval = interval self.reader = reader self.stop = False def run(self): while not self.stop: print "MONITOR:", reader.lineno time.sleep(self.interval) ...and to stop the monitor, just set m.stop=True. Note that you have a bug that needs fixing, I left it for you. ;-) -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." --Red Adair From jabronson at gmail.com Mon Jan 18 17:12:20 2010 From: jabronson at gmail.com (Joshua Bronson) Date: Mon, 18 Jan 2010 14:12:20 -0800 (PST) Subject: heapq._siftdown / decrease-key support? References: <0ba8b786-2360-4a74-84c4-97e0564fbf3c@q16g2000vbc.googlegroups.com> Message-ID: <82e3cf4e-f58c-47d7-ad21-5d5a8dd2bafa@m25g2000yqc.googlegroups.com> On Sun, Jan 17, 2010 at 11:30 PM, Raymond Hettinger wrote: > > > Raymond, do you think this technique is worth documenting in the heapq > > module? It'd be too bad if any future users incorrectly think that it > > won't meet their needs the way I did. > > Yes. ?Please assign a tracker issue to me to expand the heapq documention > to discuss the mark-as-invalid trick for priority queues. > > > Raymond I've created http://bugs.python.org/issue7734. I don't enough have privileges to assign it to you, but I added you to the nosy list. Thanks! Josh From tjreedy at udel.edu Mon Jan 18 17:12:28 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 18 Jan 2010 17:12:28 -0500 Subject: r"string" vs R"string In-Reply-To: References: <00dd21f5$0$15570$c3e8da3@news.astraweb.com> Message-ID: On 1/18/2010 8:01 AM, Colin W. wrote: > On 17-Jan-10 18:27 PM, Steven D'Aprano wrote: >> On Sun, 17 Jan 2010 11:13:48 -0500, Roy Smith wrote: >> >>> In article, >>> "Colin W." wrote: >>> >>>> On 17-Jan-10 02:16 AM, Terry Reedy wrote: >>>>> On 1/17/2010 1:55 AM, Brendan Miller wrote: >>>>>> Is there any difference whatsoever between a raw string beginning >>>>>> with the captical R or one with the lower case r e.g. r"string" vs >>>>>> R"string"? >>>>> >>>>> No. Nor is there and difference between the strings created with raw >>>>> literals and cooked literals. >>>>> >>>>> >>>> "cooked" literal ?? >>> >>> I've never seen it referred to this way in the Python literature, but >>> "cooked" is a well-known term meaning, "not raw". The usage goes back >>> decades. >> >> I think the use of "cooked" meaning "not raw" goes back a little bit more >> than decades. The verb "to cook" dates from the late 14th century, and >> the adjective form "cooked" would follow soon after. The use of "cooked" >> to mean manipulated (as in "cooking the books") comes from the 1630s. >> Extending it to strings (as in raw versus cooked strings) is an obvious >> extension. >> > Yes, I should have cottoned on, but perhaps "... between the strings, > raw and other." would have conveyed the idea. I intentionally did not say that because Python only has one type of string object (in 3.x), not raw and normal. Python *code* has string literals optionally prefixed with 'r' (or 'R', which I did not know before) to suppress the normal processing that reduces escape sequences to one char. The term 'cooked' versus 'raw' was used in Unixland at least through the 1980s. For instance, the input stream from a terminal could be cooked or left raw. I presume that is whence came Python's use of 'raw', with much the same meaning. Terry Jan Reedy From gnarlodious at gmail.com Mon Jan 18 17:43:54 2010 From: gnarlodious at gmail.com (Gnarlodious) Date: Mon, 18 Jan 2010 14:43:54 -0800 (PST) Subject: Py 3: Terminal script can't find relative path Message-ID: <58e9b1e4-049a-4a7f-bea9-8d7e7e9f191a@e27g2000yqd.googlegroups.com> I am running a script in a browser that finds the file in subfolder Data: Content=Plist('Data/Content.plist') However, running the same script in Terminal errors: IOError: [Errno 2] No such file or directory: 'Data/Content.plist' Is Py 3 unable to find relative paths? I tried all kinds of tricks to no avail. What is wrong? -- Gnarlie From kaklis at gmail.com Mon Jan 18 17:46:26 2010 From: kaklis at gmail.com (kaklis at gmail.com) Date: Mon, 18 Jan 2010 14:46:26 -0800 (PST) Subject: Parse a log file References: <92293df0-7d2e-453a-b5bc-d4a7eb905fcf@l19g2000yqb.googlegroups.com> Message-ID: <9ca93ef0-396f-4f60-9fce-d1354c0d4c32@j19g2000yqk.googlegroups.com> On Jan 18, 11:56?pm, Tim Chase wrote: > kak... at gmail.com wrote: > > I want to parse a log file with the following format for > > example: > > ? ? ? ? ? ? ? TIMESTAMPE ? ? ? ? ? ?Operation ? ? FileName > > Bytes > > 12/Jan/2010:16:04:59 +0200 ? EXISTS ? ? ? sample3.3gp ? 37151 > > 12/Jan/2010:16:04:59 +0200 ?EXISTS ? ? ? ?sample3.3gp ? 37151 > > 12/Jan/2010:16:04:59 +0200 ?EXISTS ? ? ? ?sample3.3gp ? 37151 > > 12/Jan/2010:16:04:59 +0200 ?EXISTS ? ? ? ?sample3.3gp ? 37151 > > 12/Jan/2010:16:04:59 +0200 ?EXISTS ? ? ? ?sample3.3gp ? 37151 > > 12/Jan/2010:16:05:05 +0200 ?DELETE ? ? ?sample3.3gp ? 37151 > > > How can i count the operations for a month(e.g total of 40 Operations, > > 30 exists, 10 delete?) > > It can be done pretty easily with a regexp to parse the relevant > bits: > > ? ?import re > ? ?r = re.compile(r'\d+/([^/]+)/(\d+)\S+\s+\S+\s+(\w+)') > ? ?stats = {} > ? ?for line in file('log.txt'): > ? ? ?m = r.match(line) > ? ? ?if m: > ? ? ? ?stats[m.groups()] = stats.get(m.groups(), 0) + 1 > ? ?print stats > > This prints out > > ? ?{('Jan', '2010', 'EXISTS'): 5, ('Jan', '2010', 'DELETE'): 1} > > With the resulting data structure, you can manipulate it to do > coarser-grained aggregates such as the total operations, or remap > month-name abbreviations into integers so they could be sorted > for output. > > -tkc Thank you both so much Antonis From g.bogle at auckland.no.spam.ac.nz Mon Jan 18 18:15:37 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Tue, 19 Jan 2010 12:15:37 +1300 Subject: PyQwt installation Message-ID: Should there be a problem installing PyQwt5.2.0 with PyQt4.4.4 and Python2.6 on Windows? My student is saying she can't get the Windows installer to work, and she points out that the download site says "Binary installation of PyQwt-5.2.0 on Windows for Python-2.6.x, PyQt-4.5.4" From john at castleamber.com Mon Jan 18 18:21:48 2010 From: john at castleamber.com (John Bokma) Date: Mon, 18 Jan 2010 17:21:48 -0600 Subject: Py 3: Terminal script can't find relative path References: <58e9b1e4-049a-4a7f-bea9-8d7e7e9f191a@e27g2000yqd.googlegroups.com> Message-ID: <87d417578z.fsf@castleamber.com> Gnarlodious writes: > I am running a script in a browser that finds the file in subfolder > Data: > > Content=Plist('Data/Content.plist') > > However, running the same script in Terminal errors: > > IOError: [Errno 2] No such file or directory: 'Data/Content.plist' What does: ls -l Data/Content.plist in the terminal give? -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From python at mrabarnett.plus.com Mon Jan 18 18:34:22 2010 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 18 Jan 2010 23:34:22 +0000 Subject: searching and storing large quantities of xml! In-Reply-To: References: <4b52bdb8$0$6733$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <4B54EFFE.7070208@mrabarnett.plus.com> dads wrote: [snip] > import os.path > import shutil > import zlib > import os > There's no need to import both os.path and os. Import just os; you can still refer to os.path in the rest of the code. [snip] > > def _mkMonthAndDaysDirs(self): > > ''' creates dirs for every month and day of a of specidifed > year. > Works for leap years as well. > > (specified)year/(year)month/day > > > ...2010/201001/01 > ...2010/201001/02 > ...2010/201001/03 ''' > There's nothing except a docstring in this method! > > def addZero(n): > > if len(str(n)) < 2: > return '0' + str(n) > else: > return str(n) > A shorter and quicker way is: return "%02d" % n which means "pad with leading zeros to be at least 2 characters". [snip] > > # dir struct .../(year)/(year).zip - ex. .../2008/2008.zip > zf = zipfile.ZipFile(os.path.join(self.cwd, self.year + > '.zip')) > zf.extractall() You might want to close the zipfile after use. [snip] > > if __name__ == '__main__': > os.chdir('c:/sv_zip_test/2010/') #remove I recommend that you work with the full paths instead of changing the current directory and then using relative paths. > xo = Xmlorg() > xo.ParseAndOrg() From wkfung.eric at gmail.com Mon Jan 18 19:24:11 2010 From: wkfung.eric at gmail.com (Kit) Date: Mon, 18 Jan 2010 16:24:11 -0800 (PST) Subject: What is a list compression in Python? References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> <00de11a0$0$15570$c3e8da3@news.astraweb.com> Message-ID: Thank you so much guys. Just out of curiosity: can I do something like this to "square all even numbers in the range 1-10"? print [x^2 for x in range (1,11) if x % 2 == 0] Or is there a better way of doing it? Thanks for the help, and I am really appreciate your help. Kit. On 1?19?, ??12?30?, Steven D'Aprano wrote: > On Mon, 18 Jan 2010 08:07:41 -0800, Kit wrote: > > Hello Everyone, I am not sure if I have posted this question in a > > correct board. Can anyone please teach me: > > > What is a list compression in Python? > > Google "python list comprehension". > > If Google is broken for you, try Yahoo, or any other search engine. > > > Would you mind give me some list compression examples? > > Instead of this: > > L = [] > for x in range(10): > ? ? L.append(x**2) > > you can write: > > L = [x**2 for x in range(10)] > > Instead of this example: > > L = [] > for x in range(10): > ? ? if x % 2 == 0: > ? ? ? ? L.append(x**2) > > you can write: > > L = [x**2 for x in range(10) if x % 2 == 0] > > -- > Steven From wkfung.eric at gmail.com Mon Jan 18 19:26:31 2010 From: wkfung.eric at gmail.com (Kit) Date: Mon, 18 Jan 2010 16:26:31 -0800 (PST) Subject: What is a list compression in Python? References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> <00de11a0$0$15570$c3e8da3@news.astraweb.com> Message-ID: <21602b5a-0d88-46b3-9cb9-bcfcd5c6707c@b10g2000yqa.googlegroups.com> Oops... > print [x^2 for x in range (1,11) if x % 2 == 0] print [x^2 for x in range (1,10) if x % 2 == 0] On 1?19?, ??8?24?, Kit wrote: > Thank you so much guys. > > Just out of curiosity: can I do something like this to "square all > even numbers in the range 1-10"? > print [x^2 for x in range (1,11) if x % 2 == 0] > > Or is there a better way of doing it? Thanks for the help, and I am > really appreciate your help. > > Kit. > > On 1?19?, ??12?30?, Steven D'Aprano > > > cybersource.com.au> wrote: > > On Mon, 18 Jan 2010 08:07:41 -0800, Kit wrote: > > > Hello Everyone, I am not sure if I have posted this question in a > > > correct board. Can anyone please teach me: > > > > What is a list compression in Python? > > > Google "python list comprehension". > > > If Google is broken for you, try Yahoo, or any other search engine. > > > > Would you mind give me some list compression examples? > > > Instead of this: > > > L = [] > > for x in range(10): > > ? ? L.append(x**2) > > > you can write: > > > L = [x**2 for x in range(10)] > > > Instead of this example: > > > L = [] > > for x in range(10): > > ? ? if x % 2 == 0: > > ? ? ? ? L.append(x**2) > > > you can write: > > > L = [x**2 for x in range(10) if x % 2 == 0] > > > -- > > Steven From python at mrabarnett.plus.com Mon Jan 18 19:35:04 2010 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 19 Jan 2010 00:35:04 +0000 Subject: What is a list compression in Python? In-Reply-To: References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> <00de11a0$0$15570$c3e8da3@news.astraweb.com> Message-ID: <4B54FE38.1010704@mrabarnett.plus.com> Kit wrote: > Thank you so much guys. > > Just out of curiosity: can I do something like this to "square all > even numbers in the range 1-10"? > print [x^2 for x in range (1,11) if x % 2 == 0] > > Or is there a better way of doing it? Thanks for the help, and I am > really appreciate your help. > That would be: print [x ** 2 for x in range(1, 11) if x % 2 == 0] Note that Python uses "**" for raising to a power and "^" (like in C) for bitwise exclusive-or. From gherron at islandtraining.com Mon Jan 18 19:36:45 2010 From: gherron at islandtraining.com (Gary Herron) Date: Mon, 18 Jan 2010 16:36:45 -0800 Subject: What is a list compression in Python? In-Reply-To: References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> <00de11a0$0$15570$c3e8da3@news.astraweb.com> Message-ID: <4B54FE9D.8070601@islandtraining.com> Kit wrote: > Thank you so much guys. > > Just out of curiosity: can I do something like this to "square all > even numbers in the range 1-10"? > print [x^2 for x in range (1,11) if x % 2 == 0] > > Or is there a better way of doing it? Thanks for the help, and I am > really appreciate your help. > That's a fine way to do it, although: xrange is slightly more efficient than range for large sets of values. (This isn't such a large set for it to matter.) You can get range and xrange to count by two's, eliminating the need for the "if" portion in this case. The exponent operator is ** not ^, and x*x is more efficient than x**2. So. print [x**2 for x in xrange(1,11,2)] Gary Herron > Kit. > > On 1?19?, ??12?30?, Steven D'Aprano cybersource.com.au> wrote: > >> On Mon, 18 Jan 2010 08:07:41 -0800, Kit wrote: >> >>> Hello Everyone, I am not sure if I have posted this question in a >>> correct board. Can anyone please teach me: >>> >>> What is a list compression in Python? >>> >> Google "python list comprehension". >> >> If Google is broken for you, try Yahoo, or any other search engine. >> >> >>> Would you mind give me some list compression examples? >>> >> Instead of this: >> >> L = [] >> for x in range(10): >> L.append(x**2) >> >> you can write: >> >> L = [x**2 for x in range(10)] >> >> Instead of this example: >> >> L = [] >> for x in range(10): >> if x % 2 == 0: >> L.append(x**2) >> >> you can write: >> >> L = [x**2 for x in range(10) if x % 2 == 0] >> >> -- >> Steven >> > > From wkfung.eric at gmail.com Mon Jan 18 19:41:49 2010 From: wkfung.eric at gmail.com (Kit) Date: Mon, 18 Jan 2010 16:41:49 -0800 (PST) Subject: What is a list compression in Python? References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> <00de11a0$0$15570$c3e8da3@news.astraweb.com> Message-ID: Cool! Thank you very much. You mean this instead right? print [x*x for x in xrange(1,11,2)] Kit On 1?19?, ??8?36?, Gary Herron wrote: > Kit wrote: > > Thank you so much guys. > > > Just out of curiosity: can I do something like this to "square all > > even numbers in the range 1-10"? > > print [x^2 for x in range (1,11) if x % 2 == 0] > > > Or is there a better way of doing it? Thanks for the help, and I am > > really appreciate your help. > > That's a fine way to do it, although: > > ? xrange is slightly more efficient than range for large > ? sets of values. ?(This isn't such a large set for it to matter.) > > ? You can get range and xrange to count by two's, eliminating > ? the need for the "if" portion in this case. > > ? The exponent operator is ** not ^, and ?x*x is more efficient than x**2. > > So. > > print [x**2 for x in xrange(1,11,2)] > > Gary Herron > > > > > Kit. > > > On 1?19?, ??12?30?, Steven D'Aprano > cybersource.com.au> wrote: > > >> On Mon, 18 Jan 2010 08:07:41 -0800, Kit wrote: > > >>> Hello Everyone, I am not sure if I have posted this question in a > >>> correct board. Can anyone please teach me: > > >>> What is a list compression in Python? > > >> Google "python list comprehension". > > >> If Google is broken for you, try Yahoo, or any other search engine. > > >>> Would you mind give me some list compression examples? > > >> Instead of this: > > >> L = [] > >> for x in range(10): > >> ? ? L.append(x**2) > > >> you can write: > > >> L = [x**2 for x in range(10)] > > >> Instead of this example: > > >> L = [] > >> for x in range(10): > >> ? ? if x % 2 == 0: > >> ? ? ? ? L.append(x**2) > > >> you can write: > > >> L = [x**2 for x in range(10) if x % 2 == 0] > > >> -- > >> Steven From python at mrabarnett.plus.com Mon Jan 18 19:50:42 2010 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 19 Jan 2010 00:50:42 +0000 Subject: What is a list compression in Python? In-Reply-To: <21602b5a-0d88-46b3-9cb9-bcfcd5c6707c@b10g2000yqa.googlegroups.com> References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> <00de11a0$0$15570$c3e8da3@news.astraweb.com> <21602b5a-0d88-46b3-9cb9-bcfcd5c6707c@b10g2000yqa.googlegroups.com> Message-ID: <4B5501E2.5070103@mrabarnett.plus.com> Kit wrote: > Oops... > >> print [x^2 for x in range (1,11) if x % 2 == 0] > print [x^2 for x in range (1,10) if x % 2 == 0] > The start is inclusive and the end is exclusive. This is a general rule in Python. In fact, there's only one exception that I know of: randint(a, b) in the 'random' module. From marlowequart at hotmail.com Mon Jan 18 19:53:51 2010 From: marlowequart at hotmail.com (marlowe) Date: Mon, 18 Jan 2010 16:53:51 -0800 (PST) Subject: syntax Message-ID: <3ab2b14d-3248-4570-b332-b64ac4b37758@j14g2000yqm.googlegroups.com> I wrote this program, but i have a feeling like there might be a more practical way of writing it. Can someone give me an idea of how to simplify this? Here is an example of the csv file i am using. This program calculates the exponential moving average of the 20 day range. USOtable.csv (full table found at http://ichart.finance.yahoo.com/table.csv?s=USO) Date,Open,High,Low,Close,Volume,Adj Close 2010-01-15,38.97,39.02,38.28,38.40,12615300,38.40 2010-01-14,39.30,39.44,38.88,39.06,8575900,39.06 2010-01-13,39.40,39.71,38.63,39.21,15502700,39.21 2010-01-12,40.07,40.36,39.53,39.63,11960100,39.63 2010-01-11,41.09,41.19,40.46,40.54,8902200,40.54 2010-01-08,40.63,41.17,40.45,40.93,9393500,40.93 2010-01-07,40.87,41.08,40.68,40.72,10012000,40.72 2010-01-06,40.32,41.19,39.89,40.97,19789800,40.97 2010-01-05,40.25,40.45,39.93,40.41,10450200,40.41 test.py import csv reader = open('/prog/USOtable.csv','rb') data = [row for row in csv.reader(reader)] Nvals= [1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1] H=float(data[40][2])+float(data[41][2])+float(data[42][2])+float(data [43][2])\ +float(data[44][2])+float(data[45][2])+float(data[46][2])+float(data [47][2])\ +float(data[48][2])+float(data[49][2])+float(data[50][2])+float(data [51][2])\ +float(data[52][2])+float(data[53][2])+float(data[54][2])+float(data [55][2])\ +float(data[56][2])+float(data[57][2])+float(data[58][2])+float(data [59][2])\ +float(data[60][2]) L=float(data[40][3])+float(data[41][3])+float(data[42][3])+float(data [43][3])\ +float(data[44][3])+float(data[45][3])+float(data[46][3])+float(data [47][3])\ +float(data[48][3])+float(data[49][3])+float(data[50][3])+float(data [51][3])\ +float(data[52][3])+float(data[53][3])+float(data[54][3])+float(data [55][3])\ +float(data[56][3])+float(data[57][3])+float(data[58][3])+float(data [59][3])\ +float(data[60][3]) Nvals[39]=(H-L)/2 i=1 while i <=38: high=float(data[39-i][2]) low=float(data[39-i][3]) TR=high-low Nvals[39-i]=(19*Nvals[40-i]+TR)/20 i+=1 for value in Nvals: print value From gagsl-py2 at yahoo.com.ar Mon Jan 18 19:57:15 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 18 Jan 2010 21:57:15 -0300 Subject: using super References: <4B54BB95.1090407@sequans.com> Message-ID: En Mon, 18 Jan 2010 16:50:45 -0300, Jean-Michel Pichavant escribi?: > Duncan Booth wrote: >> Jean-Michel Pichavant wrote: >> >> >>>>>> class SubClass(Base): >>>>>> colour = "Red" >>>>>> def parrot(self): >>>>>> """docstring for Subclass""" >>>>>> return super(Subclass, self).parrot() >>>>>> >>> I'm not a big fan of super, but I'm still wondering if >>> return super(self.__class__, self).parrot() >>> would have made it. >> >> No, it wouldn't. [...] >> > I see. > Then is there a reason why > return super(Subclass, self).parrot() > would be prefered over the "classic" > return Base.parrot(self) > ? > Or is it just a matter of preference ? For a longer explanation, see: James Knight: Python's Super Considered Harmful http://fuhm.net/super-harmful/ Michele Simionato: Things to Know About Python Super http://www.artima.com/weblogs/viewpost.jsp?thread=236275 http://www.artima.com/weblogs/viewpost.jsp?thread=236278 http://www.artima.com/weblogs/viewpost.jsp?thread=237121 -- Gabriel Genellina From clp2 at rebertia.com Mon Jan 18 20:23:34 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Mon, 18 Jan 2010 17:23:34 -0800 Subject: syntax In-Reply-To: <3ab2b14d-3248-4570-b332-b64ac4b37758@j14g2000yqm.googlegroups.com> References: <3ab2b14d-3248-4570-b332-b64ac4b37758@j14g2000yqm.googlegroups.com> Message-ID: <50697b2c1001181723t2fe41ca5xeb19ea048db00706@mail.gmail.com> On Mon, Jan 18, 2010 at 4:53 PM, marlowe wrote: > I wrote this program, but i have a feeling like there might be a more > practical way of writing it. Can someone give me an idea of how to > simplify this? Here is an example of the csv file i am using. This > program calculates the exponential moving average of the 20 day range. > H=float(data[40][2])+float(data[41][2])+float(data[42][2])+float(data > [43][2])\ > +float(data[44][2])+float(data[45][2])+float(data[46][2])+float(data > [47][2])\ > +float(data[48][2])+float(data[49][2])+float(data[50][2])+float(data > [51][2])\ > +float(data[52][2])+float(data[53][2])+float(data[54][2])+float(data > [55][2])\ > +float(data[56][2])+float(data[57][2])+float(data[58][2])+float(data > [59][2])\ > +float(data[60][2]) H = sum(float(data[i][2]) for i in xrange(40,61)) > L=float(data[40][3])+float(data[41][3])+float(data[42][3])+float(data > [43][3])\ > +float(data[44][3])+float(data[45][3])+float(data[46][3])+float(data > [47][3])\ > +float(data[48][3])+float(data[49][3])+float(data[50][3])+float(data > [51][3])\ > +float(data[52][3])+float(data[53][3])+float(data[54][3])+float(data > [55][3])\ > +float(data[56][3])+float(data[57][3])+float(data[58][3])+float(data > [59][3])\ > +float(data[60][3]) H = sum(float(data[i][3]) for i in xrange(40,61)) Might I suggest you learn more about comprehensions and looping?: http://docs.python.org/tutorial/datastructures.html#list-comprehensions http://www.python.org/dev/peps/pep-0289/ http://docs.python.org/library/functions.html#range http://docs.python.org/library/functions.html#sum Cheers, Chris -- http://blog.rebertia.com From clp2 at rebertia.com Mon Jan 18 20:24:59 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Mon, 18 Jan 2010 17:24:59 -0800 Subject: syntax In-Reply-To: <50697b2c1001181723t2fe41ca5xeb19ea048db00706@mail.gmail.com> References: <3ab2b14d-3248-4570-b332-b64ac4b37758@j14g2000yqm.googlegroups.com> <50697b2c1001181723t2fe41ca5xeb19ea048db00706@mail.gmail.com> Message-ID: <50697b2c1001181724ve0e2e52o51e27cde0061e22e@mail.gmail.com> On Mon, Jan 18, 2010 at 5:23 PM, Chris Rebert wrote: > On Mon, Jan 18, 2010 at 4:53 PM, marlowe wrote: >> L=float(data[40][3])+float(data[41][3])+float(data[42][3])+float(data >> [43][3])\ >> +float(data[44][3])+float(data[45][3])+float(data[46][3])+float(data >> [47][3])\ >> +float(data[48][3])+float(data[49][3])+float(data[50][3])+float(data >> [51][3])\ >> +float(data[52][3])+float(data[53][3])+float(data[54][3])+float(data >> [55][3])\ >> +float(data[56][3])+float(data[57][3])+float(data[58][3])+float(data >> [59][3])\ >> +float(data[60][3]) > > H = sum(float(data[i][3]) for i in xrange(40,61)) Erm, L for that second one obviously, not H: L = sum(float(data[i][3]) for i in xrange(40,61)) - Chris From mrabarnett at mrabarnett.plus.com Mon Jan 18 20:31:47 2010 From: mrabarnett at mrabarnett.plus.com (Matthew Barnett) Date: Tue, 19 Jan 2010 01:31:47 +0000 Subject: syntax In-Reply-To: <3ab2b14d-3248-4570-b332-b64ac4b37758@j14g2000yqm.googlegroups.com> References: <3ab2b14d-3248-4570-b332-b64ac4b37758@j14g2000yqm.googlegroups.com> Message-ID: <4B550B83.8090609@mrabarnett.plus.com> marlowe wrote: > I wrote this program, but i have a feeling like there might be a more > practical way of writing it. Can someone give me an idea of how to > simplify this? Here is an example of the csv file i am using. This > program calculates the exponential moving average of the 20 day range. > > USOtable.csv (full table found at http://ichart.finance.yahoo.com/table.csv?s=USO) > > Date,Open,High,Low,Close,Volume,Adj Close > 2010-01-15,38.97,39.02,38.28,38.40,12615300,38.40 > 2010-01-14,39.30,39.44,38.88,39.06,8575900,39.06 > 2010-01-13,39.40,39.71,38.63,39.21,15502700,39.21 > 2010-01-12,40.07,40.36,39.53,39.63,11960100,39.63 > 2010-01-11,41.09,41.19,40.46,40.54,8902200,40.54 > 2010-01-08,40.63,41.17,40.45,40.93,9393500,40.93 > 2010-01-07,40.87,41.08,40.68,40.72,10012000,40.72 > 2010-01-06,40.32,41.19,39.89,40.97,19789800,40.97 > 2010-01-05,40.25,40.45,39.93,40.41,10450200,40.41 > > > test.py > > import csv > > > reader = open('/prog/USOtable.csv','rb') > data = [row for row in csv.reader(reader)] > data = list(csv.reader(reader)) > Nvals= > [1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1] > Nvals = [1] * 40 > > H=float(data[40][2])+float(data[41][2])+float(data[42][2])+float(data > [43][2])\ > +float(data[44][2])+float(data[45][2])+float(data[46][2])+float(data > [47][2])\ > +float(data[48][2])+float(data[49][2])+float(data[50][2])+float(data > [51][2])\ > +float(data[52][2])+float(data[53][2])+float(data[54][2])+float(data > [55][2])\ > +float(data[56][2])+float(data[57][2])+float(data[58][2])+float(data > [59][2])\ > +float(data[60][2]) > H = sum(float(d[2]) for d in data[40 : 61]) > L=float(data[40][3])+float(data[41][3])+float(data[42][3])+float(data > [43][3])\ > +float(data[44][3])+float(data[45][3])+float(data[46][3])+float(data > [47][3])\ > +float(data[48][3])+float(data[49][3])+float(data[50][3])+float(data > [51][3])\ > +float(data[52][3])+float(data[53][3])+float(data[54][3])+float(data > [55][3])\ > +float(data[56][3])+float(data[57][3])+float(data[58][3])+float(data > [59][3])\ > +float(data[60][3]) > L = sum(float(d[3]) for d in data[40 : 61]) > > Nvals[39]=(H-L)/2 > > > i=1 > while i <=38: > high=float(data[39-i][2]) > low=float(data[39-i][3]) > TR=high-low > Nvals[39-i]=(19*Nvals[40-i]+TR)/20 > i+=1 > Instead of: i=1 while i <=38: ... i+=1 you could use: for i in range(1, 39): ... Note that the start value is inclusive and the end value is exclusive. > for value in Nvals: > print value From python at mrabarnett.plus.com Mon Jan 18 20:32:06 2010 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 19 Jan 2010 01:32:06 +0000 Subject: syntax In-Reply-To: <3ab2b14d-3248-4570-b332-b64ac4b37758@j14g2000yqm.googlegroups.com> References: <3ab2b14d-3248-4570-b332-b64ac4b37758@j14g2000yqm.googlegroups.com> Message-ID: <4B550B96.80303@mrabarnett.plus.com> marlowe wrote: > I wrote this program, but i have a feeling like there might be a more > practical way of writing it. Can someone give me an idea of how to > simplify this? Here is an example of the csv file i am using. This > program calculates the exponential moving average of the 20 day range. > > USOtable.csv (full table found at http://ichart.finance.yahoo.com/table.csv?s=USO) > > Date,Open,High,Low,Close,Volume,Adj Close > 2010-01-15,38.97,39.02,38.28,38.40,12615300,38.40 > 2010-01-14,39.30,39.44,38.88,39.06,8575900,39.06 > 2010-01-13,39.40,39.71,38.63,39.21,15502700,39.21 > 2010-01-12,40.07,40.36,39.53,39.63,11960100,39.63 > 2010-01-11,41.09,41.19,40.46,40.54,8902200,40.54 > 2010-01-08,40.63,41.17,40.45,40.93,9393500,40.93 > 2010-01-07,40.87,41.08,40.68,40.72,10012000,40.72 > 2010-01-06,40.32,41.19,39.89,40.97,19789800,40.97 > 2010-01-05,40.25,40.45,39.93,40.41,10450200,40.41 > > > test.py > > import csv > > > reader = open('/prog/USOtable.csv','rb') > data = [row for row in csv.reader(reader)] > data = list(csv.reader(reader)) > Nvals= > [1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1] > Nvals = [1] * 40 > > H=float(data[40][2])+float(data[41][2])+float(data[42][2])+float(data > [43][2])\ > +float(data[44][2])+float(data[45][2])+float(data[46][2])+float(data > [47][2])\ > +float(data[48][2])+float(data[49][2])+float(data[50][2])+float(data > [51][2])\ > +float(data[52][2])+float(data[53][2])+float(data[54][2])+float(data > [55][2])\ > +float(data[56][2])+float(data[57][2])+float(data[58][2])+float(data > [59][2])\ > +float(data[60][2]) > H = sum(float(d[2]) for d in data[40 : 61]) > L=float(data[40][3])+float(data[41][3])+float(data[42][3])+float(data > [43][3])\ > +float(data[44][3])+float(data[45][3])+float(data[46][3])+float(data > [47][3])\ > +float(data[48][3])+float(data[49][3])+float(data[50][3])+float(data > [51][3])\ > +float(data[52][3])+float(data[53][3])+float(data[54][3])+float(data > [55][3])\ > +float(data[56][3])+float(data[57][3])+float(data[58][3])+float(data > [59][3])\ > +float(data[60][3]) > L = sum(float(d[3]) for d in data[40 : 61]) > > Nvals[39]=(H-L)/2 > > > i=1 > while i <=38: > high=float(data[39-i][2]) > low=float(data[39-i][3]) > TR=high-low > Nvals[39-i]=(19*Nvals[40-i]+TR)/20 > i+=1 > Instead of: i=1 while i <=38: ... i+=1 you could use: for i in range(1, 39): ... Note that the start value is inclusive and the end value is exclusive. > for value in Nvals: > print value From marlowequart at hotmail.com Mon Jan 18 20:51:08 2010 From: marlowequart at hotmail.com (marlowe) Date: Mon, 18 Jan 2010 17:51:08 -0800 (PST) Subject: syntax References: <3ab2b14d-3248-4570-b332-b64ac4b37758@j14g2000yqm.googlegroups.com> Message-ID: oh, and i need to make those a new column that is added to the csv file. Thanks From gagsl-py2 at yahoo.com.ar Mon Jan 18 21:07:11 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 18 Jan 2010 23:07:11 -0300 Subject: inner methods and recursion References: Message-ID: En Mon, 18 Jan 2010 19:00:17 -0300, Steve Howell escribi?: > Hi, I have a style/design question relating to recursion and inner > methods. > > I wrote some code recently that implements a recursive algorithm that > takes several parameters from the original caller. Once the algorithm > starts running and recursing, those parameters remain the same, so I > define an inner method called "recurse" that essentially curries those > parameters. In particular, the recurse method can get passed to a > callback method that the caller supplies, so that the caller can wrap > the recursive step with their own logic. Python already have lexical scoping, you can take advantage of it. On any non-prehistoric version of Python you may write: def indent_lines(lines, branch_method, leaf_method, pass_syntax, flush_left_syntax, flush_left_empty_line, indentation_method, get_block, ): def _indent_lines(lines): output = [] while lines: # ... the real work ... # recursive call: output += branch_method(prefix, block, _indent_lines) return output return _indent_lines(lines) The real work happens inside _indent_lines, and it has access to the outer indent_lines scope, where all remaining parameters are defined. > The only thing I don't like about the technique that I'm using is that > it seems needlessly repetitive to define mostly the same parameter > list in two different places. The repetition seems mostly harmless, > but it seems like some kind of a smell that I'm overlooking a simpler > way to write the code. I just can't put my finger on what it is. Is the above technique what you were looking for? > Does anybody have any inspiration here? I vaguely recall a discussion > some time back about having some kind of @recursive decorator, but I > can't seem to find the thread. The thread I remember was about making "true" recursive calls (normal recursive calls at global or method scope are not truly recursive: they actually perform a name lookup, which may or may not yield the original function). In the above case, that doesn't happen, the recursive call resolves the _indent_lines name directly using a cell "pointing" into its container local scope. -- Gabriel Genellina From ndbecker2 at gmail.com Mon Jan 18 21:18:26 2010 From: ndbecker2 at gmail.com (Neal Becker) Date: Mon, 18 Jan 2010 21:18:26 -0500 Subject: multiprocessing question Message-ID: I'm using multiprocessing as a crude batch queuing system, like this: import my_test_program as prog (where my_test_program has a function called 'run') def run_test (args): prog.run (args[1:]) cases = [] for t in test_conditions: args = [prog.__name__]+[more args...] cases.append (args) (leaving out details, but 'cases' will be the list of test cases to run) results = pool.map (run_test, cases) Problem is, it doesn't seem to keep all my cpus busy, even though there are more test cases than cpus. Ideas? From g.bogle at auckland.no.spam.ac.nz Mon Jan 18 21:44:53 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Tue, 19 Jan 2010 15:44:53 +1300 Subject: syntax References: <3ab2b14d-3248-4570-b332-b64ac4b37758@j14g2000yqm.googlegroups.com> Message-ID: Looks like homework. From g.bogle at auckland.no.spam.ac.nz Mon Jan 18 21:48:59 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Tue, 19 Jan 2010 15:48:59 +1300 Subject: What is a list compression in Python? References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> <00de11a0$0$15570$c3e8da3@news.astraweb.com> Message-ID: Kit wrote: > Thank you so much guys. > > Just out of curiosity: can I do something like this to "square all > even numbers in the range 1-10"? > print [x^2 for x in range (1,11) if x % 2 == 0] Why not try it? From python at mrabarnett.plus.com Mon Jan 18 21:49:36 2010 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 19 Jan 2010 02:49:36 +0000 Subject: multiprocessing question In-Reply-To: References: Message-ID: <4B551DC0.2030407@mrabarnett.plus.com> Neal Becker wrote: > I'm using multiprocessing as a crude batch queuing system, like this: > > import my_test_program as prog > (where my_test_program has a function called 'run') > > def run_test (args): > prog.run (args[1:]) > > cases = [] > for t in test_conditions: > args = [prog.__name__]+[more args...] > > cases.append (args) > > (leaving out details, but 'cases' will be the list of test cases to run) > > results = pool.map (run_test, cases) > > Problem is, it doesn't seem to keep all my cpus busy, even though there are > more test cases than cpus. Ideas? > If they do a lot of I/O then that could be the bottleneck. From showell30 at yahoo.com Mon Jan 18 21:55:58 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 18 Jan 2010 18:55:58 -0800 (PST) Subject: inner methods and recursion References: Message-ID: <4d3231c3-a5c7-41b8-b2b3-ffad9201487e@l19g2000yqb.googlegroups.com> On Jan 18, 6:07?pm, "Gabriel Genellina" wrote: > En Mon, 18 Jan 2010 19:00:17 -0300, Steve Howell ? > escribi?: > > > Hi, I have a style/design question relating to recursion and inner > > methods. > > > I wrote some code recently that implements a recursive algorithm that > > takes several parameters from the original caller. ?Once the algorithm > > starts running and recursing, those parameters remain the same, so I > > define an inner method called "recurse" that essentially curries those > > parameters. ?In particular, the recurse method can get passed to a > > callback method that the caller supplies, so that the caller can wrap > > the recursive step with their own logic. > > Python already have lexical scoping, you can take advantage of it. On any ? > non-prehistoric version of Python you may write: > > def indent_lines(lines, > ? ? ? ? ? ? ?branch_method, > ? ? ? ? ? ? ?leaf_method, > ? ? ? ? ? ? ?pass_syntax, > ? ? ? ? ? ? ?flush_left_syntax, > ? ? ? ? ? ? ?flush_left_empty_line, > ? ? ? ? ? ? ?indentation_method, > ? ? ? ? ? ? ?get_block, > ? ? ? ? ? ? ?): > > ? ? ?def _indent_lines(lines): > ? ? ? ?output = [] > ? ? ? ?while lines: > ? ? ? ? ?# ... the real work ... > ? ? ? ? ?# recursive call: > ? ? ? ? ?output += branch_method(prefix, block, _indent_lines) > ? ? ? ?return output > > ? ? ?return _indent_lines(lines) > > The real work happens inside _indent_lines, and it has access to the outer ? > indent_lines scope, where all remaining parameters are defined. > > > The only thing I don't like about the technique that I'm using is that > > it seems needlessly repetitive to define mostly the same parameter > > list in two different places. ?The repetition seems mostly harmless, > > but it seems like some kind of a smell that I'm overlooking a simpler > > way to write the code. ?I just can't put my finger on what it is. > > Is the above technique what you were looking for? > That was exactly what I was looking for. I just tried it out, and it works perfectly. Thanks! From sccolbert at gmail.com Mon Jan 18 22:31:49 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Mon, 18 Jan 2010 22:31:49 -0500 Subject: PyQwt installation In-Reply-To: References: Message-ID: <7f014ea61001181931t62da8633ucb19afdef2e5769c@mail.gmail.com> On Mon, Jan 18, 2010 at 6:15 PM, Gib Bogle wrote: > Should there be a problem installing PyQwt5.2.0 with PyQt4.4.4 and > Python2.6 on Windows? My student is saying she can't get the Windows > installer to work, and she points out that the download site says "Binary > installation of PyQwt-5.2.0 on Windows for Python-2.6.x, PyQt-4.5.4" > -- > http://mail.python.org/mailman/listinfo/python-list > It's a sad day when the professor can't google. from http://pyqwt.sourceforge.net/doc5/installation.html#windows-binary-installer Windows Binary Installer Make sure that you have installed: python-2.6.2.msi numpy-1.3.0-win32-superpack-python2.6.exe PyQt-Py2.6-gpl-4.5.4-1.exe before installing PyQwt5.2.0-Python2.6-PyQt4.5.4-NumPy1.3.0-1.exe. So I would say, yes, they are probably not compatible. The PyQt4.5 brought many changes from 4.4. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven at REMOVE.THIS.cybersource.com.au Mon Jan 18 22:49:37 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 19 Jan 2010 03:49:37 GMT Subject: What is a list compression in Python? References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> <00de11a0$0$15570$c3e8da3@news.astraweb.com> Message-ID: On Mon, 18 Jan 2010 16:24:11 -0800, Kit wrote: > Thank you so much guys. > > Just out of curiosity: can I do something like this to "square all even > numbers in the range 1-10"? > print [x^2 for x in range (1,11) if x % 2 == 0] ^ is the XOR operator in Python. You want: [x**2 for x in range (1,11) if x % 2 == 0] or even better: [x**2 for x in range (2, 11, 2)] -- Steven From badouglas at gmail.com Mon Jan 18 23:04:57 2010 From: badouglas at gmail.com (tom) Date: Mon, 18 Jan 2010 20:04:57 -0800 (PST) Subject: python replace/sub/wildcard/regex issue Message-ID: <9d80d7f8-d78f-4b0b-9d8b-afae52633c8a@b10g2000yqa.googlegroups.com> hi... trying to figure out how to solve what should be an easy python/regex/ wildcard/replace issue. i've tried a number of different approaches.. so i must be missing something... my initial sample text are: Soo ChoiLONGEDITBOX">Apryl Berney Soo ChoiLONGEDITBOX">Joel Franks Joel FranksGEDITBOX">Alexander Yamato and i'm trying to get Soo Choi foo Apryl Berney Soo Choi foo Joel Franks Joel Franks foo Alexander Yamato the issue i'm facing.. is how to start at "' and substitute inclusive of the stuff inside the regex... i've tried derivations of name=re.sub("]*\">"," foo ",name) but i'm missing something... thoughts... thanks tom From badouglas at gmail.com Mon Jan 18 23:04:57 2010 From: badouglas at gmail.com (tom) Date: Mon, 18 Jan 2010 20:04:57 -0800 (PST) Subject: python replace/sub/wildcard/regex issue Message-ID: <9d80d7f8-d78f-4b0b-9d8b-afae52633c8a@b10g2000yqa.googlegroups.com> hi... trying to figure out how to solve what should be an easy python/regex/ wildcard/replace issue. i've tried a number of different approaches.. so i must be missing something... my initial sample text are: Soo ChoiLONGEDITBOX">Apryl Berney Soo ChoiLONGEDITBOX">Joel Franks Joel FranksGEDITBOX">Alexander Yamato and i'm trying to get Soo Choi foo Apryl Berney Soo Choi foo Joel Franks Joel Franks foo Alexander Yamato the issue i'm facing.. is how to start at "' and substitute inclusive of the stuff inside the regex... i've tried derivations of name=re.sub("]*\">"," foo ",name) but i'm missing something... thoughts... thanks tom From clp2 at rebertia.com Mon Jan 18 23:31:55 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Mon, 18 Jan 2010 20:31:55 -0800 Subject: python replace/sub/wildcard/regex issue In-Reply-To: <9d80d7f8-d78f-4b0b-9d8b-afae52633c8a@b10g2000yqa.googlegroups.com> References: <9d80d7f8-d78f-4b0b-9d8b-afae52633c8a@b10g2000yqa.googlegroups.com> Message-ID: <50697b2c1001182031v893cfecyfa7dc540cabb3a@mail.gmail.com> On Mon, Jan 18, 2010 at 8:04 PM, tom wrote: > hi... > > trying to figure out how to solve what should be an easy python/regex/ > wildcard/replace issue. > > i've tried a number of different approaches.. so i must be missing > something... > > my initial sample text are: > > Soo ChoiLONGEDITBOX">Apryl Berney > Soo ChoiLONGEDITBOX">Joel Franks > Joel FranksGEDITBOX">Alexander Yamato > > and i'm trying to get > > Soo Choi foo Apryl Berney > Soo Choi foo Joel Franks > Joel Franks foo Alexander Yamato > > the issue i'm facing.. is how to start at "' and > substitute inclusive of the stuff inside the regex... > > i've tried derivations of > > name=re.sub("]*\">"," foo ",name) > > but i'm missing something... > > thoughts... thanks "Some people, when confronted with a problem, think 'I know, I'll use regular expressions.' Now they have two problems." Assuming your sample text is representative of all your test: new_text = "\n".join(line[:line.index('<')] + line[line.rindex('>')+1:] for line in your_text.split('\n')) Cheers, Chris -- http://blog.rebertia.com From wuwei23 at gmail.com Mon Jan 18 23:33:53 2010 From: wuwei23 at gmail.com (alex23) Date: Mon, 18 Jan 2010 20:33:53 -0800 (PST) Subject: python replace/sub/wildcard/regex issue References: <9d80d7f8-d78f-4b0b-9d8b-afae52633c8a@b10g2000yqa.googlegroups.com> Message-ID: On Jan 19, 2:04?pm, tom wrote: > trying to figure out how to solve what should be an easy python/regex/ > wildcard/replace issue. > but i'm missing something... Well, some would say you've missed the most obvious solution of _not_ using regexps :) I'd probably do it via string methods wrapped up in a helper function: >>> def extract(text): ... first, rest = text.split('<', 1) ... ignore, last = rest.rsplit('>', 1) ... return '%s foo %s' % (first, last) ... >>> extract('Soo ChoiLONGEDITBOX">Apryl Berney') 'Soo Choi foo Apryl Berney' >>> extract('Soo ChoiLONGEDITBOX">Joel Franks') 'Soo Choi foo Joel Franks' >>> extract('Joel FranksGEDITBOX">Alexander Yamato') 'Joel Franks foo Alexander Yamato' From bedouglas at earthlink.net Mon Jan 18 23:38:18 2010 From: bedouglas at earthlink.net (tom) Date: Mon, 18 Jan 2010 20:38:18 -0800 Subject: python replace/sub/wildcard/regex issue In-Reply-To: <50697b2c1001182031v893cfecyfa7dc540cabb3a@mail.gmail.com> Message-ID: <19b501ca98c1$39ae6140$0301a8c0@tmesa.com> hi chris... should the reply you sent me be implemented all on one line? like: test='Soo ChoiLONGEDITBOX">Apryl Berney' new_text = "\n".join(line[:line.index('<')] + line[line.rindex('>')+1:] for line in test.split('\n')) and what would line be? thanks -----Original Message----- From: python-list-bounces+bedouglas=earthlink.net at python.org [mailto:python-list-bounces+bedouglas=earthlink.net at python.org]On Behalf Of Chris Rebert Sent: Monday, January 18, 2010 8:32 PM To: tom Cc: python-list at python.org Subject: Re: python replace/sub/wildcard/regex issue On Mon, Jan 18, 2010 at 8:04 PM, tom wrote: > hi... > > trying to figure out how to solve what should be an easy python/regex/ > wildcard/replace issue. > > i've tried a number of different approaches.. so i must be missing > something... > > my initial sample text are: > > Soo ChoiLONGEDITBOX">Apryl Berney > Soo ChoiLONGEDITBOX">Joel Franks > Joel FranksGEDITBOX">Alexander Yamato > > and i'm trying to get > > Soo Choi foo Apryl Berney > Soo Choi foo Joel Franks > Joel Franks foo Alexander Yamato > > the issue i'm facing.. is how to start at "' and > substitute inclusive of the stuff inside the regex... > > i've tried derivations of > > name=re.sub("]*\">"," foo ",name) > > but i'm missing something... > > thoughts... thanks "Some people, when confronted with a problem, think 'I know, I'll use regular expressions.' Now they have two problems." Assuming your sample text is representative of all your test: new_text = "\n".join(line[:line.index('<')] + line[line.rindex('>')+1:] for line in your_text.split('\n')) Cheers, Chris -- http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list From clp2 at rebertia.com Mon Jan 18 23:40:46 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Mon, 18 Jan 2010 20:40:46 -0800 Subject: python replace/sub/wildcard/regex issue In-Reply-To: <50697b2c1001182031v893cfecyfa7dc540cabb3a@mail.gmail.com> References: <9d80d7f8-d78f-4b0b-9d8b-afae52633c8a@b10g2000yqa.googlegroups.com> <50697b2c1001182031v893cfecyfa7dc540cabb3a@mail.gmail.com> Message-ID: <50697b2c1001182040v1a070bc0uf610d8e2246a74a2@mail.gmail.com> On Mon, Jan 18, 2010 at 8:31 PM, Chris Rebert wrote: > On Mon, Jan 18, 2010 at 8:04 PM, tom wrote: >> hi... >> >> trying to figure out how to solve what should be an easy python/regex/ >> wildcard/replace issue. >> >> i've tried a number of different approaches.. so i must be missing >> something... >> >> my initial sample text are: >> >> Soo ChoiLONGEDITBOX">Apryl Berney >> Soo ChoiLONGEDITBOX">Joel Franks >> Joel FranksGEDITBOX">Alexander Yamato >> >> and i'm trying to get >> >> Soo Choi foo Apryl Berney >> Soo Choi foo Joel Franks >> Joel Franks foo Alexander Yamato >> >> the issue i'm facing.. is how to start at "' and >> substitute inclusive of the stuff inside the regex... >> >> i've tried derivations of >> >> name=re.sub("]*\">"," foo ",name) >> >> but i'm missing something... >> >> thoughts... thanks > > "Some people, when confronted with a problem, think 'I know, I'll use > regular expressions.' Now they have two problems." > > Assuming your sample text is representative of all your test: > > new_text = "\n".join(line[:line.index('<')] + line[line.rindex('>')+1:] for line in your_text.split('\n')) Erm, remembering to intersperse the "foo" (should be all 1-line, bloody Gmail): new_text = "\n".join(line[:line.index('<')] + " foo " + line[line.rindex('>')+1:] for line in your_text.split('\n')) Or just use alex23's method, which seems all-round superior. :-) Cheers, Chris From anandvaidya.ml at gmail.com Tue Jan 19 00:05:26 2010 From: anandvaidya.ml at gmail.com (Anand Vaidya) Date: Mon, 18 Jan 2010 21:05:26 -0800 (PST) Subject: Generic Python Benchmark suite? References: Message-ID: <08431d7e-7ce9-40d0-8a9c-1c2b885ce7b4@p8g2000yqb.googlegroups.com> On Jan 19, 5:42?am, Terry Reedy wrote: > On 1/18/2010 4:58 AM, Anand Vaidya wrote: > > > Is there a generic python benchmark suite in active development? I am > > looking forward to comparing some code on various python > > implementations (primarily CPython 2.x, CPython 3.x, UnladenSwallow, > > Psyco). > > You might find this interesting if you have not seen it before:http://code.google.com/p/unladen-swallow/wiki/Benchmarks @Antoine, Terry, Thanks for the suggestions. I will investigate those. I just ran the pybench, doesn't run on 3.x, 2to3 fails. I was hoping to locate something like Phoronix Test Suite (that is a suite of microbenchmarks for Linux OS) Regards Anand From wissme at hotmail.com Tue Jan 19 02:09:03 2010 From: wissme at hotmail.com (Jean Guillaume Pyraksos) Date: Tue, 19 Jan 2010 08:09:03 +0100 Subject: Python IDE for MacOS-X Message-ID: What's the best one to use with beginners ? Something with integrated syntax editor, browser of doc... Thanks, JG From moekaveli at gmail.com Tue Jan 19 02:18:30 2010 From: moekaveli at gmail.com (Dr. Benjamin David Clarke) Date: Mon, 18 Jan 2010 23:18:30 -0800 (PST) Subject: Updating an OptionMenu every time the text file it reads from is updated (Tkinter) Message-ID: <45d53456-0673-4829-aa73-9c98feba7b64@s31g2000yqs.googlegroups.com> I currently have a program that reads in values for an OptionMenu from a text file. I also have an option to add a line to that text file which corresponds to a new value for that OptionMenu. How can I make that OptionMenu update its values based on that text file without restarting the program? In other words, every time I add a value to the text file, I want the OptionMenu to immediately update to take note of this change. I'll provide code if needed. From mal at egenix.com Tue Jan 19 03:52:22 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 19 Jan 2010 09:52:22 +0100 Subject: Generic Python Benchmark suite? In-Reply-To: References: Message-ID: <4B5572C6.9040009@egenix.com> Anand Vaidya wrote: > Is there a generic python benchmark suite in active development? I am > looking forward to comparing some code on various python > implementations (primarily CPython 2.x, CPython 3.x, UnladenSwallow, > Psyco). > > I am happy with something that gives me a relative number eg: ULS is > 30% faster than CPy2.x etc > > I found pybench which is probably not maintained actively. Oh, it is. In fact, I'm preparing a new version for Python 2.7. > What do you suggest? > > PS: I think a benchmark should cover file / network, database I/O, > data structures (dict, list etc), object creation/manipulation, > numbers, measure looping inefficiencies, effects of caching (memcache > etc) at the minimum pybench addresses many of the low-level aspects you're asking for. It doesn't have an I/O tests, since these usually don't have much to do with Python's performance, but rather that of the underlying OS and hardware. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 19 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From stefan_ml at behnel.de Tue Jan 19 04:41:20 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 19 Jan 2010 10:41:20 +0100 Subject: searching and storing large quantities of xml! In-Reply-To: References: <4b52bdb8$0$6733$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <4b557e40$0$6586$9b4e6d93@newsspool3.arcor-online.net> dads, 18.01.2010 22:39: > There was one thing that I forgot about - when ElementTree fails to > parse due to an element not being closed why doesn't it close the file > like object. Because it didn't open it? Stefan From rajat.dudeja at gmail.com Tue Jan 19 04:51:02 2010 From: rajat.dudeja at gmail.com (Rajat) Date: Tue, 19 Jan 2010 01:51:02 -0800 (PST) Subject: thread return code Message-ID: <08974f1d-4334-4165-8272-7c178f589bbc@m16g2000yqc.googlegroups.com> Hi, I'm using threading module in Python 2.6.4. I'm using thread's join() method. On the new thread I'm running a function which returns a code at the end. Is there a way I access that code in the parent thread after thread finishes? Simply, does join() could get me that code? Regards, Rajat From ted at sjksdjk.it Tue Jan 19 04:58:49 2010 From: ted at sjksdjk.it (ted) Date: Tue, 19 Jan 2010 10:58:49 +0100 Subject: python gui ide under linux..like visual studio ;) ? In-Reply-To: <3f3650e2-1c9d-43e1-b759-6c3d939de7bb@h9g2000yqa.googlegroups.com> References: <4b5470ce$0$1103$4fafbaef@reader4.news.tin.it> <3f3650e2-1c9d-43e1-b759-6c3d939de7bb@h9g2000yqa.googlegroups.com> Message-ID: <4b558226$0$823$4fafbaef@reader5.news.tin.it> Il 18/01/2010 21:59, Mike Driscoll ha scritto: > On Jan 18, 8:32 am, ted wrote: >> Hi at all... >> Can someone please give me some advice, about a good IDE with control >> GUI under Linux ? >> >> Actually i know QT Creator by Nokia which i can use with Python (but i >> don't know how). >> >> And, a good library for access to database (mysql, sql server, oracle) ? >> >> Thank you very much ! >> >> bye > > Check out Dabo for the database stuff: http://dabodev.com/ > > There is nothing like Visual Studio. The closest I've found are things > like the QT Creator and wxGlade / Boa Constructor / wxFormBuilder. I > know I've heard of another one that was commercial, but I can't recall > the name at the moment. > > ------------------- > Mike Driscoll > > Blog: http://blog.pythonlibrary.org > > PyCon 2010 Atlanta Feb 19-21 http://us.pycon.org/ Thank you very much. Bye! From alfps at start.no Tue Jan 19 05:25:55 2010 From: alfps at start.no (Alf P. Steinbach) Date: Tue, 19 Jan 2010 11:25:55 +0100 Subject: thread return code In-Reply-To: <08974f1d-4334-4165-8272-7c178f589bbc@m16g2000yqc.googlegroups.com> References: <08974f1d-4334-4165-8272-7c178f589bbc@m16g2000yqc.googlegroups.com> Message-ID: * Rajat: > Hi, > > I'm using threading module in Python 2.6.4. I'm using thread's join() > method. > > On the new thread I'm running a function which returns a code at the > end. Is there a way I access that code in the parent thread after > thread finishes? Simply, does join() could get me that code? join() always returns None. But you can store the code in the thread object, and then access it after the join(). Cheers & hth., - Alf From chris at simplistix.co.uk Tue Jan 19 05:56:34 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 19 Jan 2010 10:56:34 +0000 Subject: what test runner should I use? Message-ID: <4B558FE2.40701@simplistix.co.uk> Hi All, I'm wondering what test runner I should use. Here's my list of requirements: - cross platform (I develop for and on Windows, Linux and Mac) - should not prevent tests from running with other test runners (so no plugins/layers/etc that only work with one specific test runner) - should work with zc.buildout (preferably without a specialist recipe!) So far I've tried the following with the resultant problems: zope.testing - requires a special recipe to be useful - now emits deprecation warnings from itself: https://mail.zope.org/pipermail/zope-dev/2009-December/038965.html - coverage support is baroque to put it politely twisted's trial - only has old-style script definition in setup.py, so doesn't work with buildout without hackery - drops _twisted_trial folders all over the place and doesn't clear them up nose - can't see to get it to run only my packages tests, rather than including the tests of packages my package depends on - seems to be focused towards files rather than modules (which makes it not play nicely with buildout) - seems to be difficult to provide options to at configuration time that can then be overridden on the command line I did also look at py.test's homepage but found it pretty scary. What other options do people recommend? Failing that, any ideas how to fix the problems above? cheers, Chris From alfps at start.no Tue Jan 19 05:57:09 2010 From: alfps at start.no (Alf P. Steinbach) Date: Tue, 19 Jan 2010 11:57:09 +0100 Subject: Updating an OptionMenu every time the text file it reads from is updated (Tkinter) In-Reply-To: <45d53456-0673-4829-aa73-9c98feba7b64@s31g2000yqs.googlegroups.com> References: <45d53456-0673-4829-aa73-9c98feba7b64@s31g2000yqs.googlegroups.com> Message-ID: * Dr. Benjamin David Clarke: > I currently have a program that reads in values for an OptionMenu from > a text file. I also have an option to add a line to that text file > which corresponds to a new value for that OptionMenu. How can I make > that OptionMenu update its values based on that text file without > restarting the program? In other words, every time I add a value to > the text file, I want the OptionMenu to immediately update to take > note of this change. I'll provide code if needed. It's a bit unclear to me whether the update of the text file is from within the process as where the menu is, or not. If the problem is connecting file updates to menu changes: If it's the same process, perhaps you can wrap all access of the text file so that whenever an operation to change the text file is performed, it calls back on interested parties (like event handling)? If it's not the same process you need some "file changed" notification. Windows has this functionality. I don't know if it's there in *nix. Anyway, since I'm still essentially a Python newbie I don't know of any Python modules for such functionality, if that's what you need. But perhaps someone else does... Otherwise, if the problem is actually updating the menu, and it is a tkinter OptionMenu: It appears that an OptionMenu has a logical attribute 'menu' that is a tkinter Menu, representing the options; if your OptionMenu is an object 'o' then you can write 'o["menu"]' or 'o.cget( "menu" )' to get at that logical attribute. And tkinter Menu has various methods that you can use to inspect or update the menu; see . Disclaimer: I haven't tried updating, but I see no reason why it shouldn't work. :-) Cheers & hth., - Alf From michels at mps.mpg.de Tue Jan 19 06:06:12 2010 From: michels at mps.mpg.de (Helmut Michels) Date: Tue, 19 Jan 2010 12:06:12 +0100 Subject: [ANN] Data Plotting Library DISLIN 10.0 Message-ID: Dear Pytnon users, I am pleased to announce version 10.0 of the data plotting software DISLIN. DISLIN is a high-level and easy to use plotting library for displaying data as curves, bar graphs, pie charts, 3D-colour plots, surfaces, contours and maps. Several output formats are supported such as X11, VGA, PostScript, PDF, CGM, WMF, HPGL, TIFF, GIF, PNG, BMP and SVG. The software is available for the most C, Fortran 77 and Fortran 90/95 compilers. Plotting extensions for the interpreting languages Perl, Python and Java are also supported. DISLIN distributions and manuals in PDF, PostScript and HTML format are available from the DISLIN home page http://www.dislin.de and via FTP from the server ftp://ftp.gwdg.de/pub/grafik/dislin All DISLIN distributions are free for non-commercial use. Licenses for commercial use are available from the site http://www.dislin.de. ------------------- Helmut Michels Max Planck Institute for Solar System Research Phone: +49 5556 979-334 Max-Planck-Str. 2 Fax : +49 5556 979-240 D-37191 Katlenburg-Lindau Mail : michels at mps.mpg.de From fossist at gmail.com Tue Jan 19 06:16:30 2010 From: fossist at gmail.com (fossist) Date: Tue, 19 Jan 2010 03:16:30 -0800 (PST) Subject: Is HTML report of tests run using PyUnit (unittest) possible? Message-ID: <584f31d6-4c4d-4661-836f-c8552d892d38@j19g2000yqk.googlegroups.com> I am using PyUnit (unittest module) for loading test cases from our modules and executing them. Is it possible to export the test results as HTML report? Currently the results appear as text on standard output while the tests execute. But is there something out of the box available in PyUnit to make this possible? From stefan_ml at behnel.de Tue Jan 19 06:42:02 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 19 Jan 2010 12:42:02 +0100 Subject: twenty years ago Guido created Python In-Reply-To: <555a57f6-6593-47d5-9a3a-1fe01a91b51d@z7g2000vbl.googlegroups.com> References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> <87oclaxksq.fsf@benfinney.id.au> <877hrxy3zz.fsf@benfinney.id.au> <02d9bd25-5ae0-4de3-910d-dc36bec0fc11@m3g2000yqf.googlegroups.com> <555a57f6-6593-47d5-9a3a-1fe01a91b51d@z7g2000vbl.googlegroups.com> Message-ID: <4b559a8a$0$6582$9b4e6d93@newsspool3.arcor-online.net> n00m, 06.01.2010 01:25: > My mom a little girl wsas pursued by Germans alsatians dogs in a > forest > Germans just made a fun of it. > They screamead Zurich and laughed They laughed at her scare and you > now teach me. Now her hands tremble froom malntrination > She ate ate at WWII grass. And you teach me? Sorry, this thread wasn't long enough yet to put Godwin's law in action. Stefan From michele.simionato at gmail.com Tue Jan 19 06:44:09 2010 From: michele.simionato at gmail.com (Michele Simionato) Date: Tue, 19 Jan 2010 03:44:09 -0800 (PST) Subject: Inheriting methods but over-riding docstrings References: <0361ea2d$0$1336$c3e8da3@news.astraweb.com> Message-ID: <28aa31f4-7331-4ac2-918b-0c59f5a834de@e37g2000yqn.googlegroups.com> On Jan 16, 6:55?pm, Steven D'Aprano wrote: > I have a series of subclasses that inherit methods from a base class, but > I'd like them to have their own individual docstrings. The following is not tested more than you see and will not work for builtin methods, but it should work in the common cases: from types import FunctionType, CodeType def newfunc(func, docstring): c = func.func_code nc = CodeType(c.co_argcount, c.co_nlocals, c.co_stacksize, c.co_flags, c.co_code, c.co_consts, c.co_names, c.co_varnames, c.co_filename, func.__name__, c.co_firstlineno, c.co_lnotab, c.co_freevars, c.co_cellvars) nf = FunctionType(nc, func.func_globals, func.__name__) nf.__doc__ = docstring return nf def setdocstring(method, docstring): cls = method.im_class basefunc = getattr(super(cls, cls), method.__name__).im_func setattr(cls, method.__name__, newfunc(basefunc, docstring)) # example of use class B(object): def m(self): "base" return 'ok' class C(B): pass setdocstring(C.m, 'C.m docstring') print B.m.__doc__ # the base docstring print C.m.__doc__ # the new docstring From __peter__ at web.de Tue Jan 19 07:00:10 2010 From: __peter__ at web.de (Peter Otten) Date: Tue, 19 Jan 2010 13:00:10 +0100 Subject: Updating an OptionMenu every time the text file it reads from is updated (Tkinter) References: <45d53456-0673-4829-aa73-9c98feba7b64@s31g2000yqs.googlegroups.com> Message-ID: Dr. Benjamin David Clarke wrote: > I currently have a program that reads in values for an OptionMenu from > a text file. I also have an option to add a line to that text file > which corresponds to a new value for that OptionMenu. How can I make > that OptionMenu update its values based on that text file without > restarting the program? In other words, every time I add a value to > the text file, I want the OptionMenu to immediately update to take > note of this change. I'll provide code if needed. Inferred from looking into the Tkinter source code: # python 2.6 import Tkinter as tk root = tk.Tk() var = tk.StringVar() var.set("One") optionmenu = tk.OptionMenu(root, var, "One", "Two", "Three") optionmenu.grid(row=0, column=1) def add_option(): value = entry_add.get() menu = optionmenu["menu"] variable = var command = None # what you passed as command argument to optionmenu menu.add_command(label=value, command=tk._setit(variable, value, command)) label_show = tk.Label(root, text="current value") label_show.grid(row=1, column=0) entry_show = tk.Entry(root, textvariable=var) entry_show.grid(row=1, column=1) label_add = tk.Label(root, text="new option") label_add.grid(row=2, column=0) entry_add = tk.Entry(root) entry_add.grid(row=2, column=1) button_add = tk.Button(root, text="add option", command=add_option) button_add.grid(row=2, column=2) root.mainloop() Peter From solipsis at pitrou.net Tue Jan 19 07:04:13 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 19 Jan 2010 12:04:13 +0000 (UTC) Subject: Generic Python Benchmark suite? References: <08431d7e-7ce9-40d0-8a9c-1c2b885ce7b4@p8g2000yqb.googlegroups.com> Message-ID: Le Mon, 18 Jan 2010 21:05:26 -0800, Anand Vaidya a ?crit?: > @Antoine, Terry, > > Thanks for the suggestions. > > I will investigate those. I just ran the pybench, doesn't run on 3.x, > 2to3 fails. You just have to use the pybench version that is bundled with 3.x (in the Tools directory). From utente at esempio.net Tue Jan 19 07:12:14 2010 From: utente at esempio.net (superpollo) Date: Tue, 19 Jan 2010 13:12:14 +0100 Subject: [ANN] Data Plotting Library DISLIN 10.0 In-Reply-To: References: Message-ID: <4b55a19f$0$1106$4fafbaef@reader4.news.tin.it> Helmut Michels ha scritto: > Dear Pytnon users, > > I am pleased to announce version 10.0 of the data plotting software > DISLIN. why dont you make it free software (i mean. GPL'ed) bye From david.mcwright at usbfmi.com Tue Jan 19 07:23:00 2010 From: david.mcwright at usbfmi.com (dippim) Date: Tue, 19 Jan 2010 04:23:00 -0800 (PST) Subject: python replace/sub/wildcard/regex issue References: <9d80d7f8-d78f-4b0b-9d8b-afae52633c8a@b10g2000yqa.googlegroups.com> Message-ID: <6a7693f0-88d8-48b9-ac3a-896f5d02643b@l30g2000yqb.googlegroups.com> On Jan 18, 11:04?pm, tom wrote: > hi... > > trying to figure out how to solve what should be an easy python/regex/ > wildcard/replace issue. > > i've tried a number of different approaches.. so i must be missing > something... > > my initial sample text are: > > Soo ChoiLONGEDITBOX">Apryl Berney > Soo ChoiLONGEDITBOX">Joel Franks > Joel FranksGEDITBOX">Alexander Yamato > > and i'm trying to get > > Soo Choi foo Apryl Berney > Soo Choi foo Joel Franks > Joel Franks foo Alexander Yamato > > the issue i'm facing.. is how to start at "' and > substitute inclusive of the stuff inside the regex... > > i've tried derivations of > > name=re.sub("]*\">"," foo ",name) > > but i'm missing something... > > thoughts... thanks > > tom The problem here is that ]* consumes anything that's not > and then stops when it hits something that is >. So, [^>]* consumes "pan" in each case, then tries to match \">, but fails since there isn't a ", so the match ends. It never makes it to the second >. I agree with Chris Rebert, regexes are dangerous because the number of possible cases where you can match isn't always clear (see the above explanation :). Also, if the number of comparisons you have to do isn't high, they can be inefficient. However, for your limited set of examples the following should work: aList = ['Soo ChoiLONGEDITBOX">Apryl Berney', 'Soo ChoiLONGEDITBOX">Joel Franks', 'Joel FranksGEDITBOX">Alexander Yamato'] matcher = re.compile(r"<[\w\W]*>") newList = [] for x in aList: newList.append(matcher.sub(" foo ", x)) print newList David From jeanmichel at sequans.com Tue Jan 19 07:33:54 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Tue, 19 Jan 2010 13:33:54 +0100 Subject: using super In-Reply-To: References: <4B54BB95.1090407@sequans.com> Message-ID: <4B55A6B2.107@sequans.com> Gabriel Genellina wrote: >> I see. >> Then is there a reason why >> return super(Subclass, self).parrot() >> would be prefered over the "classic" >> return Base.parrot(self) >> ? >> Or is it just a matter of preference ? > > For a longer explanation, see: > > James Knight: Python's Super Considered Harmful > http://fuhm.net/super-harmful/ > > Michele Simionato: Things to Know About Python Super > http://www.artima.com/weblogs/viewpost.jsp?thread=236275 > http://www.artima.com/weblogs/viewpost.jsp?thread=236278 > http://www.artima.com/weblogs/viewpost.jsp?thread=237121 > Thanks to all who replied to this thread. I didn't remember why I didn't want to dive into super in the first place, now I remember :o) I'm sure about one thing about super: it has a misleading name. JM From gnarlodious at gmail.com Tue Jan 19 08:03:52 2010 From: gnarlodious at gmail.com (Gnarlodious) Date: Tue, 19 Jan 2010 05:03:52 -0800 (PST) Subject: Py 3: Terminal script can't find relative path References: <58e9b1e4-049a-4a7f-bea9-8d7e7e9f191a@e27g2000yqd.googlegroups.com> <87d417578z.fsf@castleamber.com> Message-ID: <9541dcf0-8060-4851-bdbd-0aed6de13bf9@v25g2000yqk.googlegroups.com> On Jan 18, 4:21?pm, John Bokma wrote: > Gnarlodious writes: > > I am running a script in a browser that finds the file in subfolder > > Data: > > > Content=Plist('Data/Content.plist') > > > However, running the same script in Terminal errors: > > > IOError: [Errno 2] No such file or directory: 'Data/Content.plist' > > What does: > > ls -l Data/Content.plist > > in the terminal give? I can replace with absolute paths and it works as expected. Could this be a Python 3 bug? Where as a CGI script it finds the relative path but not in Terminal? -- Gnarlie From rpjday at crashcourse.ca Tue Jan 19 10:29:08 2010 From: rpjday at crashcourse.ca (Robert P. J. Day) Date: Tue, 19 Jan 2010 10:29:08 -0500 (EST) Subject: can i examine the svn rev used to build a python 3 executable? Message-ID: i'm currently checking out python3 from the svn repo, configuring, building and installing under /usr/local/bin on my fedora 12 system, all good. i'm curious, though -- is there a query i can make of that executable that would tell me what svn rev it was built from? i'm guessing not, but i thought i'd ask. rday -- ======================================================================== Robert P. J. Day Waterloo, Ontario, CANADA Linux Consulting, Training and Kernel Pedantry. Web page: http://crashcourse.ca Twitter: http://twitter.com/rpjday ======================================================================== From gerald.britton at gmail.com Tue Jan 19 10:30:21 2010 From: gerald.britton at gmail.com (Gerald Britton) Date: Tue, 19 Jan 2010 10:30:21 -0500 Subject: Performance of lists vs. list comprehensions Message-ID: <5d1a32001001190730u4fa2f26cn62e79c2c73a08b@mail.gmail.com> Yesterday I stumbled across some old code in a project I was working on. It does something like this: mystring = '\n'.join( [ line for line in lines if ] ) where "lines" is a simple list of strings. I realized that the code had been written before you could put a list comprehension as an argument to join(). I figured that I could drop the '[' and ']' and leave the rest as a list comprehension since that works with current Python (and the project's base is 2.5). So I rewrote the original statement like this: mystring = '\n'.join( line for line in lines if ) It works as expected. Then I got curious as to how it performs. I was surprised to learn that the rewritten expression runs more than twice as _slow_. e.g.: >>> l ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9'] >>> Timer("' '.join([x for x in l])", 'l = map(str,range(10))').timeit() 2.9967339038848877 >>> Timer("' '.join(x for x in l)", 'l = map(str,range(10))').timeit() 7.2045478820800781 Notice that I dropped the condition testing that was in my original code. I just wanted to see the effect of two different expressions. I thought that maybe there was some lower bound on the number of the items in the list or list comprehension beyond which the comprehension would prove more efficient. There doesn't appear to be one. I scaled the length of the input list up to 1 million items and got more or less the same relative performance. Now I'm really curious and I'd like to know: 1. Can anyone else confirm this observation? 2. Why should the "pure" list comprehension be slower than the same comprehension enclosed in '[...]' ? -- Gerald Britton From rpjday at crashcourse.ca Tue Jan 19 10:32:31 2010 From: rpjday at crashcourse.ca (Robert P. J. Day) Date: Tue, 19 Jan 2010 10:32:31 -0500 (EST) Subject: can i examine the svn rev used to build a python 3 executable? In-Reply-To: References: Message-ID: On Tue, 19 Jan 2010, Robert P. J. Day wrote: > i'm currently checking out python3 from the svn repo, configuring, > building and installing under /usr/local/bin on my fedora 12 system, > all good. > > i'm curious, though -- is there a query i can make of that > executable that would tell me what svn rev it was built from? i'm > guessing not, but i thought i'd ask. never mind. just discovered that while "python3 -V" won't do it, executing it gives me: $ python3 Python 3.2a0 (py3k:77609, Jan 19 2010, 04:10:16) ... and it's that 77609 rev number i was after. rday -- ======================================================================== Robert P. J. Day Waterloo, Ontario, CANADA Linux Consulting, Training and Kernel Pedantry. Web page: http://crashcourse.ca Twitter: http://twitter.com/rpjday ======================================================================== From ryniek90 at gmail.com Tue Jan 19 10:40:02 2010 From: ryniek90 at gmail.com (ryniek90) Date: Tue, 19 Jan 2010 16:40:02 +0100 Subject: python gui ide under linux..like visual studio ;) ? In-Reply-To: References: Message-ID: <4B55D252.8020800@gmail.com> > Il 18/01/2010 21:59, Mike Driscoll ha scritto: > >> > On Jan 18, 8:32 am, ted wrote: >> >>> >> Hi at all... >>> >> Can someone please give me some advice, about a good IDE with control >>> >> GUI under Linux ? >>> >> >>> >> Actually i know QT Creator by Nokia which i can use with Python (but i >>> >> don't know how). >>> >> >>> >> And, a good library for access to database (mysql, sql server, oracle) ? >>> >> >>> >> Thank you very much ! >>> >> >>> >> bye >>> >> > >> > Check out Dabo for the database stuff:http://dabodev.com/ >> > >> > There is nothing like Visual Studio. The closest I've found are things >> > like the QT Creator and wxGlade / Boa Constructor / wxFormBuilder. I >> > know I've heard of another one that was commercial, but I can't recall >> > the name at the moment. >> > >> > ------------------- >> > Mike Driscoll >> > >> > Blog:http://blog.pythonlibrary.org >> > >> > PyCon 2010 Atlanta Feb 19-21http://us.pycon.org/ >> > Thank you very much. > > Bye! > > > Hi, check this Python Wiki pages: http://wiki.python.org/moin/IntegratedDevelopmentEnvironments Cheers. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gnarlodious at gmail.com Tue Jan 19 10:50:54 2010 From: gnarlodious at gmail.com (Gnarlodious) Date: Tue, 19 Jan 2010 07:50:54 -0800 (PST) Subject: Py 3: How to switch application to Unicode strings? Message-ID: <92288825-8b01-47fb-8592-f65a7d7aeb8a@k35g2000yqb.googlegroups.com> I am using Python 3, getting an error from SQLite: sqlite3.ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. So... how do I switch to Unicode? I thought I was doing it when I put # coding:utf-8 at the start of my script. -- Gnarlie From alfps at start.no Tue Jan 19 10:57:39 2010 From: alfps at start.no (Alf P. Steinbach) Date: Tue, 19 Jan 2010 16:57:39 +0100 Subject: Performance of lists vs. list comprehensions In-Reply-To: References: Message-ID: * Gerald Britton: > Yesterday I stumbled across some old code in a project I was working > on. It does something like this: > > mystring = '\n'.join( [ line for line in lines if depending on line> ] ) > > where "lines" is a simple list of strings. I realized that the code > had been written before you could put a list comprehension as an > argument to join(). I figured that I could drop the '[' and ']' and > leave the rest as a list comprehension since that works with current > Python (and the project's base is 2.5). So I rewrote the original > statement like this: > > mystring = '\n'.join( line for line in lines if depending on line> ) > > It works as expected. Then I got curious as to how it performs. I > was surprised to learn that the rewritten expression runs more than > twice as _slow_. e.g.: > >>>> l > ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9'] > >>>> Timer("' '.join([x for x in l])", 'l = map(str,range(10))').timeit() > 2.9967339038848877 > >>>> Timer("' '.join(x for x in l)", 'l = map(str,range(10))').timeit() > 7.2045478820800781 > > Notice that I dropped the condition testing that was in my original > code. I just wanted to see the effect of two different expressions. > > I thought that maybe there was some lower bound on the number of the > items in the list or list comprehension beyond which the comprehension > would prove more efficient. There doesn't appear to be one. I scaled > the length of the input list up to 1 million items and got more or > less the same relative performance. > > Now I'm really curious and I'd like to know: > > 1. Can anyone else confirm this observation? >>> Timer("' '.join([x for x in l])", 'l = map(str,range(10))').timeit() 5.8625191190500345 >>> Timer("' '.join(x for x in l)", 'l = map(str,range(10))').timeit() 12.093135300715574 >>> _ > 2. Why should the "pure" list comprehension be slower than the same > comprehension enclosed in '[...]' ? Regarding (2) the unparenthesized expression in join is *not* a list comprehension but a generator expression. And as such it involves join calling next() on the generator object repeatedly, with each next() call involving a light-weight context shift. In addition the docs mumble something about "lazy" evaluation, and that may also contribute to the overhead. I think that in contrast, the interpreter can evaluate a list comprehension, [x for x in blah], directly without any context shifting, just by transforming it to equivalent code and putting the target expressions innermost there. And so the main factor causing a slowdown for a list comprehension would, I think, be paging and such if the list it produced was Really Huge, while for the generator there's no memory issue but rather much calling & context shifting. Cheers & hth., - Alf From gerald.britton at gmail.com Tue Jan 19 11:10:43 2010 From: gerald.britton at gmail.com (Gerald Britton) Date: Tue, 19 Jan 2010 11:10:43 -0500 Subject: Performance of lists vs. list comprehensions In-Reply-To: References: Message-ID: <5d1a32001001190810w7d0c78b2r8723185e2d8c23f8@mail.gmail.com> Thanks! Good explanation. On Tue, Jan 19, 2010 at 10:57 AM, Alf P. Steinbach wrote: > * Gerald Britton: >> >> Yesterday I stumbled across some old code in a project I was working >> on. ?It does something like this: >> >> mystring = '\n'.join( [ line for line in lines if > depending on line> ] ) >> >> where "lines" is a simple list of strings. ?I realized that the code >> had been written before you could put a list comprehension as an >> argument to join(). ?I figured that I could drop the '[' and ']' and >> leave the rest as a list comprehension since that works with current >> Python (and the project's base is 2.5). ?So I rewrote the original >> statement like this: >> >> mystring = '\n'.join( line for line in lines if > depending on line> ) >> >> It works as expected. ?Then I got curious as to how it performs. ?I >> was surprised to learn that the rewritten expression runs more than >> twice as _slow_. ?e.g.: >> >>>>> l >> >> ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9'] >> >>>>> Timer("' '.join([x for x in l])", 'l = map(str,range(10))').timeit() >> >> 2.9967339038848877 >> >>>>> Timer("' '.join(x for x in l)", 'l = map(str,range(10))').timeit() >> >> 7.2045478820800781 >> >> Notice that I dropped the condition testing that was in my original >> code. ?I just wanted to see the effect of two different expressions. >> >> I thought that maybe there was some lower bound on the number of the >> items in the list or list comprehension beyond which the comprehension >> would prove more efficient. ?There doesn't appear to be one. ?I scaled >> the length of the input list up to 1 million items and got more or >> less the same relative performance. >> >> Now I'm really curious and I'd like to know: >> >> 1. Can anyone else confirm this observation? > >>>> Timer("' '.join([x for x in l])", 'l = map(str,range(10))').timeit() > 5.8625191190500345 >>>> Timer("' '.join(x for x in l)", 'l = map(str,range(10))').timeit() > 12.093135300715574 >>>> _ > > >> 2. Why should the "pure" list comprehension be slower than the same >> comprehension enclosed in '[...]' ? > > Regarding (2) the unparenthesized expression in join is *not* a list > comprehension but a generator expression. > > And as such it involves join calling next() on the generator object > repeatedly, with each next() call involving a light-weight context shift. > > In addition the docs mumble something about "lazy" evaluation, and that may > also contribute to the overhead. > > I think that in contrast, the interpreter can evaluate a list comprehension, > [x for x in blah], directly without any context shifting, just by > transforming it to equivalent code and putting the target expressions > innermost there. > > And so the main factor causing a slowdown for a list comprehension would, I > think, be paging and such if the list it produced was Really Huge, while for > the generator there's no memory issue but rather much calling & context > shifting. > > > Cheers & hth., > > - Alf > -- > http://mail.python.org/mailman/listinfo/python-list > -- Gerald Britton From doxalogos at gmail.com Tue Jan 19 11:15:22 2010 From: doxalogos at gmail.com (DoxaLogos) Date: Tue, 19 Jan 2010 08:15:22 -0800 (PST) Subject: multiprocessing problems Message-ID: <74594510-9366-4227-8e1e-d221a9220271@k35g2000yqb.googlegroups.com> Hi, I decided to play around with the multiprocessing module, and I'm having some strange side effects that I can't explain. It makes me wonder if I'm just overlooking something obvious or not. Basically, I have a script parses through a lot of files doing search and replace on key strings inside the file. I decided the split the work up on multiple processes on each processor core (4 total). I've tried many various ways doing this form using pool to calling out separate processes, but the result has been the same: computer crashes from endless process spawn. Here's the guts of my latest incarnation. def ProcessBatch(files): p = [] for file in files: p.append(Process(target=ProcessFile,args=file)) for x in p: x.start() for x in p: x.join() p = [] return Now, the function calling ProcessBatch looks like this: def ReplaceIt(files): """ All this does is walks through all the files passed to it and verifies the file is a legitimate file to be processed (project file). @param files: files to be processed """ processFiles = [] for replacefile in files: if(CheckSkipFile(replacefile)): processFiles.append(replacefile) if(len(processFiles) == 4): ProcessBatch(processFiles) processFiles = [] #check for left over files once main loop is done and process them if(len(processFiles) > 0): ProcessBatch(processFiles) return Specs: Windows 7 64-bit Python v2.6.2 Intel i5 Thanks From apt.shansen at gmail.com Tue Jan 19 11:18:00 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Tue, 19 Jan 2010 08:18:00 -0800 Subject: Performance of lists vs. list comprehensions In-Reply-To: <5d1a32001001190730u4fa2f26cn62e79c2c73a08b@mail.gmail.com> References: <5d1a32001001190730u4fa2f26cn62e79c2c73a08b@mail.gmail.com> Message-ID: <7a9c25c21001190818g7bd0ca6n336e6b1d52af0b2c@mail.gmail.com> On Tue, Jan 19, 2010 at 7:30 AM, Gerald Britton wrote: [snip] > mystring = '\n'.join( line for line in lines if depending on line> ) > Note, this is not a list comprehension, but a generator comprehension. A list comprehension is used to build, in one sweep, a list and return it. A generator comprehension is used to build an generator that can be iterated over to produce a sequence of items. I think you're seeing not performance of the expression, but the function call overhead which generators include. A generator requires one to call its next method to get each item: a list comprehension is just syntactical sugar for a for loop. As to which is faster, I think it depends. Your test-case is using *really* small ranges-- just ten items. In this case, just doing a simple loop to build a list and then pass it through join is probably faster. If you're using a much larger list though, the characteristics of the problem may change, where the lazy evaluation of a generator expression may be more desirable. A list comprehension includes a waste of memory, too: you have to build up a complete list before you return it, and if you have a lot of lines? That can be a problem. As you can see, the performance characteristics between the two narrow considerably if you compare a larger sample: >>> Timer("' '.join([str(x) for x in l])", 'l = xrange(100)').timeit() 50.092024087905884 >>> Timer("' '.join(str(x) for x in l)", 'l = xrange(100)').timeit() 54.591049909591675 --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From pruebauno at latinmail.com Tue Jan 19 11:18:17 2010 From: pruebauno at latinmail.com (nn) Date: Tue, 19 Jan 2010 08:18:17 -0800 (PST) Subject: Py 3: Terminal script can't find relative path References: <58e9b1e4-049a-4a7f-bea9-8d7e7e9f191a@e27g2000yqd.googlegroups.com> <87d417578z.fsf@castleamber.com> <9541dcf0-8060-4851-bdbd-0aed6de13bf9@v25g2000yqk.googlegroups.com> Message-ID: <317ebb29-8c20-4fe5-82c4-b2fe05efef38@h9g2000yqa.googlegroups.com> On Jan 19, 8:03?am, Gnarlodious wrote: > On Jan 18, 4:21?pm, John Bokma wrote: > > > Gnarlodious writes: > > > I am running a script in a browser that finds the file in subfolder > > > Data: > > > > Content=Plist('Data/Content.plist') > > > > However, running the same script in Terminal errors: > > > > IOError: [Errno 2] No such file or directory: 'Data/Content.plist' > > > What does: > > > ls -l Data/Content.plist > > > in the terminal give? > > I can replace with absolute paths and it works as expected. Could this > be a Python 3 bug? Where as a CGI script it finds the relative path > but not in Terminal? > > -- Gnarlie Stop and think for a second what you are saying: It works with absolute paths, it works as CGI script with relative paths, it doesn't work in the terminal. What is different? Do you know for sure what folder you are starting at when using the relative path? Most likely the terminal starts in a different place than the CGI script. From awilliam at opengroupware.us Tue Jan 19 11:20:17 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Tue, 19 Jan 2010 11:20:17 -0500 Subject: Closing a Curses Window??? Message-ID: <1263918017.3165.12.camel@linux-m3mt> I'm uses the curses module of Python to create a TUI. I can create windows, and subwindows, but I can't find anything on deleting or disposing of a a subwindow. How do I get rid of a subwindow? From oswald.harry at gmail.com Tue Jan 19 11:22:15 2010 From: oswald.harry at gmail.com (harryos) Date: Tue, 19 Jan 2010 08:22:15 -0800 (PST) Subject: use of super Message-ID: <8fbe9650-abee-4f0a-93f2-040fc8697e82@u41g2000yqe.googlegroups.com> hi I was going thru the weblog appln in practical django book by bennet .I came across this class Entry(Model): def save(self): dosomething() super(Entry,self).save() I couldn't make out why Entry and self are passed as arguments to super ().Can someone please explain? thanks harry From gerald.britton at gmail.com Tue Jan 19 11:26:43 2010 From: gerald.britton at gmail.com (Gerald Britton) Date: Tue, 19 Jan 2010 11:26:43 -0500 Subject: Performance of lists vs. list comprehensions In-Reply-To: <7a9c25c21001190818g7bd0ca6n336e6b1d52af0b2c@mail.gmail.com> References: <5d1a32001001190730u4fa2f26cn62e79c2c73a08b@mail.gmail.com> <7a9c25c21001190818g7bd0ca6n336e6b1d52af0b2c@mail.gmail.com> Message-ID: <5d1a32001001190826o2d5478bfm2b29a6f0f1818ab9@mail.gmail.com> Interestingly, I scaled it up to a million list items with more or less the same results. It's helpful to see that your results are different. That leads me to suspect that mine are somehow related to my own environment. Still I think the key is the overhead in calling next() for each item in the generator expression. That in itself probably accounts for the differences since function calls are somewhat expensive IIRC. On Tue, Jan 19, 2010 at 11:18 AM, Stephen Hansen wrote: > On Tue, Jan 19, 2010 at 7:30 AM, Gerald Britton > wrote: > [snip] >> >> mystring = '\n'.join( line for line in lines if > depending on line> ) > > Note, this is not a list comprehension, but a generator comprehension. > A list comprehension is used to build, in one sweep, a list and return it. > A generator comprehension is used to build an generator that can be iterated > over to produce a sequence of items. > I think you're seeing not performance of the expression, but the function > call overhead which generators include. A generator requires one to call its > next method to get each item: a list comprehension is just syntactical sugar > for a for loop. > As to which is faster, I think it depends. Your test-case is using *really* > small ranges-- just ten items. In this case, just doing a simple loop to > build a list and then pass it through join is probably faster. If you're > using a much larger list though, the characteristics of the problem may > change, where the lazy evaluation of a generator expression may be more > desirable. > A list comprehension includes a waste of memory, too: you have to build up a > complete list before you return it, and if you have a lot of lines? That can > be a problem. > As you can see, the performance characteristics between the two narrow > considerably if you compare a larger sample: > ?>>> Timer("' '.join([str(x) for x in l])", 'l = xrange(100)').timeit() > 50.092024087905884 >>>> Timer("' '.join(str(x) for x in l)", 'l = xrange(100)').timeit() > 54.591049909591675 > --S -- Gerald Britton From awilliam at opengroupware.us Tue Jan 19 11:26:49 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Tue, 19 Jan 2010 11:26:49 -0500 Subject: multiprocessing problems In-Reply-To: <74594510-9366-4227-8e1e-d221a9220271@k35g2000yqb.googlegroups.com> References: <74594510-9366-4227-8e1e-d221a9220271@k35g2000yqb.googlegroups.com> Message-ID: <1263918409.3165.15.camel@linux-m3mt> > I decided to play around with the multiprocessing module, and I'm > having some strange side effects that I can't explain. It makes me > wonder if I'm just overlooking something obvious or not. Basically, I > have a script parses through a lot of files doing search and replace > on key strings inside the file. I decided the split the work up on > multiple processes on each processor core (4 total). I've tried many > various ways doing this form using pool to calling out separate > processes, but the result has been the same: computer crashes from > endless process spawn. Are you hitting a ulimit error? The number of processes you can create is probably limited. TIP: close os.stdin on your subprocesses. > Here's the guts of my latest incarnation. > def ProcessBatch(files): > p = [] > for file in files: > p.append(Process(target=ProcessFile,args=file)) > for x in p: > x.start() > for x in p: > x.join() > p = [] > return > Now, the function calling ProcessBatch looks like this: > def ReplaceIt(files): > processFiles = [] > for replacefile in files: > if(CheckSkipFile(replacefile)): > processFiles.append(replacefile) > if(len(processFiles) == 4): > ProcessBatch(processFiles) > processFiles = [] > #check for left over files once main loop is done and process them > if(len(processFiles) > 0): > ProcessBatch(processFiles) According to this you will create files is sets of four, but an unknown number of sets of four. From doxalogos at gmail.com Tue Jan 19 11:33:25 2010 From: doxalogos at gmail.com (DoxaLogos) Date: Tue, 19 Jan 2010 08:33:25 -0800 (PST) Subject: multiprocessing problems References: <74594510-9366-4227-8e1e-d221a9220271@k35g2000yqb.googlegroups.com> Message-ID: <63c5df56-7b86-4ee7-ba4f-b14984f94097@e16g2000yqc.googlegroups.com> On Jan 19, 10:26?am, Adam Tauno Williams wrote: > > I decided to play around with the multiprocessing module, and I'm > > having some strange side effects that I can't explain. ?It makes me > > wonder if I'm just overlooking something obvious or not. ?Basically, I > > have a script parses through a lot of files doing search and replace > > on key strings inside the file. ?I decided the split the work up on > > multiple processes on each processor core (4 total). ?I've tried many > > various ways doing this form using pool to calling out separate > > processes, but the result has been the same: computer crashes from > > endless process spawn. > > Are you hitting a ulimit error? ?The number of processes you can create > is probably limited. > > TIP: close os.stdin on your subprocesses. > > > > > Here's the guts of my latest incarnation. > > def ProcessBatch(files): > > ? ? p = [] > > ? ? for file in files: > > ? ? ? ? p.append(Process(target=ProcessFile,args=file)) > > ? ? for x in p: > > ? ? ? ? x.start() > > ? ? for x in p: > > ? ? ? ? x.join() > > ? ? p = [] > > ? ? return > > Now, the function calling ProcessBatch looks like this: > > def ReplaceIt(files): > > ? ? processFiles = [] > > ? ? for replacefile in files: > > ? ? ? ? if(CheckSkipFile(replacefile)): > > ? ? ? ? ? ? processFiles.append(replacefile) > > ? ? ? ? ? ? if(len(processFiles) == 4): > > ? ? ? ? ? ? ? ? ProcessBatch(processFiles) > > ? ? ? ? ? ? ? ? processFiles = [] > > ? ? #check for left over files once main loop is done and process them > > ? ? if(len(processFiles) > 0): > > ? ? ? ? ProcessBatch(processFiles) > > According to this you will create files is sets of four, but an unknown > number of sets of four. What would be the proper way to only do a set of 4, stop, then do another set of 4? I'm trying to only 4 files at time before doing another set of 4. From gregchagnon at gmail.com Tue Jan 19 11:33:38 2010 From: gregchagnon at gmail.com (SoxFan44) Date: Tue, 19 Jan 2010 08:33:38 -0800 (PST) Subject: Create list/dict from string Message-ID: <2956b67c-c4b7-4757-bda8-48e8f799d4df@e16g2000yqc.googlegroups.com> I was wondering if there was a way to create a list (which in this case would contain several dicts) based on a string passed in by the user. Security is not an issue. Basically I want to be able to have the user pass in using optparse: --actions=[{"action_name": "action_1", "val": "asdf", "val2": "asdf"}, {"action_name": "action_2", "val": "asdf", "val2": "asdf"}, {"action_name": "action_1", "val": "asdf", "val2": "asdf"}] And have this create a list/dict. I'm aware of pickle, but it won't work as far as I can tell. Thanks. From pruebauno at latinmail.com Tue Jan 19 11:44:12 2010 From: pruebauno at latinmail.com (nn) Date: Tue, 19 Jan 2010 08:44:12 -0800 (PST) Subject: Arrrrgh! Another module broken References: Message-ID: <381e6798-f263-4db4-9aa3-0006578721eb@r5g2000yqb.googlegroups.com> On Jan 18, 11:37?am, Grant Edwards wrote: > On 2010-01-18, Jive Dadson wrote: > > > I just found another module that broke when I went to 2.6. ?Gnuplot. > > Apparently one of its routines has a parameter named "with." ?That used > > to be okay, and now it's not. > > I remember seeing depreicated warnings about that _years_ ago, > and I would have sworn it's been fixed for at least a couple > years. > > -- > Grant FWIW, "with" deprecation warnings exist since September 19, 2006 when Python 2.5 was released. From rbrt.somerville at gmail.com Tue Jan 19 11:48:23 2010 From: rbrt.somerville at gmail.com (robert somerville) Date: Tue, 19 Jan 2010 08:48:23 -0800 Subject: how to sort two dimensional array Message-ID: <2fb4a5011001190848mf8af3fcib7753783b223dc97@mail.gmail.com> Hi; i am having trouble trying to sort the rows of a 2 dimensional array by the values in the first column .. does anybody know how or have an example of how to do this ??? while leaving the remain columns remain relative to the leading column from numpy import * a=array( [ [4, 4, 3], [4, 5, 2], [3, 1, 1] ] ) i would like to generate the output (or get the output ...) b = [ [3,1,1], [4,4,3], [4,5,2] ] thanks; bob -------------- next part -------------- An HTML attachment was scrubbed... URL: From rbrt.somerville at gmail.com Tue Jan 19 11:50:21 2010 From: rbrt.somerville at gmail.com (robert somerville) Date: Tue, 19 Jan 2010 08:50:21 -0800 Subject: how to sort two dimensional array ?? Message-ID: <2fb4a5011001190850u58296a82yabb73af8b1bd21ab@mail.gmail.com> Hi; i am having trouble trying to sort the rows of a 2 dimensional array by the values in the first column .. does anybody know how or have an example of how to do this ??? while leaving the remain columns remain relative to the leading column from numpy import * a=array( [ [4, 4, 3], [4, 5, 2], [3, 1, 1] ] ) i would like to generate the output (or get the output ...) b = [ [3,1,1], [4,4,3], [4,5,2] ] thanks; bob -------------- next part -------------- An HTML attachment was scrubbed... URL: From simon at brunningonline.net Tue Jan 19 11:52:56 2010 From: simon at brunningonline.net (Simon Brunning) Date: Tue, 19 Jan 2010 16:52:56 +0000 Subject: use of super In-Reply-To: <8fbe9650-abee-4f0a-93f2-040fc8697e82@u41g2000yqe.googlegroups.com> References: <8fbe9650-abee-4f0a-93f2-040fc8697e82@u41g2000yqe.googlegroups.com> Message-ID: <8c7f10c61001190852m58eb51e3ubeec14c9b9dcdd7b@mail.gmail.com> 2010/1/19 harryos : > I was going thru the weblog appln in practical django book by > bennet .I came across this > > class Entry(Model): > ? ? ? ?def save(self): > ? ? ? ? ? ? ? ?dosomething() > ? ? ? ? ? ? ? ?super(Entry,self).save() > > I couldn't make out why Entry and self are passed as arguments to super > ().Can someone please explain? Does make anything clearer? -- Cheers, Simon B. From __peter__ at web.de Tue Jan 19 11:57:46 2010 From: __peter__ at web.de (Peter Otten) Date: Tue, 19 Jan 2010 17:57:46 +0100 Subject: Create list/dict from string References: <2956b67c-c4b7-4757-bda8-48e8f799d4df@e16g2000yqc.googlegroups.com> Message-ID: SoxFan44 wrote: > I was wondering if there was a way to create a list (which in this > case would contain several dicts) based on a string passed in by the > user. Security is not an issue. Basically I want to be able to have > the user pass in using optparse: > --actions=[{"action_name": "action_1", "val": "asdf", "val2": > "asdf"}, > {"action_name": "action_2", "val": "asdf", "val2": > "asdf"}, > {"action_name": "action_1", "val": "asdf", "val2": > "asdf"}] > > And have this create a list/dict. I'm aware of pickle, but it won't > work as far as I can tell. Both eval() and json.loads() will do. eval() is dangerous as it allows the user to run arbitrary python code. Peter From phlip2005 at gmail.com Tue Jan 19 11:58:21 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 19 Jan 2010 08:58:21 -0800 (PST) Subject: Is HTML report of tests run using PyUnit (unittest) possible? References: <584f31d6-4c4d-4661-836f-c8552d892d38@j19g2000yqk.googlegroups.com> Message-ID: <8d5070ee-1ec2-4f55-b56e-d841fb023cb9@j19g2000yqk.googlegroups.com> On Jan 19, 3:16?am, fossist wrote: > I am using PyUnit (unittest module) for loading test cases from our > modules and executing them. Is it possible to export the test results > as HTML report? Currently the results appear as text on standard > output while the tests execute. But is there something out of the box > available in PyUnit to make this possible? django-test-extensions can do this, but I'm unaware how well it works without Django. (And I _could_ complain that it comes with one or two irritations _with_ Django;) I would download it and read its source to see how the --xml option works. (Then you'd use a XSL filter to rip the XML into HTML...) -- Phlip http://c2.com/cgi/wiki?ZeekLand From rsomerville at sjgeophysics.com Tue Jan 19 12:00:26 2010 From: rsomerville at sjgeophysics.com (Robert Somerville) Date: Tue, 19 Jan 2010 09:00:26 -0800 Subject: how to sort two dimensional array ?? Message-ID: <4B55E52A.9000900@sjgeophysics.com> Hi; i am having trouble trying to sort the rows of a 2 dimensional array by the values in the first column .. does anybody know how or have an example of how to do this ??? while leaving the remain columns remain relative to the leading column from numpy import * a=array( [ [4, 4, 3], [4, 5, 2], [3, 1, 1] ] ) i would like to generate the output (or get the output ...) b = [ [3,1,1], [4,4,3], [4,5,2] ] thanks; bob From briandenzer at gmail.com Tue Jan 19 12:14:29 2010 From: briandenzer at gmail.com (Brian D) Date: Tue, 19 Jan 2010 09:14:29 -0800 (PST) Subject: Iterate over group names in a regex match? Message-ID: Here's a simple named group matching pattern: >>> s = "1,2,3" >>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") >>> m = re.match(p, s) >>> m <_sre.SRE_Match object at 0x011BE610> >>> print m.groups() ('1', '2', '3') Is it possible to call the group names, so that I can iterate over them? The result I'm looking for would be: ('one', 'two', 'three') From phlip2005 at gmail.com Tue Jan 19 12:19:55 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 19 Jan 2010 09:19:55 -0800 (PST) Subject: Python IDE for MacOS-X References: Message-ID: <6b836c4b-0297-44d1-bcad-f3423925a19c@34g2000yqp.googlegroups.com> On Jan 18, 11:09?pm, Jean Guillaume Pyraksos wrote: > What's the best one to use with beginners ? > Something with integrated syntax editor, browser of doc... > Thanks, Before this message goes stale, there's TextMate (which I have too much experience with to consider redeemable in any way)... ...and there's Komodo Edit. The problems I have with that are... - the code browsing breaks when the wind blows, and you must Find in Files (my library, Django, is symlinked below my project, so FiF can see it) - the editor refuses to let me run a script - such as a test script - each time I hit F5. For whatever reason - poor keyboard remapping, or lack of a plug-in for Django - I must use Ctrl+R to run a batch file, just to test. Testing should be ONE (1) keystroke. - FiF sees my .git folder, and everything in it. WTF?? - after searching or anything, I must click the editor with the mouse to resume editing. should move the focus out of the frame to the editor, and of course any activity that takes you out of the editor should dump you back into it by default Other than that, it has all the standard edito My problem with Mac in general is the keystrokes are always so broken they force me to switch to a mouse. Nobody seems to usability test these things and see how long you can type & manipulate windows without From slais-www at ucl.ac.uk Tue Jan 19 12:23:32 2010 From: slais-www at ucl.ac.uk (djc) Date: Tue, 19 Jan 2010 17:23:32 +0000 Subject: Iterate over group names in a regex match? In-Reply-To: References: Message-ID: Brian D wrote: > Here's a simple named group matching pattern: > >>>> s = "1,2,3" >>>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") >>>> m = re.match(p, s) >>>> m > <_sre.SRE_Match object at 0x011BE610> >>>> print m.groups() > ('1', '2', '3') > > Is it possible to call the group names, so that I can iterate over > them? > > The result I'm looking for would be: > > ('one', 'two', 'three') print(m.groupdict()) {'one': '1', 'three': '3', 'two': '2'} >>> print(m.groupdict().keys()) ['one', 'three', 'two'] -- David Clark, MSc, PhD. Dept of Information Studies Systems & Web Development Manager University College London UCL Centre for Publishing Gower Str London WCIE 6BT From oswald.harry at gmail.com Tue Jan 19 12:26:38 2010 From: oswald.harry at gmail.com (harryos) Date: Tue, 19 Jan 2010 09:26:38 -0800 (PST) Subject: use of super References: <8fbe9650-abee-4f0a-93f2-040fc8697e82@u41g2000yqe.googlegroups.com> Message-ID: <584b87a7-bc72-4a1c-8846-79c37fd1baf3@m25g2000yqc.googlegroups.com> thanks Simon..I should have checked it From gerald.britton at gmail.com Tue Jan 19 12:27:13 2010 From: gerald.britton at gmail.com (Gerald Britton) Date: Tue, 19 Jan 2010 12:27:13 -0500 Subject: Create list/dict from string In-Reply-To: <2956b67c-c4b7-4757-bda8-48e8f799d4df@e16g2000yqc.googlegroups.com> References: <2956b67c-c4b7-4757-bda8-48e8f799d4df@e16g2000yqc.googlegroups.com> Message-ID: <5d1a32001001190927n1fd0916cs78bb31d54010bf34@mail.gmail.com> Can't you just use the dict() built-in function like this: dict({"action_name": "action_1", "val": "asdf"}) Of course if the list is not properly formed, this will fail. But I guess you have thought of that already. On Tue, Jan 19, 2010 at 11:33 AM, SoxFan44 wrote: > I was wondering if there was a way to create a list (which in this > case would contain several dicts) based on a string passed in by the > user. ?Security is not an issue. ?Basically I want to be able to have > the user pass in using optparse: > --actions=[{"action_name": "action_1", "val": "asdf", "val2": > "asdf"}, > ? ? ? ? ? ? ? {"action_name": "action_2", "val": "asdf", "val2": > "asdf"}, > ? ? ? ? ? ? ? {"action_name": "action_1", "val": "asdf", "val2": > "asdf"}] > > And have this create a list/dict. ?I'm aware of pickle, but it won't > work as far as I can tell. > > Thanks. > -- > http://mail.python.org/mailman/listinfo/python-list > -- Gerald Britton From simon at brunningonline.net Tue Jan 19 12:27:33 2010 From: simon at brunningonline.net (Simon Brunning) Date: Tue, 19 Jan 2010 17:27:33 +0000 Subject: Create list/dict from string In-Reply-To: References: <2956b67c-c4b7-4757-bda8-48e8f799d4df@e16g2000yqc.googlegroups.com> Message-ID: <8c7f10c61001190927r2b973817t41e134c7b8a2ca09@mail.gmail.com> 2010/1/19 Peter Otten <__peter__ at web.de>: > Both eval() and json.loads() will do. eval() is dangerous as it allows the > user to run arbitrary python code. Something like might be worth a look too. -- Cheers, Simon B. From __peter__ at web.de Tue Jan 19 12:28:11 2010 From: __peter__ at web.de (Peter Otten) Date: Tue, 19 Jan 2010 18:28:11 +0100 Subject: Iterate over group names in a regex match? References: Message-ID: Brian D wrote: > Here's a simple named group matching pattern: > >>>> s = "1,2,3" >>>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") >>>> m = re.match(p, s) >>>> m > <_sre.SRE_Match object at 0x011BE610> >>>> print m.groups() > ('1', '2', '3') > > Is it possible to call the group names, so that I can iterate over > them? > > The result I'm looking for would be: > > ('one', 'two', 'three') >>> s = "1,2,3" >>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") >>> m = re.match(p, s) >>> dir(m) ['__copy__', '__deepcopy__', 'end', 'expand', 'group', 'groupdict', 'groups', 'span', 'start'] >>> m.groupdict().keys() ['one', 'three', 'two'] >>> sorted(m.groupdict(), key=m.span) ['one', 'two', 'three'] Peter From alfps at start.no Tue Jan 19 12:28:20 2010 From: alfps at start.no (Alf P. Steinbach) Date: Tue, 19 Jan 2010 18:28:20 +0100 Subject: Iterate over group names in a regex match? In-Reply-To: References: Message-ID: * Brian D: > Here's a simple named group matching pattern: > >>>> s = "1,2,3" >>>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") >>>> m = re.match(p, s) >>>> m > <_sre.SRE_Match object at 0x011BE610> >>>> print m.groups() > ('1', '2', '3') > > Is it possible to call the group names, so that I can iterate over > them? > > The result I'm looking for would be: > > ('one', 'two', 'three') I never used that beast (I'm in a sense pretty new to Python, although starting some months back I only investigate what's needed for my writings), but checking things in the interpreter: >>> import re >>> re >>> s = "1,2,3" >>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") >>> m = re.match(p, s) >>> m <_sre.SRE_Match object at 0x01319F70> >>> m.groups() ('1', '2', '3') >>> type( m.groups() ) >>> dir( m ) ['__copy__', '__deepcopy__', 'end', 'expand', 'group', 'groupdict', 'groups', 'span', 'start'] >>> m.groupdict >>> m.groupdict() {'one': '1', 'three': '3', 'two': '2'} >>> print( tuple( m.groupdict().keys() ) ) ('one', 'three', 'two') >>> _ Cheers & hth., - Alf From pych3m4 at gmail.com Tue Jan 19 12:36:27 2010 From: pych3m4 at gmail.com (Chema Cortes) Date: Tue, 19 Jan 2010 18:36:27 +0100 Subject: [ANN] Python-es mailing list changes home Message-ID: <2c9fb0dd1001190936q3116f0a3k221f55485daf685f@mail.gmail.com> === Python-es mailing list changes home === Due to technical problems with the site that usually ran the Python-es mailing list (Python list for the Spanish speaking community), we are setting up a new one under the python.org umbrella. Hence, the new list will become (the old one was ). Please feel free to subscribe to the new list in: http://mail.python.org/mailman/listinfo/python-es Thanks! === La lista de distribuci?n Python-es cambia de lugar === Debido a problemas t?cnicos con el sitio que normalmente albergaba la lista de Python-es (Lista de Python para la comunidad hispano-hablante), estamos configurando una nueva en el sitio python.org. As? que la nueva lista ser? (en sustituci?n de la antigua ). Por favor, si lo deseas, date de alta en la nueva lista en: http://mail.python.org/mailman/listinfo/python-es ?Gracias! Chema Cortes, Oswaldo Hern?ndez y Francesc Alted From robert.kern at gmail.com Tue Jan 19 12:39:09 2010 From: robert.kern at gmail.com (Robert Kern) Date: Tue, 19 Jan 2010 11:39:09 -0600 Subject: how to sort two dimensional array ?? In-Reply-To: <4B55E52A.9000900@sjgeophysics.com> References: <4B55E52A.9000900@sjgeophysics.com> Message-ID: On 2010-01-19 11:00 AM, Robert Somerville wrote: > Hi; > i am having trouble trying to sort the rows of a 2 dimensional array by > the values in the first column .. does anybody know how or have an > example of how to do this ??? while leaving the remain columns remain > relative to the leading column You will want to ask numpy questions on the numpy mailing list: http://www.scipy.org/Mailing_Lists > from numpy import * > > a=array( [ [4, 4, 3], [4, 5, 2], [3, 1, 1] ] ) > > i would like to generate the output (or get the output ...) > > b = [ [3,1,1], [4,4,3], [4,5,2] ] In [4]: import numpy as np In [5]: a = np.array( [ [4, 4, 3], [4, 5, 2], [3, 1, 1] ] ) In [6]: i = a[:,0].argsort() In [7]: b = a[i] In [8]: b Out[8]: array([[3, 1, 1], [4, 4, 3], [4, 5, 2]]) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From apt.shansen at gmail.com Tue Jan 19 12:39:53 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Tue, 19 Jan 2010 09:39:53 -0800 Subject: how to sort two dimensional array ?? In-Reply-To: <4B55E52A.9000900@sjgeophysics.com> References: <4B55E52A.9000900@sjgeophysics.com> Message-ID: <7a9c25c21001190939x187d685egae92ee1367272c76@mail.gmail.com> On Tue, Jan 19, 2010 at 9:00 AM, Robert Somerville < rsomerville at sjgeophysics.com> wrote: > Hi; > Hi, why did you post this three times? > i am having trouble trying to sort the rows of a 2 dimensional array by the > values in the first column .. does anybody know how or have an example of > how to do this ??? while leaving the remain columns remain relative to the > leading column > > from numpy import * > > a=array( [ [4, 4, 3], [4, 5, 2], [3, 1, 1] ] ) > > i would like to generate the output (or get the output ...) > > b = [ [3,1,1], [4,4,3], [4,5,2] ] > I don't use numpy, so this may or may not work. But for a regular python list-of-lists (2 dimensional array), you simply do: a.sort(key=lambda x: x[0]) --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From briandenzer at gmail.com Tue Jan 19 12:51:54 2010 From: briandenzer at gmail.com (Brian D) Date: Tue, 19 Jan 2010 09:51:54 -0800 (PST) Subject: Iterate over group names in a regex match? References: Message-ID: On Jan 19, 11:28?am, Peter Otten <__pete... at web.de> wrote: > Brian D wrote: > > Here's a simple named group matching pattern: > > >>>> s = "1,2,3" > >>>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") > >>>> m = re.match(p, s) > >>>> m > > <_sre.SRE_Match object at 0x011BE610> > >>>> print m.groups() > > ('1', '2', '3') > > > Is it possible to call the group names, so that I can iterate over > > them? > > > The result I'm looking for would be: > > > ('one', 'two', 'three') > >>> s = "1,2,3" > >>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") > >>> m = re.match(p, s) > >>> dir(m) > > ['__copy__', '__deepcopy__', 'end', 'expand', 'group', 'groupdict', > 'groups', 'span', 'start']>>> m.groupdict().keys() > > ['one', 'three', 'two']>>> sorted(m.groupdict(), key=m.span) > > ['one', 'two', 'three'] > > Peter groupdict() does it. I've never seen it used before. Very cool! Thank you all for taking time to answer the question. From tim.arnold at sas.com Tue Jan 19 12:53:19 2010 From: tim.arnold at sas.com (Tim Arnold) Date: Tue, 19 Jan 2010 12:53:19 -0500 Subject: Python IDE for MacOS-X References: Message-ID: "Jean Guillaume Pyraksos" wrote in message news:wissme-9248E1.08090319012010 at news.free.fr... > What's the best one to use with beginners ? > Something with integrated syntax editor, browser of doc... > Thanks, > > JG eclipse + pydev works well for me. --Tim Arnold From briandenzer at gmail.com Tue Jan 19 13:01:44 2010 From: briandenzer at gmail.com (Brian D) Date: Tue, 19 Jan 2010 10:01:44 -0800 (PST) Subject: Iterate over group names in a regex match? References: Message-ID: <5067f1f2-863f-4187-8b81-bdebe7a92df3@p24g2000yqm.googlegroups.com> On Jan 19, 11:51?am, Brian D wrote: > On Jan 19, 11:28?am, Peter Otten <__pete... at web.de> wrote: > > > > > Brian D wrote: > > > Here's a simple named group matching pattern: > > > >>>> s = "1,2,3" > > >>>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") > > >>>> m = re.match(p, s) > > >>>> m > > > <_sre.SRE_Match object at 0x011BE610> > > >>>> print m.groups() > > > ('1', '2', '3') > > > > Is it possible to call the group names, so that I can iterate over > > > them? > > > > The result I'm looking for would be: > > > > ('one', 'two', 'three') > > >>> s = "1,2,3" > > >>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") > > >>> m = re.match(p, s) > > >>> dir(m) > > > ['__copy__', '__deepcopy__', 'end', 'expand', 'group', 'groupdict', > > 'groups', 'span', 'start']>>> m.groupdict().keys() > > > ['one', 'three', 'two']>>> sorted(m.groupdict(), key=m.span) > > > ['one', 'two', 'three'] > > > Peter > > groupdict() does it. I've never seen it used before. Very cool! > > Thank you all for taking time to answer the question. FYI, here's an example of the working result ... >>> for k, v in m.groupdict().iteritems(): k, v ('one', '1') ('three', '3') ('two', '2') The use for this is that I'm pulling data from a flat text file using regex, and storing values in a dictionary that will be used to update a database. From python at mrabarnett.plus.com Tue Jan 19 13:20:42 2010 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 19 Jan 2010 18:20:42 +0000 Subject: Iterate over group names in a regex match? In-Reply-To: References: Message-ID: <4B55F7FA.4080003@mrabarnett.plus.com> Brian D wrote: > Here's a simple named group matching pattern: > >>>> s = "1,2,3" >>>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") >>>> m = re.match(p, s) >>>> m > <_sre.SRE_Match object at 0x011BE610> >>>> print m.groups() > ('1', '2', '3') > > Is it possible to call the group names, so that I can iterate over > them? > > The result I'm looking for would be: > > ('one', 'two', 'three') > The closest you can get is with groupdict(): >>> print m.groupdict() {'one': '1', 'three': '3', 'two': '2'} From lordkrandel at gmail.com Tue Jan 19 13:42:02 2010 From: lordkrandel at gmail.com (Wyrmskull) Date: Tue, 19 Jan 2010 10:42:02 -0800 (PST) Subject: substitution In-Reply-To: References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> Message-ID: <4b55fcfa.0d0db80a.1657.fffff15d@mx.google.com> Peter Otten wrote: def replace_many(s, pairs): if len(pairs): a, b = pairs[0] rest = pairs[1:] return b.join(replace_many(t, rest) for t in s.split(a)) else: return s ------------- Proves wrong, this way x -> y -> z. You call replace_many again on the central part of the split Specifics indicate that x -> y in the end. Your flowing pythonicity (if len(x):) gave me lots of inspiration. I bet this will win the prize ;) ------------- def mySubst(reps,string): if not(len(reps)): return string a,b,c = string.partition(reps[0][0]) if b: return mySubst(reps,a) + reps[0][1] + mySubst (reps,c) else: return mySubst(reps[1:],string) print mySubst( ( ('foo','bar'), ('bar','qux'), ('qux','foo') ), 'foobarquxfoo') ------- Wyrmskull From __peter__ at web.de Tue Jan 19 13:49:40 2010 From: __peter__ at web.de (Peter Otten) Date: Tue, 19 Jan 2010 19:49:40 +0100 Subject: substitution References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> Message-ID: Wyrmskull wrote: > Peter Otten wrote: >> def replace_many(s, pairs): >> if len(pairs): >> a, b = pairs[0] >> rest = pairs[1:] >> return b.join(replace_many(t, rest) for t in s.split(a)) >> else: >> return s > Proves wrong, this way x -> y -> z. > You call replace_many again on the central part of the split > Specifics indicate that x -> y in the end. Sorry, I don't understand what you want to say with the above. > Try with this: > > def mySubst(reps,string): > if not(len(reps)): > return string > current = reps[0][0] > a,b,c = string.partition(current) > if b: > return mySubst(reps,a) + reps[0][1] + mySubst (reps,c) > else: > return mySubst(reps[1:],string) > > print mySubst( ( ('foo','bar'), ('bar','qux'), ('qux','foo') ), > 'foobarquxfoo') > > ------- > Wyrmskull I don't see at first glance where the results of replace_many() and mySubst() differ. Perhaps you could give an example? Peter PS: Please keep the conversation on-list From lordkrandel at gmail.com Tue Jan 19 13:56:41 2010 From: lordkrandel at gmail.com (Wyrmskull) Date: Tue, 19 Jan 2010 10:56:41 -0800 (PST) Subject: substitution In-Reply-To: <4b55fcfa.0d0db80a.1657.fffff15d@mx.google.com> References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> <4b55fcfa.0d0db80a.1657.fffff15d@mx.google.com> Message-ID: <4b560069.100db80a.249f.ffffc009@mx.google.com> Cleaned. You can remove the 'else's if you want, because 'return' breaks the instruction flow. Should also work with other sequence types. ---------------- def mySubst(reps,seq): if reps: a,b,c = string.partition(reps[0][0]) if b: return mySubst(reps,a) + reps[0][1] + mySubst (reps,c) else: return mySubst(reps[1:], seq) else: return seq print mySubst( ( ('foo','bar'), ('bar','qux'), ('qux','foo') ), 'foobarquxfoo') print mySubst( ( ('foo','bar'), ('bar','qux'), ('qux','foo') ), 'foobarquxxxfoo') ------- Wyrmskull From __peter__ at web.de Tue Jan 19 13:58:24 2010 From: __peter__ at web.de (Peter Otten) Date: Tue, 19 Jan 2010 19:58:24 +0100 Subject: substitution References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> Message-ID: Wyrmskull wrote: > Your flowing pythonicity (if len(x):) gave me lots of inspiration. Actually, instead of if len(pairs): ... that should have been just if pairs: ... When I started writing the function I initially wanted to special-case len(pairs) == 1. The len() call is a superfluous leftover. Peter From lordkrandel at gmail.com Tue Jan 19 14:06:38 2010 From: lordkrandel at gmail.com (Wyrmskull) Date: Tue, 19 Jan 2010 11:06:38 -0800 (PST) Subject: substitution In-Reply-To: References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> Message-ID: <4b5602be.16125e0a.1a2d.0c2d@mx.google.com> Nvm, my bad, I misunderstood the split instruction. No difference :) ------- Wyrmskull P.S Sorry about the P.M., I misclicked on a GUI From gd_usenet at spamfence.net Tue Jan 19 14:10:56 2010 From: gd_usenet at spamfence.net (=?UTF-8?Q?G=C3=BCnther?= Dietrich) Date: Tue, 19 Jan 2010 20:10:56 +0100 Subject: Python IDE for MacOS-X References: Message-ID: <0nch27-m2t.ln1@spamfence.net> Jean Guillaume Pyraksos wrote: >What's the best one to use with beginners ? >Something with integrated syntax editor, browser of doc... >Thanks, I started with 'EasyEclipse for Python', but soon changed to Eclipse with PyDev, MercurialEclipse and AnyEditTools plugins installed manually. I can recommend the combination Eclipse/PyDev. Best regards, G?nther From sccolbert at gmail.com Tue Jan 19 14:27:28 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Tue, 19 Jan 2010 14:27:28 -0500 Subject: Python IDE for MacOS-X In-Reply-To: References: Message-ID: <7f014ea61001191127h6af98916l1b4ed7f6d9bb703c@mail.gmail.com> On Tue, Jan 19, 2010 at 2:09 AM, Jean Guillaume Pyraksos wrote: > What's the best one to use with beginners ? > Something with integrated syntax editor, browser of doc... > Thanks, > > JG > -- > http://mail.python.org/mailman/listinfo/python-list > I whole-heartedly recommend WingIDE. It's commercial and the only piece of commercial Linux software I use, but it is worth every penny. And support emails are answered within hours, if not minutes...they are great guys over there. **I am not affiliated with Wingware in any way. Just a happy customer.** -------------- next part -------------- An HTML attachment was scrubbed... URL: From gnarlodious at gmail.com Tue Jan 19 14:29:00 2010 From: gnarlodious at gmail.com (Gnarlodious) Date: Tue, 19 Jan 2010 11:29:00 -0800 (PST) Subject: Py 3: Terminal script can't find relative path References: <58e9b1e4-049a-4a7f-bea9-8d7e7e9f191a@e27g2000yqd.googlegroups.com> <87d417578z.fsf@castleamber.com> <9541dcf0-8060-4851-bdbd-0aed6de13bf9@v25g2000yqk.googlegroups.com> <317ebb29-8c20-4fe5-82c4-b2fe05efef38@h9g2000yqa.googlegroups.com> Message-ID: <98c70d6f-5872-436b-b160-01b39d17d270@a32g2000yqm.googlegroups.com> OK I guess that is normal, I fixed it with this: path=os.path.dirname(__file__)+"/Data/" -- Gnarlie From wolfram.hinderer at googlemail.com Tue Jan 19 14:29:15 2010 From: wolfram.hinderer at googlemail.com (Wolfram Hinderer) Date: Tue, 19 Jan 2010 11:29:15 -0800 (PST) Subject: Performance of lists vs. list comprehensions References: Message-ID: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> On 19 Jan., 16:30, Gerald Britton wrote: > >>> Timer("' '.join([x for x in l])", 'l = map(str,range(10))').timeit() > > 2.9967339038848877 > > >>> Timer("' '.join(x for x in l)", 'l = map(str,range(10))').timeit() > > 7.2045478820800781 [...] > 2. Why should the "pure" list comprehension be slower than the same > comprehension enclosed in '[...]' ? Others have already commented on "generator expression" vs. "list comprehension". I'll try to shed some light on the cause of the slowness. For me it's >>> Timer("' '.join([x for x in l])", 'l = map(str,range(10))').timeit() 0.813948839866498 >>> Timer("' '.join(x for x in l)", 'l = map(str,range(10))').timeit() 2.226825476422391 But wait! I'm on Python 3.1 and the setup statement has to be changed to make this test meaningful. >>> Timer("' '.join([x for x in l])", 'l = list(map(str,range(10)))').timeit() 2.5788493369966545 >>> Timer("' '.join(x for x in l)", 'l = list(map(str,range(10)))').timeit() 3.7431774848480472 Much smaller factor now. But wait! If we want to test list comprehension against generator comprehension, we should try a function that just consumes the iterable. >>> setup = """l = list(map(str,range(10))) ... def f(a): ... for i in a: pass ... """ >>> Timer("f([x for x in l])", setup).timeit() 3.288511528699928 >>> Timer("f(x for x in l)", setup).timeit() 2.410873798206012 Oops! Iteration over generator expressions is not inherently more expension than iteration over list comprehensions. But certainly building a list from a generator expression is more expensive than a list comprehension? >>> Timer("[x for x in l]", 'l = list(map(str,range(10)))').timeit() 2.088602950933364 >>> Timer("list(x for x in l)", 'l = list(map(str,range(10)))').timeit() 3.691566805277944 Yes, list building from a generator expression *is* expensive. And join has to do it, because it has to iterate twice over the iterable passed in: once for calculating the memory needed for the joined string, and once more to actually do the join (this is implementation dependent, of course). If the iterable is a list already, the list building is not needed. From apt.shansen at gmail.com Tue Jan 19 14:56:13 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Tue, 19 Jan 2010 11:56:13 -0800 Subject: Py 3: How to switch application to Unicode strings? In-Reply-To: <92288825-8b01-47fb-8592-f65a7d7aeb8a@k35g2000yqb.googlegroups.com> References: <92288825-8b01-47fb-8592-f65a7d7aeb8a@k35g2000yqb.googlegroups.com> Message-ID: <7a9c25c21001191156j46a7fdadt58b728477b85e651@mail.gmail.com> On Tue, Jan 19, 2010 at 7:50 AM, Gnarlodious wrote: > I am using Python 3, getting an error from SQLite: > > sqlite3.ProgrammingError: You must not use 8-bit bytestrings unless > you use a text_factory that can interpret 8-bit bytestrings (like > text_factory = str). It is highly recommended that you instead just > switch your application to Unicode strings. > > So... how do I switch to Unicode? I thought I was doing it when I put # coding:utf-8 > > at the start of my script. > All that does is mean that the script itself is encoded as utf8. In Py3, anything you do "this" or use str(), you are using unicode strings. The problem appears to be that you are passing bytestrings to sqlite; things created with b"this" or bytes(), or read from a file as bytes and not decoded before passing it to the database. To really help further, you should provide the line that threw that warning and show where any variables in it come from. As for that error message, I believe it means text_factory = bytes. --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From gerald.britton at gmail.com Tue Jan 19 15:06:13 2010 From: gerald.britton at gmail.com (Gerald Britton) Date: Tue, 19 Jan 2010 15:06:13 -0500 Subject: Performance of lists vs. list comprehensions In-Reply-To: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> Message-ID: <5d1a32001001191206x3e2cc28ey6bc5c983fb739d67@mail.gmail.com> [snip] > > Yes, list building from a generator expression *is* expensive. And > join has to do it, because it has to iterate twice over the iterable > passed in: once for calculating the memory needed for the joined > string, and once more to actually do the join (this is implementation > dependent, of course). If the iterable is a list already, the list > building is not needed. if join has to iterate twice over the iterable, how does this work? $ python3.1 Python 3.1.1+ (r311:74480, Nov 2 2009, 14:49:22) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> l = map(str, (x for x in range(10) if int(x)%2)) >>> '.'.join(l) '1.3.5.7.9' >>> If join had to iterate twice over l, it would be consumed on the first pass. If it works as you say then join would have to copy the iterable on the first pass, effectively turning it into a list. Though I didn't read through it, I would suppose that join could use a dynamic-table approach to hold the result, starting with some guesstimate then expanding the result buffer if and when needed. > -- > http://mail.python.org/mailman/listinfo/python-list > -- Gerald Britton From steve at holdenweb.com Tue Jan 19 15:52:15 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 19 Jan 2010 15:52:15 -0500 Subject: enhancing 'list' In-Reply-To: <375ab6da-58c2-4975-bc79-22b2219a3216@r24g2000yqd.googlegroups.com> References: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> <375ab6da-58c2-4975-bc79-22b2219a3216@r24g2000yqd.googlegroups.com> Message-ID: samwyse wrote: > On Jan 18, 1:56 am, Terry Reedy wrote: >> On 1/17/2010 5:37 PM, samwyse wrote: >> >> >> >> >> >>> Consider this a wish list. I know I'm unlikely to get any of these in >>> time for for my birthday, but still I felt the need to toss it out and >>> see what happens. >>> Lately, I've slinging around a lot of lists, and there are some simple >>> things I'd like to do that just aren't there. >>> s.count(x[, cmp[, key]]) >>> - return number of i?s for which s[i] == x. 'cmp' specifies a custom >>> comparison function of two arguments, as in '.sort'. 'key' specifies >>> a custom key extraction function of one argument. >>> s.index(x[, i[, j[, cmp[, key]]]]) >>> - return smallest k such that s[k] == x and i<= k< j. 'cmp' and >>> 'key' are as above. >>> s.rindex(x[, i[, j[, cmp[, key]]]]) >>> - return largest k such that s[k] == x and i<= k< j. 'cmp' and >>> 'key' are as above. >>> There are two overlapping proposals here. One is to add the .rindex >>> method, which strings already have. The other is to extend the >>> optional arguments of .sort to all other methods that test for item >>> equality. >>> One last thing, the Python 2.6.2 spec says .count and .index only >>> apply to mutable sequence types. I see no reason why they >>> (and .rindex) couldn't also apply to immutable sequences (tuples, in >>> particular). >> In 3.x, tuple does have those methods, even though the doc is not clear >> (unless fixed by now). > > That's good to hear. Perhaps I should have tried them directyly, but > my 3.1 docs still echo the 2.x docs, which only show them for > immutable sequences. The tuple IS an immutable sequence. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From arnodel at googlemail.com Tue Jan 19 16:01:31 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Tue, 19 Jan 2010 21:01:31 +0000 Subject: Performance of lists vs. list comprehensions References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> Message-ID: Gerald Britton writes: > [snip] > >> >> Yes, list building from a generator expression *is* expensive. And >> join has to do it, because it has to iterate twice over the iterable >> passed in: once for calculating the memory needed for the joined >> string, and once more to actually do the join (this is implementation >> dependent, of course). If the iterable is a list already, the list >> building is not needed. > > if join has to iterate twice over the iterable, how does this work? > > $ python3.1 > Python 3.1.1+ (r311:74480, Nov 2 2009, 14:49:22) > [GCC 4.4.1] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> l = map(str, (x for x in range(10) if int(x)%2)) >>>> '.'.join(l) > '1.3.5.7.9' >>>> > > If join had to iterate twice over l, it would be consumed on the first > pass. If it works as you say then join would have to copy the > iterable on the first pass, effectively turning it into a list. > Though I didn't read through it, I would suppose that join could use a > dynamic-table approach to hold the result, starting with some > guesstimate then expanding the result buffer if and when needed. Looking at the source (py3k): PyObject * PyUnicode_Join(PyObject *separator, PyObject *seq) { [skip declarations] fseq = PySequence_Fast(seq, ""); if (fseq == NULL) { return NULL; } [code that works out the length of the joined string then allocates memory, then fills it] } Where PySequence_Fast(seq, "") returns seq if seq is already a tuple or a list and otherwise returns a new tuple built from the elements of seq. So no, it doesn't guess the size of the joined string and yes, it iterates twice over the "sequence" (I would have thought it should be called an iterable) by copying it into a tuple. -- Arnaud From wolfram.hinderer at googlemail.com Tue Jan 19 16:15:14 2010 From: wolfram.hinderer at googlemail.com (Wolfram Hinderer) Date: Tue, 19 Jan 2010 13:15:14 -0800 (PST) Subject: Performance of lists vs. list comprehensions References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> Message-ID: <57a7f03b-7d43-484d-8952-53bf5fc2774f@c29g2000yqd.googlegroups.com> On 19 Jan., 21:06, Gerald Britton wrote: > [snip] > > > > > Yes, list building from a generator expression *is* expensive. And > > join has to do it, because it has to iterate twice over the iterable > > passed in: once for calculating the memory needed for the joined > > string, and once more to actually do the join (this is implementation > > dependent, of course). If the iterable is a list already, the list > > building is not needed. > > if join has to iterate twice over the iterable, how does this work? > > $ python3.1 > Python 3.1.1+ (r311:74480, Nov ?2 2009, 14:49:22) > [GCC 4.4.1] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > > >>> l = map(str, (x for x in range(10) if int(x)%2)) > >>> '.'.join(l) > '1.3.5.7.9' > > If join had to iterate twice over l, it would be consumed on the first > pass. Yes. (Coincidentally, l is consumed in the first execution of the Timer ()-statement, which is why I had to add the call to list. Not to mention the xrange example of Stephen. But all this is not linked to what join does internally.) > If it works as you say then join would have to copy the > iterable on the first pass, effectively turning it into a list. Yes, that's what I'm saying above in the first two lines of what you quoted. I should have added somehting like "AFAIK CPython does it that way". > Though I didn't read through it, I would suppose that join could use a > dynamic-table approach to hold the result, starting with some > guesstimate then expanding the result buffer if and when needed. Probably. But that's not what happens. Try "".join("" for x in range(10**10) and watch join eating memory. From aahz at pythoncraft.com Tue Jan 19 16:15:35 2010 From: aahz at pythoncraft.com (Aahz) Date: 19 Jan 2010 13:15:35 -0800 Subject: integer and string compare, is that correct? References: Message-ID: In article , Peter Otten <__peter__ at web.de> wrote: > >The use cases for an order that works across types like int and str are >weak to non-existent. Implementing it was considered a mistake and has >been fixed in Python 3: That is not precisely correct from my POV. The primary use case for order that works across types is sorting lists of heterogeneous data. Many people relied on that feature; however, experience showed that it caused more bugs than it fixed. But that doesn't obviate the use of the feature. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." --Red Adair From gerald.britton at gmail.com Tue Jan 19 16:20:42 2010 From: gerald.britton at gmail.com (Gerald Britton) Date: Tue, 19 Jan 2010 16:20:42 -0500 Subject: Performance of lists vs. list comprehensions In-Reply-To: References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> Message-ID: <5d1a32001001191320m549a124bw7dd392fd036409a4@mail.gmail.com> That's surprising. I wouldn't implement it that way at all. I'd use a dynamically-expanding buffer as I suggested. That way you get a single pass and don't have to calculate anything before you begin. In the best case, you'd use half the memory (the result just fits in the buffer after its last expansion and no saved tuple). In the worst case, the memory use is about the same (you just expanded the buffer using a 2x expansion rule then you hit the last item). Still I suppose the author thought of that approach and rejected it for reasons I can't yet see. On Tue, Jan 19, 2010 at 4:01 PM, Arnaud Delobelle wrote: > Gerald Britton writes: > >> [snip] >> >>> >>> Yes, list building from a generator expression *is* expensive. And >>> join has to do it, because it has to iterate twice over the iterable >>> passed in: once for calculating the memory needed for the joined >>> string, and once more to actually do the join (this is implementation >>> dependent, of course). If the iterable is a list already, the list >>> building is not needed. >> >> if join has to iterate twice over the iterable, how does this work? >> >> $ python3.1 >> Python 3.1.1+ (r311:74480, Nov ?2 2009, 14:49:22) >> [GCC 4.4.1] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >>>>> l = map(str, (x for x in range(10) if int(x)%2)) >>>>> '.'.join(l) >> '1.3.5.7.9' >>>>> >> >> If join had to iterate twice over l, it would be consumed on the first >> pass. ?If it works as you say then join would have to copy the >> iterable on the first pass, effectively turning it into a list. >> Though I didn't read through it, I would suppose that join could use a >> dynamic-table approach to hold the result, starting with some >> guesstimate then expanding the result buffer if and when needed. > > Looking at the source (py3k): > > PyObject * > PyUnicode_Join(PyObject *separator, PyObject *seq) > { > ? ?[skip declarations] > > ? ?fseq = PySequence_Fast(seq, ""); > ? ?if (fseq == NULL) { > ? ? ? ?return NULL; > ? ?} > > ? ?[code that works out the length of the joined string then allocates > ? ? memory, then fills it] > } > > Where PySequence_Fast(seq, "") returns seq if seq is already a tuple or > a list and otherwise returns a new tuple built from the elements of seq. > > So no, it doesn't guess the size of the joined string and yes, it > iterates twice over the "sequence" (I would have thought it should be > called an iterable) by copying it into a tuple. > > -- > Arnaud > -- > http://mail.python.org/mailman/listinfo/python-list > -- Gerald Britton From jgardner at jonathangardner.net Tue Jan 19 16:22:02 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Tue, 19 Jan 2010 13:22:02 -0800 (PST) Subject: Recommended "new" way for config files References: <4B46075D.8080606@mycircuit.org> <4B460CA0.20800@sequans.com> <4B4625BC.6090201@mycircuit.org> <87iqbcus0p.fsf@benfinney.id.au> Message-ID: On Jan 8, 2:54?pm, Ben Finney wrote: > Chris Rebert writes: > > JSON is one option:http://docs.python.org/library/json.html > > YAML is another contender. > Compared to JSON, it is yet to gain as much mind-share, but even more > human-friendly and no less expressive. > > Here are some discussions of YAML that can help you evaluate it: > > ? ? > ? ? > ? ? > > YAML is far too complex to be useful. I played with it a while and found the syntax even more confusing than XML, which is quite a feat. From moekaveli at gmail.com Tue Jan 19 16:23:35 2010 From: moekaveli at gmail.com (Dr. Benjamin David Clarke) Date: Tue, 19 Jan 2010 13:23:35 -0800 (PST) Subject: Updating an OptionMenu every time the text file it reads from is updated (Tkinter) References: <45d53456-0673-4829-aa73-9c98feba7b64@s31g2000yqs.googlegroups.com> Message-ID: <6aae9aa4-1a5a-45ce-8c4c-bcacca977ba7@a13g2000vbf.googlegroups.com> On Jan 19, 7:00?am, Peter Otten <__pete... at web.de> wrote: > Dr. Benjamin David Clarke wrote: > > > I currently have a program that reads in values for an OptionMenu from > > a text file. I also have an option to add a line to that text file > > which corresponds to a new value for that OptionMenu. How can I make > > that OptionMenu update its values based on that text file without > > restarting the program? In other words, every time I add a value to > > the text file, I want the OptionMenu to immediately update to take > > note of this change. I'll provide code if needed. > > Inferred from looking into the Tkinter source code: > > # python 2.6 > import Tkinter as tk > > root = tk.Tk() > > var ?= tk.StringVar() > var.set("One") > > optionmenu = tk.OptionMenu(root, var, "One", "Two", "Three") > optionmenu.grid(row=0, column=1) > > def add_option(): > ? ? value = entry_add.get() > ? ? menu = optionmenu["menu"] > ? ? variable = var > ? ? command = None # what you passed as command argument to optionmenu > ? ? menu.add_command(label=value, > ? ? ? ? ? ? ? ? ? ? ?command=tk._setit(variable, value, command)) > > label_show = tk.Label(root, text="current value") > label_show.grid(row=1, column=0) > entry_show = tk.Entry(root, textvariable=var) > entry_show.grid(row=1, column=1) > > label_add = tk.Label(root, text="new option") > label_add.grid(row=2, column=0) > entry_add = tk.Entry(root) > entry_add.grid(row=2, column=1) > > button_add = tk.Button(root, text="add option", > ? ? ? ? ? ? ? ? ? ?command=add_option) > button_add.grid(row=2, column=2) > > root.mainloop() > > Peter The problem turned out to be fairly simple. I just had to get a little creative with nesting my functions and add or remove the option from the OptionMenu right after adding/removing to the text file. Thanks for pointing me in the right direction. From python at rcn.com Tue Jan 19 16:54:02 2010 From: python at rcn.com (Raymond Hettinger) Date: Tue, 19 Jan 2010 13:54:02 -0800 (PST) Subject: Performance of lists vs. list comprehensions References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> Message-ID: [Wolfram Hinderer] > Yes, list building from a generator expression *is* expensive. And > join has to do it, because it has to iterate twice over the iterable > passed in: once for calculating the memory needed for the joined > string, and once more to actually do the join (this is implementation > dependent, of course). If the iterable is a list already, the list > building is not needed. Good analysis. That is exactly what is going on under the hood. Raymond From arnodel at googlemail.com Tue Jan 19 16:55:49 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Tue, 19 Jan 2010 21:55:49 +0000 Subject: Performance of lists vs. list comprehensions References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> Message-ID: Gerald Britton writes: > That's surprising. I wouldn't implement it that way at all. I'd use a > dynamically-expanding buffer as I suggested. That way you get a > single pass and don't have to calculate anything before you begin. In > the best case, you'd use half the memory (the result just fits in the > buffer after its last expansion and no saved tuple). In the worst > case, the memory use is about the same (you just expanded the buffer > using a 2x expansion rule then you hit the last item). > > Still I suppose the author thought of that approach and rejected it > for reasons I can't yet see. I don't know the reasons, but I'm guessing they could be historic. Before Python had iterators, str.join would mostly have been only given lists and tuples as arguments, in which case the current approach seems to be the most appropriate. Later, when things like generator functions and generator expressions were introduced, perhaps str.join wasn't optimized to accomodate them. -- Arnaud From martin at v.loewis.de Tue Jan 19 17:05:36 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Tue, 19 Jan 2010 23:05:36 +0100 Subject: can i examine the svn rev used to build a python 3 executable? In-Reply-To: References: Message-ID: <4B562CB0.9040204@v.loewis.de> > never mind. just discovered that while "python3 -V" won't do it, > executing it gives me: > > $ python3 > Python 3.2a0 (py3k:77609, Jan 19 2010, 04:10:16) > ... > > and it's that 77609 rev number i was after. If you want that in a command line fashion, do python -c 'import sys;print(sys.subversion[2])' Regards, Martin From peter.milliken at gmail.com Tue Jan 19 17:07:46 2010 From: peter.milliken at gmail.com (Peter) Date: Tue, 19 Jan 2010 14:07:46 -0800 (PST) Subject: thread return code References: <08974f1d-4334-4165-8272-7c178f589bbc@m16g2000yqc.googlegroups.com> Message-ID: On Jan 19, 9:25?pm, "Alf P. Steinbach" wrote: > * Rajat: > > > Hi, > > > I'm using threading module in Python 2.6.4. I'm using thread's join() > > method. > > > On the new thread I'm running a function which returns a code at the > > end. Is there a way I access that code in the parent thread after > > thread finishes? Simply, does join() could get me that code? > > join() always returns None. > > But you can store the code in the thread object, and then access it after the > join(). > > Cheers & hth., > > - Alf The typical way to communicate with a thread is via a queue or pipe. You can do what Alf suggests or you could create a queue (or pipe), pass it to the thread as an argument and have the thread put the "return value" into the queue as the last action prior to exit. After the join() just read the results from the queue. Using a queue or pipe is just a suggestion, the multiprocessing module offers numerous ways to communicate between tasks, have a read and pick whatever mechanism seems appropriate for your situation. Peter From fabiofz at gmail.com Tue Jan 19 18:47:51 2010 From: fabiofz at gmail.com (Fabio Zadrozny) Date: Tue, 19 Jan 2010 21:47:51 -0200 Subject: Pydev 1.5.4 Released Message-ID: Hi All, Pydev 1.5.4 has been released Details on Pydev: http://pydev.org Details on its development: http://pydev.blogspot.com Release Highlights: ------------------------------- * Actions: o Go to matching bracket (Ctrl + Shift + P) o Copy the qualified name of the current context to the clipboard. o Ctrl + Shift + T keybinding is resolved to show globals in any context (note: a conflict may occur if JDT is present -- it can be fixed at the keys preferences if wanted). o Ctrl + 2 shows a dialog with the list of available options. o Wrap paragraph is available in the source menu. o Globals browser will start with the current word if no selection is available (if possible). * Templates: o Scripting engine can be used to add template variables to Pydev. o New template variables for next, previous class or method, current module, etc. o New templates for super and super_raw. o print is now aware of Python 3.x or 2.x * Code analysis and code completion: o Fixed problem when getting builtins with multiple Python interpreters configured. o If there's a hasattr(obj, 'attr), 'attr' will be considered in the code completion and code analysis. o Fixed issue where analysis was only done once when set to only analyze open editor. o Proper namespace leakage semantic in list comprehension. o Better calltips in IronPython. o Support for code-completion in Mac OS (interpreter was crashing if _CF was not imported in the main thread). * Grammar: o Fixed issues with 'with' being used as name or keyword in 2.5. o Fixed error when using nested list comprehension. o Proper 'as' and 'with' handling in 2.4 and 2.5. o 'with' statement accepts multiple items in python 3.0. * Improved hover: o Showing the actual contents of method or class when hovering. o Link to the definition of the token being hovered (if class or method). * Others: o Completions for [{( are no longer duplicated when on block mode. o String substitution can now be configured in the interpreter. o Fixed synchronization issue that could make Pydev halt. o Fixed problem when editing with collapsed code. o Import wasn't found for auto-import location if it import started with 'import' (worked with 'from') o Fixed interactive console problem with help() function in Python 3.1 o NullPointerException fix in compare editor. What is Pydev? --------------------------- Pydev is a plugin that enables users to use Eclipse for Python, Jython and IronPython development -- making Eclipse a first class Python IDE -- It comes with many goodies such as code completion, syntax highlighting, syntax analysis, refactor, debug and many others. Cheers, -- Fabio Zadrozny ------------------------------------------------------ Software Developer Aptana http://aptana.com/python Pydev - Python Development Environment for Eclipse http://pydev.org http://pydev.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at REMOVE-THIS-cybersource.com.au Tue Jan 19 20:52:41 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 20 Jan 2010 01:52:41 GMT Subject: Performance of lists vs. list comprehensions References: <5d1a32001001190730u4fa2f26cn62e79c2c73a08b@mail.gmail.com> <7a9c25c21001190818g7bd0ca6n336e6b1d52af0b2c@mail.gmail.com> Message-ID: <00dfe6ee$0$15570$c3e8da3@news.astraweb.com> On Tue, 19 Jan 2010 11:26:43 -0500, Gerald Britton wrote: > Interestingly, I scaled it up to a million list items with more or less > the same results. A million items is not a lot of data. Depending on the size of each object, that might be as little as 4 MB of data: >>> L = ['' for _ in xrange(10**6)] >>> sys.getsizeof(L) 4348732 Try generating a billion items, or even a hundred million, and see how you go. This is a good lesson in the dangers of premature optimization. I can't think how many times I've written code using a generator expression passed to join, thinking that would surely be faster than using a list comprehension ("save building a temporary list first"). -- Steven From steve at REMOVE-THIS-cybersource.com.au Tue Jan 19 21:52:16 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 20 Jan 2010 02:52:16 GMT Subject: Performance of lists vs. list comprehensions References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> Message-ID: <00dff4e6$0$15570$c3e8da3@news.astraweb.com> On Tue, 19 Jan 2010 16:20:42 -0500, Gerald Britton wrote: > That's surprising. I wouldn't implement it that way at all. I'd use a > dynamically-expanding buffer as I suggested. That way you get a single > pass and don't have to calculate anything before you begin. In the best > case, you'd use half the memory (the result just fits in the buffer > after its last expansion and no saved tuple). In the worst case, the > memory use is about the same (you just expanded the buffer using a 2x > expansion rule then you hit the last item). In the worst case, you waste 50% of the memory allocated. And because strings are immutable (unlike lists and dicts, which also use this approach), you can never use that memory until the string is garbage collected. In the current approach, join produces a temporary sequence, but it doesn't last very long. With your suggested approach, you could end up with a large number of long-lasting strings up to twice the size necessary. Since join is particularly useful for building large strings, this could be a significant memory pessimation. The obvious fix is for join to shrink the buffer once it has finished building the string, but the success of that may be operating system dependent. I don't know -- it sounds like a recipe for memory fragmentation to me. And even if it can be done, reclaiming the memory will take time, potentially more time than the current approach. Still, it's probably worth trying, and seeing if it speeds join up. -- Steven From metolone+gmane at gmail.com Tue Jan 19 22:34:12 2010 From: metolone+gmane at gmail.com (Mark Tolonen) Date: Tue, 19 Jan 2010 19:34:12 -0800 Subject: Py 3: How to switch application to Unicode strings? References: <92288825-8b01-47fb-8592-f65a7d7aeb8a@k35g2000yqb.googlegroups.com> <7a9c25c21001191156j46a7fdadt58b728477b85e651@mail.gmail.com> Message-ID: "Stephen Hansen" wrote in message news:7a9c25c21001191156j46a7fdadt58b728477b85e651 at mail.gmail.com... > On Tue, Jan 19, 2010 at 7:50 AM, Gnarlodious > wrote: > >> I am using Python 3, getting an error from SQLite: >> >> sqlite3.ProgrammingError: You must not use 8-bit bytestrings unless >> you use a text_factory that can interpret 8-bit bytestrings (like >> text_factory = str). It is highly recommended that you instead just >> switch your application to Unicode strings. >> >> So... how do I switch to Unicode? I thought I was doing it when I put > > # coding:utf-8 >> >> at the start of my script. >> > > All that does is mean that the script itself is encoded as utf8. > Actually it means that the user has declared that the source file is encoded in utf-8. A common source of errors is that the source file is *not* encoded in utf-8. Make sure to save the source file in the encoding declared. -Mark From gnarlodious at gmail.com Tue Jan 19 23:16:40 2010 From: gnarlodious at gmail.com (Gnarlodious) Date: Tue, 19 Jan 2010 20:16:40 -0800 (PST) Subject: Py 3: How to switch application to Unicode strings? References: <92288825-8b01-47fb-8592-f65a7d7aeb8a@k35g2000yqb.googlegroups.com> <7a9c25c21001191156j46a7fdadt58b728477b85e651@mail.gmail.com> Message-ID: <646ab38b-0710-4d31-b9e1-8a6ee7bfa42f@21g2000yqj.googlegroups.com> Well, Python 3 is supposed to be all Unicode by default. I shouldn't even need to say # coding:UTF-8 And, the file is saved as Unicode. There are many mentions of this error found by Google, but none seen to clearly say what the problem is or how to fix it. FYI, the problem line says: cursor.execute('insert into Data values (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)', frameTuple) and one of the strings in the tuple contains a character like '?'. I have a version of the SQLite editor that works as expected in a browser, I don't know why. -- Gnarlie From alfps at start.no Tue Jan 19 23:25:22 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 20 Jan 2010 05:25:22 +0100 Subject: Performance of lists vs. list comprehensions In-Reply-To: <00dff4e6$0$15570$c3e8da3@news.astraweb.com> References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> <00dff4e6$0$15570$c3e8da3@news.astraweb.com> Message-ID: * Steven D'Aprano: > On Tue, 19 Jan 2010 16:20:42 -0500, Gerald Britton wrote: > >> That's surprising. I wouldn't implement it that way at all. I'd use a >> dynamically-expanding buffer as I suggested. That way you get a single >> pass and don't have to calculate anything before you begin. In the best >> case, you'd use half the memory (the result just fits in the buffer >> after its last expansion and no saved tuple). In the worst case, the >> memory use is about the same (you just expanded the buffer using a 2x >> expansion rule then you hit the last item). > > In the worst case, you waste 50% of the memory allocated. Yes. That is a good argument for not doing the expanding buffer thing. But such buffers may be generally present anyway, resulting from optimization of "+". Using CPython 2.6.4 in Windows XP: >>> def elapsed_time_for( f, n_calls ): ... return timeit.timeit( f, number = n_calls ) ... >>> def appender( n ): ... def makestr( n = n ): ... s = "" ... for i in xrange( n ): ... s = s + "A" ... return s ... return makestr ... >>> appender( 1000 )() == 1000*"A" True >>> >>> for i in xrange( 10 ): ... print( elapsed_time_for( appender( 10000*(i+1) ), 100 ) ) ... 0.782596670811 1.37728454314 2.10189898437 2.76442173517 3.34536707878 4.08251830889 4.79620119317 5.42201844089 6.12892811796 6.84236460221 >>> _ Here the linear increase of times indicate that the "+" is being optimized using an expanding buffer for the string. If only the minimal space was allocated each time then one would expect O(n^2) behavior instead of the apparently O(n) above. Example of that O(n^2) behavior given below. >>> def exact_appender( n ): ... def makestr( n = n ): ... s = "" ... for i in xrange( n ): ... new_s = s + "A" ... s = new_s ... return s ... return makestr ... >>> exact_appender( 1000 )() == 1000*"A" True >>> for i in xrange( 10 ): ... print( elapsed_time_for( exact_appender( 10000*(i+1) ), 100 ) ) ... 3.28094241027 9.30584501661 19.5319170453 33.6563767183 52.3327800042 66.5475022663 84.8809736992 Traceback (most recent call last): File "", line 2, in File "", line 2, in elapsed_time_for File "C:\Program Files\cpython\python26\lib\timeit.py", line 227, in timeit return Timer(stmt, setup, timer).timeit(number) File "C:\Program Files\cpython\python26\lib\timeit.py", line 193, in timeit timing = self.inner(it, self.timer) File "C:\Program Files\cpython\python26\lib\timeit.py", line 99, in inner _func() File "", line 5, in makestr KeyboardInterrupt >>> _ So, given that apparently the simple '+' in the first example is optimized using an expanding buffer, which then hangs around, it's not clear to me that the space optimization in 'join' really helps. It may be (but isn't necessarily) like shoveling snow in a snowstorm. Then the effort/cost could be for naught. > And because > strings are immutable (unlike lists and dicts, which also use this > approach), you can never use that memory until the string is garbage > collected. I think that the simple '+', with the apparent optimization shown in the first example above, can use that space. I know for a fact that when you control a string implementation then it can do that (since I've implemented that). But I don't know for a fact that it's practical to do so in Python. In order to use the space the implementation must know that there's only one reference to the string. And I don't know whether that information is readily available in a CPython implementation (say), although I suspect that it is. Cheers, - Alf From apt.shansen at gmail.com Tue Jan 19 23:45:11 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Tue, 19 Jan 2010 20:45:11 -0800 Subject: Py 3: How to switch application to Unicode strings? In-Reply-To: <646ab38b-0710-4d31-b9e1-8a6ee7bfa42f@21g2000yqj.googlegroups.com> References: <92288825-8b01-47fb-8592-f65a7d7aeb8a@k35g2000yqb.googlegroups.com> <7a9c25c21001191156j46a7fdadt58b728477b85e651@mail.gmail.com> <646ab38b-0710-4d31-b9e1-8a6ee7bfa42f@21g2000yqj.googlegroups.com> Message-ID: <7a9c25c21001192045t460629c0pb928c5abd24186f8@mail.gmail.com> On Tue, Jan 19, 2010 at 8:16 PM, Gnarlodious wrote: > Well, Python 3 is supposed to be all Unicode by default. I shouldn't > even need to say > # coding:UTF-8 > > And, the file is saved as Unicode. > > There are many mentions of this error found by Google, but none seen > to clearly say what the problem is or how to fix it. > > FYI, the problem line says: > > cursor.execute('insert into Data values > (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)', frameTuple) > > and one of the strings in the tuple contains a character like '?'. > I have a version of the SQLite editor that works as expected in a > browser, I don't know why. > > But is it a -unicode- string, or a -byte- string? Print it with repr(). By that error, it seems like its a bytestring. So you read it or got it from a source which provided it to you not as unicode. In that case, find out what the encoding is-- and decode it. after = before.decode("utf8") Python 3 is not 'all unicode'; or 'by default'. Python 3 has a firm line in the sand. Everything is either explicitly a byte string (bytes) or explicitly a unicode string (str). --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at REMOVE-THIS-cybersource.com.au Wed Jan 20 01:12:18 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 20 Jan 2010 06:12:18 GMT Subject: Performance of lists vs. list comprehensions References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> <00dff4e6$0$15570$c3e8da3@news.astraweb.com> Message-ID: <00e023c8$0$15570$c3e8da3@news.astraweb.com> On Wed, 20 Jan 2010 05:25:22 +0100, Alf P. Steinbach wrote: > * Steven D'Aprano: >> On Tue, 19 Jan 2010 16:20:42 -0500, Gerald Britton wrote: >> >>> That's surprising. I wouldn't implement it that way at all. I'd use a >>> dynamically-expanding buffer as I suggested. That way you get a >>> single pass and don't have to calculate anything before you begin. In >>> the best case, you'd use half the memory (the result just fits in the >>> buffer after its last expansion and no saved tuple). In the worst >>> case, the memory use is about the same (you just expanded the buffer >>> using a 2x expansion rule then you hit the last item). >> >> In the worst case, you waste 50% of the memory allocated. > > Yes. That is a good argument for not doing the expanding buffer thing. > But such buffers may be generally present anyway, resulting from > optimization of "+". As near as I can determine, the CPython optimization you are referring to doesn't use the "double the buffer when needed" technique. It operates on a completely different strategy. As near as I can tell (as a non-C speaker), it re-sizes the string in place to the size actually needed, thus reducing the amount of copying needed. The optimization patch is here: http://bugs.python.org/issue980695 and some history (including Guido's opposition to the patch) here: http://mail.python.org/pipermail/python-dev/2004-August/046686.html Nevertheless, the patch is now part of CPython since 2.4, but must be considered an implementation-specific optimization. It doesn't apply to Jython, and likely other implementations. > Using CPython 2.6.4 in Windows XP: [snip time measurements] > Here the linear increase of times indicate that the "+" is being > optimized using an expanding buffer for the string. Be careful of jumping to the conclusion from timings that a certain algorithm is used. All you can really tell is that, whatever the algorithm is, it has such-and-such big-oh behaviour. > If only the minimal > space was allocated each time then one would expect O(n^2) behavior > instead of the apparently O(n) above. I don't follow that reasoning. > Example of that O(n^2) behavior given below. The example shown demonstrates that the + optimization only applies in certain specific cases. In fact, it only applies to appending: s += t s = s + t but not prepending or concatenation with multiple strings: s = t + s s += t + u However, keep in mind that the CPython keyhole optimizer will take something like this: s += "x" + "y" and turn it into this: s += "xy" which the concatenation optimization does apply to. Optimizations make it hard to reason about the behaviour of algorithms! -- Steven From r.grimm at science-computing.de Wed Jan 20 01:39:11 2010 From: r.grimm at science-computing.de (Rainer Grimm) Date: Tue, 19 Jan 2010 22:39:11 -0800 (PST) Subject: What is a list compression in Python? References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> Message-ID: Hallo, you can also look at list comprehension as syntactic sugar for the functions map and filter. The two functions from the functional world can be expressed in a comprehensive way with list comprehension. >>> [x**2 for x in range(10) ] == map ( lambda x: x*x, range(10)) True >>> [ x for x in range(10) if x%2 == 0 ] == filter ( lambda x: x%2 == 0 , range(10)) True Greetings From dieter at handshake.de Wed Jan 20 01:41:13 2010 From: dieter at handshake.de (Dieter Maurer) Date: 20 Jan 2010 07:41:13 +0100 Subject: setattr() oddness In-Reply-To: References: <3eba07d0-fc14-4f51-a777-1c47cc866387@j14g2000yqm.googlegroups.com> <409dce36-691c-4c9f-ae69-aba358377b6a@j4g2000yqe.googlegroups.com> <4b517acc$1@dnews.tpgi.com.au> Message-ID: Steven D'Aprano writes on 18 Jan 2010 06:47:59 GMT: > On Mon, 18 Jan 2010 07:25:58 +0100, Dieter Maurer wrote: > > > Lie Ryan writes on Sat, 16 Jan 2010 19:37:29 +1100: > >> On 01/16/10 10:10, Sean DiZazzo wrote: > >> > Interesting. I can understand the "would take time" argument, but I > >> > don't see any legitimate use case for an attribute only accessible > >> > via getattr(). Well, at least not a pythonic use case. > >> > >> mostly for people (ab)using attributes instead of dictionary. > > > > Here is one use case: > > > > A query application. Queries are described by complex query objects. > > For efficiency reasons, query results should be cached. For this, it is > > not unnatural to use query objects as cache keys. Then, query objects > > must not get changed in an uncontrolled way. I use "__setattr__" to > > control access to the objects. > > > (1) Wouldn't it be more natural to store these query keys in a list or > dictionary rather than as attributes on an object? > > e.g. instead of: > > cache.__setattr__('complex query object', value) > > use: > > cache['complex query object'] = value Few will use "cache.__setattr__(...)" but "cache.attr = ..." which is nicer than "cache['attr'] = ...". Moreover, it is not the cache but the query of which I want to protect modification. My cache indeed uses "cache[query_object] = ...". But I want to prevent "query_object" from being changed after a potential caching. > (2) How does __setattr__ let you control access to the object? If a user > wants to modify the cache, and they know the complex query object, what's > stopping them from using __setattr__ too? In my specific case, "__setattr__" prevents all modifications via attribute assignment. The class uses "__dict__" access to set attributes when it knows it is still safe. Of course, this is no real protection against attackers (which could use "__dict__" as well). It only protects against accidental change of query objects. Meanwhile, I remembered a more important use case for "__setattr__": providing for transparent persistancy. The "ZODB" (Zope Object DataBase) customizes "__setattr__" in order to intercept object modifications and register automatically that the change needs to be persisted at the next transaction commit. Dieter From arnodel at googlemail.com Wed Jan 20 02:17:27 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Wed, 20 Jan 2010 07:17:27 +0000 Subject: Py 3: How to switch application to Unicode strings? References: <92288825-8b01-47fb-8592-f65a7d7aeb8a@k35g2000yqb.googlegroups.com> <7a9c25c21001191156j46a7fdadt58b728477b85e651@mail.gmail.com> <646ab38b-0710-4d31-b9e1-8a6ee7bfa42f@21g2000yqj.googlegroups.com> Message-ID: Gnarlodious writes: > Well, Python 3 is supposed to be all Unicode by default. I shouldn't > even need to say > # coding:UTF-8 > > And, the file is saved as Unicode. > When a filed is saved, shouldn't it be in a specific encoding? I don't see how you can save your file 'as unicode'. You should save your file with UTF-8 encoding. HTH -- Arnaud From metolone+gmane at gmail.com Wed Jan 20 02:22:06 2010 From: metolone+gmane at gmail.com (Mark Tolonen) Date: Tue, 19 Jan 2010 23:22:06 -0800 Subject: Py 3: How to switch application to Unicode strings? References: <92288825-8b01-47fb-8592-f65a7d7aeb8a@k35g2000yqb.googlegroups.com><7a9c25c21001191156j46a7fdadt58b728477b85e651@mail.gmail.com> <646ab38b-0710-4d31-b9e1-8a6ee7bfa42f@21g2000yqj.googlegroups.com> Message-ID: "Gnarlodious" wrote in message news:646ab38b-0710-4d31-b9e1-8a6ee7bfa42f at 21g2000yqj.googlegroups.com... > Well, Python 3 is supposed to be all Unicode by default. I shouldn't > even need to say > # coding:UTF-8 Yes, in Python 3, an absence of a 'coding' line assumes UTF-8. > And, the file is saved as Unicode. There is no such thing as "saved as Unicode". Unicode is not an encoding. For example, '?' is the Unicode codepoint 241. This can be stored in a file in a number of ways. UTF-8 is the two bytes 0xc3 0xB1. UTF-16LE is 0xF1 0x00. UTF-16BE is 0x00 0xF1. latin-1 is the single byte 0xF1. If your editor saves your file in the encoding "latin-1", and you don't use a coding line to declare it, Python 3 will throw an error if it finds a non-UTF-8 byte sequence in the file. > There are many mentions of this error found by Google, but none seen > to clearly say what the problem is or how to fix it. > FYI, the problem line says: > cursor.execute('insert into Data values > (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)', frameTuple) Is frameTuple a byte string or Unicode string. print(repr(frameTuple)). > and one of the strings in the tuple contains a character like '?'. > I have a version of the SQLite editor that works as expected in a > browser, I don't know why. Post the simplest, complete source code that exhibits the problem. -Mark From apt.shansen at gmail.com Wed Jan 20 02:32:31 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Tue, 19 Jan 2010 23:32:31 -0800 Subject: What is a list compression in Python? In-Reply-To: References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> Message-ID: <7a9c25c21001192332m17e8509blc7e3c67bf9ade364@mail.gmail.com> On Tue, Jan 19, 2010 at 10:39 PM, Rainer Grimm wrote: > Hallo, > you can also look at list comprehension as syntactic sugar for the > functions map and filter. The two functions from the functional world > can be expressed in a comprehensive way with list comprehension. > >>> [x**2 for x in range(10) ] == map ( lambda x: x*x, range(10)) > True > >>> [ x for x in range(10) if x%2 == 0 ] == filter ( lambda x: x%2 == 0 , > range(10)) > True > I really don't think you can call comprehensions as mere syntactic sugar, as there are measurable performance differences between the two. For something to be syntactic sugar, it must be essentially equivalent. A list comprehension is, IIUC, real syntactic sugar around a for loop. Meaning it, in a real way, simply creates a for-loop with special syntax. A list comprehension can be /equivalent in effect/ to the functions map and filter, but it is not syntactic sugar. It is generating entirely different code-paths with entirely different performance characteristics to achieve the same goals. But that's not sugar. Syntactic sugar is a sweet, cute, or cute way to express a more complex thought in a simpler or more concise way. But the -effect- and -result- of both thoughts are the same: if you do it manually, or with the sugar, they're essentially equivalent. That's why its sugar... its just something to make the experience sweeter, it doesn't -really- add any value beyond making the experience sweeter. Consider your tests: >>> Timer("[x**2 for x in range(10) ]").timeit() 3.4986340999603271 >>> Timer("map ( lambda x: x*x, range(10))").timeit() 4.5014309883117676 >>> Timer("[ x for x in range(10) if x%2 == 0 ]").timeit() 3.3268649578094482 >>> Timer("filter ( lambda x: x%2 == 0 , range(10))").timeit() 5.3649170398712158 The list comprehension version performs distinctly better in each case. The end result of the two are the same, but one is not sugar for the other. A list comprehension is in essence a compact way of writing a for loop that builds a list. A map (or filter) operation is a functional approach to build a list: the difference is that the latter requires you to call a function a lot, and function calls in Python are not cheap. --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From apt.shansen at gmail.com Wed Jan 20 02:34:18 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Tue, 19 Jan 2010 23:34:18 -0800 Subject: What is a list compression in Python? In-Reply-To: <7a9c25c21001192332m17e8509blc7e3c67bf9ade364@mail.gmail.com> References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> <7a9c25c21001192332m17e8509blc7e3c67bf9ade364@mail.gmail.com> Message-ID: <7a9c25c21001192334r5ed45f8fo891e37fcbe2caf2e@mail.gmail.com> On Tue, Jan 19, 2010 at 11:32 PM, Stephen Hansen wrote: > > I really don't think you can call comprehensions as mere syntactic sugar, > Err, I misspoke. I don't really think you can call comprehensions mere syntactic sugar /for map and filter/. It IS mere syntactic sugar for a "for loop". But not for the functional equivalent. --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at REMOVE-THIS-cybersource.com.au Wed Jan 20 02:36:22 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 20 Jan 2010 07:36:22 GMT Subject: Best way to convert sequence of bytes to long integer Message-ID: <00e0377c$0$15570$c3e8da3@news.astraweb.com> I have a byte string (Python 2.x string), e.g.: s = "g%$f yg\n1\05" assert len(s) == 10 I wish to convert it to a long integer, treating it as base-256. Currently I'm using: def makelong(s): n = 0 for c in s: n *= 256 n += ord(c) return n which gives: >>> makelong(s) 487088900085839492165893L Is this the best way, or have I missed some standard library function? Thanks in advance, -- Steven From stefan_ml at behnel.de Wed Jan 20 02:38:12 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 20 Jan 2010 08:38:12 +0100 Subject: Performance of lists vs. list comprehensions In-Reply-To: <00e023c8$0$15570$c3e8da3@news.astraweb.com> References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> <00dff4e6$0$15570$c3e8da3@news.astraweb.com> <00e023c8$0$15570$c3e8da3@news.astraweb.com> Message-ID: <4b56b2e5$0$7628$9b4e6d93@newsspool1.arcor-online.net> Steven D'Aprano, 20.01.2010 07:12: > On Wed, 20 Jan 2010 05:25:22 +0100, Alf P. Steinbach wrote: >> That is a good argument for not doing the expanding buffer thing. >> But such buffers may be generally present anyway, resulting from >> optimization of "+". > > As near as I can determine, the CPython optimization you are referring to > doesn't use the "double the buffer when needed" technique. It operates on > a completely different strategy. As near as I can tell (as a non-C > speaker), it re-sizes the string in place to the size actually needed, > thus reducing the amount of copying needed. > >> Using CPython 2.6.4 in Windows XP: > [snip time measurements] >> Here the linear increase of times indicate that the "+" is being >> optimized using an expanding buffer for the string. > > Be careful of jumping to the conclusion from timings that a certain > algorithm is used. All you can really tell is that, whatever the > algorithm is, it has such-and-such big-oh behaviour. Which is particularly tricky here since the algorithms depends more on the OS than on the code in CPython. The better timings come from the fact that the OS does *not* need to copy the buffer on each iteration, but does smarter things when asked to enlarge the buffer. If you ran the benchmark on an OS that *did* copy the buffer each time, the runtime would really be quadratic. BTW, I think it would actually be worth trying to apply the same approach to str.join() if the argument is not a sequence (obviously followed by a benchmark on different platforms). Stefan From stefan_ml at behnel.de Wed Jan 20 02:56:48 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 20 Jan 2010 08:56:48 +0100 Subject: Best way to convert sequence of bytes to long integer In-Reply-To: <00e0377c$0$15570$c3e8da3@news.astraweb.com> References: <00e0377c$0$15570$c3e8da3@news.astraweb.com> Message-ID: <4b56b740$0$7615$9b4e6d93@newsspool1.arcor-online.net> Steven D'Aprano, 20.01.2010 08:36: > I have a byte string (Python 2.x string), e.g.: > > s = "g%$f yg\n1\05" > assert len(s) == 10 > > I wish to convert it to a long integer, treating it as base-256. > Currently I'm using: > > def makelong(s): > n = 0 > for c in s: > n *= 256 > n += ord(c) > return n > > > which gives: > >>>> makelong(s) > 487088900085839492165893L > > > Is this the best way, or have I missed some standard library function? Have you checked if the struct module offers anything here? Stefan From alfps at start.no Wed Jan 20 03:21:30 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 20 Jan 2010 09:21:30 +0100 Subject: Performance of lists vs. list comprehensions In-Reply-To: <00e023c8$0$15570$c3e8da3@news.astraweb.com> References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> <00dff4e6$0$15570$c3e8da3@news.astraweb.com> <00e023c8$0$15570$c3e8da3@news.astraweb.com> Message-ID: * Steven D'Aprano: > On Wed, 20 Jan 2010 05:25:22 +0100, Alf P. Steinbach wrote: > >> * Steven D'Aprano: >>> On Tue, 19 Jan 2010 16:20:42 -0500, Gerald Britton wrote: >>> >>>> That's surprising. I wouldn't implement it that way at all. I'd use a >>>> dynamically-expanding buffer as I suggested. That way you get a >>>> single pass and don't have to calculate anything before you begin. In >>>> the best case, you'd use half the memory (the result just fits in the >>>> buffer after its last expansion and no saved tuple). In the worst >>>> case, the memory use is about the same (you just expanded the buffer >>>> using a 2x expansion rule then you hit the last item). >>> In the worst case, you waste 50% of the memory allocated. >> Yes. That is a good argument for not doing the expanding buffer thing. >> But such buffers may be generally present anyway, resulting from >> optimization of "+". > > > As near as I can determine, the CPython optimization you are referring to > doesn't use the "double the buffer when needed" technique. It operates on > a completely different strategy. As near as I can tell (as a non-C > speaker), it re-sizes the string in place to the size actually needed, > thus reducing the amount of copying needed. > > The optimization patch is here: > > http://bugs.python.org/issue980695 > > and some history (including Guido's opposition to the patch) here: > > http://mail.python.org/pipermail/python-dev/2004-August/046686.html > > Nevertheless, the patch is now part of CPython since 2.4, but must be > considered an implementation-specific optimization. It doesn't apply to > Jython, and likely other implementations. Provided that the CPython code is that code then you're right, it only calls PyObject_REALLOC, depending on that operation having (amortized) constant time. And PyObject_REALLOC can in principle achieve that by relying on paging, e.g. using the OS' virtual memory allocator, instead of doubling and copying. However, I'm baffled by the code I find via Google Code search; there PyObject_REALLOC simply calls malloc and copies, which if that were the code used in CPython 2.4 and greater, and if the '+' code is the code that you refer to above, would produce O(n^2) time for the first '+' example. >> Using CPython 2.6.4 in Windows XP: > [snip time measurements] >> Here the linear increase of times indicate that the "+" is being >> optimized using an expanding buffer for the string. > > Be careful of jumping to the conclusion from timings that a certain > algorithm is used. All you can really tell is that, whatever the > algorithm is, it has such-and-such big-oh behaviour. Well, as for intended meaning you're right about that, it needs not be a doubling buffer. >> If only the minimal >> space was allocated each time then one would expect O(n^2) behavior >> instead of the apparently O(n) above. > > I don't follow that reasoning. The reasoning assumes independent allocations rather than reallocation (extension of existing allocation). Then quadratic time for the example follows from sum(range(n)) = (n^2-n)/2 of the sizes of the data copied. But with PyObject_REALLOC as essentially constant time also 'join' could take advantage of this. :-) Cheers, - Alf From __peter__ at web.de Wed Jan 20 03:43:30 2010 From: __peter__ at web.de (Peter Otten) Date: Wed, 20 Jan 2010 09:43:30 +0100 Subject: Best way to convert sequence of bytes to long integer References: <00e0377c$0$15570$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > I have a byte string (Python 2.x string), e.g.: > > s = "g%$f yg\n1\05" > assert len(s) == 10 > > I wish to convert it to a long integer, treating it as base-256. > Currently I'm using: > > def makelong(s): > n = 0 > for c in s: > n *= 256 > n += ord(c) > return n > > > which gives: > >>>> makelong(s) > 487088900085839492165893L > > > Is this the best way, or have I missed some standard library function? The pickle module uses >>> import binascii >>> s = "g%$f yg\n1\05" >>> long(binascii.hexlify(s), 16) 487088900085839492165893L Peter From rpjday at crashcourse.ca Wed Jan 20 03:58:45 2010 From: rpjday at crashcourse.ca (Robert P. J. Day) Date: Wed, 20 Jan 2010 03:58:45 -0500 (EST) Subject: how to check if a module is importable? Message-ID: finally getting back to clawing my way thru the python 3 book so probably a number of newbie questions coming up. first one -- can i check if a module is importable (equivalently, exists on sys.path, i assume) without trying to import it first? i can see that i can use try/except and just run "import" -- is that the accepted way to test? rday -- ======================================================================== Robert P. J. Day Waterloo, Ontario, CANADA Linux Consulting, Training and Kernel Pedantry. Web page: http://crashcourse.ca Twitter: http://twitter.com/rpjday ======================================================================== From no.email at nospam.invalid Wed Jan 20 04:10:41 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 20 Jan 2010 01:10:41 -0800 Subject: Best way to convert sequence of bytes to long integer References: <00e0377c$0$15570$c3e8da3@news.astraweb.com> Message-ID: <7xk4vdf8fi.fsf@ruckus.brouhaha.com> Steven D'Aprano writes: > s = "g%$f yg\n1\05" > Is this the best way, or have I missed some standard library function? It is really a shame that there is not a standard library function for this. I usually do it using hexadecimal conversion: d = int(s.encode('hex'), 16) From dickinsm at gmail.com Wed Jan 20 04:10:47 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Wed, 20 Jan 2010 01:10:47 -0800 (PST) Subject: Best way to convert sequence of bytes to long integer References: <00e0377c$0$15570$c3e8da3@news.astraweb.com> Message-ID: <5858a2bd-116b-4dd8-bff3-0039b345bff7@34g2000yqp.googlegroups.com> On Jan 20, 7:36?am, Steven D'Aprano wrote: > I have a byte string (Python 2.x string), e.g.: > > s = "g%$f yg\n1\05" > assert len(s) == 10 > > I wish to convert it to a long integer, treating it as base-256. Not that it helps you right now, but provided someone finds the time, there should be an int/long method for this in Python 2.7. It's already implemented for Python 3.2, so it just needs backporting. Python 3.2a0 (py3k:77612, Jan 20 2010, 09:04:15) [GCC 4.0.1 (Apple Inc. build 5490)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> int.from_bytes(b"g%$f yg\n1\05", 'big') 487088900085839492165893 Until then, Peter Otten's solution is about as close as you can get, I think. -- Mark From mal at egenix.com Wed Jan 20 05:02:29 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 20 Jan 2010 11:02:29 +0100 Subject: setattr() oddness In-Reply-To: References: <3eba07d0-fc14-4f51-a777-1c47cc866387@j14g2000yqm.googlegroups.com> <409dce36-691c-4c9f-ae69-aba358377b6a@j4g2000yqe.googlegroups.com> <4b517acc$1@dnews.tpgi.com.au> Message-ID: <4B56D4B5.7000408@egenix.com> Dieter Maurer wrote: > Steven D'Aprano writes on 18 Jan 2010 06:47:59 GMT: >> On Mon, 18 Jan 2010 07:25:58 +0100, Dieter Maurer wrote: >> >>> Lie Ryan writes on Sat, 16 Jan 2010 19:37:29 +1100: >>>> On 01/16/10 10:10, Sean DiZazzo wrote: >>>>> Interesting. I can understand the "would take time" argument, but I >>>>> don't see any legitimate use case for an attribute only accessible >>>>> via getattr(). Well, at least not a pythonic use case. >>>> >>>> mostly for people (ab)using attributes instead of dictionary. >>> >>> Here is one use case: >>> >>> A query application. Queries are described by complex query objects. >>> For efficiency reasons, query results should be cached. For this, it is >>> not unnatural to use query objects as cache keys. Then, query objects >>> must not get changed in an uncontrolled way. I use "__setattr__" to >>> control access to the objects. >> >> >> (1) Wouldn't it be more natural to store these query keys in a list or >> dictionary rather than as attributes on an object? >> >> e.g. instead of: >> >> cache.__setattr__('complex query object', value) >> >> use: >> >> cache['complex query object'] = value > > Few will use "cache.__setattr__(...)" but "cache.attr = ..." which > is nicer than "cache['attr'] = ...". > > Moreover, it is not the cache but the query of which I want to protect > modification. My cache indeed uses "cache[query_object] = ...". > But I want to prevent "query_object" from being changed after a potential > caching. You might want to look at mxProxy which provides object-level access control: http://www.egenix.com/products/python/mxBase/mxProxy/ See the "Access Protocol" in the documentation: http://www.egenix.com/products/python/mxBase/mxProxy/doc/#_Toc162774446 >> (2) How does __setattr__ let you control access to the object? If a user >> wants to modify the cache, and they know the complex query object, what's >> stopping them from using __setattr__ too? > > In my specific case, "__setattr__" prevents all modifications via attribute > assignment. The class uses "__dict__" access to set attributes when > it knows it is still safe. > > Of course, this is no real protection against attackers (which could > use "__dict__" as well). It only protects against accidental change > of query objects. > > > Meanwhile, I remembered a more important use case for "__setattr__": > providing for transparent persistancy. The "ZODB" (Zope Object DataBase) > customizes "__setattr__" in order to intercept object modifications > and register automatically that the change needs to be persisted at > the next transaction commit. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 20 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From jeanmichel at sequans.com Wed Jan 20 05:06:08 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Wed, 20 Jan 2010 11:06:08 +0100 Subject: how to check if a module is importable? In-Reply-To: References: Message-ID: <4B56D590.1070400@sequans.com> Robert P. J. Day wrote: > finally getting back to clawing my way thru the python 3 book so > probably a number of newbie questions coming up. first one -- can i > check if a module is importable (equivalently, exists on sys.path, i > assume) without trying to import it first? > > i can see that i can use try/except and just run "import" -- is that > the accepted way to test? > > rday > -- > AFAIK, that is why ImportError is for. try: import module except ImportError: # handle me pass is commonly used. JM From bornstub at gmail.com Wed Jan 20 05:52:25 2010 From: bornstub at gmail.com (Victor Lin) Date: Wed, 20 Jan 2010 02:52:25 -0800 (PST) Subject: Memory usage problem of twisted server Message-ID: Hi, I encountered an increasing memory usage problem of my twisted server. I have posted a question on stackoverflow: http://stackoverflow.com/questions/2100192/how-to-find-the-source-of-increasing-memory-usage-of-a-twisted-server I have read the article "Improving Python's Memory Allocator" ( http://evanjones.ca/memoryallocator/ ) and Python Memory Management ( http://evanjones.ca/python-memory.html ). And I now know little about how Python manages memory. I am wondering, is that the increasing memory usage problem of my audio broadcasting caused by the how python manage memory? In my server, there are lots of audio data chunks, they are not all in fixed size. For example, the size of audio stream might looks like this: ... chunk size 822 chunk size 878 chunk size 1690 chunk size 1659 chunk size 1643 chunk size 1952 chunk size 373 chunk size 763 chunk size 1535 chunk size 1665 ... All of those chunks are str object. The size of chunks depend on mp3 encoder. You can see most the size of those chunks is bigger than 256 bytes, so that Python uses malloc to allocate spaces for those chunks. In my mind, I saw those chunks occupy different place in heap, all of them are not in fixed size, so that even when the chunk is freed, it is difficult for malloc to find the right place for the new chunk. As the result, there are so many chunks placed in large range of heap, it is difficult for OS to swap pages out. Then the RSS is always high. Is that my guessing correct? How can I monitor the memory allocation of Python? Is there any available tools for this? Or is that modifying the Python source code and recompiling the only way to monitor allocation of memory? Thanks. Victor Lin. From davea at ieee.org Wed Jan 20 06:05:50 2010 From: davea at ieee.org (Dave Angel) Date: Wed, 20 Jan 2010 06:05:50 -0500 Subject: Performance of lists vs. list comprehensions In-Reply-To: <00dfe6ee$0$15570$c3e8da3@news.astraweb.com> References: <5d1a32001001190730u4fa2f26cn62e79c2c73a08b@mail.gmail.com> <7a9c25c21001190818g7bd0ca6n336e6b1d52af0b2c@mail.gmail.com> <00dfe6ee$0$15570$c3e8da3@news.astraweb.com> Message-ID: <4B56E38E.5020907@ieee.org> Steven D'Aprano wrote: > A million items is not a lot of data. Depending on the size of each > object, that might be as little as 4 MB of data: > > >>>> L = ['' for _ in xrange(10**6)] >>>> sys.getsizeof(L) >>>> > 4348732 > > Note that this sys.getsizeof() is only saying the size of the list, not the size of the objects referenced in the list. So that number doesn't depend on "the size of the object." DaveA From rpjday at crashcourse.ca Wed Jan 20 06:59:19 2010 From: rpjday at crashcourse.ca (Robert P. J. Day) Date: Wed, 20 Jan 2010 06:59:19 -0500 (EST) Subject: examining an initial, pristine python3 shell session Message-ID: still working my way through "dive into python 3" and i've already been asked to give a newbie tutorial on it -- blind leading the blind, as it were. that should be hilarious. i'll be using python 3 and it occurred to me that it would be educational (at least for me :-) to display what an initial p3 shell session looks like before doing any imports whatsoever. as in, i run "python3" on my fedora box and, at the ">>>" prompt, i want to show what's already there for the new user. from what little i know so far, i'd start with: >>> __name__ '__main__' >>> to display the name of the current scope(?). backing up a bit, i could run either of: >>> dir() ['__builtins__', '__doc__', '__name__', '__package__'] >>> globals() {'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None} >>> then i might go a bit further to examine some of *those* objects. i admit it might seem a bit dry, but i think it would be handy to have a handle on what a clean shell session looks like before starting to import things, then seeing how that importing changes the session before getting down to actual programming. what other useful commands might i run immediately after starting a session whose output would be informative? i can certainly poke at some of those objects to see them in more detail. i'm just curious what others might recommend. thanks. rday -- ======================================================================== Robert P. J. Day Waterloo, Ontario, CANADA Linux Consulting, Training and Kernel Pedantry. Web page: http://crashcourse.ca Twitter: http://twitter.com/rpjday ======================================================================== From alfps at start.no Wed Jan 20 07:43:23 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 20 Jan 2010 13:43:23 +0100 Subject: examining an initial, pristine python3 shell session In-Reply-To: References: Message-ID: * Robert P. J. Day: > still working my way through "dive into python 3" and i've already > been asked to give a newbie tutorial on it -- blind leading the blind, > as it were. that should be hilarious. > > i'll be using python 3 and it occurred to me that it would be > educational (at least for me :-) to display what an initial p3 shell > session looks like before doing any imports whatsoever. as in, i run > "python3" on my fedora box and, at the ">>>" prompt, i want to show > what's already there for the new user. > > from what little i know so far, i'd start with: > >>>> __name__ > '__main__' > > to display the name of the current scope(?). backing up a bit, i > could run either of: > >>>> dir() > ['__builtins__', '__doc__', '__name__', '__package__'] >>>> globals() > {'__builtins__': , '__name__': > '__main__', '__doc__': None, '__package__': None} > > then i might go a bit further to examine some of *those* objects. i > admit it might seem a bit dry, but i think it would be handy to have a > handle on what a clean shell session looks like before starting to > import things, then seeing how that importing changes the session > before getting down to actual programming. > > what other useful commands might i run immediately after starting a > session whose output would be informative? i can certainly poke at > some of those objects to see them in more detail. i'm just curious > what others might recommend. thanks. That depends on what you mean by "newbie". If it's someone who knows a little bit of programming but is new to Python, then 'help' would definitely be about the first thing I'd show her. But if it's someone who doesn't even know anything about programming, then I'd recommend (blatant plug) -- its first two chapters are constructed around complete, concrete examples. However, you would have to adapt just the *sense* of the first chapter, which is only about tool usage, to *nix, since it's written for Windows. I'd not dive into 'help' for the someone who doesn't know anything because it gets technical pretty fast, and because she will get back to that on her own when it's time. Whatever you do, and whatever the background of the newbie, do introduce turtle graphics right away. The ch 2 of the above reference contains some t.g. examples that you might use (initial silly figures, graphs of functions, recursive figures). It doesn't go into the turtle module objects. But if objects are what you want to show right away, then I think the turtle module is great also for that, because those objects are simple and can be easily explored. Cheers & hth., - Alf From dickinsm at gmail.com Wed Jan 20 09:18:55 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Wed, 20 Jan 2010 06:18:55 -0800 (PST) Subject: Best way to convert sequence of bytes to long integer References: <00e0377c$0$15570$c3e8da3@news.astraweb.com> Message-ID: <9b870cc2-0966-4cb2-be93-573317c06eb0@a6g2000yqm.googlegroups.com> On Jan 20, 7:36?am, Steven D'Aprano wrote: > I have a byte string (Python 2.x string), e.g.: > > s = "g%$f yg\n1\05" > assert len(s) == 10 > > I wish to convert it to a long integer, treating it as base-256. By the way, are you willing to divulge what you're using this functionality for? When trying to hack out the API for int.to_bytes and int.from_bytes on the mailing list and bug tracker, some of the discussion about use-cases seemed a little too much like guesswork; I'd be curious to find out about real-life use-cases. Mark From lie.1296 at gmail.com Wed Jan 20 09:37:58 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Thu, 21 Jan 2010 01:37:58 +1100 Subject: how to check if a module is importable? In-Reply-To: References: Message-ID: <4b57155b@dnews.tpgi.com.au> On 01/20/10 19:58, Robert P. J. Day wrote: > > finally getting back to clawing my way thru the python 3 book so > probably a number of newbie questions coming up. first one -- can i > check if a module is importable (equivalently, exists on sys.path, i > assume) without trying to import it first? > > i can see that i can use try/except and just run "import" -- is that > the accepted way to test? check imp.find_module() From bblais at bryant.edu Wed Jan 20 09:40:32 2010 From: bblais at bryant.edu (Brian Blais) Date: Wed, 20 Jan 2010 09:40:32 -0500 Subject: turtle, ipython, and pylab don't mix Message-ID: Hello, I am noticing a hard crash of the ipython interpreter, with the pylab option, with the turtle module. I am on Win XP, Enthought Edition 6.0.0 which uses Python 2.6. I can't reproduce it without the pylab option, nor on the Mac (with an earlier Enthought Edition). The crash happens only when I try to move the turtle window, after it has drawn. The following commands are enough to give the resulting error. In [1]: from turtle import * In [2]: pendown() In [3]: forward(10) In [4]: Fatal Python error: PyEval_RestoreThread: NULL tstate This application has requested the Runtime to terminate it in an unusual way. Please contact the application's support team for more information. Any ideas? Of course, I'll use it without the pylab option, but that means that there are two icons I need instead of one. Minor inconvenience for me, but a bit more of a pain for my students. thanks, Brian Blais -- Brian Blais bblais at bryant.edu http://web.bryant.edu/~bblais http://bblais.blogspot.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From alfps at start.no Wed Jan 20 09:40:59 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 20 Jan 2010 15:40:59 +0100 Subject: examining an initial, pristine python3 shell session In-Reply-To: References: Message-ID: * Alf P. Steinbach: > * Robert P. J. Day: >> still working my way through "dive into python 3" and i've already >> been asked to give a newbie tutorial on it -- blind leading the blind, >> as it were. that should be hilarious. >> >> i'll be using python 3 and it occurred to me that it would be >> educational (at least for me :-) to display what an initial p3 shell >> session looks like before doing any imports whatsoever. as in, i run >> "python3" on my fedora box and, at the ">>>" prompt, i want to show >> what's already there for the new user. >> >> from what little i know so far, i'd start with: >> >>>>> __name__ >> '__main__' >> >> to display the name of the current scope(?). backing up a bit, i >> could run either of: >> >>>>> dir() >> ['__builtins__', '__doc__', '__name__', '__package__'] >>>>> globals() >> {'__builtins__': , '__name__': >> '__main__', '__doc__': None, '__package__': None} >> >> then i might go a bit further to examine some of *those* objects. i >> admit it might seem a bit dry, but i think it would be handy to have a >> handle on what a clean shell session looks like before starting to >> import things, then seeing how that importing changes the session >> before getting down to actual programming. >> >> what other useful commands might i run immediately after starting a >> session whose output would be informative? i can certainly poke at >> some of those objects to see them in more detail. i'm just curious >> what others might recommend. thanks. > > That depends on what you mean by "newbie". > > If it's someone who knows a little bit of programming but is new to > Python, then 'help' would definitely be about the first thing I'd show her. He he... Try "help antigravity". :-) > But if it's someone who doesn't even know anything about programming, > then I'd recommend (blatant plug) http://tinyurl.com/programmingbookP3> -- its first two chapters are > constructed around complete, concrete examples. However, you would have > to adapt just the *sense* of the first chapter, which is only about tool > usage, to *nix, since it's written for Windows. I'd not dive into 'help' > for the someone who doesn't know anything because it gets technical > pretty fast, and because she will get back to that on her own when it's > time. > > Whatever you do, and whatever the background of the newbie, do introduce > turtle graphics right away. > > The ch 2 of the above reference contains some t.g. examples that you > might use (initial silly figures, graphs of functions, recursive > figures). It doesn't go into the turtle module objects. But if objects > are what you want to show right away, then I think the turtle module is > great also for that, because those objects are simple and can be easily > explored. > > > Cheers & hth., > > - Alf From rpjday at crashcourse.ca Wed Jan 20 09:51:07 2010 From: rpjday at crashcourse.ca (Robert P. J. Day) Date: Wed, 20 Jan 2010 09:51:07 -0500 (EST) Subject: how to check if a module is importable? In-Reply-To: <4b57155b@dnews.tpgi.com.au> References: <4b57155b@dnews.tpgi.com.au> Message-ID: On Thu, 21 Jan 2010, Lie Ryan wrote: > On 01/20/10 19:58, Robert P. J. Day wrote: > > > > finally getting back to clawing my way thru the python 3 book so > > probably a number of newbie questions coming up. first one -- can i > > check if a module is importable (equivalently, exists on sys.path, i > > assume) without trying to import it first? > > > > i can see that i can use try/except and just run "import" -- is that > > the accepted way to test? > > check imp.find_module() thanks, that's what i was looking for. i'm sure i'll run across that shortly as i keep working my way thru this python 3 book. rday -- ======================================================================== Robert P. J. Day Waterloo, Ontario, CANADA Linux Consulting, Training and Kernel Pedantry. Web page: http://crashcourse.ca Twitter: http://twitter.com/rpjday ======================================================================== From rpjday at crashcourse.ca Wed Jan 20 10:02:42 2010 From: rpjday at crashcourse.ca (Robert P. J. Day) Date: Wed, 20 Jan 2010 10:02:42 -0500 (EST) Subject: examining an initial, pristine python3 shell session In-Reply-To: References: Message-ID: On Wed, 20 Jan 2010, Alf P. Steinbach wrote: > * Robert P. J. Day: ... snip ... > > what other useful commands might i run immediately after > > starting a session whose output would be informative? i can > > certainly poke at some of those objects to see them in more > > detail. i'm just curious what others might recommend. thanks. > > That depends on what you mean by "newbie". > > If it's someone who knows a little bit of programming but is new to > Python, then 'help' would definitely be about the first thing I'd > show her. > > But if it's someone who doesn't even know anything about > programming, then I'd recommend (blatant plug) http://tinyurl.com/programmingbookP3> -- its first two chapters are > constructed around complete, concrete examples. However, you would > have to adapt just the *sense* of the first chapter, which is only > about tool usage, to *nix, since it's written for Windows. I'd not > dive into 'help' for the someone who doesn't know anything because > it gets technical pretty fast, and because she will get back to that > on her own when it's time. > > Whatever you do, and whatever the background of the newbie, do > introduce turtle graphics right away. ah, thank you, i appreciate that reference. i'm expecting the small audience to be relatively tech-savvy with OO dev experience, and i'm betting that some of them will be wondering right off the very first thing *i* was wondering -- when i start that python3 shell and get dumped into it, what am i looking at? which is why i wanted to collect a few commands to give the attendees at least a vague idea of what was already there. i've collected the following to start with: >>> dir() ['__builtins__', '__doc__', '__name__', '__package__'] >>> globals() ... stuff ... then move on to examine *those* things: >>> type(dir) >>> type(globals) >>> type(__builtins__) zoom in a bit further and pick on specific examples: >>> dir(__builtins__) ... lots of output ... and so on. as i said, i know it looks dry but i figure i can take 5 minutes or so just to lay out the terrain and what a shell session looks like before you do *anything* with it. and i'm betting most of my audience will appreciate getting that high-level view before launching into some programming. they'll just want to know the initial session setup before they start importing stuff into it. rday -- ======================================================================== Robert P. J. Day Waterloo, Ontario, CANADA Linux Consulting, Training and Kernel Pedantry. Web page: http://crashcourse.ca Twitter: http://twitter.com/rpjday ======================================================================== From lie.1296 at gmail.com Wed Jan 20 10:05:44 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Thu, 21 Jan 2010 02:05:44 +1100 Subject: examining an initial, pristine python3 shell session In-Reply-To: References: Message-ID: <4b571bde$1@dnews.tpgi.com.au> On 01/20/10 22:59, Robert P. J. Day wrote: > then i might go a bit further to examine some of *those* objects. i > admit it might seem a bit dry, but i think it would be handy to have a > handle on what a clean shell session looks like before starting to > import things, then seeing how that importing changes the session > before getting down to actual programming. I would recommend to teach these introspection capabilities *after* some basic programming. If you tell them these dir(), globals(), __name__, etc before even getting into basic python, your precious newbies would sit there thinking "what the hell is he talking about?" From victorsubervi at gmail.com Wed Jan 20 10:19:01 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 20 Jan 2010 11:19:01 -0400 Subject: Interesting Problem Message-ID: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> Hi; I think I finally have an interesting problem for y'all. I need to import a script from a lower dir, forcing me to change dirs: cwd = os.getcwd() os.chdir('%s/..' % cwd) sys.path.append(os.getcwd()) from templateFrame import top, bottom os.chdir(cwd) Because I've found I must do my form evaluations *before* changing dir as above, I am forced to call these values as globals: form = cgi.FieldStorage() store = form.getfirst('store') cat = form.getfirst('cat') id = form.getfirst('id') pkg = form.getfirst('pkg') patientID = form.getfirst('patientID') Now, apparently because of python's problem with globals, when I call "id" as follows: cursor.execute('select ProductID from productsPackages where PackageID=%s' % id) I get the following error: /var/www/html/angrynates.com/cart/Store_frame2.py 135 cursor.close() 136 bottom() 137 138 Store_frame2() 139 Store_frame2 = /var/www/html/angrynates.com/cart/Store_frame2.py in Store_frame2() 119 printAProduct() 120 else: 121 cursor.execute('select ProductID from productsPackages where PackageID=%s' % id) 122 for id in [itm[0] for itm in cursor]: 123 printAProduct(id) global cursor = , cursor.execute = >, global id = '1' UnboundLocalError: local variable 'id' referenced before assignment args = ("local variable 'id' referenced before assignment",) Full code below. Please advise. TIA, beno #! /usr/bin/python import string import cgitb; cgitb.enable() import cgi import MySQLdb import sys,os from sets import Set import fpformat cwd = os.getcwd() sys.path.append(cwd) from login import login from particulars import truncate form = cgi.FieldStorage() store = form.getfirst('store') cat = form.getfirst('cat') id = form.getfirst('id') pkg = form.getfirst('pkg') patientID = form.getfirst('patientID') try: browser = form.getfirst('browser', 'all') except: browser = headers() os.chdir('%s/..' % cwd) sys.path.append(os.getcwd()) from templateFrame import top, bottom os.chdir(cwd) ourFile = string.split(__file__, "/") p = ourFile[len(ourFile) - 1] p = p[: - 9] site = ourFile[4][:-10] if site != '': site = site[:-1] user, passwd, db, host = login() db = MySQLdb.connect(host, user, passwd, db) cursor= db.cursor() cursor.execute('describe %s;' % store) descr = cursor.fetchall() cols = [] for elt in descr: cols.append(elt[0]) def printAProduct(id=id): lastID = '' cursor.execute('select * from %s where ID="%s"' % (store, id)) vals = cursor.fetchone() prodDict = dict(zip(cols, vals)) for col, item in prodDict.iteritems(): if col == 'ID': print '' % item try: # Stores such as products (but not prescriptions) will pass through from particulars import ourOptions if col in ourOptions(): print '' % (col, item) except: pass if col[:3] != 'pic': notSet = 1 if isinstance(item, (str, int, long, float, long, complex, unicode, list, buffer, xrange, tuple)): print '%s: %s
\n' % (col, item) else: try: html = "%s:
" print html except: pass # if notSet == 1: # if len(col) > 49: # colX = col[:50] + '...' # else: # colX = col # print '%s: %s
\n' % (colX, item) elif col == 'pic1': try: if (item != None): # if (item != None) and (len(item > 0)): print '
\n' % (store, col[3:], id, store, col[3:], id, store, col[3:], id) except TypeError: raise except: raise # i += 1 # try: # content = item[x].tostring() # pic = "tmp" + str(i) + ".jpg" # try: # os.remove(pic) # except: # pass # img = open(pic, "w") # img.write(content) # print '

' % pic # img.close() # except: # pass def Store_frame2(): top(browser, p) i = 0 print '\n' print '
' print "" % store print "" if pkg == 'no': printAProduct() else: cursor.execute('select ProductID from productsPackages where PackageID=%s' % id) for id in [itm[0] for itm in cursor]: printAProduct(id) if store == 'prescriptions': print "" print "" elif truncate()[store] != '': pass else: print "Quantity:
" print "" print '
' print '\n' print '\n' cursor.close() bottom() Store_frame2() -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Wed Jan 20 10:24:51 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 20 Jan 2010 11:24:51 -0400 Subject: Something More Elegant In-Reply-To: References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> <4dc0cfea1001090715md97518tf618309eda1deabb@mail.gmail.com> <4dc0cfea1001171006x743c233cy557e3930db3feba9@mail.gmail.com> Message-ID: <4dc0cfea1001200724i7415593dv7cdb3d2a1ac764c5@mail.gmail.com> On Sun, Jan 17, 2010 at 5:36 PM, Dennis Lee Bieber wrote: > I don't think that would be efficient, considering that "the above" > entails what would be something like three or four courses all by > themselves (Database Analysis, Database Design, SQL [these three are > independent of any particular implementation language, though Design and > SQL may be impacted by the database engine one is using -- for example, > foreign key integrity & cascading deletes are not enforced by MySQL > MyISAM tables, but can be enforced using InnoDB tables; SQLite doesn't > have user access levels, but all client/server engines do, etc.), and > then a course on Python with an emphasis on Web applications [since I > recall you are trying to generate HTML at some stage]) the only example > I'd be able to give would require me writing a complete application -- > the last time I supplied a large snippet of pseudo-code it took over a > month of daily questions and replies to fix your attempts to modify it; > most problems coming down to not understanding the algorithm, how to use > parameterized queries, etc. > LOL! Yeah... <:-} > > The shortest hints I can demonstrate -- using a text console for > input, AND in pseudo-code (I've not ensured this will run) would be: > > DON'T: > tname = raw_input("Enter the name of the table to be searched: ") > fname = raw_input("Enter the name of the field in %s to be searched: " > % tname) > fvalue = raw_input("Enter the value of %s defining the desired data: " > % fname) > > SQL = """select * from %s > where %s like "%%%s%%"""" % (tname, fname, fvalue) > crs.execute(SQL) > > > DO > tables = [ "Table1", > "Table2", > "Table3" ] > fields = { "Table1" : [ "t1field1", > "t1field2", > "t1field3" ], > "Table2" : [ "t2field1", > "t2field2", > "t2field3" ], > "Table3" : [ "t3field1", > "t3field2", > "t3field3" ] > } > > for i, t in enumerate(tables): > print "%s\t%s" % (i, t) > tnum = int(raw_input( > "Enter the number of the table to be searched: ")) > tname = tables[tnum] > > for i, f in enumerate(fields[tname]): > print "%s\t%s" % (i, f) > fnum = int(raw_input( > "Enter the name of the field in %s to be searched: " > % tname) > fname = fields[tname][fnum] > > fvalue = raw_input("Enter the value of %s defining the desired data: " > % fname) > > SQL = """select * from %s > where %s like %%s""" % (tname, fname) > crs.execute(SQL, ("%"+fvalue+"%",)) > > In a real application, one would use the ability of the database > engine itself to retrieve the list of table names, and the fields for > each table -- that way the code would not need to be changed if the > database has additions or deletions of tables/fields. > > Also note that I did NOT include exception handling -- all those > raw_input() in the "DO" version should have some exception handling (the > first two might not be given numeric data so the int() fails, or the > integer supplied may not be in the range of indices for the list > look-ups; all three might get an EOF entry by the user trying to abort > the look-up. > > Fancier versions would build multi-table joins, or allow for AND (or > OR) clauses in the WHERE, using a list of table.field/value pairs. > Thanks. I think I followed that :/ beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Wed Jan 20 10:26:27 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 20 Jan 2010 11:26:27 -0400 Subject: Changing var names In-Reply-To: <4B546BDD.1030602@sequans.com> References: <4dc0cfea1001150927i48b6c386w53f4487cd894d078@mail.gmail.com> <1263582938.3142.12.camel@linux-m3mt> <4dc0cfea1001151203w2d1c9520t692d2367a1882ee3@mail.gmail.com> <4B546BDD.1030602@sequans.com> Message-ID: <4dc0cfea1001200726g57a828e7vfca360357ec75cc6@mail.gmail.com> On Mon, Jan 18, 2010 at 10:10 AM, Jean-Michel Pichavant < jeanmichel at sequans.com> wrote: > Victor Subervi wrote: > >> On Fri, Jan 15, 2010 at 3:15 PM, Adam Tauno Williams < >> awilliam at opengroupware.us > wrote: >> >> On Fri, 2010-01-15 at 13:27 -0400, Victor Subervi wrote: >> > Hi; >> > Well it took me *less than a day* to fix the following problems: >> > -- bare excepts (accidentally left a couple I think) >> > -- sql injection attacks >> > -- recreating tables to make them more reasonable >> > Now, I believe someone once mentioned that there is some s/w >> tool out >> > there for changing the names of my variables more easily than going >> > through every script one-by-one. If so, please share with me again >> > what it is. >> >> This process is called 'refactoring' [a good term to Google], and >> every >> decent IDE provides some support [if it doesn't, it isn't a "decent" >> IDE] >> >> >> Thanks. I'll work with Eric, I guess, once I get my Mac. >> beno >> > Here are 2 well known IDEs, that help with refactoring (and many other > things). > http://netbeans.org/ > http://www.eclipse.org/ > > Both supports python. > Thanks. > > I personnaly will stick to vim :o) > I'll keep that in mind. I haven't used any WYSIWIGS to date. Thanks, beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From keith at nekotaku.com Wed Jan 20 10:32:36 2010 From: keith at nekotaku.com (KB) Date: Wed, 20 Jan 2010 07:32:36 -0800 (PST) Subject: Using jython to call python procedures/methods Message-ID: Hi there, I have an application that only publishes a Java API. I can use jython to access java classes, but jython currently (to the best of my knowledge) does not support numpy/scipy. Ideally I would like to have jython call a "native" python routine where I have the numpy/scipy procedures already written. Does anyone have any experience with this? Is it possible? I had toyed with the idea of having jython/java write the data to a file/database and then manually kick off the python process, but ideally I would like this as automated as possible. Thanks in advance. From robert.kern at gmail.com Wed Jan 20 11:06:26 2010 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 20 Jan 2010 10:06:26 -0600 Subject: turtle, ipython, and pylab don't mix In-Reply-To: References: Message-ID: On 2010-01-20 08:40 AM, Brian Blais wrote: > Hello, > > I am noticing a hard crash of the ipython interpreter, with the pylab > option, with the turtle module. I am on Win XP, Enthought Edition 6.0.0 > which uses Python 2.6. I can't reproduce it without the pylab option, > nor on the Mac (with an earlier Enthought Edition). The crash happens > only when I try to move the turtle window, after it has drawn. The > following commands are enough to give the resulting error. > > In [1]: from turtle import * > > In [2]: pendown() > > In [3]: forward(10) > > In [4]: Fatal Python error: PyEval_RestoreThread: NULL tstate > > This application has requested the Runtime to terminate it in an unusual > way. > Please contact the application's support team for more information. > > > Any ideas? Of course, I'll use it without the pylab option, but that > means that there are two icons I need instead of one. Minor > inconvenience for me, but a bit more of a pain for my students. The IPython list is probably a better place for this question. Is suspect that the problem is the mixing of GUIs. turtle uses Tk, but I suspect that your configured matplotlib backend is WxAgg. "ipython -pylab" will bring up a wx application as the main thread for execution of the code you type. When you try to bring in another GUI event loop, something goes awry. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From kwmsmith at gmail.com Wed Jan 20 11:08:41 2010 From: kwmsmith at gmail.com (Kurt Smith) Date: Wed, 20 Jan 2010 10:08:41 -0600 Subject: Using jython to call python procedures/methods In-Reply-To: References: Message-ID: On Wed, Jan 20, 2010 at 9:32 AM, KB wrote: > Hi there, > > I have an application that only publishes a Java API. I can use jython > to access java classes, but jython currently (to the best of my > knowledge) does not support numpy/scipy. > > Ideally I would like to have jython call a "native" python routine > where I have the numpy/scipy procedures already written. > > Does anyone have any experience with this? Is it possible? I have no experience with these technologies, and others can point you to more detailed info, but you could try using XML-RPC (see the docs for the xmlrpclib module in the standard library) or SOAP (Google it). These would be better than rolling your own. Perhaps there's a more Pythonic solution though? Kurt > > I had toyed with the idea of having jython/java write the data to a > file/database and then manually kick off the python process, but > ideally I would like this as automated as possible. > > Thanks in advance. > -- > http://mail.python.org/mailman/listinfo/python-list > From doxalogos at gmail.com Wed Jan 20 11:10:34 2010 From: doxalogos at gmail.com (DoxaLogos) Date: Wed, 20 Jan 2010 08:10:34 -0800 (PST) Subject: multiprocessing problems References: <74594510-9366-4227-8e1e-d221a9220271@k35g2000yqb.googlegroups.com> <63c5df56-7b86-4ee7-ba4f-b14984f94097@e16g2000yqc.googlegroups.com> Message-ID: <63ef9fe9-6e9b-4766-9a0c-4aebc28d4148@p24g2000yqm.googlegroups.com> On Jan 19, 10:33?am, DoxaLogos wrote: > On Jan 19, 10:26?am, Adam Tauno Williams > wrote: > > > > > > I decided to play around with the multiprocessing module, and I'm > > > having some strange side effects that I can't explain. ?It makes me > > > wonder if I'm just overlooking something obvious or not. ?Basically, I > > > have a script parses through a lot of files doing search and replace > > > on key strings inside the file. ?I decided the split the work up on > > > multiple processes on each processor core (4 total). ?I've tried many > > > various ways doing this form using pool to calling out separate > > > processes, but the result has been the same: computer crashes from > > > endless process spawn. > > > Are you hitting a ulimit error? ?The number of processes you can create > > is probably limited. > > > TIP: close os.stdin on your subprocesses. > > > > Here's the guts of my latest incarnation. > > > def ProcessBatch(files): > > > ? ? p = [] > > > ? ? for file in files: > > > ? ? ? ? p.append(Process(target=ProcessFile,args=file)) > > > ? ? for x in p: > > > ? ? ? ? x.start() > > > ? ? for x in p: > > > ? ? ? ? x.join() > > > ? ? p = [] > > > ? ? return > > > Now, the function calling ProcessBatch looks like this: > > > def ReplaceIt(files): > > > ? ? processFiles = [] > > > ? ? for replacefile in files: > > > ? ? ? ? if(CheckSkipFile(replacefile)): > > > ? ? ? ? ? ? processFiles.append(replacefile) > > > ? ? ? ? ? ? if(len(processFiles) == 4): > > > ? ? ? ? ? ? ? ? ProcessBatch(processFiles) > > > ? ? ? ? ? ? ? ? processFiles = [] > > > ? ? #check for left over files once main loop is done and process them > > > ? ? if(len(processFiles) > 0): > > > ? ? ? ? ProcessBatch(processFiles) > > > According to this you will create files is sets of four, but an unknown > > number of sets of four. > > What would be the proper way to only do a set of 4, stop, then do > another set of 4? ?I'm trying to only 4 files at time before doing > another set of 4. I found out my problems. One thing I did was followed the test queue example in the documentation, but the biggest problem turned out to be a pool instantiated globally in my script was causing most of the endless process spawn, even with the "if __name__ == "__main__":" block. From keith at nekotaku.com Wed Jan 20 11:24:38 2010 From: keith at nekotaku.com (KB) Date: Wed, 20 Jan 2010 08:24:38 -0800 (PST) Subject: Using jython to call python procedures/methods References: Message-ID: <1ab8341e-9d72-43d7-acb3-1a0559d3381a@c34g2000yqn.googlegroups.com> Hmmm, XML is an interesting angle... I'll noodle it... From apt.shansen at gmail.com Wed Jan 20 11:30:07 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Wed, 20 Jan 2010 08:30:07 -0800 Subject: Interesting Problem In-Reply-To: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> References: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> Message-ID: <7a9c25c21001200830k5d40d1dcv6813933b96d13cbb@mail.gmail.com> On Wed, Jan 20, 2010 at 7:19 AM, Victor Subervi wrote: > Hi; > I think I finally have an interesting problem for y'all. I need to import a > script from a lower dir, forcing me to change dirs: > Don't do that. If you must, then the correct way to do it is to adjust your sys.path and not change directory. "sys.path.append('..')" or whatever. But I really would just re-organize your code so you don't need such things. Have a single top-level directory that's on your path, put a blank __init__.py in all your directories, and use absolute imports everywhere. from myapp.templateFrame import top, bottom from myapp.some_directory.some_file import this, that etc. > Now, apparently because of python's problem with globals, when I call "id" > as follows: > > Python does not have problems with globals. You are repeatedly re-using the same variable names, causing confusion and errors; when you say 'id' in that line of code, Python doesn't think you are talking about the "id" that is global. It thinks you are talking about the "id" that is local-- that you define one line beneath. Local has precedence over global. Don't shadow global variables (and 'id' is a bad name, as it shadows a builtin variable) --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From gherron at islandtraining.com Wed Jan 20 11:37:55 2010 From: gherron at islandtraining.com (Gary Herron) Date: Wed, 20 Jan 2010 08:37:55 -0800 Subject: Interesting Problem In-Reply-To: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> References: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> Message-ID: <4B573163.4010607@islandtraining.com> Victor Subervi wrote: > Hi; > I think I finally have an interesting problem for y'all. I need to > import a script from a lower dir, forcing me to change dirs: Wait a moment... Your assumption that you need to change directories is invalid. (And that means everything you do below this point unnecessary.) Python provides many ways to easily import from arbitrary directories: * Before run time, modify the environment variable PYTHONPATH with the desired directory. Then import as normal. * At run time, import sys and append the desired path to sys.path, and again import as normal. * Check out the "imp" module which allows you to import from files specified via a path. Straighten that out, then we can examine the following (suspicious) clams: * that you must evaluate *before* some import * that you must use globals * python's problem with globals Gary Herron > > cwd = os.getcwd() > os.chdir('%s/..' % cwd) > sys.path.append(os.getcwd()) > from templateFrame import top, bottom > os.chdir(cwd) > > Because I've found I must do my form evaluations *before* changing dir > as above, I am forced to call these values as globals: > > form = cgi.FieldStorage() > store = form.getfirst('store') > cat = form.getfirst('cat') > id = form.getfirst('id') > pkg = form.getfirst('pkg') > patientID = form.getfirst('patientID') > > Now, apparently because of python's problem with globals, when I call > "id" as follows: > > cursor.execute('select ProductID from productsPackages where > PackageID=%s' % id) > > I get the following error: > > /var/www/html/angrynates.com/cart/Store_frame2.py > > 135 cursor.close() > 136 bottom() > 137 > 138 Store_frame2() > 139 > Store_frame2 = > /var/www/html/angrynates.com/cart/Store_frame2.py > in Store_frame2() > 119 printAProduct() > 120 else: > 121 cursor.execute('select ProductID from productsPackages where > PackageID=%s' % id) > 122 for id in [itm[0] for itm in cursor]: > 123 printAProduct(id) > global cursor = , cursor.execute = > >, > global id = '1' > > UnboundLocalError: local variable 'id' referenced before assignment > args = ("local variable 'id' referenced before assignment",) > > Full code below. Please advise. > TIA, > beno > > #! /usr/bin/python > > import string > import cgitb; cgitb.enable() > import cgi > import MySQLdb > import sys,os > from sets import Set > import fpformat > cwd = os.getcwd() > sys.path.append(cwd) > from login import login > from particulars import truncate > > form = cgi.FieldStorage() > store = form.getfirst('store') > cat = form.getfirst('cat') > id = form.getfirst('id') > pkg = form.getfirst('pkg') > patientID = form.getfirst('patientID') > > try: > browser = form.getfirst('browser', 'all') > except: > browser = headers() > > os.chdir('%s/..' % cwd) > sys.path.append(os.getcwd()) > from templateFrame import top, bottom > os.chdir(cwd) > > ourFile = string.split(__file__, "/") > p = ourFile[len(ourFile) - 1] > p = p[: - 9] > site = ourFile[4][:-10] > if site != '': > site = site[:-1] > > user, passwd, db, host = login() > db = MySQLdb.connect(host, user, passwd, db) > cursor= db.cursor() > > cursor.execute('describe %s;' % store) > descr = cursor.fetchall() > cols = [] > for elt in descr: > cols.append(elt[0]) > > def printAProduct(id=id): > lastID = '' > cursor.execute('select * from %s where ID="%s"' % (store, id)) > vals = cursor.fetchone() > prodDict = dict(zip(cols, vals)) > for col, item in prodDict.iteritems(): > if col == 'ID': > print '' % item > try: # Stores such as products (but not prescriptions) will pass > through > from particulars import ourOptions > if col in ourOptions(): > print '' % (col, item) > except: > pass > if col[:3] != 'pic': > notSet = 1 > if isinstance(item, (str, int, long, float, long, complex, > unicode, list, buffer, xrange, tuple)): > print '%s: %s
\n' % (col, item) > else: > try: > html = "%s:
" > print html > except: > pass > # if notSet == 1: > # if len(col) > 49: > # colX = col[:50] + '...' > # else: > # colX = col > # print '%s: %s
\n' % (colX, item) > elif col == 'pic1': > try: > if (item != None): > # if (item != None) and (len(item > 0)): > print ' class="highslide" href="getpic.py?store=%s&pic=%s&id=%s" > onclick="return hs.expand(this)"> src="getpic.py?store=%s&pic=%s&id=%s" width="100" height="80" alt="" > align="left" border="0" title="Click to enlarge" style="border: > 0px">
\n' % (store, col[3:], id, store, col[3:], > id, store, col[3:], id) > except TypeError: > raise > except: > raise > # i += 1 > # try: > # content = item[x].tostring() > # pic = "tmp" + str(i) + ".jpg" > # try: > # os.remove(pic) > # except: > # pass > # img = open(pic, "w") > # img.write(content) > # print '

' % pic > # img.close() > # except: > # pass > > def Store_frame2(): > top(browser, p) > i = 0 > print '\n' > print '
' > print "" % store > print "" > if pkg == 'no': > printAProduct() > else: > cursor.execute('select ProductID from productsPackages where > PackageID=%s' % id) > for id in [itm[0] for itm in cursor]: > printAProduct(id) > if store == 'prescriptions': > print "" > print "" > elif truncate()[store] != '': > pass > else: > print "Quantity: width='3' maxlength='3' />
" > print "" > print '
' > print '\n' > print '\n' > cursor.close() > bottom() > > Store_frame2() > > > -- > The Logos has come to bear > http://logos.13gems.com/ From python at mrabarnett.plus.com Wed Jan 20 11:38:07 2010 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 20 Jan 2010 16:38:07 +0000 Subject: Interesting Problem In-Reply-To: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> References: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> Message-ID: <4B57316F.4030009@mrabarnett.plus.com> Victor Subervi wrote: > Hi; > I think I finally have an interesting problem for y'all. I need to > import a script from a lower dir, forcing me to change dirs: > > cwd = os.getcwd() > os.chdir('%s/..' % cwd) > sys.path.append(os.getcwd()) > from templateFrame import top, bottom > os.chdir(cwd) > There's no need to actually change current directory. Just do this: sys.path.append(os.path.dirname(os.getcwd())) from templateFrame import top, bottom > Because I've found I must do my form evaluations *before* changing dir > as above, I am forced to call these values as globals: > > form = cgi.FieldStorage() > store = form.getfirst('store') > cat = form.getfirst('cat') > id = form.getfirst('id') > pkg = form.getfirst('pkg') > patientID = form.getfirst('patientID') > > Now, apparently because of python's problem with globals, when I call > "id" as follows: > > cursor.execute('select ProductID from productsPackages where > PackageID=%s' % id) > > I get the following error: > > /var/www/html/angrynates.com/cart/Store_frame2.py > > 135 cursor.close() > 136 bottom() > 137 > 138 Store_frame2() > 139 > Store_frame2 = > /var/www/html/angrynates.com/cart/Store_frame2.py > in Store_frame2() > 119 printAProduct() > 120 else: > 121 cursor.execute('select ProductID from productsPackages where > PackageID=%s' % id) > 122 for id in [itm[0] for itm in cursor]: > 123 printAProduct(id) > global cursor = , cursor.execute = method Cursor.execute of >, global id = '1' > > UnboundLocalError: local variable 'id' referenced before assignment > args = ("local variable 'id' referenced before assignment",) > > Full code below. Please advise. > Line 121 refers to 'id', but line 122 assigns to 'id' (the 'for' loop). In a function, if you assign to a variable then that variable defaults to being local. You're trying to use the local variable before assigning a value to it. Having a local variable called 'id' means that you can't refer to the global variable of the same name. You're also assigning to 'i' on line 113, but not referring to it elsewhere in the function. From darkneter at gmail.com Wed Jan 20 11:43:38 2010 From: darkneter at gmail.com (NighterNet) Date: Wed, 20 Jan 2010 08:43:38 -0800 (PST) Subject: Object Integer mapping Message-ID: <60e22629-c7a9-4d8f-930f-a3ed777b5f42@p24g2000yqm.googlegroups.com> Need help on python version 3.1.x. I can't seem to know what going on a bit. The code that I check that the hash is different every time. Is there a way to make the hash the same? I using as to check the class is the same variables but if it different variable in the class that it add to the array and return the index. # Generic Object->Integer mapping # the object must be usable as a dictionary key class ObjMap: def __init__(self): self.dict = {} self.next = 0 def get(self, obj): if obj in self.dict: return self.dict[obj] else: id = self.next self.next = self.next + 1 self.dict[obj] = id return id def items(self): getval = operator.itemgetter(0) getkey = operator.itemgetter(1) return map(getval, sorted(self.dict.items(), key=getkey)) class Point: def __init__(self): self.x = 0 self.y = 0 self.z = 0 points = ObjMap() Testme = Point() Testme.x = 0 print(points.get(Testme)) Testme2 = Point() Testme2.y = 1 print(points.get(Testme2)) Testme3 = Point() Testme3.y = 1 print(points.get(Testme3)) Ttestme4 = Point() Ttestme4.y = 1 print( points.get(Ttestme4)) It keep adding new array from this but doesn't match hash. Need help on how to fix this class code. From nytrokiss at gmail.com Wed Jan 20 11:49:19 2010 From: nytrokiss at gmail.com (James Matthews) Date: Wed, 20 Jan 2010 08:49:19 -0800 Subject: Interesting Problem In-Reply-To: <4B57316F.4030009@mrabarnett.plus.com> References: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> <4B57316F.4030009@mrabarnett.plus.com> Message-ID: <8a6b8e351001200849v6baf2597y3adb98d0d2d43636@mail.gmail.com> Also as a side point... It's best to anonymize the code as best you can so people cannot attach your code to your site (IP theft and security risk) On Wed, Jan 20, 2010 at 8:38 AM, MRAB wrote: > Victor Subervi wrote: > >> Hi; >> I think I finally have an interesting problem for y'all. I need to import >> a script from a lower dir, forcing me to change dirs: >> >> cwd = os.getcwd() >> os.chdir('%s/..' % cwd) >> sys.path.append(os.getcwd()) >> from templateFrame import top, bottom >> os.chdir(cwd) >> >> There's no need to actually change current directory. Just do this: > > sys.path.append(os.path.dirname(os.getcwd())) > > from templateFrame import top, bottom > > Because I've found I must do my form evaluations *before* changing dir as >> above, I am forced to call these values as globals: >> >> form = cgi.FieldStorage() >> store = form.getfirst('store') >> cat = form.getfirst('cat') >> id = form.getfirst('id') >> pkg = form.getfirst('pkg') >> patientID = form.getfirst('patientID') >> >> Now, apparently because of python's problem with globals, when I call "id" >> as follows: >> >> cursor.execute('select ProductID from productsPackages where PackageID=%s' >> % id) >> >> I get the following error: >> >> /var/www/html/angrynates.com/cart/Store_frame2.py < >> http://angrynates.com/cart/Store_frame2.py> >> >> 135 cursor.close() >> 136 bottom() >> 137 >> 138 Store_frame2() >> 139 >> Store_frame2 = >> /var/www/html/angrynates.com/cart/Store_frame2.py < >> http://angrynates.com/cart/Store_frame2.py> in Store_frame2() >> >> 119 printAProduct() >> 120 else: >> 121 cursor.execute('select ProductID from productsPackages where >> PackageID=%s' % id) >> 122 for id in [itm[0] for itm in cursor]: >> 123 printAProduct(id) >> global cursor = , cursor.execute = > method Cursor.execute of >, global id = '1' >> >> UnboundLocalError: local variable 'id' referenced before assignment >> args = ("local variable 'id' referenced before assignment",) >> >> Full code below. Please advise. >> >> Line 121 refers to 'id', but line 122 assigns to 'id' (the 'for' loop). > > In a function, if you assign to a variable then that variable defaults > to being local. You're trying to use the local variable before assigning > a value to it. Having a local variable called 'id' means that you can't > refer to the global variable of the same name. > > You're also assigning to 'i' on line 113, but not referring to it > elsewhere in the function. > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://www.astorandblack.com -- -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Wed Jan 20 12:06:05 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 20 Jan 2010 13:06:05 -0400 Subject: Interesting Problem In-Reply-To: <7a9c25c21001200830k5d40d1dcv6813933b96d13cbb@mail.gmail.com> References: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> <7a9c25c21001200830k5d40d1dcv6813933b96d13cbb@mail.gmail.com> Message-ID: <4dc0cfea1001200906h22843e0amf5b98e2ea7ad18bb@mail.gmail.com> On Wed, Jan 20, 2010 at 12:30 PM, Stephen Hansen wrote: > On Wed, Jan 20, 2010 at 7:19 AM, Victor Subervi wrote: > >> Hi; >> I think I finally have an interesting problem for y'all. I need to import >> a script from a lower dir, forcing me to change dirs: >> > > Don't do that. If you must, then the correct way to do it is to adjust your > sys.path and not change directory. "sys.path.append('..')" or whatever. > > But I really would just re-organize your code so you don't need such > things. Have a single top-level directory that's on your path, put a blank > __init__.py in all your directories, and use absolute imports everywhere. > > from myapp.templateFrame import top, bottom > from myapp.some_directory.some_file import this, that > > etc. > I didn't know I could do that. Thanks! > > > >> Now, apparently because of python's problem with globals, when I call "id" >> as follows: >> >> > Python does not have problems with globals. You are repeatedly re-using the > same variable names, causing confusion and errors; when you say 'id' in that > line of code, Python doesn't think you are talking about the "id" that is > global. It thinks you are talking about the "id" that is local-- that you > define one line beneath. > I didn't think that was a problem because it cascaded. Thanks, that fixed it! beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From chardster at gmail.com Wed Jan 20 12:18:27 2010 From: chardster at gmail.com (Richard Thomas) Date: Wed, 20 Jan 2010 09:18:27 -0800 (PST) Subject: Object Integer mapping References: <60e22629-c7a9-4d8f-930f-a3ed777b5f42@p24g2000yqm.googlegroups.com> Message-ID: On Jan 20, 4:43?pm, NighterNet wrote: > Need help on python version 3.1.x. I can't seem to know what going on > a bit. The code that I check that the hash is different every time. Is > there a way to make the hash the same? I using as to check the class > is the same variables but if it different variable in the class that > it add to the array and return the index. > > # Generic Object->Integer mapping > # the object must be usable as a dictionary key > class ObjMap: > ? ? ? ? def __init__(self): > ? ? ? ? ? ? ? ? self.dict = {} > ? ? ? ? ? ? ? ? self.next = 0 > ? ? ? ? def get(self, obj): > ? ? ? ? ? ? ? ? if obj in self.dict: > ? ? ? ? ? ? ? ? ? ? ? ? return self.dict[obj] > ? ? ? ? ? ? ? ? else: > ? ? ? ? ? ? ? ? ? ? ? ? id = self.next > ? ? ? ? ? ? ? ? ? ? ? ? self.next = self.next + 1 > ? ? ? ? ? ? ? ? ? ? ? ? self.dict[obj] = id > ? ? ? ? ? ? ? ? ? ? ? ? return id > > ? ? ? ? def items(self): > ? ? ? ? ? ? ? ? getval = operator.itemgetter(0) > ? ? ? ? ? ? ? ? getkey = operator.itemgetter(1) > ? ? ? ? ? ? ? ? return map(getval, sorted(self.dict.items(), key=getkey)) > > class Point: > ? ? ? ? def __init__(self): > ? ? ? ? ? ? ? ? self.x = 0 > ? ? ? ? ? ? ? ? self.y = 0 > ? ? ? ? ? ? ? ? self.z = 0 > > points = ObjMap() > > Testme = Point() > Testme.x = 0 > print(points.get(Testme)) > Testme2 = Point() > Testme2.y = 1 > print(points.get(Testme2)) > Testme3 = Point() > Testme3.y = 1 > print(points.get(Testme3)) > Ttestme4 = Point() > Ttestme4.y = 1 > print( points.get(Ttestme4)) > > It keep adding new array from this but doesn't match hash. Need help > on how to fix this class code. You need to define how the Point class hashes. You can do this by writing a __hash__ method like so: class Point(object): ... def __hash__(self): return hash(self.x) ^ hash(self.y) ^ hash(self.z) However you will also need to define how the Point class relates as equal using the __eq__ or __cmp__ methods. class Point(object): ... def __eq__(self, other): return self.x == other.x and self.y == other.y and self.z == other.z This addition makes sure that if two points a equivalent then they count as the same key in the dictionary. Richard. From robert.kern at gmail.com Wed Jan 20 12:43:20 2010 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 20 Jan 2010 11:43:20 -0600 Subject: Object Integer mapping In-Reply-To: References: <60e22629-c7a9-4d8f-930f-a3ed777b5f42@p24g2000yqm.googlegroups.com> Message-ID: On 2010-01-20 11:18 AM, Richard Thomas wrote: > On Jan 20, 4:43 pm, NighterNet wrote: >> Need help on python version 3.1.x. I can't seem to know what going on >> a bit. The code that I check that the hash is different every time. Is >> there a way to make the hash the same? I using as to check the class >> is the same variables but if it different variable in the class that >> it add to the array and return the index. >> >> # Generic Object->Integer mapping >> # the object must be usable as a dictionary key >> class ObjMap: >> def __init__(self): >> self.dict = {} >> self.next = 0 >> def get(self, obj): >> if obj in self.dict: >> return self.dict[obj] >> else: >> id = self.next >> self.next = self.next + 1 >> self.dict[obj] = id >> return id >> >> def items(self): >> getval = operator.itemgetter(0) >> getkey = operator.itemgetter(1) >> return map(getval, sorted(self.dict.items(), key=getkey)) >> >> class Point: >> def __init__(self): >> self.x = 0 >> self.y = 0 >> self.z = 0 >> >> points = ObjMap() >> >> Testme = Point() >> Testme.x = 0 >> print(points.get(Testme)) >> Testme2 = Point() >> Testme2.y = 1 >> print(points.get(Testme2)) >> Testme3 = Point() >> Testme3.y = 1 >> print(points.get(Testme3)) >> Ttestme4 = Point() >> Ttestme4.y = 1 >> print( points.get(Ttestme4)) >> >> It keep adding new array from this but doesn't match hash. Need help >> on how to fix this class code. > > You need to define how the Point class hashes. You can do this by > writing a __hash__ method like so: > > class Point(object): > ... > def __hash__(self): > return hash(self.x) ^ hash(self.y) ^ hash(self.z) > > However you will also need to define how the Point class relates as > equal using the __eq__ or __cmp__ methods. > > class Point(object): > ... > def __eq__(self, other): > return self.x == other.x and self.y == other.y and self.z == > other.z > > This addition makes sure that if two points a equivalent then they > count as the same key in the dictionary. I recommend a simpler approach that duplicates less code and makes it easier to maintain the __hash__/__eq__ invariants: class Point(object): ... def _key(self): # The type(self).__name__ bit is optional, but usually handy. return (type(self).__name__, self.x, self.y, self.z) def __hash__(self): return hash(self._key()) def __eq__(self, other): if not hasattr(other, '_key'): return False return self._key() == other._key() It is also worth noting that once one allows hashing by value, one should treat the objects as immutable, so one should not change the attributes as the OP does in his example. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From python at mrabarnett.plus.com Wed Jan 20 13:27:29 2010 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 20 Jan 2010 18:27:29 +0000 Subject: Interesting Problem In-Reply-To: <8a6b8e351001200849v6baf2597y3adb98d0d2d43636@mail.gmail.com> References: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> <4B57316F.4030009@mrabarnett.plus.com> <8a6b8e351001200849v6baf2597y3adb98d0d2d43636@mail.gmail.com> Message-ID: <4B574B11.80308@mrabarnett.plus.com> James Matthews wrote: > Also as a side point... It's best to anonymize the code as best you can > so people cannot attach your code to your site (IP theft and security risk) > [snip] Security risk, yes; IP theft, not so much. ;-) From alfps at start.no Wed Jan 20 13:27:34 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 20 Jan 2010 19:27:34 +0100 Subject: Best way to convert sequence of bytes to long integer In-Reply-To: <9b870cc2-0966-4cb2-be93-573317c06eb0@a6g2000yqm.googlegroups.com> References: <00e0377c$0$15570$c3e8da3@news.astraweb.com> <9b870cc2-0966-4cb2-be93-573317c06eb0@a6g2000yqm.googlegroups.com> Message-ID: * Mark Dickinson: > On Jan 20, 7:36 am, Steven D'Aprano cybersource.com.au> wrote: >> I have a byte string (Python 2.x string), e.g.: >> >> s = "g%$f yg\n1\05" >> assert len(s) == 10 >> >> I wish to convert it to a long integer, treating it as base-256. > > By the way, are you willing to divulge what you're using this > functionality for? When trying to hack out the API for int.to_bytes > and int.from_bytes on the mailing list and bug tracker, some of the > discussion about use-cases seemed a little too much like guesswork; > I'd be curious to find out about real-life use-cases. One possibility is that Steven wants to apply bitlevel and/or arithmetic operations for e.g. some custom cryptographic thing. The string that he uses as example is not completely unlike a message digest. Another possibility is custom serialization/deserialization. But it would be nice to know what it's actually about... Cheers, - Alf From ng1.ben at gmail.com Wed Jan 20 13:38:18 2010 From: ng1.ben at gmail.com (ben) Date: Wed, 20 Jan 2010 10:38:18 -0800 (PST) Subject: * operator in python tutorial Message-ID: <7b2ef0aa-8b93-4cb5-b944-beea3e2c98b1@p16g2000vbe.googlegroups.com> Hello, I am following through the python tutorial which gets to a line that uses the * operator with zip(). I searched and searched but could find no information on the operator or how to use it in general. The example from the tut is as follows: >>> x = [1, 2, 3] >>> y = [4, 5, 6] >>> zipped = zip(x, y) >>> zipped [(1, 4), (2, 5), (3, 6)] >>> x2, y2 = zip(*zipped) >>> x == list(x2) and y == list(y2) True How would i apply the * operator in general? Thanks. From ng1.ben at gmail.com Wed Jan 20 13:48:06 2010 From: ng1.ben at gmail.com (ben) Date: Wed, 20 Jan 2010 10:48:06 -0800 (PST) Subject: * operator in python tutorial References: <7b2ef0aa-8b93-4cb5-b944-beea3e2c98b1@p16g2000vbe.googlegroups.com> Message-ID: I missed it because I skipped to the explanation of zip. I hit the back arrow and went on and saw the link to "Unpacking Argument Lists" which explained what I needed to know. Thanks. From mal at egenix.com Wed Jan 20 14:11:55 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 20 Jan 2010 20:11:55 +0100 Subject: Interesting Problem In-Reply-To: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> References: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> Message-ID: <4B57557B.9070408@egenix.com> Victor Subervi wrote: > Hi; > I think I finally have an interesting problem for y'all. I need to import a > script from a lower dir, forcing me to change dirs: > > cwd = os.getcwd() > os.chdir('%s/..' % cwd) > sys.path.append(os.getcwd()) > from templateFrame import top, bottom > os.chdir(cwd) I suggest you read up on the built-in __import__() function: http://docs.python.org/library/functions.html#__import__ There's no need to change sys.path just to import a module. If you only need to execute some Python from a file, use execfile(): http://docs.python.org/library/functions.html#execfile -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 20 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From luismgz at gmail.com Wed Jan 20 14:13:07 2010 From: luismgz at gmail.com (=?ISO-8859-1?Q?Luis_M=2E_Gonz=E1lez?=) Date: Wed, 20 Jan 2010 11:13:07 -0800 (PST) Subject: What is a list compression in Python? References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> Message-ID: <55bd3026-c900-4a6a-b420-2d0d69e4b314@l30g2000yqb.googlegroups.com> On Jan 18, 1:07?pm, Kit wrote: > Hello Everyone, I am not sure if I have posted this question in a > correct board. Can anyone please teach me: > > What is a list compression in Python? > > Would you mind give me some list compression examples? > > Thanks & really appreciate that. > Kit It's also worth noting that from Python 3.0 on, you have also dict and set comprehensions. From mal at egenix.com Wed Jan 20 14:22:40 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 20 Jan 2010 20:22:40 +0100 Subject: Using jython to call python procedures/methods In-Reply-To: References: Message-ID: <4B575800.8000202@egenix.com> KB wrote: > Hi there, > > I have an application that only publishes a Java API. I can use jython > to access java classes, but jython currently (to the best of my > knowledge) does not support numpy/scipy. > > Ideally I would like to have jython call a "native" python routine > where I have the numpy/scipy procedures already written. > > Does anyone have any experience with this? Is it possible? > > I had toyed with the idea of having jython/java write the data to a > file/database and then manually kick off the python process, but > ideally I would like this as automated as possible. Have a look at this talk for some inspiration: http://www.slideshare.net/onyame/mixing-python-and-java Here's an example of using Pyro to connect Jython and CPython: http://www.razorvine.net/python/PyroAndJython As always with such solutions, there are a lot of weird problems to be expected :-/ -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 20 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From george.trojan at noaa.gov Wed Jan 20 16:03:10 2010 From: george.trojan at noaa.gov (George Trojan) Date: Wed, 20 Jan 2010 21:03:10 +0000 Subject: html code generation Message-ID: I need an advice on table generation. The table is essentially a fifo, containing about 200 rows. The rows are inserted every few minutes or so. The simplest solution is to store row data per line and write directly html code: line = "value1value2>... " each run of the program would read the previous table into a list of lines, insert the first row and drop the last one, taking care of table header and trailer. Is there a more classy solution? George From steven at REMOVE.THIS.cybersource.com.au Wed Jan 20 16:23:24 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 20 Jan 2010 21:23:24 GMT Subject: Best way to convert sequence of bytes to long integer References: <00e0377c$0$15570$c3e8da3@news.astraweb.com> <9b870cc2-0966-4cb2-be93-573317c06eb0@a6g2000yqm.googlegroups.com> Message-ID: On Wed, 20 Jan 2010 19:27:34 +0100, Alf P. Steinbach wrote: > * Mark Dickinson: >> On Jan 20, 7:36 am, Steven D'Aprano > cybersource.com.au> wrote: >>> I have a byte string (Python 2.x string), e.g.: >>> >>> s = "g%$f yg\n1\05" >>> assert len(s) == 10 >>> >>> I wish to convert it to a long integer, treating it as base-256. >> >> By the way, are you willing to divulge what you're using this >> functionality for? When trying to hack out the API for int.to_bytes >> and int.from_bytes on the mailing list and bug tracker, some of the >> discussion about use-cases seemed a little too much like guesswork; I'd >> be curious to find out about real-life use-cases. > > One possibility is that Steven wants to apply bitlevel and/or arithmetic > operations for e.g. some custom cryptographic thing. The string that he > uses as example is not completely unlike a message digest. Good guess! I'm writing a module that handles, I won't call it encryption, obfuscation using classical cryptographic algorithms. One of the functions needs a deterministic but unpredictable integer generated from a human-generated password or passphrase, so I'm using: hashlib.md5(key).digest() to get a string of bytes, then converting it to an int. int.from_bytes would be perfect for me, but in the meantime I'm using Paul Rubin's trick of int(s.encode("hex"), 16). Thanks to all who responded. -- Steven From sccolbert at gmail.com Wed Jan 20 16:52:07 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Wed, 20 Jan 2010 16:52:07 -0500 Subject: html code generation In-Reply-To: References: Message-ID: <7f014ea61001201352k639123b4m405c9e272bf628d6@mail.gmail.com> use a deque with a 'junk' as each element http://docs.python.org/library/collections.html On Wed, Jan 20, 2010 at 4:03 PM, George Trojan wrote: > I need an advice on table generation. The table is essentially a fifo, > containing about 200 rows. The rows are inserted every few minutes or so. > The simplest solution is to store row data per line and write directly html > code: > line = "value1value2>... " > each run of the program would read the previous table into a list of lines, > insert the first row and drop the last one, taking care of table header and > trailer. > Is there a more classy solution? > > George > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sccolbert at gmail.com Wed Jan 20 16:55:25 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Wed, 20 Jan 2010 16:55:25 -0500 Subject: html code generation In-Reply-To: <7f014ea61001201352k639123b4m405c9e272bf628d6@mail.gmail.com> References: <7f014ea61001201352k639123b4m405c9e272bf628d6@mail.gmail.com> Message-ID: <7f014ea61001201355i3d4071d6md171c1b7d234c859@mail.gmail.com> On Wed, Jan 20, 2010 at 4:52 PM, Chris Colbert wrote: > use a deque with a 'junk' as each element > > http://docs.python.org/library/collections.html > > On Wed, Jan 20, 2010 at 4:03 PM, George Trojan wrote: > >> I need an advice on table generation. The table is essentially a fifo, >> containing about 200 rows. The rows are inserted every few minutes or so. >> The simplest solution is to store row data per line and write directly html >> code: >> line = "value1value2>... " >> each run of the program would read the previous table into a list of >> lines, insert the first row and drop the last one, taking care of table >> header and trailer. >> Is there a more classy solution? >> >> George >> -- >> http://mail.python.org/mailman/listinfo/python-list > > In [1]: from collections import deque In [2]: a = deque(['foo', 'bar']) In [3]: ''.join(a) Out[3]: 'foobar' In [4]: a.popleft() Out[4]: 'foo' In [5]: a.append('baz') In [6]: ''.join(a) Out[6]: 'barbaz' -------------- next part -------------- An HTML attachment was scrubbed... URL: From sccolbert at gmail.com Wed Jan 20 16:59:38 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Wed, 20 Jan 2010 16:59:38 -0500 Subject: html code generation In-Reply-To: <7f014ea61001201355i3d4071d6md171c1b7d234c859@mail.gmail.com> References: <7f014ea61001201352k639123b4m405c9e272bf628d6@mail.gmail.com> <7f014ea61001201355i3d4071d6md171c1b7d234c859@mail.gmail.com> Message-ID: <7f014ea61001201359x1444106frd38f58bf9cf7dfde@mail.gmail.com> ah ok, i misread your post. Store each 'junk' as an item in the deque. the idea is the same though. On Wed, Jan 20, 2010 at 4:55 PM, Chris Colbert wrote: > > > On Wed, Jan 20, 2010 at 4:52 PM, Chris Colbert wrote: > >> use a deque with a 'junk' as each element >> >> http://docs.python.org/library/collections.html >> >> On Wed, Jan 20, 2010 at 4:03 PM, George Trojan wrote: >> >>> I need an advice on table generation. The table is essentially a fifo, >>> containing about 200 rows. The rows are inserted every few minutes or so. >>> The simplest solution is to store row data per line and write directly html >>> code: >>> line = "value1value2>... " >>> each run of the program would read the previous table into a list of >>> lines, insert the first row and drop the last one, taking care of table >>> header and trailer. >>> Is there a more classy solution? >>> >>> George >>> -- >>> http://mail.python.org/mailman/listinfo/python-list >> >> > In [1]: from collections import deque > > In [2]: a = deque(['foo', 'bar']) > > In [3]: ''.join(a) > Out[3]: 'foobar' > > In [4]: a.popleft() > Out[4]: 'foo' > > In [5]: a.append('baz') > > In [6]: ''.join(a) > Out[6]: 'barbaz' > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From darkneter at gmail.com Wed Jan 20 17:01:14 2010 From: darkneter at gmail.com (NighterNet) Date: Wed, 20 Jan 2010 14:01:14 -0800 (PST) Subject: Object Integer mapping References: <60e22629-c7a9-4d8f-930f-a3ed777b5f42@p24g2000yqm.googlegroups.com> Message-ID: <0209aa58-d355-4a6a-9061-216ea7aa8c59@e16g2000yqc.googlegroups.com> On Jan 20, 9:43?am, Robert Kern wrote: > On 2010-01-20 11:18 AM, Richard Thomas wrote: > > > > > On Jan 20, 4:43 pm, NighterNet ?wrote: > >> Need help on python version 3.1.x. I can't seem to know what going on > >> a bit. The code that I check that the hash is different every time. Is > >> there a way to make the hash the same? I using as to check the class > >> is the same variables but if it different variable in the class that > >> it add to the array and return the index. > > >> # Generic Object->Integer mapping > >> # the object must be usable as a dictionary key > >> class ObjMap: > >> ? ? ? ? ?def __init__(self): > >> ? ? ? ? ? ? ? ? ?self.dict = {} > >> ? ? ? ? ? ? ? ? ?self.next = 0 > >> ? ? ? ? ?def get(self, obj): > >> ? ? ? ? ? ? ? ? ?if obj in self.dict: > >> ? ? ? ? ? ? ? ? ? ? ? ? ?return self.dict[obj] > >> ? ? ? ? ? ? ? ? ?else: > >> ? ? ? ? ? ? ? ? ? ? ? ? ?id = self.next > >> ? ? ? ? ? ? ? ? ? ? ? ? ?self.next = self.next + 1 > >> ? ? ? ? ? ? ? ? ? ? ? ? ?self.dict[obj] = id > >> ? ? ? ? ? ? ? ? ? ? ? ? ?return id > > >> ? ? ? ? ?def items(self): > >> ? ? ? ? ? ? ? ? ?getval = operator.itemgetter(0) > >> ? ? ? ? ? ? ? ? ?getkey = operator.itemgetter(1) > >> ? ? ? ? ? ? ? ? ?return map(getval, sorted(self.dict.items(), key=getkey)) > > >> class Point: > >> ? ? ? ? ?def __init__(self): > >> ? ? ? ? ? ? ? ? ?self.x = 0 > >> ? ? ? ? ? ? ? ? ?self.y = 0 > >> ? ? ? ? ? ? ? ? ?self.z = 0 > > >> points = ObjMap() > > >> Testme = Point() > >> Testme.x = 0 > >> print(points.get(Testme)) > >> Testme2 = Point() > >> Testme2.y = 1 > >> print(points.get(Testme2)) > >> Testme3 = Point() > >> Testme3.y = 1 > >> print(points.get(Testme3)) > >> Ttestme4 = Point() > >> Ttestme4.y = 1 > >> print( points.get(Ttestme4)) > > >> It keep adding new array from this but doesn't match hash. Need help > >> on how to fix this class code. > > > You need to define how the Point class hashes. You can do this by > > writing a __hash__ method like so: > > > class Point(object): > > ? ? ?... > > ? ? ?def __hash__(self): > > ? ? ? ? ?return hash(self.x) ^ hash(self.y) ^ hash(self.z) > > > However you will also need to define how the Point class relates as > > equal using the __eq__ or __cmp__ methods. > > > class Point(object): > > ? ? ?... > > ? ? ?def __eq__(self, other): > > ? ? ? ? ?return self.x == other.x and self.y == other.y and self.z == > > other.z > > > This addition makes sure that if two points a equivalent then they > > count as the same key in the dictionary. > > I recommend a simpler approach that duplicates less code and makes it easier to > maintain the __hash__/__eq__ invariants: > > class Point(object): > ? ? ?... > ? ? ?def _key(self): > ? ? ? ? ?# The type(self).__name__ bit is optional, but usually handy. > ? ? ? ? ?return (type(self).__name__, self.x, self.y, self.z) > > ? ? ?def __hash__(self): > ? ? ? ? ?return hash(self._key()) > > ? ? ?def __eq__(self, other): > ? ? ? ? ?if not hasattr(other, '_key'): > ? ? ? ? ? ? ?return False > ? ? ? ? ?return self._key() == other._key() > > It is also worth noting that once one allows hashing by value, one should treat > the objects as immutable, so one should not change the attributes as the OP does > in his example. > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless enigma > ? that is made terrible by our own mad attempt to interpret it as though it had > ? an underlying truth." > ? ?-- Umberto Eco Thanks you, it help fix my hash check against my classes. From darcy at druid.net Wed Jan 20 17:03:42 2010 From: darcy at druid.net (D'Arcy J.M. Cain) Date: Wed, 20 Jan 2010 17:03:42 -0500 Subject: html code generation In-Reply-To: References: Message-ID: <20100120170342.ba73aeb1.darcy@druid.net> On Wed, 20 Jan 2010 21:03:10 +0000 George Trojan wrote: > I need an advice on table generation. The table is essentially a fifo, > containing about 200 rows. The rows are inserted every few minutes or > so. The simplest solution is to store row data per line and write > directly html code: > line = "value1value2>... " > each run of the program would read the previous table into a list of > lines, insert the first row and drop the last one, taking care of table > header and trailer. > Is there a more classy solution? Almost positively. It's hard to say for sure though without knowing more. However, I have a few pointers for you. First, think about a proper database. You can store the raw data into something like PostgreSQL and then you have your historical record. Then you can extract and format the latest 200 records at run time. You can change that value any time you want and even look at historical information such as the output exactly 24 hours ago. Whether you generate the page on demand or pre-calculate at intervals will depend on the ratio of updates to displays. I would start with generating on demand to start with and profile usage. Look into server side HTML. It can be a bit ugly generating your entire web page in code. Write the static part as a regular file and include your small Python script to generate the table data. The cool thing with that is that you can give your webaster the task of making pretty pages and simply deal with the variable generation. HTH. -- D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner. From iainspeed at gmail.com Wed Jan 20 17:45:44 2010 From: iainspeed at gmail.com (Iain Barnett) Date: Wed, 20 Jan 2010 22:45:44 +0000 Subject: installing psycopg2-2.0.13 with python3.1 Message-ID: <03495BC1-8736-42D5-9B59-5B0148F643AA@gmail.com> Hi, Would anyone know if it's possible to install psycopg2-2.0.13 with python3.1.1 (or similar)? When I try I get the following $ sudo python setup.py install Password: File "setup.py", line 233 except Warning, w: ^ SyntaxError: invalid syntax I can install it with python2.6 with no problems, but obviously I'd prefer to use the latest version. My system is OSX10.6, and I'm new to Python. Any help is much appreciated. Iain From gagsl-py2 at yahoo.com.ar Wed Jan 20 18:09:17 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Wed, 20 Jan 2010 20:09:17 -0300 Subject: Inheriting methods but over-riding docstrings References: <0361ea2d$0$1336$c3e8da3@news.astraweb.com> <28aa31f4-7331-4ac2-918b-0c59f5a834de@e37g2000yqn.googlegroups.com> Message-ID: En Tue, 19 Jan 2010 08:44:09 -0300, Michele Simionato escribi?: > On Jan 16, 6:55 pm, Steven D'Aprano cybersource.com.au> wrote: >> I have a series of subclasses that inherit methods from a base class, >> but >> I'd like them to have their own individual docstrings. > > from types import FunctionType, CodeType > > def newfunc(func, docstring): > c = func.func_code > nc = CodeType(c.co_argcount, c.co_nlocals, c.co_stacksize, > c.co_flags, c.co_code, c.co_consts, c.co_names, > c.co_varnames, c.co_filename, func.__name__, > c.co_firstlineno, c.co_lnotab, c.co_freevars, > c.co_cellvars) > nf = FunctionType(nc, func.func_globals, func.__name__) > nf.__doc__ = docstring > return nf > > def setdocstring(method, docstring): > cls = method.im_class > basefunc = getattr(super(cls, cls), method.__name__).im_func > setattr(cls, method.__name__, newfunc(basefunc, docstring)) > > class B(object): > def m(self): > "base" > return 'ok' > > class C(B): > pass > > setdocstring(C.m, 'C.m docstring') This is basically the same technique as in but there is a difference: you clone the function object *and* the code object it is based on. As I understand it, code objects are immutable and there is no need to clone them, but I may be wrong. Why did you feel the need to clone the code object too? -- Gabriel Genellina From joncle at googlemail.com Wed Jan 20 18:15:01 2010 From: joncle at googlemail.com (Jon Clements) Date: Wed, 20 Jan 2010 15:15:01 -0800 (PST) Subject: html code generation References: Message-ID: <5a451761-8bd8-446d-bb5d-35017661a589@r24g2000yqd.googlegroups.com> On Jan 20, 10:03?pm, "D'Arcy J.M. Cain" wrote: > On Wed, 20 Jan 2010 21:03:10 +0000 > > George Trojan wrote: > > I need an advice on table generation. The table is essentially a fifo, > > containing about 200 rows. The rows are inserted every few minutes or > > so. The simplest solution is to store row data per line and write > > directly html code: > > line = "value1value2>... " > > each run of the program would read the previous table into a list of > > lines, insert the first row and drop the last one, taking care of table > > header and trailer. > > Is there a more classy solution? > > Almost positively. ?It's hard to say for sure though without knowing > more. ?However, I have a few pointers for you. > > First, think about a proper database. ?You can store the raw data into > something like PostgreSQL and then you have your historical record. > Then you can extract and format the latest 200 records at run time. ?You > can change that value any time you want and even look at historical > information such as the output exactly 24 hours ago. > > Whether you generate the page on demand or pre-calculate at intervals > will depend on the ratio of updates to displays. ?I would start with > generating on demand to start with and profile usage. > > Look into server side HTML. ?It can be a bit ugly generating your > entire web page in code. ?Write the static part as a regular file and > include your small Python script to generate the table data. ?The cool > thing with that is that you can give your webaster the task of making > pretty pages and simply deal with the variable generation. > > HTH. > > -- > D'Arcy J.M. Cain ? ? ? ? | ?Democracy is three wolveshttp://www.druid.net/darcy/? ? ? ? ? ? ? ?| ?and a sheep voting on > +1 416 425 1212 ? ? (DoD#0082) ? ?(eNTP) ? | ?what's for dinner. I'd second this and go for a DB solution; possibly even SQLite to start with. I would tend to go with a web framework (maybe OTT) or a templating engine, so if you wish, someone can do the CSS/HTML and you just provide something that says (gimme the last 200 hundred rows). Very similar to what D'Arcy J.M. Cain has said but should hopefully formalise the possible problem and be applicable across the project. Further more you might want to look into AJAX, but ummm, that's a completely new debate :) Cheers, Jon. From nils at ccsg.de Wed Jan 20 18:30:21 2010 From: nils at ccsg.de (Nils Ruettershoff) Date: Thu, 21 Jan 2010 00:30:21 +0100 Subject: multiprocessing problems In-Reply-To: <63ef9fe9-6e9b-4766-9a0c-4aebc28d4148@p24g2000yqm.googlegroups.com> References: <74594510-9366-4227-8e1e-d221a9220271@k35g2000yqb.googlegroups.com> <63c5df56-7b86-4ee7-ba4f-b14984f94097@e16g2000yqc.googlegroups.com> <63ef9fe9-6e9b-4766-9a0c-4aebc28d4148@p24g2000yqm.googlegroups.com> Message-ID: <4B57920D.1030601@ccsg.de> Hi Doxa, DoxaLogos wrote: [...] > I found out my problems. One thing I did was followed the test queue > example in the documentation, but the biggest problem turned out to be > a pool instantiated globally in my script was causing most of the > endless process spawn, even with the "if __name__ == "__main__":" > block. > Problems who solves them self, are the best problems ;) One tip: currently your algorithm has some overhead. 'Cause you are starting 4 time an additional python interpreter, compute the files and, closing all new spawned interpreter and starting again 4 interpreter, which are processing the files. For such kind of jobs I prefer to start processes once and feeding them with data via a queue. This reduces some overhead and increase runtime performance. This could look like this: (due some pseudo functions not directly executeable -> untested) import multiprocessing import Queue class Worker(multiprocessing.Process): def __init__(self, feeder_q, queue_filled): multiprocessing.Process.__init__(self) self.feeder_q = feeder_q self.queue_filled = queue_filled def run(self): serve = True # start infinite loop while serve: try: # scan queue for work, will block process up to 5 seconds. If until then no item is in queue a Queue.Empty will be raised text = self.feeder_q.get(True, timeout=5) if text: do_stuff(text) # very important! tell the queue that the fetched work has been finished # otherwise the feeder_q.join() would block infinite self.input_queue.task_done() except Queue.Empty: # as soon as queue is empty and all work has been enqueued # process can terminate itself if self.queue_filled.is_set() and feeder_q.empty(): serve = False return if __name__ == '__main__': number_of_processes = 4 queue_filled = multiprocessing.Event() feeder_q = multiprocessing.JoinableQueue() process_list =[] # get file name which need to be processed all_files = get_all_files() # start processes for i in xrange(0,number_of_processes): process = Worker(feeder_q, queue_filled) process.start() process_list.append(thread) # start feeding for file in all_files: feeder_q.put(file) # inform processes that all work has been ordered queue_filled.set() # wait until queue is empty feeder_q.join() # wait until all processed have finished their jobs for process in process_list: process.join() Cheers, Nils From nils at ccsg.de Wed Jan 20 18:45:32 2010 From: nils at ccsg.de (Nils Ruettershoff) Date: Thu, 21 Jan 2010 00:45:32 +0100 Subject: multiprocessing problems In-Reply-To: <1263918409.3165.15.camel@linux-m3mt> References: <74594510-9366-4227-8e1e-d221a9220271@k35g2000yqb.googlegroups.com> <1263918409.3165.15.camel@linux-m3mt> Message-ID: <4B57959C.4020201@ccsg.de> Hi, Adam Tauno Williams wrote: [...] >> Here's the guts of my latest incarnation. >> def ProcessBatch(files): >> p = [] >> for file in files: >> p.append(Process(target=ProcessFile,args=file)) >> for x in p: >> x.start() >> for x in p: >> x.join() >> p = [] >> return >> Now, the function calling ProcessBatch looks like this: >> def ReplaceIt(files): >> processFiles = [] >> for replacefile in files: >> if(CheckSkipFile(replacefile)): >> processFiles.append(replacefile) >> if(len(processFiles) == 4): >> ProcessBatch(processFiles) >> processFiles = [] >> #check for left over files once main loop is done and process them >> if(len(processFiles) > 0): >> ProcessBatch(processFiles) >> > > According to this you will create files is sets of four, but an unknown > number of sets of four. > > This is not correct, 'cause of the x.join(). This will block the parent process until all processes have been terminated. So as soon as the current set of processes have finished their job, a new set will be spawned. Cheers, Nils -------------- next part -------------- An HTML attachment was scrubbed... URL: From philip at semanchuk.com Wed Jan 20 18:58:37 2010 From: philip at semanchuk.com (Philip Semanchuk) Date: Wed, 20 Jan 2010 18:58:37 -0500 Subject: installing psycopg2-2.0.13 with python3.1 In-Reply-To: <03495BC1-8736-42D5-9B59-5B0148F643AA@gmail.com> References: <03495BC1-8736-42D5-9B59-5B0148F643AA@gmail.com> Message-ID: On Jan 20, 2010, at 5:45 PM, Iain Barnett wrote: > Hi, > > Would anyone know if it's possible to install psycopg2-2.0.13 with > python3.1.1 (or similar)? When I try I get the following > > $ sudo python setup.py install > Password: > File "setup.py", line 233 > except Warning, w: > ^ > SyntaxError: invalid syntax > > > I can install it with python2.6 with no problems, but obviously I'd > prefer to use the latest version. My system is OSX10.6, and I'm new > to Python. Hi Iain, I've been out of the psycopg loop for a while now, but my guess is that the answer to your question is no. There are some Python 3- related checkins in the psycopg2 repository, but it looks like they're not ready for public use yet (unless you're adventurous). Here's the source repository, which contains a commit commented "First round of changes for Python 3": https://dndg.it/cgi-bin/gitweb.cgi?p=public/psycopg2.git If you scroll to the bottom of that page you'll see a "head" created especially for Python 2 which is where 2.0.13 came from. If you're willing to do some hacking on your own, Martin v. L?wis ported psycopg2 to Python 3 and provided a diff: http://mail.python.org/pipermail/python-porting/2008-December/000018.html Last but not least, I should note that psycopg has its own mailing list, and I bet they know more about it there: http://lists.initd.org/mailman/listinfo/psycopg Hope this helps. Psycopg & Postgres both worked wonderfully well for me. Cheers Philip From gagsl-py2 at yahoo.com.ar Wed Jan 20 19:11:49 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Wed, 20 Jan 2010 21:11:49 -0300 Subject: installing psycopg2-2.0.13 with python3.1 References: <03495BC1-8736-42D5-9B59-5B0148F643AA@gmail.com> Message-ID: En Wed, 20 Jan 2010 19:45:44 -0300, Iain Barnett escribi?: > Would anyone know if it's possible to install psycopg2-2.0.13 with > python3.1.1 (or similar)?I can install it with python2.6 with no > problems, but obviously I'd prefer to use the latest version. My system > is OSX10.6, and I'm new to Python. If you're new to Python, perhaps it's better to stay with the 2.6 version. Python 3.x introduced some incompatible changes in the language; not all third-party packages are available for Python 3.x yet. Regarding psycopg2, although some work has been done [1], there is no "official" release compatible with Python 3 yet. If you insist on using Python 3.1, there is another interface to PostgreSQL called pg8000 that claims to be Python 3.x compatible (I've not actually tested it). [1] http://mail.python.org/pipermail/python-porting/2008-December/000004.html [2] http://pybrary.net/pg8000/ -- Gabriel Genellina From g.bogle at auckland.no.spam.ac.nz Wed Jan 20 19:22:19 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Thu, 21 Jan 2010 13:22:19 +1300 Subject: py2exe and pydocs. Downloads? References: Message-ID: Gabriel Genellina wrote: > c:\temp>python -m pydoc sys > Help on built-in module sys: > [...same info...] When I do this I get: No module named tempfile From icanbob at gmail.com Wed Jan 20 20:00:10 2010 From: icanbob at gmail.com (bobicanprogram) Date: Wed, 20 Jan 2010 17:00:10 -0800 (PST) Subject: Using jython to call python procedures/methods References: Message-ID: On Jan 20, 10:32 am, KB wrote: > Hi there, > > I have an application that only publishes a Java API. I can use jython > to access java classes, but jython currently (to the best of my > knowledge) does not support numpy/scipy. > > Ideally I would like to have jython call a "native" python routine > where I have the numpy/scipy procedures already written. > > Does anyone have any experience with this? Is it possible? > > I had toyed with the idea of having jython/java write the data to a > file/database and then manually kick off the python process, but > ideally I would like this as automated as possible. > > Thanks in advance. You might want to take a look at the SIMPL toolkit (http:// www.icanprogram.com/simpl). It sports both JAVA and Python hooks so you should be able to pump the data from your JAVA module to a Python/ numpy/scipy module. bob From zuo at chopin.edu.pl Wed Jan 20 20:02:02 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Thu, 21 Jan 2010 02:02:02 +0100 Subject: Sorted dictionary Message-ID: Hello, Inspired by some my needs as well as some discussions in the net, I've implemented a sorted dictionary (i.e. a dict that returns keys, values and items always sorted by keys): http://code.activestate.com/recipes/576998/ Maybe it'll appear to be useful for somebody... And I'm curious about your opinions. Regards, *j -- Jan Kaliszewski (zuo) From someone at somewhere.com Wed Jan 20 20:30:25 2010 From: someone at somewhere.com (Gringo) Date: Wed, 20 Jan 2010 19:30:25 -0600 Subject: * operator in python tutorial In-Reply-To: <7b2ef0aa-8b93-4cb5-b944-beea3e2c98b1@p16g2000vbe.googlegroups.com> References: <7b2ef0aa-8b93-4cb5-b944-beea3e2c98b1@p16g2000vbe.googlegroups.com> Message-ID: <0036df65$0$25085$c3e8da3@news.astraweb.com> On 1/20/2010 12:38, ben wrote: > Hello, > > I am following through the python tutorial which gets to a line that > uses the * operator with zip(). I searched and searched but could find > no information on the operator or how to use it in general. The > example from the tut is as follows: >>>> x = [1, 2, 3] >>>> y = [4, 5, 6] >>>> zipped = zip(x, y) >>>> zipped > [(1, 4), (2, 5), (3, 6)] >>>> x2, y2 = zip(*zipped) >>>> x == list(x2) and y == list(y2) > True > > How would i apply the * operator in general? > Thanks. The * operator, when used in this context, unpacks the sequence and it's as if you passed each item to the function as a different parameter. For example, if you have a list x with 4 items, these two statements would be the same: f(x[0],x[1],x[2],x[3]) f(*x) Hope this helps. From steven at REMOVE.THIS.cybersource.com.au Wed Jan 20 20:50:58 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 21 Jan 2010 01:50:58 GMT Subject: Sorted dictionary References: Message-ID: On Thu, 21 Jan 2010 02:02:02 +0100, Jan Kaliszewski wrote: > Hello, > > Inspired by some my needs as well as some discussions in the net, I've > implemented a sorted dictionary (i.e. a dict that returns keys, values > and items always sorted by keys): > > http://code.activestate.com/recipes/576998/ What's the advantage of that over sorting the keys as needed? E.g. for key in sorted(dict): print key works. -- Steven From gnarlodious at gmail.com Wed Jan 20 20:56:16 2010 From: gnarlodious at gmail.com (Gnarlodious) Date: Wed, 20 Jan 2010 17:56:16 -0800 (PST) Subject: Py 3: How to switch application to Unicode strings? References: <92288825-8b01-47fb-8592-f65a7d7aeb8a@k35g2000yqb.googlegroups.com><7a9c25c21001191156j46a7fdadt58b728477b85e651@mail.gmail.com> <646ab38b-0710-4d31-b9e1-8a6ee7bfa42f@21g2000yqj.googlegroups.com> Message-ID: Thanks for the help, but I am going to skip this problem because I don't need Unicode characters in a script anyway. -- Gnarlie From gnarlodious at gmail.com Wed Jan 20 22:02:49 2010 From: gnarlodious at gmail.com (Gnarlodious) Date: Wed, 20 Jan 2010 19:02:49 -0800 (PST) Subject: Create object name from string value? Message-ID: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> I want to declare several objects from names in a list: objects=['object1', 'object2', 'object3', 'object4'] for objectName in objects: objectName=classname() That fails, and so do these: exec(objectName)=classname() eval(objectName)=classname() So how to make an object whose name is the value in the variable? -- Gnarlie From ben+python at benfinney.id.au Wed Jan 20 22:31:11 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 21 Jan 2010 14:31:11 +1100 Subject: Create object name from string value? References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> Message-ID: <87ska0dthc.fsf@benfinney.id.au> Gnarlodious writes: > I want to declare several objects from names in a list: > > objects=['object1', 'object2', 'object3', 'object4'] > for objectName in objects: > objectName=classname() Since (I presume) you are a newcomer to Python, it's best to learn the common style . I will assume the above is written as:: object_names = ['object1', 'object2', 'object3', 'object4'] for object_name in object_names: object_name = Class() > That fails Well, it succeeds (runs successfully). But I get what you mean: it fails to do what you expected. > So how to make an object whose name is the value in the variable? This is almost certainly better done with a dictionary. Like so:: object_names = ['object1', 'object2', 'object3', 'object4'] objects = {} for object_name in object_names: objects[object_name] = Class() There are shortcuts that can be used to create the dictionary without the separate list of names. But for now the above should make sense immediately and help you get to the meat of the problem. -- \ ?We now have access to so much information that we can find | `\ support for any prejudice or opinion.? ?David Suzuki, 2008-06-27 | _o__) | Ben Finney From gnarlodious at gmail.com Wed Jan 20 23:36:33 2010 From: gnarlodious at gmail.com (Gnarlodious) Date: Wed, 20 Jan 2010 20:36:33 -0800 (PST) Subject: Create object name from string value? References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> <87ska0dthc.fsf@benfinney.id.au> Message-ID: <0fa4a0fa-b813-4ba4-b032-8a4d6fbe0132@m26g2000yqb.googlegroups.com> On Jan 20, 8:31?pm, Ben Finney wrote: > Since (I presume) you are a newcomer to Python, it's best to learn the > common style . Thanks for that. > This is almost certainly better done with a dictionary. Like so:: > > ? ? object_names = ['object1', 'object2', 'object3', 'object4'] > ? ? objects = {} > ? ? for object_name in object_names: > ? ? ? ? objects[object_name] = Class() And thanks for that too. I see it returns a list of objects, Python is very cool. -- Gnarlie From qq263020776 at gmail.com Wed Jan 20 23:56:07 2010 From: qq263020776 at gmail.com (yousay) Date: Wed, 20 Jan 2010 20:56:07 -0800 (PST) Subject: why the super class can access the subclass's attribute Message-ID: I have sees aprogram like this ,i confusing why super class can access the subclass's attribute ,this is the program,thanks in advance: class MyThread(threading.Thread): def join(self): super(MyThread,self).join() return self.result class Worker(MyThread): import random import pdb pdb.set_trace() def run(self): total = 0 for i in range(random.randrange(10,100)): total +=i self.result = total From steven at REMOVE.THIS.cybersource.com.au Thu Jan 21 00:35:54 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 21 Jan 2010 05:35:54 GMT Subject: Create object name from string value? References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> Message-ID: On Wed, 20 Jan 2010 19:02:49 -0800, Gnarlodious wrote: > I want to declare several objects from names in a list: > > objects=['object1', 'object2', 'object3', 'object4'] for objectName in > objects: > objectName=classname() That's the wrong way to handle the problem. Named objects are only useful if you know the name of the object when writing the code. Otherwise, how do you know what name to use in the code? > That fails, and so do these: > > exec(objectName)=classname() > eval(objectName)=classname() The right way to solve this problem is with a dictionary: for name in ["object1", "object2", "object3"]: d = {name: classname()} print d[name] but for the record, the way to use exec is like this: exec("object1 = classname()") But beware, exec is not only much slower than the alternatives, but it risks putting a serious security flaw in your software. As a general rule, 9 times out of 10 if you think you want exec, you don't. -- Steven From wuwei23 at gmail.com Thu Jan 21 01:08:29 2010 From: wuwei23 at gmail.com (alex23) Date: Wed, 20 Jan 2010 22:08:29 -0800 (PST) Subject: Interesting Problem References: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> Message-ID: <2d6f733e-e00c-4b87-9ba1-a3d9d8478461@k35g2000yqb.googlegroups.com> Victor Subervi wrote: > I think I finally have an interesting problem for y'all. I need to import a > script from a lower dir, forcing me to change dirs: I'm surprised that no one has yet mentioned packages. Suppose you have the following folder layout and you stick an empty __init__.py in each subfolder, like so: app/ tools/ __init__.py open.py close.py extensions/ __init__.py inner.py outer.py Scripts in the 'app' folder can then import from the subfolders using dotted notation: import tools from tools import open, close import tools.extensions from tools.extensions import inner, outer However, with empty __init__ files, you can't use attribute lookup on nested modules, so if you started with an 'import tools' any attempts to reference the contents of the module in the following ways will fail: tools.open tools.close tools.ext.inner tools.ext.outer To provide this functionality, you need to import the items you want available in the __init__ of the package they belong to. So by adding the following: tools/__init__.py: import open, close, ext tools/ext/__init__.py: import inner, outer You can then do a single 'import tools' and use standard attribute lookup to access the contents of your subfolders. From michele.simionato at gmail.com Thu Jan 21 01:17:40 2010 From: michele.simionato at gmail.com (Michele Simionato) Date: Wed, 20 Jan 2010 22:17:40 -0800 (PST) Subject: Inheriting methods but over-riding docstrings References: <0361ea2d$0$1336$c3e8da3@news.astraweb.com> <28aa31f4-7331-4ac2-918b-0c59f5a834de@e37g2000yqn.googlegroups.com> Message-ID: <5ecdae50-b417-44bb-ae5b-c1d9aea865a2@y23g2000yqm.googlegroups.com> On Jan 21, 12:09?am, "Gabriel Genellina" wrote: > This is basically the same technique as in ? > but there is ? > a difference: you clone the function object *and* the code object it is ? > based on. As I understand it, code objects are immutable and there is no ? > need to clone them, but I may be wrong. Why did you feel the need to clone ? > the code object too? No need. I just had the newfunc utility in my library (I think copied from a recipe in the Python cookbook by Alex Martelli) so I used it. In this case it is overkill, though. Also, I had no read your post when I posted my solution, otherwise I would not have sent it ;) Anyway, the setdocstring utility to extract the parent method was not explicit in your post and may be of some use to somebody. M. Simionato From gagsl-py2 at yahoo.com.ar Thu Jan 21 01:40:49 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Thu, 21 Jan 2010 03:40:49 -0300 Subject: py2exe and pydocs. Downloads? References: Message-ID: En Wed, 20 Jan 2010 21:22:19 -0300, Gib Bogle escribi?: > Gabriel Genellina wrote: > >> c:\temp>python -m pydoc sys >> Help on built-in module sys: >> [...same info...] > > When I do this I get: > > No module named tempfile You found a bug. Looks like it depends on the environment, or what packages are installed, or something like that, because it worked on my other PC but not here. Please report it at http://bugs.python.org so it doesn't get forgotten. -- Gabriel Genellina From michele.simionato at gmail.com Thu Jan 21 01:51:04 2010 From: michele.simionato at gmail.com (Michele Simionato) Date: Wed, 20 Jan 2010 22:51:04 -0800 (PST) Subject: counting lines of code Message-ID: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> I need a small utility to count the lines of Python code in a directory, traversing subdirectories and ignoring comments and docstrings. I am sure there is already something doing that, what do you suggest? TIA, Michele Simionato From steven at REMOVE.THIS.cybersource.com.au Thu Jan 21 01:54:35 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 21 Jan 2010 06:54:35 GMT Subject: Sorted dictionary References: Message-ID: On Wed, 20 Jan 2010 22:21:22 -0800, Dennis Lee Bieber wrote: > On Thu, 21 Jan 2010 02:02:02 +0100, "Jan Kaliszewski" > declaimed the following in > gmane.comp.python.general: > >> Hello, >> >> Inspired by some my needs as well as some discussions in the net, I've >> implemented a sorted dictionary (i.e. a dict that returns keys, values >> and items always sorted by keys): >> > How does this differ from all the other "ordered dictionary" > schemes out there (and maybe even included in 2.7? I'm still on 2.5) Ordered dicts remember the insertion order of keys. Sorted dicts return the keys in sorted order. -- Steven From skippy.hammond at gmail.com Thu Jan 21 02:06:15 2010 From: skippy.hammond at gmail.com (Mark Hammond) Date: Thu, 21 Jan 2010 18:06:15 +1100 Subject: counting lines of code In-Reply-To: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> Message-ID: <4B57FCE7.5010704@gmail.com> On 21/01/2010 5:51 PM, Michele Simionato wrote: > I need a small utility to count the lines of Python code in a > directory, traversing subdirectories and ignoring comments and > docstrings. I am sure there is already something doing that, what do > you suggest? I suggest typing your subject line into google and hitting the "I feel lucky" button :) HTH, Mark From ben+python at benfinney.id.au Thu Jan 21 02:12:06 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 21 Jan 2010 18:12:06 +1100 Subject: counting lines of code References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> Message-ID: <87bpgodj95.fsf@benfinney.id.au> Michele Simionato writes: > I need a small utility to count the lines of Python code in a > directory, traversing subdirectories and ignoring comments and > docstrings. I am sure there is already something doing that, what do > you suggest? Any of the static code checkers (?pylint?, ?pyflakes?, etc.) would already be doing this. You would at least be able to crib from them, maybe import the functionality; ideally they can directly report what you want to see. -- \ ?My business is to teach my aspirations to conform themselves | `\ to fact, not to try and make facts harmonise with my | _o__) aspirations.? ?Thomas Henry Huxley, 1860-09-23 | Ben Finney From michele.simionato at gmail.com Thu Jan 21 02:16:47 2010 From: michele.simionato at gmail.com (Michele Simionato) Date: Wed, 20 Jan 2010 23:16:47 -0800 (PST) Subject: counting lines of code References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> Message-ID: I did not known about cloc, it does more that I need, but it looks cool (it is perl and not Python, by who cares? ;) Thanks, Michele From michele.simionato at gmail.com Thu Jan 21 02:20:13 2010 From: michele.simionato at gmail.com (Michele Simionato) Date: Wed, 20 Jan 2010 23:20:13 -0800 (PST) Subject: counting lines of code References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> <87bpgodj95.fsf@benfinney.id.au> Message-ID: <1f166703-fdfc-4f87-b336-ef80e492635b@a32g2000yqm.googlegroups.com> On Jan 21, 8:12?am, Ben Finney wrote: > Michele Simionato writes: > > I need a small utility to count the lines of Python code in a > > directory, traversing subdirectories and ignoring comments and > > docstrings. I am sure there is already something doing that, what do > > you suggest? > > Any of the static code checkers (?pylint?, ?pyflakes?, etc.) would > already be doing this. pylint does too many things, I want something fast that just counts the lines and can be run on thousands of files at once. cloc seems fine, I have just tried on 2,000 files and it gives me a report in just a few seconds. From clp2 at rebertia.com Thu Jan 21 02:38:56 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Wed, 20 Jan 2010 23:38:56 -0800 Subject: why the super class can access the subclass's attribute In-Reply-To: References: Message-ID: <50697b2c1001202338q3d944121i7a2705557aac271e@mail.gmail.com> On Wed, Jan 20, 2010 at 8:56 PM, yousay wrote: > I have sees aprogram like this ,i confusing why super class can access > the subclass's attribute To make this easy to understand, I'm going to ***drastically*** oversimplify. With that disclaimer out of the way... When you access an instance attribute, as in self.result, Python actually returns self.__dict__["result"]; that is, it does a dictionary lookup using the attribute name as a string as the key; instance attribute assignment works analogously. Every object has a special dictionary associated with it that is used to store its instance attributes; this dictionary can itself be accessed through the __dict__ attribute. (And if you're about to ask how the __dict__ attribute gets looked up, well, it's magic?!) This "instance attribute access is just dictionary manipulation" principle is universal, even in subclasses, superclasses, and unrelated objects. Thus, in both Worker and its superclass MyThread, accessing the `result` instance attribute manipulates the very same dictionary (self.__dict__) and thus works exactly the same in both cases. So, essentially, `result` is accessible to MyThread because there's nothing to prevent it from being accessible there. Other related principles that may aid in your understanding (these are not oversimplified): - Attribute lookups all happen dynamically at runtime - Python has no language-enforced notion of `protected` or `private` attributes; everything's public - Python is dynamically typed and thus does no compile-time typechecking Cheers, Chris -- Simplifications include overlooking __slots__, metaclasses, __getattribute__, and instance.class_attribute, among others. http://blog.rebertia.com > ,this is the program,thanks in advance: > class MyThread(threading.Thread): > ? ?def join(self): > ? ? ? ?super(MyThread,self).join() > ? ? ? ?return self.result > > class Worker(MyThread): > ? ?import random > ? ?import pdb > ? ?pdb.set_trace() > ? ?def run(self): > ? ? ? ?total = 0 > ? ? ? ?for i in range(random.randrange(10,100)): > ? ? ? ? ? ?total +=i > ? ? ? ?self.result = total From Martin.Drautzburg at web.de Thu Jan 21 02:43:12 2010 From: Martin.Drautzburg at web.de (Martin Drautzburg) Date: Thu, 21 Jan 2010 08:43:12 +0100 Subject: Symbols as parameters? Message-ID: <1495297.2JfmrJjJTh@beaureve.gmx.net> Hello all, When passing parameters to a function, you sometimes need a paramter which can only assume certain values, e.g. def move (direction): ... If direction can only be "up", "down", "left" or "right", you can solve this by passing strings, but this is not quite to the point: - you could pass invalid strings easily - you need to quote thigs, which is a nuisance - the parameter IS REALLY NOT A STRING, but a direction Alternatively you could export such symbols, so when you "import *" you have them available in the caller's namespace. But that forces you to "import *" which pollutes your namespace. What I am really looking for is a way - to be able to call move(up) - having the "up" symbol only in the context of the function call So it should look something like this ... magic, magic ... move(up) ... unmagic, unmagic ... print up This should complain that "up" is not defined during the "print" call, but not when move() is called. And of course there should be as little magic as possible. Any way to achieve this? From clp2 at rebertia.com Thu Jan 21 02:49:22 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Wed, 20 Jan 2010 23:49:22 -0800 Subject: Sorted dictionary In-Reply-To: References: Message-ID: <50697b2c1001202349m7b346df5q4686dfa51b29bc13@mail.gmail.com> On Wed, Jan 20, 2010 at 5:50 PM, Steven D'Aprano wrote: > On Thu, 21 Jan 2010 02:02:02 +0100, Jan Kaliszewski wrote: > >> Hello, >> >> Inspired by some my needs as well as some discussions in the net, I've >> implemented a sorted dictionary (i.e. a dict that returns keys, values >> and items always sorted by keys): >> >> http://code.activestate.com/recipes/576998/ > > > What's the advantage of that over sorting the keys as needed? > > > E.g. > > for key in sorted(dict): > ? ?print key > > > works. Well, it does spread out the cost of sorting the key list over each of the N distinct key assignments, versus sorting the entire list all at once, on-demand at the end of the process. And you avoid having to traverse the M buckets in the dictionary to locate the N keys in the first place. So it has different performance characteristics, which could theoretically matter depending on the use case; e.g. it could avoid a GUI application hanging for several seconds while it sorts a large list of keys. Cheers, Chris -- http://blog.rebertia.com From andreengels at gmail.com Thu Jan 21 03:18:32 2010 From: andreengels at gmail.com (Andre Engels) Date: Thu, 21 Jan 2010 09:18:32 +0100 Subject: Symbols as parameters? In-Reply-To: <1495297.2JfmrJjJTh@beaureve.gmx.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: <6faf39c91001210018v4a88b77eg3058f30ea41257a4@mail.gmail.com> On Thu, Jan 21, 2010 at 8:43 AM, Martin Drautzburg wrote: > Hello all, > > When passing parameters to a function, you sometimes need a paramter > which can only assume certain values, e.g. > > ? ? ? ?def move (direction): > ? ? ? ? ? ? ? ?... > If direction can only be "up", "down", "left" or "right", you can solve > this by passing strings, but this is not quite to the point: > > ? ? ? ?- you could pass invalid strings easily > ? ? ? ?- you need to quote thigs, which is a nuisance > ? ? ? ?- the parameter IS REALLY NOT A STRING, but a direction > > Alternatively you could export such symbols, so when you "import *" you > have them available in the caller's namespace. But that forces you > to "import *" which pollutes your namespace. > > What I am really looking for is a way > > ? ? ? ?- to be able to call move(up) > ? ? ? ?- having the "up" symbol only in the context of the function call > > So it should look something like this > > ... magic, magic ... > move(up) > ... unmagic, unmagic ... > print up > > This should complain that "up" is not defined during the "print" call, > but not when move() is called. And of course there should be as little > magic as possible. > > Any way to achieve this? It probably can be done, but in my opinion even it can, it would be ugly. I would just define "up" as some global constant - I don't see why it would be a problem that it's defined when you don't need it. In fact, I think it would be beneficial - you can then have direction-valued variables, so that you can (for example) implement "go the same direction I went last time". If that kind of thing happens more often, it might be useful to have a Direction class, to be able to easily program in things like "the opposite of up is down" and "right from right is back". -- Andr? Engels, andreengels at gmail.com From python at rcn.com Thu Jan 21 03:27:52 2010 From: python at rcn.com (Raymond Hettinger) Date: Thu, 21 Jan 2010 00:27:52 -0800 (PST) Subject: Sorted dictionary References: Message-ID: <1c74df0b-b22e-4aea-934a-c6d25e818f41@e37g2000yqn.googlegroups.com> On Jan 20, 5:02?pm, "Jan Kaliszewski" wrote: > Hello, > > Inspired by some my needs as well as some discussions in the net, I've ? > implemented a sorted dictionary (i.e. a dict that returns keys, values and ? > items always sorted by keys): > > http://code.activestate.com/recipes/576998/ > > Maybe it'll appear to be useful for somebody... And I'm curious about your ? > opinions. Using an underlying list to track sorted items means that insertion and deletion take O(n) time. That could be reduced to O(log n) time by using a blist or skiplist as the underlying structure for maintaining a sort. The other alternative is to just append items to the list and sort the list on demand. Since the Timsort takes advantage of existing order, the process will approach O(n) if sorting after every few updates. This is close the O(n) time it takes to iterate the list in the first place: s = SortedDict(items) list(s) # O(n log n) to sort and O(n) to iterate s[newkey] = newval list(s) # close to O(n) my two cents, Raymond From alfps at start.no Thu Jan 21 03:30:40 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 09:30:40 +0100 Subject: Symbols as parameters? In-Reply-To: <1495297.2JfmrJjJTh@beaureve.gmx.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: * Martin Drautzburg: > Hello all, > > When passing parameters to a function, you sometimes need a paramter > which can only assume certain values, e.g. > > def move (direction): > ... > If direction can only be "up", "down", "left" or "right", you can solve > this by passing strings, but this is not quite to the point: > > - you could pass invalid strings easily > - you need to quote thigs, which is a nuisance > - the parameter IS REALLY NOT A STRING, but a direction > > Alternatively you could export such symbols, so when you "import *" you > have them available in the caller's namespace. But that forces you > to "import *" which pollutes your namespace. > > What I am really looking for is a way > > - to be able to call move(up) > - having the "up" symbol only in the context of the function call > > So it should look something like this > > ... magic, magic ... > move(up) > ... unmagic, unmagic ... > print up > > This should complain that "up" is not defined during the "print" call, > but not when move() is called. And of course there should be as little > magic as possible. > > Any way to achieve this? >>> def move( direction ): ... print( "move " + str( direction ) ) ... >>> move( "up" ) move up >>> >>> class using_directions: ... up = 42 ... move( up ) ... move 42 >>> up Traceback (most recent call last): File "", line 1, in NameError: name 'up' is not defined >>> _ Of course it's an abuse of the language. :-) So I wouldn't recommend it, but it's perhaps worth having seen it. Cheers & hth., - Alf PS: I hope this was not a homework question. From nagle at animats.com Thu Jan 21 03:30:57 2010 From: nagle at animats.com (John Nagle) Date: Thu, 21 Jan 2010 00:30:57 -0800 Subject: html code generation In-Reply-To: References: Message-ID: <4b580d15$0$1621$742ec2ed@news.sonic.net> George Trojan wrote: > I need an advice on table generation. The table is essentially a fifo, > containing about 200 rows. The rows are inserted every few minutes or > so. The simplest solution is to store row data per line and write > directly html code: > line = "value1value2>... " > each run of the program would read the previous table into a list of > lines, insert the first row and drop the last one, taking care of table > header and trailer. > Is there a more classy solution? > > George The "HTMLTemplate" module is good for simple tasks like that. There are much more elaborate frameworks available, but if all you need to do is build a table periodically, it's a simple solution. This page http://www.sitetruth.com/reports/phishes.html is built with HTMLtemplate. It changes only every three hours, and computing it requires a big SQL join, so it's an automatically updated static page. John Nagle From javier.collado at gmail.com Thu Jan 21 03:38:24 2010 From: javier.collado at gmail.com (Javier Collado) Date: Thu, 21 Jan 2010 09:38:24 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: Hello, I'd say that isn't totally incorrect to use strings instead of symbols. Please note that in other programming languages symbols, atoms and the like are in fact immutable strings, which is what python provides by default. Best regards, Javier 2010/1/21 Alf P. Steinbach : > * Martin Drautzburg: >> >> Hello all, >> >> When passing parameters to a function, you sometimes need a paramter >> which can only assume certain values, e.g. >> >> ? ? ? ?def move (direction): >> ? ? ? ? ? ? ? ?... >> If direction can only be "up", "down", "left" or "right", you can solve >> this by passing strings, but this is not quite to the point: >> >> ? ? ? ?- you could pass invalid strings easily >> ? ? ? ?- you need to quote thigs, which is a nuisance >> ? ? ? ?- the parameter IS REALLY NOT A STRING, but a direction >> >> Alternatively you could export such symbols, so when you "import *" you >> have them available in the caller's namespace. But that forces you >> to "import *" which pollutes your namespace. >> >> What I am really looking for is a way >> >> ? ? ? ?- to be able to call move(up) >> ? ? ? ?- having the "up" symbol only in the context of the function call >> >> So it should look something like this >> >> ... magic, magic ... >> move(up) >> ... unmagic, unmagic ... >> print up >> >> This should complain that "up" is not defined during the "print" call, >> but not when move() is called. And of course there should be as little >> magic as possible. >> >> Any way to achieve this? > >>>> def move( direction ): > ... ? print( "move " + str( direction ) ) > ... >>>> move( "up" ) > move up >>>> >>>> class using_directions: > ... ? ? up = 42 > ... ? ? move( up ) > ... > move 42 >>>> up > Traceback (most recent call last): > ?File "", line 1, in > NameError: name 'up' is not defined >>>> _ > > > Of course it's an abuse of the language. :-) > > So I wouldn't recommend it, but it's perhaps worth having seen it. > > > Cheers & hth., > > - Alf > > PS: I hope this was not a homework question. > -- > http://mail.python.org/mailman/listinfo/python-list > From ben+python at benfinney.id.au Thu Jan 21 04:10:34 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 21 Jan 2010 20:10:34 +1100 Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: <877hrbesc5.fsf@benfinney.id.au> Martin Drautzburg writes: > When passing parameters to a function, you sometimes need a paramter > which can only assume certain values You might like to try the ?enum? library for this . -- \ ?You could augment an earwig to the point where it understood | `\ nuclear physics, but it would still be a very stupid thing to | _o__) do!? ?The Doctor, _The Two Doctors_ | Ben Finney From bearophileHUGS at lycos.com Thu Jan 21 04:15:38 2010 From: bearophileHUGS at lycos.com (Bearophile) Date: Thu, 21 Jan 2010 01:15:38 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: Martin Drautzburg, having symbols spread in the namespace is bad. And too much magic is even worse. You seem to need something like an enum that must be used with its qualified name, as: move(direction.up) That works well unless the symbols name are keywords. Creating a small class like this (that warns if you give it a string that contains a keyword) looks not too much hard: direction = Enum("up", "down", "left", "right") Or: direction = Enum("up, down, left, right") Or: direction = Enum("up down left right") Bye, bearophile From bearophileHUGS at lycos.com Thu Jan 21 04:24:25 2010 From: bearophileHUGS at lycos.com (Bearophile) Date: Thu, 21 Jan 2010 01:24:25 -0800 (PST) Subject: Sorted dictionary References: <1c74df0b-b22e-4aea-934a-c6d25e818f41@e37g2000yqn.googlegroups.com> Message-ID: Raymond Hettinger: > Using an underlying list to track sorted items > means that insertion and deletion take O(n) time. > That could be reduced to O(log n) time by using > a blist or skiplist as the underlying structure > for maintaining a sort. In the collections module it can be useful to have ordered dicts and sets based on search trees. I'm thinking about B+ trees to use CPU cache locality better. It can be fun :-) Bye, bearophile From iainking at gmail.com Thu Jan 21 04:30:19 2010 From: iainking at gmail.com (Iain King) Date: Thu, 21 Jan 2010 01:30:19 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: <5cfc086f-e812-498c-a387-c7266aa810e6@p24g2000yqm.googlegroups.com> On Jan 21, 7:43?am, Martin Drautzburg wrote: > Hello all, > > When passing parameters to a function, you sometimes need a paramter > which can only assume certain values, e.g. > > ? ? ? ? def move (direction): > ? ? ? ? ? ? ? ? ... > If direction can only be "up", "down", "left" or "right", you can solve > this by passing strings, but this is not quite to the point: > > ? ? ? ? - you could pass invalid strings easily > ? ? ? ? - you need to quote thigs, which is a nuisance > ? ? ? ? - the parameter IS REALLY NOT A STRING, but a direction > > Alternatively you could export such symbols, so when you "import *" you > have them available in the caller's namespace. But that forces you > to "import *" which pollutes your namespace. > > What I am really looking for is a way > > ? ? ? ? - to be able to call move(up) > ? ? ? ? - having the "up" symbol only in the context of the function call > > So it should look something like this > > ... magic, magic ... > move(up) > ... unmagic, unmagic ... > print up > > This should complain that "up" is not defined during the "print" call, > but not when move() is called. And of course there should be as little > magic as possible. > > Any way to achieve this? class Direction(object): pass def is_direction(d): return type(d)==Direction up = Direction() down = Direction() left = Direction() right = Direction() Now you can do your move(up), print up, etc. You can also check a valid direction was passed in by calling is_direction. 'Course, you can extend this so it does something a little more useful: class Direction(object): def __init__(self, vx=0, vy=0): self.vx = vx self.vy = vy up = Direction(0, -1) down = Direction(0, 1) left = Direction(-1, 0) right = Direction(1, 0) def move(direction): spaceship.x += direction.vx spaceship.y += direction.vy Iain From rpjday at crashcourse.ca Thu Jan 21 05:04:40 2010 From: rpjday at crashcourse.ca (Robert P. J. Day) Date: Thu, 21 Jan 2010 05:04:40 -0500 (EST) Subject: deleting, then re-importing a class method Message-ID: (once again, never ashamed to ask the dumb questions.) still playing with python3, and testing whether i can delete/unimport a specific method, then re-import it: >>> import sys >>> print(sys.__doc__) ... blah blah blah ... >>> del(sys.__doc__) >>> print(sys.__doc__) module(name[, doc]) Create a module object. The name must be a string; the optional doc argument can have any type. >>> ok, now that i've deleted just that member of "sys", can i re-import it? i know this doesn't seem to work: >>> import sys or is there an operator other than "import" that more represents a full refresh of a class? rday p.s. no, i don't have a valid application of the above, i'm just trying to break things. -- ======================================================================== Robert P. J. Day Waterloo, Ontario, CANADA Linux Consulting, Training and Kernel Pedantry. Web page: http://crashcourse.ca Twitter: http://twitter.com/rpjday ======================================================================== From alfps at start.no Thu Jan 21 05:06:39 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 11:06:39 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: Huh? I guess you meant to reply to the OP, not me. Cheers, - Alf * Javier Collado: > Hello, > > I'd say that isn't totally incorrect to use strings instead of > symbols. Please note that in other programming languages symbols, > atoms and the like are in fact immutable strings, which is what python > provides by default. > > Best regards, > Javier > > 2010/1/21 Alf P. Steinbach : >> * Martin Drautzburg: >>> Hello all, >>> >>> When passing parameters to a function, you sometimes need a paramter >>> which can only assume certain values, e.g. >>> >>> def move (direction): >>> ... >>> If direction can only be "up", "down", "left" or "right", you can solve >>> this by passing strings, but this is not quite to the point: >>> >>> - you could pass invalid strings easily >>> - you need to quote thigs, which is a nuisance >>> - the parameter IS REALLY NOT A STRING, but a direction >>> >>> Alternatively you could export such symbols, so when you "import *" you >>> have them available in the caller's namespace. But that forces you >>> to "import *" which pollutes your namespace. >>> >>> What I am really looking for is a way >>> >>> - to be able to call move(up) >>> - having the "up" symbol only in the context of the function call >>> >>> So it should look something like this >>> >>> ... magic, magic ... >>> move(up) >>> ... unmagic, unmagic ... >>> print up >>> >>> This should complain that "up" is not defined during the "print" call, >>> but not when move() is called. And of course there should be as little >>> magic as possible. >>> >>> Any way to achieve this? >>>>> def move( direction ): >> ... print( "move " + str( direction ) ) >> ... >>>>> move( "up" ) >> move up >>>>> class using_directions: >> ... up = 42 >> ... move( up ) >> ... >> move 42 >>>>> up >> Traceback (most recent call last): >> File "", line 1, in >> NameError: name 'up' is not defined >>>>> _ >> >> Of course it's an abuse of the language. :-) >> >> So I wouldn't recommend it, but it's perhaps worth having seen it. >> >> >> Cheers & hth., >> >> - Alf >> >> PS: I hope this was not a homework question. >> -- >> http://mail.python.org/mailman/listinfo/python-list >> From jeanmichel at sequans.com Thu Jan 21 05:20:09 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Thu, 21 Jan 2010 11:20:09 +0100 Subject: why the super class can access the subclass's attribute In-Reply-To: References: Message-ID: <4B582A59.9040604@sequans.com> yousay wrote: > I have sees aprogram like this ,i confusing why super class can access > the subclass's attribute > ,this is the program,thanks in advance: > class MyThread(threading.Thread): > def join(self): > super(MyThread,self).join() > return self.result > > class Worker(MyThread): > import random > import pdb > pdb.set_trace() > def run(self): > total = 0 > for i in range(random.randrange(10,100)): > total +=i > self.result = total > > > I don't thing I got your problem. But in case you are talking about 'result', you could probably say that "the subclass Worker can access to the super class MyThread 'result' attribute". Also keep in mind that there is no private attributes in python, and that any object can access any other object attribute, no matter its class. Of course, you dont want to do that. JM From tompelka at gmail.com Thu Jan 21 05:25:08 2010 From: tompelka at gmail.com (Tomas Pelka) Date: Thu, 21 Jan 2010 11:25:08 +0100 Subject: subprocess troubles Message-ID: <4B582B84.9040009@gmail.com> Hey all, have a problem with following piece of code: -------------------------------------------------- import subprocess paattern = "python" cmd = "/usr/bin/locate" arg1 = " -i" arg2 = " -d /var/www/books/mlocate.db" arg3 = str(" " + pattern) p1 = subprocess.Popen([cmd, arg1, arg2, arg3], shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE) (stdoutdata, stderrdata) = p1.communicate() print p1.returncode print "%s -- %s" % (stdoutdata, stderrdata) -------------------------------------------------- But return code is always 1 and command do not return any result/error (stdoutdata, stderrdata are None). If I run this command (/usr/bin/locate -i -d /var/www/books/mlocate.db python) from standard shell everything goes fine. Could you please give me an advice what I'm doing wrong? Thanks Cheers -- Tom From pavlovevidence at gmail.com Thu Jan 21 05:25:36 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Thu, 21 Jan 2010 02:25:36 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> On Jan 20, 11:43?pm, Martin Drautzburg wrote: > Hello all, > > When passing parameters to a function, you sometimes need a paramter > which can only assume certain values, e.g. > > ? ? ? ? def move (direction): > ? ? ? ? ? ? ? ? ... > If direction can only be "up", "down", "left" or "right", you can solve > this by passing strings, but this is not quite to the point: > > ? ? ? ? - you could pass invalid strings easily > ? ? ? ? - you need to quote thigs, which is a nuisance > ? ? ? ? - the parameter IS REALLY NOT A STRING, but a direction Nothing you can pass to the function is really a direction. A symbol is no more a direction than a string is. > Alternatively you could export such symbols, so when you "import *" you > have them available in the caller's namespace. But that forces you > to "import *" which pollutes your namespace. No it doesn't. You can still write module.UP in the function call, or you can write from module import UP. > What I am really looking for is a way > > ? ? ? ? - to be able to call move(up) > ? ? ? ? - having the "up" symbol only in the context of the function call Short answer is, you can't do it. Long answer is, you don't really want to have a symbol that's only in context during the function call, because you might just want to write some code that operates on directions (e.g., return random.choice ([UP,DOWN]), but it's a moot point since you can't do it. > So it should look something like this > > ... magic, magic ... > move(up) > ... unmagic, unmagic ... > print up > > This should complain that "up" is not defined during the "print" call, > but not when move() is called. And of course there should be as little > magic as possible. > > Any way to achieve this? Apart from writing a custom DSL (domain specific language), no. It is not, in case your were wondering, a feature that would be easily added to Python (the moratorium on new syntax notwithstanding). Python wouldn't be able to do this at compile time (since Python doesn't know what symbols refer to till run time). I convinced myself that it would be techincally possible to do it at run time, but at a penalty of higher function call overhead (which is already high enough as it is). And with the performance hit, extra scoping complexity, the inability to be used outside the function context, and no real benefit except to save typing, I don't see it ever being approved. Carl Banks From alfps at start.no Thu Jan 21 05:38:19 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 11:38:19 +0100 Subject: Symbols as parameters? In-Reply-To: <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> Message-ID: * Carl Banks: > On Jan 20, 11:43 pm, Martin Drautzburg [snip] > >> What I am really looking for is a way >> >> - to be able to call move(up) >> - having the "up" symbol only in the context of the function call > > Short answer is, you can't do it. > On the contrary, it's not difficult to do. I provided an example in my answer to the OP (first reply in the thread). However, it's IMHO an abuse of the language, not something that one should do. Cheers & hth., - Alf From javier.collado at gmail.com Thu Jan 21 05:39:11 2010 From: javier.collado at gmail.com (Javier Collado) Date: Thu, 21 Jan 2010 11:39:11 +0100 Subject: subprocess troubles In-Reply-To: <4B582B84.9040009@gmail.com> References: <4B582B84.9040009@gmail.com> Message-ID: Hello, If you set shell=False, then I think that arg2 should be separated into two different parts. Also, arg3 could be set just to pattern (no need to add extra spaces or using str function). Best regards, Javier 2010/1/21 Tomas Pelka : > Hey all, > > have a problem with following piece of code: > > -------------------------------------------------- > import subprocess > > paattern = "python" > cmd = "/usr/bin/locate" > arg1 = " -i" > arg2 = " -d /var/www/books/mlocate.db" > arg3 = str(" " + pattern) > > p1 = subprocess.Popen([cmd, arg1, arg2, arg3], shell=False, > stdout=subprocess.PIPE, stderr=subprocess.PIPE) > (stdoutdata, stderrdata) = p1.communicate() > > print p1.returncode > print "%s -- %s" % (stdoutdata, stderrdata) > -------------------------------------------------- > > But return code is always 1 and command do not return any result/error > (stdoutdata, stderrdata are None). If I run this command (/usr/bin/locate -i > -d /var/www/books/mlocate.db python) from standard shell everything goes > fine. > > Could you please give me an advice what I'm doing wrong? > > Thanks > Cheers > > -- > Tom > > -- > http://mail.python.org/mailman/listinfo/python-list > From jeanmichel at sequans.com Thu Jan 21 05:43:38 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Thu, 21 Jan 2010 11:43:38 +0100 Subject: deleting, then re-importing a class method In-Reply-To: References: Message-ID: <4B582FDA.2080601@sequans.com> Robert P. J. Day wrote: > (once again, never ashamed to ask the dumb questions.) > > still playing with python3, and testing whether i can > delete/unimport a specific method, then re-import it: > > >>>> import sys >>>> print(sys.__doc__) >>>> > ... blah blah blah ... > >>>> del(sys.__doc__) >>>> print(sys.__doc__) >>>> > module(name[, doc]) > > Create a module object. > The name must be a string; the optional doc argument can have any > type. > > > ok, now that i've deleted just that member of "sys", can i re-import > it? i know this doesn't seem to work: > > >>>> import sys >>>> > > or is there an operator other than "import" that more represents a > full refresh of a class? > > rday > > p.s. no, i don't have a valid application of the above, i'm just > trying to break things. > > -- > > > ======================================================================== > Robert P. J. Day Waterloo, Ontario, CANADA > > Linux Consulting, Training and Kernel Pedantry. > > Web page: http://crashcourse.ca > Twitter: http://twitter.com/rpjday > ======================================================================== > reload(sys) JM From __peter__ at web.de Thu Jan 21 05:51:01 2010 From: __peter__ at web.de (Peter Otten) Date: Thu, 21 Jan 2010 11:51:01 +0100 Subject: deleting, then re-importing a class method References: Message-ID: Robert P. J. Day wrote: > > (once again, never ashamed to ask the dumb questions.) > > still playing with python3, and testing whether i can > delete/unimport a specific method, then re-import it: > >>>> import sys >>>> print(sys.__doc__) > ... blah blah blah ... >>>> del(sys.__doc__) >>>> print(sys.__doc__) > module(name[, doc]) > > Create a module object. > The name must be a string; the optional doc argument can have any > type. >>>> > > ok, now that i've deleted just that member of "sys", can i re-import > it? i know this doesn't seem to work: Actually you haven't: >>> import sys >>> del sys.__doc__ >>> hasattr(sys, "__doc__") True > >>>> import sys > > or is there an operator other than "import" that more represents a > full refresh of a class? imp.reload() > rday > > p.s. no, i don't have a valid application of the above, i'm just > trying to break things. That is indeed likely to happen: Python 3.1.1+ (r311:74480, Nov 2 2009, 15:45:00) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> walk = os.walk >>> del os.walk >>> hasattr(os, "walk") False >>> import imp >>> imp.reload(os) >>> os.walk >>> os.walk is walk False So you have to two distinct walk() functions now. This becomes especially nasty when isinstance(obj, module.Class) tests begin to fail because the module and the class was reloaded, but obj is an instance of module.Class before the reload. Peter From jeanmichel at sequans.com Thu Jan 21 05:57:18 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Thu, 21 Jan 2010 11:57:18 +0100 Subject: Symbols as parameters? In-Reply-To: <1495297.2JfmrJjJTh@beaureve.gmx.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: <4B58330E.5040900@sequans.com> Martin Drautzburg wrote: > Hello all, > > When passing parameters to a function, you sometimes need a paramter > which can only assume certain values, e.g. > > def move (direction): > ... > If direction can only be "up", "down", "left" or "right", you can solve > this by passing strings, but this is not quite to the point: > > - you could pass invalid strings easily > - you need to quote thigs, which is a nuisance > - the parameter IS REALLY NOT A STRING, but a direction > > Alternatively you could export such symbols, so when you "import *" you > have them available in the caller's namespace. But that forces you > to "import *" which pollutes your namespace. > > What I am really looking for is a way > > - to be able to call move(up) > - having the "up" symbol only in the context of the function call > > So it should look something like this > > ... magic, magic ... > move(up) > ... unmagic, unmagic ... > print up > > This should complain that "up" is not defined during the "print" call, > but not when move() is called. And of course there should be as little > magic as possible. > > Any way to achieve this? > > in move.py def move(direction): print "moving %s" % direction move.UP = 'up' move.DOWN = 'down' Then >>> from move import move >>> move(move.UP) moving up JM From jarausch at igpm.rwth-aachen.de Thu Jan 21 06:02:53 2010 From: jarausch at igpm.rwth-aachen.de (Helmut Jarausch) Date: Thu, 21 Jan 2010 12:02:53 +0100 Subject: Best way to convert sequence of bytes to long integer In-Reply-To: References: <00e0377c$0$15570$c3e8da3@news.astraweb.com> <9b870cc2-0966-4cb2-be93-573317c06eb0@a6g2000yqm.googlegroups.com> Message-ID: <7rqqisFn94U1@mid.dfncis.de> On 01/20/10 22:23, Steven D'Aprano wrote: > I'm writing a module that handles, I won't call it encryption, > obfuscation using classical cryptographic algorithms. One of the > functions needs a deterministic but unpredictable integer generated from > a human-generated password or passphrase, so I'm using: > > hashlib.md5(key).digest() > > to get a string of bytes, then converting it to an int. > > int.from_bytes would be perfect for me, but in the meantime I'm using > Paul Rubin's trick of int(s.encode("hex"), 16). > Sorry, but this doesn't work for me (in Python 3.2a0) since hashlib.md5(key).digest() returns a byte string which has no .encode method. Just my 5 cents, Helmut. -- Helmut Jarausch Lehrstuhl fuer Numerische Mathematik RWTH - Aachen University D 52056 Aachen, Germany From anuhacks at gmail.com Thu Jan 21 06:18:07 2010 From: anuhacks at gmail.com (anusha k) Date: Thu, 21 Jan 2010 16:48:07 +0530 Subject: covert number into string Message-ID: <61f9a5fb1001210318r46991edake19f8008711d52e9@mail.gmail.com> Hi, Can anyone tell me how to convert number to words For example: If number =9999 then it should give me *Nine thousand nine hundred ninetynine* Is there any build-in function or something that can do this for me Thank you in advance Anusha Kadambala -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan_ml at behnel.de Thu Jan 21 06:41:04 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 21 Jan 2010 12:41:04 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: <4b583d50$0$6551$9b4e6d93@newsspool4.arcor-online.net> Alf P. Steinbach, 21.01.2010 09:30: > * Martin Drautzburg: >> - to be able to call move(up) >> - having the "up" symbol only in the context of the function call >> >> So it should look something like this >> >> ... magic, magic ... >> move(up) >> ... unmagic, unmagic ... >> print up Looks like a terribly bad design to me. >> This should complain that "up" is not defined during the "print" call, >> but not when move() is called. And of course there should be as little >> magic as possible. >> >> Any way to achieve this? > > >>> def move( direction ): > ... print( "move " + str( direction ) ) > ... > >>> move( "up" ) > move up > >>> > >>> class using_directions: > ... up = 42 > ... move( up ) > ... > move 42 > >>> up > Traceback (most recent call last): > File "", line 1, in > NameError: name 'up' is not defined > >>> _ > > Of course it's an abuse of the language. :-) And, of course, it's totally useless as the move() function doesn't know about the 'up' thing in the first place. Stefan From stefan_ml at behnel.de Thu Jan 21 06:43:08 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 21 Jan 2010 12:43:08 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> Message-ID: <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> Alf P. Steinbach, 21.01.2010 11:38: > * Carl Banks: >> On Jan 20, 11:43 pm, Martin Drautzburg > [snip] >> >>> What I am really looking for is a way >>> >>> - to be able to call move(up) >>> - having the "up" symbol only in the context of the function >>> call >> >> Short answer is, you can't do it. >> > > On the contrary, it's not difficult to do. > > I provided an example in my answer to the OP (first reply in the thread). Erm, no, you didn't. You showed a) how to pass a string constant into a function and b) how to pass a value from a bound variable. None of that is helpful to the OP's problem. Stefan From juhasecke at googlemail.com Thu Jan 21 06:48:28 2010 From: juhasecke at googlemail.com (Jan Ulrich Hasecke) Date: Thu, 21 Jan 2010 12:48:28 +0100 Subject: covert number into string In-Reply-To: <61f9a5fb1001210318r46991edake19f8008711d52e9@mail.gmail.com> References: <61f9a5fb1001210318r46991edake19f8008711d52e9@mail.gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 21.01.10 12:18, anusha k wrote: > Hi, > > Can anyone tell me how to convert number to words > For example: > If number =9999 then it should give me *Nine thousand nine hundred > ninetynine* > Is there any build-in function or something that can do this for me > Thank you in advance > Anusha Kadambala > Hey thats like an exercise in the wonderful German book "Python for Kids". Should be pretty easy to implement it by yourself. Maybe you can understand the code without the german comments. But keep in mind that this is an excercise for kids in chapter 5. I am sure there are better implementations. # Python f?r Kids -- 3. Auflage, Kapitel 5 # Autor: Gregor Lingl # Datum: 01. 03. 2004 # Loesung von Kapitel 5, Aufgabe 3: def zahlwort(zahl): einer = zahl % 10 # Einerstelle: Rest bei der Division von zahl durch 10 if einer == 1: e = "ein" elif einer == 2: e = "zwei" elif einer == 3: e = "drei" elif einer == 4: e = "vier" elif einer == 5: e = "f?nf" elif einer == 6: e = "sechs" elif einer == 7: e = "sieben" elif einer == 8: e = "acht" elif einer == 9: e = "neun" zehner = zahl // 10 # Zehnerstelle: Ergebnis der Ganzzahldivision # von zahl durch 10 if zehner == 0: z = "" elif zehner == 1: z = "zehn" elif zehner == 2: z = "zwanzig" elif zehner == 3: z = "dreissig" elif zehner == 4: z = "vierzig" elif zehner == 5: z = "f?nfzig" elif zehner == 6: z = "sechzig" elif zehner == 7: z = "siebzig" elif zehner == 8: z = "achtzig" elif zehner == 9: z ="neunzig" if zahl == 11: print "elf" elif zahl == 12: print "zw?lf" elif zahl == 17: print "siebzehn" elif einer == 0: print z elif zehner == 1: print e+z else: print e+"und"+z zahl = raw_input("Gib eine zweistellige ganze Zahl ein: ") zahl = int(zahl) # macht aus dem String zahl eine ganze Zahl zahlwort(zahl) print Cheers! juh -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.10 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAktYPwwACgkQPUzUEFbILMRLWwCgjugY7Wxzec7AH8Esz1hNIf8d WgoAoKgFo59NoBVk0jGZsqe5slOc7P9W =cP1x -----END PGP SIGNATURE----- From alfps at start.no Thu Jan 21 06:57:41 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 12:57:41 +0100 Subject: Symbols as parameters? In-Reply-To: <4b583d50$0$6551$9b4e6d93@newsspool4.arcor-online.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <4b583d50$0$6551$9b4e6d93@newsspool4.arcor-online.net> Message-ID: * Stefan Behnel: > Alf P. Steinbach, 21.01.2010 09:30: >> * Martin Drautzburg: >>> - to be able to call move(up) >>> - having the "up" symbol only in the context of the function call >>> >>> So it should look something like this >>> >>> ... magic, magic ... >>> move(up) >>> ... unmagic, unmagic ... >>> print up > > Looks like a terribly bad design to me. That it is, in Python. >>> This should complain that "up" is not defined during the "print" call, >>> but not when move() is called. And of course there should be as little >>> magic as possible. >>> >>> Any way to achieve this? >>>>> def move( direction ): >> ... print( "move " + str( direction ) ) >> ... >>>>> move( "up" ) >> move up >>>>> class using_directions: >> ... up = 42 >> ... move( up ) >> ... >> move 42 >>>>> up >> Traceback (most recent call last): >> File "", line 1, in >> NameError: name 'up' is not defined >>>>> _ >> Of course it's an abuse of the language. :-) > > And, of course, it's totally useless as the move() function doesn't know > about the 'up' thing in the first place. Hm, it seems as if you thought that the example was *literally* what the OP should do. It was not: it just demonstrated the possibility. Consider replacing the assignment with whatever, or alternatively, consider that perhaps 42 is indeed the value of "up" that the move() function knows about. Cheers & hth., - Alf From alfps at start.no Thu Jan 21 06:58:51 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 12:58:51 +0100 Subject: Symbols as parameters? In-Reply-To: <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> Message-ID: * Stefan Behnel: > Alf P. Steinbach, 21.01.2010 11:38: >> * Carl Banks: >>> On Jan 20, 11:43 pm, Martin Drautzburg >> [snip] >>>> What I am really looking for is a way >>>> >>>> - to be able to call move(up) >>>> - having the "up" symbol only in the context of the function >>>> call >>> Short answer is, you can't do it. >>> >> On the contrary, it's not difficult to do. >> >> I provided an example in my answer to the OP (first reply in the thread). > > Erm, no, you didn't. You showed a) how to pass a string constant into a > function and b) how to pass a value from a bound variable. None of that is > helpful to the OP's problem. Perhaps look again at that example. It demonstrates exactly what the OP asks for. Cheers & hth., - Alf From ndbecker2 at gmail.com Thu Jan 21 07:13:48 2010 From: ndbecker2 at gmail.com (Neal Becker) Date: Thu, 21 Jan 2010 07:13:48 -0500 Subject: multiprocessing as batch system Message-ID: I'm using multiprocessing as a poor man's batch system. It is working OK, except that it doesn't seem to do load balancing quite right. I have a 8-cpu machine. If I start, using multiprocessing pool, calling map with more than 8 elements, it will start 8 processes. It seems, though, that when some of them finish, it does not start others right away. The load average may be a lot less than 8 but still additional processes are not active. Is there a way to use multiprocessing to achieve the effect I want? (I really want a simple batch system, but without complex setup) From joel.goldstick at columbuswebmakers.com Thu Jan 21 07:49:33 2010 From: joel.goldstick at columbuswebmakers.com (Joel Goldstick) Date: Thu, 21 Jan 2010 07:49:33 -0500 Subject: covert number into string In-Reply-To: <61f9a5fb1001210318r46991edake19f8008711d52e9@mail.gmail.com> References: <61f9a5fb1001210318r46991edake19f8008711d52e9@mail.gmail.com> Message-ID: <4B584D5D.5090209@columbuswebmakers.com> anusha k wrote: > Hi, > > Can anyone tell me how to convert number to words > For example: > If number =9999 then it should give me *Nine thousand nine hundred > ninetynine* > Is there any build-in function or something that can do this for me > Thank you in advance > Anusha Kadambala > > This might help: http://snippets.dzone.com/posts/show/704 From tompelka at gmail.com Thu Jan 21 08:27:19 2010 From: tompelka at gmail.com (Tomas Pelka) Date: Thu, 21 Jan 2010 14:27:19 +0100 Subject: subprocess troubles In-Reply-To: References: <4B582B84.9040009@gmail.com> Message-ID: <4B585637.1040300@gmail.com> On 01/21/2010 11:39 AM, Javier Collado wrote: > Hello, > > If you set shell=False, then I think that arg2 should be separated > into two different parts. > > Also, arg3 could be set just to pattern (no need to add extra spaces > or using str function). > > Best regards, > Javier > > 2010/1/21 Tomas Pelka: > >> Hey all, >> >> have a problem with following piece of code: >> >> -------------------------------------------------- >> import subprocess >> >> paattern = "python" >> cmd = "/usr/bin/locate" >> arg1 = " -i" >> arg2 = " -d /var/www/books/mlocate.db" >> arg3 = str(" " + pattern) >> >> p1 = subprocess.Popen([cmd, arg1, arg2, arg3], shell=False, >> stdout=subprocess.PIPE, stderr=subprocess.PIPE) >> (stdoutdata, stderrdata) = p1.communicate() >> >> print p1.returncode >> print "%s -- %s" % (stdoutdata, stderrdata) >> -------------------------------------------------- >> >> But return code is always 1 and command do not return any result/error >> (stdoutdata, stderrdata are None). If I run this command (/usr/bin/locate -i >> -d /var/www/books/mlocate.db python) from standard shell everything goes >> fine. >> >> Could you please give me an advice what I'm doing wrong? >> >> Thanks >> Cheers >> >> -- >> Tom >> >> -- >> http://mail.python.org/mailman/listinfo/python-list >> >> Thanks Javier for advice, but sill same result. I'm running this code as cgi script from apache. Weird is that when i run it from shell as apache user, like ---------------- # su -s /bin/bash -c "/usr/bin/locate -i -d /var/www/books/mlocate.db python; echo $?" apache 0 --------------- I always get "0", but as cgi it returns "1". When I run this script by other user (tom), I'll obtain nonzero output what is OK. Additional info: # su -s /bin/bash -c "ls -l /var/www/books/mlocate.db" apache -rw-rw-r-- 1 tom books 1465653 Jan 20 13:33 /var/www/books/mlocate.db so db is readable by apache Whore source attached. -- Tom -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: search.py URL: From tim at reportlab.com Thu Jan 21 08:40:27 2010 From: tim at reportlab.com (Tim) Date: Thu, 21 Jan 2010 05:40:27 -0800 (PST) Subject: ReportLab PDF Toolkit v2.4 released Message-ID: We're pleased to announce the latest version of the ReportLab open source PDF toolkit, now available for download here: https://www.reportlab.com/software/opensource/rl-toolkit/download/ The ReportLab Toolkit is a library for programatically creating documents in PDF format. It's free, open-source software written in Python, and released under a BSD type license. Thanks, -The ReportLab Team

ReportLab Toolkit v2.4 - The Open Source Library for creating PDF Documents. (21-Jan-2010)

From wbsoft at xs4all.nl Thu Jan 21 09:18:54 2010 From: wbsoft at xs4all.nl (Wilbert Berendsen) Date: Thu, 21 Jan 2010 15:18:54 +0100 Subject: substitution In-Reply-To: References: Message-ID: <201001211518.55011.wbsoft@xs4all.nl> Op maandag 18 januari 2010 schreef Adi: > keys = [(len(key), key) for key in mapping.keys()] > keys.sort(reverse=True) > keys = [key for (_, key) in keys] > > pattern = "(%s)" % "|".join(keys) > repl = lambda x : mapping[x.group(1)] > s = "fooxxxbazyyyquuux" > > re.subn(pattern, repl, s) I managed to make it even shorted, using the key argument for sorted, not putting the whole regexp inside parentheses and pre-compiling the regular expression: import re mapping = { "foo" : "bar", "baz" : "quux", "quuux" : "foo" } # sort the keys, longest first, so 'aa' gets matched before 'a', because # in Python regexps the first match (going from left to right) in a # |-separated group is taken keys = sorted(mapping.keys(), key=len) rx = re.compile("|".join(keys)) repl = lambda x: mapping[x.group()] s = "fooxxxbazyyyquuux" rx.sub(repl, s) One thing remaining: if the replacement keys could contain non-alphanumeric characters, they should be escaped using re.escape: rx = re.compile("|".join(re.escape(key) for key in keys)) Met vriendelijke groet, Wilbert Berendsen -- http://www.wilbertberendsen.nl/ "You must be the change you wish to see in the world." -- Mahatma Gandhi From gnarlodious at gmail.com Thu Jan 21 09:23:17 2010 From: gnarlodious at gmail.com (Gnarlodious) Date: Thu, 21 Jan 2010 06:23:17 -0800 (PST) Subject: Create object name from string value? References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> Message-ID: <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> On Jan 20, 10:35?pm, Steven D'Aprano wrote: > That's the wrong way to handle the problem. Named objects are only useful > if you know the name of the object when writing the code. Otherwise, how > do you know what name to use in the code? Thank you for the help. I am gathering the names of all *.plist files in a folder, creating objects named the filename, and accessing the data like this: Data.Server.Config.BaseURL > http://Spectrumology.com/ Adding a .plist file would automatically create a plist dictionary object inside the Data module. > The right way to solve this problem is with a dictionary: > > for name in ["object1", "object2", "object3"]: > ? ? d = {name: classname()} > ? ? print d[name] This works! However I end up saying: d['Server'].Config.BaseURL to get the data, when I should be saying: Server.Config.BaseURL > but for the record, the way to use exec is like this: > > exec("object1 = classname()") I failed to make that work. So back to the original question. How to make an instance named according to a string inside a variable? I guess it should be in the top-level namespace, not inside a list or dictionary. -- Gnarlie http://Gnarlodious.com/Gnarlodious From phil at riverbankcomputing.com Thu Jan 21 09:26:55 2010 From: phil at riverbankcomputing.com (Phil Thompson) Date: Thu, 21 Jan 2010 14:26:55 +0000 Subject: SIP v4.10 Released (Python Bindings Generator) Message-ID: SIP v4.10 has been released and can be downloaded from http://www.riverbankcomputing.com/software/sip/. SIP is a tool for generating Python modules that wrap C or C++ libraries. It is similar to SWIG. It is used to generate PyQt and PyKDE. The SIP license is similar to the Python License and is also licensed under the GPL v2 and v3. SIP runs on Windows, UNIX, Linux and MacOS/X. SIP requires Python v2.3 or later and fully supports Python v3. This release adds support for keyword arguments and docstrings. Docstrings may be either explictly specified or automatically generated. Automatically generated docstrings describe the Python signatures of all available overloads for a callable. A significantly improved error reporting mechanism uses those docstrings in exceptions raised when arguments with incorrect types are passed. Other features of SIP include: - extension modules are implemented as a single binary .pyd or .so file (no Python stubs) - support for Python new-style classes - the ability to specify the super-type and meta-type used to wrap instances - generated modules are quick to import, even for large libraries - thread support - the ability to re-implement C++ abstract and virtual methods in Python - the ability to define Python classes that derive from abstract C++ classes - the ability to spread a class hierarchy across multiple Python modules - the ability to wrap a C++ class in different ways and allow an application to select a particular implementation at run-time - support for C++ namespaces - support for C++ exceptions - support for C++ operators - an extensible build system written in Python that supports over 50 platform/compiler combinations - the generation of API files for IDEs that support autocompletion and call tips. From python at mrabarnett.plus.com Thu Jan 21 09:42:02 2010 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 21 Jan 2010 14:42:02 +0000 Subject: substitution In-Reply-To: <201001211518.55011.wbsoft@xs4all.nl> References: <201001211518.55011.wbsoft@xs4all.nl> Message-ID: <4B5867BA.4070804@mrabarnett.plus.com> Wilbert Berendsen wrote: > Op maandag 18 januari 2010 schreef Adi: >> keys = [(len(key), key) for key in mapping.keys()] >> keys.sort(reverse=True) >> keys = [key for (_, key) in keys] >> >> pattern = "(%s)" % "|".join(keys) >> repl = lambda x : mapping[x.group(1)] >> s = "fooxxxbazyyyquuux" >> >> re.subn(pattern, repl, s) > > I managed to make it even shorted, using the key argument for sorted, not > putting the whole regexp inside parentheses and pre-compiling the regular > expression: > > import re > > mapping = { > "foo" : "bar", > "baz" : "quux", > "quuux" : "foo" > } > > # sort the keys, longest first, so 'aa' gets matched before 'a', because > # in Python regexps the first match (going from left to right) in a > # |-separated group is taken > keys = sorted(mapping.keys(), key=len) > For longest first you need: keys = sorted(mapping.keys(), key=len, reverse=True) > rx = re.compile("|".join(keys)) > repl = lambda x: mapping[x.group()] > s = "fooxxxbazyyyquuux" > rx.sub(repl, s) > > One thing remaining: if the replacement keys could contain non-alphanumeric > characters, they should be escaped using re.escape: > Strictly speaking, not all non-alphanumeric characters, but only the special ones. > rx = re.compile("|".join(re.escape(key) for key in keys)) > From daniel at stutzbachenterprises.com Thu Jan 21 09:42:26 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Thu, 21 Jan 2010 08:42:26 -0600 Subject: Sorted dictionary In-Reply-To: <1c74df0b-b22e-4aea-934a-c6d25e818f41@e37g2000yqn.googlegroups.com> References: <1c74df0b-b22e-4aea-934a-c6d25e818f41@e37g2000yqn.googlegroups.com> Message-ID: On Thu, Jan 21, 2010 at 2:27 AM, Raymond Hettinger wrote: > Using an underlying list to track sorted items > means that insertion and deletion take O(n) time. > That could be reduced to O(log n) time by using > a blist or skiplist as the underlying structure > for maintaining a sort. > Indeed. In fact, blist 1.1 (to be released within a month or so) will include sorteddict, sortedset, sortedlist, weaksortedset, and weaksortedlist types. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From bouncyinc at gmail.com Thu Jan 21 09:42:38 2010 From: bouncyinc at gmail.com (John Haggerty) Date: Thu, 21 Jan 2010 07:42:38 -0700 Subject: creating a python program to control the cursor and click a location on the screen at specified delay interval? Message-ID: Hi....So just testing the feasibility of doing this but I was interested in creating a primitive way of scripting gui applications w/o using a remote network client by having the application up and then almost just using the pointer position to figure out where it is and then click it and any subsequent dialogs. I would have to assume that someone has done something like this. In principle this could be done I don't know if this is possible to implement in python cross platform but the system I am using is just your standard Ubuntu Linux 9.10 system. Would be interesting to see something already in existance however to avoid duplication of work. -------------- next part -------------- An HTML attachment was scrubbed... URL: From alfps at start.no Thu Jan 21 09:43:05 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 15:43:05 +0100 Subject: Create object name from string value? In-Reply-To: <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> Message-ID: * Gnarlodious: > On Jan 20, 10:35 pm, Steven D'Aprano wrote: > >> That's the wrong way to handle the problem. Named objects are only useful >> if you know the name of the object when writing the code. Otherwise, how >> do you know what name to use in the code? > > Thank you for the help. I am gathering the names of all *.plist files > in a folder, creating objects named the filename, and accessing the > data like this: > > Data.Server.Config.BaseURL >> http://Spectrumology.com/ > > Adding a .plist file would automatically create a plist dictionary > object inside the Data module. > >> The right way to solve this problem is with a dictionary: >> >> for name in ["object1", "object2", "object3"]: >> d = {name: classname()} >> print d[name] > > This works! However I end up saying: > > d['Server'].Config.BaseURL > > to get the data, when I should be saying: > > Server.Config.BaseURL > >> but for the record, the way to use exec is like this: >> >> exec("object1 = classname()") > > I failed to make that work. So back to the original question. How to > make an instance named according to a string inside a variable? I > guess it should be in the top-level namespace, not inside a list or > dictionary. I don't understand how you intend to use a variable whose name comes from dynamic data. So I agree with Steven that it's the wrong way to handle the problem -- whatever the problem is! But, if it can help: >>> import __main__ >>> setattr( __main__, "blah", 123 ) >>> blah 123 >>> _ Cheers, - Alf From phil at riverbankcomputing.com Thu Jan 21 09:46:57 2010 From: phil at riverbankcomputing.com (Phil Thompson) Date: Thu, 21 Jan 2010 14:46:57 +0000 Subject: PyQt v4.7 Released Message-ID: <99dcfcfa3cc9487368d1684816771153@localhost> PyQt v4.7 has been released and is available from http://www.riverbankcomputing.com/software/pyqt/. PyQt is a comprehensive set of bindings for the Qt application and UI framework from Nokia. It supports the same platforms as Qt (Windows, Linux and MacOS/X). PyQt supports Python v3 and Python v2.3 and later. The highlights of this release include: - full support for Qt v4.6.1 including the new animation and state machine frameworks, gesture and multi-touch support, and advanced graphics effects (blurring, colourising, drop shadows) - all callables have docstrings that describe the Python signatures of all available overloads - keyword arguments are supported for all optional arguments. Windows installers are provided for the GPL version of PyQt which contains everything needed for PyQt development (including Qt, Qt Designer, QScintilla, and MySQL, PostgreSQL, SQLite and ODBC drivers) except Python itself. PyQt v4 is implemented as a set of 19 extension modules containing over 400 classes and over 6,000 functions and methods. QtCore The non-GUI infrastructure including event loops, threads, i18n, Unicode, signals and slots, user and application settings, mapped files and shared memory. QtDesigner A set of classes that allow the Qt Designer GUI design tool to be extended with PyQt. QtGui A rich collection of GUI widgets. QtHelp A set of classes for creating and viewing searchable documentation and being able to integrate online help with PyQt applications. It includes the C++ port of the Lucene text search engine. QtNetwork A set of classes to support TCP and UDP socket programming and higher level protocols (eg. HTTP, SSL). QtOpenGL A set of classes that allows PyOpenGL to render onto Qt widgets. QtScript A set of classes that implements a JavaScript interpreter. Python objects may be exposed in the interpreter as JavaScript objects. QtScriptTools A debugger for the JavaScript interpreter. QtSql A set of classes that implement SQL data models and interfaces to industry standard databases. The Windows installers include support for SQLite, MySQL, PostgreSQL and ODBC. QtSvg A set of classes to render SVG files onto Qt widgets. QtTest A set of classes to automate unit testing of PyQt applications and GUIs. QtWebKit This implements a web browser engine based on the WebKit engine used by Apple's Safari browser. It allows the methods and properties of Python objects to be published and appear as JavaScript objects to scripts embedded in HTML pages. QtXML A set of classes that implement DOM and SAX parsers. QtXMLPatterns A set of classes that implement XQuery and XPath support for XML and custom data models. QtAssistant A set of classes that enables the Qt Assistant online help browser to be integrated with an application. QAxContainer A set of classes for Windows that allows the integration of ActiveX controls and COM objects. phonon A cross-platform multimedia framework that enables the use of audio and video content in PyQt applications. DirectX is used as the Windows backend, QuickTime as the MacOS/X backend, and GStreamer as the Linux backend. QtMultimedia A set of classes that provide low-level multimedia functions. Application developers would normally use the phonon module. DBus PyQt includes dbus.mainloop.qt that allows the Qt event loop to be used with the standard DBus Python bindings. PyQt includes the pyuic4 utility which generates Python code to implement user interfaces created with Qt Designer in the same way that the uic utility generates C++ code. It is also able to load Designer XML files dynamically. PyQt is available under the GPL and a commercial license. Unlike Qt, PyQt is not available under the LGPL. The commercial PyQt license allows GPL applications to be relicensed at any time. From iainking at gmail.com Thu Jan 21 09:58:14 2010 From: iainking at gmail.com (Iain King) Date: Thu, 21 Jan 2010 06:58:14 -0800 (PST) Subject: substitution References: Message-ID: On Jan 21, 2:18?pm, Wilbert Berendsen wrote: > Op maandag 18 januari 2010 schreef Adi: > > > keys = [(len(key), key) for key in mapping.keys()] > > keys.sort(reverse=True) > > keys = [key for (_, key) in keys] > > > pattern = "(%s)" % "|".join(keys) > > repl = lambda x : mapping[x.group(1)] > > s = "fooxxxbazyyyquuux" > > > re.subn(pattern, repl, s) > > I managed to make it even shorted, using the key argument for sorted, not > putting the whole regexp inside parentheses and pre-compiling the regular > expression: > > import re > > mapping = { > ? ? ? ? "foo" : "bar", > ? ? ? ? "baz" : "quux", > ? ? ? ? "quuux" : "foo" > > } > > # sort the keys, longest first, so 'aa' gets matched before 'a', because > # in Python regexps the first match (going from left to right) in a > # |-separated group is taken > keys = sorted(mapping.keys(), key=len) > > rx = re.compile("|".join(keys)) > repl = lambda x: mapping[x.group()] > s = "fooxxxbazyyyquuux" > rx.sub(repl, s) > > One thing remaining: if the replacement keys could contain non-alphanumeric > characters, they should be escaped using re.escape: > > rx = re.compile("|".join(re.escape(key) for key in keys)) > > Met vriendelijke groet, > Wilbert Berendsen > > --http://www.wilbertberendsen.nl/ > "You must be the change you wish to see in the world." > ? ? ? ? -- Mahatma Gandhi Sorting it isn't the right solution: easier to hold the subs as tuple pairs and by doing so let the user specify order. Think of the following subs: "fooxx" -> "baz" "oxxx" -> "bar" does the user want "bazxbazyyyquuux" or "fobarbazyyyquuux"? Iain From wbsoft at xs4all.nl Thu Jan 21 10:08:21 2010 From: wbsoft at xs4all.nl (Wilbert Berendsen) Date: Thu, 21 Jan 2010 16:08:21 +0100 Subject: substitution In-Reply-To: <4B5867BA.4070804@mrabarnett.plus.com> References: <201001211518.55011.wbsoft@xs4all.nl> <4B5867BA.4070804@mrabarnett.plus.com> Message-ID: <201001211608.21475.wbsoft@xs4all.nl> Op donderdag 21 januari 2010 schreef MRAB: > For longest first you need: > > keys = sorted(mapping.keys(), key=len, reverse=True) Oh yes, I cut/pasted the wrong line :-) Just for clarity: import re mapping = { "foo" : "bar", "baz" : "quux", "quuux" : "foo" } # sort the keys, longest first, so 'aa' gets matched before 'a', because # in Python regexps the first match (going from left to right) in a # |-separated group is taken keys = sorted(mapping.keys(), key=len, reverse=True) rx = re.compile("|".join(keys)) repl = lambda x: mapping[x.group()] s = "fooxxxbazyyyquuux" rx.sub(repl, s) >> One thing remaining: if the replacement keys could contain non-alphanumeric >> characters, they should be escaped using re.escape: >> rx = re.compile("|".join(re.escape(key) for key in keys)) >> >Strictly speaking, not all non-alphanumeric characters, but only the >special ones. True, although the re.escape function simply escapes all non-alphanumeric characters :) And here is a factory function that returns a translator given a mapping. The translator can be called to perform replacements in a string: import re def translator(mapping): keys = sorted(mapping.keys(), key=len, reverse=True) rx = re.compile("|".join(keys)) repl = lambda m: mapping[m.group()] return lambda s: rx.sub(repl, s) #Usage: >>> t = translator(mapping) >>> t('fooxxxbazyyyquuux') 'barxxxquuxyyyfoo' w best regards, Wilbert Berendsen -- http://www.wilbertberendsen.nl/ From python.list at tim.thechases.com Thu Jan 21 10:32:52 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 21 Jan 2010 09:32:52 -0600 Subject: Create object name from string value? In-Reply-To: <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> Message-ID: <4B5873A4.6090707@tim.thechases.com> Gnarlodious wrote: >> for name in ["object1", "object2", "object3"]: >> d = {name: classname()} >> print d[name] > > This works! However I end up saying: > > d['Server'].Config.BaseURL > > to get the data, when I should be saying: > > Server.Config.BaseURL It sounds like you want a mapping of strings to class objects (not instances), so you can do something like mapping = { "object1": object1, "object2": some_module.object2, "object3": other_module.namespace.object3, } MyClass = mapping["object1"] server = MyClass(initialization_params) # the above two lines can be written as # server = mapping["object1"](init_params) print server.Config.BaseURL -tkc From jlconlin at gmail.com Thu Jan 21 10:41:20 2010 From: jlconlin at gmail.com (Jeremy) Date: Thu, 21 Jan 2010 07:41:20 -0800 (PST) Subject: distutils not finding all of my pure python modules Message-ID: <543c08a0-30c8-49fc-882f-212e099bbd96@h9g2000yqa.googlegroups.com> I have a small set of Python packages/modules that I am putting together. I'm having trouble in that when I run python setup.py sdist I don't get all of my pure python modules. The setup.py script I use is: # ===================================== from distutils.core import setup purePythonModules = ['regex', 'gnuFile'] setup(name='PythonForSafeguards', version='0.9.1', description = 'Python code for MCNP and Safeguards analysis.', author = 'Jake the Snake', author_email = 'something at blah.com', packages = ['MCNP', 'Safeguards'], url='http://lanl.gov', py_modules = purePythonModules, ) # ========================================= Everythin seems to work fine except the gnuFile.py script does not get put into the distribution. I know the file exists in the same directory as regex.py and has the same permissions. Does anyone know what is going on here? I'm using Python 2.6.4. Thanks, Jeremy From davea at ieee.org Thu Jan 21 11:01:18 2010 From: davea at ieee.org (Dave Angel) Date: Thu, 21 Jan 2010 11:01:18 -0500 Subject: Symbols as parameters? In-Reply-To: <1495297.2JfmrJjJTh@beaureve.gmx.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: <4B587A4E.9050508@ieee.org> Martin Drautzburg wrote: > Hello all, > > When passing parameters to a function, you sometimes need a paramter > which can only assume certain values, e.g. > > def move (direction): > ... > If direction can only be "up", "down", "left" or "right", you can solve > this by passing strings, but this is not quite to the point: > > - you could pass invalid strings easily > - you need to quote thigs, which is a nuisance > - the parameter IS REALLY NOT A STRING, but a direction > > Alternatively you could export such symbols, so when you "import *" you > have them available in the caller's namespace. But that forces you > to "import *" which pollutes your namespace. > > What I am really looking for is a way > > - to be able to call move(up) > - having the "up" symbol only in the context of the function call > > So it should look something like this > > ... magic, magic ... > move(up) > ... unmagic, unmagic ... > print up > > This should complain that "up" is not defined during the "print" call, > but not when move() is called. And of course there should be as little > magic as possible. > > Any way to achieve this? > > > My favorite answer provided by others was adding the attributes to the function right after the function definition, then using move(move.up) But another approach is to define the function as: def move(up=False, down=False, right=False, left=False): if up: ... if down: .... and call it as: move(up=True) DaveA From awilliam at opengroupware.us Thu Jan 21 11:12:22 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Thu, 21 Jan 2010 11:12:22 -0500 Subject: Create object name from string value? In-Reply-To: <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> Message-ID: <1264090342.8831.7.camel@linux-m3mt> > > but for the record, the way to use exec is like this: > > exec("object1 = classname()") > I failed to make that work. So back to the original question. How to > make an instance named according to a string inside a variable? I > guess it should be in the top-level namespace, not inside a list or > dictionary. I think what you want is more like: classclass = eval(classname) x = classclass(...args...) -- OpenGroupware developer: awilliam at whitemice.org OpenGroupare & Cyrus IMAPd documenation @ From awilliam at opengroupware.us Thu Jan 21 11:16:31 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Thu, 21 Jan 2010 11:16:31 -0500 Subject: multiprocessing as batch system In-Reply-To: References: Message-ID: <1264090591.8831.10.camel@linux-m3mt> On Thu, 2010-01-21 at 07:13 -0500, Neal Becker wrote: > I'm using multiprocessing as a poor man's batch system. It is working OK, > except that it doesn't seem to do load balancing quite right. > I have a 8-cpu machine. If I start, using multiprocessing pool, calling map > with more than 8 elements, it will start 8 processes. It seems, though, > that when some of them finish, it does not start others right away. The > load average may be a lot less than 8 but still additional processes are not > active. If your processes are I/O bound your system will never appear to be saturated. Multiple pending processes and a low load average usually indicated waiting on some other resource. > Is there a way to use multiprocessing to achieve the effect I want? (I > really want a simple batch system, but without complex setup) Well, honestly, that isn't possible (unless you just choose to ignore all the things that can go wrong: crashed worked, hung workers, etc...) -- OpenGroupware developer: awilliam at whitemice.org OpenGroupare & Cyrus IMAPd documenation @ From davea at ieee.org Thu Jan 21 11:21:26 2010 From: davea at ieee.org (Dave Angel) Date: Thu, 21 Jan 2010 11:21:26 -0500 Subject: Create object name from string value? In-Reply-To: <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> Message-ID: <4B587F06.2070904@ieee.org> Gnarlodious wrote: > On Jan 20, 10:35 pm, Steven D'Aprano wrote: > > >> That's the wrong way to handle the problem. Named objects are only useful >> if you know the name of the object when writing the code. Otherwise, how >> do you know what name to use in the code? >> > > Thank you for the help. I am gathering the names of all *.plist files > in a folder, creating objects named the filename, and accessing the > data like this: > > Data.Server.Config.BaseURL > >> http://Spectrumology.com/ >> > > Adding a .plist file would automatically create a plist dictionary > object inside the Data module. > > >> The right way to solve this problem is with a dictionary: >> >> for name in ["object1", "object2", "object3"]: >> d =name: classname()} >> print d[name] >> > > This works! However I end up saying: > > d['Server'].Config.BaseURL > > to get the data, when I should be saying: > > Server.Config.BaseURL > > >> but for the record, the way to use exec is like this: >> >> exec("object1 =lassname()") >> > > I failed to make that work. So back to the original question. How to > make an instance named according to a string inside a variable? I > guess it should be in the top-level namespace, not inside a list or > dictionary. > > -- Gnarlie > http://Gnarlodious.com/Gnarlodious > > I know you figure you have a specific list of files, and that they'll never conflict with the other global variables you've defined. But adding globals dynamically is "magic", and can lead to very obscure bugs. Suppose sometime in the future somebody adds another plist file with the same name as one of your functions? Put it inside a dummy class, as follows: >>> class Plist: pass ... >>> plists = Plist() >>> setattr(plists, "Server", 42) >>> plists.Server 42 >>> That avoids the ["Server"] syntax, but still doesn't risk polluting your global namespace with arbitrary names. And in the example above, you'd be using plists.Server.Config.BaseURL And if you must use the global namespace, you can simply do: >>> globals()["Server"] = 42 >>> Server 42 DaveA From apt.shansen at gmail.com Thu Jan 21 11:54:43 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Thu, 21 Jan 2010 08:54:43 -0800 Subject: Create object name from string value? In-Reply-To: <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> Message-ID: <7a9c25c21001210854o542fd9c6k9409fced12f42242@mail.gmail.com> On Thu, Jan 21, 2010 at 6:23 AM, Gnarlodious wrote: > This works! However I end up saying: > > d['Server'].Config.BaseURL > > to get the data, when I should be saying: > > Server.Config.BaseURL That's the thing: you should not actually be saying that. d['Server'].Config.BaseURL is precisely what you should be saying. You need to adjust your expectations to the language you are using and how it operates. Whatever aesthetic sense in you that is objecting to using a dictionary is something you just need to get past. If you don't like the dictionary getitem syntax, you can always do: >>> class AttrDict(dict): ... def __getattr__(self, key): ... return self[key] ... >>> a = AttrDict() >>> a["test"] = 1 >>> a.test 1 Is there any real reason you can't write Servers["Server"].Config.BaseURL or Servers.Server.Config.BaseURL, or is it just an itch that's bugging you? If its the latter, then it's better to just not scratch it and it'll go away when you get more used to the language :) Its not easy to do what you're asking for. People ask for it every once in awhile. But it's just not the Pythonic way to approach the problem, and while there are things you can do to accomplish it, they're hacks and are not guaranteed to work consistently. Its not even a oversight in Python, something the devs never considered: its intentionally designed to be this way. How come a dictionary (or class as above) isn't good enough? --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From showell30 at yahoo.com Thu Jan 21 11:54:44 2010 From: showell30 at yahoo.com (Steve Howell) Date: Thu, 21 Jan 2010 08:54:44 -0800 (PST) Subject: simple pub/sub Message-ID: <0af0eff2-50e2-44aa-81f1-b3d12cb265a7@a15g2000yqm.googlegroups.com> Hi, I'm looking for ideas on building a simple architecture that allows a bunch of independent Python processes to exchange data using files and perform calculations. One Python program would be collecting data from boat instruments on a serial port, then writing that info out to a file, and also transmitting data to instruments as needed (again through the serial port). It would be the most complex program in terms of having to poll file descriptors, or something similar, but I want to limit its job to pure communication handling. Then, I would have a bunch of programs doing various calculations on the input stream. It's totally acceptable if they just sleep a bit, wake up, see if there is any new data, and if so, do some calculations, so I will probably just use this: http://www.dabeaz.com/generators/follow.py Some of the "calculator" programs might be looking for trends in the data, and when a certain threshold is reached, they will need to notify the comm program of the new message. To avoid collisions between all the different calculator programs, I am thinking the simplest thing is just have them each write to their own output file. So then the comm program has multiple data sources. It wants to track to the serial port, but it also wants to monitor the output files from the various calculator processes. I want to do this in a mostly nonblocking way, so I would probably poll on the file descriptors. The only problems are that select "...cannot be used on regular files to determine whether a file has grown since it was last read," and it only works on Unix for files. (I'm using Unix myself, but one of the other programmers insists on Windows, and is resisting Python as well.) So a variation would be that the calculator programs talk to the comm program via sockets, which seems slightly on the heavy side, but it is certainly feasible. I know that something like twisted could probably solve my problem, but I'm also wondering if there is some solution that is a little more lightweight and batteries-included. At the heart of the matter, I just want the programs to be able to do the following tasks. 1) All programs will occasionally need to write a line of text to some data stream without having to lock it. (I don't care if the write blocks.) 2) All programs will need to be able to do a non-blocking read of a line of text from a data stream. (And there may be multiple consumers.) 3) The comm program will need to be able to poll the serial port and input data streams to see which ones are ready. Any thoughts? From zachar at awst.at Thu Jan 21 11:56:55 2010 From: zachar at awst.at (zachar at awst.at) Date: Thu, 21 Jan 2010 17:56:55 +0100 (CET) Subject: =?utf-8?Q?Python_sys=2Eprefix?= Message-ID: Hi All, I need some help about my "problem": I compiled and installed python with ./configure --prefix=/opt/pyhon2.6.4... But I have to move it to a different directory (the reason is very difficult but I must do this)... I started the interpreter from shell and I was surprised because It still worked and It changed the sys.prefix so it pointed to the correct place so the sys.path was correct.... The questions are: HOW? Is it a feature which is not documented (at least I couldn't find anything about it)? >From the official python documentation (http://docs.python.org/library/sys.html?highlight=sys#sys.prefix): "sys.prefix? A string giving the site-specific directory prefix where the platform independent Python files are installed; by default, this is the string '/usr/local'. This can be set at build time with the --prefix argument to the configure script. The main collection of Python library modules is installed in the directory prefix + '/lib/pythonversion' while the platform independent header files (all except pyconfig.h) are stored in prefix + '/include/pythonversion', where version is equal to version[:3]." In my case, this is not correct: it looks, after I call the interpreter, something change the sys.prefix variable and point it to the right place and not to that directory which I used with the configuration script... This is very useful (I am happy because it solves one of my problems) but I am afraid of it will be do it in a different way in the future (as it is not documented). Thanks, Regards, Balazs From awilliam at opengroupware.us Thu Jan 21 12:03:26 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Thu, 21 Jan 2010 12:03:26 -0500 Subject: simple pub/sub In-Reply-To: <0af0eff2-50e2-44aa-81f1-b3d12cb265a7@a15g2000yqm.googlegroups.com> References: <0af0eff2-50e2-44aa-81f1-b3d12cb265a7@a15g2000yqm.googlegroups.com> Message-ID: <1264093406.8831.25.camel@linux-m3mt> On Thu, 2010-01-21 at 08:54 -0800, Steve Howell wrote: > Hi, I'm looking for ideas on building a simple architecture that > allows a bunch of independent Python processes to exchange data using > files and perform calculations. > One Python program would be collecting data from boat instruments on a > serial port, then writing that info out to a file, and also > transmitting data to instruments as needed (again through the serial > port). It would be the most complex program in terms of having to > poll file descriptors, or something similar, but I want to limit its > job to pure communication handling. This should be pretty easy using multiprocessing. In OpenGroupware Coils we have a master process that spins up children (workers, that each provide a distinct service) it opens a Pipe to each child to send messages to the child and a Queue from which it reads [all children (clients) write to the Queue and listen on their Pipe]. Then the master just listens on its pipe and forwards messages from children to other children. All children are Service objects . Then implementing a new service [type of worker] is as easy as (our brutally simple pubsub service) Hope that helps. -- OpenGroupware developer: awilliam at whitemice.org OpenGroupare & Cyrus IMAPd documenation @ From showell30 at yahoo.com Thu Jan 21 12:28:52 2010 From: showell30 at yahoo.com (Steve Howell) Date: Thu, 21 Jan 2010 09:28:52 -0800 (PST) Subject: simple pub/sub References: <0af0eff2-50e2-44aa-81f1-b3d12cb265a7@a15g2000yqm.googlegroups.com> Message-ID: <3edfbb90-d32e-40e5-ba33-6720faaacbc9@k19g2000yqc.googlegroups.com> On Jan 21, 9:03?am, Adam Tauno Williams wrote: > On Thu, 2010-01-21 at 08:54 -0800, Steve Howell wrote: > > Hi, I'm looking for ideas on building a simple architecture that > > allows a bunch of independent Python processes to exchange data using > > files and perform calculations. > > One Python program would be collecting data from boat instruments on a > > serial port, then writing that info out to a file, and also > > transmitting data to instruments as needed (again through the serial > > port). ?It would be the most complex program in terms of having to > > poll file descriptors, or something similar, but I want to limit its > > job to pure communication handling. > > This should be pretty easy using multiprocessing. ?In OpenGroupware > Coils we have a master process > that spins up children (workers, that each provide a distinct service) it opens a Pipe to each child to send messages to the child and a Queue from which it reads [all children (clients) write to the Queue and listen on their Pipe]. ?Then the master just listens on its pipe and forwards messages from children to other children. > > All children are Service objects > . ?Then implementing a new service [type of worker] is as easy as (our brutally simple pubsub service) > > Hope that helps. > It does indeed! I'm gonna try this approach for now. From aahz at pythoncraft.com Thu Jan 21 12:37:58 2010 From: aahz at pythoncraft.com (Aahz) Date: 21 Jan 2010 09:37:58 -0800 Subject: simple pub/sub References: <0af0eff2-50e2-44aa-81f1-b3d12cb265a7@a15g2000yqm.googlegroups.com> Message-ID: In article <0af0eff2-50e2-44aa-81f1-b3d12cb265a7 at a15g2000yqm.googlegroups.com>, Steve Howell wrote: > >Hi, I'm looking for ideas on building a simple architecture that >allows a bunch of independent Python processes to exchange data using >files and perform calculations. SQLite? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From pavlovevidence at gmail.com Thu Jan 21 12:50:55 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Thu, 21 Jan 2010 09:50:55 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> Message-ID: <06bb8c18-371f-4d61-a45a-ae70d5e20f44@o28g2000yqh.googlegroups.com> On Jan 21, 2:38?am, "Alf P. Steinbach" wrote: > * Carl Banks: > > > On Jan 20, 11:43 pm, Martin Drautzburg > [snip] > > >> What I am really looking for is a way > > >> ? ? ? ? - to be able to call move(up) > >> ? ? ? ? - having the "up" symbol only in the context of the function call > > > Short answer is, you can't do it. > > On the contrary, it's not difficult to do. > > I provided an example in my answer to the OP (first reply in the thread). Your example doesn't remotely do what the OP was asking for. In fact your example is so preposterous I question your sanity. > However, it's IMHO an abuse of the language, not something that one should do. Usually people abuse the language to achieve something (ostensibly) useful. Your example is so useless I don't think I would even call it abuse. As best as I can tell, what it is is you attempting to make yourself look like some kind of badass by answering some absurdly literal interpretation of the OP's question. Except you haven't even done that: class using_directions: up = 42 move( up ) print up # <- clearly not confined to context of function call Carl Banks From garrickp at gmail.com Thu Jan 21 12:52:32 2010 From: garrickp at gmail.com (Falcolas) Date: Thu, 21 Jan 2010 09:52:32 -0800 (PST) Subject: Closures in metaclasses Message-ID: <2c126442-f0b8-4b38-82c7-7fb226b15b60@u41g2000yqe.googlegroups.com> I'm running into an issue with closures in metaclasses - that is, if I create a function with a closure in a metaclass, the closure appears to be lost when I access the final class. I end up getting the text 'param' instead of the actual tags I am expecting: ALL_TAGS = ['a', 'abbr', 'acronym', 'address', 'applet', 'b', 'bdo', 'big'] #snip def _tag_meta(name, bases, dict_): for tag in ALL_TAGS: def generic_tag(*args, **kwargs): return Tag._generate_tag(tag, *args, **kwargs) #generic_tag = eval("lambda *args, **kwargs: Tag._generate_tag ('%s', *args, **kwargs)" % tag) dict_[tag] = staticmethod(generic_tag) return type(name, bases, dict_) class Tag(object): __metaclass__ = _tag_meta @staticmethod def _generate_tag(tag_name, *args, **kwargs): # Does the expected, following is just for the example's sake return tag_name Here's the output from my actual class: $ python Python 2.6.4 (r264:75706, Dec 7 2009, 18:45:15) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from htmlgen import Tag >>> Tag.html("foo") '\nfoo\n' >>> Using the eval shown above works as expected, but I'd rather go with the closures, if someone can help me figure out what's going on. From zuo at chopin.edu.pl Thu Jan 21 13:08:45 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Thu, 21 Jan 2010 19:08:45 +0100 Subject: Sorted dictionary In-Reply-To: References: Message-ID: 21-01-2010, 07:21:22 Dennis Lee Bieber wrote: > How does this differ from all the other "ordered dictionary" schemes > out there (and maybe even included in 2.7? I'm still on 2.5) It's completely different, please read first paragraph of page I linked (http://code.activestate.com/recipes/576998/). Regards, *j -- Jan Kaliszewski (zuo) From iainspeed at gmail.com Thu Jan 21 13:13:38 2010 From: iainspeed at gmail.com (Iain Barnett) Date: Thu, 21 Jan 2010 18:13:38 +0000 Subject: installing psycopg2-2.0.13 with python3.1 In-Reply-To: References: <03495BC1-8736-42D5-9B59-5B0148F643AA@gmail.com> Message-ID: <31AC458E-97F9-48D3-9401-2D865232AE53@gmail.com> On 21 Jan 2010, at 00:11, Gabriel Genellina wrote: > En Wed, 20 Jan 2010 19:45:44 -0300, Iain Barnett escribi?: > >> Would anyone know if it's possible to install psycopg2-2.0.13 with python3.1.1 (or similar)?I can install it with python2.6 with no problems, but obviously I'd prefer to use the latest version. My system is OSX10.6, and I'm new to Python. > > If you're new to Python, perhaps it's better to stay with the 2.6 version. > > Python 3.x introduced some incompatible changes in the language; not all third-party packages are available for Python 3.x yet. Regarding psycopg2, although some work has been done [1], there is no "official" release compatible with Python 3 yet. > > If you insist on using Python 3.1, there is another interface to PostgreSQL called pg8000 that claims to be Python 3.x compatible (I've not actually tested it). > > [1] http://mail.python.org/pipermail/python-porting/2008-December/000004.html > [2] http://pybrary.net/pg8000/ > > -- > Gabriel Genellina > > -- > http://mail.python.org/mailman/listinfo/python-list On 20 Jan 2010, at 23:58, Philip Semanchuk wrote: > > On Jan 20, 2010, at 5:45 PM, Iain Barnett wrote: > >> Hi, >> >> Would anyone know if it's possible to install psycopg2-2.0.13 with python3.1.1 (or similar)? When I try I get the following >> >> $ sudo python setup.py install >> Password: >> File "setup.py", line 233 >> except Warning, w: >> ^ >> SyntaxError: invalid syntax >> >> >> I can install it with python2.6 with no problems, but obviously I'd prefer to use the latest version. My system is OSX10.6, and I'm new to Python. > > Hi Iain, > I've been out of the psycopg loop for a while now, but my guess is that the answer to your question is no. There are some Python 3-related checkins in the psycopg2 repository, but it looks like they're not ready for public use yet (unless you're adventurous). > > Here's the source repository, which contains a commit commented "First round of changes for Python 3": > https://dndg.it/cgi-bin/gitweb.cgi?p=public/psycopg2.git > > If you scroll to the bottom of that page you'll see a "head" created especially for Python 2 which is where 2.0.13 came from. > > If you're willing to do some hacking on your own, Martin v. L?wis ported psycopg2 to Python 3 and provided a diff: > http://mail.python.org/pipermail/python-porting/2008-December/000018.html > > Last but not least, I should note that psycopg has its own mailing list, and I bet they know more about it there: > http://lists.initd.org/mailman/listinfo/psycopg > > Hope this helps. Psycopg & Postgres both worked wonderfully well for me. > > Cheers > Philip > -- > http://mail.python.org/mailman/listinfo/python-list Thanks to both of you, it really is much appreciated. Regards Iain From arnodel at googlemail.com Thu Jan 21 13:24:32 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Thu, 21 Jan 2010 18:24:32 +0000 Subject: Closures in metaclasses References: <2c126442-f0b8-4b38-82c7-7fb226b15b60@u41g2000yqe.googlegroups.com> Message-ID: Falcolas writes: > I'm running into an issue with closures in metaclasses - that is, if I > create a function with a closure in a metaclass, the closure appears > to be lost when I access the final class. I end up getting the text > 'param' instead of the actual tags I am expecting: > > ALL_TAGS = ['a', 'abbr', 'acronym', 'address', 'applet', 'b', 'bdo', > 'big'] #snip > > def _tag_meta(name, bases, dict_): > for tag in ALL_TAGS: > def generic_tag(*args, **kwargs): > return Tag._generate_tag(tag, *args, **kwargs) > #generic_tag = eval("lambda *args, **kwargs: Tag._generate_tag > ('%s', *args, **kwargs)" % tag) > dict_[tag] = staticmethod(generic_tag) > return type(name, bases, dict_) This is almost a FAQ and has nothing to do with metaclasses. The simplest solution usually involves adding a default argument 'tag=tag' to the function you define (here, generic_tag), but you can't do this here because you have a **kwargs argument. Instead, you can use a closure and do this for example: def factory(tag): def generic_tag(*args, **kwargs): return Tag._generate_tag(tag, *args, **kwargs) return generic_tag def _tag_meta(name, bases, dict_): for tag in ALL_TAGS: dict_[tag] = staticmethod(factory(tag)) return type(name, bases, dict_) Then your Tag example class will work as you expect: class Tag(object): __metaclass__ = _tag_meta @staticmethod def _generate_tag(tag_name, *args, **kwargs): # Does the expected, following is just for the example's sake return tag_name I am sure that there are plenty of discussions of this issue in the archives of c.l.python. I just can't think of a good keyword to google it ATM. However, I am usure about why you are using a metaclass. HTH -- Arnaud From garrickp at gmail.com Thu Jan 21 13:37:47 2010 From: garrickp at gmail.com (Falcolas) Date: Thu, 21 Jan 2010 10:37:47 -0800 (PST) Subject: Closures in metaclasses References: <2c126442-f0b8-4b38-82c7-7fb226b15b60@u41g2000yqe.googlegroups.com> Message-ID: <7ceaaaa1-45c2-4875-a091-971c14e2d41e@f12g2000yqn.googlegroups.com> On Jan 21, 11:24?am, Arnaud Delobelle wrote: > Falcolas writes: > > I'm running into an issue with closures in metaclasses - that is, if I > > create a function with a closure in a metaclass, the closure appears > > to be lost when I access the final class. I end up getting the text > > 'param' instead of the actual tags I am expecting: > > > ALL_TAGS = ?['a', 'abbr', 'acronym', 'address', 'applet', 'b', 'bdo', > > 'big'] #snip > > > def _tag_meta(name, bases, dict_): > > ? ? for tag in ALL_TAGS: > > ? ? ? ? def generic_tag(*args, **kwargs): > > ? ? ? ? ? ? return Tag._generate_tag(tag, *args, **kwargs) > > ? ? ? ? #generic_tag = eval("lambda *args, **kwargs: Tag._generate_tag > > ('%s', *args, **kwargs)" % tag) > > ? ? ? ? dict_[tag] = staticmethod(generic_tag) > > ? ? return type(name, bases, dict_) > > This is almost a FAQ and has nothing to do with metaclasses. ?The > simplest solution usually involves adding a default argument 'tag=tag' > to the function you define (here, generic_tag), but you can't do this > here because you have a **kwargs argument. ?Instead, you can use a > closure and do this for example: > > def factory(tag): > ? ? def generic_tag(*args, **kwargs): > ? ? ? ? return Tag._generate_tag(tag, *args, **kwargs) > ? ? return generic_tag > > def _tag_meta(name, bases, dict_): > ? ? for tag in ALL_TAGS: > ? ? ? ? dict_[tag] = staticmethod(factory(tag)) > ? ? return type(name, bases, dict_) I see - I was thinking it would preserve the closure from the for statement, but I can see now why that would be wrong. > > However, I am usure about why you are using a metaclass. > > HTH > > -- > Arnaud It was the easiest way I found to add a lot of static methods to the Tag class without writing each one out. __getattr__ was not working for this application. This is for a very simple application, and I didn't want to add a lot of complexity to it's use. I'm always open for other options OTOH. From deets at nospam.web.de Thu Jan 21 13:43:41 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Thu, 21 Jan 2010 19:43:41 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> Message-ID: <7rrliuFl3gU1@mid.uni-berlin.de> Am 21.01.10 12:58, schrieb Alf P. Steinbach: > * Stefan Behnel: >> Alf P. Steinbach, 21.01.2010 11:38: >>> * Carl Banks: >>>> On Jan 20, 11:43 pm, Martin Drautzburg >>> [snip] >>>>> What I am really looking for is a way >>>>> >>>>> - to be able to call move(up) >>>>> - having the "up" symbol only in the context of the function >>>>> call >>>> Short answer is, you can't do it. >>>> >>> On the contrary, it's not difficult to do. >>> >>> I provided an example in my answer to the OP (first reply in the >>> thread). >> >> Erm, no, you didn't. You showed a) how to pass a string constant into a >> function and b) how to pass a value from a bound variable. None of >> that is >> helpful to the OP's problem. > > Perhaps look again at that example. It demonstrates exactly what the OP > asks for. Perhaps look again at the requirements: """ - the parameter IS REALLY NOT A STRING, but a direction """ Diez From zuo at chopin.edu.pl Thu Jan 21 13:43:45 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Thu, 21 Jan 2010 19:43:45 +0100 Subject: Sorted dictionary In-Reply-To: <50697b2c1001202349m7b346df5q4686dfa51b29bc13@mail.gmail.com> References: <50697b2c1001202349m7b346df5q4686dfa51b29bc13@mail.gmail.com> Message-ID: Dnia 21-01-2010 o 08:49:22 Chris Rebert napisa?(a): > On Wed, Jan 20, 2010 at 5:50 PM, Steven D'Aprano > wrote: >> On Thu, 21 Jan 2010 02:02:02 +0100, Jan Kaliszewski wrote: >>> http://code.activestate.com/recipes/576998/ >> What's the advantage of that over sorting the keys as needed? >> >> >> E.g. >> >> for key in sorted(dict): >> ? ?print key >> >> >> works. > > Well, it does spread out the cost of sorting the key list over each of > the N distinct key assignments, versus sorting the entire list all at > once, on-demand at the end of the process. And you avoid having to > traverse the M buckets in the dictionary to locate the N keys in the > first place. So it has different performance characteristics, which > could theoretically matter depending on the use case; e.g. it could > avoid a GUI application hanging for several seconds while it sorts a > large list of keys. Generally, I see 3 possible approaches: 1. What Steven wrote about: simply sort all keys when you need to get them sorted (it needs iteration over whole a dict). In many cases it's good enough (so is the best, because of simplicity) -- e.g. when a dict is not very long (very common case), or when that sorting it is a rare operation. Sort always when a key is added/deleted, maintaining an auxiliary list of sorted keys (sufficient especially when you modify a dict rarely and get from it often). 2. Sort all keys on-demand -- but only if a dict is marked as "modified". Mark a dict as "modified" when you add/delete any key; and mark a dict as "unmodified" when sorting. (Of course, it can be automatized, see e.g.: http://pypi.python.org/pypi/sorteddict). Nice, if your use-pattern is: add/del a lot, *then* only get/iter a lot... 3. Continually (at each set/del of a key) maintain auxiliary sorted list of keys -- using binary search (as I did), heap queue (see: http://pypi.python.org/pypi/HeapDict) or such an algorithm. I think this approach is the best when you have quite long dict, and you add/del fairly often and get/iter very often. Regards, *j -- Jan Kaliszewski (zuo) From zuo at chopin.edu.pl Thu Jan 21 13:45:56 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Thu, 21 Jan 2010 19:45:56 +0100 Subject: Sorted dictionary In-Reply-To: <1c74df0b-b22e-4aea-934a-c6d25e818f41@e37g2000yqn.googlegroups.com> References: <1c74df0b-b22e-4aea-934a-c6d25e818f41@e37g2000yqn.googlegroups.com> Message-ID: Dnia 21-01-2010 o 09:27:52 Raymond Hettinger napisa?(a): > On Jan 20, 5:02?pm, "Jan Kaliszewski" wrote: >> http://code.activestate.com/recipes/576998/ > Using an underlying list to track sorted items > means that insertion and deletion take O(n) time. > That could be reduced to O(log n) time by using > a blist or skiplist as the underlying structure > for maintaining a sort. Please note that I used funcions from bisect, that use binary search. Doesn't it take O(log n) time? *j -- Jan Kaliszewski (zuo) From alfps at start.no Thu Jan 21 13:46:00 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 19:46:00 +0100 Subject: Symbols as parameters? In-Reply-To: <06bb8c18-371f-4d61-a45a-ae70d5e20f44@o28g2000yqh.googlegroups.com> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <06bb8c18-371f-4d61-a45a-ae70d5e20f44@o28g2000yqh.googlegroups.com> Message-ID: * Carl Banks: > On Jan 21, 2:38 am, "Alf P. Steinbach" wrote: >> * Carl Banks: >> >>> On Jan 20, 11:43 pm, Martin Drautzburg >> [snip] >> >>>> What I am really looking for is a way >>>> - to be able to call move(up) >>>> - having the "up" symbol only in the context of the function call >>> Short answer is, you can't do it. >> On the contrary, it's not difficult to do. >> >> I provided an example in my answer to the OP (first reply in the thread). > > Your example doesn't remotely do what the OP was asking for. In fact > your example is so preposterous I question your sanity. Your first sentence is incorrect, your second sentence is a silly attempt at getting personal. >> However, it's IMHO an abuse of the language, not something that one should do. > > Usually people abuse the language to achieve something (ostensibly) > useful. Your example is so useless I don't think I would even call it > abuse. You are correct that it's useless. The OP asked for a construct to do a useless thing. The presented construct does exactly what the OP asked for: useless. > As best as I can tell, what it is is you attempting to make yourself > look like some kind of badass by answering some absurdly literal > interpretation of the OP's question. Hm, there are ways to do things, even the ad hominem thing. You just present yourself as one using strong words when you're proven wrong. Myself I'm not foreign to strong words :-), but I wouldn't dream of applying them to a person. The above is very, uh, primitive. Besides, it's quite silly to get angry when you're proved to be wrong. :-) > Except you haven't even done that: > > class using_directions: > up = 42 > move( up ) > print up # <- clearly not confined to context of function call You know, I didn't think of that ingenious thing, that it would be possible to *modify* the example so that it no longer fit the OP's literal description. Thx! Cheers & hth., - Alf From alfps at start.no Thu Jan 21 13:48:02 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 19:48:02 +0100 Subject: Symbols as parameters? In-Reply-To: <7rrliuFl3gU1@mid.uni-berlin.de> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> <7rrliuFl3gU1@mid.uni-berlin.de> Message-ID: * Diez B. Roggisch: > Am 21.01.10 12:58, schrieb Alf P. Steinbach: >> * Stefan Behnel: >>> Alf P. Steinbach, 21.01.2010 11:38: >>>> * Carl Banks: >>>>> On Jan 20, 11:43 pm, Martin Drautzburg >>>> [snip] >>>>>> What I am really looking for is a way >>>>>> >>>>>> - to be able to call move(up) >>>>>> - having the "up" symbol only in the context of the function >>>>>> call >>>>> Short answer is, you can't do it. >>>>> >>>> On the contrary, it's not difficult to do. >>>> >>>> I provided an example in my answer to the OP (first reply in the >>>> thread). >>> >>> Erm, no, you didn't. You showed a) how to pass a string constant into a >>> function and b) how to pass a value from a bound variable. None of >>> that is >>> helpful to the OP's problem. >> >> Perhaps look again at that example. It demonstrates exactly what the OP >> asks for. > > Perhaps look again at the requirements: > > """ > - the parameter IS REALLY NOT A STRING, but a direction > """ So? Cheers & hth., - Alf From deets at nospam.web.de Thu Jan 21 13:53:01 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Thu, 21 Jan 2010 19:53:01 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> <7rrliuFl3gU1@mid.uni-berlin.de> Message-ID: <7rrm4dFofqU1@mid.uni-berlin.de> Am 21.01.10 19:48, schrieb Alf P. Steinbach: > * Diez B. Roggisch: >> Am 21.01.10 12:58, schrieb Alf P. Steinbach: >>> * Stefan Behnel: >>>> Alf P. Steinbach, 21.01.2010 11:38: >>>>> * Carl Banks: >>>>>> On Jan 20, 11:43 pm, Martin Drautzburg >>>>> [snip] >>>>>>> What I am really looking for is a way >>>>>>> >>>>>>> - to be able to call move(up) >>>>>>> - having the "up" symbol only in the context of the function >>>>>>> call >>>>>> Short answer is, you can't do it. >>>>>> >>>>> On the contrary, it's not difficult to do. >>>>> >>>>> I provided an example in my answer to the OP (first reply in the >>>>> thread). >>>> >>>> Erm, no, you didn't. You showed a) how to pass a string constant into a >>>> function and b) how to pass a value from a bound variable. None of >>>> that is >>>> helpful to the OP's problem. >>> >>> Perhaps look again at that example. It demonstrates exactly what the OP >>> asks for. >> >> Perhaps look again at the requirements: >> >> """ >> - the parameter IS REALLY NOT A STRING, but a direction >> """ > > So? Oh please. You claim you provided exactly what the OP asked for. But in the body of move, all you can do is to compare the direction parameter tco "up", not to UP. So no, you provided *something*, but not what the OP asked for. Diez From alfps at start.no Thu Jan 21 14:01:04 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 20:01:04 +0100 Subject: Symbols as parameters? In-Reply-To: <7rrm4dFofqU1@mid.uni-berlin.de> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> <7rrliuFl3gU1@mid.uni-berlin.de> <7rrm4dFofqU1@mid.uni-berlin.de> Message-ID: * Diez B. Roggisch: > Am 21.01.10 19:48, schrieb Alf P. Steinbach: >> * Diez B. Roggisch: >>> Am 21.01.10 12:58, schrieb Alf P. Steinbach: >>>> * Stefan Behnel: >>>>> Alf P. Steinbach, 21.01.2010 11:38: >>>>>> * Carl Banks: >>>>>>> On Jan 20, 11:43 pm, Martin Drautzburg >>>>>> [snip] >>>>>>>> What I am really looking for is a way >>>>>>>> >>>>>>>> - to be able to call move(up) >>>>>>>> - having the "up" symbol only in the context of the function >>>>>>>> call >>>>>>> Short answer is, you can't do it. >>>>>>> >>>>>> On the contrary, it's not difficult to do. >>>>>> >>>>>> I provided an example in my answer to the OP (first reply in the >>>>>> thread). >>>>> >>>>> Erm, no, you didn't. You showed a) how to pass a string constant >>>>> into a >>>>> function and b) how to pass a value from a bound variable. None of >>>>> that is >>>>> helpful to the OP's problem. >>>> >>>> Perhaps look again at that example. It demonstrates exactly what the OP >>>> asks for. >>> >>> Perhaps look again at the requirements: >>> >>> """ >>> - the parameter IS REALLY NOT A STRING, but a direction >>> """ >> >> So? > > Oh please. You claim you provided exactly what the OP asked for. But in > the body of move, all you can do is to compare the direction parameter > tco "up", not to UP. So no, you provided *something*, but not what the > OP asked for. Pardon me for not taking you seriously or not getting the joke, if that's what it's meant as. Cheers, - Alf From deets at nospam.web.de Thu Jan 21 14:06:35 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Thu, 21 Jan 2010 20:06:35 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> <7rrliuFl3gU1@mid.uni-berlin.de> <7rrm4dFofqU1@mid.uni-berlin.de> Message-ID: <7rrmttFtgjU1@mid.uni-berlin.de> Am 21.01.10 20:01, schrieb Alf P. Steinbach: > * Diez B. Roggisch: >> Am 21.01.10 19:48, schrieb Alf P. Steinbach: >>> * Diez B. Roggisch: >>>> Am 21.01.10 12:58, schrieb Alf P. Steinbach: >>>>> * Stefan Behnel: >>>>>> Alf P. Steinbach, 21.01.2010 11:38: >>>>>>> * Carl Banks: >>>>>>>> On Jan 20, 11:43 pm, Martin Drautzburg >>>>>>> [snip] >>>>>>>>> What I am really looking for is a way >>>>>>>>> >>>>>>>>> - to be able to call move(up) >>>>>>>>> - having the "up" symbol only in the context of the function >>>>>>>>> call >>>>>>>> Short answer is, you can't do it. >>>>>>>> >>>>>>> On the contrary, it's not difficult to do. >>>>>>> >>>>>>> I provided an example in my answer to the OP (first reply in the >>>>>>> thread). >>>>>> >>>>>> Erm, no, you didn't. You showed a) how to pass a string constant >>>>>> into a >>>>>> function and b) how to pass a value from a bound variable. None of >>>>>> that is >>>>>> helpful to the OP's problem. >>>>> >>>>> Perhaps look again at that example. It demonstrates exactly what >>>>> the OP >>>>> asks for. >>>> >>>> Perhaps look again at the requirements: >>>> >>>> """ >>>> - the parameter IS REALLY NOT A STRING, but a direction >>>> """ >>> >>> So? >> >> Oh please. You claim you provided exactly what the OP asked for. But >> in the body of move, all you can do is to compare the direction >> parameter tco "up", not to UP. So no, you provided *something*, but >> not what the OP asked for. > > Pardon me for not taking you seriously or not getting the joke, if > that's what it's meant as. No joking, unless you started it somewhere earlier. The OP didn't want to compare to a string, yet your "example" forces him to exactly do that. If not, would you care to show us the fulfillment of "the parameter IS REALLY NOT A STRING, but a direction" in your example? Thanks. You see, it seems that you agreeing to be wrong is something your are so adamantly refusing to acknowledge that I'm inclined to believe that in the event of this really happen one day, a great rift in the very fabric of space and time would appear, and the world would end. If this conversation was in my native language and not in yours, I'd might give it a shot, just to see what really happens. But as it isn't, and I guess everybody else except you knows that your wrong - I wont. HTH, Diez From arnodel at googlemail.com Thu Jan 21 14:10:07 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Thu, 21 Jan 2010 11:10:07 -0800 (PST) Subject: Closures in metaclasses References: <2c126442-f0b8-4b38-82c7-7fb226b15b60@u41g2000yqe.googlegroups.com> <7ceaaaa1-45c2-4875-a091-971c14e2d41e@f12g2000yqn.googlegroups.com> Message-ID: <48807f29-3157-43b8-9b01-37ccf5f00616@a15g2000yqm.googlegroups.com> On Jan 21, 6:37?pm, Falcolas wrote: > On Jan 21, 11:24?am, Arnaud Delobelle wrote: > > > > > > > Falcolas writes: > > > I'm running into an issue with closures in metaclasses - that is, if I > > > create a function with a closure in a metaclass, the closure appears > > > to be lost when I access the final class. I end up getting the text > > > 'param' instead of the actual tags I am expecting: > > > > ALL_TAGS = ?['a', 'abbr', 'acronym', 'address', 'applet', 'b', 'bdo', > > > 'big'] #snip > > > > def _tag_meta(name, bases, dict_): > > > ? ? for tag in ALL_TAGS: > > > ? ? ? ? def generic_tag(*args, **kwargs): > > > ? ? ? ? ? ? return Tag._generate_tag(tag, *args, **kwargs) > > > ? ? ? ? #generic_tag = eval("lambda *args, **kwargs: Tag._generate_tag > > > ('%s', *args, **kwargs)" % tag) > > > ? ? ? ? dict_[tag] = staticmethod(generic_tag) > > > ? ? return type(name, bases, dict_) > > > This is almost a FAQ and has nothing to do with metaclasses. ?The > > simplest solution usually involves adding a default argument 'tag=tag' > > to the function you define (here, generic_tag), but you can't do this > > here because you have a **kwargs argument. ?Instead, you can use a > > closure and do this for example: > > > def factory(tag): > > ? ? def generic_tag(*args, **kwargs): > > ? ? ? ? return Tag._generate_tag(tag, *args, **kwargs) > > ? ? return generic_tag > > > def _tag_meta(name, bases, dict_): > > ? ? for tag in ALL_TAGS: > > ? ? ? ? dict_[tag] = staticmethod(factory(tag)) > > ? ? return type(name, bases, dict_) > > I see - I was thinking it would preserve the closure from the for > statement, but I can see now why that would be wrong. > > > > > However, I am usure about why you are using a metaclass. > > > HTH > > > -- > > Arnaud > > It was the easiest way I found to add a lot of static methods to the > Tag class without writing each one out. __getattr__ was not working > for this application. This is for a very simple application, and I > didn't want to add a lot of complexity to it's use. I'm always open > for other options OTOH. This should work (untested): class Tag(object): @staticmethod def _generate_tag(tag_name, *args, **kwargs): # Does the expected, following is just for the example's sake return tag for tag in ALL_TAGS: setattr(Tag, tag, staticmethod(factory(tag))) Or you could override __getattr__ -- Arnaud From alfps at start.no Thu Jan 21 14:24:12 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 20:24:12 +0100 Subject: Symbols as parameters? In-Reply-To: <7rrmttFtgjU1@mid.uni-berlin.de> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> <7rrliuFl3gU1@mid.uni-berlin.de> <7rrm4dFofqU1@mid.uni-berlin.de> <7rrmttFtgjU1@mid.uni-berlin.de> Message-ID: * Diez B. Roggisch: > Am 21.01.10 20:01, schrieb Alf P. Steinbach: >> * Diez B. Roggisch: >>> Am 21.01.10 19:48, schrieb Alf P. Steinbach: >>>> * Diez B. Roggisch: >>>>> Am 21.01.10 12:58, schrieb Alf P. Steinbach: >>>>>> * Stefan Behnel: >>>>>>> Alf P. Steinbach, 21.01.2010 11:38: >>>>>>>> * Carl Banks: >>>>>>>>> On Jan 20, 11:43 pm, Martin Drautzburg >>>>>>>> [snip] >>>>>>>>>> What I am really looking for is a way >>>>>>>>>> >>>>>>>>>> - to be able to call move(up) >>>>>>>>>> - having the "up" symbol only in the context of the function >>>>>>>>>> call >>>>>>>>> Short answer is, you can't do it. >>>>>>>>> >>>>>>>> On the contrary, it's not difficult to do. >>>>>>>> >>>>>>>> I provided an example in my answer to the OP (first reply in the >>>>>>>> thread). >>>>>>> >>>>>>> Erm, no, you didn't. You showed a) how to pass a string constant >>>>>>> into a >>>>>>> function and b) how to pass a value from a bound variable. None of >>>>>>> that is >>>>>>> helpful to the OP's problem. >>>>>> >>>>>> Perhaps look again at that example. It demonstrates exactly what >>>>>> the OP >>>>>> asks for. >>>>> >>>>> Perhaps look again at the requirements: >>>>> >>>>> """ >>>>> - the parameter IS REALLY NOT A STRING, but a direction >>>>> """ >>>> >>>> So? >>> >>> Oh please. You claim you provided exactly what the OP asked for. But >>> in the body of move, all you can do is to compare the direction >>> parameter tco "up", not to UP. So no, you provided *something*, but >>> not what the OP asked for. >> >> Pardon me for not taking you seriously or not getting the joke, if >> that's what it's meant as. > > No joking, unless you started it somewhere earlier. The OP didn't want > to compare to a string, yet your "example" forces him to exactly do > that. If not, would you care to show us the fulfillment of "the > parameter IS REALLY NOT A STRING, but a direction" in your example? Thanks. Just use a "direction" instead of a number (if you didn't like the number). I'm sorry that I couldn't believe that you didn't understand that. But OK, you didn't. The OP gave no definition of his "direction" though, so you'll have to use your good sense about possibilities for what it could be. From the context and later comments (especially about "import") my guess us that he's talking about numbers denoting directions, but you'd have to ask him. > You see, it seems that you agreeing to be wrong is something your are so > adamantly refusing to acknowledge that I'm inclined to believe that in > the event of this really happen one day, a great rift in the very fabric > of space and time would appear, and the world would end. If this > conversation was in my native language and not in yours, I'd might give > it a shot, just to see what really happens. But as it isn't, and I guess > everybody else except you knows that your wrong - I wont. The above is a silly ad hominem attack, containing 1 lie, and trying to confuse the issue: you're posting this in a different part of the thread as a response to Carls Banks being proved wrong by my example. Incidentally, he also resorts to ad hominem. Sadly it seems to be a common response pattern in this group. Do you understand how bad that makes you look? Cheers & hth., - Alf From ndbecker2 at gmail.com Thu Jan 21 14:27:14 2010 From: ndbecker2 at gmail.com (Neal Becker) Date: Thu, 21 Jan 2010 14:27:14 -0500 Subject: multiprocessing as batch system References: <1264090591.8831.10.camel@linux-m3mt> Message-ID: Adam Tauno Williams wrote: > On Thu, 2010-01-21 at 07:13 -0500, Neal Becker wrote: >> I'm using multiprocessing as a poor man's batch system. It is working >> OK, except that it doesn't seem to do load balancing quite right. >> I have a 8-cpu machine. If I start, using multiprocessing pool, calling >> map >> with more than 8 elements, it will start 8 processes. It seems, though, >> that when some of them finish, it does not start others right away. The >> load average may be a lot less than 8 but still additional processes are >> not active. > > If your processes are I/O bound your system will never appear to be > saturated. Multiple pending processes and a low load average usually > indicated waiting on some other resource. No, 100% compute bound. My impression was that some processes completed, but new ones were not immediately started. From daniel at stutzbachenterprises.com Thu Jan 21 14:35:55 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Thu, 21 Jan 2010 13:35:55 -0600 Subject: Sorted dictionary In-Reply-To: References: <1c74df0b-b22e-4aea-934a-c6d25e818f41@e37g2000yqn.googlegroups.com> Message-ID: On Thu, Jan 21, 2010 at 12:45 PM, Jan Kaliszewski wrote: > Please note that I used funcions from bisect, that use binary search. > > Doesn't it take O(log n) time? > It takes O(log n) time to find the point to insert, but O(n) time to perform the actual insertion. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnodel at googlemail.com Thu Jan 21 14:41:53 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Thu, 21 Jan 2010 19:41:53 +0000 Subject: Sorted dictionary References: <1c74df0b-b22e-4aea-934a-c6d25e818f41@e37g2000yqn.googlegroups.com> Message-ID: "Jan Kaliszewski" writes: > Dnia 21-01-2010 o 09:27:52 Raymond Hettinger napisa?(a): > >> On Jan 20, 5:02?pm, "Jan Kaliszewski" wrote: > >>> http://code.activestate.com/recipes/576998/ > >> Using an underlying list to track sorted items >> means that insertion and deletion take O(n) time. >> That could be reduced to O(log n) time by using >> a blist or skiplist as the underlying structure >> for maintaining a sort. > > Please note that I used funcions from bisect, that use binary search. > > Doesn't it take O(log n) time? Insertion and deletion are still O(n) as all items to the right of the inserted/deleted one have to be shifted by one place. -- Arnaud From pavlovevidence at gmail.com Thu Jan 21 14:43:12 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Thu, 21 Jan 2010 11:43:12 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <06bb8c18-371f-4d61-a45a-ae70d5e20f44@o28g2000yqh.googlegroups.com> Message-ID: On Jan 21, 10:46?am, "Alf P. Steinbach" wrote: > * Carl Banks: > > > > > > > On Jan 21, 2:38 am, "Alf P. Steinbach" wrote: > >> * Carl Banks: > > >>> On Jan 20, 11:43 pm, Martin Drautzburg > >> [snip] > > >>>> What I am really looking for is a way > >>>> ? ? ? ? - to be able to call move(up) > >>>> ? ? ? ? - having the "up" symbol only in the context of the function call > >>> Short answer is, you can't do it. > >> On the contrary, it's not difficult to do. > > >> I provided an example in my answer to the OP (first reply in the thread). > > > Your example doesn't remotely do what the OP was asking for. ?In fact > > your example is so preposterous I question your sanity. > > Your first sentence is incorrect, your second sentence is a silly attempt at > getting personal. > > >> However, it's IMHO an abuse of the language, not something that one should do. > > > Usually people abuse the language to achieve something (ostensibly) > > useful. ?Your example is so useless I don't think I would even call it > > abuse. > > You are correct that it's useless. The OP asked for a construct to do a useless > thing. The presented construct does exactly what the OP asked for: useless. > > > As best as I can tell, what it is is you attempting to make yourself > > look like some kind of badass by answering some absurdly literal > > interpretation of the OP's question. > > Hm, there are ways to do things, even the ad hominem thing. You just present > yourself as one using strong words when you're proven wrong. Myself I'm not > foreign to strong words :-), but I wouldn't dream of applying them to a person. > > The above is very, uh, primitive. > > Besides, it's quite silly to get angry when you're proved to be wrong. :-) > > > Except you haven't even done that: > > > class using_directions: > > ? ? up = 42 > > ? ? move( up ) > > ? ? print up # <- clearly not confined to context of function call > > You know, I didn't think of that ingenious thing, that it would be possible to > *modify* the example so that it no longer fit the OP's literal description. Thx! I'm not sure if you're trolling, insane, or just stupid; regardless, I'm done with you. Carl Banks From alfps at start.no Thu Jan 21 14:53:19 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 20:53:19 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <06bb8c18-371f-4d61-a45a-ae70d5e20f44@o28g2000yqh.googlegroups.com> Message-ID: * Carl Banks: > On Jan 21, 10:46 am, "Alf P. Steinbach" wrote: >> * Carl Banks: >>> On Jan 21, 2:38 am, "Alf P. Steinbach" wrote: >>>> * Carl Banks: >>>>> On Jan 20, 11:43 pm, Martin Drautzburg >>>> [snip] >>>>>> What I am really looking for is a way >>>>>> - to be able to call move(up) >>>>>> - having the "up" symbol only in the context of the function call >>>>> Short answer is, you can't do it. >>>> On the contrary, it's not difficult to do. >>>> I provided an example in my answer to the OP (first reply in the thread). >>> Your example doesn't remotely do what the OP was asking for. In fact >>> your example is so preposterous I question your sanity. >> Your first sentence is incorrect, your second sentence is a silly attempt at >> getting personal. >> >>>> However, it's IMHO an abuse of the language, not something that one should do. >>> Usually people abuse the language to achieve something (ostensibly) >>> useful. Your example is so useless I don't think I would even call it >>> abuse. >> You are correct that it's useless. The OP asked for a construct to do a useless >> thing. The presented construct does exactly what the OP asked for: useless. >> >>> As best as I can tell, what it is is you attempting to make yourself >>> look like some kind of badass by answering some absurdly literal >>> interpretation of the OP's question. >> Hm, there are ways to do things, even the ad hominem thing. You just present >> yourself as one using strong words when you're proven wrong. Myself I'm not >> foreign to strong words :-), but I wouldn't dream of applying them to a person. >> >> The above is very, uh, primitive. >> >> Besides, it's quite silly to get angry when you're proved to be wrong. :-) >> >>> Except you haven't even done that: >>> class using_directions: >>> up = 42 >>> move( up ) >>> print up # <- clearly not confined to context of function call >> You know, I didn't think of that ingenious thing, that it would be possible to >> *modify* the example so that it no longer fit the OP's literal description. Thx! > > I'm not sure if you're trolling, insane, or just stupid; regardless, > I'm done with you. Piling more ad hominem attacks on top of your previous is pretty silly. You were technically wrong, that's no big issue, everybody is wrong now and then. But reacting to a polite notification of your error, by going all out with various personal accusations, doesn't give anybody a positive impression of you. Cheers & hth., - Alf From phlip2005 at gmail.com Thu Jan 21 15:24:31 2010 From: phlip2005 at gmail.com (Phlip) Date: Thu, 21 Jan 2010 12:24:31 -0800 (PST) Subject: counting lines of code References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> <87bpgodj95.fsf@benfinney.id.au> <1f166703-fdfc-4f87-b336-ef80e492635b@a32g2000yqm.googlegroups.com> Message-ID: <7e09df6a-cda1-480e-a971-8f8a70ac4118@b9g2000yqd.googlegroups.com> On Jan 20, 11:20?pm, Michele Simionato wrote: > pylint does too many things, I want something fast that just counts > the lines and can be run on thousands of files at once. > cloc seems fine, I have just tried on 2,000 files and it gives me a > report in just a few seconds. In my experience with Python codebases that big... ...how many of those lines are duplicated, and might merge together into a better design? The LOC would go down, too. -- Phlip From aahz at pythoncraft.com Thu Jan 21 15:36:41 2010 From: aahz at pythoncraft.com (Aahz) Date: 21 Jan 2010 12:36:41 -0800 Subject: Dynamic HTML controls References: Message-ID: In article , Alan Harris-Reid wrote: > >Does anyone know where I can find any decent dynamically-constructed >HTML control classes (dropdown list, table, input field, checkbox, etc.) >written in Python. For example, for a HTML table I would like something >like... You might look at Quixote: http://quixote.ca/ -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From garrickp at gmail.com Thu Jan 21 15:38:09 2010 From: garrickp at gmail.com (Falcolas) Date: Thu, 21 Jan 2010 12:38:09 -0800 (PST) Subject: Closures in metaclasses References: <2c126442-f0b8-4b38-82c7-7fb226b15b60@u41g2000yqe.googlegroups.com> <7ceaaaa1-45c2-4875-a091-971c14e2d41e@f12g2000yqn.googlegroups.com> <48807f29-3157-43b8-9b01-37ccf5f00616@a15g2000yqm.googlegroups.com> Message-ID: On Jan 21, 12:10?pm, Arnaud Delobelle wrote: > On Jan 21, 6:37?pm, Falcolas wrote: > > > On Jan 21, 11:24?am, Arnaud Delobelle wrote: > > > It was the easiest way I found to add a lot of static methods to the > > Tag class without writing each one out. __getattr__ was not working > > for this application. This is for a very simple application, and I > > didn't want to add a lot of complexity to it's use. I'm always open > > for other options OTOH. > > This should work (untested): > > class Tag(object): > ? ? @staticmethod > ? ? def _generate_tag(tag_name, *args, **kwargs): > ? ? ? ? # Does the expected, following is just for the example's sake > ? ? ? ? return tag > > for tag in ALL_TAGS: > ? ? setattr(Tag, tag, staticmethod(factory(tag))) > > Or you could override __getattr__ > > -- > Arnaud I tried overriding __getattr__ and got an error at runtime (the instance did not have xyz key, etc), and the Tag dict is not modifiable (granted, I tried Tag.__dict__[tag] = spam, not setattr()). From __peter__ at web.de Thu Jan 21 15:55:09 2010 From: __peter__ at web.de (Peter Otten) Date: Thu, 21 Jan 2010 21:55:09 +0100 Subject: Closures in metaclasses References: <2c126442-f0b8-4b38-82c7-7fb226b15b60@u41g2000yqe.googlegroups.com> <7ceaaaa1-45c2-4875-a091-971c14e2d41e@f12g2000yqn.googlegroups.com> <48807f29-3157-43b8-9b01-37ccf5f00616@a15g2000yqm.googlegroups.com> Message-ID: Falcolas wrote: > I tried overriding __getattr__ and got an error at runtime (the You can either move __getattr__() into the metaclass or instantiate the class. I prefer the latter. Both approaches in one example: >>> class Tag: ... class __metaclass__(type): ... def __getattr__(self, name): return "<%s> via metaclass" % name ... def __getattr__(self, name): return "<%s> via class" % name ... >>> Tag.yadda ' via metaclass' >>> tag = Tag() >>> tag.yadda ' via class' From lacrima.maxim at gmail.com Thu Jan 21 15:58:58 2010 From: lacrima.maxim at gmail.com (Lacrima) Date: Thu, 21 Jan 2010 12:58:58 -0800 (PST) Subject: what test runner should I use? References: Message-ID: <3bebc5ef-f867-4566-a5b4-b1cc34fc7c81@b10g2000yqa.googlegroups.com> On Jan 19, 12:56?pm, Chris Withers wrote: > Hi All, > > I'm wondering what test runner I should use. Here's my list of requirements: > > - cross platform (I develop for and on Windows, Linux and Mac) > > - should not prevent tests from running with other test runners > ? ?(so no plugins/layers/etc that only work with one specific test > ? ? runner) > > - should work with zc.buildout (preferably without a specialist recipe!) > > So far I've tried the following with the resultant problems: > > zope.testing > > ? - requires a special recipe to be useful > ? - now emits deprecation warnings from itself: > ? ?https://mail.zope.org/pipermail/zope-dev/2009-December/038965.html > ? - coverage support is baroque to put it politely > > twisted's trial > > ? - only has old-style script definition in setup.py, so doesn't work > ? ? with buildout without hackery > > ? - drops _twisted_trial folders all over the place and doesn't clear > ? ? them up > > nose > > ? - can't see to get it to run only my packages tests, rather than > ? ? including the tests of packages my package depends on > > ? - seems to be focused towards files rather than modules > ? ? (which makes it not play nicely with buildout) > > ? - seems to be difficult to provide options to at configuration time > ? ? that can then be overridden on the command line > > I did also look at py.test's homepage but found it pretty scary. > > What other options do people recommend? > Failing that, any ideas how to fix the problems above? > > cheers, > > Chris Nose should work pretty well with buildout. You need this in your buildout.cfg [buildout] parts = test [test] recipe = pbp.recipe.noserunner eggs = yourpackage1 yourpackage2 This will generate test script (bin/test), which will search and run tests only in packages, specified in eggs option. From garrickp at gmail.com Thu Jan 21 16:01:33 2010 From: garrickp at gmail.com (Falcolas) Date: Thu, 21 Jan 2010 13:01:33 -0800 (PST) Subject: Closures in metaclasses References: <2c126442-f0b8-4b38-82c7-7fb226b15b60@u41g2000yqe.googlegroups.com> <7ceaaaa1-45c2-4875-a091-971c14e2d41e@f12g2000yqn.googlegroups.com> <48807f29-3157-43b8-9b01-37ccf5f00616@a15g2000yqm.googlegroups.com> Message-ID: <054fc7de-6706-4a41-9136-17ce0b437bf4@b10g2000yqa.googlegroups.com> On Jan 21, 1:55?pm, Peter Otten <__pete... at web.de> wrote: > Falcolas wrote: > > I tried overriding __getattr__ and got an error at runtime (the > > You can either move __getattr__() into the metaclass or instantiate the > class. I prefer the latter. > > Both approaches in one example: > > >>> class Tag: > > ... ? ? class __metaclass__(type): > ... ? ? ? ? ? ? def __getattr__(self, name): return "<%s> via metaclass" % > name > ... ? ? def __getattr__(self, name): return "<%s> via class" % name > ...>>> Tag.yadda > > ' via metaclass'>>> tag = Tag() > >>> tag.yadda > > ' via class' Very nice, thanks! From arnodel at googlemail.com Thu Jan 21 16:06:30 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Thu, 21 Jan 2010 21:06:30 +0000 Subject: Closures in metaclasses References: <2c126442-f0b8-4b38-82c7-7fb226b15b60@u41g2000yqe.googlegroups.com> <7ceaaaa1-45c2-4875-a091-971c14e2d41e@f12g2000yqn.googlegroups.com> <48807f29-3157-43b8-9b01-37ccf5f00616@a15g2000yqm.googlegroups.com> Message-ID: Falcolas writes: > On Jan 21, 12:10?pm, Arnaud Delobelle wrote: [...] >> Or you could override __getattr__ >> >> -- >> Arnaud > > I tried overriding __getattr__ and got an error at runtime (the > instance did not have xyz key, etc), and the Tag dict is not > modifiable (granted, I tried Tag.__dict__[tag] = spam, not setattr()). Yes that's because __getattr__ works on instances, not on the class itself. To achieve this, you'd have to override the *metaclass*'s __getattr__ method. But are you sure that you want lots of staticmethods? Why not instanciate your class Tag instead and have normal methods to generate tags? This would have the added benefit that you could set some options in the __init__ method of Tag to customize its behaviour. If you have a class which only contains static methods, you'd be better off with a module which contains good old functions. -- Arnaud From showell30 at yahoo.com Thu Jan 21 16:08:02 2010 From: showell30 at yahoo.com (Steve Howell) Date: Thu, 21 Jan 2010 13:08:02 -0800 (PST) Subject: simple pub/sub References: <0af0eff2-50e2-44aa-81f1-b3d12cb265a7@a15g2000yqm.googlegroups.com> Message-ID: On Jan 21, 9:37?am, a... at pythoncraft.com (Aahz) wrote: > In article <0af0eff2-50e2-44aa-81f1-b3d12cb26... at a15g2000yqm.googlegroups.com>, > Steve Howell ? wrote: > > > > >Hi, I'm looking for ideas on building a simple architecture that > >allows a bunch of independent Python processes to exchange data using > >files and perform calculations. > > SQLite? The data is just a stream of instrument readings, so putting it into a relational database doesn't really buy me much. So far "multiprocessing" seems to be the way to go. From jenn.duerr at gmail.com Thu Jan 21 16:17:59 2010 From: jenn.duerr at gmail.com (noydb) Date: Thu, 21 Jan 2010 13:17:59 -0800 (PST) Subject: Rounding up to the next 100 Message-ID: <2fff27ae-179b-4c60-95e3-c3eb6dfe94cc@n31g2000vbt.googlegroups.com> If one has a floating number as a string, is there a spiffy way to round that string-number UP to the nearest 100? XstrNmbr = 3579.127893 -- would want to round that to 3600. Thanks for any help! From aahz at pythoncraft.com Thu Jan 21 16:20:41 2010 From: aahz at pythoncraft.com (Aahz) Date: 21 Jan 2010 13:20:41 -0800 Subject: counting lines of code References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> <87bpgodj95.fsf@benfinney.id.au> <1f166703-fdfc-4f87-b336-ef80e492635b@a32g2000yqm.googlegroups.com> <7e09df6a-cda1-480e-a971-8f8a70ac4118@b9g2000yqd.googlegroups.com> Message-ID: In article <7e09df6a-cda1-480e-a971-8f8a70ac4118 at b9g2000yqd.googlegroups.com>, Phlip wrote: >On Jan 20, 11:20=A0pm, Michele Simionato >wrote: >> >> pylint does too many things, I want something fast that just counts >> the lines and can be run on thousands of files at once. >> cloc seems fine, I have just tried on 2,000 files and it gives me a >> report in just a few seconds. > >In my experience with Python codebases that big... > >...how many of those lines are duplicated, and might merge together >into a better design? Um... do you have any clue who you followed up to? If you don't, Google is your friend. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From g.bogle at auckland.no.spam.ac.nz Thu Jan 21 16:24:03 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Fri, 22 Jan 2010 10:24:03 +1300 Subject: py2exe and pydocs. Downloads? References: Message-ID: Gabriel Genellina wrote: > You found a bug. Looks like it depends on the environment, or what > packages are installed, or something like that, because it worked on my > other PC but not here. > Please report it at http://bugs.python.org so it doesn't get forgotten. > Done From Michael.Coll-Barth at VerizonWireless.com Thu Jan 21 16:30:13 2010 From: Michael.Coll-Barth at VerizonWireless.com (Michael.Coll-Barth at VerizonWireless.com) Date: Thu, 21 Jan 2010 16:30:13 -0500 Subject: Rounding up to the next 100 In-Reply-To: References: Message-ID: > From: noydb > If one has a floating number as a string, is there a spiffy way to > round that string-number UP to the nearest 100? > > XstrNmbr = 3579.127893 -- would want to round that to 3600. What's wrong with round? round( XstrNmbr, -2 ) seems to do the trick. Or do you want to get rid of the decimal point as well? The information contained in this message and any attachment may be proprietary, confidential, and privileged or subject to the work product doctrine and thus protected from disclosure. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify me immediately by replying to this message and deleting it and all copies and backups thereof. Thank you. From peter.milliken at gmail.com Thu Jan 21 16:30:29 2010 From: peter.milliken at gmail.com (Peter) Date: Thu, 21 Jan 2010 13:30:29 -0800 (PST) Subject: Python and Tkinter Programming by John Grayson References: <0738732e-d8df-469d-be4a-3699f4fd5f64@j14g2000yqm.googlegroups.com> <801d251d-ae12-4ce6-95a1-58f594cd2f81@p8g2000yqb.googlegroups.com> <4B4F96D6.2070905@codebykevin.com> Message-ID: <326dc723-3909-4293-9bf6-48a367b9251c@e16g2000yqc.googlegroups.com> On Jan 15, 9:12?am, Kevin Walzer wrote: > On 1/14/10 3:39 PM, Peter wrote: > > > > > > > On Jan 15, 6:24 am, Mark Roseman ?wrote: > >> ? Peter ?wrote: > >>> Besides, the book is mainly about using Python with Tkinter - and > >>> Tkinter hasn't changed that much since 2000, so I believe it is just > >>> as relevant today as it was back then. > > >> I'd say that Tkinter has substantially changed - with the introduction > >> of the 'ttk' themed widgets. ?I cover these in my tutorial athttp://www.tkdocs.com > > >> Mark > > > But is the ttk themed widgets a "change" to Tkinter or an "addition/ > > improvement"? i.e. does the themed widgets invalidate any of the > > Tkinter stuff in Grayson's book? That certainly isn't my impression of > > the themed widgets, I was of the impression that the themed widgets > > just added some convenient widgets (such as Scale, Spinbox etc) which > > a user could find in other GUI frameworks, for example, Pmw > > supplemented the basic Tkinter widget set and offered some (essential > > IMO) widgets that were missing in Tkinter - such as the Notebook > > widget. > > It's both a change and an improvement. The themed widgets can be used in > conjunction with the traditional widgets, but in many cases it's > possible (and desirable) to update your entire application to use the > themed widgets. > > > Lets face it, if somebody wants to get up to speed on Python and GUI > > development then the book is still very, very relevant and necessary > > (IMO). The documentation for the themed widgets still leaves a lot to > > be desired from the perspective of somebody who wants to start using > > Python to create GUI applications. As Lord Eldritch reminded me in his > > post, the book even has a section on Pmw - which is what I use mainly > > for my GUI applications - because Tkinter was missing some vital > > widgets that are now available in the ttk themed set. > > > Personally I will start to incorporate some of the ttk themed widgets > > into my applications - but Pmw will remain the 'basis' for my GUI's as > > the entire framework (IMO) supports a class oriented approach that > > allows easy creation of extensible and reconfigurable (at run time) > > GUI interfaces. > > PMW is certainly a helpful addition to the Tkinter developer's toolbox, > but it also has limitations. It suffers from some of the same > limitations as Tk itself, i.e. it is rather dated in appearance, and > even it lacks some modern UI features such as a good multicolumn > listbox, treeview, etc. In addition to the ttk themed widgets, other, > more configurable pure Tk-based megawidget packages exist, such as > BWidgets and Tablelist, and there are Python wrappers for these at the > Tkinter wiki (http://tkinter.unpythonic.net/wiki/). > > > > > Ultimately Grayson does a good job of providing information and > > reference to toolkit(s) that allow a beginner to quickly get up to > > speed on producing a GUI using Python. It is purely up to the user > > afterwards as to whether they stick with Tkinter/Pmw (and now the ttk > > themed set) or venture into wxPython or Jython (as two examples of GUI > > 'systems' that provide 'better' facilities to a Python programmer). > > Another book I've found very helpful for learning Tkinter is Programming > Python by Mark Lutz--a lot of coverage there of GUI development. > > --Kevin > > -- > Kevin Walzer > Code by Kevinhttp://www.codebykevin.com Another possible consideration when choosing a GUI to learn Python - will you want to print from within your GUI application? Admittedly, I haven't looked to see whether this situation has changed or not for some years now, but a GUI based on Tkinter (whether it is ttk or Pmw or whatever) has no support for "printing". Other GUI frameworks, such as wxPython, Jython, PyQT etc have API's that allow printing of information from within your GUI application. Now if you go the Tk route, I am sure you can use the Win32 bindings to do printing - but I personally could never work them out :-) Maybe there is some reasonable documentation or example code for doing that these days - but there wasn't when I first approached the problem (which was back in the python 2.1 days - ancient history!). Printing using Windows API was a complete mystery to me. My first GUI application required information to be printed out - I wrote the whole thing using Pmw and Tkinter - only to find that I had no way of getting stuff to a printer (I was learning Python and Tkinter at the time :-)). I briefly explored "porting" my application to wxPython or Jython because both frameworks provided printer API functions - at that time, PyQT was commercial license only for a PC, so that wasn't an option. I found wxPython stunningly hard to understand from the documentation (I have since purchased wxPython in Action by Rappin and Dunn but have never found the time to read it). I had some problems porting the code to Jython (my code relied heavily on the pickle module and Jython had some severe bugs in that area at the time), so in the end I came up with a fairly micky-mouse system that placed information that required printing into .txt files and the GUI application started Microsoft Word using mail-merge templates and I got the information to the printer that way - not ideal! But it worked :-) So consider that in your decision process - maybe somebody can (kindly :-)) hop in here and say "that's easy, just look at xyz or read this book abc" to find out how to print from within an application using Tkinter. Regards Peter From phlip2005 at gmail.com Thu Jan 21 16:31:37 2010 From: phlip2005 at gmail.com (Phlip) Date: Thu, 21 Jan 2010 13:31:37 -0800 Subject: counting lines of code In-Reply-To: References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> <87bpgodj95.fsf@benfinney.id.au> <1f166703-fdfc-4f87-b336-ef80e492635b@a32g2000yqm.googlegroups.com> <7e09df6a-cda1-480e-a971-8f8a70ac4118@b9g2000yqd.googlegroups.com> Message-ID: <%I36n.5962$4p5.2277@newsfe22.iad> Aahz wrote: > In article <7e09df6a-cda1-480e-a971-8f8a70ac4118 at b9g2000yqd.googlegroups.com>, > Phlip wrote: >> On Jan 20, 11:20=A0pm, Michele Simionato >> wrote: >>> pylint does too many things, I want something fast that just counts >>> the lines and can be run on thousands of files at once. >>> cloc seems fine, I have just tried on 2,000 files and it gives me a >>> report in just a few seconds. >> In my experience with Python codebases that big... >> >> ...how many of those lines are duplicated, and might merge together >> into a better design? > > Um... do you have any clue who you followed up to? If you don't, Google > is your friend. Oh, sorry, did I have the wrong opinion? From arnodel at googlemail.com Thu Jan 21 16:32:12 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Thu, 21 Jan 2010 21:32:12 +0000 Subject: Rounding up to the next 100 References: <2fff27ae-179b-4c60-95e3-c3eb6dfe94cc@n31g2000vbt.googlegroups.com> Message-ID: noydb writes: > If one has a floating number as a string, is there a spiffy way to > round that string-number UP to the nearest 100? > > XstrNmbr = 3579.127893 -- would want to round that to 3600. > > Thanks for any help! >>> XstrNmbr = 3579.127893 >>> round(float(XstrNmbr), -2) 3600.0 >>> HTH -- Arnaud From grahn+nntp at snipabacken.se Thu Jan 21 16:33:18 2010 From: grahn+nntp at snipabacken.se (Jorgen Grahn) Date: 21 Jan 2010 21:33:18 GMT Subject: py2exe and pydocs. Downloads? References: Message-ID: On Thu, 2010-01-21, Gib Bogle wrote: > Gabriel Genellina wrote: > >> You found a bug. Looks like it depends on the environment, or what >> packages are installed, or something like that, because it worked on my >> other PC but not here. >> Please report it at http://bugs.python.org so it doesn't get forgotten. >> > > Done And for reference, it's , "pydoc error". /Jorgen -- // Jorgen Grahn O o . From arnodel at googlemail.com Thu Jan 21 16:36:06 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Thu, 21 Jan 2010 21:36:06 +0000 Subject: Rounding up to the next 100 References: <2fff27ae-179b-4c60-95e3-c3eb6dfe94cc@n31g2000vbt.googlegroups.com> Message-ID: Arnaud Delobelle writes: > >>> XstrNmbr = 3579.127893 I meant >>> XstrNmbr = "3579.127893" > >>> round(float(XstrNmbr), -2) > 3600.0 -- Arnaud From diesch at spamfence.net Thu Jan 21 16:51:33 2010 From: diesch at spamfence.net (Florian Diesch) Date: Thu, 21 Jan 2010 22:51:33 +0100 Subject: Rounding up to the next 100 References: <2fff27ae-179b-4c60-95e3-c3eb6dfe94cc@n31g2000vbt.googlegroups.com> Message-ID: <5sum27-m3d.ln1@mid.florian-diesch.de> noydb writes: > If one has a floating number as a string, is there a spiffy way to > round that string-number UP to the nearest 100? > > XstrNmbr = 3579.127893 -- would want to round that to 3600. math.ceil(3579.127893/100)*100 Florian -- GUIs programmieren mit Python und Glade: From Martin.Drautzburg at web.de Thu Jan 21 16:51:34 2010 From: Martin.Drautzburg at web.de (Martin Drautzburg) Date: Thu, 21 Jan 2010 22:51:34 +0100 Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: <5552238.LgDixP2R6m@beaureve.gmx.net> Thanks for all the answers. Let me summarize (1) I fail to see the relevance of ?>>> def move( direction ): ... ? print( "move " + str( direction ) ) ... ?>>> move( "up" ) move up not only in the context of my question. And I don't see an abuse of the language either. Maybe this could pass as a Zen Puzzle. (2) Using enum's was suggested. That is good to know, but again it is just a way to define constants in the caller's namespace. (3) Then somone suggested to tie the constants to the function itself, as in def move(direction): ? ? print "moving %s" % direction move.UP = 'up' move.DOWN = 'down' This is quite nice. Then again the "move." is just some object which allows attributes, and which only happens to have the same name as the function. Well in this case it IS the function, alright, but I could just as well have used a Class as in class m: pass m.UP = 'up' (4) Finally someone mentioned DSLs. I guess thats absolutely correct. This is what I am struggeling to achieve. I did a little googling ("how to write DSLs in python"), but haven't found anything appealing yet. Any pointers would be appreciated. (5) Here is something I came up with myself: def symbols(aDict): aDict["foo"] = "bar" def someFunction(aFoo): print aFoo symbols(locals()) someFunction (foo) #Eh voila: foo is magically defined prints: bar The call to symbols(locals()) is the "magic, magic" I supected would be required in my original posting. If someFunction was a member of a class, the symbols would be properly tied to that class (albeit not the individual function), but still good enough. I suppose I could wrap it in a decorator, which would also do the "unmagic". In any case, getting the context right seems to be the biggest problem. If I don't want to pollute my namespace, those symbols need to be defined in some context but undefined in others. AFAIK there are not really "blocks" in python and lexical scoping is present primarily in functions. From jenn.duerr at gmail.com Thu Jan 21 16:52:36 2010 From: jenn.duerr at gmail.com (noydb) Date: Thu, 21 Jan 2010 13:52:36 -0800 (PST) Subject: Rounding up to the next 100 References: Message-ID: <74d51f20-b693-4ab2-809f-6f9df6de08c0@r19g2000yqb.googlegroups.com> On Jan 21, 4:30?pm, Michael.Coll-Ba... at VerizonWireless.com wrote: > > From: noydb > > If one has a floating number as a string, is there a spiffy way to > > round that string-number UP to the nearest 100? > > > XstrNmbr = 3579.127893 -- would want to round that to 3600. > > What's wrong with round? ?round( XstrNmbr, -2 ) seems to do the trick. > Or do you want to get rid of the decimal point as well? > > The information contained in this message and any attachment may be > proprietary, confidential, and privileged or subject to the work > product doctrine and thus protected from disclosure. ?If the reader > of this message is not the intended recipient, or an employee or > agent responsible for delivering this message to the intended > recipient, you are hereby notified that any dissemination, > distribution or copying of this communication is strictly prohibited. > If you have received this communication in error, please notify me > immediately by replying to this message and deleting it and all > copies and backups thereof. ?Thank you. Thanks Arnaud! Michael - Nothing is wrong with round -- when I tried it initially, I was confused on the base -- seeing it from this example helped clear it up. From alfps at start.no Thu Jan 21 16:53:27 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 22:53:27 +0100 Subject: Rounding up to the next 100 In-Reply-To: References: Message-ID: * Michael.Coll-Barth at VerizonWireless.com: > > >> From: noydb > >> If one has a floating number as a string, is there a spiffy way to >> round that string-number UP to the nearest 100? >> >> XstrNmbr = 3579.127893 -- would want to round that to 3600. > > > What's wrong with round? round( XstrNmbr, -2 ) seems to do the trick. > Or do you want to get rid of the decimal point as well? Perhaps completely irrelevant, but just in passing, round() changed semantics from 2.x to 3.x, in 3.x always returning int when called with just 1 argument: >>> import sys >>> sys.version '2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]' >>> print round.__doc__ round(number[, ndigits]) -> floating point number Round a number to a given precision in decimal digits (default 0 digits). This always returns a floating point number. Precision may be negative. >>> _ >>> import sys >>> sys.version '3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)]' >>> print( round.__doc__ ) round(number[, ndigits]) -> number Round a number to a given precision in decimal digits (default 0 digits). This returns an int when called with one argument, otherwise the same type as the number. ndigits may be negative. >>> _ Might be useful to know regarding "get rid of the decimal point": in 3.x round(x) does that, in 2.x it doesn't. Cheers, - Alf From jenn.duerr at gmail.com Thu Jan 21 16:58:53 2010 From: jenn.duerr at gmail.com (noydb) Date: Thu, 21 Jan 2010 13:58:53 -0800 (PST) Subject: Rounding up to the next 100 References: <74d51f20-b693-4ab2-809f-6f9df6de08c0@r19g2000yqb.googlegroups.com> Message-ID: Sorry, although what I really need is the string-number rounded UP every time. So if the number is 3890.32, it needs to go to 3900; if the number is 3811.345, it needs to go to 3900 also. So, Florian's answer works. From Michael.Coll-Barth at VerizonWireless.com Thu Jan 21 17:03:50 2010 From: Michael.Coll-Barth at VerizonWireless.com (Michael.Coll-Barth at VerizonWireless.com) Date: Thu, 21 Jan 2010 17:03:50 -0500 Subject: Rounding up to the next 100 In-Reply-To: References: <2fff27ae-179b-4c60-95e3-c3eb6dfe94cc@n31g2000vbt.googlegroups.com> Message-ID: > From: Arnaud Delobelle > > > > >>> XstrNmbr = 3579.127893 > > I meant > >>> XstrNmbr = "3579.127893" > > > >>> round(float(XstrNmbr), -2) > > 3600.0 Ah, then you will need to cast it first. >>> XstrNmbr = '3579.127893' >>> round(float(XstrNmbr) ,-2) 3600.0 The information contained in this message and any attachment may be proprietary, confidential, and privileged or subject to the work product doctrine and thus protected from disclosure. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify me immediately by replying to this message and deleting it and all copies and backups thereof. Thank you. From g.bogle at auckland.no.spam.ac.nz Thu Jan 21 17:10:46 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Fri, 22 Jan 2010 11:10:46 +1300 Subject: py2exe and pydocs. Downloads? References: Message-ID: Gabriel Genellina wrote: > En Wed, 20 Jan 2010 21:22:19 -0300, Gib Bogle > escribi?: > >> Gabriel Genellina wrote: >> >>> c:\temp>python -m pydoc sys >>> Help on built-in module sys: >>> [...same info...] >> >> When I do this I get: >> >> No module named tempfile > > You found a bug. Looks like it depends on the environment, or what > packages are installed, or something like that, because it worked on my > other PC but not here. > Please report it at http://bugs.python.org so it doesn't get forgotten. > The bug checkers are not able to reproduce the error. What is your other PC running? From alfps at start.no Thu Jan 21 17:20:08 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 23:20:08 +0100 Subject: Rounding up to the next 100 In-Reply-To: References: <74d51f20-b693-4ab2-809f-6f9df6de08c0@r19g2000yqb.googlegroups.com> Message-ID: * noydb: > Sorry, although what I really need is the string-number rounded UP > every time. So if the number is 3890.32, it needs to go to 3900; if > the number is 3811.345, it needs to go to 3900 also. > > So, Florian's answer works. You might also consider -100*(-3579.127893//100) :-) Which avoids the math.ceil but assumes the number is positive (or zero). Cheers & hth., - Alf PS: Note that this trick doesn't work with most other common languages that I'm familiar with, since the round towards zero instead of down to minus infinity, but Python has more clean semantics in this regard. From aharrisreid at googlemail.com Thu Jan 21 17:34:00 2010 From: aharrisreid at googlemail.com (Alan Harris-Reid) Date: Thu, 21 Jan 2010 22:34:00 +0000 Subject: Dynamic HTML controls In-Reply-To: References: Message-ID: <4B58D658.3080609@googlemail.com> Aahz wrote: > In article , > Alan Harris-Reid wrote: > >> Does anyone know where I can find any decent dynamically-constructed >> HTML control classes (dropdown list, table, input field, checkbox, etc.) >> written in Python. For example, for a HTML table I would like something >> like... >> > > You might look at Quixote: > http://quixote.ca/ > Thanks for that Aahz - I'll check it out. Regards, Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From alfps at start.no Thu Jan 21 17:49:59 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 23:49:59 +0100 Subject: Symbols as parameters? In-Reply-To: <5552238.LgDixP2R6m@beaureve.gmx.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> Message-ID: * Martin Drautzburg: > Thanks for all the answers. Let me summarize > > (1) I fail to see the relevance of > >>> def move( direction ): > ... print( "move " + str( direction ) ) > ... > >>> move( "up" ) > move up > > not only in the context of my question. And I don't see an abuse of the > language either. Maybe this could pass as a Zen Puzzle. Oh. You focused on the wrong details, then. Sorry for not providing a textual explanation, but I thought anyone not understanding it would just ask (that's normal in other Usenet groups that I'm familiar with). So, explanation... Here's your problem description, from the start of the thread: What I am really looking for is a way - to be able to call move(up) - having the "up" symbol only in the context of the function call So it should look something like this ... magic, magic ... move(up) ... unmagic, unmagic ... print up This should complain that "up" is not defined during the "print" call, but not when move() is called. And of course there should be as little magic as possible. The code corresponding to your first "magic, magic ..." is class using_directions: # Whatever statement (import?) defines symbolic directions goes here Then comes your -- note the added indentation move( up ) Your "... unmagic, unmagic" is simply to go back to the previous indent level. So your example's following print up at this point complaints that "up" is not defined, as you require (no general namespace pollution, although there is a slight but controllable pollution, namely the class name). Not that I recommend this technique. ;-) But it does exactly what you asked, quoted above. The best technique in my view is what you came up with yourself in the article I'm responding to (but snipped by me), namely a class with the constants and the function. It doesn't do what you required originally. But it's much better! :-) Cheers & hth., - Alf From Martin.Drautzburg at web.de Thu Jan 21 17:57:37 2010 From: Martin.Drautzburg at web.de (Martin Drautzburg) Date: Thu, 21 Jan 2010 23:57:37 +0100 Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> Message-ID: <7174001.cQaEHfC5Mm@beaureve.gmx.net> Here is a complete expample using a decorator, still a bit noisy def move(aDirection): print "moving " + aDirection #Here comes the decorator def scope(aDict): def save(locals): """Set symbols in locals and remember their original state""" setSymbols={} unsetSymbols=[] for i in ("up", "down", "left", "right"): if locals.has_key(i): setSymbols[i] = locals[i] else: unsetSymbols.append(i) # define the new symbols locals[i] = i return setSymbols, unsetSymbols def restore (locals, set, unset): """restore locals from set and unset""" for i in set.keys(): locals[i] = set[i] for i in unset: del(locals[i]) def callFunc(f): """Main decorator""" set, unset = save(aDict) f() restore(aDict, set, unset) return callFunc # -------------------------------------- # using it # -------------------------------------- # a variable defined in the outer scope up="outerScopeUp" # magic, magic (still too noisy for my taste) @scope (locals()) def _(): move(up) move(down) move(left) move(right) #verify the the outer scope variable hasn't changed print "in the outer scope up is still:", up print print "this should fail:" down # -------------------------------------- # Output # -------------------------------------- moving up moving down moving left moving right in the outer scope up is still: outerScopeUp this should fail: Traceback (most recent call last): File "", line 50, in NameError: name 'down' is not defined From deets at nospam.web.de Thu Jan 21 18:16:31 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Fri, 22 Jan 2010 00:16:31 +0100 Subject: Symbols as parameters? In-Reply-To: <5552238.LgDixP2R6m@beaureve.gmx.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> Message-ID: <7rs5ihFjqjU1@mid.uni-berlin.de> Am 21.01.10 22:51, schrieb Martin Drautzburg: > Thanks for all the answers. Let me summarize > > (1) I fail to see the relevance of > >>> def move( direction ): > ... print( "move " + str( direction ) ) > ... > >>> move( "up" ) > move up > > not only in the context of my question. And I don't see an abuse of the > language either. Maybe this could pass as a Zen Puzzle. > > (2) Using enum's was suggested. That is good to know, but again it is > just a way to define constants in the caller's namespace. > > (3) Then somone suggested to tie the constants to the function itself, > as in > def move(direction): > print "moving %s" % direction > > move.UP = 'up' > move.DOWN = 'down' > > This is quite nice. Then again the "move." is just some object which > allows attributes, and which only happens to have the same name as the > function. Well in this case it IS the function, alright, but I could > just as well have used a Class as in > > class m: pass > m.UP = 'up' > > > (4) Finally someone mentioned DSLs. I guess thats absolutely correct. > This is what I am struggeling to achieve. I did a little googling ("how > to write DSLs in python"), but haven't found anything appealing yet. > Any pointers would be appreciated. > > (5) Here is something I came up with myself: > > def symbols(aDict): > aDict["foo"] = "bar" > > def someFunction(aFoo): > print aFoo > > symbols(locals()) > someFunction (foo) #Eh voila: foo is magically defined http://docs.python.org/library/functions.html#locals If it works, it's more an accident than anything else, and can go away without prior notice. Diez From pavlovevidence at gmail.com Thu Jan 21 18:19:58 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Thu, 21 Jan 2010 15:19:58 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> Message-ID: <89e6f34c-1a7e-4c2e-88cb-ee0397678948@k17g2000yqh.googlegroups.com> On Jan 21, 1:51?pm, Martin Drautzburg wrote: > Thanks for all the answers. Let me summarize [snip] > (2) Using enum's was suggested. That is good to know, but again it is > just a way to define constants in the caller's namespace. It'll at least corral the symbols you want. [snip] > (4) Finally someone mentioned DSLs. I guess thats absolutely correct. > This is what I am struggeling to achieve. I see. Well, Python is a poor choice for defining an internal DSL (i.e., DSL using the general language's syntax), because it's (deliberately) rigid in both grammar and semantics. Soon as you want to do something a little different you're out of luck. Python is somewhat better suited for external DSLs. > I did a little googling ("how > to write DSLs in python"), but haven't found anything appealing yet. > Any pointers would be appreciated. Paul McGuire should be by to recommend PyParsing shortly. It's not really hard to write a simple DSL if you know how. It is pretty hard to learn how, though. Tools like PyParsing help a lot. > (5) Here is something I came up with myself: > > def symbols(aDict): > ? ? aDict["foo"] = "bar" > > def someFunction(aFoo): > ? ? print aFoo > > symbols(locals()) > someFunction (foo) #Eh voila: foo is magically defined > > prints: bar > > The call to symbols(locals()) is the "magic, magic" I supected would be > required in my original posting. If someFunction was a member of a > class, the symbols would be properly tied to that class (albeit not the > individual function), but still good enough. I suppose I could wrap it > in a decorator, which would also do the "unmagic". > > In any case, getting the context right seems to be the biggest problem. > If I don't want to pollute my namespace, those symbols need to be > defined in some context but undefined in others. AFAIK there are not > really "blocks" in python and lexical scoping is present primarily in > functions. That's pretty much the issue. BTW, I'm sorry that the thread got a little flamey there. Carl Banks From robert.kern at gmail.com Thu Jan 21 18:52:24 2010 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 21 Jan 2010 17:52:24 -0600 Subject: counting lines of code In-Reply-To: <%I36n.5962$4p5.2277@newsfe22.iad> References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> <87bpgodj95.fsf@benfinney.id.au> <1f166703-fdfc-4f87-b336-ef80e492635b@a32g2000yqm.googlegroups.com> <7e09df6a-cda1-480e-a971-8f8a70ac4118@b9g2000yqd.googlegroups.com> <%I36n.5962$4p5.2277@newsfe22.iad> Message-ID: On 2010-01-21 15:31 , Phlip wrote: > Aahz wrote: >> In article >> <7e09df6a-cda1-480e-a971-8f8a70ac4118 at b9g2000yqd.googlegroups.com>, >> Phlip wrote: >>> On Jan 20, 11:20=A0pm, Michele Simionato >>> wrote: >>>> pylint does too many things, I want something fast that just counts >>>> the lines and can be run on thousands of files at once. >>>> cloc seems fine, I have just tried on 2,000 files and it gives me a >>>> report in just a few seconds. >>> In my experience with Python codebases that big... >>> >>> ...how many of those lines are duplicated, and might merge together >>> into a better design? >> >> Um... do you have any clue who you followed up to? If you don't, Google >> is your friend. > > Oh, sorry, did I have the wrong opinion? You had a condescending attitude. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ben+python at benfinney.id.au Thu Jan 21 20:06:13 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 22 Jan 2010 12:06:13 +1100 Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <89e6f34c-1a7e-4c2e-88cb-ee0397678948@k17g2000yqh.googlegroups.com> Message-ID: <87d413c5iy.fsf@benfinney.id.au> Carl Banks writes: > On Jan 21, 1:51?pm, Martin Drautzburg > wrote: > > (2) Using enum's was suggested. That is good to know, but again it > > is just a way to define constants in the caller's namespace. > > It'll at least corral the symbols you want. It also satisfies the strongly-expressed demand you had for these constants to *not* be strings. It keeps the semantic meaning of these constants very distinct, and greatly reduces the possibility of an object accidentally having the same value as one of these constants. -- \ ?Roll dice!? ?Why?? ?Shut up! I don't need your fucking | `\ *input*, I need you to roll dice!? ?Luke Crane, demonstrating | _o__) his refined approach to play testing, 2009 | Ben Finney From ethan at stoneleaf.us Thu Jan 21 20:18:23 2010 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 21 Jan 2010 17:18:23 -0800 Subject: Python and Tkinter Programming by John Grayson In-Reply-To: <326dc723-3909-4293-9bf6-48a367b9251c@e16g2000yqc.googlegroups.com> References: <0738732e-d8df-469d-be4a-3699f4fd5f64@j14g2000yqm.googlegroups.com> <801d251d-ae12-4ce6-95a1-58f594cd2f81@p8g2000yqb.googlegroups.com> <4B4F96D6.2070905@codebykevin.com> <326dc723-3909-4293-9bf6-48a367b9251c@e16g2000yqc.googlegroups.com> Message-ID: <4B58FCDF.4000402@stoneleaf.us> Peter wrote: > On Jan 15, 9:12 am, Kevin Walzer wrote: >>> On Jan 15, 6:24 am, Mark Roseman wrote: >>>> Peter wrote: >>>>> Besides, the book is mainly about using Python with Tkinter - and >>>>> Tkinter hasn't changed that much since 2000, so I believe it is just >>>>> as relevant today as it was back then. >>>> I'd say that Tkinter has substantially changed - with the introduction >>>> of the 'ttk' themed widgets. I cover these in my tutorial athttp://www.tkdocs.com >> Another book I've found very helpful for learning Tkinter is Programming >> Python by Mark Lutz--a lot of coverage there of GUI development. >> > > Another possible consideration when choosing a GUI to learn Python - > will you want to print from within your GUI application? Excellent point. Many thanks to all who responded, especially for the reminder of the gui sections in Programming Python (forgot I had that book!). I'll start by going over that again, and we'll see how confident I feel afterwards. ;) ~Ethan~ From tekion at gmail.com Thu Jan 21 20:48:36 2010 From: tekion at gmail.com (tekion) Date: Thu, 21 Jan 2010 17:48:36 -0800 (PST) Subject: deriving MySQLdb class Message-ID: All, I am trying to write a class which inherits from MySQLdb class. Below is code snippet: import MySQLdb import sys class msql_connect(MySQLdb): def __init__(self): self.host = "hostname" self.user = "user" self.password = "passoword" self.database = "database name" I am running into below error: class msql_connect(MySQLdb): TypeError: Error when calling the metaclass bases module.__init__() takes at most 2 arguments (3 given) Does any one have an idea why? Thanks. From gburdell1 at gmail.com Thu Jan 21 20:56:00 2010 From: gburdell1 at gmail.com (gburdell1 at gmail.com) Date: Thu, 21 Jan 2010 17:56:00 -0800 (PST) Subject: Default return values for out-of-bounds list item Message-ID: Is there a built-in method in python that lets you specify a "default" value that will be returned whenever you try to access a list item that is out of bounds? Basically, it would be a function like this: def item(x,index,default): try: return x[index] except IndexError: return default So if a=[0,1,2,3], then item(a,0,44)=0, item(a,1,44)=1, and item(a, 1000,44)=44, item(a,-1000,44)=44 What I want to know is whether there is a built-in method or notation for this. What if, for example, we could do something like a [1000,44] ? From steve at REMOVE-THIS-cybersource.com.au Thu Jan 21 21:20:44 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 02:20:44 GMT Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> Message-ID: <0368f809$0$1357$c3e8da3@news.astraweb.com> On Thu, 21 Jan 2010 22:51:34 +0100, Martin Drautzburg wrote: > Thanks for all the answers. Let me summarize > > (1) I fail to see the relevance of > ?>>> def move( direction ): > ... ? print( "move " + str( direction ) ) ... > ?>>> move( "up" ) > move up I'm glad it's not just me then. > not only in the context of my question. And I don't see an abuse of the > language either. Maybe this could pass as a Zen Puzzle. > > (2) Using enum's was suggested. That is good to know, but again it is > just a way to define constants in the caller's namespace. I think this really is the correct solution for your problem. In Python, the standard place to have such public constants is at the module level, not the function or class. I think you're worrying unnecessarily about "namespace pollution" -- the module namespace is *exactly* the right place for them. If two functions both need UP, DOWN, etc symbols, then either: (1) they can just use the same symbols; or (2) if they can't, then they don't belong in the same module. An example from the standard library: the re module defines constants I, L, M, etc. representing flags that are passed to the re.compile. They are implemented as integers so they can easily be combined with &, but another implementation might use symbols. You will notice that they're not limited to the re.compile function itself. The caller may very well want to do something like this: # Get some flags for compile: flags = re.I & re.M # ... # much later on # x = re.compile(s, flags) You would force them to do this: # Get some flags for compile: flags = re.compile.I & re.compile.M # ... # much later on # x = re.compile(s, flags) which is, in my opinion, a needless level of indirection and possibly in violation of Demeter's Law. > (3) Then somone suggested to tie the constants to the function itself, > as in > def move(direction): > ? ? print "moving %s" % direction > > move.UP = 'up' > move.DOWN = 'down' > > This is quite nice. I would call it a horrible, horrible, horrible code smell. A stench in fact. In my opinion, such attributes tied to the function should be treated as internal to the function, and not the public interface. I wouldn't go quite so far as to say they should be treated as private, but having the caller use them should be rare and unusual. > Then again the "move." is just some object which > allows attributes, and which only happens to have the same name as the > function. Well in this case it IS the function, alright, but I could > just as well have used a Class as in > > class m: pass > m.UP = 'up' Either way, when you go to *use* the direction, you're still passing a string. There's no difference between: move(m.UP) and just move("up") Furthermore, the extra layer of indirection with the m.* doesn't give you anything useful. Think about using this: # choose a direction at random direction = random.choice([m.UP, m.DOWN, m.LEFT, m.RIGHT]) move(direction) What benefit is the extra layer of indirection? It just adds noise to the code. Surely this is better? UP, DOWN, LEFT, RIGHT = "up down left right".strip() direction = random.choice([UP, DOWN, LEFT, RIGHT]) move(direction) That's much clearer. In my opinion, if you want to prohibit users from passing a string (or integer) equal to your constants, so that move('up') does not work (in other words, they are forced to use the constants you provide) then Ben Finney's enum solution is probably the correct way to do it. But if you don't care, then the simplest solution is to define the constants you care about in the module, using either strings or ints, and then let the caller choose between using your named constants or not: move(UP) move('up') > (4) Finally someone mentioned DSLs. I guess thats absolutely correct. > This is what I am struggeling to achieve. I did a little googling ("how > to write DSLs in python"), but haven't found anything appealing yet. Any > pointers would be appreciated. That truly is using a bulldozer to crack a peanut. > (5) Here is something I came up with myself: > > def symbols(aDict): > aDict["foo"] = "bar" > > def someFunction(aFoo): > print aFoo > > symbols(locals()) > someFunction (foo) #Eh voila: foo is magically defined > prints: bar > > The call to symbols(locals()) is the "magic, magic" I supected would be > required in my original posting. If someFunction was a member of a > class, the symbols would be properly tied to that class (albeit not the > individual function), but still good enough. I disagree about it being "proper" to tie such public symbols to the class. But in any case, what you're trying to do is not supported by Python. If it works, that's a happy accident. "The contents of this dictionary should not be modified; changes may not affect the values of local and free variables used by the interpreter." http://docs.python.org/library/functions.html#locals > I suppose I could wrap it > in a decorator, which would also do the "unmagic". > > In any case, getting the context right seems to be the biggest problem. > If I don't want to pollute my namespace, It's not pollution. The module namespace is the right place for such public constants. -- Steven From steve at REMOVE-THIS-cybersource.com.au Thu Jan 21 21:21:58 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 02:21:58 GMT Subject: Create object name from string value? References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> Message-ID: <0368f853$0$1357$c3e8da3@news.astraweb.com> On Thu, 21 Jan 2010 06:23:17 -0800, Gnarlodious wrote: >> The right way to solve this problem is with a dictionary: >> >> for name in ["object1", "object2", "object3"]: >> ? ? d = {name: classname()} >> ? ? print d[name] > > This works! However I end up saying: > > d['Server'].Config.BaseURL > > to get the data, when I should be saying: > > Server.Config.BaseURL There are three moments in time that the programmer needs to care about: when you write the code, when you compile the code, and when you run the code. In this case, the two relevant ones are write-time and run-time. If you expect to be able to write Server.Config.BaseURL then that means that you MUST know the name "Server" at write-time. (Otherwise, how do you know to write "Server", instead of "MyServer" or "ServerSix"?) That means that the *name* Server isn't specified at runtime, but at write-time. Since you already know the name, then the right solution is to do something like: Server = classname() and then refer to Server, just like you would do: x = 45 print x+1 or similar. But if you are getting the name "Server" from a config file at runtime, say something like this: name=Server class=classname flag=True then you can't write: Server.Config.BaseURL in your source code, because you don't know if it will be called "Server", or "MyServer" or "Server42", or "WebServe", or "Fred". In general, you don't even know how many servers there will be: there could be one, or none, or fifty. So you need a level of indirection, hence the dictionary. -- Steven From steve at REMOVE-THIS-cybersource.com.au Thu Jan 21 21:28:23 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 02:28:23 GMT Subject: Default return values for out-of-bounds list item References: Message-ID: <0368f9d5$0$1357$c3e8da3@news.astraweb.com> On Thu, 21 Jan 2010 17:56:00 -0800, gburdell1 at gmail.com wrote: > Is there a built-in method in python that lets you specify a "default" > value that will be returned whenever you try to access a list item that > is out of bounds? No. > Basically, it would be a function like this: > > def item(x,index,default): > try: > return x[index] > except IndexError: > return default That's probably the best way to do it. > So if a=[0,1,2,3], then item(a,0,44)=0, item(a,1,44)=1, and item(a, > 1000,44)=44, item(a,-1000,44)=44 > > What I want to know is whether there is a built-in method or notation > for this. What if, for example, we could do something like a [1000,44] ? You can use slicing instead: >>> a=[0,1,2,3] >>> a[2:3] [2] >>> a[100:101] [] and then detect the empty list and use default: def item(x, index, default): a = x[index:index+1] return a[0] if a else default -- Steven From python at mrabarnett.plus.com Thu Jan 21 21:41:05 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 22 Jan 2010 02:41:05 +0000 Subject: Default return values for out-of-bounds list item In-Reply-To: References: Message-ID: <4B591041.8080905@mrabarnett.plus.com> gburdell1 at gmail.com wrote: > Is there a built-in method in python that lets you specify a "default" > value that will be returned whenever you try to access a list item > that is out of bounds? Basically, it would be a function like this: > > def item(x,index,default): > try: > return x[index] > except IndexError: > return default > > So if a=[0,1,2,3], then item(a,0,44)=0, item(a,1,44)=1, and item(a, > 1000,44)=44, item(a,-1000,44)=44 > > What I want to know is whether there is a built-in method or notation > for this. > There's no built-in method or notation for that. > What if, for example, we could do something like a [1000,44] ? That's actually using a tuple (1000, 44) as the index. Such tuples can be used as keys in a dict and are used in numpy for indexing multi-dimensional arrays, so it's definitely a bad idea. If such a method were added to the 'list' class then the best option (ie most consistent with other classes) would be get(index, default=None). From python at mrabarnett.plus.com Thu Jan 21 21:42:55 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 22 Jan 2010 02:42:55 +0000 Subject: Symbols as parameters? In-Reply-To: <0368f809$0$1357$c3e8da3@news.astraweb.com> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <0368f809$0$1357$c3e8da3@news.astraweb.com> Message-ID: <4B5910AF.3070104@mrabarnett.plus.com> Steven D'Aprano wrote: [snip] > An example from the standard library: the re module defines constants I, > L, M, etc. representing flags that are passed to the re.compile. They are > implemented as integers so they can easily be combined with &, but > another implementation might use symbols. You will notice that they're > not limited to the re.compile function itself. > > The caller may very well want to do something like this: > > # Get some flags for compile: > flags = re.I & re.M [snip] That should be: flags = re.I | re.M of course. :-) From steve at REMOVE-THIS-cybersource.com.au Thu Jan 21 22:08:57 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 03:08:57 GMT Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <0368f809$0$1357$c3e8da3@news.astraweb.com> Message-ID: <03690356$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 02:42:55 +0000, MRAB wrote: > Steven D'Aprano wrote: > [snip] >> An example from the standard library: the re module defines constants >> I, L, M, etc. representing flags that are passed to the re.compile. >> They are implemented as integers so they can easily be combined with &, >> but another implementation might use symbols. You will notice that >> they're not limited to the re.compile function itself. >> >> The caller may very well want to do something like this: >> >> # Get some flags for compile: >> flags = re.I & re.M > [snip] > > That should be: > > flags = re.I | re.M > > of course. :-) Dammit! No wonder my regexes never work as I expect!!! Thanks for the correction. -- Steven From python.list at tim.thechases.com Thu Jan 21 22:38:33 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 21 Jan 2010 21:38:33 -0600 Subject: Default return values for out-of-bounds list item In-Reply-To: <4B591041.8080905@mrabarnett.plus.com> References: <4B591041.8080905@mrabarnett.plus.com> Message-ID: <4B591DB9.7090100@tim.thechases.com> MRAB wrote: > gburdell1 at gmail.com wrote: >> Is there a built-in method in python that lets you specify a "default" >> value that will be returned whenever you try to access a list item >> that is out of bounds? Basically, it would be a function like this: >> >> def item(x,index,default): >> try: >> return x[index] >> except IndexError: >> return default >> >> So if a=[0,1,2,3], then item(a,0,44)=0, item(a,1,44)=1, and item(a, >> 1000,44)=44, item(a,-1000,44)=44 >> >> What I want to know is whether there is a built-in method or notation >> for this. > > > There's no built-in method or notation for that. > > > What if, for example, we could do something like a [1000,44] ? > > That's actually using a tuple (1000, 44) as the index. Such tuples can > be used as keys in a dict and are used in numpy for indexing > multi-dimensional arrays, so it's definitely a bad idea. > > If such a method were added to the 'list' class then the best option (ie > most consistent with other classes) would be get(index, default=None). But there's nothing stopping you from creating your own subclass of "list" that allows for defaults: class DefaultList(list): def __init__(self, default, *args, **kwargs): list.__init__(self, *args, **kwargs) self.default = default def __getitem__(self, index): try: return list.__getitem__(self, index) except IndexError: return self.default ml = DefaultList(42, [1,2,3,4]) for i in range(-5,5): print i, ml[i] (yeah, there's likely a "proper" way of using super() to do the above instead of "list.____" but the above worked for a quick 3-minute example composed in "ed"). The output of the above is -5 42 -4 1 -3 2 -2 3 -1 4 0 1 1 2 2 3 3 4 4 42 One might want to add other methods for __add__/__radd__ so that a DefaultList is returned instead of a "list", but this is python...the sky's the limit. -tkc From half.italian at gmail.com Thu Jan 21 22:42:35 2010 From: half.italian at gmail.com (Sean DiZazzo) Date: Thu, 21 Jan 2010 19:42:35 -0800 (PST) Subject: deriving MySQLdb class References: Message-ID: <9b9755ad-c3ca-42eb-b95c-0e41c5052cd7@p24g2000yqm.googlegroups.com> On Jan 21, 5:48?pm, tekion wrote: > All, > I am trying to write a class which inherits from MySQLdb class. ?Below > is code snippet: > import MySQLdb > import sys > > class ?msql_connect(MySQLdb): > ? ? def __init__(self): > ? ? ? ? self.host ? ? = ?"hostname" > ? ? ? ? self.user ? ? = "user" > ? ? ? ? self.password ?= "passoword" > ? ? ? ? self.database = "database name" > > I am running into below error: > ?class ?msql_connect(MySQLdb): > TypeError: Error when calling the metaclass bases > ? ? module.__init__() takes at most 2 arguments (3 given) > > Does any one have an idea why? ?Thanks. MySQLdb is the name of the module, not the class you want to subclass. But MySQLdb.connect() is not the class either...it's a factory function that returns instances of the class you actually want to subclass...connections.Connection(). The below works for me. from MySQLdb import connections import sys class mysql_connect(connections.Connection): def __init__(self): self.host = "host" self.user = "user" self.password = "password" self.database = "database" connections.Connection.__init__(self, host=self.host, user=self.user, passwd=self.password, db=self.database) p = mysql_connect() ~Sean From tekion at gmail.com Thu Jan 21 23:00:48 2010 From: tekion at gmail.com (tekion) Date: Thu, 21 Jan 2010 20:00:48 -0800 (PST) Subject: deriving MySQLdb class References: <9b9755ad-c3ca-42eb-b95c-0e41c5052cd7@p24g2000yqm.googlegroups.com> Message-ID: Sean, Thanks. This is useful. For future reference, how do I know what class is in MySQLdb module? From half.italian at gmail.com Thu Jan 21 23:13:16 2010 From: half.italian at gmail.com (Sean DiZazzo) Date: Thu, 21 Jan 2010 20:13:16 -0800 (PST) Subject: deriving MySQLdb class References: <9b9755ad-c3ca-42eb-b95c-0e41c5052cd7@p24g2000yqm.googlegroups.com> Message-ID: <10c80803-4934-4857-80e9-ba63648d9be4@14g2000yqp.googlegroups.com> On Jan 21, 8:00?pm, tekion wrote: > Sean, > Thanks. ?This is useful. ?For future reference, how do I know what > class is in MySQLdb module? You have to explore. ;) I found the MySQLdb module, and looked inside the __init__.py. Then looked for "connect" and followed the trail. From tekion at gmail.com Thu Jan 21 23:17:52 2010 From: tekion at gmail.com (tekion) Date: Thu, 21 Jan 2010 20:17:52 -0800 (PST) Subject: deriving MySQLdb class References: <9b9755ad-c3ca-42eb-b95c-0e41c5052cd7@p24g2000yqm.googlegroups.com> <10c80803-4934-4857-80e9-ba63648d9be4@14g2000yqp.googlegroups.com> Message-ID: <9f413105-5572-4bc7-93f4-ac7f252e3174@y23g2000yqm.googlegroups.com> Sean, I did a little investigation, there are other classes besides Connection. So, could I only set up a derived class from Connection and still be able to use the connection to query database and retrieve data? From half.italian at gmail.com Thu Jan 21 23:48:15 2010 From: half.italian at gmail.com (Sean DiZazzo) Date: Thu, 21 Jan 2010 20:48:15 -0800 (PST) Subject: deriving MySQLdb class References: <9b9755ad-c3ca-42eb-b95c-0e41c5052cd7@p24g2000yqm.googlegroups.com> <10c80803-4934-4857-80e9-ba63648d9be4@14g2000yqp.googlegroups.com> <9f413105-5572-4bc7-93f4-ac7f252e3174@y23g2000yqm.googlegroups.com> Message-ID: On Jan 21, 8:17?pm, tekion wrote: > Sean, > I did a little investigation, there are other classes besides > Connection. So, could I only set up a derived class from Connection > and still be able to use the connection to query database and retrieve > data? Im not sure I understand you completely... In theory, you could use the C API directly and subclass _mysql.connection to get at the database. But I think the point of MySQLdb is that they've done all the hard work. Why not use it? I think the other stuff in the module is in support of the Connection () class. ie. You cant get a cursor unless you already have a connection. From michele.simionato at gmail.com Fri Jan 22 00:00:30 2010 From: michele.simionato at gmail.com (Michele Simionato) Date: Thu, 21 Jan 2010 21:00:30 -0800 (PST) Subject: counting lines of code References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> <87bpgodj95.fsf@benfinney.id.au> <1f166703-fdfc-4f87-b336-ef80e492635b@a32g2000yqm.googlegroups.com> <7e09df6a-cda1-480e-a971-8f8a70ac4118@b9g2000yqd.googlegroups.com> Message-ID: <980775cc-9101-4673-af03-fc17914d63f8@c4g2000yqa.googlegroups.com> On Jan 21, 9:24?pm, Phlip wrote: > On Jan 20, 11:20?pm, Michele Simionato > wrote: > > > pylint does too many things, I want something fast that just counts > > the lines and can be run on thousands of files at once. > > cloc seems fine, I have just tried on 2,000 files and it gives me a > > report in just a few seconds. > > In my experience with Python codebases that big... > > ...how many of those lines are duplicated, and might merge together > into a better design? > > The LOC would go down, too. Actually 2,000 files is a very small portion of our code base, the one I am working on now. I have spent the last couple of months on a big refactoring project (which is still only at the beginning) and I wanted to count the difference between the lines of code before the refactoring and after the refactoring. I guess the new code is less than half than the old one. There was no cut and paste in the old code but a lot of subtle duplication, i.e. a code that could be unified in common libraries, but only after a lot of grunt work. The core parts were written 10 years ago, with a wrong architecture starting from the beginning, and then things started growing and growing on that monster. Just for fun I have run cloc on our trunk: Language files blank comment code scale 3rd gen. equiv -------------------------------------------------------------------------------- C++ 1528 67150 48251 304365 x 1.51 = 459591.15 XML 560 2769 2517 223223 x 1.90 = 424123.70 ASP 731 40136 4630 216713 x 1.29 = 279559.77 Python 2027 38825 47261 179532 x 4.20 = 754034.40 C/C++ Header 2150 51352 72619 141356 x 1.00 = 141356.00 Javascript 153 26196 9819 115311 x 1.48 = 170660.28 C 332 14147 12871 97918 x 0.77 = 75396.86 SQL 426 16432 4214 93598 x 2.29 = 214339.42 CSS 110 1493 1013 23087 x 1.00 = 23087.00 C# 83 3301 1990 19827 x 1.36 = 26964.72 Visual Basic 35 4363 5927 14633 x 2.76 = 40387.08 make 259 1617 650 8339 x 2.50 = 20847.50 Bourne Shell 52 598 1282 6557 x 3.81 = 24982.17 m4 28 611 627 5612 x 1.00 = 5612.00 IDL 23 560 0 3895 x 3.80 = 14801.00 HTML 33 354 76 3834 x 1.90 = 7284.60 MSBuild scripts 3 2 7 3419 x 1.90 = 6496.10 Lisp 33 562 648 2695 x 1.25 = 3368.75 Ruby 13 272 97 1141 x 4.20 = 4792.20 DOS Batch 77 790 410 1034 x 0.63 = 651.42 Java 4 148 181 972 x 1.36 = 1321.92 Perl 6 104 131 922 x 4.00 = 3688.00 XSD 6 0 0 506 x 1.90 = 961.40 awk 5 65 17 366 x 3.81 = 1394.46 DTD 4 117 50 351 x 1.90 = 666.90 ASP.Net 36 153 561 280 x 1.29 = 361.20 Bourne Again Shell 12 63 8 245 x 3.81 = 933.45 XSLT 1 15 14 196 x 1.90 = 372.40 NAnt scripts 3 27 0 119 x 1.90 = 226.10 Teamcenter def 10 16 0 93 x 1.00 = 93.00 -------------------------------------------------------------------------------- SUM: 8743 272238 215871 1470139 x 1.84 = 2708354.95 From casevh at gmail.com Fri Jan 22 00:02:44 2010 From: casevh at gmail.com (casevh) Date: Thu, 21 Jan 2010 21:02:44 -0800 (PST) Subject: Rounding up to the next 100 References: <74d51f20-b693-4ab2-809f-6f9df6de08c0@r19g2000yqb.googlegroups.com> Message-ID: On Jan 21, 1:58?pm, noydb wrote: > Sorry, although what I really need is the string-number rounded UP > every time. ?So if the number is 3890.32, it needs to go to 3900; if > the number is 3811.345, it needs to go to 3900 also. > > So, Florian's answer works. Another option is using math.ceil and math.floor. >>> import math >>> 100*math.ceil(1234.5678/100) 1300 >>> 100*math.floor(1234.5678/100) 1200 >>> 100*math.ceil(-1234.5678/100) -1200 >>> 100*math.floor(-1234.5678/100) -1300 casevh From gagsl-py2 at yahoo.com.ar Fri Jan 22 00:13:52 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 22 Jan 2010 02:13:52 -0300 Subject: Inheriting methods but over-riding docstrings References: <0361ea2d$0$1336$c3e8da3@news.astraweb.com> <28aa31f4-7331-4ac2-918b-0c59f5a834de@e37g2000yqn.googlegroups.com> <5ecdae50-b417-44bb-ae5b-c1d9aea865a2@y23g2000yqm.googlegroups.com> Message-ID: En Thu, 21 Jan 2010 03:17:40 -0300, Michele Simionato escribi?: > On Jan 21, 12:09 am, "Gabriel Genellina" > wrote: >> This is basically the same technique as in >> but there >> is a difference: you clone the function object *and* the code object it >> is based on. As I understand it, code objects are immutable and there >> is no need to clone them, but I may be wrong. Why did you feel the need >> to clone the code object too? > > No need. I just had the newfunc utility in my library (I think copied > from a recipe in the Python cookbook by Alex Martelli) so I used it. > In this case it is overkill, though. Also, I had no read your post > when I posted my solution, otherwise I would not have sent it ;) > Anyway, the setdocstring utility to extract the parent method was not > explicit in your post and may be of some use to somebody. Ah, ok! I was afraid I missed something -- but reusing code is a perfectly valid reason! -- Gabriel Genellina From dieter at handshake.de Fri Jan 22 01:33:49 2010 From: dieter at handshake.de (Dieter Maurer) Date: 22 Jan 2010 07:33:49 +0100 Subject: Memory usage problem of twisted server In-Reply-To: References: Message-ID: Victor Lin writes on Wed, 20 Jan 2010 02:52:25 -0800 (PST): > Hi, > > I encountered an increasing memory usage problem of my twisted server. > I have posted a question on stackoverflow: > http://stackoverflow.com/questions/2100192/how-to-find-the-source-of-increasing-memory-usage-of-a-twisted-server > > I have read the article "Improving Python's Memory Allocator" ( > http://evanjones.ca/memoryallocator/ ) and Python Memory Management > ( http://evanjones.ca/python-memory.html ). And I now know little > about how Python manages memory. I am wondering, is that the > increasing memory usage problem of my audio broadcasting caused by the > how python manage memory? Your careful reading has already told you that Python delegates memory allocation for larger blocks (>= 256 bytes) to the underlying C runtime library ("malloc" and friends). The C runtime library does not use memory compaction, i.e. it does not relocate used memory blocks in order to free space in few large chunks. Therefore, it is sensible to memory fragmentation: the free space gets scattered around in a large number of rather small blocks. The fragmentation rate is especially high when the memory request sizes have a high variance. > .... > Is that my guessing correct? How can I monitor the memory allocation > of Python? Look at "http://guppy-pe.sourceforge.net/" -- Dieter From anthra.norell at bluewin.ch Fri Jan 22 02:29:10 2010 From: anthra.norell at bluewin.ch (Anthra Norell) Date: Fri, 22 Jan 2010 08:29:10 +0100 Subject: substitution In-Reply-To: References: Message-ID: <4B5953C6.30803@bluewin.ch> Iain King wrote: > On Jan 21, 2:18 pm, Wilbert Berendsen wrote: > >> Op maandag 18 januari 2010 schreef Adi: >> >> >>> keys = [(len(key), key) for key in mapping.keys()] >>> keys.sort(reverse=True) >>> keys = [key for (_, key) in keys] >>> >>> pattern = "(%s)" % "|".join(keys) >>> repl = lambda x : mapping[x.group(1)] >>> s = "fooxxxbazyyyquuux" >>> >>> re.subn(pattern, repl, s) >>> >> I managed to make it even shorted, using the key argument for sorted, not >> putting the whole regexp inside parentheses and pre-compiling the regular >> expression: >> >> import re >> >> mapping = { >> "foo" : "bar", >> "baz" : "quux", >> "quuux" : "foo" >> >> } >> >> # sort the keys, longest first, so 'aa' gets matched before 'a', because >> # in Python regexps the first match (going from left to right) in a >> # |-separated group is taken >> keys = sorted(mapping.keys(), key=len) >> >> rx = re.compile("|".join(keys)) >> repl = lambda x: mapping[x.group()] >> s = "fooxxxbazyyyquuux" >> rx.sub(repl, s) >> >> One thing remaining: if the replacement keys could contain non-alphanumeric >> characters, they should be escaped using re.escape: >> >> rx = re.compile("|".join(re.escape(key) for key in keys)) >> >> Met vriendelijke groet, >> Wilbert Berendsen >> >> --http://www.wilbertberendsen.nl/ >> "You must be the change you wish to see in the world." >> -- Mahatma Gandhi >> > > Sorting it isn't the right solution: easier to hold the subs as tuple > pairs and by doing so let the user specify order. Think of the > following subs: > > "fooxx" -> "baz" > "oxxx" -> "bar" > > does the user want "bazxbazyyyquuux" or "fobarbazyyyquuux"? > > Iain > There is no way you can automate a user's choice. If he wants the second choice (oxxx->bar) he would have to add a third pattern: fooxxx -> fobar. In general, the rules 'upstream over downstream' and 'long over short' make sense in practically all cases. With all but simple substitution runs whose functionality is obvious, the result needs to be checked for unintended hits. To use an example from my SE manual which runs a (whimsical) text through a set of substitutions concentrating overlapping targets: >>> substitutions = [['be', 'BE'], ['being', 'BEING'], ['been', 'BEEN'], ['bee', 'BEE'], ['belong', 'BELONG'], ['long', 'LONG'], ['longer', 'LONGER']] >>> T = Translator (substitutions) # Code further up in this thread handling precedence by the two rules mentioned >>> text = "There was a bee named Mabel belonging to hive nine longing to be a beetle and thinking that being a bee was okay, but she had been a bee long enough and wouldn't be one much longer." >>> print T (text) There was a BEE named MaBEl BELONGing to hive nine LONGing to BE a BEEtle and thinking that BEING a BEE was okay, but she had BEEN a BEE LONG enough and wouldn't BE one much LONGER. All word-length substitutions resolve correctly. There are four unintended translations, though: MaBEl, BELONGing, LONGing and BEEtle. Adding the substitution Mabel->Mabel would prevent the first miss. The others could be taken care of similarly by replacing the target with itself. With large substitution sets and extensive data, this amounts to an iterative process of running, checking and fixing, many times over. That just isn't practical and may have to be abandoned when the substitutions catalog grows out of reasonable bounds. Dependable are runs where the targets are predictably singular, such as long id numbers that cannot possibly match anything but id numbers. Frederic From stefan_ml at behnel.de Fri Jan 22 02:40:23 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 22 Jan 2010 08:40:23 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> <7rrliuFl3gU1@mid.uni-berlin.de> <7rrm4dFofqU1@mid.uni-berlin.de> <7rrmttFtgjU1@mid.uni-berlin.de> Message-ID: <4b595667$0$7618$9b4e6d93@newsspool1.arcor-online.net> Alf P. Steinbach, 21.01.2010 20:24: > Do you understand how bad that makes you look? I think the right thing to say at this point is "don't feed the troll". Stefan From Martin.Drautzburg at web.de Fri Jan 22 02:43:39 2010 From: Martin.Drautzburg at web.de (Martin Drautzburg) Date: Fri, 22 Jan 2010 08:43:39 +0100 Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <89e6f34c-1a7e-4c2e-88cb-ee0397678948@k17g2000yqh.googlegroups.com> Message-ID: <1717313.IJevIQAxM0@beaureve.gmx.net> Carl Banks wrote: > I see. Well, Python is a poor choice for defining an internal DSL > (i.e., DSL using the general language's syntax), because it's > (deliberately) rigid in both grammar and semantics. I had this impression too. > Paul McGuire should be by to recommend PyParsing shortly. I looked it up and it seems to be about parsing strings. This is not what I am looking for as it would create a separate world outside of python. But I haven't looked deeply yet. From Martin.Drautzburg at web.de Fri Jan 22 03:12:46 2010 From: Martin.Drautzburg at web.de (Martin Drautzburg) Date: Fri, 22 Jan 2010 09:12:46 +0100 Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <0368f809$0$1357$c3e8da3@news.astraweb.com> Message-ID: <2002929.SUtrqfrzdN@beaureve.gmx.net> Steven D'Aprano wrote: > I think this really is the correct solution for your problem. In > Python, the standard place to have such public constants is at the > module level, not the function or class. I think you're worrying > unnecessarily about "namespace pollution" -- the module namespace is > *exactly* the right place for them. If two functions both need UP, > DOWN, etc symbols, then either: Defining those symbols at the module level is absolutely fine with me. The namespace pollution is indeed my biggest worry. You see, I want to be able to type in lots of lines with little effort. Preferably I would want to type move up I could still live with move(up) But then I need to import "from Movements import directions" or something like that. If another module defines "up" in some different way, I am in trouble. To circumvent this I would have to "import Movements", but then I's have to write move(directions.up) This is so noisy, I'd rather write move ("up") I don't like the quotes. I don't mind that "up" is a string (as someone suspected), what I dislike is that "up" was created ad-hoc by the caller. Could it be move("UP") as well? You could not tell without looking at the code of move(). Defining the symbols at module level solves THIS problem, but it leaves the above other problems. Still I like this best so far, because it is the standard way of doing this. > You would force them to do this: > > # Get some flags for compile: > flags = re.compile.I & re.compile.M > # ... > # much later on > # x = re.compile(s, flags) > > which is, in my opinion, a needless level of indirection and possibly > in violation of Demeter's Law. So scoping should be at module level? That makes some sense. > Either way, when you go to *use* the direction, you're still passing a > string. There's no difference between: > > move(m.UP) > > and just > > move("up") The difference is that move(m.UPx) would automatically raise an attribute error wheras move("upx") requires extra code in move() to raise an exception. move("up") just looks sematically wrong to me, in contrast len("up") is correct, because it really is an operation on Strings. When the caller writes move(up) should should not (need to) know what "up" really is behind the scenes. >> (4) Finally someone mentioned DSLs. I guess thats absolutely correct. >> This is what I am struggeling to achieve. I did a little googling >> ("how to write DSLs in python"), but haven't found anything appealing >> yet. Any pointers would be appreciated. > > That truly is using a bulldozer to crack a peanut. Well I guess I am really trying to implement a DSL (its about music). I just wasn't aware of that until someone mentioned DLSs here. The "symbols" problem is one problem I could not come up with anything delightful (the other one is getting rid of parenthesis). >> (5) Here is something I came up with myself: >> >> def symbols(aDict): >> aDict["foo"] = "bar" >> >> def someFunction(aFoo): >> print aFoo >> >> symbols(locals()) >> someFunction (foo) #Eh voila: foo is magically defined >> prints: bar >> >> The call to symbols(locals()) is the "magic, magic" I supected would >> be required in my original posting. If someFunction was a member of a >> class, the symbols would be properly tied to that class (albeit not >> the individual function), but still good enough. > > > I disagree about it being "proper" to tie such public symbols to the > class. But in any case, what you're trying to do is not supported by > Python. If it works, that's a happy accident. > > "The contents of this dictionary should not be modified; changes may > not affect the values of local and free variables used by the > interpreter." Two posters stronly discouraged that solution. And even with a decorator it does not look all that beautiful. I'll abandon this one. From __peter__ at web.de Fri Jan 22 03:21:48 2010 From: __peter__ at web.de (Peter Otten) Date: Fri, 22 Jan 2010 09:21:48 +0100 Subject: Default return values for out-of-bounds list item References: <0368f9d5$0$1357$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > On Thu, 21 Jan 2010 17:56:00 -0800, gburdell1 at gmail.com wrote: > >> Is there a built-in method in python that lets you specify a "default" >> value that will be returned whenever you try to access a list item that >> is out of bounds? > > No. > > >> Basically, it would be a function like this: >> >> def item(x,index,default): >> try: >> return x[index] >> except IndexError: >> return default > > That's probably the best way to do it. > > >> So if a=[0,1,2,3], then item(a,0,44)=0, item(a,1,44)=1, and item(a, >> 1000,44)=44, item(a,-1000,44)=44 >> >> What I want to know is whether there is a built-in method or notation >> for this. What if, for example, we could do something like a [1000,44] ? > > You can use slicing instead: > >>>> a=[0,1,2,3] >>>> a[2:3] > [2] >>>> a[100:101] > [] > > > and then detect the empty list and use default: > > def item(x, index, default): > a = x[index:index+1] > return a[0] if a else default You need to special-case -1: >>> for i in range(-4, 4): ... print "x[%d] -> %r" % (i, item("abc", i, "default")) ... x[-4] -> 'default' x[-3] -> 'a' x[-2] -> 'b' x[-1] -> 'default' x[0] -> 'a' x[1] -> 'b' x[2] -> 'c' x[3] -> 'default' Peter From alfps at start.no Fri Jan 22 03:27:26 2010 From: alfps at start.no (Alf P. Steinbach) Date: Fri, 22 Jan 2010 09:27:26 +0100 Subject: Symbols as parameters? In-Reply-To: <4b595667$0$7618$9b4e6d93@newsspool1.arcor-online.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> <7rrliuFl3gU1@mid.uni-berlin.de> <7rrm4dFofqU1@mid.uni-berlin.de> <7rrmttFtgjU1@mid.uni-berlin.de> <4b595667$0$7618$9b4e6d93@newsspool1.arcor-online.net> Message-ID: * Stefan Behnel: > Alf P. Steinbach, 21.01.2010 20:24: >> Do you understand how bad that makes you look? > > I think the right thing to say at this point is "don't feed the troll". I find it amazing that you continue this kind of ad hominem attack. You leave it open who you regard as trolling, but when you do ad hominem you're essentially saying "I'm technically wrong but I'll win in the personal attack domain". Do you understand how bad that makes you look? Cheers & hth., - Alf From alfps at start.no Fri Jan 22 03:29:18 2010 From: alfps at start.no (Alf P. Steinbach) Date: Fri, 22 Jan 2010 09:29:18 +0100 Subject: Symbols as parameters? In-Reply-To: <7174001.cQaEHfC5Mm@beaureve.gmx.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> Message-ID: * Martin Drautzburg: > Here is a complete expample using a decorator, still a bit noisy > > def move(aDirection): > print "moving " + aDirection > > #Here comes the decorator > def scope(aDict): > def save(locals): > """Set symbols in locals and remember their original state""" > setSymbols={} > unsetSymbols=[] > for i in ("up", "down", "left", "right"): > if locals.has_key(i): > setSymbols[i] = locals[i] > else: > unsetSymbols.append(i) > # define the new symbols > locals[i] = i > > return setSymbols, unsetSymbols > > def restore (locals, set, unset): > """restore locals from set and unset""" > for i in set.keys(): > locals[i] = set[i] > for i in unset: > del(locals[i]) > > def callFunc(f): > """Main decorator""" > set, unset = save(aDict) > f() > restore(aDict, set, unset) > return callFunc > > > # -------------------------------------- > # using it > # -------------------------------------- > # a variable defined in the outer scope > up="outerScopeUp" > > # magic, magic (still too noisy for my taste) > @scope (locals()) > def _(): > move(up) > move(down) > move(left) > move(right) > > #verify the the outer scope variable hasn't changed > print "in the outer scope up is still:", up > print > print "this should fail:" > down > > # -------------------------------------- > # Output > # -------------------------------------- > > moving up > moving down > moving left > moving right > in the outer scope up is still: outerScopeUp > > this should fail: > Traceback (most recent call last): > File "", line 50, in > NameError: name 'down' is not defined Uhm, interesting technique, technically. But have you tested this within a function or class, which is what the use of "locals" implies? The reason that I ask is that in the documentation of locals() it says this: Note The contents of this dictionary should not be modified; changes may not affect the values of local variables used by the interpreter. (There's no such note for 'globals'). I have to admit that I was afraid to post this question since my experience in [comp.lang.python] is that when some technical error is pointed out by me, then most often the person starts a personal attack and credibility attack, injecting all kinds of noise -- actually that happened yet again as I was writing this response to you! But, I figure one shouldn't give up one humanity just because of one group where that happens regularly. I'm sort of counting on you to prove that there are, counting myself and one other, and perhaps now you, at least three persons here who are happy for technical corrections from me. Or, perhaps there's some aspect of locals(), e.g. in the context of decorators, that I don't know about and can now learn. :-) Cheers & hth., - Alf From zachar at awst.at Fri Jan 22 03:39:50 2010 From: zachar at awst.at (Balazs Zachar) Date: Fri, 22 Jan 2010 09:39:50 +0100 Subject: Python sys.prefix In-Reply-To: References: Message-ID: <4B596456.90503@awst.at> It looks like the python install environment is easily movable between two machine with the same OS (I mean, with the correct libraries)... Can I use this in production? Please consider the following test, after I use ./configure --prefix=/opt/python2.6.4 && make && make install: [root at CentOS-5-4-test1 ~]# /opt/python2.6.4/bin/python Python 2.6.4 (r264:75706, Jan 20 2010, 20:26:15) [GCC 4.1.2 20080704 (Red Hat 4.1.2-46)] on linux2 Type "help", "copyright", "credits" or "license" for more information. ...import sys ...sys.prefix '/opt/python2.6.4' ...sys.path ['', '/opt/python2.6.4/lib/python26.zip', '/opt/python2.6.4/lib/python2.6', '/opt/python2.6.4/lib/python2.6/plat-linux2', '/opt/python2.6.4/lib/python2.6/lib-tk', '/opt/python2.6.4/lib/python2.6/lib-old', '/opt/python2.6.4/lib/python2.6/lib-dynload', '/opt/python2.6.4/lib/python2.6/site-packages'] [root at CentOS-5-4-test1 ~]# mv /opt/python2.6.4/ /opt/python2.6.4_newplace [root at CentOS-5-4-test1 ~]# /opt/python2.6.4_newplace/bin/python Python 2.6.4 (r264:75706, Jan 20 2010, 20:26:15) [GCC 4.1.2 20080704 (Red Hat 4.1.2-46)] on linux2 Type "help", "copyright", "credits" or "license" for more information. ...import sys ...sys.prefix '/opt/python2.6.4_newplace' ...sys.path ['', '/opt/python2.6.4_newplace/lib/python26.zip', '/opt/python2.6.4_newplace/lib/python2.6', '/opt/python2.6.4_newplace/lib/python2.6/plat-linux2', '/opt/python2.6.4_newplace/lib/python2.6/lib-tk', '/opt/python2.6.4_newplace/lib/python2.6/lib-old', '/opt/python2.6.4_newplace/lib/python2.6/lib-dynload', '/opt/python2.6.4_newplace/lib/python2.6/site-packages'] On 01/21/2010 05:56 PM, zachar at awst.at wrote: > Hi All, > > I need some help about my "problem": > I compiled and installed python with ./configure --prefix=/opt/pyhon2.6.4... But I have to move it to a different directory (the reason is very difficult but I must do this)... I started the interpreter from shell and I was surprised because It still worked and It changed the sys.prefix so it pointed to the correct place so the sys.path was correct.... The questions are: HOW? Is it a feature which is not documented (at least I couldn't find anything about it)? > > From the official python documentation (http://docs.python.org/library/sys.html?highlight=sys#sys.prefix): > "sys.prefix? > > A string giving the site-specific directory prefix where the platform independent Python files are installed; by default, this is the string '/usr/local'. This can be set at build time with the --prefix argument to the configure script. The main collection of Python library modules is installed in the directory prefix + '/lib/pythonversion' while the platform independent header files (all except pyconfig.h) are stored in prefix + '/include/pythonversion', where version is equal to version[:3]." > > In my case, this is not correct: it looks, after I call the interpreter, something change the sys.prefix variable and point it to the right place and not to that directory which I used with the configuration script... This is very useful (I am happy because it solves one of my problems) but I am afraid of it will be do it in a different way in the future (as it is not documented). > > Thanks, > Regards, > Balazs > From duncan.booth at invalid.invalid Fri Jan 22 03:39:52 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 22 Jan 2010 08:39:52 GMT Subject: substitution References: <201001211518.55011.wbsoft@xs4all.nl> <4B5867BA.4070804@mrabarnett.plus.com> Message-ID: Wilbert Berendsen wrote: > # sort the keys, longest first, so 'aa' gets matched before 'a', because > # in Python regexps the first match (going from left to right) in a > # |-separated group is taken > keys = sorted(mapping.keys(), key=len, reverse=True) > This would do just as well (although see Iain King's response for the correct answer): keys = sorted(mapping, reverse=True) You don't need to specify key=len because the default sorting for two strings where one is a prefix of the other will always sort them that way anyway, and you don't need to call mapping.keys() because that's what sorted will sort anyway. -- Duncan Booth http://kupuguy.blogspot.com From georgeolivergo at gmail.com Fri Jan 22 04:16:17 2010 From: georgeolivergo at gmail.com (George Oliver) Date: Fri, 22 Jan 2010 01:16:17 -0800 (PST) Subject: looking for Python live code reloading IDEs Message-ID: <40aa7d9f-aa35-4bfd-b0d9-46a10585731b@b9g2000yqd.googlegroups.com> hi, I'm wondering if there are any Python programming environments that enable live code reloading, for example something like the Scheme- based impromptu (but also meant for any kind of Python program, not just audio/visual generation). Currently I do this directly in my editor (for game development), but I'm curious if there are other options. From pavlovevidence at gmail.com Fri Jan 22 04:18:33 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Fri, 22 Jan 2010 01:18:33 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <89e6f34c-1a7e-4c2e-88cb-ee0397678948@k17g2000yqh.googlegroups.com> <1717313.IJevIQAxM0@beaureve.gmx.net> Message-ID: <67ffb554-8c91-4bf8-9d40-4039ad91eb75@a6g2000yqm.googlegroups.com> On Jan 21, 11:43?pm, Martin Drautzburg wrote: > > Paul McGuire should be by to recommend PyParsing shortly. > > I looked it up and it seems to be about parsing strings. This is not > what I am looking for as it would create a separate world outside of > python. But I haven't looked deeply yet. Well, it's not necessarily a whole world outside Python, since an external DSL can be tied to the underlying to some degree. For example, a simple external DSL might input a string like this: "move up" parse it (such as with PyParsing), and output a string like this: "move(direction.up)" which can then be execed. It wouldn't be a whole new language, just an altered syntax. However, you are right in that, if you don't want to go creating your own syntax, PyParsing will be of no help. You'll have to shoehorn your DSL in a language not well suited for the task as best you can. Carl Banks From rdv at roalddevries.nl Fri Jan 22 05:56:02 2010 From: rdv at roalddevries.nl (Roald de Vries) Date: Fri, 22 Jan 2010 11:56:02 +0100 Subject: Symbols as parameters? In-Reply-To: <1495297.2JfmrJjJTh@beaureve.gmx.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: <4CD97E94-B40E-481D-B367-5FCE2820F48D@roalddevries.nl> Hi Martin, On Jan 21, 2010, at 8:43 AM, Martin Drautzburg wrote: > Hello all, > > When passing parameters to a function, you sometimes need a paramter > which can only assume certain values, e.g. > > def move (direction): > ... > If direction can only be "up", "down", "left" or "right", you can > solve > this by passing strings, but this is not quite to the point: > > - you could pass invalid strings easily > - you need to quote thigs, which is a nuisance > - the parameter IS REALLY NOT A STRING, but a direction > > Alternatively you could export such symbols, so when you "import *" > you > have them available in the caller's namespace. But that forces you > to "import *" which pollutes your namespace. > > What I am really looking for is a way > > - to be able to call move(up) > - having the "up" symbol only in the context of the function > call > > So it should look something like this > > ... magic, magic ... > move(up) > ... unmagic, unmagic ... > print up > > This should complain that "up" is not defined during the "print" call, > but not when move() is called. And of course there should be as little > magic as possible. > > Any way to achieve this? You could do something like this: class Move(object): def __call__(self, direction): print(direction) return 0 def up(self): return self('up') move = Move() Now move.up() means move('up'), and you can obviously do similar things for other directions. From jeanmichel at sequans.com Fri Jan 22 05:56:16 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Fri, 22 Jan 2010 11:56:16 +0100 Subject: Symbols as parameters? In-Reply-To: <0368f809$0$1357$c3e8da3@news.astraweb.com> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <0368f809$0$1357$c3e8da3@news.astraweb.com> Message-ID: <4B598450.1090208@sequans.com> Steven D'Aprano wrote: > (3) Then somone suggested to tie the constants to the function itself, >> as in >> def move(direction): >> print "moving %s" % direction >> >> move.UP = 'up' >> move.DOWN = 'down' >> >> This is quite nice. >> > > I would call it a horrible, horrible, horrible code smell. A stench in > fact. In my opinion, such attributes tied to the function should be > treated as internal to the function, and not the public interface. > > I wouldn't go quite so far as to say they should be treated as private, > but having the caller use them should be rare and unusual. > > I don't think so, my solution is perfect :o) Let me clarify, I wouldn't write this piece of code, but the OP seems to really worry about the 'noise' around its direction definitions. That is why I removed the constant class definition of DIRECTION, and add directly those constants in the function itself, allowing to use them without additional import statements. It is not usual but it is not *that* smelling. def move(direction): """Move to the given direction. @param direction: one of the function constant move.UP or move.DOWN """ print "moving %s" % direction move.UP = 'up' move.DOWN = 'down' As soon as it is properly documented, as a public interface should be, it becomes an acceptable design, IMO. Not the design I would choose in normal circonstances though. Jean-Michel From zuo at chopin.edu.pl Fri Jan 22 05:56:26 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Fri, 22 Jan 2010 11:56:26 +0100 Subject: Symbols as parameters? In-Reply-To: <5552238.LgDixP2R6m@beaureve.gmx.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> Message-ID: 21-01-2010, 22:51:34 Martin Drautzburg wrote: > Thanks for all the answers. Let me summarize > > (1) [...] > (2) Using enum's was suggested. That is good to know, but again it is > just a way to define constants in the caller's namespace. [...] > (3) Then somone suggested to tie the constants to the function itself, > [...] > (4) Finally someone mentioned DSLs. [...] > (5) Here is something I came up with myself: > > def symbols(aDict): > aDict["foo"] = "bar" > > def someFunction(aFoo): > print aFoo > > symbols(locals()) > someFunction (foo) #Eh voila: foo is magically defined [...] And what about such a sollution (6): from functools import wraps def restrict(*arg_options): "Decorator that restricts positional arg values to a limited set." options = set(arg_options) def actual_decorator(func): @wraps(func) def wrapper(*args, **kwargs): if options.issuperset(args): func(*args, **kwargs) else: raise ValueError("possible positional arguments" " for %s() limited to: %s" % (func.__name__, ", ".join(map(repr, arg_options)))) return wrapper return actual_decorator @restrict('up', 'down', 'right', 'left') def move(direction): print(direction) # ... move('up') # OK move('down') # OK move('left') # OK move('right') # OK move('rihgtt') # raises ValueError Although it uses strings, it solves (in Pythonic, explicit way) the main problem: "you could pass invalid strings easily". Typing a two characters more ('') isn't a big effort. Please also note that you can apply not only str-based symbols but any hashable objects (obviously it could be event implemented in less efficient way to accept *any* objects, though I doubt it's worth to do...). Regards, *j -- Jan Kaliszewski (zuo) From ben+python at benfinney.id.au Fri Jan 22 06:07:45 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 22 Jan 2010 22:07:45 +1100 Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <0368f809$0$1357$c3e8da3@news.astraweb.com> Message-ID: <87eili9z3y.fsf@benfinney.id.au> Jean-Michel Pichavant writes: > Steven D'Aprano wrote: > > I would call it a horrible, horrible, horrible code smell. A stench > > in fact. [?] > As soon as it is properly documented, as a public interface should be, > it becomes an acceptable design, IMO. [?] So your position seems to be that any design fault ceases to be a fault if it is documented. If that's not your position, I would be interested to know what relevance the above statement has to Stephen's objection. -- \ ?I have one rule to live by: Don't make it worse.? ?Hazel | `\ Woodcock | _o__) | Ben Finney From zuo at chopin.edu.pl Fri Jan 22 06:09:08 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Fri, 22 Jan 2010 12:09:08 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> Message-ID: s/sollution/solution s/event implemented/even implemented Sorry *j From robin at reportlab.com Fri Jan 22 06:11:35 2010 From: robin at reportlab.com (Robin Becker) Date: Fri, 22 Jan 2010 11:11:35 +0000 Subject: decimal threading cost? Message-ID: <4B5987E7.8070401@chamonix.reportlab.co.uk> Does using the decimal module incur a penalty because it imports threading or do I have to actually start a thread? -- Robin Becker From dickinsm at gmail.com Fri Jan 22 06:50:58 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 22 Jan 2010 03:50:58 -0800 (PST) Subject: decimal threading cost? References: Message-ID: <4abc165a-eb6c-450f-ac8b-68e076115c22@n7g2000yqb.googlegroups.com> On Jan 22, 11:11?am, Robin Becker wrote: > Does using the decimal module incur a penalty because it imports threading or do > I have to actually start a thread? There is at least one threading-related performance penalty that doesn't involve the user actually starting a thread: any arithmetic operation without an explicitly specified context (e.g., as invoked by one of the standard arithmetic operators) has to make a call to threading.local() to look up the current (thread-local) context. I don't know offhand whether this penalty is significant for the Python version of decimal, though it definitely *is* significant for decimal-in-C rewrite that's in the works: it would be interesting to do some timings against a thread-unaware version of decimal.py. -- Mark From davea at ieee.org Fri Jan 22 06:56:01 2010 From: davea at ieee.org (Dave Angel) Date: Fri, 22 Jan 2010 06:56:01 -0500 Subject: Symbols as parameters? In-Reply-To: <4CD97E94-B40E-481D-B367-5FCE2820F48D@roalddevries.nl> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <4CD97E94-B40E-481D-B367-5FCE2820F48D@roalddevries.nl> Message-ID: <4B599251.6090404@ieee.org> Roald de Vries wrote: >
Hi Martin, > > On Jan 21, 2010, at 8:43 AM, Martin Drautzburg wrote: >> Hello all, >> >> When passing parameters to a function, you sometimes need a paramter >> which can only assume certain values, e.g. >> >> def move (direction): >> ... >> If direction can only be "up", "down", "left" or "right", you can solve >> this by passing strings, but this is not quite to the point: >> >> - you could pass invalid strings easily >> - you need to quote thigs, which is a nuisance >> - the parameter IS REALLY NOT A STRING, but a direction >> >> Alternatively you could export such symbols, so when you "import *" you >> have them available in the caller's namespace. But that forces you >> to "import *" which pollutes your namespace. >> >> What I am really looking for is a way >> >> - to be able to call move(up) >> - having the "up" symbol only in the context of the function call >> >> So it should look something like this >> >> ... magic, magic ... >> move(up) >> ... unmagic, unmagic ... >> print up >> >> This should complain that "up" is not defined during the "print" call, >> but not when move() is called. And of course there should be as little >> magic as possible. >> >> Any way to achieve this? > > You could do something like this: > > class Move(object): > def __call__(self, direction): > print(direction) > return 0 > > def up(self): > return self('up') > > move = Move() > > Now move.up() means move('up'), and you can obviously do similar > things for other directions. > Once pointed down that road, how about: class Move(object): def __call__(self, direction): print(direction) return 0 @property def up(self): return self('up') move = Move() Now you can just say move.up with no parentheses. When I've wanted a DSL (Domain Specific Language) in the past, I've used Forth. It has so little syntax of its own, it's not hard to design your own simple syntax for a particular problem. And one of the things you can do is to define keywords that intercept the compile process of the following token (or line, or whatever). I am not claiming it's quick to become that proficient in Forth, however. The learning curve has a couple of steep sections, and this sort of thing is one of them. DaveA From drautzburg at googlemail.com Fri Jan 22 07:06:38 2010 From: drautzburg at googlemail.com (Martin Drautzburg) Date: Fri, 22 Jan 2010 04:06:38 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: <80071018-8653-43ad-8bc8-66ddaeb971ef@k35g2000yqb.googlegroups.com> On 22 Jan., 11:56, Roald de Vries wrote: > Hi Martin, > > On Jan 21, 2010, at 8:43 AM, Martin Drautzburg wrote: > > > > > > > Hello all, > > > When passing parameters to a function, you sometimes need a paramter > > which can only assume certain values, e.g. > > > ? ? ? ?def move (direction): > > ? ? ? ? ? ? ? ?... > > If direction can only be "up", "down", "left" or "right", you can ? > > solve > > this by passing strings, but this is not quite to the point: > > > ? ? ? ?- you could pass invalid strings easily > > ? ? ? ?- you need to quote thigs, which is a nuisance > > ? ? ? ?- the parameter IS REALLY NOT A STRING, but a direction > > > Alternatively you could export such symbols, so when you "import *" ? > > you > > have them available in the caller's namespace. But that forces you > > to "import *" which pollutes your namespace. > > > What I am really looking for is a way > > > ? ? ? ?- to be able to call move(up) > > ? ? ? ?- having the "up" symbol only in the context of the function ? > > call > > > So it should look something like this > > > ... magic, magic ... > > move(up) > > ... unmagic, unmagic ... > > print up > > > This should complain that "up" is not defined during the "print" call, > > but not when move() is called. And of course there should be as little > > magic as possible. > > > Any way to achieve this? > > You could do something like this: > > class Move(object): > ? ? ?def __call__(self, direction): > ? ? ? ? ?print(direction) > ? ? ? ? ?return 0 > > ? ? ?def up(self): > ? ? ? ? ?return self('up') > > move = Move() > > Now move.up() means move('up'), and you can obviously do similar ? > things for other directions.- Zitierten Text ausblenden - > > - Zitierten Text anzeigen - I had thought about that too. It gets a bit tricky when there is more than one parameter and it completely fails whan a parameter can REALLY be a number or an arbitrary string. Think: move(direction, distance) From nospam at nospam.com Fri Jan 22 07:17:44 2010 From: nospam at nospam.com (Gilles Ganault) Date: Fri, 22 Jan 2010 13:17:44 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? Message-ID: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Hello I use a dictionary to keep a list of users connected to a web site. To avoid users from creating login names that start with digits in order to be listed at the top, I'd like to sort the list differently every minute so that it'll start with the next letter, eg. display the list from A...Zdigits the first time, then B...ZAdigits, etc. That way, users have no incentive to create login names that start with either a digit or letter A. I see that dictionaries can be sorted using the... sort() method, but is it possible to have Python start sorting from a different letter? Thank you. From rdv at roalddevries.nl Fri Jan 22 07:29:22 2010 From: rdv at roalddevries.nl (Roald de Vries) Date: Fri, 22 Jan 2010 13:29:22 +0100 Subject: Symbols as parameters? In-Reply-To: <80071018-8653-43ad-8bc8-66ddaeb971ef@k35g2000yqb.googlegroups.com> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <80071018-8653-43ad-8bc8-66ddaeb971ef@k35g2000yqb.googlegroups.com> Message-ID: <14798C82-47F6-42B0-97AA-C603B81C5BF7@roalddevries.nl> On Jan 22, 2010, at 1:06 PM, Martin Drautzburg wrote: > On 22 Jan., 11:56, Roald de Vries wrote: >> Hi Martin, >> >> On Jan 21, 2010, at 8:43 AM, Martin Drautzburg wrote: >> >> >> >> >> >>> Hello all, >> >>> When passing parameters to a function, you sometimes need a paramter >>> which can only assume certain values, e.g. >> >>> def move (direction): >>> ... >>> If direction can only be "up", "down", "left" or "right", you can >>> solve >>> this by passing strings, but this is not quite to the point: >> >>> - you could pass invalid strings easily >>> - you need to quote thigs, which is a nuisance >>> - the parameter IS REALLY NOT A STRING, but a direction >> >>> Alternatively you could export such symbols, so when you "import *" >>> you >>> have them available in the caller's namespace. But that forces you >>> to "import *" which pollutes your namespace. >> >>> What I am really looking for is a way >> >>> - to be able to call move(up) >>> - having the "up" symbol only in the context of the function >>> call >> >>> So it should look something like this >> >>> ... magic, magic ... >>> move(up) >>> ... unmagic, unmagic ... >>> print up >> >>> This should complain that "up" is not defined during the "print" >>> call, >>> but not when move() is called. And of course there should be as >>> little >>> magic as possible. >> >>> Any way to achieve this? >> >> You could do something like this: >> >> class Move(object): >> def __call__(self, direction): >> print(direction) >> return 0 >> >> def up(self): >> return self('up') >> >> move = Move() >> >> Now move.up() means move('up'), and you can obviously do similar >> things for other directions.- Zitierten Text ausblenden - >> >> - Zitierten Text anzeigen - > > I had thought about that too. It gets a bit tricky when there is more > than one parameter and it completely fails whan a parameter can REALLY > be a number or an arbitrary string. Think: move(direction, distance) For extra parameters: class Move(object): def __call__(self, direction, param1, param2, *params, **dict): print(direction) return 0 def up(self, *params, **dict): return self('up', *params, **dict) move = Move() On Jan 22, 2010, at 1:06 PM, Martin Drautzburg wrote: > On 22 Jan., 11:56, Roald de Vries wrote: >> Hi Martin, >> >> On Jan 21, 2010, at 8:43 AM, Martin Drautzburg wrote: >> >> >> >> >> >>> Hello all, >> >>> When passing parameters to a function, you sometimes need a paramter >>> which can only assume certain values, e.g. >> >>> def move (direction): >>> ... >>> If direction can only be "up", "down", "left" or "right", you can >>> solve >>> this by passing strings, but this is not quite to the point: >> >>> - you could pass invalid strings easily >>> - you need to quote thigs, which is a nuisance >>> - the parameter IS REALLY NOT A STRING, but a direction >> >>> Alternatively you could export such symbols, so when you "import *" >>> you >>> have them available in the caller's namespace. But that forces you >>> to "import *" which pollutes your namespace. >> >>> What I am really looking for is a way >> >>> - to be able to call move(up) >>> - having the "up" symbol only in the context of the function >>> call >> >>> So it should look something like this >> >>> ... magic, magic ... >>> move(up) >>> ... unmagic, unmagic ... >>> print up >> >>> This should complain that "up" is not defined during the "print" >>> call, >>> but not when move() is called. And of course there should be as >>> little >>> magic as possible. >> >>> Any way to achieve this? >> >> You could do something like this: >> >> class Move(object): >> def __call__(self, direction): >> print(direction) >> return 0 >> >> def up(self): >> return self('up') >> >> move = Move() >> >> Now move.up() means move('up'), and you can obviously do similar >> things for other directions.- Zitierten Text ausblenden - >> >> - Zitierten Text anzeigen - > > I had thought about that too. It gets a bit tricky when there is more > than one parameter For extra parameters: class Move(object): def __call__(self, direction, param1, param2, *params, **dict): print(direction) return 0 def up(self, *params, **dict): return self('up', *params, **dict) move = Move() > and it completely fails whan a parameter can REALLY > be a number or an arbitrary string. Think: move(direction, distance) For a number, move.up(10) wouldn't be too bad. For an arbitrary string, you always have __getattr__ and __getattribute__. But if you like this style of coding, you should probably switch to Ruby. From dickinsm at gmail.com Fri Jan 22 07:34:06 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 22 Jan 2010 04:34:06 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> Message-ID: <065e450f-2190-42f4-b9c6-4b95260092be@f12g2000yqn.googlegroups.com> On Jan 21, 10:57?pm, Martin Drautzburg wrote: > Here is a complete expample using a decorator, still a bit noisy > > def move(aDirection): > ? ? print "moving " + aDirection > > #Here comes the decorator > def scope(aDict): > ? ? def save(locals): > [...] Have you considered making 'scope' a context manager? Your modifying-locals hacks could be put into the __enter__ and __exit__ methods, and you'd use the context manager with something like: with scope(): # ... # use up, down, left, right here # up, down, left, right no longer defined after the with block exits. -- Mark From robin at reportlab.com Fri Jan 22 07:35:02 2010 From: robin at reportlab.com (Robin Becker) Date: Fri, 22 Jan 2010 12:35:02 +0000 Subject: decimal threading cost? In-Reply-To: <4abc165a-eb6c-450f-ac8b-68e076115c22@n7g2000yqb.googlegroups.com> References: <4abc165a-eb6c-450f-ac8b-68e076115c22@n7g2000yqb.googlegroups.com> Message-ID: <4B599B76.2000600@chamonix.reportlab.co.uk> On 22/01/2010 11:50, Mark Dickinson wrote: > On Jan 22, 11:11 am, Robin Becker wrote: >> Does using the decimal module incur a penalty because it imports threading or do >> I have to actually start a thread? > > There is at least one threading-related performance penalty that > doesn't involve the user actually starting a thread: any arithmetic > operation without an explicitly specified context (e.g., as invoked by > one of the standard arithmetic operators) has to make a call to > threading.local() to look up the current (thread-local) context. > > I don't know offhand whether this penalty is significant for the > Python version of decimal, though it definitely *is* significant for > decimal-in-C rewrite that's in the works: it would be interesting to > do some timings against a thread-unaware version of decimal.py. > > -- > Mark I guess I was more worried that the import might start the per thread instruction count check even though I'm in a single thread process. -- Robin Becker From steve at REMOVE-THIS-cybersource.com.au Fri Jan 22 07:35:40 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 12:35:40 GMT Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <0368f809$0$1357$c3e8da3@news.astraweb.com> <2002929.SUtrqfrzdN@beaureve.gmx.net> Message-ID: <03698827$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 09:12:46 +0100, Martin Drautzburg wrote: > Defining those symbols at the module level is absolutely fine with me. > The namespace pollution is indeed my biggest worry. You see, I want to > be able to type in lots of lines with little effort. Preferably I would > want to type > > move up > > I could still live with > > move(up) > > But then I need to import "from Movements import directions" or > something like that. That's one line. It's hardly any cost, and it makes the code understandable: the reader can see exactly where directions comes from. This is a Good Thing. > If another module defines "up" in some different > way, I am in trouble. Only if you do this: from movements import up, down, left, right from wossnames import up, down, left, right The solution to this problem is simple: don't do it. Honestly, you're over-thinking this problem. Such names clashes can't happen by accident. They can happen through carelessness, but that's no different from this: x = 42 # ... much later on ... x = 23 # ... and later ... assert x == 42 The only time such name clashes can happen by accident is if you do from movements import * which is precisely why that form of import is not recommended. > To circumvent this I would have to "import > Movements", but then I's have to write > > move(directions.up) You'd have to say "import directions" to write "move(directions.up)". Another solution: import directions up = directions.up move(up) > This is so noisy, I'd rather write > > move ("up") > > I don't like the quotes. I don't mind that "up" is a string (as someone > suspected), what I dislike is that "up" was created ad-hoc by the > caller. Could it be move("UP") as well? You could not tell without > looking at the code of move(). Or the documentation. This is an API design decision the designer has to make. Should the move function use strings or integers or something else to specify the direction? If strings, should they be case-sensitive or insensitive? There's no right or wrong answer, all of these things have arguments in favour and against. [...] >> Either way, when you go to *use* the direction, you're still passing a >> string. There's no difference between: >> >> move(m.UP) >> >> and just >> >> move("up") > > The difference is that move(m.UPx) would automatically raise an > attribute error and move(UPx) would raise NameError, and move(U P) would raise SyntaxError. What's your point? > wheras move("upx") requires extra code in move() to > raise an exception. What extra code? Surely move already validates its input? Surely it looks something like this? def move(direction): if direction == 'up': foo elif direction == 'down': bar elif direction == 'left': baz elif direction == 'right': foobar else: raise ValueError("invalid direction") (or any variation that does the same sort of thing). The point is, you have to validate that direction is a valid direction anyway -- you can't trust that the user will only pass valid directions. There's nothing stopping the caller from saying move([42, 23]) or move(None), so you have to validate the argument inside the function anyway. > move("up") just looks sematically wrong to me, in > contrast len("up") is correct, because it really is an operation on > Strings. When the caller writes move(up) should should not (need to) > know what "up" really is behind the scenes. Again, I point you to Ben Finney's enum module, which I think is exactly what you want. -- Steven From jeanmichel at sequans.com Fri Jan 22 07:53:58 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Fri, 22 Jan 2010 13:53:58 +0100 Subject: Symbols as parameters? In-Reply-To: <87eili9z3y.fsf@benfinney.id.au> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <0368f809$0$1357$c3e8da3@news.astraweb.com> <87eili9z3y.fsf@benfinney.id.au> Message-ID: <4B599FE6.9010004@sequans.com> Ben Finney wrote: > Jean-Michel Pichavant writes: > > >> Steven D'Aprano wrote: >> >>> I would call it a horrible, horrible, horrible code smell. A stench >>> in fact. >>> > [?] > > >> As soon as it is properly documented, as a public interface should be, >> it becomes an acceptable design, IMO. >> > [?] > > So your position seems to be that any design fault ceases to be a fault > if it is documented. If that's not your position, I would be interested > to know what relevance the above statement has to Stephen's objection. > > Well, this disign is: 1/ working 2/ simple 3/ unusual I don't now if unusual means necessarily faulty or horrible. Still I don't want to defend this design at all cost, 'cause I'm not using it anyway, and I happily acknowledge that there are better ones (commonly used). I just wanted to help the OP with writing some simple and short code to use constants instead of strings in a context of one function. Note how I used *acceptable* in my previous post to qualify this design which is a rather cautious statement. Cheers, Jean-Michel From steve at REMOVE-THIS-cybersource.com.au Fri Jan 22 07:55:40 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 12:55:40 GMT Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> Message-ID: <03698cd8$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 09:29:18 +0100, Alf P. Steinbach wrote: > But have you tested this within a function or class, which is what the > use of "locals" implies? > > The reason that I ask is that in the documentation of locals() it says > this: > > Note > The contents of this dictionary should not be modified; changes may > not affect the values of local variables used by the interpreter. > > (There's no such note for 'globals'). > > I have to admit that I was afraid to post this question since my > experience in [comp.lang.python] is that when some technical error is > pointed out by me, then most often the person starts a personal attack > and credibility attack, injecting all kinds of noise -- actually that > happened yet again as I was writing this response to you! But, I figure > one shouldn't give up one humanity just because of one group where that > happens regularly. I'm sort of counting on you to prove that there are, > counting myself and one other, and perhaps now you, at least three > persons here who are happy for technical corrections from me. I've previously said, and I'll say it again, that you do bring much of value to this community, tech-wise. Shame that it comes along with such a thin skin. It's getting so that I often find myself afraid to disagree with anything you say lest you accuse me of lying again. > Or, perhaps there's some aspect of locals(), e.g. in the context of > decorators, that I don't know about and can now learn. :-) No, you got it spot on. Not to discourage you, but you're at least the third person who pointed this out in this thread. One implementation-specific trick is that modifying locals does actually work inside a class definition (at least in Python 2.5): >>> class Foo(object): ... x = 1 ... print locals() ... locals()['x'] = 2 ... {'x': 1, '__module__': '__main__'} >>> Foo.x 2 But it doesn't work in functions. That is because the local variables in CPython functions aren't stored in a dict, for efficiency reasons, so locals() makes a copy of those variables rather than returning the actual dict used as a namespace. This suggests we can cause locals() to malfunction in a class too, by using slots, since slotted attributes aren't stored in a dictionary. That's what I would predict, but alas I'm wrong: >>> class Foo(object): ... __slots__ = 'x' ... x = 1 ... print locals() ... locals()['x'] = 2 ... {'x': 1, '__module__': '__main__', '__slots__': 'x'} >>> Foo.x 2 So I don't understand why this works. Anyone know? Bottom line is, modifying locals() is not supported as a language feature. If it works, it's an accident. (Since Python can't guarantee that modifications to locals() will take, I wonder whether it would be better to ensure that they *never* take, rather than sometimes. It would only require locals() to return a copy of the dict, a shallow copy would probably do.) -- Steven From jeanmichel at sequans.com Fri Jan 22 08:06:35 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Fri, 22 Jan 2010 14:06:35 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? In-Reply-To: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: <4B59A2DB.4080809@sequans.com> Gilles Ganault wrote: > Hello > > I use a dictionary to keep a list of users connected to a web site. > > To avoid users from creating login names that start with digits in > order to be listed at the top, I'd like to sort the list differently > every minute so that it'll start with the next letter, eg. display the > list from A...Zdigits the first time, then B...ZAdigits, etc. > > That way, users have no incentive to create login names that start > with either a digit or letter A. > > I see that dictionaries can be sorted using the... sort() method, but > is it possible to have Python start sorting from a different letter? > > Thank you. > Here is one possible solution l = ['1a', 'a', 'b','c','av','ac'] # you mentioned a dictionary in your post, if so, l = myDict.keys() l.sort() # sort your list once and for all for start in '1abcd': result = [name for name in l if name[0] >= start] + [name for name in l if name[0] < start] print result ['1a', 'a', 'b', 'c', 'av', 'ac'] ['a', 'b', 'c', 'av', 'ac', '1a'] ['b', 'c', '1a', 'a', 'av', 'ac'] ['c', '1a', 'a', 'b', 'av', 'ac'] ['1a', 'a', 'b', 'c', 'av', 'ac'] Jean-Michel From nospam at nospam.com Fri Jan 22 08:07:04 2010 From: nospam at nospam.com (Gilles Ganault) Date: Fri, 22 Jan 2010 14:07:04 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: On Fri, 22 Jan 2010 13:17:44 +0100, Gilles Ganault wrote: >I see that dictionaries can be sorted using the... sort() method, but >is it possible to have Python start sorting from a different letter? Looks like the solution is to read the list of keys into a list, sort the list, and then use this to read the dictionary: http://code.activestate.com/recipes/52306/ But I haven't found whether a list can be sorted by starting from a given letter instead of "a". From jeanmichel at sequans.com Fri Jan 22 08:09:43 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Fri, 22 Jan 2010 14:09:43 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? In-Reply-To: <4B59A2DB.4080809@sequans.com> References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <4B59A2DB.4080809@sequans.com> Message-ID: <4B59A397.4010102@sequans.com> Jean-Michel Pichavant wrote: > Gilles Ganault wrote: >> Hello >> >> I use a dictionary to keep a list of users connected to a web site. >> >> To avoid users from creating login names that start with digits in >> order to be listed at the top, I'd like to sort the list differently >> every minute so that it'll start with the next letter, eg. display the >> list from A...Zdigits the first time, then B...ZAdigits, etc. >> >> That way, users have no incentive to create login names that start >> with either a digit or letter A. >> >> I see that dictionaries can be sorted using the... sort() method, but >> is it possible to have Python start sorting from a different letter? >> >> Thank you. >> > Here is one possible solution > > l = ['1a', 'a', 'b','c','av','ac'] # you mentioned a dictionary in > your post, if so, l = myDict.keys() > l.sort() # sort your list once and for all > for start in '1abcd': > result = [name for name in l if name[0] >= start] + [name for name > in l if name[0] < start] > print result > > > ['1a', 'a', 'b', 'c', 'av', 'ac'] > ['a', 'b', 'c', 'av', 'ac', '1a'] > ['b', 'c', '1a', 'a', 'av', 'ac'] > ['c', '1a', 'a', 'b', 'av', 'ac'] > ['1a', 'a', 'b', 'c', 'av', 'ac'] > > > Jean-Michel Sorry, the code I provided produce this output: ['1a', 'a', 'ac', 'av', 'b', 'c'] ['a', 'ac', 'av', 'b', 'c', '1a'] ['b', 'c', '1a', 'a', 'ac', 'av'] ['c', '1a', 'a', 'ac', 'av', 'b'] ['1a', 'a', 'ac', 'av', 'b', 'c'] which is actually what you are searching for. I just messed up with my ipython shell history :o) JM From alfps at start.no Fri Jan 22 08:16:50 2010 From: alfps at start.no (Alf P. Steinbach) Date: Fri, 22 Jan 2010 14:16:50 +0100 Subject: Symbols as parameters? In-Reply-To: <03698cd8$0$1357$c3e8da3@news.astraweb.com> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> <03698cd8$0$1357$c3e8da3@news.astraweb.com> Message-ID: * Steven D'Aprano -> Alf P. Steinbach: > > > No, you got it spot on. Not to discourage you, but you're at least the > third person who pointed this out in this thread. I get the impression that there's some message traffic that I don't see, perhaps on the mailing list, since (a) I haven't seen that about 'locals' pointed out by anyone else in this thread, and I think I've read all messages in the thread, and (b) sometimes threads pop up that start with a reply. For example, the recent thread "Covert number into string" started with a /reply/ in my newreader, using EternalSeptember's NNTP host. It also starts with a reply in Google's archives, . Cheers, - Alf PS: No, I'm not metaphorically thin-skinned. :-) As you can see in this thread I'm a pretty calm person, not getting upset by e.g. accusations of idiocy and insanity. The reason for my comments about being afraid to post is that even though rare cases of such language and ad-hominem just reflects on the one doing it, when it becomes a group/mob phenomenon then it creates an impression, like e.g. the one implicit in your "thin skin", that somehow I'm the one reacting to things by characterizing people -- which the archives show is not so. From steve at REMOVE-THIS-cybersource.com.au Fri Jan 22 08:24:05 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 13:24:05 GMT Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: <03699381$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 13:17:44 +0100, Gilles Ganault wrote: > Hello > > I use a dictionary to keep a list of users connected to a web site. > > To avoid users from creating login names that start with digits in order > to be listed at the top, I'd like to sort the list differently every > minute so that it'll start with the next letter, eg. display the list > from A...Zdigits the first time, then B...ZAdigits, etc. > > That way, users have no incentive to create login names that start with > either a digit or letter A. If you want to prohibit users from starting their login names with a digit, prohibit them from creating a login name with a digit. > I see that dictionaries can be sorted using the... sort() method, but is > it possible to have Python start sorting from a different letter? You can write a customer sort routine by using the key parameter to sort, which will probably be messy. What I'd do is keep 27 lists of user names, according to the first letter (26 letters from A to Z, plus one extra): users = [ # keep separate lists for each starting letter ['aaron', 'avril', 'adam'], ['betty', 'bob'], ['craig', 'cathy'], ['danni', 'da5id', 'donna'], # ... and so on ['zoe', 'zach'], ['4dam', '1-4m-t00-c001'] ] # add a new user users[3].append('daniel') Sort each one individually, and display them in whatever order you like: def display(start=0): start = start % 27 all = users[start:] + users[:start] for l in all: l.sort() print l And in use: >>> display(0) ['aaron', 'adam', 'avril'] ['betty', 'bob'] ['cathy', 'craig'] ['da5id', 'daniel', 'danni', 'donna'] ['zach', 'zoe'] ['1-4m-t00-c001', '4dam'] >>> >>> display(3) ['da5id', 'daniel', 'danni', 'donna'] ['zach', 'zoe'] ['1-4m-t00-c001', '4dam'] ['aaron', 'adam', 'avril'] ['betty', 'bob'] ['cathy', 'craig'] -- Steven From steve at REMOVE-THIS-cybersource.com.au Fri Jan 22 08:32:24 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 13:32:24 GMT Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <03699381$0$1357$c3e8da3@news.astraweb.com> Message-ID: <03699574$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 13:24:05 +0000, Steven D'Aprano wrote: > On Fri, 22 Jan 2010 13:17:44 +0100, Gilles Ganault wrote: [...] >> I see that dictionaries can be sorted using the... sort() method, but >> is it possible to have Python start sorting from a different letter? > > You can write a customer sort routine by using the key parameter to > sort, which will probably be messy. What I'd do is keep 27 lists of user > names, according to the first letter (26 letters from A to Z, plus one > extra): Replying to myself... the first sign of insanity *wink* While I'd still stick to that basic strategy, I'd wrap the functionality in a class so that the whole thing was transparent to the caller. So instead of the example I gave: > # add a new user > users[3].append('daniel') I'd write the class so the caller just needed to do: users.add('daniel') and the class would calculate which inner list to append it to. I just thought I'd make that clear in case you thought I expected you to manually keep track of which inner list each name should go into. -- Steven From wolfgang at rohdewald.de Fri Jan 22 08:35:07 2010 From: wolfgang at rohdewald.de (Wolfgang Rohdewald) Date: Fri, 22 Jan 2010 14:35:07 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <03698cd8$0$1357$c3e8da3@news.astraweb.com> Message-ID: <201001221435.20607.wolfgang@rohdewald.de> On Friday 22 January 2010, Alf P. Steinbach wrote: > I get the impression that there's some message traffic that I don't > see > For example, the recent thread "Covert number into string" started > with a reply in my newreader, using EternalSeptember's NNTP host. > > It also starts with a reply in Google's archives, http://groups.google.com/group/comp.lang.python/browse_thread/threa > d/b8097d4de4a9c9b0/cb3a2e6ccd7736ef>. did you check your spam folder? I got the original. If you want to, I can mail you the headers privately -- Wolfgang From neilc at norwich.edu Fri Jan 22 08:35:26 2010 From: neilc at norwich.edu (Neil Cerutti) Date: 22 Jan 2010 13:35:26 GMT Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: <7rtnsuFu4tU1@mid.individual.net> On 2010-01-22, Gilles Ganault wrote: > Hello > > I use a dictionary to keep a list of users connected to a web > site. > > To avoid users from creating login names that start with digits > in order to be listed at the top, I'd like to sort the list > differently every minute so that it'll start with the next > letter, eg. display the list from A...Zdigits the first time, > then B...ZAdigits, etc. Resorting is more work than is needed. Just choose a different starting index each time you display the names, and set up your lister to wrap-around to your arbitrary starting index. -- Neil Cerutti From nospam at nospam.com Fri Jan 22 08:58:58 2010 From: nospam at nospam.com (Gilles Ganault) Date: Fri, 22 Jan 2010 14:58:58 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <7rtnsuFu4tU1@mid.individual.net> Message-ID: <4lbjl5p7s67orgc91isk3ccvgp9s5m78l8@4ax.com> On 22 Jan 2010 13:35:26 GMT, Neil Cerutti wrote: >Resorting is more work than is needed. Just choose a different >starting index each time you display the names, and set up your >lister to wrap-around to your arbitrary starting index. Thanks. In this case, it means that in each loop iteration, I must search the list to find which item starts with the letter I'd like to begin sorting, eg. "B". Does Python include a search method or do I have to use a for loop to locate this starting item? From malkarouri at gmail.com Fri Jan 22 09:13:30 2010 From: malkarouri at gmail.com (Muhammad Alkarouri) Date: Fri, 22 Jan 2010 06:13:30 -0800 (PST) Subject: Consume an iterable Message-ID: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> In the python help for itertools, the following function is provided: def consume(iterator, n): "Advance the iterator n-steps ahead. If n is none, consume entirely." collections.deque(islice(iterator, n), maxlen=0) What is the advantage of using a collections.deque against, say, the following code? def consume(iterator, n): for _ in islice(iterator, n): pass Regards, Muhammad Alkarouri From alfps at start.no Fri Jan 22 09:36:32 2010 From: alfps at start.no (Alf P. Steinbach) Date: Fri, 22 Jan 2010 15:36:32 +0100 Subject: Symbols as parameters? In-Reply-To: <03698cd8$0$1357$c3e8da3@news.astraweb.com> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> <03698cd8$0$1357$c3e8da3@news.astraweb.com> Message-ID: * Steven D'Aprano: > > One implementation-specific trick is that modifying locals does actually > work inside a class definition (at least in Python 2.5): > >>>> class Foo(object): > ... x = 1 > ... print locals() > ... locals()['x'] = 2 > ... > {'x': 1, '__module__': '__main__'} >>>> Foo.x > 2 > > But it doesn't work in functions. That is because the local variables in > CPython functions aren't stored in a dict, for efficiency reasons, so > locals() makes a copy of those variables rather than returning the actual > dict used as a namespace. > > This suggests we can cause locals() to malfunction in a class too, by > using slots, since slotted attributes aren't stored in a dictionary. > That's what I would predict, but alas I'm wrong: > >>>> class Foo(object): > ... __slots__ = 'x' > ... x = 1 > ... print locals() > ... locals()['x'] = 2 > ... > {'x': 1, '__module__': '__main__', '__slots__': 'x'} >>>> Foo.x > 2 > > So I don't understand why this works. Anyone know? I don't *know*, but I have a speculation. It goes like this: 1. Perhaps first the statements in the class body are evaluated using a dictionary for locals, with as yet no existing class object. 2. Perhaps that dictionary plus some other info is then passed to __new__ of the class' metaclass, e.g. by default 'type' as metaclass, which perhaps produces a class object, filling in its slots and/or __dict__ from the supplied dictionary. :-) I'd have to read documentation and PEPs to say anything more for sure. > Bottom line is, modifying locals() is not supported as a language > feature. If it works, it's an accident. By the way, when I used 'class' to create a local scope I didn't have this behavior in mind, even if it might seem to be a strange coincidence. I just learned the above about metaclasses, if it is a correct impression, by checking whether the OP's declarative-like-usage code could be rescued in some way. Unfortunately my idea, fixing up the class' __dict__ in the metaclass, didn't work because the metaclass __new__ turned out to be called after, not before. > (Since Python can't guarantee that modifications to locals() will take, I > wonder whether it would be better to ensure that they *never* take, > rather than sometimes. It would only require locals() to return a copy of > the dict, a shallow copy would probably do.) I agree. Cheers, - Alf From joncle at googlemail.com Fri Jan 22 09:37:19 2010 From: joncle at googlemail.com (Jon Clements) Date: Fri, 22 Jan 2010 06:37:19 -0800 (PST) Subject: Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <7rtnsuFu4tU1@mid.individual.net> <4lbjl5p7s67orgc91isk3ccvgp9s5m78l8@4ax.com> Message-ID: On Jan 22, 1:58?pm, Gilles Ganault wrote: > On 22 Jan 2010 13:35:26 GMT, Neil Cerutti wrote: > > >Resorting is more work than is needed. Just choose a different > >starting index each time you display the names, and set up your > >lister to wrap-around to your arbitrary starting index. > > Thanks. In this case, it means that in each loop iteration, I must > search the list to find which item starts with the letter I'd like to > begin sorting, eg. "B". Does Python include a search method or do I > have to use a for loop to locate this starting item? How about a deque of lists... untested from collections import deque; from itertools import groupby deq = deque(list(items) for key, items in groupby(sorted(usernames), lambda L: L[0].upper())) Then everytime you use deq, rotate it? hth Jon. From nospam at nospam.com Fri Jan 22 09:38:29 2010 From: nospam at nospam.com (Gilles Ganault) Date: Fri, 22 Jan 2010 15:38:29 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <4B59A2DB.4080809@sequans.com> Message-ID: On Fri, 22 Jan 2010 14:09:43 +0100, Jean-Michel Pichavant wrote: >Sorry, the code I provided produce this output: > >['1a', 'a', 'ac', 'av', 'b', 'c'] >['a', 'ac', 'av', 'b', 'c', '1a'] >['b', 'c', '1a', 'a', 'ac', 'av'] >['c', '1a', 'a', 'ac', 'av', 'b'] >['1a', 'a', 'ac', 'av', 'b', 'c'] > >which is actually what you are searching for. I just messed up with my >ipython shell history :o) Thanks for the help. I'm a Python newbie, and have a difficult time understanding what the [] + [] line does :-/ I'll simplify things by using a list instead of a dictionary: ============ connected = [] connected.append("0test") connected.append("aa") connected.append("bb") connected.append("cc") for start in '1abcd': result = [name for name in connected if name[0] >= start] + [name for name in connected if name[0] < start] print result ============ C:\>test.py ['aa', 'bb', 'cc', '0test'] ['aa', 'bb', 'cc', '0test'] ['bb', 'cc', '0test', 'aa'] ['cc', '0test', 'aa', 'bb'] ['0test', 'aa', 'bb', 'cc'] ============ Pretty close to what I need to do but.. 1. Why is the first iteration done twice? 2. How can I have just one line, save the character that I used as starting point, increment it, and save it into a file so it can be read the next time this program runs? For instance, let's say we used "b" to start looking for items, I'll save "c" in a file for the next time. Thank you. From alfps at start.no Fri Jan 22 09:41:05 2010 From: alfps at start.no (Alf P. Steinbach) Date: Fri, 22 Jan 2010 15:41:05 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <03698cd8$0$1357$c3e8da3@news.astraweb.com> Message-ID: * Wolfgang Rohdewald: > On Friday 22 January 2010, Alf P. Steinbach wrote: >> I get the impression that there's some message traffic that I don't >> see > >> For example, the recent thread "Covert number into string" started >> with a reply in my newreader, using EternalSeptember's NNTP host. >> >> It also starts with a reply in Google's archives, > http://groups.google.com/group/comp.lang.python/browse_thread/threa >> d/b8097d4de4a9c9b0/cb3a2e6ccd7736ef>. > > did you check your spam folder? No, I don't have one for Usenet traffic, using Thunderbird as client. > I got the original. If you want to, I can mail you the headers > privately :-) Thanks!, but it's not necessary. It was just an example that somehow *some* articles are evidently missing in the view that I have (and Google has). Perhaps there is some spam filter somewhere. And what such filters do can be rather arbitrary and completely ungrokkable. Cheers, - Alf From zuo at chopin.edu.pl Fri Jan 22 09:44:28 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Fri, 22 Jan 2010 15:44:28 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? In-Reply-To: <4lbjl5p7s67orgc91isk3ccvgp9s5m78l8@4ax.com> References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <7rtnsuFu4tU1@mid.individual.net> <4lbjl5p7s67orgc91isk3ccvgp9s5m78l8@4ax.com> Message-ID: 22-01-2010 Steven D'Aprano wrote: > On Fri, 22 Jan 2010 13:17:44 +0100, Gilles Ganault wrote: >> To avoid users from creating login names that start with digits in order >> to be listed at the top, I'd like to sort the list differently every >> minute so that it'll start with the next letter, eg. display the list >> from A...Zdigits the first time, then B...ZAdigits, etc. >> >> That way, users have no incentive to create login names that start with >> either a digit or letter A. > > If you want to prohibit users from starting their login names with a > digit, prohibit them from creating a login name with a digit. I understand that a real problem is to make all names -- regardless of characters they are made of -- having "the equal rights" in terms of visibility. >> I see that dictionaries can be sorted using the... sort() method List, not dictionaries (they have nothing to do with the matter). >> but is it possible to have Python start sorting from a different >> letter? > > You can write a customer sort routine by using the key parameter to sort, > which will probably be messy. It could be done e.g. in such a way: # (for Python 2.x) from collections import deque import string CHARS = string.ascii_lowercase + string.digits def rotated_key(rotated_chars=deque(CHARS)): rotated_chars.rotate(1) tr_table = string.maketrans(CHARS, ''.join(rotated_chars)) def key(item): return item.translate(tr_table) return key # generate some names import random users = [''.join(random.choice(CHARS) for i in xrange(3)) for j in xrange(50)] for i in xrange(50): users.sort(key=rotated_key()) print ','.join(users) print But it still doesn't guarantee real "equal rights in visibility" (holders of names starting with rarely used characters are in better situation). > What I'd do is keep 27 lists of user names, > according to the first letter (26 letters from A to Z, plus one extra): > users = [ # keep separate lists for each starting letter > ['aaron', 'avril', 'adam'], > ['betty', 'bob'], > ['craig', 'cathy'], > ['danni', 'da5id', 'donna'], > # ... and so on > ['zoe', 'zach'], > ['4dam', '1-4m-t00-c001'] > ] But here the second letter becomes important for "visibility" and users still are not equal. And all this mess in unnecessary, because there is one perfect and simple solution -- see the Neil Cerutti's post. 22-01-2010, 14:58:58 Gilles Ganault wrote: > On 22 Jan 2010 13:35:26 GMT, Neil Cerutti wrote: >> Resorting is more work than is needed. Just choose a different >> starting index each time you display the names, and set up your >> lister to wrap-around to your arbitrary starting index. > > Thanks. In this case, it means that in each loop iteration, I must > search the list to find which item starts with the letter I'd like to > begin sorting, eg. "B". Does Python include a search method or do I > have to use a for loop to locate this starting item? There is e.g. `bisect' module -- you can search as well as insert with its functions. But IMHO you shouldn't search for the next starting *letter*, but for the next *name* in the list (basing on name that was used recently). If the list were immutable, no searching would be needed (indexes would be sufficient), but in real life users can be added and deleted in the meantime (so index of a particular name changes). Regards, *j -- Jan Kaliszewski (zuo) From davea at ieee.org Fri Jan 22 09:49:32 2010 From: davea at ieee.org (Dave Angel) Date: Fri, 22 Jan 2010 09:49:32 -0500 Subject: [2.5.1.1/dictionary] Change sorting order? In-Reply-To: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: <4B59BAFC.9040401@ieee.org> Gilles Ganault wrote: > Hello > > I use a dictionary to keep a list of users connected to a web site. > > To avoid users from creating login names that start with digits in > order to be listed at the top, I'd like to sort the list differently > every minute so that it'll start with the next letter, eg. display the > list from A...Zdigits the first time, then B...ZAdigits, etc. > > That way, users have no incentive to create login names that start > with either a digit or letter A. > > I see that dictionaries can be sorted using the... sort() method, but > is it possible to have Python start sorting from a different letter? > > Thank you. > > Seems to me the other solutions I've seen so far are more complex than needed. I figure you either want an unordered list, in which case you could use random.shuffle(), or you want a list that's sorted, but starts somewhere in the middle, at an arbitrary place, goes to the end, and wraps back to the beginning. So use random.randint() to choose an index within the list, and concatenate two slices of the list, based on that index in reverse order. And don't bother generating it each minute, but simply generate it each time you need it. I doubt if the cost of generating it is much different than the cost of checking the time. I guess there's a third possibility, that you don't want some of the G names at the beginning, and some at the end. In that case, I'd generate the random index as above, then increment it till the first character of the item changes. Use that index as your split point. If you like any of these, I could elaborate with some code. But each approach is pretty straightforward. DaveA From nospam at nospam.com Fri Jan 22 10:01:28 2010 From: nospam at nospam.com (Gilles Ganault) Date: Fri, 22 Jan 2010 16:01:28 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: <04fjl5psivncp4187f2edvaougn8j0edai@4ax.com> On Fri, 22 Jan 2010 09:49:32 -0500, Dave Angel wrote: >Seems to me the other solutions I've seen so far are more complex than >needed. I figure you either want an unordered list, in which case you >could use random.shuffle(), or you want a list that's sorted, but starts >somewhere in the middle, at an arbitrary place, goes to the end, and >wraps back to the beginning. So use random.randint() to choose an index >within the list, and concatenate two slices of the list, based on that >index in reverse order. Yes, this is exactly what I need: Start listing items from a given index (actually, using a character since the list contains names) all the way to the end of the list; If the character wasn't the very first, go back to the first time and display the list until we get to the current character. Python is so feature-rich, I'm sure there's a much simpler way to do this than this crappy code of mine: ============= connected = [] connected.append("0dummy") connected.append("aa") connected.append("bb") connected.append("cc") index = 0 for item in connected: #For testing purposes; #Final code will read/increment character from file if item[index] == "b": break else: index = index + 1 #Print items between current character and end of list tempindex = index while(tempindex < len(connected)): print connected[tempindex] tempindex = tempindex + 1 #if current letter not first character, #display beginning of list up to current character if index != 0: tempindex = 0 while tempindex < index: print connected[tempindex] tempindex = tempindex + 1 ============= Thank you for any help From arnodel at googlemail.com Fri Jan 22 10:07:19 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Fri, 22 Jan 2010 15:07:19 +0000 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> Message-ID: Muhammad Alkarouri writes: > In the python help for itertools, the following function is provided: > > def consume(iterator, n): > "Advance the iterator n-steps ahead. If n is none, consume > entirely." > collections.deque(islice(iterator, n), maxlen=0) > > What is the advantage of using a collections.deque against, say, the > following code? > > def consume(iterator, n): > for _ in islice(iterator, n): pass > deque is written in C, and so is islice, so it's less work for the bytecode interpreter? -- Arnaud From arnodel at googlemail.com Fri Jan 22 10:19:44 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Fri, 22 Jan 2010 15:19:44 +0000 Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <04fjl5psivncp4187f2edvaougn8j0edai@4ax.com> Message-ID: Gilles Ganault writes: > On Fri, 22 Jan 2010 09:49:32 -0500, Dave Angel wrote: >>Seems to me the other solutions I've seen so far are more complex than >>needed. I figure you either want an unordered list, in which case you >>could use random.shuffle(), or you want a list that's sorted, but starts >>somewhere in the middle, at an arbitrary place, goes to the end, and >>wraps back to the beginning. So use random.randint() to choose an index >>within the list, and concatenate two slices of the list, based on that >>index in reverse order. > > Yes, this is exactly what I need: Start listing items from a given > index (actually, using a character since the list contains names) all > the way to the end of the list; If the character wasn't the very > first, go back to the first time and display the list until we get to > the current character. > > Python is so feature-rich, I'm sure there's a much simpler way to do > this than this crappy code of mine: > > ============= > connected = [] > connected.append("0dummy") > connected.append("aa") > connected.append("bb") > connected.append("cc") > > index = 0 > for item in connected: > #For testing purposes; > #Final code will read/increment character from file > if item[index] == "b": > break > else: > index = index + 1 > > #Print items between current character and end of list > tempindex = index > while(tempindex < len(connected)): > print connected[tempindex] > tempindex = tempindex + 1 > > #if current letter not first character, > #display beginning of list up to current character > if index != 0: > tempindex = 0 > while tempindex < index: > print connected[tempindex] > tempindex = tempindex + 1 > ============= > > Thank you for any help Here's a straightforward way to do it, taking advantage of negative indices (no doubt there are many others): >>> connected = '0dummy aa bb cc'.split() >>> connected ['0dummy', 'aa', 'bb', 'cc'] >>> startindex = random.randrange(len(connected)) >>> startindex 2 >>> for i in xrange(startindex - len(connected), startindex): ... print connected[i] ... bb cc 0dummy aa >>> connected[-1] is the last element of connected connected[-2] is the one before last etc... I'll let you figure out why the loop works as it does. HTH -- Arnaud From arnodel at googlemail.com Fri Jan 22 10:24:33 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Fri, 22 Jan 2010 15:24:33 +0000 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> Message-ID: Muhammad Alkarouri writes: > In the python help for itertools, the following function is provided: > > def consume(iterator, n): > "Advance the iterator n-steps ahead. If n is none, consume > entirely." > collections.deque(islice(iterator, n), maxlen=0) > > What is the advantage of using a collections.deque against, say, the > following code? > > def consume(iterator, n): > for _ in islice(iterator, n): pass > I remember suggesting this function (under the name of 'exhaust'!) a while ago - although I don't think there was an optional parameter for the number of elements consumed. I hadn't realised it had been deemed useful by others! -- Arnaud From duncan.booth at invalid.invalid Fri Jan 22 10:24:58 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 22 Jan 2010 15:24:58 GMT Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: Jean-Michel Pichavant wrote: > Here is one possible solution > > l = ['1a', 'a', 'b','c','av','ac'] # you mentioned a dictionary in your > post, if so, l = myDict.keys() > l.sort() # sort your list once and for all > for start in '1abcd': > result = [name for name in l if name[0] >= start] + [name for name > in l if name[0] < start] > print result Here's another: >>> import bisect >>> def rotated_sort(data, startch): data = sorted(data) pos = bisect.bisect_left(data, startch) return data[pos:] + data[:pos] >>> for ch in " 1abcd": print ch, rotated_sort(['1a', 'a', 'b','c','av','ac'], ch) ['1a', 'a', 'ac', 'av', 'b', 'c'] 1 ['1a', 'a', 'ac', 'av', 'b', 'c'] a ['a', 'ac', 'av', 'b', 'c', '1a'] b ['b', 'c', '1a', 'a', 'ac', 'av'] c ['c', '1a', 'a', 'ac', 'av', 'b'] d ['1a', 'a', 'ac', 'av', 'b', 'c'] >>> -- Duncan Booth http://kupuguy.blogspot.com From zuo at chopin.edu.pl Fri Jan 22 10:25:33 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Fri, 22 Jan 2010 16:25:33 +0100 Subject: Consume an iterable In-Reply-To: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> Message-ID: > In the python help for itertools, the following function is provided: > > def consume(iterator, n): > "Advance the iterator n-steps ahead. If n is none, consume > entirely." > collections.deque(islice(iterator, n), maxlen=0) > > What is the advantage of using a collections.deque against, say, the > following code? > > def consume(iterator, n): > for _ in islice(iterator, n): pass Probably the former is faster. But I haven't check it. If you are curious, use timeit module... Regards, *j -- Jan Kaliszewski (zuo) From zuo at chopin.edu.pl Fri Jan 22 10:34:11 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Fri, 22 Jan 2010 16:34:11 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? In-Reply-To: References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <7rtnsuFu4tU1@mid.individual.net> <4lbjl5p7s67orgc91isk3ccvgp9s5m78l8@4ax.com> Message-ID: PS. 22-01-2010 o 15:44:28 Jan Kaliszewski wrote: > 22-01-2010, 14:58:58 Gilles Ganault wrote: > >> On 22 Jan 2010 13:35:26 GMT, Neil Cerutti wrote: >>> Resorting is more work than is needed. Just choose a different >>> starting index each time you display the names, and set up your >>> lister to wrap-around to your arbitrary starting index. >> >> Thanks. In this case, it means that in each loop iteration, I must >> search the list to find which item starts with the letter I'd like to >> begin sorting, eg. "B". Does Python include a search method or do I >> have to use a for loop to locate this starting item? > > There is e.g. `bisect' module -- you can search as well as insert with > its functions. But IMHO you shouldn't search for the next starting > *letter*, but for the next *name* in the list (basing on name that was > used recently). Or simply choose the starting index randomly -- using random.randrange(len(userlist)) or random.randint(0, len(userlist)-1) -- and use bisect.insort() for inserting to the list keeping it sorted (if you need to list users sorted+rotated). Regards, *j From steve at REMOVE-THIS-cybersource.com.au Fri Jan 22 10:48:41 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 15:48:41 GMT Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <7rtnsuFu4tU1@mid.individual.net> Message-ID: <0369b563$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 13:35:26 +0000, Neil Cerutti wrote: > On 2010-01-22, Gilles Ganault wrote: >> Hello >> >> I use a dictionary to keep a list of users connected to a web site. >> >> To avoid users from creating login names that start with digits in >> order to be listed at the top, I'd like to sort the list differently >> every minute so that it'll start with the next letter, eg. display the >> list from A...Zdigits the first time, then B...ZAdigits, etc. > > Resorting is more work than is needed. Just choose a different starting > index each time you display the names, and set up your lister to > wrap-around to your arbitrary starting index. The original poster's requirement is to start at a new *letter* each time, not whatever name happens to be at index N. Unless you can predict what index to use for (say) names starting with "B", then your scheme doesn't work. In order to find that index, you have to do a linear search of the list after every sort, turning sorting into O(N**2) instead of O(N*log N). -- Steven From neilc at norwich.edu Fri Jan 22 10:57:07 2010 From: neilc at norwich.edu (Neil Cerutti) Date: 22 Jan 2010 15:57:07 GMT Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <7rtnsuFu4tU1@mid.individual.net> <0369b563$0$1357$c3e8da3@news.astraweb.com> Message-ID: <7ru06jFo6hU1@mid.individual.net> On 2010-01-22, Steven D'Aprano wrote: > On Fri, 22 Jan 2010 13:35:26 +0000, Neil Cerutti wrote: >> On 2010-01-22, Gilles Ganault wrote: >>> Hello >>> >>> I use a dictionary to keep a list of users connected to a web site. >>> >>> To avoid users from creating login names that start with digits in >>> order to be listed at the top, I'd like to sort the list differently >>> every minute so that it'll start with the next letter, eg. display the >>> list from A...Zdigits the first time, then B...ZAdigits, etc. >> >> Resorting is more work than is needed. Just choose a different starting >> index each time you display the names, and set up your lister to >> wrap-around to your arbitrary starting index. > > The original poster's requirement is to start at a new *letter* > each time, not whatever name happens to be at index N. > > Unless you can predict what index to use for (say) names > starting with "B", then your scheme doesn't work. In order to > find that index, you have to do a linear search of the list > after every sort, turning sorting into O(N**2) instead of > O(N*log N). O(N*Log N) + O(N) == O(N**2)? Besides, the idea was to avoid sorting altogether. -- Neil Cerutti From albert at spenarnc.xs4all.nl Fri Jan 22 11:00:54 2010 From: albert at spenarnc.xs4all.nl (Albert van der Horst) Date: 22 Jan 2010 16:00:54 GMT Subject: Writing a string.ishex function References: <5f48d4ce-8985-4188-828d-0fdf57600659@g25g2000yqd.googlegroups.com> <20100114112254.eed71352.darcy@druid.net> Message-ID: In article , MRAB wrote: >D'Arcy J.M. Cain wrote: >> On Thu, 14 Jan 2010 07:52:58 -0800 (PST) >> chandra wrote: >>> Folks, >>> >>> I am new to Python and could not find a function along the lines of >> >> Welcome. >> >>> string.ishex in Python. There is however, a string.hexdigits constant >>> in the string module. I thought I would enhance the existing modlue >>> but am unsure how I should go about it. Specifically, I have attempted >>> this much: >> >> You should always test code before posting and post the exact code that >> you tested. >> >>> ---cut--- >>> #! /usr/bin/python >>> # -*- coding: utf-8 -*- >>> >>> import string >>> >>> def ishex(string): >> >> Bad idea to name your variable after a module. This function fails >> because of that. >> >>> ishex = False >>> for i in strdef ishex(sing: >>> if i in string.hexdigits: >>> ishex = True >>> else: >>> ishex = False >>> break >>> return ishex >> >> After renaming the variable this works but you can simplify it. >> >> >>> ---cut--- >> >> Just return False once you find a non-hex digit. >> >> def ishex(s): >> for c in s: >> if not c in string.hexdigits: return False >> >> return True >> >> And here are your unit tests. Every line should print "True". >> >> print ishex('123') is True >> print ishex('abc') is True >> print ishex('xyz') is False >> print ishex('0123456789abcdefABCDEF') is True >> print ishex('0123456789abcdefABCDEFG') is False >> >Don't use 'is', use '=='. > >BTW, ishex('') should return False. You are very wrong. Not with the above statement, but the very act of issuing a statement like that is wrong. The OP didn't specify ishex(). In absence of a specification, border cases are not defined. If the specification was: any character of string s must be a hex character then ishex('') should return True. If the specification was: the string must represent a hex number then ishex('') should probably return False. I can imagine a specification where it is appropriate to throw an exception for an empty string. This is also the safest thing to do, if there is the slightest hesitation. So the correct behaviour is: "Please mister customer, what exactly did you have in mind?" Groetjes Albert -- -- Albert van der Horst, UTRECHT,THE NETHERLANDS Economic growth -- being exponential -- ultimately falters. albert at spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst From jeanmichel at sequans.com Fri Jan 22 11:02:45 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Fri, 22 Jan 2010 17:02:45 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? In-Reply-To: References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <4B59A2DB.4080809@sequans.com> Message-ID: <4B59CC25.7010302@sequans.com> Gilles Ganault wrote: > On Fri, 22 Jan 2010 14:09:43 +0100, Jean-Michel Pichavant > wrote: > >> Sorry, the code I provided produce this output: >> >> ['1a', 'a', 'ac', 'av', 'b', 'c'] >> ['a', 'ac', 'av', 'b', 'c', '1a'] >> ['b', 'c', '1a', 'a', 'ac', 'av'] >> ['c', '1a', 'a', 'ac', 'av', 'b'] >> ['1a', 'a', 'ac', 'av', 'b', 'c'] >> >> which is actually what you are searching for. I just messed up with my >> ipython shell history :o) >> > > Thanks for the help. I'm a Python newbie, and have a difficult time > understanding what the [] + [] line does :-/ > > I'll simplify things by using a list instead of a dictionary: > > ============ > connected = [] > connected.append("0test") > connected.append("aa") > connected.append("bb") > connected.append("cc") > > for start in '1abcd': > result = [name for name in connected if name[0] >= start] + [name > for name in connected if name[0] < start] > print result > ============ > C:\>test.py > ['aa', 'bb', 'cc', '0test'] > ['aa', 'bb', 'cc', '0test'] > ['bb', 'cc', '0test', 'aa'] > ['cc', '0test', 'aa', 'bb'] > ['0test', 'aa', 'bb', 'cc'] > ============ > > Pretty close to what I need to do but.. > > 1. Why is the first iteration done twice? > > 2. How can I have just one line, save the character that I used as > starting point, increment it, and save it into a file so it can be > read the next time this program runs? For instance, let's say we used > "b" to start looking for items, I'll save "c" in a file for the next > time. > > Thank you. > 1/ [] + [] is using 2 python lists comprehension, Google it for details. It is quite difficult to read until you become familiar with it. Once you get it, you can write magical stuff :o) Basically, list comrehension allows to map functions to list elements and / or filter those elements. So what I'm using is the filter feature of list comprehension: [list of names for which the first char is greater that 'start'] + [list of names for which the first char is less than 'start'] 2/ Are you sure you want to do that ? looks like you are using a hammer to smash a fly. As someone has suggested before, maybe you want to pickup some starting index random. connected = ['aa', 'bb', 'cc', '0test'] import random def getNewOrder(myList): index = random.randint(0,len(myList)-1) print index return myList[index:] + myList[:index] # using slicing instead of list comprehension (suggested by DaveA) print getNewOrder(connected) JM From aahz at pythoncraft.com Fri Jan 22 11:03:32 2010 From: aahz at pythoncraft.com (Aahz) Date: 22 Jan 2010 08:03:32 -0800 Subject: maintain 2 versions of python on my computer References: <88886fe9-31ee-4ecc-9109-b645ceb46260@c34g2000yqn.googlegroups.com> <7a1f55ee-4c12-4f56-989a-abcd1c441365@j4g2000yqe.googlegroups.com> Message-ID: In article , Duncan Booth wrote: > >That seems overkill. This does pretty much the same thing: > > @(C:\Python26\Python -x %~f0 %* || pause) && goto:EOF > import sys > print sys.version > # raise RuntimeError # uncomment to trigger the 'pause' What version of Windows is necessary to use this? Anything with cmd.exe? (Sure doesn't look anything like DOS batch files...) -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From alfps at start.no Fri Jan 22 11:15:39 2010 From: alfps at start.no (Alf P. Steinbach) Date: Fri, 22 Jan 2010 17:15:39 +0100 Subject: maintain 2 versions of python on my computer In-Reply-To: References: <88886fe9-31ee-4ecc-9109-b645ceb46260@c34g2000yqn.googlegroups.com> <7a1f55ee-4c12-4f56-989a-abcd1c441365@j4g2000yqe.googlegroups.com> Message-ID: * Aahz: > In article , > Duncan Booth wrote: >> That seems overkill. This does pretty much the same thing: >> >> @(C:\Python26\Python -x %~f0 %* || pause) && goto:EOF >> import sys >> print sys.version >> # raise RuntimeError # uncomment to trigger the 'pause' > > What version of Windows is necessary to use this? Anything with > cmd.exe? (Sure doesn't look anything like DOS batch files...) Well, all of that has been there since, I don't know, long ago... -x A Python option that skips the first line of the script. @ suppresses echo of the line. () group commands (creates a compound command) || shortcut-evaluated OR && shortcut-evaluated AND % see below... C:\test> for /? | find "%~" %~I - expands %I removing any surrounding quotes (") %~fI - expands %I to a fully qualified path name %~dI - expands %I to a drive letter only %~pI - expands %I to a path only %~nI - expands %I to a file name only %~xI - expands %I to a file extension only %~sI - expanded path contains short names only %~aI - expands %I to file attributes of file %~tI - expands %I to date/time of file %~zI - expands %I to size of file %~$PATH:I - searches the directories listed in the PATH %~dpI - expands %I to a drive letter and path only %~nxI - expands %I to a file name and extension only %~fsI - expands %I to a full path name with short names only %~dp$PATH:I - searches the directories listed in the PATH %~ftzaI - expands %I to a DIR like output line values. The %~ syntax is terminated by a valid FOR variable name. C:\test> _ Cheers & hth. (even if a bit off-topic!), - Alf From steve at REMOVE-THIS-cybersource.com.au Fri Jan 22 11:17:12 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 16:17:12 GMT Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: <0369bc12$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 09:49:32 -0500, Dave Angel wrote: > Seems to me the other solutions I've seen so far are more complex than > needed. I figure you either want an unordered list, in which case you > could use random.shuffle(), or you want a list that's sorted, but starts > somewhere in the middle, at an arbitrary place, goes to the end, and > wraps back to the beginning. So use random.randint() to choose an index > within the list, and concatenate two slices of the list, based on that > index in reverse order. > > And don't bother generating it each minute, but simply generate it each > time you need it. According to the OP's stated requirements, he needs it every minute. Personally, from a usability perspective, I think a refresh of a (potentially huge) list of users every minute would be horrible. I think the whole design needs to be rethought, but that's not my decision. > I doubt if the cost of generating it is much > different than the cost of checking the time. Checking the time is quite fast -- it's a simple system call. Slicing and copying a list containing potentially tens or hundreds of thousands of names is anything but cheap! >>> from timeit import Timer >>> t1 = Timer('x = time.time()', 'import time') >>> t2 = Timer('y = x[:1000] + x[1000:]', 'x = [None]*100000') >>> >>> t1.timeit(1000) 0.0025529861450195312 >>> t2.timeit(1000) 3.0900199413299561 Slicing is more than 1000 times slower than checking the time. If you think I'm being unfair, that the OP's application will never have 100,000 names, try it with 1000 names: >>> t3 = Timer('y = x[:1000] + x[1000:]', 'x = [None]*1000') >>> t3.timeit(1000) 0.041487932205200195 That's 20 times slower than checking the time. That's the advantage of my suggestion: the only slicing done is the tiny master list, which means copying 27 pointers. Easy and fast, and it doesn't matter whether you have one user or one million, that operation will take exactly the same amount of time. Your suggestion to keep all the users in one giant list, and slice it repeatedly, scales terribly. > I guess there's a third possibility, that you don't want some of the G > names at the beginning, and some at the end. In that case, I'd generate > the random index as above, then increment it till the first character of > the item changes. Use that index as your split point. Another solution that doesn't scale terrible well. (Although not as bad as the previous one.) And what's with the random index thing? Why are you throwing away perfectly good information every time you want to shift letters? Your solution is something like this: You have a book opened at the start of Chapter 11, and you now want to jump to the start of Chapter 12. (1) Open the book at a random page. (2) Is it the start of Chapter 12? If yes, you're done. If no, turn to the next page. If you reach the end of the book, start again at the beginning. (3) Go to step 2. There are search techniques that start with a random index, but they don't advance one position at a time! (E.g. binary search, or hunt-and- bisect search, or similar.) But why search each time when you can keep an index to the start of each chapter and jump straight there in constant time? -- Steven From jeanmichel at sequans.com Fri Jan 22 11:21:02 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Fri, 22 Jan 2010 17:21:02 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? In-Reply-To: <4B59CC25.7010302@sequans.com> References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <4B59A2DB.4080809@sequans.com> <4B59CC25.7010302@sequans.com> Message-ID: <4B59D06E.3070406@sequans.com> Jean-Michel Pichavant wrote: > Gilles Ganault wrote: >> On Fri, 22 Jan 2010 14:09:43 +0100, Jean-Michel Pichavant >> wrote: >> >>> Sorry, the code I provided produce this output: >>> >>> ['1a', 'a', 'ac', 'av', 'b', 'c'] >>> ['a', 'ac', 'av', 'b', 'c', '1a'] >>> ['b', 'c', '1a', 'a', 'ac', 'av'] >>> ['c', '1a', 'a', 'ac', 'av', 'b'] >>> ['1a', 'a', 'ac', 'av', 'b', 'c'] >>> >>> which is actually what you are searching for. I just messed up with >>> my ipython shell history :o) >>> >> >> Thanks for the help. I'm a Python newbie, and have a difficult time >> understanding what the [] + [] line does :-/ >> >> I'll simplify things by using a list instead of a dictionary: >> >> ============ >> connected = [] >> connected.append("0test") >> connected.append("aa") >> connected.append("bb") >> connected.append("cc") >> >> for start in '1abcd': >> result = [name for name in connected if name[0] >= start] + [name >> for name in connected if name[0] < start] >> print result >> ============ >> C:\>test.py >> ['aa', 'bb', 'cc', '0test'] >> ['aa', 'bb', 'cc', '0test'] >> ['bb', 'cc', '0test', 'aa'] >> ['cc', '0test', 'aa', 'bb'] >> ['0test', 'aa', 'bb', 'cc'] >> ============ >> >> Pretty close to what I need to do but.. >> >> 1. Why is the first iteration done twice? >> >> 2. How can I have just one line, save the character that I used as >> starting point, increment it, and save it into a file so it can be >> read the next time this program runs? For instance, let's say we used >> "b" to start looking for items, I'll save "c" in a file for the next >> time. >> >> Thank you. >> > > 1/ [] + [] is using 2 python lists comprehension, Google it for > details. It is quite difficult to read until you become familiar with > it. Once you get it, you can write magical stuff :o) > Basically, list comrehension allows to map functions to list elements > and / or filter those elements. > > So what I'm using is the filter feature of list comprehension: > [list of names for which the first char is greater that 'start'] + > [list of names for which the first char is less than 'start'] > > 2/ Are you sure you want to do that ? looks like you are using a > hammer to smash a fly. As someone has suggested before, maybe you want > to pickup some starting index random. > > connected = ['aa', 'bb', 'cc', '0test'] > > import random > > def getNewOrder(myList): > index = random.randint(0,len(myList)-1) > print index > return myList[index:] + myList[:index] # using slicing instead of > list comprehension (suggested by DaveA) > > print getNewOrder(connected) > > JM > > Ok I realized that picking up a random index prevent from grouping names starting with the same letter (to ease visual lookup). Then go for the random char, and use char comparison (my first example). JM From steve at REMOVE-THIS-cybersource.com.au Fri Jan 22 11:27:21 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 16:27:21 GMT Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <7rtnsuFu4tU1@mid.individual.net> <0369b563$0$1357$c3e8da3@news.astraweb.com> <7ru06jFo6hU1@mid.individual.net> Message-ID: <0369be73$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 15:57:07 +0000, Neil Cerutti wrote: > On 2010-01-22, Steven D'Aprano > wrote: >> Unless you can predict what index to use for (say) names starting with >> "B", then your scheme doesn't work. In order to find that index, you >> have to do a linear search of the list after every sort, turning >> sorting into O(N**2) instead of O(N*log N). > > O(N*Log N) + O(N) == O(N**2)? Oops! :( Of course, the sort is in fast C, and the linear search is in relatively slow Python, so it is quite conceivable that for realistic amounts of data, the time could be dominated by the searching. Or the OP might just change his requirements and allow starting the display in the middle of the letter. > Besides, the idea was to avoid sorting altogether. I don't see why you think that's necessary. Python's sort is very fast when the list is nearly sorted, so if you re-sort after any addition of a username, it might even be faster than trying to keep the list sorted all the time. That's the sort of thing that we'd need to do some timing tests to see which was faster. -- Steven From nobody at nowhere.com Fri Jan 22 11:30:54 2010 From: nobody at nowhere.com (Nobody) Date: Fri, 22 Jan 2010 16:30:54 +0000 Subject: subprocess troubles References: Message-ID: On Thu, 21 Jan 2010 11:25:08 +0100, Tomas Pelka wrote: > have a problem with following piece of code: > > -------------------------------------------------- > import subprocess > > paattern = "python" > cmd = "/usr/bin/locate" > arg1 = " -i" > arg2 = " -d /var/www/books/mlocate.db" > arg3 = str(" " + pattern) > p1 = subprocess.Popen([cmd, arg1, arg2, arg3], shell=False, > stdout=subprocess.PIPE, stderr=subprocess.PIPE) You probably want: arg1 = "-i" # no leading space arg2 = "-d" # no leading space arg3 = "/var/www/books/mlocate.db" arg4 = pattern p1 = subprocess.Popen([cmd, arg1, arg2, arg3, arg4], ... > If I run this command > (/usr/bin/locate -i -d /var/www/books/mlocate.db python) from standard > shell everything goes fine. > > Could you please give me an advice what I'm doing wrong? You're putting spaces where they aren't wanted. From neilc at norwich.edu Fri Jan 22 11:52:05 2010 From: neilc at norwich.edu (Neil Cerutti) Date: 22 Jan 2010 16:52:05 GMT Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <7rtnsuFu4tU1@mid.individual.net> <0369b563$0$1357$c3e8da3@news.astraweb.com> <7ru06jFo6hU1@mid.individual.net> <0369be73$0$1357$c3e8da3@news.astraweb.com> Message-ID: <7ru3dlFb8kU1@mid.individual.net> On 2010-01-22, Steven D'Aprano wrote: >> O(N*Log N) + O(N) == O(N**2)? > > Oops! :( > > Of course, the sort is in fast C, and the linear search is in > relatively slow Python, so it is quite conceivable that for > realistic amounts of data, the time could be dominated by the > searching. > > Or the OP might just change his requirements and allow starting > the display in the middle of the letter. That's what I would advocate. The scheme of starting at a random (or cycling through) first letters still grants an arbitrary advantage to some user names. A random starting position is thus more fair *and* more efficient. ;) -- Neil Cerutti From davea at ieee.org Fri Jan 22 11:54:58 2010 From: davea at ieee.org (Dave Angel) Date: Fri, 22 Jan 2010 11:54:58 -0500 Subject: [2.5.1.1/dictionary] Change sorting order? In-Reply-To: <04fjl5psivncp4187f2edvaougn8j0edai@4ax.com> References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <04fjl5psivncp4187f2edvaougn8j0edai@4ax.com> Message-ID: <4B59D862.3000207@ieee.org> Gilles Ganault wrote: > On Fri, 22 Jan 2010 09:49:32 -0500, Dave Angel wrote: > >> Seems to me the other solutions I've seen so far are more complex than >> needed. I figure you either want an unordered list, in which case you >> could use random.shuffle(), or you want a list that's sorted, but starts >> somewhere in the middle, at an arbitrary place, goes to the end, and >> wraps back to the beginning. So use random.randint() to choose an index >> within the list, and concatenate two slices of the list, based on that >> index in reverse order. >> > > Yes, this is exactly what I need: Start listing items from a given > index (actually, using a character since the list contains names) all > the way to the end of the list; If the character wasn't the very > first, go back to the first time and display the list until we get to > the current character. > > Python is so feature-rich, I'm sure there's a much simpler way to do > this than this crappy code of mine: > > ============= > connected = [] > connected.append("0dummy") > connected.append("aa") > connected.append("bb") > connected.append("cc") > > index = 0 > for item in connected: > #For testing purposes; > #Final code will read/increment character from file > if item[index] == "b": > break > else: > index = index + 1 > > #Print items between current character and end of list > tempindex = index > while(tempindex < len(connected)): > print connected[tempindex] > tempindex = tempindex + 1 > > #if current letter not first character, > #display beginning of list up to current character > if index != 0: > tempindex = 0 > while tempindex < index: > print connected[tempindex] > tempindex = tempindex + 1 > ============= > > Thank you for any help > > Your code would fail if you picked a letter for which there were no entries. That's easily fixed, just use >= instead of == in your if test. Jean-Michel Pichavant post implements my second option. So I'll borrow his code here. But apparently you want my third option: random starting place, but with all the names starting with the same character together. To get that, but with the distribution proportional to the number of names that start with the given character, you need to get the index in two stages. First use the random.randint(). Then do a search for the first name that starts with that same character. For that, I borrow from Duncan Booth, and use bisect.bisect(). import random, bisect connected = sorted( ['jj', '0test', '3test', 'aa', 'aj', 'bb', 'bz', 'dc', 'df'] ) def getNewOrder(myList): index = random.randint(0,len(myList)-1) startch = myList[index][0] pos = bisect.bisect_left(myList, startch) print index, myList[index], pos, myList[pos] return myList[pos:] + myList[:pos] # using slicing instead of list comprehension (suggested by DaveA) print connected for i in range(6): print getNewOrder(connected) I have the extra print inside the function to show (for example) that if 'aj' is picked, it actually will start with'aa' Note that with your constraints, there's no way it can be fair. But experiment a bit, and see if it meets your needs. DaveA From duncan.booth at invalid.invalid Fri Jan 22 12:15:28 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 22 Jan 2010 17:15:28 GMT Subject: maintain 2 versions of python on my computer References: <88886fe9-31ee-4ecc-9109-b645ceb46260@c34g2000yqn.googlegroups.com> <7a1f55ee-4c12-4f56-989a-abcd1c441365@j4g2000yqe.googlegroups.com> Message-ID: aahz at pythoncraft.com (Aahz) wrote: > In article , > Duncan Booth wrote: >> >>That seems overkill. This does pretty much the same thing: >> >> @(C:\Python26\Python -x %~f0 %* || pause) && goto:EOF >> import sys >> print sys.version >> # raise RuntimeError # uncomment to trigger the 'pause' > > What version of Windows is necessary to use this? Anything with > cmd.exe? (Sure doesn't look anything like DOS batch files...) Windows NT, Windows 2000, Windows XP, Windows Vista, Windows 7, Server 2003, Server 2008 will all handle the && and || stuff. I didn't think that NT recognised the goto:EOF as special but a quick search with Google seems to indicate that it did. If so this script probably worked since 1993.[*] Windows 7 still supports this cmd.exe syntax, but it also has Microsoft Powershell which seems to be Microsoft's latest answer to shell scripts and allows direct access to COM objects and .Net classes. [*] or at least would have worked if Guido used his time machine to take a copy of Python 2.6 back to 1993. :^) From nospam at nospam.com Fri Jan 22 12:43:00 2010 From: nospam at nospam.com (Gilles Ganault) Date: Fri, 22 Jan 2010 18:43:00 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: On 22 Jan 2010 15:24:58 GMT, Duncan Booth wrote: >Here's another: Thanks for the sample. It work great, except that it also runs when the header character doesn't match any item in the list: ======= import bisect connected = [] connected.append("_test") connected.append("-test") connected.append("0test") connected.append("1test") connected.append("Aa") connected.append("Bb") connected.append("Cc") def rotated_sort(data, startch): data = sorted(data) pos = bisect.bisect_left(data, startch) return data[pos:] + data[:pos] for ch in "_-0123456789ABCDEFGHIJKLMNOPQRSTUVWYZ": print ch, rotated_sort(connected, ch) ======= C:\>test.py _ ['_test', '-test', '0test', '1test', 'Aa', 'Bb', 'Cc'] - ['-test', '0test', '1test', 'Aa', 'Bb', 'Cc', '_test'] 0 ['0test', '1test', 'Aa', 'Bb', 'Cc', '_test', '-test'] 1 ['1test', 'Aa', 'Bb', 'Cc', '_test', '-test', '0test'] 2 ['Aa', 'Bb', 'Cc', '_test', '-test', '0test', '1test'] 3 ['Aa', 'Bb', 'Cc', '_test', '-test', '0test', '1test'] 4 ['Aa', 'Bb', 'Cc', '_test', '-test', '0test', '1test'] ======= Should I add an "if" block in the "for ch in"? From nospam at nospam.com Fri Jan 22 12:44:08 2010 From: nospam at nospam.com (Gilles Ganault) Date: Fri, 22 Jan 2010 18:44:08 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <4B59A2DB.4080809@sequans.com> <4B59CC25.7010302@sequans.com> Message-ID: On Fri, 22 Jan 2010 17:21:02 +0100, Jean-Michel Pichavant wrote: >Ok I realized that picking up a random index prevent from grouping names >starting with the same letter (to ease visual lookup). >Then go for the random char, and use char comparison (my first example). Yup, I think it's a good enough solution. Thanks much for the help. From ng1.ben at gmail.com Fri Jan 22 12:57:52 2010 From: ng1.ben at gmail.com (ben) Date: Fri, 22 Jan 2010 09:57:52 -0800 (PST) Subject: * operator in python tutorial References: <7b2ef0aa-8b93-4cb5-b944-beea3e2c98b1@p16g2000vbe.googlegroups.com> <0036df65$0$25085$c3e8da3@news.astraweb.com> Message-ID: On Jan 20, 8:30?pm, Gringo wrote: > On 1/20/2010 12:38, ben wrote: > > > > > Hello, > > > I am following through the python tutorial which gets to a line that > > uses the * operator with zip(). I searched and searched but could find > > no information on the operator or how to use it in general. The > > example from the tut is as follows: > >>>> x = [1, 2, 3] > >>>> y = [4, 5, 6] > >>>> zipped = zip(x, y) > >>>> zipped > > [(1, 4), (2, 5), (3, 6)] > >>>> x2, y2 = zip(*zipped) > >>>> x == list(x2) and y == list(y2) > > True > > > How would i apply the * operator in general? > > Thanks. > > The * operator, when used in this context, unpacks the sequence and it's > as if you passed each item to the function as a different parameter. > For example, if you have a list x with 4 items, these two statements > would be the same: > f(x[0],x[1],x[2],x[3]) > f(*x) > > Hope this helps. It does. Thanks. From davea at ieee.org Fri Jan 22 13:40:26 2010 From: davea at ieee.org (Dave Angel) Date: Fri, 22 Jan 2010 13:40:26 -0500 Subject: [2.5.1.1/dictionary] Change sorting order? In-Reply-To: <0369bc12$0$1357$c3e8da3@news.astraweb.com> References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <0369bc12$0$1357$c3e8da3@news.astraweb.com> Message-ID: <4B59F11A.7030401@ieee.org> Steven D'Aprano wrote: > On Fri, 22 Jan 2010 09:49:32 -0500, Dave Angel wrote: > > >> Seems to me the other solutions I've seen so far are more complex than >> needed. I figure you either want an unordered list, in which case you >> could use random.shuffle(), or you want a list that's sorted, but starts >> somewhere in the middle, at an arbitrary place, goes to the end, and >> wraps back to the beginning. So use random.randint() to choose an index >> within the list, and concatenate two slices of the list, based on that >> index in reverse order. >> >> And don't bother generating it each minute, but simply generate it each >> time you need it. >> > > According to the OP's stated requirements, he needs it every minute. > > I read it as he wants viewers of the page that are more than a minute apart to see a different order. > Personally, from a usability perspective, I think a refresh of a > (potentially huge) list of users every minute would be horrible. I think > the whole design needs to be rethought, but that's not my decision. > > > Certainly if there are more than 100 users or so, the idea of presenting a web page with a single list of users is unwieldy. So I assumed this problem did not have to scale. Once the list goes on multiple pages, entirely different algorithms are appropriate, along with the display mechanisms to support them. >> I doubt if the cost of generating it is much >> different than the cost of checking the time. >> > > Checking the time is quite fast -- it's a simple system call. Slicing and > copying a list containing potentially tens or hundreds of thousands of > names is anything but cheap! > > >>>> from timeit import Timer >>>> t1 = Timer('x = time.time()', 'import time') >>>> t2 = Timer('y = x[:1000] + x[1000:]', 'x = [None]*100000') >>>> >>>> t1.timeit(1000) >>>> > 0.0025529861450195312 > >>>> t2.timeit(1000) >>>> > 3.0900199413299561 > > Slicing is more than 1000 times slower than checking the time. If you > think I'm being unfair, that the OP's application will never have 100,000 > names, try it with 1000 names: > > >>>> t3 = Timer('y = x[:1000] + x[1000:]', 'x = [None]*1000') >>>> t3.timeit(1000) >>>> > 0.041487932205200195 > > That's 20 times slower than checking the time. > > That's the advantage of my suggestion: the only slicing done is the tiny > master list, which means copying 27 pointers. Easy and fast, and it > doesn't matter whether you have one user or one million, that operation > will take exactly the same amount of time. Your suggestion to keep all > the users in one giant list, and slice it repeatedly, scales terribly. > > > >> I guess there's a third possibility, that you don't want some of the G >> names at the beginning, and some at the end. In that case, I'd generate >> the random index as above, then increment it till the first character of >> the item changes. Use that index as your split point. >> > > Another solution that doesn't scale terrible well. (Although not as bad > as the previous one.) > > And what's with the random index thing? Why are you throwing away > perfectly good information every time you want to shift letters? Your > solution is something like this: > > You have a book opened at the start of Chapter 11, and you now want to > jump to the start of Chapter 12. > > (1) Open the book at a random page. > (2) Is it the start of Chapter 12? > Wrong. We're checking whether it's the start of whatever the next chapter is, so if we randomly jumped into chapter 5, we'd be going a page at a time till chapter 6. After I thought about it, I realized we should be going backwards, not forward, but the idea is the same. We're only walking through a half chapter, typically. > If yes, you're done. > If no, turn to the next page. If you reach the end of > the book, start again at the beginning. > (3) Go to step 2. > > There are search techniques that start with a random index, but they > don't advance one position at a time! (E.g. binary search, or hunt-and- > bisect search, or similar.) But why search each time when you can keep an > index to the start of each chapter and jump straight there in constant > time? > > First, the time that will be spent turning this list into a web page will far exceed any of these numbers. And if we assume he's using CGI, then the time spent reloading and unmarshaling the list will exceed by even more. Next, when it comes to an actual implementation, I used bisect(), which is O(logn). And finally, it was more important to get the concepts together, as the spec wasn't well-thought out, than it was to get some optimal solution. That's why I used slice instead of doing the index-len() trick. In real code, he'll be looping over the list, calling some logic to add

and

for example. By running that loop from index-len() to index, it can operate on the original list. In my opinion, the only "fair" answer is to choose the starting point randomly, or sequentially, but without regard to starting letter. And random has the distinct advantage of not needing to store state between CGI invocations. The rest are premature optimizations, in my opinion. But once a final spec is decided, if the letters wanted to be equally weighted, then a dictionary of sorted lists might very well be the best storage mechanism. And once each starting letter gets its own web page, it also makes sense. DaveA From phlip2005 at gmail.com Fri Jan 22 13:51:22 2010 From: phlip2005 at gmail.com (Phlip) Date: Fri, 22 Jan 2010 10:51:22 -0800 (PST) Subject: counting lines of code References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> <87bpgodj95.fsf@benfinney.id.au> <1f166703-fdfc-4f87-b336-ef80e492635b@a32g2000yqm.googlegroups.com> <7e09df6a-cda1-480e-a971-8f8a70ac4118@b9g2000yqd.googlegroups.com> <980775cc-9101-4673-af03-fc17914d63f8@c4g2000yqa.googlegroups.com> Message-ID: <0d54eff2-59f0-4d95-aafb-478c9041c1b9@p8g2000yqb.googlegroups.com> On Jan 21, 9:00?pm, Michele Simionato wrote: > Just for fun I have run cloc on our trunk: > > SUM: ? ? ? ? ? ? ? ?8743 ? ?272238 ? ?215871 ? 1470139 x ? 1.84 = > 2708354.95 Nice! My favorite version of a cloc system can distinguish test from production code. That's why I always use executable cloc to measure the ratio of test to production code (where 1.2:1 is almost comfortable an 2:1 is sacred). Just so long as nobody confuses "more lines of code!" with progress... -- Phlip From python at rcn.com Fri Jan 22 14:11:21 2010 From: python at rcn.com (Raymond Hettinger) Date: Fri, 22 Jan 2010 11:11:21 -0800 (PST) Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> Message-ID: <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> On Jan 22, 6:13?am, Muhammad Alkarouri wrote: > In the python help for itertools, the following function is provided: > > def consume(iterator, n): > ? ? "Advance the iterator n-steps ahead. If n is none, consume > entirely." > ? ? collections.deque(islice(iterator, n), maxlen=0) > > What is the advantage of using a collections.deque against, say, the > following code? > > def consume(iterator, n): > ? ? for _ in islice(iterator, n): pass The deque version is faster. Raymond From showell30 at yahoo.com Fri Jan 22 14:14:32 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 11:14:32 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue Message-ID: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> The v2.6.4 version of the tutorial says this: ''' It is also possible to use a list as a queue, where the first element added is the first element retrieved (?first-in, first-out?); however, lists are not efficient for this purpose. While appends and pops from the end of list are fast, doing inserts or pops from the beginning of a list is slow (because all of the other elements have to be shifted by one). ''' Is that really true in CPython? It seems like you could advance the pointer instead of shifting all the elements. It would create some nuances with respect to reclaiming the memory, but it seems like an easy way to make lists perform better under a pretty reasonable use case. Does anybody know off the top of their head if the "have-to-be-shifted- by-one" warning is actually valid? http://docs.python.org/tutorial/datastructures.html From Martin.Drautzburg at web.de Fri Jan 22 14:16:31 2010 From: Martin.Drautzburg at web.de (Martin Drautzburg) Date: Fri, 22 Jan 2010 20:16:31 +0100 Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> <065e450f-2190-42f4-b9c6-4b95260092be@f12g2000yqn.googlegroups.com> Message-ID: <28103200.7M7XYOBnsK@beaureve.gmx.net> Mark Dickinson wrote: > On Jan 21, 10:57?pm, Martin Drautzburg > wrote: >> Here is a complete expample using a decorator, still a bit noisy >> >> def move(aDirection): >> print "moving " + aDirection >> >> #Here comes the decorator >> def scope(aDict): >> def save(locals): >> [...] > > Have you considered making 'scope' a context manager? Your > modifying-locals hacks could be put into the __enter__ and __exit__ > methods, and you'd use the context manager with something like: > > with scope(): > # ... > # use up, down, left, right here > > # up, down, left, right no longer defined after the with block exits. Wow! no I wasn't aware of that. I found some refererences to the magical "with" statement, but I had the impressions that it was a future thing and not available in python 2.5. Other than that it looks exactly like what I was looking for. Thanks. From Martin.Drautzburg at web.de Fri Jan 22 14:29:13 2010 From: Martin.Drautzburg at web.de (Martin Drautzburg) Date: Fri, 22 Jan 2010 20:29:13 +0100 Subject: A.x vs. A["x"] Message-ID: <1718504.hrdDYYapAE@beaureve.gmx.net> This has probably been asekd a million times, but if someone could give a short answer anyways I's be most grateful. What is it that allows one to write A.x? If I have a variable A, then what to I have to assign to it to A.x becomes valid? Or even further: what do I have to do so I can write A.x=1 without having done anything magical for x (but just for A)? I know you can do this with classes, but not with plain objects, but why is that so? From susan_kijiji at yahoo.ca Fri Jan 22 14:35:35 2010 From: susan_kijiji at yahoo.ca (susan_kijiji at yahoo.ca) Date: Fri, 22 Jan 2010 11:35:35 -0800 (PST) Subject: py2exe deal with python command line inside a program Message-ID: Hi, I need to create a python subprogress, like this: myProcess = subprocess.Popen([sys.executable, 'C:\myscript.py'], env=env, stdin=subprocess.PIPE, stdout=subprocess.PIPE) sys.executable was printed out as ''C:\\Python25\\python.exe'', how can I make this work in executable package through py2exe? I have to fix the following problems: -Source code shouldn't exposed in an executable program -Since python environment is not required when running an executable program, how to deal with the situation that "C:\\Python25\ \python.exe" is required as part of command? Thanks in advance! From Martin.Drautzburg at web.de Fri Jan 22 14:39:31 2010 From: Martin.Drautzburg at web.de (Martin Drautzburg) Date: Fri, 22 Jan 2010 20:39:31 +0100 Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> <065e450f-2190-42f4-b9c6-4b95260092be@f12g2000yqn.googlegroups.com> <28103200.7M7XYOBnsK@beaureve.gmx.net> Message-ID: <9734672.qlcV6ekshW@beaureve.gmx.net> Martin Drautzburg wrote: >> with scope(): >> # ... >> # use up, down, left, right here >> >> # up, down, left, right no longer defined after the with block exits. Just looked it up again. It's a cool thing. Too bad my locals() hack would still be required. The result would be less noisy (and actually really beautiful) than the decorator implementation though. Thanks again for pointing this out to me. From showell30 at yahoo.com Fri Jan 22 14:59:29 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 11:59:29 -0800 (PST) Subject: A.x vs. A["x"] References: <1718504.hrdDYYapAE@beaureve.gmx.net> Message-ID: On Jan 22, 11:29?am, Martin Drautzburg wrote: > This has probably been asekd a million times, but if someone could give > a short answer anyways I's be most grateful. Not sure there is exactly a short answer, and I am only qualified to maybe clarify some of the things you can and cannot do, not explain the reasons they are so. > What is it that allows one to write A.x? If I have a variable A, then > what to I have to assign to it to A.x becomes valid? Although not super concise, you'll find some good reading here: http://docs.python.org/tutorial/classes.html#a-word-about-names-and-objects It maybe does not address your question exactly, but it might give you insight into the overall philosophy. > Or even further: what do I have to do so I can write A.x=1 without > having done anything magical for x (but just for A)? I know you can do > this with classes, but not with plain objects, but why is that so? Here are examples where adding attributes on the fly does not work: Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41) [GCC 4.3.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> s = '' >>> s.x = 1 Traceback (most recent call last): File "", line 1, in AttributeError: 'str' object has no attribute 'x' >>> d = {} >>> d.x = 1 Traceback (most recent call last): File "", line 1, in AttributeError: 'dict' object has no attribute 'x' >>> n = 0 >>> n.x = 1 Traceback (most recent call last): File "", line 1, in AttributeError: 'int' object has no attribute 'x' Here are examples when you can assign new attributes: >>> class C: pass ... >>> C.x = 1 >>> C().x = 1 >>> def f(): pass ... >>> f.x = 1 >>> lam = lambda n: n >>> lam.x = 1 Hope that helps or at least gets the discussion started. From clp2 at rebertia.com Fri Jan 22 15:14:25 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Fri, 22 Jan 2010 12:14:25 -0800 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: <50697b2c1001221214x2c0798bas415296ed4f61ab50@mail.gmail.com> On Fri, Jan 22, 2010 at 11:14 AM, Steve Howell wrote: > The v2.6.4 version of the tutorial says this: > > ''' > It is also possible to use a list as a queue, where the first element > added is the first element retrieved (?first-in, first-out?); however, > lists are not efficient for this purpose. While appends and pops from > the end of list are fast, doing inserts or pops from the beginning of > a list is slow (because all of the other elements have to be shifted > by one). > ''' > > Is that really true in CPython? ?It seems like you could advance the > pointer instead of shifting all the elements. ?It would create some > nuances with respect to reclaiming the memory, but it seems like an > easy way to make lists perform better under a pretty reasonable use > case. > > Does anybody know off the top of their head if the "have-to-be-shifted- > by-one" warning is actually valid? Judging by the "Sorted dictionary" thread responses: Yes. Cheers, Chris -- http://blog.rebertia.com From showell30 at yahoo.com Fri Jan 22 15:22:21 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 12:22:21 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: On Jan 22, 12:14?pm, Chris Rebert wrote: > On Fri, Jan 22, 2010 at 11:14 AM, Steve Howell wrote: > > The v2.6.4 version of the tutorial says this: > > > ''' > > It is also possible to use a list as a queue, where the first element > > added is the first element retrieved (?first-in, first-out?); however, > > lists are not efficient for this purpose. While appends and pops from > > the end of list are fast, doing inserts or pops from the beginning of > > a list is slow (because all of the other elements have to be shifted > > by one). > > ''' > > > Is that really true in CPython? ?It seems like you could advance the > > pointer instead of shifting all the elements. ?It would create some > > nuances with respect to reclaiming the memory, but it seems like an > > easy way to make lists perform better under a pretty reasonable use > > case. > > > Does anybody know off the top of their head if the "have-to-be-shifted- > > by-one" warning is actually valid? > > Judging by the "Sorted dictionary" thread responses: Yes. > I think you are referring to this comment: ''' Insertion and deletion are still O(n) as all items to the right of the inserted/deleted one have to be shifted by one place. ''' http://groups.google.com/group/comp.lang.python/browse_thread/thread/d3699724d94d5b5a I can certainly see why most reasonable implementations of a list would have insertion/deletion in the middle of the list be O(N), but I don't think that limitation has to apply for the special cases of the beginning and end of the list. From jjposner at optimum.net Fri Jan 22 15:29:17 2010 From: jjposner at optimum.net (John Posner) Date: Fri, 22 Jan 2010 15:29:17 -0500 Subject: [2.5.1.1/dictionary] Change sorting order? In-Reply-To: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: <4b5a0af2$0$4995$607ed4bc@cv.net> On 1/22/2010 7:17 AM, Gilles Ganault wrote: > Hello > > I use a dictionary to keep a list of users connected to a web site. > > To avoid users from creating login names that start with digits in > order to be listed at the top, I'd like to sort the list differently > every minute so that it'll start with the next letter, eg. display the > list from A...Zdigits the first time, then B...ZAdigits, etc. > I don't believe anyone has suggested customizing the way that the dictionary's keys are sorted. How about using a string-like object ("RotaKey"), with a changeable sort order: #-------------------------- import string, time class RotaKey(object): """ string with rotating sort order """ LETTERS = string.ascii_uppercase def __init__(self, instring): self.string = instring @staticmethod def RotateSortOrder(): """ rotate the letter sequence for sorting RotaKey objects """ RotaKey.LETTERS = RotaKey.LETTERS[1:] + RotaKey.LETTERS[0] def __repr__(self): return "<%s>" % self.string def __cmp__(self, other): self_first, self_rest = self.string[0], self.string[1:] other_first, other_rest = other.string[0], other.string[1:] result = cmp(self.LETTERS.index(self_first.upper()), self.LETTERS.index(other_first.upper())) if result != 0: # first letters differ return result else: # first letters same, compare rest of strings return cmp(self_rest, other_rest) if __name__ == "__main__": # create dictionary: keys are RotaKey's, values in range 1001 - ... names = map(RotaKey, "Ant Zebra Bob Cat2 Ant2 Eagle Bob3 Bob2 Cat Dog".split()) datadict = dict(zip(names, range(1001, 1001+len(names)))) # TEST: print, rotate, print, rotate, ... for i in range(7): if i > 0: RotaKey.RotateSortOrder() print "=" * 15, i for k in sorted(datadict): print k, datadict[k] #-------------------------- NOTE: I tried implementing RotaKey as a subclass of "str", but found that the redefinition of __cmp__() was ignored. You can invoke RotaKey.RotateSortOrder() every minute (or whatever) to change the sort order. Program output: =============== 0 1001 1005 1003 1008 1007 1009 1004 1010 1006 1002 =============== 1 1003 1008 1007 1009 1004 1010 1006 1002 1001 1005 =============== 2 1009 1004 1010 1006 1002 ... etc. -John From lists at cheimes.de Fri Jan 22 15:40:08 2010 From: lists at cheimes.de (Christian Heimes) Date: Fri, 22 Jan 2010 21:40:08 +0100 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: <4B5A0D28.5080007@cheimes.de> Steve Howell wrote: > Is that really true in CPython? It seems like you could advance the > pointer instead of shifting all the elements. It would create some > nuances with respect to reclaiming the memory, but it seems like an > easy way to make lists perform better under a pretty reasonable use > case. > > Does anybody know off the top of their head if the "have-to-be-shifted- > by-one" warning is actually valid? Why do you think the documentation has such obvious errors? CPython's list type uses an array of pointers to store its members. The type is optimized for the most common list operations in Python: iteration and appending. Python code rarely changes the head or middle of a list. The dequeue type is an optimized data structure for popping and inserting data at both ends. When you list.pop() or list.insert() the pointers in the internal array must be shifted. appending is much faster because the internal array is overallocated meaning it contains free slots at the tail of the data structure. A linked list of pointers requires more memory and iteration is slower since since it can't utilize the CPU's cache as good as an array. Christian From showell30 at yahoo.com Fri Jan 22 15:57:46 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 12:57:46 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> On Jan 22, 12:40?pm, Christian Heimes wrote: > Steve Howell wrote: > > Is that really true in CPython? ?It seems like you could advance the > > pointer instead of shifting all the elements. ?It would create some > > nuances with respect to reclaiming the memory, but it seems like an > > easy way to make lists perform better under a pretty reasonable use > > case. > > > Does anybody know off the top of their head if the "have-to-be-shifted- > > by-one" warning is actually valid? > > Why do you think the documentation has such obvious errors? I wasn't making any assumptions, hence the question mark. The Python docs are very good, but even the best of projects make advances that aren't reflected in the docs. > CPython's list type uses an array of pointers to store its members. The > type is optimized for the most common list operations in Python: > iteration and appending. Python code rarely changes the head or middle > of a list. The dequeue type is an optimized data structure for popping > and inserting data at both ends. > I disagree that Python code rarely pops elements off the top of a list. There are perfectly valid use cases for wanting a list over a dequeue without having to pay O(N) for pop(0). Maybe we are just quibbling over the meaning of "rarely." > When you list.pop() or list.insert() the pointers in the internal array > must be shifted. appending is much faster because the internal array is > overallocated meaning it contains free slots at the tail of the data > structure. A linked list of pointers requires more memory and iteration > is slower since since it can't utilize the CPU's cache as good as an array. > I am not proposing a linked list of pointers. I am wondering about something like this: p = &p[1]; (and then reclaim p[0] as free memory, I already said I understood that was the tricky bit) The pointer arithmetic for accessing each element would still work in O (1), right? From arnodel at googlemail.com Fri Jan 22 16:08:48 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Fri, 22 Jan 2010 21:08:48 +0000 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: Steve Howell writes: > On Jan 22, 12:14?pm, Chris Rebert wrote: >> On Fri, Jan 22, 2010 at 11:14 AM, Steve Howell wrote: >> > The v2.6.4 version of the tutorial says this: >> >> > ''' >> > It is also possible to use a list as a queue, where the first element >> > added is the first element retrieved (?first-in, first-out?); however, >> > lists are not efficient for this purpose. While appends and pops from >> > the end of list are fast, doing inserts or pops from the beginning of >> > a list is slow (because all of the other elements have to be shifted >> > by one). >> > ''' >> >> > Is that really true in CPython? ?It seems like you could advance the >> > pointer instead of shifting all the elements. ?It would create some >> > nuances with respect to reclaiming the memory, but it seems like an >> > easy way to make lists perform better under a pretty reasonable use >> > case. >> >> > Does anybody know off the top of their head if the "have-to-be-shifted- >> > by-one" warning is actually valid? >> >> Judging by the "Sorted dictionary" thread responses: Yes. >> > > I think you are referring to this comment: > > ''' > Insertion and deletion are still O(n) as all items to the right of the > inserted/deleted one have to be shifted by one place. > ''' > > > I can certainly see why most reasonable implementations of a list > would have insertion/deletion in the middle of the list be O(N), but I > don't think that limitation has to apply for the special cases of the > beginning and end of the list. I made the comment you quoted. In CPython, it is O(n) to delete/insert an element at the start of the list - I know it because I looked at the implementation a while ago. This is why collections.deque exists I guess. I don't know how they are implemented but insertion/deletion at either end are O(1) and so is random access. So they are the data structure that you want. If you want evidence for lists, rather than my word, try: >>> import timeit >>> timeit.Timer('while t:del t[0]', 't=[0]*100000').timeit(1) 1.8452401161193848 >>> timeit.Timer('while t:del t[-1]', 't=[0]*100000').timeit(1) 0.017747163772583008 >>> timeit.Timer( 'while t:del t[0]', 'from collections import deque; t=deque([0]*100000)' ).timeit(1) 0.022077083587646484 >>> timeit.Timer( 'while t:del t[-1]', 'from collections import deque; t=deque([0]*100000)' ).timeit(1) 0.027813911437988281 -- Arnaud From kyosohma at gmail.com Fri Jan 22 16:19:43 2010 From: kyosohma at gmail.com (Mike Driscoll) Date: Fri, 22 Jan 2010 13:19:43 -0800 (PST) Subject: looking for Python live code reloading IDEs References: <40aa7d9f-aa35-4bfd-b0d9-46a10585731b@b9g2000yqd.googlegroups.com> Message-ID: <1161e21d-489c-40f6-a239-ca5c585abb6b@y23g2000yqm.googlegroups.com> On Jan 22, 3:16?am, George Oliver wrote: > hi, I'm wondering if there are any Python programming environments > that enable live code reloading, for example something like the Scheme- > based impromptu (but also meant for any kind of Python program, not > just audio/visual generation). > > Currently I do this directly in my editor (for game development), but > I'm curious if there are other options. Maybe you could get an idea from TurboGears or Django. They seem to reload their environments when a file is saved. ------------------- Mike Driscoll Blog: http://blog.pythonlibrary.org PyCon 2010 Atlanta Feb 19-21 http://us.pycon.org/ From kyosohma at gmail.com Fri Jan 22 16:24:20 2010 From: kyosohma at gmail.com (Mike Driscoll) Date: Fri, 22 Jan 2010 13:24:20 -0800 (PST) Subject: simple pub/sub References: <0af0eff2-50e2-44aa-81f1-b3d12cb265a7@a15g2000yqm.googlegroups.com> Message-ID: <1b44693a-e91e-4ac8-93ed-4934e126fdee@g1g2000yqi.googlegroups.com> On Jan 21, 10:54?am, Steve Howell wrote: > Hi, I'm looking for ideas on building a simple architecture that > allows a bunch of independent Python processes to exchange data using > files and perform calculations. > > One Python program would be collecting data from boat instruments on a > serial port, then writing that info out to a file, and also > transmitting data to instruments as needed (again through the serial > port). ?It would be the most complex program in terms of having to > poll file descriptors, or something similar, but I want to limit its > job to pure communication handling. > > Then, I would have a bunch of programs doing various calculations on > the input stream. ?It's totally acceptable if they just sleep a bit, > wake up, see if there is any new data, and if so, do some > calculations, so I will probably just use this: > > http://www.dabeaz.com/generators/follow.py > > Some of the "calculator" programs might be looking for trends in the > data, and when a certain threshold is reached, they will need to > notify the comm program of the new message. ?To avoid collisions > between all the different calculator programs, I am thinking the > simplest thing is just have them each write to their own output file. > > So then the comm program has multiple data sources. ?It wants to track > to the serial port, but it also wants to monitor the output files from > the various calculator processes. ?I want to do this in a mostly > nonblocking way, so I would probably poll on the file descriptors. > > The only problems are that select "...cannot be used on regular files > to determine whether a file has grown since it was last read," and it > only works on Unix for files. ?(I'm using Unix myself, but one of the > other programmers insists on Windows, and is resisting Python as > well.) > > So a variation would be that the calculator programs talk to the comm > program via sockets, which seems slightly on the heavy side, but it is > certainly feasible. > > I know that something like twisted could probably solve my problem, > but I'm also wondering if there is some solution that is a little more > lightweight and batteries-included. > > At the heart of the matter, I just want the programs to be able to do > the following tasks. > > ? 1) All programs will occasionally need to write a line of text to > some data stream without having to lock it. ?(I don't care if the > write blocks.) > ? 2) All programs will need to be able to do a non-blocking read of a > line of text from a data stream. ?(And there may be multiple > consumers.) > ? 3) The comm program will need to be able to poll the serial port and > input data streams to see which ones are ready. > > Any thoughts? Did you look at the pubsub module at all? See http://pubsub.sourceforge.net/ for more information. I use it a lot in my wxPython programs. ------------------- Mike Driscoll Blog: http://blog.pythonlibrary.org PyCon 2010 Atlanta Feb 19-21 http://us.pycon.org/ From showell30 at yahoo.com Fri Jan 22 16:26:03 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 13:26:03 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: <82dcbdd1-bf17-4f3c-8158-df054e69c79b@k35g2000yqb.googlegroups.com> On Jan 22, 1:08?pm, Arnaud Delobelle wrote: > Steve Howell writes: > > On Jan 22, 12:14?pm, Chris Rebert wrote: > >> On Fri, Jan 22, 2010 at 11:14 AM, Steve Howell wrote: > >> > The v2.6.4 version of the tutorial says this: > > >> > ''' > >> > It is also possible to use a list as a queue, where the first element > >> > added is the first element retrieved (?first-in, first-out?); however, > >> > lists are not efficient for this purpose. While appends and pops from > >> > the end of list are fast, doing inserts or pops from the beginning of > >> > a list is slow (because all of the other elements have to be shifted > >> > by one). > >> > ''' > > >> > Is that really true in CPython? ?It seems like you could advance the > >> > pointer instead of shifting all the elements. ?It would create some > >> > nuances with respect to reclaiming the memory, but it seems like an > >> > easy way to make lists perform better under a pretty reasonable use > >> > case. > > >> > Does anybody know off the top of their head if the "have-to-be-shifted- > >> > by-one" warning is actually valid? > > >> Judging by the "Sorted dictionary" thread responses: Yes. > > > I think you are referring to this comment: > > > ''' > > Insertion and deletion are still O(n) as all items to the right of the > > inserted/deleted one have to be shifted by one place. > > ''' > > > I can certainly see why most reasonable implementations of a list > > would have insertion/deletion in the middle of the list be O(N), but I > > don't think that limitation has to apply for the special cases of the > > beginning and end of the list. > > I made the comment you quoted. ?In CPython, it is O(n) to delete/insert > an element at the start of the list - I know it because I looked at the > implementation a while ago. ?This is why collections.deque exists I > guess. ?I don't know how they are implemented but insertion/deletion at > either end are O(1) and so is random access. ?So they are the data > structure that you want. > > If you want evidence for lists, rather than my word, try: > > >>> import timeit > >>> timeit.Timer('while t:del t[0]', 't=[0]*100000').timeit(1) > 1.8452401161193848 > >>> timeit.Timer('while t:del t[-1]', 't=[0]*100000').timeit(1) > > 0.017747163772583008 > > >>> timeit.Timer( > > ? ? 'while t:del t[0]', > ? ? 'from collections import deque; t=deque([0]*100000)' > ).timeit(1) > 0.022077083587646484>>> timeit.Timer( > > ? ? 'while t:del t[-1]', > ? ? 'from collections import deque; t=deque([0]*100000)' > ).timeit(1) > 0.027813911437988281 > Ok, thanks, good to know. I assume it's the colorly named list_ass_slice that would have to special case ilow == 0 and you'd need a memory manager that would let you realloc from ilow:ihigh to ilow+n:high. Am I reading that much of the code correctly? From tjreedy at udel.edu Fri Jan 22 16:27:14 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 22 Jan 2010 16:27:14 -0500 Subject: * operator in python tutorial In-Reply-To: References: <7b2ef0aa-8b93-4cb5-b944-beea3e2c98b1@p16g2000vbe.googlegroups.com> <0036df65$0$25085$c3e8da3@news.astraweb.com> Message-ID: On 1/22/2010 12:57 PM, ben wrote: > On Jan 20, 8:30 pm, Gringo wrote: >>> I am following through the python tutorial which gets to a line that >>> uses the * operator with zip(). All such questions are at least briefly answered in my complete python3 symbol glossary (all syntax usages) at http://code.google.com/p/xploro/downloads/list Terry Jan Reedy From lists at cheimes.de Fri Jan 22 16:29:23 2010 From: lists at cheimes.de (Christian Heimes) Date: Fri, 22 Jan 2010 22:29:23 +0100 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> Message-ID: <4B5A18B3.20200@cheimes.de> Steve Howell wrote: > I disagree that Python code rarely pops elements off the top of a > list. There are perfectly valid use cases for wanting a list over a > dequeue without having to pay O(N) for pop(0). Maybe we are just > quibbling over the meaning of "rarely." I was speaking from my own point of view. I've written several tenths of thousands of lines of Python code in the last seven years, mostly related to data manipulation, web applications and operating system interaction but also GUI stuff and scientific code. I can't recall any performance critical or prominent code that modifies the head of a list a lot. Of course there a use cases where you may want to use list.pop(). Unless you need a FILO structure you can always replace a LILO with a FIFO -- instead of list.insert(0, value) and list.pop(0) use list.append(value) and list.pop(). It's not possible to optimize a data structure for all use cases. > I am not proposing a linked list of pointers. I am wondering about > something like this: > > p = &p[1]; > (and then reclaim p[0] as free memory, I already said I understood > that was the tricky bit) > > The pointer arithmetic for accessing each element would still work in O > (1), right? You mean it's an impossible trick unless you come up with your own memory management system. Realloc(), the standard function to change the size of chunk of allocated memory in C, doesn't support your desired operation. Christian From steve at holdenweb.com Fri Jan 22 16:30:38 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 22 Jan 2010 16:30:38 -0500 Subject: counting lines of code In-Reply-To: References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> <87bpgodj95.fsf@benfinney.id.au> <1f166703-fdfc-4f87-b336-ef80e492635b@a32g2000yqm.googlegroups.com> <7e09df6a-cda1-480e-a971-8f8a70ac4118@b9g2000yqd.googlegroups.com> <%I36n.5962$4p5.2277@newsfe22.iad> Message-ID: Robert Kern wrote: > On 2010-01-21 15:31 , Phlip wrote: >> Aahz wrote: >>> In article >>> <7e09df6a-cda1-480e-a971-8f8a70ac4118 at b9g2000yqd.googlegroups.com>, >>> Phlip wrote: >>>> On Jan 20, 11:20=A0pm, Michele Simionato >>>> wrote: >>>>> pylint does too many things, I want something fast that just counts >>>>> the lines and can be run on thousands of files at once. >>>>> cloc seems fine, I have just tried on 2,000 files and it gives me a >>>>> report in just a few seconds. >>>> In my experience with Python codebases that big... >>>> >>>> ...how many of those lines are duplicated, and might merge together >>>> into a better design? >>> >>> Um... do you have any clue who you followed up to? If you don't, Google >>> is your friend. >> >> Oh, sorry, did I have the wrong opinion? > > You had a condescending attitude. > Towards someone who is fairly obviously not a Python neophyte. Please don't think we are telling you you can't have any opinion you like. Just don't expect to get away with it when you are wrong ;-) Welcome to c.l.py. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From icanbob at gmail.com Fri Jan 22 16:30:54 2010 From: icanbob at gmail.com (bobicanprogram) Date: Fri, 22 Jan 2010 13:30:54 -0800 (PST) Subject: simple pub/sub References: <0af0eff2-50e2-44aa-81f1-b3d12cb265a7@a15g2000yqm.googlegroups.com> Message-ID: <5dcf12b7-73b1-4ec4-ac9c-b984862bdc43@b10g2000vbh.googlegroups.com> On Jan 21, 11:54 am, Steve Howell wrote: > Hi, I'm looking for ideas on building a simple architecture that > allows a bunch of independent Python processes to exchange data using > files and perform calculations. > > One Python program would be collecting data from boat instruments on a > serial port, then writing that info out to a file, and also > transmitting data to instruments as needed (again through the serial > port). It would be the most complex program in terms of having to > poll file descriptors, or something similar, but I want to limit its > job to pure communication handling. > > Then, I would have a bunch of programs doing various calculations on > the input stream. It's totally acceptable if they just sleep a bit, > wake up, see if there is any new data, and if so, do some > calculations, so I will probably just use this: > > http://www.dabeaz.com/generators/follow.py > > Some of the "calculator" programs might be looking for trends in the > data, and when a certain threshold is reached, they will need to > notify the comm program of the new message. To avoid collisions > between all the different calculator programs, I am thinking the > simplest thing is just have them each write to their own output file. > > So then the comm program has multiple data sources. It wants to track > to the serial port, but it also wants to monitor the output files from > the various calculator processes. I want to do this in a mostly > nonblocking way, so I would probably poll on the file descriptors. > > The only problems are that select "...cannot be used on regular files > to determine whether a file has grown since it was last read," and it > only works on Unix for files. (I'm using Unix myself, but one of the > other programmers insists on Windows, and is resisting Python as > well.) > > So a variation would be that the calculator programs talk to the comm > program via sockets, which seems slightly on the heavy side, but it is > certainly feasible. > > I know that something like twisted could probably solve my problem, > but I'm also wondering if there is some solution that is a little more > lightweight and batteries-included. > > At the heart of the matter, I just want the programs to be able to do > the following tasks. > > 1) All programs will occasionally need to write a line of text to > some data stream without having to lock it. (I don't care if the > write blocks.) > 2) All programs will need to be able to do a non-blocking read of a > line of text from a data stream. (And there may be multiple > consumers.) > 3) The comm program will need to be able to poll the serial port and > input data streams to see which ones are ready. > > Any thoughts? You should definitely check out the SIMPL toolkit (http:// www.icanprogram.com/simpl). It has some pretty mature Python hooks. SIMPL has been used to construct data acquisition applications not unlike the one you describe. bob From tjreedy at udel.edu Fri Jan 22 16:32:47 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 22 Jan 2010 16:32:47 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: On 1/22/2010 2:14 PM, Steve Howell wrote: > The v2.6.4 version of the tutorial says this: > Is that really true in CPython? It seems like you could advance the > pointer instead of shifting all the elements. It would create some > nuances with respect to reclaiming the memory, but it seems like an > easy way to make lists perform better under a pretty reasonable use > case. Something like this was one proposed (ie, leave space at both ends of a list) but was rejected as over-complicating the list implementation for *relatively* rare use cases. I believe deque was written subsequently to address such other use cases. From lists at cheimes.de Fri Jan 22 16:35:37 2010 From: lists at cheimes.de (Christian Heimes) Date: Fri, 22 Jan 2010 22:35:37 +0100 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: <4B5A1A29.1050506@cheimes.de> Arnaud Delobelle wrote: > I made the comment you quoted. In CPython, it is O(n) to delete/insert > an element at the start of the list - I know it because I looked at the > implementation a while ago. This is why collections.deque exists I > guess. I don't know how they are implemented but insertion/deletion at > either end are O(1) and so is random access. So they are the data > structure that you want. Your assumption is correct. The collections.dequeue type uses a double linked list of blocks. Each blocks contains a fixed amount of pointers to Python objects. The implementation makes the implementation less memory hungry than a standard double linked list with just one element for each block. Christian From wbsoft at xs4all.nl Fri Jan 22 16:40:21 2010 From: wbsoft at xs4all.nl (Wilbert Berendsen) Date: Fri, 22 Jan 2010 22:40:21 +0100 Subject: counting lines of code In-Reply-To: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> Message-ID: <201001222240.21081.wbsoft@xs4all.nl> Op donderdag 21 januari 2010 schreef Michele: > I need a small utility to count the lines of Python code in a > directory, traversing subdirectories and ignoring comments and > docstrings. sloccount can do this. http://www.dwheeler.com/sloccount/ Met vriendelijke groet, Wilbert Berendsen -- http://www.wilbertberendsen.nl/ "You must be the change you wish to see in the world." -- Mahatma Gandhi From cs at zip.com.au Fri Jan 22 16:41:52 2010 From: cs at zip.com.au (Cameron Simpson) Date: Sat, 23 Jan 2010 08:41:52 +1100 Subject: looking for Python live code reloading IDEs In-Reply-To: <1161e21d-489c-40f6-a239-ca5c585abb6b@y23g2000yqm.googlegroups.com> References: <1161e21d-489c-40f6-a239-ca5c585abb6b@y23g2000yqm.googlegroups.com> Message-ID: <20100122214151.GA29661@cskk.homeip.net> On 22Jan2010 13:19, Mike Driscoll wrote: | On Jan 22, 3:16?am, George Oliver wrote: | > hi, I'm wondering if there are any Python programming environments | > that enable live code reloading, for example something like the Scheme- | > based impromptu (but also meant for any kind of Python program, not | > just audio/visual generation). | > | > Currently I do this directly in my editor (for game development), but | > I'm curious if there are other options. | | Maybe you could get an idea from TurboGears or Django. They seem to | reload their environments when a file is saved. CherrypPy too. -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ From tjreedy at udel.edu Fri Jan 22 16:46:03 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 22 Jan 2010 16:46:03 -0500 Subject: A.x vs. A["x"] In-Reply-To: <1718504.hrdDYYapAE@beaureve.gmx.net> References: <1718504.hrdDYYapAE@beaureve.gmx.net> Message-ID: On 1/22/2010 2:29 PM, Martin Drautzburg wrote: > This has probably been asekd a million times, but if someone could give > a short answer anyways I's be most grateful. > > What is it that allows one to write A.x? If I have a variable A, You do not really have a 'variable'. You have a name A bound to an object that is an instance of class C = type(A). > what to I have to assign to it to A.x becomes valid? If C has the appropriate special methods for get/set/del attribute, then you can get/set/del attributes for instances of C (such as A). If not, you cannot. In general, builtin classes, including object, allow get but not set/del on both the class and instances thereof. Subclasses of object allow all three. > Or even further: what do I have to do so I can write A.x=1 without > having done anything magical for x (but just for A)? Make A be a user-defined class or an instance thereof. > I know you can do this with classes, but not with plain objects, but why is that so? You exact meaning here is not clear, but I suspect it is somewhat incorrect, at least for built-in classes. Terry Jan Reedy From cjns1989 at gmail.com Fri Jan 22 16:47:55 2010 From: cjns1989 at gmail.com (Chris Jones) Date: Fri, 22 Jan 2010 16:47:55 -0500 Subject: Using dictionary key as a regular expression class Message-ID: <20100122214755.GH28576@turki.gavron.org> I was writing a script that counts occurrences of characters in source code files: #!/usr/bin/python import codecs tcounters = {} f = codecs.open('/home/gavron/git/screen/src/screen.c', 'r', "utf-8") for uline in f: lline = [] for char in uline[:-1]: lline += [char] counters = {} for i in set(lline): counters[i] = lline.count(i) for c in counters.keys(): if c in tcounters: tcounters[c] += counters[c] else: tcounters.update({c: counters[c]}) counters = {} for c in tcounters.keys(): print c, '\t', tcounters[c] I was looking for a way to stat by finger actions on a US keyboard for a traditional typist, and I was thinking I could use another dictionary with keys that would be made up of the characters correponding to each finger, such as '!1QqAaZz' for the left pinky, etc., hoping I would be able to iterate the keys and match the currently processed character with the key and increment it. Is this something that makes sense, or should I look elsewhere? Suggestions to improve the above snippet are also welcome. Thanks, CJ From aahz at pythoncraft.com Fri Jan 22 16:52:34 2010 From: aahz at pythoncraft.com (Aahz) Date: 22 Jan 2010 13:52:34 -0800 Subject: Bare Excepts References: <4dc0cfea0912300209x15e5bc96nd6e6224d89df0638@mail.gmail.com> <034fd208$0$1277$c3e8da3@news.astraweb.com> <80c56956-f28e-47a3-a723-3a5e3fd294b0@j19g2000yqk.googlegroups.com> Message-ID: In article <80c56956-f28e-47a3-a723-3a5e3fd294b0 at j19g2000yqk.googlegroups.com>, sjdevnull at yahoo.com wrote: >On Jan 2, 9:35=A0pm, Dave Angel wrote: >> >> In Windows, there is a way to do it. It's just not exposed to the >> Python built-in function open(). You use the CreateFile() function, >> with /dwCreationDisposition/ of CREATE_NEW. >> >> It's atomic, and fails politely if the file already exists. >> >> No idea if Unix has a similar functionality. > >It does. In Unix, you'd pass O_CREAT|O_EXCL to the open(2) system >call (O_CREAT means create a new file, O_EXCL means exclusive mode: >fail if the file exists already). Thanks! -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From gnarlodious at gmail.com Fri Jan 22 17:00:33 2010 From: gnarlodious at gmail.com (Gnarlodious) Date: Fri, 22 Jan 2010 14:00:33 -0800 (PST) Subject: Create object name from string value? References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> Message-ID: <603b8956-bed4-4006-a466-c2e2096f37f7@c29g2000yqd.googlegroups.com> On Jan 21, 9:21?am, Dave Angel wrote: > Put it inside a dummy class, as follows: This has been very educational, thank you for all the suggestions. Here is the resulting code: class Property: pass # example of a dummy class Plist = Property() # create a Plist object from the dummy class # accumulate dictionary attributes based on every plist file dataFolder=os.path.dirname(__file__)+'/Data/' plistFiles=glob.glob(dataFolder+'*.plist') for file in range(len(plistFiles)): name=plistFiles[file].replace(dataFolder,'').replace('.plist','') setattr(Plist, name, Dict(plistFiles[file]).Config) # load the Plist object with the dictionary This is a vast improvement over my previous system. Now I can call any plist item from anywhere on my website: Data.Plist.Site.Trace where "Site.plist" is the name of the Plist file. This is such an elegant system I'm surprised it is not implemented as part of Python. -- Gnarlie http://Gnarlodious.com/Gnarlodious From arnodel at googlemail.com Fri Jan 22 17:07:13 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Fri, 22 Jan 2010 22:07:13 +0000 Subject: Using dictionary key as a regular expression class References: Message-ID: Chris Jones writes: > I was writing a script that counts occurrences of characters in source > code files: > > #!/usr/bin/python > import codecs > tcounters = {} > f = codecs.open('/home/gavron/git/screen/src/screen.c', 'r', "utf-8") > for uline in f: > lline = [] > for char in uline[:-1]: > lline += [char] > counters = {} > for i in set(lline): > counters[i] = lline.count(i) > for c in counters.keys(): > if c in tcounters: > tcounters[c] += counters[c] > else: > tcounters.update({c: counters[c]}) > counters = {} > for c in tcounters.keys(): > print c, '\t', tcounters[c] > > I was looking for a way to stat by finger actions on a US keyboard for a > traditional typist, and I was thinking I could use another dictionary > with keys that would be made up of the characters correponding to each > finger, such as '!1QqAaZz' for the left pinky, etc., hoping I would be > able to iterate the keys and match the currently processed character > with the key and increment it. > > Is this something that makes sense, or should I look elsewhere? > > Suggestions to improve the above snippet are also welcome. > > Thanks, > > CJ Why not just start with (untested): import codecs from collections import defaultdict tcounters = defaultdict(int) f = codecs.open('/home/gavron/git/screen/src/screen.c', 'r', "utf-8") for c in f.read(): tcounters[c] += 1 for c, n in tcounters.iteritems(): print "%r\t%i" % (c, n) -- Arnaud From showell30 at yahoo.com Fri Jan 22 17:38:18 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 14:38:18 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> Message-ID: <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> On Jan 22, 1:29?pm, Christian Heimes wrote: > Steve Howell wrote: > > I disagree that Python code rarely pops elements off the top of a > > list. ?There are perfectly valid use cases for wanting a list over a > > dequeue without having to pay O(N) for pop(0). ?Maybe we are just > > quibbling over the meaning of "rarely." > > I was speaking from my own point of view. I've written several tenths of > thousands of lines of Python code in the last seven years, mostly > related to data manipulation, web applications and operating system > interaction but also GUI stuff and scientific code. I can't recall any > performance critical or prominent code that modifies the head of a list > a lot. > That maybe would be an argument for just striking the paragraph from the tutorial. If it's rare that people pop the head off the list in code that is performance critical or prominent, why bother to even mention it in the tutorial? > Of course there a use cases where you may want to use list.pop(). Unless > you need a FILO structure you can always replace a LILO with a FIFO -- > instead of list.insert(0, value) and list.pop(0) use list.append(value) > and list.pop(). It's not possible to optimize a data structure for all > use cases. > > > I am not proposing a linked list of pointers. ?I am wondering about > > something like this: > > > p = &p[1]; > > (and then reclaim p[0] as free memory, I already said I understood > > that was the tricky bit) > > > The pointer arithmetic for accessing each element would still work in O > > (1), right? > > You mean it's an impossible trick unless you come up with your own > memory management system. Realloc(), the standard function to change the > size of chunk of allocated memory in C, doesn't support your desired > operation. > Impossible is a strong word. You could be lazy about giving the memory back, and just wait till the whole list is garbage collected. I don't think there's anything in Python's contract that says memory has to be freed the exact moment you stop using it, especially since we're talking about doing an O(N) memmove just to free up one pointer's worth of memory. I know the Python programmer could simply iterate through the list rather than popping off unused elements, but that just means that you not only tie up the memory for the pointers just as long, but you also prevent the objects themselves from being garbage collected. In my case I'm not really concerned about giving the memory back right away, it's more about keeping my code simple. Once I'm done with an element, I do just want to pop it off and keep all the simplicity of the lists, but I am just concerned enough speed that for a 1000 element list, I don't want to be doing 1000 memmoves for an average of 500 pointers, which effectively moves about a million bytes around for no reason. I suppose the solution is to either give up the sugar of lists, or try to wrap something like deque or list-with-offset into a sequence. From arobson73 at yahoo.co.uk Fri Jan 22 17:47:35 2010 From: arobson73 at yahoo.co.uk (G73) Date: Fri, 22 Jan 2010 14:47:35 -0800 (PST) Subject: Patch for IDLE/OS X to work with Tk-Cocoa Message-ID: <15238f19-e008-4e01-b284-721d3deba8ee@c29g2000yqd.googlegroups.com> im trying to update a patch. here is link to various patches http://bugs.python.org/issue6075 how do i update the patch, say for EditorWindow.patch. i have located my python installation the EditorWindow.py, and i can see some differences (which lines from the patch actually go into the EditorWindow.py ?. Also assuming i know which lines to edit, do i just edit the EditorWindow.py and that is it ? will i then be abler to run IDLE and this then pick up the changes i have made in EditorWindow/py ? im new to python and mac, but trying to learn it. From showell30 at yahoo.com Fri Jan 22 17:52:41 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 14:52:41 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: <0cac37c0-f858-4377-9653-4740cb655cdb@e11g2000yqe.googlegroups.com> On Jan 22, 1:32?pm, Terry Reedy wrote: > On 1/22/2010 2:14 PM, Steve Howell wrote: > > > The v2.6.4 version of the tutorial says this: > > Is that really true in CPython? ?It seems like you could advance the > > pointer instead of shifting all the elements. ?It would create some > > nuances with respect to reclaiming the memory, but it seems like an > > easy way to make lists perform better under a pretty reasonable use > > case. > > Something like this was one proposed (ie, leave space at both ends of a > list) but was rejected as over-complicating the list implementation for > *relatively* rare use cases. I believe deque was written subsequently to > address such other use cases. Bummer. I guess I get to do my own over-complicating of code, being in that unfortunate minority. From davea at ieee.org Fri Jan 22 17:54:15 2010 From: davea at ieee.org (Dave Angel) Date: Fri, 22 Jan 2010 17:54:15 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> Message-ID: <4B5A2C97.7060706@ieee.org> Steve Howell wrote: > On Jan 22, 12:40 pm, Christian Heimes wrote: > >> Steve Howell wrote: >> >>> Is that really true in CPython? It seems like you could advance the >>> pointer instead of shifting all the elements. It would create some >>> nuances with respect to reclaiming the memory, but it seems like an >>> easy way to make lists perform better under a pretty reasonable use >>> case. >>> >>> Does anybody know off the top of their head if the "have-to-be-shifted- >>> by-one" warning is actually valid? >>> >> Why do you think the documentation has such obvious errors? >> > > I wasn't making any assumptions, hence the question mark. The Python > docs are very good, but even the best of projects make advances that > aren't reflected in the docs. > > >> CPython's list type uses an array of pointers to store its members. The >> type is optimized for the most common list operations in Python: >> iteration and appending. Python code rarely changes the head or middle >> of a list. The dequeue type is an optimized data structure for popping >> and inserting data at both ends. >> >> > > I disagree that Python code rarely pops elements off the top of a > list. There are perfectly valid use cases for wanting a list over a > dequeue without having to pay O(N) for pop(0). Maybe we are just > quibbling over the meaning of "rarely." > > >> When you list.pop() or list.insert() the pointers in the internal array >> must be shifted. appending is much faster because the internal array is >> overallocated meaning it contains free slots at the tail of the data >> structure. A linked list of pointers requires more memory and iteration >> is slower since since it can't utilize the CPU's cache as good as an array. >> >> > > I am not proposing a linked list of pointers. I am wondering about > something like this: > > p =p[1]; > (and then reclaim p[0] as free memory, I already said I understood > that was the tricky bit) > > The pointer arithmetic for accessing each element would still work in O > (1), right? > > I think it was Dijkstr (sp?) who said you can accomplish anything with just one more level of indirection. Clearly each attribute (variable) that has a binding to a given list must point to a fixed piece of memory, that cannot safely be moved, because there's no efficient way to find all the attributes. That fixed piece is the list object, and I expect it's 16 or 20 bytes, on a 32bit implementation. It must in turn point to the actual malloc'ed block that contains pointers to all the elements of the list. So that block will be 4*n where n is the number of reserved cells, at least as large as len(). This is the block where copying takes place when you insert or delete from the beginning. The list object must contain a pointer to the beginning of this block, or it wouldn't be able to free() it later. So you'd be suggesting a second pointer that actually points to the current 0th pointer. And a pop would simply increment this second pointer. Such an approach might be worthwhile if you expect lots of pops and pushes, with a minimal net change. But of course each time you did a pop, you'd have to decide whether it was time to normalize/compact the block. As you say, reclaiming the 0th element of this block to the memory pool would be tricky. Doubly so, because 1) the C memory allocator has no such notion as resizing the beginning of a block. and 2) it would have nothing useful to do with the 4 bytes freed. The minimum allocated block in Python is probably 16 bytes of actual address space. I'd guess that's 4 bytes for malloc's overhead, and 8 bytes for the minimum object header, and 4 bytes for data. To check me, try: >>> a = 5.3 >>> b = 49.6 >>> id(a), id(b) (11074136, 11074152) Anyway, this could be done, where once the two pointers get some distance apart, you do a realloc, and copy everything. But of course you'd want to build some hysteresis into it, to avoid thrashing. There wouldn't be much of a performance hit, but it would increase every list size by 4 bytes minimum. So I doubt it would be a list replacement. This'd be an interesting project.to do as an addon module. DaveA From davea at ieee.org Fri Jan 22 17:57:36 2010 From: davea at ieee.org (Dave Angel) Date: Fri, 22 Jan 2010 17:57:36 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: <4B5A2D60.8080101@ieee.org> Arnaud Delobelle wrote: > Steve Howell writes: > > >> On Jan 22, 12:14 pm, Chris Rebert wrote: >> >> > I made the comment you quoted. In CPython, it is O(n) to delete/insert > an element at the start of the list - I know it because I looked at the > implementation a while ago. This is why collections.deque exists I > guess. I don't know how they are implemented but insertion/deletion at > either end are O(1) and so is random access. So they are the data > structure that you want. > > Not according to the 2.6 docs. Indexed access is O(1) at both ends but slows to O(n) in the middle. For fast random access, use lists instead. That sounds to me like a doubly-linked list implementation. From lists at cheimes.de Fri Jan 22 18:17:17 2010 From: lists at cheimes.de (Christian Heimes) Date: Sat, 23 Jan 2010 00:17:17 +0100 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> Message-ID: <4B5A31FD.5090300@cheimes.de> Steve Howell wrote: > That maybe would be an argument for just striking the paragraph from > the tutorial. If it's rare that people pop the head off the list in > code that is performance critical or prominent, why bother to even > mention it in the tutorial? How else are you going to teach new Python developers that they should favor append() and pop() in place of insert() and pop(0)? :) > Impossible is a strong word. You could be lazy about giving the > memory back, and just wait till the whole list is garbage collected. > I don't think there's anything in Python's contract that says memory > has to be freed the exact moment you stop using it, especially since > we're talking about doing an O(N) memmove just to free up one > pointer's worth of memory. Your proposal would increase the size of every list object of sizeof(ptr) or ssize_t (32 or 64bits) in order to store the information where the first element is. It would also unnecessarily complicate the code and possible slow down a lot of list operations. > I know the Python programmer could simply iterate through the list > rather than popping off unused elements, but that just means that you > not only tie up the memory for the pointers just as long, but you also > prevent the objects themselves from being garbage collected. > > In my case I'm not really concerned about giving the memory back right > away, it's more about keeping my code simple. Once I'm done with an > element, I do just want to pop it off and keep all the simplicity of > the lists, but I am just concerned enough speed that for a 1000 > element list, I don't want to be doing 1000 memmoves for an average of > 500 pointers, which effectively moves about a million bytes around for > no reason. The simplicity of Python is gained with some performance drawbacks. You have to learn to use Python algorithms. You can't simply re implement a fast C algorithm and expect it to be fast in Python, too. Christian From showell30 at yahoo.com Fri Jan 22 18:17:21 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 15:17:21 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> Message-ID: On Jan 22, 2:54?pm, Dave Angel wrote: > Steve Howell wrote: > > On Jan 22, 12:40 pm, Christian Heimes wrote: > > >> Steve Howell wrote: > > >>> Is that really true in CPython? ?It seems like you could advance the > >>> pointer instead of shifting all the elements. ?It would create some > >>> nuances with respect to reclaiming the memory, but it seems like an > >>> easy way to make lists perform better under a pretty reasonable use > >>> case. > > >>> Does anybody know off the top of their head if the "have-to-be-shifted- > >>> by-one" warning is actually valid? > > >> Why do you think the documentation has such obvious errors? > > > I wasn't making any assumptions, hence the question mark. ?The Python > > docs are very good, but even the best of projects make advances that > > aren't reflected in the docs. > > >> CPython's list type uses an array of pointers to store its members. The > >> type is optimized for the most common list operations in Python: > >> iteration and appending. Python code rarely changes the head or middle > >> of a list. The dequeue type is an optimized data structure for popping > >> and inserting data at both ends. > > > I disagree that Python code rarely pops elements off the top of a > > list. ?There are perfectly valid use cases for wanting a list over a > > dequeue without having to pay O(N) for pop(0). ?Maybe we are just > > quibbling over the meaning of "rarely." > > >> When you list.pop() or list.insert() the pointers in the internal array > >> must be shifted. appending is much faster because the internal array is > >> overallocated meaning it contains free slots at the tail of the data > >> structure. A linked list of pointers requires more memory and iteration > >> is slower since since it can't utilize the CPU's cache as good as an array. > > > I am not proposing a linked list of pointers. ?I am wondering about > > something like this: > > > p =p[1]; > > (and then reclaim p[0] as free memory, I already said I understood > > that was the tricky bit) > > > The pointer arithmetic for accessing each element would still work in O > > (1), right? > > I think it was Dijkstr (sp?) who said you can accomplish anything with > just one more level of indirection. ?Clearly each attribute (variable) > that has a binding to a given list must point to a fixed piece of > memory, that cannot safely be moved, because there's no efficient way to > find all the attributes. ? That fixed piece is the list object, and I > expect it's 16 or 20 bytes, on a 32bit implementation. ?It must in turn > point to the actual malloc'ed block that contains pointers to all the > elements of the list. ?So that block will be 4*n where n is the number > of reserved cells, at least as large as len(). ?This is the block where > copying takes place when you insert or delete from the beginning. > The indirection is already in Python, as it (at least appears to me) that everything is deferenced off of an ob_item pointer: http://svn.python.org/view/python/trunk/Objects/listobject.c?view=markup > The list object must contain a pointer to the beginning of this block, > or it wouldn't be able to free() it later. ?So you'd be suggesting a > second pointer that actually points to the current 0th pointer. ?And a > pop would simply increment this second pointer. > Yes, ob_item would point to the 0th element pointer and pop would simply increment it. The additional bookkeeping would be the original pointer. > Such an approach might be worthwhile if you expect lots of pops and > pushes, with a minimal net change. ?But of course each time you did a > pop, you'd have to decide whether it was time to normalize/compact ?the > block. > Yes, and that of course is the tricky bit. > As you say, reclaiming the 0th element of this block to the memory pool > would be tricky. ? I should clarify that a bit. Reclaiming the 0th element *cheaply* is tricky, unless you want to rewrite the memory manager. But I also think you can, of course, defer reclaiming the element. > Doubly so, because ?1) the C memory allocator has no > such notion as resizing the beginning of a block. and 2) it would have > nothing useful to do with the 4 bytes freed. ?The minimum allocated > block in Python is probably 16 bytes of actual address space. ?I'd guess > that's 4 bytes for malloc's overhead, and 8 bytes for the minimum object > header, and 4 bytes for data. ?To check me, try: > > ?>>> a = 5.3 > ?>>> b = 49.6 > ?>>> id(a), id(b) > (11074136, 11074152) > > Anyway, this could be done, where once the two pointers get some > distance apart, you do a realloc, and copy everything. ?But of course > you'd want to build some hysteresis into it, to avoid thrashing. > , but There wouldn't be any additional thrashing beyond what happens now. You'd simply avoid the first N-1 memmoves of up to kN bytes at the cost of not reclaiming those k(N-1) bytes right away. I suppose it's a more "bursty" penalty you'd be paying, but the peak of the "bursty" curve is no wider than the "constant" curve, it's just N times narrower! > There wouldn't be much of a performance hit, but it would increase every > list size by 4 bytes minimum. ?So I doubt it would be a list replacement. > I don't think that's the reason people would oppose this, although you are true about the penalty. Memory's cheap, you'd need about a quarter million lists just to fill up a meg. CPU cycles, on the other hand, are expensive, as users' demand for responsive programs seems to do a better job of keeping up with Moore's Law. I'd also argue that the memory you keep around to avoid unnecessary memmoves() will almost always be dwarfed by the memory used by the list elements themselves. From mrm at unknown.nospam Fri Jan 22 18:23:20 2010 From: mrm at unknown.nospam (Mr.M) Date: Fri, 22 Jan 2010 23:23:20 GMT Subject: PyArg_ParseTupleAndKeywords Message-ID: Hi, i can't understand what i'm doing wrong. I have a c/api that implements a new class. In (initproc) function i have somethink like this: [code] (some declarations omitted here) static char* keywordlist[] = {"service", "event_type", "free_text", "group_uid", "remote_name", "remote_abook_uid", "start_time", "end_time", "storage_time", "flags", "bytes_sent", "bytes_received", NULL}; if (!PyArg_ParseTupleAndKeywords(args, keywords, "|ssssssiiiiii", keywordlist, &service, &event_type, &free_text, &group_uid, &remote_name, &remote_abook_uid, &start_time, &end_time, &storage_time, &flags, &bytes_sent, &bytes_received)) return -1; [/code] Ok, what i want is something like this: [code] import mymodule a = mymodule.myclass() # ok, this works a = mymodule.myclass(flags = 1) # bad, this won't work > TypeError: expected string or Unicode object, tuple found [/code] I found that if i write this everything works fine: [code] import mymodule a = mymodule.myclass(service = "blabla", event_type = "event", free_text = "free", group_uid = "group", remote_name = "remote name", remote_abook_uid = "abook", start_time = 1, end_time = 61, storage_time = 61, flags = 2) [/code] in other words, the code only works if *EVERY* argument is specified in exactly the same position it appears in keyword-null-terminated array. Could you please help me? Thank you, Luca. From showell30 at yahoo.com Fri Jan 22 18:27:57 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 15:27:57 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> Message-ID: <98105b09-f522-432e-bac6-caa5010e9699@p24g2000yqm.googlegroups.com> On Jan 22, 3:17?pm, Christian Heimes wrote: > Steve Howell wrote: > > That maybe would be an argument for just striking the paragraph from > > the tutorial. ?If it's rare that people pop the head off the list in > > code that is performance critical or prominent, why bother to even > > mention it in the tutorial? > > How else are you going to teach new Python developers that they should > favor append() and pop() in place of insert() and pop(0)? :) > > > Impossible is a strong word. ?You could be lazy about giving the > > memory back, and just wait till the whole list is garbage collected. > > I don't think there's anything in Python's contract that says memory > > has to be freed the exact moment you stop using it, especially since > > we're talking about doing an O(N) memmove just to free up one > > pointer's worth of memory. > > Your proposal would increase the size of every list object of > sizeof(ptr) or ssize_t (32 or 64bits) in order to store the information > where the first element is. It would also unnecessarily complicate the > code and possible slow down a lot of list operations. > 64 bits per list is negligible. Adding a check for (ilow == 0) is even more negligible. You are not "unnecessarily" complicating code for something as widely used as Python lists if it achieves the desired benefit at minimal cost. You are complicating the code, but not "unnecessarily." > > I know the Python programmer could simply iterate through the list > > rather than popping off unused elements, but that just means that you > > not only tie up the memory for the pointers just as long, but you also > > prevent the objects themselves from being garbage collected. > > > In my case I'm not really concerned about giving the memory back right > > away, it's more about keeping my code simple. ?Once I'm done with an > > element, I do just want to pop it off and keep all the simplicity of > > the lists, but I am just concerned enough speed that for a 1000 > > element list, I don't want to be doing 1000 memmoves for an average of > > 500 pointers, which effectively moves about a million bytes around for > > no reason. > > The simplicity of Python is gained with some performance drawbacks. You > have to learn to use Python algorithms. You can't simply re implement a > fast C algorithm and expect it to be fast in Python, too. > I actually do expect Python to solve performance problems for me that are more easily solved in core than in Python itself. So I guess that's where we differ. From cjns1989 at gmail.com Fri Jan 22 18:38:44 2010 From: cjns1989 at gmail.com (Chris Jones) Date: Fri, 22 Jan 2010 18:38:44 -0500 Subject: Using dictionary key as a regular expression class In-Reply-To: References: Message-ID: <20100122233844.GJ28576@turki.gavron.org> On Fri, Jan 22, 2010 at 05:07:13PM EST, Arnaud Delobelle wrote: [..] > import codecs > from collections import defaultdict > > tcounters = defaultdict(int) > f = codecs.open('/home/gavron/git/screen/src/screen.c', 'r', "utf-8") > > for c in f.read(): > tcounters[c] += 1 > > for c, n in tcounters.iteritems(): > print "%r\t%i" % (c, n) Ah, yes.. much better - I grew suspicious of my 'effort' when I realized I could have written a shorter version in C. :-) CJ From aahz at pythoncraft.com Fri Jan 22 18:51:14 2010 From: aahz at pythoncraft.com (Aahz) Date: 22 Jan 2010 15:51:14 -0800 Subject: Efficient Running Median References: <497af344-31b5-4d1a-9b1a-c3d82feb31d3@j5g2000yqm.googlegroups.com> Message-ID: In article <497af344-31b5-4d1a-9b1a-c3d82feb31d3 at j5g2000yqm.googlegroups.com>, Raymond Hettinger wrote: > >The performance of an IndexableSkiplist is similar to a B+tree but the >implementation in pure python is much simpler. Nice! Can you summarize why IndexableSkipList is simpler? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From daniel at stutzbachenterprises.com Fri Jan 22 19:05:36 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Fri, 22 Jan 2010 18:05:36 -0600 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <98105b09-f522-432e-bac6-caa5010e9699@p24g2000yqm.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <98105b09-f522-432e-bac6-caa5010e9699@p24g2000yqm.googlegroups.com> Message-ID: On Fri, Jan 22, 2010 at 5:27 PM, Steve Howell wrote: > I actually do expect Python to solve performance problems for me that > are more easily solved in core than in Python itself. So I guess > that's where we differ. > You might be interested in the extension type I wrote (the "blist") that looks, acts, and quacks like a list, but takes worst-case O(log n) time for inserting and removing elements anywhere in the list. It's available for download here: http://pypi.python.org/pypi/blist/ And there's a detailed performance comparison with the built-in list here: http://stutzbachenterprises.com/performance-blist -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From showell30 at yahoo.com Fri Jan 22 19:09:03 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 16:09:03 -0800 (PST) Subject: medians for degree measurements Message-ID: I just saw the thread for medians, and it reminded me of a problem that I need to solve. We are writing some Python software for sailing, and we need to detect when we've departed from the median heading on the leg. Calculating arithmetic medians is straightforward, but compass bearings add a twist. The numerical median of 1, 2, 3, 4, 5, 6, 359 is 4. But for navigational purposes you would actually order the numbers 359, 1, 2, 3, 4, 5, 6, so the desired median heading of the boat is actually 3. Of course, you could express 359 better as -1 degrees to north, then the sequence would be -1, 1, 2, 3, 4, 5, and 6. And you'd be good. But that trick does not generalize if you go south instead, as you have similar issues with -179, 174, 175, 176, 177, 178, and 179. Has anybody solved this in Python, either for compass bearings or a different domain? I can think of kind of a brute force solution where you keep rotating the sequence until the endpoints are closest together mod 360, but I wonder if there is something more elegant. From robert.kern at gmail.com Fri Jan 22 19:45:46 2010 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 22 Jan 2010 18:45:46 -0600 Subject: medians for degree measurements In-Reply-To: References: Message-ID: On 2010-01-22 18:09 PM, Steve Howell wrote: > I just saw the thread for medians, and it reminded me of a problem > that I need to solve. We are writing some Python software for > sailing, and we need to detect when we've departed from the median > heading on the leg. Calculating arithmetic medians is > straightforward, but compass bearings add a twist. > > The numerical median of 1, 2, 3, 4, 5, 6, 359 is 4. But for > navigational purposes you would actually order the numbers 359, 1, 2, > 3, 4, 5, 6, so the desired median heading of the boat is actually 3. > > Of course, you could express 359 better as -1 degrees to north, then > the sequence would be -1, 1, 2, 3, 4, 5, and 6. And you'd be good. > > But that trick does not generalize if you go south instead, as you > have similar issues with -179, 174, 175, 176, 177, 178, and 179. > > Has anybody solved this in Python, either for compass bearings or a > different domain? I can think of kind of a brute force solution where > you keep rotating the sequence until the endpoints are closest > together mod 360, but I wonder if there is something more elegant. Brute force doesn't suck too much when using numpy to do the heavy lifting. In [158]: import numpy as np # Worst case. A von Mises distribution "centered" around the "South pole" # at pi/-pi. In [159]: theta = np.random.vonmises(np.pi, 1.0, size=1000) # Complex division makes circular distances easier to compute. In [160]: z = np.exp(1j * theta) # The newaxis bit plus numpy's broadcasting yields a 1000x1000 array of # the quotient of each pair of points in the dataset. In [161]: zdiv = z / z[:,np.newaxis] # Convert back to angular distances. Take the absolute value. Sum along one # dimension. Find the index of the element with the smallest mean absolute # circular deviation when used as the putative median. In [162]: i = abs(np.arctan2(zdiv.imag, zdiv.real)).sum(axis=1).argmin() # As expected, the result is close to the mode of the distribution. In [163]: theta[i] Out[163]: 3.0886753423606521 -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From python at mrabarnett.plus.com Fri Jan 22 20:12:11 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 23 Jan 2010 01:12:11 +0000 Subject: medians for degree measurements In-Reply-To: References: Message-ID: <4B5A4CEB.1020901@mrabarnett.plus.com> Steve Howell wrote: > I just saw the thread for medians, and it reminded me of a problem > that I need to solve. We are writing some Python software for > sailing, and we need to detect when we've departed from the median > heading on the leg. Calculating arithmetic medians is > straightforward, but compass bearings add a twist. > > The numerical median of 1, 2, 3, 4, 5, 6, 359 is 4. But for > navigational purposes you would actually order the numbers 359, 1, 2, > 3, 4, 5, 6, so the desired median heading of the boat is actually 3. > > Of course, you could express 359 better as -1 degrees to north, then > the sequence would be -1, 1, 2, 3, 4, 5, and 6. And you'd be good. > > But that trick does not generalize if you go south instead, as you > have similar issues with -179, 174, 175, 176, 177, 178, and 179. > > Has anybody solved this in Python, either for compass bearings or a > different domain? I can think of kind of a brute force solution where > you keep rotating the sequence until the endpoints are closest > together mod 360, but I wonder if there is something more elegant. > When you read the headings clockwise the values normally increase and you pick the middle one. However, when you cross north the values decrease. You can spot whether the set of headings crosses north by checking whether the difference between the minimum and maximum is greater than 180. If it is then make the westerly headings negative, sort the values, and pick the middle one, adding 180 if it's negative. def compass_median(points): points = sorted(points) if points[-1] - points[0] > 180: points = [(p - 360 if p > 180 else p) for p in points] points.sort() median = points[len(points) // 2] return median + 360 if median < 0 else median From tjreedy at udel.edu Fri Jan 22 20:46:35 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 22 Jan 2010 20:46:35 -0500 Subject: Using dictionary key as a regular expression class In-Reply-To: <20100122214755.GH28576@turki.gavron.org> References: <20100122214755.GH28576@turki.gavron.org> Message-ID: On 1/22/2010 4:47 PM, Chris Jones wrote: > I was writing a script that counts occurrences of characters in source code files: > > #!/usr/bin/python > import codecs > tcounters = {} > f = codecs.open('/home/gavron/git/screen/src/screen.c', 'r', "utf-8") > for uline in f: > lline = [] > for char in uline[:-1]: > lline += [char] Same but slower than lline.append(char), however, this loop just uselessless copies uline[:1] > counters = {} > for i in set(lline): > counters[i] = lline.count(i) slow way to do this > for c in counters.keys(): > if c in tcounters: > tcounters[c] += counters[c] > else: > tcounters.update({c: counters[c]}) I do not see the reason for intermediate dict > counters = {} duplicate line > for c in tcounters.keys(): > print c, '\t', tcounters[c] To only count ascii chars, as should be the case for C code, achars = [0]*63 for c in open('xxx', 'c'): try: achars[ord(c)-32] += 1 except IndexError: pass for i,n in enumerate(achars) print chr(i), n or sum subsets as desired. Terry Jan Reedy From kw at codebykevin.com Fri Jan 22 20:52:13 2010 From: kw at codebykevin.com (Kevin Walzer) Date: Fri, 22 Jan 2010 20:52:13 -0500 Subject: Patch for IDLE/OS X to work with Tk-Cocoa In-Reply-To: <15238f19-e008-4e01-b284-721d3deba8ee@c29g2000yqd.googlegroups.com> References: <15238f19-e008-4e01-b284-721d3deba8ee@c29g2000yqd.googlegroups.com> Message-ID: <3dc5d$4b5a564d$4275d90a$26905@FUSE.NET> On 1/22/10 5:47 PM, G73 wrote: > im trying to update a patch. here is link to various patches > http://bugs.python.org/issue6075 > > how do i update the patch, say for EditorWindow.patch. > i have located my python installation the EditorWindow.py, and i can > see some differences (which lines from the patch actually go into the > EditorWindow.py ?. > Also assuming i know which lines to edit, do i just edit the > EditorWindow.py and that is it ? will i then be abler to run IDLE and > this then pick up the changes i have made in EditorWindow/py ? > > im new to python and mac, but trying to learn it. To apply a patch, run "patch < patchfile," with patchfile being the patch. The command-line tool will prompt you for the name of the file to patch. I'm the author of those patches--hope you find them useful! -- Kevin Walzer Code by Kevin http://www.codebykevin.com From pavlovevidence at gmail.com Fri Jan 22 21:08:26 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Fri, 22 Jan 2010 18:08:26 -0800 (PST) Subject: PyArg_ParseTupleAndKeywords References: Message-ID: <91afd9e3-93a1-488d-8e83-299bfc222e24@22g2000yqr.googlegroups.com> On Jan 22, 3:23?pm, "Mr.M" wrote: > Hi, > > i can't understand what i'm doing wrong. I have a c/api that implements > a new class. > In (initproc) function i have somethink like this: > > [code] > > (some declarations omitted here) You probably shouldn't have, that could be where the error is.... I'd include the whole function up to the call that raises the exception. > static char* keywordlist[] = {"service", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"event_type", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"free_text", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"group_uid", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"remote_name", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"remote_abook_uid", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"start_time", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"end_time", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"storage_time", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"flags", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"bytes_sent", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"bytes_received", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? NULL}; > > if (!PyArg_ParseTupleAndKeywords(args, keywords, "|ssssssiiiiii", > keywordlist, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &service, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &event_type, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &free_text, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &group_uid, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &remote_name, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &remote_abook_uid, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &start_time, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &end_time, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &storage_time, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &flags, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &bytes_sent, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &bytes_received)) return -1; This should be return NULL in most cases, but not all (such as if it's an object's tp_init--and it does look like that's the case here, so probably not the issue). > [/code] > > Ok, what i want is something like this: > > [code] > import mymodule > a = mymodule.myclass() # ok, this works > a = mymodule.myclass(flags = 1) # bad, this won't work > ?> TypeError: expected string or Unicode object, tuple found > [/code] Are you sure that PyArg_ParseTupleAndKeywords is what's raising the error? Are you subclassing this type in Python, and passing one of the string parameters a tuple by mistake? For instance, did you do something like this: class myclass(_mycmodule._myctype): def __init__(self,*args,**kwargs): log_something_here() super(myclass,self).__init__(args,**kwargs) Note the missing * on args. > I found that if i write this everything works fine: > > [code] > import mymodule > a = mymodule.myclass(service = "blabla", event_type = "event", free_text > ? = "free", group_uid = "group", remote_name = "remote name", > remote_abook_uid = "abook", start_time = 1, end_time = 61, storage_time > = 61, flags = 2) > [/code] > > in other words, the code only works if *EVERY* argument is specified in > exactly the same position it appears in keyword-null-terminated array. That it would have to be in a certain order is strange. Are you sure it's just not merely that they all have to be present? > Could you please help me? Tricky, but I think it'd help if you provided more information. My gut feeling is that there exception is being raised somewhere other than you think it is (i.e., not by PyArg_ParseTuple), so I'd recommend adding some debugging statements to track down the exact point wher the error occurs. Carl Banks From steve at REMOVE-THIS-cybersource.com.au Fri Jan 22 21:20:14 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 23 Jan 2010 02:20:14 GMT Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> Message-ID: <036a4966$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 14:38:18 -0800, Steve Howell wrote: > I know the Python programmer could simply iterate through the list > rather than popping off unused elements, but that just means that you > not only tie up the memory for the pointers just as long, but you also > prevent the objects themselves from being garbage collected. > > In my case I'm not really concerned about giving the memory back right > away, it's more about keeping my code simple. Once I'm done with an > element, I do just want to pop it off and keep all the simplicity of the > lists, but I am just concerned enough speed that for a 1000 element > list, I don't want to be doing 1000 memmoves for an average of 500 > pointers, which effectively moves about a million bytes around for no > reason. > > I suppose the solution is to either give up the sugar of lists, or try > to wrap something like deque or list-with-offset into a sequence. I don't understand what the actual problem you're trying to solve is. Despite your disclaimer about not being concerned about reclaiming the memory, it sounds like you're trying to micro-optimize memory usage. That is, you give me the impression that you prefer this: while alist: x = alist.pop(0) process(x) over this: for x in alist: process(x) # allow alist to be garbage collected when it goes out of scope That strikes me as a pointlessly trivial optimization, even if deleting at the start of the list was efficient. But whatever your reason, if you want to insert and delete efficiently from both ends of the sequence, use a deque. If you are only doing a small number of insertions/deletions at the beginning, and so don't care about inefficiency, use a list. If you only want to insert/delete from one end, use a list. Instead of: alist.insert(0, x) alist.pop(0) use: alist.append(x) alist.pop() That's fast and efficient. In some cases it doesn't matter which order the list is, but if it does matter, the worst case is that you need to call alist.reverse() occasionally, which is quite fast. Or iterate over the list in reverse order, which is even faster. So what am I missing? -- Steven From steve at holdenweb.com Fri Jan 22 21:31:17 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 22 Jan 2010 21:31:17 -0500 Subject: [ANN] hgview 1.2.0 In-Reply-To: <4b59df90$0$6582$9b4e6d93@newsspool3.arcor-online.net> References: <4b59df90$0$6582$9b4e6d93@newsspool3.arcor-online.net> Message-ID: <4B5A5F75.2020502@holdenweb.com> Stefan Behnel wrote: > For those who are a bit less fluent in French: > [...] ... and, while having access to e-mail, have not yet come across translate.google.com? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Fri Jan 22 21:31:17 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 22 Jan 2010 21:31:17 -0500 Subject: [ANN] hgview 1.2.0 In-Reply-To: <4b59df90$0$6582$9b4e6d93@newsspool3.arcor-online.net> References: <4b59df90$0$6582$9b4e6d93@newsspool3.arcor-online.net> Message-ID: <4B5A5F75.2020502@holdenweb.com> Stefan Behnel wrote: > For those who are a bit less fluent in French: > [...] ... and, while having access to e-mail, have not yet come across translate.google.com? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Fri Jan 22 21:41:18 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 22 Jan 2010 21:41:18 -0500 Subject: Mastering Python 3 I/O - Special Preview - Feb 5, 2010 (Chicago) In-Reply-To: <4EB3255C-2B1E-44E5-BE72-D61E44E580B4@dabeaz.com> References: <4EB3255C-2B1E-44E5-BE72-D61E44E580B4@dabeaz.com> Message-ID: <4B5A61CE.2030608@holdenweb.com> Dave: New York classes went well this week, and there appears to be some demand for Chicago training. How can we satisfy this demand to our common profit? regards Steve David Beazley wrote: > Mastering Python 3 I/O > ** PyCON'2010 Tutorial Preview in Chicago ** > > with David Beazley > February 5, 2010, 12pm - 5pm > http://www.dabeaz.com/chicago/index.html > > Can't make it to PyCON, but want to attend a cutting-edge tutorial on > the latest Python features? Join David Beazley, author of the Python > Essential Reference, in Chicago for a preview of his new tutorial > "Mastering Python 3 I/O." The goal of this tutorial is to take a top > to bottom tour of the Python 3 I/O system and to focus on essential > features that you must know if you are ever going to port existing > applications to Python 3 or use it for real applications. This > tutorial promises to go far beyond what you find in the documentation > and books (Dave's included). You'll learn about tricky gotchas, see > interesting practical examples, and get a better grasp of how Python 3 > is put together. > > This tutorial preview includes a free copy of the "Python Essential > Reference, 4th Ed.", lunch at one of Chicago's finest new restaurants, > artisinal pastries and more--all for the same price as a tutorial at > PyCON. However, it's strictly limited to 8 attendees. > > More information is available at: > > http://www.dabeaz.com/chicago/index.html > > Cheers, > Dave > -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From cjns1989 at gmail.com Fri Jan 22 21:58:48 2010 From: cjns1989 at gmail.com (Chris Jones) Date: Fri, 22 Jan 2010 21:58:48 -0500 Subject: Using dictionary key as a regular expression class In-Reply-To: References: <20100122214755.GH28576@turki.gavron.org> Message-ID: <20100123025848.GC2958@turki.gavron.org> On Fri, Jan 22, 2010 at 08:46:35PM EST, Terry Reedy wrote: > On 1/22/2010 4:47 PM, Chris Jones wrote: >> I was writing a script that counts occurrences of characters in source code files: >> >> #!/usr/bin/python >> import codecs >> tcounters = {} >> f = codecs.open('/home/gavron/git/screen/src/screen.c', 'r', "utf-8") >> for uline in f: >> lline = [] >> for char in uline[:-1]: >> lline += [char] > > Same but slower than lline.append(char), however, this loop just > uselessless copies uline[:1] I'll change that. Do you mean I should just read the file one character at a time? That was my original intention but I didn't find the way to do it. >> counters = {} >> for i in set(lline): >> counters[i] = lline.count(i) > > slow way to do this > >> for c in counters.keys(): >> if c in tcounters: >> tcounters[c] += counters[c] >> else: >> tcounters.update({c: counters[c]}) > > I do not see the reason for intermediate dict Couldn't find a way to increment the counters in the 'grand total' dictionary. I always ended up with the counter values for the last input line :-( Moot point if I can do a for loop reading one character at a time till end of file. >> counters = {} > > duplicate line And totally useless since I never reference it after that. Something I move else where and forgot to delete. Sorry about that. >> for c in tcounters.keys(): >> print c, '\t', tcounters[c] Literals, comments, ?'s..? > To only count ascii chars, as should be the case for C code, > > achars = [0]*63 > for c in open('xxx', 'c'): > try: > achars[ord(c)-32] += 1 > except IndexError: > pass > > for i,n in enumerate(achars) > print chr(i), n > > or sum subsets as desired. Thanks much for the snippet, let me play with it and see if I can come up with a Unicode/utf-8 version.. since while I'm at it I might as well write something a bit more general than C code. Since utf-8 is backward-compatible with 7bit ASCII, this shouldn't be a problem. > Terry Jan Reedy Thank you for your comments! CJ From roy at panix.com Fri Jan 22 22:04:10 2010 From: roy at panix.com (Roy Smith) Date: Fri, 22 Jan 2010 22:04:10 -0500 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: In article , Christian Heimes wrote: > Steve Howell wrote: > > Is that really true in CPython? It seems like you could advance the > > pointer instead of shifting all the elements. It would create some > > nuances with respect to reclaiming the memory, but it seems like an > > easy way to make lists perform better under a pretty reasonable use > > case. > > > > Does anybody know off the top of their head if the "have-to-be-shifted- > > by-one" warning is actually valid? > > Why do you think the documentation has such obvious errors? > > CPython's list type uses an array of pointers to store its members. The > type is optimized for the most common list operations in Python: > iteration and appending. Python code rarely changes the head or middle > of a list. The dequeue type is an optimized data structure for popping > and inserting data at both ends. > > When you list.pop() or list.insert() the pointers in the internal array > must be shifted. Well, at least in theory you could make pop(0) run in O(1). All you need to do is have each list store an offset. Initially it's 0, and pop(0) would just increment the offset. Then, all references to alist[i] would turn into array[i+offset]. Of course, that's a lot of complexity to optimize a relatively rare use case. You're probably better off just using a dequeue :-) From roy at panix.com Fri Jan 22 22:09:06 2010 From: roy at panix.com (Roy Smith) Date: Fri, 22 Jan 2010 22:09:06 -0500 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> Message-ID: In article <3ac173bd-4124-434d-b726-0b9baaeec752 at 36g2000yqu.googlegroups.com>, Steve Howell wrote: > In my case I'm not really concerned about giving the memory back right > away, it's more about keeping my code simple. Once I'm done with an > element, I do just want to pop it off and keep all the simplicity of > the lists, but I am just concerned enough speed that for a 1000 > element list, I don't want to be doing 1000 memmoves for an average of > 500 pointers, which effectively moves about a million bytes around for > no reason. If you really want to pop all the elements from a long list, reverse the list and pop them off the end. Popping every element off the beginning of the list is O(n^2), as you pointed out. Reversing the list is O(n), and each pop after that is O(1), so the overall complexity is O(n). From sccolbert at gmail.com Fri Jan 22 22:47:20 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Fri, 22 Jan 2010 22:47:20 -0500 Subject: Mastering Python 3 I/O - Special Preview - Feb 5, 2010 (Chicago) In-Reply-To: <4B5A61CE.2030608@holdenweb.com> References: <4EB3255C-2B1E-44E5-BE72-D61E44E580B4@dabeaz.com> <4B5A61CE.2030608@holdenweb.com> Message-ID: <7f014ea61001221947o7e17f0ebv9784c427346f6f08@mail.gmail.com> oops :) On Fri, Jan 22, 2010 at 9:41 PM, Steve Holden wrote: > Dave: > > New York classes went well this week, and there appears to be some > demand for Chicago training. How can we satisfy this demand to our > common profit? > > regards > Steve > > David Beazley wrote: > > Mastering Python 3 I/O > > ** PyCON'2010 Tutorial Preview in Chicago ** > > > > with David Beazley > > February 5, 2010, 12pm - 5pm > > http://www.dabeaz.com/chicago/index.html > > > > Can't make it to PyCON, but want to attend a cutting-edge tutorial on > > the latest Python features? Join David Beazley, author of the Python > > Essential Reference, in Chicago for a preview of his new tutorial > > "Mastering Python 3 I/O." The goal of this tutorial is to take a top > > to bottom tour of the Python 3 I/O system and to focus on essential > > features that you must know if you are ever going to port existing > > applications to Python 3 or use it for real applications. This > > tutorial promises to go far beyond what you find in the documentation > > and books (Dave's included). You'll learn about tricky gotchas, see > > interesting practical examples, and get a better grasp of how Python 3 > > is put together. > > > > This tutorial preview includes a free copy of the "Python Essential > > Reference, 4th Ed.", lunch at one of Chicago's finest new restaurants, > > artisinal pastries and more--all for the same price as a tutorial at > > PyCON. However, it's strictly limited to 8 attendees. > > > > More information is available at: > > > > http://www.dabeaz.com/chicago/index.html > > > > Cheers, > > Dave > > > > > -- > Steve Holden +1 571 484 6266 +1 800 494 3119 > PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ > Holden Web LLC http://www.holdenweb.com/ > UPCOMING EVENTS: http://holdenweb.eventbrite.com/ > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alekseymv at gmail.com Sat Jan 23 00:23:37 2010 From: alekseymv at gmail.com (Aleksey) Date: Fri, 22 Jan 2010 21:23:37 -0800 (PST) Subject: Problems with the date of modification of files on the flash drive in windows Message-ID: Hi All, I write crossplatform program and have next problem under windows (under linux is all OK) : I'm trying to get the UTC modification date of files on the flash drive under windows. In the flash card system is FAT. In the winter time (2010 01 21) Date of modification: >>> op.getmtime('g:\\alex\\bag\\pybag.log') 1263998652.0 In the summer time (2010 07 21) Date of the modification on the hour (3600 sec) different from winter time: >>> op.getmtime('g:\\alex\\bag\\pybag.log') 1263995052.0 Also time zone is "GMT" but real is "OMST" and localtime is wrong. In windows "tm_isdst" is always "0". For files on the HDD with NTFS modification time is right. How I can get right GMT time for files in flash with FAT under windows? Winter for Windows ---------------------- >>> time.altzone -3600 >>> time.daylight 0 >>> time.localtime() time.struct_time(tm_year=2010, tm_mon=1, tm_mday=21, tm_hour=3, tm_min=32, tm_sec=42, tm_wday=3, tm_yday=21, tm_isdst=0) >>> time.gmtime() time.struct_time(tm_year=2010, tm_mon=1, tm_mday=21, tm_hour=3, tm_min=32, tm_sec=48, tm_wday=3, tm_yday=21, tm_isdst=0) >>> time.tzname ('GMT', '') >>> time.timezone 0 Summer for Windows ----------------------------- >>> time.altzone -3600 >>> time.daylight 0 >>> time.localtime() time.struct_time(tm_year=2010, tm_mon=7, tm_mday=21, tm_hour=2, tm_min=13, tm_sec=18, tm_wday=2, tm_yday=202, tm_isdst=0) >>> time.gmtime() time.struct_time(tm_year=2010, tm_mon=7, tm_mday=21, tm_hour=2, tm_min=13, tm_sec=26, tm_wday=2, tm_yday=202, tm_isdst=0) >>> time.tzname ('GMT', '') >>> time.timezone 0 >From Linux for summer: --------------------------------- >>> time.altzone -25200 >>> time.daylight 1 >>> time.localtime() time.struct_time(tm_year=2010, tm_mon=7, tm_mday=23, tm_hour=12, tm_min=7, tm_sec=24, tm_wday=4, tm_yday=204, tm_isdst=1) >>> time.gmtime() time.struct_time(tm_year=2010, tm_mon=7, tm_mday=23, tm_hour=5, tm_min=7, tm_sec=36, tm_wday=4, tm_yday=204, tm_isdst=0) >>> time.timezone -21600 >>> time.tzname ('OMST', 'OMSST') >From Linux for winter: ------------------------------------ >>> time.altzone -25200 >>> time.daylight 1 >>> time.localtime() time.struct_time(tm_year=2010, tm_mon=1, tm_mday=23, tm_hour=11, tm_min=8, tm_sec=26, tm_wday=5, tm_yday=23, tm_isdst=0) >>> time.gmtime() time.struct_time(tm_year=2010, tm_mon=1, tm_mday=23, tm_hour=5, tm_min=8, tm_sec=39, tm_wday=5, tm_yday=23, tm_isdst=0) >>> time.timezone -21600 >>> time.tzname ('OMST', 'OMSST') From showell30 at yahoo.com Sat Jan 23 00:42:43 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 21:42:43 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> Message-ID: On Jan 22, 6:20?pm, Steven D'Aprano wrote: > On Fri, 22 Jan 2010 14:38:18 -0800, Steve Howell wrote: > > I know the Python programmer could simply iterate through the list > > rather than popping off unused elements, but that just means that you > > not only tie up the memory for the pointers just as long, but you also > > prevent the objects themselves from being garbage collected. > > > In my case I'm not really concerned about giving the memory back right > > away, it's more about keeping my code simple. ?Once I'm done with an > > element, I do just want to pop it off and keep all the simplicity of the > > lists, but I am just concerned enough speed that for a 1000 element > > list, I don't want to be doing 1000 memmoves for an average of 500 > > pointers, which effectively moves about a million bytes around for no > > reason. > > > I suppose the solution is to either give up the sugar of lists, or try > > to wrap something like deque or list-with-offset into a sequence. > > I don't understand what the actual problem you're trying to solve is. > Despite your disclaimer about not being concerned about reclaiming the > memory, it sounds like you're trying to micro-optimize memory usage. My discussion of memory probably distracted you from the fact that I'm not proposing a micro-optimization of memory; I am proposing a mega- optimization of CPU. This innocent program here literally moves about a million bytes of memory around for no good reason: lst = [] for i in range(2000): lst.append(i) while lst: print lst.pop(0) Why? Because list.pop(0) is implemented in O(N) instead of O(1). Why wouldn't you get a competent C programmer simply make list_ass_slice smart enough to make list.pop(0) O(1)? The brilliant computer scientist, Christian Heimes, provides the answers, and I am paraphrasing here, of course: 1) You can save 64 bits for every list by not storing an extra pointer! 2) You can simplify the CPython implementation! 3) You can avoid the oh-so-expensive check "if ilow == 1" for all operations that don't need this optimization! Sounds like two micro-optimizations to me (and a copout to boot). > That > is, you give me the impression that you prefer this: > > while alist: > ? ? x = alist.pop(0) > ? ? process(x) > > over this: > > for x in alist: > ? ? process(x) > # allow alist to be garbage collected when it goes out of scope > No, to be more precise, I prefer this implementation of a recursive parser (using lists) to one that would have to use deque's because of the lameness of Python's list implementation: https://bitbucket.org/showell/shpaml_website/src/tip/shpaml.py From showell30 at yahoo.com Sat Jan 23 00:58:28 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 21:58:28 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> Message-ID: <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> On Jan 22, 7:09?pm, Roy Smith wrote: > In article > <3ac173bd-4124-434d-b726-0b9baaeec... at 36g2000yqu.googlegroups.com>, > ?Steve Howell wrote: > > > In my case I'm not really concerned about giving the memory back right > > away, it's more about keeping my code simple. ?Once I'm done with an > > element, I do just want to pop it off and keep all the simplicity of > > the lists, but I am just concerned enough speed that for a 1000 > > element list, I don't want to be doing 1000 memmoves for an average of > > 500 pointers, which effectively moves about a million bytes around for > > no reason. > > If you really want to pop all the elements from a long list, reverse the > list and pop them off the end. ?Popping every element off the beginning of > the list is O(n^2), as you pointed out. ?Reversing the list is O(n), and > each pop after that is O(1), so the overall complexity is O(n). I really want to use list *normally* with all its perfectly good semantics and reasonable implementation, except for its blind spot with respect to popping the first element off the list. The whole reason I use CPython vs. C in the first place is that CPython programmers can generally program basic data structures better than I can. But list.pop(0) is the exception. And, with the possible exception of dicts, lists are the most fundamental data structures that Python has. I know Python's number one concern will never be speed, but if Python makes an O(1) operation into an unnecessarily O(N) operation for no good reasons other than "it's too complicated, " or it "adds another pointer to the structure," or "it adds another conditional check to list_ass_slice for operations that aren't popping off the top," I think it's reasonable to challenge the design philosophy. From showell30 at yahoo.com Sat Jan 23 01:09:54 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 22:09:54 -0800 (PST) Subject: medians for degree measurements References: Message-ID: <044268c0-c308-406e-8d5b-a38ef5b5e381@p24g2000yqm.googlegroups.com> On Jan 22, 5:12?pm, MRAB wrote: > Steve Howell wrote: > > I just saw the thread for medians, and it reminded me of a problem > > that I need to solve. ?We are writing some Python software for > > sailing, and we need to detect when we've departed from the median > > heading on the leg. ?Calculating arithmetic medians is > > straightforward, but compass bearings add a twist. > > > The numerical median of 1, 2, 3, 4, 5, 6, 359 is 4. ?But for > > navigational purposes you would actually order the numbers 359, 1, 2, > > 3, 4, 5, 6, so the desired median heading of the boat is actually 3. > > > Of course, you could express 359 better as -1 degrees to north, then > > the sequence would be -1, 1, 2, 3, 4, 5, and 6. ?And you'd be good. > > > But that trick does not generalize if you go south instead, as you > > have similar issues with -179, 174, 175, 176, 177, 178, and 179. > > > Has anybody solved this in Python, either for compass bearings or a > > different domain? ?I can think of kind of a brute force solution where > > you keep rotating the sequence until the endpoints are closest > > together mod 360, but I wonder if there is something more elegant. > > When you read the headings clockwise the values normally increase and > you pick the middle one. However, when you cross north the values > decrease. You can spot whether the set of headings crosses north by > checking whether the difference between the minimum and maximum is > greater than 180. If it is then make the westerly headings negative, > sort the values, and pick the middle one, adding 180 if it's negative. > > def compass_median(points): > ? ? ?points = sorted(points) > ? ? ?if points[-1] - points[0] > 180: > ? ? ? ? ?points = [(p - 360 if p > 180 else p) for p in points] > ? ? ? ? ?points.sort() > ? ? ?median = points[len(points) // 2] > ? ? ?return median + 360 if median < 0 else median I like this implementation, and it would probably work 99.9999% of the time for my particular use case. The only (very contrived) edge case that I can think of is when you have 10 bearings to SSW, 10 bearings to SSE, and the two outliers are unfortunately in the NE and NW quadrants. It seems like the algorithm above would pick one of the outliers. Maybe the refinement to the algorithm above would be to find the mean first, by summing sines and cosines of the bearings, taking the quotient, and applying the arctangent. Then use the resulting angle as the equivalent of "due north" and adjust angles to be within (-180, 180) respect to the mean, pretty much as you do in the code above, with minor modifications. I realize the problem as I stated it as sort of ill-defined. The way you stated the solution made me realize more deeply that you basically have a list that needs to be rotated either clockwise or counterclockwise in certain situations. Instead of using the 180- degree check to coarsely (but usually effectively) rotate your frame of reference, you could instead use the mean to eliminate even more edge cases. From nobody at nowhere.com Sat Jan 23 01:29:01 2010 From: nobody at nowhere.com (Nobody) Date: Sat, 23 Jan 2010 06:29:01 +0000 Subject: medians for degree measurements References: Message-ID: On Fri, 22 Jan 2010 16:09:03 -0800, Steve Howell wrote: > I just saw the thread for medians, and it reminded me of a problem > that I need to solve. We are writing some Python software for > sailing, and we need to detect when we've departed from the median > heading on the leg. Calculating arithmetic medians is > straightforward, but compass bearings add a twist. > > The numerical median of 1, 2, 3, 4, 5, 6, 359 is 4. But for > navigational purposes you would actually order the numbers 359, 1, 2, > 3, 4, 5, 6, so the desired median heading of the boat is actually 3. > > Of course, you could express 359 better as -1 degrees to north, then > the sequence would be -1, 1, 2, 3, 4, 5, and 6. And you'd be good. > > But that trick does not generalize if you go south instead, as you > have similar issues with -179, 174, 175, 176, 177, 178, and 179. > > Has anybody solved this in Python, either for compass bearings or a > different domain? I can think of kind of a brute force solution where > you keep rotating the sequence until the endpoints are closest > together mod 360, but I wonder if there is something more elegant. First, there isn't always a solution; what would you consider to be the median of [0, 90, 180, 270]? In the case where the bearings are clustered, one approach is to convert each bearing from polar to cartesian coordinates, compute the centroid, then convert back to polar coordinates, i.e.: from math import degrees, radians, sin, cos, atan2 def mean(bearings): x = sum(sin(radians(a)) for a in bearings) y = sum(cos(radians(a)) for a in bearings) return degrees(atan2(x, y)) Then, subtract the mean from each bearing, coerce all angles into the range -180..+180, calculate the median, add the mean, coerce back to 0..360. def median(bearings): m = mean(bearings) bearings = [(a - m + 180) % 360 - 180 for a in bearings] bearings.sort() median = bearings[len(bearings) / 2] median += m median %= 360 return median From showell30 at yahoo.com Sat Jan 23 02:03:05 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 23:03:05 -0800 (PST) Subject: medians for degree measurements References: Message-ID: <4ba80d57-71cb-442a-8146-2e44dfdc11c9@v25g2000yqk.googlegroups.com> On Jan 22, 10:29?pm, Nobody wrote: > On Fri, 22 Jan 2010 16:09:03 -0800, Steve Howell wrote: > > I just saw the thread for medians, and it reminded me of a problem > > that I need to solve. ?We are writing some Python software for > > sailing, and we need to detect when we've departed from the median > > heading on the leg. ?Calculating arithmetic medians is > > straightforward, but compass bearings add a twist. > > > The numerical median of 1, 2, 3, 4, 5, 6, 359 is 4. ?But for > > navigational purposes you would actually order the numbers 359, 1, 2, > > 3, 4, 5, 6, so the desired median heading of the boat is actually 3. > > > Of course, you could express 359 better as -1 degrees to north, then > > the sequence would be -1, 1, 2, 3, 4, 5, and 6. ?And you'd be good. > > > But that trick does not generalize if you go south instead, as you > > have similar issues with -179, 174, 175, 176, 177, 178, and 179. > > > Has anybody solved this in Python, either for compass bearings or a > > different domain? ?I can think of kind of a brute force solution where > > you keep rotating the sequence until the endpoints are closest > > together mod 360, but I wonder if there is something more elegant. > > First, there isn't always a solution; what would you consider to be the > median of [0, 90, 180, 270]? > You probably posted before seeing my recent post. I agree that the problem is ill-defined for certain cases. > In the case where the bearings are clustered, one approach is to > convert each bearing from polar to cartesian coordinates, compute the > centroid, then convert back to polar coordinates, i.e.: > > ? ? ? ? from math import degrees, radians, sin, cos, atan2 > > ? ? ? ? def mean(bearings): > ? ? ? ? ? ? ? ? x = sum(sin(radians(a)) for a in bearings) > ? ? ? ? ? ? ? ? y = sum(cos(radians(a)) for a in bearings) > ? ? ? ? ? ? ? ? return degrees(atan2(x, y)) > > Then, subtract the mean from each bearing, coerce all angles into the > range -180..+180, calculate the median, add the mean, coerce back to > 0..360. > > ? ? ? ? def median(bearings): > ? ? ? ? ? ? ? ? m = mean(bearings) > ? ? ? ? ? ? ? ? bearings = [(a - m + 180) % 360 - 180 for a in bearings] > ? ? ? ? ? ? ? ? bearings.sort() > ? ? ? ? ? ? ? ? median = bearings[len(bearings) / 2] > ? ? ? ? ? ? ? ? median += m > ? ? ? ? ? ? ? ? median %= 360 > ? ? ? ? ? ? ? ? return median Yep, that's exactly the solution I'm leaning toward now. From aahz at pythoncraft.com Sat Jan 23 02:10:23 2010 From: aahz at pythoncraft.com (Aahz) Date: 22 Jan 2010 23:10:23 -0800 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: In article <83082e19-9130-45a8-91f2-8601c1fdaac3 at 22g2000yqr.googlegroups.com>, Steve Howell wrote: > >I really want to use list *normally* with all its perfectly good >semantics and reasonable implementation, except for its blind spot >with respect to popping the first element off the list. The whole >reason I use CPython vs. C in the first place is that CPython >programmers can generally program basic data structures better than I >can. But list.pop(0) is the exception. And, with the possible >exception of dicts, lists are the most fundamental data structures >that Python has. > >I know Python's number one concern will never be speed, but if Python >makes an O(1) operation into an unnecessarily O(N) operation for no >good reasons other than "it's too complicated, " or it "adds another >pointer to the structure," or "it adds another conditional check to >list_ass_slice for operations that aren't popping off the top," I >think it's reasonable to challenge the design philosophy. "Rough consensus and running code." You have a good point, but nobody will ever give your idea serious attention until there's a patch and benchmarks. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From nobody at nowhere.com Sat Jan 23 02:19:40 2010 From: nobody at nowhere.com (Nobody) Date: Sat, 23 Jan 2010 07:19:40 +0000 Subject: Problems with the date of modification of files on the flash drive in windows References: Message-ID: On Fri, 22 Jan 2010 21:23:37 -0800, Aleksey wrote: > I write crossplatform program and have next problem under windows > (under linux is all OK) : > > I'm trying to get the UTC modification date of files on the flash > drive under windows. In the flash card system is FAT. Timestamps stored on a FAT filesystem use an unspecified timezone. When a Windows PC writes a timestamp to a FAT filesystem, it stores the current local time. For devices such as cameras, the timestamp will be whatever's in the device's clock if it has one (even if it does, there's no guarantee that it's accurate), or some arbitrary value otherwise. When reading a timestamp, it's just used as-is, i.e. it's a local time with no timezone specified. There is no way to convert this value to UTC or to a specific timezone. If you're using FAT, timestamps are pretty much meaningless unless the device never travels between timezones (in which case, they'll probably be roughly correct or wrong by roughly an hour either way due to DST). When it comes to timezones, anything developed prior to NT basically pretends that they don't exist. From showell30 at yahoo.com Sat Jan 23 02:43:03 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 23:43:03 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: <8e4d3fe2-c4bd-4a73-9c50-7a336dab25be@o28g2000yqh.googlegroups.com> On Jan 22, 11:10?pm, a... at pythoncraft.com (Aahz) wrote: > > >I know Python's number one concern will never be speed, but if Python > >makes an O(1) operation into an unnecessarily O(N) operation for no > >good reasons other than "it's too complicated, " or it "adds another > >pointer to the structure," or "it adds another conditional check to > >list_ass_slice for operations that aren't popping off the top," I > >think it's reasonable to challenge the design philosophy. > > "Rough consensus and running code." > > You have a good point, but nobody will ever give your idea serious > attention until there's a patch and benchmarks. Here is a benchmark of O(N*N) vs. O(N) for two C programs. One does memmove; the other simply advances the pointer. showell at showell-laptop:~$ time ./slow real 0m1.446s user 0m1.444s sys 0m0.004s showell at showell-laptop:~$ time ./fast real 0m0.003s user 0m0.004s sys 0m0.000s showell at showell-laptop:~$ diff slow.c fast.c 23,24c23 < lst.size -= 1; < memmove(lst.p, lst.p + 1, lst.size); --- > lst.p += 1; showell at showell-laptop:~$ cat slow.c #include #include #include struct ob_item { int whatever; }; struct list { struct ob_item *p; int size; }; struct list make_list(int n) { struct list lst; lst.p = malloc(n); lst.size = n; return lst; } void list_pop_left(struct list lst) { lst.size -= 1; memmove(lst.p, lst.p + 1, lst.size); } int main() { struct list lst; int i; int n = 40000; int t; lst = make_list(n); for (i = 0; i < n; ++i) { list_pop_left(lst); } } From alfps at start.no Sat Jan 23 02:43:28 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sat, 23 Jan 2010 08:43:28 +0100 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: * Steve Howell: > On Jan 22, 7:09 pm, Roy Smith wrote: >> In article >> <3ac173bd-4124-434d-b726-0b9baaeec... at 36g2000yqu.googlegroups.com>, >> Steve Howell wrote: >> >>> In my case I'm not really concerned about giving the memory back right >>> away, it's more about keeping my code simple. Once I'm done with an >>> element, I do just want to pop it off and keep all the simplicity of >>> the lists, but I am just concerned enough speed that for a 1000 >>> element list, I don't want to be doing 1000 memmoves for an average of >>> 500 pointers, which effectively moves about a million bytes around for >>> no reason. >> If you really want to pop all the elements from a long list, reverse the >> list and pop them off the end. Popping every element off the beginning of >> the list is O(n^2), as you pointed out. Reversing the list is O(n), and >> each pop after that is O(1), so the overall complexity is O(n). > > I really want to use list *normally* with all its perfectly good > semantics and reasonable implementation, except for its blind spot > with respect to popping the first element off the list. The whole > reason I use CPython vs. C in the first place is that CPython > programmers can generally program basic data structures better than I > can. But list.pop(0) is the exception. And, with the possible > exception of dicts, lists are the most fundamental data structures > that Python has. Having optimized list.pop() for first element, then you would have a blind spot with respect to insertion at the front... Which could then be optimized for the cases where there is some buffer space at the front, left over from a pop. I think it would just be harder to understand and harder to explain. And I think that due to that, as usual people would build their own elaborate "explanations", with erroneous conclusions, and would then use it in inefficient ways (like, popping from the middle or inserting at the front). I think the "harder to use correctly" is the strongest argument against the optimization, but there is also a non-obvious *memory overhead*... Having popped just one element at the front, in order for the implementation to not /accumulate/ unused memory, that is, in order to avoid an ongoing /memory leak/, extending the buffer to accomodate e.g. an append() can no longer be done as a (on relevant systems) constant time reallocation (letting the OS do its virtual memory paging magic). Instead, a new buffer would have to be allocated and all data copied over. One could still have amortized constant time for appends by using a doubling buffer (which is the strategy used in C++ 'std::vector'), but at the cost of wasting some memory, a percentage... The implementation as a pure array is easy to understand and use correctly, and doesn't waste memory. But it would IMHO have been better if it wasn't called "list", which brings in the wrong associations for someone used to other languages. The name does matter. E.g. I don't think this discussion about a pop optimization would have been there except for the name, which makes that optimization sound reasonable. Perhaps some standard alternative name could be devised. Like, "array" would have been nice, except that that's already grabbed by the array module. > I know Python's number one concern will never be speed, but if Python > makes an O(1) operation into an unnecessarily O(N) operation for no > good reasons other than "it's too complicated, " or it "adds another > pointer to the structure," or "it adds another conditional check to > list_ass_slice for operations that aren't popping off the top," I > think it's reasonable to challenge the design philosophy. See above. In addition to being more difficult /to use/ correctly, that is, being much easier to misunderstand, it incurs a memory overhead -- not the one directly from the pop optimization, but by the requirements of buffer extension. Essentially, as discussed above, it would then have to use a doubling buffer. Cheers & hth., - Alf From tjreedy at udel.edu Sat Jan 23 02:45:41 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 23 Jan 2010 02:45:41 -0500 Subject: Using dictionary key as a regular expression class In-Reply-To: <20100123025848.GC2958@turki.gavron.org> References: <20100122214755.GH28576@turki.gavron.org> <20100123025848.GC2958@turki.gavron.org> Message-ID: On 1/22/2010 9:58 PM, Chris Jones wrote: > On Fri, Jan 22, 2010 at 08:46:35PM EST, Terry Reedy wrote: > Do you mean I should just read the file one character at a time? Whoops, my misdirection (you can .read(1), but this is s l o w. I meant to suggest processing it a char at a time. 1. If not too big, for c in open(x, 'rb').read() # left .read() off # 'b' will get bytes, though ord(c) same for ascii chars for byte or unicode 2. If too big for that, for line in open(): for c in line: # or left off this part >> To only count ascii chars, as should be the case for C code, >> >> achars = [0]*63 >> for c in open('xxx', 'c'): >> try: >> achars[ord(c)-32] += 1 >> except IndexError: >> pass >> >> for i,n in enumerate(achars) >> print chr(i), n >> >> or sum subsets as desired. > > Thanks much for the snippet, let me play with it and see if I can come > up with a Unicode/utf-8 version.. since while I'm at it I might as well > write something a bit more general than C code. > > Since utf-8 is backward-compatible with 7bit ASCII, this shouldn't be > a problem. For any extended ascii, use larger array without decoding (until print, if need be). For unicode, add encoding to open and 'c in line' will return unicode chars. Then use *one* dict or defaultdict. I think something like from collections import defaultdict d = defaultdict(int) ... d[c] += 1 # if c is new, d[c] defaults to int() == 0 Terry Jan Reedy From showell30 at yahoo.com Sat Jan 23 02:58:29 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 23:58:29 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: <2ce05a04-ebc9-4f26-ba32-ff787925a564@c29g2000yqd.googlegroups.com> On Jan 22, 11:10?pm, a... at pythoncraft.com (Aahz) wrote: > In article <83082e19-9130-45a8-91f2-8601c1fda... at 22g2000yqr.googlegroups.com>, > Steve Howell ? wrote: > > > > > > >I really want to use list *normally* with all its perfectly good > >semantics and reasonable implementation, except for its blind spot > >with respect to popping the first element off the list. ?The whole > >reason I use CPython vs. C in the first place is that CPython > >programmers can generally program basic data structures better than I > >can. ?But list.pop(0) is the exception. ?And, with the possible > >exception of dicts, lists are the most fundamental data structures > >that Python has. > > >I know Python's number one concern will never be speed, but if Python > >makes an O(1) operation into an unnecessarily O(N) operation for no > >good reasons other than "it's too complicated, " or it "adds another > >pointer to the structure," or "it adds another conditional check to > >list_ass_slice for operations that aren't popping off the top," I > >think it's reasonable to challenge the design philosophy. > > "Rough consensus and running code." > > You have a good point, but nobody will ever give your idea serious > attention until there's a patch and benchmarks. Another benchmark is that deques are slower than lists for accessing elements. showell at showell-laptop:~$ python foo.py 0.0215361118317 <- list 0.0429010391235 <- deque import time from collections import deque n = 40000 lst = [] for i in range(n): lst.append(i) t = time.time() for i in range(n): lst[i] print time.time() - t lst = deque(lst) t = time.time() for i in range(n): lst[i] print time.time() - t So substituting deque for list suffers not just in convenience, but also in performance. From tjreedy at udel.edu Sat Jan 23 03:13:49 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 23 Jan 2010 03:13:49 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: On 1/23/2010 12:58 AM, Steve Howell wrote: > I really want to use list *normally* with all its perfectly good > semantics and reasonable implementation, except for its blind spot > with respect to popping the first element off the list. It was not designed for that. .pop() was added to lists about 10 years ago because I asked for it (with no parameter, pop off end only) and wrote what would now be a PEP -- and because Tim Peters later supported the idea. Adding the optional parameter was something of an afterthought (never publicly discussed as far as I know) for occasional use for 'short' lists where O(n) is tolerable. You have half persuaded me that that the parameter addition was a mistake. Perhaps is is too attractice a nuisance for some people ;=). > The whole > reason I use CPython vs. C in the first place is that CPython > programmers can generally program basic data structures better than I > can. They have given us three options other that .pop(0). 1. listiterator 2. queue.Queue 3. collections.deque\ Why are you so stubborn about not using a data structure intended for your use case instead of one that was not? There is also 4. a two-list design for queues: iterator through one (or pop() from a reversed version thereof) while appending to another; switch when the first is empty. I plan to write it up with tests some day this year. > I know Python's number one concern will never be speed, but if Python > makes an O(1) operation into an unnecessarily O(N) operation for no > good reasons other than "it's too complicated, " or it "adds another > pointer to the structure," or "it adds another conditional check to > list_ass_slice for operations that aren't popping off the top," I > think it's reasonable to challenge the design philosophy. Challenge yes, mock no. Part of writing good basic data structures is not adding needless complication from featuritis and not penalizing 99.99% of access to satify a .01% need better satisfied another way. Terry Jan Reedy From tjreedy at udel.edu Sat Jan 23 03:20:24 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 23 Jan 2010 03:20:24 -0500 Subject: medians for degree measurements In-Reply-To: <044268c0-c308-406e-8d5b-a38ef5b5e381@p24g2000yqm.googlegroups.com> References: <044268c0-c308-406e-8d5b-a38ef5b5e381@p24g2000yqm.googlegroups.com> Message-ID: On 1/23/2010 1:09 AM, Steve Howell wrote: > On Jan 22, 5:12 pm, MRAB wrote: > I like this implementation, and it would probably work 99.9999% of the > time for my particular use case. The only (very contrived) edge case > that I can think of is when you have 10 bearings to SSW, 10 bearings > to SSE, and the two outliers are unfortunately in the NE and NW > quadrants. It seems like the algorithm above would pick one of the > outliers. > > Maybe the refinement to the algorithm above would be to find the mean > first, by summing sines and cosines of the bearings, taking the > quotient, and applying the arctangent. Then use the resulting angle > as the equivalent of "due north" and adjust angles to be within (-180, > 180) respect to the mean, pretty much as you do in the code above, > with minor modifications. I was going to suggest this. Let us know if it seems to work. > I realize the problem as I stated it as sort of ill-defined. Yes, I think this one reason stat books typically ignore directional data. I think it is an unfortunate omission. Terry Jan Reedy From showell30 at yahoo.com Sat Jan 23 03:23:28 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sat, 23 Jan 2010 00:23:28 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: On Jan 23, 12:13?am, Terry Reedy wrote: > > Challenge yes, mock no. > > Part of writing good basic data structures is not adding needless > complication from featuritis and not penalizing 99.99% of access to > satify a .01% need better satisfied another way. > I would like to challenge your assertion that advancing ob_item instead of doing memmove during list_ass_slice would impact the performance of list accesses in any way. It would only slow down operations that add/insert items into the list by, and then only by a single conditional statement, and those add/insert operations are already O(N) to begin with. From alfps at start.no Sat Jan 23 03:32:26 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sat, 23 Jan 2010 09:32:26 +0100 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: * Steve Howell: > On Jan 23, 12:13 am, Terry Reedy wrote: >> Challenge yes, mock no. >> >> Part of writing good basic data structures is not adding needless >> complication from featuritis and not penalizing 99.99% of access to >> satify a .01% need better satisfied another way. >> > > I would like to challenge your assertion that advancing ob_item > instead of doing memmove during list_ass_slice would impact the > performance of list accesses in any way. It would only slow down > operations that add/insert items into the list by, and then only by a > single conditional statement, and those add/insert operations are > already O(N) to begin with. I'm sorry, no, the last part is incorrect. Appending to a 'list' can currently be constant time, if OS reallocation is constant time (as the string '+' optimization relies on). With the pop optimization it can no longer be constant time without risking an accumulation of unused memory, a memory leak, although it can be amortized constant time, at the cost of wasting some percentage of memory. Cheers & hth., - Alf From showell30 at yahoo.com Sat Jan 23 03:54:19 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sat, 23 Jan 2010 00:54:19 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: <2ffb4ab8-ae8f-4b69-b449-56d0232a404b@y23g2000yqm.googlegroups.com> On Jan 23, 12:32?am, "Alf P. Steinbach" wrote: > * Steve Howell: > > > On Jan 23, 12:13 am, Terry Reedy wrote: > >> Challenge yes, mock no. > > >> Part of writing good basic data structures is not adding needless > >> complication from featuritis and not penalizing 99.99% of access to > >> satify a .01% need better satisfied another way. > > > I would like to challenge your assertion that advancing ob_item > > instead of doing memmove during list_ass_slice would impact the > > performance of list accesses in any way. ?It would only slow down > > operations that add/insert items into the list by, and then only by a > > single conditional statement, and those add/insert operations are > > already O(N) to begin with. > > I'm sorry, no, the last part is incorrect. > > Appending to a 'list' can currently be constant time, if OS reallocation is > constant time (as the string '+' optimization relies on). > That's true, but it's also irrelevant, as the pop optimization would happen in a branch of code that never gets executed during list appending: if (d < 0) { /* Delete -d items */ /* ADD CODE HERE TO AVOID memmove when ilow == 0 (i.e. ihigh + d == 0) */ memmove(&item[ihigh+d], &item[ihigh], (Py_SIZE(a) - ihigh)*sizeof(PyObject *)); list_resize(a, Py_SIZE(a) + d); item = a->ob_item; } > With the pop optimization it can no longer be constant time without risking an > accumulation of unused memory, a memory leak, although it can be amortized > constant time, at the cost of wasting some percentage of memory. list_resize already overallocates memory to allow room for growth. Whenever you did an append to the list that would force a realloc, you could first check to see if there is unused stuff at the front and do the memmove then and reclaim the unfreed memory. So instead of doing a paying for memmove on every pop, you only pay for it when the list goes to size 0, 4, 8, 16, 25, 35, 46, 58, 72, 88, etc. > From groups_ads12 at yahoo.com Sat Jan 23 04:09:40 2010 From: groups_ads12 at yahoo.com (groups_ads12) Date: Sat, 23 Jan 2010 09:09:40 -0000 Subject: www.phptutorial.me Message-ID: www.phptutorial.me php php help free php free php scripts htaccess php learn php learning php open php php and php applications php array php arrays php book php books php cgi php checkbox php class php classes php code php comments php course php curl php database php date php developer php development php doc php download php example php extension php fgets php filter php find php fopen php foreach php form php forms php function php functions php game php gd php get php header php html php http php if php include php index php insert php install php installation php isset php job php jobs php language php live php login php magic php mail php max php next php now php outsourcing php pagination php parse php pear php perl php post php print php program php programmer php programming php query php redirect php replace php rest php return php script php scripts php search php select php session php sessions php set cookie php software php source code php sql php string php strtotime php support php syntax php table php template php test php training php tutorial php update php upload php url php user php variable php video php vs php website php windows php xml smarty php -------------- next part -------------- An HTML attachment was scrubbed... URL: From cjns1989 at gmail.com Sat Jan 23 04:22:39 2010 From: cjns1989 at gmail.com (Chris Jones) Date: Sat, 23 Jan 2010 04:22:39 -0500 Subject: Using dictionary key as a regular expression class In-Reply-To: References: <20100122214755.GH28576@turki.gavron.org> <20100123025848.GC2958@turki.gavron.org> Message-ID: <20100123092239.GC2963@turki.gavron.org> On Sat, Jan 23, 2010 at 02:45:41AM EST, Terry Reedy wrote: > On 1/22/2010 9:58 PM, Chris Jones wrote: >> Do you mean I should just read the file one character at a time? > > Whoops, my misdirection (you can .read(1), but this is s l o w. > I meant to suggest processing it a char at a time. Right.. that's how I understood it - i.e. asking python for the next character, and not worrying about how much is retrieved from the disk in one pass. > 1. If not too big, > > for c in open(x, 'rb').read() # left .read() off > # 'b' will get bytes, though ord(c) same for ascii chars for byte or > unicode > > 2. If too big for that, > > for line in open(): > for c in line: # or left off this part Well the script is not going to process anything larger that a few KiloBytes, but all the same that's something I want to understand better. Isn't there any way I can tell python to retrieve a fairly large chunk of the disk file, like 4-8K, maybe.. and increment a pointer behind the scenes while I iterate so that I have access to characters one at a time. I mean, that should be pretty fast, since disk access would be minimal, and no data would actually be copied.. I would have thought that 1. above would cause python to do something like that behind the scenes. [..] >> Thanks much for the snippet, let me play with it and see if I can >> come up with a Unicode/utf-8 version.. since while I'm at it I might >> as well write something a bit more general than C code. >> >> Since utf-8 is backward-compatible with 7bit ASCII, this shouldn't be >> a problem. > For any extended ascii, You mean 8-bit encodings, like latin1 right? > use larger array without decoding (until print, if need be). For > unicode, add encoding to open and 'c in line' will return unicode > chars. Then use *one* dict or defaultdict. I think something like > from collections import defaultdict > d = defaultdict(int) > ... > d[c] += 1 # if c is new, d[c] defaults to int() == 0 I don't know python, so I basically googled for the building blocks of my little script.. and I remember seeing defaultdict(int) mentioned some place or other, but somehow I didn't understand what it did. Cool feature. Even if it's a bit wasteful, with unicode/utf-8, it looks like working with the code points, either as dictionary keys or as index values into an array might make the logic simpler - i.e. for each char, obtain its code point 'cp' and add one to dict[cp] or array[cp] - and then loop and print all non-zero values when the end-of-file condition is reached. Food for thought in any case. CJ From tjreedy at udel.edu Sat Jan 23 04:24:40 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 23 Jan 2010 04:24:40 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: On 1/23/2010 3:23 AM, Steve Howell wrote: > On Jan 23, 12:13 am, Terry Reedy wrote: >> >> Challenge yes, mock no. >> >> Part of writing good basic data structures is not adding needless >> complication from featuritis and not penalizing 99.99% of access to >> satify a .01% need better satisfied another way. >> > > I would like to challenge your assertion that advancing ob_item > instead of doing memmove during list_ass_slice would impact the > performance of list accesses in any way. It would only slow down > operations that add/insert items into the list by, and then only by a > single conditional statement, and those add/insert operations are > already O(N) to begin with. If you try writing a full patch, as I believe someone did, or at least a prototype thereof, when the idea was discussed, you might have a better idea of what the tradeoffs are and why it was rejected. For instance, when you append to a full list, it is resized. I believe it is now doubled, but the policy has varied over the years. If you turn list from essentially a stack to a deque, you complicate the resizing policy and have to consider the addition of a shift policy. Do you split the over-allocated fore and aft or increase the overallocation from double to, say, triple? If the former, then for normal usage that never uses the fore part, the over-allocation has been effectively reduced from 2x to 1.5x (which is about what it once was, I believe). This means more frequent resizings and copyings, which means slower operation for most use cases. Adding extra usually wasted space is also a nuisance. Terry Jan Reedy From showell30 at yahoo.com Sat Jan 23 04:27:12 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sat, 23 Jan 2010 01:27:12 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: On Jan 23, 12:13?am, Terry Reedy wrote: > On 1/23/2010 12:58 AM, Steve Howell wrote: > > > I really want to use list *normally* with all its perfectly good > > semantics and reasonable implementation, except for its blind spot > > with respect to popping the first element off the list. > > It was not designed for that. .pop() was added to lists about 10 years > ago because I asked for it (with no parameter, pop off end only) and > wrote what would now be a PEP -- and because Tim Peters later supported > the idea. Adding the optional parameter was something of an afterthought > (never publicly discussed as far as I know) for occasional use for > 'short' lists where O(n) is tolerable. You have half persuaded me that > that the parameter addition was a mistake. Perhaps is is too attractice > a nuisance for some people ;=). > pop(0) is a useful idiom in parsers. You can see examples in ElementTree and lib2to3. Even without pop(0), people would still write code like this, found in pstats.py: arg = args[0] args = args[1:] It is sometimes overkill (and even inappropriate) to use a queue when really you just want a list. Iterators are great, but they also have slightly different semantics than the list itself. There is nothing wrong with a language specification that allows users to do insert, delete, and pop on a list. Once you freeze the language specification, then you can turn your attention to improving the implementation. From mrm at unknown.nospam Sat Jan 23 04:34:15 2010 From: mrm at unknown.nospam (Mr.M) Date: Sat, 23 Jan 2010 09:34:15 GMT Subject: PyArg_ParseTupleAndKeywords In-Reply-To: <91afd9e3-93a1-488d-8e83-299bfc222e24@22g2000yqr.googlegroups.com> References: <91afd9e3-93a1-488d-8e83-299bfc222e24@22g2000yqr.googlegroups.com> Message-ID: Carl Banks ha scritto: >> (some declarations omitted here) > > You probably shouldn't have, that could be where the error is.... I'd > include the whole function up to the call that raises the exception. Thank you so much Carl for your help, i'll provide more info so that you can try to fix my errors! Thank you again! First of all i'll try to explain what i've discovered over a night i spent awake trying to find where the bug is. My init behaves differently in this situations: [code] # this works dummy = mymodule.myclass() # this works dummy = mymodule.myclass("string for service") # this also works dummy = mymodule.myclass("string for service", "string for event_type") # and it works if i provide arguments in the right sequence, of course # this won't work dummy = mymodule.myclass("string for service", "string for event_type", free_text = "string for free_text") # but this works dummy = mymodule.myclass(service = "string for service") # the worst thing: this doesn't work but it did and i can't # understand what is changed dummy = mymodule.myclass(free_text = "text for free_text") ok, every time the code fails i get this exception: TypeError: expected string or Unicode object, tuple found so, i think for some reason PyArg_ParseTupleAndKeywords receives "args" as a touple instead of something else. This is everything i managed to discover. This is the code: [code] /* DEBUG INFO */ printf("event INIT\n"); if (args) { int i; printf("args = \"%s\"\n", PyString_AsString(args)); printf("type = %s\n", PyString_AsString(PyObject_Repr(PyObject_Type(args)))); printf("tuple size = %d\n", PyTuple_Size(args)); for (i = 0; i < PyTuple_Size(args); i++) { printf("%d) %s\n", i, PyString_AsString(PyTuple_GetItem(args, i))); } } else { printf("args = NULL\n"); } printf("dict:\n"); if (keywords) { printf(" = %s\n", PyString_AsString(PyObject_Repr(keywords))); printf("type = %s\n", PyString_AsString(PyObject_Repr(PyObject_Type(keywords)))); } else { printf("keywords = NULL\n"); } char* service = NULL; char* event_type = NULL; char* free_text = NULL; char* group_uid = NULL; char* remote_name = NULL; char* remote_abook_uid = NULL; time_t start_time = -1L; time_t end_time = -1L; time_t storage_time = -1L; int flags = 0; int bytes_sent = -1; int bytes_received = -1; PyObject* temp; static char* keywordlist[] = {"service", "event_type", "free_text", "group_uid", "remote_name", "remote_abook_uid", "start_time", "end_time", "storage_time", "flags", "bytes_sent", "bytes_received", NULL}; if (!PyArg_ParseTupleAndKeywords(args, keywords, "|ssssssiiiiii", keywordlist, &service, &event_type, &free_text, &group_uid, &remote_name, &remote_abook_uid, &start_time, &end_time, &storage_time, &flags, &bytes_sent, &bytes_received)) { return -1; } printf("PyArg_ParseTupleAndKeywords worked fine\n"); [/code] (sorry if my code is a little messy and my english rather bad!) > Are you sure that PyArg_ParseTupleAndKeywords is what's raising the > error? Yes, i think so. I have a lot of printf in my code! > Are you subclassing this type in Python, and passing one of the string > parameters a tuple by mistake? For instance, did you do something > like this: > > class myclass(_mycmodule._myctype): > def __init__(self,*args,**kwargs): > log_something_here() > super(myclass,self).__init__(args,**kwargs) > > Note the missing * on args. no, i'm not subclassing. I have to admit that i had some testcase and a few weeks ago they worked like a charm... i can't understand what i changed, of course. > > >> I found that if i write this everything works fine: >> >> [code] >> import mymodule >> a = mymodule.myclass(service = "blabla", event_type = "event", free_text >> = "free", group_uid = "group", remote_name = "remote name", >> remote_abook_uid = "abook", start_time = 1, end_time = 61, storage_time >> = 61, flags = 2) >> [/code] >> >> in other words, the code only works if *EVERY* argument is specified in >> exactly the same position it appears in keyword-null-terminated array. > > That it would have to be in a certain order is strange. Are you sure > it's just not merely that they all have to be present? > No, i tryied to remove the "|" and Python complains that 12 argument must be present. So they are, in a sense, optional, but (and this is strange), they must be present in the right order even i provide them with keyword arguments. > Tricky, but I think it'd help if you provided more information. My > gut feeling is that there exception is being raised somewhere other > than you think it is (i.e., not by PyArg_ParseTuple), so I'd recommend > adding some debugging statements to track down the exact point wher > the error occurs. Ok, thank you! The code i posted is exactly the same i run (and it fails). I've no problem sending you the whole module or adding any other debug info you think could be helpfull to find the source of the bug. Ciao, Luca. From alfps at start.no Sat Jan 23 04:38:55 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sat, 23 Jan 2010 10:38:55 +0100 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <2ffb4ab8-ae8f-4b69-b449-56d0232a404b@y23g2000yqm.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> <2ffb4ab8-ae8f-4b69-b449-56d0232a404b@y23g2000yqm.googlegroups.com> Message-ID: * Steve Howell: > On Jan 23, 12:32 am, "Alf P. Steinbach" wrote: >> * Steve Howell: >> >>> On Jan 23, 12:13 am, Terry Reedy wrote: >>>> Challenge yes, mock no. >>>> Part of writing good basic data structures is not adding needless >>>> complication from featuritis and not penalizing 99.99% of access to >>>> satify a .01% need better satisfied another way. >>> I would like to challenge your assertion that advancing ob_item >>> instead of doing memmove during list_ass_slice would impact the >>> performance of list accesses in any way. It would only slow down >>> operations that add/insert items into the list by, and then only by a >>> single conditional statement, and those add/insert operations are >>> already O(N) to begin with. >> I'm sorry, no, the last part is incorrect. >> >> Appending to a 'list' can currently be constant time, if OS reallocation is >> constant time (as the string '+' optimization relies on). >> > > That's true, but it's also irrelevant, as the pop optimization would > happen in a branch of code that never gets executed during list > appending: > > if (d < 0) { /* Delete -d items */ > /* ADD CODE HERE TO AVOID memmove > when ilow == 0 (i.e. ihigh + d == 0) > */ > memmove(&item[ihigh+d], &item[ihigh], > (Py_SIZE(a) - ihigh)*sizeof(PyObject *)); > list_resize(a, Py_SIZE(a) + d); > item = a->ob_item; > } > > >> With the pop optimization it can no longer be constant time without risking an >> accumulation of unused memory, a memory leak, although it can be amortized >> constant time, at the cost of wasting some percentage of memory. > > list_resize already overallocates memory to allow room for growth. > Whenever you did an append to the list that would force a realloc, you > could first check to see if there is unused stuff at the front and do > the memmove then and reclaim the unfreed memory. So instead of doing > a paying for memmove on every pop [at front], you only pay for it when > the list goes to size 0, 4, 8, 16, 25, 35, 46, 58, 72, 88, etc. Oh. If 'list' already uses a doubling buffer then the only overhead from the optimization would, AFAICS, be a single add in every indexing. Which might be acceptable (at least it sounds very reasonable in the context of Python). Re terminology: I write "doubling buffer" to mean increase of buffer size by a factor. It's often 2, but might be e.g. 1.5, whatever. The point of using a constant factor is to avoid quadratic time for loops doing appending, i.e. the constant factor size increase yields amortized constant time per append. The sizes that you quote above, on the other hand, look like rather arbitrary buffer size increases where the size to increase by is limited to a certain small range. With copying or moving of everything at each buffer size increase that would not yield amortized constant time. It yield linear time, and quadratic time for a loop doing appends. But, anyway, if 'list' already uses a doubling buffer then the only overhead from the pop optimization would, AFAICS, be a single add in every indexing. On the third & gripping hand, however, a proof-of-concept actual implementation (patch) would be needed to ensure that one doesn't overlook any showstopper or serious problem, and to provide timings. And the special case would have to be documented as a special case. Hm, it would be nice if the Python docs offered complexity (time) guarantees in general... Cheers, - Alf From bearophileHUGS at lycos.com Sat Jan 23 04:55:07 2010 From: bearophileHUGS at lycos.com (Bearophile) Date: Sat, 23 Jan 2010 01:55:07 -0800 (PST) Subject: Efficient Running Median References: <497af344-31b5-4d1a-9b1a-c3d82feb31d3@j5g2000yqm.googlegroups.com> Message-ID: <6ba43727-4b13-45cd-aa0d-3831d2281721@f12g2000yqn.googlegroups.com> Very nice. I have added a comment at the bottom, using a circular queue instead of a deque may be faster. Bye, bearophile From showell30 at yahoo.com Sat Jan 23 05:37:56 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sat, 23 Jan 2010 02:37:56 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: On Jan 23, 1:24?am, Terry Reedy wrote: > On 1/23/2010 3:23 AM, Steve Howell wrote: > > > On Jan 23, 12:13 am, Terry Reedy ?wrote: > > >> Challenge yes, mock no. > > >> Part of writing good basic data structures is not adding needless > >> complication from featuritis and not penalizing 99.99% of access to > >> satify a .01% need better satisfied another way. > > > I would like to challenge your assertion that advancing ob_item > > instead of doing memmove during list_ass_slice would impact the > > performance of list accesses in any way. ?It would only slow down > > operations that add/insert items into the list by, and then only by a > > single conditional statement, and those add/insert operations are > > already O(N) to begin with. > > If you try writing a full patch, as I believe someone did, or at least a > prototype thereof, when the idea was discussed, you might have a better > idea of what the tradeoffs are and why it was rejected. > > For instance, when you append to a full list, it is resized. I believe > it is now doubled, but the policy has varied over the years. If you turn > list from essentially a stack to a deque, you complicate the resizing > policy and have to consider the addition of a shift policy. Do you split > the over-allocated fore and aft or increase the overallocation from > double to, say, triple? If the former, then for normal usage that never > uses the fore part, the over-allocation has been effectively reduced > from 2x to 1.5x (which is about what it once was, I believe). This means > more frequent resizings and copyings, which means slower operation for > most use cases. Adding extra usually wasted space is also a nuisance. > It looks like most of the complication would be in list_resize. I'm gonna oversimplify a bit, but tell me if this is the gist of it. I would have ob_item continue to always refer to first element of the list, and then I'd have to introduce another variable to refer to the start of our allocated memory, ob_start_memory, whenever you do a realloc/free/malloc. I'd have a notion of fore_wastage, which would either be a variable I maintain or something that I just calculate as needed from the difference of ob_item and ob_start_memory. In deciding whether I want to give memory back to the memory manager, I just need to adjust my calculations to account for fore and aft wastage to see if it's time to do a shrink, and before shrinking, I do the memmove. On growth, I would just always do a memmove right away if there is fore_wastage, and then do the normal procedure for aft wastage. For the most common scenario of append, append, append, the only penalty is having to skip over fore_wastage logic by checking for fore_wastage == 0 or ob_item == ob_start_memory. For the scenario of several appends followed by several pops, I get the big win of only doing log 2 N memmoves instead of N as I shrink the list down to zero. If I start alternating between pops and appends, it's basically a wash...instead of doing the memmove on the pop, I do it on the next append. If I were to pop the top element and then prepend a new element, to be truly efficient, I'd want to use reserved right away, but for simplicity, I would probably not complicate list_ass_slice and just do the normal resize() dance, which would give me memmove in one direction followed immediately by a memmove in the other direction when I come back to list_ass_slice. (But it would all still be a wash, since I would have to do the same number of memmoves in the current implementation.) A lot of the essential complexity here seems to come from the fact that realloc() isn't a very robust abstraction. It seems to be expensive to tell it you want to shrink, and it also does not have an interface to tell it to give you a little growing room. On the other hand, the code within list_resize() actually provides a nice framework for amortizing memmoves exponentially. From michele.simionato at gmail.com Sat Jan 23 06:00:19 2010 From: michele.simionato at gmail.com (Michele Simionato) Date: Sat, 23 Jan 2010 03:00:19 -0800 (PST) Subject: counting lines of code References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> <87bpgodj95.fsf@benfinney.id.au> <1f166703-fdfc-4f87-b336-ef80e492635b@a32g2000yqm.googlegroups.com> <7e09df6a-cda1-480e-a971-8f8a70ac4118@b9g2000yqd.googlegroups.com> <980775cc-9101-4673-af03-fc17914d63f8@c4g2000yqa.googlegroups.com> <0d54eff2-59f0-4d95-aafb-478c9041c1b9@p8g2000yqb.googlegroups.com> Message-ID: <187b42f9-af0a-40aa-8c2d-65b4c60e274b@c4g2000yqa.googlegroups.com> On Jan 22, 7:51?pm, Phlip wrote: > On Jan 21, 9:00?pm, Michele Simionato > wrote: > > > Just for fun I have run cloc on our trunk: > > > SUM: ? ? ? ? ? ? ? ?8743 ? ?272238 ? ?215871 ? 1470139 x ? 1.84 = > > 2708354.95 > > Nice! > > My favorite version of a cloc system can distinguish test from > production code. That's why I always use executable cloc to measure > the ratio of test to production code (where 1.2:1 is almost > comfortable an 2:1 is sacred). Most of this code base is old, before we started using automatic tests, so tests are not a significant fraction of the code. And in any case I consider tests as code, since you have to maintain them, refactor them, etc. From michele.simionato at gmail.com Sat Jan 23 06:05:54 2010 From: michele.simionato at gmail.com (Michele Simionato) Date: Sat, 23 Jan 2010 03:05:54 -0800 (PST) Subject: counting lines of code References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> <87bpgodj95.fsf@benfinney.id.au> <1f166703-fdfc-4f87-b336-ef80e492635b@a32g2000yqm.googlegroups.com> <7e09df6a-cda1-480e-a971-8f8a70ac4118@b9g2000yqd.googlegroups.com> <%I36n.5962$4p5.2277@newsfe22.iad> Message-ID: <9f15ce5c-8a97-4551-8b83-e1076861b952@y23g2000yqm.googlegroups.com> On Jan 22, 10:30?pm, Steve Holden wrote: > >> Oh, sorry, did I have the wrong opinion? > > > You had a condescending attitude. > > Towards someone who is fairly obviously not a Python neophyte. > > Please don't think we are telling you you can't have any opinion you > like. Just don't expect to get away with it when you are wrong ;-) Come on, we are all friends here! The guy just said > In my experience with Python codebases that big... > > ...how many of those lines are duplicated, and might merge together > into a better design? > > The LOC would go down, too. and it was even partially right. It is certainly right for the part I was working on. If it was good code from the beginning we would not have started this refactoring project, right? Peace, Michele From peloko45 at gmail.com Sat Jan 23 06:20:48 2010 From: peloko45 at gmail.com (Joan Miller) Date: Sat, 23 Jan 2010 03:20:48 -0800 (PST) Subject: ISC License Message-ID: <00eb248d-c9c9-430f-bc83-41ac865c5111@e11g2000yqe.googlegroups.com> There is a license approved by the OSI, the ISC License [1], which should be included in the PyPi classifiers [2]. [1] https://www.isc.org/software/license http://www.opensource.org/licenses/isc-license.txt [2] http://pypi.python.org/pypi?%3Aaction=list_classifiers From malkarouri at gmail.com Sat Jan 23 06:23:17 2010 From: malkarouri at gmail.com (Muhammad Alkarouri) Date: Sat, 23 Jan 2010 03:23:17 -0800 (PST) Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> Message-ID: <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> Thanks everyone, but not on my machine (Python 2.6.1, OS X 10.6) it's not: In [1]: from itertools import count, islice In [2]: from collections import deque In [3]: i1=count() In [4]: def consume1(iterator, n): ...: deque(islice(iterator, n), maxlen=0) ...: ...: In [5]: i2=count() In [6]: def consume2(iterator, n): ...: for _ in islice(iterator, n): pass ...: ...: In [7]: timeit consume1(i1, 10) 1000000 loops, best of 3: 1.63 us per loop In [8]: timeit consume2(i2, 10) 1000000 loops, best of 3: 846 ns per loop Can somebody please test whether it is only my machine or is this reproducible? (Thanks Jan for making me actually carry the test) From Martin.Drautzburg at web.de Sat Jan 23 06:24:44 2010 From: Martin.Drautzburg at web.de (Martin Drautzburg) Date: Sat, 23 Jan 2010 12:24:44 +0100 Subject: A.x vs. A["x"] References: <1718504.hrdDYYapAE@beaureve.gmx.net> Message-ID: <84293188.TWlGDM3umM@beaureve.gmx.net> Terry Reedy wrote: > On 1/22/2010 2:29 PM, Martin Drautzburg wrote: >> This has probably been asekd a million times, but if someone could >> give a short answer anyways I's be most grateful. >> >> What is it that allows one to write A.x? If I have a variable A, >> I know you can do this with classes, but not with plain objects, but >> why is that so? > > You exact meaning here is not clear, but I suspect it is somewhat > incorrect, at least for built-in classes. You're right. I used to think you could do this to classes: G = Strum G.x=1 But not to objects (instances): g = Strum() g.y = 2 But in fact both work. Thanks for clarifying From steve at holdenweb.com Sat Jan 23 06:35:04 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 23 Jan 2010 06:35:04 -0500 Subject: Mastering Python 3 I/O - Special Preview - Feb 5, 2010 (Chicago) In-Reply-To: <7f014ea61001221947o7e17f0ebv9784c427346f6f08@mail.gmail.com> References: <4EB3255C-2B1E-44E5-BE72-D61E44E580B4@dabeaz.com> <4B5A61CE.2030608@holdenweb.com> <7f014ea61001221947o7e17f0ebv9784c427346f6f08@mail.gmail.com> Message-ID: Chris Colbert wrote: > oops :) > Yes, but only a little oops. I think it's probably fairly well-known that David Beazley and I both supply training, and you'd hardly expect profit to be off the list of objectives for US trainers. But my failure to hit "reply to sender only" certainly did show that it's been a long week. Sorry for the noise! regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Sat Jan 23 06:37:38 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 23 Jan 2010 06:37:38 -0500 Subject: Using dictionary key as a regular expression class In-Reply-To: <20100122233844.GJ28576@turki.gavron.org> References: <20100122233844.GJ28576@turki.gavron.org> Message-ID: Chris Jones wrote: > On Fri, Jan 22, 2010 at 05:07:13PM EST, Arnaud Delobelle wrote: > > [..] > >> import codecs >> from collections import defaultdict >> >> tcounters = defaultdict(int) >> f = codecs.open('/home/gavron/git/screen/src/screen.c', 'r', "utf-8") >> >> for c in f.read(): >> tcounters[c] += 1 >> >> for c, n in tcounters.iteritems(): >> print "%r\t%i" % (c, n) > > Ah, yes.. much better - I grew suspicious of my 'effort' when I realized > I could have written a shorter version in C. :-) > Congratulations. That perception shows a sound appreciation of Python's design. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at REMOVE-THIS-cybersource.com.au Sat Jan 23 06:52:43 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 23 Jan 2010 11:52:43 GMT Subject: medians for degree measurements References: <044268c0-c308-406e-8d5b-a38ef5b5e381@p24g2000yqm.googlegroups.com> Message-ID: <036acf91$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 22:09:54 -0800, Steve Howell wrote: > On Jan 22, 5:12?pm, MRAB wrote: >> Steve Howell wrote: >> > I just saw the thread for medians, and it reminded me of a problem >> > that I need to solve. ?We are writing some Python software for >> > sailing, and we need to detect when we've departed from the median >> > heading on the leg. ?Calculating arithmetic medians is >> > straightforward, but compass bearings add a twist. [...] > I like this implementation, and it would probably work 99.9999% of the > time for my particular use case. The only (very contrived) edge case > that I can think of is when you have 10 bearings to SSW, 10 bearings to > SSE, and the two outliers are unfortunately in the NE and NW quadrants. > It seems like the algorithm above would pick one of the outliers. The trouble is that median of angular measurements is not a meaningful concept. The median depends on the values being ordered, but angles can't be sensibly ordered. Which is larger, 1 degree north or 359 degrees? Is the midpoint between them 0 degree or 180 degree? The median of the number line 0 <= x <= 360 is 180, but what's the median of the circle 0 deg <= theta <= 360 deg? With no end points, it's meaningless to talk about the middle. For this reason, I expect that taking the median of bearing measurements isn't well defined. You can probably get away with it so long as the bearings are "close", where "close" itself is ill-defined. In any case, this isn't a programming problem, it's a mathematics problem. I think you're better off taking it to a maths or statistics forum, and see what they say. -- Steven From steve at REMOVE-THIS-cybersource.com.au Sat Jan 23 07:12:29 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 23 Jan 2010 12:12:29 GMT Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> Message-ID: <036ad433$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 21:42:43 -0800, Steve Howell wrote: > This innocent program here literally moves about a million bytes of > memory around for no good reason: > > lst = [] > for i in range(2000): > lst.append(i) > while lst: > print lst.pop(0) > > Why? Because list.pop(0) is implemented in O(N) instead of O(1). > > Why wouldn't you get a competent C programmer simply make list_ass_slice > smart enough to make list.pop(0) O(1)? Because there are always trade-offs, and the competent C programmers who coded the implementation for lists choose different tradeoffs to the ones you would prefer. Seems to me that the simple solution to your problem is for you to implement your own data structure that makes whatever trade-offs you like. If it is good enough and popular enough, it might even replace the existing list implementation. >> That is, you give me the impression that you prefer this: >> >> while alist: >> ? ? x = alist.pop(0) >> ? ? process(x) >> >> over this: >> >> for x in alist: >> ? ? process(x) >> # allow alist to be garbage collected when it goes out of scope >> >> > No, to be more precise, I prefer this implementation of a recursive > parser (using lists) to one that would have to use deque's because of > the lameness of Python's list implementation: > > https://bitbucket.org/showell/shpaml_website/src/tip/shpaml.py That's a lot of code. Am I supposed to study the whole module, or can you give me a hint as to what you're referring to? The lack of docstrings and comments doesn't fill me with enthusiasm for reading it. Nevertheless, on the basis of a quick scan, I suppose that you're probably talking about the nested function called recurse: def recurse(prefix_lines): while prefix_lines: prefix, line = prefix_lines[0] if line == '': prefix_lines.pop(0) append('') else: block_size = get_block(prefix_lines) if block_size == 1: prefix_lines.pop(0) if line == pass_syntax: pass elif line.startswith(flush_left_syntax): append(line[len(flush_left_syntax):]) elif line.startswith(flush_left_empty_line): append('') else: append(prefix + leaf_method(line)) else: block = prefix_lines[:block_size] prefix_lines = prefix_lines[block_size:] branch_method(output, block, recurse) return Since you're not even looking at the results of the pop, why don't you just call del prefix_lines[0]? It probably won't perform any better, but it is more idiomatic. An alternative would be to do exactly what you want lists to do: track the start of the list. Untested: def recurse(prefix_lines): start = 0 end = len(prefix_lines) while start < end: prefix, line = prefix_lines[start] if line == '': start += 1 append('') else: block_size = get_block(prefix_lines) if block_size == 1: start += 1 if line == pass_syntax: pass elif line.startswith(flush_left_syntax): append(line[len(flush_left_syntax):]) elif line.startswith(flush_left_empty_line): append('') else: append(prefix + leaf_method(line)) else: block = prefix_lines[:block_size] start = block_size branch_method(output, block, recurse) return No more O(N) deletions. Problem solved. -- Steven From arnodel at googlemail.com Sat Jan 23 07:24:32 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sat, 23 Jan 2010 12:24:32 +0000 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: Dave Angel writes: > Arnaud Delobelle wrote: >> Steve Howell writes: >> >> >>> On Jan 22, 12:14 pm, Chris Rebert wrote: >>> >> I made the comment you quoted. In CPython, it is O(n) to delete/insert >> an element at the start of the list - I know it because I looked at the >> implementation a while ago. This is why collections.deque exists I >> guess. I don't know how they are implemented but insertion/deletion at >> either end are O(1) and so is random access. So they are the data >> structure that you want. >> >> > Not according to the 2.6 docs. > > Indexed access is O(1) at both ends but slows to O(n) in the > middle. For fast random access, use lists instead. Yes you are correct. This will teach me (again!) to check my facts. > > That sounds to me like a doubly-linked list implementation. I've just looked and it is a doubly-linked list of 'blocks' of size BLOCKLEN, which is 62 on the source I have (I guess it's 62 because then the whole block structure is 64 exactly, 62 + 1 for each link). So a small list will have near constant random access, in a way. -- Arnaud From __peter__ at web.de Sat Jan 23 07:45:43 2010 From: __peter__ at web.de (Peter Otten) Date: Sat, 23 Jan 2010 13:45:43 +0100 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> Message-ID: Muhammad Alkarouri wrote: > Thanks everyone, but not on my machine (Python 2.6.1, OS X 10.6) it's > not: > > > In [1]: from itertools import count, islice > > In [2]: from collections import deque > > In [3]: i1=count() > > In [4]: def consume1(iterator, n): > ...: deque(islice(iterator, n), maxlen=0) > ...: > ...: > > In [5]: i2=count() > > In [6]: def consume2(iterator, n): > ...: for _ in islice(iterator, n): pass > ...: > ...: > > In [7]: timeit consume1(i1, 10) > 1000000 loops, best of 3: 1.63 us per loop > > In [8]: timeit consume2(i2, 10) > 1000000 loops, best of 3: 846 ns per loop > > > Can somebody please test whether it is only my machine or is this > reproducible? I can reproduce it. The deque-based approach has a bigger constant overhead but better per-item performance. Its asymptotical behaviour is therefore better. $ python consume_timeit.py consume_deque 10: 1.77500414848 100: 3.73333001137 1000: 24.7235469818 consume_forloop 10: 1.22008490562 100: 5.86271500587 1000: 52.2449371815 consume_islice 10: 0.897439956665 100: 1.51542806625 1000: 7.70061397552 $ cat consume_timeit.py from collections import deque from itertools import islice, repeat def consume_deque(n, items): deque(islice(items, n), maxlen=0) def consume_forloop(n, items): for _ in islice(items, n): pass def consume_islice(n, items): next(islice(items, n-1, None), None) def check(fs): for consume in fs: items = iter(range(10)) consume(3, items) rest = list(items) assert rest == range(3, 10), consume.__name__ if __name__ == "__main__": fs = consume_deque, consume_forloop, consume_islice check(fs) items = repeat(None) from timeit import Timer for consume in fs: print consume.__name__ for n in (10, 100, 1000): print "%6d:" % n, print Timer("consume(%s, items)" % n, "from __main__ import consume, items").timeit() print $ With next(islice(...), None) I seem to have found a variant that beats both competitors. Peter From duncan.booth at invalid.invalid Sat Jan 23 07:58:44 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 23 Jan 2010 12:58:44 GMT Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> Message-ID: Peter Otten <__peter__ at web.de> wrote: > With next(islice(...), None) I seem to have found a variant that beats > both competitors. > It has different behaviour for n==0 but I'm sure that's easily fixed. From __peter__ at web.de Sat Jan 23 08:13:37 2010 From: __peter__ at web.de (Peter Otten) Date: Sat, 23 Jan 2010 14:13:37 +0100 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> Message-ID: Duncan Booth wrote: > Peter Otten <__peter__ at web.de> wrote: > >> With next(islice(...), None) I seem to have found a variant that beats >> both competitors. >> > It has different behaviour for n==0 but I'm sure that's easily fixed. "Different behaviour" being a euphemism for broken ;) def consume_islice(n, items): if n == 0: return next(islice(items, n-1, None), None) Peter From malkarouri at gmail.com Sat Jan 23 08:14:34 2010 From: malkarouri at gmail.com (Muhammad Alkarouri) Date: Sat, 23 Jan 2010 05:14:34 -0800 (PST) Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> Message-ID: <06153e39-7aa8-416b-93bb-32001e0cb81b@e37g2000yqn.googlegroups.com> On 23 Jan, 12:45, Peter Otten <__pete... at web.de> wrote: > Muhammad Alkarouri wrote: > > Thanks everyone, but not on my machine (Python 2.6.1, OS X 10.6) it's > > not: > > > In [1]: from itertools import count, islice > > > In [2]: from collections import deque > > > In [3]: i1=count() > > > In [4]: def consume1(iterator, n): > > ? ?...: ? ? deque(islice(iterator, n), maxlen=0) > > ? ?...: > > ? ?...: > > > In [5]: i2=count() > > > In [6]: def consume2(iterator, n): > > ? ?...: ? ? for _ in islice(iterator, n): pass > > ? ?...: > > ? ?...: > > > In [7]: timeit consume1(i1, 10) > > 1000000 loops, best of 3: 1.63 us per loop > > > In [8]: timeit consume2(i2, 10) > > 1000000 loops, best of 3: 846 ns per loop > > > Can somebody please test whether it is only my machine or is this > > reproducible? > > I can reproduce it. The deque-based approach has a bigger constant overhead > but better per-item performance. Its asymptotical behaviour is therefore > better. > > $ python consume_timeit.py > consume_deque > ? ? 10: 1.77500414848 > ? ?100: 3.73333001137 > ? 1000: 24.7235469818 > > consume_forloop > ? ? 10: 1.22008490562 > ? ?100: 5.86271500587 > ? 1000: 52.2449371815 > > consume_islice > ? ? 10: 0.897439956665 > ? ?100: 1.51542806625 > ? 1000: 7.70061397552 > > $ cat consume_timeit.py > from collections import deque > from itertools import islice, repeat > > def consume_deque(n, items): > ? ? deque(islice(items, n), maxlen=0) > > def consume_forloop(n, items): > ? ? for _ in islice(items, n): > ? ? ? ? pass > > def consume_islice(n, items): > ? ? next(islice(items, n-1, None), None) > > def check(fs): > ? ? for consume in fs: > ? ? ? ? items = iter(range(10)) > ? ? ? ? consume(3, items) > ? ? ? ? rest = list(items) > ? ? ? ? assert rest == range(3, 10), consume.__name__ > > if __name__ == "__main__": > ? ? fs = consume_deque, consume_forloop, consume_islice > ? ? check(fs) > > ? ? items = repeat(None) > > ? ? from timeit import Timer > ? ? for consume in fs: > ? ? ? ? print consume.__name__ > ? ? ? ? for n in (10, 100, 1000): > ? ? ? ? ? ? print "%6d:" % n, > ? ? ? ? ? ? print Timer("consume(%s, items)" % n, > ? ? ? ? ? ? ? ? ? ? ? ? "from __main__ import consume, items").timeit() > ? ? ? ? print > $ > > With next(islice(...), None) I seem to have found a variant that beats both ? > competitors. > > Peter Thanks Peter, I got more or less the same result on my machine (Python 2.6.1, x86_64, OS X 10.6): ~/tmp> python consume_timeit.py consume_deque 10: 1.3138859272 100: 3.54495286942 1000: 24.9603481293 consume_forloop 10: 0.658113002777 100: 2.85697007179 1000: 24.6610429287 consume_islice 10: 0.637741088867 100: 1.09042882919 1000: 5.44473600388 The next function performs much better. It is also much more direct for the purposes of consume and much more understandable (at least for me) as it doesn't require a specialized data structure which is subsequently not used as such. I am thus inclined to report it as a python documentation enhancement (bug) request. Any comments? Cheers, Muhammad From __peter__ at web.de Sat Jan 23 08:15:25 2010 From: __peter__ at web.de (Peter Otten) Date: Sat, 23 Jan 2010 14:15:25 +0100 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> Message-ID: Duncan Booth wrote: > Peter Otten <__peter__ at web.de> wrote: > >> With next(islice(...), None) I seem to have found a variant that beats >> both competitors. >> > It has different behaviour for n==0 but I'm sure that's easily fixed. "Different behaviour" being a euphemism for broken ;) def consume_islice(n, items): if n == 0: return next(islice(items, n-1, None), None) Peter From __peter__ at web.de Sat Jan 23 08:32:37 2010 From: __peter__ at web.de (Peter Otten) Date: Sat, 23 Jan 2010 14:32:37 +0100 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <06153e39-7aa8-416b-93bb-32001e0cb81b@e37g2000yqn.googlegroups.com> Message-ID: Muhammad Alkarouri wrote: > The next function performs much better. It is also much more direct > for the purposes of consume and much more understandable (at least for > me) as it doesn't require a specialized data structure which is > subsequently not used as such. > I am thus inclined to report it as a python documentation enhancement > (bug) request. Any comments? I would support that as a the deque(..., maxlen=0) trick is a bit too clever for my taste. Peter PS: Remember to include the bug fix for n=0 if you proceed. From __peter__ at web.de Sat Jan 23 08:46:16 2010 From: __peter__ at web.de (Peter Otten) Date: Sat, 23 Jan 2010 14:46:16 +0100 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> Message-ID: Peter Otten wrote: > Duncan Booth wrote: > >> Peter Otten <__peter__ at web.de> wrote: >> >>> With next(islice(...), None) I seem to have found a variant that beats >>> both competitors. >>> >> It has different behaviour for n==0 but I'm sure that's easily fixed. > > "Different behaviour" being a euphemism for broken ;) > > def consume_islice(n, items): > if n == 0: > return > next(islice(items, n-1, None), None) Even better: def consume_islice(n, items): next(islice(items, n, n), None) Peter From lists at cheimes.de Sat Jan 23 08:46:37 2010 From: lists at cheimes.de (Christian Heimes) Date: Sat, 23 Jan 2010 14:46:37 +0100 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <2ce05a04-ebc9-4f26-ba32-ff787925a564@c29g2000yqd.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> <2ce05a04-ebc9-4f26-ba32-ff787925a564@c29g2000yqd.googlegroups.com> Message-ID: <4B5AFDBD.3090200@cheimes.de> Steve Howell wrote: > Another benchmark is that deques are slower than lists for accessing > elements. deques are optimized for accessing, inserting and removing data from both ends. For anything else it's slower than the list type. The fact was explained in this very thread yesterday. Christian From malkarouri at gmail.com Sat Jan 23 08:55:12 2010 From: malkarouri at gmail.com (Muhammad Alkarouri) Date: Sat, 23 Jan 2010 05:55:12 -0800 (PST) Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <06153e39-7aa8-416b-93bb-32001e0cb81b@e37g2000yqn.googlegroups.com> Message-ID: <84254f72-2bf0-4157-990a-9880115dac0d@q4g2000yqm.googlegroups.com> On 23 Jan, 13:32, Peter Otten <__pete... at web.de> wrote: > Muhammad Alkarouri wrote: > > The next function performs much better. It is also much more direct > > for the purposes of consume and much more understandable (at least for > > me) as it doesn't require a specialized data structure which is > > subsequently not used as such. > > I am thus inclined to report it as a python documentation enhancement > > (bug) request. Any comments? > > I would support that as a the deque(..., maxlen=0) trick is a bit too clever > for my taste. > > Peter > > PS: Remember to include the bug fix for n=0 if you proceed. Done. http://bugs.python.org/issue7764 Thanks for all the effort. Muhammad From malkarouri at gmail.com Sat Jan 23 09:06:32 2010 From: malkarouri at gmail.com (Muhammad Alkarouri) Date: Sat, 23 Jan 2010 06:06:32 -0800 (PST) Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> Message-ID: <98b43693-3ba2-4d0d-b009-739d841a1c85@36g2000yqu.googlegroups.com> On 23 Jan, 13:46, Peter Otten <__pete... at web.de> wrote: > Peter Otten wrote: > > Duncan Booth wrote: > > >> Peter Otten <__pete... at web.de> wrote: > > >>> With next(islice(...), None) I seem to have found a variant that beats > >>> both ?competitors. > > >> It has different behaviour for n==0 but I'm sure that's easily fixed. > > > "Different behaviour" being a euphemism for broken ;) > > > def consume_islice(n, items): > > ? ? if n == 0: > > ? ? ? ? return > > ? ? next(islice(items, n-1, None), None) > > Even better: > > def consume_islice(n, items): > ? ? next(islice(items, n, n), None) > > Peter I submitted the bug report before considering this alternative, which is better. I may add this later in the day, but I have to wait a little as it seems you are going to optimize/improve the function almost out of existence:) If you are happy with this one, and can add the comment on the issue (http://bugs.python.org/issue7764) yourself, please do so. What I can say is, I am definitely very happy that I asked the question. You live and learn:) Cheers, Muhammad From __peter__ at web.de Sat Jan 23 09:19:56 2010 From: __peter__ at web.de (Peter Otten) Date: Sat, 23 Jan 2010 15:19:56 +0100 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <98b43693-3ba2-4d0d-b009-739d841a1c85@36g2000yqu.googlegroups.com> Message-ID: Muhammad Alkarouri wrote: > On 23 Jan, 13:46, Peter Otten <__pete... at web.de> wrote: >> def consume_islice(n, items): >> next(islice(items, n, n), None) > I submitted the bug report before considering this alternative, which > is better. I may add this later in the day, but I have to wait a > little as it seems you are going to optimize/improve the function > almost out of existence:) One problem: the above function doesn't consume the entire iterator like the original example does for n=None. Passing sys.maxint instead is not pretty. Peter From roy at panix.com Sat Jan 23 09:40:56 2010 From: roy at panix.com (Roy Smith) Date: Sat, 23 Jan 2010 09:40:56 -0500 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> Message-ID: In article , Steve Howell wrote: > This innocent program here literally moves about a million bytes of > memory around for no good reason: > > lst = [] > for i in range(2000): > lst.append(i) > while lst: > print lst.pop(0) > > Why? Because list.pop(0) is implemented in O(N) instead of O(1). I think you're being a little pedantic here. Yes, it is true that pop(0) is O(n), and that if you put an O(n) operation in a loop, you get O(n^2) run time. The problem is that while it is well-known that putting something that's O(n) in a loop gets you O(n^2), it's not well known that pop(0) for a Python list is O(n). This is where you and I apparently start to differ in what we think about this. You are arguing that this is a bug in the implementation of list. While I suppose there's some validity to that argument, I disagree. What I would argue (and have done so several times over the years, with little success) is that this is a bug in the documentation! I'm looking at http://tinyurl.com/cdbwog. It shows all the operations of a list. What it does not show is their cost. For pop(), it has a note: "The pop() method is only supported by the list and array types. The optional argument i defaults to -1, so that by default the last item is removed and returned". There's nothing there that gives any hint that pop(0) is any more expensive than pop(-1). That is "secret knowledge", which you only get by following the newsgroup discussions or looking at the implementation. You shouldn't have to do either. There's lots of possible ways list could be implemented. Without knowing the details, I'm left to guess about important stuff like the cost of operations. Every one of these operations should list the cost. Even if it's something as vague as, "While not guaranteed by the language spec, in the current implemenation of CPython ...". From rdv at roalddevries.nl Sat Jan 23 09:44:07 2010 From: rdv at roalddevries.nl (Roald de Vries) Date: Sat, 23 Jan 2010 15:44:07 +0100 Subject: enumerated while loop Message-ID: <227C7474-8CD1-4BC8-BDC3-ADBFAE01D052@roalddevries.nl> Dear all, I sometimes want to use an infinite while loop with access to the loop index, like this: def naturals(): i = 0 while True: yield i y += 1 for i in naturals(): print(i) I assume a function like 'naturals' already exists, or a similar construction for the same purpose. But what is it called? Kind regards, Roald From deets at nospam.web.de Sat Jan 23 09:49:23 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Sat, 23 Jan 2010 15:49:23 +0100 Subject: enumerated while loop In-Reply-To: References: Message-ID: <7s0gjjFehnU1@mid.uni-berlin.de> Am 23.01.10 15:44, schrieb Roald de Vries: > Dear all, > > I sometimes want to use an infinite while loop with access to the loop > index, like this: > > def naturals(): > i = 0 > while True: > yield i > y += 1 > > for i in naturals(): > print(i) > > I assume a function like 'naturals' already exists, or a similar > construction for the same purpose. But what is it called? for i, item in enumerate(iterable): .... Diez From invalid at invalid.invalid Sat Jan 23 09:50:29 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Sat, 23 Jan 2010 14:50:29 +0000 (UTC) Subject: enumerated while loop References: Message-ID: On 2010-01-23, Roald de Vries wrote: > Dear all, > > I sometimes want to use an infinite while loop with access to the loop > index, like this: > > def naturals(): > i = 0 > while True: > yield i > y += 1 > > for i in naturals(): > print(i) > > I assume a function like 'naturals' already exists, or a similar > construction for the same purpose. But what is it called? xrange(), except that you need to provde an upper limit. -- Grant From arnodel at googlemail.com Sat Jan 23 09:57:00 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sat, 23 Jan 2010 14:57:00 +0000 Subject: medians for degree measurements References: Message-ID: Steve Howell writes: > I just saw the thread for medians, and it reminded me of a problem > that I need to solve. We are writing some Python software for > sailing, and we need to detect when we've departed from the median > heading on the leg. Calculating arithmetic medians is > straightforward, but compass bearings add a twist. > > The numerical median of 1, 2, 3, 4, 5, 6, 359 is 4. But for > navigational purposes you would actually order the numbers 359, 1, 2, > 3, 4, 5, 6, so the desired median heading of the boat is actually 3. > > Of course, you could express 359 better as -1 degrees to north, then > the sequence would be -1, 1, 2, 3, 4, 5, and 6. And you'd be good. > > But that trick does not generalize if you go south instead, as you > have similar issues with -179, 174, 175, 176, 177, 178, and 179. > > Has anybody solved this in Python, either for compass bearings or a > different domain? I can think of kind of a brute force solution where > you keep rotating the sequence until the endpoints are closest > together mod 360, but I wonder if there is something more elegant. Here's a simple (too simple?) way to do it: 1. sort the bearings in ascending order 2. find the largest gap between two consecutive bearings 3. cut the circle at this point and now find the median the normal way In Python: >>> def median_bearing(bearings): ... bearings = sorted(bearings) ... n = len(bearings) ... i = max(xrange(n), key=lambda i: (bearings[(i+1)%n] - bearings[i])%360) ... return bearings[(i + (n+1)//2)%n] ... >>> median_bearing([1,2,3,4,5,6,359]) 3 >>> median_bearing([-179, 174, 175, 176, 177, 178, 179]) 177 I guess there may be cases where the results that it gives are still not very good, as in general there isn't a good notion of median for cyclic data. E.g. what would be the median of [0, 180] - it could equally be 090 or 270. Or the median of [0, 120, 240] has the same problem. But I suppose your data couldn't be like this as then it would be ill-advised to try to apply a notion of median to it. But it will work well I believe with quite localized data set with a few, even wildly inaccurate, outliers. E.g. >>> median_bearing([123, 124, 125, 125, 126, 10, 240]) 125 HTH -- Arnaud From roy at panix.com Sat Jan 23 09:57:04 2010 From: roy at panix.com (Roy Smith) Date: Sat, 23 Jan 2010 09:57:04 -0500 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: In article , "Alf P. Steinbach" wrote: > But it would IMHO have been better if it wasn't called "list", which brings > in the wrong associations for someone used to other languages. +1. When I first started using Python (back in the 1.4 days), I assumed a list was a singly-linked list. Which, of course, leads to the assumption that pop(0) is O(1), and lots of other wrong thinking(*). Oddly enough, I was going to write in the above paragraph, "like a C++ STL list", until I happened to glance at the STL docs and refreshed my memory that an STL list is doubly-linked. Which just goes to show that making assumptions based on names is a bad idea. So, we're right back to my statement earlier in this thread that the docs are deficient in that they describe behavior with no hint about cost. Given that, it should be no surprise that users make incorrect assumptions about cost. (*) I suspect somebody is going to point out that list.pop was added in some version later than 1.4, but that's a detail. From dickinsm at gmail.com Sat Jan 23 09:58:09 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 23 Jan 2010 06:58:09 -0800 (PST) Subject: enumerated while loop References: Message-ID: On Jan 23, 2:44?pm, Roald de Vries wrote: > I assume a function like 'naturals' already exists, or a similar ? > construction for the same purpose. But what is it called? itertools.count() -- Mark From zuo at chopin.edu.pl Sat Jan 23 10:04:52 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sat, 23 Jan 2010 16:04:52 +0100 Subject: enumerated while loop In-Reply-To: <7s0gjjFehnU1@mid.uni-berlin.de> References: <7s0gjjFehnU1@mid.uni-berlin.de> Message-ID: 23-01-2010 o 15:49:23 Diez B. Roggisch wrote: > Am 23.01.10 15:44, schrieb Roald de Vries: >> Dear all, >> >> I sometimes want to use an infinite while loop with access to the loop >> index, like this: >> >> def naturals(): >> i = 0 >> while True: >> yield i >> y += 1 >> >> for i in naturals(): >> print(i) >> >> I assume a function like 'naturals' already exists, or a similar >> construction for the same purpose. But what is it called? itertools.count() -- see http://docs.python.org/library/itertools.html#itertools.count > for i, item in enumerate(iterable): > .... That's completely beside the point. Regards, *j From rdv at roalddevries.nl Sat Jan 23 10:06:16 2010 From: rdv at roalddevries.nl (Roald de Vries) Date: Sat, 23 Jan 2010 16:06:16 +0100 Subject: enumerated while loop In-Reply-To: References: Message-ID: <38B9A1E5-3775-463F-86DB-3535C7D041F9@roalddevries.nl> On Jan 23, 2010, at 3:50 PM, Grant Edwards wrote: > On 2010-01-23, Roald de Vries wrote: >> Dear all, >> >> I sometimes want to use an infinite while loop with access to the >> loop >> index, like this: >> >> def naturals(): >> i = 0 >> while True: >> yield i >> y += 1 >> >> for i in naturals(): >> print(i) >> >> I assume a function like 'naturals' already exists, or a similar >> construction for the same purpose. But what is it called? > > xrange(), except that you need to provde an upper limit. I'm sorry, Grant, but you exactly miss my point ;-). The essence of this is that it's endless. From rdv at roalddevries.nl Sat Jan 23 10:07:51 2010 From: rdv at roalddevries.nl (Roald de Vries) Date: Sat, 23 Jan 2010 16:07:51 +0100 Subject: enumerated while loop In-Reply-To: <7s0gjjFehnU1@mid.uni-berlin.de> References: <7s0gjjFehnU1@mid.uni-berlin.de> Message-ID: <750D5C07-1230-4950-81A2-45A7430CDF7F@roalddevries.nl> On Jan 23, 2010, at 3:49 PM, Diez B. Roggisch wrote: > Am 23.01.10 15:44, schrieb Roald de Vries: >> Dear all, >> >> I sometimes want to use an infinite while loop with access to the >> loop >> index, like this: >> >> def naturals(): >> i = 0 >> while True: >> yield i >> y += 1 >> >> for i in naturals(): >> print(i) >> >> I assume a function like 'naturals' already exists, or a similar >> construction for the same purpose. But what is it called? > > for i, item in enumerate(iterable): > .... This only moves the problem to the variable 'iterable'. And moreover is more complex than needed; it give '(i, item)', whereas I only need the index 'i'. From ceciliaseidel at gmx.de Sat Jan 23 10:13:20 2010 From: ceciliaseidel at gmx.de (ceciliaseidel at gmx.de) Date: Sat, 23 Jan 2010 16:13:20 +0100 Subject: iterating lists Message-ID: <4B5B1210.2060300@gmx.de> As you were talking about list.pop()... Is anyone able to reproduce the following and explain why this happens by chance? (Using 3.1.1) l1 = ["ready", "steady", "go"] l2 = ["one", "two", "tree"] l3 = ["lift off"] for w in l1: print(l1.pop()) #prints only "go steady" - why not "ready"?? for w in range(len(l2)): print(l2.pop()) #prints "three two one" as expected for w in l3: print(l3.pop()) #prints "lift off" - inconsistent to first case... At least for 2.2.3 I found the first way to iterate the list as default, I guess it is deprecated now but still what happens seems weird to me... From arnodel at googlemail.com Sat Jan 23 10:19:06 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sat, 23 Jan 2010 15:19:06 +0000 Subject: iterating lists References: Message-ID: ceciliaseidel at gmx.de writes: > As you were talking about list.pop()... > > Is anyone able to reproduce the following and explain why this happens > by chance? (Using 3.1.1) > > l1 = ["ready", "steady", "go"] > l2 = ["one", "two", "tree"] > l3 = ["lift off"] > > for w in l1: > print(l1.pop()) #prints only "go steady" - why not "ready"?? > I suggest you simulate the loop above using pen and paper, writing the value of w and the value of l1 at each iteration. The behaviour you are observing should then be clearly explained. And you should also realise that it's a really bad idea to mutate a list that you are iterating on! HTH -- Arnaud From rdv at roalddevries.nl Sat Jan 23 10:30:30 2010 From: rdv at roalddevries.nl (Roald de Vries) Date: Sat, 23 Jan 2010 16:30:30 +0100 Subject: enumerated while loop In-Reply-To: References: Message-ID: On Jan 23, 2010, at 3:58 PM, Mark Dickinson wrote: > On Jan 23, 2:44 pm, Roald de Vries wrote: >> I assume a function like 'naturals' already exists, or a similar >> construction for the same purpose. But what is it called? > > itertools.count() On Jan 23, 2010, at 4:04 PM, Jan Kaliszewski wrote: > itertools.count() -- see http://docs.python.org/library/itertools.html#itertools.count > >> for i, item in enumerate(iterable): >> .... > > That's completely beside the point. Thanks guys From alfps at start.no Sat Jan 23 10:45:30 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sat, 23 Jan 2010 16:45:30 +0100 Subject: iterating lists In-Reply-To: References: Message-ID: * ceciliaseidel at gmx.de: > As you were talking about list.pop()... > > Is anyone able to reproduce the following and explain why this happens > by chance? (Using 3.1.1) > > l1 = ["ready", "steady", "go"] > l2 = ["one", "two", "tree"] > l3 = ["lift off"] > > for w in l1: > print(l1.pop()) #prints only "go steady" - why not "ready"?? > > for w in range(len(l2)): > print(l2.pop()) #prints "three two one" as expected > for w in l3: > print(l3.pop()) #prints "lift off" - inconsistent to first case... > > > At least for 2.2.3 I found the first way to iterate the list as default, > I guess it is deprecated now but still what happens seems weird to me... Arnaud Delobelle has already answered your question, but you might alternatively try this angle: l1 = ["ready", "steady", "go"] l2 = ["one", "two", "tree"] l3 = ["lift off"] for w in l1: print( w, l1 ) print(l1.pop()) #prints only "go steady" - why not "ready"?? for w in range(len(l2)): print(l2.pop()) #prints "three two one" as expected for w in l3: print( w, l3 ) print(l3.pop()) #prints "lift off" - inconsistent to first case... If the list has at least one item you always get into the first iteration of the loop. I.e. there's no inconsistency, unless you count the lack of an exception as an inconsistency. I don't know whether the behavior is clearly defined or not; there is a possibility that it might be well-defined. Cheers & hth., - Alf From thinke365 at gmail.com Sat Jan 23 10:49:46 2010 From: thinke365 at gmail.com (thinke365) Date: Sat, 23 Jan 2010 07:49:46 -0800 (PST) Subject: how can i know if a python object have a attribute such as 'attr1'? Message-ID: <27286937.post@talk.nabble.com> for example, i may define a python class: class A: def sayHello(): print 'hello' a = A() a.attr1 = 'hello' a.attr2 = 'bb' b = A() a.attr2 = 'aa' how can i know whether an object have an attribute named attr1? -- View this message in context: http://old.nabble.com/how-can-i-know-if-a-python-object-have-a-attribute-such-as-%27attr1%27--tp27286937p27286937.html Sent from the Python - python-list mailing list archive at Nabble.com. From steve at REMOVE-THIS-cybersource.com.au Sat Jan 23 10:54:26 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 23 Jan 2010 15:54:26 GMT Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: <036b0836$0$1357$c3e8da3@news.astraweb.com> On Sat, 23 Jan 2010 09:57:04 -0500, Roy Smith wrote: > In article , > "Alf P. Steinbach" wrote: > >> But it would IMHO have been better if it wasn't called "list", which >> brings in the wrong associations for someone used to other languages. > > +1. > > When I first started using Python (back in the 1.4 days), I assumed a > list was a singly-linked list. Why would you do that? I can think of at least eight different implementations of the abstract list data structure: constant-size array variable-size array variable-size array with amortised O(1) appends singly-linked list singly-linked list with CDR coding doubly-linked list skip list indexable skip list One can reasonably disregard constant-sized arrays as a possibility, given that Python lists aren't fixed size (pity the poor Pascal and Fortran coders who are stuck with static arrays!), but the rest are all reasonable possibilities. Why assume one specific implementation in the absence of documentation promising certain performance characteristics? > Oddly enough, I was going to write in the above paragraph, "like a C++ > STL list", until I happened to glance at the STL docs and refreshed my > memory that an STL list is doubly-linked. Which just goes to show that > making assumptions based on names is a bad idea. Exactly :) > So, we're right back to my statement earlier in this thread that the > docs are deficient in that they describe behavior with no hint about > cost. Given that, it should be no surprise that users make incorrect > assumptions about cost. There are quite a few problems with having the documentation specify cost: (1) Who is going to do it? Any volunteers? (2) Big-oh notation can be misleading, especially for naive users, or those whose intuition for what's fast has been shaped by other languages. Big-oh doesn't tell you whether something is fast or slow, only how it scales -- and sometimes not even then. (3) Having documented a particular performance, that discourages implementation changes. Any would-be patch or new implementation not only has to consider that the functional behaviour doesn't change, but that the performance doesn't either. In practice the Python developers are unlikely to make an implementation change which leads to radically worse performance, particularly for critical types like list and dict. But in other cases, they might choose to change big-oh behaviour, and not wish to be tied down by documentation of the cost of operations. (4) How much detail is necessary? What about degenerate cases? E.g. dict lookup in CPython is typically O(1) amortised, but if all the keys hash to the same value, it falls to O(N). (5) Should the language guarantee such degenerate behaviour? Who decides which costs are guaranteed and which are not? (6) Such performance guarantees should be implementation specific, not language specific. CPython is only one implementation of the language out of many. -- Steven From arnodel at googlemail.com Sat Jan 23 10:56:29 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sat, 23 Jan 2010 15:56:29 +0000 Subject: how can i know if a python object have a attribute such as 'attr1'? References: Message-ID: thinke365 writes: > for example, i may define a python class: > class A: > def sayHello(): > print 'hello' > > a = A() > a.attr1 = 'hello' > a.attr2 = 'bb' > > b = A() > a.attr2 = 'aa' > > how can i know whether an object have an attribute named attr1? hasattr(a, 'attr1') -- Arnaud From steve at holdenweb.com Sat Jan 23 11:05:05 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 23 Jan 2010 11:05:05 -0500 Subject: A.x vs. A["x"] In-Reply-To: <84293188.TWlGDM3umM@beaureve.gmx.net> References: <1718504.hrdDYYapAE@beaureve.gmx.net> <84293188.TWlGDM3umM@beaureve.gmx.net> Message-ID: Martin Drautzburg wrote: > Terry Reedy wrote: > >> On 1/22/2010 2:29 PM, Martin Drautzburg wrote: >>> This has probably been asekd a million times, but if someone could >>> give a short answer anyways I's be most grateful. >>> >>> What is it that allows one to write A.x? If I have a variable A, > > >>> I know you can do this with classes, but not with plain objects, but >>> why is that so? >> You exact meaning here is not clear, but I suspect it is somewhat >> incorrect, at least for built-in classes. > > You're right. I used to think you could do this to classes: > > G = Strum > G.x=1 > > But not to objects (instances): > > g = Strum() > g.y = 2 > > But in fact both work. Thanks for clarifying Both work, in fact, because a class is (like almost any other Python object), an instance of some class. In Python the classes that are defined in the interpreter are usually called types, but since "new-style objects" were introduced into Python in 2.2 you can define your own types (and subclass the system types). The name for the particular types whose instances are classes is "metaclass". But once you realize that the "regular classes" you already know about are just instances of their metaclass (the metaclass , in many cases) it might seem less surprising. There is a necessary but strange and surprising relationship between types and objects that can be summarized as isinstance(object, type) == isinstance(type,object) == True Because the built-in types are implemented as a part of the interpreter they are somewhat more restricted. Consequently not only can you not add attributes to the classes, you can't add them to the instances either: >>> int.x = 2 Traceback (most recent call last): File "", line 1, in TypeError: can't set attributes of built-in/extension type 'int' >>> i = int() >>> i.y = "hello" Traceback (most recent call last): File "", line 1, in AttributeError: 'int' object has no attribute 'y' >>> regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From ceciliaseidel at gmx.de Sat Jan 23 11:29:06 2010 From: ceciliaseidel at gmx.de (ceciliaseidel at gmx.de) Date: Sat, 23 Jan 2010 17:29:06 +0100 Subject: iterating lists In-Reply-To: References: Message-ID: <4B5B23D2.5070002@gmx.de> Arnaud Delobelle schrieb: > ceciliaseidel at gmx.de writes: > >> As you were talking about list.pop()... >> >> Is anyone able to reproduce the following and explain why this happens >> by chance? (Using 3.1.1) >> >> l1 = ["ready", "steady", "go"] >> l2 = ["one", "two", "tree"] >> l3 = ["lift off"] >> >> for w in l1: Ouch... thanks Arnaud... The stable way would've been for w in l1[:]: #use copy of l1 for iteration print(l1.pop()) #decomposite list (just in case any other newbie is reading this...) >> print(l1.pop()) #prints only "go steady" - why not "ready"?? >> > > I suggest you simulate the loop above using pen and paper, writing the > value of w and the value of l1 at each iteration. The behaviour you are > observing should then be clearly explained. And you should also realise > that it's a really bad idea to mutate a list that you are iterating on! > > HTH > From duncan.booth at invalid.invalid Sat Jan 23 11:29:23 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 23 Jan 2010 16:29:23 GMT Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> Message-ID: Roy Smith wrote: > I'm looking at http://tinyurl.com/cdbwog. It shows all the operations > of a list. What it does not show is their cost. For pop(), it has a > note: > > "The pop() method is only supported by the list and array types. The > optional argument i defaults to -1, so that by default the last item > is removed and returned". The page you should probably be looking at is http://wiki.python.org/moin/TimeComplexity From susan_kijiji at yahoo.ca Sat Jan 23 11:33:42 2010 From: susan_kijiji at yahoo.ca (im_smialing) Date: Sat, 23 Jan 2010 08:33:42 -0800 (PST) Subject: py2exe deal with python command line inside a program References: Message-ID: On Jan 22, 2:35?pm, susan_kij... at yahoo.ca wrote: > Hi, > > I need to create a python subprogress, like this: > myProcess = subprocess.Popen([sys.executable, 'C:\myscript.py'], > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?env=env, stdin=subprocess.PIPE, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?stdout=subprocess.PIPE) > > sys.executable was printed out as ''C:\\Python25\\python.exe'', how > can I make this work in executable package through py2exe? > > I have to fix the following problems: > -Source code shouldn't exposed in an executable program > -Since python ?environment is not required when running an executable > program, how to deal with the situation that "C:\\Python25\ > \python.exe" is required as part of command? > > Thanks in advance! Anyone can help? The problem took me two days and still no solutions From alfps at start.no Sat Jan 23 11:37:22 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sat, 23 Jan 2010 17:37:22 +0100 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <036b0836$0$1357$c3e8da3@news.astraweb.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> <036b0836$0$1357$c3e8da3@news.astraweb.com> Message-ID: * Steven D'Aprano: > On Sat, 23 Jan 2010 09:57:04 -0500, Roy Smith wrote: > >> In article , >> "Alf P. Steinbach" wrote: >> >>> But it would IMHO have been better if it wasn't called "list", which >>> brings in the wrong associations for someone used to other languages. >> +1. >> >> When I first started using Python (back in the 1.4 days), I assumed a >> list was a singly-linked list. > > Why would you do that? I can think of at least eight different > implementations of the abstract list data structure: > > constant-size array > variable-size array > variable-size array with amortised O(1) appends > singly-linked list > singly-linked list with CDR coding > doubly-linked list > skip list > indexable skip list > > One can reasonably disregard constant-sized arrays as a possibility, > given that Python lists aren't fixed size (pity the poor Pascal and > Fortran coders who are stuck with static arrays!), but the rest are all > reasonable possibilities. A linked list implementation would yield O(n) indexing. A great many loops in e.g. Python libraries code now having linear time would then get quadratic time, O(n^2). Those libraries would then be effectively unusable without extensive rewriting: one version for ordinary Python and one for 'list-as-list' Pythons... Thus, the linked list implementations are IMO *not* reasonable. And the reason is precisely the implied complexity guarantees, especially on indexing -- which could reasonably be O(log n), but not worse than that. > Why assume one specific implementation in the > absence of documentation promising certain performance characteristics? > > >> Oddly enough, I was going to write in the above paragraph, "like a C++ >> STL list", until I happened to glance at the STL docs and refreshed my >> memory that an STL list is doubly-linked. Which just goes to show that >> making assumptions based on names is a bad idea. > > Exactly :) > > > >> So, we're right back to my statement earlier in this thread that the >> docs are deficient in that they describe behavior with no hint about >> cost. Given that, it should be no surprise that users make incorrect >> assumptions about cost. > > There are quite a few problems with having the documentation specify cost: > > (1) Who is going to do it? Any volunteers? This problem must have been addressed at each time the documentation for some version of Python was written or updated. > (2) Big-oh notation can be misleading, especially for naive users, or > those whose intuition for what's fast has been shaped by other languages. > Big-oh doesn't tell you whether something is fast or slow, only how it > scales -- and sometimes not even then. It's how things scale that are of interest. :-) Big-oh tells you an upper asymptotic limit. That's sufficient for e.g. the C++ standard -- which, by the way, constitutes a concrete example of the practicality of specifying complexity. > (3) Having documented a particular performance, that discourages > implementation changes. Any would-be patch or new implementation not only > has to consider that the functional behaviour doesn't change, but that > the performance doesn't either. > > In practice the Python developers are unlikely to make an implementation > change which leads to radically worse performance, particularly for > critical types like list and dict. But in other cases, they might choose > to change big-oh behaviour, and not wish to be tied down by documentation > of the cost of operations. Say that there was an O(log n) documented worst complexity for 'list' indexing. Above you have described it as "reasonable" to break that, having O(n) complexity... But in light of my comments on that, and especially thinking a bit about maintainance of two or more! versions of various libraries, don't you agree that it would be Just Bad(TM)? > (4) How much detail is necessary? What about degenerate cases? E.g. dict > lookup in CPython is typically O(1) amortised, but if all the keys hash > to the same value, it falls to O(N). From N1745, the Technical Report 1 on C++ library extensions (will be part of the C++0x standard), table 21 specifying general requirements of unordered associative containers: expression: b.find(k) return type: iterator; assertion: Returns an iterator pointing to an element with key equivalent to k, or b.end() if no such element exists. complexity: Average case O(1), worst case O(b.size()). > (5) Should the language guarantee such degenerate behaviour? Who decides > which costs are guaranteed and which are not? I think the C++ standard (the latest draft of C++0x is freely available as PDF from the commitee pages) provides good guidance in this regard. :-) > (6) Such performance guarantees should be implementation specific, not > language specific. CPython is only one implementation of the language out > of many. Disagree Very Strongly. An implementation may offer stricter guarantees. But what matters regarding e.g. avoiding having to maintain two or three or umpteen versions of a library, is the set of language level complexity guarantees. Cheers, - Alf From showell30 at yahoo.com Sat Jan 23 11:41:15 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sat, 23 Jan 2010 08:41:15 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> <2ce05a04-ebc9-4f26-ba32-ff787925a564@c29g2000yqd.googlegroups.com> Message-ID: <43a0f0a9-da8b-47d5-824c-31fd5201a290@l30g2000yqb.googlegroups.com> On Jan 23, 5:46?am, Christian Heimes wrote: > Steve Howell wrote: > > Another benchmark is that deques are slower than lists for accessing > > elements. > > deques are optimized for accessing, inserting and removing data from > both ends. For anything else it's slower than the list type. The fact > was explained in this very thread yesterday. > And the benchmark confirmed it. The slowness is fairly negligible, though. From showell30 at yahoo.com Sat Jan 23 11:46:18 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sat, 23 Jan 2010 08:46:18 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> Message-ID: <38693efe-5de0-4b07-afab-bc1a0df8c490@b2g2000yqi.googlegroups.com> On Jan 23, 6:40?am, Roy Smith wrote: > In article > , > ?Steve Howell wrote: > > > This innocent program here literally moves about a million bytes of > > memory around for no good reason: > > > ? ? lst = [] > > ? ? for i in range(2000): > > ? ? ? ? lst.append(i) > > ? ? while lst: > > ? ? ? ? print lst.pop(0) > > > Why? ?Because list.pop(0) is implemented in O(N) instead of O(1). > > I think you're being a little pedantic here. ?Yes, it is true that pop(0) > is O(n), and that if you put an O(n) operation in a loop, you get O(n^2) > run time. > > The problem is that while it is well-known that putting something that's > O(n) in a loop gets you O(n^2), it's not well known that pop(0) for a > Python list is O(n). ?This is where you and I apparently start to differ in > what we think about this. > The source for Python is open. It pretty clearly shows that you move N bytes when you pop from the top of the list. Less clear is how linear the performance of memmove is. My benchmarks on the C program show that, at least on my computer, the results do not seem to contradict the "roughly linear" assumption. > You are arguing that this is a bug in the implementation of list. ?While I > suppose there's some validity to that argument, I disagree. ?What I would > argue (and have done so several times over the years, with little success) > is that this is a bug in the documentation! > > I'm looking athttp://tinyurl.com/cdbwog. ?It shows all the operations of a > list. ?What it does not show is their cost. ?For pop(), it has a note: > > "The pop() method is only supported by the list and array types. The > optional argument i defaults to -1, so that by default the last item is > removed and returned". > > There's nothing there that gives any hint that pop(0) is any more expensive > than pop(-1). ?That is "secret knowledge", which you only get by following > the newsgroup discussions or looking at the implementation. ?You shouldn't > have to do either. ?There's lots of possible ways list could be > implemented. ?Without knowing the details, I'm left to guess about > important stuff like the cost of operations. > > Every one of these operations should list the cost. ?Even if it's something > as vague as, "While not guaranteed by the language spec, in the current > implemenation of CPython ...". I agree with that. From showell30 at yahoo.com Sat Jan 23 12:03:46 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sat, 23 Jan 2010 09:03:46 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> Message-ID: <002ffd1d-15dc-4ea3-9340-6c64c09db32a@j14g2000yqm.googlegroups.com> On Jan 23, 4:12?am, Steven D'Aprano wrote: > > > An alternative would be to do exactly what you want lists to do: track > the start of the list. Untested: > > ? ? def recurse(prefix_lines): > ? ? ? ? start = 0 > ? ? ? ? end = len(prefix_lines) > ? ? ? ? while start < end: > ? ? ? ? ? ? prefix, line = prefix_lines[start] > ? ? ? ? ? ? if line == '': > ? ? ? ? ? ? ? ? start += 1 > ? ? ? ? ? ? ? ? append('') > ? ? ? ? ? ? else: > ? ? ? ? ? ? ? ? block_size = get_block(prefix_lines) > ? ? ? ? ? ? ? ? if block_size == 1: > ? ? ? ? ? ? ? ? ? ? start += 1 > ? ? ? ? ? ? ? ? ? ? if line == pass_syntax: > ? ? ? ? ? ? ? ? ? ? ? ? pass > ? ? ? ? ? ? ? ? ? ? elif line.startswith(flush_left_syntax): > ? ? ? ? ? ? ? ? ? ? ? ? append(line[len(flush_left_syntax):]) > ? ? ? ? ? ? ? ? ? ? elif line.startswith(flush_left_empty_line): > ? ? ? ? ? ? ? ? ? ? ? ? append('') > ? ? ? ? ? ? ? ? ? ? else: > ? ? ? ? ? ? ? ? ? ? ? ? append(prefix + leaf_method(line)) > ? ? ? ? ? ? ? ? else: > ? ? ? ? ? ? ? ? ? ? block = prefix_lines[:block_size] > ? ? ? ? ? ? ? ? ? ? start = block_size > ? ? ? ? ? ? ? ? ? ? branch_method(output, block, recurse) > ? ? ? ? return > > No more O(N) deletions. Problem solved. > A minor modification of your solution does work, but it also slightly + complicates the implementation. Keeping track of the start variable requires bookkeeping not just in recurse(), but also in methods that it calls. This is a pretty small program, so it's acceptable to pass around an offset variable to anybody else who might want to be consuming the list. ############ Generic indentation stuff follows -def get_indented_block(prefix_lines): - prefix, line = prefix_lines[0] +def get_indented_block(prefix_lines, start): + prefix, line = prefix_lines[start] len_prefix = len(prefix) i = 1 - while i < len(prefix_lines): - new_prefix, line = prefix_lines[i] + while i + start < len(prefix_lines): + new_prefix, line = prefix_lines[start+i] if line and len(new_prefix) <= len_prefix: break i += 1 - while i-1 > 0 and prefix_lines[i-1][1] == '': + while i-1 > 0 and prefix_lines[start+i-1][1] == '': i -= 1 return i @@ -190,15 +190,16 @@ ): append = output.append def recurse(prefix_lines): - while prefix_lines: - prefix, line = prefix_lines[0] + start = 0 + while start < len(prefix_lines): + prefix, line = prefix_lines[start] if line == '': - prefix_lines.pop(0) + start += 1 append('') else: - block_size = get_block(prefix_lines) + block_size = get_block(prefix_lines, start) if block_size == 1: - prefix_lines.pop(0) + start += 1 if line == pass_syntax: pass elif line.startswith(flush_left_syntax): @@ -208,8 +209,8 @@ else: append(prefix + leaf_method(line)) else: - block = prefix_lines[:block_size] - prefix_lines = prefix_lines[block_size:] + block = prefix_lines[start:start+block_size] + start += block_size branch_method(output, block, recurse) return prefix_lines = map(indentation_method, lines) From showell30 at yahoo.com Sat Jan 23 12:17:31 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sat, 23 Jan 2010 09:17:31 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> Message-ID: <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> On Jan 23, 4:12?am, Steven D'Aprano wrote: > On Fri, 22 Jan 2010 21:42:43 -0800, Steve Howell wrote: > > This innocent program here literally moves about a million bytes of > > memory around for no good reason: > > > ? ? lst = [] > > ? ? for i in range(2000): > > ? ? ? ? lst.append(i) > > ? ? while lst: > > ? ? ? ? print lst.pop(0) > > > Why? ?Because list.pop(0) is implemented in O(N) instead of O(1). > > > Why wouldn't you get a competent C programmer simply make list_ass_slice > > smart enough to make list.pop(0) O(1)? > > Because there are always trade-offs, and the competent C programmers who > coded the implementation for lists choose different tradeoffs to the ones > you would prefer. > > Seems to me that the simple solution to your problem is for you to > implement your own data structure that makes whatever trade-offs you > like. If it is good enough and popular enough, it might even replace the > existing list implementation. > The data structure that would make the tradeoffs I want would be implemented within CPython itself. I give a sketch of the changes elsewhere in this thread. Terry Reedy said: ''' If you try writing a full patch, as I believe someone did, or at least a prototype thereof, when the idea was discussed, you might have a better idea of what the tradeoffs are and why it was rejected. ''' I have to run, but tomorrow I will try to dig through python-dev archives and find the patch. If anybody has hints on where to look for it (anybody remember the author, for example?), it would be much appreciated. If the patch looks simple, I will try to pitch the idea that its time has come. Now that the specification of the language itself is frozen, I think there might be more room for improving implementations. Also, I might be able to make the argument that tradeoffs of memory vs. CPU vs. code complexity have different forces in the 2010s. Thanks for your reply. From detlev at die-offenbachs.de Sat Jan 23 12:28:32 2010 From: detlev at die-offenbachs.de (Detlev Offenbach) Date: Sat, 23 Jan 2010 18:28:32 +0100 Subject: pyflakes and Python3 Message-ID: Hi, does anybody know, if pyflakes is being ported to Python3? Or is there any other tool like it around that works with Python3? Regards, Detlev -- Detlev Offenbach detlev at die-offenbachs.de From rschroev_nospam_ml at fastmail.fm Sat Jan 23 12:29:33 2010 From: rschroev_nospam_ml at fastmail.fm (Roel Schroeven) Date: Sat, 23 Jan 2010 18:29:33 +0100 Subject: iterating lists In-Reply-To: References: Message-ID: <1mG6n.74060$4N3.53053@newsfe06.ams2> Op 2010-01-23 17:29, ceciliaseidel at gmx.de schreef: > > > Arnaud Delobelle schrieb: >> ceciliaseidel at gmx.de writes: >> >>> As you were talking about list.pop()... >>> >>> Is anyone able to reproduce the following and explain why this happens >>> by chance? (Using 3.1.1) >>> >>> l1 = ["ready", "steady", "go"] >>> l2 = ["one", "two", "tree"] >>> l3 = ["lift off"] >>> >>> for w in l1: > > Ouch... thanks Arnaud... The stable way would've been > > for w in l1[:]: #use copy of l1 for iteration > print(l1.pop()) #decomposite list I would prefer: while l1: print(l1.pop()) -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven From stefan_ml at behnel.de Sat Jan 23 12:34:04 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 23 Jan 2010 18:34:04 +0100 Subject: iterating lists In-Reply-To: References: Message-ID: <4b5b330c$0$6716$9b4e6d93@newsspool2.arcor-online.net> ceciliaseidel at gmx.de, 23.01.2010 17:29: > Arnaud Delobelle wrote: >> ceciliaseidel at gmx.de writes: >> >>> As you were talking about list.pop()... >>> >>> Is anyone able to reproduce the following and explain why this happens >>> by chance? (Using 3.1.1) >>> >>> l1 = ["ready", "steady", "go"] >>> l2 = ["one", "two", "tree"] >>> l3 = ["lift off"] >>> >>> for w in l1: > > Ouch... thanks Arnaud... The stable way would've been > > for w in l1[:]: #use copy of l1 for iteration > print(l1.pop()) #decomposite list If the intention is to exhaust the list during the iteration, I'd go for this: while l1: print(l1.pop()) Stefan From thinke365 at gmail.com Sat Jan 23 12:37:58 2010 From: thinke365 at gmail.com (thinke365) Date: Sat, 23 Jan 2010 09:37:58 -0800 (PST) Subject: how to generate random numbers that satisfy certain distribution Message-ID: <27288180.post@talk.nabble.com> such as uniform distribution, Normal distribution or poisson distribution. is there any package that can be used to generate such random numbers. -- View this message in context: http://old.nabble.com/how-to-generate-random-numbers-that-satisfy-certain-distribution-tp27288180p27288180.html Sent from the Python - python-list mailing list archive at Nabble.com. From showell30 at yahoo.com Sat Jan 23 12:38:29 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sat, 23 Jan 2010 09:38:29 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> <036b0836$0$1357$c3e8da3@news.astraweb.com> Message-ID: <44ccd731-4a98-4eec-a270-08f66b192d93@l19g2000yqb.googlegroups.com> On Jan 23, 7:54?am, Steven D'Aprano wrote: > On Sat, 23 Jan 2010 09:57:04 -0500, Roy Smith wrote: > > In article , > > ?"Alf P. Steinbach" wrote: > > >> But it would IMHO have been better if it wasn't called "list", which > >> brings in the wrong associations for someone used to other languages. > > > +1. > > > When I first started using Python (back in the 1.4 days), I assumed a > > list was a singly-linked list. > > Why would you do that? I can think of at least eight different > implementations of the abstract list data structure: > > constant-size array > variable-size array > variable-size array with amortised O(1) appends > singly-linked list > singly-linked list with CDR coding > doubly-linked list > skip list > indexable skip list > > One can reasonably disregard constant-sized arrays as a possibility, > given that Python lists aren't fixed size (pity the poor Pascal and > Fortran coders who are stuck with static arrays!), but the rest are all > reasonable possibilities. Why assume one specific implementation in the > absence of documentation promising certain performance characteristics? > > > Oddly enough, I was going to write in the above paragraph, "like a C++ > > STL list", until I happened to glance at the STL docs and refreshed my > > memory that an STL list is doubly-linked. ?Which just goes to show that > > making assumptions based on names is a bad idea. > > Exactly :) > > > So, we're right back to my statement earlier in this thread that the > > docs are deficient in that they describe behavior with no hint about > > cost. Given that, it should be no surprise that users make incorrect > > assumptions about cost. > > There are quite a few problems with having the documentation specify cost: > > (1) Who is going to do it? Any volunteers? > > (2) Big-oh notation can be misleading, especially for naive users, or > those whose intuition for what's fast has been shaped by other languages. > Big-oh doesn't tell you whether something is fast or slow, only how it > scales -- and sometimes not even then. > > (3) Having documented a particular performance, that discourages > implementation changes. Any would-be patch or new implementation not only > has to consider that the functional behaviour doesn't change, but that > the performance doesn't either. > > In practice the Python developers are unlikely to make an implementation > change which leads to radically worse performance, particularly for > critical types like list and dict. But in other cases, they might choose > to change big-oh behaviour, and not wish to be tied down by documentation > of the cost of operations. > > (4) How much detail is necessary? What about degenerate cases? E.g. dict > lookup in CPython is typically O(1) amortised, but if all the keys hash > to the same value, it falls to O(N). > > (5) Should the language guarantee such degenerate behaviour? Who decides > which costs are guaranteed and which are not? > > (6) Such performance guarantees should be implementation specific, not > language specific. CPython is only one implementation of the language out > of many. > Bringing this thread full circle, does it make sense to strike this passage from the tutorial?: ''' It is also possible to use a list as a queue, where the first element added is the first element retrieved (?first-in, first-out?); however, lists are not efficient for this purpose. While appends and pops from the end of list are fast, doing inserts or pops from the beginning of a list is slow (because all of the other elements have to be shifted by one). ''' I think points #3 and #6 possibly apply. Regarding points #2 and #4, the tutorial is at least not overly technical or specific; it just explains the requirement to shift other elements one by one in simple layman's terms. From steve at REMOVE-THIS-cybersource.com.au Sat Jan 23 12:44:22 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 23 Jan 2010 17:44:22 GMT Subject: iterating lists References: <1mG6n.74060$4N3.53053@newsfe06.ams2> Message-ID: <036b21fb$0$1357$c3e8da3@news.astraweb.com> On Sat, 23 Jan 2010 18:29:33 +0100, Roel Schroeven wrote: >> for w in l1[:]: #use copy of l1 for iteration >> print(l1.pop()) #decomposite list > > I would prefer: > > while l1: > print(l1.pop()) I would prefer: for x in reversed(l1): print(x) l1[:] = [] And garbage dispose of the entire list in one go, instead of an item at a time. -- Steven From python at mrabarnett.plus.com Sat Jan 23 12:47:49 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 23 Jan 2010 17:47:49 +0000 Subject: PyArg_ParseTupleAndKeywords In-Reply-To: References: <91afd9e3-93a1-488d-8e83-299bfc222e24@22g2000yqr.googlegroups.com> Message-ID: <4B5B3645.3090009@mrabarnett.plus.com> Mr.M wrote: > Carl Banks ha scritto: >>> (some declarations omitted here) >> >> You probably shouldn't have, that could be where the error is.... I'd >> include the whole function up to the call that raises the exception. > > Thank you so much Carl for your help, i'll provide more info so that you > can try to fix my errors! > Thank you again! > > First of all i'll try to explain what i've discovered over a night i > spent awake trying to find where the bug is. > > My init behaves differently in this situations: > > [code] > # this works > dummy = mymodule.myclass() > > # this works > dummy = mymodule.myclass("string for service") > > # this also works > dummy = mymodule.myclass("string for service", "string for event_type") > # and it works if i provide arguments in the right sequence, of course > > # this won't work > dummy = mymodule.myclass("string for service", "string for event_type", > free_text = "string for free_text") > > # but this works > dummy = mymodule.myclass(service = "string for service") > > # the worst thing: this doesn't work but it did and i can't > # understand what is changed > > dummy = mymodule.myclass(free_text = "text for free_text") > > ok, every time the code fails i get this exception: > > TypeError: expected string or Unicode object, tuple found > > so, i think for some reason PyArg_ParseTupleAndKeywords receives "args" > as a touple instead of something else. > > This is everything i managed to discover. > > This is the code: > > [code] > /* DEBUG INFO */ > printf("event INIT\n"); > if (args) > { > int i; > printf("args = \"%s\"\n", PyString_AsString(args)); > printf("type = %s\n", > PyString_AsString(PyObject_Repr(PyObject_Type(args)))); > printf("tuple size = %d\n", PyTuple_Size(args)); > for (i = 0; i < PyTuple_Size(args); i++) > { > printf("%d) %s\n", i, > PyString_AsString(PyTuple_GetItem(args, i))); > } > } > else > { > printf("args = NULL\n"); > } > printf("dict:\n"); > if (keywords) > { > printf(" = %s\n", > PyString_AsString(PyObject_Repr(keywords))); > printf("type = %s\n", > PyString_AsString(PyObject_Repr(PyObject_Type(keywords)))); > } > else > { > printf("keywords = NULL\n"); > } > > char* service = NULL; > char* event_type = NULL; > char* free_text = NULL; > char* group_uid = NULL; > char* remote_name = NULL; > char* remote_abook_uid = NULL; > > time_t start_time = -1L; > time_t end_time = -1L; > time_t storage_time = -1L; > > int flags = 0; > > int bytes_sent = -1; > int bytes_received = -1; > > PyObject* temp; > > static char* keywordlist[] = {"service", > "event_type", > "free_text", > "group_uid", > "remote_name", > "remote_abook_uid", > "start_time", > "end_time", > "storage_time", > "flags", > "bytes_sent", > "bytes_received", > NULL}; > > if (!PyArg_ParseTupleAndKeywords(args, keywords, "|ssssssiiiiii", > keywordlist, > &service, > &event_type, > &free_text, > &group_uid, > &remote_name, > &remote_abook_uid, > &start_time, > &end_time, > &storage_time, > &flags, > &bytes_sent, > &bytes_received)) > { > return -1; > } > > printf("PyArg_ParseTupleAndKeywords worked fine\n"); > > [/code] > > (sorry if my code is a little messy and my english rather bad!) > >> Are you sure that PyArg_ParseTupleAndKeywords is what's raising the >> error? > > Yes, i think so. I have a lot of printf in my code! > >> Are you subclassing this type in Python, and passing one of the string >> parameters a tuple by mistake? For instance, did you do something >> like this: >> >> class myclass(_mycmodule._myctype): >> def __init__(self,*args,**kwargs): >> log_something_here() >> super(myclass,self).__init__(args,**kwargs) >> >> Note the missing * on args. > > no, i'm not subclassing. > I have to admit that i had some testcase and a few weeks ago they worked > like a charm... i can't understand what i changed, of course. > >> >> >>> I found that if i write this everything works fine: >>> >>> [code] >>> import mymodule >>> a = mymodule.myclass(service = "blabla", event_type = "event", free_text >>> = "free", group_uid = "group", remote_name = "remote name", >>> remote_abook_uid = "abook", start_time = 1, end_time = 61, storage_time >>> = 61, flags = 2) >>> [/code] >>> >>> in other words, the code only works if *EVERY* argument is specified in >>> exactly the same position it appears in keyword-null-terminated array. >> >> That it would have to be in a certain order is strange. Are you sure >> it's just not merely that they all have to be present? >> > > No, i tryied to remove the "|" and Python complains that 12 argument > must be present. So they are, in a sense, optional, but (and this is > strange), they must be present in the right order even i provide them > with keyword arguments. > >> Tricky, but I think it'd help if you provided more information. My >> gut feeling is that there exception is being raised somewhere other >> than you think it is (i.e., not by PyArg_ParseTuple), so I'd recommend >> adding some debugging statements to track down the exact point wher >> the error occurs. > > Ok, thank you! > The code i posted is exactly the same i run (and it fails). > I've no problem sending you the whole module or adding any other debug > info you think could be helpfull to find the source of the bug. > Did you specify that the method accepts keywords arguments with METH_KEYWORDS? The function would take parameters for the instance (self), the positional arguments (args) and the keyword arguments (kwargs). http://docs.python.org/c-api/structures.html If you don't use METH_KEYWORDS then it'll think that all the arguments are positional, which is what seems to be happening: From mrm at unknown.nospam Sat Jan 23 13:21:25 2010 From: mrm at unknown.nospam (Mr.M) Date: Sat, 23 Jan 2010 18:21:25 GMT Subject: PyArg_ParseTupleAndKeywords In-Reply-To: References: <91afd9e3-93a1-488d-8e83-299bfc222e24@22g2000yqr.googlegroups.com> Message-ID: MRAB ha scritto: > Did you specify that the method accepts keywords arguments with > METH_KEYWORDS? The function would take parameters for the instance > (self), the positional arguments (args) and the keyword arguments > (kwargs). > > http://docs.python.org/c-api/structures.html > > If you don't use METH_KEYWORDS then it'll think that all the arguments > are positional, which is what seems to be happening: > Thank you MRAB for your reply. No, i didn't specify METH_KEYWORDS flag, but i think init method (the one that you put in tp_init slot, it is "initproc" type) doesn't have to appear in the PyMethodDef structure. Maybe i'm wrong? Luca. From info at wilbertberendsen.nl Sat Jan 23 13:44:00 2010 From: info at wilbertberendsen.nl (Wilbert Berendsen) Date: Sat, 23 Jan 2010 19:44:00 +0100 Subject: counting character occurrences In-Reply-To: References: Message-ID: <201001231944.01002.info@wilbertberendsen.nl> Op vrijdag 22 januari 2010 schreef Arnaud: > Why not just start with (untested): > > import codecs > from collections import defaultdict > > tcounters = defaultdict(int) > f = codecs.open('/home/gavron/git/screen/src/screen.c', 'r', "utf-8") > > for c in f.read(): > tcounters[c] += 1 > > for c, n in tcounters.iteritems(): > print "%r\t%i" % (c, n) Or using Counter from Python3.1 collections: import codecs from collections import Counter filename = '/home/gavron/git/screen/src/screen.c' with codecs.open(filename, 'r', 'utf-8') as f: counted = Counter(f.read()) for c, n in counted: print(c, n, sep='\t') with best regards, Wilbert Berendsen -- http://www.wilbertberendsen.nl/ From aahz at pythoncraft.com Sat Jan 23 13:45:59 2010 From: aahz at pythoncraft.com (Aahz) Date: 23 Jan 2010 10:45:59 -0800 Subject: ISC License References: <00eb248d-c9c9-430f-bc83-41ac865c5111@e11g2000yqe.googlegroups.com> Message-ID: In article <00eb248d-c9c9-430f-bc83-41ac865c5111 at e11g2000yqe.googlegroups.com>, Joan Miller wrote: > >There is a license approved by the OSI, the ISC License [1], which >should be included in the PyPi classifiers [2]. > >[1] https://www.isc.org/software/license >http://www.opensource.org/licenses/isc-license.txt >[2] http://pypi.python.org/pypi?%3Aaction=list_classifiers http://pypi.python.org/pypi has a link named "Bug Reports" -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From aahz at pythoncraft.com Sat Jan 23 13:50:42 2010 From: aahz at pythoncraft.com (Aahz) Date: 23 Jan 2010 10:50:42 -0800 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> <8e4d3fe2-c4bd-4a73-9c50-7a336dab25be@o28g2000yqh.googlegroups.com> Message-ID: In article <8e4d3fe2-c4bd-4a73-9c50-7a336dab25be at o28g2000yqh.googlegroups.com>, Steve Howell wrote: >On Jan 22, 11:10=A0pm, a... at pythoncraft.com (Aahz) wrote: >> >>>I know Python's number one concern will never be speed, but if Python >>>makes an O(1) operation into an unnecessarily O(N) operation for no >>>good reasons other than "it's too complicated, " or it "adds another >>>pointer to the structure," or "it adds another conditional check to >>>list_ass_slice for operations that aren't popping off the top," I >>>think it's reasonable to challenge the design philosophy. >> >> "Rough consensus and running code." >> >> You have a good point, but nobody will ever give your idea serious >> attention until there's a patch and benchmarks. > >Here is a benchmark of O(N*N) vs. O(N) for two C programs. One does >memmove; the other simply advances the pointer. You should provide pybench numbers and probably also use the benchmarks produced by the Unladen Swallow project. The next step is to file a patch on bugs.python.org and request review. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From ra.ravi.rav at gmail.com Sat Jan 23 13:51:38 2010 From: ra.ravi.rav at gmail.com (Ravi) Date: Sat, 23 Jan 2010 10:51:38 -0800 (PST) Subject: how to generate random numbers that satisfy certain distribution References: Message-ID: On Jan 23, 10:37?pm, thinke365 wrote: > such as uniform distribution, Normal distribution or poisson distribution. > is there any package that can be used to generate such random numbers. > > -- > View this message in context:http://old.nabble.com/how-to-generate-random-numbers-that-satisfy-cer... > Sent from the Python - python-list mailing list archive at Nabble.com. Did you try random package? From roy at panix.com Sat Jan 23 14:02:53 2010 From: roy at panix.com (Roy Smith) Date: Sat, 23 Jan 2010 14:02:53 -0500 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> Message-ID: In article , Duncan Booth wrote: > Roy Smith wrote: > > > I'm looking at http://tinyurl.com/cdbwog. It shows all the operations > > of a list. What it does not show is their cost. For pop(), it has a > > note: > > > > "The pop() method is only supported by the list and array types. The > > optional argument i defaults to -1, so that by default the last item > > is removed and returned". > > The page you should probably be looking at is > http://wiki.python.org/moin/TimeComplexity I was not aware of this page; thanks for pointing it out. From thinke365 at gmail.com Sat Jan 23 14:10:52 2010 From: thinke365 at gmail.com (thinke365) Date: Sat, 23 Jan 2010 11:10:52 -0800 (PST) Subject: how to generate random numbers that satisfy certain distribution In-Reply-To: References: <27288180.post@talk.nabble.com> Message-ID: <27288996.post@talk.nabble.com> Bugzilla from ra.ravi.rav at gmail.com wrote: > > On Jan 23, 10:37?pm, thinke365 wrote: >> such as uniform distribution, Normal distribution or poisson >> distribution. >> is there any package that can be used to generate such random numbers. >> >> -- >> View this message in >> context:http://old.nabble.com/how-to-generate-random-numbers-that-satisfy-cer... >> Sent from the Python - python-list mailing list archive at Nabble.com. > > Did you try random package? > -- > http://mail.python.org/mailman/listinfo/python-list > > of course i have tried random package, but can this package generate random sequence that satisfy possion distribution , normal distribution and uniform distribution -- View this message in context: http://old.nabble.com/how-to-generate-random-numbers-that-satisfy-certain-distribution-tp27288180p27288996.html Sent from the Python - python-list mailing list archive at Nabble.com. From python at mrabarnett.plus.com Sat Jan 23 14:14:54 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 23 Jan 2010 19:14:54 +0000 Subject: PyArg_ParseTupleAndKeywords In-Reply-To: References: <91afd9e3-93a1-488d-8e83-299bfc222e24@22g2000yqr.googlegroups.com> Message-ID: <4B5B4AAE.3050805@mrabarnett.plus.com> Mr.M wrote: > MRAB ha scritto: > >> Did you specify that the method accepts keywords arguments with >> METH_KEYWORDS? The function would take parameters for the instance >> (self), the positional arguments (args) and the keyword arguments >> (kwargs). >> >> http://docs.python.org/c-api/structures.html >> >> If you don't use METH_KEYWORDS then it'll think that all the arguments >> are positional, which is what seems to be happening: >> > > Thank you MRAB for your reply. > No, i didn't specify METH_KEYWORDS flag, but i think init method (the > one that you put in tp_init slot, it is "initproc" type) doesn't have to > appear in the PyMethodDef structure. > > Maybe i'm wrong? > I think you're right. From peteRE at MpeteOzilla.Vco.ukE Sat Jan 23 14:17:47 2010 From: peteRE at MpeteOzilla.Vco.ukE (Peter Chant) Date: Sat, 23 Jan 2010 19:17:47 +0000 Subject: how to generate random numbers that satisfy certain distribution References: Message-ID: thinke365 wrote: > > such as uniform distribution, Normal distribution or poisson distribution. > is there any package that can be used to generate such random numbers. > I remeber being told that adding up 12 random numbers in the range 0-1 (which is what most computer random number genertors at the time chucked out) and subtracted 6 gives a pretty good normal distribution. I think I did try it once and it failed, but I must have done something odd. -- http://www.petezilla.co.uk From nulla.epistola at web.de Sat Jan 23 14:36:18 2010 From: nulla.epistola at web.de (Sibylle Koczian) Date: Sat, 23 Jan 2010 20:36:18 +0100 Subject: installing psycopg2-2.0.13 with python3.1 In-Reply-To: References: <03495BC1-8736-42D5-9B59-5B0148F643AA@gmail.com> Message-ID: Iain Barnett schrieb: > On 21 Jan 2010, at 00:11, Gabriel Genellina wrote: > >> If you insist on using Python 3.1, there is another interface to PostgreSQL called pg8000 that claims to be Python 3.x compatible (I've not actually tested it). >> >> [1] http://mail.python.org/pipermail/python-porting/2008-December/000004.html >> [2] http://pybrary.net/pg8000/ >> Or there is py-postgresql: http://python.projects.postgresql.org/ I've not yet used it very much, but it seems to work quite well, is reasonably documented and the mailing list is small and helpful. HTH Sibylle From arnodel at googlemail.com Sat Jan 23 14:57:07 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sat, 23 Jan 2010 19:57:07 +0000 Subject: how to generate random numbers that satisfy certain distribution References: Message-ID: thinke365 writes: > such as uniform distribution, Normal distribution or poisson distribution. > is there any package that can be used to generate such random numbers. It's all in the standard library, the module is called -surprisingly- 'random'. - use random.uniform for the uniform distributions - use random normalvariate for normal distributions There isn't a Poisson distribution function, but there is a expovariate function. I think you can get poisson like this, but take it with a grain of salt because my probability skills are very rusty! import random import itertools def poisson(l): e = random.expovariate acc = 0.0 for n in itertools.count(): acc += e(l) if acc >= 1.0: return n -- Arnaud From no.email at nospam.invalid Sat Jan 23 15:26:03 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 23 Jan 2010 12:26:03 -0800 Subject: how to generate random numbers that satisfy certain distribution References: <27288180.post@talk.nabble.com> Message-ID: <7xy6jo1sbo.fsf@ruckus.brouhaha.com> thinke365 writes: > of course i have tried random package, but can this package generate random > sequence that satisfy possion distribution , normal distribution and uniform > distribution Did you look at the documentation? From no.email at nospam.invalid Sat Jan 23 15:29:22 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 23 Jan 2010 12:29:22 -0800 Subject: how to generate random numbers that satisfy certain distribution References: Message-ID: <7xtyuc1s65.fsf@ruckus.brouhaha.com> Peter Chant writes: > I remeber being told that adding up 12 random numbers in the range 0-1 > (which is what most computer random number genertors at the time chucked > out) and subtracted 6 gives a pretty good normal distribution. I think I > did try it once and it failed, but I must have done something odd. That gives you a binomial distribution on 12 trials, which approximates a normal distribution when the number of trials is large. 12 isn't too bad. But there's a simpler way, the Box-Muller transform, that gives you a pair drawn from a precisely normal distribution from two uniform random samples: http://en.wikipedia.org/wiki/Box-Muller_transform From thinke365 at gmail.com Sat Jan 23 15:57:15 2010 From: thinke365 at gmail.com (thinke365) Date: Sat, 23 Jan 2010 12:57:15 -0800 (PST) Subject: this customize sort did not work ,what's wrong? Message-ID: <27289860.post@talk.nabble.com> l = list() l1 = list((1, 2, 3, 4)) l2 = list((1,2)) l3 = list((1, 2, 3, 4, 5)) l.append(l1) l.append(l2) l.append(l3) print l def sort_by_list(E1, E2): print len(E1), len(E2) return len(list(E1)) > len(list(E2)) l.sort(cmp=sort_by_list) print l output: [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] 2 4 5 2 [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] the order of the elements in the list did not change! -- View this message in context: http://old.nabble.com/this-customize-sort-did-not-work-%2Cwhat%27s-wrong--tp27289860p27289860.html Sent from the Python - python-list mailing list archive at Nabble.com. From thinke365 at gmail.com Sat Jan 23 16:06:22 2010 From: thinke365 at gmail.com (thinke365) Date: Sat, 23 Jan 2010 13:06:22 -0800 (PST) Subject: this customize sort did not work ,what's wrong? In-Reply-To: <27289860.post@talk.nabble.com> References: <27289860.post@talk.nabble.com> Message-ID: <27289922.post@talk.nabble.com> i mean the output i want is: [ [1, 2], [1, 2, 3, 4], [1, 2, 3, 4, 5]], that is sort according to the length of the list element thinke365 wrote: > > l = list() > l1 = list((1, 2, 3, 4)) > l2 = list((1,2)) > l3 = list((1, 2, 3, 4, 5)) > l.append(l1) > l.append(l2) > l.append(l3) > print l > > def sort_by_list(E1, E2): > print len(E1), len(E2) > return len(list(E1)) > len(list(E2)) > > l.sort(cmp=sort_by_list) > print l > > output: > [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] > 2 4 > 5 2 > [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] > > the order of the elements in the list did not change! > -- View this message in context: http://old.nabble.com/this-customize-sort-did-not-work-%2Cwhat%27s-wrong--tp27289860p27289922.html Sent from the Python - python-list mailing list archive at Nabble.com. From thinke365 at gmail.com Sat Jan 23 16:21:06 2010 From: thinke365 at gmail.com (thinke365) Date: Sat, 23 Jan 2010 13:21:06 -0800 (PST) Subject: this customize sort did not work ,what's wrong? In-Reply-To: <27289922.post@talk.nabble.com> References: <27289860.post@talk.nabble.com> <27289922.post@talk.nabble.com> Message-ID: <27290014.post@talk.nabble.com> jesus, now i fixed it, using odd lambda sort. l.sort(lambda x,y: cmp(len(x), len(y))) print l BUT I AM STILL CONFUSED WHY COSTOMIZED SORT FAILED TO SORT AS IT IS PROGRAMMER! thinke365 wrote: > > i mean the output i want is: > [ [1, 2], [1, 2, 3, 4], [1, 2, 3, 4, 5]], that is sort according to the > length of the list element > > thinke365 wrote: >> >> l = list() >> l1 = list((1, 2, 3, 4)) >> l2 = list((1,2)) >> l3 = list((1, 2, 3, 4, 5)) >> l.append(l1) >> l.append(l2) >> l.append(l3) >> print l >> >> def sort_by_list(E1, E2): >> print len(E1), len(E2) >> return len(list(E1)) > len(list(E2)) >> >> l.sort(cmp=sort_by_list) >> print l >> >> output: >> [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] >> 2 4 >> 5 2 >> [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] >> >> the order of the elements in the list did not change! >> > > -- View this message in context: http://old.nabble.com/this-customize-sort-did-not-work-%2Cwhat%27s-wrong--tp27289860p27290014.html Sent from the Python - python-list mailing list archive at Nabble.com. From alfps at start.no Sat Jan 23 16:37:40 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sat, 23 Jan 2010 22:37:40 +0100 Subject: this customize sort did not work ,what's wrong? In-Reply-To: References: <27289860.post@talk.nabble.com> <27289922.post@talk.nabble.com> Message-ID: * thinke365: > jesus, now i fixed it, using odd lambda sort. > l.sort(lambda x,y: cmp(len(x), len(y))) > print l This uses 'cmp'. Your earlier code, quoted below, used '>'. > BUT I AM STILL CONFUSED WHY COSTOMIZED SORT FAILED TO SORT AS IT IS > PROGRAMMER! Please don't shout. > thinke365 wrote: >> i mean the output i want is: >> [ [1, 2], [1, 2, 3, 4], [1, 2, 3, 4, 5]], that is sort according to the >> length of the list element >> >> thinke365 wrote: >>> l = list() >>> l1 = list((1, 2, 3, 4)) >>> l2 = list((1,2)) >>> l3 = list((1, 2, 3, 4, 5)) >>> l.append(l1) >>> l.append(l2) >>> l.append(l3) >>> print l >>> >>> def sort_by_list(E1, E2): >>> print len(E1), len(E2) >>> return len(list(E1)) > len(list(E2)) >>> >>> l.sort(cmp=sort_by_list) >>> print l >>> >>> output: >>> [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] >>> 2 4 >>> 5 2 >>> [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] >>> >>> the order of the elements in the list did not change! >>> Cheers & hth., - Alf From duncan.booth at invalid.invalid Sat Jan 23 16:40:29 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 23 Jan 2010 21:40:29 GMT Subject: this customize sort did not work ,what's wrong? References: <27289860.post@talk.nabble.com> <27289922.post@talk.nabble.com> Message-ID: thinke365 wrote: > jesus, now i fixed it, using odd lambda sort. > l.sort(lambda x,y: cmp(len(x), len(y))) > print l > > BUT I AM STILL CONFUSED WHY COSTOMIZED SORT FAILED TO SORT AS IT IS > PROGRAMMER! Try reading the documentation: >>> help(list.sort) Help on method_descriptor: sort(...) L.sort(cmp=None, key=None, reverse=False) -- stable sort *IN PLACE*; cmp(x, y) -> -1, 0, 1 Your comparison function returned True or False, not -1, 0, +1 From steve at REMOVE-THIS-cybersource.com.au Sat Jan 23 17:05:07 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 23 Jan 2010 22:05:07 GMT Subject: how to generate random numbers that satisfy certain distribution References: <27288180.post@talk.nabble.com> Message-ID: <036b5f17$0$1357$c3e8da3@news.astraweb.com> On Sat, 23 Jan 2010 11:10:52 -0800, thinke365 wrote: > of course i have tried random package, but can this package generate > random sequence that satisfy possion distribution , normal distribution > and uniform distribution Please don't talk garbage. If you had really tried the random module, you would know the answer. What do you think random.uniform does? In an interactive Python session: >>> import random >>> help(random) and read. When you've done that, if you still have any specific questions, please ask. -- Steven From steve at REMOVE-THIS-cybersource.com.au Sat Jan 23 17:06:25 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 23 Jan 2010 22:06:25 GMT Subject: how to generate random numbers that satisfy certain distribution References: <7xtyuc1s65.fsf@ruckus.brouhaha.com> Message-ID: <036b5f64$0$1357$c3e8da3@news.astraweb.com> On Sat, 23 Jan 2010 12:29:22 -0800, Paul Rubin wrote: > Peter Chant writes: >> I remeber being told that adding up 12 random numbers in the range 0-1 >> (which is what most computer random number genertors at the time >> chucked out) and subtracted 6 gives a pretty good normal distribution. >> I think I did try it once and it failed, but I must have done something >> odd. > > That gives you a binomial distribution on 12 trials, which approximates > a normal distribution when the number of trials is large. 12 isn't too > bad. But there's a simpler way, the Box-Muller transform, that gives > you a pair drawn from a precisely normal distribution from two uniform > random samples: > > http://en.wikipedia.org/wiki/Box-Muller_transform The Box-Muller transform is reasonably simple, but you can't be serious that it is simpler than adding twelve random numbers and subtracting six! def almost_normal(): return sum([random.random() for _ in xrange(12)], -6) Not that I'm recommending that anyone use this binomial approximation for anything but the quickest and dirtiest uses, particularly since the random module already has two excellent normal distributions (gauss and normalvariate). -- Steven From no.email at nospam.invalid Sat Jan 23 17:10:10 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 23 Jan 2010 14:10:10 -0800 Subject: how to generate random numbers that satisfy certain distribution References: <7xtyuc1s65.fsf@ruckus.brouhaha.com> <036b5f64$0$1357$c3e8da3@news.astraweb.com> Message-ID: <7xr5pgxykd.fsf@ruckus.brouhaha.com> Steven D'Aprano writes: > The Box-Muller transform is reasonably simple, but you can't be serious > that it is simpler than adding twelve random numbers and subtracting six! If you want a normal distribution, using the Box-Muller transform is simpler because it spares you the complication of figuring out whether the 12-trial binomial approximation is close enough to produce reliable results for your specific application, which you obviously have to do if you are using the approximation for anything serious. It also involves writing less code than that list comprehension, since it is already implemented in the random module so you can just call it directly. From mrm at unknown.nospam Sat Jan 23 17:48:11 2010 From: mrm at unknown.nospam (Mr.M) Date: Sat, 23 Jan 2010 22:48:11 GMT Subject: PyArg_ParseTupleAndKeywords In-Reply-To: References: <91afd9e3-93a1-488d-8e83-299bfc222e24@22g2000yqr.googlegroups.com> Message-ID: MRAB ha scritto: > I think you're right. > I have rewritten my code, a piece at a time, and (and this is very annoying) now it works fine. I really can't understand what went wrong with my old code. Luca. From no.email at please.post Sat Jan 23 17:55:38 2010 From: no.email at please.post (kj) Date: Sat, 23 Jan 2010 22:55:38 +0000 (UTC) Subject: ISO module for binomial coefficients, etc. Message-ID: Before I go off to re-invent a thoroughly invented wheel, I thought I'd ask around for some existing module for computing binomial coefficient, hypergeometric coefficients, and other factorial-based combinatorial indices. I'm looking for something that can handle fairly large factorials (on the order of 10000!), using floating-point approximations as needed, and is smart about optimizations, memoizations, etc. TIA! ~K From tjreedy at udel.edu Sat Jan 23 18:04:01 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 23 Jan 2010 18:04:01 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> Message-ID: On 1/23/2010 12:17 PM, Steve Howell wrote: > Terry Reedy said: > > ''' > If you try writing a full patch, as I believe someone did, or at least > a > prototype thereof, when the idea was discussed, you might have a > better > idea of what the tradeoffs are and why it was rejected. > ''' > > I have to run, but tomorrow I will try to dig through python-dev > archives and find the patch. If anybody has hints on where to look > for it (anybody remember the author, for example?), it would be much > appreciated. The approach you outlined in your other response to me is, I believe, what was considered, investigated, and then rejected (by Guido, with agreement). The discussion may have been on the now-closed and (misspelled) pyk3 (?), or maybe on python-ideas, but my memory is more likely the former. I am sure that Raymond H. was involved also. > If the patch looks simple, I will try to pitch the idea that its time > has come. Now that the specification of the language itself is > frozen, I think there might be more room for improving > implementations. Also, I might be able to make the argument that > tradeoffs of memory vs. CPU vs. code complexity have different forces > in the 2010s. I am not opposed to a possible change, just hasty, ill-informed criticism. If there is not a PEP on this issue, it would be good to have one that recorded the proposal and the pros and cons, regardless of the outcome, so there would be something to refer people to. If that had been already done, it would have shortened this thread considerably. Terry Jan Reedy From tjreedy at udel.edu Sat Jan 23 18:08:37 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 23 Jan 2010 18:08:37 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> Message-ID: On 1/23/2010 2:02 PM, Roy Smith wrote: > In article, > Duncan Booth wrote: > >> Roy Smith wrote: >> >>> I'm looking at http://tinyurl.com/cdbwog. It shows all the operations >>> of a list. What it does not show is their cost. For pop(), it has a >>> note: >>> >>> "The pop() method is only supported by the list and array types. The >>> optional argument i defaults to -1, so that by default the last item >>> is removed and returned". >> >> The page you should probably be looking at is >> http://wiki.python.org/moin/TimeComplexity > > I was not aware of this page; thanks for pointing it out. Perhaps you could suggest on the tracker a place or places in the doc where this relatively new wiki page could be referred to. Perhaps in the introductory paragraphs of the Built-in Type section of the lib ref. Where would you have like to have found it? The page was added in response to threads like this one, but it obviously is more obscure than it should be. Terry Jan Reedy From python at mrabarnett.plus.com Sat Jan 23 18:10:00 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 23 Jan 2010 23:10:00 +0000 Subject: this customize sort did not work ,what's wrong? In-Reply-To: <27290014.post@talk.nabble.com> References: <27289860.post@talk.nabble.com> <27289922.post@talk.nabble.com> <27290014.post@talk.nabble.com> Message-ID: <4B5B81C8.9040106@mrabarnett.plus.com> thinke365 wrote: > jesus, now i fixed it, using odd lambda sort. > l.sort(lambda x,y: cmp(len(x), len(y))) > print l > > BUT I AM STILL CONFUSED WHY COSTOMIZED SORT FAILED TO SORT AS IT IS > PROGRAMMER! > > > thinke365 wrote: >> i mean the output i want is: >> [ [1, 2], [1, 2, 3, 4], [1, 2, 3, 4, 5]], that is sort according to the >> length of the list element >> >> thinke365 wrote: >>> l = list() >>> l1 = list((1, 2, 3, 4)) >>> l2 = list((1,2)) >>> l3 = list((1, 2, 3, 4, 5)) >>> l.append(l1) >>> l.append(l2) >>> l.append(l3) >>> print l >>> >>> def sort_by_list(E1, E2): >>> print len(E1), len(E2) >>> return len(list(E1)) > len(list(E2)) >>> >>> l.sort(cmp=sort_by_list) >>> print l >>> >>> output: >>> [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] >>> 2 4 >>> 5 2 >>> [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] >>> >>> the order of the elements in the list did not change! >>> When comparing [1, 2] with [1, 2, 3, 4] you want it to return -1 to say that it's shorter, so they should therefore be swapped, but you're returning 0 (False), which says it's the same length, so they're not swapped. This is also works: l.sort(lambda x,y: len(x) - len(y)) because the sort really only looks at the sign. From tjreedy at udel.edu Sat Jan 23 18:34:53 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 23 Jan 2010 18:34:53 -0500 Subject: enumerated while loop In-Reply-To: <227C7474-8CD1-4BC8-BDC3-ADBFAE01D052@roalddevries.nl> References: <227C7474-8CD1-4BC8-BDC3-ADBFAE01D052@roalddevries.nl> Message-ID: On 1/23/2010 9:44 AM, Roald de Vries wrote: > Dear all, > > I sometimes want to use an infinite while loop with access to the loop > index, like this: > > def naturals(): > i = 0 > while True: > yield i > y += 1 > > for i in naturals(): > print(i) > > I assume a function like 'naturals' already exists, or a similar > construction for the same purpose. But what is it called? itertools.count From tjreedy at udel.edu Sat Jan 23 18:38:29 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 23 Jan 2010 18:38:29 -0500 Subject: how can i know if a python object have a attribute such as 'attr1'? In-Reply-To: References: Message-ID: On 1/23/2010 10:56 AM, Arnaud Delobelle wrote: > thinke365 writes: > >> for example, i may define a python class: >> class A: >> def sayHello(): >> print 'hello' >> >> a = A() >> a.attr1 = 'hello' >> a.attr2 = 'bb' >> >> b = A() >> a.attr2 = 'aa' >> >> how can i know whether an object have an attribute named attr1? > > hasattr(a, 'attr1') or try: a.attr1 except AttributeError: pass Terry Jan Reedy From tjreedy at udel.edu Sat Jan 23 18:44:57 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 23 Jan 2010 18:44:57 -0500 Subject: this customize sort did not work ,what's wrong? In-Reply-To: <27289860.post@talk.nabble.com> References: <27289860.post@talk.nabble.com> Message-ID: On 1/23/2010 3:57 PM, thinke365 wrote: > > l = list() > l1 = list((1, 2, 3, 4)) > l2 = list((1,2)) > l3 = list((1, 2, 3, 4, 5)) > l.append(l1) > l.append(l2) > l.append(l3) > print l > > def sort_by_list(E1, E2): > print len(E1), len(E2) > return len(list(E1))> len(list(E2)) > > l.sort(cmp=sort_by_list) > print l > > output: > [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] > 2 4 > 5 2 > [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] > > the order of the elements in the list did not change! Here is what is right! >>> l = [ (1,2,3), (1,2), (1,2,3,4,5), () ] >>> l.sort(key=len) >>> l [(), (1, 2), (1, 2, 3), (1, 2, 3, 4, 5)] The cmp param is gone in 3.x in favor of key. Use the latter. Terry Jan Reedy From roy at panix.com Sat Jan 23 22:02:22 2010 From: roy at panix.com (Roy Smith) Date: Sat, 23 Jan 2010 22:02:22 -0500 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> Message-ID: In article , Terry Reedy wrote: > >> The page you should probably be looking at is > >> http://wiki.python.org/moin/TimeComplexity > > > > I was not aware of this page; thanks for pointing it out. > > Perhaps you could suggest on the tracker a place or places in the doc > where this relatively new wiki page could be referred to. Perhaps in the > introductory paragraphs of the Built-in Type section of the lib ref. > Where would you have like to have found it? I think the most logical place would have been right at the table of operations (http://tinyurl.com/cdbwog). From apt.shansen at gmail.com Sat Jan 23 22:14:47 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Sat, 23 Jan 2010 19:14:47 -0800 Subject: this customize sort did not work ,what's wrong? In-Reply-To: <27289860.post@talk.nabble.com> References: <27289860.post@talk.nabble.com> Message-ID: <7a9c25c21001231914h21ff1277ra6ca485ca4405754@mail.gmail.com> On Sat, Jan 23, 2010 at 12:57 PM, thinke365 wrote: > def sort_by_list(E1, E2): > print len(E1), len(E2) > return len(list(E1)) > len(list(E2)) > > l.sort(cmp=sort_by_list) > The cmp function is defined as returning one of three values, -1, 0 and 1. You are returning true or false, so things aren't getting sorted because you're not giving cmp what it is looking for. You could rewrite that as return cmp(len(list(E1)), len(list(E2))) if you want. However, cmp is going away in Py3. You can just do, instead: l.sort(key=len) And you'll get precisely what you're looking for. --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at REMOVE-THIS-cybersource.com.au Sat Jan 23 22:30:46 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 24 Jan 2010 03:30:46 GMT Subject: how to generate random numbers that satisfy certain distribution References: <7xtyuc1s65.fsf@ruckus.brouhaha.com> <036b5f64$0$1357$c3e8da3@news.astraweb.com> <7xr5pgxykd.fsf@ruckus.brouhaha.com> Message-ID: <036bab68$0$1357$c3e8da3@news.astraweb.com> On Sat, 23 Jan 2010 14:10:10 -0800, Paul Rubin wrote: > Steven D'Aprano writes: >> The Box-Muller transform is reasonably simple, but you can't be serious >> that it is simpler than adding twelve random numbers and subtracting >> six! > > If you want a normal distribution, using the Box-Muller transform is > simpler because it spares you the complication of figuring out whether > the 12-trial binomial approximation is close enough to produce reliable > results for your specific application, which you obviously have to do if > you are using the approximation for anything serious. But using Box-Miller gives you the complication of figuring out whether you care about being thread safety, which you have to do if you're doing anything serious. (See the comments in the random module for the gauss method). > It also involves > writing less code than that list comprehension, since it is already > implemented in the random module so you can just call it directly. By that logic, the Linux kernel is simpler than a function to add one to the argument, because the Linux kernel has already been implemented but you have to write your own add_one function. (Except in Forth, which usually comes with a word to add one to the number at the top of the stack.) We can agree that, given that the random module already has two normal distributions, there's no real point in using the binomial approximation. Everything else is quibbling. By the way, does anyone know why there is no Poisson random numbers in the module? The implementation is quite simple (but not as simple as the Linux kernel *wink*): def poisson(lambda_=1): L = math.exp(-lambda_) k = 0 p = 1 while 1: k += 1 p *= random.random() if p <= L: break return k-1 although this can be improved upon for large values of lambda_. -- Steven From python at rcn.com Sat Jan 23 23:00:37 2010 From: python at rcn.com (Raymond Hettinger) Date: Sat, 23 Jan 2010 20:00:37 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> Message-ID: <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> [Steve Howell] > Why wouldn't you get a competent C programmer simply make > list_ass_slice smart enough to make list.pop(0) O(1)? When this suggestion was discussed on python-dev years ago, it was rejected. One reason is that it was somewhat common for C code to access the list data structure directly (bypassing API accessor functions). Changing the list to have a starting offset would break existing C extensions. Another reason is that Guido is non-tolerant of space or time trade-offs for lists and tuples because they pervade the language and are heavily used internally. Any additional space or time requirement however small would impact the language performance as a whole. FWIW, that is also the reason that lists are not weak-referenceable (it would cost one extra pointer field per instance and that wasn't deemed to be worth it). > The brilliant computer scientist, Christian Heimes, provides the > answers, and I am paraphrasing here, of course: IMHO, Christian IS a brilliant computer scientist, so I'll ignore the rude intention and take the sentence literally. > ? 1) You can save 64 bits for every list by not storing an extra > pointer! > ? 2) You can simplify the CPython implementation! > ? 3) You can avoid the oh-so-expensive check "if ilow == 1" for all > operations that don't need this optimization! > > Sounds like two micro-optimizations to me (and a copout to boot). Micro or not, these reasons were part of Guido's rationale. Tim Peters and I also participated in the conversation and did not disagree. So, collections.deque() was born and the problem stopped being an issue. Also, Daniel Stuzbach has published a blist implementation that offers high performance insertions and deletions and fast handling of sparse lists. Raymond From suresh.amritapuri at gmail.com Sun Jan 24 00:57:44 2010 From: suresh.amritapuri at gmail.com (suresh.amritapuri) Date: Sat, 23 Jan 2010 21:57:44 -0800 (PST) Subject: image processing - inverse filtering Message-ID: <5bfefbb6-89a8-49f6-9f02-7d36dfbc0f16@c29g2000yqd.googlegroups.com> Hi, If I am using scipy.ndimage.gaussian_filter() for filtering an image, how to do the inverse filtering? In general how to do this using scipy.ndimage? Thanks suresh From reply-to at works.fine.invalid Sun Jan 24 03:09:24 2010 From: reply-to at works.fine.invalid (NickC) Date: 24 Jan 2010 08:09:24 GMT Subject: How to Embed PHP in HTML print Message-ID: <036becb4$0$1280$c3e8da3@news.astraweb.com> I am running a Python application under apache web server, executing as a cgi script. Most of the output is print statements that write HTML. I'd like to embed some PHP code within the HTML. The PHP is a gallery plugin script that talks to the core photo gallery application, written in php. The idea is that my python app can display a random image from a photo album, using the photo gallery application's plugin script. If I do: 'print ""', it doesn't work. I imagine because the web server never sees it as php code. Is there a way to do this? Some possible ideas: Is there a way I can get python to call functions within a PHP app? Perhaps write the plugin in a separate script in PHP, and include that page within my output so that apache recognises the page inclusion? How to get apache to pay attention to the output so it "wakes up" and does some server-side includes? Many thanks for any help. -- NickC From idrevetnom at free.fr Sun Jan 24 03:09:44 2010 From: idrevetnom at free.fr (idrevetnom) Date: Sun, 24 Jan 2010 09:09:44 +0100 Subject: ISO module for binomial coefficients, etc. References: Message-ID: <4b5bffc6$0$21815$426a74cc@news.free.fr> Maybe this could be of interest : http://tnt.math.metro-u.ac.jp/nzmath/manual/modules/combinatorial.html hope this helps Id From gagsl-py2 at yahoo.com.ar Sun Jan 24 04:54:51 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Sun, 24 Jan 2010 06:54:51 -0300 Subject: distutils not finding all of my pure python modules References: <543c08a0-30c8-49fc-882f-212e099bbd96@h9g2000yqa.googlegroups.com> Message-ID: En Thu, 21 Jan 2010 12:41:20 -0300, Jeremy escribi?: > from distutils.core import setup > > purePythonModules = ['regex', 'gnuFile'] > > setup(name='PythonForSafeguards', > version='0.9.1', > description = 'Python code for MCNP and Safeguards analysis.', > author = 'Jake the Snake', > author_email = 'something at blah.com', > packages = ['MCNP', 'Safeguards'], > url='http://lanl.gov', > py_modules = purePythonModules, > ) > > # ========================================= > > Everythin seems to work fine except the gnuFile.py script does not get > put into the distribution. I know the file exists in the same > directory as regex.py and has the same permissions. regex.py and gnuFile.py must be in the same directory as setup.py, ok? -- Gabriel Genellina From showell30 at yahoo.com Sun Jan 24 05:33:36 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sun, 24 Jan 2010 02:33:36 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> Message-ID: On Jan 23, 8:00?pm, Raymond Hettinger wrote: > [Steve Howell] > > > Why wouldn't you get a competent C programmer simply make > > list_ass_slice smart enough to make list.pop(0) O(1)? > > When this suggestion was discussed on python-dev years ago, > it was rejected. ?One reason is that it was somewhat > common for C code to access the list data structure directly > (bypassing API accessor functions). ?Changing the list to have > a starting offset would break existing C extensions. > > Another reason is that Guido is non-tolerant of space or time > trade-offs for lists and tuples because they pervade the language > and are heavily used internally. ?Any additional space or time > requirement however small would impact the language performance > as a whole. ?FWIW, that is also the reason that lists are not > weak-referenceable (it would cost one extra pointer field per > instance and that wasn't deemed to be worth it). > > > The brilliant computer scientist, Christian Heimes, provides the > > answers, and I am paraphrasing here, of course: > > IMHO, Christian IS a brilliant computer scientist, so I'll ignore > the rude intention and take the sentence literally. > You are also a brilliant computer scientist, despite the fact that you are defending a list implemenation that can't pop the first element off the list in O(1) time. From duncan.booth at invalid.invalid Sun Jan 24 05:37:51 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 24 Jan 2010 10:37:51 GMT Subject: How to Embed PHP in HTML print References: <036becb4$0$1280$c3e8da3@news.astraweb.com> Message-ID: NickC wrote: > Some possible ideas: > Is there a way I can get python to call functions within a PHP app? > Perhaps write the plugin in a separate script in PHP, and include that > page within my output so that apache recognises the page inclusion? How > to get apache to pay attention to the output so it "wakes up" and does > some server-side includes? > One option would be to ditch either the Python or the PHP and do everything in one language. Probably your simplest option is to get the web server to generate the PHP output separately. Since it's the same web server you can just use some javascript in the final web page to call in the PHP generated content with ajax. Alternatively, use urllib in Python to retrieve a page from the Apache server and insert that into its own output: that way you won't requrie javascript in the client, but that might be messy if you have authentication or sessions going on. From steve at REMOVE-THIS-cybersource.com.au Sun Jan 24 06:20:09 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 24 Jan 2010 11:20:09 GMT Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> Message-ID: <036c1967$0$1357$c3e8da3@news.astraweb.com> On Sun, 24 Jan 2010 02:33:36 -0800, Steve Howell wrote: > You are also a brilliant computer scientist, despite the fact that you > are defending a list implemenation that can't pop the first element off > the list in O(1) time. You say that like it's a bad thing. It's very simple: the trade-offs that the Python development team have deliberately chosen aren't the same trade-offs that you prefer. That doesn't make your trade-offs right and Python's wrong. They're just different, and if Python lists had your preferred implementation, I guarantee that somebody would be complaining about it right now. If you're serious about wanting O(1) pops from the start of the list, write your own list implementation and use it. You might even like to make it public, so others can use it as well. But please stop with the snide remarks and poorly disguised insults and back-handed compliments, it's getting tedious. Or just change your algorithm slightly -- it's not hard to turn an algorithm that pops from the start of a list to one that pops from the end of the list. -- Steven From tartley at tartley.com Sun Jan 24 06:28:00 2010 From: tartley at tartley.com (Jonathan Hartley) Date: Sun, 24 Jan 2010 03:28:00 -0800 (PST) Subject: py2exe deal with python command line inside a program References: Message-ID: <16c80c55-f06d-4682-9e7d-594471561f46@b10g2000yqa.googlegroups.com> On Jan 22, 7:35?pm, susan_kij... at yahoo.ca wrote: > Hi, > > I need to create a python subprogress, like this: > myProcess = subprocess.Popen([sys.executable, 'C:\myscript.py'], > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?env=env, stdin=subprocess.PIPE, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?stdout=subprocess.PIPE) > > sys.executable was printed out as ''C:\\Python25\\python.exe'', how > can I make this work in executable package through py2exe? > > I have to fix the following problems: > -Source code shouldn't exposed in an executable program > -Since python ?environment is not required when running an executable > program, how to deal with the situation that "C:\\Python25\ > \python.exe" is required as part of command? > > Thanks in advance! Hi. What does it do when you try to execute it with py2exe? Does it fail to run? What is the error? From clp2 at rebertia.com Sun Jan 24 06:35:41 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 24 Jan 2010 03:35:41 -0800 Subject: py2exe deal with python command line inside a program In-Reply-To: <16c80c55-f06d-4682-9e7d-594471561f46@b10g2000yqa.googlegroups.com> References: <16c80c55-f06d-4682-9e7d-594471561f46@b10g2000yqa.googlegroups.com> Message-ID: <50697b2c1001240335xd0b6487p5caab0ce91f7a3ff@mail.gmail.com> On Sun, Jan 24, 2010 at 3:28 AM, Jonathan Hartley wrote: > On Jan 22, 7:35?pm, susan_kij... at yahoo.ca wrote: >> Hi, >> >> I need to create a python subprogress, like this: >> myProcess = subprocess.Popen([sys.executable, 'C:\myscript.py'], >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?env=env, stdin=subprocess.PIPE, >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?stdout=subprocess.PIPE) >> >> sys.executable was printed out as ''C:\\Python25\\python.exe'', how >> can I make this work in executable package through py2exe? >> >> I have to fix the following problems: >> -Source code shouldn't exposed in an executable program >> -Since python ?environment is not required when running an executable >> program, how to deal with the situation that "C:\\Python25\ >> \python.exe" is required as part of command? >> >> Thanks in advance! > > Hi. What does it do when you try to execute it with py2exe? Does it > fail to run? What is the error? The subprocess call would fail utterly since sys.executable is apparently inaccurate for py2exe-generated executables. Cheers, Chris -- http://blog.rebertia.com From peloko45 at gmail.com Sun Jan 24 06:52:47 2010 From: peloko45 at gmail.com (Joan Miller) Date: Sun, 24 Jan 2010 03:52:47 -0800 (PST) Subject: ISC License References: <00eb248d-c9c9-430f-bc83-41ac865c5111@e11g2000yqe.googlegroups.com> Message-ID: <4c2d6744-b101-4504-bcfb-d2ec08ab68a0@m26g2000yqb.googlegroups.com> On 23 ene, 18:45, a... at pythoncraft.com (Aahz) wrote: > In article <00eb248d-c9c9-430f-bc83-41ac865c5... at e11g2000yqe.googlegroups.com>, > Joan Miller ? wrote: > > > > >There is a license approved by the OSI, the ISC License [1], which > >should be included in the PyPi classifiers [2]. > > >[1]https://www.isc.org/software/license > >http://www.opensource.org/licenses/isc-license.txt > >[2]http://pypi.python.org/pypi?%3Aaction=list_classifiers > > http://pypi.python.org/pypihas a link named "Bug Reports" > -- > Aahz (a... at pythoncraft.com) ? ? ? ? ? <*> ? ? ? ?http://www.pythoncraft.com/ > > import antigravity Added! http://sourceforge.net/tracker/?func=detail&aid=2938526&group_id=66150&atid=513503 From stefan_ml at behnel.de Sun Jan 24 07:02:48 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 24 Jan 2010 13:02:48 +0100 Subject: iterating lists In-Reply-To: <036b21fb$0$1357$c3e8da3@news.astraweb.com> References: <1mG6n.74060$4N3.53053@newsfe06.ams2> <036b21fb$0$1357$c3e8da3@news.astraweb.com> Message-ID: <4b5c36e8$0$6579$9b4e6d93@newsspool3.arcor-online.net> Steven D'Aprano, 23.01.2010 18:44: > On Sat, 23 Jan 2010 18:29:33 +0100, Roel Schroeven wrote: > >>> for w in l1[:]: #use copy of l1 for iteration >>> print(l1.pop()) #decomposite list >> I would prefer: >> >> while l1: >> print(l1.pop()) > > > I would prefer: > > for x in reversed(l1): > print(x) > l1[:] = [] > > > And garbage dispose of the entire list in one go, instead of an item at a > time. IIRC, that's what CPython does anyway, so no need to make this more complex than necessary. Stefan From davea at ieee.org Sun Jan 24 07:07:55 2010 From: davea at ieee.org (Dave Angel) Date: Sun, 24 Jan 2010 07:07:55 -0500 Subject: ISO module for binomial coefficients, etc. In-Reply-To: References: Message-ID: <4B5C381B.8070909@ieee.org> kj wrote: > Before I go off to re-invent a thoroughly invented wheel, I thought > I'd ask around for some existing module for computing binomial > coefficient, hypergeometric coefficients, and other factorial-based > combinatorial indices. I'm looking for something that can handle > fairly large factorials (on the order of 10000!), using floating-point > approximations as needed, and is smart about optimizations, > memoizations, etc. > > TIA! > > ~K > > You do realize that a standard. python floating point number cannot possibly approximate a number like 10000! Better use longs. I'd check out the gamma function, which matches factorial for integer arguments (plus or minus 1). DaveA From alfps at start.no Sun Jan 24 07:55:50 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 24 Jan 2010 13:55:50 +0100 Subject: ISO module for binomial coefficients, etc. In-Reply-To: References: Message-ID: * Dave Angel: > kj wrote: >> Before I go off to re-invent a thoroughly invented wheel, I thought >> I'd ask around for some existing module for computing binomial >> coefficient, hypergeometric coefficients, and other factorial-based >> combinatorial indices. I'm looking for something that can handle >> fairly large factorials (on the order of 10000!), using floating-point >> approximations as needed, and is smart about optimizations, >> memoizations, etc. >> >> TIA! >> >> ~K >> >> > You do realize that a standard. python floating point number cannot > possibly approximate a number like 10000! I think what kj is looking for, provided she/he is knowledgable about the subject, is code that does something like >>> from math import * >>> log_fac = 0 >>> for i in range( 1, 10000+1 ): ... log_fac += log( i, 10 ) ... >>> print( "10000! = {}e{}".format( 10**(log_fac % 1), int( log_fac ) ) ) 10000! = 2.84625968062e35659 >>> _ which turned out to be accurate to 10 digits. > Better use longs. That would involve incredible overhead. E.g., how many bytes for the number above? Those bytes translate into arithmetic overhead. > I'd check out the gamma function, which matches factorial for integer > arguments (plus or minus 1). Or, e.g., logarithms... ;-) Cheers & hth., - Alf From jim.waddle at boeing.com Sun Jan 24 08:54:20 2010 From: jim.waddle at boeing.com (Waddle, Jim) Date: Sun, 24 Jan 2010 05:54:20 -0800 Subject: ctypes for AIX Message-ID: <9C240725D2ABD949A80B46DD93AB107042E8D44101@XCH-NW-08V.nw.nos.boeing.com> I need to use ctypes with python running on AIX. It appears that python is being developed mostly for windows. Is there a policy concerning getting functions like ctypes working on AIX. Jim Waddle KIT-D 425-785-5194 From reply-to at works.fine.invalid Sun Jan 24 08:59:46 2010 From: reply-to at works.fine.invalid (NickC) Date: 24 Jan 2010 13:59:46 GMT Subject: How to Embed PHP in HTML print References: <036becb4$0$1280$c3e8da3@news.astraweb.com> Message-ID: <036c3ed3$0$1277$c3e8da3@news.astraweb.com> On Sun, 24 Jan 2010 10:37:51 +0000, Duncan Booth wrote: > content with ajax. Alternatively, use urllib in Python to retrieve a > page from the Apache server and insert that into its own output: that Thanks for hint on urllib. I shake my head in amazement with python sometimes. I'll write it here: print urllib.urlopen('http://myserver.com/phpscript.php').read() That's it. *One* line. The output from the php script is: random image and the one-liner seamlessly prints that to insert it into the html output. And I thought it would be hard; I should have known better. -- NickC From clp2 at rebertia.com Sun Jan 24 09:30:57 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 24 Jan 2010 06:30:57 -0800 Subject: ctypes for AIX In-Reply-To: <9C240725D2ABD949A80B46DD93AB107042E8D44101@XCH-NW-08V.nw.nos.boeing.com> References: <9C240725D2ABD949A80B46DD93AB107042E8D44101@XCH-NW-08V.nw.nos.boeing.com> Message-ID: <50697b2c1001240630r33845a1bl7f94b40a4f0c4436@mail.gmail.com> On Sun, Jan 24, 2010 at 5:54 AM, Waddle, Jim wrote: > I need to use ctypes with python running on AIX. According to the ctypes readme, ctypes is based on libffi, which according to its website, supports AIX for PowerPC64. So, perhaps you could state what the actual error or problem you're encountering is? It is theoretically possible the ctypes-bundled libffi is either outdated or had the AIX-specific bits removed; I don't know, I'm not a CPython dev. > It appears that python is being developed mostly for windows. No, not really; your statement is especially ironic considering one of Python's primary areas of use is for web applications as part of a LAMP stack. > Is there a policy concerning getting functions like ctypes working on AIX. No idea. Someone will probably chime in though. Cheers, Chris -- http://blog.rebertia.com From sg552 at hotmail.co.uk Sun Jan 24 10:08:15 2010 From: sg552 at hotmail.co.uk (Rotwang) Date: Sun, 24 Jan 2010 15:08:15 +0000 Subject: Default path for files Message-ID: Hi all, can anybody tell me whether there's a way to change the default location for files to be opened by open()? I'd like to be able to create files somewhere other than my Python folder without having to write the full path in the filename every time. Sorry if this is a stupid question, I don't know much about programming. From krister.svanlund at gmail.com Sun Jan 24 10:14:25 2010 From: krister.svanlund at gmail.com (Krister Svanlund) Date: Sun, 24 Jan 2010 16:14:25 +0100 Subject: Default path for files In-Reply-To: References: Message-ID: <2cf430a61001240714x21ddac6av46145a10eec434ab@mail.gmail.com> On Sun, Jan 24, 2010 at 4:08 PM, Rotwang wrote: > Hi all, can anybody tell me whether there's a way to change the default > location for files to be opened by open()? I'd like to be able to create > files somewhere other than my Python folder without having to write the full > path in the filename every time. Sorry if this is a stupid question, I don't > know much about programming. Check out http://docs.python.org/library/os.html and the function chdir it is what you are looking for. From rpjday at crashcourse.ca Sun Jan 24 10:28:26 2010 From: rpjday at crashcourse.ca (Robert P. J. Day) Date: Sun, 24 Jan 2010 10:28:26 -0500 (EST) Subject: how to list the attributes of a class, not an object? Message-ID: once again, probably a trivial question but i googled and didn't get an obvious solution. how to list the attributes of a *class*? eg., i was playing with dicts and noticed that the type returned by the keys() method was "dict_keys". so i'm now curious as to the attributes of the dict_keys class. but i don't know how to look at that without first *creating* such an instance, then asking for "dir(dk)". surely there's a simpler way just using the class name, no? rday p.s. any recommendations for the most concise reference sheet for python 3 that exists? being able to print off the entire language spec on two or four pages and tacking it up in front of me would be just ducky. thanks. -- ======================================================================== Robert P. J. Day Waterloo, Ontario, CANADA Linux Consulting, Training and Kernel Pedantry. Web page: http://crashcourse.ca Twitter: http://twitter.com/rpjday ======================================================================== From bblais at bryant.edu Sun Jan 24 10:29:20 2010 From: bblais at bryant.edu (Brian Blais) Date: Sun, 24 Jan 2010 10:29:20 -0500 Subject: some turtle questions Message-ID: <8F7BCECA-CD55-425D-A173-94D7F30DFB53@bryant.edu> Hello, I am trying to think of things to do with the turtle module with my students, and I have some ideas where I am not sure whether the turtle module can do it. 1) is there a way to determine the current screen pixel color? I am thinking about having the turtle go forward until it reaches an object, say a red circle. I can probably do this by making circle objects (drawn with turtles themselves) which know their own position, and check against this info. But I thought it might be useful also for the turtle to know. 2) is there a way to put a limit on the extent the turtle can travel? it seems I can keep moving off of the screen. Is there a way to make it so that a forward(50) command, at the edge, either raises an exception (at the wall) or simply doesn't move the turtle because of the limit? thanks! bb -- Brian Blais bblais at bryant.edu http://web.bryant.edu/~bblais http://bblais.blogspot.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From zuo at chopin.edu.pl Sun Jan 24 10:34:47 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sun, 24 Jan 2010 16:34:47 +0100 Subject: Consume an iterable In-Reply-To: References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <98b43693-3ba2-4d0d-b009-739d841a1c85@36g2000yqu.googlegroups.com> Message-ID: Dnia 23-01-2010 o 15:19:56 Peter Otten <__peter__ at web.de> napisa?(a): >>> def consume_islice(n, items): >>> next(islice(items, n, n), None) > > One problem: the above function doesn't consume the entire iterator like > the original example does for n=None. Passing sys.maxint instead is not > pretty. Not very pretty, but noticeably (though not dramatically) faster for n=None. Consider a modified version of the script from http://bugs.python.org/issue7764: import collections, sys from itertools import islice, repeat def consume0(iterator, n): # the old one collections.deque(islice(iterator, n), maxlen=0) def consume1(iterator, n): # similar to the primary proposal if n is None: collections.deque(iterator, maxlen=0) elif n != 0: next(islice(iterator, n-1, None), None) def consume2(iterator, n): # the approved proposal (see #7764) if n is None: collections.deque(iterator, maxlen=0) else: next(islice(iterator, n, n), None) def consume3(iterator, n): # with sys.maxint if n is None: n = sys.maxint # (maybe should be sys.maxsize instead?) next(islice(iterator, n, n), None) def test(fs): for consume in fs: iterator = iter(range(10)) consume(iterator, 3) rest = list(iterator) assert rest == range(3, 10), consume.__name__ iterator = iter(range(10)) consume(iterator, 0) rest = list(iterator) assert rest == range(10), consume.__name__ iterator = iter(range(10)) consume(iterator, None) rest = list(iterator) assert rest == [], consume.__name__ if __name__ == "__main__": from timeit import Timer fs = (consume0, consume1, consume2, consume3) test(fs) iterator = repeat(None, 1000) for consume in fs: print consume.__name__ for n in (10, 100, 1000, None): print "%6s:" % n, print Timer("consume(iterator, %s)" % n, "import collections, sys\n" "from __main__ import consume, iterator").timeit() print Results [Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41) [GCC 4.3.3] on linux2 pentium4 2.4 GHz]: consume0 10: 2.94313001633 100: 2.91833305359 1000: 2.93242096901 None: 2.90090417862 consume1 10: 1.80793309212 100: 1.7936270237 1000: 1.83439803123 None: 2.37652015686 consume2 10: 1.58784389496 100: 1.5890610218 1000: 1.58557391167 None: 2.37005710602 consume3 10: 1.6071870327 100: 1.61109304428 1000: 1.60717701912 None: 1.81885385513 Regards, *j -- Jan Kaliszewski (zuo) From sg552 at hotmail.co.uk Sun Jan 24 10:49:00 2010 From: sg552 at hotmail.co.uk (Rotwang) Date: Sun, 24 Jan 2010 15:49:00 +0000 Subject: Default path for files In-Reply-To: References: Message-ID: Krister Svanlund wrote: > On Sun, Jan 24, 2010 at 4:08 PM, Rotwang wrote: >> Hi all, can anybody tell me whether there's a way to change the default >> location for files to be opened by open()? I'd like to be able to create >> files somewhere other than my Python folder without having to write the full >> path in the filename every time. Sorry if this is a stupid question, I don't >> know much about programming. > > Check out http://docs.python.org/library/os.html and the function > chdir it is what you are looking for. Thank you. So would adding import os os.chdir() to site.py (or any other module which is automatically imported during initialisation) change the default location to every time I used Python? From zuo at chopin.edu.pl Sun Jan 24 10:56:42 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sun, 24 Jan 2010 16:56:42 +0100 Subject: how to list the attributes of a class, not an object? In-Reply-To: References: Message-ID: 24-01-2010, 16:28:26 Robert P. J. Day wrote > once again, probably a trivial question but i googled and didn't > get an obvious solution. how to list the attributes of a *class*? dir(type(an_obj)) or more reliable: list(vars(type(an_obj))) (dir() uses __dir__ which can be implemented in any way, and default implementation, accordinto to the docs, "attempts to produce the most relevant, rather than complete, information"). > eg., i was playing with dicts and noticed that the type returned by > the keys() method was "dict_keys". so i'm now curious as to the > attributes of the dict_keys class. but i don't know how to look at > that without first *creating* such an instance, then asking for > "dir(dk)". Why you bother about creating an instance? Just do it: list(vars(type({}.keys()))) or dir(type({}.keys())) if dir() satisfies you. Regards, *j -- Jan Kaliszewski (zuo) From alfps at start.no Sun Jan 24 10:56:46 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 24 Jan 2010 16:56:46 +0100 Subject: how to list the attributes of a class, not an object? In-Reply-To: References: Message-ID: * Robert P. J. Day: > once again, probably a trivial question but i googled and didn't > get an obvious solution. how to list the attributes of a *class*? > > eg., i was playing with dicts and noticed that the type returned by > the keys() method was "dict_keys". so i'm now curious as to the > attributes of the dict_keys class. but i don't know how to look at > that without first *creating* such an instance, then asking for > "dir(dk)". Like, dir( list ) where 'list' is the built-in type. There's a pretty-printer for that somewhere, but I can't recall. And as I also recommended in your thread "examining an initial, pristine python3 shell session", help( list ) or more generally help( "list" ) > surely there's a simpler way just using the class name, no? Yes. :-) > rday > > p.s. any recommendations for the most concise reference sheet for > python 3 that exists? being able to print off the entire language > spec on two or four pages and tacking it up in front of me would be > just ducky. thanks. Sorry, don't know. Cheers & hth., - Alf From lists at cheimes.de Sun Jan 24 11:01:27 2010 From: lists at cheimes.de (Christian Heimes) Date: Sun, 24 Jan 2010 17:01:27 +0100 Subject: Default path for files In-Reply-To: References: Message-ID: Rotwang wrote: > import os > os.chdir() > > to site.py (or any other module which is automatically imported during > initialisation) change the default location to every time I used > Python? First of all you shouldn't alter the site module ever! The optional sitecustomize module exists to make global changes. A library must never change the current working directory. It's up to the application to choose the right working directory. If you mess with the working directory in a library or global module like site you *will* break applications. Python has multiple ways to modify the list of importable locations, either globally, for the current user or the current application. Use them wisely! Christian From rpjday at crashcourse.ca Sun Jan 24 11:02:48 2010 From: rpjday at crashcourse.ca (Robert P. J. Day) Date: Sun, 24 Jan 2010 11:02:48 -0500 (EST) Subject: how to list the attributes of a class, not an object? In-Reply-To: References: Message-ID: On Sun, 24 Jan 2010, Alf P. Steinbach wrote: > * Robert P. J. Day: > > once again, probably a trivial question but i googled and didn't > > get an obvious solution. how to list the attributes of a *class*? > > > > eg., i was playing with dicts and noticed that the type returned by > > the keys() method was "dict_keys". so i'm now curious as to the > > attributes of the dict_keys class. but i don't know how to look at > > that without first *creating* such an instance, then asking for > > "dir(dk)". > > Like, > > dir( list ) > > where 'list' is the built-in type. > > There's a pretty-printer for that somewhere, but I can't recall. except that doesn't work for >>> dir(dict_keys) so what's the difference there? rday -- ======================================================================== Robert P. J. Day Waterloo, Ontario, CANADA Linux Consulting, Training and Kernel Pedantry. Web page: http://crashcourse.ca Twitter: http://twitter.com/rpjday ======================================================================== From __peter__ at web.de Sun Jan 24 11:05:58 2010 From: __peter__ at web.de (Peter Otten) Date: Sun, 24 Jan 2010 17:05:58 +0100 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <98b43693-3ba2-4d0d-b009-739d841a1c85@36g2000yqu.googlegroups.com> Message-ID: Jan Kaliszewski wrote: > Dnia 23-01-2010 o 15:19:56 Peter Otten <__peter__ at web.de> napisa?(a): > >>>> def consume_islice(n, items): >>>> next(islice(items, n, n), None) >> >> One problem: the above function doesn't consume the entire iterator like >> the original example does for n=None. Passing sys.maxint instead is not >> pretty. > > Not very pretty, but noticeably (though not dramatically) faster for > n=None. Consider a modified version of the script from > http://bugs.python.org/issue7764: > > import collections, sys > from itertools import islice, repeat > > def consume0(iterator, n): # the old one > collections.deque(islice(iterator, n), maxlen=0) > > def consume1(iterator, n): # similar to the primary proposal > if n is None: > collections.deque(iterator, maxlen=0) > elif n != 0: > next(islice(iterator, n-1, None), None) > > def consume2(iterator, n): # the approved proposal (see #7764) > if n is None: > collections.deque(iterator, maxlen=0) > else: > next(islice(iterator, n, n), None) > > def consume3(iterator, n): # with sys.maxint > if n is None: > n = sys.maxint # (maybe should be sys.maxsize instead?) > next(islice(iterator, n, n), None) > > def test(fs): > for consume in fs: > iterator = iter(range(10)) > consume(iterator, 3) > rest = list(iterator) > assert rest == range(3, 10), consume.__name__ > > iterator = iter(range(10)) > consume(iterator, 0) > rest = list(iterator) > assert rest == range(10), consume.__name__ > > iterator = iter(range(10)) > consume(iterator, None) > rest = list(iterator) > assert rest == [], consume.__name__ > > if __name__ == "__main__": > from timeit import Timer > > fs = (consume0, consume1, > consume2, consume3) > test(fs) > > iterator = repeat(None, 1000) > for consume in fs: > print consume.__name__ > for n in (10, 100, 1000, None): > print "%6s:" % n, > print Timer("consume(iterator, %s)" % n, > "import collections, sys\n" > "from __main__ import consume, > iterator").timeit() > print > > > Results [Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41) [GCC 4.3.3] > on linux2 pentium4 2.4 GHz]: > > consume0 > 10: 2.94313001633 > 100: 2.91833305359 > 1000: 2.93242096901 > None: 2.90090417862 > > consume1 > 10: 1.80793309212 > 100: 1.7936270237 > 1000: 1.83439803123 > None: 2.37652015686 > > consume2 > 10: 1.58784389496 > 100: 1.5890610218 > 1000: 1.58557391167 > None: 2.37005710602 > > consume3 > 10: 1.6071870327 > 100: 1.61109304428 > 1000: 1.60717701912 > None: 1.81885385513 > > > Regards, > *j > Don't the results look suspicious to you? Try measuring with iterator = iter([]) I'm sure you'll get the same result. An "easy" fix which introduces some constant overhead but keeps the results comparable: for consume in fs: print consume.__name__ for n in (10, 100, 1000, None): print "%6s:" % n, print Timer("consume(repeat(None, 1000), %s)" % n, "import collections, sys\n" "from __main__ import consume, repeat").timeit() print Just for fun, here's a variant of consume3 for the paranoid: _sentinel = object() def consume4(iterator, n): if n is None: n = sys.maxint while next(islice(iterator, n, n), _sentinel) is not _sentinel: pass Peter From zuo at chopin.edu.pl Sun Jan 24 11:17:07 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sun, 24 Jan 2010 17:17:07 +0100 Subject: PS. In-Reply-To: References: Message-ID: 24-01-2010, 16:56:42 Jan Kaliszewski wrote: > 24-01-2010, 16:28:26 Robert P. J. Day wrote > >> once again, probably a trivial question but i googled and didn't >> get an obvious solution. how to list the attributes of a *class*? > > dir(type(an_obj)) > > or more reliable: > > list(vars(type(an_obj))) > > (dir() uses __dir__ which can be implemented in any way, and default > implementation, accordinto to the docs, "attempts to produce the most > relevant, rather than complete, information"). I missed one important thing: * dir(a_type) mostly applies to attributes of a_type *and* of its base types/classes [1]. * vars() applies only to attributes of this particular type (AFAIN vars(sth) and sth.__dict__ are practically the same). Example: >>> class D(dict): pass ... >>> dir(D) ['__class__', '__contains__', '__delattr__', '__delitem__', '__dict__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'clear', 'copy', 'fromkeys', 'get', 'items', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values'] >>> list(vars(D)) ['__dict__', '__module__', '__weakref__', '__doc__'] Regards, *j [1] In Python 3.x *type* and *class* is practically the same. (though built-in ones are denoted as *types* rather than *classes* -- using this naming convention a *class* is simply a user-defined *type*). -- Jan Kaliszewski (zuo) From sg552 at hotmail.co.uk Sun Jan 24 11:33:25 2010 From: sg552 at hotmail.co.uk (Rotwang) Date: Sun, 24 Jan 2010 16:33:25 +0000 Subject: Default path for files In-Reply-To: References: Message-ID: Christian Heimes wrote: > Rotwang wrote: >> import os >> os.chdir() >> >> to site.py (or any other module which is automatically imported during >> initialisation) change the default location to every time I used >> Python? > > First of all you shouldn't alter the site module ever! The optional > sitecustomize module exists to make global changes. > > A library must never change the current working directory. It's up to > the application to choose the right working directory. If you mess with > the working directory in a library or global module like site you *will* > break applications. Python has multiple ways to modify the list of > importable locations, either globally, for the current user or the > current application. Use them wisely! > > Christian > OK, thanks. From alfps at start.no Sun Jan 24 11:37:41 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 24 Jan 2010 17:37:41 +0100 Subject: how to list the attributes of a class, not an object? In-Reply-To: References: Message-ID: * Robert P. J. Day: > On Sun, 24 Jan 2010, Alf P. Steinbach wrote: > >> * Robert P. J. Day: >>> once again, probably a trivial question but i googled and didn't >>> get an obvious solution. how to list the attributes of a *class*? >>> >>> eg., i was playing with dicts and noticed that the type returned by >>> the keys() method was "dict_keys". so i'm now curious as to the >>> attributes of the dict_keys class. but i don't know how to look at >>> that without first *creating* such an instance, then asking for >>> "dir(dk)". >> Like, >> >> dir( list ) >> >> where 'list' is the built-in type. >> >> There's a pretty-printer for that somewhere, but I can't recall. > > except that doesn't work for > > >>> dir(dict_keys) > > so what's the difference there? 'list' is a built-in type that by default is available. 'dict_keys' is a type that you're not meant to use directly, so it's not made available by default. The 'type' function yields the type of its argument, so you *can* do e.g. DictKeys = type( {}.keys() ) dir( DictKeys ) list( vars( DictKeys ) ) help( DictKeys ) It doesn't help much though because the only method of interrest is __iter__, which produces an iterator that you can use e.g. in a for loop or to construct a list, whatever. The relevant place to find out more about keys() is in the documentation. Cheers & hth., - Alf From zuo at chopin.edu.pl Sun Jan 24 11:49:32 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sun, 24 Jan 2010 17:49:32 +0100 Subject: Consume an iterable In-Reply-To: References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <98b43693-3ba2-4d0d-b009-739d841a1c85@36g2000yqu.googlegroups.com> Message-ID: > Don't the results look suspicious to you? Try measuring with > > iterator = iter([]) You are obviously right, my brain doesn't work well today :-( But the corret results even more distinctly support my thesis -- that for n=None using n=sys.maxint (consume3) is noticeably faster than 0-maxlen deque (consume1 and consume2): consume0 10: 4.06217813492 100: 8.45529103279 1000: 53.237226963 None: 54.0063519478 consume1 10: 2.59927105904 100: 3.47109603882 1000: 12.7196500301 None: 26.0995740891 consume2 10: 2.36225390434 100: 3.22979712486 1000: 12.5794699192 None: 28.5096430779 consume3 10: 2.39173388481 100: 3.43043398857 1000: 14.3361399174 None: 14.8560190201 Regards, *j -- Jan Kaliszewski (zuo) From zuo at chopin.edu.pl Sun Jan 24 11:58:54 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sun, 24 Jan 2010 17:58:54 +0100 Subject: how to list the attributes of a class, not an object? In-Reply-To: References: Message-ID: 24-01-2010, 17:37:41 Alf P. Steinbach wrote: > DictKeys = type( {}.keys() ) > > dir( DictKeys ) > list( vars( DictKeys ) ) > help( DictKeys ) > > It doesn't help much though because the only method of interrest is > __iter__ Not only. Please, consider: >>> dictkeys = type({}.keys()) >>> set(dir(dictkeys)).difference(dir(object)) {'__ror__', '__rsub__', '__and__', '__rand__', '__contains__', '__len__', '__iter__', '__or__', '__rxor__', '__xor__', '__sub__'} And e.g. comparision-related mehods (__gt__, __le__ and so on) also are more interensing in dict keys views than in plain object() instances... Regards, *j From python at bdurham.com Sun Jan 24 12:05:36 2010 From: python at bdurham.com (python at bdurham.com) Date: Sun, 24 Jan 2010 12:05:36 -0500 Subject: Splitting text at whitespace but keeping the whitespace in the returned list Message-ID: <1264352736.30280.1356298585@webmail.messagingengine.com> I need to parse some ASCII text into 'word' sized chunks of text AND collect the whitespace that seperates the split items. By 'word' I mean any string of characters seperated by whitespace (newlines, carriage returns, tabs, spaces, soft-spaces, etc). This means that my split text can contain punctuation and numbers - just not whitespace. The split( None ) method works fine for returning the word sized chunks of text, but destroys the whitespace separators that I need. Is there a variation of split() that returns delimiters as well as tokens? Thank you, Malcolm -------------- next part -------------- An HTML attachment was scrubbed... URL: From gagsl-py2 at yahoo.com.ar Sun Jan 24 12:09:03 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Sun, 24 Jan 2010 14:09:03 -0300 Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> <03698cd8$0$1357$c3e8da3@news.astraweb.com> Message-ID: En Fri, 22 Jan 2010 10:16:50 -0300, Alf P. Steinbach escribi?: > I get the impression that there's some message traffic that I don't see, > perhaps on the mailing list, since (a) I haven't seen that about > 'locals' pointed out by anyone else in this thread, and I think I've > read all messages in the thread, and (b) sometimes threads pop up that > start with a reply. > > For example, the recent thread "Covert number into string" started with > a /reply/ in my newreader, using EternalSeptember's NNTP host. > > It also starts with a reply in Google's archives, http://groups.google.com/group/comp.lang.python/browse_thread/thread/b8097d4de4a9c9b0/cb3a2e6ccd7736ef>. gmane (news.gmane.org) is a newsserver that acts as a bridge news <-> mailing lists. The post that started the thread you mention is available there, as well as those previous posts in this thread about the danger of using locals() -- Gabriel Genellina From python at mrabarnett.plus.com Sun Jan 24 12:24:40 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 24 Jan 2010 17:24:40 +0000 Subject: Splitting text at whitespace but keeping the whitespace in the returned list In-Reply-To: <1264352736.30280.1356298585@webmail.messagingengine.com> References: <1264352736.30280.1356298585@webmail.messagingengine.com> Message-ID: <4B5C8258.6090508@mrabarnett.plus.com> python at bdurham.com wrote: > I need to parse some ASCII text into 'word' sized chunks of text AND > collect the whitespace that seperates the split items. By 'word' I mean > any string of characters seperated by whitespace (newlines, carriage > returns, tabs, spaces, soft-spaces, etc). This means that my split text > can contain punctuation and numbers - just not whitespace. > > The split( None ) method works fine for returning the word sized chunks > of text, but destroys the whitespace separators that I need. > > Is there a variation of split() that returns delimiters as well as tokens? > I'd use the re module: >>> import re >>> re.split(r'(\s+)', "Hello world!") ['Hello', ' ', 'world!'] From __peter__ at web.de Sun Jan 24 12:24:49 2010 From: __peter__ at web.de (Peter Otten) Date: Sun, 24 Jan 2010 18:24:49 +0100 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <98b43693-3ba2-4d0d-b009-739d841a1c85@36g2000yqu.googlegroups.com> Message-ID: Jan Kaliszewski wrote: > But the corret results even more distinctly support my thesis -- that for > n=None using n=sys.maxint (consume3) is noticeably faster than 0-maxlen > deque (consume1 and consume2): That advantage may not survive the next release: http://svn.python.org/view/python/trunk/Modules/_collectionsmodule.c?r1=68145&r2=70296&pathrev=70296 Peter From alfps at start.no Sun Jan 24 12:25:47 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 24 Jan 2010 18:25:47 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> <03698cd8$0$1357$c3e8da3@news.astraweb.com> Message-ID: * Gabriel Genellina: > En Fri, 22 Jan 2010 10:16:50 -0300, Alf P. Steinbach > escribi?: > >> I get the impression that there's some message traffic that I don't >> see, perhaps on the mailing list, since (a) I haven't seen that about >> 'locals' pointed out by anyone else in this thread, and I think I've >> read all messages in the thread, and (b) sometimes threads pop up that >> start with a reply. >> >> For example, the recent thread "Covert number into string" started >> with a /reply/ in my newreader, using EternalSeptember's NNTP host. >> >> It also starts with a reply in Google's archives, > http://groups.google.com/group/comp.lang.python/browse_thread/thread/b8097d4de4a9c9b0/cb3a2e6ccd7736ef>. >> > > gmane (news.gmane.org) is a newsserver that acts as a bridge news <-> > mailing lists. The post that started the thread you mention is available > there, as well as those previous posts in this thread about the danger > of using locals() Thanks. I have gmane in Thunderbird (after all, IIRC gmane is Norwegian! :-) ), but only with one obscure list. Cheers, - Alf From lognaturel at gmail.com Sun Jan 24 12:35:42 2010 From: lognaturel at gmail.com (Helene Martin) Date: Sun, 24 Jan 2010 09:35:42 -0800 Subject: [Edu-sig] some turtle questions In-Reply-To: <8F7BCECA-CD55-425D-A173-94D7F30DFB53@bryant.edu> References: <8F7BCECA-CD55-425D-A173-94D7F30DFB53@bryant.edu> Message-ID: I'm almost sure that there's no way for a turtle to know anything about the background. That's an unfortunate limitation! As for putting a limit on a turtle's travel, you need to write an appropriate conditional. For example, if you want your turtle to stay within a 200x200 square centered around the origin and stop if it gets out, do something roughly like: while(math.abs(t.xcor()) < 100 and math.abs(t.ycor()) < 100): move turtle Of course, you could instead use if statements and simulate bouncing (if my turtle's x coordinate is beyond my bounding box, subtract from its x coordinate). Best, H?l?ne. Computer Science Teacher Garfield High School http://garfieldcs.com On Sun, Jan 24, 2010 at 7:29 AM, Brian Blais wrote: > Hello, > I am trying to think of things to do with the turtle module with my > students, and I have some ideas where I am not sure whether the turtle > module can do it. > 1) is there a way to determine the current screen pixel color? ?I am > thinking about having the turtle go forward until it reaches an object, say > a red circle. ?I can probably do this by making circle objects (drawn with > turtles themselves) which know their own position, and check against this > info. ?But I thought it might be useful also for the turtle to know. > 2) is there a way to put a limit on the extent the turtle can travel? ?it > seems I can keep moving off of the screen. ?Is there a way to make it so > that a forward(50) command, at the edge, either raises an exception (at the > wall) or simply doesn't move the turtle because of the limit? > > thanks! > > bb > > -- > Brian Blais > bblais at bryant.edu > http://web.bryant.edu/~bblais > http://bblais.blogspot.com/ > > > > _______________________________________________ > Edu-sig mailing list > Edu-sig at python.org > http://mail.python.org/mailman/listinfo/edu-sig > > From python at bdurham.com Sun Jan 24 12:36:33 2010 From: python at bdurham.com (python at bdurham.com) Date: Sun, 24 Jan 2010 12:36:33 -0500 Subject: Splitting text at whitespace but keeping the whitespace in the returned list In-Reply-To: <4B5C8258.6090508@mrabarnett.plus.com> References: <1264352736.30280.1356298585@webmail.messagingengine.com> <4B5C8258.6090508@mrabarnett.plus.com> Message-ID: <1264354593.1381.1356302439@webmail.messagingengine.com> MRAB, "MRAB" wrote: > >>> import re > >>> re.split(r'(\s+)', "Hello world!") > ['Hello', ' ', 'world!'] That was exactly (EXACTLY!) the solution I was looking for. Thank you! Malcolm From zuo at chopin.edu.pl Sun Jan 24 13:01:59 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sun, 24 Jan 2010 19:01:59 +0100 Subject: Consume an iterable In-Reply-To: References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <98b43693-3ba2-4d0d-b009-739d841a1c85@36g2000yqu.googlegroups.com> Message-ID: 24-01-2010, 18:24:49 Peter Otten <__peter__ at web.de> wrote: >> n=None using n=sys.maxint (consume3) is noticeably faster than 0-maxlen >> deque (consume1 and consume2): > > That advantage may not survive the next release: > > http://svn.python.org/view/python/trunk/Modules/_collectionsmodule.c?r1=68145&r2=70296&pathrev=70296 Nice. :) *j -- Jan Kaliszewski (zuo) From gd_usenet at spamfence.net Sun Jan 24 13:04:48 2010 From: gd_usenet at spamfence.net (=?UTF-8?Q?G=C3=BCnther?= Dietrich) Date: Sun, 24 Jan 2010 19:04:48 +0100 Subject: Default path for files References: Message-ID: <0neu27-csn.ln1@spamfence.net> Rotwang wrote: >> Check out http://docs.python.org/library/os.html and the function >> chdir it is what you are looking for. > >Thank you. So would adding > >import os >os.chdir() > >to site.py (or any other module which is automatically imported during >initialisation) change the default location to every time I used >Python? Don't change the library modules. It would catch you anytime when you expect it least. See for the environment variable PYTHONSTARTUP and the associated startup file. Best regards, G?nther From george.sakkis at gmail.com Sun Jan 24 13:11:07 2010 From: george.sakkis at gmail.com (George Sakkis) Date: Sun, 24 Jan 2010 10:11:07 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> <065e450f-2190-42f4-b9c6-4b95260092be@f12g2000yqn.googlegroups.com> <28103200.7M7XYOBnsK@beaureve.gmx.net> <9734672.qlcV6ekshW@beaureve.gmx.net> Message-ID: <6ddbbbfe-402d-4b87-8b61-a5042071dd08@e37g2000yqn.googlegroups.com> On Jan 22, 8:39?pm, Martin Drautzburg wrote: > Martin Drautzburg wrote: > >> with scope(): > >> ? ? # ... > >> ? ? # use up, down, left, right here > > >> # up, down, left, right no longer defined after the with block exits. > > Just looked it up again. It's a cool thing. Too bad my locals() hack > would still be required. The result would be less noisy (and actually > really beautiful) than the decorator implementation though. Thanks > again for pointing this out to me. Both in your example and by using a context manager, you can get away with not passing locals() explicitly by introspecting the stack frame. Here's a context manager that does the trick: from __future__ import with_statement from contextlib import contextmanager import sys @contextmanager def enums(*consts): # 2 levels up the stack to bypass the contextmanager frame f_locals = sys._getframe(2).f_locals new_names = set() reset_locals, updated_locals = {}, {} for const in consts: updated_locals[const] = const if const in f_locals: reset_locals[const] = f_locals[const] else: new_names.add(const) f_locals.update(updated_locals) try: yield finally: for name in new_names: del f_locals[name] f_locals.update(reset_locals) if __name__ == '__main__': def move(aDirection): print "moving " + aDirection up = "outerScopeUp" with enums("up", "down", "left", "right"): move(up) move(down) move(left) move(right) print "in the outer scope up is still:", up print "this should fail:" down Of course, as all other attempts to mess with locals() shown in this thread, this only "works" when locals() is globals(). If you try it within a function, it fails: def test(): up = "outerScopeUp" with enums("up", "down", "left", "right"): move(up) move(down) move(left) move(right) print "in the outer scope up is still:", up print "this should fail:" down ## XXX: doesn't work within a function test() So it's utility is pretty limited; a custom DSL is probably better suited to your problem. George From aahz at pythoncraft.com Sun Jan 24 13:14:29 2010 From: aahz at pythoncraft.com (Aahz) Date: 24 Jan 2010 10:14:29 -0800 Subject: simple cgi program References: <4b531bf9$0$1140$4fafbaef@reader1.news.tin.it> Message-ID: In article <4b531bf9$0$1140$4fafbaef at reader1.news.tin.it>, superpollo wrote: > >i would like to submit the following code for review. it is a simple >common gateway interface program, which uses the least possible >libraries for the sake of mechanism undertanding. You should probably factor this into two or three functions. I would not rename urllib here -- it's too common to want to use "url" for an actual URL. You should not use readline() -- just use read(). Which then obviates the need for the test (there's no real need for using CONTENT_LENGTH for this purpose IIRC). The value for "me" is wrong unless your cgi is installed as a top-level script. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From susan_kijiji at yahoo.ca Sun Jan 24 13:14:37 2010 From: susan_kijiji at yahoo.ca (im_smialing) Date: Sun, 24 Jan 2010 10:14:37 -0800 (PST) Subject: py2exe deal with python command line inside a program References: <16c80c55-f06d-4682-9e7d-594471561f46@b10g2000yqa.googlegroups.com> Message-ID: <8de9116a-0b09-4919-aec2-a9c41e37e177@y12g2000yqh.googlegroups.com> Hi Jonathan, Here is the traceback I got, 'test.py' is where "main" starts, and I replaced 'sys.executable' with string 'python': args: ['python', 'C:\\myscript.py'] Traceback (most recent call last): File "test.py", line 22, in File "subprocess.pyc", line 594, in __init__ File "subprocess.pyc", line 816, in _execute_child WindowsError: [Error 2] The system cannot find the file specified On Jan 24, 6:28?am, Jonathan Hartley wrote: > On Jan 22, 7:35?pm, susan_kij... at yahoo.ca wrote: > > > > > > > Hi, > > > I need to create a python subprogress, like this: > > myProcess = subprocess.Popen([sys.executable, 'C:\myscript.py'], > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?env=env, stdin=subprocess.PIPE, > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?stdout=subprocess.PIPE) > > > sys.executable was printed out as ''C:\\Python25\\python.exe'', how > > can I make this work in executable package through py2exe? > > > I have to fix the following problems: > > -Source code shouldn't exposed in an executable program > > -Since python ?environment is not required when running an executable > > program, how to deal with the situation that "C:\\Python25\ > > \python.exe" is required as part of command? > > > Thanks in advance! > > Hi. What does it do when you try to execute it with py2exe? Does it > fail to run? What is the error?- Hide quoted text - > > - Show quoted text - From susan_kijiji at yahoo.ca Sun Jan 24 13:25:34 2010 From: susan_kijiji at yahoo.ca (im_smialing) Date: Sun, 24 Jan 2010 10:25:34 -0800 (PST) Subject: py2exe deal with python command line inside a program References: <16c80c55-f06d-4682-9e7d-594471561f46@b10g2000yqa.googlegroups.com> Message-ID: <08003ed8-04f5-4f33-a49f-a143da5d3882@h2g2000yqj.googlegroups.com> On Jan 24, 6:35?am, Chris Rebert wrote: > On Sun, Jan 24, 2010 at 3:28 AM, Jonathan Hartley wrote: > > On Jan 22, 7:35?pm, susan_kij... at yahoo.ca wrote: > >> Hi, > > >> I need to create a python subprogress, like this: > >> myProcess = subprocess.Popen([sys.executable, 'C:\myscript.py'], > >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?env=env, stdin=subprocess.PIPE, > >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?stdout=subprocess.PIPE) > > >> sys.executable was printed out as ''C:\\Python25\\python.exe'', how > >> can I make this work in executable package through py2exe? > > >> I have to fix the following problems: > >> -Source code shouldn't exposed in an executable program > >> -Since python ?environment is not required when running an executable > >> program, how to deal with the situation that "C:\\Python25\ > >> \python.exe" is required as part of command? > > >> Thanks in advance! > > > Hi. What does it do when you try to execute it with py2exe? Does it > > fail to run? What is the error? > Thanks for pointing that, this time I try to use 'python' as the arg, I got an error: WindowsError: [Error 2] The system cannot find the file specified Because the subprocess is looking for a source code location, and which was hard coded, any suggestion to work out the issue? > The subprocess call would fail utterly since sys.executable is > apparently inaccurate for py2exe-generated executables. > > Cheers, > Chris > --http://blog.rebertia.com- Hide quoted text - > > - Show quoted text - From clp2 at rebertia.com Sun Jan 24 13:50:50 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 24 Jan 2010 10:50:50 -0800 Subject: py2exe deal with python command line inside a program In-Reply-To: <08003ed8-04f5-4f33-a49f-a143da5d3882@h2g2000yqj.googlegroups.com> References: <16c80c55-f06d-4682-9e7d-594471561f46@b10g2000yqa.googlegroups.com> <08003ed8-04f5-4f33-a49f-a143da5d3882@h2g2000yqj.googlegroups.com> Message-ID: <50697b2c1001241050g108c9055u8d62ff36b039801c@mail.gmail.com> On Sun, Jan 24, 2010 at 10:25 AM, im_smialing wrote: > On Jan 24, 6:35?am, Chris Rebert wrote: >> On Sun, Jan 24, 2010 at 3:28 AM, Jonathan Hartley wrote: >> > On Jan 22, 7:35?pm, susan_kij... at yahoo.ca wrote: >> >> Hi, >> >> >> I need to create a python subprogress, like this: >> >> myProcess = subprocess.Popen([sys.executable, 'C:\myscript.py'], >> >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?env=env, stdin=subprocess.PIPE, >> >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?stdout=subprocess.PIPE) >> >> >> sys.executable was printed out as ''C:\\Python25\\python.exe'', how >> >> can I make this work in executable package through py2exe? >> >> >> I have to fix the following problems: >> >> -Source code shouldn't exposed in an executable program >> >> -Since python ?environment is not required when running an executable >> >> program, how to deal with the situation that "C:\\Python25\ >> >> \python.exe" is required as part of command? >> >> >> Thanks in advance! >> >> > Hi. What does it do when you try to execute it with py2exe? Does it >> > fail to run? What is the error? >> > Thanks for pointing that, this time I try to use 'python' as the arg, > I got an error: > WindowsError: [Error 2] The system cannot find the file specified > > Because the subprocess is looking for a source code location, and > which was hard coded, any suggestion to work out the issue? Famous last words, but I think it's impossible. You'd have to somehow specify the Python interpreter inside the py2exe-generated executable as the program for subprocess.Popen to run, but I sincerely doubt that can be done. I would suggest something involving os.fork(), but you're clearly on Windows, which doesn't support fork(), so that option's out. The closest thing that leaves is execfile(): http://docs.python.org/library/functions.html#execfile Or you could relax your constraints: You could require Python to be installed on the system (I think there are ways to have your program's installer run the Python installer without any user interaction), or you could give up trying to keep the source code secret (it's illegal for your users to redistribute or modify it anyway, assuming you use the right EULA, and py2exe doesn't keep your sourcecode secret anyway - http://stackoverflow.com/questions/261638/how-do-i-protect-python-code) Cheers, Chris -- http://blog.rebertia.com >> The subprocess call would fail utterly since sys.executable is >> apparently inaccurate for py2exe-generated executables. From showell30 at yahoo.com Sun Jan 24 13:59:32 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sun, 24 Jan 2010 10:59:32 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <036c1967$0$1357$c3e8da3@news.astraweb.com> Message-ID: On Jan 24, 3:20?am, Steven D'Aprano wrote: > On Sun, 24 Jan 2010 02:33:36 -0800, Steve Howell wrote: > > You are also a brilliant computer scientist, despite the fact that you > > are defending a list implemenation that can't pop the first element off > > the list in O(1) time. > > You say that like it's a bad thing. It is. > It's very simple: the trade-offs that the Python development team have > deliberately chosen aren't the same trade-offs that you prefer. That > doesn't make your trade-offs right and Python's wrong. They're just > different, and if Python lists had your preferred implementation, I > guarantee that somebody would be complaining about it right now. > > If you're serious about wanting O(1) pops from the start of the list, > write your own list implementation and use it. You might even like to > make it public, so others can use it as well. But please stop with the > snide remarks and poorly disguised insults and back-handed compliments, > it's getting tedious. I will stop being snide, but I will be blunt, and if anybody interprets my criticism as an insult, so be it. The current algorithm is broken. It's a 20th century implementation of lists built on top of a 20th century memory manager. It's at least ten years behind the times. > Or just change your algorithm slightly -- it's not hard to turn an > algorithm that pops from the start of a list to one that pops from the > end of the list. > The fact that you are proposing to reverse a list to work around its performance deficiencies just confirms to me that the algorithm is broken. I will concede the fact that most of CPython's tradeoffs are driven by the limitations of the underlying memory manager. If realloc () allowed you to easily release memory from the front of a previously allocated block, we'd be talking about maybe a 10-line patch here, and it wouldn't impact even list_resize() in a meaningful way. Even with realloc()'s brokenness, you could improve pop(0) in a way that does not impact list access at all, and the patch would not change the time complexity of any operation; it would just add negligible extract bookkeeping within list_resize() and a few other places. The objection that the extra pointer would increase the size of list objects is totally 20th century thinking. It would be totally negligible for any real world program. From showell30 at yahoo.com Sun Jan 24 14:26:32 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sun, 24 Jan 2010 11:26:32 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> Message-ID: On Jan 23, 3:04?pm, Terry Reedy wrote: > On 1/23/2010 12:17 PM, Steve Howell wrote: > > > Terry Reedy said: > > > ''' > > If you try writing a full patch, as I believe someone did, or at least > > a > > prototype thereof, when the idea was discussed, you might have a > > better > > idea of what the tradeoffs are and why it was rejected. > > ''' > > > I have to run, but tomorrow I will try to dig through python-dev > > archives and find the patch. ?If anybody has hints on where to look > > for it (anybody remember the author, for example?), it would be much > > appreciated. > > The approach you outlined in your other response to me is, I believe, > what was considered, investigated, and then rejected (by Guido, with > agreement). The discussion may have been on the now-closed and > (misspelled) pyk3 (?), or maybe on python-ideas, but my memory is more > likely the former. I am sure that Raymond H. was involved also. > > > If the patch looks simple, I will try to pitch the idea that its time > > has come. ?Now that the specification of the language itself is > > frozen, I think there might be more room for improving > > implementations. ?Also, I might be able to make the argument that > > tradeoffs of memory vs. CPU vs. code complexity have different forces > > in the 2010s. > > I am not opposed to a possible change, just hasty, ill-informed > criticism. If there is not a PEP on this issue, it would be good to have > one that recorded the proposal and the pros and cons, regardless of the > outcome, so there would be something to refer people to. If that had > been already done, it would have shortened this thread considerably. > I think it's a good idea to write a PEP on this issue, and I will attempt a first draft. I think I should submit the first draft to python-ideas, correct? I expect the PEP to be at least initially, if not permanently, rejected, but it would not be an exercise in futility, as I agree it's good to record pros and cons of the proposal in one place. The PEP probably would not include a proposed patch until there was a little bit of consensus behind it, but it would not take me a lot of time to present the basic argument. Here is my sketch of what the PEP would look like. Proposal: Improve list's implementation so that deleting elements from the front of the list does not require an O(N) memmove operation. Rationale: Some Python programs that process lists have multiple methods that consume the first element of the list and pop it off. The pattern comes up with parsers in particular, but there are other examples. It is possible now, of course, to use a data structure in Python that has O(1) for deleting off the top of the list, but none of the alternatives fully replicate the benefits of list itself. Specification: Improving CPython's performance does not affect the language itself, so there are no bikeshed arguments to be had with respect to syntax, etc. Any patch would, of course, affect the performance of nearly every Python program in existence, so any patch would have to, at a bare minimum: 1) Not increase the time or memory complexity of any other list operation. 2) Not affect list access at all. 3) Minimally affect list operations that mutate the list. 4) Be reasonably simple within CPython itself. 5) Not be grossly wasteful of memory. Backwards Compatibility: See above. An implementation of this PEP would not change the definition of the language in any way, but it would have to minimally impact the performance of lists for the normal use cases. Implementation: There are two ways to make deleting the first item of the list run more efficiently. The most ambitious proposal is to fix the memory manager itself to allow the release of memory from the start of the chunk. The advantage of this proposal is that it would simplify the changes to list itself, and possibly have collateral benefits for other CPython internal data structures. The disadvantage of the proposal is that there is a strong tradition in CPython to use native memory management, particularly with respect to the fact that it runs on many platforms. The less ambitious proposal is to change the memory management scheme within list itself. There is already precedent in list_resize() to optimistically allocate memory, so it is not a great break from tradition to optimistically defer the release of memory. But it would complicate things. References: I would refer to this thread on comp.lang.python for discussion, and I would also try to dig up older threads on python-dev or elsewhere. From babedoudi at yahoo.fr Sun Jan 24 14:27:44 2010 From: babedoudi at yahoo.fr (=?ISO-8859-1?Q?R=E9mi?=) Date: Sun, 24 Jan 2010 20:27:44 +0100 Subject: Terminal application with non-standard print Message-ID: Hello everyone, I would like to do a Python application that prints data to stdout, but not the common way. I do not want the lines to be printed after each other, but the old lines to be replaced with the new ones, like wget does it for example (when downloading a file you can see the percentage increasing on a same line). I looked into the curses module, but this seems adapted only to do a whole application, and the terminal history is not visible anymore when the application starts. Any ideas? Thanks, Remi From aahz at pythoncraft.com Sun Jan 24 14:28:53 2010 From: aahz at pythoncraft.com (Aahz) Date: 24 Jan 2010 11:28:53 -0800 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> Message-ID: In article , Steve Howell wrote: > >Even with realloc()'s brokenness, you could improve pop(0) in a way >that does not impact list access at all, and the patch would not change >the time complexity of any operation; it would just add negligible >extract bookkeeping within list_resize() and a few other places. Again, your responsibility is to provide a patch and a spectrum of benchmarking tests to prove it. Then you would still have to deal with the objection that extensions use the list internals -- that might be an okay sell given the effort otherwise required to port extensions to Python 3, but that's not the way to bet. Have you actually read the discussions you were pointed at? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From invalid at invalid.invalid Sun Jan 24 14:33:16 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Sun, 24 Jan 2010 19:33:16 +0000 (UTC) Subject: Terminal application with non-standard print References: Message-ID: On 2010-01-24, R?mi wrote: > I would like to do a Python application that prints data to stdout, but > not the common way. I do not want the lines to be printed after each > other, but the old lines to be replaced with the new ones, like wget > does it for example (when downloading a file you can see the percentage > increasing on a same line). sys.stdout.write("Here's the first line") time.sleep(1) sys.stdout.write("\rAnd this line replaces it.") -- Grant From babedoudi at yahoo.fr Sun Jan 24 14:39:53 2010 From: babedoudi at yahoo.fr (=?ISO-8859-1?Q?R=E9mi?=) Date: Sun, 24 Jan 2010 20:39:53 +0100 Subject: Terminal application with non-standard print In-Reply-To: References: Message-ID: Thank you for your answer, but that does not work : the second line is printed after the first one. -- R?mi Grant Edwards wrote: > On 2010-01-24, R?mi wrote: > >> I would like to do a Python application that prints data to stdout, but >> not the common way. I do not want the lines to be printed after each >> other, but the old lines to be replaced with the new ones, like wget >> does it for example (when downloading a file you can see the percentage >> increasing on a same line). > > sys.stdout.write("Here's the first line") > time.sleep(1) > sys.stdout.write("\rAnd this line replaces it.") > From martin at v.loewis.de Sun Jan 24 14:50:15 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 24 Jan 2010 20:50:15 +0100 Subject: [ANN] Python 2.5.5 Release Candidate 2. Message-ID: <4B5CA477.1060807@v.loewis.de> On behalf of the Python development team and the Python community, I'm happy to announce the release candidate 2 of Python 2.5.5. This is a source-only release that only includes security fixes. The last full bug-fix release of Python 2.5 was Python 2.5.4. Users are encouraged to upgrade to the latest release of Python 2.6 (which is 2.6.4 at this point). This releases fixes issues with the logging and tarfile modules, and with thread-local variables. Since the release candidate 1, additional bugs have been fixed in the expat module. See the detailed release notes at the website (also available as Misc/NEWS in the source distribution) for details of bugs fixed. For more information on Python 2.5.5, including download links for various platforms, release notes, and known issues, please see: http://www.python.org/2.5.5 Highlights of the previous major Python releases are available from the Python 2.5 page, at http://www.python.org/2.5/highlights.html Enjoy this release, Martin Martin v. Loewis martin at v.loewis.de Python Release Manager (on behalf of the entire python-dev team) From showell30 at yahoo.com Sun Jan 24 14:53:15 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sun, 24 Jan 2010 11:53:15 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> Message-ID: <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> On Jan 24, 11:28?am, a... at pythoncraft.com (Aahz) wrote: > In article , > Steve Howell ? wrote: > > > > >Even with realloc()'s brokenness, you could improve pop(0) in a way > >that does not impact list access at all, and the patch would not change > >the time complexity of any operation; it would just add negligible > >extract bookkeeping within list_resize() and a few other places. > > Again, your responsibility is to provide a patch and a spectrum of > benchmarking tests to prove it. ?Then you would still have to deal with > the objection that extensions use the list internals -- that might be an > okay sell given the effort otherwise required to port extensions to > Python 3, but that's not the way to bet. Ok. > Have you actually read the discussions you were pointed at? I don't think anybody provided an actual link, but please correct me if I overlooked it. From invalid at invalid.invalid Sun Jan 24 14:53:32 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Sun, 24 Jan 2010 19:53:32 +0000 (UTC) Subject: Terminal application with non-standard print References: Message-ID: On 2010-01-24, R?mi wrote: > Thank you for your answer, but that does not work: Works fine for me. > the second line is printed after the first one. Not when I run it. There's not much more I can say given the level of detail you've provided. -- Grant From python at rcn.com Sun Jan 24 14:57:01 2010 From: python at rcn.com (Raymond Hettinger) Date: Sun, 24 Jan 2010 11:57:01 -0800 (PST) Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <98b43693-3ba2-4d0d-b009-739d841a1c85@36g2000yqu.googlegroups.com> Message-ID: <6f7ef176-f9ea-4975-86d0-009fd01b8308@h34g2000yqm.googlegroups.com> > ? ? ?def consume2(iterator, n): ?# the approved proposal (see #7764) > ? ? ? ? ?if n is None: > ? ? ? ? ? ? ?collections.deque(iterator, maxlen=0) > ? ? ? ? ?else: > ? ? ? ? ? ? ?next(islice(iterator, n, n), None) FWIW, the deque() approach becomes even faster in Py2.7 and Py3.1 which has a high-speed path for the case where maxlen is zero. Here's a snippet from Modules/_collectionsmodule.c: /* Run an iterator to exhaustion. Shortcut for the extend/extendleft methods when maxlen == 0. */ static PyObject* consume_iterator(PyObject *it) { PyObject *item; while ((item = PyIter_Next(it)) != NULL) { Py_DECREF(item); } Py_DECREF(it); if (PyErr_Occurred()) return NULL; Py_RETURN_NONE; } This code consumes an iterator to exhaustion. It is short, sweet, and hard to beat. Raymond From tjreedy at udel.edu Sun Jan 24 15:13:16 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 24 Jan 2010 15:13:16 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> Message-ID: On 1/24/2010 2:26 PM, Steve Howell wrote: > I think it's a good idea to write a PEP on this issue, and I will > attempt a first draft. I think I should submit the first draft to > python-ideas, correct? That is not a *requirement* for drafts in general, but it is a good idea for a community or community-person generated proposal, such as this one. > I expect the PEP to be at least initially, if not permanently, > rejected, Guido sometimes rejects 'no-chance' proposals without waiting to be asked, but he usually waits until the PEP author feels the issue is ripe and asks for a pronouncement. tjr From alfps at start.no Sun Jan 24 15:14:32 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 24 Jan 2010 21:14:32 +0100 Subject: Symbols as parameters? In-Reply-To: <6ddbbbfe-402d-4b87-8b61-a5042071dd08@e37g2000yqn.googlegroups.com> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> <065e450f-2190-42f4-b9c6-4b95260092be@f12g2000yqn.googlegroups.com> <28103200.7M7XYOBnsK@beaureve.gmx.net> <9734672.qlcV6ekshW@beaureve.gmx.net> <6ddbbbfe-402d-4b87-8b61-a5042071dd08@e37g2000yqn.googlegroups.com> Message-ID: Just top-posting for clarity. :-) up = "UP" left = "LEFT" down = "DOWN" right = "RIGHT" # This code is not guaranteed to work by the language specification. # But it is one way to do the solution I presented earlier in the thread. import sys def import_from( module_name ): local_variables = sys._getframe( 1 ).f_locals m = __import__( module_name, globals(), local_variables, "*" ) for name in local_variables: if not name.startswith( "_" ): local_variables[name] = getattr( m, name ) def move( direction ): print( "Moving " + str( direction ) ) def test(): up = "outerScopeUp" class using_directions: (up, left, down, right) = 4*[None]; import_from( "directions" ) move( up ) move( down ) move( left ) move( right ) print( "in the outer scope up is still: " + up ) print( "this should fail:" ) down test() Moving UP Moving DOWN Moving LEFT Moving RIGHT in the outer scope up is still: outerScopeUp this should fail: Traceback (most recent call last): File "locals.py", line 29, in test() File "locals.py", line 27, in test down NameError: global name 'down' is not defined Cheers & hth., - Alf * George Sakkis: > On Jan 22, 8:39 pm, Martin Drautzburg > wrote: > >> Martin Drautzburg wrote: >>>> with scope(): >>>> # ... >>>> # use up, down, left, right here >>>> # up, down, left, right no longer defined after the with block exits. >> Just looked it up again. It's a cool thing. Too bad my locals() hack >> would still be required. The result would be less noisy (and actually >> really beautiful) than the decorator implementation though. Thanks >> again for pointing this out to me. > > Both in your example and by using a context manager, you can get away > with not passing locals() explicitly by introspecting the stack frame. > Here's a context manager that does the trick: > > from __future__ import with_statement > from contextlib import contextmanager > import sys > > @contextmanager > def enums(*consts): > # 2 levels up the stack to bypass the contextmanager frame > f_locals = sys._getframe(2).f_locals > new_names = set() > reset_locals, updated_locals = {}, {} > for const in consts: > updated_locals[const] = const > if const in f_locals: > reset_locals[const] = f_locals[const] > else: > new_names.add(const) > f_locals.update(updated_locals) > try: > yield > finally: > for name in new_names: > del f_locals[name] > f_locals.update(reset_locals) > > > if __name__ == '__main__': > def move(aDirection): > print "moving " + aDirection > > up = "outerScopeUp" > with enums("up", "down", "left", "right"): > move(up) > move(down) > move(left) > move(right) > print "in the outer scope up is still:", up > print "this should fail:" > down > > > Of course, as all other attempts to mess with locals() shown in this > thread, this only "works" when locals() is globals(). If you try it > within a function, it fails: > > def test(): > up = "outerScopeUp" > with enums("up", "down", "left", "right"): > move(up) > move(down) > move(left) > move(right) > print "in the outer scope up is still:", up > print "this should fail:" > down > > ## XXX: doesn't work within a function > test() > > So it's utility is pretty limited; a custom DSL is probably better > suited to your problem. From tekion at gmail.com Sun Jan 24 15:26:30 2010 From: tekion at gmail.com (tekion) Date: Sun, 24 Jan 2010 12:26:30 -0800 (PST) Subject: easy_install error ... Message-ID: <5e9c94df-531f-4725-bcd5-87c603d8df1c@o26g2000vbd.googlegroups.com> All, I am running into issue with easy install error, see error below: python setup.py easy_install -m 'docutils==0.4' running easy_install error: Not a URL, existing file, or requirement spec: "'docutils==0.4'" any idea as to why? thanks. From babedoudi at yahoo.fr Sun Jan 24 15:35:09 2010 From: babedoudi at yahoo.fr (=?ISO-8859-1?Q?R=E9mi?=) Date: Sun, 24 Jan 2010 12:35:09 -0800 (PST) Subject: Terminal application with non-standard print References: Message-ID: <8901726e-d32d-48bc-8a6b-39a05b65a46d@g29g2000yqe.googlegroups.com> My apologies, I did not run the lines properly. Thanks, that works great now. If I understand well, \r erases the last line. How about erasing the previous lines? For example when writing sys.stdout.write("1\n2\n") sys.stdout.write("\r3") the "1" is still visible. -- R?mi On 24 jan, 20:53, Grant Edwards wrote: > On 2010-01-24, R?mi wrote: > > > Thank you for your answer, but that does not work: > > Works fine for me. > > > the second line is printed after the first one. > > Not when I run it. > > There's not much more I can say given the level of detail > you've provided. > > -- > Grant From no.email at nospam.invalid Sun Jan 24 15:44:25 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 24 Jan 2010 12:44:25 -0800 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> Message-ID: <7xaaw3b5cm.fsf@ruckus.brouhaha.com> Steve Howell writes: > Proposal: Improve list's implementation so that deleting elements from > the front of the list does not require an O(N) memmove operation. ... > It is possible now, of course, to use a data structure in > Python that has O(1) for deleting off the top of the list, but none of > the alternatives fully replicate the benefits of list itself. I think you are mostly referring to deque. Why don't you instead say what you think is wrong with using deque, and how deque can be improved? > See above. An implementation of this PEP would not change the > definition of the language in any way, but it would have to minimally > impact the performance of lists for the normal use cases. But you're talking about adding one or two words to EVERY list, and many normal use cases allocate a LOT of lists. Those use cases are likely more common than use cases that pop from the front of the list but for some reason can't use deque. > The most ambitious proposal is to fix the memory manager itself to > allow the release of memory from the start of the chunk. That's inappropriate given the memory fragmentation it would cause. Really, you're describing a problem that arises in a few programs but up til now, as far as I know, everyone has found deque to be an adequate solution. Your approach of snarling against list is not persuading anyone that list needs to be changed, because most everyone is satisfied with the existing solution. You might change approaches and discuss deque, what's wrong with it, and whether it can be fixed. Getting a change approved for deque is probably much easier than getting one approved for list, just because nowhere near as many things depend on deque's performance. And when discussing performance in this context, additive constants do matter. From alfps at start.no Sun Jan 24 15:44:36 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 24 Jan 2010 21:44:36 +0100 Subject: [Edu-sig] some turtle questions In-Reply-To: References: <8F7BCECA-CD55-425D-A173-94D7F30DFB53@bryant.edu> Message-ID: * Helene Martin: > I'm almost sure that there's no way for a turtle to know anything > about the background. That's an unfortunate limitation! The "background" for the turtle is just a Tkinter canvas. So yes, it's technically possible to inspect things there, since there is method to obtain the canvas. But unfortunately the canvas is not a bitmapped picture: it just maintains a list of objects to draw (vector graphics, metafile graphics). And so there's no practical way to obtain the "current screen pixel color" as Brian asks for: there are no pixels in that canvas... Blatant plug: I included some perhaps interesting turtle examples in ch 2 of my writings at . It starts with some silly figures, then graph drawing, then some recursive figures (idealized tree, C-curve, dragoncurve). Helene: since I'm not on the original list, could you perhaps forward to that list or to the original poster? Thanks, - Alf > As for putting a limit on a turtle's travel, you need to write an > appropriate conditional. For example, if you want your turtle to stay > within a 200x200 square centered around the origin and stop if it gets > out, do something roughly like: > > while(math.abs(t.xcor()) < 100 and math.abs(t.ycor()) < 100): > move turtle > > Of course, you could instead use if statements and simulate bouncing > (if my turtle's x coordinate is beyond my bounding box, subtract from > its x coordinate). > > Best, > > H?l?ne. > Computer Science Teacher > Garfield High School > http://garfieldcs.com > > On Sun, Jan 24, 2010 at 7:29 AM, Brian Blais wrote: >> Hello, >> I am trying to think of things to do with the turtle module with my >> students, and I have some ideas where I am not sure whether the turtle >> module can do it. >> 1) is there a way to determine the current screen pixel color? I am >> thinking about having the turtle go forward until it reaches an object, say >> a red circle. I can probably do this by making circle objects (drawn with >> turtles themselves) which know their own position, and check against this >> info. But I thought it might be useful also for the turtle to know. >> 2) is there a way to put a limit on the extent the turtle can travel? it >> seems I can keep moving off of the screen. Is there a way to make it so >> that a forward(50) command, at the edge, either raises an exception (at the >> wall) or simply doesn't move the turtle because of the limit? >> >> thanks! From no.email at nospam.invalid Sun Jan 24 15:45:19 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 24 Jan 2010 12:45:19 -0800 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <98b43693-3ba2-4d0d-b009-739d841a1c85@36g2000yqu.googlegroups.com> <6f7ef176-f9ea-4975-86d0-009fd01b8308@h34g2000yqm.googlegroups.com> Message-ID: <7x636rb5b4.fsf@ruckus.brouhaha.com> Raymond Hettinger writes: > This code consumes an iterator to exhaustion. > It is short, sweet, and hard to beat. I've always used sum(1 for x in iterator) or some such. From davea at ieee.org Sun Jan 24 16:11:03 2010 From: davea at ieee.org (Dave Angel) Date: Sun, 24 Jan 2010 16:11:03 -0500 Subject: ISO module for binomial coefficients, etc. In-Reply-To: References: Message-ID: <4B5CB767.6010100@ieee.org> Alf P. Steinbach wrote: >
* Dave > Angel: >> kj wrote: >>> Before I go off to re-invent a thoroughly invented wheel, I thought >>> I'd ask around for some existing module for computing binomial >>> coefficient, hypergeometric coefficients, and other factorial-based >>> combinatorial indices. I'm looking for something that can handle >>> fairly large factorials (on the order of 10000!), using floating-point >>> approximations as needed, and is smart about optimizations, >>> memoizations, etc. >>> >>> TIA! >>> >>> ~K >>> >>> >> You do realize that a standard. python floating point number cannot >> possibly approximate a number like 10000! > > I think what kj is looking for, provided she/he is knowledgable about > the subject, is code that does something like > > >>> from math import * > >>> log_fac = 0 > >>> for i in range( 1, 10000+1 ): > ... log_fac += log( i, 10 ) > ... > >>> print( "10000! = {}e{}".format( 10**(log_fac % 1), int( log_fac > ) ) ) > 10000! = 2.84625968062e35659 > >>> _ > > which turned out to be accurate to 10 digits. > > >> Better use longs. > > That would involve incredible overhead. E.g., how many bytes for the > number above? Those bytes translate into arithmetic overhead. > About 14k. > >> I'd check out the gamma function, which matches factorial for integer >> arguments (plus or minus 1). > > Or, e.g., logarithms... ;-) > > > Cheers & hth., > > - Alf > I didn't think of simply summing the logs. I did have some optimizations in mind for the multiply of the longs. If you do lots of partial products, you can do a good bit of the work with smaller numbers, and only get to longs when those partial products get big enough. You could also use scaling when the numbers do start getting bigger. But I still think there must be code for the gamma function that would be quicker. But I haven't chased that lead. DaveA From susan_kijiji at yahoo.ca Sun Jan 24 16:17:44 2010 From: susan_kijiji at yahoo.ca (im_smiling) Date: Sun, 24 Jan 2010 13:17:44 -0800 (PST) Subject: py2exe deal with python command line inside a program References: <16c80c55-f06d-4682-9e7d-594471561f46@b10g2000yqa.googlegroups.com> <08003ed8-04f5-4f33-a49f-a143da5d3882@h2g2000yqj.googlegroups.com> Message-ID: <489fc373-e245-4519-a374-b836f480c553@k35g2000yqb.googlegroups.com> Thanks for taking time to help me. If I use either way, requiring python being installed, or allow source codes exposure, it seems in some degree, there's not necessarily to make the executable package any more, which is very frustrating.... Suppose I take the first way, python environment must be on a machine, how can I have py2exe find a specific file path, for instance as below, 2 up levels from running director, how to find myscript.py in py2exe? Since in customers' machines, C:\dev\mysricpt.py doesn't exist.... python environment: running directory - C:\dev\level1\level2\test.py, suprocess directory-C:\dev\mysricpt.py py2exe C:\dev\level1\level2\dist On Jan 24, 1:50?pm, Chris Rebert wrote: > On Sun, Jan 24, 2010 at 10:25 AM, im_smialing wrote: > > On Jan 24, 6:35?am, Chris Rebert wrote: > >> On Sun, Jan 24, 2010 at 3:28 AM, Jonathan Hartley wrote: > >> > On Jan 22, 7:35?pm, susan_kij... at yahoo.ca wrote: > >> >> Hi, > > >> >> I need to create a python subprogress, like this: > >> >> myProcess = subprocess.Popen([sys.executable, 'C:\myscript.py'], > >> >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?env=env, stdin=subprocess.PIPE, > >> >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?stdout=subprocess.PIPE) > > >> >> sys.executable was printed out as ''C:\\Python25\\python.exe'', how > >> >> can I make this work in executable package through py2exe? > > >> >> I have to fix the following problems: > >> >> -Source code shouldn't exposed in an executable program > >> >> -Since python ?environment is not required when running an executable > >> >> program, how to deal with the situation that "C:\\Python25\ > >> >> \python.exe" is required as part of command? > > >> >> Thanks in advance! > > >> > Hi. What does it do when you try to execute it with py2exe? Does it > >> > fail to run? What is the error? > > > Thanks for pointing that, this time I try to use 'python' as the arg, > > I got an error: > > WindowsError: [Error 2] The system cannot find the file specified > > > Because the subprocess is looking for a source code location, and > > which was hard coded, any suggestion to work out the issue? > > Famous last words, but I think it's impossible. You'd have to somehow > specify the Python interpreter inside the py2exe-generated executable > as the program for subprocess.Popen to run, but I sincerely doubt that > can be done. > > I would suggest something involving os.fork(), but you're clearly on > Windows, which doesn't support fork(), so that option's out. > > The closest thing that leaves is execfile():http://docs.python.org/library/functions.html#execfile > > Or you could relax your constraints: > You could require Python to be installed on the system (I think there > are ways to have your program's installer run the Python installer > without any user interaction), or you could give up trying to keep the > source code secret (it's illegal for your users to redistribute or > modify it anyway, assuming you use the right EULA, and py2exe doesn't > keep your sourcecode secret anyway -http://stackoverflow.com/questions/261638/how-do-i-protect-python-code) > > Cheers, > Chris > --http://blog.rebertia.com > > >> The subprocess call would fail utterly since sys.executable is > >> apparently inaccurate for py2exe-generated executables. > > From news123 at free.fr Sun Jan 24 16:19:38 2010 From: news123 at free.fr (News123) Date: Sun, 24 Jan 2010 22:19:38 +0100 Subject: start .pyo files with doubleclick on windows Message-ID: <4b5cb96a$0$23933$426a74cc@news.free.fr> Hi, I'd like to start .pyo files under windows with a double click. (I know I can just write a .bat wrapper, but somehow it would be more fun to start with a direct double click) Currently this works if the file does not import any other .pyo file. The problem is, that a dobleclick performs a "python.exe myfile.pyo" In order to import precompiled .pyo files however it seems, that "python.exe -OO myfile.pyo" would be required. Is there any trick to do this under windows? The only trick, that I can think of is to write a small wrapper executable, that would then call python -OO. Wouldn't it be useful if python searched also for .pyo files when trying to import and being invoked with "python.exe mycode.pyo" thanks in advance for any ideas suggestion From alfps at start.no Sun Jan 24 16:25:53 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 24 Jan 2010 22:25:53 +0100 Subject: start .pyo files with doubleclick on windows In-Reply-To: <4b5cb96a$0$23933$426a74cc@news.free.fr> References: <4b5cb96a$0$23933$426a74cc@news.free.fr> Message-ID: * News123: > Hi, > > > I'd like to start .pyo files under windows with a double click. > (I know I can just write a .bat wrapper, but somehow it would be more > fun to start with a direct double click) > > > Currently this works if the file does not import any other .pyo file. > > > The problem is, that a dobleclick performs a > "python.exe myfile.pyo" > > In order to import precompiled .pyo files however it seems, that > "python.exe -OO myfile.pyo" > > would be required. > > > Is there any trick to do this under windows? > The only trick, that I can think of is to write a small wrapper > executable, that would then call python -OO. > > > Wouldn't it be useful if python searched also for .pyo files when trying > to import and being invoked with > > "python.exe mycode.pyo" > > > thanks in advance for any ideas suggestion C:\> assoc .pyo .pyo=Python.CompiledFile C:\> ftype python.compiledfile python.compiledfile="C:\Program Files\cpython\python31\python.exe" "%1" %* C:\> _ Use ftype to change the association. Cheers & hth., - Alf From nospam at nospam.com Sun Jan 24 16:33:39 2010 From: nospam at nospam.com (Gilles Ganault) Date: Sun, 24 Jan 2010 22:33:39 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: On Fri, 22 Jan 2010 13:17:44 +0100, Gilles Ganault wrote: >To avoid users from creating login names that start with digits in >order to be listed at the top, I'd like to sort the list differently >every minute so that it'll start with the next letter, eg. display the >list from A...Zdigits the first time, then B...ZAdigits, etc. Thanks everyone for the great feedback. I ended up using - a list instead of a dictionary to hold the list of names - use the random library to pick a character from which to start listing names (simpler than saving the next letter into a file, and starting from this the next time the loop is gone through) For those interested, here's some working code: ========== import random connected = ["_test","-test","0test","1test","Aa","Aab","Bb","Bbbbb","Cc","Ccaaaaaa"] connected.sort() #Fill list with non-duplicate first letter of all items characters=[] for name in connected: char = name[0] #if this character not in list, add to array if not char in characters: characters.append(char) #Pick a random character from which to start iterating index = random.randint(0,len(characters)-1) startch = characters[index] print "Will start from %s" % startch #Go through list starting with items that start with 'starch', and rotate from beginning result = [name for name in connected if name[0] >= startch] + [name for name in connected if name[0] < startch] print result ========== Thanks again. From casevh at gmail.com Sun Jan 24 16:49:36 2010 From: casevh at gmail.com (casevh) Date: Sun, 24 Jan 2010 13:49:36 -0800 (PST) Subject: ISO module for binomial coefficients, etc. References: Message-ID: On Jan 23, 2:55?pm, kj wrote: > Before I go off to re-invent a thoroughly invented wheel, I thought > I'd ask around for some existing module for computing binomial > coefficient, hypergeometric coefficients, and other factorial-based > combinatorial indices. ?I'm looking for something that can handle > fairly large factorials (on the order of 10000!), using floating-point > approximations as needed, and is smart about optimizations, > memoizations, etc. > > TIA! > > ~K If you need exact values, gmpy (http://code.google.com/p/gmpy/) has basic, but fast, support for calculating binomial coefficients and factorials. If you are floating point approximations, check out mpmath (http://code.google.com/p/mpmath/). casevh From daniel at stutzbachenterprises.com Sun Jan 24 16:51:00 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Sun, 24 Jan 2010 15:51:00 -0600 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> Message-ID: On Sun, Jan 24, 2010 at 1:53 PM, Steve Howell wrote: > I don't think anybody provided an actual link, but please correct me > if I overlooked it. I have to wonder if my messages are all ending up in your spam folder for some reason. :-) PEP 3128 (which solves your problem, but not using the implementation you suggest) http://www.python.org/dev/peps/pep-3128/ Implementation as an extension module: http://pypi.python.org/pypi/blist/ Related discussion: http://mail.python.org/pipermail/python-3000/2007-April/006757.html http://mail.python.org/pipermail/python-3000/2007-May/007491.html Detailed performance comparison: http://stutzbachenterprises.com/performance-blist I maintain a private fork of Python 3 with the blist replacing the regular list, as a way of rigorously testing the blist implementation. Although I originally proposed a PEP, I am content to have the blist exist as a third-party module. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC From news123 at free.fr Sun Jan 24 17:36:53 2010 From: news123 at free.fr (News123) Date: Sun, 24 Jan 2010 23:36:53 +0100 Subject: start .pyo files with doubleclick on windows In-Reply-To: References: <4b5cb96a$0$23933$426a74cc@news.free.fr> Message-ID: <4b5ccb85$0$22008$426a74cc@news.free.fr> Hi Alf, Alf P. Steinbach wrote: > * News123: >> Hi, >> >> >> I'd like to start .pyo files under windows with a double click. > > C:\> assoc .pyo > .pyo=Python.CompiledFile > > C:\> ftype python.compiledfile > python.compiledfile="C:\Program Files\cpython\python31\python.exe" "%1" %* > > C:\> _ > > > Use ftype to change the association. Thanks a lot, I leared something new about Windows What I did now is this: assoc .pyo=Python.CompiledOptimizedFile ftype Python.CompiledOptimizedFile="C:\Python26\python.exe" -OO "%1" %* bye N > > > Cheers & hth., > > - Alf From ladasky at my-deja.com Sun Jan 24 17:53:14 2010 From: ladasky at my-deja.com (John Ladasky) Date: Sun, 24 Jan 2010 14:53:14 -0800 (PST) Subject: Broken Python 2.6 installation on Ubuntu Linux 8.04 Message-ID: Hello everyone, I've posted this same question over on ubuntuforums.org, so I'm trying to get help in all of the logical places. I'm running Ubuntu Linux 8.04 (Hardy) on a fairly new x86 box, with two hard disks in a software RAID 1 configuration. Hardy comes with Python 2.5 as a standard package, but not 2.6. I would really like to have the "combinations" function that is included with itertools in Python 2.6. I tried writing a combinations function of my own, but it's SLOW and uses a HUGE amount of memory. So, in my case the Linux Synaptic Package Manager cannot be used to install Python 2.6. I therefore attempted a manual installation. I downloaded the Linux tarball for Python 2.6 from python.org. I followed the installation instructions, and they appeared to execute fine. But when I started IDLE, I still had Python 2.5. No good. I do most of my editing in SCIte. Apparently SCIte knows that I have Python 2.6, and is trying to use it. Alas, my programs depend on extra Python packages such as biopython, numpy and matplotlib. My Python 2.6 distro does not have these yet. None of my programs will run from SCIte! Looking down into the details of the install, I've discovered that Hardy placed the Python 2.4 and 2.5 executables in /usr/bin. My Python 2.6 installation ended up in /usr/local/bin. This may be contributing to my problems. Given the mess I've made by trying to just install plain-old Python, I don't know whether I should attempt to back out, or to press on. Can I convince IDLE to connect to Python 2.6? How do I manually install site packages? Alternately, I COULD upgrade my Ubuntu Linux to 9.4 (Jaunty) or 9.10 (Karmic). Python 2.6 comes standard with both of these. But this is why I mentioned that my storage is RAID1. Apparently, upgrading with RAID present is a serious headache. The Linux wizards are supposed to be fixing these problems in the next release, due in April. I could wait, I suppose. In the mean time, I may have to uninstall Python 2.6 and get my 2.5 running again. I have not found any instructions for how to do that. Help! From arnodel at googlemail.com Sun Jan 24 17:55:53 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sun, 24 Jan 2010 22:55:53 +0000 Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: Gilles Ganault writes: > On Fri, 22 Jan 2010 13:17:44 +0100, Gilles Ganault > wrote: >>To avoid users from creating login names that start with digits in >>order to be listed at the top, I'd like to sort the list differently >>every minute so that it'll start with the next letter, eg. display the >>list from A...Zdigits the first time, then B...ZAdigits, etc. > > Thanks everyone for the great feedback. I ended up using > - a list instead of a dictionary to hold the list of names > - use the random library to pick a character from which to start > listing names (simpler than saving the next letter into a file, and > starting from this the next time the loop is gone through) > > For those interested, here's some working code: > > ========== > import random > > connected = > ["_test","-test","0test","1test","Aa","Aab","Bb","Bbbbb","Cc","Ccaaaaaa"] > connected.sort() > > #Fill list with non-duplicate first letter of all items > characters=[] > for name in connected: > char = name[0] > #if this character not in list, add to array > if not char in characters: > characters.append(char) characters = list(set(name[0] for name in connected)) > #Pick a random character from which to start iterating > index = random.randint(0,len(characters)-1) > startch = characters[index] startch = random.choice(characters) > print "Will start from %s" % startch > > #Go through list starting with items that start with 'starch', and > rotate from beginning > result = [name for name in connected if name[0] >= startch] + [name > for name in connected if name[0] < startch] It would make sense to keep track of the indices where the first word starting with a certain letter starts in the earlier part of the code and to use it now to 'rotate' the list. > print result > ========== > > Thanks again. -- Arnaud From benjamin.kaplan at case.edu Sun Jan 24 18:05:16 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Sun, 24 Jan 2010 18:05:16 -0500 Subject: Broken Python 2.6 installation on Ubuntu Linux 8.04 In-Reply-To: References: Message-ID: On Sun, Jan 24, 2010 at 5:53 PM, John Ladasky wrote: > Hello everyone, > > I've posted this same question over on ubuntuforums.org, so I'm trying > to get help in all of the logical places. > > I'm running Ubuntu Linux 8.04 (Hardy) on a fairly new x86 box, with > two hard disks in a software RAID 1 configuration. > > Hardy comes with Python 2.5 as a standard package, but not 2.6. ?I > would really like to have the "combinations" function that is included > with itertools in Python 2.6. ?I tried writing a combinations function > of my own, but it's SLOW and uses a HUGE amount of memory. > > So, in my case the Linux Synaptic Package Manager cannot be used to > install Python 2.6. ?I therefore attempted a manual installation. ?I > downloaded the Linux tarball for Python 2.6 from python.org. ?I > followed the installation instructions, and they appeared to execute > fine. ?But when I started IDLE, I still had Python 2.5. ?No good. > How did you start IDLE? If you started it by going to the Applications menu, it still points to the Python 2.5 idle. If you open up a terminal and run idle, it should run Python 2.6. If it doesn't, make a .bashrc file in your home directory and add the line "$PATH=/usr/local/bin:$PATH". > I do most of my editing in SCIte. ?Apparently SCIte knows that I have > Python 2.6, and is trying to use it. Alas, my programs depend on extra > Python packages such as biopython, numpy and matplotlib. My Python 2.6 > distro does not have these yet. None of my programs will run from > SCIte! Extensions written in C must be recompiled for every version of Python. Since you're using a version of Python not available through the package manager, your packages are also not available through that. You'll have to download the sources for those and compile them by hand to. This is why most people stick with the precompiled binaries. > > Looking down into the details of the install, I've discovered that > Hardy placed the Python 2.4 and 2.5 executables in /usr/bin. ?My > Python 2.6 installation ended up in /usr/local/bin. ?This may be > contributing to my problems. > It shouldn't be. /usr/local/bin should already be on your path in front of /usr/bin. If it isn't put it there (that's what the .bashrc file I listed before does) > Given the mess I've made by trying to just install plain-old Python, I > don't know whether I should attempt to back out, or to press on. Can I > convince IDLE to connect to Python 2.6? ?How do I manually install > site packages? > Download the package sources and compile them yourself. Python packages have a very easy way to do that- just cd into the source folder and run "python setup.py install" > Alternately, I COULD upgrade my Ubuntu Linux to 9.4 (Jaunty) or 9.10 > (Karmic). ?Python 2.6 comes standard with both of these. ?But this is > why I mentioned that my storage is RAID1. ?Apparently, upgrading with > RAID present is a serious headache. ?The Linux wizards are supposed to > be fixing these problems in the next release, due in April. ?I could > wait, I suppose. > > In the mean time, I may have to uninstall Python 2.6 and get my 2.5 > running again. ?I have not found any instructions for how to do that. > Your Python2.5 installation is just fine. You can get to it by running python2.5 at the command line (as opposed to "python" which should run python2.6) > Help! > -- > http://mail.python.org/mailman/listinfo/python-list > From steve at holdenweb.com Sun Jan 24 18:07:46 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 24 Jan 2010 18:07:46 -0500 Subject: How to Embed PHP in HTML print In-Reply-To: <036c3ed3$0$1277$c3e8da3@news.astraweb.com> References: <036becb4$0$1280$c3e8da3@news.astraweb.com> <036c3ed3$0$1277$c3e8da3@news.astraweb.com> Message-ID: <4B5CD2C2.4080309@holdenweb.com> NickC wrote: > On Sun, 24 Jan 2010 10:37:51 +0000, Duncan Booth wrote: > >> content with ajax. Alternatively, use urllib in Python to retrieve a >> page from the Apache server and insert that into its own output: that > > Thanks for hint on urllib. I shake my head in amazement with python > sometimes. I'll write it here: > > print urllib.urlopen('http://myserver.com/phpscript.php').read() > > That's it. *One* line. > > The output from the php script is: > > alt="random image" > IMG_0759" /> > > and the one-liner seamlessly prints that to insert it into the html output. > > And I thought it would be hard; I should have known better. > > Always great to see Python doing impressive things for new users. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Sun Jan 24 18:07:46 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 24 Jan 2010 18:07:46 -0500 Subject: How to Embed PHP in HTML print In-Reply-To: <036c3ed3$0$1277$c3e8da3@news.astraweb.com> References: <036becb4$0$1280$c3e8da3@news.astraweb.com> <036c3ed3$0$1277$c3e8da3@news.astraweb.com> Message-ID: <4B5CD2C2.4080309@holdenweb.com> NickC wrote: > On Sun, 24 Jan 2010 10:37:51 +0000, Duncan Booth wrote: > >> content with ajax. Alternatively, use urllib in Python to retrieve a >> page from the Apache server and insert that into its own output: that > > Thanks for hint on urllib. I shake my head in amazement with python > sometimes. I'll write it here: > > print urllib.urlopen('http://myserver.com/phpscript.php').read() > > That's it. *One* line. > > The output from the php script is: > > alt="random image" > IMG_0759" /> > > and the one-liner seamlessly prints that to insert it into the html output. > > And I thought it would be hard; I should have known better. > > Always great to see Python doing impressive things for new users. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From __peter__ at web.de Sun Jan 24 18:21:36 2010 From: __peter__ at web.de (Peter Otten) Date: Mon, 25 Jan 2010 00:21:36 +0100 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <98b43693-3ba2-4d0d-b009-739d841a1c85@36g2000yqu.googlegroups.com> <6f7ef176-f9ea-4975-86d0-009fd01b8308@h34g2000yqm.googlegroups.com> Message-ID: Raymond Hettinger wrote: > FWIW, the deque() approach becomes even faster in Py2.7 and Py3.1 > which has a high-speed path for the case where maxlen is zero. > Here's a snippet from Modules/_collectionsmodule.c: > > /* Run an iterator to exhaustion. Shortcut for > the extend/extendleft methods when maxlen == 0. */ > static PyObject* > consume_iterator(PyObject *it) > { > PyObject *item; > > while ((item = PyIter_Next(it)) != NULL) { > Py_DECREF(item); > } > Py_DECREF(it); > if (PyErr_Occurred()) > return NULL; > Py_RETURN_NONE; > } > > > This code consumes an iterator to exhaustion. > It is short, sweet, and hard to beat. islice() is still a tad faster. A possible optimization: static PyObject* consume_iterator(PyObject *it) { PyObject *item; PyObject *(*iternext)(PyObject *); iternext = *Py_TYPE(it)->tp_iternext; while ((item = iternext(it)) != NULL) { Py_DECREF(item); } Py_DECREF(it); if (PyErr_Occurred()) { if(PyErr_ExceptionMatches(PyExc_StopIteration)) PyErr_Clear(); else return NULL; } Py_RETURN_NONE; } Before: $ ./python -m timeit -s"from itertools import repeat, islice; from collections import deque; from sys import maxint" "next(islice(repeat(None, 1000), maxint, maxint), None)" 100000 loops, best of 3: 6.49 usec per loop $ ./python -m timeit -s"from itertools import repeat, islice; from collections import deque; from sys import maxint" "deque(repeat(None, 1000), maxlen=0)" 100000 loops, best of 3: 9.93 usec per loop After: $ ./python -m timeit -s"from itertools import repeat, islice; from collections import deque; from sys import maxint" "deque(repeat(None, 1000), maxlen=0)" 100000 loops, best of 3: 6.31 usec per loop Peter PS: Two more, for Paul Rubin: $ ./python -m timeit -s"from itertools import repeat" "sum(0 for _ in repeat(None, 1000))" 10000 loops, best of 3: 125 usec per loop $ ./python -m timeit -s"from itertools import repeat" "sum(0 for _ in repeat(None, 1000) if 0)" 10000 loops, best of 3: 68.3 usec per loop From andymac at bullseye.apana.org.au Sun Jan 24 18:43:50 2010 From: andymac at bullseye.apana.org.au (Andrew MacIntyre) Date: Mon, 25 Jan 2010 10:43:50 +1100 Subject: ctypes for AIX In-Reply-To: <9C240725D2ABD949A80B46DD93AB107042E8D44101@XCH-NW-08V.nw.nos.boeing.com> References: <9C240725D2ABD949A80B46DD93AB107042E8D44101@XCH-NW-08V.nw.nos.boeing.com> Message-ID: <4B5CDB36.4020904@bullseye.andymac.org> Waddle, Jim wrote: > Is there a policy concerning getting functions like ctypes working on AIX. If you can get it working, post a patch on the bug tracker. -- ------------------------------------------------------------------------- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac at bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac at pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From lists at cheimes.de Sun Jan 24 18:52:38 2010 From: lists at cheimes.de (Christian Heimes) Date: Mon, 25 Jan 2010 00:52:38 +0100 Subject: Broken Python 2.6 installation on Ubuntu Linux 8.04 In-Reply-To: References: Message-ID: <4B5CDD46.30303@cheimes.de> Benjamin Kaplan wrote: > Extensions written in C must be recompiled for every version of > Python. Since you're using a version of Python not available through > the package manager, your packages are also not available through > that. You'll have to download the sources for those and compile them > by hand to. This is why most people stick with the precompiled > binaries. As far as I remember you can use the Debian build system to create binaries for your Python version. You have to add Python 2.6 to /usr/share/python/debian_defaults and recompile the desired packages with "apt-get -b python-yourmodule". This should generate one to several deb files. Install them with "dpkg -i filename.deb". It's all documented in /usr/share/doc/python/python-policy.txt.gz, read the file with zless. By the way you mustn't install your own Python with "make install", use "make altinstall"! Your /usr/local/bin/python binary masks the original python command in /usr/bin. You should remove all /usr/local/bin/py* binaries that do not end with 2.6. Otherwise you may and will break existing programs on your system. Christian From mwilson at the-wire.com Sun Jan 24 19:03:08 2010 From: mwilson at the-wire.com (Mel) Date: Sun, 24 Jan 2010 19:03:08 -0500 Subject: ISO module for binomial coefficients, etc. References: Message-ID: Dave Angel wrote: > But I still think there must be code for the gamma function that would > be quicker. But I haven't chased that lead. Log of the gamma function is also an algorithm in its own right (e.g. published in _Numerical Recipes_.) Mel. From steven at REMOVE.THIS.cybersource.com.au Sun Jan 24 19:05:13 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 25 Jan 2010 00:05:13 GMT Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> <065e450f-2190-42f4-b9c6-4b95260092be@f12g2000yqn.googlegroups.com> <28103200.7M7XYOBnsK@beaureve.gmx.net> <9734672.qlcV6ekshW@beaureve.gmx.net> <6ddbbbfe-402d-4b87-8b61-a5042071dd08@e37g2000yqn.googlegroups.com> Message-ID: On Sun, 24 Jan 2010 10:11:07 -0800, George Sakkis wrote: > Both in your example and by using a context manager, you can get away > with not passing locals() explicitly by introspecting the stack frame. You say that as if it were less of an ugly hack than the locals() trick. But sys._getframe is a private implementation detail, so you're no better off. > So it's utility is pretty limited; a custom DSL is probably better > suited to your problem. Personally, I think the OP is worrying too much about namespace pollution. If you're concerned about the enums of one function clashing with the enums of another, that's a strong hint that they don't belong in the same module. -- Steven From ladasky at my-deja.com Sun Jan 24 19:13:16 2010 From: ladasky at my-deja.com (John Ladasky) Date: Sun, 24 Jan 2010 16:13:16 -0800 (PST) Subject: Broken Python 2.6 installation on Ubuntu Linux 8.04 References: Message-ID: <2c4702dd-1c66-46f4-ad40-44798a54abdc@21g2000yqj.googlegroups.com> Thanks, Benjamin, I am getting a handle on this. I've written my own Python modules before, and have installed them using distutils. So I know that procedure. I just downloaded the Numpy 1.4.0 tarball, and I succeeded in installing it. A program I wrote which depends on numpy ran successfully from SCIte. I'll move on to matplotlib and biopython next. I also succeeded in running both the Python 2.6 and the Python 2.5 interpreters from the terminal prompt as you suggested. Not sure whether I'll need to play with .bashrc yet. From george.sakkis at gmail.com Sun Jan 24 19:46:08 2010 From: george.sakkis at gmail.com (George Sakkis) Date: Sun, 24 Jan 2010 16:46:08 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> <065e450f-2190-42f4-b9c6-4b95260092be@f12g2000yqn.googlegroups.com> <28103200.7M7XYOBnsK@beaureve.gmx.net> <9734672.qlcV6ekshW@beaureve.gmx.net> <6ddbbbfe-402d-4b87-8b61-a5042071dd08@e37g2000yqn.googlegroups.com> Message-ID: <9aba7f45-6ed8-4397-ae63-71c1bfc23568@v25g2000yqk.googlegroups.com> On Jan 25, 1:05?am, Steven D'Aprano wrote: > On Sun, 24 Jan 2010 10:11:07 -0800, George Sakkis wrote: > > Both in your example and by using a context manager, you can get away > > with not passing locals() explicitly by introspecting the stack frame. > > You say that as if it were less of an ugly hack than the locals() trick. > But sys._getframe is a private implementation detail, so you're no better > off. The client *is* better off in that he doesn't have to pass locals() to some magic decorator to achieve his goal; how this is implemented is a different issue. As for being an ugly hack, you may use inspect.stack () if the underscore in sys._getframe() makes you feel dirty. George From robert.kern at gmail.com Sun Jan 24 20:26:08 2010 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 24 Jan 2010 19:26:08 -0600 Subject: medians for degree measurements In-Reply-To: <036acf91$0$1357$c3e8da3@news.astraweb.com> References: <044268c0-c308-406e-8d5b-a38ef5b5e381@p24g2000yqm.googlegroups.com> <036acf91$0$1357$c3e8da3@news.astraweb.com> Message-ID: On 2010-01-23 05:52 , Steven D'Aprano wrote: > On Fri, 22 Jan 2010 22:09:54 -0800, Steve Howell wrote: > >> On Jan 22, 5:12 pm, MRAB wrote: >>> Steve Howell wrote: >>>> I just saw the thread for medians, and it reminded me of a problem >>>> that I need to solve. We are writing some Python software for >>>> sailing, and we need to detect when we've departed from the median >>>> heading on the leg. Calculating arithmetic medians is >>>> straightforward, but compass bearings add a twist. > [...] >> I like this implementation, and it would probably work 99.9999% of the >> time for my particular use case. The only (very contrived) edge case >> that I can think of is when you have 10 bearings to SSW, 10 bearings to >> SSE, and the two outliers are unfortunately in the NE and NW quadrants. >> It seems like the algorithm above would pick one of the outliers. > > The trouble is that median of angular measurements is not a meaningful > concept. The median depends on the values being ordered, but angles can't > be sensibly ordered. Which is larger, 1 degree north or 359 degrees? Is > the midpoint between them 0 degree or 180 degree? Then don't define the median that way. Instead, define the median as the point that minimizes the sum of the absolute deviations of the data from that point (the L1 norm of the deviations, for those familiar with that terminology). For 1-D data on the real number line, that corresponds to sorting the data and taking the middle element (or the artithmetic mean of the middle two in the case of even-numbered data). My definition applies to other spaces, too, that don't have a total order attached to them including the space of angles. The "circular median" is a real, well-defined statistic that is used for exactly what the OP intends to use it for. > The median of the number line 0<= x<= 360 is 180, but what's the median > of the circle 0 deg<= theta<= 360 deg? With no end points, it's > meaningless to talk about the middle. > > For this reason, I expect that taking the median of bearing measurements > isn't well defined. You can probably get away with it so long as the > bearings are "close", where "close" itself is ill-defined. > > In any case, this isn't a programming problem, it's a mathematics > problem. I think you're better off taking it to a maths or statistics > forum, and see what they say. Same answer with my statistics hat on or off. :-) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From robert.kern at gmail.com Sun Jan 24 20:33:42 2010 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 24 Jan 2010 19:33:42 -0600 Subject: how to generate random numbers that satisfy certain distribution In-Reply-To: <036bab68$0$1357$c3e8da3@news.astraweb.com> References: <7xtyuc1s65.fsf@ruckus.brouhaha.com> <036b5f64$0$1357$c3e8da3@news.astraweb.com> <7xr5pgxykd.fsf@ruckus.brouhaha.com> <036bab68$0$1357$c3e8da3@news.astraweb.com> Message-ID: On 2010-01-23 21:30 , Steven D'Aprano wrote: > On Sat, 23 Jan 2010 14:10:10 -0800, Paul Rubin wrote: > >> Steven D'Aprano writes: >>> The Box-Muller transform is reasonably simple, but you can't be serious >>> that it is simpler than adding twelve random numbers and subtracting >>> six! >> >> If you want a normal distribution, using the Box-Muller transform is >> simpler because it spares you the complication of figuring out whether >> the 12-trial binomial approximation is close enough to produce reliable >> results for your specific application, which you obviously have to do if >> you are using the approximation for anything serious. > > But using Box-Miller gives you the complication of figuring out whether > you care about being thread safety, which you have to do if you're doing > anything serious. (See the comments in the random module for the gauss > method). If you care about thread-safety, each thread should get its own Random instance anyways. > By the way, does anyone know why there is no Poisson random numbers in > the module? The implementation is quite simple (but not as simple as the > Linux kernel *wink*): > > > def poisson(lambda_=1): > L = math.exp(-lambda_) > k = 0 > p = 1 > while 1: > k += 1 > p *= random.random() > if p<= L: > break > return k-1 > > > although this can be improved upon for large values of lambda_. Where large is about 10. That's where my implementation in numpy.random switches over to a more complicated, but more efficient implementation. It does require a log-gamma special function implementation, though. #define LS2PI 0.91893853320467267 #define TWELFTH 0.083333333333333333333333 long rk_poisson_ptrs(rk_state *state, double lam) { long k; double U, V, slam, loglam, a, b, invalpha, vr, us; slam = sqrt(lam); loglam = log(lam); b = 0.931 + 2.53*slam; a = -0.059 + 0.02483*b; invalpha = 1.1239 + 1.1328/(b-3.4); vr = 0.9277 - 3.6224/(b-2); while (1) { U = rk_double(state) - 0.5; V = rk_double(state); us = 0.5 - fabs(U); k = (long)floor((2*a/us + b)*U + lam + 0.43); if ((us >= 0.07) && (V <= vr)) { return k; } if ((k < 0) || ((us < 0.013) && (V > us))) { continue; } if ((log(V) + log(invalpha) - log(a/(us*us)+b)) <= (-lam + k*loglam - loggam(k+1))) { return k; } } } -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From tekion at gmail.com Sun Jan 24 22:12:02 2010 From: tekion at gmail.com (tekion) Date: Sun, 24 Jan 2010 19:12:02 -0800 (PST) Subject: easy_install error ... References: <5e9c94df-531f-4725-bcd5-87c603d8df1c@o26g2000vbd.googlegroups.com> Message-ID: <2d3c4b85-c0fe-4f65-94ea-d246acdd0345@d30g2000vbl.googlegroups.com> I am also having another issue with easy_install, python setup.py easy_install --find-links thirdparty 'docutils==0.4' running easy_install error: Not a URL, existing file, or requirement spec: "'docutils==0.4'" I think both problem are related. I have setuptool version .6C11 install and I am running python 2.5 on window OS. Is any one else having similar issue with setuptool version .6C11? Thanks. From ursusmaximus at gmail.com Sun Jan 24 22:18:12 2010 From: ursusmaximus at gmail.com (Ron) Date: Sun, 24 Jan 2010 19:18:12 -0800 (PST) Subject: Sikuli: the coolest Python project I have yet seen... Message-ID: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> Sikuli is the coolest Python project I have ever seen in my ten year hobbyist career. An MIT oepn source project, Sikuli uses Python to automate GUI tasks (in any GUI or GUI baed app that runs the JVM) by simply drag and dropping GUI elements into Python scripts as function arguments. Download at http://sikuli.csail.mit.edu/ I also did this podcast about Sikuli http://media.libsyn.com/media/awaretek/Python411_20100124_Sikuli.mp3 From showell30 at yahoo.com Sun Jan 24 23:12:11 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sun, 24 Jan 2010 20:12:11 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> Message-ID: <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> On Jan 24, 12:44?pm, Paul Rubin wrote: > Steve Howell writes: > > Proposal: Improve list's implementation so that deleting elements from > > the front of the list does not require an O(N) memmove operation. ... > > It is possible now, of course, to use a data structure in > > Python that has O(1) for deleting off the top of the list, but none of > > the alternatives fully replicate the benefits of list itself. > > I think you are mostly referring to deque. ?Why don't you instead > say what you think is wrong with using deque, and how deque can > be improved? > There is nothing wrong with deque, at least as far as I know, if the data strucure actually applies to your use case. It does not apply to my use case. > > See above. ?An implementation of this PEP would not change the > > definition of the language in any way, but it would have to minimally > > impact the performance of lists for the normal use cases. > > But you're talking about adding one or two words to EVERY list, and many > normal use cases allocate a LOT of lists. ?Those use cases are likely > more common than use cases that pop from the front of the list but for > some reason can't use deque. For EVERY list, you are not only allocating memory for the list itself, but you are also allocating memory for the objects within the list. So the extra one or two words are NEGLIGIBLE. > > The most ambitious proposal is to fix the memory manager itself to > > allow the release of memory from the start of the chunk. > > That's inappropriate given the memory fragmentation it would cause. > Bullshit. Memory managers consolidate free memory chunks all the time. That's their job. > Really, you're describing a problem that arises in a few programs but up > til now, as far as I know, everyone has found deque to be an adequate > solution. ? Wrong. Many programs delete the first element of the list. > Your approach of snarling against list is not persuading > anyone that list needs to be changed, because most everyone is satisfied > with the existing solution. ? Please provide evidence of that. I am pretty sure that everybody who chooses alternatives to Python would disagree. > You might change approaches and discuss > deque, what's wrong with it, and whether it can be fixed. ?Getting a > change approved for deque is probably much easier than getting one > approved for list, just because nowhere near as many things depend on > deque's performance. ? Again...I am not looking to improve deque, which is a perfectly valid data structure for a limited set of problems. > And when discussing performance in this contextc > additive constants do matter. Wrong again. Operations that mutate lists are already expensive, and a few checks to see if unreleased memory can be reclaimed are totally NEGLIGIBLE. From steven at REMOVE.THIS.cybersource.com.au Mon Jan 25 01:07:48 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 25 Jan 2010 06:07:48 GMT Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> Message-ID: On Sun, 24 Jan 2010 20:12:11 -0800, Steve Howell wrote: >> > The most ambitious proposal is to fix the memory manager itself to >> > allow the release of memory from the start of the chunk. >> >> That's inappropriate given the memory fragmentation it would cause. >> >> > Bullshit. Memory managers consolidate free memory chunks all the time. > That's their job. So let me get this straight... You've complained that Python's list.pop(0) is lame because it moves memory around. And your solution to that is to have the memory manager move the memory around instead? Perhaps I'm missing something, but I don't see the advantage here. At best, you consolidate all those moves you wanted to avoid and do them all at once instead of a few at a time. At worst, you get a situation where the application periodically, and unpredictably, grinds to a halt while the memory manager tries to defrag all those lists. >> Really, you're describing a problem that arises in a few programs but >> up til now, as far as I know, everyone has found deque to be an >> adequate solution. > > Wrong. Many programs delete the first element of the list. I'm sure they do. Many programs do all sorts of things, of varying sensibleness. But I'm pretty sure I've never written a program that deleted the first element of a list. Even if I have, it's a vanishingly small use-case for me. YMMV. >> Your approach of snarling against list is not persuading anyone that >> list needs to be changed, because most everyone is satisfied with the >> existing solution. > > Please provide evidence of that. I am pretty sure that everybody who > chooses alternatives to Python would disagree. Do you honestly believe that "everybody" who prefers another language over Python does so because they dislike the performance of list.pop(0)? >> You might change approaches and discuss deque, what's wrong with it, >> and whether it can be fixed. ?Getting a change approved for deque is >> probably much easier than getting one approved for list, just because >> nowhere near as many things depend on deque's performance. > > Again...I am not looking to improve deque, which is a perfectly valid > data structure for a limited set of problems. > >> And when discussing performance in this contextc additive constants do >> matter. > > Wrong again. Operations that mutate lists are already expensive, and a > few checks to see if unreleased memory can be reclaimed are totally > NEGLIGIBLE. Popping from the end of the list isn't expensive. Reversing lists is relatively cheap. In-place modifications are very cheap. -- Steven From arnodel at googlemail.com Mon Jan 25 01:57:42 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 25 Jan 2010 06:57:42 +0000 Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: Arnaud Delobelle writes: > Gilles Ganault writes: > >> On Fri, 22 Jan 2010 13:17:44 +0100, Gilles Ganault >> wrote: >>>To avoid users from creating login names that start with digits in >>>order to be listed at the top, I'd like to sort the list differently >>>every minute so that it'll start with the next letter, eg. display the >>>list from A...Zdigits the first time, then B...ZAdigits, etc. >> >> Thanks everyone for the great feedback. I ended up using >> - a list instead of a dictionary to hold the list of names >> - use the random library to pick a character from which to start >> listing names (simpler than saving the next letter into a file, and >> starting from this the next time the loop is gone through) >> >> For those interested, here's some working code: >> >> ========== >> import random >> >> connected = >> ["_test","-test","0test","1test","Aa","Aab","Bb","Bbbbb","Cc","Ccaaaaaa"] >> connected.sort() >> >> #Fill list with non-duplicate first letter of all items >> characters=[] >> for name in connected: >> char = name[0] >> #if this character not in list, add to array >> if not char in characters: >> characters.append(char) > > characters = list(set(name[0] for name in connected)) > >> #Pick a random character from which to start iterating >> index = random.randint(0,len(characters)-1) >> startch = characters[index] > > startch = random.choice(characters) > >> print "Will start from %s" % startch >> >> #Go through list starting with items that start with 'starch', and >> rotate from beginning >> result = [name for name in connected if name[0] >= startch] + [name >> for name in connected if name[0] < startch] I forgot this line: result = sorted(connected, key=lambda name: name[0] < startch) This takes advantage of the fact that timsort is stable. So you could do it like this, assuming connected is sorted: characters = list(set(name[0] for name in connected)) startch = random.choice(characters) result = sorted(connected, key=lambda name: name[0] < startch) If connected is not sorted, you could replace the last line with: result = sorted(connected, key=lambda name: (name[0] < startch, name)) -- Arnaud From no.email at nospam.invalid Mon Jan 25 02:24:30 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 24 Jan 2010 23:24:30 -0800 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> Message-ID: <7xpr4ylk9d.fsf@ruckus.brouhaha.com> Steve Howell writes: > There is nothing wrong with deque, at least as far as I know, if the > data strucure actually applies to your use case. It does not apply to > my use case. You haven't explained why deque doesn't apply to your use case. Until a convincing explanation emerges, the sentiment you're creating seems to be "what's wrong with that guy and why doesn't he just use deque?". So, why aren't you using deque? If deque somehow isn't adequate for your use case, maybe it can be improved. >> Your approach of snarling against list is not persuading >> anyone that list needs to be changed, because most everyone is satisfied >> with the existing solution. ? > > Please provide evidence of that. I am pretty sure that everybody who > chooses alternatives to Python would disagree. I've heard of many reasons to choose alternatives to Python, and have chosen alternatives to Python in various situations myself. The list.pop(0) issue has never been one of those reasons for me or anyone else I know of to choose an alternative until you came along. Anyway, you're welcome to switch to another language; nobody's heart will be broken if you do that. I'd be interested to know which languages handle list.pop(0) the way you're proposing for Python. >> And when discussing performance in this context, additive constants >> do matter. > Wrong again. Operations that mutate lists are already expensive I'm talking about memory consumption, which is part of Python's concept of performance. You're proposing adding a word or two to every list, with insufficient justification presented so far. Any such justification would have to include a clear and detailed explanation of why using deque is insufficient, so that would be a good place to start. On another note: the idea you're suggesting, while not yet 100% convincing, is not crazy, which is why people are willing to discuss it with you reasonably. But your confrontational style is making discussion unpleasant. Can you dial it back a little? Your current approach is perhaps leading you towards people's ignore lists. From tim.yellow at gmail.com Mon Jan 25 02:29:25 2010 From: tim.yellow at gmail.com (tim) Date: Sun, 24 Jan 2010 23:29:25 -0800 (PST) Subject: Sikuli: the coolest Python project I have yet seen... References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> Message-ID: <3abeeada-28fb-4e04-8a8b-0b814918d7e0@33g2000yqo.googlegroups.com> On Jan 25, 11:18?am, Ron wrote: > Sikuli is the coolest Python project I have ever seen in my ten year > hobbyist career. An MIT oepn source project, Sikuli uses Python to > automate GUI tasks (in any GUI or GUI baed app that runs the JVM) by > simply drag and dropping GUI elements into Python scripts as function > arguments. Download athttp://sikuli.csail.mit.edu/I also did this > podcast about Sikulihttp://media.libsyn.com/media/awaretek/Python411_20100124_Sikuli.mp3 Wow , It look likes like better than autoIt !! Very impressed. From siddhartha.veedaluru at gmail.com Mon Jan 25 03:35:48 2010 From: siddhartha.veedaluru at gmail.com (siddu) Date: Mon, 25 Jan 2010 00:35:48 -0800 (PST) Subject: TypeError not caught by except statement Message-ID: <73caff4c-243e-4ee6-aa6a-6b14f46a6706@a6g2000yqm.googlegroups.com> Hi, except not able to caught the TypeError exception occured in the below code log.info("refer",ret) in the try block throws a TypeError which is not caught . Also sometimes process is getting hanged. ------------------------------------------------------------------------------------------------------------------------------------------------ import logging log = logging.getLogger() fileName = strftime("%d-%b-%Y-", gmtime()) + str(int(time.time())) + "- Log.log" log = logging.getLogger() log.setLevel(logging.NOTSET) fh = logging.FileHandler(logFile) logFileLevel = logging.DEBUG fh.setLevel(logFileLevel) format_string = '%(process)d %(thread)d %(asctime)-15s %(levelname)-5s at %(filename)-15s in %(funcName)-10s at line %(lineno)-3d "%(message) s"' fh.setFormatter(logging.Formatter(format_string)) log.addHandler(fh) try: log.info("start") log.info("refer",ret) log.info("end") except TypeError: log.exception("Exception raised") ---------------------------------------------------------------------------------------------------------------------------------------------- OUTPUT message: Traceback (most recent call last): File "C:\Python26\lib\logging\__init__.py", line 768, in emit msg = self.format(record) File "C:\Python26\lib\logging\__init__.py", line 648, in format return fmt.format(record) File "C:\Python26\lib\logging\__init__.py", line 436, in format record.message = record.getMessage() File "C:\Python26\lib\logging\__init__.py", line 306, in getMessage msg = msg % self.args TypeError: not all arguments converted during string formatting From alfps at start.no Mon Jan 25 03:55:45 2010 From: alfps at start.no (Alf P. Steinbach) Date: Mon, 25 Jan 2010 09:55:45 +0100 Subject: TypeError not caught by except statement In-Reply-To: <73caff4c-243e-4ee6-aa6a-6b14f46a6706@a6g2000yqm.googlegroups.com> References: <73caff4c-243e-4ee6-aa6a-6b14f46a6706@a6g2000yqm.googlegroups.com> Message-ID: * siddu: > Hi, > > except not able to caught the TypeError exception occured in the below > code > > log.info("refer",ret) in the try block > > throws a TypeError which is not caught . > Also sometimes process is getting hanged. > > ------------------------------------------------------------------------------------------------------------------------------------------------ > import logging > log = logging.getLogger() > fileName = strftime("%d-%b-%Y-", gmtime()) + str(int(time.time())) + "- > Log.log" > log = logging.getLogger() > log.setLevel(logging.NOTSET) > fh = logging.FileHandler(logFile) Where does 'logFile' come from. > logFileLevel = logging.DEBUG > fh.setLevel(logFileLevel) > format_string = '%(process)d %(thread)d %(asctime)-15s %(levelname)-5s > at %(filename)-15s in %(funcName)-10s at line %(lineno)-3d "%(message) > s"' > fh.setFormatter(logging.Formatter(format_string)) > log.addHandler(fh) > > try: > log.info("start") > log.info("refer",ret) > log.info("end") > except TypeError: > log.exception("Exception raised") Try to reduce the example to a small, complete program that exhibits the problem. > > ---------------------------------------------------------------------------------------------------------------------------------------------- > OUTPUT message: > > Traceback (most recent call last): > File "C:\Python26\lib\logging\__init__.py", line 768, in emit > msg = self.format(record) > File "C:\Python26\lib\logging\__init__.py", line 648, in format > return fmt.format(record) > File "C:\Python26\lib\logging\__init__.py", line 436, in format > record.message = record.getMessage() > File "C:\Python26\lib\logging\__init__.py", line 306, in getMessage > msg = msg % self.args > TypeError: not all arguments converted during string formatting Is this a complete listing? CHheers & hth., - Alf From frank at chagford.com Mon Jan 25 03:59:01 2010 From: frank at chagford.com (Frank Millman) Date: Mon, 25 Jan 2010 10:59:01 +0200 Subject: Is defaultdict thread safe? Message-ID: Hi all Is defaultdict thread safe? Assume I have - from collections import defaultdict my_dict = defaultdict(list) If two threads call "my_dict['abc'].append(...)" simultaneously, is it guaranteed that my_dict['abc'] will end up containing two elements? Thanks Frank Millman From pavlovevidence at gmail.com Mon Jan 25 04:19:57 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Mon, 25 Jan 2010 01:19:57 -0800 (PST) Subject: Sikuli: the coolest Python project I have yet seen... References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> Message-ID: On Jan 24, 7:18?pm, Ron wrote: > Sikuli is the coolest Python project I have ever seen in my ten year > hobbyist career. An MIT oepn source project, Sikuli uses Python to > automate GUI tasks (in any GUI or GUI baed app that runs the JVM) by > simply drag and dropping GUI elements into Python scripts as function > arguments. Download athttp://sikuli.csail.mit.edu/I also did this > podcast about Sikulihttp://media.libsyn.com/media/awaretek/Python411_20100124_Sikuli.mp3 Nice, thanks for the link. Very happy to see people using Python for cool stuff like this, also to see the alternate implementations in use. And it couldn't come at a better time for me as I am trying to figure out how to automate some GUI-only program I am forced to use at work. Carl Banks From python at rcn.com Mon Jan 25 04:26:52 2010 From: python at rcn.com (Raymond Hettinger) Date: Mon, 25 Jan 2010 01:26:52 -0800 (PST) Subject: Is defaultdict thread safe? References: Message-ID: <502c108e-7bed-46b6-a24b-e25b7b92773f@e25g2000yqh.googlegroups.com> On Jan 25, 12:59?am, "Frank Millman" wrote: > Hi all > > Is defaultdict thread safe? Sometimes. It depends on whether an operation has callbacks to pure Python. > Assume I have - > > ? ? from collections import defaultdict > ? ? my_dict = defaultdict(list) > > If two threads call "my_dict['abc'].append(...)" simultaneously, is it > guaranteed that my_dict['abc'] will end up containing two elements? Yes. But, if the constructor is a user defined class, the pure python code runs for the instantiation and all bets are off. class A: def __init__(self): . . . my_dict = defaultdict(A) # not thread-safe. Raymond From olof.bjarnason at gmail.com Mon Jan 25 04:40:31 2010 From: olof.bjarnason at gmail.com (Olof Bjarnason) Date: Mon, 25 Jan 2010 10:40:31 +0100 Subject: Sikuli: the coolest Python project I have yet seen... In-Reply-To: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> Message-ID: 2010/1/25 Ron : > Sikuli is the coolest Python project I have ever seen in my ten year > hobbyist career. An MIT oepn source project, Sikuli uses Python to > automate GUI tasks (in any GUI or GUI baed app that runs the JVM) by > simply drag and dropping GUI elements into Python scripts as function > arguments. Download at http://sikuli.csail.mit.edu/ I also did this > podcast about Sikuli http://media.libsyn.com/media/awaretek/Python411_20100124_Sikuli.mp3 > -- > http://mail.python.org/mailman/listinfo/python-list > It looks really nice, but the screenhost-taking did not work on my computer (Win7). Innovative yet simple idea this mix-visual-and-code. -- http://olofb.wordpress.com From frank at chagford.com Mon Jan 25 04:45:02 2010 From: frank at chagford.com (Frank Millman) Date: Mon, 25 Jan 2010 11:45:02 +0200 Subject: Is defaultdict thread safe? References: <502c108e-7bed-46b6-a24b-e25b7b92773f@e25g2000yqh.googlegroups.com> Message-ID: On Jan 25, 11:26 am, Raymond Hettinger wrote: > On Jan 25, 12:59 am, "Frank Millman" wrote: > > > Hi all > > > Is defaultdict thread safe? > > Sometimes. It depends on whether an operation has callbacks to pure > Python. > > > Assume I have - > > > from collections import defaultdict > > my_dict = defaultdict(list) > > > If two threads call "my_dict['abc'].append(...)" simultaneously, is it > > guaranteed that my_dict['abc'] will end up containing two elements? > > Yes. > > But, if the constructor is a user defined class, the pure python code > runs for the instantiation and all bets are off. > > class A: > def __init__(self): > . . . > my_dict = defaultdict(A) # not thread-safe. > > Raymond Many thanks Frank From technic.tec at gmail.com Mon Jan 25 04:56:56 2010 From: technic.tec at gmail.com (tec) Date: Mon, 25 Jan 2010 17:56:56 +0800 Subject: TypeError not caught by except statement In-Reply-To: <73caff4c-243e-4ee6-aa6a-6b14f46a6706@a6g2000yqm.googlegroups.com> References: <73caff4c-243e-4ee6-aa6a-6b14f46a6706@a6g2000yqm.googlegroups.com> Message-ID: On 2010-1-25 16:35, siddu wrote: > > Hi, > > except not able to caught the TypeError exception occured in the below > code > > log.info("refer",ret) in the try block > > throws a TypeError which is not caught . > Also sometimes process is getting hanged. > > ------------------------------------------------------------------------------------------------------------------------------------------------ > import logging > log = logging.getLogger() > fileName = strftime("%d-%b-%Y-", gmtime()) + str(int(time.time())) + "- > Log.log" > log = logging.getLogger() > log.setLevel(logging.NOTSET) > fh = logging.FileHandler(logFile) > logFileLevel = logging.DEBUG > fh.setLevel(logFileLevel) > format_string = '%(process)d %(thread)d %(asctime)-15s %(levelname)-5s > at %(filename)-15s in %(funcName)-10s at line %(lineno)-3d "%(message) > s"' > fh.setFormatter(logging.Formatter(format_string)) > log.addHandler(fh) > > try: > log.info("start") > log.info("refer",ret) ~~~~~~~~~~~~~~~~~~~~~~~ Seems this line causes the exception, and it is handled inside log.info(), which prints those traceback info. Usually log.info(msg, args) raises the same exception as print(msg%args). > log.info("end") > except TypeError: > log.exception("Exception raised") > > ---------------------------------------------------------------------------------------------------------------------------------------------- > OUTPUT message: > > Traceback (most recent call last): > File "C:\Python26\lib\logging\__init__.py", line 768, in emit > msg = self.format(record) > File "C:\Python26\lib\logging\__init__.py", line 648, in format > return fmt.format(record) > File "C:\Python26\lib\logging\__init__.py", line 436, in format > record.message = record.getMessage() > File "C:\Python26\lib\logging\__init__.py", line 306, in getMessage > msg = msg % self.args > TypeError: not all arguments converted during string formatting From pavlovevidence at gmail.com Mon Jan 25 05:03:49 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Mon, 25 Jan 2010 02:03:49 -0800 (PST) Subject: TypeError not caught by except statement References: <73caff4c-243e-4ee6-aa6a-6b14f46a6706@a6g2000yqm.googlegroups.com> Message-ID: <8100e8d0-9c30-4f74-9af5-e65c187fe415@a5g2000yqi.googlegroups.com> On Jan 25, 12:35?am, siddu wrote: > Hi, > > except not able to caught the TypeError exception occured in the below > code > > ? ? ? ? log.info("refer",ret) in the try block > > throws a TypeError which is not caught . > Also sometimes process is getting hanged. > > ------------------------------------------------------------------------------------------------------------------------------------------------ > import logging > log = logging.getLogger() > fileName = strftime("%d-%b-%Y-", gmtime()) + str(int(time.time())) + "- > Log.log" > log = logging.getLogger() > log.setLevel(logging.NOTSET) > fh = logging.FileHandler(logFile) > logFileLevel = logging.DEBUG > fh.setLevel(logFileLevel) > format_string = '%(process)d %(thread)d %(asctime)-15s %(levelname)-5s > at %(filename)-15s in %(funcName)-10s at line %(lineno)-3d "%(message) > s"' > fh.setFormatter(logging.Formatter(format_string)) > log.addHandler(fh) > > try: > ? ? log.info("start") > ? ? log.info("refer",ret) > ? ? log.info("end") > except TypeError: > ? ? log.exception("Exception raised") > > ---------------------------------------------------------------------------------------------------------------------------------------------- > OUTPUT message: > > Traceback (most recent call last): > ? File "C:\Python26\lib\logging\__init__.py", line 768, in emit > ? ? msg = self.format(record) > ? File "C:\Python26\lib\logging\__init__.py", line 648, in format > ? ? return fmt.format(record) > ? File "C:\Python26\lib\logging\__init__.py", line 436, in format > ? ? record.message = record.getMessage() > ? File "C:\Python26\lib\logging\__init__.py", line 306, in getMessage > ? ? msg = msg % self.args > TypeError: not all arguments converted during string formatting The logging module is swallowing the exception (and printing a traceback), I guess because logging is considered something that shouldn't bring your program down on error. You can apparently define a logging handler that overrides "handleError" to propogate the exception if you want. Can't tell you why it's hanging, but the logging error you're getting is probably because your string formatter is trying to perform the following operation: "refer" % (ret,) Carl Banks From carlisdjfriends at yahoo.com Mon Jan 25 05:18:29 2010 From: carlisdjfriends at yahoo.com (carlisdjfriends) Date: Mon, 25 Jan 2010 10:18:29 -0000 Subject: [Private Photo Share] Cali Girl- Has sent you private photos. Message-ID: I do not want the entire group seeing these photos.Because some may recognize me. Here's the link: http://www.ourlivespace.com/hotgirl/photos.htm Enjoy babe :) From stefan_ml at behnel.de Mon Jan 25 05:55:19 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 25 Jan 2010 11:55:19 +0100 Subject: Is defaultdict thread safe? In-Reply-To: References: Message-ID: <4b5d7897$0$6581$9b4e6d93@newsspool3.arcor-online.net> Frank Millman, 25.01.2010 09:59: > Is defaultdict thread safe? > > Assume I have - > > from collections import defaultdict > my_dict = defaultdict(list) > > If two threads call "my_dict['abc'].append(...)" simultaneously, is it > guaranteed that my_dict['abc'] will end up containing two elements? Thread-safety is implementation specific. Other runtime environments than CPython (e.g. Jython, IronPython, PyPy) may or may not provide any guarantees on thread safety here. Stefan From richard.lamboj at bilcom.at Mon Jan 25 05:59:13 2010 From: richard.lamboj at bilcom.at (Richard Lamboj) Date: Mon, 25 Jan 2010 11:59:13 +0100 Subject: reading from pipe Message-ID: <201001251159.13237.richard.lamboj@bilcom.at> Hello, is there any solution to catch if a pipe has closed? Maybe the signal modul? For Simulation: #!/usr/bin/env python # -*- coding: utf-8 -*- import sys while True: line = sys.stdin.readline() sys.stdout.write(line) sys.stdout.flush() time cat /tmp/proxy.test | test.py Solution: if line == "": break Kind Regards, Richi From lutz.horn at fastmail.fm Mon Jan 25 06:06:18 2010 From: lutz.horn at fastmail.fm (Lutz Horn) Date: Mon, 25 Jan 2010 12:06:18 +0100 Subject: reading from pipe In-Reply-To: References: Message-ID: Hi, Richard Lamboj schrieb: > is there any solution to catch if a pipe has closed? Maybe the signal modul? Since sys.stdin is a file object, you can use sys.stdin.closed to check if it has been closed. Lutz From ladasky at my-deja.com Mon Jan 25 06:18:10 2010 From: ladasky at my-deja.com (ladasky at my-deja.com) Date: Mon, 25 Jan 2010 03:18:10 -0800 (PST) Subject: WxPython upgrade trouble on Ubuntu 8.04 Message-ID: Hello everyone, You know the old saying, "in for a penny, in for a pound." Several hours ago I posted this... http://groups.google.com/group/comp.lang.python/msg/0a86e792c674adc8 ...in which I described my desire to acquire Python 2.6 without upgrading my Ubuntu Linux installation from 8.04. Since Python 2.6 is not part of the Ubuntu 8.04 repository, I installed Python 2.6 manually. Things are a little messy right now, since invoking IDLE from the desktop still defaults to Python 2.5. But I'm getting there. At least I can access Python 2.6 from the command prompt, and SCIte also finds 2.6. Of course, I also have to reinstall all of the Python modules that I use on 2.6. I have succeeded with numpy and biopython, both installed manually. WxPython is giving me trouble, though. I visited wxpython.org and noted that wx version 2.8.10.1 fixes some important bugs that can occur when using Python 2.6. The instructions here... http://wiki.wxpython.org/InstallingOnUbuntuOrDebian ...allowed me to add the wxWidgets apt repository to the list of web sites that my Synaptic Package Manager checks for updates. It found wx 2.8.10.1 and auto-installed it. I got error messages from Python 2.5 until I also installed python-wxtools, wx2.8-i18n, libwxgtk2.8- dev, and libgtk2.0-dev as recommended on the web page mentioned above. Now "import wx" worked again, with no error messages, from Python 2.5. Installing wx on Python 2.6 is my ultimate goal, and here's where I'm stuck. I actually tried "import wx" from Python 2.6 first, and I got this result: >>> import wx Traceback (most recent call last): File "", line 1, in File "wx/__init__.py", line 45, in from wx._core import * File "wx/_core.py", line 4, in import _core_ ImportError: No module named _core_ I went back to Synaptic, said that I wanted to reinstall wx 2.8.10.1 -- but not really, as I then selected "download package files only." I received the tarball. The top-level folder has the wx binary stuff, which I believe is already installed. I selected the wxPython directory at the second level. Tried "python setup.py build", and after several minutes and dozens of warning messages, I finally got this fatal error: In file included from /usr/include/wx-2.8/wx/glcanvas.h:54, from contrib/glcanvas/gtk/glcanvas_wrap.cpp:2661: /usr/include/wx-2.8/wx/gtk/glcanvas.h: At global scope: /usr/include/wx-2.8/wx/gtk/glcanvas.h:47: error: ?GLXContext? does not name a type /usr/include/wx-2.8/wx/gtk/glcanvas.h:124: error: ISO C++ forbids declaration of ?GLXFBConfig? with no type /usr/include/wx-2.8/wx/gtk/glcanvas.h:124: error: expected ?;? before ?*? token error: command 'gcc' failed with exit status 1 Worse -- NOW if I start Python 2.5, I get the same error message I got from 2.6. Now I'm stuck again. Any advice? Many thanks! From ladasky at my-deja.com Mon Jan 25 06:21:42 2010 From: ladasky at my-deja.com (John Ladasky) Date: Mon, 25 Jan 2010 03:21:42 -0800 (PST) Subject: Broken Python 2.6 installation on Ubuntu Linux 8.04 References: Message-ID: <79c8e373-e48d-42c3-ad97-08bee0f192d6@b2g2000yqi.googlegroups.com> On Jan 24, 3:52?pm, Christian Heimes wrote: > By the way you mustn't install your own Python with "make install", use > "make altinstall"! Your /usr/local/bin/python binary masks the original > python command in /usr/bin. You should remove all /usr/local/bin/py* > binaries that do not end with 2.6. Otherwise you may and will break > existing programs on your system. > > Christian Hello Christian, In my earlier response to Benjamin, I thought I was going to solve this problem quickly. Maybe not! I know for a fact that my Linux printer management program, HPLIP toolbox, uses wxPython. And now HPLIP won't start! However, my usr/local/bin ONLY contains references to Python 2.6. So I think this is a problem with me installing wx... see this other post... http://groups.google.com/group/comp.soft-sys.wxwindows/msg/f33e245eb0956067 Sigh. All this, just so I could use some itertools functions. From solipsis at pitrou.net Mon Jan 25 06:29:09 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 25 Jan 2010 11:29:09 +0000 (UTC) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> Message-ID: Le Sun, 24 Jan 2010 11:28:53 -0800, Aahz a ?crit?: > > Again, your responsibility is to provide a patch and a spectrum of > benchmarking tests to prove it. Then you would still have to deal with > the objection that extensions use the list internals -- that might be an > okay sell given the effort otherwise required to port extensions to > Python 3, but that's not the way to bet. IMO, code accessing the list internals should be considered broken. The macros (PyList_GET_ITEM, etc.) are there for a reason. We can't just freeze every internal characteristic of the interpreter just because someone might be messing around with it in unrecommended ways. From modpoc at gmail.com Mon Jan 25 07:27:13 2010 From: modpoc at gmail.com (mbpoc mbpoc) Date: Mon, 25 Jan 2010 17:57:13 +0530 Subject: modbus Message-ID: <38e2a7731001250427g4e6139a2sdd210242e1248a23@mail.gmail.com> Hi all, I started using pymodbus. I am trying to gain pointers as to how the communcation between devices be achieved through modbus. Suggestions on Simulators/ master-slave codes would be of gr8 help Regards -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeanmichel at sequans.com Mon Jan 25 07:27:37 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 25 Jan 2010 13:27:37 +0100 Subject: Sikuli: the coolest Python project I have yet seen... In-Reply-To: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> Message-ID: <4B5D8E39.1040600@sequans.com> Ron wrote: > Sikuli is the coolest Python project I have ever seen in my ten year > hobbyist career. An MIT oepn source project, Sikuli uses Python to > automate GUI tasks (in any GUI or GUI baed app that runs the JVM) by > simply drag and dropping GUI elements into Python scripts as function > arguments. Download at http://sikuli.csail.mit.edu/ I also did this > podcast about Sikuli http://media.libsyn.com/media/awaretek/Python411_20100124_Sikuli.mp3 > It looks like your web site is down. JM From ian at excess.org Mon Jan 25 07:53:05 2010 From: ian at excess.org (Ian Ward) Date: Mon, 25 Jan 2010 07:53:05 -0500 Subject: ANN: Urwid 0.9.9.1 - Console UI Library Message-ID: <4B5D9431.9060109@excess.org> Announcing Urwid 0.9.9.1 ------------------------ Urwid home page: http://excess.org/urwid/ Screen shots: http://excess.org/urwid/examples.html Tarball: http://excess.org/urwid/urwid-0.9.9.1.tar.gz About this release: =================== This maintenance release fixes a number of bugs including a backwards incompatibility introduced in the last release and a poor ListBox snapping behaviour. New in this release: ==================== * Fix for ListBox snapping to selectable widgets taller than the ListBox itself * raw_display switching to alternate buffer now works properly with Terminal.app * Fix for BoxAdapter backwards incompatibility introduced in 0.9.9 * Fix for a doctest failure under powerpc * Fix for systems with gpm_mev installed but not running gpm About Urwid =========== Urwid is a console UI library for Python. It features fluid interface resizing, UTF-8 support, multiple text layouts, simple attribute markup, powerful scrolling list boxes and flexible interface design. Urwid is released under the GNU LGPL. From robin at reportlab.com Mon Jan 25 08:34:32 2010 From: robin at reportlab.com (Robin Becker) Date: Mon, 25 Jan 2010 13:34:32 +0000 Subject: Sikuli: the coolest Python project I have yet seen... In-Reply-To: <4B5D8E39.1040600@sequans.com> References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> <4B5D8E39.1040600@sequans.com> Message-ID: <4B5D9DE8.7040303@chamonix.reportlab.co.uk> On 25/01/2010 12:27, Jean-Michel Pichavant wrote: > Ron wrote: >> Sikuli is the coolest Python project I have ever seen in my ten year >> hobbyist career. An MIT oepn source project, Sikuli uses Python to >> automate GUI tasks (in any GUI or GUI baed app that runs the JVM) by >> simply drag and dropping GUI elements into Python scripts as function >> arguments. Download at http://sikuli.csail.mit.edu/ I also did this >> podcast about Sikuli >> http://media.libsyn.com/media/awaretek/Python411_20100124_Sikuli.mp3 > It looks like your web site is down. > > JM there's a u-tube here http://www.youtube.com/watch?v=FxDOlhysFcM -- Robin Becker From vs at it.uu.se Mon Jan 25 08:39:21 2010 From: vs at it.uu.se (Virgil Stokes) Date: Mon, 25 Jan 2010 14:39:21 +0100 Subject: Sikuli: the coolest Python project I have yet seen... In-Reply-To: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> Message-ID: <4B5D9F09.7050607@it.uu.se> On 25-Jan-2010 04:18, Ron wrote: > Sikuli is the coolest Python project I have ever seen in my ten year > hobbyist career. An MIT oepn source project, Sikuli uses Python to > automate GUI tasks (in any GUI or GUI baed app that runs the JVM) by > simply drag and dropping GUI elements into Python scripts as function > arguments. Download at http://sikuli.csail.mit.edu/ I also did this > This link is broken! --V From simon at brunningonline.net Mon Jan 25 08:57:00 2010 From: simon at brunningonline.net (Simon Brunning) Date: Mon, 25 Jan 2010 13:57:00 +0000 Subject: Is python not good enough? In-Reply-To: References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> Message-ID: <8c7f10c61001250557u6b37f284x202574f20becd326@mail.gmail.com> 2010/1/25 Albert van der Horst : > If Go was to compete with anything, they would have give it a name > that was Googleable. ;-) If they want it Googleable, it will be. ;-) -- Cheers, Simon B. From groups_ads12 at yahoo.com Mon Jan 25 09:10:05 2010 From: groups_ads12 at yahoo.com (groups_ads12) Date: Mon, 25 Jan 2010 14:10:05 -0000 Subject: www.visualstudio2010.learn.net.in Message-ID: www.visualstudio2010.learn.net.in visual studio 2010 visual studio 2010 download visual studio 2010 release date microsoft visual studio 2010 visual studio 2010 release visual studio 2010 screenshots visual studio 2010 features visual studio 2010 wiki visual studio 2010 blog visual studio 2010 rapidshare visual studio 2010 review visual studio 2010 screenshot visual studio 2010 video microsoft visual studio 2010 download visual studio 2010 iso visual studio 2010 new features visual studio 2010 preview what's new in visual studio 2010 visual studio 2010 mfc visual studio 2010 msdn visual studio 2010 requirements visual studio 2010 wikipedia visual studio 2010 ctp2 visual studio 2010 news visual studio 2010 password visual studio 2010 professional visual studio 2010 sdk visual studio 2010 what's new visual studio 2010 downloads visual studio 2010 expired visual studio 2010 launch visual studio 2010 trial visual studio 2010 videos whats new in visual studio 2010 new features in visual studio 2010 new in visual studio 2010 visual studio 2010 c# visual studio 2010 demo visual studio 2010 extensibility visual studio 2010 ide visual studio 2010 intellisense visual studio 2010 launch date visual studio 2010 multi monitor visual studio 2010 price visual studio 2010 screen shots visual studio 2010 slow visual studio 2010 tutorial features of visual studio 2010 release date for visual studio 2010 visual studio 2010 activation visual studio 2010 forum visual studio 2010 parallel buy visual studio 2010 date visual studio 2010 how to install visual studio 2010 install visual studio 2010 installing visual studio 2010 release date of visual studio 2010 visual studio 2010 .net visual studio 2010 64 visual studio 2010 architecture visual studio 2010 asp net visual studio 2010 asp net mvc visual studio 2010 changes visual studio 2010 expiration visual studio 2010 expire visual studio 2010 extension visual studio 2010 feedback visual studio 2010 free visual studio 2010 installation visual studio 2010 jquery visual studio 2010 mef visual studio 2010 performance visual studio 2010 released visual studio 2010 screen visual studio 2010 ship date visual studio 2010 timeline visual studio 2010 training visual studio 2010 upgrade visual studio 2010 virtual visual studio 2010 web visual studio 2010 windows xp welcome to visual studio 2010 when visual studio 2010 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeanmichel at sequans.com Mon Jan 25 09:12:11 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 25 Jan 2010 15:12:11 +0100 Subject: ANN: Urwid 0.9.9.1 - Console UI Library In-Reply-To: <4B5D9431.9060109@excess.org> References: <4B5D9431.9060109@excess.org> Message-ID: <4B5DA6BB.8020106@sequans.com> Ian Ward wrote: > Announcing Urwid 0.9.9.1 > ------------------------ > > Urwid home page: > http://excess.org/urwid/ > > Screen shots: > http://excess.org/urwid/examples.html > > Tarball: > http://excess.org/urwid/urwid-0.9.9.1.tar.gz > > > About this release: > =================== > > This maintenance release fixes a number of bugs including a backwards > incompatibility introduced in the last release and a poor ListBox > snapping behaviour. > > > New in this release: > ==================== > > * Fix for ListBox snapping to selectable widgets taller than the > ListBox itself > > * raw_display switching to alternate buffer now works properly with > Terminal.app > > * Fix for BoxAdapter backwards incompatibility introduced in 0.9.9 > > * Fix for a doctest failure under powerpc > > * Fix for systems with gpm_mev installed but not running gpm > > > About Urwid > =========== > > Urwid is a console UI library for Python. It features fluid interface > resizing, UTF-8 support, multiple text layouts, simple attribute markup, > powerful scrolling list boxes and flexible interface design. > > Urwid is released under the GNU LGPL. > > > I tried to use urwid to provide some friendly interface to some of our command line tools. It's pretty much effective, but I had difficulties redirecting stdout/stdin to one of the widget. In fact I don't know how do it properly, so it's currently broken. Most of ours tools are written so they log and take input from stdout/stdin, is there a widget you can use to act as stdin/stdout ? Google didn't find any receip for that, it would have been great. If it exists, any code sample would be much appreciated. Jean-Michel From javier.collado at gmail.com Mon Jan 25 09:14:34 2010 From: javier.collado at gmail.com (Javier Collado) Date: Mon, 25 Jan 2010 15:14:34 +0100 Subject: Sikuli: the coolest Python project I have yet seen... In-Reply-To: <4B5D9F09.7050607@it.uu.se> References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> <4B5D9F09.7050607@it.uu.se> Message-ID: Hello, I think the site is under maintenance. I tried a couple of hours ago and it worked fine. As an alternative, I found that this link also worked: http://www.sikuli.org/ Unfortunately, it seems it's not working right now. Best regards, Javier 2010/1/25 Virgil Stokes : > On 25-Jan-2010 04:18, Ron wrote: >> >> Sikuli is the coolest Python project I have ever seen in my ten year >> hobbyist career. An MIT oepn source project, Sikuli uses Python to >> automate GUI tasks (in any GUI or GUI baed app that runs the JVM) by >> simply drag and dropping GUI elements into Python scripts as function >> arguments. Download at http://sikuli.csail.mit.edu/ I also did this >> > > This link is broken! > > --V > > -- > http://mail.python.org/mailman/listinfo/python-list > From kyosohma at gmail.com Mon Jan 25 09:16:26 2010 From: kyosohma at gmail.com (Mike Driscoll) Date: Mon, 25 Jan 2010 06:16:26 -0800 (PST) Subject: WxPython upgrade trouble on Ubuntu 8.04 References: Message-ID: On Jan 25, 5:18?am, "lada... at my-deja.com" wrote: > Hello everyone, > > You know the old saying, "in for a penny, in for a pound." > > Several hours ago I posted this... > > http://groups.google.com/group/comp.lang.python/msg/0a86e792c674adc8 > > ...in which I described my desire to acquire Python 2.6 without > upgrading my Ubuntu Linux installation from 8.04. > > Since Python 2.6 is not part of the Ubuntu 8.04 repository, I > installed Python 2.6 manually. ?Things are a little messy right now, > since invoking IDLE from the desktop still defaults to Python 2.5. > But I'm getting there. ?At least I can access Python 2.6 from the > command prompt, and SCIte also finds 2.6. > > Of course, I also have to reinstall all of the Python modules that I > use on 2.6. ?I have succeeded with numpy and biopython, both installed > manually. ?WxPython is giving me trouble, though. > > I visited wxpython.org and noted that wx version 2.8.10.1 fixes some > important bugs that can occur when using Python 2.6. ?The instructions > here... > > http://wiki.wxpython.org/InstallingOnUbuntuOrDebian > > ...allowed me to add the wxWidgets apt repository to the list of web > sites that my Synaptic Package Manager checks for updates. ?It found > wx 2.8.10.1 and auto-installed it. ?I got error messages from Python > 2.5 until I also installed python-wxtools, wx2.8-i18n, libwxgtk2.8- > dev, and libgtk2.0-dev as recommended on the web page mentioned > above. ?Now "import wx" worked again, with no error messages, from > Python 2.5. > > Installing wx on Python 2.6 is my ultimate goal, and here's where I'm > stuck. ?I actually tried "import wx" from Python 2.6 first, and I got > this result: > > >>> import wx > > Traceback (most recent call last): > ? File "", line 1, in > ? File "wx/__init__.py", line 45, in > ? ? from wx._core import * > ? File "wx/_core.py", line 4, in > ? ? import _core_ > ImportError: No module named _core_ > > I went back to Synaptic, said that I wanted to reinstall wx 2.8.10.1 > -- but not really, as I then selected "download package files only." > I received the tarball. ?The top-level folder has the wx binary stuff, > which I believe is already installed. ?I selected the wxPython > directory at the second level. ?Tried "python setup.py build", and > after several minutes and dozens of warning messages, I finally got > this fatal error: > > In file included from /usr/include/wx-2.8/wx/glcanvas.h:54, > ? ? ? ? ? ? ? ? ?from contrib/glcanvas/gtk/glcanvas_wrap.cpp:2661: > /usr/include/wx-2.8/wx/gtk/glcanvas.h: At global scope: > /usr/include/wx-2.8/wx/gtk/glcanvas.h:47: error: ?GLXContext? does not > name a type > /usr/include/wx-2.8/wx/gtk/glcanvas.h:124: error: ISO C++ forbids > declaration of ?GLXFBConfig? with no type > /usr/include/wx-2.8/wx/gtk/glcanvas.h:124: error: expected ?;? before > ?*? token > error: command 'gcc' failed with exit status 1 > > Worse -- NOW if I start Python 2.5, I get the same error message I got > from 2.6. > > Now I'm stuck again. ?Any advice? ?Many thanks! Try re-posting to the wxPython mailing list. Someone there will know what the next step is. ------------------- Mike Driscoll Blog: http://blog.pythonlibrary.org PyCon 2010 Atlanta Feb 19-21 http://us.pycon.org/ From albert at spenarnc.xs4all.nl Mon Jan 25 09:26:19 2010 From: albert at spenarnc.xs4all.nl (Albert van der Horst) Date: 25 Jan 2010 14:26:19 GMT Subject: Is python not good enough? References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> Message-ID: In article , Aahz wrote: >In article <1b42700d-139a-4653-8669-d4ee2fc488b6 at r5g2000yqb.googlegroups.com>, >ikuta liu wrote: >> >>Is python not good enough? for google, enhance python performance is >>the good way better then choose build Go language? > >It is not at all clear that -- despite some comments to the contrary -- >the Go developers are intending to compete with Python. Go seems much >more intended to compete with C++/Java. If they're successful, we may >eventually see GoPython. ;-) As far as I can tell, Go was not intended to compete with anything. It was their own itch they scratched. Then they opened it to the world, which I applaud. If Go was to compete with anything, they would have give it a name that was Googleable. ;-) >-- >Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Groetjes Albert -- -- Albert van der Horst, UTRECHT,THE NETHERLANDS Economic growth -- being exponential -- ultimately falters. albert at spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst From albert at spenarnc.xs4all.nl Mon Jan 25 09:50:35 2010 From: albert at spenarnc.xs4all.nl (Albert van der Horst) Date: 25 Jan 2010 14:50:35 GMT Subject: A simple-to-use sound file writer References: Message-ID: In article , Mel wrote: >Alf P. Steinbach wrote: >> * Steve Holden: > >>> It's not clear to me that you can approximate any waveform with a >>> suitable combination of square waves, >> >> Oh. It's simple to prove. At least conceptually! :-) >> >> Consider first that you need an infinite number of sine waves to create a >> perfect square wave. >> >> The opposite also holds: infinite number of square waves to create a >> perfect sine wave (in a way sines and squares are opposites, the most >> incompatible). > >No, it doesn't. The infinite set of sine waves that make a square wave >leave out the sine waves of frequency 2f, 4f, 6f, 8f, ... (2*n*f) ... . >Once you've left them out, you can never get them back. So sawtooth waves, >for example, can't generally be built out of sets of square waves. Bullshit. My Boehm (B\"ohm) electronic organ does exactly that. They even have a chip for it. In the 70's it was a great hype, a sawtooth organ. Well not exactly a hype, the sound, especially the low registers, is dramatically better. If you're interested in frequencies above audible (organ builders aren't), you need an infinity of squares to build a perfect sawtooth. But then you need an inifinity of sines to build a perfect square wave. > Mel. Groetjes Albert -- -- Albert van der Horst, UTRECHT,THE NETHERLANDS Economic growth -- being exponential -- ultimately falters. albert at spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst From ursusmaximus at gmail.com Mon Jan 25 10:38:32 2010 From: ursusmaximus at gmail.com (Ron) Date: Mon, 25 Jan 2010 07:38:32 -0800 (PST) Subject: Sikuli: the coolest Python project I have yet seen... References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> Message-ID: Thew link at MIT does appear to be down right now, but I presume it will come back up. Well, those of you who find it underwhelming are in good company. See the blog post at Lambda the Ultimate http://lambda-the-ultimate.org/node/3783 I was impressed though by the application to notify you when your bus gets close to the pickup point, using Google maps, and by the app to automatically chart a course to Houston from LA on I-10, again using Google maps. And perhaps most of all, the app to notify you when your sleeping baby wakes up, from a picture on a digital camera. Hey, most of life is non-deterministic. I am in the analog engineering world and simple, deterministic black and white situations are all fine and useful, but I can see this very easy to use and simple technology being useful also ;-)) All of the above apps are but a few lines of code. Ron From tavares at fe.up.pt Mon Jan 25 10:59:33 2010 From: tavares at fe.up.pt (tavares at fe.up.pt) Date: Mon, 25 Jan 2010 07:59:33 -0800 (PST) Subject: CompIMAGE 2010: Call for Short Communications Message-ID: Dear Colleague, In the framework of the International Symposium CompIMAGE'10 ?Computational Modeling of Objects Presented in Images: Fundamentals, Methods, and Applications,? May 5-7, 2010, Buffalo-Niagara, NY, USA (www.fe.up.pt/CompIMAGE), prospective authors are invited to submit short communications to be published in the Symposium proceedings and presented at the Symposium. Deadline for submission: February 23, 2010. This deadline is firm and will not be extended. Requirements: The manuscripts must be prepared according to the LNCS requirements and submitted electronically in PDF format, together with all source files and figures. The length should be between 2 and 6 pages. The papers will be reviewed by the Editors. The authors of accepted papers will be notified by February 26, 2010. The accepted papers will be published by Research Publishing Services (Singapore) in an edited book with an ISBN. A paper will be included in the proceedings only if at least one of its author registers by February 28, 2010 and sends a symposium fee for short communication, which includes access to all symposium materials, banquet, and tour of Niagara Falls (please, visit the symposium webpage at: www.fe.up.pt/CompIMAGE). The participants are supposed to present their work orally or at a poster session. Yours sincerely, Jo?o Manuel R. S. Tavares (University of Porto, symposium co-chair) From ladasky at my-deja.com Mon Jan 25 11:10:54 2010 From: ladasky at my-deja.com (John Ladasky) Date: Mon, 25 Jan 2010 08:10:54 -0800 (PST) Subject: WxPython upgrade trouble on Ubuntu 8.04 References: Message-ID: Hi Mike, Thanks, I forgot that wxPython-users is distinct from comp.soft- sys.wxwindows. I'll give it a try. From wegwerp at gmail.com Mon Jan 25 11:16:28 2010 From: wegwerp at gmail.com (Bas) Date: Mon, 25 Jan 2010 08:16:28 -0800 (PST) Subject: medians for degree measurements References: Message-ID: <5b9beaa6-f707-46f6-b439-c9adb1fd7794@r19g2000yqb.googlegroups.com> On Jan 23, 1:09?am, Steve Howell wrote: [snip problem with angle data wrapping around at 360 degrees] Hi, This problem is trivial to solve if you can assume that you that your data points are measured consecutively and that your boat does not turn by more than 180 degrees between two samples, which seems a reasonable use case. If you cannot make this assumption, the answer seems pretty arbitrary to me anyhow. The standard trick in this situation is to 'unwrap' the data (fix > 180 deg jumps by adding or subtracting 360 to subsequent points), do your thing and then 'rewrap' to your desired interval ([0-355] or [-180,179] degrees). In [1]: from numpy import * In [2]: def median_degree(degrees): ...: return mod(rad2deg(median(unwrap(deg2rad(degrees)))),360) ...: In [3]: print(median_degree([1, 2, 3, 4, 5, 6, 359])) 3.0 In [4]: print(median_degree([-179, 174, 175, 176, 177, 178, 179])) 177.0 If the deg2rad and rad2deg bothers you, you should write your own unwrap function that handles data in degrees. Hope this helps, Bas P.S. Slightly off-topic rant against both numpy and matlab implementation of unwrap: They always assume data is in radians. There is some option to specify the maximum jump size in radians, but to me it would be more useful to specify the interval of a complete cycle, so that you can do unwrapped_radians = unwrap(radians) unwrapped_degrees = unwrap(degrees, 360) unwrapped_32bit_counter = unwrap(overflowing_counter, 2**32) From robert.kern at gmail.com Mon Jan 25 11:34:48 2010 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 25 Jan 2010 10:34:48 -0600 Subject: medians for degree measurements In-Reply-To: <5b9beaa6-f707-46f6-b439-c9adb1fd7794@r19g2000yqb.googlegroups.com> References: <5b9beaa6-f707-46f6-b439-c9adb1fd7794@r19g2000yqb.googlegroups.com> Message-ID: On 2010-01-25 10:16 AM, Bas wrote: > P.S. > Slightly off-topic rant against both numpy and matlab implementation > of unwrap: They always assume data is in radians. There is some option > to specify the maximum jump size in radians, but to me it would be > more useful to specify the interval of a complete cycle, so that you > can do > > unwrapped_radians = unwrap(radians) > unwrapped_degrees = unwrap(degrees, 360) > unwrapped_32bit_counter = unwrap(overflowing_counter, 2**32) Rants accompanied with patches are more effective. :-) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From wegwerp at gmail.com Mon Jan 25 12:06:34 2010 From: wegwerp at gmail.com (Bas) Date: Mon, 25 Jan 2010 09:06:34 -0800 (PST) Subject: medians for degree measurements References: <5b9beaa6-f707-46f6-b439-c9adb1fd7794@r19g2000yqb.googlegroups.com> Message-ID: <5f4a7bb8-5e5e-408c-98a1-0d2077590ce5@v25g2000yqk.googlegroups.com> > On 2010-01-25 10:16 AM, Bas wrote: > > > P.S. > > Slightly off-topic rant against both numpy and matlab implementation > > of unwrap: They always assume data is in radians. There is some option > > to specify the maximum jump size in radians, but to me it would be > > more useful to specify the interval of a complete cycle, so that you > > can do > > > unwrapped_radians = unwrap(radians) > > unwrapped_degrees = unwrap(degrees, 360) > > unwrapped_32bit_counter = unwrap(overflowing_counter, 2**32) On Jan 25, 5:34 pm, Robert Kern wrote:> > Rants accompanied with patches are more effective. :-) As you wish (untested): def unwrap(p, cycle=2*pi, axis=-1): """docstring to be updated""" p = asarray(p) half_cycle = cycle / 2 nd = len(p.shape) dd = diff(p, axis=axis) slice1 = [slice(None, None)]*nd # full slices slice1[axis] = slice(1, None) ddmod = mod(dd+half_cycle, cycle)-half_cycle _nx.putmask(ddmod, (ddmod==-half_cycle) & (dd > 0), half_cycle) ph_correct = ddmod - dd; _nx.putmask(ph_correct, abs(dd) References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> <8c7f10c61001250557u6b37f284x202574f20becd326@mail.gmail.com> Message-ID: <4eb0089f1001250922l2138556cl91c6091a0f97d402@mail.gmail.com> On Mon, Jan 25, 2010 at 8:57 AM, Simon Brunning wrote: > 2010/1/25 Albert van der Horst : >> If Go was to compete with anything, they would have give it a name >> that was Googleable. ;-) > If they want it Googleable, it will be. ;-) http://www.google.com/search?q=go+language From showell30 at yahoo.com Mon Jan 25 12:31:15 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 09:31:15 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> <7xpr4ylk9d.fsf@ruckus.brouhaha.com> Message-ID: On Jan 24, 11:24?pm, Paul Rubin wrote: > Steve Howell writes: > > There is nothing wrong with deque, at least as far as I know, if the > > data strucure actually applies to your use case. ?It does not apply to > > my use case. > > You haven't explained why deque doesn't apply to your use case. ?Until a > convincing explanation emerges, the sentiment you're creating seems to > be "what's wrong with that guy and why doesn't he just use deque?". ?So, > why aren't you using deque? ?If deque somehow isn't adequate for your > use case, maybe it can be improved. > These are the reasons I am not using deque: 1) I want to use native lists, so that downstream methods can use them as lists. 2) Lists are faster for accessing elements. 3) I want to be able to insert elements into the middle of the list. 4) I have no need for rotating elements. I also have reasons for not using the other workarounds, such as reversing the list. > >> And when discussing performance in this context, additive constants > >> do matter. > > Wrong again. ?Operations that mutate lists are already expensive > > I'm talking about memory consumption, which is part of Python's concept > of performance. ?You're proposing adding a word or two to every list, > with insufficient justification presented so far. ?Any such > justification would have to include a clear and detailed explanation of > why using deque is insufficient, so that would be a good place to start. > Adding a word or two to a list is an O(1) addition to a data structure that takes O(N) memory to begin with. That extra pointer should really be taken not just in context of the list itself taking O(N) memory, but also the fact that all the elements in the list are also consuming memory (until they get popped off). So adding the pointer has neglible cost. Another way of looking at it is that you would need to have 250 or so lists in memory at the same time before the extra pointer was even costing you kilobytes of memory. My consumer laptop has 3027908k of memory. From robert.kern at gmail.com Mon Jan 25 12:39:04 2010 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 25 Jan 2010 11:39:04 -0600 Subject: medians for degree measurements In-Reply-To: <5f4a7bb8-5e5e-408c-98a1-0d2077590ce5@v25g2000yqk.googlegroups.com> References: <5b9beaa6-f707-46f6-b439-c9adb1fd7794@r19g2000yqb.googlegroups.com> <5f4a7bb8-5e5e-408c-98a1-0d2077590ce5@v25g2000yqk.googlegroups.com> Message-ID: On 2010-01-25 11:06 AM, Bas wrote: >> On 2010-01-25 10:16 AM, Bas wrote: >> >>> P.S. >>> Slightly off-topic rant against both numpy and matlab implementation >>> of unwrap: They always assume data is in radians. There is some option >>> to specify the maximum jump size in radians, but to me it would be >>> more useful to specify the interval of a complete cycle, so that you >>> can do >> >>> unwrapped_radians = unwrap(radians) >>> unwrapped_degrees = unwrap(degrees, 360) >>> unwrapped_32bit_counter = unwrap(overflowing_counter, 2**32) > > On Jan 25, 5:34 pm, Robert Kern wrote:> >> Rants accompanied with patches are more effective. :-) > > As you wish (untested): > > def unwrap(p, cycle=2*pi, axis=-1): > """docstring to be updated""" > p = asarray(p) > half_cycle = cycle / 2 > nd = len(p.shape) > dd = diff(p, axis=axis) > slice1 = [slice(None, None)]*nd # full slices > slice1[axis] = slice(1, None) > ddmod = mod(dd+half_cycle, cycle)-half_cycle > _nx.putmask(ddmod, (ddmod==-half_cycle)& (dd> 0), half_cycle) > ph_correct = ddmod - dd; > _nx.putmask(ph_correct, abs(dd) up = array(p, copy=True, dtype='d') > up[slice1] = p[slice1] + ph_correct.cumsum(axis) > return up > > I never saw a use case for the discontinuity argument, so in my > preferred version it would be removed. Of course this breaks old code > (by who uses this option anyhow??) and breaks compatibility between > matlab and numpy. Sometimes legitimate features have phase discontinuities greater than pi. If you want your feature to be accepted, please submit a patch that does not break backwards compatibility and which updates the docstring and tests appropriately. I look forward to seeing the complete patch! Thank you. http://projects.scipy.org/numpy -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From tim.arnold at sas.com Mon Jan 25 12:47:25 2010 From: tim.arnold at sas.com (Tim Arnold) Date: Mon, 25 Jan 2010 12:47:25 -0500 Subject: Splitting text at whitespace but keeping the whitespace in thereturned list References: <1264352736.30280.1356298585@webmail.messagingengine.com> Message-ID: "MRAB" wrote in message news:mailman.1362.1264353878.28905.python-list at python.org... > python at bdurham.com wrote: >> I need to parse some ASCII text into 'word' sized chunks of text AND >> collect the whitespace that seperates the split items. By 'word' I mean >> any string of characters seperated by whitespace (newlines, carriage >> returns, tabs, spaces, soft-spaces, etc). This means that my split text >> can contain punctuation and numbers - just not whitespace. >> The split( None ) method works fine for returning the word sized chunks >> of text, but destroys the whitespace separators that I need. >> Is there a variation of split() that returns delimiters as well as >> tokens? >> > I'd use the re module: > > >>> import re > >>> re.split(r'(\s+)', "Hello world!") > ['Hello', ' ', 'world!'] also, partition works though it returns a tuple instead of a list. >>> s = 'hello world' >>> s.partition(' ') ('hello', ' ', 'world') >>> --Tim Arnold From xahlee at gmail.com Mon Jan 25 12:55:12 2010 From: xahlee at gmail.com (Xah Lee) Date: Mon, 25 Jan 2010 09:55:12 -0800 (PST) Subject: Xah's Edu Corner: Unix Pipe As Functional Language Message-ID: <4c5dad6d-d8e9-46af-8ed8-5bc6dce057df@l11g2000yqb.googlegroups.com> Dear comrades, Hot from the press: ? Unix Pipe As Functional Language http://xahlee.org/comp/unix_pipes_and_functional_lang.html plain text version follows: -------------------------------------------------- Unix Pipe As Functional Language Xah Lee, 2010-01-25 Found the following juicy interview snippet today: Q: Is there a connection between the idea of composing programs together from the command line throught pipes and the idea of writing little languages, each for a specific domain? Alfred Aho: I think there's a connection. Certainly in the early days of Unix, pipes facilitated function composition on the command line. You could take an input, perform some transformation on it, and then pipe the output into another program. ... Q: When you say ?function composition?, that brings to mind the mathematical approach of function composition. Alfred Aho: That's exactly what I mean. Q: Was that mathematical formalism in mind at the invention of the pipe, or was that a metaphor added later when someone realized it worked the same way? Alfred Aho: I think it was right there from the start. Doug McIlroy, at least in my book, deserves the credit for pipes. He thought like a mathematician and I think he had this connection right from the start. I think of the Unix command line as a prototypical functional language. It is from a interview with Alfred Aho, one of the creators of AWK. The source is from this book: Masterminds of Programming: Conversations with the Creators of Major Programming Languages (2009), by Federico Biancuzzi et al. (amazon) Since about 1998, when i got into the unix programing industry, i see the pipe as a post fix notation, and sequencing pipes as a form of functional programing, but finding it overall extremely badly designed. I've wrote a few essays explaining the functional programing connection and exposing the lousy syntax. (mostly in years around 2000) However, i've never seen another person expressing the idea that unix pipes is a form of postfix notation and functional programing. It is a great satisfaction to see one of the main unix author state so. -------------------------------------------------- Unix Pipe As Functional Programing The following email content (slighted edited) is posted to Mac OS X mailing list, 2002-05. Source From: xah / xahlee.org Subject: Re: mail handling/conversion between OSes/apps Date: May 12, 2002 8:41:58 PM PDT Cc: macosx-talk / omnigroup.com Yes, unix have this beautiful philosophy. The philosophy is functional programing. For example, define: power(x) := x*x so ?power(3)? returns ?9?. Here ?power? is a function that takes 2 arguments. First parameter specifies the number to be raised to power, the second the number of times to multiply itself. functions can be nested, f(g(h(x))) or composed compose(f,g,h)(x) Here the ?compose? itself is a function, which take other functions as arguments, and the output of compose is a new function that is equivalent to nesting f g h. Nesting does not necessarily involved nested syntax. Here's a post fix notation in Mathematica for example: x // h // g // h or prefix notation: f @ g @ h @ x or in lisp (f (g (h x))) The principle is that everything is either a function definition or function application, and function's behavior is strictly determined by its argument. Apple around 1997 or so have this OpenDoc technology, which is similar idea applied more broadly across OS. That is, instead of one monolithic browser or big image editors or other software, but have lots of small tools or components that each does one specific thing and all can call each other or embedded in a application framework as services or the like. For example, in a email apps, you can use BBEdit to write you email, use Microsoft's spell checker, use XYZ brand of recorder to record a message, without having to open many applications or use the Finder the way we would do today. This multiplies flexibility. (OpenDoc was killed when Steve Jobs become the iCEO around 1998 and did some serious house-keeping, against the ghastly anger of Mac developers and fanatics, I'm sure many of you remember this piece of history.) The unix pipe syntax ?|?, is a postfix notation for nesting. e.g. ps auwwx | awk '{print $2}' | sort -n | xargs echo in conventional syntax it might look like this: xargs( echo, sort(n, awk('print $2', ps(auwwx))) ) So when you use ?pipe? to string many commands in unix, you are doing supreme functional programing. That's why it is so flexible and useful, because each component or function does one thing, and you can combine them in myriad of ways. However, this beautiful functional programing idea, when it is implemented by the unix heads, becomes a fucking mess. Nothing works and nothing works right. I don't feel like writing a comprehensive exposition on this at the moment. Here's a quick summary: * Fantastically stupid syntax. * Inconsistencies everywhere. Everywhere. * Fucking stupid global variables reliance called environment variables, which fucks up the whole functional programing paradigm. * Implicit stuff everywhere. * Totally incompetent commands and their parameters. (promiscuously non-orthogonal, and missing things, and fucked up in just more ways than one can possibly imagine. there are one million way to do one thing, and none are correct, and much simple needs CANNOT be done! (that's why there are gazillion shells each smart-ass improving upon the other, and that's why Perl is born too! But asinine Larry Wall don't know shit but smugly created another complexity that don't do much.)) Maybe some other day when i'm pissed, i'll write a better exposition on this issue. I've been wanting to write a final-say essay on this for long. Don't feel like it now. -------------------------------------------------- Unix Syntatical and Semantical Stupidity Exposition The following is posted to a Mac OS X mailing list. Source From: xah at xahlee.org Subject: unix inanity: shell env syntax Date: June 7, 2002 12:00:29 AM PDT To: macosx-talk at omnigroup.com Unix Syntatical and Semantical Stupidity Exposition. (this is one of the many technical expositions of unix stupidity) (this is currently unpolished, but the meat is there. Input welcome.) arguments are given with a dash prefix. e.g. ls -a -l Order (usually) does not matter. So, ls -a -l is the same as ls -l -a but arguments can be combined, e.g. ls -al means the same thing as ls -a -l However, some option consists of more than one character. e.g. perl -version perl -help therefore, the meaning of a option string "-ab" is ad hoc dependent on the program. It can be "-a -b" or just a option named "ab". Then, sometimes there are two versions of the same optional argument. e.g. perl -help perl -h perl -version perl -v this equivalence is ad hoc. Different program will disagree on common options. For example, to get the version, here are common varieties: -v -V -version sometimes v/V stands for "verbose mode", i.e. to output more detail. Sometimes, if a option is given more than once, then it specifies a degree of that option. For example, some command accept the -v for "verbose", meaning that it will output more detail. Sometimes there are few levels of detail. The number of times a option is given determines the level of detail. e.g. on Solaris 8, /usr/ucb/ps -w /usr/ucb/ps -w -w Thus, meaning of repeated option may have special meaning depending on the program. Oftentimes some options automatically turn on or surpress a bunch of others. e.g. Solaris 8, /usr/bin/ls -f When a named optional parameter is of a boolean type, that is a toggle of yes/no, true/false, exist/nonexist, then it is often times that instead of taking a boolean value, their sole existence or non- existence defines their value. Toggle options are sometimes represented by one option name for yes, while another option name for no, and when both are present, the behavior is program dependent. Toggle options are represented by different option names. For named options, their syntax is slack but behavior is usually dependent on the program. i.e. not all of the following works for every program command -o="myvalue" command -omyvalue comand -o myvalue Often one option may have many synonyms... A example of a better design... (Mathematica, Scheme, Dylan, Python, Ruby... there's quite a lot elegance and practicality yet distinct designs and purposes and styles ...) (recall that unix have a bad design to begin with; it's a donkey shit pile from the beginning and continuation. Again, unix is not simply technically incompetent. If that, then that's easy to improve, and i don't have a problem with, since there are things in one way or another considered horrendous by today's standard like COBOL or FORTRAN or DOS etc. But, unix is a brain-washing idiot-making machine, churning out piles and piles of religiously idiotic and pigheaded keyboard punchers. For EVERY aspects of good engineering methodology improvement or language design progress opportunity, unixers will unanimously turn it down. Inevitably someone will ask me what's my point. My point in my series of unix-showcasing articles have always been clear for those who studies it: Unix is a crime that caused society inordinate harm, and i want unix geeks to wake up and realize it. Microsoft PowerShell Note: Microsoft's new shell programing language, PowerShell (b2006), adopted much of unix shell's syntax and the pipe paradigm, but with a much consistent and formal design. (see: Xah's PowerShell Tutorial) Xah ? http://xahlee.org/ ? From showell30 at yahoo.com Mon Jan 25 13:00:03 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 10:00:03 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> Message-ID: <18177f2a-ca15-4d3c-9f6c-09beb21a4947@y12g2000yqh.googlegroups.com> On Jan 24, 10:07?pm, Steven D'Aprano wrote: > On Sun, 24 Jan 2010 20:12:11 -0800, Steve Howell wrote: > >> > The most ambitious proposal is to fix the memory manager itself to > >> > allow the release of memory from the start of the chunk. > > >> That's inappropriate given the memory fragmentation it would cause. > > > Bullshit. ?Memory managers consolidate free memory chunks all the time. > > That's their job. > > So let me get this straight... > > You've complained that Python's list.pop(0) is lame because it moves > memory around. And your solution to that is to have the memory manager > move the memory around instead? > > Perhaps I'm missing something, but I don't see the advantage here. At > best, you consolidate all those moves you wanted to avoid and do them all > at once instead of a few at a time. At worst, you get a situation where > the application periodically, and unpredictably, grinds to a halt while > the memory manager tries to defrag all those lists. > You are misunderstanding what I meant, because I did not explain it very well. When you release memory from the front of the list, if the memory before it was also free, the memory manager could consolidate the two chunks as one free chunk. There is no rational scenario where the memory manager grinds to a halt tries to "defrag all those lists." Of course, once the list gets fully garbage collected, then entire chunk of memory is freed up. > >> Your approach of snarling against list is not persuading anyone that > >> list needs to be changed, because most everyone is satisfied with the > >> existing solution. > > > Please provide evidence of that. ?I am pretty sure that everybody who > > chooses alternatives to Python would disagree. > > Do you honestly believe that "everybody" who prefers another language > over Python does so because they dislike the performance of list.pop(0)? > No I don't believe any statement that makes gross generalizations, so I also don't believe "most everyone is satisfied with the existing solution." > >> You might change approaches and discuss deque, what's wrong with it, > >> and whether it can be fixed. ?Getting a change approved for deque is > >> probably much easier than getting one approved for list, just because > >> nowhere near as many things depend on deque's performance. > > > Again...I am not looking to improve deque, which is a perfectly valid > > data structure for a limited set of problems. > > >> And when discussing performance in this contextc additive constants do > >> matter. > > > Wrong again. ?Operations that mutate lists are already expensive, and a > > few checks to see if unreleased memory can be reclaimed are totally > > NEGLIGIBLE. > > Popping from the end of the list isn't expensive. Reversing lists is > relatively cheap. In-place modifications are very cheap. > I am talking in relative terms here. I am saying that checking a single flag in C code isn't gonna significantly slow down any operation that calls list_resize(). Delete operations would already be doing a memmove operation, and insert operations already have to decide whether to optimistically allocate memory and create the new list element. Regarding the extra use of memory, I addressed this in my prior posting. Here is code for list_resize: static int list_resize(PyListObject *self, Py_ssize_t newsize) { PyObject **items; size_t new_allocated; Py_ssize_t allocated = self->allocated; /* Bypass realloc() when a previous overallocation is large enough to accommodate the newsize. If the newsize falls lower than half the allocated size, then proceed with the realloc() to shrink the list. */ if (allocated >= newsize && newsize >= (allocated >> 1)) { assert(self->ob_item != NULL || newsize == 0); Py_SIZE(self) = newsize; return 0; } /* This over-allocates proportional to the list size, making room * for additional growth. The over-allocation is mild, but is * enough to give linear-time amortized behavior over a long * sequence of appends() in the presence of a poorly-performing * system realloc(). * The growth pattern is: 0, 4, 8, 16, 25, 35, 46, 58, 72, 88, ... */ new_allocated = (newsize >> 3) + (newsize < 9 ? 3 : 6); /* check for integer overflow */ if (new_allocated > PY_SIZE_MAX - newsize) { PyErr_NoMemory(); return -1; } else { new_allocated += newsize; } if (newsize == 0) new_allocated = 0; items = self->ob_item; if (new_allocated <= ((~(size_t)0) / sizeof(PyObject *))) PyMem_RESIZE(items, PyObject *, new_allocated); else items = NULL; if (items == NULL) { PyErr_NoMemory(); return -1; } self->ob_item = items; Py_SIZE(self) = newsize; self->allocated = new_allocated; return 0; } Here is the code for list_ass_slice: static int list_ass_slice(PyListObject *a, Py_ssize_t ilow, Py_ssize_t ihigh, PyObject *v) { /* Because [X]DECREF can recursively invoke list operations on this list, we must postpone all [X]DECREF activity until after the list is back in its canonical shape. Therefore we must allocate an additional array, 'recycle', into which we temporarily copy the items that are deleted from the list. :-( */ PyObject *recycle_on_stack[8]; PyObject **recycle = recycle_on_stack; /* will allocate more if needed */ PyObject **item; PyObject **vitem = NULL; PyObject *v_as_SF = NULL; /* PySequence_Fast(v) */ Py_ssize_t n; /* # of elements in replacement list */ Py_ssize_t norig; /* # of elements in list getting replaced */ Py_ssize_t d; /* Change in size */ Py_ssize_t k; size_t s; int result = -1; /* guilty until proved innocent */ #define b ((PyListObject *)v) if (v == NULL) n = 0; else { if (a == b) { /* Special case "a[i:j] = a" -- copy b first */ v = list_slice(b, 0, Py_SIZE(b)); if (v == NULL) return result; result = list_ass_slice(a, ilow, ihigh, v); Py_DECREF(v); return result; } v_as_SF = PySequence_Fast(v, "can only assign an iterable"); if(v_as_SF == NULL) goto Error; n = PySequence_Fast_GET_SIZE(v_as_SF); vitem = PySequence_Fast_ITEMS(v_as_SF); } if (ilow < 0) ilow = 0; else if (ilow > Py_SIZE(a)) ilow = Py_SIZE(a); if (ihigh < ilow) ihigh = ilow; else if (ihigh > Py_SIZE(a)) ihigh = Py_SIZE(a); norig = ihigh - ilow; assert(norig >= 0); d = n - norig; if (Py_SIZE(a) + d == 0) { Py_XDECREF(v_as_SF); return list_clear(a); } item = a->ob_item; /* recycle the items that we are about to remove */ s = norig * sizeof(PyObject *); if (s > sizeof(recycle_on_stack)) { recycle = (PyObject **)PyMem_MALLOC(s); if (recycle == NULL) { PyErr_NoMemory(); goto Error; } } memcpy(recycle, &item[ilow], s); if (d < 0) { /* Delete -d items */ memmove(&item[ihigh+d], &item[ihigh], (Py_SIZE(a) - ihigh)*sizeof(PyObject *)); list_resize(a, Py_SIZE(a) + d); item = a->ob_item; } else if (d > 0) { /* Insert d items */ k = Py_SIZE(a); if (list_resize(a, k+d) < 0) goto Error; item = a->ob_item; memmove(&item[ihigh+d], &item[ihigh], (k - ihigh)*sizeof(PyObject *)); } for (k = 0; k < n; k++, ilow++) { PyObject *w = vitem[k]; Py_XINCREF(w); item[ilow] = w; } for (k = norig - 1; k >= 0; --k) Py_XDECREF(recycle[k]); result = 0; Error: if (recycle != recycle_on_stack) PyMem_FREE(recycle); Py_XDECREF(v_as_SF); return result; #undef b } From showell30 at yahoo.com Mon Jan 25 13:13:57 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 10:13:57 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> <7xpr4ylk9d.fsf@ruckus.brouhaha.com> Message-ID: <1e6c08e2-3d37-484c-a137-eeb1cd3fed59@c4g2000yqa.googlegroups.com> On Jan 25, 9:31?am, Steve Howell wrote: > On Jan 24, 11:24?pm, Paul Rubin wrote: > > > Steve Howell writes: > > > There is nothing wrong with deque, at least as far as I know, if the > > > data strucure actually applies to your use case. ?It does not apply to > > > my use case. > > > You haven't explained why deque doesn't apply to your use case. ?Until a > > convincing explanation emerges, the sentiment you're creating seems to > > be "what's wrong with that guy and why doesn't he just use deque?". ?So, > > why aren't you using deque? ?If deque somehow isn't adequate for your > > use case, maybe it can be improved. > > These are the reasons I am not using deque: > > ? 1) I want to use native lists, so that downstream methods can use > them as lists. > ? 2) Lists are faster for accessing elements. > ? 3) I want to be able to insert elements into the middle of the list. > ? 4) I have no need for rotating elements. > > I also have reasons for not using the other workarounds, such as > reversing the list. > > > >> And when discussing performance in this context, additive constants > > >> do matter. > > > Wrong again. ?Operations that mutate lists are already expensive > > > I'm talking about memory consumption, which is part of Python's concept > > of performance. ?You're proposing adding a word or two to every list, > > with insufficient justification presented so far. ?Any such > > justification would have to include a clear and detailed explanation of > > why using deque is insufficient, so that would be a good place to start. > > Adding a word or two to a list is an O(1) addition to a data structure > that takes O(N) memory to begin with. > > That extra pointer should really be taken not just in context of the > list itself taking O(N) memory, but also the fact that all the > elements in the list are also consuming memory (until they get popped > off). ?So adding the pointer has neglible cost. > > Another way of looking at it is that you would need to have 250 or so > lists in memory at the same time before the extra pointer was even > costing you kilobytes of memory. ?My consumer laptop has 3027908k of > memory. I should also point out that my telephone has gigabytes of memory. It's a fairly expensive device, but I regularly carry multiple gigabytes of memory around in my front pants pocket. There are some valid reasons to reject a proposal to make deleting elements off the top of the list be O(1). Memory consumption is not one of them. Even the most naive patch to make pop(0) and del lst[0] advance the pointer would eventually reclaim memory once the list is garbage collected. Also, by allowing users to pop elements off the list without a memmove, you encourage users to discard elements earlier in the process, which means you can amortize the garbage collection for the list elements themselves (i.e. less spiky), and do it earlier. From showell30 at yahoo.com Mon Jan 25 13:24:04 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 10:24:04 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> Message-ID: On Jan 24, 1:51?pm, Daniel Stutzbach wrote: > On Sun, Jan 24, 2010 at 1:53 PM, Steve Howell wrote: > > I don't think anybody provided an actual link, but please correct me > > if I overlooked it. > > I have to wonder if my messages are all ending up in your spam folder > for some reason. :-) > > PEP 3128 (which solves your problem, but not using the implementation > you suggest)http://www.python.org/dev/peps/pep-3128/ > > Implementation as an extension module:http://pypi.python.org/pypi/blist/ > > Related discussion:http://mail.python.org/pipermail/python-3000/2007-April/006757.htmlhttp://mail.python.org/pipermail/python-3000/2007-May/007491.html > be > Detailed performance comparison:http://stutzbachenterprises.com/performance-blist > > I maintain a private fork of Python 3 with the blist replacing the > regular list, as a way of rigorously testing the blist implementation. > > Although I originally proposed a PEP, I am content to have the blist > exist as a third-party module. > Hi Daniel, I agree with what Raymond Hettinger says toward the top of the PEP. Blist, while extremely useful, does seem to have to trade off performance of common operations, notably "get item", in order to get better performance for other operations (notably insert/delete). My algorithm does exactly N pops and roughly N list accesses, so I would be going from N*N + N to N + N log N if switched to blist. That would be at least a theoretical gain over the current performance, but if pop() were O(1), I could get the whole thing down to N time. From wegwerp at gmail.com Mon Jan 25 13:31:04 2010 From: wegwerp at gmail.com (Bas) Date: Mon, 25 Jan 2010 10:31:04 -0800 (PST) Subject: medians for degree measurements References: <5b9beaa6-f707-46f6-b439-c9adb1fd7794@r19g2000yqb.googlegroups.com> <5f4a7bb8-5e5e-408c-98a1-0d2077590ce5@v25g2000yqk.googlegroups.com> Message-ID: <6da539f0-0b63-4467-ae3a-4e7b9d86dc98@l19g2000yqb.googlegroups.com> > >> On 2010-01-25 10:16 AM, Bas wrote: > >>> P.S. > >>> Slightly off-topic rant against both numpy and matlab implementation > >>> of unwrap: They always assume data is in radians. There is some option > >>> to specify the maximum jump size in radians, but to me it would be > >>> more useful to specify the interval of a complete cycle, so that you > >>> can do [snip] > > I never saw a use case for the discontinuity argument, so in my > > preferred version it would be removed. Of course this breaks old code > > (by who uses this option anyhow??) and breaks compatibility between > > matlab and numpy. On Jan 25, 6:39 pm, Robert Kern wrote: > Sometimes legitimate features have phase discontinuities greater than pi. We are dwelling more and more off-topic here, but anyhow: According to me, the use of unwrap is inherently related to measurement instruments that wrap around, like rotation encoders, interferometers or up/down counters. Say you have a real phase step of +1.5 pi, how could you possibly discern if from a real phase step of -pi/2? This is like an aliasing problem, so the only real solution would be to increase the sampling speed of your system. To me, the discontinuity parameter is serving some hard to explain corner case (see matlab manual), which is better left to be solved by hand in cases it appears. I regret matlab ever added the feature. > If you want your feature to be accepted, please submit a patch that does not break > backwards compatibility and which updates the docstring and tests appropriately. > I look forward to seeing the complete patch! Thank you. I think my 'cycle' argument does have real uses, like the degrees in this thread and the digital-counter example (which comes from own experience and required me to write my own unwrap). I'll try to submit a non-breaking patch if I ever have time. Bas From showell30 at yahoo.com Mon Jan 25 13:36:19 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 10:36:19 -0800 (PST) Subject: medians for degree measurements References: <044268c0-c308-406e-8d5b-a38ef5b5e381@p24g2000yqm.googlegroups.com> <036acf91$0$1357$c3e8da3@news.astraweb.com> Message-ID: <4131428a-8497-4b4d-84ab-fdfce1dee270@j14g2000yqm.googlegroups.com> On Jan 24, 5:26?pm, Robert Kern wrote: > On 2010-01-23 05:52 , Steven D'Aprano wrote: > > On Fri, 22 Jan 2010 22:09:54 -0800, Steve Howell wrote: > > >> On Jan 22, 5:12 pm, MRAB ?wrote: > >>> Steve Howell wrote: > >>>> I just saw the thread for medians, and it reminded me of a problem > >>>> that I need to solve. ?We are writing some Python software for > >>>> sailing, and we need to detect when we've departed from the median > >>>> heading on the leg. ?Calculating arithmetic medians is > >>>> straightforward, but compass bearings add a twist. > > [...] > >> I like this implementation, and it would probably work 99.9999% of the > >> time for my particular use case. ?The only (very contrived) edge case > >> that I can think of is when you have 10 bearings to SSW, 10 bearings to > >> SSE, and the two outliers are unfortunately in the NE and NW quadrants. > >> It seems like the algorithm above would pick one of the outliers. > > > The trouble is that median of angular measurements is not a meaningful > > concept. The median depends on the values being ordered, but angles can't > > be sensibly ordered. Which is larger, 1 degree north or 359 degrees? Is > > the midpoint between them 0 degree or 180 degree? > > Then don't define the median that way. Instead, define the median as the point > that minimizes the sum of the absolute deviations of the data from that point > (the L1 norm of the deviations, for those familiar with that terminology). For > 1-D data on the real number line, that corresponds to sorting the data and > taking the middle element (or the artithmetic mean of the middle two in the case > of even-numbered data). My definition applies to other spaces, too, that don't > have a total order attached to them including the space of angles. > > The "circular median" is a real, well-defined statistic that is used for exactly > what the OP intends to use it for. > I admitted pretty early in the thread that I did not define the statistic with much rigor, although most people got the gist of the problem, and as Robert points out, you can more clearly the define the problem, although I think under any definition, some inputs will have multiple solutions, such as (0, 90, 180, 270) and (0, 120, 240). If you've ever done lake sailing, you probably have encountered days where the wind seems to be coming from those exact angles. This is the code that I'll be using (posted by "Nobody"). I'll report back it if it has any issues. def mean(bearings): x = sum(sin(radians(a)) for a in bearings) y = sum(cos(radians(a)) for a in bearings) return degrees(atan2(x, y)) def median(bearings): m = mean(bearings) bearings = [(a - m + 180) % 360 - 180 for a in bearings] bearings.sort() median = bearings[len(bearings) / 2] median += m median %= 360 return median From moibenko at fnal.gov Mon Jan 25 14:05:28 2010 From: moibenko at fnal.gov (Alexander Moibenko) Date: Mon, 25 Jan 2010 13:05:28 -0600 Subject: Total maximal size of data Message-ID: <4B5DEB78.10406@fnal.gov> I have a simple question to which I could not find an answer. What is the total maximal size of list including size of its elements? I do not like to look into python source. Here is a code example: import struct KB=1024 MB=KB*KB GB=MB*KB buf=[] bs=32*KB n=4*GB/bs print "N",n i=0 size=0L while i < n: data = struct.pack("%ss" % (bs,), "") buf.append(data) size = size+bs if size % (100*MB) == 0: print "SIZE", size/MB, "MB" i=i+1 while 1: # to keep script running while I am looking at the machine status pass Here is what I get on 32-bit architecture: cat /proc/meminfo MemTotal: 8309860 kB MemFree: 5964888 kB Buffers: 84396 kB Cached: 865644 kB SwapCached: 0 kB ...... The program output: N 131072 SIZE 100 MB SIZE 200 MB SIZE 300 MB SIZE 400 MB SIZE 500 MB SIZE 600 MB SIZE 700 MB SIZE 800 MB SIZE 900 MB SIZE 1000 MB SIZE 1100 MB SIZE 1200 MB SIZE 1300 MB SIZE 1400 MB SIZE 1500 MB SIZE 1600 MB SIZE 1700 MB SIZE 1800 MB SIZE 1900 MB SIZE 2000 MB SIZE 2100 MB SIZE 2200 MB SIZE 2300 MB SIZE 2400 MB SIZE 2500 MB SIZE 2600 MB SIZE 2700 MB SIZE 2800 MB SIZE 2900 MB SIZE 3000 MB Traceback (most recent call last): File "bs.py", line 14, in ? data = struct.pack("%ss" % (bs,), "") MemoryError ++++++++++++++++++++ The number of list elements for a given block size is 131072. If I change block size the script traces back at the same total size 3000MB. Somewhere I read that list could have 2147483647 items, on most platforms. Somewhere else that it is *536,870,912 (http://stackoverflow.com/questions/855191/how-big-can-a-python-array-get)** *But what is the maximal size of the whole list including the size of its elements? Thanks. From deets at nospam.web.de Mon Jan 25 14:23:50 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Mon, 25 Jan 2010 20:23:50 +0100 Subject: Total maximal size of data In-Reply-To: References: Message-ID: <4B5DEFC6.9080907@nospam.web.de> Am 25.01.10 20:05, schrieb Alexander Moibenko: > I have a simple question to which I could not find an answer. > What is the total maximal size of list including size of its elements? > I do not like to look into python source. But it would answer that question pretty fast. Because then you'd see that all list-object-methods are defined in terms of Py_ssize_t, which is an alias for ssize_t of your platform. 64bit that should be a 64bit long. Diez From deets at nospam.web.de Mon Jan 25 14:23:57 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Mon, 25 Jan 2010 20:23:57 +0100 Subject: Total maximal size of data In-Reply-To: References: Message-ID: <7s69efF2u0U1@mid.uni-berlin.de> Am 25.01.10 20:05, schrieb Alexander Moibenko: > I have a simple question to which I could not find an answer. > What is the total maximal size of list including size of its elements? > I do not like to look into python source. But it would answer that question pretty fast. Because then you'd see that all list-object-methods are defined in terms of Py_ssize_t, which is an alias for ssize_t of your platform. 64bit that should be a 64bit long. Diez From tjreedy at udel.edu Mon Jan 25 14:27:17 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 25 Jan 2010 14:27:17 -0500 Subject: Sikuli: the coolest Python project I have yet seen... In-Reply-To: References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> <4B5D9F09.7050607@it.uu.se> Message-ID: On 1/25/2010 9:14 AM, Javier Collado wrote: > I think the site is under maintenance. I tried a couple of hours ago > and it worked fine. > > As an alternative, I found that this link also worked: > http://www.sikuli.org/ This just redirects to the link below >>> http://sikuli.csail.mit.edu/ I also did this >> This link is broken! Worked for me both yesterday and now. From daniel at stutzbachenterprises.com Mon Jan 25 14:28:10 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 25 Jan 2010 13:28:10 -0600 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> Message-ID: On Mon, Jan 25, 2010 at 12:24 PM, Steve Howell wrote: > Hi Daniel, I agree with what Raymond Hettinger says toward the top of > the PEP. Blist, while extremely useful, does seem to have to trade > off performance of common operations, notably "get item", in order to > get better performance for other operations (notably insert/delete). > Actually, the latest version of blist is competitive for "get item" and similar operations. See: http://stutzbachenterprises.com/performance-blist/item http://stutzbachenterprises.com/performance-blist/set-item http://stutzbachenterprises.com/performance-blist/lifo http://stutzbachenterprises.com/performance-blist/shuffle I added a flat cache of the leaf nodes, yielding O(1) get/set item operations whenever those operations dominate over insert/delete operations. The cache adds around 1.5% memory overhead. > My algorithm does exactly N pops and roughly N list accesses, so I > would be going from N*N + N to N + N log N if switched to blist. That > would be at least a theoretical gain over the current performance, but > if pop() were O(1), I could get the whole thing down to N time. > If I understand correctly, you feel strongly that a list.pop(0) that runs in O(n) time is "broken", but you're comfortable with a list.pop(1) that runs in O(n) time. Is that correct? How do you feel about a bisect.insort(list, item) that takes O(n) time? Different people are bound to have different opinions about which operations are most important and where lies the best tradeoff between different operations (as well as code complexity). I am not sure why you feel so strongly that particular spot is best. Obviously, I prefer a slightly different spot, but I also respect the core developers' choice. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From anm12345 at gmail.com Mon Jan 25 14:39:09 2010 From: anm12345 at gmail.com (AlexM) Date: Mon, 25 Jan 2010 11:39:09 -0800 (PST) Subject: Total maximal size of data References: <7s69efF2u0U1@mid.uni-berlin.de> Message-ID: <96b3f223-2f5f-4af1-b896-2d926986441f@g29g2000yqe.googlegroups.com> On Jan 25, 1:23?pm, "Diez B. Roggisch" wrote: > Am 25.01.10 20:05, schrieb Alexander Moibenko: > > > I have a simple question to which I could not find an answer. > > What is the total maximal size of list including size of its elements? > > I do not like to look into python source. > > But it would answer that question pretty fast. Because then you'd see > that all list-object-methods are defined in terms of Py_ssize_t, which > is an alias for ssize_t of your platform. 64bit that should be a 64bit long. > > Diez Then how do explain the program output? Alex. From gagsl-py2 at yahoo.com.ar Mon Jan 25 14:44:14 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 25 Jan 2010 16:44:14 -0300 Subject: start .pyo files with doubleclick on windows References: <4b5cb96a$0$23933$426a74cc@news.free.fr> <4b5ccb85$0$22008$426a74cc@news.free.fr> Message-ID: En Sun, 24 Jan 2010 19:36:53 -0300, News123 escribi?: > Hi Alf, > > > > > > Alf P. Steinbach wrote: >> * News123: >>> Hi, >>> >>> >>> I'd like to start .pyo files under windows with a double click. >> >> C:\> assoc .pyo >> .pyo=Python.CompiledFile >> >> C:\> ftype python.compiledfile >> python.compiledfile="C:\Program Files\cpython\python31\python.exe" "%1" >> %* >> >> C:\> _ >> >> >> Use ftype to change the association. > > Thanks a lot, I leared something new about Windows > > > What I did now is this: > > assoc .pyo=Python.CompiledOptimizedFile > ftype Python.CompiledOptimizedFile="C:\Python26\python.exe" -OO "%1" %* This looks like a bug (or two) - "python foo.pyo" should fail; a .pyo has a different magic number than a .pyc file. Importing a .pyo file fails in this case. A .pyo file should be run with "python -O foo.pyo" (or -OO) - on Windows, .pyo files should be associated with python -O by default, you should not need to do that by yourself. -- Gabriel Genellina From gagsl-py2 at yahoo.com.ar Mon Jan 25 14:47:57 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 25 Jan 2010 16:47:57 -0300 Subject: Default path for files References: <0neu27-csn.ln1@spamfence.net> Message-ID: En Sun, 24 Jan 2010 15:04:48 -0300, G?nther Dietrich escribi?: > Rotwang wrote: > >>> Check out http://docs.python.org/library/os.html and the function >>> chdir it is what you are looking for. >> >> Thank you. So would adding >> >> import os >> os.chdir() >> >> to site.py (or any other module which is automatically imported during >> initialisation) change the default location to every time I used >> Python? > > Don't change the library modules. It would catch you anytime when you > expect it least. > > See for the environment variable PYTHONSTARTUP and the associated > startup file. sitecustomize.py would be a better place. PYTHONSTARTUP is only used when running in interactive mode. Anyway, I'd do that explicitely on each script that requires it; after upgrading the Python version, or moving to another PC, those scripts would start failing... -- Gabriel Genellina From deets at nospam.web.de Mon Jan 25 15:03:51 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Mon, 25 Jan 2010 21:03:51 +0100 Subject: Total maximal size of data In-Reply-To: <96b3f223-2f5f-4af1-b896-2d926986441f@g29g2000yqe.googlegroups.com> References: <7s69efF2u0U1@mid.uni-berlin.de> <96b3f223-2f5f-4af1-b896-2d926986441f@g29g2000yqe.googlegroups.com> Message-ID: <7s6bp7Fgl6U1@mid.uni-berlin.de> Am 25.01.10 20:39, schrieb AlexM: > On Jan 25, 1:23 pm, "Diez B. Roggisch" wrote: >> Am 25.01.10 20:05, schrieb Alexander Moibenko: >> >>> I have a simple question to which I could not find an answer. >>> What is the total maximal size of list including size of its elements? >>> I do not like to look into python source. >> >> But it would answer that question pretty fast. Because then you'd see >> that all list-object-methods are defined in terms of Py_ssize_t, which >> is an alias for ssize_t of your platform. 64bit that should be a 64bit long. >> >> Diez > > Then how do explain the program output? What exactly? That after 3GB it ran out of memory? Because you don't have 4GB memory available for processes. Diez From tjreedy at udel.edu Mon Jan 25 15:07:09 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 25 Jan 2010 15:07:09 -0500 Subject: Total maximal size of data In-Reply-To: <4B5DEB78.10406@fnal.gov> References: <4B5DEB78.10406@fnal.gov> Message-ID: On 1/25/2010 2:05 PM, Alexander Moibenko wrote: > I have a simple question to which I could not find an answer. Because it has no finite answer > What is the total maximal size of list including size of its elements? In theory, unbounded. In practice, limited by the memory of the interpreter. The maximum # of elements depends on the interpreter. Each element can be a list whose maximum # of elements ..... and recursively so on... Terry Jan Reedy From anm12345 at gmail.com Mon Jan 25 15:15:09 2010 From: anm12345 at gmail.com (AlexM) Date: Mon, 25 Jan 2010 12:15:09 -0800 (PST) Subject: Total maximal size of data References: <7s69efF2u0U1@mid.uni-berlin.de> <96b3f223-2f5f-4af1-b896-2d926986441f@g29g2000yqe.googlegroups.com> <7s6bp7Fgl6U1@mid.uni-berlin.de> Message-ID: On Jan 25, 2:03?pm, "Diez B. Roggisch" wrote: > Am 25.01.10 20:39, schrieb AlexM: > > > On Jan 25, 1:23 pm, "Diez B. Roggisch" ?wrote: > >> Am 25.01.10 20:05, schrieb Alexander Moibenko: > > >>> I have a simple question to which I could not find an answer. > >>> What is the total maximal size of list including size of its elements? > >>> I do not like to look into python source. > > >> But it would answer that question pretty fast. Because then you'd see > >> that all list-object-methods are defined in terms of Py_ssize_t, which > >> is an alias for ssize_t of your platform. 64bit that should be a 64bit long. > > >> Diez > > > Then how do explain the program output? > > What exactly? That after 3GB it ran out of memory? Because you don't > have 4GB memory available for processes. > > Diez Did you see my posting? .... Here is what I get on 32-bit architecture: cat /proc/meminfo MemTotal: 8309860 kB MemFree: 5964888 kB Buffers: 84396 kB Cached: 865644 kB SwapCached: 0 kB ..... I have more than 5G in memory not speaking of swap space. From anm12345 at gmail.com Mon Jan 25 15:21:48 2010 From: anm12345 at gmail.com (AlexM) Date: Mon, 25 Jan 2010 12:21:48 -0800 (PST) Subject: Total maximal size of data References: <4B5DEB78.10406@fnal.gov> Message-ID: <48f81b87-661e-4b3b-9dd6-3511bc84f607@36g2000yqu.googlegroups.com> On Jan 25, 2:07?pm, Terry Reedy wrote: > On 1/25/2010 2:05 PM, Alexander Moibenko wrote: > > > I have a simple question to which I could not find an answer. > > Because it has no finite answer > > > What is the total maximal size of list including size of its elements? > > In theory, unbounded. In practice, limited by the memory of the interpreter. > > The maximum # of elements depends on the interpreter. Each element can > be a list whose maximum # of elements ..... and recursively so on... > > Terry Jan Reedy I am not asking about maximum numbers of elements I am asking about total maximal size of list including size of its elements. In other words: if size of each list element is ELEMENT_SIZE and all elements have the same size what would be the maximal number of these elements in 32 - bit architecture? I see 3 GB, and wonder why? Why not 2 GB or not 4 GB? AlexM AlexM From alfps at start.no Mon Jan 25 15:37:51 2010 From: alfps at start.no (Alf P. Steinbach) Date: Mon, 25 Jan 2010 21:37:51 +0100 Subject: Total maximal size of data In-Reply-To: <48f81b87-661e-4b3b-9dd6-3511bc84f607@36g2000yqu.googlegroups.com> References: <4B5DEB78.10406@fnal.gov> <48f81b87-661e-4b3b-9dd6-3511bc84f607@36g2000yqu.googlegroups.com> Message-ID: * AlexM: > On Jan 25, 2:07 pm, Terry Reedy wrote: >> On 1/25/2010 2:05 PM, Alexander Moibenko wrote: >> >>> I have a simple question to which I could not find an answer. >> Because it has no finite answer >> >>> What is the total maximal size of list including size of its elements? >> In theory, unbounded. In practice, limited by the memory of the interpreter. >> >> The maximum # of elements depends on the interpreter. Each element can >> be a list whose maximum # of elements ..... and recursively so on... >> >> Terry Jan Reedy > > I am not asking about maximum numbers of elements I am asking about > total maximal size of list including size of its elements. In other > words: > if size of each list element is ELEMENT_SIZE and all elements have the > same size what would be the maximal number of these elements in 32 - > bit architecture? > I see 3 GB, and wonder why? Why not 2 GB or not 4 GB? At a guess you were running this in 32-bit Windows. By default it reserves the upper two gig of address space for mapping system DLLs. It can be configured to use just 1 gig for that, and it seems like your system is, or you're using some other system with that kind of behavior, or, it's just arbitrary... Cheers & hth., - Alf (by what mechanism do socks disappear from the washer?) From deets at nospam.web.de Mon Jan 25 15:42:35 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Mon, 25 Jan 2010 21:42:35 +0100 Subject: Total maximal size of data In-Reply-To: References: <7s69efF2u0U1@mid.uni-berlin.de> <96b3f223-2f5f-4af1-b896-2d926986441f@g29g2000yqe.googlegroups.com> <7s6bp7Fgl6U1@mid.uni-berlin.de> Message-ID: <7s6e1sFu8nU1@mid.uni-berlin.de> Am 25.01.10 21:15, schrieb AlexM: > On Jan 25, 2:03 pm, "Diez B. Roggisch" wrote: >> Am 25.01.10 20:39, schrieb AlexM: >> >>> On Jan 25, 1:23 pm, "Diez B. Roggisch" wrote: >>>> Am 25.01.10 20:05, schrieb Alexander Moibenko: >> >>>>> I have a simple question to which I could not find an answer. >>>>> What is the total maximal size of list including size of its elements? >>>>> I do not like to look into python source. >> >>>> But it would answer that question pretty fast. Because then you'd see >>>> that all list-object-methods are defined in terms of Py_ssize_t, which >>>> is an alias for ssize_t of your platform. 64bit that should be a 64bit long. >> >>>> Diez >> >>> Then how do explain the program output? >> >> What exactly? That after 3GB it ran out of memory? Because you don't >> have 4GB memory available for processes. >> >> Diez > > Did you see my posting? > .... > Here is what I get on 32-bit architecture: > cat /proc/meminfo > MemTotal: 8309860 kB > MemFree: 5964888 kB > Buffers: 84396 kB > Cached: 865644 kB > SwapCached: 0 kB > ..... > > I have more than 5G in memory not speaking of swap space. Yes, I saw your posting. 32Bit is 32Bit. Do you know about PAE? http://de.wikipedia.org/wiki/Physical_Address_Extension Just because the system can deal with more overall memory - one process can't get more than 4 GB (or even less, through re-mapped memory). Except it uses specific APIs like the old hi-mem-stuff under DOS. Diez From anm12345 at gmail.com Mon Jan 25 15:49:05 2010 From: anm12345 at gmail.com (AlexM) Date: Mon, 25 Jan 2010 12:49:05 -0800 (PST) Subject: Total maximal size of data References: <4B5DEB78.10406@fnal.gov> <48f81b87-661e-4b3b-9dd6-3511bc84f607@36g2000yqu.googlegroups.com> Message-ID: <662d9b37-ac93-4360-81d9-1562b0235836@o28g2000yqh.googlegroups.com> On Jan 25, 2:37?pm, "Alf P. Steinbach" wrote: > * AlexM: > > > > > On Jan 25, 2:07 pm, Terry Reedy wrote: > >> On 1/25/2010 2:05 PM, Alexander Moibenko wrote: > > >>> I have a simple question to which I could not find an answer. > >> Because it has no finite answer > > >>> What is the total maximal size of list including size of its elements? > >> In theory, unbounded. In practice, limited by the memory of the interpreter. > > >> The maximum # of elements depends on the interpreter. Each element can > >> be a list whose maximum # of elements ..... and recursively so on... > > >> Terry Jan Reedy > > > I am not asking about maximum numbers of elements I am asking about > > total maximal size of list including size of its elements. In other > > words: > > if size of each list element is ELEMENT_SIZE and all elements have the > > same size what would be the maximal number of these elements in 32 - > > bit architecture? > > I see 3 GB, and wonder why? Why not 2 GB or not 4 GB? > > At a guess you were running this in 32-bit Windows. By default it reserves the > upper two gig of address space for mapping system DLLs. It can be configured to > use just 1 gig for that, and it seems like your system is, or you're using some > other system with that kind of behavior, or, it's just arbitrary... > > Cheers & hth., > > - Alf (by what mechanism do socks disappear from the washer?) No, it is 32-bit Linux. Alex From hansmu at xs4all.nl Mon Jan 25 15:55:03 2010 From: hansmu at xs4all.nl (Hans Mulder) Date: Mon, 25 Jan 2010 21:55:03 +0100 Subject: Terminal application with non-standard print In-Reply-To: References: Message-ID: <4b5e059f$0$22944$e4fe514c@news.xs4all.nl> Grant Edwards wrote: > On 2010-01-24, R?mi wrote: > >> I would like to do a Python application that prints data to stdout, but >> not the common way. I do not want the lines to be printed after each >> other, but the old lines to be replaced with the new ones, like wget >> does it for example (when downloading a file you can see the percentage >> increasing on a same line). > > sys.stdout.write("Here's the first line") > time.sleep(1) > sys.stdout.write("\rAnd this line replaces it.") That does not work on my system, because sys.stdout is line buffered. This causes both strings to be written when sys.stdout is closed because Python is shutting down. This works better: import sys, time sys.stdout.write("Here's the first line") sys.stdout.flush() time.sleep(1) sys.stdout.write("\rAnd this line replaces it.") sys.stdout.flush() Hope this helps, -- HansM From deets at nospam.web.de Mon Jan 25 15:56:58 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Mon, 25 Jan 2010 21:56:58 +0100 Subject: Total maximal size of data In-Reply-To: <662d9b37-ac93-4360-81d9-1562b0235836@o28g2000yqh.googlegroups.com> References: <4B5DEB78.10406@fnal.gov> <48f81b87-661e-4b3b-9dd6-3511bc84f607@36g2000yqu.googlegroups.com> <662d9b37-ac93-4360-81d9-1562b0235836@o28g2000yqh.googlegroups.com> Message-ID: <7s6esqF3q7U1@mid.uni-berlin.de> Am 25.01.10 21:49, schrieb AlexM: > On Jan 25, 2:37 pm, "Alf P. Steinbach" wrote: >> * AlexM: >> >> >> >>> On Jan 25, 2:07 pm, Terry Reedy wrote: >>>> On 1/25/2010 2:05 PM, Alexander Moibenko wrote: >> >>>>> I have a simple question to which I could not find an answer. >>>> Because it has no finite answer >> >>>>> What is the total maximal size of list including size of its elements? >>>> In theory, unbounded. In practice, limited by the memory of the interpreter. >> >>>> The maximum # of elements depends on the interpreter. Each element can >>>> be a list whose maximum # of elements ..... and recursively so on... >> >>>> Terry Jan Reedy >> >>> I am not asking about maximum numbers of elements I am asking about >>> total maximal size of list including size of its elements. In other >>> words: >>> if size of each list element is ELEMENT_SIZE and all elements have the >>> same size what would be the maximal number of these elements in 32 - >>> bit architecture? >>> I see 3 GB, and wonder why? Why not 2 GB or not 4 GB? >> >> At a guess you were running this in 32-bit Windows. By default it reserves the >> upper two gig of address space for mapping system DLLs. It can be configured to >> use just 1 gig for that, and it seems like your system is, or you're using some >> other system with that kind of behavior, or, it's just arbitrary... >> >> Cheers& hth., >> >> - Alf (by what mechanism do socks disappear from the washer?) > > No, it is 32-bit Linux. > Alex I already answered that (as did Alf, the principle applies for both OSs) - kernel memory space is mapped into the address-space, reducing it by 1 or 2 GB. Diez From zuo at chopin.edu.pl Mon Jan 25 15:57:48 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Mon, 25 Jan 2010 21:57:48 +0100 Subject: how can i know if a python object have a attribute such as 'attr1'? In-Reply-To: References: Message-ID: 24-01-2010, 00:38:29 Terry Reedy wrote: > On 1/23/2010 10:56 AM, Arnaud Delobelle wrote: >> thinke365 writes: >> >>> for example, i may define a python class: >>> class A: >>> def sayHello(): >>> print 'hello' >>> >>> a = A() >>> a.attr1 = 'hello' >>> a.attr2 = 'bb' >>> >>> b = A() >>> a.attr2 = 'aa' >>> >>> how can i know whether an object have an attribute named attr1? >> >> hasattr(a, 'attr1') > > or > try: a.attr1 > except AttributeError: pass or -- if you are interested only in attributes contained by attributes dict of this particular object (and no in attributes of its type, base types nor attributes calculated on-demand by __getattr__/__getattribute__ methods) -- you can check its __dict__ -- * using vars(), e.g.: if 'attr1' in vars(a)... * or directly (less elegant?), e.g.: if 'attr1' in a.__dict__... But please remember that it doesn't work for instances of types with __slots__ defined (see: http://docs.python.org/reference/datamodel.html#slots). Regards, *j -- Jan Kaliszewski (zuo) From no.email at nospam.invalid Mon Jan 25 16:00:02 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 25 Jan 2010 13:00:02 -0800 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> <7xpr4ylk9d.fsf@ruckus.brouhaha.com> Message-ID: <7xy6jlnbn1.fsf@ruckus.brouhaha.com> Steve Howell writes: > These are the reasons I am not using deque: Thanks for these. Now we are getting somewhere. > 1) I want to use native lists, so that downstream methods can use > them as lists. It sounds like that could be fixed by making the deque API a proper superset of the list API. > 2) Lists are faster for accessing elements. It sounds like that could be fixed by optimizing deque somewhat. Also, have you profiled your application to show that accessing list elements is actually using a significant fraction of its runtime and that it would be slowed down noticably by deque? If not, it's a red herring. > 3) I want to be able to insert elements into the middle of the list. I just checked, and was surprised to find that deque doesn't support this. I'd say go ahead and file a feature request to add it to deque. > 4) I have no need for rotating elements. That's unpersuasive since you're advocating adding a feature to list that many others have no need for. > Adding a word or two to a list is an O(1) addition to a data structure > that takes O(N) memory to begin with. Yes, as mentioned, additive constants matter. > Another way of looking at it is that you would need to have 250 or so > lists in memory at the same time before the extra pointer was even > costing you kilobytes of memory. I've often run applications with millions of lists, maybe tens of millions. Of course it would be 100's of millions if the machines were big enough. > My consumer laptop has 3027908k of memory. I thought the idea of buying bigger machines was to solve bigger problems, not to solve the same problems more wastefully. From anm12345 at gmail.com Mon Jan 25 16:22:17 2010 From: anm12345 at gmail.com (AlexM) Date: Mon, 25 Jan 2010 13:22:17 -0800 (PST) Subject: Total maximal size of data References: <7s69efF2u0U1@mid.uni-berlin.de> <96b3f223-2f5f-4af1-b896-2d926986441f@g29g2000yqe.googlegroups.com> <7s6bp7Fgl6U1@mid.uni-berlin.de> <7s6e1sFu8nU1@mid.uni-berlin.de> Message-ID: <8255c8da-4028-42a7-a48b-03ea8fccf971@v25g2000yqk.googlegroups.com> On Jan 25, 2:42?pm, "Diez B. Roggisch" wrote: > Am 25.01.10 21:15, schrieb AlexM: > > > > > On Jan 25, 2:03 pm, "Diez B. Roggisch" ?wrote: > >> Am 25.01.10 20:39, schrieb AlexM: > > >>> On Jan 25, 1:23 pm, "Diez B. Roggisch" ? ?wrote: > >>>> Am 25.01.10 20:05, schrieb Alexander Moibenko: > > >>>>> I have a simple question to which I could not find an answer. > >>>>> What is the total maximal size of list including size of its elements? > >>>>> I do not like to look into python source. > > >>>> But it would answer that question pretty fast. Because then you'd see > >>>> that all list-object-methods are defined in terms of Py_ssize_t, which > >>>> is an alias for ssize_t of your platform. 64bit that should be a 64bit long. > > >>>> Diez > > >>> Then how do explain the program output? > > >> What exactly? That after 3GB it ran out of memory? Because you don't > >> have 4GB memory available for processes. > > >> Diez > > > Did you see my posting? > > .... > > Here is what I get on 32-bit architecture: > > cat /proc/meminfo > > MemTotal: ? ? ?8309860 kB > > MemFree: ? ? ? 5964888 kB > > Buffers: ? ? ? ? 84396 kB > > Cached: ? ? ? ? 865644 kB > > SwapCached: ? ? ? ? ?0 kB > > ..... > > > I have more than 5G in memory not speaking of swap space. > > Yes, I saw your posting. 32Bit is 32Bit. Do you know about PAE? > > ? ?http://de.wikipedia.org/wiki/Physical_Address_Extension > > Just because the system can deal with more overall memory - one process > can't get more than 4 GB (or even less, through re-mapped memory). > Except it uses specific APIs like the old hi-mem-stuff under DOS. > > Diez Yes, I do. Good catch! I have PAE enabled, but I guess I have compiled python without extended memory. So I was looking in the wrong place. Thanks! AlexM From deets at nospam.web.de Mon Jan 25 16:31:49 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Mon, 25 Jan 2010 22:31:49 +0100 Subject: Total maximal size of data In-Reply-To: <8255c8da-4028-42a7-a48b-03ea8fccf971@v25g2000yqk.googlegroups.com> References: <7s69efF2u0U1@mid.uni-berlin.de> <96b3f223-2f5f-4af1-b896-2d926986441f@g29g2000yqe.googlegroups.com> <7s6bp7Fgl6U1@mid.uni-berlin.de> <7s6e1sFu8nU1@mid.uni-berlin.de> <8255c8da-4028-42a7-a48b-03ea8fccf971@v25g2000yqk.googlegroups.com> Message-ID: <7s6gu5FgdfU1@mid.uni-berlin.de> Am 25.01.10 22:22, schrieb AlexM: > On Jan 25, 2:42 pm, "Diez B. Roggisch" wrote: >> Am 25.01.10 21:15, schrieb AlexM: >> >> >> >>> On Jan 25, 2:03 pm, "Diez B. Roggisch" wrote: >>>> Am 25.01.10 20:39, schrieb AlexM: >> >>>>> On Jan 25, 1:23 pm, "Diez B. Roggisch" wrote: >>>>>> Am 25.01.10 20:05, schrieb Alexander Moibenko: >> >>>>>>> I have a simple question to which I could not find an answer. >>>>>>> What is the total maximal size of list including size of its elements? >>>>>>> I do not like to look into python source. >> >>>>>> But it would answer that question pretty fast. Because then you'd see >>>>>> that all list-object-methods are defined in terms of Py_ssize_t, which >>>>>> is an alias for ssize_t of your platform. 64bit that should be a 64bit long. >> >>>>>> Diez >> >>>>> Then how do explain the program output? >> >>>> What exactly? That after 3GB it ran out of memory? Because you don't >>>> have 4GB memory available for processes. >> >>>> Diez >> >>> Did you see my posting? >>> .... >>> Here is what I get on 32-bit architecture: >>> cat /proc/meminfo >>> MemTotal: 8309860 kB >>> MemFree: 5964888 kB >>> Buffers: 84396 kB >>> Cached: 865644 kB >>> SwapCached: 0 kB >>> ..... >> >>> I have more than 5G in memory not speaking of swap space. >> >> Yes, I saw your posting. 32Bit is 32Bit. Do you know about PAE? >> >> http://de.wikipedia.org/wiki/Physical_Address_Extension >> >> Just because the system can deal with more overall memory - one process >> can't get more than 4 GB (or even less, through re-mapped memory). >> Except it uses specific APIs like the old hi-mem-stuff under DOS. >> >> Diez > > Yes, I do. Good catch! I have PAE enabled, but I guess I have compiled > python without extended memory. So I was looking in the wrong place. You can't compile it with PAE. It's an extension that doesn't make sense in a general purpose language. It is used by Databases or some such, that can hold large structures in memory that don't need random access, but can cope with windowing. Diez From arnodel at googlemail.com Mon Jan 25 16:32:10 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 25 Jan 2010 21:32:10 +0000 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> Message-ID: Steve Howell writes: [...] > My algorithm does exactly N pops and roughly N list accesses, so I > would be going from N*N + N to N + N log N if switched to blist. Can you post your algorithm? It would be interesting to have a concrete use case to base this discussion on. -- Arnaud From jeremy+complangpythonannounce at jeremysanders.net Mon Jan 25 16:42:52 2010 From: jeremy+complangpythonannounce at jeremysanders.net (Jeremy Sanders) Date: Mon, 25 Jan 2010 21:42:52 +0000 Subject: ANN: Veusz 1.6 Message-ID: Veusz 1.6 --------- Velvet Ember Under Sky Zenith ----------------------------- http://home.gna.org/veusz/ Veusz is Copyright (C) 2003-2010 Jeremy Sanders Licenced under the GPL (version 2 or greater). Veusz is a Qt4 based scientific plotting package. It is written in Python, using PyQt4 for display and user-interfaces, and numpy for handling the numeric data. Veusz is designed to produce publication-ready Postscript/PDF/SVG output. The user interface aims to be simple, consistent and powerful. Veusz provides a GUI, command line, embedding and scripting interface (based on Python) to its plotting facilities. It also allows for manipulation and editing of datasets. Data can be captured from external sources such as internet sockets or other programs. Changes in 1.6: * User defined constants, functions or external Python imports can be defined for use when evaluating expressions. * Import descriptor is much more tolerant of syntax, e.g. "x,+- y,+,-" can now be specified as "x +- y + -". * New SVG export (PyQt >= 4.6). Supports clipping and exports text as paths for full WYSIWYG. * Dataset names can now contain any character except "`". Names containing non-alphanumeric characters can be quoted in expressions `like so`*1.23 * Widget names can contain any character except "/" * A transparency dataset can be provided to specify the per-pixel transparency of the image widget. * A polygon widget has been added. * There is a new option to place axis ticks outside the plot (outer ticks setting on axis widget) * Several new line styles have been added. * Several new plotting markers have been added. * The capture dialog can optionally retain the last N values captured. Minor changes: * Use of flat cap line style for plotting error bars for exactness. * Add fixes for saving imported unicode text. * Fix image colors for big endian systems (e.g. Mac PPC). * Add boxfill error bar style, plotting errors as filled boxes. * Positive and negative error bars are forced to have the correct sign. Features of package: * X-Y plots (with errorbars) * Line and function plots * Contour plots * Images (with colour mappings and colorbars) * Stepped plots (for histograms) * Bar graphs * Plotting dates * Fitting functions to data * Stacked plots and arrays of plots * Plot keys * Plot labels * Shapes and arrows on plots * LaTeX-like formatting for text * EPS/PDF/PNG/SVG/EMF export * Scripting interface * Dataset creation/manipulation * Embed Veusz within other programs * Text, CSV and FITS importing * Data can be captured from external sources Requirements for source install: Python (2.4 or greater required) http://www.python.org/ Qt >= 4.3 (free edition) http://www.trolltech.com/products/qt/ PyQt >= 4.3 (SIP is required to be installed first) http://www.riverbankcomputing.co.uk/pyqt/ http://www.riverbankcomputing.co.uk/sip/ numpy >= 1.0 http://numpy.scipy.org/ Optional: Microsoft Core Fonts (recommended for nice output) http://corefonts.sourceforge.net/ PyFITS >= 1.1 (optional for FITS import) http://www.stsci.edu/resources/software_hardware/pyfits pyemf >= 2.0.0 (optional for EMF export) http://pyemf.sourceforge.net/ For EMF and better SVG export, PyQt >= 4.6 or better is required, to fix a bug in the C++ wrapping For documentation on using Veusz, see the "Documents" directory. The manual is in PDF, HTML and text format (generated from docbook). The examples are also useful documentation. Issues with the current version: * Due to Qt, hatched regions sometimes look rather poor when exported to PostScript, PDF or SVG. * Due to a bug in Qt, some long lines, or using log scales, can lead to very slow plot times under X11. It is fixed by upgrading to Qt-4.5.1 (or using a binary). Switching off antialiasing in the options may help. If you enjoy using Veusz, I would love to hear from you. Please join the mailing lists at https://gna.org/mail/?group=veusz to discuss new features or if you'd like to contribute code. The latest code can always be found in the SVN repository. Jeremy Sanders From apt.shansen at gmail.com Mon Jan 25 16:43:30 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Mon, 25 Jan 2010 13:43:30 -0800 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> <7xpr4ylk9d.fsf@ruckus.brouhaha.com> Message-ID: <7a9c25c21001251343q7d1ee864oafcb0e58e1c3b71e@mail.gmail.com> On Mon, Jan 25, 2010 at 9:31 AM, Steve Howell wrote: > Another way of looking at it is that you would need to have 250 or so > lists in memory at the same time before the extra pointer was even > costing you kilobytes of memory. My consumer laptop has 3027908k of > memory. > Umm, I think the issue here is that some people have use-cases which are talking of number of lists whole orders of magnitude higher then you're talking about here. In your program, maybe you only count the number of lists in the hundreds, and so a few extra words wouldn't matter. I have applications that have hundreds of thousands to millions of lists in memory-- and which have to be managed somewhat carefully to avoid the 32-bit memory allocation limit not smacking them (64-bit python isn't an option for me presently). I've never had an algorithm which needed to pop off the top of a list that I couldn't with utter triviality simply operate in the reverse. If Python's gonna get more memory hungry, I'd like to see how it benefits me in some way. I mean, Unladen Swallow is talking about boosting Python's memory need for the JIT, but I'm getting distinct performance improvements out of that. That sounds like a fair trade. You want Python to eat up a few more of my megs that I'd rather put to use elsewhere, because... you don't want to just reverse your algorithm to treat the FIFO as a LILO? Sure, I can break my program up to run in separate processes and double how much data I can have at once, with some IPC overhead. And if I got something out of it, I'd be happy to! Or you can alter your algorithm. Why must I be the one to change? :) --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From anm12345 at gmail.com Mon Jan 25 16:46:17 2010 From: anm12345 at gmail.com (AlexM) Date: Mon, 25 Jan 2010 13:46:17 -0800 (PST) Subject: Total maximal size of data References: <7s69efF2u0U1@mid.uni-berlin.de> <96b3f223-2f5f-4af1-b896-2d926986441f@g29g2000yqe.googlegroups.com> <7s6bp7Fgl6U1@mid.uni-berlin.de> <7s6e1sFu8nU1@mid.uni-berlin.de> <8255c8da-4028-42a7-a48b-03ea8fccf971@v25g2000yqk.googlegroups.com> <7s6gu5FgdfU1@mid.uni-berlin.de> Message-ID: <9fb9ce60-7d0e-4990-a7ca-b23751bdaabe@f12g2000yqn.googlegroups.com> On Jan 25, 3:31?pm, "Diez B. Roggisch" wrote: > Am 25.01.10 22:22, schrieb AlexM: > > > > > On Jan 25, 2:42 pm, "Diez B. Roggisch" ?wrote: > >> Am 25.01.10 21:15, schrieb AlexM: > > >>> On Jan 25, 2:03 pm, "Diez B. Roggisch" ? ?wrote: > >>>> Am 25.01.10 20:39, schrieb AlexM: > > >>>>> On Jan 25, 1:23 pm, "Diez B. Roggisch" ? ? ?wrote: > >>>>>> Am 25.01.10 20:05, schrieb Alexander Moibenko: > > >>>>>>> I have a simple question to which I could not find an answer. > >>>>>>> What is the total maximal size of list including size of its elements? > >>>>>>> I do not like to look into python source. > > >>>>>> But it would answer that question pretty fast. Because then you'd see > >>>>>> that all list-object-methods are defined in terms of Py_ssize_t, which > >>>>>> is an alias for ssize_t of your platform. 64bit that should be a 64bit long. > > >>>>>> Diez > > >>>>> Then how do explain the program output? > > >>>> What exactly? That after 3GB it ran out of memory? Because you don't > >>>> have 4GB memory available for processes. > > >>>> Diez > > >>> Did you see my posting? > >>> .... > >>> Here is what I get on 32-bit architecture: > >>> cat /proc/meminfo > >>> MemTotal: ? ? ?8309860 kB > >>> MemFree: ? ? ? 5964888 kB > >>> Buffers: ? ? ? ? 84396 kB > >>> Cached: ? ? ? ? 865644 kB > >>> SwapCached: ? ? ? ? ?0 kB > >>> ..... > > >>> I have more than 5G in memory not speaking of swap space. > > >> Yes, I saw your posting. 32Bit is 32Bit. Do you know about PAE? > > >> ? ?http://de.wikipedia.org/wiki/Physical_Address_Extension > > >> Just because the system can deal with more overall memory - one process > >> can't get more than 4 GB (or even less, through re-mapped memory). > >> Except it uses specific APIs like the old hi-mem-stuff under DOS. > > >> Diez > > > Yes, I do. Good catch! I have PAE enabled, but I guess I have compiled > > python without extended memory. So I was looking in the wrong place. > > You can't compile it with PAE. It's an extension that doesn't make sense > in a general purpose language. It is used by Databases or some such, > that can hold large structures in memory that don't need random access, > but can cope with windowing. > > Diez Well, there actually is a way of building programs that may use more than 4GB of memory on 32 machines for Linux with higmem kernels, but I guess this would not work for python. I'll just switch to 64-bit architecture. Thanks again. AlexM From peteRE at MpeteOzilla.Vco.ukE Mon Jan 25 17:04:18 2010 From: peteRE at MpeteOzilla.Vco.ukE (Peter Chant) Date: Mon, 25 Jan 2010 22:04:18 +0000 Subject: Python, PIL and 16 bit per channel images Message-ID: <24h137-e99.ln1@phoenix.fire> Does anyone know whether PIL can handle 16 bit per channel RGB images? PyPNG site (http://packages.python.org/pypng/ca.html) states PIL uses 8 bits per channel internally. Thanks, Pete -- http://www.petezilla.co.uk From showell30 at yahoo.com Mon Jan 25 17:05:59 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 14:05:59 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> Message-ID: <7655fa8f-bdc1-4bfd-8bd0-f3bba2babc91@a6g2000yqm.googlegroups.com> On Jan 25, 1:32?pm, Arnaud Delobelle wrote: > Steve Howell writes: > > [...] > > > My algorithm does exactly N pops and roughly N list accesses, so I > > would be going from N*N + N to N + N log N if switched to blist. > > Can you post your algorithm? ?It would be interesting to have a concrete > use case to base this discussion on. > It is essentially this, in list_ass_slice: if (d < 0) { /* Delete -d items */ if (ilow == 0) { a->popped -= d; a->ob_item -= d * sizeof(PyObject *); list_resize(a, Py_SIZE(a)); } else { memmove(&item[ihigh+d], &item[ihigh], (Py_SIZE(a) - ihigh)*sizeof(PyObject *)); list_resize(a, Py_SIZE(a) + d); } item = a->ob_item; } I am still working through the memory management issues, but when I have a complete working patch, I will give more detail. From showell30 at yahoo.com Mon Jan 25 17:09:00 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 14:09:00 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> Message-ID: <7ea02185-0bc1-4f49-9931-0974d8a0db5c@a5g2000yqi.googlegroups.com> On Jan 25, 1:32?pm, Arnaud Delobelle wrote: > Steve Howell writes: > > [...] > > > My algorithm does exactly N pops and roughly N list accesses, so I > > would be going from N*N + N to N + N log N if switched to blist. > > Can you post your algorithm? ?It would be interesting to have a concrete > use case to base this discussion on. > I just realized you meant the Python code itself. It is here: https://bitbucket.org/showell/shpaml_website/src/tip/shpaml.py From deets at nospam.web.de Mon Jan 25 17:12:29 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Mon, 25 Jan 2010 23:12:29 +0100 Subject: Total maximal size of data In-Reply-To: <9fb9ce60-7d0e-4990-a7ca-b23751bdaabe@f12g2000yqn.googlegroups.com> References: <7s69efF2u0U1@mid.uni-berlin.de> <96b3f223-2f5f-4af1-b896-2d926986441f@g29g2000yqe.googlegroups.com> <7s6bp7Fgl6U1@mid.uni-berlin.de> <7s6e1sFu8nU1@mid.uni-berlin.de> <8255c8da-4028-42a7-a48b-03ea8fccf971@v25g2000yqk.googlegroups.com> <7s6gu5FgdfU1@mid.uni-berlin.de> <9fb9ce60-7d0e-4990-a7ca-b23751bdaabe@f12g2000yqn.googlegroups.com> Message-ID: <7s6jadFmfdU1@mid.uni-berlin.de> > > Well, there actually is a way of building programs that may use more > than 4GB of memory on 32 machines for Linux with higmem kernels, but I > guess this would not work for python. As I said, it's essentially paging: http://kerneltrap.org/node/2450 And it's not something you can just compile in, you need explicit code-support for it. Which python hasn't. And most other programs. So there is not a magic compile option. > I'll just switch to 64-bit architecture. That's the solution, yes :) Diez From invalid at invalid.invalid Mon Jan 25 17:20:54 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Mon, 25 Jan 2010 22:20:54 +0000 (UTC) Subject: Terminal application with non-standard print References: <4b5e059f$0$22944$e4fe514c@news.xs4all.nl> Message-ID: On 2010-01-25, Hans Mulder wrote: > Grant Edwards wrote: >> On 2010-01-24, R?mi wrote: >> >>> I would like to do a Python application that prints data to stdout, but >>> not the common way. I do not want the lines to be printed after each >>> other, but the old lines to be replaced with the new ones, like wget >>> does it for example (when downloading a file you can see the percentage >>> increasing on a same line). >> >> sys.stdout.write("Here's the first line") >> time.sleep(1) >> sys.stdout.write("\rAnd this line replaces it.") > > That does not work on my system, because sys.stdout is line buffered. That's correct of course. > This causes both strings to be written when sys.stdout is closed because > Python is shutting down. > > This works better: > > import sys, time > > sys.stdout.write("Here's the first line") > sys.stdout.flush() > time.sleep(1) > sys.stdout.write("\rAnd this line replaces it.") > sys.stdout.flush() Or you can tell Python to do unbuffered output: #!/usr/bin/python -u -- Grant Edwards grante Yow! I'm using my X-RAY at VISION to obtain a rare visi.com glimpse of the INNER WORKINGS of this POTATO!! From george.trojan at noaa.gov Mon Jan 25 17:27:19 2010 From: george.trojan at noaa.gov (George Trojan) Date: Mon, 25 Jan 2010 22:27:19 +0000 Subject: site.py confusion Message-ID: Inspired by the 'Default path for files' thread I tried to use sitecustomize in my code. What puzzles me is that the site.py's main() is not executed. My sitecustomize.py is def main(): print 'In Main()' main() and the test program is import site #site.main() print 'Hi' The output is $ python try.py Hi When I uncomment the site.main() line the output is $ python try.py In Main() Hi If I change import site to import sitecustomize the output is as above. What gives? Adding to the confusion, I found http://code.activestate.com/recipes/552729/ which contradicts http://docs.python.org/library/site.html George From half.italian at gmail.com Mon Jan 25 17:30:58 2010 From: half.italian at gmail.com (Sean DiZazzo) Date: Mon, 25 Jan 2010 14:30:58 -0800 (PST) Subject: Terminal application with non-standard print References: Message-ID: On Jan 24, 11:27?am, R?mi wrote: > Hello everyone, > > I would like to do a Python application that prints data to stdout, but > not the common way. I do not want the lines to be printed after each > other, but the old lines to be replaced with the new ones, like wget > does it for example (when downloading a file you can see the percentage > increasing on a same line). > > I looked into the curses module, but this seems adapted only to do a > whole application, and the terminal history is not visible anymore when > the application starts. > > Any ideas? > > Thanks, > > Remi You might want to take a look at the readline module. ~Sean From sccolbert at gmail.com Mon Jan 25 17:42:38 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Mon, 25 Jan 2010 17:42:38 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <7ea02185-0bc1-4f49-9931-0974d8a0db5c@a5g2000yqi.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> <7ea02185-0bc1-4f49-9931-0974d8a0db5c@a5g2000yqi.googlegroups.com> Message-ID: <7f014ea61001251442y430019d8qdee626cff1f01a7f@mail.gmail.com> On Mon, Jan 25, 2010 at 5:09 PM, Steve Howell wrote: > On Jan 25, 1:32 pm, Arnaud Delobelle wrote: > > Steve Howell writes: > > > > [...] > > > > > My algorithm does exactly N pops and roughly N list accesses, so I > > > would be going from N*N + N to N + N log N if switched to blist. > > > > Can you post your algorithm? It would be interesting to have a concrete > > use case to base this discussion on. > > > > I just realized you meant the Python code itself. It is here: > > https://bitbucket.org/showell/shpaml_website/src/tip/shpaml.py > > -- > http://mail.python.org/mailman/listinfo/python-list > looking at that code, i think you could solve your whole problem with a single called to reversed() (which is NOT the same as list.reverse()) -------------- next part -------------- An HTML attachment was scrubbed... URL: From sccolbert at gmail.com Mon Jan 25 17:57:07 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Mon, 25 Jan 2010 17:57:07 -0500 Subject: Python, PIL and 16 bit per channel images In-Reply-To: <24h137-e99.ln1@phoenix.fire> References: <24h137-e99.ln1@phoenix.fire> Message-ID: <7f014ea61001251457u33db2a6at981f55503065ef55@mail.gmail.com> On Mon, Jan 25, 2010 at 5:04 PM, Peter Chant wrote: > Does anyone know whether PIL can handle 16 bit per channel RGB images? > PyPNG site (http://packages.python.org/pypng/ca.html) states PIL uses 8 > bits > per channel internally. > > Thanks, > > Pete > > > -- > http://www.petezilla.co.uk > > -- > http://mail.python.org/mailman/listinfo/python-list > Mode The mode of an image defines the type and depth of a pixel in the image. The current release supports the following standard modes: - *1* (1-bit pixels, black and white, stored with one pixel per byte) - *L* (8-bit pixels, black and white) - *P* (8-bit pixels, mapped to any other mode using a colour palette) - *RGB* (3x8-bit pixels, true colour) - *RGBA* (4x8-bit pixels, true colour with transparency mask) - *CMYK* (4x8-bit pixels, colour separation) - *YCbCr* (3x8-bit pixels, colour video format) - *I* (32-bit signed integer pixels) - *F* (32-bit floating point pixels) http://www.pythonware.com/library/pil/handbook/concepts.htm -------------- next part -------------- An HTML attachment was scrubbed... URL: From showell30 at yahoo.com Mon Jan 25 18:00:48 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 15:00:48 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> <7xpr4ylk9d.fsf@ruckus.brouhaha.com> <7xy6jlnbn1.fsf@ruckus.brouhaha.com> Message-ID: <81319a79-cf25-4444-ab71-bfe982ae7fee@m16g2000yqc.googlegroups.com> On Jan 25, 1:00?pm, Paul Rubin wrote: > Steve Howell writes: > > These are the reasons I am not using deque: > > Thanks for these. ?Now we are getting somewhere. > > > ? 1) I want to use native lists, so that downstream methods can use > > them as lists. > > It sounds like that could be fixed by making the deque API a proper > superset of the list API. That is probably a good idea. > > ? 2) Lists are faster for accessing elements. > > It sounds like that could be fixed by optimizing deque somewhat. ?Also, > have you profiled your application to show that accessing list elements > is actually using a significant fraction of its runtime and that it > would be slowed down noticably by deque? ?If not, it's a red herring. I haven't profiled deque vs. list, but I think you are correct about pop() possibly being a red herring. It appears that the main bottleneck might still be the processing I do on each line of text, which in my cases is regexes. For really large lists, I suppose memmove() would eventually start to become a bottleneck, but it's brutally fast when it just moves a couple kilobytes of data around. > > ? 3) I want to be able to insert elements into the middle of the list. > > I just checked, and was surprised to find that deque doesn't support > this. ?I'd say go ahead and file a feature request to add it to deque. > It might be a good thing to add just for consistency sake. If somebody first implements an algorithm with lists, then discovers it has overhead relating to inserting/appending at the end of the list, then the more deque behaves like a list, the more easily they could switch over their code to deque. Not knowing much about deque's internals, I assume its performance for insert() would O(N) just like list, although maybe a tiny bit slower. > > ? 4) I have no need for rotating elements. > > That's unpersuasive since you're advocating adding a feature to list > that many others have no need for. ? > To be precise, I wasn't really advocating for a new feature but an internal optimization of a feature that already exists. > > Adding a word or two to a list is an O(1) addition to a data structure > > that takes O(N) memory to begin with. > > Yes, as mentioned, additive constants matter. > > > Another way of looking at it is that you would need to have 250 or so > > lists in memory at the same time before the extra pointer was even > > costing you kilobytes of memory. > > I've often run applications with millions of lists, maybe tens of > millions. ?Of course it would be 100's of millions if the machines were > big enough. > I bet even in your application, the amount of memory consumed by the PyListObjects themselves is greatly dwarfed by other objects, notably the list elements themselves, not to mention any dictionaries that your app uses. > > My consumer laptop has 3027908k of memory. > > I thought the idea of buying bigger machines was to solve bigger > problems, not to solve the same problems more wastefully. Well, I am not trying to solve problems wastefully here. CPU cycles are also scarce, so it seems wasteful to do an O(N) memmove that could be avoided by storing an extra pointer per list. I also think that encouraging the use of pop(0) would actually make many programs more memory efficient, in the sense that you can garbage collect list elements earlier. Thanks for your patience in responding to me, despite the needlessly abrasive tone of my earlier postings. I am coming around to this thinking: 1) Summarize all this discussion and my lessons learned in some kind of document. It does not have to be a PEP per se, but I could provide a useful service to the community by listing pros/cons/etc. 2) I would still advocate for removing the warning against list.pop (0) from the tutorial. I agree with Steven D'Aprano that docs really should avoid describing implementation details in many instances (although I do not know what he thinks about this particular case). I also think that the performance penalty for pop(0) is negligible for most medium-sized programs. For large-sized programs where you really want to swap in deque, I think most authors are beyond reading the tutorial and are looking elsewhere for insight on Python data structures. 3) I am gonna try to implement the patch anyway for my own edification. 4) I do think that there are ways that deque could be improved, but it is not high on my priority list. I will try to mention it in the PEP, though. From showell30 at yahoo.com Mon Jan 25 18:30:16 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 15:30:16 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> Message-ID: <55d2d213-7779-4344-ad79-d9142c8d2a3e@a5g2000yqi.googlegroups.com> On Jan 25, 1:32?pm, Arnaud Delobelle wrote: > Steve Howell writes: > > [...] > > > My algorithm does exactly N pops and roughly N list accesses, so I > > would be going from N*N + N to N + N log N if switched to blist. > > Can you post your algorithm? ?It would be interesting to have a concrete > use case to base this discussion on. > These are the profile results for an admittedly very large file (430,000 lines), which shows that pop() consumes more time than any other low level method. So pop() is not a total red herring. But I have to be honest and admit that I grossly overestimated the penalty for smaller files. Typical files are a couple hundred lines, and for that use case, pop()'s expense gets totally drowned out by regex handling. In other words, it's a lot cheaper to move a couple hundred pointers per list element pop than it is to apply a series of regexes to them, which shouldn't be surprising. ncalls tottime percall cumtime percall filename:lineno (function) 230001/1 149.508 0.001 222.432 222.432 /home/showell/workspace/ shpaml_website/shpaml.py:192(recurse) 429999 17.667 0.000 17.667 0.000 {method 'pop' of 'list' objects} 530000 8.428 0.000 14.125 0.000 /home/showell/workspace/ shpaml_website/shpaml.py:143(get_indented_block) 3700008 7.877 0.000 7.877 0.000 {built-in method match} 5410125/5410121 5.697 0.000 5.697 0.000 {len} 300000 3.938 0.000 22.286 0.000 /home/showell/workspace/ shpaml_website/shpaml.py:96(convert_line) 959999 3.847 0.000 6.759 0.000 /home/showell/workspace/ shpaml_website/shpaml.py:29(INDENT) 959999 3.717 0.000 12.547 0.000 /home/showell/workspace/ shpaml_website/shpaml.py:138(find_indentation) 370000 3.495 0.000 20.204 0.000 /home/showell/workspace/ shpaml_website/shpaml.py:109(apply_jquery) 370000 3.322 0.000 6.528 0.000 {built-in method sub} 1469999 2.575 0.000 2.575 0.000 {built-in method groups} As an aside, I am a little surprised by how often I call len() and that it also takes a large chunk of time, but that's my problem to fix. From jim.waddle at boeing.com Mon Jan 25 18:36:32 2010 From: jim.waddle at boeing.com (Waddle, Jim) Date: Mon, 25 Jan 2010 15:36:32 -0800 Subject: ctypes for AIX In-Reply-To: <50697b2c1001240630r33845a1bl7f94b40a4f0c4436@mail.gmail.com> References: <9C240725D2ABD949A80B46DD93AB107042E8D44101@XCH-NW-08V.nw.nos.boeing.com> <50697b2c1001240630r33845a1bl7f94b40a4f0c4436@mail.gmail.com> Message-ID: <9C240725D2ABD949A80B46DD93AB107042E8D44574@XCH-NW-08V.nw.nos.boeing.com> Chris, Thanks for responding to my email. I apologize for the remark about python only being developed for windows. I got the impression when I was looking at the ActivePython web site and saw that the version of python that they had available was not supported on very many unix systems. I should not make general statement based on only one web site. After reading your email I decided to see for myself what the issue was about compiling python on AIX 5.3. This is the error I saw the first time I tried to use ctypes. Python 2.4.3 (#1, Jul 17 2006, 20:00:23) [C] on aix5 Type "help", "copyright", "credits" or "license" for more information. >>> import ctypes Traceback (most recent call last): File "", line 1, in ? ImportError: No module named ctypes This version of python was downloaded and installed from ActivePython and when I checked their webpage it states that ctypes is not available on AIX. I then figured I would get a new copy of python and install it on AIX. I downloaded python.2.5.5c2 from http://www.python.org. I did the configure and make which posted many errors in the ctypes function which I guess is the reason that is does not get include in the final make. an example of the build error I get when doing the make is: xlc_r -q64 -DNDEBUG -O -I. -I/s/users/cz030a/xferjunk/python/Python-2.5.5c2/./Include -Ibuild/temp.aix-5.3-2.5/libffi/inclu de -Ibuild/temp.aix-5.3-2.5/libffi -I/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/libffi/src -I/s/users/c z030a/xferjunk/ots/python2.5/include -I. -IInclude -I./Include -I/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Include -I/ s/users/cz030a/xferjunk/python/Python-2.5.5c2 -c /s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.c - o build/temp.aix-5.3-2.5/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.o "/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.c", line 2820.31: 1506-068 (W) Operation between ty pes "void*" and "int(*)(void)" is not allowed. "/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.c", line 3363.28: 1506-280 (W) Function argument as signment between types "int(*)(void)" and "void*" is not allowed. "/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.c", line 4768.67: 1506-280 (W) Function argument as signment between types "void*" and "void*(*)(void*,const void*,unsigned long)" is not allowed. "/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.c", line 4769.66: 1506-280 (W) Function argument as signment between types "void*" and "void*(*)(void*,int,unsigned long)" is not allowed. I do not have sufficient knowledge to know how to fix this. I would think that this error somehow is related to compiling on aix. If you have any suggestions on how to correct this problem , I would appreciate it Jim Waddle KIT-D 425-785-5194 -----Original Message----- From: chris at rebertia.com [mailto:chris at rebertia.com] On Behalf Of Chris Rebert Sent: Sunday, January 24, 2010 7:31 AM To: Waddle, Jim Cc: python-list at python.org Subject: Re: ctypes for AIX On Sun, Jan 24, 2010 at 5:54 AM, Waddle, Jim wrote: > I need to use ctypes with python running on AIX. According to the ctypes readme, ctypes is based on libffi, which according to its website, supports AIX for PowerPC64. So, perhaps you could state what the actual error or problem you're encountering is? It is theoretically possible the ctypes-bundled libffi is either outdated or had the AIX-specific bits removed; I don't know, I'm not a CPython dev. > It appears that python is being developed mostly for windows. No, not really; your statement is especially ironic considering one of Python's primary areas of use is for web applications as part of a LAMP stack. > Is there a policy concerning getting functions like ctypes working on AIX. No idea. Someone will probably chime in though. Cheers, Chris -- http://blog.rebertia.com From showell30 at yahoo.com Mon Jan 25 18:40:29 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 15:40:29 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <7f014ea61001251442y430019d8qdee626cff1f01a7f@mail.gmail.com> Message-ID: <7965.79187.qm@web33505.mail.mud.yahoo.com> --- On Mon, 1/25/10, Chris Colbert wrote: > > looking at that code, i think you could solve > your whole problem with a single called to reversed() (which > is NOT the same as list.reverse())? > I do not think that's actually true. It does no good to pop elements off a copy of the list if there is still code that refers to the original list. So I think you really do want list.reverse(). The problem with reversing the lists is that it gets sliced and diced and passed around to other methods, one of which, html_block_tag, recursively calls back to the main method. So you could say that everybody just has to work with a reversed list, but in my mind, that would be just backward and overly complicated. I am not completely ruling out the approach, though. The idea of modelling the program essentially as a stack has some validity, and it probably would run faster. https://bitbucket.org/showell/shpaml_website/src/tip/shpaml.py From debatem1 at gmail.com Mon Jan 25 18:45:00 2010 From: debatem1 at gmail.com (geremy condra) Date: Mon, 25 Jan 2010 18:45:00 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> <2ffb4ab8-ae8f-4b69-b449-56d0232a404b@y23g2000yqm.googlegroups.com> Message-ID: On Sat, Jan 23, 2010 at 4:38 AM, Alf P. Steinbach wrote: > Hm, it would be nice if the Python docs offered complexity (time) > guarantees in general... > > Cheers, > > - Alf This would be a very welcome improvement IMHO- especially in collections. Geremy Condra From ethan at stoneleaf.us Mon Jan 25 19:25:31 2010 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 25 Jan 2010 16:25:31 -0800 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <44ccd731-4a98-4eec-a270-08f66b192d93@l19g2000yqb.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> <036b0836$0$1357$c3e8da3@news.astraweb.com> <44ccd731-4a98-4eec-a270-08f66b192d93@l19g2000yqb.googlegroups.com> Message-ID: <4B5E367B.9060705@stoneleaf.us> Steve Howell wrote: >> On Sat, 23 Jan 2010 09:57:04 -0500, Roy Smith wrote: >>> So, we're right back to my statement earlier in this thread that the >>> docs are deficient in that they describe behavior with no hint about >>> cost. Given that, it should be no surprise that users make incorrect >>> assumptions about cost. No hint? Looking at the below snippet of docs -- "not efficient" and "slow" sound like pretty good hints to me. > Bringing this thread full circle, does it make sense to strike this > passage from the tutorial?: > > ''' > It is also possible to use a list as a queue, where the first element > added is the first element retrieved (?first-in, first-out?); however, > lists are not efficient for this purpose. While appends and pops from > the end of list are fast, doing inserts or pops from the beginning of > a list is slow (because all of the other elements have to be shifted > by one). > ''' > > I think points #3 and #6 possibly apply. Regarding points #2 and #4, > the tutorial is at least not overly technical or specific; it just > explains the requirement to shift other elements one by one in simple > layman's terms. > I think the paragraph is fine. Instead of waiting for the (hundreds of?) posts wondering why making a FIFO queue from a list is so slow, and what's wrong with Python, etc, etc, it points out up front that yes you can, and here's why you don't want to. This does not strike me as too much knowledge. ~Ethan~ From alfps at start.no Mon Jan 25 20:13:18 2010 From: alfps at start.no (Alf P. Steinbach) Date: Tue, 26 Jan 2010 02:13:18 +0100 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> <036b0836$0$1357$c3e8da3@news.astraweb.com> <44ccd731-4a98-4eec-a270-08f66b192d93@l19g2000yqb.googlegroups.com> Message-ID: * Ethan Furman: > Steve Howell wrote: >>> On Sat, 23 Jan 2010 09:57:04 -0500, Roy Smith wrote: >>>> So, we're right back to my statement earlier in this thread that the >>>> docs are deficient in that they describe behavior with no hint about >>>> cost. Given that, it should be no surprise that users make incorrect >>>> assumptions about cost. > > No hint? Looking at the below snippet of docs -- "not efficient" and > "slow" sound like pretty good hints to me. > >> Bringing this thread full circle, does it make sense to strike this >> passage from the tutorial?: >> >> ''' >> It is also possible to use a list as a queue, where the first element >> added is the first element retrieved (?first-in, first-out?); however, >> lists are not efficient for this purpose. While appends and pops from >> the end of list are fast, doing inserts or pops from the beginning of >> a list is slow (because all of the other elements have to be shifted >> by one). >> ''' >> >> I think points #3 and #6 possibly apply. Regarding points #2 and #4, >> the tutorial is at least not overly technical or specific; it just >> explains the requirement to shift other elements one by one in simple >> layman's terms. >> > > I think the paragraph is fine. Instead of waiting for the (hundreds > of?) posts wondering why making a FIFO queue from a list is so slow, and > what's wrong with Python, etc, etc, it points out up front that yes you > can, and here's why you don't want to. This does not strike me as too > much knowledge. Is the tutorial regarded as part of the language specification? I understand that the standard library docs are part (e.g. 'object' is only described there), and that at least some PEPs are. Cheers, - Alf From malaclypse2 at gmail.com Mon Jan 25 20:58:00 2010 From: malaclypse2 at gmail.com (Jerry Hill) Date: Mon, 25 Jan 2010 20:58:00 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> <2ffb4ab8-ae8f-4b69-b449-56d0232a404b@y23g2000yqm.googlegroups.com> Message-ID: <16651e81001251758j6fc2c2edj550edc25bbee96f@mail.gmail.com> On Sat, Jan 23, 2010 at 4:38 AM, Alf P. Steinbach wrote: > Hm, it would be nice if > the Python docs offered complexity (time) guarantees in general... Last time it came up, I don't think there was any core developer interest in putting complexity guarantees in the Python Language Reference. Some folks did document the behavior of most of the common CPython containers though: http://wiki.python.org/moin/TimeComplexity -- Jerry From siddhartha.veedaluru at gmail.com Mon Jan 25 23:17:51 2010 From: siddhartha.veedaluru at gmail.com (siddhartha veedaluru) Date: Tue, 26 Jan 2010 09:47:51 +0530 Subject: TypeError not caught by except statement Message-ID: <424b71ec1001252017i64d28375oc3644433c66ae40b@mail.gmail.com> Hi, except not able to caught the TypeError exception occured in the below code log.info("refer",ret) in the try block throws a TypeError which is not caught . Also sometimes process is getting hanged. ------------------------------------------------------------------------------------------------------------------------------------------------ import logging log = logging.getLogger() fileName = strftime("%d-%b-%Y-", gmtime()) + str(int(time.time())) + "- Log.log" log = logging.getLogger() log.setLevel(logging.NOTSET) fh = logging.FileHandler(logFile) logFileLevel = logging.DEBUG fh.setLevel(logFileLevel) format_string = '%(process)d %(thread)d %(asctime)-15s %(levelname)-5s at %(filename)-15s in %(funcName)-10s at line %(lineno)-3d "%(message) s"' fh.setFormatter(logging.Formatter(format_string)) log.addHandler(fh) try: log.info("start") log.info("refer",ret) log.info("end") except TypeError: log.exception("Exception raised") ---------------------------------------------------------------------------------------------------------------------------------------------- OUTPUT message: Traceback (most recent call last): File "C:\Python26\lib\logging\__init__.py", line 768, in emit msg = self.format(record) File "C:\Python26\lib\logging\__init__.py", line 648, in format return fmt.format(record) File "C:\Python26\lib\logging\__init__.py", line 436, in format record.message = record.getMessage() File "C:\Python26\lib\logging\__init__.py", line 306, in getMessage msg = msg % self.args TypeError: not all arguments converted during string formatting -------------- next part -------------- An HTML attachment was scrubbed... URL: From ursusmaximus at gmail.com Mon Jan 25 23:25:47 2010 From: ursusmaximus at gmail.com (Ron) Date: Mon, 25 Jan 2010 20:25:47 -0800 (PST) Subject: Sikuli: the coolest Python project I have yet seen... References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> Message-ID: <5acc067c-f650-4e2f-90f0-ee73ed7382cc@a5g2000yqi.googlegroups.com> OK, here's an idea. I used to do screen scraping scripts and run them as CGI scripts with an HTMl user interface. Why not run Sikuli on Jython on a JVM running on my server, so that I can do my screen scraping with Sikuli? I can take user inputs by using CGI forms from a web client, process the requests using a Sikuli script on the server, and send the results back to the web client. This sounds like fun to me, and easier to highlight and capture the appropriate screen information on targeted web sites using Sikuli than to hand code location information or even using Beautiful Soup. From no.email at nospam.invalid Mon Jan 25 23:31:07 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 25 Jan 2010 20:31:07 -0800 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> <7xpr4ylk9d.fsf@ruckus.brouhaha.com> <7xy6jlnbn1.fsf@ruckus.brouhaha.com> <81319a79-cf25-4444-ab71-bfe982ae7fee@m16g2000yqc.googlegroups.com> Message-ID: <7xeildij1w.fsf@ruckus.brouhaha.com> Steve Howell writes: > I haven't profiled deque vs. list, but I think you are correct about > pop() possibly being a red herring.... > For really large lists, I suppose memmove() would eventually start to > become a bottleneck, but it's brutally fast when it just moves a > couple kilobytes of data around. One way to think of Python is as a scripting wrapper around a bunch of C functions, rather than as a full-fledged programming language. Viewed that way, list operations like pop(0) are essentially constant time unless the list is quite large. By that I mean you can implement classic structures like doubly-linked lists using Python tuples, but even though inserting into the middle of them is theoretically O(1), the memmove's of the native list operations will be much faster in practice. Programs dealing with large lists (more than a few thousand elements) are obviously different and if your program is using such large lists, you have to plan a little differently when writing the code. >> I've often run applications with millions of lists > I bet even in your application, the amount of memory consumed by the > PyListObjects themselves is greatly dwarfed by other objects, notably > the list elements themselves Such lists often would just one element or even be empty. For example, you might have a dictionary mapping names to addresses. Most people have just one address, but some might have no address, and a few might have more than one address, so you would have a list of addresses for each name. Of course the dictionary slots in that example would also use space. > Well, I am not trying to solve problems wastefully here. CPU cycles > are also scarce, so it seems wasteful to do an O(N) memmove that could > be avoided by storing an extra pointer per list. Realistically the CPython interpreter is so slow that the memmove is unnoticable, and Python (at least CPython) just isn't all that conductive to writing fast code. It makes up for this in programmer productivity for the many sorts of problems in which moderate speed is acceptable. > Thanks for your patience in responding to me, despite the needlessly > abrasive tone of my earlier postings. I wondered whether you might have come over from the Lisp newsgroups, which are pretty brutal. We try to be friendlier here (not that we're always successful). Anyway, welcome. > 1) Summarize all this discussion and my lessons learned in some kind > of document. It does not have to be a PEP per se, but I could provide > a useful service to the community by listing pros/cons/etc. I suppose that can't hurt, but there are probably other areas (multicore parallelism is a perennial one) of much higher community interest. http://wiki.python.org/moin/ is probably a good place to put such a document. > 2) I would still advocate for removing the warning against list.pop > (0) from the tutorial. I agree with Steven D'Aprano that docs really > should avoid describing implementation details in many instances On general principles I agree with Alex Stepanov that the running time of a function should be part of its interface (nobody wants to use a stack of popping an element takes quadratic time) and therefore should be stated in the docs. Python just has a weird incongruence between the interpreter layer and the C layer, combined with a library well-evolved for everyday problem sizes, so the traditional asymptotic approach to algorithm selection often doesn't give the best practical choice. I don't feel like looking up what the tutorial says about pop(0), but if it just warns against it without qualification, it should probably be updated. From tekion at gmail.com Mon Jan 25 23:31:38 2010 From: tekion at gmail.com (tekion) Date: Mon, 25 Jan 2010 20:31:38 -0800 (PST) Subject: easy_install error ... References: <5e9c94df-531f-4725-bcd5-87c603d8df1c@o26g2000vbd.googlegroups.com> <2d3c4b85-c0fe-4f65-94ea-d246acdd0345@d30g2000vbl.googlegroups.com> Message-ID: FYI, I figured out what I was doing wrong. After reading the setuptools docs, I noticed took out the quotes around the package name and it works, see details below: python setup.py easy_install -m docutils==0.4 running easy_install Searching for docutils==0.4 Best match: docutils 0.4 Processing docutils-0.4-py2.6.egg Removing docutils 0.4 from easy-install.pth file Installing rst2html.py script to C:\Python26\Scripts Installing rst2latex.py script to C:\Python26\Scripts Installing rst2newlatex.py script to C:\Python26\Scripts Installing rst2pseudoxml.py script to C:\Python26\Scripts Installing rst2s5.py script to C:\Python26\Scripts Installing rst2xml.py script to C:\Python26\Scripts Using c:\python26\lib\site-packages\docutils-0.4-py2.6.egg Because this distribution was installed --multi-version, before you can import modules from this package in an application, you will need to 'import pkg_resources' and then use a 'require()' call similar to one of these examples, in order to select the desired version: pkg_resources.require("docutils") # latest installed version pkg_resources.require("docutils==0.4") # this exact version pkg_resources.require("docutils>=0.4") # this version or higher Processing dependencies for docutils==0.4 Finished processing dependencies for docutils==0.4 I hope this helps other having the similar issue as me. From showell30 at yahoo.com Tue Jan 26 00:00:30 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 21:00:30 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> Message-ID: <0b8ed84f-0a42-42c5-a1ba-e70ed048f39d@k19g2000yqc.googlegroups.com> On Jan 24, 11:28?am, a... at pythoncraft.com (Aahz) wrote: > In article , > Steve Howell ? wrote: > > > > >Even with realloc()'s brokenness, you could improve pop(0) in a way > >that does not impact list access at all, and the patch would not change > >the time complexity of any operation; it would just add negligible > >extract bookkeeping within list_resize() and a few other places. > > Again, your responsibility is to provide a patch and a spectrum of > benchmarking tests to prove it. ?Then you would still have to deal with > the objection that extensions use the list internals -- that might be an > okay sell given the effort otherwise required to port extensions to > Python 3, but that's not the way to bet. > Ok, I just submitted a patch to python-dev that illustrates a 100x speedup on an admittedly artificial program. It still has a long way to go, but it demonstrates proof of concept. I'm done for the day, but tomorrow I will try to polish it up and improve it, even if its doomed for rejection. Apologies to all I have offended in this thread. I frankly found some of the pushback to be a bit hasty and disrespectful, but I certainly overreacted to some of the criticism. And now I'm in the awkward position of asking the people I offended to help me with the patch. If anybody can offer me a hand in understanding some of CPython's internals, particularly with regard to memory management, it would be greatly appreciated. (Sorry I don't have a link to the python-dev posting; it is not showing up in the archives yet for some reason.) From showell30 at yahoo.com Tue Jan 26 00:13:00 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 21:13:00 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> <7xpr4ylk9d.fsf@ruckus.brouhaha.com> <7xy6jlnbn1.fsf@ruckus.brouhaha.com> <81319a79-cf25-4444-ab71-bfe982ae7fee@m16g2000yqc.googlegroups.com> <7xeildij1w.fsf@ruckus.brouhaha.com> Message-ID: On Jan 25, 8:31?pm, Paul Rubin wrote: > Steve Howell writes: > > I haven't profiled deque vs. list, but I think you are correct about > > pop() possibly being a red herring.... > > For really large lists, I suppose memmove() would eventually start to > > become a bottleneck, but it's brutally fast when it just moves a > > couple kilobytes of data around. > > One way to think of Python is as a scripting wrapper around a bunch of C > functions, rather than as a full-fledged programming language. ?Viewed > that way, list operations like pop(0) are essentially constant time > unless the list is quite large. ?By that I mean you can implement > classic structures like doubly-linked lists using Python tuples, but > even though inserting into the middle of them is theoretically O(1), the > memmove's of the native list operations will be much faster in practice. > Programs dealing with large lists (more than a few thousand elements) > are obviously different and if your program is using such large lists, > you have to plan a little differently when writing the code. Thanks. That is a good way of looking at. > > Realistically the CPython interpreter is so slow that the memmove is > unnoticable, and Python (at least CPython) just isn't all that > conductive to writing fast code. ?It makes up for this in programmer > productivity for the many sorts of problems in which moderate speed > is acceptable. > Definitely, and moderate speed is enough in a surprisingly large number of applications. > > Thanks for your patience in responding to me, despite the needlessly > > abrasive tone of my earlier postings. ? > > I wondered whether you might have come over from the Lisp newsgroups, > which are pretty brutal. ?We try to be friendlier here (not that we're > always successful). ?Anyway, welcome. > :) > > ? 1) Summarize all this discussion and my lessons learned in some kind > > of document. ?It does not have to be a PEP per se, but I could provide > > a useful service to the community by listing pros/cons/etc. > > I suppose that can't hurt, but there are probably other areas (multicore > parallelism is a perennial one) of much higher community interest. > > http://wiki.python.org/moin/is probably a good place to put such > a document. > Ok, that's where I'll start. > > ? 2) I would still advocate for removing the warning against list.pop > > (0) from the tutorial. ?I agree with Steven D'Aprano that docs really > > should avoid describing implementation details in many instances > > On general principles I agree with Alex Stepanov that the running time > of a function should be part of its interface (nobody wants to use a > stack of popping an element takes quadratic time) and therefore should > be stated in the docs. ?Python just has a weird incongruence between the > interpreter layer and the C layer, combined with a library well-evolved > for everyday problem sizes, so the traditional asymptotic approach to > algorithm selection often doesn't give the best practical choice. > > I don't feel like looking up what the tutorial says about pop(0), but if > it just warns against it without qualification, it should probably > be updated. Here it is: http://docs.python.org/tutorial/datastructures.html#using-lists-as-queues My opinion is that the warning should be either removed or qualified, but it is probably fine as written. ''' It is also possible to use a list as a queue, where the first element added is the first element retrieved (?first-in, first-out?); however, lists are not efficient for this purpose. While appends and pops from the end of list are fast, doing inserts or pops from the beginning of a list is slow (because all of the other elements have to be shifted by one). ''' The qualifications would be that deque lacks some features that list has, and that the shift-by-one operation is actually a call to memmove () and may not apply to all implementations. From millerdev at gmail.com Tue Jan 26 00:26:49 2010 From: millerdev at gmail.com (Daniel) Date: Mon, 25 Jan 2010 21:26:49 -0800 (PST) Subject: Authenticated encryption with PyCrypto Message-ID: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> Just got done reading this thread: http://groups.google.com/group/comp.lang.python/browse_thread/thread/b31a5b5f58084f12/0e09f5f5542812c3 and I'd appreciate feedback on this recipe: http://code.activestate.com/recipes/576980/ Of course, it does not meet all of the requirements set forth by the OP in the referenced thread (the pycrypto dependency is a problem), but it is an attempt to provide a simple interface for performing strong, password-based encryption. Are there already modules out there that provide such a simple interface? If there are, they seem to be hiding somewhere out of Google's view. I looked at ezPyCrypto, but it seemed to require public and private keys, which was not convenient in my situation... maybe password-based encryption is trivial to do with ezPyCrypto as well? In addition to ezPyCrypto, I looked at Google's keyczar, but despite the claims of the documentation, the API seemed overly complicated. Is it possible to have a simple API for an industry-strength encryption module? If not, is it possible to document that complicated API such that a non- cryptographer could use it and feel confident that he hadn't made a critical mistake? Also, slightly related, is there an easy way to get the sha/md5 deprecation warnings emitted by PyCrypto in Python 2.6 to go away? ~ Daniel From kirby.urner at gmail.com Tue Jan 26 01:10:32 2010 From: kirby.urner at gmail.com (kirby urner) Date: Mon, 25 Jan 2010 22:10:32 -0800 Subject: [Edu-sig] some turtle questions In-Reply-To: <8F7BCECA-CD55-425D-A173-94D7F30DFB53@bryant.edu> References: <8F7BCECA-CD55-425D-A173-94D7F30DFB53@bryant.edu> Message-ID: Hi Brian -- If you wanna go to a lot of work, but not a huge amount, write wrapper class for the Standard Library turtle that intercepts its commands and updates an on-board data structure, representing pixels x pixels, specifying self position, keep color info stashed per each one. That's a lot of data, depending on screen resolution. Consider a thick line option if you have one, make your turtle "wide body". Or stay with thin. So then if you go turtle.forward(10) you will send it to your self-made forward method. Stop and smell the pixels, see what color was stashed there, either by another turtle (! -- shared data structure) or by this turtle, or maybe it's still the default untrammeled color. You can add new methods, like "glide" or "explode" that translate to the underlying turtle somehow -- use your imagination. Kirby On Sun, Jan 24, 2010 at 7:29 AM, Brian Blais wrote: > Hello, > > I am trying to think of things to do with the turtle module with my > students, and I have some ideas where I am not sure whether the turtle > module can do it. > > 1) is there a way to determine the current screen pixel color? I am > thinking about having the turtle go forward until it reaches an object, say > a red circle. I can probably do this by making circle objects (drawn with > turtles themselves) which know their own position, and check against this > info. But I thought it might be useful also for the turtle to know. > > 2) is there a way to put a limit on the extent the turtle can travel? it > seems I can keep moving off of the screen. Is there a way to make it so > that a forward(50) command, at the edge, either raises an exception (at the > wall) or simply doesn't move the turtle because of the limit? > > > thanks! > > > bb > > -- > Brian Blais > bblais at bryant.edu > http://web.bryant.edu/~bblais > http://bblais.blogspot.com/ > > > > > _______________________________________________ > Edu-sig mailing list > Edu-sig at python.org > http://mail.python.org/mailman/listinfo/edu-sig > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gabriel.rossetti at arimaz.com Tue Jan 26 03:14:48 2010 From: gabriel.rossetti at arimaz.com (Gabriel Rossetti) Date: Tue, 26 Jan 2010 09:14:48 +0100 Subject: Any Swisses here? In-Reply-To: <145e6acd-4260-495b-b562-da634fccfa1a@j19g2000yqk.googlegroups.com> References: <145e6acd-4260-495b-b562-da634fccfa1a@j19g2000yqk.googlegroups.com> Message-ID: <4B5EA478.607@arimaz.com> n00m wrote: > Congrats! > Your choice -- to ban building of muslim mosques -- is the only choice > to save our civililazation. > This yellow plague (incl. Chinese etc) must be eliminated from our > Planet, > They are very cunning critters, they can play on strings of > compassion, > but its riffraffs. What do you know about Russia? Who are yuo against > Russia? > Nothing. Dust. We are simply tired people. > Actually, I am Swiss and I have to say that the vote does not represent Switzerland. In fact, a lot of people didn't vote because they were sure it would never pass. They just came out with an official analysis/statistics of the vote results and it shows that people with higher education voted against the ban, that Christians voted for the ban and atheists against it. Also, if you look at the outcome for each Canton (State), the rural ones voted for the ban and the metropolitan/cosmopolitan ones against it. What happened is basically what happened in Nazi Germany, a strong campaign for the masses and sadly it worked. I have several Russian friends and I have no idea what their view is on the subject. Russia has several Muslim ethnic groups and with most there seems to be no problem (I'm excluding Chechnya, even thought the conflict is more about power than religion). Gabriel Disclaimer: the views expressed in this email are solely mine and not in any way representative of my company's views or ideologies. From showell30 at yahoo.com Tue Jan 26 04:27:54 2010 From: showell30 at yahoo.com (Steve Howell) Date: Tue, 26 Jan 2010 01:27:54 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <0b8ed84f-0a42-42c5-a1ba-e70ed048f39d@k19g2000yqc.googlegroups.com> Message-ID: <2ba54ccf-b0b5-4a30-a31a-fa82571af20e@a6g2000yqm.googlegroups.com> On Jan 25, 9:00?pm, Steve Howell wrote: > On Jan 24, 11:28?am, a... at pythoncraft.com (Aahz) wrote: > > > > > In article , > > Steve Howell ? wrote: > > > >Even with realloc()'s brokenness, you could improve pop(0) in a way > > >that does not impact list access at all, and the patch would not change > > >the time complexity of any operation; it would just add negligible > > >extract bookkeeping within list_resize() and a few other places. > > > Again, your responsibility is to provide a patch and a spectrum of > > benchmarking tests to prove it. ?Then you would still have to deal with > > the objection that extensions use the list internals -- that might be an > > okay sell given the effort otherwise required to port extensions to > > Python 3, but that's not the way to bet. > > Ok, I just submitted a patch to python-dev that illustrates a 100x > speedup on an admittedly artificial program. ?It still has a long way > to go, but it demonstrates proof of concept. ?I'm done for the day, > but tomorrow I will try to polish it up and improve it, even if its > doomed for rejection. ?Apologies to all I have offended in this > thread. ?I frankly found some of the pushback to be a bit hasty and > disrespectful, but I certainly overreacted to some of the criticism. > And now I'm in the awkward position of asking the people I offended to > help me with the patch. ?If anybody can offer me a hand in > understanding some of CPython's internals, particularly with regard to > memory management, it would be greatly appreciated. > > (Sorry I don't have a link to the python-dev posting; it is not > showing up in the archives yet for some reason.) Here is the latest version of the patch, which passes all the tests on my debug build. Not exactly trivial, but not super complicated either. Index: Include/listobject.h =================================================================== --- Include/listobject.h (revision 77751) +++ Include/listobject.h (working copy) @@ -36,6 +36,7 @@ * the list is not yet visible outside the function that builds it. */ Py_ssize_t allocated; + Py_ssize_t orphans; } PyListObject; PyAPI_DATA(PyTypeObject) PyList_Type; Index: Objects/listobject.c =================================================================== --- Objects/listobject.c (revision 77751) +++ Objects/listobject.c (working copy) @@ -27,13 +27,25 @@ PyObject **items; size_t new_allocated; Py_ssize_t allocated = self->allocated; + Py_ssize_t needed; + if (self->orphans >= newsize) { + items = self->ob_item - self->orphans; + memmove(items, &items[self->orphans], + (newsize)*sizeof(PyObject *)); + self->ob_item = items; + self->orphans = 0; + } + + needed = newsize + self->orphans; + items = self->ob_item - self->orphans; + /* Bypass realloc() when a previous overallocation is large enough to accommodate the newsize. If the newsize falls lower than half the allocated size, then proceed with the realloc() to shrink the list. */ - if (allocated >= newsize && newsize >= (allocated >> 1)) { - assert(self->ob_item != NULL || newsize == 0); + if (allocated >= needed && needed >= (allocated >> 1)) { + assert(items != NULL || newsize == 0); Py_SIZE(self) = newsize; return 0; } @@ -45,28 +57,32 @@ * system realloc(). * The growth pattern is: 0, 4, 8, 16, 25, 35, 46, 58, 72, 88, ... */ - new_allocated = (newsize >> 3) + (newsize < 9 ? 3 : 6); + new_allocated = (needed >> 3) + (needed < 9 ? 3 : 6); /* check for integer overflow */ - if (new_allocated > PY_SIZE_MAX - newsize) { + if (new_allocated > PY_SIZE_MAX - needed) { PyErr_NoMemory(); return -1; } else { - new_allocated += newsize; + new_allocated += needed; } - if (newsize == 0) + if (needed == 0) new_allocated = 0; - items = self->ob_item; - if (new_allocated <= ((~(size_t)0) / sizeof(PyObject *))) + /* + fprintf(stderr, "ob_item: %p", self->ob_item); + fprintf(stderr, "items: %p", items); + */ + if (new_allocated <= ((~(size_t)0) / sizeof(PyObject *))) { PyMem_RESIZE(items, PyObject *, new_allocated); + } else items = NULL; if (items == NULL) { PyErr_NoMemory(); return -1; } - self->ob_item = items; + self->ob_item = items + self->orphans; Py_SIZE(self) = newsize; self->allocated = new_allocated; return 0; @@ -158,6 +174,7 @@ } Py_SIZE(op) = size; op->allocated = size; + op->orphans = 0; _PyObject_GC_TRACK(op); return (PyObject *) op; } @@ -304,9 +321,10 @@ There's a simple test case where somehow this reduces thrashing when a *very* large list is created and immediately deleted. */ + op->ob_item -= op->orphans; i = Py_SIZE(op); while (--i >= 0) { - Py_XDECREF(op->ob_item[i]); + Py_XDECREF(op->ob_item[i+op->orphans]); } PyMem_FREE(op->ob_item); } @@ -548,12 +566,13 @@ if (item != NULL) { /* Because XDECREF can recursively invoke operations on this list, we make it empty first. */ + item -= a->orphans; i = Py_SIZE(a); Py_SIZE(a) = 0; a->ob_item = NULL; a->allocated = 0; while (--i >= 0) { - Py_XDECREF(item[i]); + Py_XDECREF(item[i + a->orphans]); } PyMem_FREE(item); } @@ -638,18 +657,32 @@ memcpy(recycle, &item[ilow], s); if (d < 0) { /* Delete -d items */ - memmove(&item[ihigh+d], &item[ihigh], - (Py_SIZE(a) - ihigh)*sizeof(PyObject *)); + if (ilow == 0) { + a->orphans += (-1 * d); + a->ob_item += (-1 * d); + } + else { + memmove(&item[ihigh+d], &item[ihigh], + (Py_SIZE(a) - ihigh)*sizeof(PyObject *)); + } list_resize(a, Py_SIZE(a) + d); item = a->ob_item; } else if (d > 0) { /* Insert d items */ - k = Py_SIZE(a); - if (list_resize(a, k+d) < 0) - goto Error; - item = a->ob_item; - memmove(&item[ihigh+d], &item[ihigh], - (k - ihigh)*sizeof(PyObject *)); + if (ilow == 0 && d <= a->orphans) { + a->orphans += (-1 * d); + a->ob_item += (-1 * d); + item = a->ob_item; + list_resize(a, Py_SIZE(a) + d); + } + else { + k = Py_SIZE(a); + if (list_resize(a, k+d) < 0) + goto Error; + item = a->ob_item; + memmove(&item[ihigh+d], &item[ihigh], + (k - ihigh)*sizeof(PyObject *)); + } } for (k = 0; k < n; k++, ilow++) { PyObject *w = vitem[k]; @@ -838,7 +871,7 @@ } break; } - if (Py_SIZE(self) < self->allocated) { + if (Py_SIZE(self) + self->orphans < self->allocated) { /* steals ref */ PyList_SET_ITEM(self, Py_SIZE(self), item); ++Py_SIZE(self); @@ -852,7 +885,7 @@ } /* Cut back result list if initial guess was too large. */ - if (Py_SIZE(self) < self->allocated) + if (Py_SIZE(self) + self->orphans < self->allocated) list_resize(self, Py_SIZE(self)); /* shrinking can't fail */ Py_DECREF(it); @@ -2290,7 +2323,7 @@ { Py_ssize_t res; - res = sizeof(PyListObject) + self->allocated * sizeof(void*); + res = sizeof(PyListObject) + (self->allocated + self->orphans) * sizeof(void*); return PyLong_FromSsize_t(res); } Index: Lib/test/test_list.py =================================================================== --- Lib/test/test_list.py (revision 77751) +++ Lib/test/test_list.py (working copy) @@ -51,6 +51,15 @@ self.assertEqual(len([0]), 1) self.assertEqual(len([0, 1, 2]), 3) + def test_pop_and_prepend(self): + # This guards against faulty optimizations on list that + # attempt to makes pops and prepends at the beginning of the + # list work faster. + lst = [5] * 100 + del lst[0] + lst.insert(0, 4) + self.assertEqual(lst, [4] + [5] * 99) + def test_overflow(self): lst = [4, 5, 6, 7] n = int((sys.maxsize*2+2) // len(lst)) Index: Lib/test/test_iter.py =================================================================== --- Lib/test/test_iter.py (revision 77751) +++ Lib/test/test_iter.py (working copy) @@ -875,7 +875,18 @@ except TypeError: pass + def test_extends(self): + # This test would break on an incomplete patch to listobject.c + def gen(): + for i in range(500): + yield i + lst = [0] * 500 + for i in range(240): + lst.pop(0) + lst.extend(gen()) + + def test_main(): run_unittest(TestCase) Index: Lib/test/test_sys.py =================================================================== --- Lib/test/test_sys.py (revision 77751) +++ Lib/test/test_sys.py (working copy) @@ -515,7 +515,7 @@ # bool objects are not gc tracked self.assertEqual(sys.getsizeof(True), size(vh) + self.longdigit) # but lists are - self.assertEqual(sys.getsizeof([]), size(vh + 'PP') + gc_header_size) + self.assertEqual(sys.getsizeof([]), size(vh + 'PPi') + gc_header_size) def test_default(self): h = self.header @@ -638,7 +638,7 @@ # list samples = [[], [1,2,3], ['1', '2', '3']] for sample in samples: - check(sample, size(vh + 'PP') + len(sample)*self.P) + check(sample, size(vh + 'PPi') + len(sample)*self.P) # sortwrapper (list) # XXX # cmpwrapper (list) From jeanmichel at sequans.com Tue Jan 26 04:49:54 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Tue, 26 Jan 2010 10:49:54 +0100 Subject: Sikuli: the coolest Python project I have yet seen... In-Reply-To: References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> <4B5D9F09.7050607@it.uu.se> Message-ID: <4B5EBAC2.4080209@sequans.com> Terry Reedy wrote: > On 1/25/2010 9:14 AM, Javier Collado wrote: > >> I think the site is under maintenance. I tried a couple of hours ago >> and it worked fine. >> >> As an alternative, I found that this link also worked: >> http://www.sikuli.org/ > > This just redirects to the link below > >>>> http://sikuli.csail.mit.edu/ I also did this > >>> This link is broken! > > Worked for me both yesterday and now. > I confirmed it now works. The demo is pretty impressive. However one should not be the kind of guy changing its desktop theme on a regular basis. If I got it well, all is based on bitmap recognition (with some tolerance though). I'll still give it a try. JM From lanyjie at yahoo.com Tue Jan 26 06:17:23 2010 From: lanyjie at yahoo.com (Yingjie Lan) Date: Tue, 26 Jan 2010 03:17:23 -0800 (PST) Subject: announcement: expy release 0.5 Message-ID: <290176.43047.qm@web54204.mail.re2.yahoo.com> Hi all, This is to announce expy release 0.5. expy is an express way to extend Python. For more information and tutorials on expy, see: http://expy.sf.net/ Cheers, Yingjie From mail at hellmutweber.de Tue Jan 26 06:48:32 2010 From: mail at hellmutweber.de (Hellmut Weber) Date: Tue, 26 Jan 2010 12:48:32 +0100 Subject: Accessing the name of an actual parameter Message-ID: <4B5ED690.1020706@hellmutweber.de> Hi, consider the following piece of code, please ----- ----- def f(param): nameOfParam = ??? # here I want to access the name of the variable # which was given as parameter to the function print nameOfParam, param return if __name__ == __main__: a = 1 f(a) b = 'abcd' f(a) ----- ----- The output should be: 'a' 1 'b' 'abcd' ----- ----- I tried to look at globals() and locals(), gave a look to the frames (sys._getframe(0) and sys._getframe(1), but did not see a possibility to access the information a want How can this be done? TIA Hellmut -- Dr. Hellmut Weber mail at hellmutweber.de Degenfeldstra?e 2 tel +49-89-3081172 D-80803 M?nchen-Schwabing mobil +49-172-8450321 please: No DOCs, no PPTs. why: tinyurl.com/cbgq From clp2 at rebertia.com Tue Jan 26 06:58:42 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Tue, 26 Jan 2010 03:58:42 -0800 Subject: Accessing the name of an actual parameter In-Reply-To: <4B5ED690.1020706@hellmutweber.de> References: <4B5ED690.1020706@hellmutweber.de> Message-ID: <50697b2c1001260358h4cc93b8cva11b9e524ed77eae@mail.gmail.com> On Tue, Jan 26, 2010 at 3:48 AM, Hellmut Weber wrote: > Hi, > > consider the following piece of code, please > > ----- ----- > > def f(param): > ?nameOfParam = ??? > ?# here I want to access the name of the variable > ?# which was given as parameter to the function > ?print nameOfParam, param > ?return > > if __name__ == __main__: > > ?a = 1 > ?f(a) > > ?b = 'abcd' > ?f(a) > > ----- ----- > > The output should be: > > 'a' 1 > 'b' 'abcd' > > ----- ----- > > I tried to look at globals() and locals(), gave a look to the frames > (sys._getframe(0) and sys._getframe(1), > but did not see a possibility to access the information a want > > How can this be done? It's an FAQ that's not in the FAQ. The popular answer is "Don't try to do that! There's usually a less magic way to accomplish your goal.". Alternatives can usually be proffered if the exact use case is explained specifically. Cheers, Chris -- http://blog.rebertia.com From alfps at start.no Tue Jan 26 07:01:13 2010 From: alfps at start.no (Alf P. Steinbach) Date: Tue, 26 Jan 2010 13:01:13 +0100 Subject: Accessing the name of an actual parameter In-Reply-To: References: Message-ID: * Hellmut Weber: > > consider the following piece of code, please > > ----- ----- > > def f(param): > nameOfParam = ??? > # here I want to access the name of the variable > # which was given as parameter to the function > print nameOfParam, param > return > > if __name__ == __main__: > > a = 1 > f(a) > > b = 'abcd' > f(a) > > ----- ----- > > The output should be: > > 'a' 1 > 'b' 'abcd' > > ----- ----- > > I tried to look at globals() and locals(), gave a look to the frames > (sys._getframe(0) and sys._getframe(1), > but did not see a possibility to access the information a want > > How can this be done? Not in any efficient way. A debugger can do it, and you can do it in the same way as a debugger, checking stack frames and the source code. But if it's debugging that you're after then use a debugger -- that's what they're for. Otherwise, just change the way that you invoke the routine. For example, >>> def f( **kwa ): ... print( kwa ) ... >>> a = 1234 >>> b = "huh" >>> f( a = a, b = b ) {'a': 1234, 'b': 'huh'} >>> _ Cheers & hth., - Alf From iurisilvio at gmail.com Tue Jan 26 07:06:18 2010 From: iurisilvio at gmail.com (Iuri) Date: Tue, 26 Jan 2010 10:06:18 -0200 Subject: Sikuli: the coolest Python project I have yet seen... In-Reply-To: <4B5EBAC2.4080209@sequans.com> References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> <4B5D9F09.7050607@it.uu.se> <4B5EBAC2.4080209@sequans.com> Message-ID: <789aac5a1001260406s12005ffcye8be6f2bfd2d59ed@mail.gmail.com> I liked this tool! I'm thinking about how I can write acceptance tests with Sikuli. []s iurisilvio On Tue, Jan 26, 2010 at 7:49 AM, Jean-Michel Pichavant < jeanmichel at sequans.com> wrote: > Terry Reedy wrote: > >> On 1/25/2010 9:14 AM, Javier Collado wrote: >> >> I think the site is under maintenance. I tried a couple of hours ago >>> and it worked fine. >>> >>> As an alternative, I found that this link also worked: >>> http://www.sikuli.org/ >>> >> >> This just redirects to the link below >> >> http://sikuli.csail.mit.edu/ I also did this >>>>> >>>> >> This link is broken! >>>> >>> >> Worked for me both yesterday and now. >> >> I confirmed it now works. > The demo is pretty impressive. > > However one should not be the kind of guy changing its desktop theme on a > regular basis. If I got it well, all is based on bitmap recognition (with > some tolerance though). I'll still give it a try. > > JM > > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeanmichel at sequans.com Tue Jan 26 07:06:44 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Tue, 26 Jan 2010 13:06:44 +0100 Subject: Accessing the name of an actual parameter In-Reply-To: <4B5ED690.1020706@hellmutweber.de> References: <4B5ED690.1020706@hellmutweber.de> Message-ID: <4B5EDAD4.4030902@sequans.com> Hellmut Weber wrote: > Hi, > > consider the following piece of code, please > > ----- ----- > > def f(param): > nameOfParam = ??? > # here I want to access the name of the variable > # which was given as parameter to the function > print nameOfParam, param > return > > if __name__ == __main__: > > a = 1 > f(a) > > b = 'abcd' > f(a) > > ----- ----- > > The output should be: > > 'a' 1 > 'b' 'abcd' > > ----- ----- > > I tried to look at globals() and locals(), gave a look to the frames > (sys._getframe(0) and sys._getframe(1), > but did not see a possibility to access the information a want > > How can this be done? > > TIA > > Hellmut > > I don't know if it's possible (maybe using smelling hackish code). What I know for sure, is that your function should ask the required informations as parameters. That what parameters are for. Tell us whant you want to achieve precisely with the parameter names, maybe we'll find something suitable. JM From roy at panix.com Tue Jan 26 07:40:15 2010 From: roy at panix.com (Roy Smith) Date: Tue, 26 Jan 2010 07:40:15 -0500 Subject: Splitting text at whitespace but keeping the whitespace in thereturned list References: <1264352736.30280.1356298585@webmail.messagingengine.com> Message-ID: In article , "Tim Arnold" wrote: > also, partition works though it returns a tuple instead of a list. > >>> s = 'hello world' > >>> s.partition(' ') > ('hello', ' ', 'world') I've never used partition() before; my first thought on reading the above was, "That's weird, it should be returning a list". Then I went and looked at the docs. Given the description (returns specifically a 3-tuple), I guess a tuple makes sense, but now I'm wondering what the use case was for this method when it was invented? Having a variant of split() which either leaves the delimiter on the end of each word, or returns a list of alternating [word, delimiter, word, delimiter, word] seems logical and orthogonal. In fact, partition() is really just the hypothetical whitespace-preserving variant of split(), with maxsplit=1, except that it returns a tuple instead of a list. So, what was the original problem partition() was trying to solve? From jeanmichel at sequans.com Tue Jan 26 08:43:07 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Tue, 26 Jan 2010 14:43:07 +0100 Subject: pylint and the 'missing docstring' warning Message-ID: <4B5EF16B.6080505@sequans.com> Hello, Does anyone using pylint knows a way to make pylint ignore these 'missing docstring' warnings when the base class version of the method actually defines the docstring ? 'Cause my doc builder (epydoc) handle it properly and propagate docstrings if asked to. Too bad pylint is complaining about it. I don't want to deactivate this warning. Any hint would be apreciated JM From mlee3891 at gmail.com Tue Jan 26 08:51:12 2010 From: mlee3891 at gmail.com (Cascade3891) Date: Tue, 26 Jan 2010 05:51:12 -0800 (PST) Subject: My experiences building a small app on Python Message-ID: <62c367db-49e3-485b-80d6-ed2f99065f48@22g2000yqr.googlegroups.com> Hello Python community, I built a small app with Python for the first time. It has served a very useful and productive purpose for me and my colleagues at work. I have shared my experiences on the following link due to the better web presentation tools and coding presentation tools: http://www.daniweb.com/forums/thread255458.html It's a bit of a read. But insightful. If you have any feedback or tips and advice, please let me know. I've fallen in love with programming because of Python. =) Thank you in advance. Cheers. From frankborell at gmail.com Tue Jan 26 08:52:56 2010 From: frankborell at gmail.com (frankborell at gmail.com) Date: Tue, 26 Jan 2010 05:52:56 -0800 (PST) Subject: PIL: Wiping out metadata on save only on progressive JPG's Message-ID: Hello, I've tested on PIL 1.16 and 1.17. Metadata(IPTC and EXIF) are saved fine when the jpg image is NOT progressive. But when the jpg is progressive, the metadata gets wiped out. Does anyone have any ideas on how to preserve? im = Image.open('progressive jpeg path') im.save('jpeg path') or im.save('jpeg path',option={'progression':False,'quality': 60,'optimize':True}) Thanks, Frank From jjposner at optimum.net Tue Jan 26 08:56:50 2010 From: jjposner at optimum.net (John Posner) Date: Tue, 26 Jan 2010 08:56:50 -0500 Subject: pylint and the 'missing docstring' warning In-Reply-To: References: Message-ID: <4b5ef47f$0$31262$607ed4bc@cv.net> On 1/26/2010 8:43 AM, Jean-Michel Pichavant wrote: > Hello, > > Does anyone using pylint knows a way to make pylint ignore these > 'missing docstring' warnings when the base class version of the method > actually defines the docstring ? > 'Cause my doc builder (epydoc) handle it properly and propagate > docstrings if asked to. Too bad pylint is complaining about it. > I don't want to deactivate this warning. > Look for this setting in the pylint configuration file: # not require a docstring no-docstring-rgx=__.*__ -John From jeanmichel at sequans.com Tue Jan 26 09:22:33 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Tue, 26 Jan 2010 15:22:33 +0100 Subject: pylint and the 'missing docstring' warning In-Reply-To: <4b5ef47f$0$31262$607ed4bc@cv.net> References: <4b5ef47f$0$31262$607ed4bc@cv.net> Message-ID: <4B5EFAA9.8090608@sequans.com> John Posner wrote: > On 1/26/2010 8:43 AM, Jean-Michel Pichavant wrote: >> Hello, >> >> Does anyone using pylint knows a way to make pylint ignore these >> 'missing docstring' warnings when the base class version of the method >> actually defines the docstring ? >> 'Cause my doc builder (epydoc) handle it properly and propagate >> docstrings if asked to. Too bad pylint is complaining about it. >> I don't want to deactivate this warning. >> > > Look for this setting in the pylint configuration file: > > # not require a docstring > no-docstring-rgx=__.*__ > > -John > > If I'm no wrong this just disables docstring checks for methods matching __.*__ I'd like to disable it for any method which base class version already defines the docstring. JM From simon at brunningonline.net Tue Jan 26 10:02:55 2010 From: simon at brunningonline.net (Simon Brunning) Date: Tue, 26 Jan 2010 15:02:55 +0000 Subject: My experiences building a small app on Python In-Reply-To: <62c367db-49e3-485b-80d6-ed2f99065f48@22g2000yqr.googlegroups.com> References: <62c367db-49e3-485b-80d6-ed2f99065f48@22g2000yqr.googlegroups.com> Message-ID: <8c7f10c61001260702xf4f71c2t538661f73f66399f@mail.gmail.com> 2010/1/26 Cascade3891 : > It's a bit of a read. But insightful. We'll be the judge of that, surely? ;-) -- Cheers, Simon B. From gherron at islandtraining.com Tue Jan 26 10:50:31 2010 From: gherron at islandtraining.com (Gary Herron) Date: Tue, 26 Jan 2010 07:50:31 -0800 Subject: Accessing the name of an actual parameter In-Reply-To: <4B5ED690.1020706@hellmutweber.de> References: <4B5ED690.1020706@hellmutweber.de> Message-ID: <4B5F0F47.4080007@islandtraining.com> Hellmut Weber wrote: > Hi, > > consider the following piece of code, please > > ----- ----- > > def f(param): > nameOfParam = ??? > # here I want to access the name of the variable > # which was given as parameter to the function > print nameOfParam, param > return > > if __name__ == __main__: > > a = 1 > f(a) > > b = 'abcd' > f(a) > > ----- ----- > > The output should be: > > 'a' 1 > 'b' 'abcd' > > ----- ----- > > I tried to look at globals() and locals(), gave a look to the frames > (sys._getframe(0) and sys._getframe(1), > but did not see a possibility to access the information a want > > How can this be done? > > TIA > > Hellmut > > It's naive to think this question even makes sense. There are many ways f can be called which don't involve a parameter: f(42) f(time()) f(a+123) f(sin(a)) f(f(1)) and so on. Cheers, Gary Herron From gherron at islandtraining.com Tue Jan 26 10:54:06 2010 From: gherron at islandtraining.com (Gary Herron) Date: Tue, 26 Jan 2010 07:54:06 -0800 Subject: Accessing the name of an actual parameter In-Reply-To: <4B5F0F47.4080007@islandtraining.com> References: <4B5ED690.1020706@hellmutweber.de> <4B5F0F47.4080007@islandtraining.com> Message-ID: <4B5F101E.2080501@islandtraining.com> Gary Herron wrote: > Hellmut Weber wrote: >> Hi, >> >> consider the following piece of code, please >> >> ----- ----- >> >> def f(param): >> nameOfParam = ??? >> # here I want to access the name of the variable >> # which was given as parameter to the function >> print nameOfParam, param >> return >> >> if __name__ == __main__: >> >> a = 1 >> f(a) >> >> b = 'abcd' >> f(a) >> >> ----- ----- >> >> The output should be: >> >> 'a' 1 >> 'b' 'abcd' >> >> ----- ----- >> >> I tried to look at globals() and locals(), gave a look to the frames >> (sys._getframe(0) and sys._getframe(1), >> but did not see a possibility to access the information a want >> >> How can this be done? >> >> TIA >> >> Hellmut >> >> > > It's naive to think this question even makes sense. There are many > ways f can be called which don't involve a parameter: That's worded poorly, Rather I should have said: "don't involve a parameter that comes from a single variable." > > f(42) > f(time()) > f(a+123) > f(sin(a)) > f(f(1)) > > and so on. > Cheers, > Gary Herron > From pruebauno at latinmail.com Tue Jan 26 11:07:51 2010 From: pruebauno at latinmail.com (nn) Date: Tue, 26 Jan 2010 08:07:51 -0800 (PST) Subject: ctypes for AIX References: <9C240725D2ABD949A80B46DD93AB107042E8D44101@XCH-NW-08V.nw.nos.boeing.com> <50697b2c1001240630r33845a1bl7f94b40a4f0c4436@mail.gmail.com> Message-ID: On Jan 25, 6:36?pm, "Waddle, Jim" wrote: > Chris, > Thanks for responding to my email. > I apologize for the remark about python only being developed for windows. I got the impression when I was looking at the ActivePython web site and saw that the version of python that they had available was not supported on very many unix systems. I should not make general statement based on only one web site. After reading your email I decided to see for myself what the issue was about compiling python on AIX 5.3. > > This is the error I saw the first time I tried to use ctypes. > > Python 2.4.3 (#1, Jul 17 2006, 20:00:23) [C] on aix5 > Type "help", "copyright", "credits" or "license" for more information.>>> import ctypes > > Traceback (most recent call last): > ? File "", line 1, in ? > ImportError: No module named ctypes > > This version of python was downloaded and installed from ActivePython and when I checked their webpage it states that ctypes is not available on AIX. > I then figured I would get a new copy of python and install it on AIX. I downloaded python.2.5.5c2 fromhttp://www.python.org. I did the configure and make which posted many errors in the ctypes function which I guess is the reason that is does not get include in the final make. > > an example of the build error I get when doing the make is: > xlc_r -q64 -DNDEBUG -O -I. -I/s/users/cz030a/xferjunk/python/Python-2.5.5c2/./Include -Ibuild/temp.aix-5.3-2.5/libffi/inclu > de -Ibuild/temp.aix-5.3-2.5/libffi -I/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/libffi/src -I/s/users/c > z030a/xferjunk/ots/python2.5/include -I. -IInclude -I./Include -I/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Include -I/ > s/users/cz030a/xferjunk/python/Python-2.5.5c2 -c /s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.c - > o build/temp.aix-5.3-2.5/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.o > "/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.c", line 2820.31: 1506-068 (W) Operation between ty > pes "void*" and "int(*)(void)" is not allowed. > "/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.c", line 3363.28: 1506-280 (W) Function argument as > signment between types "int(*)(void)" and "void*" is not allowed. > "/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.c", line 4768.67: 1506-280 (W) Function argument as > signment between types "void*" and "void*(*)(void*,const void*,unsigned long)" is not allowed. > "/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.c", line 4769.66: 1506-280 (W) Function argument as > signment between types "void*" and "void*(*)(void*,int,unsigned long)" is not allowed. > > I do not have sufficient knowledge to know how to fix this. I would think that this error somehow is related to compiling on aix. If you have any suggestions on how to correct this problem , I would appreciate it > > Jim Waddle > KIT-D > 425-785-5194 > > -----Original Message----- > From: ch... at rebertia.com [mailto:ch... at rebertia.com] On Behalf Of Chris Rebert > Sent: Sunday, January 24, 2010 7:31 AM > To: Waddle, Jim > > Cc: python-l... at python.org > Subject: Re: ctypes for AIX > > On Sun, Jan 24, 2010 at 5:54 AM, Waddle, Jim wrote: > > I need to use ctypes with python running on AIX. > > According to the ctypes readme, ctypes is based on libffi, which > according to its website, supports AIX for PowerPC64. > So, perhaps you could state what the actual error or problem you're > encountering is? > It is theoretically possible the ctypes-bundled libffi is either > outdated or had the AIX-specific bits removed; I don't know, I'm not a > CPython dev. > > > It appears that python is being developed mostly for windows. > > No, not really; your statement is especially ironic considering one of > Python's primary areas of use is for web applications as part of a > LAMP stack. > > > Is there a policy concerning getting functions like ctypes working on AIX. > > No idea. Someone will probably chime in though. > > Cheers, > Chris > --http://blog.rebertia.com > > Look at this website: http://pware.hvcc.edu/news.html "8/1/2009 Python with ctypes!" Good luck. From duncan.booth at invalid.invalid Tue Jan 26 11:13:16 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 26 Jan 2010 16:13:16 GMT Subject: Accessing the name of an actual parameter References: <4B5ED690.1020706@hellmutweber.de> Message-ID: Gary Herron wrote: > It's naive to think this question even makes sense. There are many ways > f can be called which don't involve a parameter: > > f(42) > f(time()) > f(a+123) > f(sin(a)) > f(f(1)) > > and so on. So long as the OP doesn't care if they get no names or several name that might not matter. However, explicitly passing in the name you want to use is likely to be more useful. >>> def getcallersnames(obj): f = inspect.currentframe().f_back.f_back return [name for name in f.f_locals if f.f_locals[name] is obj] >>> def bip(x): print getcallersnames(x) >>> def foo(bar): baz = bar bip(baz) bip(baz+1) >>> foo(3) ['bar', 'baz'] [] >>> bip(bip) ['bip'] -- Duncan Booth http://kupuguy.blogspot.com From jjposner at optimum.net Tue Jan 26 11:19:04 2010 From: jjposner at optimum.net (John Posner) Date: Tue, 26 Jan 2010 11:19:04 -0500 Subject: pylint and the 'missing docstring' warning In-Reply-To: References: <4b5ef47f$0$31262$607ed4bc@cv.net> Message-ID: <4B5F15F8.3010004@optimum.net> On 1/26/2010 9:22 AM, Jean-Michel Pichavant wrote: > John Posner wrote: >> On 1/26/2010 8:43 AM, Jean-Michel Pichavant wrote: >>> Hello, >>> >>> Does anyone using pylint knows a way to make pylint ignore these >>> 'missing docstring' warnings when the base class version of the method >>> actually defines the docstring ? >>> 'Cause my doc builder (epydoc) handle it properly and propagate >>> docstrings if asked to. Too bad pylint is complaining about it. >>> I don't want to deactivate this warning. >>> >> >> Look for this setting in the pylint configuration file: >> >> # not require a docstring >> no-docstring-rgx=__.*__ >> >> -John >> >> > If I'm no wrong this just disables docstring checks for methods matching > __.*__ > I'd like to disable it for any method which base class version already > defines the docstring. > I'm a Pylint newbie myself. The only (half-baked) idea I can come up with is to use a naming convention for such derived classes. For example, if all such class names ended with "_d" (ugh!), you could modify the configuration setting like this: no-docstring-rgx=(__.*__|.*_d) -John From steve at holdenweb.com Tue Jan 26 11:54:23 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 26 Jan 2010 11:54:23 -0500 Subject: TypeError not caught by except statement In-Reply-To: <424b71ec1001252017i64d28375oc3644433c66ae40b@mail.gmail.com> References: <424b71ec1001252017i64d28375oc3644433c66ae40b@mail.gmail.com> Message-ID: <4B5F1E3F.3010800@holdenweb.com> siddhartha veedaluru wrote: > Hi, > > except not able to caught the TypeError exception occured in the below > code > > log.info ("refer",ret) in the try block > > throws a TypeError which is not caught . The traceback you provide does not show the line in question, so I am confused as to why you would think it was causing the problem. The purpose of the traceback is to show you where the exception occurred, and I question whether that is happening here. Anyway, I suspect your error might go away if you turned the first argument of hte log.info() call into a format string such as log.info("refer: %s", ret) regards Steve > Also sometimes process is getting hanged. > > ------------------------------------------------------------------------------------------------------------------------------------------------ > import logging > log = logging.getLogger() > fileName = strftime("%d-%b-%Y-", gmtime()) + str(int(time.time())) + "- > Log.log" > log = logging.getLogger() > log.setLevel(logging.NOTSET) > fh = logging.FileHandler(logFile) > logFileLevel = logging.DEBUG > fh.setLevel(logFileLevel) > format_string = '%(process)d %(thread)d %(asctime)-15s %(levelname)-5s > at %(filename)-15s in %(funcName)-10s at line %(lineno)-3d "%(message) > s"' > fh.setFormatter(logging.Formatter(format_string)) > log.addHandler(fh) > > try: > log.info ("start") > log.info ("refer",ret) > log.info ("end") > except TypeError: > log.exception("Exception raised") > > ---------------------------------------------------------------------------------------------------------------------------------------------- > OUTPUT message: > > Traceback (most recent call last): > File "C:\Python26\lib\logging\__init__.py", line 768, in emit > msg = self.format(record) > File "C:\Python26\lib\logging\__init__.py", line 648, in format > return fmt.format(record) > File "C:\Python26\lib\logging\__init__.py", line 436, in format > record.message = record.getMessage() > File "C:\Python26\lib\logging\__init__.py", line 306, in getMessage > msg = msg % self.args > TypeError: not all arguments converted during string formatting > -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Tue Jan 26 11:54:23 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 26 Jan 2010 11:54:23 -0500 Subject: TypeError not caught by except statement In-Reply-To: <424b71ec1001252017i64d28375oc3644433c66ae40b@mail.gmail.com> References: <424b71ec1001252017i64d28375oc3644433c66ae40b@mail.gmail.com> Message-ID: <4B5F1E3F.3010800@holdenweb.com> siddhartha veedaluru wrote: > Hi, > > except not able to caught the TypeError exception occured in the below > code > > log.info ("refer",ret) in the try block > > throws a TypeError which is not caught . The traceback you provide does not show the line in question, so I am confused as to why you would think it was causing the problem. The purpose of the traceback is to show you where the exception occurred, and I question whether that is happening here. Anyway, I suspect your error might go away if you turned the first argument of hte log.info() call into a format string such as log.info("refer: %s", ret) regards Steve > Also sometimes process is getting hanged. > > ------------------------------------------------------------------------------------------------------------------------------------------------ > import logging > log = logging.getLogger() > fileName = strftime("%d-%b-%Y-", gmtime()) + str(int(time.time())) + "- > Log.log" > log = logging.getLogger() > log.setLevel(logging.NOTSET) > fh = logging.FileHandler(logFile) > logFileLevel = logging.DEBUG > fh.setLevel(logFileLevel) > format_string = '%(process)d %(thread)d %(asctime)-15s %(levelname)-5s > at %(filename)-15s in %(funcName)-10s at line %(lineno)-3d "%(message) > s"' > fh.setFormatter(logging.Formatter(format_string)) > log.addHandler(fh) > > try: > log.info ("start") > log.info ("refer",ret) > log.info ("end") > except TypeError: > log.exception("Exception raised") > > ---------------------------------------------------------------------------------------------------------------------------------------------- > OUTPUT message: > > Traceback (most recent call last): > File "C:\Python26\lib\logging\__init__.py", line 768, in emit > msg = self.format(record) > File "C:\Python26\lib\logging\__init__.py", line 648, in format > return fmt.format(record) > File "C:\Python26\lib\logging\__init__.py", line 436, in format > record.message = record.getMessage() > File "C:\Python26\lib\logging\__init__.py", line 306, in getMessage > msg = msg % self.args > TypeError: not all arguments converted during string formatting > -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Tue Jan 26 11:57:17 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 26 Jan 2010 11:57:17 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <0b8ed84f-0a42-42c5-a1ba-e70ed048f39d@k19g2000yqc.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <0b8ed84f-0a42-42c5-a1ba-e70ed048f39d@k19g2000yqc.googlegroups.com> Message-ID: <4B5F1EED.5000503@holdenweb.com> Steve Howell wrote: > On Jan 24, 11:28 am, a... at pythoncraft.com (Aahz) wrote: >> In article , >> Steve Howell wrote: >> >> >> >>> Even with realloc()'s brokenness, you could improve pop(0) in a way >>> that does not impact list access at all, and the patch would not change >>> the time complexity of any operation; it would just add negligible >>> extract bookkeeping within list_resize() and a few other places. >> Again, your responsibility is to provide a patch and a spectrum of >> benchmarking tests to prove it. Then you would still have to deal with >> the objection that extensions use the list internals -- that might be an >> okay sell given the effort otherwise required to port extensions to >> Python 3, but that's not the way to bet. >> > > Ok, I just submitted a patch to python-dev that illustrates a 100x > speedup on an admittedly artificial program. It still has a long way > to go, but it demonstrates proof of concept. I'm done for the day, > but tomorrow I will try to polish it up and improve it, even if its > doomed for rejection. Apologies to all I have offended in this > thread. I frankly found some of the pushback to be a bit hasty and > disrespectful, but I certainly overreacted to some of the criticism. > And now I'm in the awkward position of asking the people I offended to > help me with the patch. If anybody can offer me a hand in > understanding some of CPython's internals, particularly with regard to > memory management, it would be greatly appreciated. > > (Sorry I don't have a link to the python-dev posting; it is not > showing up in the archives yet for some reason.) > > Fortunately for you this is a very welcoming group, and particularly responsive to individuals who have seen the error of their ways ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Tue Jan 26 11:57:17 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 26 Jan 2010 11:57:17 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <0b8ed84f-0a42-42c5-a1ba-e70ed048f39d@k19g2000yqc.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <0b8ed84f-0a42-42c5-a1ba-e70ed048f39d@k19g2000yqc.googlegroups.com> Message-ID: <4B5F1EED.5000503@holdenweb.com> Steve Howell wrote: > On Jan 24, 11:28 am, a... at pythoncraft.com (Aahz) wrote: >> In article , >> Steve Howell wrote: >> >> >> >>> Even with realloc()'s brokenness, you could improve pop(0) in a way >>> that does not impact list access at all, and the patch would not change >>> the time complexity of any operation; it would just add negligible >>> extract bookkeeping within list_resize() and a few other places. >> Again, your responsibility is to provide a patch and a spectrum of >> benchmarking tests to prove it. Then you would still have to deal with >> the objection that extensions use the list internals -- that might be an >> okay sell given the effort otherwise required to port extensions to >> Python 3, but that's not the way to bet. >> > > Ok, I just submitted a patch to python-dev that illustrates a 100x > speedup on an admittedly artificial program. It still has a long way > to go, but it demonstrates proof of concept. I'm done for the day, > but tomorrow I will try to polish it up and improve it, even if its > doomed for rejection. Apologies to all I have offended in this > thread. I frankly found some of the pushback to be a bit hasty and > disrespectful, but I certainly overreacted to some of the criticism. > And now I'm in the awkward position of asking the people I offended to > help me with the patch. If anybody can offer me a hand in > understanding some of CPython's internals, particularly with regard to > memory management, it would be greatly appreciated. > > (Sorry I don't have a link to the python-dev posting; it is not > showing up in the archives yet for some reason.) > > Fortunately for you this is a very welcoming group, and particularly responsive to individuals who have seen the error of their ways ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From jeanmichel at sequans.com Tue Jan 26 12:12:48 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Tue, 26 Jan 2010 18:12:48 +0100 Subject: Accessing the name of an actual parameter In-Reply-To: References: <4B5ED690.1020706@hellmutweber.de> Message-ID: <4B5F2290.2040806@sequans.com> Duncan Booth wrote: > Gary Herron wrote: > > >> It's naive to think this question even makes sense. There are many ways >> f can be called which don't involve a parameter: >> >> f(42) >> f(time()) >> f(a+123) >> f(sin(a)) >> f(f(1)) >> >> and so on. >> > > So long as the OP doesn't care if they get no names or several name that > might not matter. However, explicitly passing in the name you want to use > is likely to be more useful. > > >>>> def getcallersnames(obj): >>>> > f = inspect.currentframe().f_back.f_back > return [name for name in f.f_locals if f.f_locals[name] is obj] > > >>>> def bip(x): >>>> > print getcallersnames(x) > > > >>>> def foo(bar): >>>> > baz = bar > bip(baz) > bip(baz+1) > > > >>>> foo(3) >>>> > ['bar', 'baz'] > [] > How do you know which between baz and bar has been used ? Here you get all names (in locals()) referencing the object passed in argument, you don't get the one actually used for the call. Another issue: class Foo: test = 'test' def foo(bar): bip(bar.test) f = Foo() foo(f) [] That restricts pretty much the use of getcallersnames. As said before, there is hardly any legit design which would require to know the actual refrence used to pass a parameter. JM From jeanmichel at sequans.com Tue Jan 26 12:20:42 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Tue, 26 Jan 2010 18:20:42 +0100 Subject: pylint and the 'missing docstring' warning In-Reply-To: <4B5F15F8.3010004@optimum.net> References: <4b5ef47f$0$31262$607ed4bc@cv.net> <4B5F15F8.3010004@optimum.net> Message-ID: <4B5F246A.6050009@sequans.com> John Posner wrote: > On 1/26/2010 9:22 AM, Jean-Michel Pichavant wrote: >> John Posner wrote: >>> On 1/26/2010 8:43 AM, Jean-Michel Pichavant wrote: >>>> Hello, >>>> >>>> Does anyone using pylint knows a way to make pylint ignore these >>>> 'missing docstring' warnings when the base class version of the method >>>> actually defines the docstring ? >>>> 'Cause my doc builder (epydoc) handle it properly and propagate >>>> docstrings if asked to. Too bad pylint is complaining about it. >>>> I don't want to deactivate this warning. >>>> >>> >>> Look for this setting in the pylint configuration file: >>> >>> # not require a docstring >>> no-docstring-rgx=__.*__ >>> >>> -John >>> >>> >> If I'm no wrong this just disables docstring checks for methods matching >> __.*__ >> I'd like to disable it for any method which base class version already >> defines the docstring. >> > > I'm a Pylint newbie myself. The only (half-baked) idea I can come up > with is to use a naming convention for such derived classes. For > example, if all such class names ended with "_d" (ugh!), you could > modify the configuration setting like this: > > no-docstring-rgx=(__.*__|.*_d) > > -John That would work but I definitely don't want to do that. JM From mal at egenix.com Tue Jan 26 12:21:09 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 26 Jan 2010 18:21:09 +0100 Subject: ctypes for AIX In-Reply-To: <9C240725D2ABD949A80B46DD93AB107042E8D44101@XCH-NW-08V.nw.nos.boeing.com> References: <9C240725D2ABD949A80B46DD93AB107042E8D44101@XCH-NW-08V.nw.nos.boeing.com> Message-ID: <4B5F2485.8070909@egenix.com> Waddle, Jim wrote: > I need to use ctypes with python running on AIX. It appears that python is being developed mostly for windows. Is there a policy concerning getting functions like ctypes working on AIX. If you can provide patches to get ctypes working on AIX, we'd consider adding those to the core. Please file patch or bug describing the problems you are having on AIX with the tracker: http://bugs.python.org/ Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 26 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From mal at egenix.com Tue Jan 26 12:37:53 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 26 Jan 2010 18:37:53 +0100 Subject: Authenticated encryption with PyCrypto In-Reply-To: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> Message-ID: <4B5F2871.6010102@egenix.com> Daniel wrote: > Just got done reading this thread: > > http://groups.google.com/group/comp.lang.python/browse_thread/thread/b31a5b5f58084f12/0e09f5f5542812c3 > > and I'd appreciate feedback on this recipe: > > http://code.activestate.com/recipes/576980/ > > Of course, it does not meet all of the requirements set forth by the > OP in the referenced thread (the pycrypto dependency is a problem), > but it is an attempt to provide a simple interface for performing > strong, password-based encryption. Are there already modules out there > that provide such a simple interface? If there are, they seem to be > hiding somewhere out of Google's view. > > I looked at ezPyCrypto, but it seemed to require public and private > keys, which was not convenient in my situation... maybe password-based > encryption is trivial to do with ezPyCrypto as well? In addition to > ezPyCrypto, I looked at Google's keyczar, but despite the claims of > the documentation, the API seemed overly complicated. Is it possible > to have a simple API for an industry-strength encryption module? If > not, is it possible to document that complicated API such that a non- > cryptographer could use it and feel confident that he hadn't made a > critical mistake? Yes, it is possible, but whatever you come up with will usually be bound to just one (or a few) different use cases, e.g. just look at the different cipher modes there are, the different key sizes, block sizes (for block ciphers), IV strings, padding, etc. etc. Note that your code has a padding bug: the decoder doesn't undo the padding. You're lucky though, since pickle will only read as much data as it needs and not complain about the extra data it finds. You are also using CBC mode, even though you are really after ECB mode (your code doesn't use chaining). With ECB mode, you don't need the IV string. > Also, slightly related, is there an easy way to get the sha/md5 > deprecation warnings emitted by PyCrypto in Python 2.6 to go away? Yes: you silence them via the warnings module. I suppose that the latest version of PyCrypto fixes these warnings. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 26 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From tmohr at s.netic.de Tue Jan 26 12:41:44 2010 From: tmohr at s.netic.de (Torsten Mohr) Date: Tue, 26 Jan 2010 18:41:44 +0100 Subject: deriving from array.array Message-ID: Hello, i try to derive a class from array.array: import array class Abc(array.array): def __init__(self, a, b): array.array.__init__(self, 'B') self.a = a self.b = b a = Abc(4, 5) print a print a.a I get an error for "a = Abc(4, 5)", seems the parameters are forwarded to array's __init__ as they are. Though i explicitly call __init__() for array. I'd like to use array and make sure it's type is always 'B'. I'd like to derive because i don't want to rewrite all the methods like __getiem__ for my class and then call array's __getitem__. How do i best derive from array.array? Thanks for any hints, Torsten. From alfps at start.no Tue Jan 26 13:24:27 2010 From: alfps at start.no (Alf P. Steinbach) Date: Tue, 26 Jan 2010 19:24:27 +0100 Subject: deriving from array.array In-Reply-To: References: Message-ID: * Torsten Mohr: > Hello, > > i try to derive a class from array.array: > > > import array > > class Abc(array.array): > def __init__(self, a, b): > array.array.__init__(self, 'B') > self.a = a > self.b = b > > > a = Abc(4, 5) > print a > print a.a > > > I get an error for "a = Abc(4, 5)", seems the parameters are > forwarded to array's __init__ as they are. No, with CPython they're forwarded to __new__. > Though i explicitly > call __init__() for array. That's the constructor inherited from 'object', it takes no args (except the self arg). > I'd like to use array and make sure it's type is always 'B'. > I'd like to derive because i don't want to rewrite all the methods like > __getiem__ for my class and then call array's __getitem__. > > How do i best derive from array.array? import array class ByteArray( array.array ): def __new__( self, *args ): return array.array.__new__( self, "B" ) def __init__( self, a, b ): array.array.__init__( self ) self.a = a self.b = b a = ByteArray( 4, 5 ) print( a ) print( a.a ) Disclaimer: I'm not a Python programmer. :-) Cheers & hth., - Alf From gslindstrom at gmail.com Tue Jan 26 13:58:47 2010 From: gslindstrom at gmail.com (gslindstrom at gmail.com) Date: Tue, 26 Jan 2010 18:58:47 +0000 Subject: Python or Ant Message-ID: <000e0cd70e4e3246b9047e15e0fb@google.com> My company is looking at creating a tool to allow us to define and manage a process for each job we run (a typical job may be look on a customers ftp site for a file, download it, decrypt it and load it into our database). We would like something which would allow us to glue together various existing processes we currently use into a single unit with multiple steps. Along the way, this new routine would need to log its progress and be able to report and even handle errors. A coworker has suggested we look at Ant ("Another Neat Tool") and, though it looks promising, I have reservations. If I recall correctly, it was intended as a replacement for "Make" and I worry that we may be trying to force Ant to be something it is not. Also, most of our code base is in Python and I'd really like to stay that way, of possible. Are there any systems out there that will allow me to run multiple programs as one process? We could write our own, of course, and the Twisted package looks like it would be fun to use. Or, is Ant a viable solution to our problem? Your constructive comments would be appreciated Greg Lindstrom Novasys Health Little Rock, Arkansas -------------- next part -------------- An HTML attachment was scrubbed... URL: From cmpython at gmail.com Tue Jan 26 13:59:10 2010 From: cmpython at gmail.com (CM) Date: Tue, 26 Jan 2010 10:59:10 -0800 (PST) Subject: Sikuli: the coolest Python project I have yet seen... References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> Message-ID: <89c2550d-057d-4cc2-a855-d4f3f11cb272@b10g2000vbh.googlegroups.com> On Jan 24, 10:18?pm, Ron wrote: > Sikuli is the coolest Python project I have ever seen in my ten year > hobbyist career. An MIT oepn source project, Sikuli uses Python to > automate GUI tasks (in any GUI or GUI baed app that runs the JVM) by > simply drag and dropping GUI elements into Python scripts as function > arguments. Download athttp://sikuli.csail.mit.edu/I also did this > podcast about Sikulihttp://media.libsyn.com/media/awaretek/Python411_20100124_Sikuli.mp3 How is this preferable to a macro recorder? From alfps at start.no Tue Jan 26 14:23:57 2010 From: alfps at start.no (Alf P. Steinbach) Date: Tue, 26 Jan 2010 20:23:57 +0100 Subject: deriving from array.array In-Reply-To: References: Message-ID: * Alf P. Steinbach: > * Torsten Mohr: >> Hello, >> >> i try to derive a class from array.array: >> >> >> import array >> >> class Abc(array.array): >> def __init__(self, a, b): >> array.array.__init__(self, 'B') >> self.a = a >> self.b = b >> >> >> a = Abc(4, 5) >> print a >> print a.a >> >> >> I get an error for "a = Abc(4, 5)", seems the parameters are >> forwarded to array's __init__ as they are. > > No, with CPython they're forwarded to __new__. > > >> Though i explicitly >> call __init__() for array. > > That's the constructor inherited from 'object', it takes no args (except > the self arg). > > >> I'd like to use array and make sure it's type is always 'B'. >> I'd like to derive because i don't want to rewrite all the methods >> like __getiem__ for my class and then call array's __getitem__. >> >> How do i best derive from array.array? > > > import array > > class ByteArray( array.array ): > def __new__( self, *args ): > return array.array.__new__( self, "B" ) > > def __init__( self, a, b ): > array.array.__init__( self ) > self.a = a > self.b = b > > a = ByteArray( 4, 5 ) > print( a ) > print( a.a ) > > > > Disclaimer: I'm not a Python programmer. :-) Hm, good that I included a disclaimer. The above code is technically OK but it is misleading. The first argument to '__new__' is not a self argument but a type argument, better called 'cls' or some such. From the docs, "__new__() is a static method (special-cased so you need not declare it as such) that takes the class of which an instance was requested as its first argument" Cheers, - Alf (self-correcting) From tmohr at s.netic.de Tue Jan 26 14:28:26 2010 From: tmohr at s.netic.de (Torsten Mohr) Date: Tue, 26 Jan 2010 20:28:26 +0100 Subject: deriving from array.array References: Message-ID: Hello, thanks a lot for your hint, it works fine. >> I get an error for "a = Abc(4, 5)", seems the parameters are >> forwarded to array's __init__ as they are. > > No, with CPython they're forwarded to __new__. Not sure if i understand this correctly, if i derive from other classes (like wxpython widgets) i always call the bases __init__ . >> Though i explicitly >> call __init__() for array. > > That's the constructor inherited from 'object', it takes no args (except > the self arg). Is there a way to find out what i need to call? I haven't found much in the documentation. From writing C extensions i knew about the "new" entry in the PyTypeObject struct but it seems there's more behind it. In docs.python.org i did not find much, is there an URL where i can read more? Best regards, Torsten. From aahz at pythoncraft.com Tue Jan 26 14:35:06 2010 From: aahz at pythoncraft.com (Aahz) Date: 26 Jan 2010 11:35:06 -0800 Subject: Py 3: Terminal script can't find relative path References: <58e9b1e4-049a-4a7f-bea9-8d7e7e9f191a@e27g2000yqd.googlegroups.com> <9541dcf0-8060-4851-bdbd-0aed6de13bf9@v25g2000yqk.googlegroups.com> <317ebb29-8c20-4fe5-82c4-b2fe05efef38@h9g2000yqa.googlegroups.com> <98c70d6f-5872-436b-b160-01b39d17d270@a32g2000yqm.googlegroups.com> Message-ID: In article <98c70d6f-5872-436b-b160-01b39d17d270 at a32g2000yqm.googlegroups.com>, Gnarlodious wrote: > >OK I guess that is normal, I fixed it with this: >path=os.path.dirname(__file__)+"/Data/" Use os.path.join() instead -- just in case you ever port to Windows -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From debatem1 at gmail.com Tue Jan 26 14:40:08 2010 From: debatem1 at gmail.com (geremy condra) Date: Tue, 26 Jan 2010 14:40:08 -0500 Subject: Authenticated encryption with PyCrypto In-Reply-To: <4B5F2871.6010102@egenix.com> References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> <4B5F2871.6010102@egenix.com> Message-ID: On Tue, Jan 26, 2010 at 12:37 PM, M.-A. Lemburg wrote: > You are also using CBC mode, even though you are really after > ECB mode (your code doesn't use chaining). With ECB mode, you > don't need the IV string. However, ECB mode is not as secure- the IV is the right way to go here. I'd also note that you aren't supposed to use RandomPool anymore, and that AES-192 is frequently recommended over AES-256 for new applications due to a number of recent developments in the cryptanalysis of its key schedule. Geremy Condra From no.email at nospam.invalid Tue Jan 26 14:45:14 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Tue, 26 Jan 2010 11:45:14 -0800 Subject: Authenticated encryption with PyCrypto References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> Message-ID: <7xwrz4abw5.fsf@ruckus.brouhaha.com> Daniel writes: > Of course, it does not meet all of the requirements set forth by the > OP in the referenced thread (the pycrypto dependency is a problem), > but it is an attempt to provide a simple interface for performing > strong, password-based encryption. Are there already modules out there > that provide such a simple interface? If there are, they seem to be > hiding somewhere out of Google's view. http://www.nightsong.com/phr/crypto/p3.py I need to update it to handle 64-bit OS's and use Python 3.x-style print statements, but that is pretty trivial. From steve at holdenweb.com Tue Jan 26 14:46:55 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 26 Jan 2010 14:46:55 -0500 Subject: Python or Ant In-Reply-To: <000e0cd70e4e3246b9047e15e0fb@google.com> References: <000e0cd70e4e3246b9047e15e0fb@google.com> Message-ID: <4B5F46AF.7020407@holdenweb.com> gslindstrom at gmail.com wrote: > My company is looking at creating a tool to allow us to define and > manage a process for each job we run (a typical job may be look on a > customers ftp site for a file, download it, decrypt it and load it into > our database). We would like something which would allow us to glue > together various existing processes we currently use into a single unit > with multiple steps. Along the way, this new routine would need to log > its progress and be able to report and even handle errors. A coworker > has suggested we look at Ant ("Another Neat Tool") and, though it looks > promising, I have reservations. If I recall correctly, it was intended > as a replacement for "Make" and I worry that we may be trying to force > Ant to be something it is not. Also, most of our code base is in Python > and I'd really like to stay that way, of possible. > > Are there any systems out there that will allow me to run multiple > programs as one process? We could write our own, of course, and the > Twisted package looks like it would be fun to use. Or, is Ant a viable > solution to our problem? > > Your constructive comments would be appreciated > > Greg Lindstrom > Novasys Health > Little Rock, Arkansas > Take a look at fabric. It was designed for installation, but it might do what you want. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Tue Jan 26 14:46:55 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 26 Jan 2010 14:46:55 -0500 Subject: Python or Ant In-Reply-To: <000e0cd70e4e3246b9047e15e0fb@google.com> References: <000e0cd70e4e3246b9047e15e0fb@google.com> Message-ID: <4B5F46AF.7020407@holdenweb.com> gslindstrom at gmail.com wrote: > My company is looking at creating a tool to allow us to define and > manage a process for each job we run (a typical job may be look on a > customers ftp site for a file, download it, decrypt it and load it into > our database). We would like something which would allow us to glue > together various existing processes we currently use into a single unit > with multiple steps. Along the way, this new routine would need to log > its progress and be able to report and even handle errors. A coworker > has suggested we look at Ant ("Another Neat Tool") and, though it looks > promising, I have reservations. If I recall correctly, it was intended > as a replacement for "Make" and I worry that we may be trying to force > Ant to be something it is not. Also, most of our code base is in Python > and I'd really like to stay that way, of possible. > > Are there any systems out there that will allow me to run multiple > programs as one process? We could write our own, of course, and the > Twisted package looks like it would be fun to use. Or, is Ant a viable > solution to our problem? > > Your constructive comments would be appreciated > > Greg Lindstrom > Novasys Health > Little Rock, Arkansas > Take a look at fabric. It was designed for installation, but it might do what you want. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From alfps at start.no Tue Jan 26 14:51:38 2010 From: alfps at start.no (Alf P. Steinbach) Date: Tue, 26 Jan 2010 20:51:38 +0100 Subject: deriving from array.array In-Reply-To: References: Message-ID: * Torsten Mohr: > > thanks a lot for your hint, it works fine. > >>> I get an error for "a = Abc(4, 5)", seems the parameters are >>> forwarded to array's __init__ as they are. >> No, with CPython they're forwarded to __new__. > > Not sure if i understand this correctly, if i derive from other > classes (like wxpython widgets) i always call the bases __init__ . Well, creation of an object happens in two phases: allocation and initialization. Allocation reserves memory for the object and creates a sort of default object in that memory. This is the job of __new__. The object doesn't exist at all before __new__ is called. Initialization then outfits the object with all that's required for a proper object of class T. This is the job of __init__. __init__ as passed as argument the default object created by __new__. CPython's array.array does both in __new__. It does not override the __init__ it inherits from 'object'. To wit: >>> import array >>> id( array.array.__init__ ) 10965560 >>> id( object.__init__ ) 10965560 >>> array.array.__init__ is object.__init__ True >>> _ So overriding __init__ doesn't override any of array.array's functionality. But you'd have to override __new__ anyway, because with array.array the first argument has to be a type code specifying the element type of the array object the __new__ should create. An alternative design instead of those type codes could have had one subclass for each element type, essentially what you're doing, except you're only doing one of those subclasses. Then there would be no issue with __new__. >>> Though i explicitly >>> call __init__() for array. >> That's the constructor inherited from 'object', it takes no args (except >> the self arg). > > Is there a way to find out what i need to call? Testing. :-) > I haven't found much in > the documentation. From writing C extensions i knew about the "new" entry > in the PyTypeObject struct but it seems there's more behind it. > In docs.python.org i did not find much, is there an URL where i can read > more? Don't know, sorry. Google? Cheers, - Alf From malaclypse2 at gmail.com Tue Jan 26 14:53:02 2010 From: malaclypse2 at gmail.com (Jerry Hill) Date: Tue, 26 Jan 2010 14:53:02 -0500 Subject: deriving from array.array In-Reply-To: References: Message-ID: <16651e81001261153k7556ad50l79fe7b6afa12c4ff@mail.gmail.com> On Tue, Jan 26, 2010 at 2:28 PM, Torsten Mohr wrote: > Is there a way to find out what i need to call? I haven't found much in > the documentation. From writing C extensions i knew about the "new" entry > in the PyTypeObject struct but it seems there's more behind it. > In docs.python.org i did not find much, is there an URL where i can read > more? > http://docs.python.org/reference/datamodel.html#basic-customization talks about __new__() and __init__(). The basic rule of thumb is that when you inherit from an immutable type, you need to override __new__, and when you inherit from a mutable type you need to override __init__. I don't think that rule of thumb applies here, though, since an array.array is mutable. The actual rule is that you need to override __new__ if the class you're inheriting from overrides __new__. I don't think there's a straightforward way to know that ahead of time unless you look at the source code for the class. Either that, or you just see that you get a TypeError when initializing your object and realize it means you need to override the constructor. It would be nice if the TypeError was a little more explicit about what was going on, though. Here's what I get in IDLE for a class similar to yours: >>> import array >>> class my_a(array.array): def __init__(self, a, b): array.array.__init__(self, 'B') >>> a = my_a(1, 2) Traceback (most recent call last): File "", line 1, in a = my_a(1, 2) TypeError: array() argument 1 must be char, not int It would be nice if the traceback mentioned something about array.__new__() instead of just array(). -- Jerry -------------- next part -------------- An HTML attachment was scrubbed... URL: From john at castleamber.com Tue Jan 26 15:21:15 2010 From: john at castleamber.com (John Bokma) Date: Tue, 26 Jan 2010 14:21:15 -0600 Subject: Python or Ant References: <000e0cd70e4e3246b9047e15e0fb@google.com> Message-ID: <87ljfk1utg.fsf@castleamber.com> Steve Holden writes: Apologies for the piggy back, my Usenet provider can't get the OP's message. > gslindstrom at gmail.com wrote: >> My company is looking at creating a tool to allow us to define and >> manage a process for each job we run (a typical job may be look on a >> customers ftp site for a file, download it, decrypt it and load it into >> our database). We would like something which would allow us to glue >> together various existing processes we currently use into a single unit >> with multiple steps. Along the way, this new routine would need to log >> its progress and be able to report and even handle errors. A coworker >> has suggested we look at Ant ("Another Neat Tool") and, though it looks >> promising, I have reservations. If I recall correctly, it was intended >> as a replacement for "Make" and I worry that we may be trying to force >> Ant to be something it is not. It was intended as a replacement. But you can use it for many other things, without making it feel out of place in my opinion. I do use it exactly the way you describe for various tasks: calling external programs, uploading a file, etc. My personal site, see sig, is generated by a Perl program, and uploaded by another, and some other programs have to run as well, and all is glued together using ant. If I modify an XML file (my site is defined using XML), I do: ant local to update a local version. If I am happy with how it looks, I do ant upload Ant makes several things really easy. The things I do with it would take me more lines in either Perl or Python. So I use Perl (or Python) when it makes things easier, and ant to glue everything together if that's easier. -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From ursusmaximus at gmail.com Tue Jan 26 15:24:48 2010 From: ursusmaximus at gmail.com (Ron) Date: Tue, 26 Jan 2010 12:24:48 -0800 (PST) Subject: Sikuli: the coolest Python project I have yet seen... References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> <89c2550d-057d-4cc2-a855-d4f3f11cb272@b10g2000vbh.googlegroups.com> Message-ID: <73d864ff-bd11-46af-ba80-c5081189c92c@e25g2000yqh.googlegroups.com> On Jan 26, 10:59?am, CM wrote: > On Jan 24, 10:18?pm, Ron wrote: > > > Sikuli is the coolest Python project I have ever seen in my ten year > > hobbyist career. An MIT open source project, Sikuli uses Python to > > automate GUI tasks (in any GUI or GUI based app that runs the JVM) by > > simply drag and dropping GUI elements into Python scripts as function > > Arguments. Download athttp://sikuli.csail.mit.edu/Ialso did this > > podcast about Sikulihttp://media.libsyn.com/media/awaretek/Python411_20100124_Sikuli.mp3 > > How is this preferable to a macro recorder? Well, the pattern recognition engine allows you to recognize partial matches to any image, to any desired degree of accuracy. In other words, you can specify to take action only when an exact match is found, or else when a 50% match is found. This allows applications like the baby monitor (to tell you when your sleeping baby wakes up, and the imminent bus arrival monitor (to tell you when your bus is within one mile or any distance you want), and the route mapper from one city to another on a digital map. Another thing, Sikuli works with web pages. In other words, you can automate interaction with web sites as well as with desktop applications. You can also automate the entry of text. And it works (theoretically) on any graphical platform (Mac, Linux, Window, smartphones, etc). Probably other advantages. Those are just the ones I see off the top of my head. Good question. Ron From clp2 at rebertia.com Tue Jan 26 15:31:24 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Tue, 26 Jan 2010 12:31:24 -0800 Subject: Python or Ant In-Reply-To: <000e0cd70e4e3246b9047e15e0fb@google.com> References: <000e0cd70e4e3246b9047e15e0fb@google.com> Message-ID: <50697b2c1001261231g4a7dd9a4w388c5ac766fe6d84@mail.gmail.com> On Tue, Jan 26, 2010 at 10:58 AM, wrote: > My company is looking at creating a tool to allow us to define and manage a > process for each job we run (a typical job may be look on a customers ftp > site for a file, download it, decrypt it and load it into our database). We > would like something which would allow us to glue together various existing > processes we currently use into a single unit with multiple steps. Along the > way, this new routine would need to log its progress and be able to report > and even handle errors. A coworker has suggested we look at Ant ("Another > Neat Tool") and, though it looks promising, I have reservations. If I recall > correctly, it was intended as a replacement for "Make" and I worry that we > may be trying to force Ant to be something it is not. Also, most of our code > base is in Python and I'd really like to stay that way, of possible. > > Are there any systems out there that will allow me to run multiple programs > as one process? We could write our own, of course, and the Twisted package > looks like it would be fun to use. Or, is Ant a viable solution to our > problem? > > Your constructive comments would be appreciated There are several make-replacements written in Python. They could be an option. Here's a list of some of them (courtesy some googling): - Paver (http://www.blueskyonmars.com/projects/paver/) - SCons (http://www.scons.org/) - Vellum (https://launchpad.net/vellum) - Aap (http://www.a-a-p.org/) (List is in no particular order and likely incomplete; I have not tried any of these myself.) Cheers, Chris -- http://blog.rebertia.com From javier.collado at gmail.com Tue Jan 26 15:42:31 2010 From: javier.collado at gmail.com (Javier Collado) Date: Tue, 26 Jan 2010 21:42:31 +0100 Subject: Python or Ant In-Reply-To: <50697b2c1001261231g4a7dd9a4w388c5ac766fe6d84@mail.gmail.com> References: <000e0cd70e4e3246b9047e15e0fb@google.com> <50697b2c1001261231g4a7dd9a4w388c5ac766fe6d84@mail.gmail.com> Message-ID: Hello, One tool that I really like is doit: http://python-doit.sourceforge.net/ If you need to execute jobs remotely, you may like to take a look at STAF: http://staf.sourceforge.net/index.php Best regards, Javier 2010/1/26 Chris Rebert : > On Tue, Jan 26, 2010 at 10:58 AM, ? wrote: >> My company is looking at creating a tool to allow us to define and manage a >> process for each job we run (a typical job may be look on a customers ftp >> site for a file, download it, decrypt it and load it into our database). We >> would like something which would allow us to glue together various existing >> processes we currently use into a single unit with multiple steps. Along the >> way, this new routine would need to log its progress and be able to report >> and even handle errors. A coworker has suggested we look at Ant ("Another >> Neat Tool") and, though it looks promising, I have reservations. If I recall >> correctly, it was intended as a replacement for "Make" and I worry that we >> may be trying to force Ant to be something it is not. Also, most of our code >> base is in Python and I'd really like to stay that way, of possible. >> >> Are there any systems out there that will allow me to run multiple programs >> as one process? We could write our own, of course, and the Twisted package >> looks like it would be fun to use. Or, is Ant a viable solution to our >> problem? >> >> Your constructive comments would be appreciated > > There are several make-replacements written in Python. They could be an option. > > Here's a list of some of them (courtesy some googling): > - Paver (http://www.blueskyonmars.com/projects/paver/) > - SCons (http://www.scons.org/) > - Vellum (https://launchpad.net/vellum) > - Aap (http://www.a-a-p.org/) > > (List is in no particular order and likely incomplete; I have not > tried any of these myself.) > > Cheers, > Chris > -- > http://blog.rebertia.com > -- > http://mail.python.org/mailman/listinfo/python-list > From mal at egenix.com Tue Jan 26 15:52:31 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 26 Jan 2010 21:52:31 +0100 Subject: Authenticated encryption with PyCrypto In-Reply-To: <4B5F2871.6010102@egenix.com> References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> <4B5F2871.6010102@egenix.com> Message-ID: <4B5F560F.2010201@egenix.com> M.-A. Lemburg wrote: > Daniel wrote: >> Just got done reading this thread: >> >> http://groups.google.com/group/comp.lang.python/browse_thread/thread/b31a5b5f58084f12/0e09f5f5542812c3 >> >> and I'd appreciate feedback on this recipe: >> >> http://code.activestate.com/recipes/576980/ >> > [...] > You are also using CBC mode, even though you are really after > ECB mode (your code doesn't use chaining). With ECB mode, you > don't need the IV string. Sorry. Forget that last comment - your code does use chaining. It's hidden away in the PyCrypto code for block cipher encoding: the .encode() method will automatically apply the cipher to all blocks of 32 byte input data and apply chaining to all of them (initializing the data area with the IV string). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 26 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From mal at egenix.com Tue Jan 26 16:00:22 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 26 Jan 2010 22:00:22 +0100 Subject: Authenticated encryption with PyCrypto In-Reply-To: References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> <4B5F2871.6010102@egenix.com> Message-ID: <4B5F57E6.9050200@egenix.com> geremy condra wrote: > On Tue, Jan 26, 2010 at 12:37 PM, M.-A. Lemburg wrote: > > > >> You are also using CBC mode, even though you are really after >> ECB mode (your code doesn't use chaining). With ECB mode, you >> don't need the IV string. > > However, ECB mode is not as secure- the IV is the right way to go > here. Right - I forgot that PyCrypto applies the chaining internally when being passed data of more than 32 bytes. > I'd also note that you aren't supposed to use RandomPool anymore, > and that AES-192 is frequently recommended over AES-256 for > new applications due to a number of recent developments in > the cryptanalysis of its key schedule. Do you have pointers for this ? I could only find http://en.wikipedia.org/wiki/Advanced_Encryption_Standard#Security and https://cryptolux.org/Block My reading of their FAQ (https://cryptolux.org/FAQ_on_the_attacks) is that using AES-128 is the way to go (and it's faster too) - at least for the time being. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 26 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From tjreedy at udel.edu Tue Jan 26 16:09:25 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 26 Jan 2010 16:09:25 -0500 Subject: Sikuli: the coolest Python project I have yet seen... In-Reply-To: <89c2550d-057d-4cc2-a855-d4f3f11cb272@b10g2000vbh.googlegroups.com> References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> <89c2550d-057d-4cc2-a855-d4f3f11cb272@b10g2000vbh.googlegroups.com> Message-ID: On 1/26/2010 1:59 PM, CM wrote: > On Jan 24, 10:18 pm, Ron wrote: >> Sikuli is the coolest Python project I have ever seen in my ten year >> hobbyist career. An MIT oepn source project, Sikuli uses Python to >> automate GUI tasks (in any GUI or GUI baed app that runs the JVM) by >> simply drag and dropping GUI elements into Python scripts as function >> arguments. Download athttp://sikuli.csail.mit.edu/I also did this >> podcast about Sikulihttp://media.libsyn.com/media/awaretek/Python411_20100124_Sikuli.mp3 > > How is this preferable to a macro recorder? One can add program logic to the interaction. For instance, one of the demos on YouTube uses sikuli to read a Bejeweled (game) board. Program logic then calculates a move. From tjreedy at udel.edu Tue Jan 26 16:22:53 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 26 Jan 2010 16:22:53 -0500 Subject: My experiences building a small app on Python In-Reply-To: <62c367db-49e3-485b-80d6-ed2f99065f48@22g2000yqr.googlegroups.com> References: <62c367db-49e3-485b-80d6-ed2f99065f48@22g2000yqr.googlegroups.com> Message-ID: On 1/26/2010 8:51 AM, Cascade3891 wrote: > Hello Python community, > > I built a small app with Python for the first time. It has served a > very useful and productive purpose for me and my colleagues at work. > > I have shared my experiences on the following link due to the better > web presentation tools and coding presentation tools: > http://www.daniweb.com/forums/thread255458.html > > It's a bit of a read. But insightful. > > If you have any feedback or tips and advice, please let me know. When one has multiple small images that are all the same size, a standard trick is to have *one* image (and one file -- a lossless .bmp or .png) that is a tiled composite of the images. Read in the image and 'snip' apart the tiles. I have also seen bitmaps encoded in code as a hex string (for use with tk), but I do not know how well that would work with wx. > I've fallen in love with programming because of Python. =) You are not the only one. Welcome to the club. Terry Jan Reedy From millerdev at gmail.com Tue Jan 26 17:51:00 2010 From: millerdev at gmail.com (Daniel) Date: Tue, 26 Jan 2010 14:51:00 -0800 (PST) Subject: Authenticated encryption with PyCrypto References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> Message-ID: <837d2986-29b4-4cf7-b6a7-347008281d11@p6g2000vbl.googlegroups.com> On Jan 26, 12:37?pm, "M.-A. Lemburg" wrote: > Note that your code has a padding bug: the decoder doesn't > undo the padding. You're lucky though, since pickle will only > read as much data as it needs and not complain about the extra > data it finds. Doesn't the last line in decrypt() do it? return data[:-ord(data[-1])] Given, it's a bit cryptic... no pun intended :) > > Also, slightly related, is there an easy way to get the sha/md5 > > deprecation warnings emitted by PyCrypto in Python 2.6 to go away? > > Yes: you silence them via the warnings module. I suppose that the > latest version of PyCrypto fixes these warnings. The version that gets installed by easy_install or pip (2.0.1) emits those warnings. Is there a more recent version? Thanks for the feedback. ~ Daniel From mal at egenix.com Tue Jan 26 18:22:15 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 27 Jan 2010 00:22:15 +0100 Subject: Authenticated encryption with PyCrypto In-Reply-To: <837d2986-29b4-4cf7-b6a7-347008281d11@p6g2000vbl.googlegroups.com> References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> <837d2986-29b4-4cf7-b6a7-347008281d11@p6g2000vbl.googlegroups.com> Message-ID: <4B5F7927.1010900@egenix.com> Daniel wrote: > On Jan 26, 12:37 pm, "M.-A. Lemburg" wrote: >> Note that your code has a padding bug: the decoder doesn't >> undo the padding. You're lucky though, since pickle will only >> read as much data as it needs and not complain about the extra >> data it finds. > > Doesn't the last line in decrypt() do it? > > return data[:-ord(data[-1])] > > Given, it's a bit cryptic... no pun intended :) That's cryptic indeed... I just found that you're not padding with zero bytes, but instead with char(pad) where pad is the number of bytes you add: pad = AES_BLOCK_SIZE - len(data) % AES_BLOCK_SIZE data = data + pad * chr(pad) This code will pad with 16 bytes of chr(16) in case len(data) is in fact on a block size boundary. When using pickle, you don't need this, since pickle includes all necessary length information in the serialized data stream. I'd just pad with \0 and not worry about the extra bytes at the end when using pickle to serialize the objects. It's more important to worry about whether you really want to unpickle the data or not, since pickle opens up lots of possibilities of executing code on the decoding side of the communication channel. >>> Also, slightly related, is there an easy way to get the sha/md5 >>> deprecation warnings emitted by PyCrypto in Python 2.6 to go away? >> >> Yes: you silence them via the warnings module. I suppose that the >> latest version of PyCrypto fixes these warnings. > > The version that gets installed by easy_install or pip (2.0.1) emits > those warnings. Is there a more recent version? This is the most recent version: http://www.dlitz.net/software/pycrypto/ -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 27 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From aahz at pythoncraft.com Tue Jan 26 18:36:26 2010 From: aahz at pythoncraft.com (Aahz) Date: 26 Jan 2010 15:36:26 -0800 Subject: Splitting text at whitespace but keeping the whitespace in thereturned list References: <1264352736.30280.1356298585@webmail.messagingengine.com> Message-ID: In article , Roy Smith wrote: > >I've never used partition() before; my first thought on reading the above >was, "That's weird, it should be returning a list". Then I went and looked >at the docs. Given the description (returns specifically a 3-tuple), I >guess a tuple makes sense, but now I'm wondering what the use case was for >this method when it was invented? http://docs.python.org/whatsnew/2.5.html -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From xahlee at gmail.com Tue Jan 26 18:47:35 2010 From: xahlee at gmail.com (Xah Lee) Date: Tue, 26 Jan 2010 15:47:35 -0800 (PST) Subject: python 3's adoption Message-ID: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Some thoughts about Python 3 Adoption. Xah Lee, 2010-01-26 Some notes of Wikipedia readings related to Python. Unladen Swallow, a new project from Google. It is a new python compiler with the goal of 5 times faster than the de facto standand implementation CPython. Also note Stackless Python, which is already been used in some major commercial projects. Was looking into what's new in Python 3. See: http://docs.python.org/dev/3.0/whatsnew/3.0.html. >From a quick reading, i don't really like it. Here's some highlights: * Print is now a function. Great, much improvement. * Many functions that return lists now returns ?Views? or ?Iterators? Instead. A fucking fuck all fucked up shit. A extraneous ?oop engineering? complication. (See: Lambda in Python 3000) * The cmp() function used in sort is basically gone, users are now supposed to use the ?key? parameter instead. This is a flying-face- fuck to computer science. This would be the most serious fuckup in python 3. (See: Sorting in Python and Perl) * Integers by default is long. Great! * Much more integrated unicode support, rewrite of most its text or string semantics. Fantastic. Finally. Am looking because i wonder if i should switch to python 3 for my own few scripts, and rewrite my Python Tutorial for version 3. Am also interested to know how python 3 is received by the computing industry. Apparantly, a little search on the web indicates that vast majority of python base have not switched, as expected, for many good reasons. Vast majority of major python modules and tools have not switched. Most linux distro have not switched, i don't find any large corporation having adopted Python 3 (Google, Yahoo, Facebook, NASA,... ). (sources: Source, Source) Basically, such a incompatible change with trivial, ideological improvements, is too costy to switch. I wonder, if by 2015, will most large corporate users have switched to python 3. I give it a maybe. In today's Proliferation of Computing Languages, such a major antic by Guido can just hurt itself. What is he thinking? He of course thought himself as a god of lang designer, who sincerely wants to push towards perfection, all future-looking. Unfortunately, the tens of other major language designers all think similarly. perm archive of this post with possible updates here: http://xahlee.org/comp/python3.html Any comment on this? Xah ? http://xahlee.org/ ? From millerdev at gmail.com Tue Jan 26 19:07:23 2010 From: millerdev at gmail.com (Daniel) Date: Tue, 26 Jan 2010 16:07:23 -0800 (PST) Subject: Authenticated encryption with PyCrypto References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> <4B5F2871.6010102@egenix.com> Message-ID: <0eb13c9a-29d0-4019-b221-794fdb500213@22g2000yqr.googlegroups.com> geremy condra wrote: > I'd also note that you aren't supposed to use RandomPool anymore, OK, I updated the recipe to use os.urandom() > and that AES-192 is frequently recommended over AES-256 for > new applications due to a number of recent developments in > the cryptanalysis of its key schedule. Changed easily enough. The updated recipe defaults to AES-192. I also made one other minor tweak: the global constants were moved to class-level so they can be overridden more easily if needed. Daniel From millerdev at gmail.com Tue Jan 26 19:23:02 2010 From: millerdev at gmail.com (Daniel) Date: Tue, 26 Jan 2010 16:23:02 -0800 (PST) Subject: Authenticated encryption with PyCrypto References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> <837d2986-29b4-4cf7-b6a7-347008281d11@p6g2000vbl.googlegroups.com> Message-ID: <7169bb89-c22b-4826-83bc-b72b77150c30@e25g2000yqh.googlegroups.com> M.-A. Lemburg wrote: > Daniel wrote: > > On Jan 26, 12:37 pm, "M.-A. Lemburg" wrote: > >> Note that your code has a padding bug: the decoder doesn't > >> undo the padding. You're lucky though, since pickle will only > >> read as much data as it needs and not complain about the extra > >> data it finds. > > > Doesn't the last line in decrypt() do it? > > > ? ? return data[:-ord(data[-1])] > > > Given, it's a bit cryptic... no pun intended :) > > That's cryptic indeed... I just found that you're not padding > with zero bytes, but instead with char(pad) where pad is the > number of bytes you add: > > ? ? ? ? pad = AES_BLOCK_SIZE - len(data) % AES_BLOCK_SIZE > ? ? ? ? data = data + pad * chr(pad) > > This code will pad with 16 bytes of chr(16) in case len(data) > is in fact on a block size boundary. > > When using pickle, you don't need this, since pickle includes > all necessary length information in the serialized data stream. > > I'd just pad with \0 and not worry about the extra bytes > at the end when using pickle to serialize the objects. I think I'll leave the padding in there since it keeps the encrypt/ decrypt methods usable for non-pickle data. > It's more important to worry about whether you really > want to unpickle the data or not, since pickle opens > up lots of possibilities of executing code on the decoding > side of the communication channel. I understand the risks of unpickle. With strong, authenticated encryption I think it is reasonably safe to send an encrypted pickle through an untrusted medium (the Internet) and know that it has not been modified enroute. That is, unless someone has obtained the key, in which case I have a bigger problem to worry about. > >>> Also, slightly related, is there an easy way to get the sha/md5 > >>> deprecation warnings emitted by PyCrypto in Python 2.6 to go away? > > >> Yes: you silence them via the warnings module. I suppose that the > >> latest version of PyCrypto fixes these warnings. > > > The version that gets installed by easy_install or pip (2.0.1) emits > > those warnings. Is there a more recent version? > > This is the most recent version: > > ? ? ? ?http://www.dlitz.net/software/pycrypto/ Thanks. Daniel From fordhaivat at gmail.com Tue Jan 26 19:36:24 2010 From: fordhaivat at gmail.com (Someone Something) Date: Tue, 26 Jan 2010 19:36:24 -0500 Subject: Just drawing lines and plotting points? Message-ID: Hello, I need a python library that makes drawing lines and plotting points (these two things are the only things I need to do) easy. Or, how can I do something like this with pygame? Basically, what I want to do is make graphs. In pygame, since the coordinate system switches the x's and the y's I would have to switch them again when I plot my points so my graphs look okay. I hope this was enough info for me to get a good answer. -------------- next part -------------- An HTML attachment was scrubbed... URL: From clp2 at rebertia.com Tue Jan 26 19:54:56 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Tue, 26 Jan 2010 16:54:56 -0800 Subject: Just drawing lines and plotting points? In-Reply-To: References: Message-ID: <50697b2c1001261654i7a64adaai3df49eac8a859eae@mail.gmail.com> On Tue, Jan 26, 2010 at 4:36 PM, Someone Something wrote: > Hello, > > I need a python library that makes drawing lines and plotting points (these > two things are the only things I need to do) easy. Or, how can I do > something like this with pygame? Basically, what I want to do is make > graphs. In pygame, since the coordinate system switches the x's and the y's > I would have to switch them again when I plot my points so my graphs look > okay. I hope this was enough info for me to get a good answer. matplotlib, perhaps?: http://matplotlib.sourceforge.net/ Cheers, Chris -- http://blog.rebertia.com From aharrisreid at googlemail.com Tue Jan 26 19:55:41 2010 From: aharrisreid at googlemail.com (Alan Harris-Reid) Date: Wed, 27 Jan 2010 00:55:41 +0000 Subject: python 3's adoption In-Reply-To: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: <4B5F8F0D.3060205@googlemail.com> Xah Lee wrote: > Some thoughts about Python 3 Adoption. > > Xah Lee, 2010-01-26 > > Some notes of Wikipedia readings related to Python. > > Unladen Swallow, a new project from Google. It is a new python > compiler with the goal of 5 times faster than the de facto standand > implementation CPython. Also note Stackless Python, which is already > been used in some major commercial projects. > > Was looking into what's new in Python 3. See: http://docs.python.org/dev/3.0/whatsnew/3.0.html. > >From a quick reading, i don't really like it. Here's some highlights: > > * Print is now a function. Great, much improvement. > * Many functions that return lists now returns ?Views? or > ?Iterators? Instead. A fucking fuck all fucked up shit. A extraneous > ?oop engineering? complication. (See: Lambda in Python 3000) > * The cmp() function used in sort is basically gone, users are now > supposed to use the ?key? parameter instead. This is a flying-face- > fuck to computer science. This would be the most serious fuckup in > python 3. (See: Sorting in Python and Perl) > * Integers by default is long. Great! > * Much more integrated unicode support, rewrite of most its text > or string semantics. Fantastic. Finally. > > Am looking because i wonder if i should switch to python 3 for my own > few scripts, and rewrite my Python Tutorial for version 3. Am also > interested to know how python 3 is received by the computing industry. > Apparantly, a little search on the web indicates that vast majority of > python base have not switched, as expected, for many good reasons. > Vast majority of major python modules and tools have not switched. > Most linux distro have not switched, i don't find any large > corporation having adopted Python 3 (Google, Yahoo, Facebook, > NASA,... ). (sources: Source, Source) Basically, such a incompatible > change with trivial, ideological improvements, is too costy to switch. > > I wonder, if by 2015, will most large corporate users have switched to > python 3. I give it a maybe. In today's Proliferation of Computing > Languages, such a major antic by Guido can just hurt itself. What is > he thinking? He of course thought himself as a god of lang designer, > who sincerely wants to push towards perfection, all future-looking. > Unfortunately, the tens of other major language designers all think > similarly. > > perm archive of this post with possible updates here: > http://xahlee.org/comp/python3.html > > Any comment on this? > > Xah > ? http://xahlee.org/ Hello Xah, I have no figures to base this on (just what I have read on the web), but although the vast majority of comanies with big Python investments are probably waiting for the 'critical mass' to use Python3 regularly (oil-tanker effect), I would like to think that smaller operations are experimenting with it more-and-more. I think that for beginners who have dived into Python in the last 6-12 months (like me), it doesn't make sense to start with an older version. I do not want to learn 'old' syntax and functions of a language, only to have to learn the new versions when most developers upgrade to 3 - I might as well learn the new syntax now and be ahead-of-the-game ;-) . I know my choice of related packages (web-framework, ORM, templating-engine) is very limited at present, but there are enough branches and beta-versions around to help me with my learning-curve, and I figure there will be some full-production-releases around by the time I am 'fluent' with Python (12-18 months?). Currently I am using Python 3.1 and CherryPy (3.20 rc1) every day, and have had no serious problems (yet). I would be interested to hear how other people are using Python 3, and with what compatible packages. Regards, Alan Harris-Reid From alfps at start.no Tue Jan 26 20:28:00 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 02:28:00 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: I'm responding to the original message by Xah Lee, which is not carried by my Usenet provider. * Alan Harris-Reid: > Xah Lee wrote: >> Some thoughts about Python 3 Adoption. >> >> Xah Lee, 2010-01-26 >> >> Some notes of Wikipedia readings related to Python. >> >> Unladen Swallow, a new project from Google. It is a new python >> compiler with the goal of 5 times faster than the de facto standand >> implementation CPython. Also note Stackless Python, which is already >> been used in some major commercial projects. >> >> Was looking into what's new in Python 3. See: >> http://docs.python.org/dev/3.0/whatsnew/3.0.html. >> >From a quick reading, i don't really like it. Here's some highlights: >> >> * Print is now a function. Great, much improvement. Actually not, IMHO. All it does is is to provide incompatibility. They forgot Ronald Reagan's old maxim: if it don't need fixin', don't fix it. >> * Many functions that return lists now returns ?Views? or >> ?Iterators? Instead. A fucking fuck all fucked up shit. A extraneous >> ?oop engineering? complication. (See: Lambda in Python 3000) On the contrary, this is a great improvement. It makes no sense to have to write more code and less readable code for the common efficient case. And the default, the way that takes 0.5 seconds less to do, does have a great influence on what people choose, even in elections (Obama's latest "opt out" proposal is an example that this principle is recognized even by the President of the United States). When you want an explicit collection such as a 'list', introducing some overhead for presumably a good reason, it's as easy as e.g. 'list(range(42))'. >> * The cmp() function used in sort is basically gone, users are now >> supposed to use the ?key? parameter instead. This is a flying-face- >> fuck to computer science. This would be the most serious fuckup in >> python 3. (See: Sorting in Python and Perl) I agree. :-) Probably there must have been some rationale, but to put it bluntly removing the comparator is more like moronic than pythonic. If the rationale was efficiency, then a rational solution could be to provide two sort methods, like 'sort' using direct comparisions and 'custom_sort' using a specified comparator. Come to think of it, that door is still open, except for the language spec freeze. It's possible to work around the removal of 'cmp' in various kludgy ways, but you shouldn't have to "work around" a library sort routine. >> * Integers by default is long. Great! Yes, totally agree. Nothing confuses the newbie or the now-and-then programmer so much as limited range integers. Well, perhaps floating point, but... >> * Much more integrated unicode support, rewrite of most its text >> or string semantics. Fantastic. Finally. I'm not sure if the Unicode support is fantastic. I have the impression that there are still some edges to iron out and corners to smooth, + bugs to kill. But it's Very Good. [snip] Cheers, - Alf From invalid at invalid.invalid Tue Jan 26 20:38:37 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Wed, 27 Jan 2010 01:38:37 +0000 (UTC) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: On 2010-01-27, Alf P. Steinbach wrote: > I'm responding to the original message by Xah Lee, which is > not carried by my Usenet provider. A Usenet provider that doesn't carry messages from Xah Lee. So... many... jokes..... -- Grant From wuwei23 at gmail.com Tue Jan 26 20:54:15 2010 From: wuwei23 at gmail.com (alex23) Date: Tue, 26 Jan 2010 17:54:15 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> "Alf P. Steinbach" wrote: > Actually not, IMHO. All it does is is to provide incompatibility. They forgot > Ronald Reagan's old maxim: if it don't need fixin', don't fix it. [...] > Probably there must have been some rationale, but to put it bluntly removing the > comparator is more like moronic than pythonic. So in both cases, you haven't put any effort into researching why these decisions were made, but you're happy to jump to conclusions about the intelligence of the implementer(s) while regularly trotting out complaints of ad hominem attacks against yourself. The saddest part of all is you seem to consider yourself some kind of educator. From python at mrabarnett.plus.com Tue Jan 26 21:03:32 2010 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 27 Jan 2010 02:03:32 +0000 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: <4B5F9EF4.7070304@mrabarnett.plus.com> Alf P. Steinbach wrote: > I'm responding to the original message by Xah Lee, which is not carried > by my Usenet provider. > > * Alan Harris-Reid: >> Xah Lee wrote: >>> Some thoughts about Python 3 Adoption. >>> >>> Xah Lee, 2010-01-26 >>> >>> Some notes of Wikipedia readings related to Python. >>> >>> Unladen Swallow, a new project from Google. It is a new python >>> compiler with the goal of 5 times faster than the de facto standand >>> implementation CPython. Also note Stackless Python, which is already >>> been used in some major commercial projects. >>> >>> Was looking into what's new in Python 3. See: >>> http://docs.python.org/dev/3.0/whatsnew/3.0.html. >>> >From a quick reading, i don't really like it. Here's some highlights: >>> >>> * Print is now a function. Great, much improvement. > > Actually not, IMHO. All it does is is to provide incompatibility. They > forgot Ronald Reagan's old maxim: if it don't need fixin', don't fix it. > It was an inconsistency in the language. BTW, the usual form of the maxim is: if it ain't broke, don't fix it. > >>> * Many functions that return lists now returns ?Views? or >>> ?Iterators? Instead. A fucking fuck all fucked up shit. A extraneous >>> ?oop engineering? complication. (See: Lambda in Python 3000) > > On the contrary, this is a great improvement. It makes no sense to have > to write more code and less readable code for the common efficient case. > And the default, the way that takes 0.5 seconds less to do, does have a > great influence on what people choose, even in elections (Obama's latest > "opt out" proposal is an example that this principle is recognized even > by the President of the United States). When you want an explicit > collection such as a 'list', introducing some overhead for presumably a > good reason, it's as easy as e.g. 'list(range(42))'. > > >>> * The cmp() function used in sort is basically gone, users are now >>> supposed to use the ?key? parameter instead. This is a flying-face- >>> fuck to computer science. This would be the most serious fuckup in >>> python 3. (See: Sorting in Python and Perl) > > I agree. :-) > > Probably there must have been some rationale, but to put it bluntly > removing the comparator is more like moronic than pythonic. If the > rationale was efficiency, then a rational solution could be to provide > two sort methods, like 'sort' using direct comparisions and > 'custom_sort' using a specified comparator. Come to think of it, that > door is still open, except for the language spec freeze. > > It's possible to work around the removal of 'cmp' in various kludgy > ways, but you shouldn't have to "work around" a library sort routine. > Why two sort methods? It would be simpler just to retain the 'cmp' argument, like it is in Python 2.6, but deprecated. > >>> * Integers by default is long. Great! > > Yes, totally agree. > > Nothing confuses the newbie or the now-and-then programmer so much as > limited range integers. > Limited range? That hasn't been the case with 'int' for long time! (It's automatically promoted to long.) [snip] From alfps at start.no Tue Jan 26 21:08:29 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 03:08:29 +0100 Subject: python 3's adoption In-Reply-To: <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> Message-ID: * alex23: > "Alf P. Steinbach" wrote: >> Actually not, IMHO. All it does is is to provide incompatibility. They forgot >> Ronald Reagan's old maxim: if it don't need fixin', don't fix it. > [...] >> Probably there must have been some rationale, but to put it bluntly removing the >> comparator is more like moronic than pythonic. > > So in both cases, you haven't put any effort into researching why > these decisions were made, but you're happy to jump to conclusions > about the intelligence of the implementer(s) while regularly trotting > out complaints of ad hominem attacks against yourself. > > The saddest part of all is you seem to consider yourself some kind of > educator. Hi "alex23". Ad hominem attacks like you try above, just reflect on the one doing it. Adding in some obviously false assertions just makes you look even worse. And posting anonymously, well, that article's got it all. Such attacks constitute noise and, except for how they reflect on you, only noise. Cheers & hth., - Alf From steve at holdenweb.com Tue Jan 26 21:09:53 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 26 Jan 2010 21:09:53 -0500 Subject: python 3's adoption In-Reply-To: <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> Message-ID: [Off-list] alex23 wrote: > "Alf P. Steinbach" wrote: >> Actually not, IMHO. All it does is is to provide incompatibility. They forgot >> Ronald Reagan's old maxim: if it don't need fixin', don't fix it. > [...] >> Probably there must have been some rationale, but to put it bluntly removing the >> comparator is more like moronic than pythonic. > > So in both cases, you haven't put any effort into researching why > these decisions were made, but you're happy to jump to conclusions > about the intelligence of the implementer(s) while regularly trotting > out complaints of ad hominem attacks against yourself. > > The saddest part of all is you seem to consider yourself some kind of > educator. ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From alfps at start.no Tue Jan 26 21:12:37 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 03:12:37 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> Message-ID: * Steve Holden: > [Off-list] > alex23 wrote: >> "Alf P. Steinbach" wrote: >>> Actually not, IMHO. All it does is is to provide incompatibility. They forgot >>> Ronald Reagan's old maxim: if it don't need fixin', don't fix it. >> [...] >>> Probably there must have been some rationale, but to put it bluntly removing the >>> comparator is more like moronic than pythonic. >> So in both cases, you haven't put any effort into researching why >> these decisions were made, but you're happy to jump to conclusions >> about the intelligence of the implementer(s) while regularly trotting >> out complaints of ad hominem attacks against yourself. >> >> The saddest part of all is you seem to consider yourself some kind of >> educator. > > ;-) Please don't post more noise and ad hominem attacks to the group, Steve. - Alf From debatem1 at gmail.com Tue Jan 26 21:43:17 2010 From: debatem1 at gmail.com (geremy condra) Date: Tue, 26 Jan 2010 21:43:17 -0500 Subject: Authenticated encryption with PyCrypto In-Reply-To: <7169bb89-c22b-4826-83bc-b72b77150c30@e25g2000yqh.googlegroups.com> References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> <837d2986-29b4-4cf7-b6a7-347008281d11@p6g2000vbl.googlegroups.com> <7169bb89-c22b-4826-83bc-b72b77150c30@e25g2000yqh.googlegroups.com> Message-ID: On Tue, Jan 26, 2010 at 7:23 PM, Daniel wrote: > I understand the risks of unpickle. With strong, authenticated > encryption I think it is reasonably safe to send an encrypted pickle > through an untrusted medium (the Internet) and know that it has not > been modified enroute. That is, unless someone has obtained the key, > in which case I have a bigger problem to worry about. > Not to sound pedantic, but the fact that somebody is executing exactly the malware they sent you is kind of cold comfort. Might I suggest that a warning to that effect would be prudent? Geremy Condra From john at castleamber.com Tue Jan 26 22:04:53 2010 From: john at castleamber.com (John Bokma) Date: Tue, 26 Jan 2010 21:04:53 -0600 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> Message-ID: <878wbk1c4q.fsf@castleamber.com> "Alf P. Steinbach" writes: > Please don't post more noise and ad hominem attacks to the group, > Steve. Funny that you talk about noise while replying yourself to noise. Xah Lee is just a pathetic spammer. He's not going to reply in this thread. He just shits out his stuff in as many groups as possible to promote his website. -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From pydota at gmail.com Tue Jan 26 22:11:10 2010 From: pydota at gmail.com (moon sky) Date: Wed, 27 Jan 2010 11:11:10 +0800 Subject: python for net manager Message-ID: <6e5636721001261911i6d5ca5fma2dff5b8a89ed277@mail.gmail.com> hi there, i just want to write a small tool to manage the network, including detect the ip addr,netmask, and send the arp request to find out locale's ip-mac turtple, is there any way to succeed this exlude 'subprocess call'? thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From alfps at start.no Tue Jan 26 22:12:07 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 04:12:07 +0100 Subject: python 3's adoption In-Reply-To: <878wbk1c4q.fsf@castleamber.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> <878wbk1c4q.fsf@castleamber.com> Message-ID: * John Bokma: > "Alf P. Steinbach" writes: > >> Please don't post more noise and ad hominem attacks to the group, >> Steve. > > Funny that you talk about noise while replying yourself to noise. Xah > Lee is just a pathetic spammer. He's not going to reply in this > thread. He just shits out his stuff in as many groups as possible to > promote his website. Sorry, I didn't know. Thanks for the heads-up. Cheers, - Alf From john at castleamber.com Tue Jan 26 22:18:04 2010 From: john at castleamber.com (John Bokma) Date: Tue, 26 Jan 2010 21:18:04 -0600 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> <878wbk1c4q.fsf@castleamber.com> Message-ID: <874om81bir.fsf@castleamber.com> "Alf P. Steinbach" writes: > * John Bokma: >> "Alf P. Steinbach" writes: >> >>> Please don't post more noise and ad hominem attacks to the group, >>> Steve. >> >> Funny that you talk about noise while replying yourself to noise. Xah >> Lee is just a pathetic spammer. He's not going to reply in this >> thread. He just shits out his stuff in as many groups as possible to >> promote his website. > > Sorry, I didn't know. > > Thanks for the heads-up. My apologies for the harsh tone of my reply. I am more than tired of how Xah Lee spams weekly, and shouldn't have assumed you were aware of that. -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From tjreedy at udel.edu Tue Jan 26 22:37:45 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 26 Jan 2010 22:37:45 -0500 Subject: python 3's adoption In-Reply-To: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: On 1/26/2010 6:47 PM, Xah Lee wrote: > Some thoughts about Python 3 Adoption. > > Xah Lee, 2010-01-26 > > Some notes of Wikipedia readings related to Python. > > Unladen Swallow, a new project from Google. It is a new python > compiler with the goal of 5 times faster than the de facto standand > implementation CPython. Also note Stackless Python, which is already > been used in some major commercial projects. > > Was looking into what's new in Python 3. See: http://docs.python.org/dev/3.0/whatsnew/3.0.html. > From a quick reading, i don't really like it. Here's some highlights: > > * Print is now a function. Great, much improvement. Something we agree on. For simple scripts, it is a bit of a nuisance, but it eliminates kludgy special-case syntax for options, which are now normal keyword arguments. > * Many functions that return lists now returns ?Views? or > ?Iterators? Instead. A fucking fuck all fucked up shit. A extraneous > ?oop engineering? complication. (See: Lambda in Python 3000) Something we disagree on. I consider the shift from list or iterator orientation a major advance. If you ever want to iterate a 10 million item dict or map or filter a 100 million item sequence, you might then see the point. > * The cmp() function used in sort is basically gone, users are now > supposed to use the ?key? parameter instead. This is a flying-face- > fuck to computer science. This would be the most serious fuckup in > python 3. (See: Sorting in Python and Perl) It is completely gone. Cmp was kludgy. Sort paid no attention to the tri-state return but only paid attention to the less-than branch. Which means a) that it was redundant with the less than method and that the effort to split == from > was completely wasted. It is difficult to find use cases where cmp= is superior to key= > * Integers by default is long. Great! > * Much more integrated unicode support, rewrite of most its text > or string semantics. Fantastic. Finally. What is missing now on Windows is better support from the OS, at least with XP. I have not tried 3.1 on Win7 yet (and never touched Vista). > > Am looking because i wonder if i should switch to python 3 for my own > few scripts, and rewrite my Python Tutorial for version 3. I am writing my book on algorithms using 3.1 because of the above and other improvements (consistent / and elimination of old classes, for instance). > Am also > interested to know how python 3 is received by the computing industry. > Apparantly, a little search on the web indicates that vast majority of > python base have not switched, Some people have panicked prematurely. Guido never expected that working old apps would all be switched. There is probably code still being run with 1.5 from 15 years ago. And there will be 2.7 code running at least that far in the future. Even before 3.0 was out, he also never expected major uptake until 3.2, which is a year away. 3.0 was, well, a .0 release with a couple of .0 problems. 3.1 is pretty solid and modules are appearing for it. > as expected, for many good reasons. > Vast majority of major python modules and tools have not switched. A few have, more will in the next year or after 3.2 is released, some never will. The most important one, in my opinion, is numpy, since some module authors who *want* to do a 3.x version are waiting on that. > Most linux distro have not switched, Have any switched to 2.6 yet, as the default? Some do have 3.1 version available. > i don't find any large > corporation having adopted Python 3 (Google, Yahoo, Facebook, > NASA,... ). (sources: Source, Source) Basically, such a incompatible > change with trivial, ideological improvements, is too costy to switch. The major reduction of redundancy by the overdue removal of obsolete features makes it a much nicer language to learn than, say, 2.6. It is much closer to the simplicity of 1.4, which is what I started with. Terry Jan Reedy From tjreedy at udel.edu Tue Jan 26 22:38:45 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 26 Jan 2010 22:38:45 -0500 Subject: Just drawing lines and plotting points? In-Reply-To: <50697b2c1001261654i7a64adaai3df49eac8a859eae@mail.gmail.com> References: <50697b2c1001261654i7a64adaai3df49eac8a859eae@mail.gmail.com> Message-ID: On 1/26/2010 7:54 PM, Chris Rebert wrote: > On Tue, Jan 26, 2010 at 4:36 PM, Someone Something wrote: >> Hello, >> >> I need a python library that makes drawing lines and plotting points (these >> two things are the only things I need to do) easy. Or, how can I do >> something like this with pygame? Basically, what I want to do is make >> graphs. In pygame, since the coordinate system switches the x's and the y's >> I would have to switch them again when I plot my points so my graphs look >> okay. I hope this was enough info for me to get a good answer. > > matplotlib, perhaps?: http://matplotlib.sourceforge.net/ or dislin? From debatem1 at gmail.com Tue Jan 26 22:55:16 2010 From: debatem1 at gmail.com (geremy condra) Date: Tue, 26 Jan 2010 22:55:16 -0500 Subject: Authenticated encryption with PyCrypto In-Reply-To: <4B5F57E6.9050200@egenix.com> References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> <4B5F2871.6010102@egenix.com> <4B5F57E6.9050200@egenix.com> Message-ID: On Tue, Jan 26, 2010 at 4:00 PM, M.-A. Lemburg wrote: > > > geremy condra wrote: >> On Tue, Jan 26, 2010 at 12:37 PM, M.-A. Lemburg wrote: >> >> >> >>> You are also using CBC mode, even though you are really after >>> ECB mode (your code doesn't use chaining). With ECB mode, you >>> don't need the IV string. >> >> However, ECB mode is not as secure- the IV is the right way to go >> here. > > Right - I forgot that PyCrypto applies the chaining internally > when being passed data of more than 32 bytes. > >> I'd also note that you aren't supposed to use RandomPool anymore, >> and that AES-192 is frequently recommended over AES-256 for >> new applications due to a number of recent developments in >> the cryptanalysis of its key schedule. > > Do you have pointers for this ? > > I could only find > http://en.wikipedia.org/wiki/Advanced_Encryption_Standard#Security > and > https://cryptolux.org/Block > Sure. The best source is the papers themselves: http://eprint.iacr.org/2009/374.pdf http://eprint.iacr.org/2009/317.pdf http://eprint.iacr.org/2009/242.pdf If you're interested I can provide more. The other thing to keep in mind is that several of the new attacks abuse algebraic properties of the key schedule, and that older attacks like XSL (eXtended Sparse Linearisation) and models (like the BES) are useful tools. There's also been some recent research into using bent functions for cryptanalysis, including cryptanalysis over GF(p), which falls into the lap of older models like the BES, which views AES in terms of GF(2^8), although I don't seem to be able to find free links to those papers. > My reading of their FAQ (https://cryptolux.org/FAQ_on_the_attacks) > is that using AES-128 is the way to go (and it's faster too) - at > least for the time being. In applications where speed is not of the essence, there doesn't appear to be a good reason to prefer AES-128 to AES-192, and while the margin is small, recent attacks have consistently led to results that are 2^10 or 2^12 larger for 192 than for 128. I also doubt highly that the wells opened by these newer results are the end of this question- so my advice would be to use 192 where possible, and keep an eye out just in case. Geremy Condra From rantingrick at gmail.com Tue Jan 26 23:50:08 2010 From: rantingrick at gmail.com (rantingrick) Date: Tue, 26 Jan 2010 20:50:08 -0800 (PST) Subject: Just drawing lines and plotting points? References: <50697b2c1001261654i7a64adaai3df49eac8a859eae@mail.gmail.com> Message-ID: On Jan 26, 9:38?pm, Terry Reedy wrote: > On 1/26/2010 7:54 PM, Chris Rebert wrote: > > > On Tue, Jan 26, 2010 at 4:36 PM, Someone Something ?wrote: > >> Hello, > > >> I need a python library that makes drawing lines and plotting points (these > >> two things are the only things I need to do) easy. Or, how can I do > >> something like this with pygame? Basically, what I want to do is make > >> graphs. In pygame, since the coordinate system switches the x's and the y's > >> I would have to switch them again when I plot my points so my graphs look > >> okay. I hope this was enough info for me to get a good answer. > > > matplotlib, perhaps?:http://matplotlib.sourceforge.net/ > > or dislin? it doesn't get any easier than the tkCanvas widget... import Tkinter as tk #import tkinter as tk #python3.0+ app = tk.Tk() canvas = tk.Canvas(app) canvas.pack() canvas.create_line(sx,sy, ex,ey) #... #... app.mainloop() From alfps at start.no Tue Jan 26 23:52:13 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 05:52:13 +0100 Subject: Just drawing lines and plotting points? In-Reply-To: References: <50697b2c1001261654i7a64adaai3df49eac8a859eae@mail.gmail.com> Message-ID: * rantingrick: > On Jan 26, 9:38 pm, Terry Reedy wrote: >> On 1/26/2010 7:54 PM, Chris Rebert wrote: >> >>> On Tue, Jan 26, 2010 at 4:36 PM, Someone Something wrote: >>>> Hello, >>>> I need a python library that makes drawing lines and plotting points (these >>>> two things are the only things I need to do) easy. Or, how can I do >>>> something like this with pygame? Basically, what I want to do is make >>>> graphs. In pygame, since the coordinate system switches the x's and the y's >>>> I would have to switch them again when I plot my points so my graphs look >>>> okay. I hope this was enough info for me to get a good answer. >>> matplotlib, perhaps?:http://matplotlib.sourceforge.net/ >> or dislin? > > it doesn't get any easier than the tkCanvas widget... > > import Tkinter as tk > #import tkinter as tk #python3.0+ > app = tk.Tk() > canvas = tk.Canvas(app) > canvas.pack() > canvas.create_line(sx,sy, ex,ey) > #... > #... > app.mainloop() He he, it's even easier to draw a graph using the turtle module. Cheers, - Alf From rantingrick at gmail.com Wed Jan 27 00:05:46 2010 From: rantingrick at gmail.com (rantingrick) Date: Tue, 26 Jan 2010 21:05:46 -0800 (PST) Subject: Just drawing lines and plotting points? References: <50697b2c1001261654i7a64adaai3df49eac8a859eae@mail.gmail.com> Message-ID: <67a2a20b-b17d-4677-b8cc-4793adf077f4@m16g2000yqc.googlegroups.com> On Jan 26, 10:52?pm, "Alf P. Steinbach" wrote: > * rantingrick: > > > > > > > On Jan 26, 9:38 pm, Terry Reedy wrote: > >> On 1/26/2010 7:54 PM, Chris Rebert wrote: > > >>> On Tue, Jan 26, 2010 at 4:36 PM, Someone Something ?wrote: > >>>> Hello, > >>>> I need a python library that makes drawing lines and plotting points (these > >>>> two things are the only things I need to do) easy. Or, how can I do > >>>> something like this with pygame? Basically, what I want to do is make > >>>> graphs. In pygame, since the coordinate system switches the x's and the y's > >>>> I would have to switch them again when I plot my points so my graphs look > >>>> okay. I hope this was enough info for me to get a good answer. > >>> matplotlib, perhaps?:http://matplotlib.sourceforge.net/ > >> or dislin? > > > it doesn't get any easier than the tkCanvas widget... > > > import Tkinter as tk > > #import tkinter as tk #python3.0+ > > app = tk.Tk() > > canvas = tk.Canvas(app) > > canvas.pack() > > canvas.create_line(sx,sy, ex,ey) > > #... > > #... > > app.mainloop() > > He he, it's even easier to draw a graph using the turtle module. > > Cheers, > > - Alf Yes Alf please forgive me, i had completely forgotten about the "visual" learners amongst us. turtle, turtle, watch him go... turtle, turtle, why he so slow... turtle, turtle, not even he don't know...? ;-) From whyadd9 at gmail.com Wed Jan 27 00:26:14 2010 From: whyadd9 at gmail.com (Patrick) Date: Tue, 26 Jan 2010 21:26:14 -0800 (PST) Subject: scraping with urllib2 Message-ID: I'm trying to scrape the attached link for the price listed $99.99: http://bananarepublic.gap.com/browse/product.do?cid=41559&vid=1&pid=692392 I can see the price if I view the source(I even turned off java and javascript), but when I use urllib2, the price doesn't show up. Is there another library other than urllib2 that would work? From mallik.v.arjun at gmail.com Wed Jan 27 00:48:10 2010 From: mallik.v.arjun at gmail.com (=?UTF-8?B?TWFsbGlrYXJqdW4o4LKu4LKy4LON4LKy4LK/4LKV4LK+4LKw4LON4LKc4LOB4LKo4LONKQ==?=) Date: Wed, 27 Jan 2010 11:18:10 +0530 Subject: Simple Password Strength Checker Review Help needed Message-ID: <158b9ef31001262148g1926d067ja8ba840ba074eca4@mail.gmail.com> Dear friends, I am newbie to Python + pygtk + Glade and recently wrote a simple password strength checker app. Since this is my first app/program, can someone review my code (just over 150 lines) and help me improve my programming capabilities Thank you Mallikarjun -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PyPassChecker.glade Type: application/x-glade Size: 6363 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: somaSoftware.py Type: text/x-python Size: 5956 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PyPassChecker.xml Type: text/xml Size: 5445 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PyPassChecker.zip Type: application/zip Size: 3844 bytes Desc: not available URL: From mallik.v.arjun at gmail.com Wed Jan 27 00:56:21 2010 From: mallik.v.arjun at gmail.com (=?UTF-8?B?TWFsbGlrYXJqdW4o4LKu4LKy4LON4LKy4LK/4LKV4LK+4LKw4LON4LKc4LOB4LKo4LONKQ==?=) Date: Wed, 27 Jan 2010 11:26:21 +0530 Subject: Simple Password Strength Checker Review Help needed In-Reply-To: <158b9ef31001262148g1926d067ja8ba840ba074eca4@mail.gmail.com> References: <158b9ef31001262148g1926d067ja8ba840ba074eca4@mail.gmail.com> Message-ID: <158b9ef31001262156j18af3367sd05938338696a880@mail.gmail.com> I am really sorry, I uploaded a wrong python file. Here is the correct file 2010/1/27 Mallikarjun > Dear friends, > I am newbie to Python + pygtk + Glade and recently wrote a simple password > strength checker app. > Since this is my first app/program, can someone review my code (just over > 150 lines) and help me improve my programming capabilities > > Thank you > Mallikarjun > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PyPassChecker.py Type: text/x-python Size: 5578 bytes Desc: not available URL: From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 01:12:50 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 27 Jan 2010 06:12:50 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: On Wed, 27 Jan 2010 02:28:00 +0100, Alf P. Steinbach wrote: >>> * Print is now a function. Great, much improvement. > > Actually not, IMHO. All it does is is to provide incompatibility. They > forgot Ronald Reagan's old maxim: if it don't need fixin', don't fix it. print as a function is more consistent and more convenient than print as a statement. Or to put it another way, print as a statement is less consistent and convenient, and therefore in need of fixing. By Ronald Reagan's aphorism, it should be fixed, and now does. Of course there's a bit of a learning curve for those used to the old ways, but in two years from now, I expect no-one will remember what the fuss was about. Python 2.6 users will start their script with "from __future__ import print_function", Python 2.7 users may not even need to do that, and we'll all pity those stuck with 2.5 or 2.4. >>> * Many functions that return lists now returns ?Views? or >>> ?Iterators? Instead. A fucking fuck all fucked up shit. A extraneous >>> ?oop engineering? complication. (See: Lambda in Python 3000) > > On the contrary, this is a great improvement. Agreed. >>> * The cmp() function used in sort is basically gone, users are now >>> supposed to use the ?key? parameter instead. This is a flying-face- >>> fuck to computer science. This would be the most serious fuckup in >>> python 3. (See: Sorting in Python and Perl) > > I agree. :-) I did too, when I first heard cmp was to be dumped. But I changed my mind and now agree with the decision to drop cmp. Custom sorts are nearly always much better written with key rather than cmp: key adds an O(N) overheard to the sorting, while cmp makes sorting O(N**2). In other words, key is *much* faster, and generally easier to write as well. There may be a handful of rare applications where there is no obvious way to convert a cmp function to a key, but they are vanishingly rare. I think, if you search the archives, Paul Rubin may have come up with one example. There are a number of recipes for easily converting cmp sorts to key sorts. I think, in my perfect world, list.sort() and sorted() should continue being key based, while the standard library contained (perhaps in the functools module?) a sort function that contains all the bells and whistles. You want cmp, it's there, and you can pay the extra cost if you need it. You want to sort multiple lists by the contents of one? One import away. But keep the general sort() function lean and super-fast. -- Steven From no.email at nospam.invalid Wed Jan 27 01:23:11 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Tue, 26 Jan 2010 22:23:11 -0800 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: <7xr5pcyskw.fsf@ruckus.brouhaha.com> Steven D'Aprano writes: > print as a function is more consistent and more convenient than print as > a statement. Convenience is subjective, but the 3.x 'print' behavior is definitely inconsistent (i.e. different from 2.x). The change makes a lot of my code silently produce wrong results, too. I often print tuples to show what a program is doing: print (timestamp, 'transmogrified', blob) which in 2.x prints a parenthesized tuple that I can later read back in with eval. That line of code still prints a message, but in a different format, instead of throwing an error. From andreengels at gmail.com Wed Jan 27 01:45:48 2010 From: andreengels at gmail.com (Andre Engels) Date: Wed, 27 Jan 2010 07:45:48 +0100 Subject: scraping with urllib2 In-Reply-To: References: Message-ID: <6faf39c91001262245t7c99120fy1e3c990c079b7f61@mail.gmail.com> On Wed, Jan 27, 2010 at 6:26 AM, Patrick wrote: > I'm trying to scrape the attached link for the price listed $99.99: > http://bananarepublic.gap.com/browse/product.do?cid=41559&vid=1&pid=692392 > > I can see the price if I view the source(I even turned off java and > javascript), but when I use urllib2, the price doesn't show up. > > Is there another library other than urllib2 that would work? To see that page you need to accept cookies from the site and send them back. -- Andr? Engels, andreengels at gmail.com From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 01:57:06 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 27 Jan 2010 06:57:06 GMT Subject: Simple Password Strength Checker Review Help needed References: <158b9ef31001262148g1926d067ja8ba840ba074eca4@mail.gmail.com> Message-ID: On Wed, 27 Jan 2010 11:26:21 +0530, Mallikarjun wrote: I don't know much about pygtk and Glade, but I can make a couple of comments: > class PassChecker: > > def on_check_button_toggled(self, button): > if self.pass_entry.get_visibility(): > self.pass_entry.set_visibility(False) > else: > self.pass_entry.set_visibility(True) This can be written as: self.pass_entry.set_visibility(not self.pass_entry.get_visibility()) Better still, get rid of the "set_visiblity" and "get_visibility" methods. This isn't Java, setters and getters are rarely needed. Just make an attribute "visible" and toggle it: self.pass_entry.visible = not self.pass_entry.visible > def on_guidelines_button_clicked(self, button): > message = "Here are some guidelines to be followed while > choosing password.\n\ [...] It is best to write such long strings as a triple-quoted string, to avoid needing all those backslashes. Instead of: message = "Here are some guidelines... \n\ blah blah blah \n\ blah blah" it is better to write: message = """Here are some guidelines... blah blah blah blah blah""" > def on_pass_entry_changed(self, text_entry): > password = self.pass_entry.get_text() strength = > self.password_strength_checker(password) > > if strength >= 0.0 and strength <= 0.20: > strength_in_str = "Very Week" Python allows you to write multiple comparisons like this: if 0.0 <= strength <= 0.20: strength_in_str = "Very Weak" elif 0.20 < strength <= 0.40: strength_in_str = "Weak" Note also the correct spelling of weak. ("Week" is seven days.) > def password_strength_checker(self, password): > > if not password: > return 0.0 > > # Check for having same Characters > has_same_characters = True > first_letter = password[0] > for i in range(1, password.__len__()): You nearly never need to explicitly call double-underscore methods like __len__. The above is better written as: for i in range(1, len(password)): > if not first_letter == password[i]: > has_same_characters = False > if not has_same_characters: > break > if has_same_characters: > return 0.0 I'm not sure what you're trying to do here. By the names, I think you're looking for any repeated characters, but judging by the code, you're only checking to see if the *first* character is repeated. If you want to check only the first character, this is best written as: # get the first character, and see it if is in the rest of the string first_repeated = password[0] in password[1:] If you want to check for any duplicate: any_duplicate = len(set(password)) != len(password) If that's a bit too mysterious for you, try this: for c in password: # check each character for duplicates if password.count(c) != 1: print "character %s is duplicated" % c I'm sure you can adapt that to do what you want. > lower_alphabets_count = 0 > upper_alphabets_count = 0 > digits_count = 0 > special_chars_count = 0 > for i in range(0, password.__len__()): Again, use len(password). > char = password[i] Since you *only* use i as an index to grab the ith character, this is better written by iterating over the string itself: for char in password: ... > if char.isalpha(): > if char.isupper(): > upper_alphabets_count += 1 > else: > lower_alphabets_count += 1 > elif char.isdigit(): > digits_count += 1 > else: > special_chars_count += 1 if char.isupper(): upper_alphabets_count += 1 elif char.isupper(): lower_alphabets_count += 1 elif char.isdigit(): digits_count += 1 else: special_chars_count += 1 > chars_count = [upper_alphabets_count, lower_alphabets_count, > digits_count, special_chars_count] > > if password.__len__() < 8: len(password) > if not chars_count.__contains__(0): > strength = 0.5 if not 0 in chars_count: strength = 0.5 Hope this helps. -- Steven From no.email at nospam.invalid Wed Jan 27 02:14:31 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Tue, 26 Jan 2010 23:14:31 -0800 Subject: Simple Password Strength Checker Review Help needed References: <158b9ef31001262148g1926d067ja8ba840ba074eca4@mail.gmail.com> Message-ID: <7xfx5sxbmw.fsf@ruckus.brouhaha.com> "Mallikarjun(?????????????)" writes: > Since this is my first app/program, can someone review my code (just > over 150 lines) and help me improve my programming capabilities I didn't look at the code. From a security point of view, the concept of "password strength checking" is pretty dubious. If you want secure passwords, generate them from a random number source and assign them to the users. Don't have the users make up their own passwords. It's relatively (compared to using a computer file exposed to remote internet attacks) for users to write down the the random passwords on paper, as long as they're a little bit careful. As Bruce Schneier put it: "My wallet is already a secure container; it has valuable things in it, and I have a lifetime of experience keeping it safe. Adding a piece of paper with my passwords seems like a natural thing to do." From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 02:26:42 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 27 Jan 2010 07:26:42 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> Message-ID: On Tue, 26 Jan 2010 22:23:11 -0800, Paul Rubin wrote: > Steven D'Aprano writes: >> print as a function is more consistent and more convenient than print >> as a statement. > > Convenience is subjective, but the 3.x 'print' behavior is definitely > inconsistent (i.e. different from 2.x). Sorry, I meant consistent with the rest of Python, which mostly uses functions/methods and only rarely statements (e.g. del and import). > The change makes a lot of my > code silently produce wrong results, too. I often print tuples to show > what a program is doing: > > print (timestamp, 'transmogrified', blob) > > which in 2.x prints a parenthesized tuple that I can later read back in > with eval. That line of code still prints a message, but in a different > format, instead of throwing an error. I don't pretend that the transition between statement and function syntax will be anything but inconvenient, but I believe the end result will be worth it. -- Steven From arnodel at googlemail.com Wed Jan 27 02:34:56 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Wed, 27 Jan 2010 07:34:56 +0000 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> <7ea02185-0bc1-4f49-9931-0974d8a0db5c@a5g2000yqi.googlegroups.com> Message-ID: Steve Howell writes: > On Jan 25, 1:32?pm, Arnaud Delobelle wrote: >> Steve Howell writes: >> >> [...] >> >> > My algorithm does exactly N pops and roughly N list accesses, so I >> > would be going from N*N + N to N + N log N if switched to blist. >> >> Can you post your algorithm? ?It would be interesting to have a concrete >> use case to base this discussion on. >> > > I just realized you meant the Python code itself. It is here: > > https://bitbucket.org/showell/shpaml_website/src/tip/shpaml.py Hi - I didn't have the time to look at it before today. You scan through a list called prefix_lines, and you use pop(0) as a means to keep track of your position in this list. It seems to me that it would be more effective to explicitely keep track of it - and it would remove the need to the numerous copies of sublists of indent_lines that you have to create. I have rewritten part of the three relevant functions (html_block_tag, get_indented_block and recurse, within indent_lines) to show you what I mean (see below). I have tried to keep the changes to a minimum - you can see that the code is no more complex like this. The advantage is that only one list is created, prefix_lines, and there is no need to mutate it or copy parts of it during the algorithm. I have not had the time to test it though (only on one of the examples on the examples webpage). Code follows: [...] def html_block_tag(output, block, start, end, recurse): append = output.append prefix, tag = block[start] if RAW_HTML.regex.match(tag): append(prefix + tag) recurse(block, start + 1, end) else: start_tag, end_tag = apply_jquery_sugar(tag) append(prefix + start_tag) recurse(block, start + 1, end) append(prefix + end_tag) [...] def get_indented_block(prefix_lines, start, end): prefix, line = prefix_lines[start] len_prefix = len(prefix) i = start + 1 while i < end: new_prefix, line = prefix_lines[i] if line and len(new_prefix) <= len_prefix: break i += 1 while i-1 > start and prefix_lines[i-1][1] == '': i -= 1 return i [...] def indent_lines(lines, output, branch_method, leaf_method, pass_syntax, flush_left_syntax, flush_left_empty_line, indentation_method, get_block, ): append = output.append def recurse(prefix_lines, start, end): while start < end: prefix, line = prefix_lines[start] if line == '': start += 1 append('') else: block_end = get_block(prefix_lines, start, end) if block_end == start + 1: start += 1 if line == pass_syntax: pass elif line.startswith(flush_left_syntax): append(line[len(flush_left_syntax):]) elif line.startswith(flush_left_empty_line): append('') else: append(prefix + leaf_method(line)) else: branch_method(output, prefix_lines, start, block_end, recurse) start = block_end return prefix_lines = map(indentation_method, lines) recurse(prefix_lines, 0, len(prefix_lines)) From no.email at nospam.invalid Wed Jan 27 02:37:00 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Tue, 26 Jan 2010 23:37:00 -0800 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> Message-ID: <7x636oc82r.fsf@ruckus.brouhaha.com> Steven D'Aprano writes: > Sorry, I meant consistent with the rest of Python, which mostly uses > functions/methods and only rarely statements (e.g. del and import). yield, assert, if/else, return, etc. If we're after that kind of consistency, why not get rid of all those statements too? They have already partly done it with yield, and they ended up introducing a new separate if/else expression syntax as an alternative to the statement. > I don't pretend that the transition between statement and function syntax > will be anything but inconvenient, but I believe the end result will be > worth it. This just seems like a gratuitous change with very little benefit. Changing a bunch of list functions to instead return iterators is a much deeper change that will cause larger amounts of breakage, but it has quite a lot of benefits, so there's a stronger case for it. From apt.shansen at gmail.com Wed Jan 27 02:41:32 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Tue, 26 Jan 2010 23:41:32 -0800 Subject: python 3's adoption In-Reply-To: <7xr5pcyskw.fsf@ruckus.brouhaha.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> Message-ID: <7a9c25c21001262341o47742922n801a802ef9708a4e@mail.gmail.com> On Tue, Jan 26, 2010 at 10:23 PM, Paul Rubin wrote: > Steven D'Aprano writes: > > print as a function is more consistent and more convenient than print as > > a statement. > > Convenience is subjective, but the 3.x 'print' behavior is definitely > inconsistent (i.e. different from 2.x). This is a strange definition of "consistent" in my mind. To me, consistent means that it matches the patterns and idioms of the rest of the language at that point in time, not that it implies anything about backwards-compatibility. "print" as a statement was always inconsistent with the rest of the Python language; the rules of how it behaved had to be defined in a unique way (e.g., the behavior of trailing commas, the >> syntax to redirect what you print to). Everything about the statement was inconsistent. Yes, 3.0 changing the print statement to a function made it inconsistent with previous versions, but at the same time it made it MORE consistent with the rest of the language and Python code. If we are defining "consistent" as "backwards-compatible", we have a problem. Python3 is about fixing -language- inconsistencies, and allowing for backwards incompatibility when that meant the language itself was more consistent. > The change makes a lot of my > code silently produce wrong results, too. I often print tuples to show > what a program is doing: > > print (timestamp, 'transmogrified', blob) > > which in 2.x prints a parenthesized tuple that I can later read back in > with eval. That line of code still prints a message, but in a different > format, instead of throwing an error. > That seems like bad code. If one needs to print out what a program is doing, use the logging module. Maybe its just me, but in my experience-- real professional code never uses print unless it was a small script with a very narrowly focused purpose. Changing that focus to use Python3's print function vs a statement isn't really all that hard. And the language is more consistent. --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From timr at probo.com Wed Jan 27 02:47:25 2010 From: timr at probo.com (Tim Roberts) Date: Tue, 26 Jan 2010 23:47:25 -0800 Subject: Sikuli: the coolest Python project I have yet seen... References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> <89c2550d-057d-4cc2-a855-d4f3f11cb272@b10g2000vbh.googlegroups.com> Message-ID: CM wrote: > >How is this preferable to a macro recorder? Macro recorders work by measuring mouse motion and capturing click locations, or by recording the control IDs of the clicked windows. The former is sensitive to changing window locations, the latter to application updates. Sikuli works by using image analysis to locate the regions on the screen to be tickled. It's a novel idea, although others have correctly pointed out that it's not the most efficient way to automate applications. -- Tim Roberts, timr at probo.com Providenza & Boekelheide, Inc. From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 02:47:47 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 27 Jan 2010 07:47:47 GMT Subject: Simple Password Strength Checker Review Help needed References: <158b9ef31001262148g1926d067ja8ba840ba074eca4@mail.gmail.com> <7xfx5sxbmw.fsf@ruckus.brouhaha.com> Message-ID: On Tue, 26 Jan 2010 23:14:31 -0800, Paul Rubin wrote: > "Mallikarjun(?????????????)" writes: >> Since this is my first app/program, can someone review my code (just >> over 150 lines) and help me improve my programming capabilities > > I didn't look at the code. From a security point of view, the concept > of "password strength checking" is pretty dubious. If you want secure > passwords, generate them from a random number source But why do they need to be random, if password strength is dubious? *wink* > and assign them to > the users. Don't have the users make up their own passwords. It's > relatively (compared to using a computer file exposed to remote internet > attacks) for users to write down the the random passwords on paper, as > long as they're a little bit careful. I think you're missing a word there. Relatively secure perhaps? The problem is that most users will not be a little bit careful. They will stick the password on a Post-it note on the side of the monitor, or write it down and lose it, or leave the paper sitting on their desk while they go to lunch. > As Bruce Schneier put it: > > "My wallet is already a secure container; it has valuable things in > it, and I have a lifetime of experience keeping it safe. Adding a > piece of paper with my passwords seems like a natural thing to do." And people frequently lose their wallets. Besides, with the number of on-line identities and passwords many people need, you'll need a separate wallet just for the passwords. I have something of the order of 80 or 90 passwords written down, and another dozen in my head. Because there are so many, I need to keep account information with them: there's no way I'd be able to remember what the passwords were for otherwise. That's way too much to put in my wallet, and even if I did, if I lost it, I'd lose *everything*. Not only would some stranger likely have access to all my accounts, but I wouldn't even be able to identify what those accounts were, let alone authenticate into them. So I need at least one (and likely more) password I can keep in my head, so I can encrypt my list of rarely-used passwords. Because it needs to be something I can remember, it can't be a random string of digits, but it needs to *look* random. In other words, it needs to be a good password that is meaningful to me, but not to anyone else, and since I'm really bad at judging randomness (like nearly all humans), I'll take all the help I can get. -- Steven From no.email at nospam.invalid Wed Jan 27 03:03:36 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 27 Jan 2010 00:03:36 -0800 Subject: Simple Password Strength Checker Review Help needed References: <158b9ef31001262148g1926d067ja8ba840ba074eca4@mail.gmail.com> <7xfx5sxbmw.fsf@ruckus.brouhaha.com> Message-ID: <7xd40wrn3b.fsf@ruckus.brouhaha.com> Steven D'Aprano writes: > I think you're missing a word there. Relatively secure perhaps? Yes, something like that, oops. > The problem is that most users will not be a little bit careful. They > will stick the password on a Post-it note on the side of the monitor,... Right, that's what I mean about careful. But, people do generally develop skills at keeping important bits of paper (like their drivers' license) safe. > Besides, with the number of on-line identities and passwords many people > need, you'll need a separate wallet just for the passwords. I have > something of the order of 80 or 90 passwords written down.. I don't have anywhere that many important ones. For the less important ones I just use my browser's auto-fill feature (i.e. the passwords are stored in the browser). I generate important ones with a program (2-6 words selected at random from a dictionary). When I generate a new one, I write it on a slip of paper that I keep in my pocket and refer to as necessary. After referring to the paper a few times I usually remember the password and don't need the paper any more, so I dispose of it. > So I need at least one (and likely more) password I can keep in my > head, so I can encrypt my list of rarely-used passwords. If there's just one, and it's a phrase, you can remember it. > since I'm really bad at judging randomness (like nearly all humans), Right, humans are no good at generating or judging randomness. It's best to use an entropy source. www.diceware.com explains how to do it with dice and a wordlist they supply. I use a program something like the following instead: from math import log dictfile = '/usr/share/dict/words' def genrandom(nbytes): with open('/dev/urandom') as f: return int(f.read(nbytes).encode('hex'), 16) def main(): wordlist = list(x.strip() for x in open(dictfile) if len(x) < 7) nwords = len(wordlist) print "%d words, entropy=%.3f bits/word"% ( nwords, log(nwords, 2)) print '-'.join(wordlist[genrandom(10)%nwords] for i in xrange(5)) main() You can also use the diceware word list instead of the unix wordlist, of course. From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 03:20:57 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 27 Jan 2010 08:20:57 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> Message-ID: On Tue, 26 Jan 2010 23:37:00 -0800, Paul Rubin wrote: > Steven D'Aprano writes: >> Sorry, I meant consistent with the rest of Python, which mostly uses >> functions/methods and only rarely statements (e.g. del and import). > > yield, assert, if/else, return, etc. If we're after that kind of > consistency, why not get rid of all those statements too? They have > already partly done it with yield, and they ended up introducing a new > separate if/else expression syntax as an alternative to the statement. Without becoming a purely functional language, you won't get rid of all statements. In my opinion, outside of such purely functional languages and unconventional languages like Forth, statements play a valuable role in that they are controlling syntax. For example: for, while, if, try, break, yield, return are all used for flow control, and should remain as statements. But print is not -- it's a function disguised as a statement. print, after all, is just sugar for stdout.write, which is fine as far as it goes, but when you want to do something slightly different from what the sugar does, it becomes terribly inconvenient. A consequence of print being a statement is that my modules are often filled with functions and methods called "print_". Suppose I want to print the type of each argument as well as it's value. I would like to do this: def decorator(pr): def inner(*args, **kwargs): args = [(type(a), a) for a in args] pr(*args, **kwargs) return inner print = decorator(print) and then use print as normal, but of course I can't. So instead I do something like this: def print_(*args, **kwargs): args = [(type(a), a) for a in args] kw = {'file': sys.stdout, 'sep': ' ', 'end': '\n'} kw.update(kwargs) if len(kw) != 3: raise TypeError('invalid keyword argument') file = kw['file'] sep = kw['sep'] end = kw['end'] for arg in args: print >>file, (str(arg) + sep), # Don't forget the comma. print >>file, end, # Another comma. And then, I have to remember to call print_ instead of print. -- Steven From no.email at nospam.invalid Wed Jan 27 03:36:52 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 27 Jan 2010 00:36:52 -0800 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> Message-ID: <7x636ovt97.fsf@ruckus.brouhaha.com> Steven D'Aprano writes: > Without becoming a purely functional language, you won't get rid of all > statements. Why not? GCC lets you use any statement in an expression: #include main() { int i, x, p=0; x = ({ for (i=1; i<=10; i++) p += i; p;}); printf ("x=%d\n", x); } and C is certainly not a purely functional language. > for, while, if, try, break, yield, return > are all used for flow control, and should remain as statements. What about assert, import, and pass? From wuwei23 at gmail.com Wed Jan 27 03:45:22 2010 From: wuwei23 at gmail.com (alex23) Date: Wed, 27 Jan 2010 00:45:22 -0800 (PST) Subject: Sikuli: the coolest Python project I have yet seen... References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> <89c2550d-057d-4cc2-a855-d4f3f11cb272@b10g2000vbh.googlegroups.com> Message-ID: <9546c710-0a24-4dc2-aa07-e92d4bb72eea@m16g2000yqc.googlegroups.com> Tim Roberts wrote: > it's not the most efficient way to automate applications Sikuli doesn't seem that much different from Python in this way: it may not be the most efficient use of the computer's time, but I dare say it's significantly less demanding on the end user's. I can see Sikuli easily progressing to a full visual programming interface, replacing the 'click' keyword et al with iconic representations. Simple "script-less" GUI macro-ing for the masses? Fantastic. From fetchinson at googlemail.com Wed Jan 27 04:35:00 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Wed, 27 Jan 2010 10:35:00 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: >>> * Print is now a function. Great, much improvement. > > Actually not, IMHO. All it does is is to provide incompatibility. What incompatibility are you exactly talking about? Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> print( 'hello' ) hello >>> print 'hello' hello >>> Yes, this is with python 2.6.2 which is in the 2.x line of releases. So? Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From mmm286 at gmail.com Wed Jan 27 04:43:05 2010 From: mmm286 at gmail.com (mierdatutis mi) Date: Wed, 27 Jan 2010 10:43:05 +0100 Subject: Help parsing a page with python Message-ID: Hi, I would like to parse a webpage to can get the url of the video download. I use pyhton and firebug but I cant get the url link. Example: The url where I have to get the video link is: http://www.rtve.es/mediateca/videos/20100125/saber-comer---salsa-verde-judiones-25-01-10/676590.shtml " The video is http://www.rtve.es/resources/TE_SSAC011/flv/8/2/1264426362028.flv Could you help me please? Many thanks and sorry for my english! -------------- next part -------------- An HTML attachment was scrubbed... URL: From simon at brunningonline.net Wed Jan 27 05:13:32 2010 From: simon at brunningonline.net (Simon Brunning) Date: Wed, 27 Jan 2010 10:13:32 +0000 Subject: Help parsing a page with python In-Reply-To: References: Message-ID: <8c7f10c61001270213p10ad7d12ra39c75bea1f5e0f4@mail.gmail.com> 2010/1/27 mierdatutis mi : > Hi, > > I would like to parse a webpage to can get the url of the video download. I > use pyhton and firebug but I cant get the url link. > > Example: > > The url where I have to get the video link is: > http://www.rtve.es/mediateca/videos/20100125/saber-comer---salsa-verde-judiones-25-01-10/676590.shtml" > > The video is > http://www.rtve.es/resources/TE_SSAC011/flv/8/2/1264426362028.flv > Could you help me please? That URL doesn't appear to be in the HTML - it must be being brought in by the JavaScript somehow. -- Cheers, Simon B. From steve at holdenweb.com Wed Jan 27 05:19:35 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 27 Jan 2010 05:19:35 -0500 Subject: python 3's adoption In-Reply-To: <7x636ovt97.fsf@ruckus.brouhaha.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> Message-ID: Paul Rubin wrote: > Steven D'Aprano writes: >> Without becoming a purely functional language, you won't get rid of all >> statements. > > Why not? GCC lets you use any statement in an expression: > > #include > > main() > { > int i, x, p=0; > x = ({ for (i=1; i<=10; i++) p += i; p;}); > printf ("x=%d\n", x); > } > > and C is certainly not a purely functional language. > >> for, while, if, try, break, yield, return >> are all used for flow control, and should remain as statements. > > What about assert, import, and pass? Remember that Guido stated (I think in the original FAQ entry for why assignments don't produce results, but I could be wrong) that he explicitly wanted Python to be a statement-based language. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From floris.bruynooghe at gmail.com Wed Jan 27 05:20:53 2010 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Wed, 27 Jan 2010 02:20:53 -0800 (PST) Subject: Ad hoc lists vs ad hoc tuples Message-ID: <8a57c64f-0c67-4ec1-b4be-34d76c7e7571@q4g2000yqm.googlegroups.com> One thing I ofter wonder is which is better when you just need a throwaway sequence: a list or a tuple? E.g.: if foo in ['some', 'random', 'strings']: ... if [bool1, bool2, boo3].count(True) != 1: ... (The last one only works with tuples since python 2.6) Is a list or tuple better or more efficient in these situations? Regards Floris PS: This is inspired by some of the space-efficiency comments from the list.pop(0) discussion. From fetchinson at googlemail.com Wed Jan 27 05:32:08 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Wed, 27 Jan 2010 11:32:08 +0100 Subject: myths about python 3 Message-ID: Hi folks, I was going to write this post for a while because all sorts of myths periodically come up on this list about python 3. I don't think the posters mean to spread false information on purpose, they simply are not aware of the facts. My list is surely incomplete, please feel free to post your favorite misconception about python 3 that people periodically state, claim or ask about. 1. Print statement/function creates incompatibility between 2.x and 3.x! Certainly false or misleading, if one uses 2.6 and 3.x the incompatibility is not there. Print as a function works in 2.6: Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> print( 'hello' ) hello >>> print 'hello' hello >>> 2. Integer division creates incompatibility between 2.x and 3.x! Again false or misleading, because one can get the 3.x behavior with 2.6: Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 6/5 1 >>> from __future__ import division >>> 6/5 1.2 Please feel free to post your favorite false or misleading claim about python 3! Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From stefan_ml at behnel.de Wed Jan 27 05:40:27 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 27 Jan 2010 11:40:27 +0100 Subject: myths about python 3 In-Reply-To: References: Message-ID: <4b60181c$0$6732$9b4e6d93@newsspool2.arcor-online.net> Daniel Fetchinson, 27.01.2010 11:32: > 1. Print statement/function creates incompatibility between 2.x and 3.x! > > Certainly false or misleading, if one uses 2.6 and 3.x the > incompatibility is not there. Print as a function works in 2.6: > > Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) > [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> print( 'hello' ) > hello > >>> print 'hello' > hello This is actually misleading by itself, as the first statement is not a function call in Py2: Python 2.6.4 (r264:75706, Dec 7 2009, 18:43:55) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> print(1,2) (1, 2) It can, however, be made a function call through a future import in 2.6: >>> from __future__ import print_function >>> print(1,2) 1 2 Stefan From andreengels at gmail.com Wed Jan 27 05:45:48 2010 From: andreengels at gmail.com (Andre Engels) Date: Wed, 27 Jan 2010 11:45:48 +0100 Subject: myths about python 3 In-Reply-To: References: Message-ID: <6faf39c91001270245q155a932aj3e0bb98d377ddad1@mail.gmail.com> On Wed, Jan 27, 2010 at 11:32 AM, Daniel Fetchinson wrote: > Hi folks, > > I was going to write this post for a while because all sorts of myths > periodically come up on this list about python 3. I don't think the > posters mean to spread false information on purpose, they simply are > not aware of the facts. > > My list is surely incomplete, please feel free to post your favorite > misconception about python 3 that people periodically state, claim or > ask about. > > 1. Print statement/function creates incompatibility between 2.x and 3.x! > > Certainly false or misleading, if one uses 2.6 and 3.x the > incompatibility is not there. Print as a function works in 2.6: > > Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) > [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> print( 'hello' ) > hello >>>> print 'hello' > hello >>>> > > > 2. Integer division creates incompatibility between 2.x and 3.x! > > Again false or misleading, because one can get the 3.x behavior with 2.6: > > Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) > [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> 6/5 > 1 >>>> from __future__ import division >>>> 6/5 > 1.2 > > > Please feel free to post your favorite false or misleading claim about python 3! Well, I see two false or misleading claims just above - namely that the two claims above are false or misleading. They tell just half of the story, and that half is indeed easy. A Python 3 program can be unchanged (in the case of print) or with only trivial modifications (in the case of integer division) be made to run on Python 2.6. The other way around this is _not_ the case. To say that two things are compatible if one can be used for the other, but the other not for the first, is false or misleading. -- Andr? Engels, andreengels at gmail.com From babedoudi at yahoo.fr Wed Jan 27 06:23:34 2010 From: babedoudi at yahoo.fr (=?ISO-8859-1?Q?R=E9mi?=) Date: Wed, 27 Jan 2010 03:23:34 -0800 (PST) Subject: Terminal application with non-standard print References: Message-ID: On 25 jan, 23:30, Sean DiZazzo wrote: > On Jan 24, 11:27?am, R?mi wrote: > > > > > Hello everyone, > > > I would like to do a Python application that prints data to stdout, but > > not the common way. I do not want the lines to be printed after each > > other, but the old lines to be replaced with the new ones, like wget > > does it for example (when downloading a file you can see the percentage > > increasing on a same line). > > > I looked into the curses module, but this seems adapted only to do a > > whole application, and the terminal history is not visible anymore when > > the application starts. > > > Any ideas? > > > Thanks, > > > Remi > > You might want to take a look at the readline module. > > ~Sean Thanks everyone for your answers, that helped a lot. From mmm286 at gmail.com Wed Jan 27 06:33:17 2010 From: mmm286 at gmail.com (mierdatutis mi) Date: Wed, 27 Jan 2010 12:33:17 +0100 Subject: Help parsing a page with python In-Reply-To: <8c7f10c61001270213p10ad7d12ra39c75bea1f5e0f4@mail.gmail.com> References: <8c7f10c61001270213p10ad7d12ra39c75bea1f5e0f4@mail.gmail.com> Message-ID: Those videos are generated by javascript. There is some parser with python for javascript??? Thanks a lot! 2010/1/27 Simon Brunning > 2010/1/27 mierdatutis mi : > > Hi, > > > > I would like to parse a webpage to can get the url of the video download. > I > > use pyhton and firebug but I cant get the url link. > > > > Example: > > > > The url where I have to get the video link is: > > > http://www.rtve.es/mediateca/videos/20100125/saber-comer---salsa-verde-judiones-25-01-10/676590.shtml > " > > > > The video is > > http://www.rtve.es/resources/TE_SSAC011/flv/8/2/1264426362028.flv > > Could you help me please? > > That URL doesn't appear to be in the HTML - it must be being brought > in by the JavaScript somehow. > > -- > Cheers, > Simon B. > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From xahlee at gmail.com Wed Jan 27 06:40:38 2010 From: xahlee at gmail.com (Xah Lee) Date: Wed, 27 Jan 2010 03:40:38 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> <878wbk1c4q.fsf@castleamber.com> Message-ID: <0c142d80-5f54-4b63-a8ce-f826ec9ad82d@r19g2000yqb.googlegroups.com> Someone is badmouthing me, and it has been doing so over the years. I feel obliged to post a off topic relpy. See: ? DreamHost.com and A Incidence of Harassment http://xahlee.org/Periodic_dosage_dir/t2/harassment.html ? Why Can't You Be Normal? http://xahlee.org/Netiquette_dir/why_cant_you_be_normal.html Xah On Jan 26, 7:12 pm, "Alf P. Steinbach" wrote: > * John Bokma: > > > "Alf P. Steinbach" writes: > > >> Please don't post more noise and ad hominem attacks to the group, > >> Steve. > > > Funny that you talk about noise while replying yourself to noise. Xah > > Lee is just a pathetic spammer. He's not going to reply in this > > thread. He just shits out his stuff in as many groups as possible to > > promote his website. > > Sorry, I didn't know. > > Thanks for the heads-up. > > Cheers, > > - Alf From simon at brunningonline.net Wed Jan 27 06:51:05 2010 From: simon at brunningonline.net (Simon Brunning) Date: Wed, 27 Jan 2010 11:51:05 +0000 Subject: Help parsing a page with python In-Reply-To: References: <8c7f10c61001270213p10ad7d12ra39c75bea1f5e0f4@mail.gmail.com> Message-ID: <8c7f10c61001270351q5ff78b6bv30a2aa03c5cd98a3@mail.gmail.com> 2010/1/27 mierdatutis mi : > Those videos are generated by javascript. > There is some parser with python for javascript??? There is , but simulating the whole context of a browser is going to be a horror. You are probably far better off automating a real browser. WebDriver () has Python bindings these days. It's primarily intended for functional testing, but it might be a good fit here too. -- Cheers, Simon B. From javier.collado at gmail.com Wed Jan 27 06:58:34 2010 From: javier.collado at gmail.com (Javier Collado) Date: Wed, 27 Jan 2010 12:58:34 +0100 Subject: scraping with urllib2 In-Reply-To: <6faf39c91001262245t7c99120fy1e3c990c079b7f61@mail.gmail.com> References: <6faf39c91001262245t7c99120fy1e3c990c079b7f61@mail.gmail.com> Message-ID: Hello, To accept cookies, use the HTTPCookieProcessor as explained here: http://www.nomadjourney.com/2009/03/automatic-site-login-using-python-urllib2/ Best regards, Javier 2010/1/27 Andre Engels : > On Wed, Jan 27, 2010 at 6:26 AM, Patrick wrote: >> I'm trying to scrape the attached link for the price listed $99.99: >> http://bananarepublic.gap.com/browse/product.do?cid=41559&vid=1&pid=692392 >> >> I can see the price if I view the source(I even turned off java and >> javascript), but when I use urllib2, the price doesn't show up. >> >> Is there another library other than urllib2 that would work? > > To see that page you need to accept cookies from the site and send them back. > > > > -- > Andr? Engels, andreengels at gmail.com > -- > http://mail.python.org/mailman/listinfo/python-list > From javier.collado at gmail.com Wed Jan 27 07:00:53 2010 From: javier.collado at gmail.com (Javier Collado) Date: Wed, 27 Jan 2010 13:00:53 +0100 Subject: Help parsing a page with python In-Reply-To: References: <8c7f10c61001270213p10ad7d12ra39c75bea1f5e0f4@mail.gmail.com> Message-ID: Hello, A test case for Windmill might also be used to extract the information that you're looking for. Best regards, Javier 2010/1/27 mierdatutis mi : > Those videos are generated by javascript. > There is some parser with python for javascript??? > > Thanks a lot! > > > 2010/1/27 Simon Brunning >> >> 2010/1/27 mierdatutis mi : >> > Hi, >> > >> > I would like to parse a webpage to can get the url of the video >> > download. I >> > use pyhton and firebug but I cant get the url link. >> > >> > Example: >> > >> > The url where I have to get the video link is: >> > >> > http://www.rtve.es/mediateca/videos/20100125/saber-comer---salsa-verde-judiones-25-01-10/676590.shtml" >> > >> > The video is >> > http://www.rtve.es/resources/TE_SSAC011/flv/8/2/1264426362028.flv >> > Could you help me please? >> >> That URL doesn't appear to be in the HTML - it must be being brought >> in by the JavaScript somehow. >> >> -- >> Cheers, >> Simon B. >> -- >> http://mail.python.org/mailman/listinfo/python-list > > > -- > http://mail.python.org/mailman/listinfo/python-list > > From mmm286 at gmail.com Wed Jan 27 07:08:14 2010 From: mmm286 at gmail.com (mierdatutis mi) Date: Wed, 27 Jan 2010 13:08:14 +0100 Subject: Help parsing a page with python In-Reply-To: References: <8c7f10c61001270213p10ad7d12ra39c75bea1f5e0f4@mail.gmail.com> Message-ID: Hello again, What test case for Windmill? Can you say me the link, please? Many thanks 2010/1/27 Javier Collado > Hello, > > A test case for Windmill might also be used to extract the information > that you're looking for. > > Best regards, > Javier > > 2010/1/27 mierdatutis mi : > > Those videos are generated by javascript. > > There is some parser with python for javascript??? > > > > Thanks a lot! > > > > > > 2010/1/27 Simon Brunning > >> > >> 2010/1/27 mierdatutis mi : > >> > Hi, > >> > > >> > I would like to parse a webpage to can get the url of the video > >> > download. I > >> > use pyhton and firebug but I cant get the url link. > >> > > >> > Example: > >> > > >> > The url where I have to get the video link is: > >> > > >> > > http://www.rtve.es/mediateca/videos/20100125/saber-comer---salsa-verde-judiones-25-01-10/676590.shtml > " > >> > > >> > The video is > >> > http://www.rtve.es/resources/TE_SSAC011/flv/8/2/1264426362028.flv > >> > Could you help me please? > >> > >> That URL doesn't appear to be in the HTML - it must be being brought > >> in by the JavaScript somehow. > >> > >> -- > >> Cheers, > >> Simon B. > >> -- > >> http://mail.python.org/mailman/listinfo/python-list > > > > > > -- > > http://mail.python.org/mailman/listinfo/python-list > > > > > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From simon at brunningonline.net Wed Jan 27 07:14:23 2010 From: simon at brunningonline.net (Simon Brunning) Date: Wed, 27 Jan 2010 12:14:23 +0000 Subject: Help parsing a page with python In-Reply-To: References: <8c7f10c61001270213p10ad7d12ra39c75bea1f5e0f4@mail.gmail.com> Message-ID: <8c7f10c61001270414x37c7c1dci93a8a5667c923e20@mail.gmail.com> 2010/1/27 mierdatutis mi : > Hello again, > > What test case for Windmill? Can you say me the link, please? http://lmgtfy.com/?q=windmill+test -- Cheers, Simon B. From iainking at gmail.com Wed Jan 27 07:22:30 2010 From: iainking at gmail.com (Iain King) Date: Wed, 27 Jan 2010 04:22:30 -0800 (PST) Subject: Ad hoc lists vs ad hoc tuples References: <8a57c64f-0c67-4ec1-b4be-34d76c7e7571@q4g2000yqm.googlegroups.com> Message-ID: On Jan 27, 10:20?am, Floris Bruynooghe wrote: > One thing I ofter wonder is which is better when you just need a > throwaway sequence: a list or a tuple? ?E.g.: > > if foo in ['some', 'random', 'strings']: > ? ? ... > if [bool1, bool2, boo3].count(True) != 1: > ? ?... > > (The last one only works with tuples since python 2.6) > > Is a list or tuple better or more efficient in these situations? > > Regards > Floris > > PS: This is inspired by some of the space-efficiency comments from the > list.pop(0) discussion. I tend to use tuples unless using a list makes it easier to read. For example: if foo in ('some', 'random', 'strings'): draw.text((10,30), "WHICH IS WHITE", font=font) draw.line([(70,25), (85,25), (105,45)]) I've no idea what the performance difference is; I've always assumed it's negligible. Iain From xahlee at gmail.com Wed Jan 27 07:26:00 2010 From: xahlee at gmail.com (Xah Lee) Date: Wed, 27 Jan 2010 04:26:00 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> On Jan 26, 3:47 pm, Xah Lee wrote: > * Many functions that return lists now returns ?Views? or > ?Iterators? Instead. A fucking fuck all fucked up shit. A extraneous > ?oop engineering? complication. (See: Lambda in Python 3000) See also: ?Iterators: Signs of Weakness in Object-Oriented Languages? (1992) By Henry G Baker. http://home.pipeline.com/~hbaker1/Iterator.html Xah wrote: > * The cmp() function used in sort is basically gone, users are now > supposed to use the ?key? parameter instead. This is a flying-face- > fuck to computer science. This would be the most serious fuckup in > python 3. (See: Sorting in Python and Perl) Steven D'Aprano wrote: > I did too, when I first heard cmp was to be dumped. But I changed my mind > and now agree with the decision to drop cmp. Custom sorts are nearly > always much better written with key rather than cmp: key adds an O(N) > overheard to the sorting, while cmp makes sorting O(N**2). In other > words, key is *much* faster, and generally easier to write as well. The reason cmp is a order slower is due to Python lang's limitation and compiler implementation limitation. ? Sorting in Python and Perl http://xahlee.org/perl-python/sort_list.html the python lang, by the very nature of being a dynamic lang, makes it hard for compiler analize and optimize a sort function call with cmp argument that is equivalent to a key. So, for practical reasons, i think a ?key? parameter is fine. But chopping off ?cmp? is damaging. When your data structure is complex, its order is not embedded in some ?key?. Taking out ?cmp? makes it impossible to sort your data structure. in the essay, i also gave a few examples of data structures where you need a general means to specify the ordering function in order to sort. Without a cmp, you'll need to munge your data by some decorate- sort-dedecorate technique. >From another perspective, often it is considered a good principle of design that the lang be a bit flexible to let programers do what they want, instead of force them into one way. Python, being a loosely typed lang, with so-called ?duck typing?, in contrast to Java or OCaml in different ways, certainly is more on the lose extreme of the lang spectrum with respect to code construction. So, i don't see how python 3 people made the decision to removed cmp. (am pretty sure, at least part of the reason, is a ill attitude towards functional programing and lambda, lead by Guido.) > There may be a handful of rare applications where there is no obvious way > to convert a cmp function to a key, but they are vanishingly rare. I > think, if you search the archives, Paul Rubin may have come up with one > example. There are a number of recipes for easily converting cmp sorts to > key sorts. You say that it is rare. You are probably right. Though, that may just be caused by the language itself and consequently the type of people who uses it. If Python's lambda is not limited, and the python community doesn't look down on lambda, it is likely cmp will used more. The more your data structure becomes complex, the more cmp will be needed. > I think, in my perfect world, list.sort() and sorted() should continue > being key based, while the standard library contained (perhaps in the > functools module?) a sort function that contains all the bells and > whistles. You want cmp, it's there, and you can pay the extra cost if you > need it. You want to sort multiple lists by the contents of one? One > import away. But keep the general sort() function lean and super-fast. btw, is something like cmp still available in some module for sort? Xah ? http://xahlee.org/ ? From wissme at hotmail.com Wed Jan 27 08:01:40 2010 From: wissme at hotmail.com (Jean Guillaume Pyraksos) Date: Wed, 27 Jan 2010 14:01:40 +0100 Subject: Python and Ruby Message-ID: What are the arguments for choosing Python against Ruby for introductory programming ? Python has no provisions for tail recursion, Ruby is going to... So what ? Thanks, JG From stefan_ml at behnel.de Wed Jan 27 08:04:41 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 27 Jan 2010 14:04:41 +0100 Subject: Python and Ruby In-Reply-To: References: Message-ID: <4b6039ea$0$6725$9b4e6d93@newsspool2.arcor-online.net> Jean Guillaume Pyraksos, 27.01.2010 14:01: > What are the arguments for choosing Python against Ruby > for introductory programming ? PEP 20: http://www.python.org/dev/peps/pep-0020/ Stefan From richard.lamboj at bilcom.at Wed Jan 27 08:06:23 2010 From: richard.lamboj at bilcom.at (Richard Lamboj) Date: Wed, 27 Jan 2010 14:06:23 +0100 Subject: starting a thread in a nother thread Message-ID: <201001271406.23584.richard.lamboj@bilcom.at> hello, just for _curiosity_. What would be if i start a thread in a nother thread and acquire a lock in the "child" thread. Is there anything that could go wrong if someone try to start threads in threads? Kind Regards, Richi From stefan_ml at behnel.de Wed Jan 27 08:10:13 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 27 Jan 2010 14:10:13 +0100 Subject: starting a thread in a nother thread In-Reply-To: References: Message-ID: <4b603b35$0$6725$9b4e6d93@newsspool2.arcor-online.net> Richard Lamboj, 27.01.2010 14:06: > just for _curiosity_. What would be if i start a thread in a nother thread and > acquire a lock in the "child" thread. Is there anything that could go wrong > if someone try to start threads in threads? There's usually tons of things that can go wrong w.r.t. threads: http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf However, there's nothing special to a thread that was started from another thread, so the problems don't change. Stefan From simon at brunningonline.net Wed Jan 27 08:47:25 2010 From: simon at brunningonline.net (Simon Brunning) Date: Wed, 27 Jan 2010 13:47:25 +0000 Subject: Python and Ruby In-Reply-To: References: Message-ID: <8c7f10c61001270547t10c15144wbc372407621b738f@mail.gmail.com> 2010/1/27 Jean Guillaume Pyraksos : > What are the arguments for choosing Python against Ruby > for introductory programming ? Frankly, either would be a good choice. I think Python is a little cleaner, but I'm sure you'd find Ruby fans who'd argue the complete opposite. Both have good ecosystems, (i.e. good communities, and plenty of good libraries and frameworks) - but Python is probably a bit ahead here having been around a bit longer. > Python has no provisions > for tail recursion, Ruby is going to... So what ? This would be a very strange reason to pick one language over the other - it's a very minor point. -- Cheers, Simon B. From alfps at start.no Wed Jan 27 09:05:40 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 15:05:40 +0100 Subject: Just drawing lines and plotting points? In-Reply-To: <67a2a20b-b17d-4677-b8cc-4793adf077f4@m16g2000yqc.googlegroups.com> References: <50697b2c1001261654i7a64adaai3df49eac8a859eae@mail.gmail.com> <67a2a20b-b17d-4677-b8cc-4793adf077f4@m16g2000yqc.googlegroups.com> Message-ID: * rantingrick: > On Jan 26, 10:52 pm, "Alf P. Steinbach" wrote: >> * rantingrick: >> >> >> >> >> >>> On Jan 26, 9:38 pm, Terry Reedy wrote: >>>> On 1/26/2010 7:54 PM, Chris Rebert wrote: >>>>> On Tue, Jan 26, 2010 at 4:36 PM, Someone Something wrote: >>>>>> Hello, >>>>>> I need a python library that makes drawing lines and plotting points (these >>>>>> two things are the only things I need to do) easy. Or, how can I do >>>>>> something like this with pygame? Basically, what I want to do is make >>>>>> graphs. In pygame, since the coordinate system switches the x's and the y's >>>>>> I would have to switch them again when I plot my points so my graphs look >>>>>> okay. I hope this was enough info for me to get a good answer. >>>>> matplotlib, perhaps?:http://matplotlib.sourceforge.net/ >>>> or dislin? >>> it doesn't get any easier than the tkCanvas widget... >>> import Tkinter as tk >>> #import tkinter as tk #python3.0+ >>> app = tk.Tk() >>> canvas = tk.Canvas(app) >>> canvas.pack() >>> canvas.create_line(sx,sy, ex,ey) >>> #... >>> #... >>> app.mainloop() >> He he, it's even easier to draw a graph using the turtle module. >> >> Cheers, >> >> - Alf > > Yes Alf please forgive me, i had completely forgotten about the > "visual" learners amongst us. > > turtle, turtle, watch him go... > turtle, turtle, why he so slow... > turtle, turtle, not even he don't know...? > > ;-) Uhm, that is a misconception. Just do turtle.hideturtle() turtle.tracer( 0 ) to turn off the animation stuff. That said, the turtle module bundled with Python is somewhat limited, but for plotting simple graphs it's certainly simpler than using tkinter directly. Cheers & hth., - Alf From javier.collado at gmail.com Wed Jan 27 09:07:15 2010 From: javier.collado at gmail.com (Javier Collado) Date: Wed, 27 Jan 2010 15:07:15 +0100 Subject: Help parsing a page with python In-Reply-To: References: <8c7f10c61001270213p10ad7d12ra39c75bea1f5e0f4@mail.gmail.com> Message-ID: Hello, You can find some advice here: http://www.packtpub.com/article/web-scraping-with-python-part-2 Best regards, Javier 2010/1/27 mierdatutis mi : > Hello again, > > What test case for Windmill? Can you say me the link, please? > > Many thanks > > 2010/1/27 Javier Collado >> >> Hello, >> >> A test case for Windmill might also be used to extract the information >> that you're looking for. >> >> Best regards, >> ? ?Javier >> >> 2010/1/27 mierdatutis mi : >> > Those videos are generated by javascript. >> > There is some parser with python for javascript??? >> > >> > Thanks a lot! >> > >> > >> > 2010/1/27 Simon Brunning >> >> >> >> 2010/1/27 mierdatutis mi : >> >> > Hi, >> >> > >> >> > I would like to parse a webpage to can get the url of the video >> >> > download. I >> >> > use pyhton and firebug but I cant get the url link. >> >> > >> >> > Example: >> >> > >> >> > The url where I have to get the video link is: >> >> > >> >> > >> >> > http://www.rtve.es/mediateca/videos/20100125/saber-comer---salsa-verde-judiones-25-01-10/676590.shtml" >> >> > >> >> > The video is >> >> > http://www.rtve.es/resources/TE_SSAC011/flv/8/2/1264426362028.flv >> >> > Could you help me please? >> >> >> >> That URL doesn't appear to be in the HTML - it must be being brought >> >> in by the JavaScript somehow. >> >> >> >> -- >> >> Cheers, >> >> Simon B. >> >> -- >> >> http://mail.python.org/mailman/listinfo/python-list >> > >> > >> > -- >> > http://mail.python.org/mailman/listinfo/python-list >> > >> > >> -- >> http://mail.python.org/mailman/listinfo/python-list > > > -- > http://mail.python.org/mailman/listinfo/python-list > > From fetchinson at googlemail.com Wed Jan 27 09:22:39 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Wed, 27 Jan 2010 15:22:39 +0100 Subject: myths about python 3 In-Reply-To: <6faf39c91001270245q155a932aj3e0bb98d377ddad1@mail.gmail.com> References: <6faf39c91001270245q155a932aj3e0bb98d377ddad1@mail.gmail.com> Message-ID: >> Hi folks, >> >> I was going to write this post for a while because all sorts of myths >> periodically come up on this list about python 3. I don't think the >> posters mean to spread false information on purpose, they simply are >> not aware of the facts. >> >> My list is surely incomplete, please feel free to post your favorite >> misconception about python 3 that people periodically state, claim or >> ask about. >> >> 1. Print statement/function creates incompatibility between 2.x and 3.x! >> >> Certainly false or misleading, if one uses 2.6 and 3.x the >> incompatibility is not there. Print as a function works in 2.6: >> >> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >>>>> print( 'hello' ) >> hello >>>>> print 'hello' >> hello >>>>> >> >> >> 2. Integer division creates incompatibility between 2.x and 3.x! >> >> Again false or misleading, because one can get the 3.x behavior with 2.6: >> >> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >>>>> 6/5 >> 1 >>>>> from __future__ import division >>>>> 6/5 >> 1.2 >> >> >> Please feel free to post your favorite false or misleading claim about >> python 3! > > Well, I see two false or misleading claims just above - namely that > the two claims above are false or misleading. They tell just half of > the story, and that half is indeed easy. A Python 3 program can be > unchanged (in the case of print) or with only trivial modifications > (in the case of integer division) be made to run on Python 2.6. Okay, so we agree that as long as print and integer division is concerned, a program can easily be written that runs on both 2.6 and 3.x. My statements are exactly this, so I don't understand why you disagree. > The other way around this is _not_ the case. What do you mean? > To say that two things are > compatible if one can be used for the other, but the other not for the > first, is false or misleading. I'm not sure what you mean here. Maybe I didn't make myself clear enough, but what I mean is this: as long as print and integer division is concerned, it is trivial to write code that runs on both 2.6 and 3.x. Hence if someone wants to highlight incompatibility (which surely exists) between 2.6 and 3.x he/she has to look elsewhere. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From richard.lamboj at bilcom.at Wed Jan 27 09:23:35 2010 From: richard.lamboj at bilcom.at (Richard Lamboj) Date: Wed, 27 Jan 2010 15:23:35 +0100 Subject: starting a thread in a nother thread In-Reply-To: <4b603b35$0$6725$9b4e6d93@newsspool2.arcor-online.net> References: <4b603b35$0$6725$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <201001271523.35060.richard.lamboj@bilcom.at> Am Wednesday 27 January 2010 14:10:13 schrieb Stefan Behnel: > Richard Lamboj, 27.01.2010 14:06: > > just for _curiosity_. What would be if i start a thread in a nother > > thread and acquire a lock in the "child" thread. Is there anything that > > could go wrong if someone try to start threads in threads? > > There's usually tons of things that can go wrong w.r.t. threads: > > http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf > > However, there's nothing special to a thread that was started from another > thread, so the problems don't change. > > Stefan Hello, i have tried a little bit around with psycopg2 and threads, I'am sharing one connection for all threads. When i'am starting the threads "normal" everything works without any Problem. When i'am starting the threads from another thread than i got a "segmentation fault" - i'am using locks. I'am sending 2048 Reqeuest at once. The "normal" Methode with one thread for every Request needs 10 seconds. The other Methode starting a thread and starting two other threads from this thread crashes after 10 to 40 requests with the segmentation fault error. Any Idea why? Its a 64 bit maschine. Maybe i'am making something wrong? Kind Regards From fetchinson at googlemail.com Wed Jan 27 09:30:00 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Wed, 27 Jan 2010 15:30:00 +0100 Subject: myths about python 3 In-Reply-To: <4b60181c$0$6732$9b4e6d93@newsspool2.arcor-online.net> References: <4b60181c$0$6732$9b4e6d93@newsspool2.arcor-online.net> Message-ID: >> 1. Print statement/function creates incompatibility between 2.x and 3.x! >> >> Certainly false or misleading, if one uses 2.6 and 3.x the >> incompatibility is not there. Print as a function works in 2.6: >> >> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >> >>> print( 'hello' ) >> hello >> >>> print 'hello' >> hello > > This is actually misleading by itself, as the first statement is not a > function call in Py2: > > Python 2.6.4 (r264:75706, Dec 7 2009, 18:43:55) > [GCC 4.4.1] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> print(1,2) > (1, 2) > > It can, however, be made a function call through a future import in 2.6: > > >>> from __future__ import print_function > >>> print(1,2) > 1 2 Thanks! This is true, luckily you provided a better solution and the conclusion is not changed, as long as print is concerned, 2.6 and 3.x can trivially be made compatible. Surely there are incompatibilities, but first of all there are many tools that help the transition such as 2to3 and there is a clear and officially documented migration guide too (quoted by Steve Holden in another thread not so long ago), second of all the most vocal arguments that one hears mostly from ill-informed people are related to print and similar non-issues. These then get quoted over and over again, which led me to write this post :) Cheers, Daniel Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From stefan_ml at behnel.de Wed Jan 27 09:30:17 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 27 Jan 2010 15:30:17 +0100 Subject: starting a thread in a nother thread In-Reply-To: References: <4b603b35$0$6725$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <4b604df9$0$6723$9b4e6d93@newsspool2.arcor-online.net> Richard Lamboj, 27.01.2010 15:23: > Am Wednesday 27 January 2010 14:10:13 schrieb Stefan Behnel: >> Richard Lamboj, 27.01.2010 14:06: >>> just for _curiosity_. What would be if i start a thread in a nother >>> thread and acquire a lock in the "child" thread. Is there anything that >>> could go wrong if someone try to start threads in threads? >> There's usually tons of things that can go wrong w.r.t. threads: >> >> http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf >> >> However, there's nothing special to a thread that was started from another >> thread, so the problems don't change. > > i have tried a little bit around with psycopg2 and threads, > > I'am sharing one connection for all threads. When i'am starting the > threads "normal" everything works without any Problem. When i'am starting the > threads from another thread than i got a "segmentation fault" Sounds like a bug that you might want to report to the maintainers of psycopg2. Stefan From jeanmichel at sequans.com Wed Jan 27 09:32:17 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Wed, 27 Jan 2010 15:32:17 +0100 Subject: myths about python 3 In-Reply-To: References: <6faf39c91001270245q155a932aj3e0bb98d377ddad1@mail.gmail.com> Message-ID: <4B604E71.5010706@sequans.com> Daniel Fetchinson wrote: >>> Hi folks, >>> >>> I was going to write this post for a while because all sorts of myths >>> periodically come up on this list about python 3. I don't think the >>> posters mean to spread false information on purpose, they simply are >>> not aware of the facts. >>> >>> My list is surely incomplete, please feel free to post your favorite >>> misconception about python 3 that people periodically state, claim or >>> ask about. >>> >>> 1. Print statement/function creates incompatibility between 2.x and 3.x! >>> >>> Certainly false or misleading, if one uses 2.6 and 3.x the >>> incompatibility is not there. Print as a function works in 2.6: >>> >>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>> >>>>>> print( 'hello' ) >>>>>> >>> hello >>> >>>>>> print 'hello' >>>>>> >>> hello >>> >>> 2. Integer division creates incompatibility between 2.x and 3.x! >>> >>> Again false or misleading, because one can get the 3.x behavior with 2.6: >>> >>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>> >>>>>> 6/5 >>>>>> >>> 1 >>> >>>>>> from __future__ import division >>>>>> 6/5 >>>>>> >>> 1.2 >>> >>> >>> Please feel free to post your favorite false or misleading claim about >>> python 3! >>> >> Well, I see two false or misleading claims just above - namely that >> the two claims above are false or misleading. They tell just half of >> the story, and that half is indeed easy. A Python 3 program can be >> unchanged (in the case of print) or with only trivial modifications >> (in the case of integer division) be made to run on Python 2.6. >> > > Okay, so we agree that as long as print and integer division is > concerned, a program can easily be written that runs on both 2.6 and > 3.x. > > My statements are exactly this, so I don't understand why you disagree. > > >> The other way around this is _not_ the case. >> > > What do you mean? > > >> To say that two things are >> compatible if one can be used for the other, but the other not for the >> first, is false or misleading. >> > > I'm not sure what you mean here. Maybe I didn't make myself clear > enough, but what I mean is this: as long as print and integer division > is concerned, it is trivial to write code that runs on both 2.6 and > 3.x. Hence if someone wants to highlight incompatibility (which surely > exists) between 2.6 and 3.x he/she has to look elsewhere. > > Cheers, > Daniel > > How would you write in python 2.6 if print: print('Hello') --- def myPrint(*args): for arg in args: sys.stdout.write(str(arg)) print = myPrint JM From spam2008 at nney.com Wed Jan 27 09:41:40 2010 From: spam2008 at nney.com (D HANNEY) Date: Wed, 27 Jan 2010 06:41:40 -0800 (PST) Subject: Generic proxy (that proxies methods like __iter__) Message-ID: <635f48e9-b8de-402b-8215-1e5c26a33ed1@k35g2000yqb.googlegroups.com> >>> def show(opened): ... with opened as file: ... for line in file: ... print line.strip() ... >>> show(open("test.txt")) blah1 blah2 blah3 # # Good! I wonder if I can do that with StringIO ... # >>> import StringIO >>> show(StringIO.StringIO("blah1\nblah2\nblah3\n")) Traceback (most recent call last): File "", line 1, in File "", line 2, in show AttributeError: StringIO instance has no attribute '__exit__' # # Oh dear ... I'll need to proxy it ... # >>> class IterNoGuardProxy(object): ... def __init__(self, t): ... self.t = t ... def __enter__(self): ... return self ... def __exit__(self, type, value, traceback): ... return False ... def __iter__(self): ... return self.t.__iter__() ... >>> show(IterNoGuardProxy(StringIO.StringIO("blah1\nblah2\nblah3\n"))) blah1 blah2 blah3 >>> # # Great! That works well. Now I wonder if I can do it generically # in a way that can proxy for ALL methods (including, but not limited to, __iter__) # >>> class NoGuardProxy(object): ... def __init__(self, t): ... self.t = t ... def __enter__(self): ... return self ... def __exit__(self, type, value, traceback): ... return False ... def __getattr__(self): ... return self.t.__getattr__(self) ... >>> show(NoGuardProxy(StringIO.StringIO("blah1\nblah2\nblah3\n"))) Traceback (most recent call last): File "", line 1, in File "", line 3, in show TypeError: 'NoGuardProxy' object is not iterable # # Now I'm stuck. # Can anyone help? # From lie.1296 at gmail.com Wed Jan 27 09:42:21 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Thu, 28 Jan 2010 01:42:21 +1100 Subject: myths about python 3 In-Reply-To: References: <6faf39c91001270245q155a932aj3e0bb98d377ddad1@mail.gmail.com> Message-ID: <4b6050ef$1@dnews.tpgi.com.au> On 01/28/10 01:32, Jean-Michel Pichavant wrote: > Daniel Fetchinson wrote: >>>> Hi folks, >>>> >>>> I was going to write this post for a while because all sorts of myths >>>> periodically come up on this list about python 3. I don't think the >>>> posters mean to spread false information on purpose, they simply are >>>> not aware of the facts. >>>> >>>> My list is surely incomplete, please feel free to post your favorite >>>> misconception about python 3 that people periodically state, claim or >>>> ask about. >>>> >>>> 1. Print statement/function creates incompatibility between 2.x and >>>> 3.x! >>>> >>>> Certainly false or misleading, if one uses 2.6 and 3.x the >>>> incompatibility is not there. Print as a function works in 2.6: >>>> >>>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >>>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >>>> Type "help", "copyright", "credits" or "license" for more information. >>>> >>>>>>> print( 'hello' ) >>>>>>> >>>> hello >>>> >>>>>>> print 'hello' >>>>>>> >>>> hello >>>> 2. Integer division creates incompatibility between 2.x and 3.x! >>>> >>>> Again false or misleading, because one can get the 3.x behavior with >>>> 2.6: >>>> >>>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >>>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >>>> Type "help", "copyright", "credits" or "license" for more information. >>>> >>>>>>> 6/5 >>>>>>> >>>> 1 >>>> >>>>>>> from __future__ import division >>>>>>> 6/5 >>>>>>> >>>> 1.2 >>>> >>>> >>>> Please feel free to post your favorite false or misleading claim about >>>> python 3! >>>> >>> Well, I see two false or misleading claims just above - namely that >>> the two claims above are false or misleading. They tell just half of >>> the story, and that half is indeed easy. A Python 3 program can be >>> unchanged (in the case of print) or with only trivial modifications >>> (in the case of integer division) be made to run on Python 2.6. >>> >> >> Okay, so we agree that as long as print and integer division is >> concerned, a program can easily be written that runs on both 2.6 and >> 3.x. >> >> My statements are exactly this, so I don't understand why you disagree. >> >> >>> The other way around this is _not_ the case. >>> >> >> What do you mean? >> >> >>> To say that two things are >>> compatible if one can be used for the other, but the other not for the >>> first, is false or misleading. >>> >> >> I'm not sure what you mean here. Maybe I didn't make myself clear >> enough, but what I mean is this: as long as print and integer division >> is concerned, it is trivial to write code that runs on both 2.6 and >> 3.x. Hence if someone wants to highlight incompatibility (which surely >> exists) between 2.6 and 3.x he/she has to look elsewhere. >> >> Cheers, >> Daniel >> >> > How would you write in python 2.6 > > if print: > print('Hello') > > --- > > def myPrint(*args): > for arg in args: > sys.stdout.write(str(arg)) > > print = myPrint > > JM from __future__ import print_function if print: print('Hello') def myPrint(*args): for arg in args: sys.stdout.write(str(arg)) print = myPrint From mal at egenix.com Wed Jan 27 09:47:06 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 27 Jan 2010 15:47:06 +0100 Subject: Ad hoc lists vs ad hoc tuples In-Reply-To: References: <8a57c64f-0c67-4ec1-b4be-34d76c7e7571@q4g2000yqm.googlegroups.com> Message-ID: <4B6051EA.50508@egenix.com> Iain King wrote: > On Jan 27, 10:20 am, Floris Bruynooghe > wrote: >> One thing I ofter wonder is which is better when you just need a >> throwaway sequence: a list or a tuple? E.g.: >> >> if foo in ['some', 'random', 'strings']: >> ... >> if [bool1, bool2, boo3].count(True) != 1: >> ... >> >> (The last one only works with tuples since python 2.6) >> >> Is a list or tuple better or more efficient in these situations? >> >> Regards >> Floris >> >> PS: This is inspired by some of the space-efficiency comments from the >> list.pop(0) discussion. > > I tend to use tuples unless using a list makes it easier to read. For > example: > > if foo in ('some', 'random', 'strings'): > > draw.text((10,30), "WHICH IS WHITE", font=font) > draw.line([(70,25), (85,25), (105,45)]) > > I've no idea what the performance difference is; I've always assumed > it's negligible. Tuples are a faster to create than lists, so if you create lots of them and don't need the list features, use tuples. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 27 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From showell30 at yahoo.com Wed Jan 27 09:53:24 2010 From: showell30 at yahoo.com (Steve Howell) Date: Wed, 27 Jan 2010 06:53:24 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> <7ea02185-0bc1-4f49-9931-0974d8a0db5c@a5g2000yqi.googlegroups.com> Message-ID: On Jan 26, 11:34?pm, Arnaud Delobelle wrote: > Steve Howell writes: > > On Jan 25, 1:32?pm, Arnaud Delobelle wrote: > >> Steve Howell writes: > > >> [...] > > >> > My algorithm does exactly N pops and roughly N list accesses, so I > >> > would be going from N*N + N to N + N log N if switched to blist. > > >> Can you post your algorithm? ?It would be interesting to have a concrete > >> use case to base this discussion on. > > > I just realized you meant the Python code itself. ?It is here: > > >https://bitbucket.org/showell/shpaml_website/src/tip/shpaml.py > > Hi - I didn't have the time to look at it before today. ?You scan > through a list called prefix_lines, and you use pop(0) as a means to > keep track of your position in this list. ?It seems to me that it would > be more effective to explicitely keep track of it - and it would remove > the need to the numerous copies of sublists of indent_lines that you > have to create. > > I have rewritten part of the three relevant functions (html_block_tag, > get_indented_block and recurse, within indent_lines) to show you what I > mean (see below). ?I have tried to keep the changes to a minimum - you > can see that the code is no more complex like this. ?The advantage is > that only one list is created, prefix_lines, and there is no need to > mutate it or copy parts of it during the algorithm. ?I have not had the > time to test it though (only on one of the examples on the examples > webpage). > > Code follows: > > [...] > > def html_block_tag(output, block, start, end, recurse): > ? ? append = output.append > ? ? prefix, tag = block[start] > ? ? if RAW_HTML.regex.match(tag): > ? ? ? ? append(prefix + tag) > ? ? ? ? recurse(block, start + 1, end) > ? ? else: > ? ? ? ? start_tag, end_tag = apply_jquery_sugar(tag) > ? ? ? ? append(prefix + start_tag) > ? ? ? ? recurse(block, start + 1, end) > ? ? ? ? append(prefix + end_tag) > > [...] > > def get_indented_block(prefix_lines, start, end): > ? ? prefix, line = prefix_lines[start] > ? ? len_prefix = len(prefix) > ? ? i = start + 1 > ? ? while i < end: > ? ? ? ? new_prefix, line = prefix_lines[i] > ? ? ? ? if line and len(new_prefix) <= len_prefix: > ? ? ? ? ? ? break > ? ? ? ? i += 1 > ? ? while i-1 > start and prefix_lines[i-1][1] == '': > ? ? ? ? i -= 1 > ? ? return i > > [...] > > def indent_lines(lines, > ? ? ? ? ? ? output, > ? ? ? ? ? ? branch_method, > ? ? ? ? ? ? leaf_method, > ? ? ? ? ? ? pass_syntax, > ? ? ? ? ? ? flush_left_syntax, > ? ? ? ? ? ? flush_left_empty_line, > ? ? ? ? ? ? indentation_method, > ? ? ? ? ? ? get_block, > ? ? ? ? ? ? ): > ? ? append = output.append > ? ? def recurse(prefix_lines, start, end): > ? ? ? ? while start < end: > ? ? ? ? ? ? prefix, line = prefix_lines[start] > ? ? ? ? ? ? if line == '': > ? ? ? ? ? ? ? ? start += 1 > ? ? ? ? ? ? ? ? append('') > ? ? ? ? ? ? else: > ? ? ? ? ? ? ? ? block_end = get_block(prefix_lines, start, end) > ? ? ? ? ? ? ? ? if block_end == start + 1: > ? ? ? ? ? ? ? ? ? ? start += 1 > ? ? ? ? ? ? ? ? ? ? if line == pass_syntax: > ? ? ? ? ? ? ? ? ? ? ? ? pass > ? ? ? ? ? ? ? ? ? ? elif line.startswith(flush_left_syntax): > ? ? ? ? ? ? ? ? ? ? ? ? append(line[len(flush_left_syntax):]) > ? ? ? ? ? ? ? ? ? ? elif line.startswith(flush_left_empty_line): > ? ? ? ? ? ? ? ? ? ? ? ? append('') > ? ? ? ? ? ? ? ? ? ? else: > ? ? ? ? ? ? ? ? ? ? ? ? append(prefix + leaf_method(line)) > ? ? ? ? ? ? ? ? else: > ? ? ? ? ? ? ? ? ? ? branch_method(output, prefix_lines, start, block_end, recurse) > ? ? ? ? ? ? ? ? ? ? start = block_end > ? ? ? ? return > ? ? prefix_lines = map(indentation_method, lines) > ? ? recurse(prefix_lines, 0, len(prefix_lines)) I think your rewrite makes sense for the way that Python is implemented today. Instead of mutating the list as you consume elements, you propose to advance the "start" variable, which is essentially a pointer. There are only two disadvantage of the "start" approach that I can think of: 1) You have to pass around two parameters where before there was only one. (Aside: for stylistic concerns, would you bundle them in a tuple, since they kind of go hand in hand?) 2) You hold on to memory from the elements longer. Pushing this complexity down into CPython of course would have similar disadvantages: 1) you have to store two pointers where before there was only one 2) you hold on to memory from pointers to orphaned elements longer Disadvantage #1 is more than offset by the fact that you would have had to waste memory with the "start" in the Python program. Disadvantage #2 is offset by the fact that you would have been holding on to the elements themselves in the Python program. Admittedly this a pretty narrow context where pushing the logic down into CPython gains overall simplicity and performance. Disadvantage #1 is the sticky point when you consider the patch in the broader context of all programs. Thanks for looking at the code, by the way. From rdv at roalddevries.nl Wed Jan 27 09:56:24 2010 From: rdv at roalddevries.nl (Roald de Vries) Date: Wed, 27 Jan 2010 15:56:24 +0100 Subject: Python and Ruby In-Reply-To: References: Message-ID: <589DBDBD-5847-442A-B124-806D742B4189@roalddevries.nl> On Jan 27, 2010, at 2:01 PM, Jean Guillaume Pyraksos wrote: > What are the arguments for choosing Python against Ruby > for introductory programming ? Python has no provisions > for tail recursion, Ruby is going to... So what ? > Thanks, I think the main difference is in culture, especially for *introductory* programming. For example, I have the impression that Rubyists like to write the same thing in as many ways as possible (example: rubyquiz.com), Python prefers one obvious way; this (arguably) makes Ruby more writable and Python more readable. From alfps at start.no Wed Jan 27 09:57:14 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 15:57:14 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: * Daniel Fetchinson: >>>> * Print is now a function. Great, much improvement. >> Actually not, IMHO. All it does is is to provide incompatibility. > > > What incompatibility are you exactly talking about? > > Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) > [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> print( 'hello' ) > hello >>>> print 'hello' > hello > > Yes, this is with python 2.6.2 which is in the 2.x line of releases. So? I gather that your example is about code that technically executes fine with both versions and produces the same result, i.e. that there is a subset with the same syntax and semantics. But 'print' calls that technically execute fine with both versions may and will in general produce different results. I.e. not just the syntax but also the semantics have changed: >>> import sys >>> sys.version '2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]' >>> >>> print( "2+2 =", 2+2 ) ('2+2 =', 4) >>> _ >>> import sys >>> sys.version '3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)]' >>> >>> print( "2+2 =", 2+2 ) 2+2 = 4 >>> _ Cheers & hth., - Alf From arnodel at googlemail.com Wed Jan 27 10:07:23 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Wed, 27 Jan 2010 07:07:23 -0800 (PST) Subject: Generic proxy (that proxies methods like __iter__) References: <635f48e9-b8de-402b-8215-1e5c26a33ed1@k35g2000yqb.googlegroups.com> Message-ID: On 27 Jan, 14:41, D HANNEY wrote: [...] > >>> class NoGuardProxy(object): > > ... def __init__(self, t): > ... self.t = t > ... def __enter__(self): > ... return self > ... def __exit__(self, type, value, traceback): > ... return False > ... ? ? ? ? def __getattr__(self): > ... ? ? ? ? ? ? ? ? return self.t.__getattr__(self) > ...>>> show(NoGuardProxy(StringIO.StringIO("blah1\nblah2\nblah3\n"))) > > Traceback (most recent call last): > ? File "", line 1, in > ? File "", line 3, in show > TypeError: 'NoGuardProxy' object is not iterable > > # > # Now I'm stuck. > # Can anyone help? > # See [1] for an explanation. Here is an idea: you could get round that by generating a class on the fly, if you don't mind changing the class of the object (untested): def noguardproxy(obj): class NoGuardProxy(type(obj)): def __enter__(self): return self def __exit__(self, type, value, traceback): return False obj.__class__ = NoGuardProxy return obj If you want to keep obj untouched, you could add the line obj = copy(obj) before changing the class. -- Arnaud [1] http://docs.python.org/reference/datamodel.html#new-style-special-lookup From steve at holdenweb.com Wed Jan 27 10:18:20 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 27 Jan 2010 10:18:20 -0500 Subject: Ad hoc lists vs ad hoc tuples In-Reply-To: References: <8a57c64f-0c67-4ec1-b4be-34d76c7e7571@q4g2000yqm.googlegroups.com> Message-ID: <4B60593C.8050202@holdenweb.com> Iain King wrote: > On Jan 27, 10:20 am, Floris Bruynooghe > wrote: >> One thing I ofter wonder is which is better when you just need a >> throwaway sequence: a list or a tuple? E.g.: >> >> if foo in ['some', 'random', 'strings']: >> ... >> if [bool1, bool2, boo3].count(True) != 1: >> ... >> >> (The last one only works with tuples since python 2.6) >> >> Is a list or tuple better or more efficient in these situations? >> >> Regards >> Floris >> >> PS: This is inspired by some of the space-efficiency comments from the >> list.pop(0) discussion. > > I tend to use tuples unless using a list makes it easier to read. For > example: > > if foo in ('some', 'random', 'strings'): > > draw.text((10,30), "WHICH IS WHITE", font=font) > draw.line([(70,25), (85,25), (105,45)]) > > I've no idea what the performance difference is; I've always assumed > it's negligible. > The fact that you have felt able to neglect the performance difference makes it quite literally negligible. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Wed Jan 27 10:18:20 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 27 Jan 2010 10:18:20 -0500 Subject: Ad hoc lists vs ad hoc tuples In-Reply-To: References: <8a57c64f-0c67-4ec1-b4be-34d76c7e7571@q4g2000yqm.googlegroups.com> Message-ID: <4B60593C.8050202@holdenweb.com> Iain King wrote: > On Jan 27, 10:20 am, Floris Bruynooghe > wrote: >> One thing I ofter wonder is which is better when you just need a >> throwaway sequence: a list or a tuple? E.g.: >> >> if foo in ['some', 'random', 'strings']: >> ... >> if [bool1, bool2, boo3].count(True) != 1: >> ... >> >> (The last one only works with tuples since python 2.6) >> >> Is a list or tuple better or more efficient in these situations? >> >> Regards >> Floris >> >> PS: This is inspired by some of the space-efficiency comments from the >> list.pop(0) discussion. > > I tend to use tuples unless using a list makes it easier to read. For > example: > > if foo in ('some', 'random', 'strings'): > > draw.text((10,30), "WHICH IS WHITE", font=font) > draw.line([(70,25), (85,25), (105,45)]) > > I've no idea what the performance difference is; I've always assumed > it's negligible. > The fact that you have felt able to neglect the performance difference makes it quite literally negligible. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From kw at codebykevin.com Wed Jan 27 10:23:17 2010 From: kw at codebykevin.com (Kevin Walzer) Date: Wed, 27 Jan 2010 10:23:17 -0500 Subject: Library support for Python 3.x Message-ID: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> I'm going to be starting some new Python projects in Python 2.6, but am concerned that at least three of the libraries I will be using--pycrypto, paramiko and feedparser--are not currently supported in Python 3.x. The authors of these programs have not given any indication that work is underway to support Python 3.x. Eventually I plan to migrate to Python 3.x. I don't want to be stuck using libraries that may not be updated for the next generation of Python. How are others handling this issue? --Kevin -- Kevin Walzer Code by Kevin http://www.codebykevin.com From fetchinson at googlemail.com Wed Jan 27 10:26:37 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Wed, 27 Jan 2010 16:26:37 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: >>>>> * Print is now a function. Great, much improvement. >>> Actually not, IMHO. All it does is is to provide incompatibility. >> >> >> What incompatibility are you exactly talking about? >> >> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >>>>> print( 'hello' ) >> hello >>>>> print 'hello' >> hello >> >> Yes, this is with python 2.6.2 which is in the 2.x line of releases. So? > > I gather that your example is about code that technically executes fine with > both versions and produces the same result, i.e. that there is a subset with > the > same syntax and semantics. > > But 'print' calls that technically execute fine with both versions may and > will > in general produce different results. > > I.e. not just the syntax but also the semantics have changed: > > > >>> import sys > >>> sys.version > '2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]' > >>> > >>> print( "2+2 =", 2+2 ) > ('2+2 =', 4) > >>> _ > > > >>> import sys > >>> sys.version > '3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)]' > >>> > >>> print( "2+2 =", 2+2 ) > 2+2 = 4 > >>> _ True. However, as someone else pointed out in a neighbouring thread you can do Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from __future__ import print_function >>> print( "2+2 =", 2+2 ) 2+2 = 4 >>> which gives 100% compatibility as far as print is concerned between 2.6 and 3.x. So again, what sort of an incompatibility are you talking about concerning 'print' between 2.6 and 3.x? Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From alfps at start.no Wed Jan 27 10:47:04 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 16:47:04 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: * Daniel Fetchinson: >>>>>> * Print is now a function. Great, much improvement. >>>> Actually not, IMHO. All it does is is to provide incompatibility. >>> >>> What incompatibility are you exactly talking about? >>> >>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>>>>> print( 'hello' ) >>> hello >>>>>> print 'hello' >>> hello >>> >>> Yes, this is with python 2.6.2 which is in the 2.x line of releases. So? >> I gather that your example is about code that technically executes fine with >> both versions and produces the same result, i.e. that there is a subset with >> the >> same syntax and semantics. >> >> But 'print' calls that technically execute fine with both versions may and >> will >> in general produce different results. >> >> I.e. not just the syntax but also the semantics have changed: >> >> >> >>> import sys >> >>> sys.version >> '2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]' >> >>> >> >>> print( "2+2 =", 2+2 ) >> ('2+2 =', 4) >> >>> _ >> >> >> >>> import sys >> >>> sys.version >> '3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)]' >> >>> >> >>> print( "2+2 =", 2+2 ) >> 2+2 = 4 >> >>> _ > > True. However, as someone else pointed out in a neighbouring thread you can do > > Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) > [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> from __future__ import print_function >>>> print( "2+2 =", 2+2 ) > 2+2 = 4 > > which gives 100% compatibility as far as print is concerned between 2.6 and 3.x. That makes the code behave with 3.x syntax and semantics regarding print. I.e. it chooses one language. It doesn't make them compatible: if they were, then you wouldn't have to choose. For example, it doesn't fix up 2.x code that prints tuples: incompatible, it will still execute but no longer do what it was supposed to do. But lest you focus down on that rare case, consider 2.x code that contains print "2+2", 2 All such statements need to be fixed up in a 2->3 transition. Adding __future__ just makes the code invalid, it doesn't give you any shred of compatibility for this. A code rewrite can be partially automated like 2to3 but the incompatibility needlessly introduces an additional set of things that Can Go Wrong(TM), and with Murphy present, as He always is, well. > So again, what sort of an incompatibility are you talking about > concerning 'print' between 2.6 and 3.x? Syntax and semantics. They're different. Cheers & hth., - Alf From solipsis at pitrou.net Wed Jan 27 10:56:10 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 27 Jan 2010 15:56:10 +0000 (UTC) Subject: ctypes for AIX References: <9C240725D2ABD949A80B46DD93AB107042E8D44101@XCH-NW-08V.nw.nos.boeing.com> <50697b2c1001240630r33845a1bl7f94b40a4f0c4436@mail.gmail.com> <9C240725D2ABD949A80B46DD93AB107042E8D44574@XCH-NW-08V.nw.nos.boeing.com> Message-ID: Hello, > I then figured I would get a new copy of python and install it > on AIX. I downloaded python.2.5.5c2 from http://www.python.org. I did > the configure and make which posted many errors in the ctypes function > which I guess is the reason that is does not get include in the final > make. I would suggest two things: 1) try with the latest Python 2.6.x, because 2.5 only receives security fixes now. 2.6 is the stable branch which receives regular bug fixes (which includes build problems) 2) if Python 2.6.x also fails compiling, please file a bug report on http://bugs.python.org Regards Antoine. From Eric_Dexter at msn.com Wed Jan 27 11:11:29 2010 From: Eric_Dexter at msn.com (Eric_Dexter at msn.com) Date: Wed, 27 Jan 2010 08:11:29 -0800 (PST) Subject: myths about python 3 References: <6faf39c91001270245q155a932aj3e0bb98d377ddad1@mail.gmail.com> <4b6050ef$1@dnews.tpgi.com.au> Message-ID: <2cd4f02e-ffef-4c50-b4d5-78d92111c7fa@a32g2000yqm.googlegroups.com> On Jan 27, 8:42?am, Lie Ryan wrote: > On 01/28/10 01:32, Jean-Michel Pichavant wrote: > > > > > Daniel Fetchinson wrote: > >>>> Hi folks, > > >>>> I was going to write this post for a while because all sorts of myths > >>>> periodically come up on this list about python 3. I don't think the > >>>> posters mean to spread false information on purpose, they simply are > >>>> not aware of the facts. > > >>>> My list is surely incomplete, please feel free to post your favorite > >>>> misconception about python 3 that people periodically state, claim or > >>>> ask about. > > >>>> 1. Print statement/function creates incompatibility between 2.x and > >>>> 3.x! > > >>>> Certainly false or misleading, if one uses 2.6 and 3.x the > >>>> incompatibility is not there. Print as a function works in 2.6: > > >>>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) > >>>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 > >>>> Type "help", "copyright", "credits" or "license" for more information. > > >>>>>>> print( 'hello' ) > > >>>> hello > > >>>>>>> print 'hello' > > >>>> hello > >>>> ? ? ? 2. Integer division creates incompatibility between 2.x and 3.x! > > >>>> Again false or misleading, because one can get the 3.x behavior with > >>>> 2.6: > > >>>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) > >>>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 > >>>> Type "help", "copyright", "credits" or "license" for more information. > > >>>>>>> 6/5 > > >>>> 1 > > >>>>>>> from __future__ import division > >>>>>>> 6/5 > > >>>> 1.2 > > >>>> Please feel free to post your favorite false or misleading claim about > >>>> python 3! > > >>> Well, I see two false or misleading claims just above - namely that > >>> the two claims above are false or misleading. They tell just half of > >>> the story, and that half is indeed easy. A Python 3 program can be > >>> unchanged (in the case of print) or with only trivial modifications > >>> (in the case of integer division) be made to run on Python 2.6. > > >> Okay, so we agree that as long as print and integer division is > >> concerned, a program can easily be written that runs on both 2.6 and > >> 3.x. > > >> My statements are exactly this, so I don't understand why you disagree. > > >>> The other way around this is _not_ the case. > > >> What do you mean? > > >>> To say that two things are > >>> compatible if one can be used for the other, but the other not for the > >>> first, is false or misleading. > > >> I'm not sure what you mean here. Maybe I didn't make myself clear > >> enough, but what I mean is this: as long as print and integer division > >> is concerned, it is trivial to write code that runs on both 2.6 and > >> 3.x. Hence if someone wants to highlight incompatibility (which surely > >> exists) between 2.6 and 3.x he/she has to look elsewhere. > > >> Cheers, > >> Daniel > > > How would you write in python 2.6 > > > if print: > > ? ?print('Hello') > > > --- > > > def myPrint(*args): > > ? ?for arg in args: > > ? ? ? ?sys.stdout.write(str(arg)) > > > print = myPrint > > > JM > > from __future__ import print_function > > if print: > ? ? print('Hello') > > def myPrint(*args): > ? ?for arg in args: > ? ? ? ?sys.stdout.write(str(arg)) > > print = myPrint I can't say that I am that keen on 2.6 all my favorite graphics libraries are in 2.5. If there was money involved I would probably think y'all were doing it to stay employed so I am thinking I should wait till 3.4 and 3.5 to get involved with this but much sooner than python 4.0. I did notice that I had trouble compiling a library because some version of microsoft c is no longer available...sort of forced migration. ______________________________________ http://dextracker.blogspot.com/ From stefan_ml at behnel.de Wed Jan 27 11:24:37 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 27 Jan 2010 17:24:37 +0100 Subject: python 3's adoption In-Reply-To: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: <4b6068c6$0$6725$9b4e6d93@newsspool2.arcor-online.net> Xah Lee, 27.01.2010 00:47: > Any comment on this? No, sorry. Not worth bothering. Stefan From mmm286 at gmail.com Wed Jan 27 11:49:39 2010 From: mmm286 at gmail.com (mierdatutis mi) Date: Wed, 27 Jan 2010 17:49:39 +0100 Subject: Help parsing a page with python In-Reply-To: References: <8c7f10c61001270213p10ad7d12ra39c75bea1f5e0f4@mail.gmail.com> Message-ID: Hello, Thanks Javier, But I think that the page embeds a viewer. Only the viewer knows the URL to the FLV file itself. I can't see any direct correspondence between the elements of the two URLs, I cant see a way to construct the FLV's URL from the contents of that page :-( I have to do manually, I dont see other way :-( Many thanks for your answer Muchas gracias 2010/1/27 Javier Collado > Hello, > > You can find some advice here: > http://www.packtpub.com/article/web-scraping-with-python-part-2 > > Best regards, > Javier > > 2010/1/27 mierdatutis mi : > > Hello again, > > > > What test case for Windmill? Can you say me the link, please? > > > > Many thanks > > > > 2010/1/27 Javier Collado > >> > >> Hello, > >> > >> A test case for Windmill might also be used to extract the information > >> that you're looking for. > >> > >> Best regards, > >> Javier > >> > >> 2010/1/27 mierdatutis mi : > >> > Those videos are generated by javascript. > >> > There is some parser with python for javascript??? > >> > > >> > Thanks a lot! > >> > > >> > > >> > 2010/1/27 Simon Brunning > >> >> > >> >> 2010/1/27 mierdatutis mi : > >> >> > Hi, > >> >> > > >> >> > I would like to parse a webpage to can get the url of the video > >> >> > download. I > >> >> > use pyhton and firebug but I cant get the url link. > >> >> > > >> >> > Example: > >> >> > > >> >> > The url where I have to get the video link is: > >> >> > > >> >> > > >> >> > > http://www.rtve.es/mediateca/videos/20100125/saber-comer---salsa-verde-judiones-25-01-10/676590.shtml > " > >> >> > > >> >> > The video is > >> >> > http://www.rtve.es/resources/TE_SSAC011/flv/8/2/1264426362028.flv > >> >> > Could you help me please? > >> >> > >> >> That URL doesn't appear to be in the HTML - it must be being brought > >> >> in by the JavaScript somehow. > >> >> > >> >> -- > >> >> Cheers, > >> >> Simon B. > >> >> -- > >> >> http://mail.python.org/mailman/listinfo/python-list > >> > > >> > > >> > -- > >> > http://mail.python.org/mailman/listinfo/python-list > >> > > >> > > >> -- > >> http://mail.python.org/mailman/listinfo/python-list > > > > > > -- > > http://mail.python.org/mailman/listinfo/python-list > > > > > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fetchinson at googlemail.com Wed Jan 27 11:58:10 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Wed, 27 Jan 2010 17:58:10 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: >>>>>>> * Print is now a function. Great, much improvement. >>>>> Actually not, IMHO. All it does is is to provide incompatibility. >>>> >>>> What incompatibility are you exactly talking about? >>>> >>>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >>>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >>>> Type "help", "copyright", "credits" or "license" for more information. >>>>>>> print( 'hello' ) >>>> hello >>>>>>> print 'hello' >>>> hello >>>> >>>> Yes, this is with python 2.6.2 which is in the 2.x line of releases. So? >>> I gather that your example is about code that technically executes fine >>> with >>> both versions and produces the same result, i.e. that there is a subset >>> with >>> the >>> same syntax and semantics. >>> >>> But 'print' calls that technically execute fine with both versions may >>> and >>> will >>> in general produce different results. >>> >>> I.e. not just the syntax but also the semantics have changed: >>> >>> >>> >>> import sys >>> >>> sys.version >>> '2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]' >>> >>> >>> >>> print( "2+2 =", 2+2 ) >>> ('2+2 =', 4) >>> >>> _ >>> >>> >>> >>> import sys >>> >>> sys.version >>> '3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)]' >>> >>> >>> >>> print( "2+2 =", 2+2 ) >>> 2+2 = 4 >>> >>> _ >> >> True. However, as someone else pointed out in a neighbouring thread you >> can do >> >> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >>>>> from __future__ import print_function >>>>> print( "2+2 =", 2+2 ) >> 2+2 = 4 >> >> which gives 100% compatibility as far as print is concerned between 2.6 >> and 3.x. > > That makes the code behave with 3.x syntax and semantics regarding print. > I.e. it chooses one language. > > It doesn't make them compatible: Of course it makes them compatible. I'm not saying any print-related code in python 2.6 is valid python 3 code, but that it is *possible* to write print-related code in python 2.6 that is also valid in python 3. > if they were, then you wouldn't have to choose. It seems to me you are arguing with the statement "Any print-related python 2.6 code is valid python 3 code". Nobody is making this statement. Let me repeat, what you should be arguing with is "It is possible to write print-related python 2.6 code that is also valid python 3 code". Perhaps I didn't make myself clear before, but at least now I hope it's clear what I mean. Cheers, Daniel > For example, it doesn't fix up 2.x code that prints tuples: incompatible, it > will still execute but no longer do what it was supposed to do. > > But lest you focus down on that rare case, consider 2.x code that contains > > print "2+2", 2 > > All such statements need to be fixed up in a 2->3 transition. Adding > __future__ > just makes the code invalid, it doesn't give you any shred of compatibility > for > this. A code rewrite can be partially automated like 2to3 but the > incompatibility needlessly introduces an additional set of things that Can > Go > Wrong(TM), and with Murphy present, as He always is, well. > > >> So again, what sort of an incompatibility are you talking about >> concerning 'print' between 2.6 and 3.x? > > Syntax and semantics. They're different. -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From aahz at pythoncraft.com Wed Jan 27 12:25:09 2010 From: aahz at pythoncraft.com (Aahz) Date: 27 Jan 2010 09:25:09 -0800 Subject: Library support for Python 3.x References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> Message-ID: In article <9f3c3$4b605a65$4275d90a$30451 at FUSE.NET>, Kevin Walzer wrote: > >I'm going to be starting some new Python projects in Python 2.6, but am >concerned that at least three of the libraries I will be >using--pycrypto, paramiko and feedparser--are not currently supported in >Python 3.x. The authors of these programs have not given any indication >that work is underway to support Python 3.x. Eventually I plan to >migrate to Python 3.x. > >I don't want to be stuck using libraries that may not be updated for the >next generation of Python. How are others handling this issue? >From my POV, your question would be precisely identical if you had started your project when Python 2.3 was just released and wanted to know if the libraries you selected would be available for Python 2.6. Choose the best libraries for your circumstances now and worry about the future later. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From chiragdhyani at gmail.com Wed Jan 27 12:27:43 2010 From: chiragdhyani at gmail.com (Chirag Dhyani) Date: Wed, 27 Jan 2010 22:57:43 +0530 Subject: GUI-pyqt solution needed Message-ID: <83ab036c1001270927p4cc07a3dna3c5a08b2c5432d1@mail.gmail.com> Hi, My sincere apologies if I am mailing in wrong list. My mission: I have to develop GUI based tool on python and will be using PyQT with qtdesigner. I am automating creation of test script for call flows just by drawing them from tool. Challenges: In this tool there will be a drawing area where I should have to utility to create call flow by drag and droping the elements. I will have a small database from which relevent piece of codes will be retrieved during the execution time and the script will be retrieved. I know that I will be quering the database based on cordinates so think that cordinate programming will be the correct solution. However, can anybody have idea for the ideal solution that can be? Thank you in advance. Chi -------------- next part -------------- An HTML attachment was scrubbed... URL: From aahz at pythoncraft.com Wed Jan 27 12:27:47 2010 From: aahz at pythoncraft.com (Aahz) Date: 27 Jan 2010 09:27:47 -0800 Subject: Simple Password Strength Checker Review Help needed References: <158b9ef31001262148g1926d067ja8ba840ba074eca4@mail.gmail.com> <7xfx5sxbmw.fsf@ruckus.brouhaha.com> Message-ID: In article <7xfx5sxbmw.fsf at ruckus.brouhaha.com>, Paul Rubin wrote: > >From a security point of view, the concept >of "password strength checking" is pretty dubious. If you want secure >passwords, generate them from a random number source and assign them to >the users. Don't have the users make up their own passwords. It's >relatively (compared to using a computer file exposed to remote internet >attacks) for users to write down the the random passwords on paper, as >long as they're a little bit careful. As Bruce Schneier put it: > > "My wallet is already a secure container; it has valuable things in > it, and I have a lifetime of experience keeping it safe. Adding a > piece of paper with my passwords seems like a natural thing to do." Actually, I treat my wallet as a source of trouble and only keep replaceable items in it. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From alfps at start.no Wed Jan 27 12:29:25 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 18:29:25 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: * Daniel Fetchinson: >>>>>>>> * Print is now a function. Great, much improvement. >>>>>> Actually not, IMHO. All it does is is to provide incompatibility. >>>>> What incompatibility are you exactly talking about? >>>>> >>>>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >>>>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >>>>> Type "help", "copyright", "credits" or "license" for more information. >>>>>>>> print( 'hello' ) >>>>> hello >>>>>>>> print 'hello' >>>>> hello >>>>> >>>>> Yes, this is with python 2.6.2 which is in the 2.x line of releases. So? >>>> I gather that your example is about code that technically executes fine >>>> with >>>> both versions and produces the same result, i.e. that there is a subset >>>> with >>>> the >>>> same syntax and semantics. >>>> >>>> But 'print' calls that technically execute fine with both versions may >>>> and >>>> will >>>> in general produce different results. >>>> >>>> I.e. not just the syntax but also the semantics have changed: >>>> >>>> >>>> >>> import sys >>>> >>> sys.version >>>> '2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]' >>>> >>> >>>> >>> print( "2+2 =", 2+2 ) >>>> ('2+2 =', 4) >>>> >>> _ >>>> >>>> >>>> >>> import sys >>>> >>> sys.version >>>> '3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)]' >>>> >>> >>>> >>> print( "2+2 =", 2+2 ) >>>> 2+2 = 4 >>>> >>> _ >>> True. However, as someone else pointed out in a neighbouring thread you >>> can do >>> >>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>>>>> from __future__ import print_function >>>>>> print( "2+2 =", 2+2 ) >>> 2+2 = 4 >>> >>> which gives 100% compatibility as far as print is concerned between 2.6 >>> and 3.x. >> That makes the code behave with 3.x syntax and semantics regarding print. >> I.e. it chooses one language. >> >> It doesn't make them compatible: > > Of course it makes them compatible. I'm not saying any print-related > code in python 2.6 is valid python 3 code, but that it is *possible* > to write print-related code in python 2.6 that is also valid in python > 3. > >> if they were, then you wouldn't have to choose. > > It seems to me you are arguing with the statement "Any print-related > python 2.6 code is valid python 3 code". Nobody is making this > statement. Let me repeat, what you should be arguing with is "It is > possible to write print-related python 2.6 code that is also valid > python 3 code". Perhaps I didn't make myself clear before, but at > least now I hope it's clear what I mean. Perhaps we're in violent agreement. :-) However, the root node of this subthread was my statement about the needless incompatibility introduced by changing print in 3.x, whether that statement was correct or reasonable / whatever. The main problem with the incompatibility is for porting code, not for writing code from scratch. It's also a problem wrt. learning the language. And I see no good reason for it: print can't really do more, or less, or more conveniently (rather, one has to write a bit more now for same effect). Cheers, - Alf From solipsis at pitrou.net Wed Jan 27 12:32:27 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 27 Jan 2010 17:32:27 +0000 (UTC) Subject: Ad hoc lists vs ad hoc tuples References: <8a57c64f-0c67-4ec1-b4be-34d76c7e7571@q4g2000yqm.googlegroups.com> Message-ID: Le Wed, 27 Jan 2010 02:20:53 -0800, Floris Bruynooghe a ?crit?: > > Is a list or tuple better or more efficient in these situations? Tuples are faster to allocate (they are allocated in one single step) and quite a bit smaller too. In some situations, in Python 2.7 and 3.1, they can also be ignored by the garbage collector, yielding faster collections. (not to mention that they are hashable, which can be useful) From adityashukla1983 at gmail.com Wed Jan 27 12:37:33 2010 From: adityashukla1983 at gmail.com (aditya shukla) Date: Wed, 27 Jan 2010 11:37:33 -0600 Subject: Extract half screen of a video Message-ID: <73045cca1001270937k5894ae2dg92f472fe89b7b8e2@mail.gmail.com> Hello Guys, I have a video and what I want is to extract only half of the screen of it.By half screen i mean when i run the video i can say the left half(from the monitor's screen) of the video and dump the right half. Thanks Aditya -------------- next part -------------- An HTML attachment was scrubbed... URL: From luismgz at gmail.com Wed Jan 27 12:38:36 2010 From: luismgz at gmail.com (=?ISO-8859-1?Q?Luis_M=2E_Gonz=E1lez?=) Date: Wed, 27 Jan 2010 09:38:36 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> Message-ID: > Please don't post more noise and ad hominem attacks to the group, Steve. "Ad hominem"? Please, operor non utor lingua non notus per vulgaris populus. Gratias ago vos... From steve at holdenweb.com Wed Jan 27 12:50:10 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 27 Jan 2010 12:50:10 -0500 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: <4B607CD2.1060408@holdenweb.com> Alf P. Steinbach wrote: [...] > The main problem with the incompatibility is for porting code, not for > writing code from scratch. It's also a problem wrt. learning the > language. And I see no good reason for it: print can't really do more, > or less, or more conveniently (rather, one has to write a bit more now > for same effect). Of course it can do more: it allows you to layer your own print functionality into the system much more easily than you could with the print statement. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From alfps at start.no Wed Jan 27 12:52:57 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 18:52:57 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: * Steve Holden: > Alf P. Steinbach wrote: > [...] >> The main problem with the incompatibility is for porting code, not for >> writing code from scratch. It's also a problem wrt. learning the >> language. And I see no good reason for it: print can't really do more, >> or less, or more conveniently (rather, one has to write a bit more now >> for same effect). > > Of course it can do more: it allows you to layer your own print > functionality into the system much more easily than you could with the > print statement. Yeah, point. Very minor though. :-) Cheers, - Alf From arnodel at googlemail.com Wed Jan 27 12:57:59 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Wed, 27 Jan 2010 17:57:59 +0000 Subject: site.py confusion References: Message-ID: George Trojan writes: > Inspired by the 'Default path for files' thread I tried to use > sitecustomize in my code. What puzzles me is that the site.py's main() > is not executed. My sitecustomize.py is > def main(): > print 'In Main()' > main() > and the test program is > import site > #site.main() > print 'Hi' > The output is > $ python try.py > Hi That's normal as site.py is automatically imported on initialisation. So when you do: import site the module is not re-executed as it already has been imported. Try this: ---- file: foo.py print 'Foo' ---- end --- Interactive session >>> import foo # First time, print statement executed Foo >>> import foo # Second time, print statement not executed >>> > When I uncomment the site.main() line the output is > $ python try.py > In Main() > Hi Now you explicitely call site.main(), so it executes it! > If I change import site to import sitecustomize the output is as > above. What gives? It's normal, this time it's the first time you import it so its content is executed. HTH -- Arnaud From MLMDev at Comcast.net Wed Jan 27 13:29:41 2010 From: MLMDev at Comcast.net (Mitchell L Model) Date: Wed, 27 Jan 2010 13:29:41 -0500 Subject: I really need webbrowser.open('file://') to open a web browser In-Reply-To: References: Message-ID: <5A08CF3C-63E4-4103-B4B2-6C17E1D6B6D0@Comcast.net> On Jan 15, 2010, at 3:59 PM, Timur Tabi > After reading several web pages and mailing list threads, I've learned > that the webbrowser module does not really support opening local > files, even if I use a file:// URL designator. In most cases, > webbrowser.open() will indeed open the default web browser, but with > Python 2.6 on my Fedora 10 system, it opens a text editor instead. On > Python 2.5, it opens the default web browser. > > This is a problem because my Python script creates a local HTML file > and I want it displayed on the web browser. > > So is there any way to force webbrowser.open() to always use an actual > web browser? I had some discussions with the Python documentation writers that led to the following note being included in the Python 3.1 library documentation for webbrowser.open: "Note that on some platforms, trying to open a filename using this function, may work and start the operating system?s associated program. However, this is neither supported nor portable." The discussions suggested that this lack of support and portability was actually always the case and that the webbrowser module is simply not meant to handle file URLs. I had taken advantage of the accidental functionality to generate HTML reports and open them, as well as to open specific documentation pages from within a program. You can control which browser opens the URL by using webbrowser.get to obtain a controller for a particular browser, specified by its argument, then call the open method on the controller instead of the module. For opening files reliability and the ability to pick a particular program (browser or otherwise) to open it with you might have to resort to invoking a command line via subprocess.Popen. From george.trojan at noaa.gov Wed Jan 27 13:48:23 2010 From: george.trojan at noaa.gov (George Trojan) Date: Wed, 27 Jan 2010 18:48:23 +0000 Subject: site.py confusion In-Reply-To: References: Message-ID: Arnaud Delobelle wrote: > George Trojan writes: > >> Inspired by the 'Default path for files' thread I tried to use >> sitecustomize in my code. What puzzles me is that the site.py's main() >> is not executed. My sitecustomize.py is >> def main(): >> print 'In Main()' >> main() >> and the test program is >> import site >> #site.main() >> print 'Hi' >> The output is >> $ python try.py >> Hi > > That's normal as site.py is automatically imported on initialisation. > So when you do: > > import site > > the module is not re-executed as it already has been imported. > Try this: > > ---- file: foo.py > print 'Foo' > ---- end > > --- Interactive session > >>> import foo # First time, print statement executed > Foo > >>> import foo # Second time, print statement not executed > >>> > >> When I uncomment the site.main() line the output is >> $ python try.py >> In Main() >> Hi > > Now you explicitely call site.main(), so it executes it! > >> If I change import site to import sitecustomize the output is as >> above. What gives? > > It's normal, this time it's the first time you import it so its content > is executed. > > HTH > I understand that importing a module repeatedly does nothing. Also, I made a typo in my previous posting - I meant sitecustomize.main(), not site.main(). My understanding of the code in site.py is that when the module is imported, main() is executed. main() calls execsitecustomize() that attempts to import sitecustomize. That action should trigger execution of code in sitecustomize.py, which is located in the current directory. But that does not work. I changed execsitecustomize() to def execsitecustomize(): """Run custom site specific code, if available.""" try: import sitecustomize except ImportError: pass import sys print sys.path raise That gave me the explanation why the above happens: when site is imported, the current directory is not yet prepended to sys.path. $ python2.6 -v try.py ... ['/usr/local/Python-2.6.3/lib/python26.zip', '/usr/local/Python-2.6.3/lib/python2.6', '/usr/local/Python-2.6.3/lib/python2.6/plat-linux2', '/usr/local/Python-2.6.3/lib/python2.6/lib-tk', '/usr/local/Python-2.6.3/lib/python2.6/lib-old', '/usr/local/Python-2.6.3/lib/python2.6/lib-dynload', '/usr/local/Python-2.6.3/lib/python2.6/site-packages'] 'import site' failed; traceback: Traceback (most recent call last): File "/usr/local/Python-2.6.3/lib/python2.6/site.py", line 516, in main() File "/usr/local/Python-2.6.3/lib/python2.6/site.py", line 507, in main execsitecustomize() File "/usr/local/Python-2.6.3/lib/python2.6/site.py", line 472, in execsitecustomize import sitecustomize ... This also explains the recipe http://code.activestate.com/recipes/552729/ I wanted to have library location specific to application without having to explicitly change sys.path in all App-Top-Dir/bin/*.py. I thought creating bin/sitecustomize.py would do the trick. I guess the documentation should mention this fact. The comment in recipe 552729 is: Since Python 2.5 the automatic import of the module "sitecustomize.py" in the directory of the main program is not supported any more (even if the documentation says that it is). George From no.email at nospam.invalid Wed Jan 27 14:03:44 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 27 Jan 2010 11:03:44 -0800 Subject: Library support for Python 3.x References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> Message-ID: <7xk4v3mktr.fsf@ruckus.brouhaha.com> aahz at pythoncraft.com (Aahz) writes: > From my POV, your question would be precisely identical if you had > started your project when Python 2.3 was just released and wanted to > know if the libraries you selected would be available for Python 2.6. I didn't realize 2.6 broke libraries that had worked in 2.3, at least on any scale. Did I miss something? From exarkun at twistedmatrix.com Wed Jan 27 14:23:36 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Wed, 27 Jan 2010 19:23:36 -0000 Subject: Library support for Python 3.x In-Reply-To: <7xk4v3mktr.fsf@ruckus.brouhaha.com> References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> Message-ID: <20100127192336.1898.1244562725.divmod.xquotient.811@localhost.localdomain> On 07:03 pm, no.email at nospam.invalid wrote: >aahz at pythoncraft.com (Aahz) writes: >> From my POV, your question would be precisely identical if you had >>started your project when Python 2.3 was just released and wanted to >>know if the libraries you selected would be available for Python 2.6. > >I didn't realize 2.6 broke libraries that had worked in 2.3, at least >on >any scale. Did I miss something? Lots of tiny things change. Any of these can break a library. With the 3 releases between 2.3 and 2.6, there are lots of opportunities for these changes. I don't know what you mean by "any scale", but I know that I've seen lots of things break on 2.6 that worked on 2.3, 2.4, or 2.5. Jean-Paul From sjdevnull at yahoo.com Wed Jan 27 14:26:25 2010 From: sjdevnull at yahoo.com (sjdevnull at yahoo.com) Date: Wed, 27 Jan 2010 11:26:25 -0800 (PST) Subject: Library support for Python 3.x References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> Message-ID: <29a1c902-0fba-4736-8c24-649e768a5ef2@c29g2000yqd.googlegroups.com> On Jan 27, 2:03?pm, Paul Rubin wrote: > a... at pythoncraft.com (Aahz) writes: > > From my POV, your question would be precisely identical if you had > > started your project when Python 2.3 was just released and wanted to > > know if the libraries you selected would be available for Python 2.6. > > I didn't realize 2.6 broke libraries that had worked in 2.3, at least on > any scale. ?Did I miss something? I certainly had to update several modules I use (C extensions) to work with the new memory management in a recent release (changing PyMem_Del to Py_DECREF being a pretty common alteration); I can't remember whether that was for 2.6 or 2.5. From kyosohma at gmail.com Wed Jan 27 14:29:04 2010 From: kyosohma at gmail.com (Mike Driscoll) Date: Wed, 27 Jan 2010 11:29:04 -0800 (PST) Subject: Pyowa Meeting Next Monday Message-ID: <0834b458-e941-45d4-9be3-280daf97db46@21g2000yqj.googlegroups.com> Hi, Next Monday, February 1st, we will be having our first Pyowa meeting of 2010! It will be held at the Marshall County Sheriff's Office and start at 7 p.m. (barring inclement weather). I will be demoing some software I created for our Sheriff's office and will talk about the various modules used to put it all together. We'll probably have plenty of free time to chat and you can show off anything neat that you've been doing too. I hope to see you there! ----------------- Mike Driscoll Website: www.pyowa.org Twitter: www.twitter.com/pyowa From invalid at invalid.invalid Wed Jan 27 15:05:35 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Wed, 27 Jan 2010 20:05:35 +0000 (UTC) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: On 2010-01-27, Alf P. Steinbach wrote: > * Steve Holden: >> Alf P. Steinbach wrote: >> [...] >>> The main problem with the incompatibility is for porting code, not for >>> writing code from scratch. It's also a problem wrt. learning the >>> language. And I see no good reason for it: print can't really do more, >>> or less, or more conveniently (rather, one has to write a bit more now >>> for same effect). >> >> Of course it can do more: it allows you to layer your own print >> functionality into the system much more easily than you could with the >> print statement. > > Yeah, point. Very minor though. :-) I don't think it's minor at all. Being able to globally redefine the behavior of "print" for all modules is a big win when you want to import a module that's too chatty about what it's doing. The "right" way for modules to chatter about internals is using the logging module, but not everybody follows that convention. -- Grant Edwards grante Yow! Four thousand at different MAGNATES, MOGULS visi.com & NABOBS are romping in my gothic solarium!! From mcfletch at vrplumber.com Wed Jan 27 15:23:25 2010 From: mcfletch at vrplumber.com (Mike C. Fletcher) Date: Wed, 27 Jan 2010 15:23:25 -0500 Subject: Library support for Python 3.x In-Reply-To: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> Message-ID: <4B60A0BD.5050301@vrplumber.com> Kevin Walzer wrote: > I'm going to be starting some new Python projects in Python 2.6, but > am concerned that at least three of the libraries I will be > using--pycrypto, paramiko and feedparser--are not currently supported > in Python 3.x. The authors of these programs have not given any > indication that work is underway to support Python 3.x. Eventually I > plan to migrate to Python 3.x. > > I don't want to be stuck using libraries that may not be updated for > the next generation of Python. How are others handling this issue? > > --Kevin Some of us are saying: * When Python 3.x has something compelling to offer: o Users will start asking for Python 3.x with a *reason* to justify the cost o Libraries will begin to see the pain of porting, and most importantly, *maintaining*, as a reasonable trade-off o Libraries will port (and maintain) o Applications should *then* port * or When everyone is already on a (basically) compatible 2.x version (i.e. 2.6+); say in 3 years: o Conversion and maintenance costs will lower to the point where we can justify them for libraries Python 3.x has very little that is a compelling use-case (versus 2.x) for most people I've talked to. My paying customers aren't screaming "we want to spend a week of your time to rewrite, re-test and re-deploy our working code into Python 3.x so that it can do exactly the same thing with no visible benefit and lots of likely subtle failures". Unicode-as-default doesn't really make customers say "wow" when all their Unicode-needing code is already Unicode-using. A few syntax changes here and there... well, no, they certainly don't care (can't say I *really* do either). The initial marked slowdown for 3.0 (which I gather should be mostly fixed in 3.1) didn't do much of a sales-job either. Possible compelling arguments that would get people to convert (with the projects working on them): * 5x (or more) speedup (PyPy, Unladen Swallow) * adaptive parallelization/execution-kernel mechanism as a first-class primitive (Apple's C extensions for OpenCL) * continuation-like mechanisms, anonymous code blocks a-la Ruby (PyPy, Stackless) * (free) threading, GIL removal (IronPython, Jython) * compilation-to-native (PyPy) * compilation to mobile-platform native (PyPy?) None of those in in Python 3.x, though there's talk of merging Unladen Swallow into CPython to provide a carrot for conversions (though AFAIK it's not yet a 5x improvement across the board). To compound the problem, Python 3.x doesn't include any of the syntax changes you'd want to see to support e.g. anonymous blocks, continuations, OpenCL integration, etceteras... so if we want those, we're going to have to either introduce new syntax (which is blocked) or hack it in... which we could do on Python 2.x too. I don't know about other maintainers, but I've started running PyOpenGL tests with -3 on Python 2.6 to get a feel for how many issues are going to come up. Most have been minimal. But when I sit down and actually consider *maintaining* a 3.x release when I'm already *barely* able to keep up with the 2.x maintenance in my tiny amounts of free time... well... I do *that* stuff for *fun* after all, and double the headaches for no noticeable benefit doesn't really sound like fun... oh, and numpy isn't ported, so screw it ;) ... Interestingly, at PyGTA Myles mentioned that he'd found his first-ever Python 3.x-only library! Which he then converted to Python 2.x, because he actually wanted to use it in real code :) . Projects which haven't ported to Python 3.x aren't necessarily dead, they're just not nailed to that particular perch (yet), Mike -- ________________________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://www.vrplumber.com http://blog.vrplumber.com From awilliam at opengroupware.us Wed Jan 27 15:28:34 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Wed, 27 Jan 2010 15:28:34 -0500 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: <1264624114.3103.3.camel@linux-m3mt> On Wed, 2010-01-27 at 18:52 +0100, Alf P. Steinbach wrote: > * Steve Holden: > > Alf P. Steinbach wrote: > > [...] > >> The main problem with the incompatibility is for porting code, not for > >> writing code from scratch. It's also a problem wrt. learning the > >> language. And I see no good reason for it: print can't really do more, > >> or less, or more conveniently (rather, one has to write a bit more now > >> for same effect). > > Of course it can do more: it allows you to layer your own print > > functionality into the system much more easily than you could with the > > print statement. > Yeah, point. Very minor though. :-) So you get to determine that? I'd call the whole print thing (a) a legitimate change to increase consistency and (b) a fairly minor porting nuisance since application code [as in big-chunks-o-code] almost never contains print statements. I know at least two shops that have scripts they run on all Python code, prior to it entering production, to ensure there are no print statements. From timur at freescale.com Wed Jan 27 15:31:37 2010 From: timur at freescale.com (Timur Tabi) Date: Wed, 27 Jan 2010 14:31:37 -0600 Subject: I really need webbrowser.open('file://') to open a web browser In-Reply-To: <5A08CF3C-63E4-4103-B4B2-6C17E1D6B6D0@Comcast.net> References: <5A08CF3C-63E4-4103-B4B2-6C17E1D6B6D0@Comcast.net> Message-ID: On Wed, Jan 27, 2010 at 12:29 PM, Mitchell L Model wrote: > I had some discussions with the Python documentation writers that led to the > following note being included in the Python 3.1 library documentation for > webbrowser.open: "Note that on some platforms, trying to open a filename > using this function, may work and start the operating system?s associated > program. However, this is neither supported nor portable." Then they should have renamed the API. I appreciate that they're finally documenting this, but I still think it's a bunch of baloney. > You can control which browser opens the URL by using webbrowser.get to > obtain a controller for a particular browser, specified by its argument, > then call the open method on the controller instead of the module. How can I know which controller (application) the system will use when it opens an http URL? I depend on webbrowser.open('http') to choose the best web browser on the installed system. Does webbrowser.get() tell me which application that will be? > For opening files reliability and the ability to pick a particular program > (browser or otherwise) to open it with you might have to resort to invoking > a command line via subprocess.Popen. But that only works if I know which application to open. -- Timur Tabi Linux kernel developer at Freescale From nagle at animats.com Wed Jan 27 15:41:57 2010 From: nagle at animats.com (John Nagle) Date: Wed, 27 Jan 2010 12:41:57 -0800 Subject: Simple Password Strength Checker Review Help needed In-Reply-To: References: Message-ID: <4b60a16e$0$1670$742ec2ed@news.sonic.net> Mallikarjun(?????????????) wrote: > Dear friends, > I am newbie to Python + pygtk + Glade and recently wrote a simple password > strength checker app. > Since this is my first app/program, can someone review my code (just over > 150 lines) and help me improve my programming capabilities Here's my classic "Obvious password detector": http://www.animats.com/source/obvious/obvious.c This prevents dictionary attacks using an English dictionary, but needs only a small bit table and does no I/O. John Nagle From invalid at invalid.invalid Wed Jan 27 15:45:21 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Wed, 27 Jan 2010 20:45:21 +0000 (UTC) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On 2010-01-27, John Nagle wrote: > Arguably, Python 3 has been rejected by the market. Let's just say that it hasn't yet been accepted by the market. ;) > Instead, there's now Python 2.6, Python 2.7, and Python 2.8. > Python 3 has turned into a debacle like Perl 6, now 10 years > old. I think I'd have to wait a couple more years before making that sort of pronouncement. That said, I don't expect to start using Python 3 until library availability or my Linux distro forces me to. -- Grant Edwards grante Yow! Inside, I'm already at SOBBING! visi.com From nagle at animats.com Wed Jan 27 15:56:10 2010 From: nagle at animats.com (John Nagle) Date: Wed, 27 Jan 2010 12:56:10 -0800 Subject: myths about python 3 In-Reply-To: References: Message-ID: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Daniel Fetchinson wrote: > Hi folks, > > I was going to write this post for a while because all sorts of myths > periodically come up on this list about python 3. I don't think the > posters mean to spread false information on purpose, they simply are > not aware of the facts. > > My list is surely incomplete, please feel free to post your favorite > misconception about python 3 that people periodically state, claim or > ask about. Myths about Python 3: 1. Python 3 is supported by major Linux distributions. FALSE - most distros are shipping with Python 2.4, or 2.5 at best. 2. Python 3 is supported by multiple Python implementations. FALSE - Only CPython supports 3.x. Iron Python, Unladen Swallow, PyPy, and Jython have all stayed with 2.x versions of Python. 3. Python 3 is supported by most 3rd party Python packages. FALSE - it's not supported by MySQLdb, OpenSSL, feedparser, etc. Arguably, Python 3 has been rejected by the market. Instead, there's now Python 2.6, Python 2.7, and Python 2.8. Python 3 has turned into a debacle like Perl 6, now 10 years old. That's the reality, Python 3 fanboys. John Nagle From nagle at animats.com Wed Jan 27 16:03:04 2010 From: nagle at animats.com (John Nagle) Date: Wed, 27 Jan 2010 13:03:04 -0800 Subject: starting a thread in a nother thread In-Reply-To: <4b604df9$0$6723$9b4e6d93@newsspool2.arcor-online.net> References: <4b603b35$0$6725$9b4e6d93@newsspool2.arcor-online.net> <4b604df9$0$6723$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <4b60a661$0$1598$742ec2ed@news.sonic.net> Stefan Behnel wrote: > Richard Lamboj, 27.01.2010 15:23: >> Am Wednesday 27 January 2010 14:10:13 schrieb Stefan Behnel: >>> Richard Lamboj, 27.01.2010 14:06: >>>> just for _curiosity_. What would be if i start a thread in a nother >>>> thread and acquire a lock in the "child" thread. Is there anything that >>>> could go wrong if someone try to start threads in threads? >>> There's usually tons of things that can go wrong w.r.t. threads: >>> >>> http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf >>> >>> However, there's nothing special to a thread that was started from another >>> thread, so the problems don't change. >> i have tried a little bit around with psycopg2 and threads, >> >> I'am sharing one connection for all threads. When i'am starting the >> threads "normal" everything works without any Problem. When i'am starting the >> threads from another thread than i got a "segmentation fault" > > Sounds like a bug that you might want to report to the maintainers of psycopg2. > > Stefan If a C package called from Python crashes, the package is defective. Nothing you can do from Python should be able to cause a segmentation fault. Google search: "Results 1 - 10 of about 29,400 for psycopg2 crash". John Nagle From nagle at animats.com Wed Jan 27 16:04:48 2010 From: nagle at animats.com (John Nagle) Date: Wed, 27 Jan 2010 13:04:48 -0800 Subject: Library support for Python 3.x In-Reply-To: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> Message-ID: <4b60a6c9$0$1598$742ec2ed@news.sonic.net> Kevin Walzer wrote: > I'm going to be starting some new Python projects in Python 2.6, but am > concerned that at least three of the libraries I will be > using--pycrypto, paramiko and feedparser--are not currently supported in > Python 3.x. The authors of these programs have not given any indication > that work is underway to support Python 3.x. Eventually I plan to > migrate to Python 3.x. Maybe by 2015 or so, that might be feasible. Wait until Python 3.x ships as standard with major Linux distros. Right now, 2.4 or 2.5 is the production version of Python. John Nagle From awilliam at opengroupware.us Wed Jan 27 16:06:24 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Wed, 27 Jan 2010 16:06:24 -0500 Subject: myths about python 3 In-Reply-To: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <1264626384.3103.8.camel@linux-m3mt> On Wed, 2010-01-27 at 12:56 -0800, John Nagle wrote: > Daniel Fetchinson wrote: > > Hi folks, > > I was going to write this post for a while because all sorts of myths > > periodically come up on this list about python 3. I don't think the > > posters mean to spread false information on purpose, they simply are > > not aware of the facts. > > My list is surely incomplete, please feel free to post your favorite > > misconception about python 3 that people periodically state, claim or > > ask about. > Myths about Python 3: > 1. Python 3 is supported by major Linux distributions. > FALSE - most distros are shipping with Python 2.4, or 2.5 at best. CentOS: python26-2.6.4-1.ius.parallel.el5 openSUSE: python-2.6.2-6.3.i586, python3-3.1-3.3.i586 Darn, those pesky facts. > 2. Python 3 is supported by multiple Python implementations. > FALSE - Only CPython supports 3.x. Iron Python, Unladen Swallow, > PyPy, and Jython have all stayed with 2.x versions of Python. And of all Python development what percentage takes place on all those combined? 2%? Maybe. From sjdevnull at yahoo.com Wed Jan 27 16:14:23 2010 From: sjdevnull at yahoo.com (sjdevnull at yahoo.com) Date: Wed, 27 Jan 2010 13:14:23 -0800 (PST) Subject: myths about python 3 References: <6faf39c91001270245q155a932aj3e0bb98d377ddad1@mail.gmail.com> Message-ID: On Jan 27, 9:22?am, Daniel Fetchinson wrote: > >> Hi folks, > > >> I was going to write this post for a while because all sorts of myths > >> periodically come up on this list about python 3. I don't think the > >> posters mean to spread false information on purpose, they simply are > >> not aware of the facts. > > >> My list is surely incomplete, please feel free to post your favorite > >> misconception about python 3 that people periodically state, claim or > >> ask about. > > >> 1. Print statement/function creates incompatibility between 2.x and 3.x! > > >> Certainly false or misleading, if one uses 2.6 and 3.x the > >> incompatibility is not there. Print as a function works in 2.6: > > >> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) > >> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 > >> Type "help", "copyright", "credits" or "license" for more information. > >>>>> print( 'hello' ) > >> hello > >>>>> print 'hello' > >> hello > > >> 2. Integer division creates incompatibility between 2.x and 3.x! > > >> Again false or misleading, because one can get the 3.x behavior with 2.6: > > >> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) > >> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 > >> Type "help", "copyright", "credits" or "license" for more information. > >>>>> 6/5 > >> 1 > >>>>> from __future__ import division > >>>>> 6/5 > >> 1.2 > > >> Please feel free to post your favorite false or misleading claim about > >> python 3! > > > Well, I see two false or misleading claims just above - namely that > > the two claims above are false or misleading. They tell just half of > > the story, and that half is indeed easy. A Python 3 program can be > > unchanged (in the case of print) or with only trivial modifications > > (in the case of integer division) be made to run on Python 2.6. > > Okay, so we agree that as long as print and integer division is > concerned, a program can easily be written that runs on both 2.6 and > 3.x. > > My statements are exactly this, so I don't understand why you disagree. > > > The other way around this is _not_ the case. > > What do you mean? > > > To say that two things are > > compatible if one can be used for the other, but the other not for the > > first, is false or misleading. > > I'm not sure what you mean here. Maybe I didn't make myself clear > enough, but what I mean is this: as long as print and integer division > is concerned, it is trivial to write code that runs on both 2.6 and > 3.x. Hence if someone wants to highlight incompatibility (which surely > exists) between 2.6 and 3.x he/she has to look elsewhere. I think you're misunderstanding what "compatibility" means in a programming language context. Python 3 and Python 2 are not mutually compatible, as arbitrary programs written in one will not run in the other. The most important fallout of that is that Python 3 is not backwards compatible, in that existing Python 2 programs won't run unaltered in Python 3--while it's easy to write a new program in a subset of the languages that runs on both Python 2 and 3, the huge existing codebase of Python 2 code won't run under Python 3. That there exists an intersection of the two languages that is compatible with both doesn't make the two languages compatible with each other--although it being a fairly large subset does help mitigate a sizable chunk of the problems caused by incompatibility (as do tools like 2to3). In your example, a python2 program that uses print and division will fail in python3. The print problem is less significant, since the failure will probably be a syntax error or a rapid exception raised. The division problem is more problematic, since a program may appear to run fine but silently misbehave; such errors are much more likely to result in significant damage to data or other long-term badness. From nobody at nowhere.org Wed Jan 27 16:17:08 2010 From: nobody at nowhere.org (Georg) Date: Wed, 27 Jan 2010 22:17:08 +0100 Subject: C Structure rebuild with ctypes References: <7p6ksnFkg1U1@mid.individual.net><7qo9avFivmU1@mid.individual.net> Message-ID: <7sboqmFe9sU1@mid.individual.net> Hi Mark, many thanks for your hints. > --------------- func.py ------------------ > import ctypes as c > > # int func (int numVars, char **varNames, int *varTypes) > > INT = c.c_int > PINT = c.POINTER(INT) > PCHAR = c.c_char_p > PPCHAR = c.POINTER(PCHAR) > > func = c.CDLL('func').func > func.restype = None > func.argtypes = [INT,PPCHAR,PINT] > > numVars = 3 > varNames = (PCHAR * numVars)('abc','def','ghi') > varTypes = (INT * numVars)(1,2,3) > > func(numVars,varNames,varTypes) The function to call has the following signiture: int SetVarLabel(int handle, const char *varName, const char *varLabel) I did as you suggested by doing the following: INT = c_int PCHAR = c_char_p libc.argtypes = [INT,PCHAR,PCHAR] vn = create_string_buffer("var1") vl = create_string_buffer("Label") print "vn: ", vn.value print "vl: ", vl.value returnCode = libc.SetVarLabel(h,byref(vn), byref(vl)) print "Return Code: ", returnCode The return code is always "variable not found". Thus my call to the C library is not working. What do I do wrong? Best regards Georg From ben+python at benfinney.id.au Wed Jan 27 16:20:30 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 28 Jan 2010 08:20:30 +1100 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <878wbj4541.fsf@benfinney.id.au> John Nagle writes: > Myths about Python 3: > > 1. Python 3 is supported by major Linux distributions. > > FALSE - most distros are shipping with Python 2.4, or 2.5 at > best. There's a big difference between ?What list of versions of Python does ship with?? versus ?Which one version of Python does use for its implementation of ?python???. I think the latter is the more pertinent question. Do you have evidence to back up a claim (not that I'm saying you have necessarily made the claim; it's not clear from what you wrote) that most operating systems use Python 2.4 as their implementation of ?python?? -- \ ?Anyone who believes exponential growth can go on forever in a | `\ finite world is either a madman or an economist.? ?Kenneth | _o__) Boulding | Ben Finney From benjamin.kaplan at case.edu Wed Jan 27 16:25:46 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Wed, 27 Jan 2010 16:25:46 -0500 Subject: myths about python 3 In-Reply-To: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On Wed, Jan 27, 2010 at 3:56 PM, John Nagle wrote: > Daniel Fetchinson wrote: >> >> Hi folks, >> >> I was going to write this post for a while because all sorts of myths >> periodically come up on this list about python 3. I don't think the >> posters mean to spread false information on purpose, they simply are >> not aware of the facts. >> >> My list is surely incomplete, please feel free to post your favorite >> misconception about python 3 that people periodically state, claim or >> ask about. > > Myths about Python 3: > > 1. ?Python 3 is supported by major Linux distributions. > > ? ? ? ?FALSE - most distros are shipping with Python 2.4, or 2.5 at best. > The latest versions of Ubuntu Jaunty and Karmic, Fedora 11 and 12, and OpenSUSE 11.2 all use Python 2.6. Ubuntu has been shipping python 3 since Jaunty came out last April. According to Fedora's package index, Python 3 is in the devel version which probably means it will be in upcoming versions of Fedora as well. > 2. ?Python 3 is supported by multiple Python implementations. > > ? ? ? ?FALSE - Only CPython supports 3.x. ?Iron Python, Unladen Swallow, > ? ? ? ?PyPy, and Jython have all stayed with 2.x versions of Python. > When Python 2.6 came out, Jython was still on 2.2. The difference between 2.2 and 2.6 is almost as big of a difference as between 2.6 and 3.0. In that time, you had the introduction of the boolean type, generators, list comprehensions, the addition of the "yield" and "with" keywords, universal newline support, and decorators in addition to the large number of changes to the standard library such as the introduction of the subprocess module. > 3. ?Python 3 is supported by most 3rd party Python packages. > > ? ? ? ?FALSE - it's not supported by MySQLdb, OpenSSL, feedparser, etc. > > Arguably, Python 3 has been rejected by the market. ?Instead, there's > now Python 2.6, Python 2.7, and Python 2.8. ?Python 3 has turned into > a debacle like Perl 6, now 10 years old. > Give the package maintainers time to update. There were some pretty big changes to the C API. Most of the major 3rd party packages like numpy and MySQLdb have already commited to having a Python 3 version. They just haven't gotten them out yet. > That's the reality, Python 3 fanboys. > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?John Nagle > -- > http://mail.python.org/mailman/listinfo/python-list > From lists at cheimes.de Wed Jan 27 16:26:04 2010 From: lists at cheimes.de (Christian Heimes) Date: Wed, 27 Jan 2010 22:26:04 +0100 Subject: myths about python 3 In-Reply-To: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <4B60AF6C.3090304@cheimes.de> John Nagle wrote: > 1. Python 3 is supported by major Linux distributions. > > FALSE - most distros are shipping with Python 2.4, or 2.5 at best. You are wrong. Modern versions of Debian / Ubuntu are using Python 2.6. My Ubuntu box has python3.0, too. > 2. Python 3 is supported by multiple Python implementations. > > FALSE - Only CPython supports 3.x. Iron Python, Unladen Swallow, > PyPy, and Jython have all stayed with 2.x versions of Python. You are wrong again. We have an active discussion on the Python developer mailinglist to integrate Unladen Swallow into py3k, see http://jessenoller.com/2010/01/06/unladen-swallow-python-3s-best-feature/ . The other implementations have been planing Python 3.x support for more than a year. > > 3. Python 3 is supported by most 3rd party Python packages. > > FALSE - it's not supported by MySQLdb, OpenSSL, feedparser, etc. Python 3.x has builtin support for several aspects of OpenSSL. Other important projects like Cython, lxml, some Postgres DBAs, Django, mod_wsgi and many more have been ported to Python 3.x for more than a year. In fact Graham has helped us to identify several issues in 3.0 beta while he was porting mod_wsgi to Python 3.0. (Personally I neither see MySQL as mission critical nor as a powerful RDBMS. Just try to combine foreign keys with triggers and you'll see what I'm talking about. *scnr*) Christian From ben+python at benfinney.id.au Wed Jan 27 16:30:58 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 28 Jan 2010 08:30:58 +1100 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <874om744ml.fsf@benfinney.id.au> Adam Tauno Williams writes: > On Wed, 2010-01-27 at 12:56 -0800, John Nagle wrote: > > 2. Python 3 is supported by multiple Python implementations. > > FALSE - Only CPython supports 3.x. Iron Python, Unladen Swallow, > > PyPy, and Jython have all stayed with 2.x versions of Python. > > And of all Python development what percentage takes place on all those > combined? That's irrelevant to the point, though. Regardless of how small the minority of developers on the platform, it clearly matters to those developers which versions of Python they can use. -- \ ?When I get new information, I change my position. What, sir, | `\ do you do with new information?? ?John Maynard Keynes | _o__) | Ben Finney From ben+python at benfinney.id.au Wed Jan 27 16:50:12 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 28 Jan 2010 08:50:12 +1100 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <87zl3z2p63.fsf@benfinney.id.au> Christian Heimes writes: > John Nagle wrote: > > 1. Python 3 is supported by major Linux distributions. > > > > FALSE - most distros are shipping with Python 2.4, or 2.5 at best. > > You are wrong. Modern versions of Debian / Ubuntu are using Python > 2.6. Only if by ?modern? you mean ?not released yet?. The latest stable Debian (Debian 5.0, Lenny) has only Python 2.4 and Python 2.5. It does not have Python 2.6 at all, and until this month Python 2.6 was not even in the in-development suite of Debian. Fortunately, Debian Squeeze now finally has added Python 2.6 (though currently ?python? still uses Python 2.5). But Squeeze is currently a long way from being released. Python 3 is in the play-pen of Debian ?unstable?, but only arrived this week. It's even further from being released; it has to pass the filter from ?unstable? to ?testing? before it gets consideration for that. So I think your statement above is at least a mischaracterisation of the truth. -- \ ?As scarce as truth is, the supply has always been in excess of | `\ the demand.? ?Josh Billings | _o__) | Ben Finney From alfps at start.no Wed Jan 27 16:50:28 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 22:50:28 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: * Adam Tauno Williams: > On Wed, 2010-01-27 at 18:52 +0100, Alf P. Steinbach wrote: >> * Steve Holden: >>> Alf P. Steinbach wrote: >>> [...] >>>> The main problem with the incompatibility is for porting code, not for >>>> writing code from scratch. It's also a problem wrt. learning the >>>> language. And I see no good reason for it: print can't really do more, >>>> or less, or more conveniently (rather, one has to write a bit more now >>>> for same effect). >>> Of course it can do more: it allows you to layer your own print >>> functionality into the system much more easily than you could with the >>> print statement. >> Yeah, point. Very minor though. :-) > > So you get to determine that? I'm sorry, I don't find that question meaningful. > I'd call the whole print thing (a) a legitimate change to increase > consistency and (b) a fairly minor porting nuisance since application > code [as in big-chunks-o-code] almost never contains print statements. > I know at least two shops that have scripts they run on all Python code, > prior to it entering production, to ensure there are no print > statements. Considering that in your opinion "application code [as in big-chunks-o-code] almost never contains print statements", is the point about being able to replace print, as you see it, more than a minor point? Cheers & sorry for not grokking your question, - Alf From MLMDev at comcast.net Wed Jan 27 17:00:45 2010 From: MLMDev at comcast.net (Mitchell L Model) Date: Wed, 27 Jan 2010 17:00:45 -0500 Subject: I really need webbrowser.open('file://') to open a web browser In-Reply-To: References: <5A08CF3C-63E4-4103-B4B2-6C17E1D6B6D0@Comcast.net> Message-ID: <813FC2BA-3F3D-4C08-B6F4-F44837CBB275@comcast.net> On Jan 27, 2010, at 3:31 PM, Timur Tabi wrote: > On Wed, Jan 27, 2010 at 12:29 PM, Mitchell L Model > wrote: > >> I had some discussions with the Python documentation writers that >> led to the >> following note being included in the Python 3.1 library >> documentation for >> webbrowser.open: "Note that on some platforms, trying to open a >> filename >> using this function, may work and start the operating system?s >> associated >> program. However, this is neither supported nor portable." > > Then they should have renamed the API. I appreciate that they're > finally documenting this, but I still think it's a bunch of baloney. I agree, but I am pretty sure that, based on the discussions I had with the Python documenters and developers, that there's no hope of winning this argument. I suppose that since a file: URL is not, strictly speaking, on the web, that it shouldn't be opened with a "web" browser. It's just that the "web" part of "web browser" became more or less obsolete a long time ago since there are so many more ways of using browsers and so many more things they can do then just browse the web. So if you interpret the name "webbrowser" to mean that it browses the web, as opposed to files, which means going through some kind of server-based protocol, the module does what it says. But I still like the idea of using it to open files, especially when I want the file to be opened by its associated application and not a browser. > >> You can control which browser opens the URL by using webbrowser.get >> to >> obtain a controller for a particular browser, specified by its >> argument, >> then call the open method on the controller instead of the module. > > How can I know which controller (application) the system will use when > it opens an http URL? I depend on webbrowser.open('http') to choose > the best web browser on the installed system. Does webbrowser.get() > tell me which application that will be? webbrowser.get() with no arguments gives you the default kind of browser controller, just as if you had used webbrowser.open() directly. > >> For opening files reliability and the ability to pick a particular >> program >> (browser or otherwise) to open it with you might have to resort to >> invoking >> a command line via subprocess.Popen. > > But that only works if I know which application to open. Aha. You could use subprocess to specify the application from within your Python code, but not to indicate "the user's default browser", unless the platform has a command for that. On OS X, for instance, the command line: open file.html opens file.html with the application the user has associated with html files, whereas open -a safari file.html will open it with Safari even if the user has chosen Firefox for html files. There's stuff like this for Windows, I suppose, but hardly as convenient. And I think that Linux environments are all over the place on this, but I'm not sure. webbrowser.get() returns a control object of the default class for the user's environment -- the one that means "use the default browser" so it won't help. From awilliam at opengroupware.us Wed Jan 27 17:00:56 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Wed, 27 Jan 2010 17:00:56 -0500 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <1264629656.3103.10.camel@linux-m3mt> On Wed, 2010-01-27 at 16:25 -0500, Benjamin Kaplan wrote: > On Wed, Jan 27, 2010 at 3:56 PM, John Nagle wrote: > Give the package maintainers time to update. There were some pretty > big changes to the C API. Most of the major 3rd party packages like > numpy and MySQLdb have already commited to having a Python 3 version. > They just haven't gotten them out yet. PostgreSQL support is available for Python3. Just switch to using a real database and one of you problems is solved. :) From pavlovevidence at gmail.com Wed Jan 27 17:07:15 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Wed, 27 Jan 2010 14:07:15 -0800 (PST) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> On Jan 27, 12:56?pm, John Nagle wrote: > Arguably, Python 3 has been rejected by the market. No it's not fathomably arguable, because there's no reasonable way that Python 3 could have fully replaced Python 2 so quickly. At best, you could reasonably argue there hasn't been enough time to tell. >?Instead, there's > now Python 2.6, Python 2.7, and Python 2.8. It was always the plan to continue developing Python 2.x alongside Python 3.x during the transition period. Last I heard, don't remember where, the plan was for Python 2.7 to be the last version in the Python 2 line. If that's true, Python 3 acceptance is further along at this point than anticipated, since they originally thought they might have to go up to 2.9. >?Python 3 has turned into > a debacle like Perl 6, now 10 years old. Perl 6 has never been released. The situations aren't even comparable. > That's the reality, Python 3 fanboys. You're the fanboy, fanboi. You are so hellbent on badmouthing Python 3 that you throw three ridiculous, straw-grasping arguments at us. Here's the real reality. Python 3 is going to replace Python 2, and it has nothing to do with technical merit. The developers are planning to stop development on 2.x line, and only continue with 3.x, so anyone who wants to stay current--which is most people--with Python will have to use 3.x. There is no hope that developers will be pressured by the market to change their plans; we would have seen enough of a backlash by now. There is also no hope someone will fork Python 2.x and continue it in perpetuity. Well, someone might try to fork it, but they won't be able to call it Python. No, don't be silly, a fork of Python not called Python won't gain market share. So rail if it makes you feel better but you've already lost. Carl Banks From tjreedy at udel.edu Wed Jan 27 17:15:10 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 27 Jan 2010 17:15:10 -0500 Subject: Ad hoc lists vs ad hoc tuples In-Reply-To: References: <8a57c64f-0c67-4ec1-b4be-34d76c7e7571@q4g2000yqm.googlegroups.com> Message-ID: On 1/27/2010 12:32 PM, Antoine Pitrou wrote: > Le Wed, 27 Jan 2010 02:20:53 -0800, Floris Bruynooghe a ?crit : >> >> Is a list or tuple better or more efficient in these situations? > > Tuples are faster to allocate (they are allocated in one single step) and > quite a bit smaller too. > In some situations, in Python 2.7 and 3.1, they can also be ignored by > the garbage collector, yielding faster collections. > > (not to mention that they are hashable, which can be useful) Constant tuples (a tuple whose members are all seen as constants by the compiler) are now pre-compiled and constructed once and put into the code object as such rather than re-constructed with each run of the code. >>> from dis import dis >>> def l(): return [1,2,3] >>> def t(): return 1,2,3 >>> dis(l) 1 0 LOAD_CONST 1 (1) 3 LOAD_CONST 2 (2) 6 LOAD_CONST 3 (3) 9 BUILD_LIST 3 12 RETURN_VALUE >>> dis(t) 1 0 LOAD_CONST 4 ((1, 2, 3)) 3 RETURN_VALUE >>> # 3.1 Terry Jan Reedy From exarkun at twistedmatrix.com Wed Jan 27 17:19:00 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Wed, 27 Jan 2010 22:19:00 -0000 Subject: myths about python 3 In-Reply-To: <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> Message-ID: <20100127221900.1898.256361359.divmod.xquotient.848@localhost.localdomain> On 10:07 pm, pavlovevidence at gmail.com wrote: >On Jan 27, 12:56?pm, John Nagle wrote: >>Arguably, Python 3 has been rejected by the market. > >No it's not fathomably arguable, because there's no reasonable way >that Python 3 could have fully replaced Python 2 so quickly. > >At best, you could reasonably argue there hasn't been enough time to >tell. >>?Instead, there's >>now Python 2.6, Python 2.7, and Python 2.8. > >It was always the plan to continue developing Python 2.x alongside >Python 3.x during the transition period. > >Last I heard, don't remember where, the plan was for Python 2.7 to be >the last version in the Python 2 line. If that's true, Python 3 >acceptance is further along at this point than anticipated, since they >originally thought they might have to go up to 2.9. This assumes that the decision to stop making new 2.x releases is based on Python 3 adoption, rather than on something else. As far as I can tell, it's based on the personal desire of many of the core developers to stop bothering with 2.x. In other words, it's more a gauge of adoption of Python 3 amongst Python core developers. Jean-Paul From paul at boddie.org.uk Wed Jan 27 17:19:32 2010 From: paul at boddie.org.uk (Paul Boddie) Date: Wed, 27 Jan 2010 14:19:32 -0800 (PST) Subject: I really need webbrowser.open('file://') to open a web browser References: <5A08CF3C-63E4-4103-B4B2-6C17E1D6B6D0@Comcast.net> Message-ID: <38cbe4b0-3a23-4c68-a2ef-d7d7acec3270@c29g2000yqd.googlegroups.com> On 27 Jan, 23:00, Mitchell L Model wrote: > > I suppose that since a file: URL is not, strictly speaking, on the ? > web, that it shouldn't be opened with a "web" browser. But anything with a URL is (or should be regarded as being) on the Web. It may not be anything more than a local resource and thus have no universal or common meaning - someone else may not be able to resolve the URL to a file at all, or it may resolve to a different file - but it's part of the Web as observed by one party. Paul From missive at hotmail.com Wed Jan 27 17:24:01 2010 From: missive at hotmail.com (Lee Harr) Date: Thu, 28 Jan 2010 02:54:01 +0430 Subject: some turtle questions Message-ID: > I am trying to think of things to do with the turtle module > 1) is there a way to determine the current screen pixel color? This would not use the included turtle module, but you could use the turtle module from the pygsear collection: http://www.nongnu.org/pygsear/ It requires pygame, but if you have that installed already it is pretty easy to get it working. Code to get a drawn color would look like: >>> from pygsear.Drawable import Turtle >>> t=Turtle() >>> t.bg.get_at((0,0)) (0, 0, 0, 255) >>> t.forward(100) >>> t.position [400.0, 200.0] >>> t.bg.get_at((400,300)) (255, 255, 255, 255) >>> t.left(90) >>> t.set_color((255,0,0)) >>> t.forward(100) >>> t.position [300.0, 200.0] >>> t.bg.get_at((350,200)) (255, 0, 0, 255) >>> > 2) is there a way to put a limit on the extent the turtle can > travel? it seems I can keep moving off of the screen. You could write a subclass for interactive use. Maybe like this: from pygsear.Drawable import Turtle class T2(Turtle): ??? def forward(self, dist): ??????? Turtle.forward(self, dist) ??????? if not self.onscreen(slack=10): ??????????? self.backward(dist) ... but I'm not sure that's the best approach. Often when doing something interesting the turtle will go offscreen for a bit and come back later to finish up the visible portion of his drawing. _________________________________________________________________ Hotmail: Free, trusted and rich email service. https://signup.live.com/signup.aspx?id=60969 From mensanator at aol.com Wed Jan 27 17:24:38 2010 From: mensanator at aol.com (Mensanator) Date: Wed, 27 Jan 2010 14:24:38 -0800 (PST) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On Jan 27, 2:56?pm, John Nagle wrote: > Daniel Fetchinson wrote: > > Hi folks, > > > I was going to write this post for a while because all sorts of myths > > periodically come up on this list about python 3. I don't think the > > posters mean to spread false information on purpose, they simply are > > not aware of the facts. > > > My list is surely incomplete, please feel free to post your favorite > > misconception about python 3 that people periodically state, claim or > > ask about. > > Myths about Python 3: > > 1. ?Python 3 is supported by major Linux distributions. > > ? ? ? ? FALSE - most distros are shipping with Python 2.4, or 2.5 at best. So? I use Mac OSX 10.6, not Linux. And that comes with 2.6. Nothing stopped me from adding 3.1. > > 2. ?Python 3 is supported by multiple Python implementations. > > ? ? ? ? FALSE - Only CPython supports 3.x. ?Iron Python, Unladen Swallow, > ? ? ? ? PyPy, and Jython have all stayed with 2.x versions of Python. So? I only use CPython. > > 3. ?Python 3 is supported by most 3rd party Python packages. > > ? ? ? ? FALSE - it's not supported by MySQLdb, OpenSSL, feedparser, etc. So? The only 3rd party module I use is gmpy, and that's been updated to 3.x. > > Arguably, Python 3 has been rejected by the market. ?Instead, there's > now Python 2.6, Python 2.7, and Python 2.8. ?Python 3 has turned into > a debacle like Perl 6, now 10 years old. > > That's the reality, Python 3 fanboys. Maybe in *your* world. I'm perfectly happy in my world using 3.1. > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? John Nagle From tjreedy at udel.edu Wed Jan 27 17:34:23 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 27 Jan 2010 17:34:23 -0500 Subject: myths about python 3 In-Reply-To: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On 1/27/2010 3:56 PM, John Nagle wrote: > 2. Python 3 is supported by multiple Python implementations. > > FALSE - Only CPython supports 3.x. Iron Python, Unladen Swallow, > PyPy, and Jython have all stayed with 2.x versions of Python. Actually, Unladen Swallow is now targeted at 3.1; its developers have conservatively proposed its integration in CPython 3.3. I would not be completely shocked if it happens in 3.2. > Arguably, Python 3 has been rejected by the market. Almost everything is 'arguable'. Based on experience, Guido never expected major uptake until 3.2 (a year away). > Instead, there's now Python 2.6, Just who produced that? and why? > Python 2.7, Does not exist yet, but again, coming from the same devs for the purpose of helping transition to 3.x. > and Python 2.8. Unlikely to ever exist. > Python 3 has turned into a debacle like Perl 6, now 10 years old. You have to wait another 9 years to really say that. However, my impression is that Python 3 alreays surpasses Perl 6. > That's the reality, Python 3 fanboys. Why are you such a Python 3 hateboy? Terry Jan Reedy From tjreedy at udel.edu Wed Jan 27 17:38:31 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 27 Jan 2010 17:38:31 -0500 Subject: Library support for Python 3.x In-Reply-To: <7xk4v3mktr.fsf@ruckus.brouhaha.com> References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> Message-ID: On 1/27/2010 2:03 PM, Paul Rubin wrote: > aahz at pythoncraft.com (Aahz) writes: >> From my POV, your question would be precisely identical if you had >> started your project when Python 2.3 was just released and wanted to >> know if the libraries you selected would be available for Python 2.6. > > I didn't realize 2.6 broke libraries that had worked in 2.3, at least on > any scale. Did I miss something? For a windows user who depends on pre-built binaries, every new release breaks *every* library that is not pure Python and needs to be compiled. From dmalcolm at redhat.com Wed Jan 27 17:58:54 2010 From: dmalcolm at redhat.com (David Malcolm) Date: Wed, 27 Jan 2010 17:58:54 -0500 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <1264633134.5245.198.camel@brick> On Wed, 2010-01-27 at 16:25 -0500, Benjamin Kaplan wrote: > On Wed, Jan 27, 2010 at 3:56 PM, John Nagle wrote: > > Daniel Fetchinson wrote: > >> > >> Hi folks, > >> > >> I was going to write this post for a while because all sorts of myths > >> periodically come up on this list about python 3. I don't think the > >> posters mean to spread false information on purpose, they simply are > >> not aware of the facts. > >> > >> My list is surely incomplete, please feel free to post your favorite > >> misconception about python 3 that people periodically state, claim or > >> ask about. > > > > Myths about Python 3: > > > > 1. Python 3 is supported by major Linux distributions. > > > > FALSE - most distros are shipping with Python 2.4, or 2.5 at best. > > > The latest versions of Ubuntu Jaunty and Karmic, Fedora 11 and 12, and > OpenSUSE 11.2 all use Python 2.6. Ubuntu has been shipping python 3 > since Jaunty came out last April. According to Fedora's package index, > Python 3 is in the devel version which probably means it will be in > upcoming versions of Fedora as well. FWIW, more information on Fedora python 3 status here: https://fedoraproject.org/wiki/Features/Python3F13 [snip] > Give the package maintainers time to update. There were some pretty > big changes to the C API. Most of the major 3rd party packages like > numpy and MySQLdb have already commited to having a Python 3 version. > They just haven't gotten them out yet. I'll take this opportunity to make a shameless plug for my 2to3c tool: http://dmalcolm.livejournal.com/3935.html which takes some of the grunt work out of taking C code and making it compilable against both 2 and 3. (it will still require a human to do some of the work, alas). Hope this is helpful Dave From john at castleamber.com Wed Jan 27 18:03:12 2010 From: john at castleamber.com (John Bokma) Date: Wed, 27 Jan 2010 17:03:12 -0600 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> <878wbk1c4q.fsf@castleamber.com> <0c142d80-5f54-4b63-a8ce-f826ec9ad82d@r19g2000yqb.googlegroups.com> Message-ID: <87zl3z40cv.fsf@castleamber.com> Xah Lee writes: > Someone is badmouthing me, and it has been doing so over the years. I > feel obliged to post a off topic relpy. See: > > ? DreamHost.com and A Incidence of Harassment > http://xahlee.org/Periodic_dosage_dir/t2/harassment.html Dreamhost is not the only ISP that has agreed with me in the past regarding your spamming. And yes, it's spamming if you copy paste an article from your web site with the link to that site and cross post it to as many groups as your Usenet provider allows for. Thank goodness that GG has a limit on how many groups you can spam at a time. Post to on-topic groups only, really on-topic groups, not as many as you can select. And participate like you know do, and nobody will complain about /how/ you post. And if you cross-post /set a follow-up-to/. That's netiquette. -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From jgardner at jonathangardner.net Wed Jan 27 18:29:05 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Wed, 27 Jan 2010 15:29:05 -0800 (PST) Subject: Python and Ruby References: Message-ID: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> On Jan 27, 5:47?am, Simon Brunning wrote: > > I think Python is a little cleaner, but I'm sure you'd find Ruby fans > who'd argue the complete opposite. > Are you sure about that? There's a lot of line noise in Ruby. How are you supposed to pronounce "@@"? What about "{|..| ... }"? There's a lot of "magic" in Ruby as well. For instance, function calls are made without parentheses. Blocks can only appear as the first argument. There's a lot more, if you put your mind to it. Indentation is also optional in Ruby. You can quickly fool a newbie by not indenting your code properly, which is impossible in Python. Python is much, much cleaner. I don't know how anyone can honestly say Ruby is cleaner than Python. From jgardner at jonathangardner.net Wed Jan 27 18:31:18 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Wed, 27 Jan 2010 15:31:18 -0800 (PST) Subject: Python and Ruby References: Message-ID: <43cad4e7-6b54-4bd9-9ca2-217cb109560f@g29g2000yqe.googlegroups.com> On Jan 27, 6:56?am, Roald de Vries wrote: > On Jan 27, 2010, at 2:01 PM, Jean Guillaume Pyraksos wrote: > > > What are the arguments for choosing Python against Ruby > > for introductory programming? > > I think the main difference is in culture, especially for ? > *introductory* programming. To add to that, Python is the type of language where experienced programmers can pick it up by reading code, and newbies won't get hopelessly lost. I've taught less-than-formal introductory programming classes to people who are new to programming. Python gets out of the way, and allows you to focus on the programming concepts, such as variable, functions, parameters, classes, and algorithms. From fetchinson at googlemail.com Wed Jan 27 18:38:14 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Thu, 28 Jan 2010 00:38:14 +0100 Subject: myths about python 3 In-Reply-To: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: >> Hi folks, >> >> I was going to write this post for a while because all sorts of myths >> periodically come up on this list about python 3. I don't think the >> posters mean to spread false information on purpose, they simply are >> not aware of the facts. >> >> My list is surely incomplete, please feel free to post your favorite >> misconception about python 3 that people periodically state, claim or >> ask about. > > Myths about Python 3: I said myths, not facts! :) s/Myths/Facts/ > 1. Python 3 is supported by major Linux distributions. > > FALSE - most distros are shipping with Python 2.4, or 2.5 at best. This latter statement is false, Fedora 11 and 12 come with python 2.6. > 2. Python 3 is supported by multiple Python implementations. > > FALSE - Only CPython supports 3.x. Iron Python, Unladen Swallow, > PyPy, and Jython have all stayed with 2.x versions of Python. This latter statement is false, unladen swallow is currently targeting 3.x and may even be merged into cpython 3.x. > 3. Python 3 is supported by most 3rd party Python packages. > > FALSE - it's not supported by MySQLdb, OpenSSL, feedparser, etc. s/most/my favorite/ Actually, tons of 3rd party packages are already ported, postgres and django are just 2 examples, the next release of PIL will be another. > Arguably, Python 3 has been rejected by the market. Instead, there's > now Python 2.6, Python 2.7, and Python 2.8. Python 3 has turned into > a debacle like Perl 6, now 10 years old. These are the kinds of myths I had in mind when starting the thread. All sorts of BS is kept circulating without any facts to back up the claims. Actually, in this thread tons of rebuttals can be found to your statements but I doubt you will change your mind :) Hopefully others reading all of this will at least see what is BS and what is the actual situation about python 3. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From falk at mauve.rahul.net Wed Jan 27 18:50:19 2010 From: falk at mauve.rahul.net (Edward A. Falk) Date: Wed, 27 Jan 2010 23:50:19 +0000 (UTC) Subject: myths about python 3 References: Message-ID: In article , Daniel Fetchinson wrote: >Hi folks, > >1. Print statement/function creates incompatibility between 2.x and 3.x! > >Certainly false or misleading, if one uses 2.6 and 3.x the >incompatibility is not there. Print as a function works in 2.6: Yes, but does print as a statement work? Your argument is that python 2.x code can be ported to 3.x and still run under 2.6 if you're careful about how you do the port. That's not the same as saying they're compatible. >2. Integer division creates incompatibility between 2.x and 3.x! Same as above. Saying you can make it work by rewriting your code is not the same as saying that it works. If they'd wanted wide adoption of python 3, they should have made it as compatible as possible with python 2 code. If they dropped the print statement, then they did not do so. -- -Ed Falk, falk at despams.r.us.com http://thespamdiaries.blogspot.com/ From jgardner at jonathangardner.net Wed Jan 27 18:51:15 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Wed, 27 Jan 2010 15:51:15 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> Message-ID: <642d67e6-781b-4157-b917-449139854bad@b9g2000yqd.googlegroups.com> On Jan 27, 9:38?am, Luis M. Gonz?lez wrote: > > Please don't post more noise and ad hominem attacks to the group, Steve. > > "Ad hominem"? > Please, operor non utor lingua non notus per vulgaris populus. > Gratias ago vos... My rough, machine-assisted translation: "Don't try to use language that the people don't know. Thanks." From falk at mauve.rahul.net Wed Jan 27 18:52:07 2010 From: falk at mauve.rahul.net (Edward A. Falk) Date: Wed, 27 Jan 2010 23:52:07 +0000 (UTC) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: In article , Grant Edwards wrote: > >That said, I don't expect to start using Python 3 until library >availability or my Linux distro forces me to. If python 3 is much more efficient than python 2, or it has features I really need for some application I'll write in the future, I might be tempted to switch. Maybe some future version of python 3 will be compatible with python 2 source code. That would help. -- -Ed Falk, falk at despams.r.us.com http://thespamdiaries.blogspot.com/ From jgardner at jonathangardner.net Wed Jan 27 18:54:23 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Wed, 27 Jan 2010 15:54:23 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: On Jan 26, 10:12?pm, Steven D'Aprano wrote: > > I did too, when I first heard cmp was to be dumped. But I changed my mind > and now agree with the decision to drop cmp. Custom sorts are nearly > always much better written with key rather than cmp: key adds an O(N) > overheard to the sorting, while cmp makes sorting O(N**2). In other > words, key is *much* faster, and generally easier to write as well. > Imagine what sort() could do if all the keys were platform-native types. In other words, if you could bypass the Python compare because all the keys were native int, char, char*, or float. You can't do that with cmp either. From no.email at nospam.invalid Wed Jan 27 18:54:55 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 27 Jan 2010 15:54:55 -0800 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: <7xvdenje7k.fsf@ruckus.brouhaha.com> Steven D'Aprano writes: > always much better written with key rather than cmp: key adds an O(N) > overheard to the sorting, while cmp makes sorting O(N**2). Whaaaaaaaaaa ...... ???? No I don't think so. From cookiecaper at gmail.com Wed Jan 27 19:00:57 2010 From: cookiecaper at gmail.com (cookiecaper) Date: Wed, 27 Jan 2010 16:00:57 -0800 (PST) Subject: Updating form action attribute with mechanize.Browser Message-ID: <662d367b-267e-4c9a-b53a-201b64a036c4@r19g2000yqb.googlegroups.com> Hi all. I am using mechanize.Browser to download web pages and such. One of these has the wrong URL in the action attribute of the form I need to submit. Anyone know how I can update this quickly? ClientForm.__init__ seems to take the action attribute but that is passed by b.select_form(), I think. From jgardner at jonathangardner.net Wed Jan 27 19:12:04 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Wed, 27 Jan 2010 16:12:04 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> Message-ID: On Jan 27, 12:36?am, Paul Rubin wrote: > Steven D'Aprano writes: > > Without becoming a purely functional language, you won't get rid of all > > statements. > > Why not? ?GCC lets you use any statement in an expression: > > ? ? #include > > ? ? main() > ? ? { > ? ? ? int i, x, p=0; > ? ? ? x = ({ for (i=1; i<=10; i++) p += i; p;}); > ? ? ? printf ("x=%d\n", x); > ? ? } > > and C is certainly not a purely functional language. > > > for, while, if, try, break, yield, return > > are all used for flow control, and should remain as statements. > > What about assert, import, and pass? I think you missed the point about what a functional language is and is not. Let me see if I can help clear up that confusion a bit. In a functional language, you don't use statements. For instance, when you declare a new function, you call the function-declaring-function with parameters describing the parameters and body of that function. In such a language, flow control is very, very strange for those not familiar with it. For instance, how can you call an "if" function that only evaluates one or the other branches? In such a case, you're not making a function call, at least not as you know it. What you're doing is calling a special kind of function that doesn't evaluate its parameters until later, if at all. With the above paradigm, you have to start thinking about the programming task in a completely different light. Rather than organizing your code into a sequence of evaluated statements, you organize it into one giant function that will evaluate its parameters and return some result. This is a difficult paradigm for programmers to grasp, and is one reason why Lisp and other functional languages aren't know for being newbie-friendly. Whether or not a statement can be used as an expression is really orthogonal to this conversation. If "yield", "break", and "continue" were functions, then you could "call" them from within another function and have the call-ee yield, break, or continue. For instance: def do_something(): break() while 1: do_something() # The next line is never reached do_something_else() This could be useful for some situations, but it is really, really confusing and so it's not allowed. Such a function would have to rely on some type of magic so that it is correctly identified as a yield, break, or continue, and not a typical function call. "import" is another case. It does something very special. It assigns to values in the namespace of the code from which it was called. No function can do that, at least not without some weird magic. It's better to leave it as a statement and teach new users that it is very, very special, than to redefine how functions work with the call-ees namespace, or make it a common task to muck with such things. If "while", "for", "try", "if", "def", and "class" were functions, they would have to be written like so: if(expr, if-true-expr, if-false-expr) Note, however, that you can't execute if-true of if-false until AFTER the if() function is called. That means either we re-define what it means to call a function, or we have some notation that means "Don't execute this yet". Either way, it's really, really confusing. Also, note that statements allow you to combine many statements into one. Python's rule is that you can't pass statements to a function, only expression, so you'd have to re-define how that works to allow some function that would combine multiple statements together. In the end, print is a great candidate for a function, while the others are most definitely not. Adding any of them as functions would mean other, fundamental aspects of Python would have to be re-defined as well. I hope this clears up Steve's point about functional languages. From ethan at stoneleaf.us Wed Jan 27 19:14:53 2010 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 27 Jan 2010 16:14:53 -0800 Subject: myths about python 3 In-Reply-To: References: <6faf39c91001270245q155a932aj3e0bb98d377ddad1@mail.gmail.com> Message-ID: <4B60D6FD.4070708@stoneleaf.us> Daniel Fetchinson wrote: >>> Hi folks, >>> >>> I was going to write this post for a while because all sorts of myths >>> periodically come up on this list about python 3. I don't think the >>> posters mean to spread false information on purpose, they simply are >>> not aware of the facts. >>> >>> My list is surely incomplete, please feel free to post your favorite >>> misconception about python 3 that people periodically state, claim or >>> ask about. >>> >>> 1. Print statement/function creates incompatibility between 2.x and 3.x! >>> >>> Certainly false or misleading, if one uses 2.6 and 3.x the >>> incompatibility is not there. Print as a function works in 2.6: >>> >>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>>>>> print( 'hello' ) >>> hello >>>>>> print 'hello' >>> hello >>> >>> 2. Integer division creates incompatibility between 2.x and 3.x! >>> >>> Again false or misleading, because one can get the 3.x behavior with 2.6: >>> >>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>>>>> 6/5 >>> 1 >>>>>> from __future__ import division >>>>>> 6/5 >>> 1.2 >>> >>> >>> Please feel free to post your favorite false or misleading claim about >>> python 3! >> Well, I see two false or misleading claims just above - namely that >> the two claims above are false or misleading. They tell just half of >> the story, and that half is indeed easy. A Python 3 program can be >> unchanged (in the case of print) or with only trivial modifications >> (in the case of integer division) be made to run on Python 2.6. > > Okay, so we agree that as long as print and integer division is > concerned, a program can easily be written that runs on both 2.6 and > 3.x. > > My statements are exactly this, so I don't understand why you disagree. > >> The other way around this is _not_ the case. > > What do you mean? > >> To say that two things are >> compatible if one can be used for the other, but the other not for the >> first, is false or misleading. > > I'm not sure what you mean here. Maybe I didn't make myself clear > enough, but what I mean is this: as long as print and integer division > is concerned, it is trivial to write code that runs on both 2.6 and > 3.x. Hence if someone wants to highlight incompatibility (which surely > exists) between 2.6 and 3.x he/she has to look elsewhere. > > Cheers, > Daniel > I think what Andre is saying is that you can't get 2.x behavior in 3.x, only the other way 'round. In the integer division instance, the 2.x behavior of 6/5 = 1 is not going to happen in 3.x. ~Ethan~ From jgardner at jonathangardner.net Wed Jan 27 19:16:59 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Wed, 27 Jan 2010 16:16:59 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xvdenje7k.fsf@ruckus.brouhaha.com> Message-ID: <5b9e69bc-b261-4e81-84bc-045936011e82@o9g2000yqa.googlegroups.com> On Jan 27, 3:54?pm, Paul Rubin wrote: > Steven D'Aprano writes: > > always much better written with key rather than cmp: key adds an O(N) > > overheard to the sorting, while cmp makes sorting O(N**2). > > Whaaaaaaaaaa ...... ???? ?No I don't think so. You're referring to the O(N**2) bit, right? I am sure he knew it was O (N*log(N)), which is still worse than O(N) for key. If he didn't, well, Python has some fundamental flaws in its basic sort algorithm. From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 19:19:24 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 28 Jan 2010 00:19:24 GMT Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On Wed, 27 Jan 2010 12:56:10 -0800, John Nagle wrote: > Daniel Fetchinson wrote: >> Hi folks, >> >> I was going to write this post for a while because all sorts of myths >> periodically come up on this list about python 3. I don't think the >> posters mean to spread false information on purpose, they simply are >> not aware of the facts. >> >> My list is surely incomplete, please feel free to post your favorite >> misconception about python 3 that people periodically state, claim or >> ask about. > > Myths about Python 3: > > 1. Python 3 is supported by major Linux distributions. > 2. Python 3 is supported by multiple Python implementations. > 3. Python 3 is supported by most 3rd party Python packages. A myth actually needs to be believed by some sector of the population, even if a small one. (Isolated nut cases don't count.) Star Wars is not a myth, because nobody -- not even those wacky people who put down "Jedi" as their religion on census forms -- actually believes it is a documentary. I've never heard anyone claiming any of those three "myths". I conclude that you just made them up, that they are fictional claims rather than genuine myths mistakenly believed by some people. So, in that spirit, here are three more for your collection. Perhaps you could start a list on a website somewhere. 4. Python 3 will make you irresistible to women. FALSE - Python 3 coders are no more likely to get a date than any other programmer. 5. Python 3 programs cannot be buggy, no matter how poorly you code. FALSE - programs written in Python 3 can contain bugs. 6. The code for Python 3 was handed down to Guido from the Heavens, carved into stone tablets by the Gods themselves. FALSE - Python 3 was designed and written by fallible human beings, and consequently there is no guarantee that it will be perfect in all ways for all purposes. > That's the reality, Python 3 fanboys. Speaks for itself. -- Steven From no.email at nospam.invalid Wed Jan 27 19:25:20 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 27 Jan 2010 16:25:20 -0800 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> Message-ID: <7xockfjcsv.fsf@ruckus.brouhaha.com> Jonathan Gardner writes: >> What about assert, import, and pass? >... > For instance, how can you call an "if" function ... > If "yield", "break", and "continue" were functions, ... > "import" ... does something very special. It assigns > to values in the namespace of the code from which it was called. > In the end, print is a great candidate for a function, Reasonable point about import. "If", "yield", "break", and "continue" are irrelevant if one accepts the flow control argument, although there is already expression syntax for "if" and "yield". What about assert and pass? > Also, note that statements allow you to combine many statements into > one. Python's rule is that you can't pass statements to a function, But if we're going around introducing changes that break things, why not get rid of that silly restriction? I don't mind that 3.x is breaking stuff for the sake of improving things. That's the whole idea of 3.x, after all. What bugs me is that the improvements are mostly quite superficial, and the breakage seems often gratuitous. I'd rather see much more drastic changes and correspondingly much larger improvements. Right now I rent an apartment in a city where real estate values have dropped a couple percent in the past few years. So in principle, I could move to a 2% bigger apartment while paying the same rent, or move to a similarly sized apartment while paying 2% less. Either of those is objectively an improvement, but am I going to do it? Hell no--moving even down the same street is way too big a hassle to even consider doing it for such a tiny gain. But if the "2%" were replaced by 3x or 5x, the proposition would be a heck of a lot more attractive, even if I had to move a larger distance. That's sort of the way I feel about switching from Python 2.x to 3.x. From no.email at nospam.invalid Wed Jan 27 19:28:08 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 27 Jan 2010 16:28:08 -0800 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <7xk4v3jco7.fsf@ruckus.brouhaha.com> Steven D'Aprano writes: > 6. The code for Python 3 was handed down to Guido from the Heavens, > carved into stone tablets by the Gods themselves. That is heresy. The direction was up, not down. From briandenzer at gmail.com Wed Jan 27 19:28:35 2010 From: briandenzer at gmail.com (Brian D) Date: Wed, 27 Jan 2010 16:28:35 -0800 (PST) Subject: Stuck on a three word street name regex Message-ID: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> I've tackled this kind of problem before by looping through a patterns dictionary, but there must be a smarter approach. Two addresses. Note that the first has incorrectly transposed the direction and street name. The second has an extra space in it before the street type. Clearly done by someone who didn't know how to concatenate properly -- or didn't care. 1000 RAMPART S ST 100 JOHN CHURCHILL CHASE ST I want to parse the elements into an array of values that can be inserted into new database fields. Anyone who loves solving these kinds of puzzles care to relieve my frazzled brain? The pattern I'm using doesn't keep the "CHASE" with the "JOHN CHURCHILL": >>> p = re.compile(r'(?P\d+)\s(?P[A-Z\s]*)\s(?P\w*)\s(?P\w{2})$') >>> s = '1405 RAMPART S ST' >>> m = re.search(p, s) >>> m <_sre.SRE_Match object at 0x011A4440> >>> print m.groups() ('1405', 'RAMPART', 'S', 'ST') >>> s = '45 JOHN CHURCHILL CHASE ST' >>> m = re.search(p, s) >>> m <_sre.SRE_Match object at 0x011A43E8> >>> print m.groups() ('45', 'JOHN CHURCHILL', 'CHASE', 'ST') From no.email at nospam.invalid Wed Jan 27 19:30:40 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 27 Jan 2010 16:30:40 -0800 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xvdenje7k.fsf@ruckus.brouhaha.com> <5b9e69bc-b261-4e81-84bc-045936011e82@o9g2000yqa.googlegroups.com> Message-ID: <7xfx5rjcjz.fsf@ruckus.brouhaha.com> Jonathan Gardner writes: > You're referring to the O(N**2) bit, right? I am sure he knew it was O > (N*log(N)), which is still worse than O(N) for key. It's O(n log n) for both key and cmp. key is usually more convenient and usually gives a constant-factor speedup (DSU pattern), but it uses more memory and there are some types of orderings that it can't handle without a bunch of contortion. From pavlovevidence at gmail.com Wed Jan 27 19:32:39 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Wed, 27 Jan 2010 16:32:39 -0800 (PST) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> Message-ID: On Jan 27, 2:19?pm, exar... at twistedmatrix.com wrote: > On 10:07 pm, pavlovevide... at gmail.com wrote: > >Last I heard, don't remember where, the plan was for Python 2.7 to be > >the last version in the Python 2 line. ?If that's true, Python 3 > >acceptance is further along at this point than anticipated, since they > >originally thought they might have to go up to 2.9. > > This assumes that the decision to stop making new 2.x releases is based > on Python 3 adoption, rather than on something else. I should have said, "If anything...". Carl Banks From no.email at nospam.invalid Wed Jan 27 19:35:27 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 27 Jan 2010 16:35:27 -0800 Subject: Stuck on a three word street name regex References: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> Message-ID: <7xbpgfjcc0.fsf@ruckus.brouhaha.com> Brian D writes: > I've tackled this kind of problem before by looping through a patterns > dictionary, but there must be a smarter approach.> > Two addresses. Note that the first has incorrectly transposed the > direction and street name. .... If you're really serious about it (e.g. you are the post office trying to program automatic mail sorting machines) there is no simple regex trick anything like what you want. A lot of addresses will be ambiguous. You have use all the info you have about your entire address corpus (e.g. you need a complete street directory of the whole US) and do a bunch of Bayesian inference. As a very simple example, for an address like "1000 RAMPART S ST" you'd use the zip code to identify the address's geographic neighborhood, and then use your street directory to find candidate correct addresses within that zip code. The USPS does an amazing job of delivering mail to completely mangled addresses based on methods like that. From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 19:35:42 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 28 Jan 2010 00:35:42 GMT Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On Wed, 27 Jan 2010 16:25:46 -0500, Benjamin Kaplan wrote: > When Python 2.6 came out, Jython was still on 2.2. The difference > between 2.2 and 2.6 is almost as big of a difference as between 2.6 and > 3.0. In that time, you had the introduction of the boolean type, > generators, list comprehensions, the addition of the "yield" and "with" > keywords, universal newline support, and decorators in addition to the > large number of changes to the standard library such as the introduction > of the subprocess module. THANK YOU Benjamin for injecting this note of sanity into the discussion. I believe that, with the possible exception of the change from byte strings to unicode strings, virtually *all* the hoo-har over Python 3 is simply due to the tactical mistake of Guido and the Python Dev team of *calling* Python 3 a backward incompatible release. Python has had previous major changes in the past (e.g. 1.5 to 2.0 and 2.1 to 2.2) and hardly anyone made a complaint. Certainly the move from 2.x to 3.x is a big move. If you have to support both series simultaneously, I don't envy your job, but if CherryPy can do it, so can others. But it's not qualitatively different from supporting (say) 2.4 through 2.6. Targeting multiple versions is always a PITA. I also find it telling that perhaps the biggest change of all, the one from byte strings to unicode, hardly rates a mention from the skeptics and haters. Instead we get rants about how division behaves differently (forgetting that "from __future__ import division" has worked since at least 2.4 and possibly older) and complaints that print is different. -- Steven From pavlovevidence at gmail.com Wed Jan 27 19:44:18 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Wed, 27 Jan 2010 16:44:18 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xvdenje7k.fsf@ruckus.brouhaha.com> <5b9e69bc-b261-4e81-84bc-045936011e82@o9g2000yqa.googlegroups.com> Message-ID: On Jan 27, 4:16?pm, Jonathan Gardner wrote: > On Jan 27, 3:54?pm, Paul Rubin wrote: > > > Steven D'Aprano writes: > > > always much better written with key rather than cmp: key adds an O(N) > > > overheard to the sorting, while cmp makes sorting O(N**2). > > > Whaaaaaaaaaa ...... ???? ?No I don't think so. > > You're referring to the O(N**2) bit, right? I am sure he knew it was O > (N*log(N)), which is still worse than O(N) for key. > > If he didn't, well, Python has some fundamental flaws in its basic > sort algorithm. Quicksort is O(N**2) worst case, perhaps that's what he meant. I'm not sure about timsort but since it's based on quicksort I'd guess it is O(N**2) or worse, worst case, as well. Typical case of a sort is still O(N*log(N)) whether using key or cmp. People recommended against cmp not because of overall algorithmic time considerations, but because of the number of function calls. cmp function was called a minimum of N-1 times and typically around N*log2 (N) times, whereas key was called exactly N times. Since the overhead of a Python function call was very large compared to the built-in cmp operation, it usually outweighed the algorithmic time considerations. Carl Banks From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 19:47:41 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 28 Jan 2010 00:47:41 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: On Wed, 27 Jan 2010 18:29:25 +0100, Alf P. Steinbach wrote: > The main problem with the incompatibility is for porting code, not for > writing code from scratch. Correct. It's a trivial problem, but still a problem. > It's also a problem wrt. learning the language. This makes no sense. Why is it harder to learn print(x, y, z) than this? print x, y, z The first case is like the other functions you have to learn, like len(). In fact, many newbies to Python put unneeded parentheses around arguments to print simply because they assume it is a function. I would argue that the new print function is *simpler* to learn. It is more consistent with other built-ins, and has fewer magic idioms to learn. Instead of: print >>fileObj, x, y, z you use regular function syntax with a meaningful keyword: print(x, y, z, file=fileObj) If you want suppress the newline at the end of each print: print x, y, z, # note the final comma compared to: print(x, y, z, end='') If you want to change the space between elements, instead of: sys.stdout.write(str(x) + "*" + str(y) + "*" + str(z) + '\n') you use: print(x, y, z, sep='*') If you want to override the behaviour of print in a module, instead of having to edit the source code of the module (which might not even be available), all you need to do is monkey-patch it: import module module.print = myprint -- Steven From drsalists at gmail.com Wed Jan 27 20:02:33 2010 From: drsalists at gmail.com (Dan Stromberg) Date: Wed, 27 Jan 2010 17:02:33 -0800 Subject: Orange County, California Python User Group Message-ID: <5def086a1001271702t2d371eeficb0d2cda7e830d4c@mail.gmail.com> http://www.meetup.com/Orange-County-CA-Python-User-Group/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From drsalists at gmail.com Wed Jan 27 20:03:39 2010 From: drsalists at gmail.com (Dan Stromberg) Date: Wed, 27 Jan 2010 17:03:39 -0800 Subject: Orange County, California Python User Group In-Reply-To: <5def086a1001271702t2d371eeficb0d2cda7e830d4c@mail.gmail.com> References: <5def086a1001271702t2d371eeficb0d2cda7e830d4c@mail.gmail.com> Message-ID: <5def086a1001271703v3ea5f932ra0d4b986df4ea33b@mail.gmail.com> On Wed, Jan 27, 2010 at 5:02 PM, Dan Stromberg wrote: > http://www.meetup.com/Orange-County-CA-Python-User-Group/ > Sigh. 'little quick on the trigger there. Anyway, as you might have inferred, I'm attempting to start a Python User Group for Orange County California. If you're interested, please visit the URL above. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 20:09:58 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 28 Jan 2010 01:09:58 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> Message-ID: On Wed, 27 Jan 2010 00:36:52 -0800, Paul Rubin wrote: > Steven D'Aprano writes: >> Without becoming a purely functional language, you won't get rid of all >> statements. > > Why not? GCC lets you use any statement in an expression: I stand corrected. Whether it is a good idea or not is another story. >> for, while, if, try, break, yield, return are all used for flow >> control, and should remain as statements. > > What about assert, import, and pass? Both assert and pass are special and need to be treated specially by the compiler. Arguably we could replace pass with None, but pass makes the intent more obvious. Although assert could be a function, it is treated specially by the compiler and so making it a statement emphases that it is special. The regular form of import could easily be a function returning the module object. After all, that's exactly what __import__ is! But using __import__ for anything but the simplest cases is quite awkward, hence we have syntactic sugar in the form of the import statement. You also missed del as another statement-based command. All the current statements have good strong reasons for being statements. print on the other hand lacks a strong reason for being a statement, and the only good argument against changing is to avoid an incompatible change. But since it is not a gratuitous incompatible change, the long term benefit out-weighs the short-term pain, in my opinion. -- Steven From python at mrabarnett.plus.com Wed Jan 27 20:27:42 2010 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 28 Jan 2010 01:27:42 +0000 Subject: Stuck on a three word street name regex In-Reply-To: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> References: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> Message-ID: <4B60E80E.4010301@mrabarnett.plus.com> Brian D wrote: > I've tackled this kind of problem before by looping through a patterns > dictionary, but there must be a smarter approach. > > Two addresses. Note that the first has incorrectly transposed the > direction and street name. The second has an extra space in it before > the street type. Clearly done by someone who didn't know how to > concatenate properly -- or didn't care. > > 1000 RAMPART S ST > > 100 JOHN CHURCHILL CHASE ST > > I want to parse the elements into an array of values that can be > inserted into new database fields. > > Anyone who loves solving these kinds of puzzles care to relieve my > frazzled brain? > > The pattern I'm using doesn't keep the "CHASE" with the "JOHN > CHURCHILL": > [snip] Regex doesn't gain you much. I'd split the string and then fix the parts as necessary: >>> def parse_address(address): ... parts = address.split() ... if parts[-2] == "S": ... parts[1 : -1] = [parts[-2]] + parts[1 : -2] ... parts[1 : -1] = [" ".join(parts[1 : -1])] ... return parts ... >>> print parse_address("1000 RAMPART S ST") ['1000', 'S RAMPART', 'ST'] >>> print parse_address("100 JOHN CHURCHILL CHASE ST") ['100', 'JOHN CHURCHILL CHASE', 'ST'] From wuwei23 at gmail.com Wed Jan 27 20:36:29 2010 From: wuwei23 at gmail.com (alex23) Date: Wed, 27 Jan 2010 17:36:29 -0800 (PST) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: Terry Reedy wrote: > Actually, Unladen Swallow is now targeted at 3.1; its developers have > conservatively proposed its integration in CPython 3.3. I would not be > completely shocked if it happens in 3.2. Why do I feel like there's less of an onus on Unladen Swallow to _actually prove itself in substantial real world usage_ before integration into CPython than there is on even the smallest of modules for inclusion in the standard library? Are we really expected to just ditch everything we know about CPython's performance characteristics just for some questionable and possibly uneven gains? I've been a big supporter of Py3 from the beginning, but this repeated claim of US becoming the mainline interpreter for 3.x pretty much kills dead a lot of my interest. What am I not seeing amidst the high memory usage and variable performance results of US's _custom-made_ benchmarks? Doesn't that fact alone worry anyone else? Or that LLVM is listed as only having "partial support" with non-Cygwin x86 Windows? Yes, I'd _love_ Python to be faster, who wouldn't? But I'm not seeing the evidence here to support the almost unbridled eagerness that's being demonstrated. From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 20:44:48 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 28 Jan 2010 01:44:48 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xvdenje7k.fsf@ruckus.brouhaha.com> <5b9e69bc-b261-4e81-84bc-045936011e82@o9g2000yqa.googlegroups.com> Message-ID: On Wed, 27 Jan 2010 16:16:59 -0800, Jonathan Gardner wrote: > On Jan 27, 3:54?pm, Paul Rubin wrote: >> Steven D'Aprano writes: >> > always much better written with key rather than cmp: key adds an O(N) >> > overheard to the sorting, while cmp makes sorting O(N**2). >> >> Whaaaaaaaaaa ...... ???? ?No I don't think so. > > You're referring to the O(N**2) bit, right? I am sure he knew it was O > (N*log(N)), which is still worse than O(N) for key. > > If he didn't, well, Python has some fundamental flaws in its basic sort > algorithm. I may have the details wrong, but the performance of sort with cmp is very poor. sort with key calls the key function once for each element in the list, which is O(N), and then sorts O(N*log N), giving a total of O(N*log N). sort with cmp has to compare the comparison function multiple times for each element. I *thought* it ended up calling it N times each, giving O(N**2), but I may be delusional. Whatever it is though, it's quite slow. Here's my test code (for Python 2.5): from timeit import Timer from string import letters from random import shuffle def mycmp(s, t): return cmp(s.upper(), t.upper()) def randstr(): L = list(letters) shuffle(L) return ''.join(L) setup = "from __main__ import mycmp, data" N = 300 data = [randstr() for _ in xrange(N)] t1 = Timer("d = data[:]; d.sort()", setup) # raw sort t2 = Timer("d = data[:]; d.sort(cmp=mycmp)", setup) t3 = Timer("d = data[:]; d.sort(key=str.upper)", setup) for t in (t1, t2, t3): print min(t.repeat(number=500)) N *= 10 # Do it again with bigger N. data = [randstr() for _ in xrange(N)] for t in (t1, t2, t3): print min(t.repeat(number=500)) And here are the results on my PC: # N = 300 0.0631489753723 2.36756515503 0.226485967636 # N = 3000 1.03457903862 35.3092911243 2.77242517471 This doesn't support my claim of O(N**2), but it does demonstrate that sort with cmp should be avoided if at all possible. -- Steven From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 20:46:12 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 28 Jan 2010 01:46:12 GMT Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <7xk4v3jco7.fsf@ruckus.brouhaha.com> Message-ID: On Wed, 27 Jan 2010 16:28:08 -0800, Paul Rubin wrote: > Steven D'Aprano writes: >> 6. The code for Python 3 was handed down to Guido from the Heavens, >> carved into stone tablets by the Gods themselves. > > That is heresy. The direction was up, not down. SPLITTER!!! -- Steven From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 20:56:00 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 28 Jan 2010 01:56:00 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xvdenje7k.fsf@ruckus.brouhaha.com> <5b9e69bc-b261-4e81-84bc-045936011e82@o9g2000yqa.googlegroups.com> Message-ID: On Wed, 27 Jan 2010 16:44:18 -0800, Carl Banks wrote: >> You're referring to the O(N**2) bit, right? I am sure he knew it was O >> (N*log(N)), which is still worse than O(N) for key. >> >> If he didn't, well, Python has some fundamental flaws in its basic sort >> algorithm. > > Quicksort is O(N**2) worst case, perhaps that's what he meant. No, apparently I was delusional. > I'm not > sure about timsort but since it's based on quicksort I'd guess it is > O(N**2) or worse, worst case, as well. timsort is actually a mergesort, not a quicksort. You may like to read this: http://svn.python.org/projects/python/trunk/Objects/listsort.txt > Typical case of a sort is still O(N*log(N)) whether using key or cmp. > People recommended against cmp not because of overall algorithmic time > considerations, but because of the number of function calls. cmp > function was called a minimum of N-1 times and typically around N*log2 > (N) times, whereas key was called exactly N times. Since the overhead > of a Python function call was very large compared to the built-in cmp > operation, it usually outweighed the algorithmic time considerations. Yes, that sounds more like it. Thanks for the corrections. -- Steven From pavlovevidence at gmail.com Wed Jan 27 20:59:39 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Wed, 27 Jan 2010 17:59:39 -0800 (PST) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <384da2bb-1130-4bfe-8fd8-a343ca4a000c@x10g2000prk.googlegroups.com> On Jan 27, 5:36?pm, alex23 wrote: > Terry Reedy wrote: > > Actually, Unladen Swallow is now targeted at 3.1; its developers have > > conservatively proposed its integration in CPython 3.3. I would not be > > completely shocked if it happens in 3.2. > > Why do I feel like there's less of an onus on Unladen Swallow to > _actually prove itself in substantial real world usage_ before > integration into CPython than there is on even the smallest of modules > for inclusion in the standard library? I don't sense that. I get a sense that there's a lot of people licking their chops because it's sponsored by Google and everything Google touches turns to gold, but that's just nameless plebians. I trust the developers not to be easily convinced. If GvR allows this into CPython without something like a typical 4x speed increase I'll eat my hat. Carl Banks From martien.friedeman at gmail.com Wed Jan 27 21:00:33 2010 From: martien.friedeman at gmail.com (hans moleman) Date: Wed, 27 Jan 2010 18:00:33 -0800 (PST) Subject: CodeInvestigator testers Message-ID: I need Python programmers to test my Python tracing tool CodeInvestigator please. It records all debug information in a recording phase. The program runs as per usual, albeit a lot slower, while this recording takes place. You can then move anywhere in your code and click words in the code to obtain debug information. Useful if you have no idea where to start to debug or you want to see how a program works. Your Python installation isn't changed by this tool. It can easily be uninstalled or forgotten. It requires Python 2.6 and a Firefox browser. It comes with an installer for Windows, Linux and Mac OSX. Any bugs, ideas and remarks please. http://sourceforge.net/projects/cde-investigatr/ Thank you. Martien Friedeman From sg552 at hotmail.co.uk Wed Jan 27 21:06:28 2010 From: sg552 at hotmail.co.uk (Rotwang) Date: Thu, 28 Jan 2010 02:06:28 +0000 Subject: List weirdness - what the heck is going on here? Message-ID: Hi all, I've been trying to make a class with which to manipulate sound data, and have run into some behaviour I don't understand which I hope somebody here can explain. The class has an attribute called data, which is a list with two elements, one for each audio channel, each of which is a list containing the audio data for that channel. It also has various methods to write data such as sine waves and so on, and a method to insert data from one sound at the start of data from another. Schematically, the relevant bits look like this: class sound: def f(self): self.data = [[0]]*2 def insert(self, other): for c in xrange(2): self.data[c][0:0] = other.data[c] However, the insert method doesn't work properly; x.insert(y) adds two copies of y's data to the start of x's data, instead of one. From a session in IDLE: >>> x = sound() >>> y = sound() >>> x.f() >>> y.f() >>> x.data [[0], [0]] >>> x.insert(y) >>> x.data [[0, 0, 0], [0, 0, 0]] But suppose I replace the line self.data = [[0]]*2 with self.data = [[0] for c in xrange(2)] Then it works fine: >>> x = sound() >>> y = sound() >>> x.f() >>> y.f() >>> x.data [[0], [0]] >>> x.insert(y) >>> x.data [[0, 0], [0, 0]] Can anybody tell me what's going on? From tekion at gmail.com Wed Jan 27 21:18:21 2010 From: tekion at gmail.com (tekion) Date: Wed, 27 Jan 2010 18:18:21 -0800 (PST) Subject: deriving MySQLdb class References: <9b9755ad-c3ca-42eb-b95c-0e41c5052cd7@p24g2000yqm.googlegroups.com> <10c80803-4934-4857-80e9-ba63648d9be4@14g2000yqp.googlegroups.com> <9f413105-5572-4bc7-93f4-ac7f252e3174@y23g2000yqm.googlegroups.com> Message-ID: On Jan 21, 11:48?pm, Sean DiZazzo wrote: > On Jan 21, 8:17?pm, tekion wrote: > > > Sean, > > I did a little investigation, there are other classes besides > > Connection. So, could I only set up a derived class from Connection > > and still be able to use the connection to query database and retrieve > > data? > > Im not sure I understand you completely... > > In theory, you could use the C API directly and subclass > _mysql.connection to get at the database. ?But I think the point of > MySQLdb is that they've done all the hard work. ?Why not use it? Regarding above statement, Yeah; what you said is true. I am just playing with inheritance to get a better understanding of it and since I am working with database connection, then I might as learn it by doing it. > I think the other stuff in the module is in support of the Connection > () class. ?ie. ?You cant get a cursor unless you already have a > connection. Yes, I have confirmed it just by inheriting the connection class, I am able to connect and perform normal database functions. Sweet!!! This is why I like Python. Thanks for your help. From nyamatongwe+thunder at gmail.com Wed Jan 27 21:18:33 2010 From: nyamatongwe+thunder at gmail.com (Neil Hodgson) Date: Thu, 28 Jan 2010 02:18:33 GMT Subject: myths about python 3 In-Reply-To: <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> Message-ID: Carl Banks: > There is also no hope someone will fork Python 2.x and continue it in > perpetuity. Well, someone might try to fork it, but they won't be > able to call it Python. Over time there may be more desire from those unable or unwilling to upgrade to 3.x to work on improvements to 2.x, perhaps leading to a version 2.8. One of the benefits of open source is that you are not trapped into following vendor decisions like Microsoft abandoning classic VB in favour of VB.NET. It would be unreasonable for the core developers to try to block this. Refusing use of the Python trademark for a version that was reasonably compatible in both directions would be particularly petty. Neil From angrybaldguy at gmail.com Wed Jan 27 21:18:34 2010 From: angrybaldguy at gmail.com (Owen Jacobson) Date: Wed, 27 Jan 2010 21:18:34 -0500 Subject: List weirdness - what the heck is going on here? References: Message-ID: <2010012721183416807-angrybaldguy@gmailcom> On 2010-01-27 21:06:28 -0500, Rotwang said: > Hi all, I've been trying to make a class with which to manipulate sound > data, and have run into some behaviour I don't understand which I hope > somebody here can explain. The class has an attribute called data, > which is a list with two elements, one for each audio channel, each of > which is a list containing the audio data for that channel. It also has > various methods to write data such as sine waves and so on, and a > method to insert data from one sound at the start of data from another. > Schematically, the relevant bits look like this: > > class sound: > def f(self): > self.data = [[0]]*2 Consider that this is equivalent to def f(self): x = [0] self.data = [x, x] self.data is now a list containing two references to the list referenced by x -- so changes via either of the elements of self.data will affect both elements. Your comprehension version creates a list containing two distinct list objects, so this doesn't happen. > Can anybody tell me what's going on? -o From apt.shansen at gmail.com Wed Jan 27 21:22:25 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Wed, 27 Jan 2010 18:22:25 -0800 Subject: List weirdness - what the heck is going on here? In-Reply-To: References: Message-ID: <7a9c25c21001271822g5a5df06dg2e8b5471951562b4@mail.gmail.com> On Wed, Jan 27, 2010 at 6:06 PM, Rotwang wrote: > But suppose I replace the line > self.data = [[0]]*2 > > with > > self.data = [[0] for c in xrange(2)] > The first line does not do what you think it does: it doesn't make a copy of that internal [0]. Python almost never implicitly copies anything, things that look like they may make a copy of a mutable object-- usually don't, unless there's something very explicit in the action that says 'making a new object out of that other object' What it is doing is creating a list which contains a reference to the same [0] twice, which you can see as: print self.data[0] is self.data[1] The "is" operator doesn't test equality, but precise identity-- those two objects are exactly the same. So when you loop over later to insert data, what you're doing is inserting the same data into the same exact list... that just happens to be known by two different names right now. :) The list comprehension, on the other hand, is creating a new [0] each time it loops over your xrange. --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From groups_ads12 at yahoo.com Wed Jan 27 21:35:45 2010 From: groups_ads12 at yahoo.com (groups_ads12) Date: Thu, 28 Jan 2010 02:35:45 -0000 Subject: www.visualstudio.me Message-ID: www.visualstudio.me buy visual studio buy visual studio 2008 download microsoft visual studio 2005 download visual studio 2003 download visual studio 2005 download visual studio 2008 download visual studio 6.0 download visual studio net free visual studio learn visual studio microsoft visual studio microsoft visual studio .net microsoft visual studio .net 2005 microsoft visual studio 2003 microsoft visual studio 2005 microsoft visual studio 2005 express edition microsoft visual studio 2005 free download microsoft visual studio 2005 professional microsoft visual studio 2005 professional edition microsoft visual studio 2005 standard edition microsoft visual studio 2008 microsoft visual studio 2008 express microsoft visual studio 2008 professional microsoft visual studio 2008 professional edition microsoft visual studio 2008 professional with msdn professional microsoft visual studio 2008 programming microsoft visual studio 2008 standard microsoft visual studio 2008 standard edition microsoft visual studio 2008 standard upgrade microsoft visual studio 6 microsoft visual studio 6.0 microsoft visual studio 8 microsoft visual studio download microsoft visual studio net microsoft visual studio net 2003 microsoft visual studio net 2005 microsoft visual studio pro 2008 microsoft visual studio tips ms visual studio ms visual studio 2005 ms visual studio 2008 visual studio visual studio 10 visual studio 2..8 visual studio 2002 visual studio 2003 visual studio 2003 service pack visual studio 2005 visual studio 2005 c# visual studio 2005 download visual studio 2005 pro visual studio 2005 professional visual studio 2005 professional edition visual studio 2005 rapidshare visual studio 2005 service pack visual studio 2005 service pack 1 visual studio 2005 service pack 2 visual studio 2005 sp visual studio 2005 sp1 visual studio 2005 sp2 visual studio 2005 standard edition visual studio 2005 team suite visual studio 2005 trial visual studio 2005 tutorial visual studio 2005 vista visual studio 2006 visual studio 2007 visual studio 2008 visual studio 2008 book visual studio 2008 c# visual studio 2008 free download visual studio 2008 price visual studio 2008 professional visual studio 2008 professional edition visual studio 2008 sdk visual studio 2008 service pack visual studio 2008 standard visual studio 2008 standard edition visual studio 2008 team system visual studio 2008 trial visual studio 2008 tutorial visual studio 2008 upgrade visual studio 2009 visual studio 2010 visual studio 2010 download visual studio 6 visual studio 6 download visual studio 6 service pack visual studio 6.0 visual studio 7 visual studio 8 visual studio c visual studio download visual studio enterprise visual studio gallery visual studio ide visual studio magazine visual studio msdn visual studio net visual studio net 2003 visual studio net 2003 download visual studio net 2005 visual studio net 2008 visual studio pro visual studio pro 2008 visual studio professional visual studio professional 2008 visual studio sdk visual studio setup project visual studio software visual studio standard visual studio team edition visual studio team suite visual studio team system visual studio tools visual studio tools for office visual studio training visual studio trial visual studio tutorial visual studio tutorials visual studio upgrade visual studio wiki visual studios -------------- next part -------------- An HTML attachment was scrubbed... URL: From tanix at mongo.net Wed Jan 27 21:36:41 2010 From: tanix at mongo.net (tanix) Date: Thu, 28 Jan 2010 02:36:41 GMT Subject: Python Goldmine has been updated: http://preciseinfo.org/Convert/index_Convert_Python.html References: Message-ID: In article , tanix at mongo.net (tanix) wrote: Good news: Site search has been fully implemented. You can search the entire Python collection or a single chapter related to specific context to find what you are looking for. Bad news: We are currently out of sync with google. So, if you do a google search you might see a different article than what google shows. >Python Goldmine collection contains the extensive collection of articles >going back several years. It includes thousands of code >examples and expert discussions on all major topics. > >The information is organized by relevant topics, covered >by the corresponding chapters. > >The information was filtered with sophisticated filters and vast >majority of artilces with little relevance have been filtered out. > >If you have any specific requests for some new chapters to be added >and it is of interest to others, please post your requests on this >thread. > >If anyone feels he has above average level of competence, or can >reccommend someone who posts on this group, you may request to be >included in the expert chapters. > >The Python Goldmine is at: > >http://preciseinfo.org/Convert/index_Convert_Python.html > -- Programmer's Goldmine collections: http://preciseinfo.org Tens of thousands of code examples and expert discussions on C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, organized by major topics of language, tools, methods, techniques. All collections are fully searchable down to specific chapter. From steve at holdenweb.com Wed Jan 27 21:37:21 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 27 Jan 2010 21:37:21 -0500 Subject: myths about python 3 In-Reply-To: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <4B60F861.1060408@holdenweb.com> John Nagle wrote: > Daniel Fetchinson wrote: >> Hi folks, >> >> I was going to write this post for a while because all sorts of myths >> periodically come up on this list about python 3. I don't think the >> posters mean to spread false information on purpose, they simply are >> not aware of the facts. >> >> My list is surely incomplete, please feel free to post your favorite >> misconception about python 3 that people periodically state, claim or >> ask about. > > Myths about Python 3: > > 1. Python 3 is supported by major Linux distributions. > > FALSE - most distros are shipping with Python 2.4, or 2.5 at best. > Why would a "major Linux distribution" want to saddle themselves with such a new technology so erly in its lifetime? > 2. Python 3 is supported by multiple Python implementations. > > FALSE - Only CPython supports 3.x. Iron Python, Unladen Swallow, > PyPy, and Jython have all stayed with 2.x versions of Python. > Your selective information here is particularly partial to your case. I have spoken with developers from IronPython and Jython, and both teams are committed to eventual support of 3.x. > 3. Python 3 is supported by most 3rd party Python packages. > > FALSE - it's not supported by MySQLdb, OpenSSL, feedparser, etc. > I would argue it's up to Python to support those facilities rather than the other way round. > Arguably, Python 3 has been rejected by the market. Instead, there's > now Python 2.6, Python 2.7, and Python 2.8. Python 3 has turned into > a debacle like Perl 6, now 10 years old. > > That's the reality, Python 3 fanboys. > Kindly confine your debate to the facts and keep the snide remarks to yourself. Like it or not Python 3 is the future, and unladen swallow's recent announcement that they would target only Python 3 represented a ground-breaking advance for the language. Happily my Python 2.x interpreters all continue to work just as they have since they were installed. If you have to stretch as far as Perl 6 for an analogy then you have clearly stretched a little too far. The situations are not even closely comparable, and I defy you to argue otherwise. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Wed Jan 27 21:37:21 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 27 Jan 2010 21:37:21 -0500 Subject: myths about python 3 In-Reply-To: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <4B60F861.1060408@holdenweb.com> John Nagle wrote: > Daniel Fetchinson wrote: >> Hi folks, >> >> I was going to write this post for a while because all sorts of myths >> periodically come up on this list about python 3. I don't think the >> posters mean to spread false information on purpose, they simply are >> not aware of the facts. >> >> My list is surely incomplete, please feel free to post your favorite >> misconception about python 3 that people periodically state, claim or >> ask about. > > Myths about Python 3: > > 1. Python 3 is supported by major Linux distributions. > > FALSE - most distros are shipping with Python 2.4, or 2.5 at best. > Why would a "major Linux distribution" want to saddle themselves with such a new technology so erly in its lifetime? > 2. Python 3 is supported by multiple Python implementations. > > FALSE - Only CPython supports 3.x. Iron Python, Unladen Swallow, > PyPy, and Jython have all stayed with 2.x versions of Python. > Your selective information here is particularly partial to your case. I have spoken with developers from IronPython and Jython, and both teams are committed to eventual support of 3.x. > 3. Python 3 is supported by most 3rd party Python packages. > > FALSE - it's not supported by MySQLdb, OpenSSL, feedparser, etc. > I would argue it's up to Python to support those facilities rather than the other way round. > Arguably, Python 3 has been rejected by the market. Instead, there's > now Python 2.6, Python 2.7, and Python 2.8. Python 3 has turned into > a debacle like Perl 6, now 10 years old. > > That's the reality, Python 3 fanboys. > Kindly confine your debate to the facts and keep the snide remarks to yourself. Like it or not Python 3 is the future, and unladen swallow's recent announcement that they would target only Python 3 represented a ground-breaking advance for the language. Happily my Python 2.x interpreters all continue to work just as they have since they were installed. If you have to stretch as far as Perl 6 for an analogy then you have clearly stretched a little too far. The situations are not even closely comparable, and I defy you to argue otherwise. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From rantingrick at gmail.com Wed Jan 27 21:41:37 2010 From: rantingrick at gmail.com (rantingrick) Date: Wed, 27 Jan 2010 18:41:37 -0800 (PST) Subject: Python and Ruby References: <43cad4e7-6b54-4bd9-9ca2-217cb109560f@g29g2000yqe.googlegroups.com> Message-ID: <1d501ad6-39f8-4e5d-aabc-222b71553940@e25g2000yqh.googlegroups.com> On Jan 27, 5:31?pm, Jonathan Gardner wrote: > To add to that, Python is the type of language where experienced > programmers can pick it up by reading code, and newbies won't get > hopelessly lost. I've taught less-than-formal introductory programming > classes to people who are new to programming. Python gets out of the > way, and allows you to focus on the programming concepts, such as > variable, functions, parameters, classes, and algorithms. Well said Jonathan!! Well said!! (both posts) From no.email at nospam.invalid Wed Jan 27 21:44:05 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 27 Jan 2010 18:44:05 -0800 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <7xzl3zq77u.fsf@ruckus.brouhaha.com> Steve Holden writes: > Kindly confine your debate to the facts and keep the snide remarks to > yourself. Like it or not Python 3 is the future, and unladen swallow's > recent announcement that they would target only Python 3 represented a > ground-breaking advance for the language. My take on things is that doing unladen swallow really "right" will require yet more incompatible changes; i.e., the result will either still leave quite a bit of performance on the table, or else it won't be compatible with the current specification of Python 3 and they'll presumably have to call it Python 4. And if Python 4 is as good as I believe it could possibly be, then it might get wide acceptance before Python 3 really has all that much uptake. If I have to accept incompatibility anyway, and Python 4 gives huge improvements while Python 3's improvements are tiny or moderate, why not skip over Python 3? From wuwei23 at gmail.com Wed Jan 27 21:59:26 2010 From: wuwei23 at gmail.com (alex23) Date: Wed, 27 Jan 2010 18:59:26 -0800 (PST) Subject: List weirdness - what the heck is going on here? References: Message-ID: <4702e667-c089-4349-a349-76ec844b6828@a17g2000pre.googlegroups.com> Rotwang wrote: > Can anybody tell me what's going on? Your problem is basically this: >>> a = [1] >>> b = [a] * 2 >>> b [[1], [1]] >>> a.append(2) >>> b [[1, 2], [1, 2]] The expression '[a] * 2' doesn't make two copies of list of a list of a, it makes two nested _references_ to it. When you modify the 'first' list, you're seeing that changed reflected in the second reference. The list comprehension, however, is returning a _new_ list each time. Changing its contents doesn't affect any of the other independently created lists. From cournape at gmail.com Wed Jan 27 22:27:41 2010 From: cournape at gmail.com (David Cournapeau) Date: Thu, 28 Jan 2010 12:27:41 +0900 Subject: python 3's adoption In-Reply-To: <7xockfjcsv.fsf@ruckus.brouhaha.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> <7xockfjcsv.fsf@ruckus.brouhaha.com> Message-ID: <5b8d13221001271927l11dfd032s3b8159c831444691@mail.gmail.com> On Thu, Jan 28, 2010 at 9:25 AM, Paul Rubin wrote: > I don't mind that 3.x is breaking stuff for the sake of improving > things. ?That's the whole idea of 3.x, after all. ?What bugs me is that > the improvements are mostly quite superficial, and the breakage seems > often gratuitous. ?I'd rather see much more drastic changes and > correspondingly much larger improvements. At the risk of saying something trivial, that depends on what you consider superficial or not. It totally depends on your usage. I know I would have taken something to improve python packaging (dropping distutils, stable ABI across the whole 3.x range) over any feature in py3k. That's why your real estate comparison is not adequate IMHO - what is 2 % for you is maybe 30 % for someone else, and my own 30 % for packaging may be 1 % for most people who don't care about C extensions at all. David. From briandenzer at gmail.com Wed Jan 27 22:39:44 2010 From: briandenzer at gmail.com (Brian D) Date: Wed, 27 Jan 2010 19:39:44 -0800 (PST) Subject: Stuck on a three word street name regex References: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> <7xbpgfjcc0.fsf@ruckus.brouhaha.com> Message-ID: On Jan 27, 6:35?pm, Paul Rubin wrote: > Brian D writes: > > I've tackled this kind of problem before by looping through a patterns > > dictionary, but there must be a smarter approach.> > > Two addresses. Note that the first has incorrectly transposed the > > direction and street name. .... > > If you're really serious about it (e.g. you are the post office trying > to program automatic mail sorting machines) there is no simple regex > trick anything like what you want. ?A lot of addresses will be > ambiguous. ?You have use all the info you have about your entire address > corpus (e.g. you need a complete street directory of the whole US) and > do a bunch of Bayesian inference. ?As a very simple example, for an > address like "1000 RAMPART S ST" you'd use the zip code to identify the > address's geographic neighborhood, and then use your street directory to > find candidate correct addresses within that zip code. ?The USPS does > an amazing job of delivering mail to completely mangled addresses > based on methods like that. Paul, That's a sound methodology. I actually have a routine that will compare an address to a list of all streets in the city using a Short Distance function. I have used that in circumstances when there are a lot of problems with addresses. In this case, however, the streets are actually structured very well -- except for the transposed street directions. I was really hoping to see if there's a solution that handles one, two, and three word strings, followed by an occasional single character, and then a two character suffix. I'm still hoping for that kind of a solution if it exists. The reason? It's actually a very small number of addresses that aren't being captured with the current regex. I don't see the need for overkill, and I'm always stretching to learn something I haven't already succeeded at accomplishing. I may just make a second pass at the data with a different regex. From briandenzer at gmail.com Wed Jan 27 22:57:25 2010 From: briandenzer at gmail.com (Brian D) Date: Wed, 27 Jan 2010 19:57:25 -0800 (PST) Subject: Stuck on a three word street name regex References: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> Message-ID: <0a712e20-facf-4a9b-8852-6136a8bad7dc@n7g2000yqb.googlegroups.com> On Jan 27, 7:27?pm, MRAB wrote: > Brian D wrote: > > I've tackled this kind of problem before by looping through a patterns > > dictionary, but there must be a smarter approach. > > > Two addresses. Note that the first has incorrectly transposed the > > direction and street name. The second has an extra space in it before > > the street type. Clearly done by someone who didn't know how to > > concatenate properly -- or didn't care. > > > 1000 RAMPART S ST > > > 100 JOHN CHURCHILL CHASE ?ST > > > I want to parse the elements into an array of values that can be > > inserted into new database fields. > > > Anyone who loves solving these kinds of puzzles care to relieve my > > frazzled brain? > > > The pattern I'm using doesn't keep the "CHASE" with the "JOHN > > CHURCHILL": > > [snip] > Regex doesn't gain you much. I'd split the string and then fix the parts > as necessary: > > ?>>> def parse_address(address): > ... ? ? parts = address.split() > ... ? ? if parts[-2] == "S": > ... ? ? ? ? parts[1 : -1] = [parts[-2]] + parts[1 : -2] > ... ? ? parts[1 : -1] = [" ".join(parts[1 : -1])] > ... ? ? return parts > ... > ?>>> print parse_address("1000 RAMPART S ST") > ['1000', 'S RAMPART', 'ST'] > ?>>> print parse_address("100 JOHN CHURCHILL CHASE ?ST") > ['100', 'JOHN CHURCHILL CHASE', 'ST'] This is a nice approach I wouldn't have thought to pursue. I've never seen this referencing of list elements in reverse order with negative values, so that certainly expands my knowledge of Python. Of course, I'd want to check for other directionals -- probably with a list check, e.g., if parts[-2] in ('E', 'W', 'N', 'S'): Thanks for sharing your approach. From g.bogle at auckland.no.spam.ac.nz Wed Jan 27 23:03:14 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Thu, 28 Jan 2010 17:03:14 +1300 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> Message-ID: Luis M. Gonz?lez wrote: >> Please don't post more noise and ad hominem attacks to the group, Steve. > > "Ad hominem"? > Please, operor non utor lingua non notus per vulgaris populus. > Gratias ago vos... > "ad hominem" is "lingua notus per vulgaris populus", the vulgar pop of these parts, anyway. From g.bogle at auckland.no.spam.ac.nz Wed Jan 27 23:04:27 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Thu, 28 Jan 2010 17:04:27 +1300 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: Steven D'Aprano wrote: > On Wed, 27 Jan 2010 02:28:00 +0100, Alf P. Steinbach wrote: > >>>> * Print is now a function. Great, much improvement. >> Actually not, IMHO. All it does is is to provide incompatibility. They >> forgot Ronald Reagan's old maxim: if it don't need fixin', don't fix it. The aphorism needs fixing: If it ain't broke, don't fix it. From hackingkk at gmail.com Wed Jan 27 23:12:22 2010 From: hackingkk at gmail.com (hackingKK) Date: Thu, 28 Jan 2010 09:42:22 +0530 Subject: Python and Ruby In-Reply-To: <1d501ad6-39f8-4e5d-aabc-222b71553940@e25g2000yqh.googlegroups.com> References: <43cad4e7-6b54-4bd9-9ca2-217cb109560f@g29g2000yqe.googlegroups.com> <1d501ad6-39f8-4e5d-aabc-222b71553940@e25g2000yqh.googlegroups.com> Message-ID: <4B610EA6.2050607@gmail.com> On Thursday 28 January 2010 08:11 AM, rantingrick wrote: > On Jan 27, 5:31 pm, Jonathan Gardner > wrote: > > >> To add to that, Python is the type of language where experienced >> programmers can pick it up by reading code, and newbies won't get >> hopelessly lost. I've taught less-than-formal introductory programming >> classes to people who are new to programming. Python gets out of the >> way, and allows you to focus on the programming concepts, such as >> variable, functions, parameters, classes, and algorithms. >> > Well said Jonathan!! Well said!! (both posts) > I share the same experience. As pointed out in the "how to think like a computer scientist in Python" I found that when I was teaching first time programmers, Python was the best choice. Not just the clean syntax but preciseness of the language itself was very helpful. as I callit "get in get your work done and get out!" methodology works very well for absolute beginner students for programming. new comers to this list might be happy to note that python reads very closely to spoken English. But I want to add more. while it is great tool for beginners, it is a power box for advanced users who have the attitude of "cut the crap off and come to the point" For such programmers python is extremly good, I teach all types of students and now have convinced a few univercities here in India to include python in engineering curriculam. Happy hacking. Krishnakant. From ppearson at nowhere.invalid Wed Jan 27 23:33:13 2010 From: ppearson at nowhere.invalid (Peter Pearson) Date: 28 Jan 2010 04:33:13 GMT Subject: ISO module for binomial coefficients, etc. References: Message-ID: <7scic9F5s8U1@mid.individual.net> On Sun, 24 Jan 2010 16:11:03 -0500, Dave Angel wrote: > I didn't think of simply summing the logs. A couple terms of Stirling's approximation work pretty well: def log_fact_half( N ): """log_fact_half( n ) returns the natural logarithm of the factorial of n/2. n need not be an integer. Domain: n from 0 to infinity Range: from something around -0.12 to infinity """ SmallFacts = ( 0.0000000000000000, -0.1207822376352453, 0.0000000000000000, 0.2846828704729192, 0.6931471805599453, 1.2009736023470743, 1.7917594692280550, 2.4537365708424423, 3.1780538303479458, 3.9578139676187165, 4.7874917427820458, 5.6625620598571418, 6.5792512120101012, 7.5343642367587327, 8.5251613610654147, 9.5492672573009969, 10.6046029027452509, 11.6893334207972686, 12.8018274800814691, 13.9406252194037634 ) if N < 0: raise RuntimeError, \ "Negative argument in LogHalfFact!" if N < len( SmallFacts ): RetVal = SmallFacts[ N ] else: n = 0.5 * N RetVal = (n+0.5)*math.log(n) - n + HALFLOG2PI + 1.0/(12*n) - \ 1.0/(360*n*n*n) return RetVal -- To email me, substitute nowhere->spamcop, invalid->net. From daniel at stutzbachenterprises.com Thu Jan 28 00:11:25 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Wed, 27 Jan 2010 23:11:25 -0600 Subject: ISO module for binomial coefficients, etc. In-Reply-To: References: Message-ID: On Sat, Jan 23, 2010 at 4:55 PM, kj wrote: > Before I go off to re-invent a thoroughly invented wheel, I thought > I'd ask around for some existing module for computing binomial > coefficient, hypergeometric coefficients, and other factorial-based > combinatorial indices. I'm looking for something that can handle > fairly large factorials (on the order of 10000!), using floating-point > approximations as needed, and is smart about optimizations, > memoizations, etc. > I don't have code for any of the other stuff, but I have some efficient code (see below) for computing exact integer factorials. It's a divide-and-conquer algorithm. I'm not sure of the exact time complexity, but it appears to somewhere between n*log n and n**2. python2.6 -m timeit -s 'from utils import factorial' 'x = factorial(10000)' 10 loops, best of 3: 27.7 msec per loop If that's not fast enough, I suggest using the gamma function to compute floating-point approximations, as gamma(n) == (n-1)!. Gamma isn't yet included in the Python standard library (http://bugs.python.org/issue3366), but you can use the ctypes module to get it from the system C library on many platforms. For computing binomial coefficients, you can use the lgamma function (log of gamma), also found in my system C libraries. Since choose(n, k) = exp(lgamma(n) - lgamma(k) - lgamma(n-k)). If you go with floating point, watch out for floating point rounding error. ;-) def factorial(n): "http://www.luschny.de/math/factorial/binarysplitfact.html" p = 1 r = 1 p, r = factorial_loop(n, p, r) return r << n_minus_num_of_bits(n) def factorial_loop(n, p, r): if n <= 2: return p, r p, r = factorial_loop(n // 2, p, r) p *= part_product(n // 2 + 1 + ((n // 2) & 1), n - 1 + (n & 1)) r *= p return p, r def part_product(n, m): if (m <= (n+1)): return n if (m == (n+2)): return n*m k = (n+m) // 2 if ((k & 1) != 1): k -= 1 return part_product(n, k) * part_product(k+2, m) def n_minus_num_of_bits(v): w = v if w >= 2**32-1: raise OverflowError w -= (0xaaaaaaaa & w) >> 1 w = (w & 0x33333333) + ((w >> 2) & 0x33333333) w = w + (w >> 4) & 0x0f0f0f0f w += w >> 8 w += w >> 16 return v - (w & 0xff) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From cournape at gmail.com Thu Jan 28 00:58:15 2010 From: cournape at gmail.com (David Cournapeau) Date: Thu, 28 Jan 2010 14:58:15 +0900 Subject: Library support for Python 3.x In-Reply-To: References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> Message-ID: <5b8d13221001272158l96082d8sf96521ad00adee43@mail.gmail.com> On Thu, Jan 28, 2010 at 7:38 AM, Terry Reedy wrote: > > For a windows user who depends on pre-built binaries, every new release > breaks *every* library that is not pure Python and needs to be compiled. That's not windows specific - most packages which distribute binary packages need to package binaries for every minor version (2.4, 2.5, etc...). That's certainly the case for numpy and scipy. Python does not have a stable ABI across minor releases, only micro releases. I doubt that's what Paul was referring to, though - he seemed more concern with API/language changes than ABI issues. cheers, David From no.email at nospam.invalid Thu Jan 28 01:37:52 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 27 Jan 2010 22:37:52 -0800 Subject: Library support for Python 3.x References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> Message-ID: <7xk4v2g2f3.fsf@ruckus.brouhaha.com> David Cournapeau writes: > That's not windows specific - most packages which distribute binary > packages need to package binaries for every minor version (2.4, 2.5, > etc...).... > I doubt that's what Paul was referring to, though - he seemed more > concern with API/language changes than ABI issues. I didn't realize the ABI situation was that unstable. I thought you could just package up a .so or .dll and people could keep using it. I tend to not want to use extension modules that are not in the stdlib, and I guess this is another reason to keep staying away from them. From tjreedy at udel.edu Thu Jan 28 01:39:14 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 28 Jan 2010 01:39:14 -0500 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On 1/27/2010 8:36 PM, alex23 wrote: > Terry Reedy wrote: >> Actually, Unladen Swallow is now targeted at 3.1; its developers have >> conservatively proposed its integration in CPython 3.3. This statement was to counter the 'myth' that US was only targeted at 2.x when the current situation is quite the opposite. >> I would not be completely shocked if it happens in 3.2. I was initially rather dubious about the idea. I based the above on the team's acceptance of and response to reasonable requirements. In particular, several people said that the speed/space traceoff should be optional, and that compilation 'without llvm' should really be without, not just with llvm present but disabled. Instead of arguing, Colin went ahead and patched the build process to make it be this way. > Why do I feel like there's less of an onus on Unladen Swallow to > _actually prove itself in substantial real world usage_ before > integration into CPython than there is on even the smallest of modules > for inclusion in the standard library? I have no idea. It will have to improve its speedup more before adoption. I will not be surprised if that happens. > Are we really expected to just ditch everything we know about > CPython's performance characteristics just for some questionable and > possibly uneven gains? > > I've been a big supporter of Py3 from the beginning, but this repeated > claim of US becoming the mainline interpreter for 3.x US is not a new or separate interpreter. It will be an optional jit replacement for one component of CPython, the eval loop. All the code for builting functions, types, and modules will be untouched, as will their big O performance characteristics. > pretty much kills dead a lot of my interest. If you can still have a binary free of the traceoff, why would you care? > What am I not seeing amidst the high > memory usage and variable performance results of US's _custom-made_ > benchmarks? Doesn't that fact alone worry anyone else? Or that LLVM is > listed as only having "partial support" with non-Cygwin x86 Windows? They claim they have pretty well fixed that. They know that complete Windows support, including 64 bit versions, is a necessity. Terry Jan Reedy From python at bdurham.com Thu Jan 28 01:43:57 2010 From: python at bdurham.com (python at bdurham.com) Date: Thu, 28 Jan 2010 01:43:57 -0500 Subject: Python interface to Google translate? Message-ID: <1264661037.19646.1356985801@webmail.messagingengine.com> Looking for any recommendations on a Python module/package that would allow me to submit small HTML Unicode documents to Google and retrieve the translated results. Or is this the type of task that one should use a urllib-like module and do manually? Note: By small HTML Unicode documents I mean files with about 50 paragraphs of text marked up with basic HTML tags. Any suggestions appreciated. Thanks, Malcolm -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnodel at googlemail.com Thu Jan 28 02:21:28 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Thu, 28 Jan 2010 07:21:28 +0000 Subject: List weirdness - what the heck is going on here? References: Message-ID: Rotwang writes: > Hi all, I've been trying to make a class with which to manipulate > sound data, and have run into some behaviour I don't understand which > I hope somebody here can explain. The class has an attribute called > data, which is a list with two elements, one for each audio channel, > each of which is a list containing the audio data for that channel. It > also has various methods to write data such as sine waves and so on, > and a method to insert data from one sound at the start of data from > another. Schematically, the relevant bits look like this: > > class sound: > def f(self): > self.data = [[0]]*2 > > def insert(self, other): > for c in xrange(2): > self.data[c][0:0] = other.data[c] > > However, the insert method doesn't work properly; x.insert(y) adds two > copies of y's data to the start of x's data, instead of one. From a > session in IDLE: > >>>> x = sound() >>>> y = sound() >>>> x.f() >>>> y.f() >>>> x.data > [[0], [0]] >>>> x.insert(y) >>>> x.data > [[0, 0, 0], [0, 0, 0]] > > But suppose I replace the line > > self.data = [[0]]*2 > > with > > self.data = [[0] for c in xrange(2)] > > Then it works fine: > >>>> x = sound() >>>> y = sound() >>>> x.f() >>>> y.f() >>>> x.data > [[0], [0]] >>>> x.insert(y) >>>> x.data > [[0, 0], [0, 0]] > > Can anybody tell me what's going on? It's a FAQ! http://www.python.org/doc/faq/programming/#how-do-i-create-a-multidimensional-list -- Arnaud From cournape at gmail.com Thu Jan 28 02:28:53 2010 From: cournape at gmail.com (David Cournapeau) Date: Thu, 28 Jan 2010 16:28:53 +0900 Subject: Library support for Python 3.x In-Reply-To: <7xk4v2g2f3.fsf@ruckus.brouhaha.com> References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> <7xk4v2g2f3.fsf@ruckus.brouhaha.com> Message-ID: <5b8d13221001272328u55b26275j68277b555c1058e8@mail.gmail.com> On Thu, Jan 28, 2010 at 3:37 PM, Paul Rubin wrote: > David Cournapeau writes: >> That's not windows specific - most packages which distribute binary >> packages need to package binaries for every minor version (2.4, 2.5, >> etc...).... >> I doubt that's what Paul was referring to, though - he seemed more >> concern with API/language changes than ABI issues. > > I didn't realize the ABI situation was that unstable. Unstable may be strong - every minor version of python has a lifespan of several years. But yes, that's an hindrance for packagers: you need to package binaries for every minor version of python, although I guess for trivial extensions, you may get away with it on some platforms. That's why as far as I am concerned, something like PEP 384 worths more than any feature in py3k I am aware of. I think it will have more impact than py3k's features for the scientific python, if the stable API is rich enough. It would certainly make more incentive for me to work on porting packages to py3k than just doing it because we will have to at some point. cheers, David From floris.bruynooghe at gmail.com Thu Jan 28 02:32:23 2010 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Wed, 27 Jan 2010 23:32:23 -0800 (PST) Subject: Ad hoc lists vs ad hoc tuples References: <8a57c64f-0c67-4ec1-b4be-34d76c7e7571@q4g2000yqm.googlegroups.com> Message-ID: <8e2f9215-ee5e-4114-a955-82a88d2be9fe@q4g2000yqm.googlegroups.com> On Jan 27, 10:15?pm, Terry Reedy wrote: > On 1/27/2010 12:32 PM, Antoine Pitrou wrote: > > > Le Wed, 27 Jan 2010 02:20:53 -0800, Floris Bruynooghe a ?crit : > > >> Is a list or tuple better or more efficient in these situations? > > > Tuples are faster to allocate (they are allocated in one single step) and > > quite a bit smaller too. Thanks for all the answers! This is what I was expecting but it's nice to see it confirmed. Regards Floris From richard.lamboj at bilcom.at Thu Jan 28 02:42:26 2010 From: richard.lamboj at bilcom.at (Richard Lamboj) Date: Thu, 28 Jan 2010 08:42:26 +0100 Subject: starting a thread in a nother thread In-Reply-To: <4b604df9$0$6723$9b4e6d93@newsspool2.arcor-online.net> References: <4b604df9$0$6723$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <201001280842.26177.richard.lamboj@bilcom.at> Am Wednesday 27 January 2010 15:30:17 schrieb Stefan Behnel: > Richard Lamboj, 27.01.2010 15:23: > > Am Wednesday 27 January 2010 14:10:13 schrieb Stefan Behnel: > >> Richard Lamboj, 27.01.2010 14:06: > >>> just for _curiosity_. What would be if i start a thread in a nother > >>> thread and acquire a lock in the "child" thread. Is there anything that > >>> could go wrong if someone try to start threads in threads? > >> > >> There's usually tons of things that can go wrong w.r.t. threads: > >> > >> http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf > >> > >> However, there's nothing special to a thread that was started from > >> another thread, so the problems don't change. > > > > i have tried a little bit around with psycopg2 and threads, > > > > I'am sharing one connection for all threads. When i'am starting the > > threads "normal" everything works without any Problem. When i'am starting > > the threads from another thread than i got a "segmentation fault" > > Sounds like a bug that you might want to report to the maintainers of > psycopg2. > > Stefan Yes i should make a bug report and thanks for your help. Just some Debugginginfo: [New Thread 0x429b5950 (LWP 27486)] python-dbg: ../Objects/stringobject.c:116: PyString_FromString: Assertion `str != ((void *)0)' failed. Program received signal SIGABRT, Aborted. [Switching to Thread 0x431b6950 (LWP 27484)] 0x00007f6d932feed5 in raise () from /lib/libc.so.6 From jgardner at jonathangardner.net Thu Jan 28 02:50:55 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Wed, 27 Jan 2010 23:50:55 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> <7xockfjcsv.fsf@ruckus.brouhaha.com> Message-ID: <6b89dc67-ceee-4b83-a11f-c7376185dcd1@b9g2000pri.googlegroups.com> On Jan 27, 4:25?pm, Paul Rubin wrote: > What about assert and pass? > If you're going to have statements, you're going to need the null statement. That's "pass". It could be renamed "null_statement" but "pass" is a better description. "None" and "pass" are cousins of sorts, since "None" is the null expression and "pass" the null statement. I agree on "assert". I don't like running a program in test mode and then running it in production mode with different code. I would rather test what I am going to actually run. "assert" should be a function, and support for removing assert statements should be eliminated. I simply don't use assert statements at all. From stefan_ml at behnel.de Thu Jan 28 03:00:23 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 28 Jan 2010 09:00:23 +0100 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <4b614417$0$6715$9b4e6d93@newsspool2.arcor-online.net> Benjamin Kaplan, 27.01.2010 22:25: > On Wed, Jan 27, 2010 at 3:56 PM, John Nagle wrote: >> 2. Python 3 is supported by multiple Python implementations. >> >> FALSE - Only CPython supports 3.x. Iron Python, Unladen Swallow, >> PyPy, and Jython have all stayed with 2.x versions of Python. >> > When Python 2.6 came out, Jython was still on 2.2. The difference > between 2.2 and 2.6 is almost as big of a difference as between 2.6 > and 3.0. >From an implementors point of view, it's actually quite the opposite. Most syntax features of Python 3 can be easily implemented on top of an existing Py2 Implementation (we have most of them in Cython already, and I really found them fun to write), and the shifting-around in the standard library can hardly be called non-trivial. All the hard work that went into the design of CPython 3.x (and into its test suite) now makes it easy to just steal from what's there already. The amount of work that the Jython project put into catching up from 2.1 to 2.5/6 (new style classes! generators!) is really humongous compared to the adaptations that an implementation needs to do to support Python 3 code. I have great respect for the Jython project for what they achieved in the last couple of years. (I also have great respect for the IronPython project for fighting the One Microsoft Way into opening up, but that's a different kind of business.) If there was enough interest from the respective core developers, I wouldn't be surprised if we had more than one 'mostly compatible' alternative Python 3 implementation in a couple of months. But it's the obvious vicious circle business. As long as there aren't enough important users of Py3, alternative implementations won't have enough incentives to refocus their scarce developer time. Going for 2.6/7 first means that most of the Py3 work gets done anyway, so it'll be even easier then. That makes 2.6->2.7->3.2/3 the most natural implementation path. (And that, again, makes it a *really* good decision that 2.7 will be the last 2.x release line.) >> 3. Python 3 is supported by most 3rd party Python packages. >> >> FALSE - it's not supported by MySQLdb, OpenSSL, feedparser, etc. >> >> Arguably, Python 3 has been rejected by the market. Instead, there's >> now Python 2.6, Python 2.7, and Python 2.8. Python 3 has turned into >> a debacle like Perl 6, now 10 years old. > > Give the package maintainers time to update. There were some pretty > big changes to the C API. Most of the major 3rd party packages like > numpy and MySQLdb have already commited to having a Python 3 version. > They just haven't gotten them out yet. I second that. NumPy has already announced that they'll rewrite some of their code in Cython to make it more maintainable and portable (and to simplify the port to Py3, I guess). Other projects will equally well find their ways to get their code ready. It's just a matter of time. I think there's enough pressure on the maintainers by now (both from users and from personal pride) to consider their packages tainted if they aren't portable enough to run on all major (C)Python versions (and Py3.1 certainly is one of them), even if they don't use them themselves. That appears to be an impressively good incentive. Stefan From durumdara at gmail.com Thu Jan 28 03:07:00 2010 From: durumdara at gmail.com (Durumdara) Date: Thu, 28 Jan 2010 09:07:00 +0100 Subject: Data exchange between Delphi and Python (Win) Message-ID: <9e384ef61001280007j38f231e4we085e9af6d1b725d@mail.gmail.com> Hi! I have an exotic db, with exotic drivers, and it have buggy ODBC driver. But I have native driver - under Delphi. I need to access this DB under Pylons (or mod_python). I wrote one solution that working with XML. But I search for easier way to transform and move data between apps. I saw Python for Delphi, but the installer is showing only Python 2.3 as selectable engine. I think to COM/OLE, because it is accessable from all program, and I think to DLL (but DLL have problematic parameterisation). The input data (what Delphi got) are SQL commands, and the output are the rows (if got). What do you thinking about it? Have anyone experience in this theme? Thanks for it! dd -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben+python at benfinney.id.au Thu Jan 28 03:08:41 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 28 Jan 2010 19:08:41 +1100 Subject: Library support for Python 3.x References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> <7xk4v2g2f3.fsf@ruckus.brouhaha.com> Message-ID: <873a1q3b3q.fsf@benfinney.id.au> David Cournapeau writes: > Unstable may be strong - every minor version of python has a lifespan > of several years. But yes, that's an hindrance for packagers: you need > to package binaries for every minor version of python It's important to note that this is mitigated, ironically enough, by intentionally targeting a minimum Python minor version because the code base makes use of Python features not available in older versions. That is, any minor version of Python that doesn't have the features your code base uses can be ignored (given the set of supported versions is explicitly declared) ? and hence one doesn't need to package binaries for every minor version. -- \ ?Our products just aren't engineered for security.? ?Brian | `\ Valentine, senior vice-president of Microsoft Windows | _o__) development, 2002 | Ben Finney From stefan_ml at behnel.de Thu Jan 28 03:11:31 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 28 Jan 2010 09:11:31 +0100 Subject: myths about python 3 In-Reply-To: <87zl3z2p63.fsf@benfinney.id.au> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <87zl3z2p63.fsf@benfinney.id.au> Message-ID: <4b6146b3$0$6715$9b4e6d93@newsspool2.arcor-online.net> Ben Finney, 27.01.2010 22:50: > Christian Heimes writes: > >> John Nagle wrote: >>> 1. Python 3 is supported by major Linux distributions. >>> >>> FALSE - most distros are shipping with Python 2.4, or 2.5 at best. >> You are wrong. Modern versions of Debian / Ubuntu are using Python >> 2.6. > > Only if by ?modern? you mean ?not released yet?. > > The latest stable Debian (Debian 5.0, Lenny) has only Python 2.4 and > Python 2.5. It does not have Python 2.6 at all, and until this month > Python 2.6 was not even in the in-development suite of Debian. 'Stable Debian' has a long tradition of being late and outdated on arrival. That doesn't mean you can't use existing Debian packages on it. And there certainly are .deb packages available for Py3.1.1 - Ubuntu uses them, and other Debian based distributions do, too. And Ubuntu Karmic definitely uses Py2.6 as 'python'. Stefan From no.email at nospam.invalid Thu Jan 28 03:37:54 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 28 Jan 2010 00:37:54 -0800 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> <7xockfjcsv.fsf@ruckus.brouhaha.com> <6b89dc67-ceee-4b83-a11f-c7376185dcd1@b9g2000pri.googlegroups.com> Message-ID: <7xy6ji62vx.fsf@ruckus.brouhaha.com> Jonathan Gardner writes: > If you're going to have statements, you're going to need the null > statement. That's "pass". Why? Expressions are statements, so you could just say "pass" (in quotes, denoting a string literal), or 0, or None, os anything else like that, instead of having a special statement. From no.email at nospam.invalid Thu Jan 28 03:40:09 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 28 Jan 2010 00:40:09 -0800 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <4b614417$0$6715$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <7xtyu662s6.fsf@ruckus.brouhaha.com> Stefan Behnel writes: > The amount of work that the Jython project put into catching up from 2.1 to > 2.5/6 (new style classes! generators!) is really humongous compared to the > adaptations that an implementation needs to do to support Python 3 code. I wonder whether Jython could borrow code from Clojure for some of this stuff, to save a little work. Cross-fertilization between free software projects is usually a good thing. From stefan_ml at behnel.de Thu Jan 28 03:40:42 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 28 Jan 2010 09:40:42 +0100 Subject: Library support for Python 3.x In-Reply-To: References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> Message-ID: <4b614d8a$0$6718$9b4e6d93@newsspool2.arcor-online.net> David Cournapeau, 28.01.2010 06:58: > On Thu, Jan 28, 2010 at 7:38 AM, Terry Reedy wrote: > >> For a windows user who depends on pre-built binaries, every new release >> breaks *every* library that is not pure Python and needs to be compiled. > > That's not windows specific - most packages which distribute binary > packages need to package binaries for every minor version (2.4, 2.5, > etc...). That's certainly the case for numpy and scipy. Python does > not have a stable ABI across minor releases, only micro releases. That doesn't completely match my experience. It's true that there is no guarantee that the ABI will stay compatible, but when you compile lxml against Py2.4 on a 32bit machine, it will continue to import in Py2.5 and (IIRC) Py2.6. It won't be as fast and it won't use some newer features, but it will work. Don't remember my experience with 2.3, though. It obviously can't work the other way round, i.e. when compiling against 2.6, it will never work in 2.5 or earlier. But there is definitely a certain degree of ABI compatibility available. Stefan From cournape at gmail.com Thu Jan 28 03:48:47 2010 From: cournape at gmail.com (David Cournapeau) Date: Thu, 28 Jan 2010 17:48:47 +0900 Subject: Library support for Python 3.x In-Reply-To: <873a1q3b3q.fsf@benfinney.id.au> References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> <7xk4v2g2f3.fsf@ruckus.brouhaha.com> <873a1q3b3q.fsf@benfinney.id.au> Message-ID: <5b8d13221001280048u10596800xfe68108902b0ecb3@mail.gmail.com> On Thu, Jan 28, 2010 at 5:08 PM, Ben Finney wrote: > > It's important to note that this is mitigated, ironically enough, by > intentionally targeting a minimum Python minor version because the code > base makes use of Python features not available in older versions. > > That is, any minor version of Python that doesn't have the features your > code base uses can be ignored (given the set of supported versions is > explicitly declared) ? and hence one doesn't need to package binaries > for every minor version. This has nothing to do whatsoever with feature, since we are talking about ABI issues. David From cournape at gmail.com Thu Jan 28 03:54:02 2010 From: cournape at gmail.com (David Cournapeau) Date: Thu, 28 Jan 2010 17:54:02 +0900 Subject: Library support for Python 3.x In-Reply-To: <4b614d8a$0$6718$9b4e6d93@newsspool2.arcor-online.net> References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> <4b614d8a$0$6718$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <5b8d13221001280054w54d07eb3s6be5384644c01b38@mail.gmail.com> On Thu, Jan 28, 2010 at 5:40 PM, Stefan Behnel wrote: > > That doesn't completely match my experience. It's true that there is no > guarantee that the ABI will stay compatible, but when you compile lxml > against Py2.4 on a 32bit machine, it will continue to import in Py2.5 and > (IIRC) Py2.6. It won't be as fast and it won't use some newer features, but > it will work. Don't remember my experience with 2.3, though. Importing fine is a very low expectation for ABI compatibility :) Since python does not make ABI guarantees between minor releases, you don't know whether some structures layouts are changed between versions, and in general, tracking crashes due to those is no fun. It really depends on how much you depend on the C API, but for something extensive like NumPy, I don't think it would ever work. So yes, you could say "just try and if it crashes, check that it is not ABI-related". In practice, this is very poor engineering in my book... David From alfps at start.no Thu Jan 28 04:12:18 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 28 Jan 2010 10:12:18 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: * Steven D'Aprano: > On Wed, 27 Jan 2010 18:29:25 +0100, Alf P. Steinbach wrote: > >> The main problem with the incompatibility is for porting code, not for >> writing code from scratch. > > Correct. It's a trivial problem, but still a problem. > >> It's also a problem wrt. learning the language. > > This makes no sense. Why is it harder to learn > > print(x, y, z) > > than this? > > print x, y, z I think it's actually easier to learn just the 3.x form. But it's more difficult to learn that there are /two different/ syntactic forms, depending on which version of Python you're using and in addition depending on __future__ declarations! E.g., picking up some example code from the web, and it does not work... > The first case is like the other functions you have to learn, like len(). > In fact, many newbies to Python put unneeded parentheses around arguments > to print simply because they assume it is a function. > > I would argue that the new print function is *simpler* to learn. It is > more consistent with other built-ins, and has fewer magic idioms to > learn. Yes, yes, yes, I agree. > Instead of: > > print >>fileObj, x, y, z > > you use regular function syntax with a meaningful keyword: > > print(x, y, z, file=fileObj) > > If you want suppress the newline at the end of each print: > > print x, y, z, # note the final comma > > compared to: > > print(x, y, z, end='') Actually I thought the final comma thing was nice. It was like Basic. I think the 2.x 'print' must have been modeled on Basic's 'print'. > If you want to change the space between elements, instead of: > > sys.stdout.write(str(x) + "*" + str(y) + "*" + str(z) + '\n') > > you use: > > print(x, y, z, sep='*') > > > If you want to override the behaviour of print in a module, instead of > having to edit the source code of the module (which might not even be > available), all you need to do is monkey-patch it: > > import module > module.print = myprint >>> import builtins >>> >>> org_print = print >>> builtins.print = 666 >>> >>> print( "trallala" ) Traceback (most recent call last): File "", line 1, in TypeError: 'int' object is not callable >>> org_print( "but is that really so smart?" ) but is that really so smart? >>> _ Cheers, - Alf From no.email at nospam.invalid Thu Jan 28 04:21:23 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 28 Jan 2010 01:21:23 -0800 Subject: Library support for Python 3.x References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> <4b614d8a$0$6718$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <7xiqamziss.fsf@ruckus.brouhaha.com> David Cournapeau writes: > So yes, you could say "just try and if it crashes, check that it is > not ABI-related". In practice, this is very poor engineering in my > book... I just looked at PEP 384 and I don't see anything in it about version numbers in the interfaces. I certainly think something like that should be added if it's not too late. Basically any extension module should check that the CPython loading it is new enough, and CPython should (when feasible) continue to support old interfaces when changes are made. This is pretty standard stuff as done in COM, Java, and presumably .NET, along with many communications protocols. From stefan_ml at behnel.de Thu Jan 28 04:25:26 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 28 Jan 2010 10:25:26 +0100 Subject: Library support for Python 3.x In-Reply-To: References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> <4b614d8a$0$6718$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <4b615807$0$6724$9b4e6d93@newsspool2.arcor-online.net> David Cournapeau, 28.01.2010 09:54: > On Thu, Jan 28, 2010 at 5:40 PM, Stefan Behnel wrote: > >> That doesn't completely match my experience. It's true that there is no >> guarantee that the ABI will stay compatible, but when you compile lxml >> against Py2.4 on a 32bit machine, it will continue to import in Py2.5 and >> (IIRC) Py2.6. It won't be as fast and it won't use some newer features, but >> it will work. Don't remember my experience with 2.3, though. > > Importing fine is a very low expectation for ABI compatibility :) Ok, I should have said "imports and runs its test suite successfully". I just wanted to add a "works for me" to counter your rather pessimistic comments. > Since python does not make ABI guarantees between minor releases, you > don't know whether some structures layouts are changed between > versions, and in general, tracking crashes due to those is no fun. It > really depends on how much you depend on the C API, but for something > extensive like NumPy, I don't think it would ever work. I wouldn't be so sure that NumPy is so much more "extensive" in C-API usage than lxml. > So yes, you could say "just try and if it crashes, check that it is > not ABI-related". In practice, this is very poor engineering in my > book... Well, I don't know if there is any 'official' core developer statement regarding ABI compatibility, but I know that Guido doesn't take it easy to break it for a release. He tried pretty hard to keep it up for 2.5->2.6, at least, even if he was aware that it would be futile for 2.x->3.x. Stefan From duncan.booth at invalid.invalid Thu Jan 28 04:45:21 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 28 Jan 2010 09:45:21 GMT Subject: Ad hoc lists vs ad hoc tuples References: <8a57c64f-0c67-4ec1-b4be-34d76c7e7571@q4g2000yqm.googlegroups.com> Message-ID: Terry Reedy wrote: > Constant tuples (a tuple whose members are all seen as constants by the > compiler) are now pre-compiled and constructed once and put into the > code object as such rather than re-constructed with each run of the code. Sadly that's not entirely accurate. Tuples whose members are all simple constants are pre-compiled, but here's an example of a tuple whose members are all constants but doesn't get optimised: >>> def foo(): data = ( ('hello', 42), ('world', 24), ) return data >>> import dis >>> dis.dis(foo) 3 0 LOAD_CONST 5 (('hello', 42)) 4 3 LOAD_CONST 6 (('world', 24)) 6 BUILD_TUPLE 2 9 STORE_FAST 0 (data) 6 12 LOAD_FAST 0 (data) 15 RETURN_VALUE -- Duncan Booth http://kupuguy.blogspot.com From gandalf at shopzeus.com Thu Jan 28 05:50:16 2010 From: gandalf at shopzeus.com (Laszlo Nagy) Date: Thu, 28 Jan 2010 15:20:16 +0430 Subject: Portable way to tell if a process is still alive In-Reply-To: References: Message-ID: <4B616BE8.7080202@shopzeus.com> >> Suppose we have a program that writes its process id into a pid file. >> Usually the program deletes the pid file when it exists... But in some >> cases (for example, killed with kill -9 or TerminateProcess) pid file is >> left there. I would like to know if a process (given with its process >> id) is still running or not. I know that this can be done with OS >> specific calls. But that is not portable. It can also be done by >> executing "ps -p 23423" with subprocess module, but that is not portable >> either. Is there a portable way to do this? >> >> If not, would it be a good idea to implement this (I think very >> primitive) function in the os module? >> > > Not only is there no way to do it portably, there is no way to do it > reliably for the general case. The problem is that processes do not have > unique identifiers. A PID only uniquely identifies a running process; once > the process terminates, its PID becomes available for re-use. > Non-general case: the process is a service and only one instance should be running. There could be a pid file left on the disk. It is possible to e.g. mount procfs, and check if the given PID belongs to a command line / executed program that is in question. It cannot be guaranteed that a service will always delete its pid file when it exists. It happens for example, somebody kills it with "kill -9" or exits on signal 11 etc. It actually did happened to me, and then the service could not be restarted because the PID file was there. (It is an error to run two instances of the same service, but it is also an error to not run it....) Whan I would like to do upon startup is to check if the process is already running. This way I could create a "guardian" that checks other services, and (re)starts them if they stopped working. And no, it is not a solution to write "good" a service that will never stop, because: 1. It is particulary not possible in my case - there is a software bug in a third party library that causes my service exit on various wreid signals. 2. It is not possible anyway. There are users killing processes accidentally, and other unforeseen bugs. 3. In a mission critical environment, I would use a guardian even if guarded services are not likely to stop I understand that this is a whole different question now, and possibly there is no portable way to do it. Just I wonder if there are others facing a similar problem here. Any thoughs or comments - is it bad that I would like to achieve? Is there a better approach? Thanks, Laszlo From denis-bz-gg at t-online.de Thu Jan 28 06:54:43 2010 From: denis-bz-gg at t-online.de (denis) Date: Thu, 28 Jan 2010 03:54:43 -0800 (PST) Subject: Total maximal size of data References: Message-ID: <7647d906-f9dd-43cc-9732-11f0d4e46669@n7g2000yqb.googlegroups.com> On Jan 25, 8:05?pm, Alexander Moibenko wrote: > I have a simple question to which I could not find an answer. > What is the [total maximal] size of list including size of its elements? Beware, sys.getsizeof(alist) is 4*len(alist) + a bit, regardless of alists's contents ?! See http://stackoverflow.com/questions/2117255/python-deep-getsizeof-list-with-contents cheers -- denis From sg552 at hotmail.co.uk Thu Jan 28 07:11:05 2010 From: sg552 at hotmail.co.uk (Rotwang) Date: Thu, 28 Jan 2010 12:11:05 +0000 Subject: List weirdness - what the heck is going on here? In-Reply-To: <2010012721183416807-angrybaldguy@gmailcom> References: <2010012721183416807-angrybaldguy@gmailcom> Message-ID: Owen Jacobson wrote: > On 2010-01-27 21:06:28 -0500, Rotwang said: > >> Hi all, I've been trying to make a class with which to manipulate >> sound data, and have run into some behaviour I don't understand which >> I hope somebody here can explain. The class has an attribute called >> data, which is a list with two elements, one for each audio channel, >> each of which is a list containing the audio data for that channel. It >> also has various methods to write data such as sine waves and so on, >> and a method to insert data from one sound at the start of data from >> another. Schematically, the relevant bits look like this: >> >> class sound: >> def f(self): >> self.data = [[0]]*2 > > Consider that this is equivalent to > > def f(self): > x = [0] > self.data = [x, x] > > self.data is now a list containing two references to the list referenced > by x -- so changes via either of the elements of self.data will affect > both elements. Your comprehension version creates a list containing two > distinct list objects, so this doesn't happen. Thanks, and likewise to everyone else who replied. From lie.1296 at gmail.com Thu Jan 28 07:12:22 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Thu, 28 Jan 2010 23:12:22 +1100 Subject: interaction of mode 'r+', file.write(), and file.tell(): a bug or undefined behavior? Message-ID: <4b617f4a$1@dnews.tpgi.com.au> In the code: """ f = open('input.txt', 'r+') for line in f: s = line.replace('python', 'PYTHON') # f.tell() f.write(s) """ When f.tell() is commented, 'input.txt' does not change; but when uncommented, the f.write() succeeded writing into the 'input.txt' (surprisingly, but not entirely unexpected, at the end of the file). $ ##################################### $ $ cp orig.txt input.txt $ cat input.txt abcde abc python abc python abc python $ python Python 2.6.4 (r264:75706, Jan 12 2010, 05:24:27) [GCC 4.3.4] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> f = open('input.txt', 'r+') >>> for line in f: ... s = line.replace('python', 'PYTHON') ... f.write(s) ... >>> $ cat input.txt abcde abc python abc python abc python $ $ ##################################### $ $ cp orig.txt input.txt $ cat input.txt abcde abc python abc python abc python $ python Python 2.6.4 (r264:75706, Jan 12 2010, 05:24:27) [GCC 4.3.4] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> f = open('input.txt', 'r+') >>> for line in f: ... s = line.replace('python', 'PYTHON') ... f.tell() ... f.write(s) ... 39 45 60 >>> $ cat input.txt abcde abc python abc python abc python abcde abc PYTHON abc $ $ ##################################### Do you think this should be a bug or undefined behavior governed by the underlying OS and C library? Shouldn't file.tell() be purely informational, and not have side effect? The machine is Gentoo (amd64, gcc-4.3.4, glibc-2.10.1-r1), Linux (2.6.31-gentoo-r6), and Python 2.6.4 From robert.kiwanuka at gmail.com Thu Jan 28 07:29:01 2010 From: robert.kiwanuka at gmail.com (kiwanuka) Date: Thu, 28 Jan 2010 04:29:01 -0800 (PST) Subject: is there any alternative to savefig? Message-ID: Hi all, I wonder if anyone knows any alternative function in pylab (or otherwise) that could be used to save an image. My problem is as follows: --------------- from pylab import * ... figure(1) fig1 = gca() figure(2) fig2 = gca() figure(3) fig3 = gca() for i,data_file in enumerate(data_file_list): time,x, y,x2, y2 = read_csv_file_4(open (data_file),elements=num_of_elements) fig1.plot(-x,-y,color=colours[i],label=labellist[i]) fig2.plot(time,-y,color=colours[i],label=labellist[i]) fig3.plot(time,-x,color=colours[i],label=labellist[i]) fig1.legend(loc='best') fig1.set_title("y1 - x1") fig1.set_ylabel("y1") fig1.set_xlabel("x1") #savefig("y1-x1.png") fig2.legend(loc='best') fig2.set_title("y1 - time") fig2.set_ylabel("y1") fig2.set_xlabel("time[s]") #savefig("y1-time.png") fig3.legend(loc='best') fig3.set_title("x1 - time") fig3.set_ylabel("x1") fig3.set_xlabel("time[s]") #savefig("x1-time.png") show() --------------------------- In the above code, I read multiple data files and plot three separate figures. Now I would like to save each of the figures to a file as the commented savefig satements suggest. The trouble is that if I uncomment all those savefig statements, I get three saved images all containing the plot belonging to figure(3), which was the last figure declared. I understand this to be happening because savefig will save the "current" figure, which in this case happens to be the last one declared. If I could do something like fig1.savefig("y1-x1.png") or savefig("y1- x1.png").fig1, this would solve the problem but I'm not aware of any such methods or modules to enable this. This is thus a flaw in the general design/implementation of the savefig function, but is there an alternative function to enable me achieve what I need? Is there perhaps a possible tweak to savefig to make it do the same? Thanks in advance, Robert From fetchinson at googlemail.com Thu Jan 28 07:41:47 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Thu, 28 Jan 2010 13:41:47 +0100 Subject: Library support for Python 3.x In-Reply-To: <4b60a6c9$0$1598$742ec2ed@news.sonic.net> References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <4b60a6c9$0$1598$742ec2ed@news.sonic.net> Message-ID: >> I'm going to be starting some new Python projects in Python 2.6, but am >> concerned that at least three of the libraries I will be >> using--pycrypto, paramiko and feedparser--are not currently supported in >> Python 3.x. The authors of these programs have not given any indication >> that work is underway to support Python 3.x. Eventually I plan to >> migrate to Python 3.x. > > Maybe by 2015 or so, that might be feasible. Wait until Python 3.x > ships as standard with major Linux distros. Right now, 2.4 or 2.5 is > the production version of Python. You keep repeating this nonsense even though it has been pointed out to you in a neighbouring thread that many (most?) of the main linux distros ship python 2.6 and not 2.5 or 2.4. For example Fedora 11 and 12 both ship python 2.6, others mentioned lots of other examples in said other thread, anyone can look them up. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From richard.lamboj at bilcom.at Thu Jan 28 08:25:30 2010 From: richard.lamboj at bilcom.at (Richard Lamboj) Date: Thu, 28 Jan 2010 14:25:30 +0100 Subject: Killing a Thread Message-ID: <201001281425.30434.richard.lamboj@bilcom.at> Hello, which Method is better to kill a Thread? Using Thread Events, or a raising a Exception? Maybe someone has a small example for me? Kind Regards, Richi From spam2008 at nney.com Thu Jan 28 08:31:36 2010 From: spam2008 at nney.com (D HANNEY) Date: Thu, 28 Jan 2010 05:31:36 -0800 (PST) Subject: Generic proxy (that proxies methods like __iter__) References: <635f48e9-b8de-402b-8215-1e5c26a33ed1@k35g2000yqb.googlegroups.com> Message-ID: <40f612b7-8b9c-4cf8-81df-65d4386e52de@r24g2000yqd.googlegroups.com> On Jan 27, 3:07?pm, Arnaud Delobelle wrote: > On 27 Jan, 14:41, D HANNEY wrote: > [...] > > See [1] for an explanation. ?Here is an idea: you could get round that > by generating a class on the fly, if you don't mind changing the class > of the object (untested): > > def noguardproxy(obj): > ? ? class NoGuardProxy(type(obj)): > ? ? ? ? def __enter__(self): > ? ? ? ? ? ? ? ?return self > ? ? ? ? def __exit__(self, type, value, traceback): > ? ? ? ? ? ? ? ?return False > ? ? obj.__class__ = NoGuardProxy > ? ? return obj > > If you want to keep obj untouched, you could add the line > > ? ? obj = copy(obj) > > before changing the class. > > -- > Arnaud > > [1]http://docs.python.org/reference/datamodel.html#new-style-special-lookup Thank you! Your solution works if I change "type(obj)" to say "obj.__class__". If I don't make this change Python complains "TypeError: Error when calling the metaclass bases type 'instance' is not an acceptable base type". So, I've got something that works but I don't understand why it works. I've read your docs.python reference but that hasn't helped yet. It might come to me but in the meantime, can you (or anyone) offer an explanation for this? Thanks in advance, David From briandenzer at gmail.com Thu Jan 28 08:40:09 2010 From: briandenzer at gmail.com (Brian D) Date: Thu, 28 Jan 2010 05:40:09 -0800 (PST) Subject: Stuck on a three word street name regex References: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> <0a712e20-facf-4a9b-8852-6136a8bad7dc@n7g2000yqb.googlegroups.com> Message-ID: <4e222e90-e088-4a82-a7d2-c62f509e2517@g29g2000yqe.googlegroups.com> > > [snip] > > Regex doesn't gain you much. I'd split the string and then fix the parts > > as necessary: > > > ?>>> def parse_address(address): > > ... ? ? parts = address.split() > > ... ? ? if parts[-2] == "S": > > ... ? ? ? ? parts[1 : -1] = [parts[-2]] + parts[1 : -2] > > ... ? ? parts[1 : -1] = [" ".join(parts[1 : -1])] > > ... ? ? return parts > > ... > > ?>>> print parse_address("1000 RAMPART S ST") > > ['1000', 'S RAMPART', 'ST'] > > ?>>> print parse_address("100 JOHN CHURCHILL CHASE ?ST") > > ['100', 'JOHN CHURCHILL CHASE', 'ST'] > > This is a nice approach I wouldn't have thought to pursue. I've never > seen this referencing of list elements in reverse order with negative > values, so that certainly expands my knowledge of Python. Of course, > I'd want to check for other directionals -- probably with a list > check, e.g., > > if parts[-2] in ('E', 'W', 'N', 'S'): > > Thanks for sharing your approach. After studying this again today, I realized the ingeniousness of reverse slicing the list (or perhaps right slicing) -- that one doesn't have to worry about the number of words in the string. To translate for those who may follow, the expression "parts[1 : -1]" means gather list items from position one in the list (index position 2) to one index position before the end of the list. The value in this is that we already know the first list element after a split() will be the street number. The last element will be the street type. Everything in between, no matter how many words, will be the street name -- excepting, of course, the instances where there's a street direction added in, as captured in example above. A very nice solution. Thanks again! From briandenzer at gmail.com Thu Jan 28 08:48:48 2010 From: briandenzer at gmail.com (Brian D) Date: Thu, 28 Jan 2010 05:48:48 -0800 (PST) Subject: Stuck on a three word street name regex References: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> <0a712e20-facf-4a9b-8852-6136a8bad7dc@n7g2000yqb.googlegroups.com> <4e222e90-e088-4a82-a7d2-c62f509e2517@g29g2000yqe.googlegroups.com> Message-ID: <1b7dba4d-3f12-4d31-8e9a-b36cd4f40217@u41g2000yqe.googlegroups.com> On Jan 28, 7:40?am, Brian D wrote: > > > [snip] > > > Regex doesn't gain you much. I'd split the string and then fix the parts > > > as necessary: > > > > ?>>> def parse_address(address): > > > ... ? ? parts = address.split() > > > ... ? ? if parts[-2] == "S": > > > ... ? ? ? ? parts[1 : -1] = [parts[-2]] + parts[1 : -2] > > > ... ? ? parts[1 : -1] = [" ".join(parts[1 : -1])] > > > ... ? ? return parts > > > ... > > > ?>>> print parse_address("1000 RAMPART S ST") > > > ['1000', 'S RAMPART', 'ST'] > > > ?>>> print parse_address("100 JOHN CHURCHILL CHASE ?ST") > > > ['100', 'JOHN CHURCHILL CHASE', 'ST'] > > > This is a nice approach I wouldn't have thought to pursue. I've never > > seen this referencing of list elements in reverse order with negative > > values, so that certainly expands my knowledge of Python. Of course, > > I'd want to check for other directionals -- probably with a list > > check, e.g., > > > if parts[-2] in ('E', 'W', 'N', 'S'): > > > Thanks for sharing your approach. > > After studying this again today, I realized the ingeniousness of > reverse slicing the list (or perhaps right slicing) -- that one > doesn't have to worry about the number of words in the string. > > To translate for those who may follow, the expression "parts[1 : -1]" > means gather list items from position one in the list (index position > 2) to one index position before the end of the list. The value in this > is that we already know the first list element after a split() will be > the street number. The last element will be the street type. > Everything in between, no matter how many words, will be the street > name -- excepting, of course, the instances where there's a street > direction added in, as captured in example above. > > A very nice solution. Thanks again! Correction: [snip] the expression "parts[1 : -1]" means gather list items from the second element in the list (index value 1) to one index position before the end of the list. [snip] From spam2008 at nney.com Thu Jan 28 08:54:16 2010 From: spam2008 at nney.com (D HANNEY) Date: Thu, 28 Jan 2010 05:54:16 -0800 (PST) Subject: Generic proxy (that proxies methods like __iter__) References: <635f48e9-b8de-402b-8215-1e5c26a33ed1@k35g2000yqb.googlegroups.com> <40f612b7-8b9c-4cf8-81df-65d4386e52de@r24g2000yqd.googlegroups.com> Message-ID: <5c153055-d88b-49d2-8938-2efac70f0548@k19g2000yqc.googlegroups.com> On Jan 28, 1:31 pm, D HANNEY wrote: > > Your solution works if I change "type(obj)" to say "obj.__class__". > If I don't make this change Python complains "TypeError: Error when > calling the metaclass bases type 'instance' is not an acceptable base > type". > So, I've got something that works but I don't understand why it works. > I've read your docs.python reference but that hasn't helped yet. It > might come to me but in the meantime, can you (or anyone) offer an > explanation for this? > > Thanks in advance, > > David Oh it looks like this is py3K thing. Python 3.0.1+ (r301:69556, Apr 15 2009, 15:59:22) >>> from io import StringIO >>> type(StringIO("x")) >>> StringIO("x").__class__ >>> Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41) >>> from StringIO import StringIO >>> type(StringIO("x")) >>> StringIO("x").__class__ >>> I was using 2.6.2. Your suggestion would have been fine in py3k. Even if I stuck with 2.6.2 and StringIO was a "new style class" there then it would have been fine ... but it isn't. As it stands __class__ seems the only way that works on both 3k and 2k. Bit weird, but I get it now. Thanks again! David From lie.1296 at gmail.com Thu Jan 28 09:27:59 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Fri, 29 Jan 2010 01:27:59 +1100 Subject: Stuck on a three word street name regex In-Reply-To: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> References: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> Message-ID: <4b619f12$1@dnews.tpgi.com.au> On 01/28/10 11:28, Brian D wrote: > I've tackled this kind of problem before by looping through a patterns > dictionary, but there must be a smarter approach. > > Two addresses. Note that the first has incorrectly transposed the > direction and street name. The second has an extra space in it before > the street type. Clearly done by someone who didn't know how to > concatenate properly -- or didn't care. > > 1000 RAMPART S ST > > 100 JOHN CHURCHILL CHASE ST > > I want to parse the elements into an array of values that can be > inserted into new database fields. > > Anyone who loves solving these kinds of puzzles care to relieve my > frazzled brain? > > The pattern I'm using doesn't keep the "CHASE" with the "JOHN > CHURCHILL": How does the following perform? pat = re.compile(r'(?P\d+)\s+(?P[A-Z\s]+)\s+(?PN|S|W|E|)\s+(?PST|RD|AVE?|)$') or more legibly: pat = re.compile( r''' (?P \d+ ) #M series of digits \s+ (?P [A-Z\s]+ ) #M one-or-more word \s+ (?P S?E|SW?|N?W|NE?| ) #O direction or nothing \s+ (?P ST|RD|AVE? ) #M street type $ #M END ''', re.VERBOSE) From exarkun at twistedmatrix.com Thu Jan 28 09:33:23 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Thu, 28 Jan 2010 14:33:23 -0000 Subject: Portable way to tell if a process is still alive In-Reply-To: <4B616BE8.7080202@shopzeus.com> References: <4B616BE8.7080202@shopzeus.com> Message-ID: <20100128143323.1898.2126720850.divmod.xquotient.889@localhost.localdomain> On 10:50 am, gandalf at shopzeus.com wrote: > >>>Suppose we have a program that writes its process id into a pid file. >>>Usually the program deletes the pid file when it exists... But in >>>some cases (for example, killed with kill -9 or TerminateProcess) pid >>>file is left there. I would like to know if a process (given with its >>>process id) is still running or not. I know that this can be done >>>with OS specific calls. But that is not portable. It can also be done >>>by executing "ps -p 23423" with subprocess module, but that is not >>>portable either. Is there a portable way to do this? >>> >>>If not, would it be a good idea to implement this (I think very >>>primitive) function in the os module? >> >>Not only is there no way to do it portably, there is no way to do it >>reliably for the general case. The problem is that processes do not >>have >>unique identifiers. A PID only uniquely identifies a running process; >>once >>the process terminates, its PID becomes available for re-use. > >Non-general case: the process is a service and only one instance should >be running. There could be a pid file left on the disk. It is possible >to e.g. mount procfs, and check if the given PID belongs to a command >line / executed program that is in question. It cannot be guaranteed >that a service will always delete its pid file when it exists. It >happens for example, somebody kills it with "kill -9" or exits on >signal 11 etc. It actually did happened to me, and then the service >could not be restarted because the PID file was there. (It is an error >to run two instances of the same service, but it is also an error to >not run it....) Whan I would like to do upon startup is to check if the >process is already running. This way I could create a "guardian" that >checks other services, and (re)starts them if they stopped working. > >And no, it is not a solution to write "good" a service that will never >stop, because: > >1. It is particulary not possible in my case - there is a software bug >in a third party library that causes my service exit on various wreid >signals. >2. It is not possible anyway. There are users killing processes >accidentally, and other unforeseen bugs. >3. In a mission critical environment, I would use a guardian even if >guarded services are not likely to stop > >I understand that this is a whole different question now, and possibly >there is no portable way to do it. Just I wonder if there are others >facing a similar problem here. Any thoughs or comments - is it bad that >I would like to achieve? Is there a better approach? I've been pondering using a listening unix socket for this. As long as the process is running, a client can connect to the unix socket. As soon as the process isn't, no matter the cause, clients can no longer connect to it. A drawback of this approach in some cases is probably that the process should be accepting these connections (and then dropping them). This may not always be easy to add to an existing app. Jean-Paul From anthony.tolle at gmail.com Thu Jan 28 09:38:54 2010 From: anthony.tolle at gmail.com (Anthony Tolle) Date: Thu, 28 Jan 2010 06:38:54 -0800 (PST) Subject: interaction of mode 'r+', file.write(), and file.tell(): a bug or undefined behavior? References: <4b617f4a$1@dnews.tpgi.com.au> Message-ID: <35ee0c46-f4e5-4dcb-ac32-7fe9d0b68104@15g2000vbg.googlegroups.com> On Jan 28, 7:12?am, Lie Ryan wrote: > In the code: > > """ > f = open('input.txt', 'r+') > for line in f: > ? ? s = line.replace('python', 'PYTHON') > ? ? # f.tell() > ? ? f.write(s) > """ > [snip] My guess is that there are a few possible problems: 1) In this case, writing to file opened with 'r+' without an explicit f.seek is probably not a good idea. The file iterator (for line in f) uses a readahead buffer, which means you can't guarantee what the current file position will be. 2) It may be necessary to do an explicit f.flush or f.close when writing to an 'r+' file. In your case, the close should automatically happen when the f object falls out of scope, which tells me that were still looking at some other problem, like not using f.seek 3) It is possible that f.tell implicitly flushes buffers used by the file object. That would explain why uncommenting the f.tell causes the writes to show up. What are you trying to accomplish? Overwrite the original file, or append to it? If you want to overwrite the file, it may be better to generate a new file, delete the old one, then rename the new one. If you want to append, then it would be better to open the file with append mode ('a') From alfps at start.no Thu Jan 28 10:00:37 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 28 Jan 2010 16:00:37 +0100 Subject: interaction of mode 'r+', file.write(), and file.tell(): a bug or undefined behavior? In-Reply-To: <35ee0c46-f4e5-4dcb-ac32-7fe9d0b68104@15g2000vbg.googlegroups.com> References: <4b617f4a$1@dnews.tpgi.com.au> <35ee0c46-f4e5-4dcb-ac32-7fe9d0b68104@15g2000vbg.googlegroups.com> Message-ID: * Anthony Tolle: > On Jan 28, 7:12 am, Lie Ryan wrote: >> In the code: >> >> """ >> f = open('input.txt', 'r+') >> for line in f: >> s = line.replace('python', 'PYTHON') >> # f.tell() >> f.write(s) >> """ >> [snip] > > My guess is that there are a few possible problems: > > 1) In this case, writing to file opened with 'r+' without an explicit > f.seek is probably not a good idea. The file iterator (for line in f) > uses a readahead buffer, which means you can't guarantee what the > current file position will be. > > 2) It may be necessary to do an explicit f.flush or f.close when > writing to an 'r+' file. In your case, the close should automatically > happen when the f object falls out of scope, which tells me that were > still looking at some other problem, like not using f.seek > > 3) It is possible that f.tell implicitly flushes buffers used by the > file object. That would explain why uncommenting the f.tell causes > the writes to show up. As far as I understand it the behavior stems from CPython file operations being implemented fairly directly as forwarding to C library FILE* operations, and the C standard prescribes Undefined Behavior to the case above. I think the Python language/library specification should specify the effect (perhaps just as UB, but anyway, specified). For as it is, it may/will be different with different Python implementations, meaning that code that works OK with one implementation may fail with another implementation. > What are you trying to accomplish? Overwrite the original file, or > append to it? If you want to overwrite the file, it may be better to > generate a new file, delete the old one, then rename the new one. If > you want to append, then it would be better to open the file with > append mode ('a') Cheers, - Alf From karthikramaswamy88 at gmail.com Thu Jan 28 10:07:04 2010 From: karthikramaswamy88 at gmail.com (evilweasel) Date: Thu, 28 Jan 2010 07:07:04 -0800 (PST) Subject: Need help with a program Message-ID: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Hi folks, I am a newbie to python, and I would be grateful if someone could point out the mistake in my program. Basically, I have a huge text file similar to the format below: AAAAAGACTCGAGTGCGCGGA 0 AAAAAGATAAGCTAATTAAGCTACTGG 0 AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 AAAAAGGTCGCCTGACGGCTGC 0 The text is nothing but DNA sequences, and there is a number next to it. What I will have to do is, ignore those lines that have 0 in it, and print all other lines (excluding the number) in a new text file (in a particular format called as FASTA format). This is the program I wrote for that: seq1 = [] list1 = [] lister = [] listers = [] listers1 = [] a = [] d = [] i = 0 j = 0 num = 0 file1 = open(sys.argv[1], 'r') for line in file1: if not line.startswith('\n'): seq1 = line.split() if len(seq1) == 0: continue a = seq1[0] list1.append(a) d = seq1[1] lister.append(d) b = len(lister) for j in range(0, b): if lister[j] == 0: listers.append(j) else: listers1.append(j) print listers1 resultsfile = open("sequences1.txt", 'w') for i in listers1: resultsfile.write('\n>seq' + str(i) + '\n' + list1[i] + '\n') But this isn't working. I am not able to find the bug in this. I would be thankful if someone could point it out. Thanks in advance! Cheers! From alfps at start.no Thu Jan 28 10:17:41 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 28 Jan 2010 16:17:41 +0100 Subject: Need help with a program In-Reply-To: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: * evilweasel: > Hi folks, > > I am a newbie to python, and I would be grateful if someone could > point out the mistake in my program. Basically, I have a huge text > file similar to the format below: > > AAAAAGACTCGAGTGCGCGGA 0 > AAAAAGATAAGCTAATTAAGCTACTGG 0 > AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 > AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 > AAAAAGGTCGCCTGACGGCTGC 0 > > The text is nothing but DNA sequences, and there is a number next to > it. What I will have to do is, ignore those lines that have 0 in it, > and print all other lines (excluding the number) in a new text file > (in a particular format called as FASTA format). This is the program I > wrote for that: > > seq1 = [] > list1 = [] > lister = [] > listers = [] > listers1 = [] > a = [] > d = [] > i = 0 > j = 0 > num = 0 > > file1 = open(sys.argv[1], 'r') > for line in file1: > if not line.startswith('\n'): > seq1 = line.split() > if len(seq1) == 0: > continue > > a = seq1[0] > list1.append(a) > > d = seq1[1] > lister.append(d) > > > b = len(lister) > for j in range(0, b): > if lister[j] == 0: > listers.append(j) > else: > listers1.append(j) > > > print listers1 > resultsfile = open("sequences1.txt", 'w') > for i in listers1: > resultsfile.write('\n>seq' + str(i) + '\n' + list1[i] + '\n') > > But this isn't working. What do you mean by "isn't working"? > I am not able to find the bug in this. I would > be thankful if someone could point it out. Thanks in advance! What do you expect as output, and what do you actually get as output? Cheers, - Alf From dickinsm at gmail.com Thu Jan 28 10:22:00 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Thu, 28 Jan 2010 07:22:00 -0800 (PST) Subject: Need help with a program References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: <2181c046-f9fe-48d2-b6cf-95b9c5f9c946@l19g2000yqb.googlegroups.com> On Jan 28, 3:07?pm, evilweasel wrote: > Hi folks, > > I am a newbie to python, and I would be grateful if someone could > point out the mistake in my program. > for j in range(0, b): > ? ? if lister[j] == 0: At a guess, this line should be: if lister[j] == '0': ... -- Mark From krister.svanlund at gmail.com Thu Jan 28 10:28:11 2010 From: krister.svanlund at gmail.com (Krister Svanlund) Date: Thu, 28 Jan 2010 16:28:11 +0100 Subject: Need help with a program In-Reply-To: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: <2cf430a61001280728h1a2f4977xe1fee5c269c009c7@mail.gmail.com> On Thu, Jan 28, 2010 at 4:07 PM, evilweasel wrote: > Hi folks, > > I am a newbie to python, and I would be grateful if someone could > point out the mistake in my program. Basically, I have a huge text > file similar to the format below: > > AAAAAGACTCGAGTGCGCGGA ? 0 > AAAAAGATAAGCTAATTAAGCTACTGG ? ? 0 > AAAAAGATAAGCTAATTAAGCTACTGGGTT ? 1 > AAAAAGGGGGCTCACAGGGGAGGGGTAT ? ? 1 > AAAAAGGTCGCCTGACGGCTGC ?0 > > The text is nothing but DNA sequences, and there is a number next to > it. What I will have to do is, ignore those lines that have 0 in it, > and print all other lines (excluding the number) in a new text file > (in a particular format called as FASTA format). This is the program I > wrote for that: > > seq1 = [] > list1 = [] > lister = [] > listers = [] > listers1 = [] > a = [] > d = [] > i = 0 > j = 0 > num = 0 > > file1 = open(sys.argv[1], 'r') > for line in file1: > ? ?if not line.startswith('\n'): > ? ? ? ?seq1 = line.split() > ? ? ? ?if len(seq1) == 0: > ? ? ? ? ? ?continue > > ? ? ? ?a = seq1[0] > ? ? ? ?list1.append(a) > > ? ? ? ?d = seq1[1] > ? ? ? ?lister.append(d) > > > b = len(lister) > for j in range(0, b): > ? ?if lister[j] == 0: > ? ? ? ?listers.append(j) > ? ?else: > ? ? ? ?listers1.append(j) > > > print listers1 > resultsfile = open("sequences1.txt", 'w') > for i in listers1: > ? ?resultsfile.write('\n>seq' + str(i) + '\n' + list1[i] + '\n') > > But this isn't working. I am not able to find the bug in this. I would > be thankful if someone could point it out. Thanks in advance! > > Cheers! I'm not totaly sure what you want to do but try this (python2.6+): newlines = [] with open(sys.argv[1], 'r') as f: text = f.read(); for line in text.splitlines(): if not line.strip() and line.strip().endswith('1'): newlines.append('seq'+line) with open(sys.argv[2], 'w') as f: f.write('\n'.join(newlines)) From krister.svanlund at gmail.com Thu Jan 28 10:31:49 2010 From: krister.svanlund at gmail.com (Krister Svanlund) Date: Thu, 28 Jan 2010 16:31:49 +0100 Subject: Need help with a program In-Reply-To: <2cf430a61001280728h1a2f4977xe1fee5c269c009c7@mail.gmail.com> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> <2cf430a61001280728h1a2f4977xe1fee5c269c009c7@mail.gmail.com> Message-ID: <2cf430a61001280731w1ea33884r5d09d48b9fa3eb6c@mail.gmail.com> On Thu, Jan 28, 2010 at 4:28 PM, Krister Svanlund wrote: > On Thu, Jan 28, 2010 at 4:07 PM, evilweasel > wrote: >> Hi folks, >> >> I am a newbie to python, and I would be grateful if someone could >> point out the mistake in my program. Basically, I have a huge text >> file similar to the format below: >> >> AAAAAGACTCGAGTGCGCGGA ? 0 >> AAAAAGATAAGCTAATTAAGCTACTGG ? ? 0 >> AAAAAGATAAGCTAATTAAGCTACTGGGTT ? 1 >> AAAAAGGGGGCTCACAGGGGAGGGGTAT ? ? 1 >> AAAAAGGTCGCCTGACGGCTGC ?0 >> >> The text is nothing but DNA sequences, and there is a number next to >> it. What I will have to do is, ignore those lines that have 0 in it, >> and print all other lines (excluding the number) in a new text file >> (in a particular format called as FASTA format). This is the program I >> wrote for that: >> >> seq1 = [] >> list1 = [] >> lister = [] >> listers = [] >> listers1 = [] >> a = [] >> d = [] >> i = 0 >> j = 0 >> num = 0 >> >> file1 = open(sys.argv[1], 'r') >> for line in file1: >> ? ?if not line.startswith('\n'): >> ? ? ? ?seq1 = line.split() >> ? ? ? ?if len(seq1) == 0: >> ? ? ? ? ? ?continue >> >> ? ? ? ?a = seq1[0] >> ? ? ? ?list1.append(a) >> >> ? ? ? ?d = seq1[1] >> ? ? ? ?lister.append(d) >> >> >> b = len(lister) >> for j in range(0, b): >> ? ?if lister[j] == 0: >> ? ? ? ?listers.append(j) >> ? ?else: >> ? ? ? ?listers1.append(j) >> >> >> print listers1 >> resultsfile = open("sequences1.txt", 'w') >> for i in listers1: >> ? ?resultsfile.write('\n>seq' + str(i) + '\n' + list1[i] + '\n') >> >> But this isn't working. I am not able to find the bug in this. I would >> be thankful if someone could point it out. Thanks in advance! >> >> Cheers! > > I'm not totaly sure what you want to do but try this (python2.6+): > > newlines = [] > > with open(sys.argv[1], 'r') as f: > ? ?text = f.read(); > ? ?for line in text.splitlines(): > ? ? ? ?if not line.strip() and line.strip().endswith('1'): ? ? ? ? ? ?newlines.append('seq'+line.strip()[:-1].strip()) > > with open(sys.argv[2], 'w') as f: > ? ?f.write('\n'.join(newlines)) > Gah, made some errors From krister.svanlund at gmail.com Thu Jan 28 10:39:16 2010 From: krister.svanlund at gmail.com (Krister Svanlund) Date: Thu, 28 Jan 2010 16:39:16 +0100 Subject: Need help with a program In-Reply-To: <2cf430a61001280737p58fcfe2dj606a6c08f78a121f@mail.gmail.com> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> <2cf430a61001280728h1a2f4977xe1fee5c269c009c7@mail.gmail.com> <2cf430a61001280731w1ea33884r5d09d48b9fa3eb6c@mail.gmail.com> <2cf430a61001280737x73202974w358804b4580a3a7f@mail.gmail.com> <2cf430a61001280737p58fcfe2dj606a6c08f78a121f@mail.gmail.com> Message-ID: <2cf430a61001280739n44c11f67gb642d0a47c7509cc@mail.gmail.com> On Thu, Jan 28, 2010 at 4:31 PM, Krister Svanlund wrote: > On Thu, Jan 28, 2010 at 4:28 PM, Krister Svanlund > wrote: >> On Thu, Jan 28, 2010 at 4:07 PM, evilweasel >> wrote: >>> Hi folks, >>> >>> I am a newbie to python, and I would be grateful if someone could >>> point out the mistake in my program. Basically, I have a huge text >>> file similar to the format below: >>> >>> AAAAAGACTCGAGTGCGCGGA ? 0 >>> AAAAAGATAAGCTAATTAAGCTACTGG ? ? 0 >>> AAAAAGATAAGCTAATTAAGCTACTGGGTT ? 1 >>> AAAAAGGGGGCTCACAGGGGAGGGGTAT ? ? 1 >>> AAAAAGGTCGCCTGACGGCTGC ?0 >>> >>> The text is nothing but DNA sequences, and there is a number next to >>> it. What I will have to do is, ignore those lines that have 0 in it, >>> and print all other lines (excluding the number) in a new text file >>> (in a particular format called as FASTA format). This is the program I >>> wrote for that: >>> >>> seq1 = [] >>> list1 = [] >>> lister = [] >>> listers = [] >>> listers1 = [] >>> a = [] >>> d = [] >>> i = 0 >>> j = 0 >>> num = 0 >>> >>> file1 = open(sys.argv[1], 'r') >>> for line in file1: >>> ? ?if not line.startswith('\n'): >>> ? ? ? ?seq1 = line.split() >>> ? ? ? ?if len(seq1) == 0: >>> ? ? ? ? ? ?continue >>> >>> ? ? ? ?a = seq1[0] >>> ? ? ? ?list1.append(a) >>> >>> ? ? ? ?d = seq1[1] >>> ? ? ? ?lister.append(d) >>> >>> >>> b = len(lister) >>> for j in range(0, b): >>> ? ?if lister[j] == 0: >>> ? ? ? ?listers.append(j) >>> ? ?else: >>> ? ? ? ?listers1.append(j) >>> >>> >>> print listers1 >>> resultsfile = open("sequences1.txt", 'w') >>> for i in listers1: >>> ? ?resultsfile.write('\n>seq' + str(i) + '\n' + list1[i] + '\n') >>> >>> But this isn't working. I am not able to find the bug in this. I would >>> be thankful if someone could point it out. Thanks in advance! >>> >>> Cheers! I'm trying this again: newlines = [] with open(sys.argv[1], 'r') as f: ? ?text = f.read(); ? ?for line in (l.strip() for l in text.splitlines()): ? ? ? ?if line: ? ? ? ? ? ?line_elem = line.split() ? ? ? ? ? ?if len(line_elem) == 2 and line_elem[1] == '1': ? ? ? ? ? ? ? ?newlines.append('seq'+line_elem[0]) with open(sys.argv[2], 'w') as f: ? ?f.write('\n'.join(newlines)) From darcy at druid.net Thu Jan 28 10:44:37 2010 From: darcy at druid.net (D'Arcy J.M. Cain) Date: Thu, 28 Jan 2010 10:44:37 -0500 Subject: Need help with a program In-Reply-To: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: <20100128104437.3c48ec68.darcy@druid.net> On Thu, 28 Jan 2010 07:07:04 -0800 (PST) evilweasel wrote: > I am a newbie to python, and I would be grateful if someone could Welcome. > point out the mistake in my program. Basically, I have a huge text > file similar to the format below: You don't say how it isn't working. As a first step you should read http://catb.org/~esr/faqs/smart-questions.html. > The text is nothing but DNA sequences, and there is a number next to > it. What I will have to do is, ignore those lines that have 0 in it, Your code doesn't completely ignore them. See below. > and print all other lines (excluding the number) in a new text file > (in a particular format called as FASTA format). This is the program I > wrote for that: > > seq1 = [] > list1 = [] > lister = [] > listers = [] > listers1 = [] > a = [] > d = [] > i = 0 > j = 0 > num = 0 This seems like an awful lot of variables for such a simple task. > > file1 = open(sys.argv[1], 'r') > for line in file1: This is good. You aren't trying to load the whole file into memory at once. If the file is huge as you say then that would have been bad. I would have made one small optimization that saves one assignment and one extra variable. for line in open(sys.argv[1], 'r'): > if not line.startswith('\n'): > seq1 = line.split() > if len(seq1) == 0: > continue This is redundant and perhaps not even correct at the end of the file. It assumes that the last line ends with a newline. Look at what '\n'.split() gives you and see if you can't improve the above code. Another small optimization - "if seq1" is better than "if len(seq1)". > > a = seq1[0] > list1.append(a) Aha! I may have found your bug. Are you mixing tabs and spaces? Don't do that. Either always use spaces or always use tabs. My suggestion is to use spaces and choose a short indent such as three or even two but that's a religious issue. > > d = seq1[1] > lister.append(d) You can also do "a, d = seq1". Of course you must be sure that you have two fields. Perhaps that's guaranteed for your input but a quick sanity test wouldn't hurt here. However, I don't understand all of the above. It may also be a source of problems. You say the files are huge. Are you filling up memory here? You did the smart thing reading the file but you lose it here. In any case, see below. > b = len(lister) > for j in range(0, b): Go lookup zip() > if lister[j] == 0: I think that you will find that lister[j] is "0", not 0. > listers.append(j) > else: > listers1.append(j) Why are you collecting the input? Just toss the '0' ones and write the others lines directly to the output. Hope this helps with this script and in further understanding the power and simplicity of Python. Good luck. -- D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner. From robert.kiwanuka at gmail.com Thu Jan 28 10:45:28 2010 From: robert.kiwanuka at gmail.com (kiwanuka) Date: Thu, 28 Jan 2010 07:45:28 -0800 (PST) Subject: is there any alternative to savefig? References: Message-ID: <887d573b-a32b-4cf7-a262-1675c971a316@m25g2000yqc.googlegroups.com> On Jan 28, 12:29?pm, kiwanuka wrote: > Hi all, > > I wonder if anyone knows any alternative function in pylab (or > otherwise) that could be used to save an image. My problem is as > follows: > > --------------- > from pylab import * > ... > > figure(1) > fig1 = gca() > figure(2) > fig2 = gca() > figure(3) > fig3 = gca() > > for i,data_file in enumerate(data_file_list): > ? ?time,x, y,x2, y2 = read_csv_file_4(open > (data_file),elements=num_of_elements) > ? ?fig1.plot(-x,-y,color=colours[i],label=labellist[i]) > ? ?fig2.plot(time,-y,color=colours[i],label=labellist[i]) > ? ?fig3.plot(time,-x,color=colours[i],label=labellist[i]) > > fig1.legend(loc='best') > fig1.set_title("y1 - x1") > fig1.set_ylabel("y1") > fig1.set_xlabel("x1") > #savefig("y1-x1.png") > > fig2.legend(loc='best') > fig2.set_title("y1 - time") > fig2.set_ylabel("y1") > fig2.set_xlabel("time[s]") > #savefig("y1-time.png") > > fig3.legend(loc='best') > fig3.set_title("x1 - time") > fig3.set_ylabel("x1") > fig3.set_xlabel("time[s]") > #savefig("x1-time.png") > show() > --------------------------- > > In the above code, I read multiple data files and plot three separate > figures. Now I would like to save each of the figures to a file as the > commented savefig satements suggest. The trouble is that if I > uncomment all those savefig statements, I get three saved images all > containing the plot belonging to figure(3), which was the last figure > declared. > > I understand this to be happening because savefig will save the > "current" figure, which in this case happens to be the last one > declared. > > If I could do something like fig1.savefig("y1-x1.png") or savefig("y1- > x1.png").fig1, this would solve the problem but I'm not aware of any > such methods ?or modules to enable this. This is thus a flaw in the > general design/implementation of the savefig function, but is there an > alternative function to enable me achieve what I need? Is there > perhaps a possible tweak to savefig to make it do the same? > > Thanks in advance, > > Robert Problem solved. From alfps at start.no Thu Jan 28 10:47:33 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 28 Jan 2010 16:47:33 +0100 Subject: is there any alternative to savefig? In-Reply-To: <887d573b-a32b-4cf7-a262-1675c971a316@m25g2000yqc.googlegroups.com> References: <887d573b-a32b-4cf7-a262-1675c971a316@m25g2000yqc.googlegroups.com> Message-ID: * kiwanuka: > On Jan 28, 12:29 pm, kiwanuka wrote: >> Hi all, >> >> I wonder if anyone knows any alternative function in pylab (or >> otherwise) that could be used to save an image. My problem is as >> follows: >> >> --------------- >> from pylab import * >> ... >> >> figure(1) >> fig1 = gca() >> figure(2) >> fig2 = gca() >> figure(3) >> fig3 = gca() >> >> for i,data_file in enumerate(data_file_list): >> time,x, y,x2, y2 = read_csv_file_4(open >> (data_file),elements=num_of_elements) >> fig1.plot(-x,-y,color=colours[i],label=labellist[i]) >> fig2.plot(time,-y,color=colours[i],label=labellist[i]) >> fig3.plot(time,-x,color=colours[i],label=labellist[i]) >> >> fig1.legend(loc='best') >> fig1.set_title("y1 - x1") >> fig1.set_ylabel("y1") >> fig1.set_xlabel("x1") >> #savefig("y1-x1.png") >> >> fig2.legend(loc='best') >> fig2.set_title("y1 - time") >> fig2.set_ylabel("y1") >> fig2.set_xlabel("time[s]") >> #savefig("y1-time.png") >> >> fig3.legend(loc='best') >> fig3.set_title("x1 - time") >> fig3.set_ylabel("x1") >> fig3.set_xlabel("time[s]") >> #savefig("x1-time.png") >> show() >> --------------------------- >> >> In the above code, I read multiple data files and plot three separate >> figures. Now I would like to save each of the figures to a file as the >> commented savefig satements suggest. The trouble is that if I >> uncomment all those savefig statements, I get three saved images all >> containing the plot belonging to figure(3), which was the last figure >> declared. >> >> I understand this to be happening because savefig will save the >> "current" figure, which in this case happens to be the last one >> declared. >> >> If I could do something like fig1.savefig("y1-x1.png") or savefig("y1- >> x1.png").fig1, this would solve the problem but I'm not aware of any >> such methods or modules to enable this. This is thus a flaw in the >> general design/implementation of the savefig function, but is there an >> alternative function to enable me achieve what I need? Is there >> perhaps a possible tweak to savefig to make it do the same? >> >> Thanks in advance, >> >> Robert > > Problem solved. That's nice, but would you mind telling the group the solution? Assuming that you haven't (I don't see all messages posted to the e-mail list, only what's propagated to Usenet or originating there). Cheers, - Alf From karthikramaswamy88 at gmail.com Thu Jan 28 10:50:31 2010 From: karthikramaswamy88 at gmail.com (evilweasel) Date: Thu, 28 Jan 2010 07:50:31 -0800 (PST) Subject: Need help with a program References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: I will make my question a little more clearer. I have close to 60,000 lines of the data similar to the one I posted. There are various numbers next to the sequence (this is basically the number of times the sequence has been found in a particular sample). So, I would need to ignore the ones containing '0' and write all other sequences (excluding the number, since it is trivial) in a new text file, in the following format: >seq59902 TTTTTTTATAAAATATATAGT >seq59903 TTTTTTTATTTCTTGGCGTTGT >seq59904 TTTTTTTGGTTGCCCTGCGTGG >seq59905 TTTTTTTGTTTATTTTTGGG The number next to 'seq' is the line number of the sequence. When I run the above program, what I expect is an output file that is similar to the above output but with the ones containing '0' ignored. But, I am getting all the sequences printed in the file. Kindly excuse the 'newbieness' of the program. :) I am hoping to improve in the next few months. Thanks to all those who replied. I really appreciate it. :) From rdv at roalddevries.nl Thu Jan 28 11:01:38 2010 From: rdv at roalddevries.nl (Roald de Vries) Date: Thu, 28 Jan 2010 17:01:38 +0100 Subject: Symbols as parameters? In-Reply-To: <4CD97E94-B40E-481D-B367-5FCE2820F48D@roalddevries.nl> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <4CD97E94-B40E-481D-B367-5FCE2820F48D@roalddevries.nl> Message-ID: On Jan 22, 2010, at 11:56 AM, Roald de Vries wrote: > Hi Martin, > > On Jan 21, 2010, at 8:43 AM, Martin Drautzburg wrote: >> Hello all, >> >> When passing parameters to a function, you sometimes need a paramter >> which can only assume certain values, e.g. >> >> def move (direction): >> ... >> If direction can only be "up", "down", "left" or "right", you can >> solve >> this by passing strings, but this is not quite to the point: >> >> - you could pass invalid strings easily >> - you need to quote thigs, which is a nuisance >> - the parameter IS REALLY NOT A STRING, but a direction >> >> Alternatively you could export such symbols, so when you "import *" >> you >> have them available in the caller's namespace. But that forces you >> to "import *" which pollutes your namespace. >> >> What I am really looking for is a way >> >> - to be able to call move(up) >> - having the "up" symbol only in the context of the function >> call >> >> So it should look something like this >> >> ... magic, magic ... >> move(up) >> ... unmagic, unmagic ... >> print up >> >> This should complain that "up" is not defined during the "print" >> call, >> but not when move() is called. And of course there should be as >> little >> magic as possible. >> >> Any way to achieve this? > > You could do something like this: > > class Move(object): > def __call__(self, direction): > print(direction) > return 0 > > def up(self): > return self('up') > > move = Move() > > Now move.up() means move('up'), and you can obviously do similar > things for other directions. Question out of general interest in the language: If I would want to generate such functions in a for-loop, what would I have to do? This doesn't work: class Move(object): def __call__(self, direction): return direction move = Move() for f in ['up', 'down', 'right', 'left']: move.__dict__[f] = lambda: move(f) ... because now 'move.up()' returns 'left' because thats the current value of f. Is there a way to 'expand' f in the loop? Or a reason that you never should use this? From MLMDev at Comcast.net Thu Jan 28 11:03:20 2010 From: MLMDev at Comcast.net (Mitchell L Model) Date: Thu, 28 Jan 2010 11:03:20 -0500 Subject: python 3's adoption In-Reply-To: References: Message-ID: <2021A225-2BEE-4DE9-AE88-B9199E5D4491@Comcast.net> I have been working with Python 3 for over a year. I used it in writing my book "Bioinformatics Programming Using Python" (http://oreilly.com/catalog/9780596154509 ). I didn't see any point in teaching an incompatible earlier version of a language in transition. In preparing the book and its examples I explored a large number of Python modules in some depth and encountered most of the differences between the language and libraries of Python 2 and Python 3. The change was a bit awkward for a while, and there were some surprises, but in the end I have found nothing in Python 3 for which I would prefer Python 2's version. Removal of old-style classes is a big win. Having print as a function provides a tremendous amount of flexibility. I use the sep and end keywords all the time. There is no reason for print to be a statement, and it was an awkward inconsistency in a language that leans towards functional styles. Likewise the elimination of cmp, while shocking, leads to much simpler comparison arguments to sort, since all the function does is return a key; then, sort uses __lt__ (I think) so it automatically uses each class's definition of that. The weird objects returned from things like sorted, dict.keys/values/items, and so on are values that in practice are used primarily in iterations; you can always turn the result into a list, though I have to admit that while developing and debugging I trip trying to pick out a specific element from one of these using indexing (typically [0]); I've learned to think of them as generators, even though they aren't. The rearrangements and name changes in the libraries are quite helpful. I could go on, but basically the language and library changes are on the whole large improvements with little, if any, downside. Conversion of old code is greatly facilitied by the 2to3 tool that comes with Python 3. The big issue in moving from 2 to 3 is the external libraries and development tools you use. Different IDEs have released versions that support Python 3 at different times. (I believe Wing was the first.) If you use numpy, for example, or one of the many libraries that require it, you are stuck. Possibly some important facilities will never be ported to Python 3, but probably most active projects will eventually produce a Python 3 version -- for example, according to its web page, a Python 3 version of PIL is on the way. I was able to cover all the topics in my book using only Python library modules, something I felt would be best for readers -- I used libraries such as elementree, sqlite3, and tkinter. The only disappointment was that I couldn't include a chapter on BioPython, since there is no Python 3 version. By now, many large facilities support both Python 2 and Python 3. I am currently building a complex GUI/Visualization application based on the Python 3 version of PyQt4 and Wing IDE and am delighted with all of it. It may well be that some very important large From aahz at pythoncraft.com Thu Jan 28 11:06:33 2010 From: aahz at pythoncraft.com (Aahz) Date: 28 Jan 2010 08:06:33 -0800 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: In article , Steven D'Aprano wrote: >On Wed, 27 Jan 2010 16:25:46 -0500, Benjamin Kaplan wrote: >> >> When Python 2.6 came out, Jython was still on 2.2. The difference >> between 2.2 and 2.6 is almost as big of a difference as between 2.6 and >> 3.0. In that time, you had the introduction of the boolean type, >> generators, list comprehensions, the addition of the "yield" and "with" >> keywords, universal newline support, and decorators in addition to the >> large number of changes to the standard library such as the introduction >> of the subprocess module. > >I believe that, with the possible exception of the change from byte >strings to unicode strings, virtually *all* the hoo-har over Python 3 is >simply due to the tactical mistake of Guido and the Python Dev team of >*calling* Python 3 a backward incompatible release. Python has had >previous major changes in the past (e.g. 1.5 to 2.0 and 2.1 to 2.2) and >hardly anyone made a complaint. But as Steven points out, the difference from 2.2 to 2.6 is roughly the same as 2.6 to 3.1. Python has never before had such a large difference from one release to the next, and I think few people try to support serious apps on the full range from 2.2 to 2.6. Moreover, the task of using a single codebase without 2to3 is much easier in 1.4 through 2.6. Admittedly, it wouldn't be much fun to write 1.4 code these days without all the neat features that have been added, but you can't argue that it's hard. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From allah.27 at gmail.com Thu Jan 28 11:09:50 2010 From: allah.27 at gmail.com (talal awadh) Date: Thu, 28 Jan 2010 08:09:50 -0800 (PST) Subject: Good Book Message-ID: <7dbf7dff-cf08-463c-b7ff-ef83ff65a304@36g2000yqu.googlegroups.com> Many people who want to learn Islam or are new converts find it hard to have a simplified guide that explains to them the basics of Islam in a nutshell; so I decided to collect the basic guidelines and gather them in an e-book I named it "Basic Islam" for Introducing Islam http://www.saaid.net/book/9/2012.doc From roy at panix.com Thu Jan 28 11:09:58 2010 From: roy at panix.com (Roy Smith) Date: Thu, 28 Jan 2010 11:09:58 -0500 Subject: python 3's adoption References: Message-ID: In article , Mitchell L Model wrote: > I use the sep and end keywords all the time. What are 'sep' and 'end'? I'm looking in http://docs.python.org/3.1/genindex-all.html and don't see those mentioned at all. Am I just looking in the wrong place? From aahz at pythoncraft.com Thu Jan 28 11:10:15 2010 From: aahz at pythoncraft.com (Aahz) Date: 28 Jan 2010 08:10:15 -0800 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> Message-ID: In article , Neil Hodgson wrote: >Carl Banks: >> >> There is also no hope someone will fork Python 2.x and continue it in >> perpetuity. Well, someone might try to fork it, but they won't be >> able to call it Python. > > Over time there may be more desire from those unable or unwilling to >upgrade to 3.x to work on improvements to 2.x, perhaps leading to a >version 2.8. One of the benefits of open source is that you are not >trapped into following vendor decisions like Microsoft abandoning >classic VB in favour of VB.NET. > > It would be unreasonable for the core developers to try to block >this. Refusing use of the Python trademark for a version that was >reasonably compatible in both directions would be particularly petty. Agreed, and as a PSF member, I'd certainly be opposed to anyone trying to prevent the release of Python 2.8, and I would actively favor providing PSF and python.org resources to them. OTOH, I would also be likely to push anyone working on Python 2.8 to come up with a solid release plan first. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From pruebauno at latinmail.com Thu Jan 28 11:13:09 2010 From: pruebauno at latinmail.com (nn) Date: Thu, 28 Jan 2010 08:13:09 -0800 (PST) Subject: Need help with a program References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: On Jan 28, 10:50?am, evilweasel wrote: > I will make my question a little more clearer. I have close to 60,000 > lines of the data similar to the one I posted. There are various > numbers next to the sequence (this is basically the number of times > the sequence has been found in a particular sample). So, I would need > to ignore the ones containing '0' and write all other sequences > (excluding the number, since it is trivial) in a new text file, in the > following format: > > >seq59902 > > TTTTTTTATAAAATATATAGT > > >seq59903 > > TTTTTTTATTTCTTGGCGTTGT > > >seq59904 > > TTTTTTTGGTTGCCCTGCGTGG > > >seq59905 > > TTTTTTTGTTTATTTTTGGG > > The number next to 'seq' is the line number of the sequence. When I > run the above program, what I expect is an output file that is similar > to the above output but with the ones containing '0' ignored. But, I > am getting all the sequences printed in the file. > > Kindly excuse the 'newbieness' of the program. :) I am hoping to > improve in the next few months. Thanks to all those who replied. I > really appreciate it. :) People have already given you some pointers to your problem. In the end you will have to "tweak the details" because only you have access to the data not us. Just as example here is another way to do what you are doing: with open('dnain.dat') as infile, open('dnaout.dat','w') as outfile: partgen=(line.split() for line in infile) dnagen=(str(i+1)+'\n'+part[0]+'\n' for i,part in enumerate(partgen) if len(part)>1 and part[1]!='0') outfile.writelines(dnagen) From alfps at start.no Thu Jan 28 11:13:22 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 28 Jan 2010 17:13:22 +0100 Subject: python 3's adoption In-Reply-To: References: Message-ID: * Roy Smith: > In article , > Mitchell L Model wrote: > >> I use the sep and end keywords all the time. > > What are 'sep' and 'end'? I'm looking in > http://docs.python.org/3.1/genindex-all.html and don't see those mentioned > at all. Am I just looking in the wrong place? >>> print( print.__doc__ ) print(value, ..., sep=' ', end='\n', file=sys.stdout) Prints the values to a stream, or to sys.stdout by default. Optional keyword arguments: file: a file-like object (stream); defaults to the current sys.stdout. sep: string inserted between values, default a space. end: string appended after the last value, default a newline. >>> _ Cheers & hth., - Alf From dreadpiratejeff at gmail.com Thu Jan 28 11:34:15 2010 From: dreadpiratejeff at gmail.com (J) Date: Thu, 28 Jan 2010 11:34:15 -0500 Subject: Good Book In-Reply-To: <7dbf7dff-cf08-463c-b7ff-ef83ff65a304@36g2000yqu.googlegroups.com> References: <7dbf7dff-cf08-463c-b7ff-ef83ff65a304@36g2000yqu.googlegroups.com> Message-ID: <36dec4ff1001280834n41aa5268o13d8bfe8e0acffc8@mail.gmail.com> On Thu, Jan 28, 2010 at 11:09, talal awadh wrote: > Many people who want to learn Islam or are new converts find it hard I just wanted to thank you for reminding me that I needed to write a mail filter to delete this kind of drivel. I appreciate the reminder! Cheers, Jeff From lie.1296 at gmail.com Thu Jan 28 11:35:36 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Fri, 29 Jan 2010 03:35:36 +1100 Subject: python 3's adoption In-Reply-To: <7xy6ji62vx.fsf@ruckus.brouhaha.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> <7xockfjcsv.fsf@ruckus.brouhaha.com> <6b89dc67-ceee-4b83-a11f-c7376185dcd1@b9g2000pri.googlegroups.com> <7xy6ji62vx.fsf@ruckus.brouhaha.com> Message-ID: <4b61bd02@dnews.tpgi.com.au> On 01/28/10 19:37, Paul Rubin wrote: > Jonathan Gardner writes: >> If you're going to have statements, you're going to need the null >> statement. That's "pass". > > Why? Expressions are statements, so you could just say "pass" (in > quotes, denoting a string literal), or 0, or None, os anything else like > that, instead of having a special statement. or, if the null statement "pass" is removed, you could define: pass = None or pass = object() # sentinel and have essentially the same thing. hmm... on second thought, not special-casing pass means doing a LOAD_GLOBAL or LOAD_CONST for operation that is supposed to be doing nothing. From dinov at microsoft.com Thu Jan 28 11:37:24 2010 From: dinov at microsoft.com (Dino Viehland) Date: Thu, 28 Jan 2010 16:37:24 +0000 Subject: myths about python 3 In-Reply-To: <4b614417$0$6715$9b4e6d93@newsspool2.arcor-online.net> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <4b614417$0$6715$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <1A472770E042064698CB5ADC83A12ACD34CAEAB2@TK5EX14MBXC116.redmond.corp.microsoft.com> Stefan wrote: > >From an implementors point of view, it's actually quite the opposite. Most > syntax features of Python 3 can be easily implemented on top of an existing > Py2 Implementation (we have most of them in Cython already, and I really > found them fun to write), and the shifting-around in the standard library > can hardly be called non-trivial. All the hard work that went into the > design of CPython 3.x (and into its test suite) now makes it easy to just > steal from what's there already. > > The amount of work that the Jython project put into catching up from 2.1 to > 2.5/6 (new style classes! generators!) is really humongous compared to the > adaptations that an implementation needs to do to support Python 3 code. I > have great respect for the Jython project for what they achieved in the > last couple of years. (I also have great respect for the IronPython project > for fighting the One Microsoft Way into opening up, but that's a different > kind of business.) > > If there was enough interest from the respective core developers, I > wouldn't be surprised if we had more than one 'mostly compatible' > alternative Python 3 implementation in a couple of months. But it's the > obvious vicious circle business. As long as there aren't enough important > users of Py3, alternative implementations won't have enough incentives to > refocus their scarce developer time. Going for 2.6/7 first means that most > of the Py3 work gets done anyway, so it'll be even easier then. That makes > 2.6->2.7->3.2/3 the most natural implementation path. (And that, again, > makes it a *really* good decision that 2.7 will be the last 2.x release line.) I just want to echo this as I completely agree. Last time I went through the list it looked like there were around 10 major new features (some of them even not so major) that we needed to implement to bring IronPython up to the 3.0 level. It shouldn't be too time consuming, and it greatly improves our compatibility by finally having the same string types, but our users don't yet want us to stop supporting 2.x. From solipsis at pitrou.net Thu Jan 28 11:52:19 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 28 Jan 2010 16:52:19 +0000 (UTC) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: Le Thu, 28 Jan 2010 00:19:24 +0000, Steven D'Aprano a ?crit?: > 4. Python 3 will make you irresistible to women. > > FALSE - Python 3 coders are no more likely to get a date than any > other programmer. They spend less time coding, so they /can/ get more "dates" (what a strange English word) :-) Those dates don't have to be with women of course. From lie.1296 at gmail.com Thu Jan 28 11:54:26 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Fri, 29 Jan 2010 03:54:26 +1100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: <4b61c166$1@dnews.tpgi.com.au> On 01/28/10 20:12, Alf P. Steinbach wrote: > * Steven D'Aprano: >> On Wed, 27 Jan 2010 18:29:25 +0100, Alf P. Steinbach wrote:> >> Instead of: >> >> print >>fileObj, x, y, z >> >> you use regular function syntax with a meaningful keyword: >> >> print(x, y, z, file=fileObj) >> >> If you want suppress the newline at the end of each print: >> >> print x, y, z, # note the final comma >> >> compared to: >> >> print(x, y, z, end='') > > Actually I thought the final comma thing was nice. It was like Basic. I > think the 2.x 'print' must have been modeled on Basic's 'print'. if that was true, then python missed the final semicolon >> If you want to change the space between elements, instead of: >> >> sys.stdout.write(str(x) + "*" + str(y) + "*" + str(z) + '\n') >> >> you use: >> >> print(x, y, z, sep='*') >> >> >> If you want to override the behaviour of print in a module, instead of >> having to edit the source code of the module (which might not even be >> available), all you need to do is monkey-patch it: >> >> import module >> module.print = myprint > > >>> import builtins > >>> > >>> org_print = print > >>> builtins.print = 666 > >>> > >>> print( "trallala" ) > Traceback (most recent call last): > File "", line 1, in > TypeError: 'int' object is not callable > >>> org_print( "but is that really so smart?" ) > but is that really so smart? > >>> _ Monkey patching follows (or should follow) the same rule as class inheritance overriding: the overrider's input domain must be a superset of the overriden's input domain and the overrider's output range must be a subset of the overriden's output range. 666 object (int) is not even remotely compatible with function object. From solipsis at pitrou.net Thu Jan 28 11:55:29 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 28 Jan 2010 16:55:29 +0000 (UTC) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: Le Wed, 27 Jan 2010 17:36:29 -0800, alex23 a ?crit?: > > I've been a big supporter of Py3 from the beginning, but this repeated > claim of US becoming the mainline interpreter for 3.x pretty much kills > dead a lot of my interest. As long as the U-S JIT can be disabled at compile-time (and also at runtime), I don't think there's much of a contention actually. The other changes probably aren't controversial, although I haven't looked at them. Antoine. From arnodel at googlemail.com Thu Jan 28 12:00:54 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Thu, 28 Jan 2010 17:00:54 +0000 Subject: Need help with a program References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: nn writes: > On Jan 28, 10:50?am, evilweasel wrote: >> I will make my question a little more clearer. I have close to 60,000 >> lines of the data similar to the one I posted. There are various >> numbers next to the sequence (this is basically the number of times >> the sequence has been found in a particular sample). So, I would need >> to ignore the ones containing '0' and write all other sequences >> (excluding the number, since it is trivial) in a new text file, in the >> following format: >> >> >seq59902 >> >> TTTTTTTATAAAATATATAGT >> >> >seq59903 >> >> TTTTTTTATTTCTTGGCGTTGT >> >> >seq59904 >> >> TTTTTTTGGTTGCCCTGCGTGG >> >> >seq59905 >> >> TTTTTTTGTTTATTTTTGGG >> >> The number next to 'seq' is the line number of the sequence. When I >> run the above program, what I expect is an output file that is similar >> to the above output but with the ones containing '0' ignored. But, I >> am getting all the sequences printed in the file. >> >> Kindly excuse the 'newbieness' of the program. :) I am hoping to >> improve in the next few months. Thanks to all those who replied. I >> really appreciate it. :) > > People have already given you some pointers to your problem. In the > end you will have to "tweak the details" because only you have access > to the data not us. > > Just as example here is another way to do what you are doing: > > with open('dnain.dat') as infile, open('dnaout.dat','w') as outfile: > partgen=(line.split() for line in infile) > dnagen=(str(i+1)+'\n'+part[0]+'\n' > for i,part in enumerate(partgen) > if len(part)>1 and part[1]!='0') > outfile.writelines(dnagen) I think that generator expressions are overrated :) What's wrong with: with open('dnain.dat') as infile, open('dnaout.dat','w') as outfile: for i, line in enumerate(infile): parts = line.split() if len(parts) > 1 and parts[1] != '0': outfile.write(">seq%s\n%s\n" % (i+1, parts[0])) (untested) -- Arnaud From jjposner at optimum.net Thu Jan 28 12:23:36 2010 From: jjposner at optimum.net (John Posner) Date: Thu, 28 Jan 2010 12:23:36 -0500 Subject: Need help with a program In-Reply-To: References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: <4B61C818.3040302@optimum.net> On 1/28/2010 10:50 AM, evilweasel wrote: > I will make my question a little more clearer. I have close to 60,000 > lines of the data similar to the one I posted. There are various > numbers next to the sequence (this is basically the number of times > the sequence has been found in a particular sample). So, I would need > to ignore the ones containing '0' and write all other sequences > (excluding the number, since it is trivial) in a new text file, in the > following format: > >> seq59902 > TTTTTTTATAAAATATATAGT > >> seq59903 > TTTTTTTATTTCTTGGCGTTGT > >> seq59904 > TTTTTTTGGTTGCCCTGCGTGG > >> seq59905 > TTTTTTTGTTTATTTTTGGG > > The number next to 'seq' is the line number of the sequence. When I > run the above program, what I expect is an output file that is similar > to the above output but with the ones containing '0' ignored. But, I > am getting all the sequences printed in the file. > > Kindly excuse the 'newbieness' of the program. :) I am hoping to > improve in the next few months. Thanks to all those who replied. I > really appreciate it. :) Your program is a good first try. It contains a newbie error (looking for the number 0 instead of the string "0"). But more importantly, you're doing too much work yourself, rather than letting Python do the heavy lifting for you. These practices and tools make life a lot easier: * As others have noted, don't accumulate output in a list. Just write data to the output file line-by-line. * You don't need to initialize every variable at the beginning of the program. But there's no harm in it. * Use the enumerate() function to provide a line counter: for counter, line in enumerate(file1): This eliminates the need to accumulate output data in a list, then use the index variable "j" as the line counter. * Use string formatting. Each chunk of output is a two-line string, with the line-counter and the DNA sequence as variables: outformat = """seq%05d %s """ ... later, inside your loop ... resultsfile.write(outformat % (counter, sequence)) HTH, John From ethan at stoneleaf.us Thu Jan 28 12:35:53 2010 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 28 Jan 2010 09:35:53 -0800 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <4B61CAF9.6090701@stoneleaf.us> Steven D'Aprano wrote: > 4. Python 3 will make you irresistible to women. > > FALSE What?!? Drat!!! Guess I'll have to learn Lisp... ;) ~Ethan~ From briandenzer at gmail.com Thu Jan 28 12:36:52 2010 From: briandenzer at gmail.com (Brian D) Date: Thu, 28 Jan 2010 09:36:52 -0800 (PST) Subject: Stuck on a three word street name regex References: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> <0a712e20-facf-4a9b-8852-6136a8bad7dc@n7g2000yqb.googlegroups.com> <4e222e90-e088-4a82-a7d2-c62f509e2517@g29g2000yqe.googlegroups.com> <1b7dba4d-3f12-4d31-8e9a-b36cd4f40217@u41g2000yqe.googlegroups.com> Message-ID: <82c1cf49-eeff-42a1-96fd-012ee50922f2@c29g2000yqd.googlegroups.com> > Correction: > > [snip] the expression "parts[1 : -1]" means gather list items from the > second element in the list (index value 1) to one index position > before the end of the list. [snip] MRAB's solution was deserving of a more complete solution: >>> def parse_address(address): # Handles poorly-formatted addresses: # 100 RAMPART S ST -- direction in wrong position # 45 JOHN CHURCHILL CHASE ST -- two spaces before type #addresslist = ['num', 'dir', 'name', 'type'] addresslist = ['', '', '', ''] parts = address.split() if parts[-2] in ('E', 'W', 'N', 'S'): addresslist[1] = parts[-2] addresslist[2] = ' '.join(parts[1 : -2]) else: addresslist[2] = ' '.join(parts[1 : -1]) addresslist[0] = parts[0] addresslist[3] = parts[-1] return addresslist >>> parse_address('45 John Churchill Chase N St') ['45', 'N', 'John Churchill Chase', 'St'] >>> parse_address('45 John Churchill Chase St') ['45', '', 'John Churchill Chase', 'St'] From alfps at start.no Thu Jan 28 12:37:56 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 28 Jan 2010 18:37:56 +0100 Subject: python 3's adoption In-Reply-To: <4b61c166$1@dnews.tpgi.com.au> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <4b61c166$1@dnews.tpgi.com.au> Message-ID: * Lie Ryan: > On 01/28/10 20:12, Alf P. Steinbach wrote: >> >>> import builtins >> >>> >> >>> org_print = print >> >>> builtins.print = 666 >> >>> >> >>> print( "trallala" ) >> Traceback (most recent call last): >> File "", line 1, in >> TypeError: 'int' object is not callable >> >>> org_print( "but is that really so smart?" ) >> but is that really so smart? >> >>> _ > > Monkey patching follows (or should follow) the same rule as class > inheritance overriding: the overrider's input domain must be a superset > of the overriden's input domain and the overrider's output range must be > a subset of the overriden's output range. 666 object (int) is not even > remotely compatible with function object. Yes, that's my point. A 'print' replacement should ideally provide all the guarantees on behavior that standard 'print' does. And with that it's not so easy to put in a /correct/ replacement of 'print'; in particular, it has to honor the 'file' keyword argument. Thus the 3.x design makes it easy to replace 'print' incorrectly. I'd rather still had 'print' as keyword... ;-) Cheers, - Alf From jeanmichel at sequans.com Thu Jan 28 12:49:02 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Thu, 28 Jan 2010 18:49:02 +0100 Subject: Need help with a program In-Reply-To: References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: <4B61CE0E.5000801@sequans.com> evilweasel wrote: > I will make my question a little more clearer. I have close to 60,000 > lines of the data similar to the one I posted. There are various > numbers next to the sequence (this is basically the number of times > the sequence has been found in a particular sample). So, I would need > to ignore the ones containing '0' and write all other sequences > (excluding the number, since it is trivial) in a new text file, in the > following format: > > >> seq59902 >> > TTTTTTTATAAAATATATAGT > > >> seq59903 >> > TTTTTTTATTTCTTGGCGTTGT > > >> seq59904 >> > TTTTTTTGGTTGCCCTGCGTGG > > >> seq59905 >> > TTTTTTTGTTTATTTTTGGG > > The number next to 'seq' is the line number of the sequence. When I > run the above program, what I expect is an output file that is similar > to the above output but with the ones containing '0' ignored. But, I > am getting all the sequences printed in the file. > > Kindly excuse the 'newbieness' of the program. :) I am hoping to > improve in the next few months. Thanks to all those who replied. I > really appreciate it. :) > Using regexp may increase readability (if you are familiar with it). What about import re output = open("sequences1.txt", 'w') for index, line in enumerate(open(sys.argv[1], 'r')): match = re.match('(?P[GATC]+)\s+1') if match: output.write('seq%s\n%s\n' % (index, match.group('sequence'))) Jean-Michel From briandenzer at gmail.com Thu Jan 28 12:50:37 2010 From: briandenzer at gmail.com (Brian D) Date: Thu, 28 Jan 2010 09:50:37 -0800 (PST) Subject: Stuck on a three word street name regex References: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> <4b619f12$1@dnews.tpgi.com.au> Message-ID: <3b3b1880-70f1-4a18-994c-40c3d6ba7f85@f12g2000yqn.googlegroups.com> On Jan 28, 8:27?am, Lie Ryan wrote: > On 01/28/10 11:28, Brian D wrote: > > > > > I've tackled this kind of problem before by looping through a patterns > > dictionary, but there must be a smarter approach. > > > Two addresses. Note that the first has incorrectly transposed the > > direction and street name. The second has an extra space in it before > > the street type. Clearly done by someone who didn't know how to > > concatenate properly -- or didn't care. > > > 1000 RAMPART S ST > > > 100 JOHN CHURCHILL CHASE ?ST > > > I want to parse the elements into an array of values that can be > > inserted into new database fields. > > > Anyone who loves solving these kinds of puzzles care to relieve my > > frazzled brain? > > > The pattern I'm using doesn't keep the "CHASE" with the "JOHN > > CHURCHILL": > > How does the following perform? > > pat = > re.compile(r'(?P\d+)\s+(?P[A-Z\s]+)\s+(?PN|S|W|E|)\s+(?PST|RD|AVE?|)$') > > or more legibly: > > pat = re.compile( > ? ? r''' > ? ? ? (?P ?\d+ ? ? ? ? ? ? ?) ?#M series of digits > ? ? ? \s+ > ? ? ? (?P [A-Z\s]+ ? ? ? ? ) ?#M one-or-more word > ? ? ? \s+ > ? ? ? (?P ?S?E|SW?|N?W|NE?| ) ?#O direction or nothing > ? ? ? \s+ > ? ? ? (?P ST|RD|AVE? ? ? ? ) ?#M street type > ? ? ? $ ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? #M END > ? ? ''', re.VERBOSE) Is that all? That little empty space after the "|" OR metacharacter? Wow. As a test, to create a failure, if I remove that last "|" metacharacter from the "N|S|W|E|" string (i.e., "N|S|W|E"), the match fails on addresses that do not have that malformed direction after the street name (e.g., '45 JOHN CHURCHILL CHASE ST') Very clever. I don't think I've ever seen documentation showing that little trick. Thanks for enlightening me! From darcy at druid.net Thu Jan 28 13:03:44 2010 From: darcy at druid.net (D'Arcy J.M. Cain) Date: Thu, 28 Jan 2010 13:03:44 -0500 Subject: Need help with a program In-Reply-To: <4B61CE0E.5000801@sequans.com> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> <4B61CE0E.5000801@sequans.com> Message-ID: <20100128130344.af14ef8e.darcy@druid.net> On Thu, 28 Jan 2010 18:49:02 +0100 Jean-Michel Pichavant wrote: > Using regexp may increase readability (if you are familiar with it). If you have a problem and you think that regular expressions are the solution then now you have two problems. Regex is really overkill for the OP's problem and it certainly doesn't improve readability. -- D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner. From tjreedy at udel.edu Thu Jan 28 13:06:52 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 28 Jan 2010 13:06:52 -0500 Subject: Library support for Python 3.x In-Reply-To: <7xk4v2g2f3.fsf@ruckus.brouhaha.com> References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> <7xk4v2g2f3.fsf@ruckus.brouhaha.com> Message-ID: On 1/28/2010 1:37 AM, Paul Rubin wrote: > David Cournapeau writes: >> That's not windows specific - most packages which distribute binary >> packages need to package binaries for every minor version (2.4, 2.5, >> etc...).... >> I doubt that's what Paul was referring to, though - he seemed more >> concern with API/language changes than ABI issues. > > I didn't realize the ABI situation was that unstable. I thought you > could just package up a .so or .dll and people could keep using it. I > tend to not want to use extension modules that are not in the stdlib, > and I guess this is another reason to keep staying away from them. My impression is that there is something 'special' about Windows (msvc) such that binaries compiled against x.y automatically do not work for x.y+1, even is the ABI is unchanged from Python's viewpoint. The point of my post that David responded to is that most Windows users have always been effectively dependent on 3rd party module/package developers to produce a new binary for each new version, whereas many *nix users could download the source and compile, or at least give it a go. tjr From tjreedy at udel.edu Thu Jan 28 13:20:02 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 28 Jan 2010 13:20:02 -0500 Subject: python 3's adoption In-Reply-To: <7xy6ji62vx.fsf@ruckus.brouhaha.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> <7xockfjcsv.fsf@ruckus.brouhaha.com> <6b89dc67-ceee-4b83-a11f-c7376185dcd1@b9g2000pri.googlegroups.com> <7xy6ji62vx.fsf@ruckus.brouhaha.com> Message-ID: On 1/28/2010 3:37 AM, Paul Rubin wrote: > Jonathan Gardner writes: >> If you're going to have statements, you're going to need the null >> statement. That's "pass". > > Why? Expressions are statements, so you could just say "pass" (in > quotes, denoting a string literal), or 0, or None, os anything else like > that, instead of having a special statement. As Python is currently compiled, you are right, pass is not needed. A string becomes the doc attribute, and becomes local var 0, but 0 is just ignored. I actually expected a load_const but that is now optimized away. I am not sure this was always true. Perhaps 'pass' is easier than '0' for mewcomers reading the tutorial, but I have no data. >>> def f(): '' >>> def g(): pass >>> def h(): 0 >>> from dis import dis >>> dis(f) 1 0 LOAD_CONST 1 (None) 3 RETURN_VALUE >>> dis(g) 1 0 LOAD_CONST 0 (None) 3 RETURN_VALUE >>> dis(h) 1 0 LOAD_CONST 0 (None) 3 RETURN_VALUE >>> f.__doc__ '' >>> g.__doc__ >>> Terry Jan Reedy From peloko45 at gmail.com Thu Jan 28 13:20:46 2010 From: peloko45 at gmail.com (Joan Miller) Date: Thu, 28 Jan 2010 10:20:46 -0800 (PST) Subject: Wrap a function Message-ID: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> I've to call to many functions with the format: >>> run("cmd") were "cmd" is a command with its arguments to pass them to the shell and run it, i.e. >>> run("pwd") or >>> run("ls /home") Does anybody knows any library to help me to avoid the use of the main quotes, and brackets? I would to use anything as: $ ls /home => run("ls /home") or, at least run pwd => run("pwd") From jeanmichel at sequans.com Thu Jan 28 13:21:42 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Thu, 28 Jan 2010 19:21:42 +0100 Subject: Need help with a program In-Reply-To: <20100128130344.af14ef8e.darcy@druid.net> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> <4B61CE0E.5000801@sequans.com> <20100128130344.af14ef8e.darcy@druid.net> Message-ID: <4B61D5B6.9010004@sequans.com> D'Arcy J.M. Cain wrote: > On Thu, 28 Jan 2010 18:49:02 +0100 > Jean-Michel Pichavant wrote: > >> Using regexp may increase readability (if you are familiar with it). >> > > If you have a problem and you think that regular expressions are the > solution then now you have two problems. Regex is really overkill for > the OP's problem and it certainly doesn't improve readability. > > It depends on the reader ability to understand a *simple* regexp. It is also strange to get such answer after taking so much precautions, so let me quote myself: "Using regexp *may* increase readability (*if* you are *familiar* with it)." I honestly find it quite readable in the sample code I provided and spares all the if-len-startwith-strip logic, but If the OP does not agree, fine with me. But there's no need to get certain that I'm completly wrong. JM From pavlovevidence at gmail.com Thu Jan 28 13:21:52 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Thu, 28 Jan 2010 10:21:52 -0800 (PST) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> Message-ID: On Jan 28, 8:10?am, a... at pythoncraft.com (Aahz) wrote: > In article , > Neil Hodgson ? wrote: > > >Carl Banks: > > >> There is also no hope someone will fork Python 2.x and continue it in > >> perpetuity. ?Well, someone might try to fork it, but they won't be > >> able to call it Python. > > > ? Over time there may be more desire from those unable or unwilling to > >upgrade to 3.x to work on improvements to 2.x, perhaps leading to a > >version 2.8. One of the benefits of open source is that you are not > >trapped into following vendor decisions like Microsoft abandoning > >classic VB in favour of VB.NET. > > > ? It would be unreasonable for the core developers to try to block > >this. Refusing use of the Python trademark for a version that was > >reasonably compatible in both directions would be particularly petty. > > Agreed, and as a PSF member, I'd certainly be opposed to anyone trying to > prevent the release of Python 2.8, and I would actively favor providing > PSF and python.org resources to them. ?OTOH, I would also be likely to > push anyone working on Python 2.8 to come up with a solid release plan > first. Well, I'd consider that an official release. Note that I didn't claim there was no hope PSF wouldn't change it's mind on 2.8. All I saying is that if PSF decides to shut down 2.x there's no hope of a rogue Python 2.x series replacing Python 3.x. Regardless of how magnaminous the people of PSF are, the unfortunate reality is that trademark owners are forced by the law to be "particularly petty". PSF's IP lawyer will advise not to allow unsanctioned fork of Python 2.7 to call itself Python 2.8. Carl Banks From howe.steven at gmail.com Thu Jan 28 13:28:54 2010 From: howe.steven at gmail.com (Steven Howe) Date: Thu, 28 Jan 2010 10:28:54 -0800 Subject: Need help with a program In-Reply-To: <4B61CE0E.5000801@sequans.com> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> <4B61CE0E.5000801@sequans.com> Message-ID: <4B61D766.70300@gmail.com> On 01/28/2010 09:49 AM, Jean-Michel Pichavant wrote: > evilweasel wrote: >> I will make my question a little more clearer. I have close to 60,000 >> lines of the data similar to the one I posted. There are various >> numbers next to the sequence (this is basically the number of times >> the sequence has been found in a particular sample). So, I would need >> to ignore the ones containing '0' and write all other sequences >> (excluding the number, since it is trivial) in a new text file, in the >> following format: >> >>> seq59902 >> TTTTTTTATAAAATATATAGT >> >>> seq59903 >> TTTTTTTATTTCTTGGCGTTGT >> >>> seq59904 >> TTTTTTTGGTTGCCCTGCGTGG >> >>> seq59905 >> TTTTTTTGTTTATTTTTGGG >> >> The number next to 'seq' is the line number of the sequence. When I >> run the above program, what I expect is an output file that is similar >> to the above output but with the ones containing '0' ignored. But, I >> am getting all the sequences printed in the file. >> >> Kindly excuse the 'newbieness' of the program. :) I am hoping to >> improve in the next few months. Thanks to all those who replied. I >> really appreciate it. :) > Using regexp may increase readability (if you are familiar with it). > What about > > import re > > output = open("sequences1.txt", 'w') > > for index, line in enumerate(open(sys.argv[1], 'r')): > match = re.match('(?P[GATC]+)\s+1') > if match: > output.write('seq%s\n%s\n' % (index, match.group('sequence'))) > > > Jean-Michel Finally! After ready 8 or 9 messages about find a line ending with '1', someone suggests Regex. It was my first thought. Steven From tjreedy at udel.edu Thu Jan 28 13:32:11 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 28 Jan 2010 13:32:11 -0500 Subject: python 3's adoption In-Reply-To: <2021A225-2BEE-4DE9-AE88-B9199E5D4491@Comcast.net> References: <2021A225-2BEE-4DE9-AE88-B9199E5D4491@Comcast.net> Message-ID: On 1/28/2010 11:03 AM, Mitchell L Model wrote: > I have been working with Python 3 for over a year. I used it in writing > my book "Bioinformatics Programming Using Python" > (http://oreilly.com/catalog/9780596154509). I didn't see any point in > teaching an incompatible earlier version of a language in transition. In > preparing the book and its examples I explored a large number of Python > modules in some depth and encountered most of the differences between > the language and libraries of Python 2 and Python 3. The change was a > bit awkward for a while, and there were some surprises, but in the end I > have found nothing in Python 3 for which I would prefer Python 2's version. > > Removal of old-style classes is a big win. Having print as a function > provides a tremendous amount of flexibility. I use the sep and end > keywords all the time. There is no reason for print to be a statement, > and it was an awkward inconsistency in a language that leans towards > functional styles. Likewise the elimination of cmp, while shocking, > leads to much simpler comparison arguments to sort, since all the > function does is return a key; then, sort uses __lt__ (I think) so it > automatically uses each class's definition of that. The weird objects > returned from things like sorted, dict.keys/values/items, and so on are > values that in practice are used primarily in iterations; you can always > turn the result into a list, though I have to admit that while > developing and debugging I trip trying to pick out a specific element > from one of these using indexing (typically [0]); I've learned to think > of them as generators, even though they aren't. The rearrangements and > name changes in the libraries are quite helpful. I could go on, but > basically the language and library changes are on the whole large > improvements with little, if any, downside. I agree completely. > Conversion of old code is greatly facilitied by the 2to3 tool that comes > with Python 3. The big issue in moving from 2 to 3 is the external > libraries and development tools you use. Different IDEs have released > versions that support Python 3 at different times. (I believe Wing was > the first.) If you use numpy, for example, or one of the many libraries > that require it, you are stuck. Possibly some important facilities will > never be ported to Python 3, but probably most active projects will > eventually produce a Python 3 version -- for example, according to its > web page, a Python 3 version of PIL is on the way. I was able to cover > all the topics in my book using only Python library modules, something I > felt would be best for readers -- I used libraries such as elementree, > sqlite3, and tkinter. The only disappointment was that I couldn't > include a chapter on BioPython, since there is no Python 3 version. > > By now, many large facilities support both Python 2 and Python 3. I am > currently building a complex GUI/Visualization application based on the > Python 3 version of PyQt4 and Wing IDE and am delighted with all of it. > It may well be that some very important large Something got clipped ;-) Anyway, thank you for the report. Terry Jan Reedy From mensanator at aol.com Thu Jan 28 13:38:17 2010 From: mensanator at aol.com (Mensanator) Date: Thu, 28 Jan 2010 10:38:17 -0800 (PST) Subject: Need help with a program References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> <4B61CE0E.5000801@sequans.com> Message-ID: <2e0ae97b-3aef-4ebf-afdf-0ed53c022b02@k19g2000yqc.googlegroups.com> On Jan 28, 12:28?pm, Steven Howe wrote: > On 01/28/2010 09:49 AM, Jean-Michel Pichavant wrote: > > > > > > > evilweasel wrote: > >> I will make my question a little more clearer. I have close to 60,000 > >> lines of the data similar to the one I posted. There are various > >> numbers next to the sequence (this is basically the number of times > >> the sequence has been found in a particular sample). So, I would need > >> to ignore the ones containing '0' and write all other sequences > >> (excluding the number, since it is trivial) in a new text file, in the > >> following format: > > >>> seq59902 > >> TTTTTTTATAAAATATATAGT > > >>> seq59903 > >> TTTTTTTATTTCTTGGCGTTGT > > >>> seq59904 > >> TTTTTTTGGTTGCCCTGCGTGG > > >>> seq59905 > >> TTTTTTTGTTTATTTTTGGG > > >> The number next to 'seq' is the line number of the sequence. When I > >> run the above program, what I expect is an output file that is similar > >> to the above output but with the ones containing '0' ignored. But, I > >> am getting all the sequences printed in the file. > > >> Kindly excuse the 'newbieness' of the program. :) I am hoping to > >> improve in the next few months. Thanks to all those who replied. I > >> really appreciate it. :) > > Using regexp may increase readability (if you are familiar with it). > > What about > > > import re > > > output = open("sequences1.txt", 'w') > > > for index, line in enumerate(open(sys.argv[1], 'r')): > > ? ?match = re.match('(?P[GATC]+)\s+1') > > ? ?if match: > > ? ? ? ?output.write('seq%s\n%s\n' % (index, match.group('sequence'))) > > > Jean-Michel > > Finally! > > After ready 8 or 9 messages about find a line ending with '1', someone > suggests Regex. > It was my first thought. And as a first thought, it is, of course, wrong. You don't want lines ending in '1', you want ANY non-'0' amount. Likewise, you don't want to exclude lines ending in '0' because you'll end up excluding counts of 10, 20, 30, etc. You need a regex that extracts ALL the numeric characters at the end of the line and exclude those that evaluate to 0. > > Steven From nagle at animats.com Thu Jan 28 13:38:25 2010 From: nagle at animats.com (John Nagle) Date: Thu, 28 Jan 2010 10:38:25 -0800 Subject: Why doesn't "filterwarnings" suppress this message? Message-ID: <4b61d5fb$0$1658$742ec2ed@news.sonic.net> Feedparser isn't supported for Python 3.0, so in Python 2.6, many warning messages appear. I'm trying, in Python 2.6, to suppress the warning message: ./feedparser\feedparser.py:69: DeprecationWarning: the sgmllib module has been removed in Python 3.0 import sgmllib, re, sys, copy, urlparse, time, rfc822, types, cgi, urllib, urllib2 with this warning filter: warnings.filterwarnings(action='ignore', category=DeprecationWarning, module='feedparser') The filter suppresses other deprecation warnings associated with the feedparser module, but not that one. How do I get rid of that one? And no, warnings.filterwarnings(action='ignore', category=DeprecationWarning, module='sgmllib') doesn't do it. John Nagle From __peter__ at web.de Thu Jan 28 14:01:51 2010 From: __peter__ at web.de (Peter Otten) Date: Thu, 28 Jan 2010 20:01:51 +0100 Subject: Why doesn't "filterwarnings" suppress this message? References: <4b61d5fb$0$1658$742ec2ed@news.sonic.net> Message-ID: John Nagle wrote: > Feedparser isn't supported for Python 3.0, so in Python 2.6, many > warning > messages appear. I'm trying, in Python 2.6, to suppress the warning > message: > > ./feedparser\feedparser.py:69: DeprecationWarning: > the sgmllib module has been removed in Python 3.0 > import sgmllib, re, sys, copy, urlparse, time, rfc822, types, > cgi, urllib, urllib2 > > with this warning filter: > > warnings.filterwarnings(action='ignore', > category=DeprecationWarning, module='feedparser') > > The filter suppresses other deprecation warnings associated with > the feedparser module, but not that one. How do I get rid of that > one? > > And no, > > warnings.filterwarnings(action='ignore', > category=DeprecationWarning, module='sgmllib') > > doesn't do it. > > John Nagle I can't confirm that: $ cat nowarning.py import sys if "--off" in sys.argv: import warnings warnings.filterwarnings(action="ignore", category=DeprecationWarning, module="feedparser") import feedparser $ python nowarning.py $ python -3 nowarning.py /usr/lib/python2.6/site.py:1: DeprecationWarning: The 'new' module has been removed in Python 3.0; use the 'types' module instead. """Append module search paths for third-party packages to sys.path. /usr/lib/pymodules/python2.6/feedparser.py:69: DeprecationWarning: the sgmllib module has been removed in Python 3.0 import sgmllib, re, sys, copy, urlparse, time, rfc822, types, cgi, urllib, urllib2 /usr/lib/pymodules/python2.6/feedparser.py:69: DeprecationWarning: in 3.x, rfc822 has been removed in favor of the email package import sgmllib, re, sys, copy, urlparse, time, rfc822, types, cgi, urllib, urllib2 $ python -3 nowarning.py --off /usr/lib/python2.6/site.py:1: DeprecationWarning: The 'new' module has been removed in Python 3.0; use the 'types' module instead. """Append module search paths for third-party packages to sys.path. $ python -V Python 2.6.4 Peter From jrh at joshh.co.uk Thu Jan 28 14:16:11 2010 From: jrh at joshh.co.uk (Josh Holland) Date: Thu, 28 Jan 2010 19:16:11 +0000 (UTC) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> Message-ID: On 2010-01-28, Joan Miller wrote: > I've to call to many functions with the format: > >>>> run("cmd") Check the docs on os.system(). -- Josh "dutchie" Holland http://www.joshh.co.uk/ http://twitter.com/jshholland http://identi.ca/jshholland From stu.doherty at gmail.com Thu Jan 28 14:17:49 2010 From: stu.doherty at gmail.com (Big Stu) Date: Thu, 28 Jan 2010 11:17:49 -0800 (PST) Subject: Great example of a python module/package following up to date conventions. Message-ID: <0876c5e7-e8c5-4334-a000-df310b056723@h2g2000yqj.googlegroups.com> I'm hoping someone on here can point me to an example of a python package that is a great example of how to put it all together. I'm hoping for example code that demonstrates: -Strict adherence to PEP 8 -thorough use of Docstrings -Conventional directory structure/package layout -Appropriate use of the latest accepted coding guidelines in the python community (e.g., new classes versus old classes, Python 3000 compatibility, newer language features, etc. etc.) -Some amount of object oriented design Bonus: -Unit tests -Logging mechanism I can't imagine a package that's been around longer than a few years will hit upon all these things well unless the maintainer went back and did some serious refactoring and re-tooling. Is this question possible to answer? From peloko45 at gmail.com Thu Jan 28 14:24:28 2010 From: peloko45 at gmail.com (Joan Miller) Date: Thu, 28 Jan 2010 11:24:28 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> Message-ID: On 28 ene, 19:16, Josh Holland wrote: > On 2010-01-28, Joan Miller wrote: > > > I've to call to many functions with the format: > > >>>> run("cmd") > > Check the docs on os.system(). No. I've a function that uses subprocess to run commands on the same shell and so substitute to bash scrips. But a script full of run ("shell_command --with --arguments") is too verbose. From jrh at joshh.co.uk Thu Jan 28 14:28:54 2010 From: jrh at joshh.co.uk (Josh Holland) Date: Thu, 28 Jan 2010 19:28:54 +0000 (UTC) Subject: Great example of a python module/package following up to date conventions. References: <0876c5e7-e8c5-4334-a000-df310b056723@h2g2000yqj.googlegroups.com> Message-ID: On 2010-01-28, Big Stu wrote: > I'm hoping someone on here can point me to an example of a python > package that is a great example of how to put it all together. I'm > hoping for example code that demonstrates: Surely most of the Standard Library should satisfy all your requirements? -- Josh "dutchie" Holland http://www.joshh.co.uk/ http://twitter.com/jshholland http://identi.ca/jshholland From no.i.dont at want.mail.from.spammers.com Thu Jan 28 14:31:27 2010 From: no.i.dont at want.mail.from.spammers.com (Fencer) Date: Thu, 28 Jan 2010 20:31:27 +0100 Subject: python 3's adoption In-Reply-To: References: Message-ID: <7se70fFd0mU1@mid.individual.net> On 2010-01-28 17:03, Mitchell L Model wrote: > I have been working with Python 3 for over a year. I used it in writing > my book "Bioinformatics Programming Using Python" > (http://oreilly.com/catalog/9780596154509). That book sounds very interesting, even though I am more interested in the bioinformatic parts, I already know some python. I'll show my boss the link, thanks! - Fencer From stu.doherty at gmail.com Thu Jan 28 14:35:43 2010 From: stu.doherty at gmail.com (Big Stu) Date: Thu, 28 Jan 2010 11:35:43 -0800 (PST) Subject: Great example of a python module/package following up to date conventions. References: <0876c5e7-e8c5-4334-a000-df310b056723@h2g2000yqj.googlegroups.com> Message-ID: <40409dfe-05df-4f4a-b2d0-d7527d650ecf@n7g2000yqb.googlegroups.com> On Jan 28, 2:28?pm, Josh Holland wrote: > On 2010-01-28, Big Stu wrote: > > > I'm hoping someone on here can point me to an example of a python > > package that is a great example of how to put it all together. ?I'm > > hoping for example code that demonstrates: > > Surely most of the Standard Library should satisfy all your > requirements? > > -- > Josh "dutchie" Holland http://www.joshh.co.uk/http://twitter.com/jshhollandhttp://identi.ca/jshholland That's definitely a place I've started to poke around, but the standard library stuff always comes to me by way of my standard python installation. I was hoping to have a template of a 3rd party package to follow. Complete with conventions to follow for easily packaging and distributing via the usual python channels (pypi, easy_install, egg, etc.). From exarkun at twistedmatrix.com Thu Jan 28 14:38:58 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Thu, 28 Jan 2010 19:38:58 -0000 Subject: Great example of a python module/package following up to date conventions. In-Reply-To: References: <0876c5e7-e8c5-4334-a000-df310b056723@h2g2000yqj.googlegroups.com> Message-ID: <20100128193858.1898.1910918008.divmod.xquotient.906@localhost.localdomain> On 07:28 pm, jrh at joshh.co.uk wrote: >On 2010-01-28, Big Stu wrote: >>I'm hoping someone on here can point me to an example of a python >>package that is a great example of how to put it all together. I'm >>hoping for example code that demonstrates: > >Surely most of the Standard Library should satisfy all your >requirements? Have you actually looked at any of the standard library? Jean-Paul From peloko45 at gmail.com Thu Jan 28 14:41:06 2010 From: peloko45 at gmail.com (Joan Miller) Date: Thu, 28 Jan 2010 11:41:06 -0800 (PST) Subject: Great example of a python module/package following up to date conventions. References: <0876c5e7-e8c5-4334-a000-df310b056723@h2g2000yqj.googlegroups.com> Message-ID: On 28 ene, 19:17, Big Stu wrote: > I'm hoping someone on here can point me to an example of a python > package that is a great example of how to put it all together. ?I'm > hoping for example code that demonstrates: > > -Strict adherence to PEP 8 > -thorough use of Docstrings > -Conventional directory structure/package layout > -Appropriate use of the latest accepted coding guidelines in the > python community (e.g., new classes versus old classes, Python 3000 > compatibility, newer language features, etc. etc.) > -Some amount of object oriented design > > Bonus: > -Unit tests > -Logging mechanism > > I can't imagine a package that's been around longer than a few years > will hit upon all these things well unless the maintainer went back > and did some serious refactoring and re-tooling. > > Is this question possible to answer? Look here: http://bitbucket.org/ares/scripy/src/ From jrh at joshh.co.uk Thu Jan 28 14:47:26 2010 From: jrh at joshh.co.uk (Josh Holland) Date: Thu, 28 Jan 2010 19:47:26 +0000 (UTC) Subject: Great example of a python module/package following up to date conventions. References: <0876c5e7-e8c5-4334-a000-df310b056723@h2g2000yqj.googlegroups.com> Message-ID: On 2010-01-28, exarkun at twistedmatrix.com wrote: > Have you actually looked at any of the standard library? Not recently or in depth, no. I would have thought that it would be of high quality. I must have been mistaken. -- Josh "dutchie" Holland http://www.joshh.co.uk/ http://twitter.com/jshholland http://identi.ca/jshholland From stu.doherty at gmail.com Thu Jan 28 14:49:13 2010 From: stu.doherty at gmail.com (Big Stu) Date: Thu, 28 Jan 2010 11:49:13 -0800 (PST) Subject: Great example of a python module/package following up to date conventions. References: <0876c5e7-e8c5-4334-a000-df310b056723@h2g2000yqj.googlegroups.com> Message-ID: > Have you actually looked at any of the standard library? > > Jean-Paul I'm looking at urllib2 right now and it is covering a bunch of the bases I'm looking for. And grepping in the /usr/lib/python2.5/ folder for import statements on various things I'm interested in is bringing up some good examples to check out as well. Given that I'm still fairly novice to this I'm not yet in the position to make a good judgment on what is and isn't a good python practice so I was hoping someone on here might be able to point at a module or 2 that has really done a good job of following best practices. Seems like a reasonable question with an answer that others in a similar position to me might find useful. From steve at holdenweb.com Thu Jan 28 14:51:48 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 28 Jan 2010 14:51:48 -0500 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> Message-ID: <4B61EAD4.3040102@holdenweb.com> Carl Banks wrote: > On Jan 28, 8:10 am, a... at pythoncraft.com (Aahz) wrote: >> In article , >> Neil Hodgson wrote: >> >>> Carl Banks: >>>> There is also no hope someone will fork Python 2.x and continue it in >>>> perpetuity. Well, someone might try to fork it, but they won't be >>>> able to call it Python. >>> Over time there may be more desire from those unable or unwilling to >>> upgrade to 3.x to work on improvements to 2.x, perhaps leading to a >>> version 2.8. One of the benefits of open source is that you are not >>> trapped into following vendor decisions like Microsoft abandoning >>> classic VB in favour of VB.NET. >>> It would be unreasonable for the core developers to try to block >>> this. Refusing use of the Python trademark for a version that was >>> reasonably compatible in both directions would be particularly petty. >> Agreed, and as a PSF member, I'd certainly be opposed to anyone trying to >> prevent the release of Python 2.8, and I would actively favor providing >> PSF and python.org resources to them. OTOH, I would also be likely to >> push anyone working on Python 2.8 to come up with a solid release plan >> first. > > Well, I'd consider that an official release. Note that I didn't claim > there was no hope PSF wouldn't change it's mind on 2.8. All I saying > is that if PSF decides to shut down 2.x there's no hope of a rogue > Python 2.x series replacing Python 3.x. > > Regardless of how magnaminous the people of PSF are, the unfortunate > reality is that trademark owners are forced by the law to be > "particularly petty". PSF's IP lawyer will advise not to allow > unsanctioned fork of Python 2.7 to call itself Python 2.8. > But if it were sanctioned ... ? We *are* pretty magnanimous ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Thu Jan 28 14:51:48 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 28 Jan 2010 14:51:48 -0500 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> Message-ID: <4B61EAD4.3040102@holdenweb.com> Carl Banks wrote: > On Jan 28, 8:10 am, a... at pythoncraft.com (Aahz) wrote: >> In article , >> Neil Hodgson wrote: >> >>> Carl Banks: >>>> There is also no hope someone will fork Python 2.x and continue it in >>>> perpetuity. Well, someone might try to fork it, but they won't be >>>> able to call it Python. >>> Over time there may be more desire from those unable or unwilling to >>> upgrade to 3.x to work on improvements to 2.x, perhaps leading to a >>> version 2.8. One of the benefits of open source is that you are not >>> trapped into following vendor decisions like Microsoft abandoning >>> classic VB in favour of VB.NET. >>> It would be unreasonable for the core developers to try to block >>> this. Refusing use of the Python trademark for a version that was >>> reasonably compatible in both directions would be particularly petty. >> Agreed, and as a PSF member, I'd certainly be opposed to anyone trying to >> prevent the release of Python 2.8, and I would actively favor providing >> PSF and python.org resources to them. OTOH, I would also be likely to >> push anyone working on Python 2.8 to come up with a solid release plan >> first. > > Well, I'd consider that an official release. Note that I didn't claim > there was no hope PSF wouldn't change it's mind on 2.8. All I saying > is that if PSF decides to shut down 2.x there's no hope of a rogue > Python 2.x series replacing Python 3.x. > > Regardless of how magnaminous the people of PSF are, the unfortunate > reality is that trademark owners are forced by the law to be > "particularly petty". PSF's IP lawyer will advise not to allow > unsanctioned fork of Python 2.7 to call itself Python 2.8. > But if it were sanctioned ... ? We *are* pretty magnanimous ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Thu Jan 28 14:54:09 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 28 Jan 2010 14:54:09 -0500 Subject: Wrap a function In-Reply-To: References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> Message-ID: Joan Miller wrote: > On 28 ene, 19:16, Josh Holland wrote: >> On 2010-01-28, Joan Miller wrote: >> >>> I've to call to many functions with the format: >>>>>> run("cmd") >> Check the docs on os.system(). > No. I've a function that uses subprocess to run commands on the same > shell and so substitute to bash scrips. But a script full of run > ("shell_command --with --arguments") is too verbose. So rewrite the script to read the commands from a file and execute them one by one? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From mrholtsr at sbcglobal.net Thu Jan 28 14:57:05 2010 From: mrholtsr at sbcglobal.net (Ray Holt) Date: Thu, 28 Jan 2010 14:57:05 -0500 Subject: Why am I getting this Error message Message-ID: <61E1CBCB55F4401AAEAA0976543E5C0E@ray> Why am I getting the following error message. Area has been declared as an attribute of Circle. Thanks, Ray class Circle: def __init__(self): self.radius = 1 def area(self): return self.radius * self.radius * 3.14159 c = Circle() c.radius = 3 print c.area() Traceback (most recent call last): File "", line 1, in class Circle: File "", line 8, in Circle print c.area() AttributeError: Circle instance has no attribute 'area' -------------- next part -------------- An HTML attachment was scrubbed... URL: From jjposner at optimum.net Thu Jan 28 14:58:50 2010 From: jjposner at optimum.net (John Posner) Date: Thu, 28 Jan 2010 14:58:50 -0500 Subject: Wrap a function In-Reply-To: References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> Message-ID: <4B61EC7A.7020406@optimum.net> On 1/28/2010 2:24 PM, Joan Miller wrote: > On 28 ene, 19:16, Josh Holland wrote: >> On 2010-01-28, Joan Miller wrote: >> >>> I've to call to many functions with the format: >> >>>>>> run("cmd") >> >> Check the docs on os.system(). > No. I've a function that uses subprocess to run commands on the same > shell and so substitute to bash scrips. But a script full of run > ("shell_command --with --arguments") is too verbose. I'm suspicious of your original intent. Essentially, you want to write code in which a literal string, such as ... ls -l ... is *not* enclosed in quotes. Why run the risk of creating confusion (in other people who look at your code, in syntax-checking tools, etc.) between variables and literals? But I'm in sympathy with your desire to make the code as clean as possible and to minimize the number of times you have to type a quote character. My suggestions: 1. Create a function (say, "Run") that encapsulates as much of the syntax as possible: os.system(), subprocess.call(), string-splitting, whatever. So an invocation would look like this: Run("ls -l *.txt") (I think you've already done this step.) 2. Find a text editor that supports keyboard macros, so that a single keystroke turns this text line: ls -l *.txt ... into this one: Run("ls -l *.txt") HTH, John From python at mrabarnett.plus.com Thu Jan 28 14:59:44 2010 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 28 Jan 2010 19:59:44 +0000 Subject: Need help with a program In-Reply-To: <4B61D766.70300@gmail.com> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> <4B61CE0E.5000801@sequans.com> <4B61D766.70300@gmail.com> Message-ID: <4B61ECB0.4030707@mrabarnett.plus.com> Steven Howe wrote: > On 01/28/2010 09:49 AM, Jean-Michel Pichavant wrote: >> evilweasel wrote: >>> I will make my question a little more clearer. I have close to 60,000 >>> lines of the data similar to the one I posted. There are various >>> numbers next to the sequence (this is basically the number of times >>> the sequence has been found in a particular sample). So, I would need >>> to ignore the ones containing '0' and write all other sequences >>> (excluding the number, since it is trivial) in a new text file, in the >>> following format: >>> >>>> seq59902 >>> TTTTTTTATAAAATATATAGT >>> >>>> seq59903 >>> TTTTTTTATTTCTTGGCGTTGT >>> >>>> seq59904 >>> TTTTTTTGGTTGCCCTGCGTGG >>> >>>> seq59905 >>> TTTTTTTGTTTATTTTTGGG >>> >>> The number next to 'seq' is the line number of the sequence. When I >>> run the above program, what I expect is an output file that is similar >>> to the above output but with the ones containing '0' ignored. But, I >>> am getting all the sequences printed in the file. >>> >>> Kindly excuse the 'newbieness' of the program. :) I am hoping to >>> improve in the next few months. Thanks to all those who replied. I >>> really appreciate it. :) >> Using regexp may increase readability (if you are familiar with it). >> What about >> >> import re >> >> output = open("sequences1.txt", 'w') >> >> for index, line in enumerate(open(sys.argv[1], 'r')): >> match = re.match('(?P[GATC]+)\s+1') >> if match: >> output.write('seq%s\n%s\n' % (index, match.group('sequence'))) >> >> >> Jean-Michel > > Finally! > > After ready 8 or 9 messages about find a line ending with '1', someone > suggests Regex. > It was my first thought. > I'm a great fan of regexes, but I never though of using them for this because it doesn't look like a regex type of problem to me. From peloko45 at gmail.com Thu Jan 28 15:06:04 2010 From: peloko45 at gmail.com (Joan Miller) Date: Thu, 28 Jan 2010 12:06:04 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> Message-ID: <012da98d-959e-4de4-a4a1-17ec9fdcc209@a5g2000yqi.googlegroups.com> On 28 ene, 19:54, Steve Holden wrote: > Joan Miller wrote: > > On 28 ene, 19:16, Josh Holland wrote: > >> On 2010-01-28, Joan Miller wrote: > > >>> I've to call to many functions with the format: > >>>>>> run("cmd") > >> Check the docs on os.system(). > > No. I've a function that uses subprocess to run commands on the same > > shell and so substitute to bash scrips. But a script full of run > > ("shell_command --with --arguments") is too verbose. > > So rewrite the script to read the commands from a file and execute them > one by one? > I had thinked about that but the problem is that I would that were mixed with python code, so can be get the output from a system command and manipulate it from python From exarkun at twistedmatrix.com Thu Jan 28 15:11:14 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Thu, 28 Jan 2010 20:11:14 -0000 Subject: Great example of a python module/package following up to date conventions. In-Reply-To: References: <0876c5e7-e8c5-4334-a000-df310b056723@h2g2000yqj.googlegroups.com> Message-ID: <20100128201114.1898.1971415063.divmod.xquotient.918@localhost.localdomain> On 07:49 pm, stu.doherty at gmail.com wrote: > >>Have you actually looked at any of the standard library? >> >>Jean-Paul > >I'm looking at urllib2 right now and it is covering a bunch of the >bases I'm looking for. And grepping in the /usr/lib/python2.5/ folder >for import statements on various things I'm interested in is bringing >up some good examples to check out as well. Given that I'm still >fairly novice to this I'm not yet in the position to make a good >judgment on what is and isn't a good python practice so I was hoping >someone on here might be able to point at a module or 2 that has >really done a good job of following best practices. > >Seems like a reasonable question with an answer that others in a >similar position to me might find useful. You're right. I was actually wondering if Josh had looked before suggesting it. :) The stdlib varies wildly in quality, with much of it not serving as a particular good example of most of the points you mentioned. urllib2 is probably better than a lot, but, for example, even it only manages about 75% line coverage by its test suite. Jean-Paul From peloko45 at gmail.com Thu Jan 28 15:13:14 2010 From: peloko45 at gmail.com (Joan Miller) Date: Thu, 28 Jan 2010 12:13:14 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> <4B61EC7A.7020406@optimum.net> Message-ID: On 28 ene, 19:58, John Posner wrote: > On 1/28/2010 2:24 PM, Joan Miller wrote: > > > On 28 ene, 19:16, Josh Holland ?wrote: > >> On 2010-01-28, Joan Miller ?wrote: > > >>> I've to call to many functions with the format: > > >>>>>> run("cmd") > > >> Check the docs on os.system(). > > No. I've a function that uses subprocess to run commands on the same > > shell and so substitute to bash scrips. But a script full of run > > ("shell_command --with --arguments") is too verbose. > > I'm suspicious of your original intent. Essentially, you want to write > code in which a literal string, such as ... > > ? ?ls -l > > ... is *not* enclosed in quotes. Why run the risk of creating confusion > (in other people who look at your code, in syntax-checking tools, etc.) > between variables and literals? Yes but to that code could be prepend a sign as '$' to be identified and so be parsed. > > But I'm in sympathy with your desire to make the code as clean as > possible and to minimize the number of times you have to type a quote > character. My suggestions: > > 1. Create a function (say, "Run") that encapsulates as much of the > syntax as possible: os.system(), subprocess.call(), string-splitting, > whatever. So an invocation would look like this: > > ? ?Run("ls -l *.txt") > > (I think you've already done this step.) Yes, I made a funtion very cool to call to system commands, that works well with pipes and passes the variables (i.e. "LANG=C grep -e 'foo' / home") > 2. Find a text editor that supports keyboard macros, so that a single > keystroke turns this text line: > > ? ?ls -l *.txt > > ... into this one: > > ? ?Run("ls -l *.txt") This is not what I'm looking for. I'm supposing that could be solved with a DSL or a macro library, any? From clp2 at rebertia.com Thu Jan 28 15:16:14 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Thu, 28 Jan 2010 12:16:14 -0800 Subject: Why am I getting this Error message In-Reply-To: <61E1CBCB55F4401AAEAA0976543E5C0E@ray> References: <61E1CBCB55F4401AAEAA0976543E5C0E@ray> Message-ID: <50697b2c1001281216y5f89145csd38f2a0e5fff0c70@mail.gmail.com> On Thu, Jan 28, 2010 at 11:57 AM, Ray Holt wrote: > Why am I getting the following error message. Area has been declared as an > attribute of Circle. Thanks, Ray > > class Circle: > ?def __init__(self): > ???? self.radius = 1 > ?def area(self): > ???? return self.radius * self.radius * 3.14159 > ?c = Circle() > ?c.radius = 3 > ?print c.area() > > Traceback (most recent call last): > ? File "", line 1, in > ??? class Circle: > ? File "", line 8, in Circle > ??? print c.area() > AttributeError: Circle instance has no attribute 'area' Unable to reproduce: Python 2.6.4 (r264:75706, Dec 20 2009, 15:52:35) [GCC 4.2.1 (Apple Inc. build 5646) (dot 1)] on darwin >>> class Circle: ... def __init__(self): ... self.radius = 1 ... def area(self): ... return self.radius * self.radius * 3.14159 ... >>> c = Circle() >>> c.radius = 3 >>> print c.area() 28.27431 Cheers, Chris -- http://blog.rebertia.com From half.italian at gmail.com Thu Jan 28 15:18:26 2010 From: half.italian at gmail.com (Sean DiZazzo) Date: Thu, 28 Jan 2010 12:18:26 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> <4B61EC7A.7020406@optimum.net> Message-ID: On Jan 28, 12:13?pm, Joan Miller wrote: > On 28 ene, 19:58, John Posner wrote: > > > > > On 1/28/2010 2:24 PM, Joan Miller wrote: > > > > On 28 ene, 19:16, Josh Holland ?wrote: > > >> On 2010-01-28, Joan Miller ?wrote: > > > >>> I've to call to many functions with the format: > > > >>>>>> run("cmd") > > > >> Check the docs on os.system(). > > > No. I've a function that uses subprocess to run commands on the same > > > shell and so substitute to bash scrips. But a script full of run > > > ("shell_command --with --arguments") is too verbose. > > > I'm suspicious of your original intent. Essentially, you want to write > > code in which a literal string, such as ... > > > ? ?ls -l > > > ... is *not* enclosed in quotes. Why run the risk of creating confusion > > (in other people who look at your code, in syntax-checking tools, etc.) > > between variables and literals? > > Yes but to that code could be prepend a sign as '$' to be identified > and so be parsed. > > > > > But I'm in sympathy with your desire to make the code as clean as > > possible and to minimize the number of times you have to type a quote > > character. My suggestions: > > > 1. Create a function (say, "Run") that encapsulates as much of the > > syntax as possible: os.system(), subprocess.call(), string-splitting, > > whatever. So an invocation would look like this: > > > ? ?Run("ls -l *.txt") > > > (I think you've already done this step.) > > Yes, I made a funtion very cool to call to system commands, that works > well with pipes and passes the variables (i.e. "LANG=C grep -e 'foo' / > home") > > > 2. Find a text editor that supports keyboard macros, so that a single > > keystroke turns this text line: > > > ? ?ls -l *.txt > > > ... into this one: > > > ? ?Run("ls -l *.txt") > > This is not what I'm looking for. I'm supposing that could be solved > with a DSL or a macro library, any? Python is not perl. Thank God/Guido. From peter.milliken at gmail.com Thu Jan 28 15:20:48 2010 From: peter.milliken at gmail.com (Peter) Date: Thu, 28 Jan 2010 12:20:48 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> <4B61EC7A.7020406@optimum.net> Message-ID: On Jan 29, 6:58?am, John Posner wrote: > On 1/28/2010 2:24 PM, Joan Miller wrote: > > > On 28 ene, 19:16, Josh Holland ?wrote: > >> On 2010-01-28, Joan Miller ?wrote: > > >>> I've to call to many functions with the format: > > >>>>>> run("cmd") > > >> Check the docs on os.system(). > > No. I've a function that uses subprocess to run commands on the same > > shell and so substitute to bash scrips. But a script full of run > > ("shell_command --with --arguments") is too verbose. > > I'm suspicious of your original intent. Essentially, you want to write > code in which a literal string, such as ... > > ? ?ls -l > > ... is *not* enclosed in quotes. Why run the risk of creating confusion > (in other people who look at your code, in syntax-checking tools, etc.) > between variables and literals? > > But I'm in sympathy with your desire to make the code as clean as > possible and to minimize the number of times you have to type a quote > character. My suggestions: > > 1. Create a function (say, "Run") that encapsulates as much of the > syntax as possible: os.system(), subprocess.call(), string-splitting, > whatever. So an invocation would look like this: > > ? ?Run("ls -l *.txt") > > (I think you've already done this step.) > > 2. Find a text editor that supports keyboard macros, so that a single > keystroke turns this text line: > > ? ?ls -l *.txt > > ... into this one: > > ? ?Run("ls -l *.txt") > > HTH, > John I can't see you avoiding quotes etc, but extending on John's comment, the obvious next step would be to run everything in a loop i.e. place all the commands into a list and create a loop that ran each command in the list. Almost all editors support macros - most editors support some form of language sensitive editing (NOT the prompt call parameters style but rather help with the syntax via a 'form' style of fill-in) that will allow you to reduce typing effort. But macros would be the first and easiest choice for this activity. Peter From steve at holdenweb.com Thu Jan 28 15:25:19 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 28 Jan 2010 15:25:19 -0500 Subject: Why am I getting this Error message In-Reply-To: <61E1CBCB55F4401AAEAA0976543E5C0E@ray> References: <61E1CBCB55F4401AAEAA0976543E5C0E@ray> Message-ID: Ray Holt wrote: > Why am I getting the following error message. Area has been declared as > an attribute of Circle. Thanks, Ray > > > class Circle: > def __init__(self): > self.radius = 1 > def area(self): > return self.radius * self.radius * 3.14159 > c = Circle() > c.radius = 3 > print c.area() > > > > Traceback (most recent call last): > File "", line 1, in > class Circle: > File "", line 8, in Circle > print c.area() > AttributeError: Circle instance has no attribute 'area' > Because you have indented the last three lines to make them a part of the Circle definition. Generally four spaces makes a better indent, and that kind of problem is then much more obvious. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From python at mrabarnett.plus.com Thu Jan 28 15:33:50 2010 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 28 Jan 2010 20:33:50 +0000 Subject: Why am I getting this Error message In-Reply-To: <61E1CBCB55F4401AAEAA0976543E5C0E@ray> References: <61E1CBCB55F4401AAEAA0976543E5C0E@ray> Message-ID: <4B61F4AE.8080803@mrabarnett.plus.com> Ray Holt wrote: > Why am I getting the following error message. Area has been declared as > an attribute of Circle. Thanks, Ray > > > class Circle: > def __init__(self): > self.radius = 1 > def area(self): > return self.radius * self.radius * 3.14159 > c = Circle() > c.radius = 3 > print c.area() > > > > Traceback (most recent call last): > File "", line 1, in > class Circle: > File "", line 8, in Circle > print c.area() > AttributeError: Circle instance has no attribute 'area' > Probably incorrect indentation. This works: class Circle: def __init__(self): self.radius = 1 def area(self): return self.radius * self.radius * 3.14159 c = Circle() c.radius = 3 print c.area() From peloko45 at gmail.com Thu Jan 28 15:34:50 2010 From: peloko45 at gmail.com (Joan Miller) Date: Thu, 28 Jan 2010 12:34:50 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> <4B61EC7A.7020406@optimum.net> Message-ID: <64dc1630-6aa5-4983-b061-261457bf0153@g29g2000yqe.googlegroups.com> On 28 ene, 20:20, Peter wrote: > On Jan 29, 6:58?am, John Posner wrote: > > > > > On 1/28/2010 2:24 PM, Joan Miller wrote: > > > > On 28 ene, 19:16, Josh Holland ?wrote: > > >> On 2010-01-28, Joan Miller ?wrote: > > > >>> I've to call to many functions with the format: > > > >>>>>> run("cmd") > > > >> Check the docs on os.system(). > > > No. I've a function that uses subprocess to run commands on the same > > > shell and so substitute to bash scrips. But a script full of run > > > ("shell_command --with --arguments") is too verbose. > > > I'm suspicious of your original intent. Essentially, you want to write > > code in which a literal string, such as ... > > > ? ?ls -l > > > ... is *not* enclosed in quotes. Why run the risk of creating confusion > > (in other people who look at your code, in syntax-checking tools, etc.) > > between variables and literals? > > > But I'm in sympathy with your desire to make the code as clean as > > possible and to minimize the number of times you have to type a quote > > character. My suggestions: > > > 1. Create a function (say, "Run") that encapsulates as much of the > > syntax as possible: os.system(), subprocess.call(), string-splitting, > > whatever. So an invocation would look like this: > > > ? ?Run("ls -l *.txt") > > > (I think you've already done this step.) > > > 2. Find a text editor that supports keyboard macros, so that a single > > keystroke turns this text line: > > > ? ?ls -l *.txt > > > ... into this one: > > > ? ?Run("ls -l *.txt") > > > HTH, > > John > > I can't see you avoiding quotes etc, but extending on John's comment, > the obvious next step would be to run everything in a loop i.e. place > all the commands into a list and create a loop that ran each command > in the list. Yes, but could be necessary that were mixed with python code. > Almost all editors support macros - most editors support some form of > language sensitive editing (NOT the prompt call parameters style but > rather help with the syntax via a 'form' style of fill-in) that will > allow you to reduce typing effort. But macros would be the first and > easiest choice for this activity. The goal of my program is substitute to bash scripts, so the macros in editors are irrelevant fo this one. From peloko45 at gmail.com Thu Jan 28 15:45:24 2010 From: peloko45 at gmail.com (Joan Miller) Date: Thu, 28 Jan 2010 12:45:24 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> <4B61EC7A.7020406@optimum.net> <64dc1630-6aa5-4983-b061-261457bf0153@g29g2000yqe.googlegroups.com> Message-ID: <8508278f-0cf2-46c2-8313-2c640a71cdbd@q4g2000yqm.googlegroups.com> On 28 ene, 20:34, Joan Miller wrote: > On 28 ene, 20:20, Peter wrote: > > > On Jan 29, 6:58?am, John Posner wrote: > > > > On 1/28/2010 2:24 PM, Joan Miller wrote: > > > > > On 28 ene, 19:16, Josh Holland ?wrote: > > > >> On 2010-01-28, Joan Miller ?wrote: > > > > >>> I've to call to many functions with the format: > > > > >>>>>> run("cmd") > > > > >> Check the docs on os.system(). > > > > No. I've a function that uses subprocess to run commands on the same > > > > shell and so substitute to bash scrips. But a script full of run > > > > ("shell_command --with --arguments") is too verbose. > > > > I'm suspicious of your original intent. Essentially, you want to write > > > code in which a literal string, such as ... > > > > ? ?ls -l > > > > ... is *not* enclosed in quotes. Why run the risk of creating confusion > > > (in other people who look at your code, in syntax-checking tools, etc.) > > > between variables and literals? > > > > But I'm in sympathy with your desire to make the code as clean as > > > possible and to minimize the number of times you have to type a quote > > > character. My suggestions: > > > > 1. Create a function (say, "Run") that encapsulates as much of the > > > syntax as possible: os.system(), subprocess.call(), string-splitting, > > > whatever. So an invocation would look like this: > > > > ? ?Run("ls -l *.txt") > > > > (I think you've already done this step.) > > > > 2. Find a text editor that supports keyboard macros, so that a single > > > keystroke turns this text line: > > > > ? ?ls -l *.txt > > > > ... into this one: > > > > ? ?Run("ls -l *.txt") > > > > HTH, > > > John > > > I can't see you avoiding quotes etc, but extending on John's comment, > > the obvious next step would be to run everything in a loop i.e. place > > all the commands into a list and create a loop that ran each command > > in the list. > > Yes, but could be necessary that were mixed with python code. > > > Almost all editors support macros - most editors support some form of > > language sensitive editing (NOT the prompt call parameters style but > > rather help with the syntax via a 'form' style of fill-in) that will > > allow you to reduce typing effort. But macros would be the first and > > easiest choice for this activity. > > The goal of my program is substitute to bash scripts, so the macros in > editors are irrelevant fo this one. I think that the best solution that I've is to build a program that parses the script to convert *$ command* to run("command") before of be called by python. From ps.ohms at gmail.com Thu Jan 28 15:53:30 2010 From: ps.ohms at gmail.com (PS.OHM) Date: Thu, 28 Jan 2010 12:53:30 -0800 (PST) Subject: get error install MySQLdb on Mac OS X Message-ID: <30c0946e-fa6a-4a54-930c-1962d167fffe@x10g2000prk.googlegroups.com> Hello Guys I have get some error when i install MySQLdb on Mac OS X after i key command $python setup.py build rusult is : : error: command 'gcc-4.0' failed with exit status 1 How to config this poblem? From lanyjie at yahoo.com Thu Jan 28 16:12:55 2010 From: lanyjie at yahoo.com (Yingjie Lan) Date: Thu, 28 Jan 2010 13:12:55 -0800 (PST) Subject: mix statically typed with dynamically typed In-Reply-To: <64dc1630-6aa5-4983-b061-261457bf0153@g29g2000yqe.googlegroups.com> Message-ID: <708505.12453.qm@web54201.mail.re2.yahoo.com> We all know that Python is dynamically typed, and dynamically typed languages are generally slower than statically typed ones. I wonder if it is possible at all for Python to mix statically-typed-ness with dynamically-typed-ness to boost up its speed a little bit, especially when speed is needed. For example, you define a function like this: def speed(float dist, float time): return dist/time then the compiler would generate code to first check parameter types (or even do some casts if appropriate, say cast an int into float) in the beginning of this function. and the rest of the function would then be compiled with the assumption that 'dist' and 'time' are of the type float. Of course, dynamically-typed-ness is still the same as before. Python is well known for providing multiple programming paradigms, I wonder if we could also sneak this in nicely. Any thoughts? From tjreedy at udel.edu Thu Jan 28 16:21:59 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 28 Jan 2010 16:21:59 -0500 Subject: myths about python 3 In-Reply-To: <4B61EAD4.3040102@holdenweb.com> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> <4B61EAD4.3040102@holdenweb.com> Message-ID: On 1/28/2010 2:51 PM, Steve Holden wrote: > Carl Banks wrote: >> Regardless of how magnaminous the people of PSF are, the unfortunate >> reality is that trademark owners are forced by the law to be >> "particularly petty". PSF's IP lawyer will advise not to allow >> unsanctioned fork of Python 2.7 to call itself Python 2.8. >> > But if it were sanctioned ... ? We *are* pretty magnanimous ;-) I think it foolish to speculate in the absence of specifics. If some people wanted to coninue bug-fix maintainance of 2.7 after the main group of developers is done with it, in 5 years, then no new name is needed. If some people wanted to backport additional 3.x features, while still keeping the old, obsolete stuff around, then '2.8' would be appropriate. If some people wanted to add a collection of incompatible new features, perhaps some that Guido has rejected for 'Python', so that they were producing a real fork, then a new name should be used. I consider the first option possible, assuming that significant bugs still remain in 5 years. The second seems more dubious, as the developers have already backported most of what they thought sensible. The third has always been possible, and has been done, and there would be nothing really special about using 2.7 as a base. Terry Jan Reedy From pruebauno at latinmail.com Thu Jan 28 16:22:10 2010 From: pruebauno at latinmail.com (nn) Date: Thu, 28 Jan 2010 13:22:10 -0800 (PST) Subject: Need help with a program References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: Arnaud Delobelle wrote: > nn writes: > > > On Jan 28, 10:50?am, evilweasel wrote: > >> I will make my question a little more clearer. I have close to 60,000 > >> lines of the data similar to the one I posted. There are various > >> numbers next to the sequence (this is basically the number of times > >> the sequence has been found in a particular sample). So, I would need > >> to ignore the ones containing '0' and write all other sequences > >> (excluding the number, since it is trivial) in a new text file, in the > >> following format: > >> > >> >seq59902 > >> > >> TTTTTTTATAAAATATATAGT > >> > >> >seq59903 > >> > >> TTTTTTTATTTCTTGGCGTTGT > >> > >> >seq59904 > >> > >> TTTTTTTGGTTGCCCTGCGTGG > >> > >> >seq59905 > >> > >> TTTTTTTGTTTATTTTTGGG > >> > >> The number next to 'seq' is the line number of the sequence. When I > >> run the above program, what I expect is an output file that is similar > >> to the above output but with the ones containing '0' ignored. But, I > >> am getting all the sequences printed in the file. > >> > >> Kindly excuse the 'newbieness' of the program. :) I am hoping to > >> improve in the next few months. Thanks to all those who replied. I > >> really appreciate it. :) > > > > People have already given you some pointers to your problem. In the > > end you will have to "tweak the details" because only you have access > > to the data not us. > > > > Just as example here is another way to do what you are doing: > > > > with open('dnain.dat') as infile, open('dnaout.dat','w') as outfile: > > partgen=(line.split() for line in infile) > > dnagen=(str(i+1)+'\n'+part[0]+'\n' > > for i,part in enumerate(partgen) > > if len(part)>1 and part[1]!='0') > > outfile.writelines(dnagen) > > I think that generator expressions are overrated :) What's wrong with: > > with open('dnain.dat') as infile, open('dnaout.dat','w') as outfile: > for i, line in enumerate(infile): > parts = line.split() > if len(parts) > 1 and parts[1] != '0': > outfile.write(">seq%s\n%s\n" % (i+1, parts[0])) > > (untested) > > -- > Arnaud Nothing really, After posting I was thinking I should have posted a more straightforward version like the one you wrote. Now there is! It probably is more efficient too. I just have a tendency to think in terms of pipes: "pipe this junk in here, then in here, get output". Probably damage from too much Unix scripting.Since I can't resist the urge to post crazy code here goes the bonus round (don't do this at work): open('dnaout.dat','w').writelines( 'seq%s\n%s\n'%(i+1,part[0]) for i,part in enumerate(line.split() for line in open('dnain.dat')) if len(part)>1 and part[1]!='0') From deets at nospam.web.de Thu Jan 28 16:27:21 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Thu, 28 Jan 2010 22:27:21 +0100 Subject: mix statically typed with dynamically typed In-Reply-To: References: Message-ID: <4B620139.2080004@nospam.web.de> Am 28.01.10 22:12, schrieb Yingjie Lan: > We all know that Python is dynamically typed, and dynamically typed languages are generally slower than statically typed ones. I wonder if it is possible at all for Python to mix statically-typed-ness with dynamically-typed-ness to boost up its speed a little bit, especially when speed is needed. For example, you define a function like this: > > def speed(float dist, float time): > return dist/time > > then the compiler would generate code to first check parameter types (or even do some casts if appropriate, say cast an int into float) in the beginning of this function. and the rest of the function would then be compiled with the assumption that 'dist' and 'time' are of the type float. > > Of course, dynamically-typed-ness is still the same as before. Python is well known for providing multiple programming paradigms, I wonder if we could also sneak this in nicely. There are various attempts to achieve this. The most generic one, which is most promising in the long run is PyPy, the implementation of Python in itself, with the added benefit of making code-generators that emit e.g. C based on Python-code. Then there is Cython, which blends Python with C & integrates very nicely. Last but not least, for you actual example, psyco is the easiest thing to use, it's a JIT aimed to especially optimize numeric operations as the one you present. Diez From ben+python at benfinney.id.au Thu Jan 28 16:33:58 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 29 Jan 2010 08:33:58 +1100 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <87tyu529tl.fsf@benfinney.id.au> Antoine Pitrou writes: > Le Thu, 28 Jan 2010 00:19:24 +0000, Steven D'Aprano a ?crit?: > > 4. Python 3 will make you irresistible to women. > > > > FALSE - Python 3 coders are no more likely to get a date than > > any other programmer. > > They spend less time coding, so they /can/ get more "dates" (what a > strange English word) :-) Perhaps Steven could tell you about a lovely Australian meaning for the word ?date?. -- \ ?Even if the voices in my head are not real, they have pretty | `\ good ideas.? ?anonymous | _o__) | Ben Finney From steve at holdenweb.com Thu Jan 28 16:37:31 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 28 Jan 2010 16:37:31 -0500 Subject: mix statically typed with dynamically typed In-Reply-To: <708505.12453.qm@web54201.mail.re2.yahoo.com> References: <64dc1630-6aa5-4983-b061-261457bf0153@g29g2000yqe.googlegroups.com> <708505.12453.qm@web54201.mail.re2.yahoo.com> Message-ID: Yingjie Lan wrote: > We all know that Python is dynamically typed, and dynamically typed languages are generally slower than statically typed ones. I wonder if it is possible at all for Python to mix statically-typed-ness with dynamically-typed-ness to boost up its speed a little bit, especially when speed is needed. For example, you define a function like this: > > def speed(float dist, float time): > return dist/time > > then the compiler would generate code to first check parameter types (or even do some casts if appropriate, say cast an int into float) in the beginning of this function. and the rest of the function would then be compiled with the assumption that 'dist' and 'time' are of the type float. > > Of course, dynamically-typed-ness is still the same as before. Python is well known for providing multiple programming paradigms, I wonder if we could also sneak this in nicely. > > Any thoughts? > > > Google for "Python function annotations": the features you want are already there in the language specification. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From jgardner at jonathangardner.net Thu Jan 28 16:40:20 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Thu, 28 Jan 2010 13:40:20 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> Message-ID: <9a28cf01-20af-4ce2-9b78-d8529800e4b9@y7g2000prc.googlegroups.com> On Jan 28, 10:20?am, Joan Miller wrote: > I've to call to many functions with the format: > > >>> run("cmd") > > were "cmd" is a command with its arguments to pass them to the shell > and run it, i.e. > > > > >>> ?run("pwd") > or > >>> run("ls /home") > > Does anybody knows any library to help me to avoid the use of the main > quotes, and brackets? > > I would to use anything as: > > $ ls /home => run("ls /home") > > or, at least > > run pwd => run("pwd") How about this? def pwd(): return run("pwd") pwd() def ls(l=False, files=()): args = [] if l: args.insert(0, '-l') args.append(files) return run("ls", args) ls(l=True, "/foo") From arnodel at googlemail.com Thu Jan 28 16:42:17 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Thu, 28 Jan 2010 21:42:17 +0000 Subject: Need help with a program References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: nn writes: > After posting I was thinking I should have posted a more > straightforward version like the one you wrote. Now there is! It > probably is more efficient too. I just have a tendency to think in > terms of pipes: "pipe this junk in here, then in here, get output". > Probably damage from too much Unix scripting. This is funny, I did think *exactly* this when I saw your code :) -- Arnaud From ben+python at benfinney.id.au Thu Jan 28 16:42:38 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 29 Jan 2010 08:42:38 +1100 Subject: Great example of a python module/package following up to date conventions. References: <0876c5e7-e8c5-4334-a000-df310b056723@h2g2000yqj.googlegroups.com> Message-ID: <87pr4t29f5.fsf@benfinney.id.au> Josh Holland writes: > On 2010-01-28, exarkun at twistedmatrix.com wrote: > > Have you actually looked at any of the standard library? > Not recently or in depth, no. I would have thought that it would be of > high quality. I must have been mistaken. Style conventions were introduced relatively late in the history of Python (PEP 7, PEP 8, and PEP 257 were created in 2001). The existing standard library code works as-is, so no particular effort has gone into cleaning it up to retroactively conform. Also, existing APIs in the standard library tend to be preserved as non-conformant (I'm looking at you, ?logging? and ?unittest?) rather than breaking existing code by changing the API. The end result is that there are huge swaths of the standard library that do not adhere to the style conventions. It could even be argued that part of the reason for introducing the official conventions was to limit the scale of the damage already done. -- \ ?Facts do not cease to exist because they are ignored.? ?Aldous | `\ Huxley | _o__) | Ben Finney From alfps at start.no Thu Jan 28 16:43:37 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 28 Jan 2010 22:43:37 +0100 Subject: mix statically typed with dynamically typed In-Reply-To: References: Message-ID: * Yingjie Lan: [snip] > > def speed(float dist, float time): > return dist/time > > then the compiler would generate code to first check parameter types > (or even do some casts if appropriate, say cast an int into float) in > the beginning of this function. and the rest of the function would then > be compiled with the assumption that 'dist' and 'time' are of the type > float. > > Of course, dynamically-typed-ness is still the same as before. Python > is well known for providing multiple programming paradigms, I wonder if > we could also sneak this in nicely. > > Any thoughts? Python already has the /syntax/, e.g. >>> def speed( dist: float, time: float ) -> float: ... return dist/time ... >>> print( speed.__annotations__ ) {'dist': , 'return': , 'time': } >>> _ However, this syntax, while exploitable, is by default nothing but an annotation device, like doc strings. I'm not sure I like your idea of introducing static typing to increase speed, but it could be done without introducing new syntax simply by defining a special meaning to such annotation expressions that are 'type' invocations, say, then like def speed( dist: type( float ), time: type( float ) ) -> type( float ) Since there are umpteen projects to increase speed of Python this idea may already have been explored... Cheers & hth., - Alf (who has some other ideas) From ben+python at benfinney.id.au Thu Jan 28 16:52:52 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 29 Jan 2010 08:52:52 +1100 Subject: Great example of a python module/package following up to date conventions. References: <0876c5e7-e8c5-4334-a000-df310b056723@h2g2000yqj.googlegroups.com> Message-ID: <87ljfh28y3.fsf@benfinney.id.au> Big Stu writes: > I'm hoping someone on here can point me to an example of a python > package that is a great example of how to put it all together. I'm > hoping for example code that demonstrates [good adherence to style and > practice conventions] I find the code base for Bazaar to be particularly good quality : $ sudo aptitude install bzr $ bzr branch lp:bzr $ cd bzr/ $ # browse browse browse -- \ ?Probably the earliest flyswatters were nothing more than some | `\ sort of striking surface attached to the end of a long stick.? | _o__) ?Jack Handey | Ben Finney From duncan.booth at invalid.invalid Thu Jan 28 16:53:45 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 28 Jan 2010 21:53:45 GMT Subject: mix statically typed with dynamically typed References: Message-ID: "Alf P. Steinbach" wrote: > I'm not sure I like your idea of introducing static typing to increase > speed, but it could be done without introducing new syntax simply by > defining a special meaning to such annotation expressions that are > 'type' invocations, say, then like > > def speed( dist: type( float ), time: type( float ) ) -> type( > float ) > That would be particularly useless: >>> type(float) is type True So your declaration is identical to: def speed(dist: type, time: type) -> type: Much better just to stick to something like: def speed( dist: float, time: float ) -> float: where at least you can tell from the annotations what types were actually used. From MLMDev at Comcast.net Thu Jan 28 16:54:38 2010 From: MLMDev at Comcast.net (Mitchell L Model) Date: Thu, 28 Jan 2010 16:54:38 -0500 Subject: python 3's adoption In-Reply-To: References: Message-ID: On Jan 28, 2010, at 12:00 PM, python-list-request at python.org wrote: > From: Roy Smith > Date: January 28, 2010 11:09:58 AM EST > To: python-list at python.org > Subject: Re: python 3's adoption > > > In article , > Mitchell L Model wrote: > >> I use the sep and end keywords all the time. > > What are 'sep' and 'end'? I'm looking in > http://docs.python.org/3.1/genindex-all.html and don't see those > mentioned > at all. Am I just looking in the wrong place? > Sorry -- I wasn't clear. They are keyword arguments to the print function. From MLMDev at Comcast.net Thu Jan 28 17:00:10 2010 From: MLMDev at Comcast.net (Mitchell L Model) Date: Thu, 28 Jan 2010 17:00:10 -0500 Subject: python 3's adoption In-Reply-To: References: Message-ID: On Jan 28, 2010, at 1:40 PM, Terry Reedy wrote > ... > On 1/28/2010 11:03 AM, Mitchell L Model wrote: >> I have been working with Python 3 for over a year. ... > > I agree completely. Such sweet words to read! > >> Conversion of old code is greatly facilitied by the 2to3 tool that >> comes >> with Python 3. The big issue in moving from 2 to 3 is the external >> libraries and development tools you use. Different IDEs have released >> versions that support Python 3 at different times. (I believe Wing >> was >> the first.) If you use numpy, for example, or one of the many >> libraries >> that require it, you are stuck. Possibly some important facilities >> will >> never be ported to Python 3, but probably most active projects will >> eventually produce a Python 3 version -- for example, according to >> its >> web page, a Python 3 version of PIL is on the way. I was able to >> cover >> all the topics in my book using only Python library modules, >> something I >> felt would be best for readers -- I used libraries such as >> elementree, >> sqlite3, and tkinter. The only disappointment was that I couldn't >> include a chapter on BioPython, since there is no Python 3 version. >> >> By now, many large facilities support both Python 2 and Python 3. I >> am >> currently building a complex GUI/Visualization application based on >> the >> Python 3 version of PyQt4 and Wing IDE and am delighted with all of >> it. >> It may well be that some very important large > > Something got clipped ;-) Thanks for noticing. Actually, I had abandoned that sentence and went back and added more to the prior paragraph. Just never went back and deleted the false start. > > Anyway, thank you for the report. > Glad to contribute; gladder to be appreciated. From half.italian at gmail.com Thu Jan 28 17:02:39 2010 From: half.italian at gmail.com (Sean DiZazzo) Date: Thu, 28 Jan 2010 14:02:39 -0800 (PST) Subject: get error install MySQLdb on Mac OS X References: <30c0946e-fa6a-4a54-930c-1962d167fffe@x10g2000prk.googlegroups.com> Message-ID: On Jan 28, 12:53?pm, "PS.OHM" wrote: > Hello Guys > > I have get some error when i install MySQLdb on Mac OS X > > after i key command $python setup.py build > > rusult is > : > : > error: command 'gcc-4.0' failed with exit status 1 > > How to config this poblem? Please show a little bit more of the error From alfps at start.no Thu Jan 28 17:07:24 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 28 Jan 2010 23:07:24 +0100 Subject: mix statically typed with dynamically typed In-Reply-To: References: Message-ID: * Duncan Booth: > "Alf P. Steinbach" wrote: > >> I'm not sure I like your idea of introducing static typing to increase >> speed, but it could be done without introducing new syntax simply by >> defining a special meaning to such annotation expressions that are >> 'type' invocations, say, then like >> >> def speed( dist: type( float ), time: type( float ) ) -> type( >> float ) >> > > That would be particularly useless: > >>>> type(float) is type > True > > So your declaration is identical to: > > def speed(dist: type, time: type) -> type: That's the point. > Much better just to stick to something like: > > def speed( dist: float, time: float ) -> float: > > where at least you can tell from the annotations what types were actually > used. No, you do not want to redefine the meaning of existing (actually used) constructs. That would be particularly useless, to use your own words. :-) Cheers & hth., - Alf From peloko45 at gmail.com Thu Jan 28 17:16:52 2010 From: peloko45 at gmail.com (Joan Miller) Date: Thu, 28 Jan 2010 14:16:52 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> <9a28cf01-20af-4ce2-9b78-d8529800e4b9@y7g2000prc.googlegroups.com> Message-ID: On 28 ene, 21:40, Jonathan Gardner wrote: > On Jan 28, 10:20?am, Joan Miller wrote: > > > > > I've to call to many functions with the format: > > > >>> run("cmd") > > > were "cmd" is a command with its arguments to pass them to the shell > > and run it, i.e. > > > >>> ?run("pwd") > > or > > >>> run("ls /home") > > > Does anybody knows any library to help me to avoid the use of the main > > quotes, and brackets? > > > I would to use anything as: > > > $ ls /home => run("ls /home") > > > or, at least > > > run pwd => run("pwd") > > How about this? > > def pwd(): return run("pwd") > > pwd() > > def ls(l=False, files=()): > ? ? args = [] > ? ? if l: args.insert(0, '-l') > ? ? args.append(files) > ? ? return run("ls", args) > > ls(l=True, "/foo") There would be to make a function for each system command to use so it would be too inefficient, and follow the problem with the quotes. The best is make a parser into a compiled language From jjposner at optimum.net Thu Jan 28 17:57:09 2010 From: jjposner at optimum.net (John Posner) Date: Thu, 28 Jan 2010 17:57:09 -0500 Subject: Wrap a function In-Reply-To: <8508278f-0cf2-46c2-8313-2c640a71cdbd@q4g2000yqm.googlegroups.com> References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> <4B61EC7A.7020406@optimum.net> <64dc1630-6aa5-4983-b061-261457bf0153@g29g2000yqe.googlegroups.com> <8508278f-0cf2-46c2-8313-2c640a71cdbd@q4g2000yqm.googlegroups.com> Message-ID: <4B621645.1000906@optimum.net> On 1/28/2010 3:45 PM, Joan Miller wrote: > On 28 ene, 20:34, Joan Miller wrote: >> On 28 ene, 20:20, Peter wrote: >> >>> On Jan 29, 6:58 am, John Posner wrote: >> >>>> On 1/28/2010 2:24 PM, Joan Miller wrote: >> >>>>> On 28 ene, 19:16, Josh Holland wrote: >>>>>> On 2010-01-28, Joan Miller wrote: >> >>>>>>> I've to call to many functions with the format: >> >>>>>>>>>> run("cmd") >> >>>>>> Check the docs on os.system(). >>>>> No. I've a function that uses subprocess to run commands on the same >>>>> shell and so substitute to bash scrips. But a script full of run >>>>> ("shell_command --with --arguments") is too verbose. >> >>>> I'm suspicious of your original intent. Essentially, you want to write >>>> code in which a literal string, such as ... >> >>>> ls -l >> >>>> ... is *not* enclosed in quotes. Why run the risk of creating confusion >>>> (in other people who look at your code, in syntax-checking tools, etc.) >>>> between variables and literals? >> >>>> But I'm in sympathy with your desire to make the code as clean as >>>> possible and to minimize the number of times you have to type a quote >>>> character. My suggestions: >> >>>> 1. Create a function (say, "Run") that encapsulates as much of the >>>> syntax as possible: os.system(), subprocess.call(), string-splitting, >>>> whatever. So an invocation would look like this: >> >>>> Run("ls -l *.txt") >> >>>> (I think you've already done this step.) >> >>>> 2. Find a text editor that supports keyboard macros, so that a single >>>> keystroke turns this text line: >> >>>> ls -l *.txt >> >>>> ... into this one: >> >>>> Run("ls -l *.txt") >> >>>> HTH, >>>> John >> >>> I can't see you avoiding quotes etc, but extending on John's comment, >>> the obvious next step would be to run everything in a loop i.e. place >>> all the commands into a list and create a loop that ran each command >>> in the list. >> >> Yes, but could be necessary that were mixed with python code. >> >>> Almost all editors support macros - most editors support some form of >>> language sensitive editing (NOT the prompt call parameters style but >>> rather help with the syntax via a 'form' style of fill-in) that will >>> allow you to reduce typing effort. But macros would be the first and >>> easiest choice for this activity. >> >> The goal of my program is substitute to bash scripts, so the macros in >> editors are irrelevant fo this one. > > I think that the best solution that I've is to build a program that > parses the script to convert *$ command* to run("command") before of > be called by python. I believe you're working on Linux, so how about using "sed"? Here's a (prettified) BASH transcript of a sed script (edit.sed) transforming a 6-line text file (myprog.py). The text file has both Python statements and "special commands", which have "$ " at the beginning of the line. >>> cat myprog.py print "hello" $ ls -l r = range(10) $ grep foo bar.data pass print "bye" >>> cat edit.sed s/^\$ \(.*\)/Run("\1")/ >>> sed -f edit.sed data.txt print "hello" Run("ls -l") r = range(10) Run("grep foo bar.data") pass print "bye" -John From steve at REMOVE-THIS-cybersource.com.au Thu Jan 28 18:33:19 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 28 Jan 2010 23:33:19 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> <7xockfjcsv.fsf@ruckus.brouhaha.com> <6b89dc67-ceee-4b83-a11f-c7376185dcd1@b9g2000pri.googlegroups.com> <7xy6ji62vx.fsf@ruckus.brouhaha.com> Message-ID: <03720b25$0$1309$c3e8da3@news.astraweb.com> On Thu, 28 Jan 2010 13:20:02 -0500, Terry Reedy wrote: > On 1/28/2010 3:37 AM, Paul Rubin wrote: >> Jonathan Gardner writes: >>> If you're going to have statements, you're going to need the null >>> statement. That's "pass". >> >> Why? Expressions are statements, so you could just say "pass" (in >> quotes, denoting a string literal), or 0, or None, os anything else >> like that, instead of having a special statement. > > As Python is currently compiled, you are right, pass is not needed. A > string becomes the doc attribute, and becomes local var 0, but 0 is just > ignored. I actually expected a load_const but that is now optimized > away. I am not sure this was always true. Perhaps 'pass' is easier than > '0' for mewcomers reading the tutorial, but I have no data. As I said earlier, a dedicated statement `pass` indicates the intent of the author better than some arbitrary constant. if flag: 0 else: do_something() could be a mistake, perhaps the programmer intended to write "x = 0", but there is no ambiguity with the pass statement. I would hope that PyChecker and the equivalent would flag the use of bare constants in code and give a warning. In any case, while such a idiom works in code, it plays havoc with the interactive interpreter: >>> while 1: ... "pass" ... 'pass' 'pass' 'pass' 'pass' 'pass' Traceback (most recent call last): File "", line 2, in KeyboardInterrupt I have trimmed the output for convenience -- in reality, it fills my terminal's output buffer faster than I can hit ctrl-C. Using a constant as a proxy for the pass statement would get obnoxious really fast. -- Steven From morganhein at gmail.com Thu Jan 28 18:34:22 2010 From: morganhein at gmail.com (JohnnyFive) Date: Thu, 28 Jan 2010 15:34:22 -0800 (PST) Subject: Sleep timer but still responsive? Message-ID: I need help with something that is probably fairly simple, but i'm having a heck of a time getting it work. Basically, I need my program to sleep for a certain amount of time, but I don't want the console to become unresponsive while sleeping. As soon as the time is up, I want the main program to run it's course again. I tried using a Timer, threads, etc, but I really can't figure it out. What am I missing? I can post what I have, but I don't want to get caught up on how i'm doing it wrong (as none of it works), but rather the correct way to do it. Thanks in advance! From jgardner at jonathangardner.net Thu Jan 28 18:36:37 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Thu, 28 Jan 2010 15:36:37 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> <9a28cf01-20af-4ce2-9b78-d8529800e4b9@y7g2000prc.googlegroups.com> Message-ID: On Jan 28, 2:16?pm, Joan Miller wrote: > > There would be to make a function for each system command to use so it > would be too inefficient, and follow the problem with the quotes. > > The best is make a parser into a compiled language > Yeah, you could do that. Or you can simply rely on /bin/sh to do the parsing and everything else for you. No need to re-invent the wheel. I don't think Python will ever beat sh as a shell replacement. When people say that Python is great for some situations, but not so much for others, I think they thought of running commands like this as "other", From paul at boddie.org.uk Thu Jan 28 18:47:53 2010 From: paul at boddie.org.uk (Paul Boddie) Date: Thu, 28 Jan 2010 15:47:53 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> Message-ID: On 27 Jan, 13:26, Xah Lee wrote: > > So, for practical reasons, i think a ?key? parameter is fine. But > chopping off ?cmp? is damaging. When your data structure is complex, > its order is not embedded in some ?key?. Taking out ?cmp? makes it > impossible to sort your data structure. What would annoy me if I used Python 3.x would be the apparent lack of the __cmp__ method for conveniently defining comparisons between instances of my own classes. Having to define all the rich comparison methods frequently isn't even as much fun as it sounds. Paul From mensanator at aol.com Thu Jan 28 18:49:55 2010 From: mensanator at aol.com (Mensanator) Date: Thu, 28 Jan 2010 15:49:55 -0800 (PST) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On Jan 28, 11:35?am, Ethan Furman wrote: > Steven D'Aprano wrote: > > 4. Python 3 will make you irresistible to women. > > > ? ? FALSE > > What?!? ?Drat!!! ?Guess I'll have to learn Lisp... ?;) Irresisible? Ha! The chicks will think you have a harelip. > > ~Ethan~ From kerensaelise at hotmail.com Thu Jan 28 18:52:14 2010 From: kerensaelise at hotmail.com (elsa) Date: Thu, 28 Jan 2010 15:52:14 -0800 (PST) Subject: For loop searching takes too long! Message-ID: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> Hi guys, I've got a problem with my program, in that the code just takes too long to run. Here's what I'm doing. If anyone has any tips, they'd be much appreciated! So, say I have a list of lists that looks something like this (I'm using a list of lists, rather than a list of tuples, as I need it to be mutable): myList = [[786,0],[45, 1],[673,1],...................[23,46]] there are enough entries in the outer list, that the sum of myList[i] [0] across all i could be as high as 10^7. Now, what I need to do is randomly choose one myList[i], however the distribution of my random choice needs to be proportional to the values of myList[i][0]. So, for this list above, I'd have a much higher chance of choosing myList[0] than myList[1]. Here is how I'm doing it at the moment: def chooseI(myList): mySum=0 choice = random.choice(range(1,sum([i[0] for i in myList])+1)) for i in range(len(myList)): mySum+=myList[i][0] if mySum>=choice: return i break This works just fine if sum([i[0] for i in myList]) is less than 10,000, say. However if its up around 10^7, the whole thing crashes. Is there a faster way of doing this, that doesn't involve as many computational steps? Thanks! elsa From steve at REMOVE-THIS-cybersource.com.au Thu Jan 28 19:00:57 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 00:00:57 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <4b61c166$1@dnews.tpgi.com.au> Message-ID: <0372119f$0$1309$c3e8da3@news.astraweb.com> On Thu, 28 Jan 2010 18:37:56 +0100, Alf P. Steinbach wrote: > * Lie Ryan: >> On 01/28/10 20:12, Alf P. Steinbach wrote: >>> >>> import builtins >>> >>> >>> >>> org_print = print >>> >>> builtins.print = 666 >>> >>> >>> >>> print( "trallala" ) >>> Traceback (most recent call last): >>> File "", line 1, in >>> TypeError: 'int' object is not callable >>> >>> org_print( "but is that really so smart?" ) >>> but is that really so smart? >>> >>> _ >> >> Monkey patching follows (or should follow) the same rule as class >> inheritance overriding: the overrider's input domain must be a superset >> of the overriden's input domain and the overrider's output range must >> be a subset of the overriden's output range. 666 object (int) is not >> even remotely compatible with function object. > > Yes, that's my point. > > A 'print' replacement should ideally provide all the guarantees on > behavior that standard 'print' does. > > And with that it's not so easy to put in a /correct/ replacement of > 'print'; in particular, it has to honor the 'file' keyword argument. Oh come on, that's easy. This is Python we're talking about. Any object with a write() method is valid. >>> class MyIO: ... def __init__(self): ... self.buffer = [] ... def write(self, *args): ... self.buffer.extend(args) ... >>> io = MyIO() >>> print("Say goodnight Gracie", "Goodnight Gracie!", file=io) >>> io.buffer ['Say goodnight Gracie', ' ', 'Goodnight Gracie!', '\n'] In many cases, probably most, you won't be writing your own full-blown print replacement, but merely decorating the existing function. How easy is that? Trivial. Here's how to shut down a noisy module that prints too much: >>> def make_quieter(func): ... def inner(*args, **kwargs): ... out = kwargs.get('file') ... if out in (None, sys.stdout): ... pass ... else: ... func(*args, **kwargs) ... return inner ... >>> print = make_quieter(print) >>> print("spam spam spam") >>> print("spam spam spam", file=io) >>> io.buffer ['Say goodnight Gracie', ' ', 'Goodnight Gracie!', '\n', 'spam spam spam', '\n'] (except of course you would monkey-patch the module). Because print is a built-in, getting the original version back is even easier: >>> del print >>> print("spam spam spam") 'spam spam spam' > Thus the 3.x design makes it easy to replace 'print' incorrectly. This is Python -- the compiler won't stop you from shooting yourself in the foot. >>> len = 666 >>> len("hello world") Traceback (most recent call last): File "", line 1, in TypeError: 'int' object is not callable > I'd rather still had 'print' as keyword... ;-) We've shown many benefits of print as a function. Can you give any benefits of it being a statement, other than backward compatibility for those who learned it from Python 2.x? If you were designing your own language from scratch, so that backwards compatibility wasn't an issue, why would you make print a statement? -- Steven From ben+python at benfinney.id.au Thu Jan 28 19:01:21 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 29 Jan 2010 11:01:21 +1100 Subject: Sorting a list with entries of unequal types Message-ID: <87sk9pzsmm.fsf@benfinney.id.au> Howdy all, Python 2.6 tells me that, in Python 3, sorting a list with entries of incompatible types is not allowed: ===== $ python2.5 -c "foo = [1, True, 'green', 4, -27, 15.3]; foo.sort(); print foo;" [-27, 1, True, 4, 15.300000000000001, 'green'] $ python2.6 -c "foo = [1, True, 'green', 4, -27, 15.3]; foo.sort(); print foo;" [-27, 1, True, 4, 15.300000000000001, 'green'] $ python2.6 -3 -c "foo = [1, True, 'green', 4, -27, 15.3]; foo.sort(); print foo;" -c:1: DeprecationWarning: comparing unequal types not supported in 3.x [-27, 1, True, 4, 15.300000000000001, 'green'] ===== So how should I be sorting a list with entries of ?unequal types? such that it will work in Python 3? -- \ ?The idea that He would take his attention away from the | `\ universe in order to give me a bicycle with three speeds is | _o__) just so unlikely that I can't go along with it.? ?Quentin Crisp | Ben Finney From aahz at pythoncraft.com Thu Jan 28 19:18:43 2010 From: aahz at pythoncraft.com (Aahz) Date: 28 Jan 2010 16:18:43 -0800 Subject: py2exe deal with python command line inside a program References: Message-ID: In article , wrote: > >I need to create a python subprogress, like this: >myProcess = subprocess.Popen([sys.executable, 'C:\myscript.py'], > env=env, stdin=subprocess.PIPE, > stdout=subprocess.PIPE) You need to change your setup.py so that myscript.py is also an exe. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From steve at REMOVE-THIS-cybersource.com.au Thu Jan 28 19:27:10 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 00:27:10 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> <7xockfjcsv.fsf@ruckus.brouhaha.com> <6b89dc67-ceee-4b83-a11f-c7376185dcd1@b9g2000pri.googlegroups.com> Message-ID: <037217c3$0$1309$c3e8da3@news.astraweb.com> On Wed, 27 Jan 2010 23:50:55 -0800, Jonathan Gardner wrote: > I agree on "assert". I don't like running a program in test mode and > then running it in production mode with different code. I would rather > test what I am going to actually run. "assert" should be a function, and > support for removing assert statements should be eliminated. But then it wouldn't be an assert, it would be a test, and tests are best written explicitly. The whole point of assertions (outside of test code) is that they should ALWAYS pass. Since they should always pass, they're safe to optimize away, if the user explicitly runs Python with the "optimize" runtime switch. This is under the end-user's control, not the application writer. If you don't trust optimized code, don't use it. Assertions should be treated as "this will never happen" code, and are only there to catch coding errors and logic mistakes. If you expect that a test could fail, it shouldn't be written as an assert but as an explicit test. > I simply don't use assert statements at all. Outside of explicit test code (e.g. unit tests or equivalent), asserts should be rare. If you never use them at all, it probably means you're either not programming defensively enough, or you're needlessly writing explicit tests to cover situations that can't happen. assert exists to cover the middle ground between those two extremes. -- Steven From no.email at nospam.invalid Thu Jan 28 19:28:43 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 28 Jan 2010 16:28:43 -0800 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <4b61c166$1@dnews.tpgi.com.au> <0372119f$0$1309$c3e8da3@news.astraweb.com> Message-ID: <7xockdycsk.fsf@ruckus.brouhaha.com> Steven D'Aprano writes: > If you were designing your own language from scratch, so that backwards > compatibility wasn't an issue, why would you make print a statement? As another real estate analogy, my apartment has some problems with its plumbing, plus an ugly spot on the kitchen wall that could use a coat of paint to fix. The plumbing problem is fairly serious (they keep shutting off the water in the building when it acts up) but the kitchen spot is a minor annoyance that is mostly hidden by the refrigerator anyway. Fixing either of those things would be a hassle: I'd have to cover all my stuff with plastic and stay out of the apartment for a day or so while the maintenance guys hacked on things, and I'd expect some of my stuff to get damaged inadvertently no matter how careful everyone was. It's worth dealing with the repair hassles to get the plumbing fixed, and if I'm going to have to deal with that disruption anyway, then sure, I'd like them to paint the kitchen spot at the same time. But if they say they want me to cover up all my stuff and leave so they can JUST fix the kitchen spot, and then do the same thing a second time so they can fix the plumbing at some unspecified date in the future, then I'd rather just live with the kitchen spot the way it is. Yes it's a cosmetic blemish, but it's not causing any real problems, and I'd rather not deal with the hassle and risk of fixing it if there's no other benefit. In Python terms, the print statement is the spot on the wall, while the plumbing is something like the GIL and the legacy codebase that would break in a hundred ways if Python had real parallelism and a tracing garbage collector and a native-code compiler and the various language changes it would take to make all that stuff really fly rather than just limp along. If they are going to make everyone deal with the disruption of migrating to an incompatible version, they should do it once rather than twice. In short, there's a mythical Python 4 that only exists in my imagination, but it already interests me a lot more than Python 3. From no.email at nospam.invalid Thu Jan 28 19:32:49 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 28 Jan 2010 16:32:49 -0800 Subject: Sorting a list with entries of unequal types References: <87sk9pzsmm.fsf@benfinney.id.au> Message-ID: <7xk4v1yclq.fsf@ruckus.brouhaha.com> Ben Finney writes: > So how should I be sorting a list with entries of ?unequal types? such > that it will work in Python 3? Um, what ordering do you want? Basically you have to write a custom key function (they removed the option of writing a comparison function). Maybe something like foo.sort(key=lambda x: (type(x), x)) would be enough to fool the sorting function? I don't have python 3 so I haven't tried it. From __peter__ at web.de Thu Jan 28 19:35:52 2010 From: __peter__ at web.de (Peter Otten) Date: Fri, 29 Jan 2010 01:35:52 +0100 Subject: Sorting a list with entries of unequal types References: <87sk9pzsmm.fsf@benfinney.id.au> Message-ID: Ben Finney wrote: > Python 2.6 tells me that, in Python 3, sorting a list with entries of > incompatible types is not allowed: > $ python2.6 -3 -c "foo = [1, True, 'green', 4, -27, 15.3]; foo.sort(); > print foo;" -c:1: DeprecationWarning: comparing unequal types not > supported in 3.x > [-27, 1, True, 4, 15.300000000000001, 'green'] > ===== > > So how should I be sorting a list with entries of ?unequal types? such > that it will work in Python 3? I can't find the relevant part of the 2.6 documentation, but something like >>> def key(x): ... t = type(x) ... t = compat.get(t, t) ... return t.__name__, id(t), x ... >>> compat = {bool: float, int: float} >>> sorted([1, True, 'green', 4, -27, 15.3], key=key) [-27, 1, True, 4, 15.300000000000001, 'green'] should work. Peter From __peter__ at web.de Thu Jan 28 19:41:58 2010 From: __peter__ at web.de (Peter Otten) Date: Fri, 29 Jan 2010 01:41:58 +0100 Subject: For loop searching takes too long! References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> Message-ID: elsa wrote: > Hi guys, > > I've got a problem with my program, in that the code just takes too > long to run. Here's what I'm doing. If anyone has any tips, they'd be > much appreciated! > > So, say I have a list of lists that looks something like this (I'm > using a list of lists, rather than a list of tuples, as I need it to > be mutable): > > myList = [[786,0],[45, 1],[673,1],...................[23,46]] > > there are enough entries in the outer list, that the sum of myList[i] > [0] across all i could be as high as 10^7. > > Now, what I need to do is randomly choose one myList[i], however the > distribution of my random choice needs to be proportional to the > values of myList[i][0]. So, for this list above, I'd have a much > higher chance of choosing myList[0] than myList[1]. > > Here is how I'm doing it at the moment: > > def chooseI(myList): > mySum=0 > choice = random.choice(range(1,sum([i[0] for i in myList])+1)) > for i in range(len(myList)): > mySum+=myList[i][0] > if mySum>=choice: > return i > break > > This works just fine if sum([i[0] for i in myList]) is less than > 10,000, say. However if its up around 10^7, the whole thing crashes. > Is there a faster way of doing this, that doesn't involve as many > computational steps? Do the first items of the inner lists change often? If not you can put the running sum into them, i. e. myList = [[768, ...], [786+45, ...], [786+45+673, ...], ...] and use bisect.bisect() to choose the item. Peter From jjposner at optimum.net Thu Jan 28 19:50:32 2010 From: jjposner at optimum.net (John Posner) Date: Thu, 28 Jan 2010 19:50:32 -0500 Subject: For loop searching takes too long! In-Reply-To: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> Message-ID: <4B6230D8.30704@optimum.net> On 1/28/2010 6:52 PM, elsa wrote: > Hi guys, > > I've got a problem with my program, in that the code just takes too > long to run. Here's what I'm doing. If anyone has any tips, they'd be > much appreciated! > > So, say I have a list of lists that looks something like this (I'm > using a list of lists, rather than a list of tuples, as I need it to > be mutable): > > myList = [[786,0],[45, 1],[673,1],...................[23,46]] > > there are enough entries in the outer list, that the sum of myList[i] > [0] across all i could be as high as 10^7. > > Now, what I need to do is randomly choose one myList[i], however the > distribution of my random choice needs to be proportional to the > values of myList[i][0]. So, for this list above, I'd have a much > higher chance of choosing myList[0] than myList[1]. > > Here is how I'm doing it at the moment: > > def chooseI(myList): > mySum=0 > choice = random.choice(range(1,sum([i[0] for i in myList])+1)) > for i in range(len(myList)): > mySum+=myList[i][0] > if mySum>=choice: > return i > break > > This works just fine if sum([i[0] for i in myList]) is less than > 10,000, say. However if its up around 10^7, the whole thing crashes. > Is there a faster way of doing this, that doesn't involve as many > computational steps? > This way of generating *choice* might be less resource-intensive: list_total = 0 for x in myList: list_total += x[0] choice = random.randint(1, list_total) This avoids creating another list (the list comprehension) as big as myList. BTW, I don't think you need to *break* after you've already *return*ed. -John From gagsl-py2 at yahoo.com.ar Thu Jan 28 19:55:53 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Thu, 28 Jan 2010 21:55:53 -0300 Subject: Sleep timer but still responsive? References: Message-ID: En Thu, 28 Jan 2010 20:34:22 -0300, JohnnyFive escribi?: > I need help with something that is probably fairly simple, but i'm > having a heck of a time getting it work. > > Basically, I need my program to sleep for a certain amount of time, > but I don't want the console to become unresponsive while sleeping. Please provide more details. What do you want your program to do while sleeping? What kind of actions do you want a response to? Do you have a GUI? A curses-based interfase? -- Gabriel Genellina From python at mrabarnett.plus.com Thu Jan 28 19:56:55 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 29 Jan 2010 00:56:55 +0000 Subject: Sorting a list with entries of unequal types In-Reply-To: <87sk9pzsmm.fsf@benfinney.id.au> References: <87sk9pzsmm.fsf@benfinney.id.au> Message-ID: <4B623257.6030106@mrabarnett.plus.com> Ben Finney wrote: > Howdy all, > > Python 2.6 tells me that, in Python 3, sorting a list with entries of > incompatible types is not allowed: > > ===== > $ python2.5 -c "foo = [1, True, 'green', 4, -27, 15.3]; foo.sort(); print foo;" > [-27, 1, True, 4, 15.300000000000001, 'green'] > > $ python2.6 -c "foo = [1, True, 'green', 4, -27, 15.3]; foo.sort(); print foo;" > [-27, 1, True, 4, 15.300000000000001, 'green'] > > $ python2.6 -3 -c "foo = [1, True, 'green', 4, -27, 15.3]; foo.sort(); print foo;" > -c:1: DeprecationWarning: comparing unequal types not supported in 3.x > [-27, 1, True, 4, 15.300000000000001, 'green'] > ===== > > So how should I be sorting a list with entries of ?unequal types? such > that it will work in Python 3? > First you need to decide how they should be ordered, for example by name of class then value. Then make tuple keys for the values. Python compares tuple members one by one, only looking at the next members if the current members are equal, so if the first members are the class names (str) and the second members are the values (possibly unequal types), then Python will compare the values only if the types are the same. >>> sorted([1, True, 'green', 4, -27, 15.3], key=lambda value: (type(value).__name__, value)) [True, 15.3, -27, 1, 4, 'green'] This does mean that Boolean < float < int, but that's easy to fix: >>> def make_key(value): ... if isinstance(value, (int, float)): ... return "float", float(value) ... return type(value).__name__, value ... >>> sorted([1, True, 'green', 4, -27, 15.3], key=make_key) [-27, 1, True, 4, 15.3, 'green'] From g.bogle at auckland.no.spam.ac.nz Thu Jan 28 19:57:18 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Fri, 29 Jan 2010 13:57:18 +1300 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: Ethan Furman wrote: > Steven D'Aprano wrote: >> 4. Python 3 will make you irresistible to women. >> >> FALSE > > What?!? Drat!!! Guess I'll have to learn Lisp... ;) > > ~Ethan~ Learn to say this fast, you'll impress the hell out of them: Chaps with chapped lips lisp. From python at mrabarnett.plus.com Thu Jan 28 19:58:34 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 29 Jan 2010 00:58:34 +0000 Subject: Sorting a list with entries of unequal types In-Reply-To: <7xk4v1yclq.fsf@ruckus.brouhaha.com> References: <87sk9pzsmm.fsf@benfinney.id.au> <7xk4v1yclq.fsf@ruckus.brouhaha.com> Message-ID: <4B6232BA.1070608@mrabarnett.plus.com> Paul Rubin wrote: > Ben Finney writes: >> So how should I be sorting a list with entries of ?unequal types? such >> that it will work in Python 3? > > Um, what ordering do you want? Basically you have to write a custom key > function (they removed the option of writing a comparison function). > > Maybe something like > > foo.sort(key=lambda x: (type(x), x)) > > would be enough to fool the sorting function? I don't have python 3 > so I haven't tried it. You can't compare types: TypeError: unorderable types: type() < type() but you can compare their names. From steve at REMOVE-THIS-cybersource.com.au Thu Jan 28 20:00:52 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 01:00:52 GMT Subject: For loop searching takes too long! References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> Message-ID: <03721fab$0$1309$c3e8da3@news.astraweb.com> On Thu, 28 Jan 2010 15:52:14 -0800, elsa wrote: > Now, what I need to do is randomly choose one myList[i], however the > distribution of my random choice needs to be proportional to the values > of myList[i][0]. So, for this list above, I'd have a much higher chance > of choosing myList[0] than myList[1]. > > Here is how I'm doing it at the moment: > > def chooseI(myList): > mySum=0 > choice = random.choice(range(1,sum([i[0] for i in myList])+1)) > for i in range(len(myList)): > mySum+=myList[i][0] > if mySum>=choice: > return i > break This isn't related to your problem, but you don't need the break after the return -- the return will leave the loop and the function, and the break will never be reached. You could probably speed the code up a little by changing all the calls to range into xrange. range actually generates a list of integers, which is time consuming, while xrange is a lazy generator which just produces each integer one at a time. (You can ignore this if you are using Python 3.0 or 3.1.) Another small optimization you could use is to use a generator expression instead of a list comprehension inside the call to sum. That should generate the sum lazily, instead of calculating a giant list first and then adding it up. But I'd try something like this: # Untested. def chooseI(myList): total = sum(i[0] for i in myList) mySum = 0 choice = random.randrange(total) for i in myList: mySum += i[0] if mySum >= choice: return i -- Steven From steve at REMOVE-THIS-cybersource.com.au Thu Jan 28 20:11:05 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 01:11:05 GMT Subject: Sorting a list with entries of unequal types References: <87sk9pzsmm.fsf@benfinney.id.au> Message-ID: <03722210$0$1309$c3e8da3@news.astraweb.com> On Fri, 29 Jan 2010 11:01:21 +1100, Ben Finney wrote: > So how should I be sorting a list with entries of ?unequal types? such > that it will work in Python 3? That depends on how you want the items to be sorted. Python 2.x sorted unequal types in some arbitrary but consistent order. If that's all you want, then a key function like this will work in Python 3.1: >>> foo = [1, True, 'green', 4, -27, 15.3] >>> foo.sort() Traceback (most recent call last): File "", line 1, in TypeError: unorderable types: str() < bool() >>> foo.sort(key=lambda x: (str(type(x)), x)) >>> foo [True, 15.3, -27, 1, 4, 'green'] If you want to match the sort order from a specific version of Python, or if you want a specific ordering, you'll need to write your own key function. -- Steven From ben+python at benfinney.id.au Thu Jan 28 20:13:50 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 29 Jan 2010 12:13:50 +1100 Subject: Sorting a list with entries of unequal types References: <87sk9pzsmm.fsf@benfinney.id.au> Message-ID: <87fx5pzp9t.fsf@benfinney.id.au> Paul Rubin writes: > Ben Finney writes: > > So how should I be sorting a list with entries of ?unequal types? > > such that it will work in Python 3? > > Um, what ordering do you want? The same ordering I'd get in Python 2; that is, determined by the types of the elements. Peter Otten <__peter__ at web.de> writes: > I can't find the relevant part of the 2.6 documentation, but something like > > >>> def key(x): > ... t = type(x) > ... t = compat.get(t, t) > ... return t.__name__, id(t), x > ... > >>> compat = {bool: float, int: float} The problem with this approach is that it assumes I know all the types of elements that will be sorted; I don't. I want to sort a list coming from some other part of the code, and I don't want to arbitrarily limit the types of the list elements. Essentially, I want the same behaviour I'd get from Python 2's sort: the order of the elements is determined by the ordering built into the types of the elements themselves, with sensible defaults. -- \ ?I distrust those people who know so well what God wants them | `\ to do to their fellows, because it always coincides with their | _o__) own desires.? ?Susan Brownell Anthony, 1896 | Ben Finney From davea at ieee.org Thu Jan 28 20:15:29 2010 From: davea at ieee.org (Dave Angel) Date: Thu, 28 Jan 2010 20:15:29 -0500 Subject: For loop searching takes too long! In-Reply-To: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> Message-ID: <4B6236B1.1040208@ieee.org> elsa wrote: > Hi guys, > > I've got a problem with my program, in that the code just takes too > long to run. Here's what I'm doing. If anyone has any tips, they'd be > much appreciated! > > So, say I have a list of lists that looks something like this (I'm > using a list of lists, rather than a list of tuples, as I need it to > be mutable): > > myList = [[786,0],[45, 1],[673,1],...................[23,46]] > > there are enough entries in the outer list, that the sum of myList[i] > [0] across all i could be as high as 10^7. > > Now, what I need to do is randomly choose one myList[i], however the > distribution of my random choice needs to be proportional to the > values of myList[i][0]. So, for this list above, I'd have a much > higher chance of choosing myList[0] than myList[1]. > > Here is how I'm doing it at the moment: > > def chooseI(myList): > mySum=0 > choice = random.choice(range(1,sum([i[0] for i in myList])+1)) > for i in range(len(myList)): > mySum+=myList[i][0] > if mySum>=choice: > return i > break > > This works just fine if sum([i[0] for i in myList]) is less than > 10,000, say. However if its up around 10^7, the whole thing crashes. > Is there a faster way of doing this, that doesn't involve as many > computational steps? > > Thanks! > > elsa > > "the whole thing crashes" -- give us the specific crash you see, including the stack trace. "Crash" is far too vague. At first glance you could be running out of memory. But not with the values you're quoting. If a typical average value for i[0] is 50, that's only 200k elements. Anyway, you could print out the len(myList) to see how big the list is. There are quicker ways to do the sum, but the first thing to change is the random.choice() stuff. Once you have a sum, you can use random.randint() on it directly. No need to make a range list. Another approach might be to build a new list with a running sum in it. Then after doing the randint() on the last (highest) item, you can do a bisect.bisect on that list. The index that returns will be your return value. DaveA From steve at REMOVE-THIS-cybersource.com.au Thu Jan 28 20:20:03 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 01:20:03 GMT Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <87tyu529tl.fsf@benfinney.id.au> Message-ID: <03722429$0$1309$c3e8da3@news.astraweb.com> On Fri, 29 Jan 2010 08:33:58 +1100, Ben Finney wrote: > Antoine Pitrou writes: > >> Le Thu, 28 Jan 2010 00:19:24 +0000, Steven D'Aprano a ?crit?: >> > 4. Python 3 will make you irresistible to women. >> > >> > FALSE - Python 3 coders are no more likely to get a date than any >> > other programmer. >> >> They spend less time coding, so they /can/ get more "dates" (what a >> strange English word) :-) > > Perhaps Steven could tell you about a lovely Australian meaning for the > word ?date?. This is a family list, so perhaps I shouldn't. :) In Australia slang, "date" is short for "date hole", which is the part of the anatomy which is also known as "the [one] brown eye". In parts of the US, it is also known as the "corn hole", and in Cockney rhyming slang it is a jam role. I trust I don't need to be any more explicit... -- Steven From sccolbert at gmail.com Thu Jan 28 20:20:22 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Thu, 28 Jan 2010 20:20:22 -0500 Subject: For loop searching takes too long! In-Reply-To: <03721fab$0$1309$c3e8da3@news.astraweb.com> References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> <03721fab$0$1309$c3e8da3@news.astraweb.com> Message-ID: <7f014ea61001281720l3d06273cwc0b1f7f9c500a445@mail.gmail.com> if you're open to other libraries, this could be done extremely fast in numpy. On my machine summing that whole array takes 75ms. On Thu, Jan 28, 2010 at 8:00 PM, Steven D'Aprano < steve at remove-this-cybersource.com.au> wrote: > On Thu, 28 Jan 2010 15:52:14 -0800, elsa wrote: > > > Now, what I need to do is randomly choose one myList[i], however the > > distribution of my random choice needs to be proportional to the values > > of myList[i][0]. So, for this list above, I'd have a much higher chance > > of choosing myList[0] than myList[1]. > > > > Here is how I'm doing it at the moment: > > > > def chooseI(myList): > > mySum=0 > > choice = random.choice(range(1,sum([i[0] for i in myList])+1)) > > for i in range(len(myList)): > > mySum+=myList[i][0] > > if mySum>=choice: > > return i > > break > > This isn't related to your problem, but you don't need the break after > the return -- the return will leave the loop and the function, and the > break will never be reached. > > You could probably speed the code up a little by changing all the calls > to range into xrange. range actually generates a list of integers, which > is time consuming, while xrange is a lazy generator which just produces > each integer one at a time. (You can ignore this if you are using Python > 3.0 or 3.1.) > > Another small optimization you could use is to use a generator expression > instead of a list comprehension inside the call to sum. That should > generate the sum lazily, instead of calculating a giant list first and > then adding it up. > > But I'd try something like this: > > # Untested. > def chooseI(myList): > total = sum(i[0] for i in myList) > mySum = 0 > choice = random.randrange(total) > for i in myList: > mySum += i[0] > if mySum >= choice: > return i > > > > -- > Steven > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at REMOVE-THIS-cybersource.com.au Thu Jan 28 20:24:33 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 01:24:33 GMT Subject: Sorting a list with entries of unequal types References: <87sk9pzsmm.fsf@benfinney.id.au> <87fx5pzp9t.fsf@benfinney.id.au> Message-ID: <03722537$0$1309$c3e8da3@news.astraweb.com> On Fri, 29 Jan 2010 12:13:50 +1100, Ben Finney wrote: > Paul Rubin writes: > >> Ben Finney writes: >> > So how should I be sorting a list with entries of ?unequal types? >> > such that it will work in Python 3? >> >> Um, what ordering do you want? > > The same ordering I'd get in Python 2; that is, determined by the types > of the elements. The ordering has not been consistent across minor versions of Python 2: $ python2.0 Python 2.0.1 (#1, Jan 14 2010, 15:43:17) [GCC 4.1.2 20070925 (Red Hat 4.1.2-27)] on linux2 Type "copyright", "credits" or "license" for more information. >>> L = [None, {}, [], 1, "x"] >>> L.sort() >>> L [1, None, {}, [], 'x'] $ python2.6 Python 2.6.1 (r261:67515, Dec 24 2008, 00:33:13) [GCC 4.1.2 20070502 (Red Hat 4.1.2-12)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> L = [None, {}, [], 1, "x"] >>> L.sort() >>> L [None, 1, {}, [], 'x'] Python has never guaranteed a particular ordering for unequal types, only that it would be consistent between repeated calls to sort. -- Steven From steve at REMOVE-THIS-cybersource.com.au Thu Jan 28 20:30:05 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 01:30:05 GMT Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <4CD97E94-B40E-481D-B367-5FCE2820F48D@roalddevries.nl> Message-ID: <03722683$0$1309$c3e8da3@news.astraweb.com> On Thu, 28 Jan 2010 17:01:38 +0100, Roald de Vries wrote: > Question out of general interest in the language: If I would want to > generate such functions in a for-loop, what would I have to do? This > doesn't work: > > class Move(object): > def __call__(self, direction): > return direction > > move = Move() > > for f in ['up', 'down', 'right', 'left']: > move.__dict__[f] = lambda: move(f) > > ... because now 'move.up()' returns 'left' because thats the current > value of f. Is there a way to 'expand' f in the loop? Or a reason that > you never should use this? Possibly the simplest way is to use Python's handling of default values to get the result you want: for f in ['up', 'down', 'right', 'left']: move.__dict__[f] = lambda f=f: move(f) BTW, there's no need to explicitly reference move.__dict__: for f in ['up', 'down', 'right', 'left']: setattr(move, f, lambda f=f: move(f)) -- Steven From mdj.mdj at gmail.com Thu Jan 28 20:38:23 2010 From: mdj.mdj at gmail.com (mdj) Date: Thu, 28 Jan 2010 17:38:23 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> Message-ID: <184f0182-acbc-4a85-a8d9-7f813ebd2eb0@k2g2000pro.googlegroups.com> On Jan 29, 9:47?am, Paul Boddie wrote: > On 27 Jan, 13:26, Xah Lee wrote: > > > > > So, for practical reasons, i think a ?key? parameter is fine. But > > chopping off ?cmp? is damaging. When your data structure is complex, > > its order is not embedded in some ?key?. Taking out ?cmp? makes it > > impossible to sort your data structure. > > What would annoy me if I used Python 3.x would be the apparent lack of > the __cmp__ method for conveniently defining comparisons between > instances of my own classes. Having to define all the rich comparison > methods frequently isn't even as much fun as it sounds. OT, but you can always define the other operators in terms of a cmp and mix it in, restoring the original behaviour. Unfortunately it won't restore the original performance until someone comes to their senses and restores __cmp__ Matt From ben+python at benfinney.id.au Thu Jan 28 21:16:11 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 29 Jan 2010 13:16:11 +1100 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <87tyu529tl.fsf@benfinney.id.au> <03722429$0$1309$c3e8da3@news.astraweb.com> Message-ID: <87ockdy7tg.fsf@benfinney.id.au> Steven D'Aprano writes: > On Fri, 29 Jan 2010 08:33:58 +1100, Ben Finney wrote: > > Perhaps Steven could tell you about a lovely Australian meaning for > > the word ?date?. > > This is a family list, so perhaps I shouldn't. :) > > In Australia slang, "date" is short for "date hole", which is the part > of the anatomy which is also known as "the [one] brown eye". In parts > of the US, it is also known as the "corn hole", and in Cockney rhyming > slang it is a jam role. > > I trust I don't need to be any more explicit... I think the reason ?date? was initially used is because dates are most familiar to us as fleshy, dark brown, wrinkled, compressed points. My interests in etymology and scatology unite here. -- \ ?In the long run, the utility of all non-Free software | `\ approaches zero. All non-Free software is a dead end.? ?Mark | _o__) Pilgrim, 2006 | Ben Finney From steve at REMOVE-THIS-cybersource.com.au Thu Jan 28 21:21:28 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 02:21:28 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> <184f0182-acbc-4a85-a8d9-7f813ebd2eb0@k2g2000pro.googlegroups.com> Message-ID: <0372328c$0$1309$c3e8da3@news.astraweb.com> On Thu, 28 Jan 2010 17:38:23 -0800, mdj wrote: > On Jan 29, 9:47?am, Paul Boddie wrote: >> On 27 Jan, 13:26, Xah Lee wrote: >> >> >> >> > So, for practical reasons, i think a ?key? parameter is fine. But >> > chopping off ?cmp? is damaging. When your data structure is complex, >> > its order is not embedded in some ?key?. Taking out ?cmp? makes it >> > impossible to sort your data structure. >> >> What would annoy me if I used Python 3.x would be the apparent lack of >> the __cmp__ method for conveniently defining comparisons between >> instances of my own classes. Having to define all the rich comparison >> methods frequently isn't even as much fun as it sounds. > > OT, but you can always define the other operators in terms of a cmp and > mix it in, restoring the original behaviour. Unfortunately it won't > restore the original performance until someone comes to their senses and > restores __cmp__ "Comes to their senses"? There's nothing you can do with __cmp__ that you can't do better with rich comparisons, and plenty that rich comparisons can do that __cmp__ is utterly incapable of dealing with. __cmp__ is crippled since it can only be used for defining classes where the operators < etc return flags. It can't be used if you want to implement some other behaviour for the operators. E.g. here's a silly example: >>> class Silly(object): ... def __init__(self): ... self.link = None ... def __gt__(self, other): ... self.link = other ... >>> x = Silly() >>> x > Silly() >>> x.link <__main__.X object at 0xb7cda74c> More importantly, __cmp__ is only suitable for classes that implement total ordering. If you have a data type that does not have total ordering, for example sets, you can't implement it using __cmp__. E.g.: >>> s = set([1, 2, 3, 4]) >>> t = set([3, 4, 5, 6]) >>> s < t False >>> s > t False >>> s == t False >>> cmp(s, t) Traceback (most recent call last): File "", line 1, in TypeError: cannot compare sets using cmp() Sets have partial ordering, and __cmp__ is simply not up to the job of dealing with it. Having two mechanisms for implementing comparisons is unnecessary. It adds complications to the language that we are better off without. The only advantage of the obsolete __cmp__ is that lazy programmers only need to write one method instead of six. This is an advantage, I accept that (hey, I'm a lazy programmer too, that's why I use Python!) but it's not a big advantage. If you really care about it you can create a mixin class, a decorator, or a metaclass to simplify creation of the methods. For example, a quick and dirty decorator: >>> def make_comparisons(cls): ... cls.__gt__ = lambda self, other: self.__cmp__(other) == 1 ... cls.__ge__ = lambda self, other: self.__cmp__(other) >= 0 ... cls.__eq__ = lambda self, other: self.__cmp__(other) == 0 ... cls.__ne__ = lambda self, other: self.__cmp__(other) != 0 ... cls.__le__ = lambda self, other: self.__cmp__(other) <= 0 ... cls.__lt__ = lambda self, other: self.__cmp__(other) == -1 ... return cls ... >>> @make_comparisons ... class BiggerThanEverything(object): ... def __cmp__(self, other): ... return 1 ... >>> x = BiggerThanEverything() >>> x > 1000 True >>> x < 0 False -- Steven From steve at holdenweb.com Thu Jan 28 21:22:38 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 28 Jan 2010 21:22:38 -0500 Subject: For loop searching takes too long! In-Reply-To: <4B6236B1.1040208@ieee.org> References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> <4B6236B1.1040208@ieee.org> Message-ID: Dave Angel wrote: > elsa wrote: >> Hi guys, >> >> I've got a problem with my program, in that the code just takes too >> long to run. Here's what I'm doing. If anyone has any tips, they'd be >> much appreciated! >> >> So, say I have a list of lists that looks something like this (I'm >> using a list of lists, rather than a list of tuples, as I need it to >> be mutable): >> >> myList = [[786,0],[45, 1],[673,1],...................[23,46]] >> >> there are enough entries in the outer list, that the sum of myList[i] >> [0] across all i could be as high as 10^7. >> >> Now, what I need to do is randomly choose one myList[i], however the >> distribution of my random choice needs to be proportional to the >> values of myList[i][0]. So, for this list above, I'd have a much >> higher chance of choosing myList[0] than myList[1]. >> >> Here is how I'm doing it at the moment: >> >> def chooseI(myList): >> mySum=0 >> choice = random.choice(range(1,sum([i[0] for i in myList])+1)) >> for i in range(len(myList)): >> mySum+=myList[i][0] >> if mySum>=choice: >> return i >> break >> >> This works just fine if sum([i[0] for i in myList]) is less than >> 10,000, say. However if its up around 10^7, the whole thing crashes. >> Is there a faster way of doing this, that doesn't involve as many >> computational steps? >> >> Thanks! >> >> elsa >> >> > "the whole thing crashes" -- give us the specific crash you see, > including the stack trace. "Crash" is far too vague. > > At first glance you could be running out of memory. But not with the > values you're quoting. If a typical average value for i[0] is 50, > that's only 200k elements. Anyway, you could print out the len(myList) > to see how big the list is. > > There are quicker ways to do the sum, but the first thing to change is > the random.choice() stuff. Once you have a sum, you can use > random.randint() on it directly. No need to make a range list. > > Another approach might be to build a new list with a running sum in it. > Then after doing the randint() on the last (highest) item, you can do a > bisect.bisect on that list. The index that returns will be your return > value. > Your approach seems to validate an informal impression I had that with N choices one ought to be able to build a binary tree where each decision went to left or right according to whether a number drawn from a linear [0,1) distribution was greater that some arbitrary margin, or not. There are then arithmetical questions of exactly how to arrive at the correct threshold values for each bifurcation. If the most probable paths are deliberately placed at the top of the tree then the most frequent cases will be fastest to generate, also. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Thu Jan 28 21:26:04 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 28 Jan 2010 21:26:04 -0500 Subject: myths about python 3 In-Reply-To: <03722429$0$1309$c3e8da3@news.astraweb.com> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <87tyu529tl.fsf@benfinney.id.au> <03722429$0$1309$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > On Fri, 29 Jan 2010 08:33:58 +1100, Ben Finney wrote: > >> Antoine Pitrou writes: >> >>> Le Thu, 28 Jan 2010 00:19:24 +0000, Steven D'Aprano a ?crit : >>>> 4. Python 3 will make you irresistible to women. >>>> >>>> FALSE - Python 3 coders are no more likely to get a date than any >>>> other programmer. >>> They spend less time coding, so they /can/ get more "dates" (what a >>> strange English word) :-) >> Perhaps Steven could tell you about a lovely Australian meaning for the >> word ?date?. > > This is a family list, so perhaps I shouldn't. :) > > In Australia slang, "date" is short for "date hole", which is the part of > the anatomy which is also known as "the [one] brown eye". In parts of the > US, it is also known as the "corn hole", and in Cockney rhyming slang it > is a jam role. > > I trust I don't need to be any more explicit... > > > My God, and I just nominated you for membership of the PSF. Trust an Australian to descend to normally unplumbed depths (pun intended) the very second you stake your reputation on them. I guess that means I know my place ... there-goes-my-american-citizenship-ly y'rs - steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From roy at panix.com Thu Jan 28 21:26:43 2010 From: roy at panix.com (Roy Smith) Date: Thu, 28 Jan 2010 21:26:43 -0500 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> <7xockfjcsv.fsf@ruckus.brouhaha.com> <6b89dc67-ceee-4b83-a11f-c7376185dcd1@b9g2000pri.googlegroups.com> <7xy6ji62vx.fsf@ruckus.brouhaha.com> <03720b25$0$1309$c3e8da3@news.astraweb.com> Message-ID: In article <03720b25$0$1309$c3e8da3 at news.astraweb.com>, Steven D'Aprano wrote: > In any case, while such a idiom works in code, it plays havoc with the > interactive interpreter: > > >>> while 1: > ... "pass" > ... > 'pass' > 'pass' > 'pass' > 'pass' > 'pass' It's not particularly useful without the quotes either :-) From roy at panix.com Thu Jan 28 21:30:45 2010 From: roy at panix.com (Roy Smith) Date: Thu, 28 Jan 2010 21:30:45 -0500 Subject: python 3's adoption References: Message-ID: In article , "Alf P. Steinbach" wrote: > * Roy Smith: > > In article , > > Mitchell L Model wrote: > > > >> I use the sep and end keywords all the time. > > > > What are 'sep' and 'end'? I'm looking in > > http://docs.python.org/3.1/genindex-all.html and don't see those mentioned > > at all. Am I just looking in the wrong place? > > >>> print( print.__doc__ ) > print(value, ..., sep=' ', end='\n', file=sys.stdout) > > Prints the values to a stream, or to sys.stdout by default. > Optional keyword arguments: > file: a file-like object (stream); defaults to the current sys.stdout. > sep: string inserted between values, default a space. > end: string appended after the last value, default a newline. > >>> _ I'm inclined to call it a docs bug that these keywords are not in the HTML index. Yes? From wuwei23 at gmail.com Thu Jan 28 21:38:20 2010 From: wuwei23 at gmail.com (alex23) Date: Thu, 28 Jan 2010 18:38:20 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> Message-ID: <8a7053f7-43ad-4766-a106-ef1b94826028@k2g2000pro.googlegroups.com> Joan Miller wrote: > Does anybody knows any library to help me to avoid the use of the main > quotes, and brackets? > I would to use anything as: > $ ls /home => run("ls /home") It's not a library, but IPython[1] provides a lot of what you're after: IPython 0.9.1 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object'. ?object also works, ?? prints more. In [1]: ls /home bb/ dcallan/ ehornsby/ ftp/ uqamartl/ uqckorte/ uqmtrev2/ In [2]: path = '/home' In [3]: ls $path bb/ dcallan/ ehornsby/ ftp/ uqamartl/ uqckorte/ uqmtrev2/ In [4]: output = !ls $path In [5]: output Out[5]: SList (.p, .n, .l, .s, .grep(), .fields(), sort() available): 0: bb 1: dcallan 2: ehornsby 3: ftp 4: uqamartl 5: uqckorte 6: uqmtrev2 In [6]: output[6] Out[6]: 'uqmtrev2' You'll need to run your scripts with IPython, so this may not be a solution if you plan on distributing them. [1] http://ipython.scipy.org/ From wuwei23 at gmail.com Thu Jan 28 21:56:11 2010 From: wuwei23 at gmail.com (alex23) Date: Thu, 28 Jan 2010 18:56:11 -0800 (PST) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <284dc9bc-6e7f-47d2-9982-a09dc3cb9dc6@b9g2000pri.googlegroups.com> Terry Reedy wrote: > This statement was to counter the 'myth' that US was only targeted at > 2.x when the current situation is quite the opposite. Not so much 'myth' as 'outdated information', they were very clear that 2.x was the initial target. > In particular, several people said that the speed/space traceoff > should be optional, and that compilation 'without llvm' should really > be without, not just with llvm present but disabled. Instead of arguing, > Colin went ahead and patched the build process to make it be this way. Ah, that's excellent. The impression being given off is that it's a total replacement. > I have no idea. It will have to improve its speedup more before > adoption. I will not be surprised if that happens. It's not so much about being surprised or not, it's wanting actual evidence and not just claims, and moreso _extensive real world usage_ before it's integrated. This seems far more intimate a change than adding a module to the stdlib, I expect it to have at _least_ the evaluation time & vague consensus of approval expected of those. > US is not a new or separate interpreter. It will be an optional jit > replacement for one component of CPython, the eval loop. All the code > for builting functions, types, and modules will be untouched, as will > their big O performance characteristics. As long as there aren't any related decreases in performance in other areas, I'll be happy. > If you can still have a binary free of the traceoff, why would you care? Well, I didn't know I could, so now I don't quite as much :) > They claim they have pretty well fixed that. They know that complete > Windows support, including 64 bit versions, is a necessity. Maybe I'll be a lot more convinced after the Q4 report. The 'incomplete' Windows support may not be as big an issue as I thought, it seems to refer to a lack of support for older Windows versions rather than an incomplete implementation on the supported ones. Cheers, Terry, this addressed a lot of my concerns, although I'm still keen to see more facts & real-world usage over custom-crafted benchmarks and enthusiastic claims. From steve at holdenweb.com Thu Jan 28 22:02:01 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 28 Jan 2010 22:02:01 -0500 Subject: For loop searching takes too long! In-Reply-To: References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> <4B6236B1.1040208@ieee.org> Message-ID: Steve Holden wrote: > Dave Angel wrote: >> elsa wrote: >>> Hi guys, >>> >>> I've got a problem with my program, in that the code just takes too >>> long to run. Here's what I'm doing. If anyone has any tips, they'd be >>> much appreciated! >>> >>> So, say I have a list of lists that looks something like this (I'm >>> using a list of lists, rather than a list of tuples, as I need it to >>> be mutable): >>> >>> myList = [[786,0],[45, 1],[673,1],...................[23,46]] >>> >>> there are enough entries in the outer list, that the sum of myList[i] >>> [0] across all i could be as high as 10^7. >>> >>> Now, what I need to do is randomly choose one myList[i], however the >>> distribution of my random choice needs to be proportional to the >>> values of myList[i][0]. So, for this list above, I'd have a much >>> higher chance of choosing myList[0] than myList[1]. >>> >>> Here is how I'm doing it at the moment: >>> >>> def chooseI(myList): >>> mySum=0 >>> choice = random.choice(range(1,sum([i[0] for i in myList])+1)) >>> for i in range(len(myList)): >>> mySum+=myList[i][0] >>> if mySum>=choice: >>> return i >>> break >>> >>> This works just fine if sum([i[0] for i in myList]) is less than >>> 10,000, say. However if its up around 10^7, the whole thing crashes. >>> Is there a faster way of doing this, that doesn't involve as many >>> computational steps? >>> >>> Thanks! >>> >>> elsa >>> >>> >> "the whole thing crashes" -- give us the specific crash you see, >> including the stack trace. "Crash" is far too vague. >> >> At first glance you could be running out of memory. But not with the >> values you're quoting. If a typical average value for i[0] is 50, >> that's only 200k elements. Anyway, you could print out the len(myList) >> to see how big the list is. >> >> There are quicker ways to do the sum, but the first thing to change is >> the random.choice() stuff. Once you have a sum, you can use >> random.randint() on it directly. No need to make a range list. >> >> Another approach might be to build a new list with a running sum in it. >> Then after doing the randint() on the last (highest) item, you can do a >> bisect.bisect on that list. The index that returns will be your return >> value. >> > Your approach seems to validate an informal impression I had that with > N choices one ought to be able to build a binary tree where each > decision went to left or right according to whether a number drawn from > a linear [0,1) distribution was greater that some arbitrary margin, or not. > > There are then arithmetical questions of exactly how to arrive at the > correct threshold values for each bifurcation. If the most probable > paths are deliberately placed at the top of the tree then the most > frequent cases will be fastest to generate, also. > Bad form, I know, to reply to oneself, but since I was egotistical enough to read what I wrote again when it was published I must also record the conjecture that the resulting alphabet, expressed as the binary history of the bifurcations chosen for any given symbol, must surely be a Hamming code to be most efficient. No proof is adduced. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Thu Jan 28 22:03:00 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 28 Jan 2010 22:03:00 -0500 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> <7xockfjcsv.fsf@ruckus.brouhaha.com> <6b89dc67-ceee-4b83-a11f-c7376185dcd1@b9g2000pri.googlegroups.com> <7xy6ji62vx.fsf@ruckus.brouhaha.com> <03720b25$0$1309$c3e8da3@news.astraweb.com> Message-ID: Roy Smith wrote: > In article <03720b25$0$1309$c3e8da3 at news.astraweb.com>, > Steven D'Aprano wrote: > >> In any case, while such a idiom works in code, it plays havoc with the >> interactive interpreter: >> >>>>> while 1: >> ... "pass" >> ... >> 'pass' >> 'pass' >> 'pass' >> 'pass' >> 'pass' > > It's not particularly useful without the quotes either :-) Perhaps not. but at least it keeps its misery to itself. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From davea at ieee.org Thu Jan 28 22:09:06 2010 From: davea at ieee.org (Dave Angel) Date: Thu, 28 Jan 2010 22:09:06 -0500 Subject: For loop searching takes too long! In-Reply-To: References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> <4B6236B1.1040208@ieee.org> Message-ID: <4B625152.7050403@ieee.org> Steve Holden wrote: > Dave Angel wrote: > >> elsa wrote: >> >>> Hi guys, >>> >>> I've got a problem with my program, in that the code just takes too >>> long to run. Here's what I'm doing. If anyone has any tips, they'd be >>> much appreciated! >>> >>> So, say I have a list of lists that looks something like this (I'm >>> using a list of lists, rather than a list of tuples, as I need it to >>> be mutable): >>> >>> myList = [[786,0],[45, 1],[673,1],...................[23,46]] >>> >>> there are enough entries in the outer list, that the sum of myList[i] >>> [0] across all i could be as high as 10^7. >>> >>> Now, what I need to do is randomly choose one myList[i], however the >>> distribution of my random choice needs to be proportional to the >>> values of myList[i][0]. So, for this list above, I'd have a much >>> higher chance of choosing myList[0] than myList[1]. >>> >>> Here is how I'm doing it at the moment: >>> >>> def chooseI(myList): >>> mySum=0 >>> choice = random.choice(range(1,sum([i[0] for i in myList])+1)) >>> for i in range(len(myList)): >>> mySum+=myList[i][0] >>> if mySum>=choice: >>> return i >>> break >>> >>> This works just fine if sum([i[0] for i in myList]) is less than >>> 10,000, say. However if its up around 10^7, the whole thing crashes. >>> Is there a faster way of doing this, that doesn't involve as many >>> computational steps? >>> >>> Thanks! >>> >>> elsa >>> >>> >>> >> "the whole thing crashes" -- give us the specific crash you see, >> including the stack trace. "Crash" is far too vague. >> >> At first glance you could be running out of memory. But not with the >> values you're quoting. If a typical average value for i[0] is 50, >> that's only 200k elements. Anyway, you could print out the len(myList) >> to see how big the list is. >> >> There are quicker ways to do the sum, but the first thing to change is >> the random.choice() stuff. Once you have a sum, you can use >> random.randint() on it directly. No need to make a range list. >> >> Another approach might be to build a new list with a running sum in it. >> Then after doing the randint() on the last (highest) item, you can do a >> bisect.bisect on that list. The index that returns will be your return >> value. >> >> > Your approach seems to validate an informal impression I had that with > N choices one ought to be able to build a binary tree where each > decision went to left or right according to whether a number drawn from > a linear [0,1) distribution was greater that some arbitrary margin, or not. > > There are then arithmetical questions of exactly how to arrive at the > correct threshold values for each bifurcation. If the most probable > paths are deliberately placed at the top of the tree then the most > frequent cases will be fastest to generate, also. > > regards > If the list will remain the same, it may be worth pre-computing the running sum into it, and then the function to get a random value just has to do the bisect. But as you say, if the values are not evenly distributed, a binary split (which is what bisect does) might not give the best average time. I doubt if it would help for small lists (Python code would be slower than the presumably C code in bisect), but perhaps for larger lists, an index something like this would be better: Assuming the list had 40 items, and the sum was 1000 for the sake of example. you'd build a tree with 500 as the top node. that would "point" not to 20, but to whatever index had a value that was closest to 500. Then you'd build child nodes that similarly split each portion. And you'd repeat it till the remaining size was fairly small, at which point you'd use standard bisect logic. On a different tack, someone could probably improve on the bisect() function, for cases where the list is a sorted list of numbers. Typically a binary algorithm is used because a comparison function gives either less than, equal, or greater. But if you know they are all well-behaved numbers, you could do a formula to narrow down the choices faster. Let's say that at some point you know the value is between items[5] and items[105], and the values respectively are 40 and 140. You're searching for a value of 50. Instead of trying items[55], you do a scale factor, and do 5 + ( 105-5 ) * (50-40) / (140-40). So you'd try element 15 next. All these values are plus/minus 1, but I think I've come close to an algorithm. Kind of an analogy to Newton's method. DaveA From steve at holdenweb.com Thu Jan 28 22:15:53 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 28 Jan 2010 22:15:53 -0500 Subject: python 3's adoption In-Reply-To: <0372328c$0$1309$c3e8da3@news.astraweb.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> <184f0182-acbc-4a85-a8d9-7f813ebd2eb0@k2g2000pro.googlegroups.com> <0372328c$0$1309$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > On Thu, 28 Jan 2010 17:38:23 -0800, mdj wrote: > >> On Jan 29, 9:47 am, Paul Boddie wrote: >>> On 27 Jan, 13:26, Xah Lee wrote: >>> >>> >>> >>>> So, for practical reasons, i think a ?key? parameter is fine. But >>>> chopping off ?cmp? is damaging. When your data structure is complex, >>>> its order is not embedded in some ?key?. Taking out ?cmp? makes it >>>> impossible to sort your data structure. >>> What would annoy me if I used Python 3.x would be the apparent lack of >>> the __cmp__ method for conveniently defining comparisons between >>> instances of my own classes. Having to define all the rich comparison >>> methods frequently isn't even as much fun as it sounds. >> OT, but you can always define the other operators in terms of a cmp and >> mix it in, restoring the original behaviour. Unfortunately it won't >> restore the original performance until someone comes to their senses and >> restores __cmp__ > > "Comes to their senses"? > > There's nothing you can do with __cmp__ that you can't do better with > rich comparisons, and plenty that rich comparisons can do that __cmp__ is > utterly incapable of dealing with. __cmp__ is crippled since it can only > be used for defining classes where the operators < etc return flags. It > can't be used if you want to implement some other behaviour for the > operators. E.g. here's a silly example: > >>>> class Silly(object): > ... def __init__(self): > ... self.link = None > ... def __gt__(self, other): > ... self.link = other > ... >>>> x = Silly() >>>> x > Silly() >>>> x.link > <__main__.X object at 0xb7cda74c> > > > More importantly, __cmp__ is only suitable for classes that implement > total ordering. If you have a data type that does not have total > ordering, for example sets, you can't implement it using __cmp__. > > E.g.: > >>>> s = set([1, 2, 3, 4]) >>>> t = set([3, 4, 5, 6]) >>>> s < t > False >>>> s > t > False >>>> s == t > False >>>> cmp(s, t) > Traceback (most recent call last): > File "", line 1, in > TypeError: cannot compare sets using cmp() > > > Sets have partial ordering, and __cmp__ is simply not up to the job of > dealing with it. > > Having two mechanisms for implementing comparisons is unnecessary. It > adds complications to the language that we are better off without. The > only advantage of the obsolete __cmp__ is that lazy programmers only need > to write one method instead of six. This is an advantage, I accept that > (hey, I'm a lazy programmer too, that's why I use Python!) but it's not a > big advantage. If you really care about it you can create a mixin class, > a decorator, or a metaclass to simplify creation of the methods. For > example, a quick and dirty decorator: > > >>>> def make_comparisons(cls): > ... cls.__gt__ = lambda self, other: self.__cmp__(other) == 1 > ... cls.__ge__ = lambda self, other: self.__cmp__(other) >= 0 > ... cls.__eq__ = lambda self, other: self.__cmp__(other) == 0 > ... cls.__ne__ = lambda self, other: self.__cmp__(other) != 0 > ... cls.__le__ = lambda self, other: self.__cmp__(other) <= 0 > ... cls.__lt__ = lambda self, other: self.__cmp__(other) == -1 > ... return cls > ... >>>> @make_comparisons > ... class BiggerThanEverything(object): > ... def __cmp__(self, other): > ... return 1 > ... >>>> x = BiggerThanEverything() >>>> x > 1000 > True >>>> x < 0 > False > > While I am fully aware that premature optimization, etc., but I cannot resist an appeal to efficiency if it finally kills off this idea that "they took 'cmp()' away" is a bad thing. Passing a cmp= argument to sort provides the interpreter with a function that will be called each time any pair of items have to be compared. The key= argument, however, specifies a transformation from [x0, x1, ..., xN] to [(key(x0), x0), (key(x1), x1), ..., (key(xN), xN)] (which calls the key function precisely once per sortable item). >From a C routine like sort() [in CPython, anyway] calling out from C to a Python function to make a low-level decision like "is A less than B?" turns out to be disastrous for execution efficiency (unlike the built-in default comparison, which can be called directly from C in CPython). If your data structures have a few hundred items in them it isn't going to make a huge difference. If they have a few million thenit is already starting to affect performance ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From mdj.mdj at gmail.com Thu Jan 28 23:38:22 2010 From: mdj.mdj at gmail.com (mdj) Date: Thu, 28 Jan 2010 20:38:22 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> <184f0182-acbc-4a85-a8d9-7f813ebd2eb0@k2g2000pro.googlegroups.com> <0372328c$0$1309$c3e8da3@news.astraweb.com> Message-ID: On Jan 29, 12:21?pm, Steven D'Aprano wrote: > "Comes to their senses"? > > There's nothing you can do with __cmp__ that you can't do better with > rich comparisons, and plenty that rich comparisons can do that __cmp__ is > utterly incapable of dealing with. __cmp__ is crippled since it can only > be used for defining classes where the operators < etc return flags. It > can't be used if you want to implement some other behaviour for the > operators. E.g. here's a silly example: [example snipped] Okay, but I think that overloading to change the semantic meaning of operators is another issue, and one that there's a great deal of difference of opinion on. Personally, I find arbitrarily redefining operators with well understood semantics to mean something totally different to be poor style, and those who disagree would find my style poor :-) > Having two mechanisms for implementing comparisons is unnecessary. It > adds complications to the language that we are better off without. The > only advantage of the obsolete __cmp__ is that lazy programmers only need > to write one method instead of six. This is an advantage, I accept that > (hey, I'm a lazy programmer too, that's why I use Python!) but it's not a > big advantage. If you really care about it you can create a mixin class, > a decorator, or a metaclass to simplify creation of the methods. For > example, a quick and dirty decorator: I agree it's unnecessary, but deleting a time tested idiom in the name of consistency seems excessive to me. Is there not a compromise? For those of us who prefer limiting operator overloads to semantically compatible cases the extra flexibility is seldom necessary, and the price extracted in terms of inconvenience is therefore quite high. You can (as I mentioned and you demonstrated) work around the problem, but I'd hate to see 50 different implementations of a common idiom wasting brain space in future projects. Matt (Note I remove the crosspost to comp.lang.lisp where I saw this - one of the reasons I love lisp is that discussions like this become totally unnecessary) From daniel at stutzbachenterprises.com Thu Jan 28 23:51:16 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Thu, 28 Jan 2010 22:51:16 -0600 Subject: For loop searching takes too long! In-Reply-To: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> Message-ID: On Thu, Jan 28, 2010 at 5:52 PM, elsa wrote: > choice = random.choice(range(1,sum([i[0] for i in myList])+1)) > That's the line causing the problem. It's generating all of the numbers between 1 and your sum, then picking one. Try the following instead, which will pick a number between 1 and your sum without generating each and every one. choice = random.randrange(1, sum(i[0] for i in myList)+1) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From timr at probo.com Fri Jan 29 00:21:05 2010 From: timr at probo.com (Tim Roberts) Date: Thu, 28 Jan 2010 21:21:05 -0800 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: John Nagle wrote: > >Arguably, Python 3 has been rejected by the market. Instead, there's >now Python 2.6, Python 2.7, and Python 2.8. Python 3 has turned into >a debacle like Perl 6, now 10 years old. Although I happen to be one of the folks who are reluctant to switch to Python 3, I have to say that this comparison is entirely unfair. Python 3 exists in the wild. It has been released, and has even had a couple of updates. Eventually, it will prevail. Resistance is futile, you WILL be assimilated. Perl 6, on the other hand, is still fantasyware a decade after its announcement. It is, for the most part, THE canonical example of the wrong way to conduct a development effort. -- Tim Roberts, timr at probo.com Providenza & Boekelheide, Inc. From wuwei23 at gmail.com Fri Jan 29 00:33:01 2010 From: wuwei23 at gmail.com (alex23) Date: Thu, 28 Jan 2010 21:33:01 -0800 (PST) Subject: python 3's adoption References: Message-ID: Roy Smith wrote: > I'm inclined to call it a docs bug that these keywords are not in the HTML > index. ?Yes? No. The indices don't list the named parameters for any other built- in, so why would they for print()? From steve at REMOVE-THIS-cybersource.com.au Fri Jan 29 00:34:01 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 05:34:01 GMT Subject: [OT] Perl 6 [was Re: myths about python 3] References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <03725fae$0$1309$c3e8da3@news.astraweb.com> On Thu, 28 Jan 2010 21:21:05 -0800, Tim Roberts wrote: > Perl 6, on the other hand, is still fantasyware a decade after its > announcement. It is, for the most part, THE canonical example of the > wrong way to conduct a development effort. Out of curiosity, and completely off-topic, why has Perl 6 gone so badly? -- Steven From steve at REMOVE-THIS-cybersource.com.au Fri Jan 29 00:34:13 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 05:34:13 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> <7xockfjcsv.fsf@ruckus.brouhaha.com> <6b89dc67-ceee-4b83-a11f-c7376185dcd1@b9g2000pri.googlegroups.com> <7xy6ji62vx.fsf@ruckus.brouhaha.com> <03720b25$0$1309$c3e8da3@news.astraweb.com> Message-ID: <03725fba$0$1309$c3e8da3@news.astraweb.com> On Thu, 28 Jan 2010 21:26:43 -0500, Roy Smith wrote: > In article <03720b25$0$1309$c3e8da3 at news.astraweb.com>, > Steven D'Aprano wrote: > >> In any case, while such a idiom works in code, it plays havoc with the >> interactive interpreter: >> >> >>> while 1: >> ... "pass" >> ... >> 'pass' >> 'pass' >> 'pass' >> 'pass' >> 'pass' > > It's not particularly useful without the quotes either :-) Of course "while 1: pass" is useless, but it doesn't fill the terminal with thousands of lines of "pass". The point I was making is that the statement pass does nothing, while in the interactive interpreter a string or other object prints to the screen. -- Steven From raghavendra.gv.vanam at gmail.com Fri Jan 29 00:36:27 2010 From: raghavendra.gv.vanam at gmail.com (vanam) Date: Thu, 28 Jan 2010 21:36:27 -0800 (PST) Subject: [File Input Module]Replacing string in a file Message-ID: <9d24353e-b48f-4869-9e3b-f5e7555de47f@u15g2000prd.googlegroups.com> Hi all, For replacing string in the file, i have used the module File input for replacing the string in the file. For understanding and execution purpose, i have just included Python as a string in the file and want it to be replaced to PYTHON. Below are my queries and code: (Correct me if my understanding is wrong???????) 1)) import fileinput x = fileinput.input('data.txt',inplace=0) for line in x: line = line.replace('Python','PYTHON) print line, x.close() The above piece of code will not create any backup file but it will replace PYTHON (Print on the console) but not physically write to the file. 2))) import fileinput x = fileinput.input('data.txt',inplace=1) for line in x: line = line.replace('Python','PYTHON') print line, x.close() The above piece of code will create backup file but hidden (in the form of bak file) and it will physically write to the file -- I have verified the contents of data.txt after the file operation and it had written successfully.But why it is not printing line i.e. string in the file on the console. 3))) import fileinput x = fileinput.input('data.txt',inplace=1) for line in x: line = line.replace('Python','PYTHON') x.close() The above piece of code after execution is wiping out the full contents. But addition of print line, is exactly replacing the string, what exactly addition of print is making difference??? 4))) import fileinput x = fileinput.input('data.txt',inplace=1,backup='content.txt') for line in x: line = line.replace('Python','PYTHON') print line, x.close() The above piece is creating a backup file by name data.txtcontent.txt (I am not sure whether created file name is correct or not?) and to the back up file it had added previous content i.e., Python and it had replaced the contents in data.txt to PYTHON From tjreedy at udel.edu Fri Jan 29 00:56:26 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 29 Jan 2010 00:56:26 -0500 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> Message-ID: On 1/28/2010 6:47 PM, Paul Boddie wrote: > On 27 Jan, 13:26, Xah Lee wrote: >> >> So, for practical reasons, i think a ?key? parameter is fine. But >> chopping off ?cmp? is damaging. When your data structure is complex, >> its order is not embedded in some ?key?. Taking out ?cmp? makes it >> impossible to sort your data structure. > > What would annoy me if I used Python 3.x would be the apparent lack of > the __cmp__ method for conveniently defining comparisons between > instances of my own classes. Having to define all the rich comparison > methods frequently isn't even as much fun as it sounds. You define __eq__, which automatically generates __ne__. You define __lt__, which is all sort and heap need. This should be about as easier than __eq__, which is really needed, and __cmp__. If you need the other 3, either copy the recipe in the Cookbook, or, even faster def __ge__(s,o): return o.__lt__(s) def __le__(s,o): return s < o or s == o def __ge__(s,o): return s > o or s == o All done. Terry Jan Reedy From ben+python at benfinney.id.au Fri Jan 29 01:01:36 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 29 Jan 2010 17:01:36 +1100 Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> Message-ID: <87vdelwitb.fsf@benfinney.id.au> Dennis Lee Bieber writes: > On Thu, 28 Jan 2010 11:24:28 -0800 (PST), Joan Miller: > > On 28 ene, 19:16, Josh Holland wrote: > > > Check the docs on os.system(). > > No. I've a function that uses subprocess to run commands on the same > > shell and so substitute to bash scrips. But a script full of run > > ("shell_command --with --arguments") is too verbose. > > I shall blaspheme, and suggest that maybe the language you want > to use is REXX (ooREXX or Regina). Heh. That isn't blasphemy, because no true Pythonista [0] would claim Python to be the god of that domain. It's no sin to say that Python isn't a good choice for specific things; and ?I want to write programs by indistinguishably mixing statements with external system calls? is one of them, IMO. [0] fully aware of , thanks in advance. -- \ ?Natural catastrophes are rare, but they come often enough. We | `\ need not force the hand of nature.? ?Carl Sagan, _Cosmos_, 1980 | _o__) | Ben Finney From steve at REMOVE-THIS-cybersource.com.au Fri Jan 29 01:01:40 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 06:01:40 GMT Subject: [File Input Module]Replacing string in a file References: <9d24353e-b48f-4869-9e3b-f5e7555de47f@u15g2000prd.googlegroups.com> Message-ID: <03726626$0$1309$c3e8da3@news.astraweb.com> On Thu, 28 Jan 2010 21:36:27 -0800, vanam wrote: > Hi all, > > For replacing string in the file, i have used the module File input for > replacing the string in the file. If you're processing a single file, using fileinput is overkill, and slow as well. Here is a quick-and-dirty way of processing one file, with no error checking: text = open("some file", "r").read() # modify the text however you like text = text.replace("ham", "spam") # replace in place open("some file", "w").write(text) In a real application, you would need to do significant error checking. But it shows the basic technique: you read a file, modify it in memory, then write it back to disk. > For understanding and execution purpose, i have just included Python as > a string in the file and want it to be replaced to PYTHON. > > Below are my queries and code: (Correct me if my understanding is > wrong???????) > > 1)) > > import fileinput > x = fileinput.input('data.txt',inplace=0) > for line in x: > line = line.replace('Python','PYTHON) > print line, > x.close() > > The above piece of code will not create any backup file but it will > replace PYTHON (Print on the console) but not physically write to the > file. Correct. > 2))) > > import fileinput > x = fileinput.input('data.txt',inplace=1) > for line in x: > line = line.replace('Python','PYTHON') > print line, > x.close() > > The above piece of code will create backup file but hidden (in the form > of bak file) and it will physically write to the file -- I have verified > the contents of data.txt after the file operation and it had written > successfully.But why it is not printing line i.e. string in the file on > the console. You should read the documentation to fileinput. The inplace=1 argument tells fileinput to redirect stdout to the current file, so the output of print will go to the file instead of the console. Personally, I find that an incredibly unintuitive and dangerous API, and would recommend against fileinput just for that reason alone. > 3))) > > import fileinput > x = fileinput.input('data.txt',inplace=1) > for line in x: > line = line.replace('Python','PYTHON') > x.close() > > The above piece of code after execution is wiping out the full contents. > But addition of print line, is exactly replacing the string, what > exactly addition of print is making difference??? That's how the module works: printing will go to the file instead of the console. Since you don't print anything, the file gets written with the empty string. > 4))) > > import fileinput > x = fileinput.input('data.txt',inplace=1,backup='content.txt') > for line in x: > line = line.replace('Python','PYTHON') print line, > x.close() > > The above piece is creating a backup file by name data.txtcontent.txt Yes, that's write, you told it to use "content.txt" as the file extension on the backup file. So it starts with the existing name, "data.txt" and adds "content.txt" as the extension, giving "data.txtcontent.txt". You really should read the documentation. In the interactive interpreter: import fileinput help(fileinput) > (I am not sure whether created file name is correct or not?) and to the > back up file it had added previous content i.e., Python and it had > replaced the contents in data.txt to PYTHON Is there a question there? -- Steven From debatem1 at gmail.com Fri Jan 29 01:02:46 2010 From: debatem1 at gmail.com (geremy condra) Date: Fri, 29 Jan 2010 01:02:46 -0500 Subject: [OT] Perl 6 [was Re: myths about python 3] In-Reply-To: <03725fae$0$1309$c3e8da3@news.astraweb.com> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <03725fae$0$1309$c3e8da3@news.astraweb.com> Message-ID: On Fri, Jan 29, 2010 at 12:34 AM, Steven D'Aprano wrote: > On Thu, 28 Jan 2010 21:21:05 -0800, Tim Roberts wrote: > >> Perl 6, on the other hand, is still fantasyware a decade after its >> announcement. ?It is, for the most part, THE canonical example of the >> wrong way to conduct a development effort. > > Out of curiosity, and completely off-topic, why has Perl 6 gone so badly? > > > -- > Steven > -- Too much like Perl. Geremy Condra From clp2 at rebertia.com Fri Jan 29 01:07:35 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Thu, 28 Jan 2010 22:07:35 -0800 Subject: [OT] Perl 6 [was Re: myths about python 3] In-Reply-To: <03725fae$0$1309$c3e8da3@news.astraweb.com> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <03725fae$0$1309$c3e8da3@news.astraweb.com> Message-ID: <50697b2c1001282207h7c671f16j1f6ed6064731f970@mail.gmail.com> On Thu, Jan 28, 2010 at 9:34 PM, Steven D'Aprano < steve at remove-this-cybersource.com.au> wrote: > On Thu, 28 Jan 2010 21:21:05 -0800, Tim Roberts wrote: > >> Perl 6, on the other hand, is still fantasyware a decade after its >> announcement. It is, for the most part, THE canonical example of the >> wrong way to conduct a development effort. > > Out of curiosity, and completely off-topic, why has Perl 6 gone so badly? - Super-dramatic changes to the language (some people think Python 3 is/was radical, but it's got nothing on Perl 6) - Complete from-scratch rewrite (c.f. Joel Spolsky's "Things You Should Never Do") - Ambitious implementation strategy, i.e. Parrot. It will be powerful and useful beyond just Perl when complete, but that extra complexity comes at a price. => Although why Pugs and the other implementations are likewise held up, I have no idea. But then I'm not a Perlite (Is that the right word?). For that matter, it remains to be seen whether the Python 3 changeover will fare better. But I'm still pretty hopeful. At least we have a complete reference implementation. Cheers, Chris -- What we need is some sort of coordinated library porting effort. Or a 3.x-specific killer app. http://blog.rebertia.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From alfps at start.no Fri Jan 29 01:10:01 2010 From: alfps at start.no (Alf P. Steinbach) Date: Fri, 29 Jan 2010 07:10:01 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> <184f0182-acbc-4a85-a8d9-7f813ebd2eb0@k2g2000pro.googlegroups.com> <0372328c$0$1309$c3e8da3@news.astraweb.com> Message-ID: * Steve Holden: >> > While I am fully aware that premature optimization, etc., but I cannot > resist an appeal to efficiency if it finally kills off this idea that > "they took 'cmp()' away" is a bad thing. > > Passing a cmp= argument to sort provides the interpreter with a function > that will be called each time any pair of items have to be compared. The > key= argument, however, specifies a transformation from [x0, x1, ..., > xN] to [(key(x0), x0), (key(x1), x1), ..., (key(xN), xN)] (which calls > the key function precisely once per sortable item). > >>From a C routine like sort() [in CPython, anyway] calling out from C to > a Python function to make a low-level decision like "is A less than B?" > turns out to be disastrous for execution efficiency (unlike the built-in > default comparison, which can be called directly from C in CPython). > > If your data structures have a few hundred items in them it isn't going > to make a huge difference. If they have a few million thenit is already > starting to affect performance ;-) It's not either/or, it's do programmers still need the cmp functionality? Consider that *correctness* is a bit more important than efficiency, and that sorting strings is quite common... Possibly you can show me the way forward towards sorting these strings (shown below) correctly for a Norwegian locale. Possibly you can't. But one thing is for sure, if there was a cmp functionality it would not be a problem. Python 3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> L = ["?", "?", "?"] # This is in SORTED ORDER in Norwegian >>> L ['?', '?', '?'] >>> L.sort() >>> L ['?', '?', '?'] >>> >>> import locale >>> locale.getdefaultlocale() ('nb_NO', 'cp1252') >>> locale.setlocale( locale.LC_ALL ) # Just checking... 'C' >>> locale.setlocale( locale.LC_ALL, "" ) # Setting default locale, Norwgian. 'Norwegian (Bokm?l)_Norway.1252' >>> locale.strxfrm( "???" ) '???' >>> L.sort( key = locale.strxfrm ) >>> L ['?', '?', '?'] >>> locale.strcoll( "?", "?" ) 1 >>> locale.strcoll( "?", "?" ) -1 >>> Note that strcoll correctly orders the strings as ["?", "?", "?"], that is, it would have if it could have been used as cmp function to sort (or better, to a separate routine named e.g. custom_sort). And strcoll can be so used in 2.x: C:\Documents and Settings\Alf\test> py2 Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> def show( list ): ... print "[" + ", ".join( list ) + "]" ... >>> L = [u"?", u"?", u"?"] >>> show( L ) [?, ?, ?] >>> L.sort() >>> show( L ) [?, ?, ?] >>> import locale >>> locale.setlocale( locale.LC_ALL, "" ) 'Norwegian (Bokm\xe5l)_Norway.1252' >>> L.sort( cmp = locale.strcoll ) >>> show( L ) [?, ?, ?] >>> L [u'\xe6', u'\xf8', u'\xe5'] >>> _ The above may just be a bug in the 3.x stxfrm. But it illustrates that sometimes you have your sort order defined by a comparision function. Transforming that into a key can be practically impossible (it can also be quite inefficient). Cheers & hth., - Alf From clp2 at rebertia.com Fri Jan 29 01:14:09 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Thu, 28 Jan 2010 22:14:09 -0800 Subject: Wrap a function In-Reply-To: References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> Message-ID: <50697b2c1001282214h7100ec75v374bfdc81aa985c1@mail.gmail.com> On Thu, Jan 28, 2010 at 9:44 PM, Dennis Lee Bieber wrote: > On Thu, 28 Jan 2010 11:24:28 -0800 (PST), Joan Miller > declaimed the following in > gmane.comp.python.general: > > > On 28 ene, 19:16, Josh Holland wrote: > > > On 2010-01-28, Joan Miller wrote: > > > > > > > I've to call to many functions with the format: > > > > > > >>>> run("cmd") > > > > > > Check the docs on os.system(). > > No. I've a function that uses subprocess to run commands on the same > > shell and so substitute to bash scrips. But a script full of run > > ("shell_command --with --arguments") is too verbose. > > ? ? ? ?I shall blaspheme, and suggest that maybe the language you want to > use is REXX (ooREXX or Regina). Sounds like the REXX designers already got the blaspheming covered when they came up with such an inelegant-sounding feature... > ? ? ? ?By default, ANY statement that can not be confused for a REXX > language statement is sent to the currently defined command handler > (Which on most OSs is equivalent to Python's os.system() call; the late > Amiga, and IBM's mainframe OS had features that support defining other > applications as command handlers). > > ? ? ? ?A common practice is to put quotes about the first word of the > command to ensure it gets treated as external command. Cheers, Chris -- http://blog.rebertia.com From wuwei23 at gmail.com Fri Jan 29 01:16:08 2010 From: wuwei23 at gmail.com (alex23) Date: Thu, 28 Jan 2010 22:16:08 -0800 (PST) Subject: Perl 6 [was Re: myths about python 3] References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <03725fae$0$1309$c3e8da3@news.astraweb.com> Message-ID: geremy condra wrote: > Steven D'Aprano wrote: > > Out of curiosity, and completely off-topic, why has Perl 6 gone so badly? > > Too much like Perl. I was going to suggest that it's probably due to the multitude of ways to it could be done :) From tjreedy at udel.edu Fri Jan 29 01:18:22 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 29 Jan 2010 01:18:22 -0500 Subject: Sorting a list with entries of unequal types In-Reply-To: <03722537$0$1309$c3e8da3@news.astraweb.com> References: <87sk9pzsmm.fsf@benfinney.id.au> <87fx5pzp9t.fsf@benfinney.id.au> <03722537$0$1309$c3e8da3@news.astraweb.com> Message-ID: On 1/28/2010 8:24 PM, Steven D'Aprano wrote: >>> Um, what ordering do you want? >> >> The same ordering I'd get in Python 2; that is, determined by the types >> of the elements. > > The ordering has not been consistent across minor versions of Python 2: And complex numbers, and probably some other things, cannot be sorted even in Python2. Universal sorting is a broken idea, so Python3 leaves it to you to say what *you* mean. From 54wutong at gmail.com Fri Jan 29 01:29:53 2010 From: 54wutong at gmail.com (Stephen.Wu) Date: Thu, 28 Jan 2010 22:29:53 -0800 (PST) Subject: which one is faster? Message-ID: str.find(targetStr) str.index(targetStr) with exception str.count(targetStr) targetStr in str which is the fastest way to check whether targetStr is in str? thanks all From wuwei23 at gmail.com Fri Jan 29 01:39:32 2010 From: wuwei23 at gmail.com (alex23) Date: Thu, 28 Jan 2010 22:39:32 -0800 (PST) Subject: which one is faster? References: Message-ID: <140a7cf6-b4fa-4bab-a31d-aacf2c7678c5@k2g2000pro.googlegroups.com> On Jan 29, 4:29?pm, "Stephen.Wu" <54wut... at gmail.com> wrote: > str.find(targetStr) > str.index(targetStr) with exception > str.count(targetStr) > targetStr in str > > which is the fastest way to check whether targetStr is in str? It's generally a lot quicker to investigate this kind of question yourself using the interpreter & the timeit module. You'll need skills like these to be able to profile your code to look for actual performance bottlenecks, generic advice on the fastest of a set of functions will only get you so far. IPython is pretty handy for simple timing tests as it provides convenience wrappers around timeit: In [1]: t = 'foo' In [2]: s = 'djoemdmsllsodmedmsoskemozpleaoleodspsfooosoapxooeplaapakekoda' In [3]: timeit s.find(t) 1000000 loops, best of 3: 374 ns per loop In [4]: timeit s.index(t) 1000000 loops, best of 3: 381 ns per loop In [7]: timeit s.count(t) 1000000 loops, best of 3: 397 ns per loop In [8]: timeit t in s 1000000 loops, best of 3: 219 ns per loop >From the looks of those results, using 'in' seems to be the fastest. From ben+python at benfinney.id.au Fri Jan 29 02:08:35 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 29 Jan 2010 18:08:35 +1100 Subject: Sorting a list with entries of unequal types References: <87sk9pzsmm.fsf@benfinney.id.au> <87fx5pzp9t.fsf@benfinney.id.au> <03722537$0$1309$c3e8da3@news.astraweb.com> Message-ID: <87r5p9wfpo.fsf@benfinney.id.au> Terry Reedy writes: > And complex numbers, and probably some other things, cannot be sorted > even in Python2. Universal sorting is a broken idea, so Python3 leaves > it to you to say what *you* mean. Okay. I guess I'd better figure out what that is, for this code base, before porting it to Python 3 :-) Thanks for the helpful responses. -- \ ?Most people, I think, don't even know what a rootkit is, so | `\ why should they care about it?? ?Thomas Hesse, Sony BMG, 2006 | _o__) | Ben Finney From nyamatongwe+thunder at gmail.com Fri Jan 29 02:22:03 2010 From: nyamatongwe+thunder at gmail.com (Neil Hodgson) Date: Fri, 29 Jan 2010 07:22:03 GMT Subject: [OT] Perl 6 [was Re: myths about python 3] In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <03725fae$0$1309$c3e8da3@news.astraweb.com> Message-ID: Looks to me like the problem with Perl 6 was that it was too ambitious, wanting to fix all perceived problems with the language. Python 3 is much more limited in scope: at its core its Python with Unicode fixed and old junk removed. Neil From rdv at roalddevries.nl Fri Jan 29 03:08:56 2010 From: rdv at roalddevries.nl (Roald de Vries) Date: Fri, 29 Jan 2010 09:08:56 +0100 Subject: Symbols as parameters? In-Reply-To: <03722683$0$1309$c3e8da3@news.astraweb.com> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <4CD97E94-B40E-481D-B367-5FCE2820F48D@roalddevries.nl> <03722683$0$1309$c3e8da3@news.astraweb.com> Message-ID: <21315400-1778-4F2F-A644-06BDA9FCBDA8@roalddevries.nl> On Jan 29, 2010, at 2:30 AM, Steven D'Aprano wrote: > On Thu, 28 Jan 2010 17:01:38 +0100, Roald de Vries wrote: >> Question out of general interest in the language: If I would want to >> generate such functions in a for-loop, what would I have to do? This >> doesn't work: >> >> class Move(object): >> def __call__(self, direction): >> return direction >> >> move = Move() >> >> for f in ['up', 'down', 'right', 'left']: >> move.__dict__[f] = lambda: move(f) >> >> ... because now 'move.up()' returns 'left' because thats the current >> value of f. Is there a way to 'expand' f in the loop? Or a reason >> that >> you never should use this? > > > Possibly the simplest way is to use Python's handling of default > values > to get the result you want: > > for f in ['up', 'down', 'right', 'left']: > move.__dict__[f] = lambda f=f: move(f) This still leaves open the possibility of move.up('down') resulting in move('down'), but for the rest I like it. > BTW, there's no need to explicitly reference move.__dict__: > > for f in ['up', 'down', 'right', 'left']: > setattr(move, f, lambda f=f: move(f)) I expected something like that. Thanks. Roald From martin at v.loewis.de Fri Jan 29 03:16:17 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Fri, 29 Jan 2010 09:16:17 +0100 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <4B629951.6020700@v.loewis.de> > Python has had > previous major changes in the past (e.g. 1.5 to 2.0 and 2.1 to 2.2) and > hardly anyone made a complaint. I think this is actually false for the switch from 1.5 to 2.0. People complained a lot, and announced that they won't switch to Python 2 in any foreseeable future, and indeed, they stayed with Python 1.5.2 for several years after Python 2 was released. Of course, the Python user base was much smaller then than it is now, so the absolute number of complainers surely was never as high is it is now. Only when Python 4 gets released, even more people will complain and announce that Python 4 has failed and that they stay with Python 3.8 forever. Regards, Martin From martin at v.loewis.de Fri Jan 29 03:25:15 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Fri, 29 Jan 2010 09:25:15 +0100 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> Message-ID: <4B629B6B.3000100@v.loewis.de> > Well, I'd consider that an official release. Note that I didn't claim > there was no hope PSF wouldn't change it's mind on 2.8. I'd like to point out that the PSF formally doesn't have any say in this. Instead, releases are created by the release manager, who gets appointed by Guido van Rossum. Those two listen primarily to the opinions of the fellow committers (which may or may not happen to be PSF members as well). > Regardless of how magnaminous the people of PSF are, the unfortunate > reality is that trademark owners are forced by the law to be > "particularly petty". PSF's IP lawyer will advise not to allow > unsanctioned fork of Python 2.7 to call itself Python 2.8. However, calling it "Passive Python", or "John Smith Python", for example, would certainly be fine. Calling something "Python 2.8" may not be unreasonable, also "Python 2.9", but surely "Python 2.10" will hit strong objections (GvR always announced that there will never be a two-digit minor release number). Regards, Martin From martin at v.loewis.de Fri Jan 29 03:33:45 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Fri, 29 Jan 2010 09:33:45 +0100 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <4B629D69.3070905@v.loewis.de> > Why do I feel like there's less of an onus on Unladen Swallow to > _actually prove itself in substantial real world usage_ before > integration into CPython than there is on even the smallest of modules > for inclusion in the standard library? Because it's a VM change, not an end-user change. VM changes can be reviewed and evaluated by core developers, without requiring necessarily feedback from end users (although end users can and do certainly evaluate VM changes themselves also and provide feedback). For library changes, it's more difficult to evaluate them, because you not only need to find out whether the change is correct, but also whether it is useful. > Are we really expected to just ditch everything we know about > CPython's performance characteristics just for some questionable and > possibly uneven gains? That's the point of writing a PEP. Provide feedback to the PEP authors, and ask them to incorporate your objections into the PEP in case they forget. Then, when the PEP is about to be approved, that feedback gets taken into consideration. Posting in the 665th message on a Usenet thread is unlikely to have any effect on the PEP process, though. > I've been a big supporter of Py3 from the beginning, but this repeated > claim of US becoming the mainline interpreter for 3.x pretty much > kills dead a lot of my interest. It's not a claim, it's a PEP. Not being interested in the PEP process is your choice, of course, but you shouldn't complain afterwards that your opinion wasn't considered if you didn't actually voice it appropriately. Regards, Martin From peloko45 at gmail.com Fri Jan 29 03:41:33 2010 From: peloko45 at gmail.com (Joan Miller) Date: Fri, 29 Jan 2010 00:41:33 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> <9a28cf01-20af-4ce2-9b78-d8529800e4b9@y7g2000prc.googlegroups.com> Message-ID: On 28 ene, 23:36, Jonathan Gardner wrote: > On Jan 28, 2:16?pm, Joan Miller wrote: > > > > > There would be to make a function for each system command to use so it > > would be too inefficient, and follow the problem with the quotes. > > > The best is make a parser into a compiled language > > Yeah, you could do that. Or you can simply rely on /bin/sh to do the > parsing and everything else for you. No need to re-invent the wheel. I > don't think Python will ever beat sh as a shell replacement. > > When people say that Python is great for some situations, but not so > much for others, I think they thought of running commands like this as > "other", I started to working on this project (Scripy [1]) because I wanted to hacking cryptsetup in my ubuntu. The funcionts to manage its initialization are in bash and it goes to be non-maintainable code, cryptic and very hard to debug (as whatever bash script of medium size). Here you have the beast: http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/karmic/cryptsetup/karmic/annotate/head%3A/debian/cryptdisks.functions Using Scripy I can debug easily the commands run from the shell, and log all if I would. Now, thanks to Scripy I've created a script for easily disks partitioning [2] using a simple cofiguration in YAML [3]. The great thing is that I can add volumes and encrypted partitions :) The only problem is that it's too verbose but I could rename *run()* by a simple function as *_()* or *r()* [1] http://bitbucket.org/ares/scripy/src/ [2] http://bitbucket.org/ares/scripypartition/src/tip/lib/scripy/part/disk.py#cl-22 [3] http://bitbucket.org/ares/scripypartition/src/tip/bin/init_crypto.py#cl-46 From nagle at animats.com Fri Jan 29 03:41:54 2010 From: nagle at animats.com (John Nagle) Date: Fri, 29 Jan 2010 00:41:54 -0800 Subject: Why doesn't "filterwarnings" suppress this message? In-Reply-To: References: <4b61d5fb$0$1658$742ec2ed@news.sonic.net> Message-ID: <4b629bac$0$1620$742ec2ed@news.sonic.net> It turned out to be a loading order issue. That message is generated at import time, and "filterwarnings" hadn't been called yet. John Nagle Peter Otten wrote: > John Nagle wrote: > >> Feedparser isn't supported for Python 3.0, so in Python 2.6, many >> warning >> messages appear. I'm trying, in Python 2.6, to suppress the warning >> message: >> >> ./feedparser\feedparser.py:69: DeprecationWarning: >> the sgmllib module has been removed in Python 3.0 >> import sgmllib, re, sys, copy, urlparse, time, rfc822, types, >> cgi, urllib, urllib2 >> >> with this warning filter: >> >> warnings.filterwarnings(action='ignore', >> category=DeprecationWarning, module='feedparser') >> >> The filter suppresses other deprecation warnings associated with >> the feedparser module, but not that one. How do I get rid of that >> one? From peloko45 at gmail.com Fri Jan 29 03:43:19 2010 From: peloko45 at gmail.com (Joan Miller) Date: Fri, 29 Jan 2010 00:43:19 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> <4B61EC7A.7020406@optimum.net> <64dc1630-6aa5-4983-b061-261457bf0153@g29g2000yqe.googlegroups.com> <8508278f-0cf2-46c2-8313-2c640a71cdbd@q4g2000yqm.googlegroups.com> <4B621645.1000906@optimum.net> Message-ID: <8595827f-f698-46d2-9178-8138f9ceac24@f12g2000yqn.googlegroups.com> On 28 ene, 22:57, John Posner wrote: > On 1/28/2010 3:45 PM, Joan Miller wrote: > > > > > On 28 ene, 20:34, Joan Miller ?wrote: > >> On 28 ene, 20:20, Peter ?wrote: > > >>> On Jan 29, 6:58 am, John Posner ?wrote: > > >>>> On 1/28/2010 2:24 PM, Joan Miller wrote: > > >>>>> On 28 ene, 19:16, Josh Holland ? ?wrote: > >>>>>> On 2010-01-28, Joan Miller ? ?wrote: > > >>>>>>> I've to call to many functions with the format: > > >>>>>>>>>> run("cmd") > > >>>>>> Check the docs on os.system(). > >>>>> No. I've a function that uses subprocess to run commands on the same > >>>>> shell and so substitute to bash scrips. But a script full of run > >>>>> ("shell_command --with --arguments") is too verbose. > > >>>> I'm suspicious of your original intent. Essentially, you want to write > >>>> code in which a literal string, such as ... > > >>>> ? ? ls -l > > >>>> ... is *not* enclosed in quotes. Why run the risk of creating confusion > >>>> (in other people who look at your code, in syntax-checking tools, etc.) > >>>> between variables and literals? > > >>>> But I'm in sympathy with your desire to make the code as clean as > >>>> possible and to minimize the number of times you have to type a quote > >>>> character. My suggestions: > > >>>> 1. Create a function (say, "Run") that encapsulates as much of the > >>>> syntax as possible: os.system(), subprocess.call(), string-splitting, > >>>> whatever. So an invocation would look like this: > > >>>> ? ? Run("ls -l *.txt") > > >>>> (I think you've already done this step.) > > >>>> 2. Find a text editor that supports keyboard macros, so that a single > >>>> keystroke turns this text line: > > >>>> ? ? ls -l *.txt > > >>>> ... into this one: > > >>>> ? ? Run("ls -l *.txt") > > >>>> HTH, > >>>> John > > >>> I can't see you avoiding quotes etc, but extending on John's comment, > >>> the obvious next step would be to run everything in a loop i.e. place > >>> all the commands into a list and create a loop that ran each command > >>> in the list. > > >> Yes, but could be necessary that were mixed with python code. > > >>> Almost all editors support macros - most editors support some form of > >>> language sensitive editing (NOT the prompt call parameters style but > >>> rather help with the syntax via a 'form' style of fill-in) that will > >>> allow you to reduce typing effort. But macros would be the first and > >>> easiest choice for this activity. > > >> The goal of my program is substitute to bash scripts, so the macros in > >> editors are irrelevant fo this one. > > > I think that the best solution that I've is to build a program that > > parses the script to convert *$ command* to run("command") before of > > be called by python. > > I believe you're working on Linux, so how about using "sed"? Here's a > (prettified) BASH transcript of a sed script (edit.sed) transforming a > 6-line text file (myprog.py). The text file has both Python statements > and "special commands", which have "$ " at the beginning of the line. > > ?>>> cat myprog.py > print "hello" > $ ls -l > r = range(10) > $ grep foo bar.data > pass > print "bye" > > ?>>> cat edit.sed > s/^\$ \(.*\)/Run("\1")/ > > ?>>> sed -f edit.sed data.txt > print "hello" > Run("ls -l") > r = range(10) > Run("grep foo bar.data") > pass > print "bye" > > -John Yes, this would a well solution. Simple and fast to build. From ben+python at benfinney.id.au Fri Jan 29 03:48:26 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 29 Jan 2010 19:48:26 +1100 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <4B629D69.3070905@v.loewis.de> Message-ID: <87k4v1wb39.fsf@benfinney.id.au> "Martin v. Loewis" writes: > Not being interested in the PEP process is your choice, of course, but > you shouldn't complain afterwards that your opinion wasn't considered > if you didn't actually voice it appropriately. +1 QOTW -- \ ?I installed a skylight in my apartment. The people who live | `\ above me are furious!? ?Steven Wright | _o__) | Ben Finney From nagle at animats.com Fri Jan 29 03:50:18 2010 From: nagle at animats.com (John Nagle) Date: Fri, 29 Jan 2010 00:50:18 -0800 Subject: mix statically typed with dynamically typed In-Reply-To: References: Message-ID: <4b629da4$0$1593$742ec2ed@news.sonic.net> Yingjie Lan wrote: > We all know that Python is dynamically typed, and dynamically typed languages > are generally slower than statically typed ones. That's mostly a problem with the CPython interpreter, which is a naive interpreter. Many dynamically typed languages have implementations which optimize out much of the run-time type handling. Shed Skin does this for Python. LISP has been doing this better for decades. The JIT systems for Javascript also do this. Psyco has some explicit typing capability, but it doesn't do much about eliminating redundant attribute lookups. The two big wins that Python needs for performance are 1) at least recognize when a variable can be represented as "long" "double", or "char", and 2) recognize when an object or module doesn't need dynamic attribute lookup and the attribute slots can be nailed down at compile time. (Plus, of course, something so that multithreaded programs don't suck so bad on multicore CPUs.) John Nagle From __peter__ at web.de Fri Jan 29 03:59:03 2010 From: __peter__ at web.de (Peter Otten) Date: Fri, 29 Jan 2010 09:59:03 +0100 Subject: Sorting a list with entries of unequal types References: <87sk9pzsmm.fsf@benfinney.id.au> <87fx5pzp9t.fsf@benfinney.id.au> Message-ID: Ben Finney wrote: > Peter Otten <__peter__ at web.de> writes: > >> I can't find the relevant part of the 2.6 documentation, but something >> like >> >> >>> def key(x): >> ... t = type(x) >> ... t = compat.get(t, t) >> ... return t.__name__, id(t), x >> ... >> >>> compat = {bool: float, int: float} > > The problem with this approach is that it assumes I know all the types > of elements that will be sorted; I don't. I want to sort a list coming > from some other part of the code, and I don't want to arbitrarily limit > the types of the list elements. I don't see the limitation here. The above key functions sorts unknown types by name, then id(type) then instance. Peter From peloko45 at gmail.com Fri Jan 29 04:06:56 2010 From: peloko45 at gmail.com (Joan Miller) Date: Fri, 29 Jan 2010 01:06:56 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> Message-ID: On 29 ene, 05:44, Dennis Lee Bieber wrote: > On Thu, 28 Jan 2010 11:24:28 -0800 (PST), Joan Miller > declaimed the following in > gmane.comp.python.general: > > > On 28 ene, 19:16, Josh Holland wrote: > > > On 2010-01-28, Joan Miller wrote: > > > > > I've to call to many functions with the format: > > > > >>>> run("cmd") > > > > Check the docs on os.system(). > > No. I've a function that uses subprocess to run commands on the same > > shell and so substitute to bash scrips. But a script full of run > > ("shell_command --with --arguments") is too verbose. > > ? ? ? ? I shall blaspheme, and suggest that maybe the language you want to > use is REXX (ooREXX or Regina). > > ? ? ? ? By default, ANY statement that can not be confused for a REXX > language statement is sent to the currently defined command handler > (Which on most OSs is equivalent to Python's os.system() call; the late > Amiga, and IBM's mainframe OS had features that support defining other > applications as command handlers). > > ? ? ? ? A common practice is to put quotes about the first word of the > command to ensure it gets treated as external command. I prefer Python since that has a great standard library where I get a well logging system between another things. In addition, the main intended audience will be for system administrators who many of them already have used it. From siddhartha.veedaluru at gmail.com Fri Jan 29 04:17:56 2010 From: siddhartha.veedaluru at gmail.com (siddhartha veedaluru) Date: Fri, 29 Jan 2010 14:47:56 +0530 Subject: Python-list Digest, Vol 76, Issue 303 In-Reply-To: References: Message-ID: <424b71ec1001290117y259a134dkeb31abf906cd32bf@mail.gmail.com> Hi, Thats the output i got in output window.it is not listing the statement which caused it. copy paste and try it on you setup. it behaves the same way. I know that error is in log.info() function call.but it is not caught by "except". which is not getting logged using log.exception Thanks Siddhartha ---------- Forwarded message ---------- From: Steve Holden To: Date: Tue, 26 Jan 2010 11:54:23 -0500 Subject: Re: TypeError not caught by except statement siddhartha veedaluru wrote: > Hi, > > except not able to caught the TypeError exception occured in the below > code > > log.info ("refer",ret) in the try block > > throws a TypeError which is not caught . The traceback you provide does not show the line in question, so I am confused as to why you would think it was causing the problem. The purpose of the traceback is to show you where the exception occurred, and I question whether that is happening here. Anyway, I suspect your error might go away if you turned the first argument of hte log.info() call into a format string such as log.info("refer: %s", ret) regards Steve > Also sometimes process is getting hanged. > > ------------------------------------------------------------------------------------------------------------------------------------------------ > import logging > log = logging.getLogger() > fileName = strftime("%d-%b-%Y-", gmtime()) + str(int(time.time())) + "- > Log.log" > log = logging.getLogger() > log.setLevel(logging.NOTSET) > fh = logging.FileHandler(logFile) > logFileLevel = logging.DEBUG > fh.setLevel(logFileLevel) > format_string = '%(process)d %(thread)d %(asctime)-15s %(levelname)-5s > at %(filename)-15s in %(funcName)-10s at line %(lineno)-3d "%(message) > s"' > fh.setFormatter(logging.Formatter(format_string)) > log.addHandler(fh) > > try: > log.info ("start ") > log.info ("refer",ret ) > log.info ("end ") > except TypeError: > log.exception("Exception raised") > > ---------------------------------------------------------------------------------------------------------------------------------------------- > OUTPUT message: > > Traceback (most recent call last): > File "C:\Python26\lib\logging\__init__.py", line 768, in emit > msg = self.format(record) > File "C:\Python26\lib\logging\__init__.py", line 648, in format > return fmt.format(record) > File "C:\Python26\lib\logging\__init__.py", line 436, in format > record.message = record.getMessage() > File "C:\Python26\lib\logging\__init__.py", line 306, in getMessage > msg = msg % self.args > TypeError: not all arguments converted during string formatting > -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From siddhartha.veedaluru at gmail.com Fri Jan 29 04:18:42 2010 From: siddhartha.veedaluru at gmail.com (siddhartha veedaluru) Date: Fri, 29 Jan 2010 14:48:42 +0530 Subject: TypeError not caught by except statement Message-ID: <424b71ec1001290118n5492939x1f740471b3c17bc0@mail.gmail.com> Hi, Thats the output i got in output window.it is not listing the statement which caused it. copy paste and try it on you setup. it behaves the same way. I know that error is in log.info() function call.but it is not caught by "except". which is not getting logged using log.exception Thanks Siddhartha ---------- Forwarded message ---------- From: Steve Holden To: Date: Tue, 26 Jan 2010 11:54:23 -0500 Subject: Re: TypeError not caught by except statement siddhartha veedaluru wrote: > Hi, > > except not able to caught the TypeError exception occured in the below > code > > log.info ("refer",ret) in the try block > > throws a TypeError which is not caught . The traceback you provide does not show the line in question, so I am confused as to why you would think it was causing the problem. The purpose of the traceback is to show you where the exception occurred, and I question whether that is happening here. Anyway, I suspect your error might go away if you turned the first argument of hte log.info() call into a format string such as log.info("refer: %s", ret) regards Steve > Also sometimes process is getting hanged. > > ------------------------------------------------------------------------------------------------------------------------------------------------ > import logging > log = logging.getLogger() > fileName = strftime("%d-%b-%Y-", gmtime()) + str(int(time.time())) + "- > Log.log" > log = logging.getLogger() > log.setLevel(logging.NOTSET) > fh = logging.FileHandler(logFile) > logFileLevel = logging.DEBUG > fh.setLevel(logFileLevel) > format_string = '%(process)d %(thread)d %(asctime)-15s %(levelname)-5s > at %(filename)-15s in %(funcName)-10s at line %(lineno)-3d "%(message) > s"' > fh.setFormatter(logging.Formatter(format_string)) > log.addHandler(fh) > > try: > log.info ("start ") > log.info ("refer",ret ) > log.info ("end ") > except TypeError: > log.exception("Exception raised") > > ---------------------------------------------------------------------------------------------------------------------------------------------- > OUTPUT message: > > Traceback (most recent call last): > File "C:\Python26\lib\logging\__init__.py", line 768, in emit > msg = self.format(record) > File "C:\Python26\lib\logging\__init__.py", line 648, in format > return fmt.format(record) > File "C:\Python26\lib\logging\__init__.py", line 436, in format > record.message = record.getMessage() > File "C:\Python26\lib\logging\__init__.py", line 306, in getMessage > msg = msg % self.args > TypeError: not all arguments converted during string formatting > -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jspies at sun.ac.za Fri Jan 29 04:23:54 2010 From: jspies at sun.ac.za (Johann Spies) Date: Fri, 29 Jan 2010 11:23:54 +0200 Subject: Need help with a program In-Reply-To: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: <20100129092354.GG1718@sun.ac.za> On Thu, Jan 28, 2010 at 07:07:04AM -0800, evilweasel wrote: > Hi folks, > > I am a newbie to python, and I would be grateful if someone could > point out the mistake in my program. Basically, I have a huge text > file similar to the format below: > > AAAAAGACTCGAGTGCGCGGA 0 > AAAAAGATAAGCTAATTAAGCTACTGG 0 > AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 > AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 > AAAAAGGTCGCCTGACGGCTGC 0 I know this is a python list but if you really want to get the job done quickly this is one method without writing python code: $ cat /tmp/y AAAAAGACTCGAGTGCGCGGA 0 AAAAAGATAAGCTAATTAAGCTACTGG 0 AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 AAAAAGGTCGCCTGACGGCTGC 0 $ grep -v 0 /tmp/y > tmp/z $ cat /tmp/z AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 Regards Johann -- Johann Spies Telefoon: 021-808 4599 Informasietegnologie, Universiteit van Stellenbosch "My son, if sinners entice thee, consent thou not." Proverbs 1:10 From steve at REMOVE-THIS-cybersource.com.au Fri Jan 29 05:04:33 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 10:04:33 GMT Subject: Need help with a program References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: <03729f14$0$1309$c3e8da3@news.astraweb.com> On Fri, 29 Jan 2010 11:23:54 +0200, Johann Spies wrote: > On Thu, Jan 28, 2010 at 07:07:04AM -0800, evilweasel wrote: >> Hi folks, >> >> I am a newbie to python, and I would be grateful if someone could point >> out the mistake in my program. Basically, I have a huge text file >> similar to the format below: >> >> AAAAAGACTCGAGTGCGCGGA 0 >> AAAAAGATAAGCTAATTAAGCTACTGG 0 >> AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 >> AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 >> AAAAAGGTCGCCTGACGGCTGC 0 > > I know this is a python list but if you really want to get the job done > quickly this is one method without writing python code: > > $ cat /tmp/y > AAAAAGACTCGAGTGCGCGGA 0 > AAAAAGATAAGCTAATTAAGCTACTGG 0 > AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 > AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 > AAAAAGGTCGCCTGACGGCTGC 0 > $ grep -v 0 /tmp/y > tmp/z > $ cat /tmp/z > AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 > AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 That will do the wrong thing for lines like: AAAAAGATAAGCTAATTAAGCTACTGGGTT 10 -- Steven From bruno.42.desthuilliers at websiteburo.invalid Fri Jan 29 06:00:35 2010 From: bruno.42.desthuilliers at websiteburo.invalid (Bruno Desthuilliers) Date: Fri, 29 Jan 2010 12:00:35 +0100 Subject: which one is faster? In-Reply-To: <140a7cf6-b4fa-4bab-a31d-aacf2c7678c5@k2g2000pro.googlegroups.com> References: <140a7cf6-b4fa-4bab-a31d-aacf2c7678c5@k2g2000pro.googlegroups.com> Message-ID: <4b62bfb8$0$23526$426a74cc@news.free.fr> alex23 a ?crit : > On Jan 29, 4:29 pm, "Stephen.Wu" <54wut... at gmail.com> wrote: >> str.find(targetStr) >> str.index(targetStr) with exception >> str.count(targetStr) >> targetStr in str >> >> which is the fastest way to check whether targetStr is in str? > > It's generally a lot quicker to investigate this kind of question > yourself using the interpreter & the timeit module. You'll need skills > like these to be able to profile your code to look for actual > performance bottlenecks, generic advice on the fastest of a set of > functions will only get you so far. Indeed. Another point to take into consideration is the _real_ dataset on which the functions under test will have to work. And possibly the available resources at runtime. As an example, an iterator-based solution is likely to be slower than a more straightforward "load everything in ram" one, but will scale better for large datasets and/or heavy concurrent access to the system. From rajat.dudeja at gmail.com Fri Jan 29 06:30:18 2010 From: rajat.dudeja at gmail.com (Rajat) Date: Fri, 29 Jan 2010 03:30:18 -0800 (PST) Subject: Thread get off from CPU occasionally. I suspect. Message-ID: <964a0563-cd3f-407e-8c32-9ebd07ef8829@a16g2000pre.googlegroups.com> Hi, I'm using Python 2.6 and using the threading module. In a python module (parent module) I'm importing main() function from another python module. I'm runnin this main() function on a Python thread so that I can periodically check in the parent thread/module if the new thread is up and running. This information that the python script running on the new thread is still up and running is passed to a 3rd party application (in a manner known to 3rd party). As 3rd party is waiting for this new thread to finish. It waits for 3 mins and on told to it by the parent thread that new thread is still running, the 3rd party then starts to wait again for 3 mins. I've successfully implemented this functionality and it runs well except for a rare chance when the new thread does not completely runs. However, most of the cases the new thread runs completely. I cannot understand why and when the new thread gets off the CPU or crashes or any other thing? I'm not able to figure out what is happening.? The code in the parent thread that starts the new thread is as follows? exec("from " + passed_script_file + " import main") #imported main function. It works. import threading import time p = RunTestScript(main) p.start() t_nought = time.time() logSystemInfo(t_nought) seconds_passed = 0 keep_alive_set = 0 while p.isAlive() == True: p.join(60.0) if p.isAlive() == True: seconds_passed = time.time() - t_nought if seconds_passed >= 120: t_nought = time.time() if keep_alive_set != 1: # 3rd party way of informing that the new thread is alive. request = 'SET SHARED VAR KEEP_ALIVE=%d' %(1) res = handle.submit('local', 'VAR', request) if (res.rc != 0): logSystemError('[PyInt] Error: STAF local VAR %s failed, RC: %s, Result: %s\n' % (request, res.rc, res.result)) return res.rc keep_alive_set = 1 Earlier, when I did not run the main function on a new thread, it always used to run completely. I used to run it as: exec("from " + passed_script_file + " import main") retCode = main() Thanks and regards, Rajat From steve at REMOVE-THIS-cybersource.com.au Fri Jan 29 06:47:26 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 11:47:26 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> <184f0182-acbc-4a85-a8d9-7f813ebd2eb0@k2g2000pro.googlegroups.com> <0372328c$0$1309$c3e8da3@news.astraweb.com> Message-ID: <0372b731$0$1309$c3e8da3@news.astraweb.com> On Fri, 29 Jan 2010 07:10:01 +0100, Alf P. Steinbach wrote: > >>> L = ["?", "?", "?"] # This is in SORTED ORDER in Norwegian L [...] > >>> L.sort( key = locale.strxfrm ) > >>> L > ['?', '?', '?'] > >>> locale.strcoll( "?", "?" ) > 1 > >>> locale.strcoll( "?", "?" ) > -1 > > Note that strcoll correctly orders the strings as ["?", "?", "?"], that > is, it would have if it could have been used as cmp function to sort (or > better, to a separate routine named e.g. custom_sort). This is in Python2.5, so I'm explicitly specifying unicode strings: >>> L = [u"?", u"?", u"?"] >>> assert sorted(L) == [u'?', u'?', u'?'] The default C-locale sorting of L does not equal to L. Now let's change to Norwegian locale: >>> import locale >>> locale.setlocale(locale.LC_ALL, 'nb_NO') 'nb_NO' >>> print u''.join(sorted(L, cmp=locale.strcoll)) ??? So far so good, we've confirmed that in Python 2.5 we can sort in a locale-aware form. Now, can we do this with a key function? Thanks to Raymond Hettinger's recipe here: http://code.activestate.com/recipes/576653/ >>> print u''.join(sorted(L, key=CmpToKey(locale.strcoll))) ??? Success! Let's try it in Python 3.1: >>> L = ["?", "?", "?"] >>> assert sorted(L) == ['?', '?', '?'] >>> >>> import locale >>> locale.setlocale(locale.LC_ALL, 'nb_NO') 'nb_NO' >>> ''.join(sorted(L, key=CmpToKey(locale.strcoll))) '???' The definition of CmpToKey can be found at the URL above. It's not very exciting, but here it is: def CmpToKey(mycmp): 'Convert a cmp= function into a key= function' class K(object): def __init__(self, obj, *args): self.obj = obj def __lt__(self, other): return mycmp(self.obj, other.obj) == -1 def __gt__(self, other): return mycmp(self.obj, other.obj) == 1 def __eq__(self, other): return mycmp(self.obj, other.obj) == 0 def __le__(self, other): return mycmp(self.obj, other.obj) != 1 def __ge__(self, other): return mycmp(self.obj, other.obj) != -1 def __ne__(self, other): return mycmp(self.obj, other.obj) != 0 return K If that's too verbose for you, stick this as a helper function in your application: def CmpToKey(mycmp): 'Convert a cmp= function into a key= function' class K(object): def __init__(self, obj, *args): self.obj = obj __lt__ = lambda s, o: mycmp(s.obj, o.obj) == -1 __gt__ = lambda s, o: mycmp(s.obj, o.obj) == 1 __eq__ = lambda s, o: mycmp(s.obj, o.obj) == 0 __le__ = lambda s, o: mycmp(s.obj, o.obj) != 1 __ge__ = lambda s, o: mycmp(s.obj, o.obj) != -1 __ne__ = lambda s, o: mycmp(s.obj, o.obj) != 0 return K [...] > The above may just be a bug in the 3.x stxfrm. But it illustrates that > sometimes you have your sort order defined by a comparision function. > Transforming that into a key can be practically impossible (it can also > be quite inefficient). This might be true, but you haven't demonstrated it. With one little helper function, you should be able to convert any comparison function into a key function, with relatively little overhead. -- Steven From solipsis at pitrou.net Fri Jan 29 06:48:26 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Jan 2010 11:48:26 +0000 (UTC) Subject: which one is faster? References: <140a7cf6-b4fa-4bab-a31d-aacf2c7678c5@k2g2000pro.googlegroups.com> Message-ID: Le Thu, 28 Jan 2010 22:39:32 -0800, alex23 a ?crit?: > On Jan 29, 4:29?pm, "Stephen.Wu" <54wut... at gmail.com> wrote: >> str.find(targetStr) >> str.index(targetStr) with exception >> str.count(targetStr) >> targetStr in str >> >> which is the fastest way to check whether targetStr is in str? > [...] > >>From the looks of those results, using 'in' seems to be the fastest. To answer the question more precisely: * "in" is the fastest because it doesn't have the method call overhead. This is only a fixed cost, though, and doesn't depend on the inputs. * all four alternatives use the same underlying algorithm, *but* count() has to go to the end of the input string in order to count all occurrences. The other expressions can stop as soon as the first occurence is found, which can of course be a big win if the occurrence is near the start of the string and the string is very long So, to sum it up: * "in" is faster by a small fixed cost advantage * "find" and "index" are almost exactly equivalent * "count" will often be slower because it can't early exit Regards Antoine. From invalid at invalid.invalid Fri Jan 29 06:52:53 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 29 Jan 2010 11:52:53 +0000 (UTC) Subject: [OT] Perl 6 [was Re: myths about python 3] References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <03725fae$0$1309$c3e8da3@news.astraweb.com> Message-ID: On 2010-01-29, Neil Hodgson wrote: > Looks to me like the problem with Perl 6 was that it was too > ambitious, wanting to fix all perceived problems with the > language. I thought Python was Perl with all the perceived problems fixed. -- Grant From solipsis at pitrou.net Fri Jan 29 06:55:02 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Jan 2010 11:55:02 +0000 (UTC) Subject: [OT] myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <87tyu529tl.fsf@benfinney.id.au> <03722429$0$1309$c3e8da3@news.astraweb.com> <87ockdy7tg.fsf@benfinney.id.au> Message-ID: Le Fri, 29 Jan 2010 13:16:11 +1100, Ben Finney a ?crit?: > > I think the reason ?date? was initially used is because dates are most > familiar to us as fleshy, dark brown, wrinkled, compressed points. > > My interests in etymology and scatology unite here. Ah, I suppose it explains the strange ASCII art drawing in your signature then? (Here) | | v > -- > \ ?In the long run, the utility of all non-Free software | > `\ approaches zero. All non-Free software is a dead end.? ?Mark | > _o__) Pilgrim, 2006 | > Ben Finney From as at sci.fi Fri Jan 29 08:00:08 2010 From: as at sci.fi (Anssi Saari) Date: Fri, 29 Jan 2010 15:00:08 +0200 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <87zl3z2p63.fsf@benfinney.id.au> <4b6146b3$0$6715$9b4e6d93@newsspool2.arcor-online.net> Message-ID: Stefan Behnel writes: > 'Stable Debian' has a long tradition of being late and outdated on arrival. > That doesn't mean you can't use existing Debian packages on it. Yes, but that's beside the point. No released version of Debian ships with Python3 or even 2.6. Oh, and RHEL5 and CentOS5 ship with 2.4. From as at sci.fi Fri Jan 29 08:13:01 2010 From: as at sci.fi (Anssi Saari) Date: Fri, 29 Jan 2010 15:13:01 +0200 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: Daniel Fetchinson writes: >> 1. Python 3 is supported by major Linux distributions. >> >> FALSE - most distros are shipping with Python 2.4, or 2.5 at best. > > This latter statement is false, Fedora 11 and 12 come with python 2.6. How does your mention of one distro counter that claim? Personally, I'd like to see a study of what version of Python ships with what Linux distribution. Say, include the top 100 distros from distrowatch.com? I think there may a surprising number of distros that ship with no version of Python what so ever. From jspies at sun.ac.za Fri Jan 29 08:21:40 2010 From: jspies at sun.ac.za (Johann Spies) Date: Fri, 29 Jan 2010 15:21:40 +0200 Subject: Need help with a program In-Reply-To: <03729f14$0$1309$c3e8da3@news.astraweb.com> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> <03729f14$0$1309$c3e8da3@news.astraweb.com> Message-ID: <20100129132140.GH10729@sun.ac.za> On Fri, Jan 29, 2010 at 10:04:33AM +0000, Steven D'Aprano wrote: > > I know this is a python list but if you really want to get the job done > > quickly this is one method without writing python code: > > > > $ cat /tmp/y > > AAAAAGACTCGAGTGCGCGGA 0 > > AAAAAGATAAGCTAATTAAGCTACTGG 0 > > AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 > > AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 > > AAAAAGGTCGCCTGACGGCTGC 0 > > $ grep -v 0 /tmp/y > tmp/z > > $ cat /tmp/z > > AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 > > AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 > > That will do the wrong thing for lines like: > > AAAAAGATAAGCTAATTAAGCTACTGGGTT 10 In that case change the grep to ' 0$' then only the lines with a singel digit '0' at the end of the line will be excluded. One can do the same using regulare expressions in Python and it will probably a lot slower on large files. Regards Johann -- Johann Spies Telefoon: 021-808 4599 Informasietegnologie, Universiteit van Stellenbosch "My son, if sinners entice thee, consent thou not." Proverbs 1:10 From andrew at acooke.org Fri Jan 29 08:30:27 2010 From: andrew at acooke.org (andrew cooke) Date: Fri, 29 Jan 2010 05:30:27 -0800 (PST) Subject: Function name unchanged in error message Message-ID: Is there any way to change the name of the function in an error message? In the example below I'd like the error to refer to bar(), for example (the motivation is related function decorators - I'd like the wrapper function to give the same name) >>> def foo(): ... return 7 ... >>> foo.__name__ = 'bar' >>> foo(123) Traceback (most recent call last): File "", line 1, in TypeError: foo() takes no arguments (1 given) Thanks, Andrew From duncan.booth at invalid.invalid Fri Jan 29 08:41:38 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 29 Jan 2010 13:41:38 GMT Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: Anssi Saari wrote: > Daniel Fetchinson writes: > >>> 1. Python 3 is supported by major Linux distributions. >>> >>> FALSE - most distros are shipping with Python 2.4, or 2.5 at best. >> >> This latter statement is false, Fedora 11 and 12 come with python 2.6. > > How does your mention of one distro counter that claim? Personally, > I'd like to see a study of what version of Python ships with what > Linux distribution. Say, include the top 100 distros from > distrowatch.com? I think there may a surprising number of distros that > ship with no version of Python what so ever. > Is there any easier way to build that list than working through all the distributions by hand? Maybe someone should create a wiki page to record this. Here's what I see in the Ubuntu packages. Python 3 seems only to be in the universe repositories so far. Dapper: Python 2.4.2 Hardy: Python 2.5.2 Intrepid: Python 2.5.2, 3.0~b3 (universe) Jaunty: Python 2.6.2, 3.0.1 (universe) Karmic: Python 2.6.4rc1, 3.1 (universe) Lucid: Python 2.6.4, 3.1 (universe) WTF is rc anything doing in the main repository? -- Duncan Booth http://kupuguy.blogspot.com From jeanmichel at sequans.com Fri Jan 29 08:44:49 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Fri, 29 Jan 2010 14:44:49 +0100 Subject: Function name unchanged in error message In-Reply-To: References: Message-ID: <4B62E651.80007@sequans.com> andrew cooke wrote: > Is there any way to change the name of the function in an error > message? In the example below I'd like the error to refer to bar(), > for example (the motivation is related function decorators - I'd like > the wrapper function to give the same name) > > >>>> def foo(): >>>> > ... return 7 > ... > >>>> foo.__name__ = 'bar' >>>> foo(123) >>>> > Traceback (most recent call last): > File "", line 1, in > TypeError: foo() takes no arguments (1 given) > > Thanks, > Andrew > In [9]: def foo(): ...: return 7 ...: In [10]: bar = foo In [11]: bar(54) TypeError: foo() takes no arguments (1 given) JM From benjamin.kaplan at case.edu Fri Jan 29 08:54:25 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Fri, 29 Jan 2010 08:54:25 -0500 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On Fri, Jan 29, 2010 at 8:13 AM, Anssi Saari wrote: > Daniel Fetchinson writes: > >>> 1. ?Python 3 is supported by major Linux distributions. >>> >>> ? ? ?FALSE - most distros are shipping with Python 2.4, or 2.5 at best. >> >> This latter statement is false, Fedora 11 and 12 come with python 2.6. > > How does your mention of one distro counter that claim? Personally, > I'd like to see a study of what version of Python ships with what > Linux distribution. Say, include the top 100 distros from > distrowatch.com? I think there may a surprising number of distros that > ship with no version of Python what so ever. > -- Just from a quick look, 6 of the top 10 distros on distrowatch have Python 2.6 including all of the top 4. > http://mail.python.org/mailman/listinfo/python-list > From duncan.booth at invalid.invalid Fri Jan 29 08:59:54 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 29 Jan 2010 13:59:54 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> <184f0182-acbc-4a85-a8d9-7f813ebd2eb0@k2g2000pro.googlegroups.com> <0372328c$0$1309$c3e8da3@news.astraweb.com> <0372b731$0$1309$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > If that's too verbose for you, stick this as a helper function in your > application: > > > def CmpToKey(mycmp): > 'Convert a cmp= function into a key= function' > class K(object): > def __init__(self, obj, *args): > self.obj = obj > __lt__ = lambda s, o: mycmp(s.obj, o.obj) == -1 > __gt__ = lambda s, o: mycmp(s.obj, o.obj) == 1 > __eq__ = lambda s, o: mycmp(s.obj, o.obj) == 0 > __le__ = lambda s, o: mycmp(s.obj, o.obj) != 1 > __ge__ = lambda s, o: mycmp(s.obj, o.obj) != -1 > __ne__ = lambda s, o: mycmp(s.obj, o.obj) != 0 > return K > > Shouldn't that be: def CmpToKey(mycmp): 'Convert a cmp= function into a key= function' class K(object): def __init__(self, obj, *args): self.obj = obj __lt__ = lambda s, o: mycmp(s.obj, o.obj) < 0 __gt__ = lambda s, o: mycmp(s.obj, o.obj) > 0 __eq__ = lambda s, o: mycmp(s.obj, o.obj) == 0 __le__ = lambda s, o: mycmp(s.obj, o.obj) <= 0 __ge__ = lambda s, o: mycmp(s.obj, o.obj) >= 0 __ne__ = lambda s, o: mycmp(s.obj, o.obj) != 0 return K After all the cmp function needn't return -1,0,1 only negative, 0, positive. And if that's still too verbose for you try: def CmpToKey(mycmp): 'Convert a cmp= function into a key= function' class K(object): def __init__(self, obj, *args): self.obj = obj __lt__ = lambda s, o: mycmp(s.obj, o.obj) < 0 return K That works on 3.1 and 2.6 though of course there's no guarantee that a future Python version won't use a different comparison operator so only use it if you like living dangerously and have problems typing. :^) -- Duncan Booth http://kupuguy.blogspot.com From clp2 at rebertia.com Fri Jan 29 09:10:59 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Fri, 29 Jan 2010 06:10:59 -0800 Subject: Function name unchanged in error message In-Reply-To: References: Message-ID: <50697b2c1001290610t1a2ac5c8hcd5f9856aa85392b@mail.gmail.com> On Fri, Jan 29, 2010 at 5:30 AM, andrew cooke wrote: > Is there any way to change the name of the function in an error > message? ?In the example below I'd like the error to refer to bar(), > for example (the motivation is related function decorators - I'd like > the wrapper function to give the same name) > >>>> def foo(): > ... ? ? return 7 > ... >>>> foo.__name__ = 'bar' >>>> foo(123) > Traceback (most recent call last): > ?File "", line 1, in > TypeError: foo() takes no arguments (1 given) It gets weirder: >>> print(foo) Cheers, Chris -- http://blog.rebertia.com From dirknbr at googlemail.com Fri Jan 29 09:22:01 2010 From: dirknbr at googlemail.com (dirknbr) Date: Fri, 29 Jan 2010 06:22:01 -0800 (PST) Subject: Python 3.1 simplejson install Message-ID: <2be8271a-f97d-4ff8-a806-c6e1d1d4adfd@n7g2000yqb.googlegroups.com> I am trying to install simplejson on Python 3.1 on Windows. When I do 'python setup.py install' I get 'except DisutilsPlatformError, x: SyntaxError' with a dash under the comma. Any ideas? Dirk From __peter__ at web.de Fri Jan 29 09:22:56 2010 From: __peter__ at web.de (Peter Otten) Date: Fri, 29 Jan 2010 15:22:56 +0100 Subject: Function name unchanged in error message References: Message-ID: andrew cooke wrote: > Is there any way to change the name of the function in an error > message? In the example below I'd like the error to refer to bar(), > for example (the motivation is related function decorators - I'd like > the wrapper function to give the same name) > >>>> def foo(): > ... return 7 > ... >>>> foo.__name__ = 'bar' >>>> foo(123) > Traceback (most recent call last): > File "", line 1, in > TypeError: foo() takes no arguments (1 given) The name is looked up in the code object. As that is immutable you have to make a new one: argnames = 'argcount nlocals stacksize flags code consts names varnames filename name firstlineno lnotab'.split() def f(): return 42 code = type(f.func_code) function = type(f) def make_code(proto, **kw): for name in argnames: if name not in kw: kw[name] = getattr(proto, "co_" + name) values = [kw[name] for name in argnames] return code(*values) if __name__ == "__main__": def foo(): print "foo" c = make_code(foo.func_code, name="bar") foo.func_code = c foo(42) Peter From darcy at druid.net Fri Jan 29 09:32:52 2010 From: darcy at druid.net (D'Arcy J.M. Cain) Date: Fri, 29 Jan 2010 09:32:52 -0500 Subject: Need help with a program In-Reply-To: <20100129092354.GG1718@sun.ac.za> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> <20100129092354.GG1718@sun.ac.za> Message-ID: <20100129093252.204146d0.darcy@druid.net> On Fri, 29 Jan 2010 11:23:54 +0200 Johann Spies wrote: > I know this is a python list but if you really want to get the job > done quickly this is one method without writing python code: > [...] > $ grep -v 0 /tmp/y > tmp/z There's plenty of ways to do it without writing Python. C, C++, Perl, Forth, Awk, BASIC, Intercal, etc. So what? Besides, your solution doesn't work. You want "grep -vw 0 /tmp/y > tmp/z" and even then it doesn't meet the requirements. It extracts the lines the OP wants but doesn't reformat them. It also assumes a Unix system or at least something with grep installed so it isn't portable. If you want to see how the same task can be done in many different languages see http://www.roesler-ac.de/wolfram/hello.htm. -- D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner. From __peter__ at web.de Fri Jan 29 09:33:11 2010 From: __peter__ at web.de (Peter Otten) Date: Fri, 29 Jan 2010 15:33:11 +0100 Subject: Python 3.1 simplejson install References: <2be8271a-f97d-4ff8-a806-c6e1d1d4adfd@n7g2000yqb.googlegroups.com> Message-ID: dirknbr wrote: > I am trying to install simplejson on Python 3.1 on Windows. When I do > 'python setup.py install' I get 'except DisutilsPlatformError, x: > SyntaxError' with a dash under the comma. You are trying to install a package written for Python 2.x on 3.x, and some of the 2.x Syntax is illegal in Python 3. I recommend that you use the json module instead that is already included in the 3.1 distribution, see http://docs.python.org/3.1/library/json.html Peter From austin.bingham at gmail.com Fri Jan 29 09:37:09 2010 From: austin.bingham at gmail.com (Austin Bingham) Date: Fri, 29 Jan 2010 15:37:09 +0100 Subject: Some C-API functions clear the error indicator? Message-ID: I've noticed that several (many?) python functions seem to clear the error/exception indicators when they're called from a C/C++ program. For example, both PyImport_ImportModule and traceback.extract_tb() (called via the function call methods) do this: if error indicators are set prior to their call (as indicated by PyErr_Fetch, and including a call to PyErr_Restore), I see that they are unset (using the same method) after the call. This happens even when the functions succeed. The functions that do this don't seem to indicate in their documentation that this will happen. So first, does anyone know why this is happening? Is it because of the context in which I'm making the calls? Is there any pattern or reason behind which functions will do this? Or am I just doing something wrong? If the problem is context-dependent (e.g. I haven't properly established a call stack, or something of that flavor), any pointers on doing things properly would be great. Here's some example code demonstrating the problem: --- #include int main(int argc, char** argv) { Py_Initialize(); // Cause an IndexError PyObject* list = PyList_New(0); PyObject* obj = PyList_GetItem(list, 100); PyObject *t = NULL, *v = NULL, *tb = NULL; // Verify that we see the error PyErr_Fetch(&t, &v, &tb); assert(t); PyErr_Restore(t, v, tb); // Import a module, which seems to be clearing the error indicator PyObject* mod = PyImport_ImportModule("sys"); assert(PyObject_HasAttrString(mod, "path")); // Verify that the error indicator has been cleared PyErr_Fetch(&t, &v, &tb); assert(!t); // <=== The error is gone! PyErr_Restore(t, v, tb); Py_Finalize(); return 0; } --- Thanks in advance. Austin From lists at cheimes.de Fri Jan 29 09:39:06 2010 From: lists at cheimes.de (Christian Heimes) Date: Fri, 29 Jan 2010 15:39:06 +0100 Subject: Python 3.1 simplejson install In-Reply-To: <2be8271a-f97d-4ff8-a806-c6e1d1d4adfd@n7g2000yqb.googlegroups.com> References: <2be8271a-f97d-4ff8-a806-c6e1d1d4adfd@n7g2000yqb.googlegroups.com> Message-ID: dirknbr wrote: > I am trying to install simplejson on Python 3.1 on Windows. When I do > 'python setup.py install' I get 'except DisutilsPlatformError, x: > SyntaxError' with a dash under the comma. There is no need to install simplejson yourself. Python 3 and 2.6 already have a JSON package called "json". Christian From pruebauno at latinmail.com Fri Jan 29 09:41:27 2010 From: pruebauno at latinmail.com (nn) Date: Fri, 29 Jan 2010 06:41:27 -0800 (PST) Subject: Need help with a program References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: <89a02db5-4e9f-4c0e-937c-b37ddea90283@l30g2000yqb.googlegroups.com> Johann Spies wrote: > On Thu, Jan 28, 2010 at 07:07:04AM -0800, evilweasel wrote: > > Hi folks, > > > > I am a newbie to python, and I would be grateful if someone could > > point out the mistake in my program. Basically, I have a huge text > > file similar to the format below: > > > > AAAAAGACTCGAGTGCGCGGA 0 > > AAAAAGATAAGCTAATTAAGCTACTGG 0 > > AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 > > AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 > > AAAAAGGTCGCCTGACGGCTGC 0 > > I know this is a python list but if you really want to get the job > done quickly this is one method without writing python code: > > $ cat /tmp/y > AAAAAGACTCGAGTGCGCGGA 0 > AAAAAGATAAGCTAATTAAGCTACTGG 0 > AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 > AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 > AAAAAGGTCGCCTGACGGCTGC 0 > $ grep -v 0 /tmp/y > tmp/z > $ cat /tmp/z > AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 > AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 > > Regards > Johann > -- > Johann Spies Telefoon: 021-808 4599 > Informasietegnologie, Universiteit van Stellenbosch > > "My son, if sinners entice thee, consent thou not." > Proverbs 1:10 I would rather use awk for this: awk 'NF==2 && $2!~/^0$/ {printf("seq%s\n%s\n",NR,$1)}' dnain.dat but I think that is getting a bit off topic... From clp2 at rebertia.com Fri Jan 29 09:41:28 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Fri, 29 Jan 2010 06:41:28 -0800 Subject: Python 3.1 simplejson install In-Reply-To: References: <2be8271a-f97d-4ff8-a806-c6e1d1d4adfd@n7g2000yqb.googlegroups.com> Message-ID: <50697b2c1001290641p646d23d3r412179d4c28ec5a7@mail.gmail.com> On Fri, Jan 29, 2010 at 6:33 AM, Peter Otten <__peter__ at web.de> wrote: > dirknbr wrote: >> I am trying to install simplejson on Python 3.1 on Windows. When I do >> 'python setup.py install' I get 'except DisutilsPlatformError, x: >> SyntaxError' with a dash under the comma. > > You are trying to install a package written for Python 2.x on 3.x, and some > of the 2.x Syntax is illegal in Python 3. To be specific (http://www.python.org/dev/peps/pep-3110/): "the ambiguous except A, B: which would mean different things in Python 2.x and 3.x -- leading to hard-to-catch bugs -- cannot legally occur in 3.x code." The except statement in your error would be properly written in 3.x as: except DisutilsPlatformError as x: Cheers, Chris -- http://blog.rebertia.com From exarkun at twistedmatrix.com Fri Jan 29 09:50:34 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Fri, 29 Jan 2010 14:50:34 -0000 Subject: Function name unchanged in error message In-Reply-To: <50697b2c1001290610t1a2ac5c8hcd5f9856aa85392b@mail.gmail.com> References: <50697b2c1001290610t1a2ac5c8hcd5f9856aa85392b@mail.gmail.com> Message-ID: <20100129145034.1898.1276929215.divmod.xquotient.937@localhost.localdomain> On 02:10 pm, clp2 at rebertia.com wrote: >On Fri, Jan 29, 2010 at 5:30 AM, andrew cooke >wrote: >>Is there any way to change the name of the function in an error >>message? ?In the example below I'd like the error to refer to bar(), >>for example (the motivation is related function decorators - I'd like >>the wrapper function to give the same name) >>>>>def foo(): >>... ? ? return 7 >>... >>>>>foo.__name__ = 'bar' >>>>>foo(123) >>Traceback (most recent call last): >>?File "", line 1, in >>TypeError: foo() takes no arguments (1 given) > >It gets weirder: >>>>print(foo) > The name is represented in (at least) two places, on the function object and on the code object: >>> def foo(): pass ... >>> foo.func_name 'foo' >>> foo.func_code.co_name 'foo' >>> foo.func_name = 'bar' >>> foo >>> foo.func_code.co_name = 'baz' Traceback (most recent call last): File "", line 1, in TypeError: readonly attribute >>> new.function and new.code will let you construct new objects with different values (and copying over whichever existing attributes you want to preserve). Jean-Paul From duncan.booth at invalid.invalid Fri Jan 29 09:58:42 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 29 Jan 2010 14:58:42 GMT Subject: Some C-API functions clear the error indicator? References: Message-ID: Austin Bingham wrote: > The functions that do this don't seem to indicate in their > documentation that this will happen. So first, does anyone know why > this is happening? Is it because of the context in which I'm making > the calls? Is there any pattern or reason behind which functions will > do this? Or am I just doing something wrong? > (Just guessing here) I would expect that any function that executes Python code will clear the error. I think that has to happen otherwise the Python code will throw an exception whenever it gets round to checking for errors. In the past I've found that if you fail to check for an error in C code before returning to the interpreter you get the exception thrown a few instructions later, so something similar would probably happen if you call other Python code from C. If it is anything that executes Python then that would include any function that creates or destroys an object with Python constructor or destructor code. or that compares or otherwise operates on instances defined in Python. In particular it might mean that any function that doesn't appear to clear the error could do so in a slightly different situation. -- Duncan Booth http://kupuguy.blogspot.com From mattsteel at hotmail.it Fri Jan 29 10:32:58 2010 From: mattsteel at hotmail.it (Mattsteel) Date: Fri, 29 Jan 2010 07:32:58 -0800 (PST) Subject: Funny behaviour with __future__ and doctest between 2.6 and 3.1 Message-ID: Hello all. I'm using Python 2.6.4 and Python 3.1.1. My wish is to code in a 3.1-compliant way using 2.6, so I'm importing the __future__ module. I've found a funny thing comparing the two folliwing snippets that differ for one line only, that is the position of __future__ import (before or after the doc string). Well, I understand the subtle difference but still I wander what really happen behind the scenes. Comments are welcome. --------------------------------------- #!/usr/bin/env python ''' >>> concat('hello','world') 'hello world' ''' from __future__ import unicode_literals def concat( first, second ): return first + ' ' + second if __name__ == "__main__": import doctest doctest.testmod() --------------------------------------- #!/usr/bin/env python from __future__ import unicode_literals ''' >>> concat('hello','world') 'hello world' ''' def concat( first, second ): return first + ' ' + second if __name__ == "__main__": import doctest doctest.testmod() --------------------------------------- The first way shows the following failure: --------------------------------------- Failed example: concat('hello','world') Expected: 'hello world' Got: u'hello world' --------------------------------------- Regards. Matt. From paul at boddie.org.uk Fri Jan 29 10:34:31 2010 From: paul at boddie.org.uk (Paul Boddie) Date: Fri, 29 Jan 2010 07:34:31 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> Message-ID: <35eac267-620a-410d-9a01-ae2b3c5790d7@e37g2000yqn.googlegroups.com> On 29 Jan, 06:56, Terry Reedy wrote: > On 1/28/2010 6:47 PM, Paul Boddie wrote: > > > What would annoy me if I used Python 3.x would be the apparent lack of > > the __cmp__ method for conveniently defining comparisons between > > instances of my own classes. Having to define all the rich comparison > > methods frequently isn't even as much fun as it sounds. > > You define __eq__, which automatically generates __ne__. >From the Python 3.1 language reference: "There are no implied relationships among the comparison operators. The truth of x==y does not imply that x!=y is false. Accordingly, when defining __eq__(), one should also define __ne__() so that the operators will behave as expected." Maybe Python 3.1 plugs a default method in, anyway. > You define __lt__, which is all sort and heap need. > This should be about as easier than __eq__, which is really needed, and > __cmp__. If you need the other 3, either copy the recipe in the > Cookbook, or, even faster > > def __ge__(s,o): return o.__lt__(s) > def __le__(s,o): return s < o or s == o > def __ge__(s,o): return s > o or s == o Spot the error in the above. ;-) Of course, this could be defined in a base class and be inherited everywhere, but the case that made me raise this issue actually involved instances of a class delegating comparison/sorting to another object. With __cmp__ this can be done concisely and only involve the delegation of one method - not so with the rich comparison "protocol". Paul From alfps at start.no Fri Jan 29 10:35:50 2010 From: alfps at start.no (Alf P. Steinbach) Date: Fri, 29 Jan 2010 16:35:50 +0100 Subject: python 3's adoption In-Reply-To: <0372b731$0$1309$c3e8da3@news.astraweb.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> <184f0182-acbc-4a85-a8d9-7f813ebd2eb0@k2g2000pro.googlegroups.com> <0372328c$0$1309$c3e8da3@news.astraweb.com> <0372b731$0$1309$c3e8da3@news.astraweb.com> Message-ID: * Steven D'Aprano: > On Fri, 29 Jan 2010 07:10:01 +0100, Alf P. Steinbach wrote: > >> >>> L = ["?", "?", "?"] # This is in SORTED ORDER in Norwegian L > > [...] > >> >>> L.sort( key = locale.strxfrm ) >> >>> L >> ['?', '?', '?'] >> >>> locale.strcoll( "?", "?" ) >> 1 >> >>> locale.strcoll( "?", "?" ) >> -1 >> >> Note that strcoll correctly orders the strings as ["?", "?", "?"], that >> is, it would have if it could have been used as cmp function to sort (or >> better, to a separate routine named e.g. custom_sort). > > This is in Python2.5, so I'm explicitly specifying unicode strings: > >>>> L = [u"?", u"?", u"?"] >>>> assert sorted(L) == [u'?', u'?', u'?'] > > The default C-locale sorting of L does not equal to L. Now let's change > to Norwegian locale: > >>>> import locale >>>> locale.setlocale(locale.LC_ALL, 'nb_NO') > 'nb_NO' >>>> print u''.join(sorted(L, cmp=locale.strcoll)) > ??? > > So far so good, we've confirmed that in Python 2.5 we can sort in a > locale-aware form. Now, can we do this with a key function? Thanks to > Raymond Hettinger's recipe here: > > http://code.activestate.com/recipes/576653/ > > >>>> print u''.join(sorted(L, key=CmpToKey(locale.strcoll))) > ??? > > > Success! > > Let's try it in Python 3.1: > >>>> L = ["?", "?", "?"] >>>> assert sorted(L) == ['?', '?', '?'] >>>> >>>> import locale >>>> locale.setlocale(locale.LC_ALL, 'nb_NO') > 'nb_NO' Hm. A bit off-topic, but... >>> import locale >>> locale.setlocale(locale.LC_ALL, 'nb_NO') Traceback (most recent call last): File "", line 1, in File "C:\Program Files\cpython\python31\lib\locale.py", line 527, in return _setlocale(category, locale) locale.Error: unsupported locale setting >>> _ This on a machine where the Python default locale is Norwegian. >>>> ''.join(sorted(L, key=CmpToKey(locale.strcoll))) > '???' > > > The definition of CmpToKey can be found at the URL above. It's not very > exciting, but here it is: > > > def CmpToKey(mycmp): > 'Convert a cmp= function into a key= function' > class K(object): > def __init__(self, obj, *args): > self.obj = obj > def __lt__(self, other): > return mycmp(self.obj, other.obj) == -1 > def __gt__(self, other): > return mycmp(self.obj, other.obj) == 1 > def __eq__(self, other): > return mycmp(self.obj, other.obj) == 0 > def __le__(self, other): > return mycmp(self.obj, other.obj) != 1 > def __ge__(self, other): > return mycmp(self.obj, other.obj) != -1 > def __ne__(self, other): > return mycmp(self.obj, other.obj) != 0 > return K This is pretty smart as a generic solution. Thanks! :-) I was thinking more of sticking those comparisions in some custom string class or such, which would be rather ugly... The above does have quite high overhead though! That is, it's /inefficient/ compared to using a 'cmp' function directly. > If that's too verbose for you, stick this as a helper function in your > application: > > > def CmpToKey(mycmp): > 'Convert a cmp= function into a key= function' > class K(object): > def __init__(self, obj, *args): > self.obj = obj > __lt__ = lambda s, o: mycmp(s.obj, o.obj) == -1 > __gt__ = lambda s, o: mycmp(s.obj, o.obj) == 1 > __eq__ = lambda s, o: mycmp(s.obj, o.obj) == 0 > __le__ = lambda s, o: mycmp(s.obj, o.obj) != 1 > __ge__ = lambda s, o: mycmp(s.obj, o.obj) != -1 > __ne__ = lambda s, o: mycmp(s.obj, o.obj) != 0 > return K > > > [...] >> The above may just be a bug in the 3.x stxfrm. But it illustrates that >> sometimes you have your sort order defined by a comparision function. >> Transforming that into a key can be practically impossible (it can also >> be quite inefficient). > > This might be true, but you haven't demonstrated it. The "can be ... practically impossible" was hogwash. I posted late, sorry. The "quite inefficient" holds. > With one little > helper function, you should be able to convert any comparison function > into a key function, with relatively little overhead. I wouldn't call an extra Python method call per comparision "relatively little overhead". Note that the same number of comparisions as with a 'cmp' based sort is performed. But with the wrapper every comparision is indirected through a Python method call (I think, but not sure, that the creation of those comparision objects does not introduce significant overhead, but the calls must). # -*- coding: utf-8 -*- from __future__ import print_function from __future__ import unicode_literals try: range = xrange except: pass import locale import sys import random import timeit def CmpToKey( mycmp ): "Convert a cmp= function into a key= function" class K( object ): def __init__( self, obj, *args ): self.obj = obj def __lt__( self, other ): return mycmp( self.obj, other.obj ) == -1 def __gt__( self, other ): return mycmp( self.obj, other.obj ) == 1 def __eq__( self, other ): return mycmp( self.obj, other.obj ) == 0 def __le__( self, other ): return mycmp( self.obj, other.obj ) != 1 def __ge__( self, other ): return mycmp( self.obj, other.obj ) != -1 def __ne__( self, other ): return mycmp( self.obj, other.obj ) != 0 return K def elapsed_time_for( f ): n_calls = 1 return timeit.timeit( f, number = n_calls ) def major_pyversion(): return sys.version_info[0] def sorting_of( string_list ): def sort2x(): string_list.sort( cmp = locale.strcoll ) def sort3x(): string_list.sort( key = CmpToKey( locale.strcoll ) ) return sort2x if major_pyversion() <= 2 else sort3x alphabet = "abcdefghijklmnopqrstuvwxyz???" n_strings = int( 1e5 ) nbno_locale = "" # "nb_NO" raises exception on my machine. def rnd_string(): len = random.randint( 1, 80 ) s = "" for i in range( len ): s = s + random.choice( alphabet ) return s locale.setlocale( locale.LC_ALL, nbno_locale ) random.seed( 666 ) assert( rnd_string() == "?moxqudxlsz?a?cte?rmuoc?jrd?pbvyabwhn" ) print( "Creating array of pseudo-random strings..." ) L = [rnd_string() for i in range( n_strings )] print( "Sorting..." ) time = elapsed_time_for( sorting_of( L ) ) print( "Last string is '{0}'".format( L[-1] ) ) print( "{0}".format( time ) ) assert( L[-1][0] == "?" ) C:\Documents and Settings\Alf\test> py3 sort.py Creating array of pseudo-random strings... Sorting... Last string is '???ywxuybrxgvnstoyvcmd?jdlnxwwfsbnzsinxncmtrxuxgtqtduzv?ixqovtjxiflfesuvfa' 8.45987772189 C:\Documents and Settings\Alf\test> py2 sort.py Creating array of pseudo-random strings... Sorting... Last string is '???ywxuybrxgvnstoyvcmd?jdlnxwwfsbnzsinxncmtrxuxgtqtduzv?ixqovtjxiflfesuvfa' 3.41336790011 C:\Documents and Settings\Alf\test> _ In the run above Python 3.x was only roughly 2.5 times slower. On my machine it ranges from 2.5 times slower to 3 times slower. Most often around 2.5 x slower. Of course this does not measure how much faster a Python 3.x cmp/"<" based sort /could be/, only how slow, sluggish and snail-like it is compared to 2.x. :-) In other words, that's the speed price, 2.5 to 3 times slower sorting, for ditching cmp instead of e.g. replacing it with something better. Cheers & hth., - Alf From Eric_Dexter at msn.com Fri Jan 29 10:48:43 2010 From: Eric_Dexter at msn.com (Eric_Dexter at msn.com) Date: Fri, 29 Jan 2010 07:48:43 -0800 (PST) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <8a830b83-d949-4af1-b0d7-25f68ec5f47a@e25g2000yqh.googlegroups.com> On Jan 27, 2:56?pm, John Nagle wrote: > Daniel Fetchinson wrote: > > Hi folks, > > > I was going to write this post for a while because all sorts of myths > > periodically come up on this list about python 3. I don't think the > > posters mean to spread false information on purpose, they simply are > > not aware of the facts. > > > My list is surely incomplete, please feel free to post your favorite > > misconception about python 3 that people periodically state, claim or > > ask about. > > Myths about Python 3: > > 1. ?Python 3 is supported by major Linux distributions. > > ? ? ? ? FALSE - most distros are shipping with Python 2.4, or 2.5 at best. > > 2. ?Python 3 is supported by multiple Python implementations. > > ? ? ? ? FALSE - Only CPython supports 3.x. ?Iron Python, Unladen Swallow, > ? ? ? ? PyPy, and Jython have all stayed with 2.x versions of Python. > > 3. ?Python 3 is supported by most 3rd party Python packages. > > ? ? ? ? FALSE - it's not supported by MySQLdb, OpenSSL, feedparser, etc. > > Arguably, Python 3 has been rejected by the market. ?Instead, there's > now Python 2.6, Python 2.7, and Python 2.8. ?Python 3 has turned into > a debacle like Perl 6, now 10 years old. > > That's the reality, Python 3 fanboys. > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? John Nagle I am seeing alot of bleeding edge linux distro's with 2.5 and 2.6 I would perfer they stayed with 2.4 or 2.5 .. or if there is a version that I can come up with a convincing cross- platform os.startfile I would want to use a vmware version of that linux. _______________________________________ http://dextracker.blogspot.com/ From __peter__ at web.de Fri Jan 29 10:50:12 2010 From: __peter__ at web.de (Peter Otten) Date: Fri, 29 Jan 2010 16:50:12 +0100 Subject: Funny behaviour with __future__ and doctest between 2.6 and 3.1 References: Message-ID: Mattsteel wrote: > Hello all. > I'm using Python 2.6.4 and Python 3.1.1. > My wish is to code in a 3.1-compliant way using 2.6, so I'm importing > the __future__ module. > I've found a funny thing comparing the two folliwing snippets that > differ for one line only, that is the position of __future__ import > (before or after the doc string). > > Well, I understand the subtle difference but still I wander what > really happen behind the scenes. Are you sure? The second script has no module docstring, just a string literal somewhere in the module, and therefore no tests. You can see that by running it with the verbose option -v. Also, from __future__ import unicode_literals doesn't affect the repr() of a unicode instance. But the interactive interpreter invokes repr() on the result before it is printed: >>> from __future__ import unicode_literals >>> "yadda" u'yadda' > Comments are welcome. > > --------------------------------------- > #!/usr/bin/env python > ''' > >>> concat('hello','world') > 'hello world' > ''' > from __future__ import unicode_literals > def concat( first, second ): > return first + ' ' + second > if __name__ == "__main__": > import doctest > doctest.testmod() > --------------------------------------- > #!/usr/bin/env python > from __future__ import unicode_literals > ''' > >>> concat('hello','world') > 'hello world' > ''' > def concat( first, second ): > return first + ' ' + second > if __name__ == "__main__": > import doctest > doctest.testmod() > --------------------------------------- > > > The first way shows the following failure: > > --------------------------------------- > Failed example: > concat('hello','world') > Expected: > 'hello world' > Got: > u'hello world' > > --------------------------------------- > > Regards. > > Matt. From thomasmallen at gmail.com Fri Jan 29 10:54:23 2010 From: thomasmallen at gmail.com (Thomas Allen) Date: Fri, 29 Jan 2010 07:54:23 -0800 (PST) Subject: SimpleXMLRPCServer daemon Message-ID: <1e689e5c-8e2a-4896-b867-717316fa783e@o26g2000vbd.googlegroups.com> I have a script that runs an instance of SimpleXMLRPCServer and in general it works as expected. In its __del__, it is supposed to clean up its PID file (written on boot). I have two problems with this server instance: The first is that tt doesn't always clean up its PID file; is there a more reliable way to do this than how I am currently? The second is that when it does crash, I don't know about it...what would be sufficient as a "keep-alive" script to restart it? I suppose I could use something like EventMachine (already installed on my server) to watch the PID file if it were deleted reliably. Thomas Allen From awilliam at opengroupware.us Fri Jan 29 11:07:07 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Fri, 29 Jan 2010 11:07:07 -0500 Subject: SimpleXMLRPCServer daemon In-Reply-To: <1e689e5c-8e2a-4896-b867-717316fa783e@o26g2000vbd.googlegroups.com> References: <1e689e5c-8e2a-4896-b867-717316fa783e@o26g2000vbd.googlegroups.com> Message-ID: <1264781227.6660.17.camel@linux-m3mt> On Fri, 2010-01-29 at 07:54 -0800, Thomas Allen wrote: > I have a script that runs an instance of SimpleXMLRPCServer and in > general it works as expected. In its __del__, it is supposed to clean > up its PID file (written on boot). I have two problems with this > server instance: The first is that tt doesn't always clean up its PID > file; is there a more reliable way to do this than how I am currently? > The second is that when it does crash, I don't know about it...what > would be sufficient as a "keep-alive" script to restart it? I suppose > I could use something like EventMachine (already installed on my > server) to watch the PID file if it were deleted reliably. Why don't you wrap the server in a try/except block? Then if an exception occurs killing the server you can automatically restart and/or send an e-mail to an admin address. From michele.simionato at gmail.com Fri Jan 29 11:09:40 2010 From: michele.simionato at gmail.com (Michele Simionato) Date: Fri, 29 Jan 2010 08:09:40 -0800 (PST) Subject: Function name unchanged in error message References: Message-ID: On Jan 29, 2:30?pm, andrew cooke wrote: > Is there any way to change the name of the function in an error > message? ?In the example below I'd like the error to refer to bar(), > for example (the motivation is related function decorators - I'd like > the wrapper function to give the same name) Use the decorator module which does the right thing: http://pypi.python.org/pypi/decorator From mattsteel at hotmail.it Fri Jan 29 11:21:55 2010 From: mattsteel at hotmail.it (Mattsteel) Date: Fri, 29 Jan 2010 08:21:55 -0800 (PST) Subject: Funny behaviour with __future__ and doctest between 2.6 and 3.1 References: Message-ID: <3a02ac42-337b-4778-99e0-1309e012163d@g29g2000yqe.googlegroups.com> Hi Peter. Sadly (for me), you're right... then the only way to use doctest to work both in 2.6 and 3.1 (without modifications between them) is something like this: #!/usr/bin/env python ''' >>> str(concat('hello','world')) 'hello world' ''' from __future__ import unicode_literals def concat( first, second ): return first + ' ' + second if __name__ == "__main__": import doctest doctest.testmod() Is there any way to avoid using str(...) to protect the string? M. --------------------------------------- #!/usr/bin/env python ''' >>> concat('hello','world') 'hello world' ''' from __future__ import unicode_literals def concat( first, second ): return first + ' ' + second if __name__ == "__main__": import doctest doctest.testmod() On 29 Gen, 16:50, Peter Otten <__pete... at web.de> wrote: > Mattsteel wrote: > > Hello all. > > I'm using Python 2.6.4 and Python 3.1.1. > > My wish is to code in a 3.1-compliant way using 2.6, so I'm importing > > the __future__ module. > > I've found a funny thing comparing the two folliwing snippets that > > differ for one line only, that is the position of __future__ import > > (before or after the doc string). > > > Well, I understand the subtle difference but still I wander what > > really happen behind the scenes. > > Are you sure? The second script has no module docstring, just a string > literal somewhere in the module, and therefore no tests. You can see that by > running it with the verbose option -v. Also, > > from __future__ import unicode_literals > > doesn't affect the repr() of a unicode instance. But the interactive > interpreter invokes repr() on the result before it is printed: > > >>> from __future__ import unicode_literals > >>> "yadda" > > u'yadda' > > > Comments are welcome. > > > --------------------------------------- > > ? ? #!/usr/bin/env python > > ? ? ''' > > ? ? >>> concat('hello','world') > > ? ? 'hello world' > > ? ? ''' > > ? ? from __future__ ?import unicode_literals > > ? ? def concat( first, second ): > > ? ? ? ? return first + ' ' + second > > ? ? if __name__ == "__main__": > > ? ? ? ? import doctest > > ? ? ? ? doctest.testmod() > > --------------------------------------- > > ? ? #!/usr/bin/env python > > ? ? from __future__ ?import unicode_literals > > ? ? ''' > > ? ? >>> concat('hello','world') > > ? ? 'hello world' > > ? ? ''' > > ? ? def concat( first, second ): > > ? ? ? ? return first + ' ' + second > > ? ? if __name__ == "__main__": > > ? ? ? ? import doctest > > ? ? ? ? doctest.testmod() > > --------------------------------------- > > > The first way shows the following failure: > > > --------------------------------------- > > ? Failed example: > > ? ? ? concat('hello','world') > > ? Expected: > > ? ? ? 'hello world' > > ? Got: > > ? ? ? u'hello world' > > > --------------------------------------- > > > Regards. > > > Matt. > > From __peter__ at web.de Fri Jan 29 11:30:30 2010 From: __peter__ at web.de (Peter Otten) Date: Fri, 29 Jan 2010 17:30:30 +0100 Subject: Funny behaviour with __future__ and doctest between 2.6 and 3.1 References: <3a02ac42-337b-4778-99e0-1309e012163d@g29g2000yqe.googlegroups.com> Message-ID: Mattsteel wrote: > Sadly (for me), you're right... then the only way to use doctest to > work both in 2.6 and 3.1 (without modifications between them) is > something like this: > > #!/usr/bin/env python > ''' >>>> str(concat('hello','world')) > 'hello world' > ''' > from __future__ import unicode_literals > def concat( first, second ): > return first + ' ' + second > if __name__ == "__main__": > import doctest > doctest.testmod() > > Is there any way to avoid using str(...) to protect the string? > M. I think you can work around the problem. The following should pass in Python 2.6 and 3.1: ''' >>> concat('hello','world') == 'hello world' True ''' from __future__ import unicode_literals def concat( first, second ): return first + ' ' + second if __name__ == "__main__": import doctest doctest.testmod() Peter From tim.wintle at teamrubber.com Fri Jan 29 11:35:18 2010 From: tim.wintle at teamrubber.com (Tim Wintle) Date: Fri, 29 Jan 2010 16:35:18 +0000 Subject: SimpleXMLRPCServer daemon In-Reply-To: <1e689e5c-8e2a-4896-b867-717316fa783e@o26g2000vbd.googlegroups.com> References: <1e689e5c-8e2a-4896-b867-717316fa783e@o26g2000vbd.googlegroups.com> Message-ID: <1264782918.28992.3.camel@localhost> On Fri, 2010-01-29 at 07:54 -0800, Thomas Allen wrote: > The second is that when it does crash, I don't know about it...what > would be sufficient as a "keep-alive" script to restart it? I suppose > I could use something like EventMachine (already installed on my > server) to watch the PID file if it were deleted reliably. If the server crashes then it clearly won't get around to deleting it's pid file. The way I do it is to use os.kill (with signal 0 IIRC) to check if the process is still alive when the script starts. If it's not then I delete the pid file and carry on starting up. Tim From aharrisreid at googlemail.com Fri Jan 29 11:37:39 2010 From: aharrisreid at googlemail.com (Alan Harris-Reid) Date: Fri, 29 Jan 2010 16:37:39 +0000 Subject: Threading issue with SQLite Message-ID: <4B630ED3.5000706@googlemail.com> Hi, I am creating a web application (using Python 3.1 and CherryPy 3.2) where a SQLite connection and cursor object are created using the following code (simplified from the original): class MainSite: con = sqlite.connect('MyDatabase.db') cursor = con.cursor() def index_page(): some HTML code cursor.execute(some SQL statement) more HTML code def another_page(): some HTML code cursor.execute(anotherSQL statement) more HTML code When I call a URL which launches the another_page() method I get the error message "sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread." Questions... 1. Is there a large overhead in opening a new SQLite connection for each thread (ie. within each method)? 2. Is there any way to use the same connection for the whole class (or should I forget that idea completely?) 3. When a method returns to the calling method, is the connection automatically closed (assuming the object is local, of course) or does it have to be done explicitly using connection.close()? TIA, Alan Harris-Reid From mr.spoon21 at gmail.com Fri Jan 29 11:53:43 2010 From: mr.spoon21 at gmail.com (Mr.SpOOn) Date: Fri, 29 Jan 2010 16:53:43 +0000 Subject: Keyboard input Message-ID: <8f67b6f81001290853i39bab1acs49d02ac5be948d13@mail.gmail.com> Hi, I need to get keyboard input in a python program. I need it to let the user choose some options, for example: 1) option 1 2) option 2 3) option 3 and then to input some data to the program. I'm using the raw_input method and it works fine, but I noted that I can't use backspace and when I accidentally press shift space (because I need to input uppercase letters separated by a space) it writes some strange characters. So, is there another way to get keyboard input? Thanks, Carlo From mrm at unknown.nospam Fri Jan 29 12:00:34 2010 From: mrm at unknown.nospam (Mr.M) Date: Fri, 29 Jan 2010 17:00:34 GMT Subject: get PyObject* knowing its string name Message-ID: I think this sounds like a stupid question, but I searched the C/Api doc and google but I wasn't able to find any hint: how can I retrive PyObject pointer if I only know it's name? What I'd like to do is something like this: [code] PyObject* obj = PyFindWhatImLookingFor("w.z.y.x"); [/code] Of course if I know that w is a module, z is a class and so on I can search w dictionary than z members than... But I wonder if there is a way to retrive without knowing what w.z.y.x means. Thank you, Luca. From morganhein at gmail.com Fri Jan 29 12:13:28 2010 From: morganhein at gmail.com (JohnnyFive) Date: Fri, 29 Jan 2010 09:13:28 -0800 (PST) Subject: Sleep timer but still responsive? References: Message-ID: <8c76dbf2-8853-48cc-9ca9-37929936f73a@b35g2000vbc.googlegroups.com> On Jan 28, 4:55?pm, "Gabriel Genellina" wrote: > Please provide more details. What do you want your program to do while ? > sleeping? What kind of actions do you want a response to? > Do you have a GUI? A curses-based interfase? > > -- > Gabriel Genellina My app is purely console based. I just don't want the console to lock up (on Windows using time.sleep(x) causes the console to become unresponsive until the timer is done), and I want people to be able to CTRL+C to stop the script if need be (which can't be done if it's unresponsive!). Thanks. From jakecjacobson at gmail.com Fri Jan 29 12:25:19 2010 From: jakecjacobson at gmail.com (jakecjacobson) Date: Fri, 29 Jan 2010 09:25:19 -0800 (PST) Subject: Processing XML File Message-ID: <6bac2d7b-509d-4f92-912f-5e8dc68c3f30@36g2000yqu.googlegroups.com> I need to take a XML web resource and split it up into smaller XML files. I am able to retrieve the web resource but I can't find any good XML examples. I am just learning Python so forgive me if this question has been answered many times in the past. My resource is like: ... ... ... ... So in this example, I would need to output 2 files with the contents of each file what is between the open and close document tag. From andrej.mitrovich at gmail.com Fri Jan 29 12:27:06 2010 From: andrej.mitrovich at gmail.com (Andrej Mitrovic) Date: Fri, 29 Jan 2010 09:27:06 -0800 (PST) Subject: Changing tab behavior in Python Interactive mode Message-ID: <93ae66b7-7fe2-4542-b8d0-039f9423cdd8@n7g2000yqb.googlegroups.com> I've noticed that when running Python in interactive mode (via cmd on windows), the first time I hit tab it will move 4 spaces to the right, however each new tab will move 8 spaces instead of 4. Why this inconsistent behavior? And how could I change this to be consistent and always move only 4 spaces? From robert.kern at gmail.com Fri Jan 29 12:29:01 2010 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 29 Jan 2010 11:29:01 -0600 Subject: get PyObject* knowing its string name In-Reply-To: References: Message-ID: On 2010-01-29 11:00 AM, Mr.M wrote: > I think this sounds like a stupid question, but I searched the C/Api doc > and google but I wasn't able to find any hint: > > how can I retrive PyObject pointer if I only know it's name? > > What I'd like to do is something like this: > > [code] > PyObject* obj = PyFindWhatImLookingFor("w.z.y.x"); > [/code] > > Of course if I know that w is a module, z is a class and so on I can > search w dictionary than z members than... > > But I wonder if there is a way to retrive without knowing what w.z.y.x > means. There is probably a clever/careful way to do it, but most systems that do this just separate the package/module parts from the object parts: "w:z.y.x" Import the module part, then getattr() your way down the object part. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From andreas.tawn at ubisoft.com Fri Jan 29 12:33:19 2010 From: andreas.tawn at ubisoft.com (Andreas Tawn) Date: Fri, 29 Jan 2010 18:33:19 +0100 Subject: Sleep timer but still responsive? In-Reply-To: <8c76dbf2-8853-48cc-9ca9-37929936f73a@b35g2000vbc.googlegroups.com> References: <8c76dbf2-8853-48cc-9ca9-37929936f73a@b35g2000vbc.googlegroups.com> Message-ID: <654D9D97DA51AD479973BC2D5578603C06AAD6AC51@PDC-MAIL-CMS01.ubisoft.org> > On Jan 28, 4:55?pm, "Gabriel Genellina" > wrote: > > Please provide more details. What do you want your program to do > while > > sleeping? What kind of actions do you want a response to? > > Do you have a GUI? A curses-based interfase? > > > > -- > > Gabriel Genellina > > My app is purely console based. I just don't want the console to lock > up (on Windows using time.sleep(x) causes the console to become > unresponsive until the timer is done), and I want people to be able to > CTRL+C to stop the script if need be (which can't be done if it's > unresponsive!). > > Thanks. How about this? Responds to ctrl+c, but still sleeps. import time def responsiveSleep(n): while n > 0: time.sleep(1) n -= 1 Cheers, Drea From python at mrabarnett.plus.com Fri Jan 29 12:39:31 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 29 Jan 2010 17:39:31 +0000 Subject: Sleep timer but still responsive? In-Reply-To: <8c76dbf2-8853-48cc-9ca9-37929936f73a@b35g2000vbc.googlegroups.com> References: <8c76dbf2-8853-48cc-9ca9-37929936f73a@b35g2000vbc.googlegroups.com> Message-ID: <4B631D53.6030602@mrabarnett.plus.com> JohnnyFive wrote: > On Jan 28, 4:55 pm, "Gabriel Genellina" > wrote: >> Please provide more details. What do you want your program to do while >> sleeping? What kind of actions do you want a response to? >> Do you have a GUI? A curses-based interfase? >> >> -- >> Gabriel Genellina > > My app is purely console based. I just don't want the console to lock > up (on Windows using time.sleep(x) causes the console to become > unresponsive until the timer is done), and I want people to be able to > CTRL+C to stop the script if need be (which can't be done if it's > unresponsive!). > > Thanks. Which version of Python are you using? time.sleep(x) can be interrupted in Python 2.6. If the version you're using can't be interrupted then you could use multiple sleeps: # Wait for a total of 10 secs. for i in range(10): time.sleep(1) From steve at holdenweb.com Fri Jan 29 12:47:46 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 29 Jan 2010 12:47:46 -0500 Subject: Changing tab behavior in Python Interactive mode In-Reply-To: <93ae66b7-7fe2-4542-b8d0-039f9423cdd8@n7g2000yqb.googlegroups.com> References: <93ae66b7-7fe2-4542-b8d0-039f9423cdd8@n7g2000yqb.googlegroups.com> Message-ID: Andrej Mitrovic wrote: > I've noticed that when running Python in interactive mode (via cmd on > windows), the first time I hit tab it will move 4 spaces to the right, > however each new tab will move 8 spaces instead of 4. Why this > inconsistent behavior? And how could I change this to be consistent > and always move only 4 spaces? The first tab you are starting four characters in due to the interpreter already having printed ">>> ". regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From awilliam at opengroupware.us Fri Jan 29 13:04:44 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Fri, 29 Jan 2010 13:04:44 -0500 Subject: Processing XML File In-Reply-To: <6bac2d7b-509d-4f92-912f-5e8dc68c3f30@36g2000yqu.googlegroups.com> References: <6bac2d7b-509d-4f92-912f-5e8dc68c3f30@36g2000yqu.googlegroups.com> Message-ID: <1264788284.3027.0.camel@linux-m3mt> On Fri, 2010-01-29 at 09:25 -0800, jakecjacobson wrote: > I need to take a XML web resource and split it up into smaller XML > files. I am able to retrieve the web resource but I can't find any > good XML examples. I am just learning Python so forgive me if this > question has been answered many times in the past. > My resource is like: > > ... > ... > > > ... > ... > > So in this example, I would need to output 2 files with the contents > of each file what is between the open and close document tag. Do you want to parse the document or SaX? I have a SaX example at From gagsl-py2 at yahoo.com.ar Fri Jan 29 13:19:19 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 15:19:19 -0300 Subject: Sleep timer but still responsive? References: <8c76dbf2-8853-48cc-9ca9-37929936f73a@b35g2000vbc.googlegroups.com> <4B631D53.6030602@mrabarnett.plus.com> Message-ID: En Fri, 29 Jan 2010 14:39:31 -0300, MRAB escribi?: > JohnnyFive wrote: >> My app is purely console based. I just don't want the console to lock >> up (on Windows using time.sleep(x) causes the console to become >> unresponsive until the timer is done), and I want people to be able to >> CTRL+C to stop the script if need be (which can't be done if it's >> unresponsive!). >> Thanks. > > Which version of Python are you using? time.sleep(x) can be interrupted > in Python 2.6. I'm able to be more precise: time.sleep() can be interrupted in any Python version since 2.3. To the OP: beware of any unqualified 'except' clauses; a block like this: try: ... except: do_something_or_pass may "swallow" the KeyboardInterrupt exception (generated by a Ctrl-C press). From Python 2.5 and up, the most generic exception clause should read `except Exception: ...` In previous versions, you had to explicitely re-raise KeyboardInterrupt and SystemExit in any catch-all block: try: ... except (KeyboardInterrupt, SystemExit): raise except: ... -- Gabriel Genellina From morganhein at gmail.com Fri Jan 29 13:24:48 2010 From: morganhein at gmail.com (JohnnyFive) Date: Fri, 29 Jan 2010 10:24:48 -0800 (PST) Subject: Sleep timer but still responsive? References: <8c76dbf2-8853-48cc-9ca9-37929936f73a@b35g2000vbc.googlegroups.com> Message-ID: On Jan 29, 9:33?am, Andreas Tawn wrote: > > On Jan 28, 4:55?pm, "Gabriel Genellina" > > wrote: > > > Please provide more details. What do you want your program to do > > while > > > sleeping? What kind of actions do you want a response to? > > > Do you have a GUI? A curses-based interfase? > > > > -- > > > Gabriel Genellina > > > My app is purely console based. I just don't want the console to lock > > up (on Windows using time.sleep(x) causes the console to become > > unresponsive until the timer is done), and I want people to be able to > > CTRL+C to stop the script if need be (which can't be done if it's > > unresponsive!). > > > Thanks. > > How about this? Responds to ctrl+c, but still sleeps. > > import time > > def responsiveSleep(n): > ? ? while n > 0: > ? ? ? ? time.sleep(1) > ? ? ? ? n -= 1 > > Cheers, > > Drea Thanks for the ideas! Maybe it's just my computer, but using your solution still causes the prompt to become unresponsive during the sleeps. I am using 2.6.4 btw. It's not a major deal though, I just thought there had to be a way to do this fairly easily. From jakecjacobson at gmail.com Fri Jan 29 13:34:17 2010 From: jakecjacobson at gmail.com (jakecjacobson) Date: Fri, 29 Jan 2010 10:34:17 -0800 (PST) Subject: Processing XML File References: <6bac2d7b-509d-4f92-912f-5e8dc68c3f30@36g2000yqu.googlegroups.com> Message-ID: On Jan 29, 1:04?pm, Adam Tauno Williams wrote: > On Fri, 2010-01-29 at 09:25 -0800, jakecjacobson wrote: > > I need to take a XML web resource and split it up into smaller XML > > files. ?I am able to retrieve the web resource but I can't find any > > good XML examples. ?I am just learning Python so forgive me if this > > question has been answered many times in the past. > > My resource is like: > > > > ? ? ?... > > ? ? ?... > > > > > > ? ? ?... > > ? ? ?... > > > > So in this example, I would need to output 2 files with the contents > > of each file what is between the open and close document tag. > > Do you want to parse the document or SaX? > > I have a SaX example at > Thanks but I am way over my head with XML, Python. I am working with DDMS and need to output the individual resource nodes to their own file. I hope that this helps and I need a good example and how to use it. Here is what a resource node looks like: Sample Taxonomy This is a sample taxonomy created for the Help page. Sample Developer FGM, Inc. 703-885-1000 sampleDeveloper at fgm.com You can see the DDMS site at https://metadata.dod.mil/. From awilliam at opengroupware.us Fri Jan 29 13:54:26 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Fri, 29 Jan 2010 13:54:26 -0500 Subject: Processing XML File In-Reply-To: References: <6bac2d7b-509d-4f92-912f-5e8dc68c3f30@36g2000yqu.googlegroups.com> Message-ID: <1264791266.3492.3.camel@linux-m3mt> On Fri, 2010-01-29 at 10:34 -0800, jakecjacobson wrote: > On Jan 29, 1:04 pm, Adam Tauno Williams > wrote: > > On Fri, 2010-01-29 at 09:25 -0800, jakecjacobson wrote: > > > I need to take a XML web resource and split it up into smaller XML > > > files. I am able to retrieve the web resource but I can't find any > > > good XML examples. I am just learning Python so forgive me if this > > > question has been answered many times in the past. > > > My resource is like: > > > > > > ... > > > ... > > > > > > > > > > > > So in this example, I would need to output 2 files with the contents > > > of each file what is between the open and close document tag. > > Do you want to parse the document or SaX? > > I have a SaX example at > > > Thanks but I am way over my head with XML, Python. I am working with > DDMS and need to output the individual resource nodes to their own > file. I hope that this helps and I need a good example and how to use > it. If that is all you need XPath will spit it apart for you like doc = etree.parse(self._rfile) results = doc.xpath(xpath) for result in results: print str(result) For example if your XML has an outermost element of ResultSet with inner row elements just do: for record in doc.xpath(u'/ResultSet/row') Implied import for these examples is "from lxml import etree" > Here is what a resource node looks like: > xsi:schemaLocation="https://metadata.dod.mil/mdr/ns/DDMS/1.4/ > https://metadata.dod.mil/mdr/ns/DDMS/1.4/" > xmlns:ddms="https://metadata.dod.mil/mdr/ns/DDMS/1.4/" > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" > xmlns:ICISM="urn:us:gov:ic:ism:v2"> > > > > ICISM:classification="U">Sample Taxonomy > ICISM:classification="U"> > This is a sample taxonomy created for the Help page. > > > ICISM:classification="U"> > > Sample > Developer > FGM, Inc. > 703-885-1000 > sampleDeveloper at fgm.com > > > ICISM:classification="U" ICISM:nonICmarkings="DIST_STMT_A" /> > > > > You can see the DDMS site at https://metadata.dod.mil/. -- OpenGroupware developer: awilliam at whitemice.org OpenGroupare & Cyrus IMAPd documenation @ From stefan_ml at behnel.de Fri Jan 29 14:24:35 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 29 Jan 2010 20:24:35 +0100 Subject: Processing XML File In-Reply-To: <6bac2d7b-509d-4f92-912f-5e8dc68c3f30@36g2000yqu.googlegroups.com> References: <6bac2d7b-509d-4f92-912f-5e8dc68c3f30@36g2000yqu.googlegroups.com> Message-ID: <4b6335f3$0$6573$9b4e6d93@newsspool3.arcor-online.net> jakecjacobson, 29.01.2010 18:25: > I need to take a XML web resource and split it up into smaller XML > files. I am able to retrieve the web resource but I can't find any > good XML examples. I am just learning Python so forgive me if this > question has been answered many times in the past. > > My resource is like: > > > ... > ... > > > ... > ... > Is this what you get as a document or is this just /contained/ in the document? Note that XML does not allow more than one root element, so the above is not XML. Each of the two ... parts form an XML document by themselves, though. > So in this example, I would need to output 2 files with the contents > of each file what is between the open and close document tag. Are the two files formatted as you show above? In that case, you can simply iterate over the lines and cut the document when you see "". Or, if you are sure that "" only appears as top-most elements and not inside of the documents, you can search for "" in the content (a string, I guess) and split it there. As was pointed out before, once you have these two documents, use the xml.etree package to work with them. Something like this might work: import xml.etree.ElementTree as ET data = urllib2.urlopen(url).read() for part in data.split(''): document = ET.fromstring(''+part) print(document.tag) # ... do other stuff Stefan From fred.sells at adventistcare.org Fri Jan 29 14:31:56 2010 From: fred.sells at adventistcare.org (Sells, Fred) Date: Fri, 29 Jan 2010 14:31:56 -0500 Subject: Processing XML File In-Reply-To: <4b6335f3$0$6573$9b4e6d93@newsspool3.arcor-online.net> References: <6bac2d7b-509d-4f92-912f-5e8dc68c3f30@36g2000yqu.googlegroups.com> <4b6335f3$0$6573$9b4e6d93@newsspool3.arcor-online.net> Message-ID: Google is your friend. Elementtree is one of the better documented IMHO, but there are many modules to do this. > -----Original Message----- > From: python-list-bounces+frsells=adventistcare.org at python.org > [mailto:python-list-bounces+frsells=adventistcare.org at python.org] On > Behalf Of Stefan Behnel > Sent: Friday, January 29, 2010 2:25 PM > To: python-list at python.org > Subject: Re: Processing XML File > > jakecjacobson, 29.01.2010 18:25: > > I need to take a XML web resource and split it up into smaller XML > > files. I am able to retrieve the web resource but I can't find any > > good XML examples. I am just learning Python so forgive me if this > > question has been answered many times in the past. > > > > My resource is like: > > > > > > ... > > ... > > > > > > ... > > ... > > > > Is this what you get as a document or is this just /contained/ in the > document? > > Note that XML does not allow more than one root element, so the above is > not XML. Each of the two ... parts form an XML > document by themselves, though. > > > > So in this example, I would need to output 2 files with the contents > > of each file what is between the open and close document tag. > > Are the two files formatted as you show above? In that case, you can > simply > iterate over the lines and cut the document when you see "". Or, > if you are sure that "" only appears as top-most elements and > not > inside of the documents, you can search for "" in the content (a > string, I guess) and split it there. > > As was pointed out before, once you have these two documents, use the > xml.etree package to work with them. > > Something like this might work: > > import xml.etree.ElementTree as ET > > data = urllib2.urlopen(url).read() > > for part in data.split(''): > document = ET.fromstring(''+part) > print(document.tag) > # ... do other stuff > > Stefan > -- > http://mail.python.org/mailman/listinfo/python-list From stefan_ml at behnel.de Fri Jan 29 14:41:58 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 29 Jan 2010 20:41:58 +0100 Subject: Processing XML File In-Reply-To: References: <6bac2d7b-509d-4f92-912f-5e8dc68c3f30@36g2000yqu.googlegroups.com> <4b6335f3$0$6573$9b4e6d93@newsspool3.arcor-online.net> Message-ID: <4b633a07$0$6582$9b4e6d93@newsspool3.arcor-online.net> Sells, Fred, 29.01.2010 20:31: > Google is your friend. Elementtree is one of the better documented > IMHO, but there are many modules to do this. Unless the OP provides some more information, "do this" is rather underdefined. And sending someone off to Google who is just learning the basics of Python and XML and trying to solve a very specific problem with them is not exactly the spirit I'm used to in this newsgroup. Stefan From half.italian at gmail.com Fri Jan 29 15:04:08 2010 From: half.italian at gmail.com (Sean DiZazzo) Date: Fri, 29 Jan 2010 12:04:08 -0800 (PST) Subject: SimpleXMLRPCServer daemon References: <1e689e5c-8e2a-4896-b867-717316fa783e@o26g2000vbd.googlegroups.com> Message-ID: <9f5d00f3-8485-43d0-bf55-4e71fb389065@a5g2000prg.googlegroups.com> On Jan 29, 7:54?am, Thomas Allen wrote: > I have a script that runs an instance of SimpleXMLRPCServer and in > general it works as expected. In its __del__, it is supposed to clean > up its PID file (written on boot). I have two problems with this > server instance: The first is that tt doesn't always clean up its PID > file; is there a more reliable way to do this than how I am currently? > The second is that when it does crash, I don't know about it...what > would be sufficient as a "keep-alive" script to restart it? I suppose > I could use something like EventMachine (already installed on my > server) to watch the PID file if it were deleted reliably. > > Thomas Allen You should check out python-daemon. I use home grown daemons all the time, and have only played a little with the python-daemon library, but it is surely written well, and handles lockfiles and pidfiles. I believe you can map a function to any signal that the daemon receives to do any kind of cleanup etc. Ssometimes those PID files might be left around, but the runner included with the module (i think) checks for stale pid files. There's all kinds of useful stuff. In my home grown daemons I use Adam's technique, and wrap the "while 1:" block in a try except clause. ~Sean From gagsl-py2 at yahoo.com.ar Fri Jan 29 15:04:49 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 17:04:49 -0300 Subject: Some C-API functions clear the error indicator? References: Message-ID: En Fri, 29 Jan 2010 11:37:09 -0300, Austin Bingham escribi?: > I've noticed that several (many?) python functions seem to clear the > error/exception indicators when they're called from a C/C++ program. > For example, both PyImport_ImportModule and traceback.extract_tb() > (called via the function call methods) do this: if error indicators > are set prior to their call (as indicated by PyErr_Fetch, and > including a call to PyErr_Restore), I see that they are unset (using > the same method) after the call. This happens even when the functions > succeed. It's simple: you have to check *every* function call for failure. Many functions return new object references and you have to properly decrement them in case of failure, so in most cases this means that you have to check each and every call. -- Gabriel Genellina From gagsl-py2 at yahoo.com.ar Fri Jan 29 15:18:42 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 17:18:42 -0300 Subject: site.py confusion References: Message-ID: En Wed, 27 Jan 2010 15:48:23 -0300, George Trojan escribi?: > Arnaud Delobelle wrote: >> George Trojan writes: >> >>> Inspired by the 'Default path for files' thread I tried to use >>> sitecustomize in my code. What puzzles me is that the site.py's main() >>> is not executed. My sitecustomize.py is > > That gave me the explanation why the above happens: when site is > imported, the current directory is not yet prepended to sys.path. > > I wanted to have library location specific to application without having > to explicitly change sys.path in all App-Top-Dir/bin/*.py. I thought > creating bin/sitecustomize.py would do the trick. Put sitecustomize.py in any of those directories already in sys.path; on Python 2.6, you may put sitecustomize.py on your "user site directory", ~/.local/lib/python2.6/site-packages (see PEP370; it won't show up in sys.path unless the directory actually exists) -- Gabriel Genellina From gagsl-py2 at yahoo.com.ar Fri Jan 29 15:28:24 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 17:28:24 -0300 Subject: SimpleXMLRPCServer daemon References: <1e689e5c-8e2a-4896-b867-717316fa783e@o26g2000vbd.googlegroups.com> Message-ID: En Fri, 29 Jan 2010 12:54:23 -0300, Thomas Allen escribi?: > I have a script that runs an instance of SimpleXMLRPCServer and in > general it works as expected. In its __del__, it is supposed to clean > up its PID file (written on boot). I have two problems with this > server instance: The first is that tt doesn't always clean up its PID > file; is there a more reliable way to do this than how I am currently? > The second is that when it does crash, I don't know about it...what > would be sufficient as a "keep-alive" script to restart it? I suppose > I could use something like EventMachine (already installed on my > server) to watch the PID file if it were deleted reliably. I agree with the recommendation of using some daemon library; doing it right is better left to the experts :) But if you can't or don't want to alter your code so much, I suggest: - use atexit.register, instead of __del__, to delete the PID file - keep a lock on the pid file; if a second instance is able to write to it, it means it's an orphan from a previous, aborted run. (Checking whether a process with such pid exists is not enough - pids are recycled rather fast) -- Gabriel Genellina From andrej.mitrovich at gmail.com Fri Jan 29 15:35:58 2010 From: andrej.mitrovich at gmail.com (Andrej Mitrovic) Date: Fri, 29 Jan 2010 12:35:58 -0800 (PST) Subject: Changing tab behavior in Python Interactive mode References: <93ae66b7-7fe2-4542-b8d0-039f9423cdd8@n7g2000yqb.googlegroups.com> Message-ID: <073949c2-787d-44a4-a471-c77e605d32de@y12g2000yqh.googlegroups.com> On Jan 29, 6:47?pm, Steve Holden wrote: > Andrej Mitrovic wrote: > > I've noticed that when running Python in interactive mode (via cmd on > > windows), the first time I hit tab it will move 4 spaces to the right, > > however each new tab will move 8 spaces instead of 4. Why this > > inconsistent behavior? And how could I change this to be consistent > > and always move only 4 spaces? > > The first tab you are starting four characters in due to the interpreter > already having printed ">>> ". > > regards > ?Steve > -- > Steve Holden ? ? ? ? ? +1 571 484 6266 ? +1 800 494 3119 > PyCon is coming! Atlanta, Feb 2010 ?http://us.pycon.org/ > Holden Web LLC ? ? ? ? ? ? ? ?http://www.holdenweb.com/ > UPCOMING EVENTS: ? ? ? ?http://holdenweb.eventbrite.com/ I see. I wonder if I could modify this behavior.. I guess I'll have to take a look at the source. Thanks anyway! From gagsl-py2 at yahoo.com.ar Fri Jan 29 15:37:18 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 17:37:18 -0300 Subject: Function name unchanged in error message References: Message-ID: En Fri, 29 Jan 2010 13:09:40 -0300, Michele Simionato escribi?: > On Jan 29, 2:30 pm, andrew cooke wrote: >> Is there any way to change the name of the function in an error >> message? In the example below I'd like the error to refer to bar(), >> for example (the motivation is related function decorators - I'd like >> the wrapper function to give the same name) > > Use the decorator module which does the right thing: > http://pypi.python.org/pypi/decorator The decorator module is a very fine addition to anyone's tool set -- but in this case it is enough to use the wraps() function from the functools standard module. -- Gabriel Genellina From gagsl-py2 at yahoo.com.ar Fri Jan 29 15:43:43 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 17:43:43 -0300 Subject: python for net manager References: <6e5636721001261911i6d5ca5fma2dff5b8a89ed277@mail.gmail.com> Message-ID: En Wed, 27 Jan 2010 00:11:10 -0300, moon sky escribi?: > i just want to write a small tool to manage the network, including detect > the ip addr,netmask, and send the arp request to find out locale's > ip-mac > turtple, > is there any way to succeed this exlude 'subprocess call'? Try scapy: http://www.secdev.org/projects/scapy/ -- Gabriel Genellina From steve at holdenweb.com Fri Jan 29 16:07:09 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 29 Jan 2010 16:07:09 -0500 Subject: Changing tab behavior in Python Interactive mode In-Reply-To: <073949c2-787d-44a4-a471-c77e605d32de@y12g2000yqh.googlegroups.com> References: <93ae66b7-7fe2-4542-b8d0-039f9423cdd8@n7g2000yqb.googlegroups.com> <073949c2-787d-44a4-a471-c77e605d32de@y12g2000yqh.googlegroups.com> Message-ID: <4B634DFD.20309@holdenweb.com> Andrej Mitrovic wrote: > On Jan 29, 6:47 pm, Steve Holden wrote: >> Andrej Mitrovic wrote: >>> I've noticed that when running Python in interactive mode (via cmd on >>> windows), the first time I hit tab it will move 4 spaces to the right, >>> however each new tab will move 8 spaces instead of 4. Why this >>> inconsistent behavior? And how could I change this to be consistent >>> and always move only 4 spaces? >> The first tab you are starting four characters in due to the interpreter >> already having printed ">>> ". >> >> regards >> Steve >> -- >> Steve Holden +1 571 484 6266 +1 800 494 3119 >> PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ >> Holden Web LLC http://www.holdenweb.com/ >> UPCOMING EVENTS: http://holdenweb.eventbrite.com/ > > I see. I wonder if I could modify this behavior.. I guess I'll have to > take a look at the source. Thanks anyway! I's nothing to do with the source: it isn't the interpreter that's deciding where the "tab stops" are, it's the console window the interpreter is running in. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Fri Jan 29 16:07:09 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 29 Jan 2010 16:07:09 -0500 Subject: Changing tab behavior in Python Interactive mode In-Reply-To: <073949c2-787d-44a4-a471-c77e605d32de@y12g2000yqh.googlegroups.com> References: <93ae66b7-7fe2-4542-b8d0-039f9423cdd8@n7g2000yqb.googlegroups.com> <073949c2-787d-44a4-a471-c77e605d32de@y12g2000yqh.googlegroups.com> Message-ID: <4B634DFD.20309@holdenweb.com> Andrej Mitrovic wrote: > On Jan 29, 6:47 pm, Steve Holden wrote: >> Andrej Mitrovic wrote: >>> I've noticed that when running Python in interactive mode (via cmd on >>> windows), the first time I hit tab it will move 4 spaces to the right, >>> however each new tab will move 8 spaces instead of 4. Why this >>> inconsistent behavior? And how could I change this to be consistent >>> and always move only 4 spaces? >> The first tab you are starting four characters in due to the interpreter >> already having printed ">>> ". >> >> regards >> Steve >> -- >> Steve Holden +1 571 484 6266 +1 800 494 3119 >> PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ >> Holden Web LLC http://www.holdenweb.com/ >> UPCOMING EVENTS: http://holdenweb.eventbrite.com/ > > I see. I wonder if I could modify this behavior.. I guess I'll have to > take a look at the source. Thanks anyway! I's nothing to do with the source: it isn't the interpreter that's deciding where the "tab stops" are, it's the console window the interpreter is running in. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From gagsl-py2 at yahoo.com.ar Fri Jan 29 16:17:52 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 18:17:52 -0300 Subject: Killing a Thread References: <201001281425.30434.richard.lamboj@bilcom.at> Message-ID: En Thu, 28 Jan 2010 10:25:30 -0300, Richard Lamboj escribi?: > which Method is better to kill a Thread? Using Thread Events, or a > raising a > Exception? Maybe someone has a small example for me? The best way is simply NOT to do that. You don't kill a thread, you ask it to commit suicide. There is no reliable way to forcefully shut down another thread. The thread must periodically check some value (perhaps an Event object; in simple cases any variable will do) and (cleanly) exit when asked to. Threads that run pure Python code may be interrupted using the API call PyThreadState_SetAsyncExc; look for a recipe using ctypes to call it from Python. -- Gabriel Genellina From austin.bingham at gmail.com Fri Jan 29 16:25:14 2010 From: austin.bingham at gmail.com (Austin Bingham) Date: Fri, 29 Jan 2010 22:25:14 +0100 Subject: Some C-API functions clear the error indicator? In-Reply-To: References: Message-ID: Maybe I'm not following what you're saying. In my case, I already know that an exception has been thrown. In the course of processing that exception, I call another function which, for whatever reason and even when it succeeds, clears the exception indicators. How can I address this issue by checking function calls for failure? Austin On Fri, Jan 29, 2010 at 9:04 PM, Gabriel Genellina wrote: > En Fri, 29 Jan 2010 11:37:09 -0300, Austin Bingham > escribi?: > >> I've noticed that several (many?) python functions seem to clear the >> error/exception indicators when they're called from a C/C++ program. >> For example, both PyImport_ImportModule and traceback.extract_tb() >> (called via the function call methods) do this: if error indicators >> are set prior to their call (as indicated by PyErr_Fetch, and >> including a call to PyErr_Restore), I see that they are unset (using >> the same method) after the call. This happens even when the functions >> succeed. > > It's simple: you have to check *every* function call for failure. Many > functions return new object references and you have to properly decrement > them in case of failure, so in most cases this means that you have to check > each and every call. > > -- > Gabriel Genellina > > -- > http://mail.python.org/mailman/listinfo/python-list > From python at bdurham.com Fri Jan 29 16:33:11 2010 From: python at bdurham.com (python at bdurham.com) Date: Fri, 29 Jan 2010 16:33:11 -0500 Subject: Thread safe locale techniques? Message-ID: <1264800791.976.1357306091@webmail.messagingengine.com> We're currently writing a web application based on a threaded python web server framework (cherrypy) and would like to simultaneously support users from multiple locales. The locale module doesn't appear to be thread safe. Are there 3rd party libraries or modules that provide locale parsing and formatting functionality in a thread-safe way? Suggestions appreciated! Regards, Malcolm -------------- next part -------------- An HTML attachment was scrubbed... URL: From gagsl-py2 at yahoo.com.ar Fri Jan 29 16:38:47 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 18:38:47 -0300 Subject: Keyboard input References: <8f67b6f81001290853i39bab1acs49d02ac5be948d13@mail.gmail.com> Message-ID: En Fri, 29 Jan 2010 13:53:43 -0300, Mr.SpOOn escribi?: > I need to get keyboard input in a python program. I need it to let the > user choose some options, for example: > > 1) option 1 > 2) option 2 > 3) option 3 > > and then to input some data to the program. > > I'm using the raw_input method and it works fine, but I noted that I > can't use backspace and when I accidentally press shift space (because > I need to input uppercase letters separated by a space) it writes some > strange characters. That's strange. If you're using Linux, make sure you have the readline package installed. > So, is there another way to get keyboard input? If readline capabilities are not enough, you could try Urwid, a library for writing console programs: http://excess.org/urwid/ - but it's a big move. wxPython would let you write a graphical interface (and it's a bigger move!) -- Gabriel Genellina From andrej.mitrovich at gmail.com Fri Jan 29 16:43:05 2010 From: andrej.mitrovich at gmail.com (Andrej Mitrovic) Date: Fri, 29 Jan 2010 13:43:05 -0800 (PST) Subject: Changing tab behavior in Python Interactive mode References: <93ae66b7-7fe2-4542-b8d0-039f9423cdd8@n7g2000yqb.googlegroups.com> <073949c2-787d-44a4-a471-c77e605d32de@y12g2000yqh.googlegroups.com> Message-ID: <2c8c062f-0622-49ea-950b-80afde5371de@u41g2000yqe.googlegroups.com> On Jan 29, 10:07?pm, Steve Holden wrote: > Andrej Mitrovic wrote: > > On Jan 29, 6:47 pm, Steve Holden wrote: > >> Andrej Mitrovic wrote: > >>> I've noticed that when running Python in interactive mode (via cmd on > >>> windows), the first time I hit tab it will move 4 spaces to the right, > >>> however each new tab will move 8 spaces instead of 4. Why this > >>> inconsistent behavior? And how could I change this to be consistent > >>> and always move only 4 spaces? > >> The first tab you are starting four characters in due to the interpreter > >> already having printed ">>> ". > > >> regards > >> ?Steve > >> -- > >> Steve Holden ? ? ? ? ? +1 571 484 6266 ? +1 800 494 3119 > >> PyCon is coming! Atlanta, Feb 2010 ?http://us.pycon.org/ > >> Holden Web LLC ? ? ? ? ? ? ? ?http://www.holdenweb.com/ > >> UPCOMING EVENTS: ? ? ? ?http://holdenweb.eventbrite.com/ > > > I see. I wonder if I could modify this behavior.. I guess I'll have to > > take a look at the source. Thanks anyway! > > I's nothing to do with the source: it isn't the interpreter that's > deciding where the "tab stops" are, it's the console window the > interpreter is running in. > > regards > ?Steve > -- > Steve Holden ? ? ? ? ? +1 571 484 6266 ? +1 800 494 3119 > PyCon is coming! Atlanta, Feb 2010 ?http://us.pycon.org/ > Holden Web LLC ? ? ? ? ? ? ? ?http://www.holdenweb.com/ > UPCOMING EVENTS: ? ? ? ?http://holdenweb.eventbrite.com/ I see. Well I'm nitpicking here, it's not all that important (I just use cmd when I'm trying out small pieces of code). And I doubt there's a tabstop setting for cmd that I could use. Thanks for all your help! From gagsl-py2 at yahoo.com.ar Fri Jan 29 17:08:45 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 19:08:45 -0300 Subject: Data exchange between Delphi and Python (Win) References: <9e384ef61001280007j38f231e4we085e9af6d1b725d@mail.gmail.com> Message-ID: En Thu, 28 Jan 2010 05:07:00 -0300, Durumdara escribi?: > I have an exotic db, with exotic drivers, and it have buggy ODBC driver. > > But I have native driver - under Delphi. > > I need to access this DB under Pylons (or mod_python). > [...] > I think to COM/OLE, because it is accessable from all program, and I > think > to DLL (but DLL have problematic parameterisation). So, you can query the database in Delphi, and you want to be able to do that from Python. Your two ideas are feasible: you may write a DLL (or a COM object) in Delphi, that takes a query string, executes it, and returns the resulting data. All in Delphi code, no Python involved. Keep the memory allocation in Delphi - allocate and return a plain PChar buffer, and include a free() function to deallocate it when the other side is done using it. From Python, you may use ctypes [1] to call the DLL functions, or pywin32 [2] to invoke the COM object. Both ways are easy enough. [1] http://docs.python.org/library/ctypes.html [2] http://sourceforge.net/projects/pywin32/ -- Gabriel Genellina From rantingrick at gmail.com Fri Jan 29 17:12:45 2010 From: rantingrick at gmail.com (rantingrick) Date: Fri, 29 Jan 2010 14:12:45 -0800 (PST) Subject: Keyboard input References: <8f67b6f81001290853i39bab1acs49d02ac5be948d13@mail.gmail.com> Message-ID: On Jan 29, 3:38?pm, "Gabriel Genellina" wrote: > but it's a big ? > move. wxPython would let you write a graphical interface (and it's a ? > bigger move!) > > -- > Gabriel Genellina *ahem*!, tkinter is just a slightly medium move, and my god it's in the stdlib for crying out loud!! From jgardner at jonathangardner.net Fri Jan 29 17:15:30 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Fri, 29 Jan 2010 14:15:30 -0800 (PST) Subject: Keyboard input References: Message-ID: <24cdaadb-a8bd-47c1-8347-8229ccd3caba@b36g2000pri.googlegroups.com> On Jan 29, 8:53?am, "Mr.SpOOn" wrote: > Hi, > I need to get keyboard input in a python program. I need it to let the > user choose some options, for example: > > 1) option 1 > 2) option 2 > 3) option 3 > > and then to input some data to the program. > > I'm using the raw_input method and it works fine, but I noted that I > can't use backspace and when I accidentally press shift space (because > I need to input uppercase letters separated by a space) it writes some > strange characters. > > So, is there another way to get keyboard input? > > Maybe curses is overkill, but it will do what you need. Most interactive console apps use curses. You've run into some of the problems of simply reading STDIN. From jgardner at jonathangardner.net Fri Jan 29 17:18:04 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Fri, 29 Jan 2010 14:18:04 -0800 (PST) Subject: Threading issue with SQLite References: Message-ID: <09ca16b8-4e10-40f5-a69d-29fcdf6c2593@b9g2000pri.googlegroups.com> On Jan 29, 8:37?am, Alan Harris-Reid wrote: > > Questions... > 1. ?Is there a large overhead in opening a new SQLite connection for > each thread (ie. within each method)? Yes, but not as bad as some other DBs. > 2. ?Is there any way to use the same connection for the whole class (or > should I forget that idea completely?) Forget it. The thread stuff is pretty important. > 3. ?When a method returns to the calling method, is the connection > automatically closed (assuming the object is local, of course) or does > it have to be done explicitly using connection.close()? > If the object falls out of scope, that is, no code anywhere can reference it, it will be garbage collected and thus closed. Suggestion: Use something like SQLAlchemy to manage you DB interactions. One day, you'll move away from SQLite, and you'll be glad you were programming at a higher level. SQLAlchemy also does things like connection pooling and such for you. From pavlovevidence at gmail.com Fri Jan 29 17:23:48 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Fri, 29 Jan 2010 14:23:48 -0800 (PST) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> <4B629B6B.3000100@v.loewis.de> Message-ID: <5cd5a9f7-a42d-4f5a-a3f5-814a4992e700@b7g2000pro.googlegroups.com> On Jan 29, 12:25?am, "Martin v. Loewis" wrote: > > Well, I'd consider that an official release. ?Note that I didn't claim > > there was no hope PSF wouldn't change it's mind on 2.8. > > I'd like to point out that the PSF formally doesn't have any say in > this. Doesn't PSF own the Python trademark? Then it has to have a say, not over whether someone can fork the project or make another official release, but over whether they can do so and still call it Python. Carl Banks From jgardner at jonathangardner.net Fri Jan 29 17:27:19 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Fri, 29 Jan 2010 14:27:19 -0800 (PST) Subject: which one is faster? References: Message-ID: <002c2bc5-ed0a-4351-937a-358b987cadda@h9g2000prn.googlegroups.com> On Jan 28, 10:29?pm, "Stephen.Wu" <54wut... at gmail.com> wrote: > str.find(targetStr) > str.index(targetStr) with exception > str.count(targetStr) > targetStr in str > > which is the fastest way to check whether targetStr is in str? > The fastest way of all is to forget about this and finish the rest of your program. Developer time is much, much more valuable than processor time. When you are all done, and have solved all the other problems in the world, you can come back and pontificate on how many nanoseconds you can save by using "find" or "in". From ben+python at benfinney.id.au Fri Jan 29 17:29:31 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 30 Jan 2010 09:29:31 +1100 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <873a1ownn8.fsf@benfinney.id.au> Duncan Booth writes: > Here's what I see in the Ubuntu packages. Python 3 seems only to be in the > universe repositories so far. > > Dapper: Python 2.4.2 > Hardy: Python 2.5.2 > Intrepid: Python 2.5.2, 3.0~b3 (universe) > Jaunty: Python 2.6.2, 3.0.1 (universe) > Karmic: Python 2.6.4rc1, 3.1 (universe) > Lucid: Python 2.6.4, 3.1 (universe) > > WTF is rc anything doing in the main repository? It's to be expected if the release team process specifies announcement messages of the form ?RELEASED: Python X.Y.Zrc2?. I have long argued, without much traction, that the process should reserve the term ?RELEASED? for use only in reference to releases, not other things. -- \ ?First they came for the verbs, and I said nothing, for verbing | `\ weirds language. Then, they arrival for the nouns and I speech | _o__) nothing, for I no verbs.? ?Peter Ellis | Ben Finney From jgardner at jonathangardner.net Fri Jan 29 17:49:06 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Fri, 29 Jan 2010 14:49:06 -0800 (PST) Subject: For loop searching takes too long! References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> Message-ID: On Jan 28, 3:52?pm, elsa wrote: > > I've got a problem with my program, in that the code just takes too > long to run. Here's what I'm doing. If anyone has any tips, they'd be > much appreciated! > First of all, don't play with large lists. Large lists have a tendency to grow larger over time, until they get absurdly large. If you're dealing with a long list, work with it like you'd work with a file. If you need random access, stick it in some form of database, such as BDB or PostgreSQL. If you need an index, stick it in some form of DB. Eventually, large lists end up like that anyway. Don't fool yourself early on---prepare for the worst and solve the problems of tomorrow today. > So, say I have a list of lists that looks something like this (I'm > using a list of lists, rather than a list of tuples, as I need it to > be mutable): > > myList = [[786,0],[45, 1],[673,1],...................[23,46]] > > there are enough entries in the outer list, that the sum of myList[i] > [0] across all i could be as high as 10^7. > > Now, what I need to do is randomly choose one myList[i], however the > distribution of my random choice needs to be proportional to the > values of myList[i][0]. So, for this list above, I'd have a much > higher chance of choosing myList[0] than myList[1]. > Let's do some thinking here. Let's say you have a list of N items. You need to choose one, but you don't know how many there are. One algorithm that works is you start with the first item. If that's the only item, you've chosen it. If there's another item, you roll the dice. There's a 1/2 chance you drop the first item and take the sedon. If there's a third item, you roll the dice, There's a 1/3 chance you drop the first or second item, and take the third. You see the pattern? Keep the nth item 1/n of the time. In your case, you're actually pulling off N items each time, all with the same value. Your chance of keeping the next item is (number of items in the next one)/(total items seen so far, including the next one). Do the math for this. It's really simple. Let's walk through this: [[786,0],[45, 1],[673,2],...................[23,46]] Step 1: See [786,0]. Remember 786. Keep 0. Step 2: See [45,1]. Remember 786+45=831. Keep 1 45/831 of the time. Step 3: See [673,2]. Remember 831+673=1504. Keep 2 673/1504 of the time. Etc... Now, the algorithm I've just described is much less memory intensive and can deal with very long lists of numbers. However, you are calling rand() a lot. rand() is not a cheap operation. One optimization is to roll rand() once, and keep using that number over and over again. Granted, for very large numbers of items, you're going to need a very precise random number of some very small probabilities will never be chosen. This is left as an exercise for the reader. From mrm at unknown.nospam Fri Jan 29 17:50:52 2010 From: mrm at unknown.nospam (Mr.M) Date: Fri, 29 Jan 2010 22:50:52 GMT Subject: C API: module cleanup function Message-ID: Hi, I can't figure out if there is a way to run a specialized cleanup function when a module needs to be "unloaded" (i.e. just before a reload() or when i quit the interpreter). I'm thinking of something like tp_dealloc. If I call Py_InitModule3 and look at module->ob_type->tp_dealloc, I find that Python provides a default tp_dealloc for me. Now, suppose my module needs to allocate some resources at startup, I'm not sure, but I think I'd have to do it in my PyMODINIT_FUNC, right? But, if I reload() my module or if I quit the Python interpreter, I'd like to free those resources (before allocate them again, in case of a reload). Is there a way to make this work? Thank you, Luca. From gagsl-py2 at yahoo.com.ar Fri Jan 29 18:45:47 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 20:45:47 -0300 Subject: C API: module cleanup function References: Message-ID: En Fri, 29 Jan 2010 19:50:52 -0300, Mr.M escribi?: > I can't figure out if there is a way to run a specialized cleanup > function when a module needs to be "unloaded" (i.e. just before a > reload() or when i quit the interpreter). I think what you want to do isn't possible with Python 2, and it's one of the reasons the module handling was redesigned in Python 3.x; see PEP 3121. > I'm thinking of something like tp_dealloc. m_free (PyModuleDef member, in Python 3) might work for this. -- Gabriel Genellina From mrm at unknown.nospam Fri Jan 29 18:52:49 2010 From: mrm at unknown.nospam (Mr.M) Date: Fri, 29 Jan 2010 23:52:49 GMT Subject: C API: module cleanup function In-Reply-To: References: Message-ID: Gabriel Genellina ha scritto: > I think what you want to do isn't possible with Python 2, and it's one > of the reasons the module handling was redesigned in Python 3.x; see PEP > 3121. Thank you Gabriel for your help. Unlucky I can't use Python 3.x in my project, sob! Luca. From gagsl-py2 at yahoo.com.ar Fri Jan 29 19:02:14 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 21:02:14 -0300 Subject: Python-list Digest, Vol 76, Issue 303 References: <424b71ec1001290117y259a134dkeb31abf906cd32bf@mail.gmail.com> Message-ID: [top posting corrected] > From: Steve Holden > Date: Tue, 26 Jan 2010 11:54:23 -0500 > Anyway, I suspect your error might go away if you turned the first > argument of hte log.info() call into a format string such as > > log.info("refer: %s", ret) > >> try: >> log.info("start") >> log.info("refer",ret) >> log.info("end") >> except TypeError: >> log.exception("Exception raised") En Fri, 29 Jan 2010 06:17:56 -0300, siddhartha veedaluru escribi?: > Thats the output i got in output window.it is not listing the statement > which caused it. copy paste and try it on you setup. it behaves the same > way. > > I know that error is in log.info() function call.but it is not caught by > "except". which is not getting logged using log.exception Errors that occur inside the logging system are not logged themselves (users aren't usually interested on those kind of errors, but on application errors; also, if logging errors were logged too, this could create an infinite loop). To handle those errors in the logging system yourself, override Handler.handleError() -- Gabriel Genellina From gagsl-py2 at yahoo.com.ar Fri Jan 29 19:11:08 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 21:11:08 -0300 Subject: Some C-API functions clear the error indicator? References: Message-ID: En Fri, 29 Jan 2010 18:25:14 -0300, Austin Bingham escribi?: > Maybe I'm not following what you're saying. In my case, I already know > that an exception has been thrown. In the course of processing that > exception, I call another function which, for whatever reason and even > when it succeeds, clears the exception indicators. How can I address > this issue by checking function calls for failure? Maybe if you provide an actual use case we can suggest how to handle it. The code in your original post does not make any sense to me (except by showing that PyImport_ImportModule does clear the error indicator). If you already know there was an error, and you even have retrieved the error details, why do you care if the error indicator gets reset? -- Gabriel Genellina From tjreedy at udel.edu Fri Jan 29 19:24:15 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 29 Jan 2010 19:24:15 -0500 Subject: Changing tab behavior in Python Interactive mode In-Reply-To: <2c8c062f-0622-49ea-950b-80afde5371de@u41g2000yqe.googlegroups.com> References: <93ae66b7-7fe2-4542-b8d0-039f9423cdd8@n7g2000yqb.googlegroups.com> <073949c2-787d-44a4-a471-c77e605d32de@y12g2000yqh.googlegroups.com> <2c8c062f-0622-49ea-950b-80afde5371de@u41g2000yqe.googlegroups.com> Message-ID: On 1/29/2010 4:43 PM, Andrej Mitrovic wrote: > On Jan 29, 10:07 pm, Steve Holden wrote: >> Andrej Mitrovic wrote: >>> On Jan 29, 6:47 pm, Steve Holden wrote: >>>> Andrej Mitrovic wrote: >>>>> I've noticed that when running Python in interactive mode (via cmd on >>>>> windows), the first time I hit tab it will move 4 spaces to the right, >> I's nothing to do with the source: it isn't the interpreter that's >> deciding where the "tab stops" are, it's the console window the >> interpreter is running in. > I see. Well I'm nitpicking here, it's not all that important (I just > use cmd when I'm trying out small pieces of code). And I doubt there's > a tabstop setting for cmd that I could use. Nope. I checked the properties dialog (rt click upper left icon). You might want to turn on quickeditmode (as admin). Terry Jan Reedy From aahz at pythoncraft.com Fri Jan 29 21:04:31 2010 From: aahz at pythoncraft.com (Aahz) Date: 29 Jan 2010 18:04:31 -0800 Subject: image processing - inverse filtering References: <5bfefbb6-89a8-49f6-9f02-7d36dfbc0f16@c29g2000yqd.googlegroups.com> Message-ID: In article <5bfefbb6-89a8-49f6-9f02-7d36dfbc0f16 at c29g2000yqd.googlegroups.com>, suresh.amritapuri wrote: > >If I am using scipy.ndimage.gaussian_filter() for filtering an image, >how to do the inverse filtering? In general how to do this using >scipy.ndimage? http://projects.scipy.org/mailman/listinfo/scipy-user -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From benjamin.kaplan at case.edu Fri Jan 29 21:28:03 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Fri, 29 Jan 2010 21:28:03 -0500 Subject: myths about python 3 In-Reply-To: <873a1ownn8.fsf@benfinney.id.au> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <873a1ownn8.fsf@benfinney.id.au> Message-ID: On Fri, Jan 29, 2010 at 5:29 PM, Ben Finney wrote: > Duncan Booth writes: > >> Here's what I see in the Ubuntu packages. Python 3 seems only to be in the >> universe repositories so far. >> >> Dapper: Python 2.4.2 >> Hardy: Python 2.5.2 >> Intrepid: Python 2.5.2, 3.0~b3 (universe) >> Jaunty: Python 2.6.2, 3.0.1 (universe) >> Karmic: Python 2.6.4rc1, 3.1 (universe) >> Lucid: Python 2.6.4, 3.1 (universe) >> >> WTF is rc anything doing in the main repository? > > It's to be expected if the release team process specifies announcement > messages of the form ?RELEASED: Python X.Y.Zrc2?. I have long argued, > without much traction, that the process should reserve the term > ?RELEASED? for use only in reference to releases, not other things. > Don't worry, it's not just Python. Ubuntu Hardy (IIRC) had a Firefox 3 beta in the repositories until a little while after the final release. I don't know what's taking them so long to upgrade to the final 2.6.4 though. > -- > ?\ ? ? ?First they came for the verbs, and I said nothing, for verbing | > ?`\ ? ?weirds language. Then, they arrival for the nouns and I speech | > _o__) ? ? ? ? ? ? ? ? ? ? ? ? ? nothing, for I no verbs.? ?Peter Ellis | > Ben Finney > -- > http://mail.python.org/mailman/listinfo/python-list > From steve at REMOVE-THIS-cybersource.com.au Fri Jan 29 22:03:04 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 30 Jan 2010 03:03:04 GMT Subject: Keyboard input References: <8f67b6f81001290853i39bab1acs49d02ac5be948d13@mail.gmail.com> Message-ID: <03738dc7$0$1309$c3e8da3@news.astraweb.com> On Fri, 29 Jan 2010 18:38:47 -0300, Gabriel Genellina wrote: >> I'm using the raw_input method and it works fine, but I noted that I >> can't use backspace and when I accidentally press shift space (because >> I need to input uppercase letters separated by a space) it writes some >> strange characters. > > That's strange. If you're using Linux, make sure you have the readline > package installed. Yes, I have this same issue too, I've just recently installed Python 2.0 (as well as 1.5, 2.4, 2.5, 2.6, 3.0 and 3.1), and while all the other versions recognise the readline package, this doesn't: >>> x = 1 >>> # hit up arrow to get previous line ... ^[[A I'm pretty sure I built 2.0 the same way I built all the others, anyone have any hints as to what I missed? -- Steven From pavlovevidence at gmail.com Fri Jan 29 22:03:28 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Fri, 29 Jan 2010 19:03:28 -0800 (PST) Subject: Perl 6 [was Re: myths about python 3] References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <03725fae$0$1309$c3e8da3@news.astraweb.com> Message-ID: <9e3c036a-b1ba-4d12-b47b-64e8aca8e69b@33g2000vbe.googlegroups.com> On Jan 28, 9:34?pm, Steven D'Aprano wrote: > On Thu, 28 Jan 2010 21:21:05 -0800, Tim Roberts wrote: > > Perl 6, on the other hand, is still fantasyware a decade after its > > announcement. ?It is, for the most part, THE canonical example of the > > wrong way to conduct a development effort. > > Out of curiosity, and completely off-topic, why has Perl 6 gone so badly? Because Larry Wall saw the writing on the wall that his "nice little scripting language" was way out of its league in terms of what it was trying to be (i.e., a real programming language suitable for building applications), and so he bailed out and left the Perl community with no leadership. This is just my impression, but some of the efforts to create Perl 6 seem to be clinging to a notion that Perl has always been a good language and are trying to ensure that it doesn't stray very far from Perl's core principles. Problem is, Perl wasn't ever a good language so they won't succeed. IOW, as Geremy Condra said, "it's too much like Perl". Carl Banks From steve at REMOVE-THIS-cybersource.com.au Fri Jan 29 22:07:52 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 30 Jan 2010 03:07:52 GMT Subject: For loop searching takes too long! References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> Message-ID: <03738ee7$0$1309$c3e8da3@news.astraweb.com> On Fri, 29 Jan 2010 14:49:06 -0800, Jonathan Gardner wrote: > On Jan 28, 3:52?pm, elsa wrote: >> >> I've got a problem with my program, in that the code just takes too >> long to run. Here's what I'm doing. If anyone has any tips, they'd be >> much appreciated! >> >> > First of all, don't play with large lists. Large lists have a tendency > to grow larger over time, until they get absurdly large. > > If you're dealing with a long list, work with it like you'd work with a > file. If you need random access, stick it in some form of database, such > as BDB or PostgreSQL. If you need an index, stick it in some form of DB. > Eventually, large lists end up like that anyway. Don't fool yourself > early on---prepare for the worst and solve the problems of tomorrow > today. Talk about premature optimization. The OP probably has a few hundreds of thousands of items, millions at most, which is trivial on a modern PC. Your way of thinking is what has led to Firefox using a database to manage a few thousand bookmarks and cookies, which in turn leads to consistent data corruption problems. (I've been keeping note, and so far my installation of Firefox 3 has had database corruption at startup one time out of five.) -- Steven From apt.shansen at gmail.com Fri Jan 29 22:23:29 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Fri, 29 Jan 2010 19:23:29 -0800 Subject: Threading issue with SQLite In-Reply-To: <4B630ED3.5000706@googlemail.com> References: <4B630ED3.5000706@googlemail.com> Message-ID: <7a9c25c21001291923u339f7363oda95b783f4030135@mail.gmail.com> On Fri, Jan 29, 2010 at 8:37 AM, Alan Harris-Reid < aharrisreid at googlemail.com> wrote: > Hi, > > I am creating a web application (using Python 3.1 and CherryPy 3.2) where a > SQLite connection and cursor object are created using the following code > (simplified from the original): > > class MainSite: > con = sqlite.connect('MyDatabase.db') > cursor = con.cursor() > This is bad. For one thing, every thread should have its own connection at least-- you can store it in thread-local storage (see the threading module) for re-use if you want, if cherrypy does thread pooling techniques or some such (I'd be sorta surprised if it didn't). For another thing, cursors are NOT meant to be long-running interfaces. You should not leave one open beyond one concrete action (e.g., one user request, with however many SQL statements it takes to accomplish that work) even if you are re-using the same connection across multiple methods/invocations. Questions... > 1. Is there a large overhead in opening a new SQLite connection for each > thread (ie. within each method)? > You're making a web-app in Python, man. :) The overhead of a SQLite connection is not exactly significant at that point :) Though you could use thread pooling techniques with each thread having its own dedicated connection to mitigate what overhead there is. > 2. Is there any way to use the same connection for the whole class (or > should I forget that idea completely?) > You shouldn't think of "connection" and "class" as having any kind of meaningful link, it just doesn't make sense. It won't work and what you're -trying- to do doesn't actually even make sense. Besides the fact that you're worrying prematurely about overhead and trying to optimize it away, connections (and from them, cursors) to databases are things that are used for binding together a bunch of statements into transactions and maintaining 'current state' of your interaction with the database: its a a logical abstraction of "what I'm doing Now to the database", whereas the controller class is, "what can be done." 3. When a method returns to the calling method, is the connection > automatically closed (assuming the object is local, of course) or does it > have to be done explicitly using connection.close()? > If it goes out of scope, it will be closed if its a local variable (be sure you commit if you need to first!). Assuming you aren't caching/saving the connection in the thread for future calls it handles, if you have a thread pool doing your work (again, I'd be -really- surprised if CherryPy with threads didn't involve thread pools as opposed to starting up and stopping threads continually as requests come in). HTH, --S > > TIA, > Alan Harris-Reid > > > -- > http://mail.python.org/mailman/listinfo/python-list > --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From Blogtest77 at gmail.com Fri Jan 29 22:33:12 2010 From: Blogtest77 at gmail.com (Blog) Date: Sat, 30 Jan 2010 09:33:12 +0600 Subject: myths about python 3 In-Reply-To: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On 1/28/2010 2:56 AM, John Nagle wrote: > Daniel Fetchinson wrote: > 1. Python 3 is supported by major Linux distributions. > > FALSE - most distros are shipping with Python 2.4, or 2.5 at best. Where did you come up with that information? Almost all of the major distros ship with 2.6.x - CentOS, OpenSuSe, Ubuntu, Fedora. (Debian does ship with 2.5, but the next major release "sid' is due out in Q2) > Arguably, Python 3 has been rejected by the market. Instead, there's > now Python 2.6, Python 2.7, and Python 2.8. Python 3 has turned into > a debacle like Perl 6, now 10 years old. WTF? Where'd you hear about version 2.8? FRI, 2.7 is and will be THE LAST version of the 2.x series - "the" End-Of-Life for Python 2. At least py3k is a real product - unlike the perl 6 vaporware. That said, I think python 2 will be the dominant player in the market for the next couple of years. However, there's been some momentum behind Py3k recently. Unladen swallow is going to be merged with Py3k (not 2.x for which it was originally developed). I hear the django guys have made huge progress in porting the framework to Py3k (although it will be a good few months before the code is released in the wild) From Blogtest77 at gmail.com Fri Jan 29 22:50:39 2010 From: Blogtest77 at gmail.com (Blog) Date: Sat, 30 Jan 2010 09:50:39 +0600 Subject: myths about python 3 In-Reply-To: <7xzl3zq77u.fsf@ruckus.brouhaha.com> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <7xzl3zq77u.fsf@ruckus.brouhaha.com> Message-ID: On 1/28/2010 8:44 AM, Paul Rubin wrote: > Steve Holden writes: >> Kindly confine your debate to the facts and keep the snide remarks to >> yourself. Like it or not Python 3 is the future, and unladen swallow's >> recent announcement that they would target only Python 3 represented a >> ground-breaking advance for the language. > > My take on things is that doing unladen swallow really "right" will > require yet more incompatible changes; i.e., the result will either > still leave quite a bit of performance on the table, or else it won't be > compatible with the current specification of Python 3 and they'll > presumably have to call it Python 4. And if Python 4 is as good as I > believe it could possibly be, then it might get wide acceptance before > Python 3 really has all that much uptake. If I have to accept > incompatibility anyway, and Python 4 gives huge improvements while > Python 3's improvements are tiny or moderate, why not skip over Python 3? There's a prime example - it's called Windows Vista! ;) From ben+python at benfinney.id.au Fri Jan 29 23:06:29 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 30 Jan 2010 15:06:29 +1100 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <87zl3wtewq.fsf@benfinney.id.au> Blog writes: > (Debian does ship with 2.5, but the next major release "sid' is due > out in Q2) Sid is the perpetual development playground (?unstable?), never released as a suite, but a proving ground for packages to determine their fitness for going to the next level of testing. The next-to-be-released suite is Squeeze (currently ?testing?), which has Python 2.5 (the default ?python?) and Python 2.6. -- \ ?If you define cowardice as running away at the first sign of | `\ danger, screaming and tripping and begging for mercy, then yes, | _o__) Mr. Brave man, I guess I'm a coward.? ?Jack Handey | Ben Finney From steve at REMOVE-THIS-cybersource.com.au Sat Jan 30 01:06:18 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 30 Jan 2010 06:06:18 GMT Subject: Code snippet: dualmethod descriptor Message-ID: <0373b8b9$0$1309$c3e8da3@news.astraweb.com> I came up with this descriptor a few years ago, but never used it for anything. I've just found an actual use for it in my own code, so I thought I'd make it public for anyone else who might have a use for it. Use-case: if you have a class without an __init__ or __new__ method, then this may be of use to you. It enables the caller to call methods on either the class, or an instance, and the first argument passed the method will be the class or the instance respectively. This differs from classmethods, which always passes the class, and staticmethods, which don't pass either. The name "dualmethod" is my own. Here's the implementation: class dualmethod(object): """Descriptor implementing dualmethods (combination class/instance method). Returns a method which takes either an instance or a class as the first argument. When called on an instance, the instance is passed as the first argument. When called as a class, the class itself is passed instead. >>> class Example(object): ... @dualmethod ... def method(this): ... if type(this) is type: ... print("I am the class '%s'." % this.__name__) ... else: ... print("I am an instance of the class '%s'." % ... this.__class__.__name__) ... >>> Example.method() I am the class 'Example'. >>> Example().method() I am an instance of the class 'Example'. """ def __init__(self, func): self.func = func def __get__(self, obj, cls=None): if obj is None: obj = cls or type(obj) def newfunc(*args, **kwargs): return self.func(obj, *args, **kwargs) return newfunc (Tested with Python 2.4, 2.5, 2.6, 3.0 and 3.1.) My use-case is a class that has constants in class attributes, and methods which refer to those constants. Because they are constants, there is no reason to give each instance a separate reference to the attribute, hence the attribute is on the class: class Example: ATTR = "something" @dualmethod def method(this): return this.ATTR + " else" Normally I have no reason to instantiate the class: there is no __init__ or __new__ method. Normally I would use classmethod, but every now and again I want to change the behaviour by modifying ATTR, and using class- methods would force me to subclass in order to make that change. dualmethod lets me get the best of both worlds. If I'm using the default value of ATTR, I can just call the method on the class without bothering to instantiate it. But if I want to customise the behaviour of the method, rather than subclassing, I can instantiate and add an instance attribute: >>> Example.method() 'something else' >>> x = Example() >>> x.ATTR = "nothing" >>> x.method() 'nothing else' >>> Example.method() # default still works as expected 'something else' without effecting the default behaviour. I hope this is useful to you. You can use this without condition, although I would be grateful for credit. -- Steven From stefan_ml at behnel.de Sat Jan 30 01:09:26 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 30 Jan 2010 07:09:26 +0100 Subject: C API: module cleanup function In-Reply-To: References: Message-ID: <4b63cd1b$0$7618$9b4e6d93@newsspool1.arcor-online.net> Mr.M, 29.01.2010 23:50: > I can't figure out if there is a way to run a specialized cleanup > function when a module needs to be "unloaded" (i.e. just before a > reload() or when i quit the interpreter). > > I'm thinking of something like tp_dealloc. > > If I call Py_InitModule3 and look at module->ob_type->tp_dealloc, I find > that Python provides a default tp_dealloc for me. > > Now, suppose my module needs to allocate some resources at startup, I'm > not sure, but I think I'd have to do it in my PyMODINIT_FUNC, right? > > But, if I reload() my module or if I quit the Python interpreter, I'd > like to free those resources (before allocate them again, in case of a > reload). > > Is there a way to make this work? Gabriel already pointed you to the module cleanup support in Py3, which can be used to provide reload capabilities to your module. In Py2, there are at least some ways to free resources when terminating the interpreter. See the "atexit" module and the Py_AtExit() function: http://docs.python.org/library/atexit.html http://docs.python.org/c-api/sys.html Note that both have their specific limitations, though, as you can see from the docs. Also note that it might help you to take a look at Cython, a Python-to-C compiler for writing fast C extensions. It has an option for generating module-level cleanup code automatically, and generally simplifies writing binary extension modules quite a bit. Stefan From austin.bingham at gmail.com Sat Jan 30 02:10:39 2010 From: austin.bingham at gmail.com (Austin Bingham) Date: Sat, 30 Jan 2010 08:10:39 +0100 Subject: Some C-API functions clear the error indicator? In-Reply-To: References: Message-ID: The original post was, in a nutshell, the "use case"; it's very scaled down the from the real example, and not intended to "make sense". The notion on which I was (apparently incorrectly) basing my exception translation was that I could 1) get and reset references to the error indicators, 2) call other python methods that don't themselves throw, and 3) later retrieve the same "active" exceptions. I was relying on this ability to "re-fetch" exceptions insofar as the functions in my code which dealt with python exceptions all looked up the exception separately. The predicate that "a successful function won't modify the error indicators" appears to be wrong, however, and I've modified my code accordingly. Austin On Sat, Jan 30, 2010 at 1:11 AM, Gabriel Genellina wrote: > En Fri, 29 Jan 2010 18:25:14 -0300, Austin Bingham > escribi?: > >> Maybe I'm not following what you're saying. In my case, I already know >> that an exception has been thrown. In the course of processing that >> exception, I call another function which, for whatever reason and even >> when it succeeds, clears the exception indicators. How can I address >> this issue by checking function calls for failure? > > Maybe if you provide an actual use case we can suggest how to handle it. The > code in your original post does not make any sense to me (except by showing > that PyImport_ImportModule does clear the error indicator). If you already > know there was an error, and you even have retrieved the error details, why > do you care if the error indicator gets reset? > > -- > Gabriel Genellina > > -- > http://mail.python.org/mailman/listinfo/python-list > From austin.bingham at gmail.com Sat Jan 30 02:17:30 2010 From: austin.bingham at gmail.com (Austin Bingham) Date: Sat, 30 Jan 2010 08:17:30 +0100 Subject: Some C-API functions clear the error indicator? In-Reply-To: References: Message-ID: That makes a lot of sense. And if I take the approach that any Py* function might do this, it actually looks like I can simplify my code (rather than managing some list of ill-behaved functions or something.) Thanks! On Fri, Jan 29, 2010 at 3:58 PM, Duncan Booth wrote: > Austin Bingham wrote: > >> The functions that do this don't seem to indicate in their >> documentation that this will happen. So first, does anyone know why >> this is happening? Is it because of the context in which I'm making >> the calls? Is there any pattern or reason behind which functions will >> do this? Or am I just doing something wrong? >> > (Just guessing here) > I would expect that any function that executes Python code will clear the > error. > > I think that has to happen otherwise the Python code will throw an > exception whenever it gets round to checking for errors. In the past I've > found that if you fail to check for an error in C code before returning to > the interpreter you get the exception thrown a few instructions later, so > something similar would probably happen if you call other Python code from > C. > > If it is anything that executes Python then that would include any function > that creates or destroys an object with Python constructor or destructor > code. or that compares or otherwise operates on instances defined in > Python. In particular it might mean that any function that doesn't appear > to clear the error could do so in a slightly different situation. > > -- > Duncan Booth http://kupuguy.blogspot.com > -- > http://mail.python.org/mailman/listinfo/python-list > From __peter__ at web.de Sat Jan 30 04:28:19 2010 From: __peter__ at web.de (Peter Otten) Date: Sat, 30 Jan 2010 10:28:19 +0100 Subject: Function name unchanged in error message References: Message-ID: Gabriel Genellina wrote: > En Fri, 29 Jan 2010 13:09:40 -0300, Michele Simionato > escribi?: > >> On Jan 29, 2:30 pm, andrew cooke wrote: >>> Is there any way to change the name of the function in an error >>> message? In the example below I'd like the error to refer to bar(), >>> for example (the motivation is related function decorators - I'd like >>> the wrapper function to give the same name) >> >> Use the decorator module which does the right thing: >> http://pypi.python.org/pypi/decorator > > The decorator module is a very fine addition to anyone's tool set -- but > in this case it is enough to use the wraps() function from the functools > standard module. I don't know about the decorator module, but functools.wraps() doesn't affect the error message: >>> from functools import wraps >>> def f(): pass ... >>> @wraps(f) ... def g(): pass ... >>> g.__name__ 'f' >>> try: g(42) ... except TypeError as e: ... print e ... g() takes no arguments (1 given) Peter From davea at ieee.org Sat Jan 30 05:07:17 2010 From: davea at ieee.org (Dave Angel) Date: Sat, 30 Jan 2010 05:07:17 -0500 Subject: Sleep timer but still responsive? In-Reply-To: References: <8c76dbf2-8853-48cc-9ca9-37929936f73a@b35g2000vbc.googlegroups.com> Message-ID: <4B6404D5.4070809@ieee.org> JohnnyFive wrote: > On Jan 29, 9:33 am, Andreas Tawn wrote: > >>> On Jan 28, 4:55 pm, "Gabriel Genellina" >>> wrote: >>> >>>> Please provide more details. What do you want your program to do >>>> >>> while >>> >>>> sleeping? What kind of actions do you want a response to? >>>> Do you have a GUI? A curses-based interfase? >>>> >>>> -- >>>> Gabriel Genellina >>>> >>> My app is purely console based. I just don't want the console to lock >>> up (on Windows using time.sleep(x) causes the console to become >>> unresponsive until the timer is done), and I want people to be able to >>> CTRL+C to stop the script if need be (which can't be done if it's >>> unresponsive!). >>> >>> Thanks. >>> >> How about this? Responds to ctrl+c, but still sleeps. >> >> import time >> >> def responsiveSleep(n): >> while n > 0: >> time.sleep(1) >> n -= >> >> Cheers, >> >> Drea >> > > Thanks for the ideas! Maybe it's just my computer, but using your > solution still causes the prompt to become unresponsive during the > sleeps. > > I am using 2.6.4 btw. It's not a major deal though, I just thought > there had to be a way to do this fairly easily. > > You responded to my message off-list, so I have to paste it here. """ Dave, Thanks for the response. Here's my scenario. I have a program that checks an ftp site every hour for updated files, downloads them, and then sleeps again for another hour when done. It's 100% console. When I run the app, no matter how I try, I can't get the window to act normal during the sleep cycle. I've tried putting the sleep in another thread and have the main thread.join(), but it all has the same behavior: the console window behaves like it is crashing. Using the small sleep() increments also causes the same thing, the becomes undraggable for periods of time, and again acts like the program has crashed (even though it's just asleep). I am running a version of it right now, and i've managed to minimize it, but it won't maximize so I can ctrl+c it. All it's doing is exactly what Andreas recommended. This is not user-friendly behavior! I've also tried having the main thread sit at a raw_input, and have another thread have the timer, but there was odd behavior when I wanted the program to continue what it was doing, even if the user hadn't pressed "enter" to get passed the command prompt. """ You probably need to tell us your complete environment. I know you're running 2.6.4, but don't know which version of Windows, so I'll guess XP. You're running in a cmd.exe console. There must be something else going on in your system, since a console does not become unresponsive during a sleep. The python program is sleeping, but the console is very much alive; it's a separate process. So dragging, minimizing, restoring and maximizing is unaffected. Try the following simple script from a cmd console: import time print print print "going to sleep" print "Use Ctrl-C to get my attention, and end the program", time.sleep(30) print "done" On XP SP3, running Python 2.6.4, this ignores regular keystrokes, but responds nicely to control C. And it can be dragged around, resized, minimized, etc. with no problem. If you get different behavior, tell us more precisely how your environment differs from my guesses. Once we've solved Ctrl-C, drag the console, minimize the console, then maybe you're going to request "respond to other keystrokes". It can all be done, but only with more careful wording than "behaves like its crashing." Regards, DaveA From mattsteel at hotmail.it Sat Jan 30 05:25:01 2010 From: mattsteel at hotmail.it (Mattsteel) Date: Sat, 30 Jan 2010 02:25:01 -0800 (PST) Subject: Funny behaviour with __future__ and doctest between 2.6 and 3.1 References: <3a02ac42-337b-4778-99e0-1309e012163d@g29g2000yqe.googlegroups.com> Message-ID: <20136170-8eda-41c2-a0ff-7a701e6c006b@z26g2000yqm.googlegroups.com> On 29 Gen, 17:30, Peter Otten <__pete... at web.de> wrote: > I think you can work around the problem. The following should pass in Python > 2.6 and 3.1: > > '''>>> concat('hello','world') == 'hello world' > True > ''' I see. Thank for the concern. M. From solipsis at pitrou.net Sat Jan 30 08:11:24 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 30 Jan 2010 13:11:24 +0000 (UTC) Subject: Some C-API functions clear the error indicator? References: Message-ID: Le Fri, 29 Jan 2010 22:25:14 +0100, Austin Bingham a ?crit?: > Maybe I'm not following what you're saying. In my case, I already know > that an exception has been thrown. In the course of processing that > exception, I call another function which, for whatever reason and even > when it succeeds, clears the exception indicators. If you know an error occurred and need to retain it somewhere, just use PyErr_Fetch() and PyErr_Restore(). Regards Antoine. From mrm at unknown.nospam Sat Jan 30 08:24:51 2010 From: mrm at unknown.nospam (Mr.M) Date: Sat, 30 Jan 2010 13:24:51 GMT Subject: C API: module cleanup function In-Reply-To: <4b63cd1b$0$7618$9b4e6d93@newsspool1.arcor-online.net> References: <4b63cd1b$0$7618$9b4e6d93@newsspool1.arcor-online.net> Message-ID: Stefan Behnel ha scritto: > Gabriel already pointed you to the module cleanup support in Py3, which can > be used to provide reload capabilities to your module. > > In Py2, there are at least some ways to free resources when terminating the > interpreter. See the "atexit" module and the Py_AtExit() function: > > http://docs.python.org/library/atexit.html > http://docs.python.org/c-api/sys.html > > Note that both have their specific limitations, though, as you can see from > the docs. > > Also note that it might help you to take a look at Cython, a Python-to-C > compiler for writing fast C extensions. It has an option for generating > module-level cleanup code automatically, and generally simplifies writing > binary extension modules quite a bit. > > Stefan Thank you very much Stefan for your reply, I'll study the sources you have pointed me to. Could I allocate my resources in a "static" object (without publishing the type of that object so that I can't instantiate another one) linked to my module? This way, when I stop the interpreter, the object will be destroyed calling its destructor. There's something I'm missing? Thank you, Luca. From keith at nekotaku.com Sat Jan 30 09:21:01 2010 From: keith at nekotaku.com (KB) Date: Sat, 30 Jan 2010 06:21:01 -0800 (PST) Subject: Utility to screenscrape sites using javascript ? Message-ID: <4c4298c0-b93d-4691-bceb-e982373f87e3@f17g2000prh.googlegroups.com> Hi there, I have a service I subscribe to that uses javascript to stream news. Ideally I would like to use python to parse the information for me. Note there is an option to take a static snapshot of the current stream but that is still done via Javascript. (I can reference the snapshot with a unique URL though, so I can pass that to a parser as long as it can "resolve" the javascript and get at the content) I had a quick look at Windmill but it doesn't appear to be what I am looking for. Does anyone else have any experience in screenscraping sites that utilise javascript? Can you share how you did it and perhaps some sample code if possible? Thanks a bunch! From 3lvss0809 at gmail.com Sat Jan 30 09:37:48 2010 From: 3lvss0809 at gmail.com (peskar.miha@hotmail.com) Date: Sat, 30 Jan 2010 06:37:48 -0800 (PST) Subject: Microsoft Office Word and Python (Win XP) References: Message-ID: <85ee4831-3b9c-4096-b32a-387e59d6e7c1@r6g2000yqn.googlegroups.com> 21 days has passed and still noone is willing to help :-( From stefan_ml at behnel.de Sat Jan 30 09:44:56 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 30 Jan 2010 15:44:56 +0100 Subject: C API: module cleanup function In-Reply-To: References: <4b63cd1b$0$7618$9b4e6d93@newsspool1.arcor-online.net> Message-ID: <4b6445e9$0$7621$9b4e6d93@newsspool1.arcor-online.net> Mr.M, 30.01.2010 14:24: > Could I allocate my resources in a "static" object linked to my module? Sure, but you will have to let CPython know about it so that it can see that the reference is only held by the module that it is cleaning up. Otherwise it can't collect the reference. This works easily in Py3 but not in Py2. In Cython, you would just write cdef ResourceKeeper resource_keeper = ResourceKeeper() at the module level and let Cython generate the cleanup code for it. In C, you have to implement an atexit function and decref the resource object either there or in the module cleanup function of Py3. > (without publishing > the type of that object so that I can't instantiate another one) Note that there is the type() builtin function which returns the type given an instance. So you can't hide the type. Stefan From alfps at start.no Sat Jan 30 09:54:21 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sat, 30 Jan 2010 15:54:21 +0100 Subject: Microsoft Office Word and Python (Win XP) In-Reply-To: <85ee4831-3b9c-4096-b32a-387e59d6e7c1@r6g2000yqn.googlegroups.com> References: <85ee4831-3b9c-4096-b32a-387e59d6e7c1@r6g2000yqn.googlegroups.com> Message-ID: * peskar.miha at hotmail.com: > 21 days has passed and still noone is willing to help :-( Did you see the reply from Marco Nawin? If you don't see that reply, dated (a bit less than) 2 hours after your original posting on the 9th, I can repost it here. If you have any follow-up questions just post them. Cheers & hth., - Alf From steve at holdenweb.com Sat Jan 30 10:01:33 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 30 Jan 2010 10:01:33 -0500 Subject: Microsoft Office Word and Python (Win XP) In-Reply-To: <85ee4831-3b9c-4096-b32a-387e59d6e7c1@r6g2000yqn.googlegroups.com> References: <85ee4831-3b9c-4096-b32a-387e59d6e7c1@r6g2000yqn.googlegroups.com> Message-ID: peskar.miha at hotmail.com wrote: > 21 days has passed and still noone is willing to help :-( Y'know, the Internet isn't a magic lantern. Perhaps you need to ask your question differently, or provide more information. Maybe there's something *you* could do to help you get closer to the answer to *your* question. Read the question at the time, don't remember having much to contribute right then. But you lucked out today: it's Saturday morning and I am at home in my family room, and it's snowing outside. Hey, at least you do know already that people will respond to your questions. Look for a (second-hand?) copy of Andy Robinson's and Mark Hammond's "Python Programming on Win32", which taught me a lot about COM I have thankfully since been able to forget. Go to Microsoft channels for details of the Office object models, and good luck with that (the quality and quantity of that used to be something of a movable feast, though I imagine things have improved since). You might also ask yourself whether IronPython and the .NET interfaces aren't a better and more modern way of controlling Microsoft products. The .NET documentation is pretty well organized and complete, I believe. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From mrm at unknown.nospam Sat Jan 30 10:02:25 2010 From: mrm at unknown.nospam (Mr.M) Date: Sat, 30 Jan 2010 15:02:25 GMT Subject: C API: module cleanup function In-Reply-To: <4b6445e9$0$7621$9b4e6d93@newsspool1.arcor-online.net> References: <4b63cd1b$0$7618$9b4e6d93@newsspool1.arcor-online.net> <4b6445e9$0$7621$9b4e6d93@newsspool1.arcor-online.net> Message-ID: <5SX8n.18688$Ua.307@twister2.libero.it> Stefan Behnel ha scritto: > Note that there is the type() builtin function which returns the type given > an instance. So you can't hide the type. Argh! Yes, you are right. So I'd better have a look to Cython, right? L- From fsenzeru at gmail.com Sat Jan 30 10:32:41 2010 From: fsenzeru at gmail.com (Dan Brown) Date: Sat, 30 Jan 2010 07:32:41 -0800 (PST) Subject: list.extend([]) Question Message-ID: <088e7a24-b0d0-4d43-bee7-193e5eaefe57@b7g2000pro.googlegroups.com> Why does extending a list with the empty list result in None? It seems very counterintuitive to me, at least --- I expected ['a'].extend ([]) to result in ['a'], not None. From alfps at start.no Sat Jan 30 10:38:07 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sat, 30 Jan 2010 16:38:07 +0100 Subject: list.extend([]) Question In-Reply-To: <088e7a24-b0d0-4d43-bee7-193e5eaefe57@b7g2000pro.googlegroups.com> References: <088e7a24-b0d0-4d43-bee7-193e5eaefe57@b7g2000pro.googlegroups.com> Message-ID: * Dan Brown: > Why does extending a list with the empty list result in None? It > seems very counterintuitive to me, at least --- I expected ['a'].extend > ([]) to result in ['a'], not None. It does. 'extend' is an operation that /modifies/ the array. It just returns None as its expression result, in the same way as e.g. the Python 3.x 'print' (another pure "doer" operation). >>> L = ['a'] >>> L ['a'] >>> L2 = L.extend( [] ) >>> L2 >>> L2 is None True >>> L ['a'] >>> _ Cheers & hth., - Alf From andreengels at gmail.com Sat Jan 30 10:41:45 2010 From: andreengels at gmail.com (Andre Engels) Date: Sat, 30 Jan 2010 16:41:45 +0100 Subject: list.extend([]) Question In-Reply-To: <088e7a24-b0d0-4d43-bee7-193e5eaefe57@b7g2000pro.googlegroups.com> References: <088e7a24-b0d0-4d43-bee7-193e5eaefe57@b7g2000pro.googlegroups.com> Message-ID: <6faf39c91001300741u7f3973at30ac32e8f421cbf@mail.gmail.com> On Sat, Jan 30, 2010 at 4:32 PM, Dan Brown wrote: > Why does extending a list with the empty list result in None? ?It > seems very counterintuitive to me, at least --- I expected ['a'].extend > ([]) to result in ['a'], not None. Extend is a method of the list. The list itself is changed, it does not return itself: >>> A = [1,2] >>> B = [3,4] >>> C = A.extend(B) >>> C >>> C is None True >>> A [1, 2, 3, 4] Thus, nothing special about extend([]), this is the general behaviour of extend -- Andr? Engels, andreengels at gmail.com From fsenzeru at gmail.com Sat Jan 30 10:43:37 2010 From: fsenzeru at gmail.com (Dan Brown) Date: Sat, 30 Jan 2010 07:43:37 -0800 (PST) Subject: list.extend([]) Question References: <088e7a24-b0d0-4d43-bee7-193e5eaefe57@b7g2000pro.googlegroups.com> Message-ID: <1dbe4fc8-c712-4524-87b0-41ec19c9117e@v37g2000prh.googlegroups.com> On Jan 30, 8:38?am, "Alf P. Steinbach" wrote: > It does. > > 'extend' is an operation that /modifies/ the array. > > It just returns None as its expression result, in the same way as e.g. the > Python 3.x 'print' (another pure "doer" operation). > > ? ?>>> L = ['a'] > ? ?>>> L > ? ?['a'] > ? ?>>> L2 = L.extend( [] ) > ? ?>>> L2 > ? ?>>> L2 is None > ? ?True > ? ?>>> L > ? ?['a'] > ? ?>>> _ > > Cheers & hth., > > - Alf Aha. Well, I feel a bit silly for not thinking to try it that way. Thanks! From steve at holdenweb.com Sat Jan 30 10:45:45 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 30 Jan 2010 10:45:45 -0500 Subject: list.extend([]) Question In-Reply-To: <088e7a24-b0d0-4d43-bee7-193e5eaefe57@b7g2000pro.googlegroups.com> References: <088e7a24-b0d0-4d43-bee7-193e5eaefe57@b7g2000pro.googlegroups.com> Message-ID: Dan Brown wrote: > Why does extending a list with the empty list result in None? It > seems very counterintuitive to me, at least --- I expected ['a'].extend > ([]) to result in ['a'], not None. How very inconvenient of Python! What it actually does is create an anonymous list containing only the element 'a', and leave it unchanged by extending it with an empty list. Since there is no longer any reference to the list it has become garbage. Contrast that with: >>> lst = ['a'] >>> lst.extend([]) >>> lst ['a'] >>> lst.append([]) >>> lst ['a', []] >>> lst.extend(['1']) >>> lst ['a', [], '1'] >>> As you can see by the absence of output, both the .extend() and .append() list methods return None. They mutate the list instance upon which they are called. In your example you were expecting the methods to return the mutated list. They don't. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From nobody at nowhere.com Sat Jan 30 11:01:19 2010 From: nobody at nowhere.com (Nobody) Date: Sat, 30 Jan 2010 16:01:19 +0000 Subject: Default path for files References: Message-ID: On Sun, 24 Jan 2010 15:08:15 +0000, Rotwang wrote: > Hi all, can anybody tell me whether there's a way to change the default > location for files to be opened by open()? I'd like to be able to create > files somewhere other than my Python folder without having to write the > full path in the filename every time. Sorry if this is a stupid question, > I don't know much about programming. If you pass a relative pathname to open() (or any other function which expects a filename), it will be interpreted relative to the current directory. Given that the current directory always seems to be the Python directory, and you refer to it as a "folder", I'm guessing that you're running Python on Windows via a shortcut in the Start Menu or on the desktop. In which case, the ideal solution is probably to open the Properties dialog for the shortcut and change the "Start in" field to your "My Documents" directory (or some subdirectory of it). Python itself won't care which directory it starts in. From nobody at nowhere.com Sat Jan 30 11:29:02 2010 From: nobody at nowhere.com (Nobody) Date: Sat, 30 Jan 2010 16:29:02 +0000 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On Wed, 27 Jan 2010 12:56:10 -0800, John Nagle wrote: > Arguably, Python 3 has been rejected by the market. Arguably, Python 3 has not yet been accepted by the market. Part of it is down to a catch-22: applications won't use Python 3 if the libraries on which they depend don't support it, and support for Python 3 by libraries will be influenced by the perceived demand. OTOH, it's safe to assume that there will remain areas where Python 2 is preferred. Primarily Unix scripting, where most data is byte strings with the encoding either unknown or irrelevant. That alone will ensure that Python 2 is alive and well even as Python 4 is released. Even if python.org doesn't support Python 2, it's a safe bet that e.g. ActiveState will. From nobody at nowhere.com Sat Jan 30 11:43:18 2010 From: nobody at nowhere.com (Nobody) Date: Sat, 30 Jan 2010 16:43:18 +0000 Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> Message-ID: On Wed, 27 Jan 2010 15:29:05 -0800, Jonathan Gardner wrote: > There's a lot of "magic" in Ruby as well. For instance, function calls are > made without parentheses. That's also true for most functional languages, e.g. Haskell and ML, as well as e.g. Tcl and most shells. Why require "f(x)" or "(f x)" if "f x" will suffice? > Python is much, much cleaner. I don't know how anyone can honestly say > Ruby is cleaner than Python. I'm not familiar with Ruby, but most languages are cleaner than Python once you get beyond the "10-minute introduction" stage. From kw at codebykevin.com Sat Jan 30 11:45:17 2010 From: kw at codebykevin.com (Kevin Walzer) Date: Sat, 30 Jan 2010 11:45:17 -0500 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On 1/30/10 11:29 AM, Nobody wrote: > > Arguably, Python 3 has not yet been accepted by the market. > > Part of it is down to a catch-22: applications won't use Python 3 if the > libraries on which they depend don't support it, and support for Python 3 > by libraries will be influenced by the perceived demand. This is part of my reason for not yet moving to Python 3--several libraries that I will need do not currently support Python 3. -- Kevin Walzer Code by Kevin http://www.codebykevin.com From nobody at nowhere.com Sat Jan 30 11:54:52 2010 From: nobody at nowhere.com (Nobody) Date: Sat, 30 Jan 2010 16:54:52 +0000 Subject: Keyboard input References: Message-ID: On Fri, 29 Jan 2010 16:53:43 +0000, Mr.SpOOn wrote: > I'm using the raw_input method and it works fine, but I noted that I can't > use backspace and when I accidentally press shift space (because I need to > input uppercase letters separated by a space) it writes some strange > characters. > > So, is there another way to get keyboard input? sys.stdin.readline() From tanix at mongo.net Sat Jan 30 11:58:34 2010 From: tanix at mongo.net (tanix) Date: Sat, 30 Jan 2010 16:58:34 GMT Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> Message-ID: In article , Nobody wrote: >On Wed, 27 Jan 2010 15:29:05 -0800, Jonathan Gardner wrote: > >> There's a lot of "magic" in Ruby as well. For instance, function calls are >> made without parentheses. > >That's also true for most functional languages, e.g. Haskell and ML, as >well as e.g. Tcl and most shells. Why require "f(x)" or "(f x)" if "f x" >will suffice? > >> Python is much, much cleaner. I don't know how anyone can honestly say >> Ruby is cleaner than Python. > >I'm not familiar with Ruby, but most languages are cleaner than Python >once you get beyond the "10-minute introduction" stage. I'd have to agree. The only ones that beat Python in that department are Javascript and PHP. Plus CSS and HTML if you can call those languages. The very idea of using a number of blanks to identify your block level is as insane as it gets. First of all, combinations of blanks and tabs, depending on how your ide is setup to expand tabs, may get you bugs, you'd never imagine in your wild dreams. Braces is the most reliable way to identify blocks. Sane compilers ignore blanks altogether. -- Programmer's Goldmine collections: http://preciseinfo.org Tens of thousands of code examples and expert discussions on C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, organized by major topics of language, tools, methods, techniques. All collections are fully searchable down to specific chapter. From nobody at nowhere.com Sat Jan 30 12:00:43 2010 From: nobody at nowhere.com (Nobody) Date: Sat, 30 Jan 2010 17:00:43 +0000 Subject: Utility to screenscrape sites using javascript ? References: <4c4298c0-b93d-4691-bceb-e982373f87e3@f17g2000prh.googlegroups.com> Message-ID: On Sat, 30 Jan 2010 06:21:01 -0800, KB wrote: > I have a service I subscribe to that uses javascript to stream news. > Ideally I would like to use python to parse the information for me. Note > there is an option to take a static snapshot of the current stream but > that is still done via Javascript. (I can reference the snapshot with a > unique URL though, so I can pass that to a parser as long as it can > "resolve" the javascript and get at the content) > > I had a quick look at Windmill but it doesn't appear to be what I am > looking for. Does anyone else have any experience in screenscraping sites > that utilise javascript? Can you share how you did it and perhaps some > sample code if possible? There's a Python interface to SpiderMonkey (Mozilla's JavaScript interpreter): http://pypi.python.org/pypi/python-spidermonkey From e_d_k at yahoo.com Sat Jan 30 12:06:26 2010 From: e_d_k at yahoo.com (Ed Keith) Date: Sat, 30 Jan 2010 09:06:26 -0800 (PST) Subject: Python and Ruby In-Reply-To: Message-ID: <162882.48205.qm@web58906.mail.re1.yahoo.com> --- On Sat, 1/30/10, Nobody wrote: > From: Nobody > > Python is much, much cleaner. I don't know how anyone > can honestly say > > Ruby is cleaner than Python. > > I'm not familiar with Ruby, but most languages are cleaner > than Python > once you get beyond the "10-minute introduction" stage. You need to be clear about what you mean by "clean". Is Python scoping "clean"? I suspect lots of people would argue either side. -EdK Ed Keith e_d_k at yahoo.com Blog: edkeith.blogspot.com From lists at cheimes.de Sat Jan 30 12:47:17 2010 From: lists at cheimes.de (Christian Heimes) Date: Sat, 30 Jan 2010 18:47:17 +0100 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <4B6470A5.5080308@cheimes.de> Blog wrote: > WTF? Where'd you hear about version 2.8? FRI, 2.7 is and will be THE > LAST version of the 2.x series - "the" End-Of-Life for Python 2 Where do you get your information from? Your answer is the first that clearly marks the end of lifetime for the 2.x series. I didn't know that and I'm a Python core dev as well as a PSF member ... *scnr* Christian From Blogtest77 at gmail.com Sat Jan 30 13:08:49 2010 From: Blogtest77 at gmail.com (Blog) Date: Sun, 31 Jan 2010 00:08:49 +0600 Subject: myths about python 3 In-Reply-To: <4B6470A5.5080308@cheimes.de> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <4B6470A5.5080308@cheimes.de> Message-ID: On 1/30/2010 11:47 PM, Christian Heimes wrote: > Blog wrote: >> WTF? Where'd you hear about version 2.8? FRI, 2.7 is and will be THE >> LAST version of the 2.x series - "the" End-Of-Life for Python 2 > > Where do you get your information from? Your answer is the first that > clearly marks the end of lifetime for the 2.x series. I didn't know that > and I'm a Python core dev as well as a PSF member ... *scnr* > > Christian Well, here's what the Python 2.7 alpha 2 (January 9th, 2010) release note says: "Python 2.7 is scheduled to be the last major version in the 2.x series before it moves into 5 years of bugfix-only mode. This release contains many of the features that were first released in Python 3.1..." Ref: http://www.python.org/download/releases/2.7/ Here are some more resources: "Python 2 nears end of life" "Python 2.7, expected to be the last major version of the 2.x series of the dynamic language, was released as a second alpha earlier month by the Python Software Foundation, with the final release set for June. When 2.7 is released, the 2.x line will move into five years of a bug fix-only mode." http://www.computerworlduk.com/technology/development/software/news/index.cfm?newsid=18331 "2.7's the end of the line. There was some discussion around the release of 2.6 as to how far the 2.x series should go, and the conclusion which came out of it was that 2.7 is it. Beyond this, the world is 3.x (which, given the time it takes OS distributors to catch up -- most are still on 2.5 -- is about right with the projected time frame for most projects to port)." http://www.reddit.com/r/Python/comments/aoloc/python_27_alpha_2_has_been_released/ "Let?s also account for the fact that, as of this writing, Python 2.7 (scheduled for next year) is *intended* to be the End of Life release of the Python 2.x syntax ? Python 3 being the next evolutionary step." http://jessenoller.com/2009/12/04/pythons-moratorium-lets-think-about-this/ Of course, this isn't written in stone - there may very well be another version after 2.7. However, at this instant, it does look like 2.7 will be the veritable last version. Best regards. From Blogtest77 at gmail.com Sat Jan 30 13:09:52 2010 From: Blogtest77 at gmail.com (Blog) Date: Sun, 31 Jan 2010 00:09:52 +0600 Subject: myths about python 3 In-Reply-To: <87zl3wtewq.fsf@benfinney.id.au> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <87zl3wtewq.fsf@benfinney.id.au> Message-ID: On 1/30/2010 10:06 AM, Ben Finney wrote: > Blog writes: > >> (Debian does ship with 2.5, but the next major release "sid' is due >> out in Q2) > > Sid is the perpetual development playground (?unstable?), never released > as a suite, but a proving ground for packages to determine their fitness > for going to the next level of testing. > > The next-to-be-released suite is Squeeze (currently ?testing?), which > has Python 2.5 (the default ?python?) and Python 2.6. > Oops! My bad! I actually meant Squeeze. Thanks for catching the "typo". From nagle at animats.com Sat Jan 30 13:23:39 2010 From: nagle at animats.com (John Nagle) Date: Sat, 30 Jan 2010 10:23:39 -0800 Subject: Threading issue with SQLite In-Reply-To: <09ca16b8-4e10-40f5-a69d-29fcdf6c2593@b9g2000pri.googlegroups.com> References: <09ca16b8-4e10-40f5-a69d-29fcdf6c2593@b9g2000pri.googlegroups.com> Message-ID: <4b647586$0$1636$742ec2ed@news.sonic.net> Jonathan Gardner wrote: > On Jan 29, 8:37 am, Alan Harris-Reid > wrote: >> Questions... >> 1. Is there a large overhead in opening a new SQLite connection for >> each thread (ie. within each method)? > Suggestion: Use something like SQLAlchemy to manage you DB > interactions. One day, you'll move away from SQLite, and you'll be > glad you were programming at a higher level. SQLAlchemy also does > things like connection pooling and such for you. Generally, if you get to the point where you're concerned about concurrency performance issues with SQLite, it's time to upgrade to a more serious database. Especially if you're doing much updating. SQLite can do multiple SELECT operations in parallel, but the entire database is locked for all operations that write. John Nagle From uraniumore238 at gmail.com Sat Jan 30 13:33:42 2010 From: uraniumore238 at gmail.com (uche) Date: Sat, 30 Jan 2010 10:33:42 -0800 (PST) Subject: unencountered error in FFT python Message-ID: Hi, I have the following FFT python code and it doesn't seem to compile correctly. To run it, please create a file called output.csv with 1,2,3,4,5,6,7,8. simply run the main function. I get an error such as the following: x[a], x[b] = x[(a)] + W[(n % N)] * x[(b)], x[(a)] - W[(n % N)] * x [(b)] TypeError: list indices must be integers, not float How can I fixe this problem ? I have tried puttin int on all of the variables, but I don't think that is the intension of the person who wore the original code. #!/usr/bin/python """ FFT using Cooley-Tukey algorithm where N = 2^n. The choice of e^{-j2\pi/N} or e^{j2\pi/N} is made by 'sign=-1' or 'sign=1' respectively. Since I prefer Engineering convention, I chose 'sign=-1' as the default. FFT is performed as follows: 1. bit-reverse the array. 2. partition the data into group of m = 2, 4, 8, ..., N data points. 3. for each group with m data points, 1. divide into upper half (section A) and lower half (section B), each containing m/2 data points. 2. divide unit circle by m. 3. apply "butterfly" operation |a| = |1 w||a| or a, b = a+w*b, a-w*b |b| |1 -w||b| where a and b are data points of section A and B starting from the top of each section, and w is data points along the unit circle starting from z = 1+0j. FFT ends after applying "butterfly" operation on the entire data array as whole, when m = N. """ def main(): array = [] array2 = [] import os import time os.path.exists("input.csv") fin=open('input.csv', 'r') for line in fin: #read the line from the file array=line.split(',') for a in range(len(array)): #convert into integers array2.append((array[a])) Ti=time.time() print (fft(array2)) Tf=time.time() print (("It took"),Tf-Ti,("seconds to calculate an FFT of size"),len(array)) #end timer """ Find 2^n that is equal to or greater than. """ def nextpow2(i): n = 2 while n < i: n = n * 2 return n """ Return bit-reversed list, whose length is assumed to be 2^n: eg. 0111 <--> 1110 for N=2^4. """ def bitrev(x): N, x = len(x), x[:] if N != nextpow2(N): raise ValueError ('N is not power of 2') for i in range(N): k, b, a = 0, N>>1, 1 while b >= a: if b & i: k = k | a if a & i: k = k | b b, a = b>>1, a<<1 if i < k: x[i], x[k] = (x[k],x[i]) return x def fft(x, sign=-1): from cmath import pi, exp N, W = len(x), [] for i in range(N): # exp(-j...) is default W.append(exp(sign * 2j * pi * i / N)) x = bitrev(x) m = 2 while m <= N: for s in range(0,N,m): for i in range (int(m/2)): n = i * N / m a, b = s + i, s + i + m/2 x[a], x[b] = x[(a)] + W[(n % N)] * x[(b)], x[(a)] - W [(n % N)] * x[(b)] m = m * 2 return x From martin at v.loewis.de Sat Jan 30 13:39:24 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Sat, 30 Jan 2010 19:39:24 +0100 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <4B647CDC.2070409@v.loewis.de> Christian Heimes wrote: > Blog wrote: >> WTF? Where'd you hear about version 2.8? FRI, 2.7 is and will be THE >> LAST version of the 2.x series - "the" End-Of-Life for Python 2 > > Where do you get your information from? It was discussed repeatedly on python-dev, last time when the release announcements for the 2.7 alpha releases were discussed. See http://www.python.org/download/releases/2.7/ Regards, Martin From stefan_ml at behnel.de Sat Jan 30 13:52:57 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 30 Jan 2010 19:52:57 +0100 Subject: unencountered error in FFT python In-Reply-To: References: Message-ID: <4b64800a$0$6735$9b4e6d93@newsspool2.arcor-online.net> uche, 30.01.2010 19:33: > I have the following FFT python code You didn't seriously implement an FFT in plain Python code, did you? FFTs are amongst the first thing that come to my mind when I try to imagine what I'd use NumPy for. (and I *never* used it!) Stefan From aahz at pythoncraft.com Sat Jan 30 13:59:47 2010 From: aahz at pythoncraft.com (Aahz) Date: 30 Jan 2010 10:59:47 -0800 Subject: Default path for files References: Message-ID: In article , Rotwang wrote: > >Hi all, can anybody tell me whether there's a way to change the default >location for files to be opened by open()? I'd like to be able to create >files somewhere other than my Python folder without having to write the >full path in the filename every time. Sorry if this is a stupid >question, I don't know much about programming. from os.path import join BASE = '/path/to/root' f = open(join(BASE, filename)) Trust me, you'll be much happier with this. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From groups_ads12 at yahoo.com Sat Jan 30 14:05:25 2010 From: groups_ads12 at yahoo.com (groups_ads12) Date: Sat, 30 Jan 2010 19:05:25 -0000 Subject: www.assembly.learn.net.in Message-ID: http://www.assembly.learn.net.in/ assembly assembly of god assemble assembly jobs assembly language assembly work assembler electronic assembly pcb assembly assemblies crib assembly cable assembly christian assembly contract assembly furniture assembly hub assembly the assembly assembly code assembly instructions assembly language programming assembly programming document assembly smt assembly x86 assembly axle assembly computer assembly electronics assembly urban assembly wheel bearing assembly assembly magazine assembly services bearing assembly mechanical assembly arm assembly assembly required assembly table assembly tools car assembly medical assembly mips assembly wheel assembly ikea assembly assembly program assembly service assembly worker product assembly assembly instruction assembly language step by step assembly systems assembly testing c# assembly central assembly distributor assembly learn assembly language pc assembly american assembly assembly solutions component assembly hand assembly windows assembly assembly area assembly language program assembly test assembly to c parts assembly 8051 assembler free assembly grand assembly industrial assembly office furniture assembly rear assembly system assembly assembly technologies front assembly 8051 assembly assembly example flat pack furniture assembly hardware assembly full assembly x86 assembly programming 8086 assembly a happy assembly -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Sat Jan 30 14:08:03 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 30 Jan 2010 19:08:03 +0000 Subject: unencountered error in FFT python In-Reply-To: References: Message-ID: <4B648393.4000204@mrabarnett.plus.com> uche wrote: > Hi, > I have the following FFT python code and it doesn't seem to compile > correctly. To run it, please create a file called output.csv with > 1,2,3,4,5,6,7,8. simply run the main function. I get an error such as > the following: > > x[a], x[b] = x[(a)] + W[(n % N)] * x[(b)], x[(a)] - W[(n % N)] * x > [(b)] > TypeError: list indices must be integers, not float > > How can I fixe this problem ? I have tried puttin int on all of the > variables, but I don't think that is the intension of the person who > wore the original code. > Which version of Python are you using? In Python 3 the division operator '/' returns a float, whereas in Python 2 it returns an int if both operands are int. In Python 3 the int division operator is '//', which is also accepted in recent versions of Python 2. [snip] > > os.path.exists("input.csv") > > fin=open('input.csv', 'r') > > for line in fin: #read the line from the file > > array=line.split(',') > These lines should be indented more: > for a in range(len(array)): #convert into integers > > array2.append((array[a])) > array2.append(int(array[a])) [snip] From steve at REMOVE-THIS-cybersource.com.au Sat Jan 30 14:16:04 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 30 Jan 2010 19:16:04 GMT Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> Message-ID: <037471d0$0$1309$c3e8da3@news.astraweb.com> On Sat, 30 Jan 2010 16:58:34 +0000, tanix wrote: > In article , Nobody > wrote: >>On Wed, 27 Jan 2010 15:29:05 -0800, Jonathan Gardner wrote: >> >>> There's a lot of "magic" in Ruby as well. For instance, function calls >>> are made without parentheses. >> >>That's also true for most functional languages, e.g. Haskell and ML, as >>well as e.g. Tcl and most shells. Why require "f(x)" or "(f x)" if "f x" >>will suffice? >> >>> Python is much, much cleaner. I don't know how anyone can honestly say >>> Ruby is cleaner than Python. >> >>I'm not familiar with Ruby, but most languages are cleaner than Python >>once you get beyond the "10-minute introduction" stage. > > I'd have to agree. The only ones that beat Python in that department are > Javascript and PHP. Plus CSS and HTML if you can call those languages. Your sentence makes no sense. You agree that "most" languages are cleaner than Python, and then in the very next sentence, out of the hundreds if not thousands of languages invented, you can only list TWO that are better than Python -- and those are Javascript and PHP!!! That's like saying "most things are harder than diamond -- the only things that beat diamond are jelly and talc". > The very idea of using a number of blanks to identify your block level > is as insane as it gets. Not at all. People do it all the time. The very idea of expecting people to count nested braces to identify block level is what is crazy, which is why in languages with braces people still indent the blocks. > First of all, combinations of blanks and tabs, > depending on how your ide is setup to expand tabs, may get you bugs, > you'd never imagine in your wild dreams. Not really. The bugs are quite simple, and generally easy to fix. To describe them as unimaginable is stupid. >>> for x in [1, 2, 3]: ... print x ... print x+1 File "", line 3 print x+1 ^ IndentationError: unexpected indent If you can't imagine getting an IndentationError from making an indentation error, there's something wrong with you. In any case, if your IDE mixes tabs and spaces, your IDE is broken and you should fix your tools rather than blame the language. > Braces is the most reliable way to identify blocks. Nonsense. For the compiler, both are equally reliable, and for the human reader, indents beat braces easily. > Sane compilers ignore blanks altogether. Really? So a "sane compiler" sees no difference between: for x in mylist: and forxinmylist: I'm glad I don't have to program using a compiler you consider "sane". -- Steven From uraniumore238 at gmail.com Sat Jan 30 14:18:51 2010 From: uraniumore238 at gmail.com (uche) Date: Sat, 30 Jan 2010 11:18:51 -0800 (PST) Subject: unencountered error in FFT python References: Message-ID: On Jan 30, 2:08?pm, MRAB wrote: > uche wrote: > > Hi, > > I have the following FFT python code and it doesn't seem to compile > > correctly. To run it, please create a file called output.csv with > > 1,2,3,4,5,6,7,8. simply run the main function. I get an error such as > > the following: > > > ? ? x[a], x[b] = x[(a)] + W[(n % N)] * x[(b)], x[(a)] - W[(n % N)] * x > > [(b)] > > TypeError: list indices must be integers, not float > > > How can I fixe this problem ? I have tried puttin int on all of the > > variables, but I don't think that is the intension of the person who > > wore the original code. > > Which version of Python are you using? In Python 3 the division operator > '/' returns a float, whereas in Python 2 it returns an int if both > operands are int. In Python 3 the int division operator is '//', which > is also accepted in recent versions of Python 2. > > [snip] > > > ? ? os.path.exists("input.csv") > > > ? ? fin=open('input.csv', 'r') > > > ? ? for line in fin: ? ?#read the line from the file > > > ? ? ? ? array=line.split(',') > > These lines should be indented more: > > > ? ? for a in range(len(array)): #convert into integers > > > ? ? ? ? array2.append((array[a])) > > ? ? ? ? ? array2.append(int(array[a])) > > [snip] Thanks, I just got this code from a site and trying to compile it. it's just a nightmare! I got another problem after changing / to // . Yes, I am using 3.1. W.append(exp(sign * 2j * pi * i // N)) TypeError: can't take floor of complex number. From stefan_ml at behnel.de Sat Jan 30 14:21:58 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 30 Jan 2010 20:21:58 +0100 Subject: unencountered error in FFT python In-Reply-To: <4b64800a$0$6735$9b4e6d93@newsspool2.arcor-online.net> References: <4b64800a$0$6735$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <4b6486d6$0$6724$9b4e6d93@newsspool2.arcor-online.net> Stefan Behnel, 30.01.2010 19:52: > uche, 30.01.2010 19:33: >> I have the following FFT python code > > You didn't seriously implement an FFT in plain Python code, did you? Sorry, no, you didn't. Should have read your post a little closer. > FFTs > are amongst the first thing that come to my mind when I try to imagine what > I'd use NumPy for. (and I *never* used it!) On second thought, I'd probably use FFTW for that, and there seem to be Python bindings for it: http://developer.berlios.de/projects/pyfftw/ Stefan From keith at nekotaku.com Sat Jan 30 14:28:47 2010 From: keith at nekotaku.com (KB) Date: Sat, 30 Jan 2010 11:28:47 -0800 (PST) Subject: Utility to screenscrape sites using javascript ? References: <4c4298c0-b93d-4691-bceb-e982373f87e3@f17g2000prh.googlegroups.com> Message-ID: <3df16722-7726-4b05-b8c3-d1b6a9f7e98b@k6g2000prg.googlegroups.com> > On Sat, 30 Jan 2010 06:21:01 -0800, KB wrote: > > I have a service I subscribe to that uses javascript to stream news. > There's a Python interface to SpiderMonkey (Mozilla's JavaScript > interpreter): > > http://pypi.python.org/pypi/python-spidermonkey Thanks! I don't see a documentation page, but how would one use this? Would you download the HTML using urllib2/mechanize, then parse for the .js script and use spider-monkey to "execute" the script and the output is passed back to python? TIA. From stefan_ml at behnel.de Sat Jan 30 14:28:47 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 30 Jan 2010 20:28:47 +0100 Subject: unencountered error in FFT python In-Reply-To: References: Message-ID: <4b64886f$0$6724$9b4e6d93@newsspool2.arcor-online.net> uche, 30.01.2010 20:18: > I got another problem after changing / to // . Yes, I am using 3.1. > > W.append(exp(sign * 2j * pi * i // N)) > TypeError: can't take floor of complex number. Don't change it everywhere, just where it deals with integers. In the above, "/" is perfectly right. Stefan From uraniumore238 at gmail.com Sat Jan 30 14:30:15 2010 From: uraniumore238 at gmail.com (uche) Date: Sat, 30 Jan 2010 11:30:15 -0800 (PST) Subject: unencountered error in FFT python References: <4b64800a$0$6735$9b4e6d93@newsspool2.arcor-online.net> <4b6486d6$0$6724$9b4e6d93@newsspool2.arcor-online.net> Message-ID: On Jan 30, 2:21?pm, Stefan Behnel wrote: > Stefan Behnel, 30.01.2010 19:52: > > > uche, 30.01.2010 19:33: > >> I have the following FFT python code > > > You didn't seriously implement an FFT in plain Python code, did you? > > Sorry, no, you didn't. Should have read your post a little closer. > > > FFTs > > are amongst the first thing that come to my mind when I try to imagine what > > I'd use NumPy for. (and I *never* used it!) > > On second thought, I'd probably use FFTW for that, and there seem to be > Python bindings for it: > > http://developer.berlios.de/projects/pyfftw/ > > Stefan Thanks for the suggestions and site. I just wanted to know what the heck is going wrong with this code. Please take a look at the code , thanks From apt.shansen at gmail.com Sat Jan 30 14:36:08 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Sat, 30 Jan 2010 11:36:08 -0800 Subject: Python and Ruby In-Reply-To: References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> Message-ID: <7a9c25c21001301136r25747681oefd7dc0a380db526@mail.gmail.com> On Sat, Jan 30, 2010 at 8:58 AM, tanix wrote: > The very idea of using a number of blanks to identify your block level > is as insane as it gets. First of all, combinations of blanks and tabs, > depending on how your ide is setup to expand tabs, may get you bugs, > you'd never imagine in your wild dreams. > > Braces is the most reliable way to identify blocks. > > Sane compilers ignore blanks altogether. > God forbid the compiler use the same syntactical structure as the human eye in reading code and assigning meaning. I mean, everyone knows that in C, no one ever has any issues with indentation levels causing human confusion and misunderstanding between what they think the structure of the code is and what the compiler interprets it as. Also, we all know very well that no modern IDE's or text editors let you manage tabs and spaces in a consistent fashion, and Python doesn't error out when you use inconsistent or nonsensical indentation. It just lets you create anonymous blocks of code when you indent too far in for no reason at all. --S P.S. Yes, my response is juvenile and snarky. However, I am weary of the imbecilic OMG WITESPACE IS RONG AND STUPID response. If you don't like whitespace being structurally significant, so be it-- aesthetics are subjective. If its not for you, its not for you. Python isn't for everyone. "Its insane", "Its stupid", "Its causes all kinds of confusion", "Its causes all kinds of hard to find bugs" is just complete, utter, unsupportable nonsense. -------------- next part -------------- An HTML attachment was scrubbed... URL: From uraniumore238 at gmail.com Sat Jan 30 15:20:48 2010 From: uraniumore238 at gmail.com (uche) Date: Sat, 30 Jan 2010 12:20:48 -0800 (PST) Subject: unencountered error in FFT python References: <4b64800a$0$6735$9b4e6d93@newsspool2.arcor-online.net> <4b6486d6$0$6724$9b4e6d93@newsspool2.arcor-online.net> Message-ID: On Jan 30, 2:30?pm, uche wrote: > On Jan 30, 2:21?pm, Stefan Behnel wrote: > > > > > > > Stefan Behnel, 30.01.2010 19:52: > > > > uche, 30.01.2010 19:33: > > >> I have the following FFT python code > > > > You didn't seriously implement an FFT in plain Python code, did you? > > > Sorry, no, you didn't. Should have read your post a little closer. > > > > FFTs > > > are amongst the first thing that come to my mind when I try to imagine what > > > I'd use NumPy for. (and I *never* used it!) > > > On second thought, I'd probably use FFTW for that, and there seem to be > > Python bindings for it: > > >http://developer.berlios.de/projects/pyfftw/ > > > Stefan > > Thanks for the suggestions and site. I just wanted to know what the > heck is going wrong with this code. Please take a look at the code , > thanks- Hide quoted text - > > - Show quoted text - Thanks Stephan. You are the best! Another issue: x[a], x[b] = x[(a)] + W[(n % N)] * x[(b)], x[(a)] - W[(n % N)] * x [(b)] TypeError: can't multiply sequence by non-int of type 'complex' From marcmagransdeabril at gmail.com Sat Jan 30 15:36:59 2010 From: marcmagransdeabril at gmail.com (marc magrans de abril) Date: Sat, 30 Jan 2010 12:36:59 -0800 (PST) Subject: Slow down while creating a big list and iterating over it Message-ID: <2fdd4e1b-db58-45d7-9b7d-73dcdca6384a@c29g2000yqd.googlegroups.com> Dear colleagues, I was doing a small program to classify log files for a cluster of PCs, I just wanted to simplify a quite repetitive task in order to find errors and so. My first naive implementation was something like: patterns = [] while(logs): pattern = logs[0] new_logs = [l for l in logs if dist(pattern,l)>THERESHOLD] entry = (len(logs)-len(new_logs),pattern) patterns.append(entry) logs = new_logs Where dist(...) is the levenshtein distance (i.e. edit distance) and logs is something like 1.5M logs (700 MB file). I thought that python will be an easy choice although not really fast.. I was not surprised when the first iteration of the while loop was taking ~10min. I thought "not bad, let's how much it takes". However, it seemed that the second iteration never finished. My surprise was big when I added a print instead of the list comprehension: new_logs=[] for count,l in enumerate(logs): print count if dist(pattern,l)>THERESHOLD: new_logs.append(l) The surprise was that the displayed counter was running ~10 times slower on the second iteration of the while loop. I am a little lost. Anyone knows the reson of this behavior? How should I write a program that deals with large data sets in python? Thanks a lot! marc magrans de abril From alfps at start.no Sat Jan 30 17:08:56 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sat, 30 Jan 2010 23:08:56 +0100 Subject: Slow down while creating a big list and iterating over it In-Reply-To: <2fdd4e1b-db58-45d7-9b7d-73dcdca6384a@c29g2000yqd.googlegroups.com> References: <2fdd4e1b-db58-45d7-9b7d-73dcdca6384a@c29g2000yqd.googlegroups.com> Message-ID: * marc magrans de abril: > Dear colleagues, > > I was doing a small program to classify log files for a cluster of > PCs, I just wanted to simplify a quite repetitive task in order to > find errors and so. > > My first naive implementation was something like: > patterns = [] > while(logs): > pattern = logs[0] > new_logs = [l for l in logs if dist(pattern,l)>THERESHOLD] > entry = (len(logs)-len(new_logs),pattern) > patterns.append(entry) > logs = new_logs > > Where dist(...) is the levenshtein distance (i.e. edit distance) and > logs is something like 1.5M logs (700 MB file). I thought that python > will be an easy choice although not really fast.. > > I was not surprised when the first iteration of the while loop was > taking ~10min. I thought "not bad, let's how much it takes". However, > it seemed that the second iteration never finished. > > My surprise was big when I added a print instead of the list > comprehension: > new_logs=[] > for count,l in enumerate(logs): > print count > if dist(pattern,l)>THERESHOLD: > new_logs.append(l) > > The surprise was that the displayed counter was running ~10 times > slower on the second iteration of the while loop. > > I am a little lost. Anyone knows the reson of this behavior? It's on line 42 of your program. :-) That is, it's in the dist function. Evidently it doesn't like a more complex 'pattern'. > How should I write a program that deals with large data sets in python? As in any other language. Try to avoid repeating the same computations. Try to make the data fit the computational task. Cheers & hth., - Alf From johnroth1 at gmail.com Sat Jan 30 17:14:54 2010 From: johnroth1 at gmail.com (John Roth) Date: Sat, 30 Jan 2010 14:14:54 -0800 (PST) Subject: PEP 3147 - new .pyc format Message-ID: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> PEP 3147 has just been posted, proposing that, beginning in release 3.2 (and possibly 2.7) compiled .pyc and .pyo files be placed in a directory with a .pyr extension. The reason is so that compiled versions of a program can coexist, which isn't possible now. Frankly, I think this is a really good idea, although I've got a few comments. 1. Apple's MAC OS X should be mentioned, since 10.5 (and presumably 10.6) ship with both Python release 2.3 and 2.5 installed. 2. I think the proposed logic is too complex. If this is installed in 3.2, then that release should simply store its .pyc file in the .pyr directory, without the need for either a command line switch or an environment variable (both are mentioned in the PEP.) 3. Tool support. There are tools that look for the .pyc files; these need to be upgraded somehow. The ones that ship with Python should, of course, be fixed with the PEP, but there are others. 4. I'm in favor of putting the source in the .pyr directory as well, but that's got a couple more issues. One is tool support, which is likely to be worse for source, and the other is some kind of algorithm for identifying which source goes with which object. Summary: I like it, but I think it needs a bit more work. John Roth From andrew at acooke.org Sat Jan 30 17:16:19 2010 From: andrew at acooke.org (andrew cooke) Date: Sat, 30 Jan 2010 14:16:19 -0800 (PST) Subject: Function name unchanged in error message References: Message-ID: <29d9b330-a57f-48a8-9d1e-bdad7c038ad9@k19g2000yqc.googlegroups.com> On Jan 29, 11:22?am, Peter Otten <__pete... at web.de> wrote: > The name is looked up in the code object. As that is immutable you have to > make a new one: [details snipped] thanks very much! sorry i didn't reply earlier - been travelling. (also, thanks to any other replies - i'm just reading through at the moment and this is the first one i've got to that will help me solve it, but i don't mean to exclude anything later...!) andrew From andrew at acooke.org Sat Jan 30 17:17:39 2010 From: andrew at acooke.org (andrew cooke) Date: Sat, 30 Jan 2010 14:17:39 -0800 (PST) Subject: Function name unchanged in error message References: Message-ID: <8e7d79dc-5f91-4e8c-921b-399fe0da77bf@m16g2000yqc.googlegroups.com> On Jan 29, 5:37?pm, "Gabriel Genellina" wrote: > The decorator module is a very fine addition to anyone's tool set -- but ? > in this case it is enough to use the wraps() function from the functools ? > standard module. ah, thanks! i thought something like this existed in the standard lib, but couldn't find it. andrew From andrew at acooke.org Sat Jan 30 17:26:43 2010 From: andrew at acooke.org (andrew cooke) Date: Sat, 30 Jan 2010 14:26:43 -0800 (PST) Subject: Function name unchanged in error message References: <50697b2c1001290610t1a2ac5c8hcd5f9856aa85392b@mail.gmail.com> Message-ID: <5303e556-e9df-4ef3-bd4d-fa4928399c6e@r19g2000yqb.googlegroups.com> On Jan 29, 11:50?am, exar... at twistedmatrix.com wrote: > new.function and new.code will let you construct new objects with > different values (and copying over whichever existing attributes you > want to preserve). unfortunately new is deprecated and dropped from 3. i can't see how the same functionality is available in the types module for 3 - am i missing something obvious? http://docs.python.org/library/new.html http://docs.python.org/3.1/library/types.html#module-types thanks, andrew From andrew at acooke.org Sat Jan 30 17:31:51 2010 From: andrew at acooke.org (andrew cooke) Date: Sat, 30 Jan 2010 14:31:51 -0800 (PST) Subject: Function name unchanged in error message References: <8e7d79dc-5f91-4e8c-921b-399fe0da77bf@m16g2000yqc.googlegroups.com> Message-ID: On Jan 30, 7:17?pm, andrew cooke wrote: > On Jan 29, 5:37?pm, "Gabriel Genellina" > wrote: > > > The decorator module is a very fine addition to anyone's tool set -- but ? > > in this case it is enough to use the wraps() function from the functools ? > > standard module. > > ah, thanks! ?i thought something like this existed in the standard > lib, but couldn't find it. > > andrew ah, sorry, peter's code uses types, so i assume that's the way to go (i was hoping that there was something a bit simpler - i don't like the fact that the code in peter's code has a fixed list of special names). andrew From mensanator at aol.com Sat Jan 30 17:36:52 2010 From: mensanator at aol.com (Mensanator) Date: Sat, 30 Jan 2010 14:36:52 -0800 (PST) Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> Message-ID: On Jan 30, 4:14?pm, John Roth wrote: > PEP 3147 has just been posted, proposing that, beginning in release > 3.2 (and possibly 2.7) compiled .pyc and .pyo files be placed in a > directory with a .pyr extension. The reason is so that compiled > versions of a program can coexist, which isn't possible now. > > Frankly, I think this is a really good idea, although I've got a few > comments. > > 1. Apple's MAC OS X should be mentioned, since 10.5 (and presumably > 10.6) ship with both Python release 2.3 and 2.5 installed. Mac OSX 10.6 has 2.6 installed. > > 2. I think the proposed logic is too complex. If this is installed in > 3.2, then that release should simply store its .pyc file in the .pyr > directory, without the need for either a command line switch or an > environment variable (both are mentioned in the PEP.) > > 3. Tool support. There are tools that look for the .pyc files; these > need to be upgraded somehow. The ones that ship with Python should, of > course, be fixed with the PEP, but there are others. > > 4. I'm in favor of putting the source in the .pyr directory as well, > but that's got a couple more issues. One is tool support, which is > likely to be worse for source, and the other is some kind of algorithm > for identifying which source goes with which object. > > Summary: I like it, but I think it needs a bit more work. > > John Roth From python at mrabarnett.plus.com Sat Jan 30 17:57:57 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 30 Jan 2010 22:57:57 +0000 Subject: PEP 3147 - new .pyc format In-Reply-To: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> Message-ID: <4B64B975.7060404@mrabarnett.plus.com> John Roth wrote: > PEP 3147 has just been posted, proposing that, beginning in release > 3.2 (and possibly 2.7) compiled .pyc and .pyo files be placed in a > directory with a .pyr extension. The reason is so that compiled > versions of a program can coexist, which isn't possible now. > > Frankly, I think this is a really good idea, although I've got a few > comments. > > 1. Apple's MAC OS X should be mentioned, since 10.5 (and presumably > 10.6) ship with both Python release 2.3 and 2.5 installed. > > 2. I think the proposed logic is too complex. If this is installed in > 3.2, then that release should simply store its .pyc file in the .pyr > directory, without the need for either a command line switch or an > environment variable (both are mentioned in the PEP.) > > 3. Tool support. There are tools that look for the .pyc files; these > need to be upgraded somehow. The ones that ship with Python should, of > course, be fixed with the PEP, but there are others. > > 4. I'm in favor of putting the source in the .pyr directory as well, > but that's got a couple more issues. One is tool support, which is > likely to be worse for source, and the other is some kind of algorithm > for identifying which source goes with which object. > > Summary: I like it, but I think it needs a bit more work. > The PEP has a .pyr directory for each .py file: foo.py foo.pyr/ f2b30a0d.pyc # Python 2.5 f2d10a0d.pyc # Python 2.6 f2d10a0d.pyo # Python 2.6 -O f2d20a0d.pyc # Python 2.6 -U 0c4f0a0d.pyc # Python 3.1 Other possibilities are: 1. A single pyr directory: foo.py pyr/ foo.f2b30a0d.pyc # Python 2.5 foo.f2d10a0d.pyc # Python 2.6 foo.f2d10a0d.pyo # Python 2.6 -O foo.f2d20a0d.pyc # Python 2.6 -U foo.0c4f0a0d.pyc # Python 3.1 2. A .pyr directory for each version of Python: foo.py f2b30a0d.pyr/ # Python 2.5 foo.pyc f2d10a0d.pyr/ # Python 2.6/Python 2.6 -O foo.pyc foo.pyo f2d20a0d.pyr/ # Python 2.6 -U foo.pyc 0c4f0a0d.pyr/ # Python 3.1 foo.pyc From kerensaelise at hotmail.com Sat Jan 30 18:08:20 2010 From: kerensaelise at hotmail.com (elsa) Date: Sat, 30 Jan 2010 15:08:20 -0800 (PST) Subject: Still too slow Message-ID: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> Hello again, Thanks for the tips r.e random.ranint(). This improved matters somewhat, however my program is still too slow. If anyone has any further tips on how to speed it up, they would be much appreciated! So, I'm calling evolve(L,limit) from the interactive prompt. L is initally [[100],['NA']]. Ideally limit would be 10^7. Here is my program: import random n=100 def evolve(L,limit): global n while n References: <2fdd4e1b-db58-45d7-9b7d-73dcdca6384a@c29g2000yqd.googlegroups.com> Message-ID: <4B64BCBA.5040905@mrabarnett.plus.com> Alf P. Steinbach wrote: > * marc magrans de abril: >> Dear colleagues, >> >> I was doing a small program to classify log files for a cluster of >> PCs, I just wanted to simplify a quite repetitive task in order to >> find errors and so. >> >> My first naive implementation was something like: >> patterns = [] >> while(logs): >> pattern = logs[0] >> new_logs = [l for l in logs if dist(pattern,l)>THERESHOLD] >> entry = (len(logs)-len(new_logs),pattern) >> patterns.append(entry) >> logs = new_logs >> >> Where dist(...) is the levenshtein distance (i.e. edit distance) and >> logs is something like 1.5M logs (700 MB file). I thought that python >> will be an easy choice although not really fast.. >> >> I was not surprised when the first iteration of the while loop was >> taking ~10min. I thought "not bad, let's how much it takes". However, >> it seemed that the second iteration never finished. >> >> My surprise was big when I added a print instead of the list >> comprehension: >> new_logs=[] >> for count,l in enumerate(logs): >> print count >> if dist(pattern,l)>THERESHOLD: >> new_logs.append(l) >> >> The surprise was that the displayed counter was running ~10 times >> slower on the second iteration of the while loop. >> >> I am a little lost. Anyone knows the reson of this behavior? > > It's on line 42 of your program. :-) That is, it's in the dist function. > Evidently it doesn't like a more complex 'pattern'. > Find out which pattern is being used on the second iteration and then try it on the first iteration. Is it just as slow? If so, then it's down to the length/complexity of that pattern - a much longer/more complex pattern might take much longer when computing the distance. >> How should I write a program that deals with large data sets in python? > > As in any other language. Try to avoid repeating the same computations. > Try to make the data fit the computational task. > True. Basically, you're computing the distance between every pair of logs! From andrew at acooke.org Sat Jan 30 18:13:06 2010 From: andrew at acooke.org (andrew cooke) Date: Sat, 30 Jan 2010 15:13:06 -0800 (PST) Subject: Function name unchanged in error message References: Message-ID: <33a3e0e9-2051-41a9-9ddd-f1c8db713bd3@z26g2000yqm.googlegroups.com> On Jan 29, 1:09?pm, Michele Simionato wrote: > On Jan 29, 2:30?pm, andrew cooke wrote: > > > Is there any way to change the name of the function in an error > > message? ?In the example below I'd like the error to refer to bar(), > > for example (the motivation is related function decorators - I'd like > > the wrapper function to give the same name) > > Use the decorator module which does the right thing:http://pypi.python.org/pypi/decorator curiously, decorator doesn't have this issue, because the way it defines decorators uses *args. so the error i gave cannot occur at the level of the decorator - the extra arg is passed to the wrapped function, and so the error message is correct because it is generated by the inner function. i need to look at my code; this might be the simplest solution of all. thanks, andrew From philip at semanchuk.com Sat Jan 30 18:31:41 2010 From: philip at semanchuk.com (Philip Semanchuk) Date: Sat, 30 Jan 2010 18:31:41 -0500 Subject: Still too slow In-Reply-To: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> References: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> Message-ID: On Jan 30, 2010, at 6:08 PM, elsa wrote: > Hello again, > > Thanks for the tips r.e random.ranint(). This improved matters > somewhat, however my program is still too slow. If anyone has any > further tips on how to speed it up, they would be much appreciated! > > So, I'm calling evolve(L,limit) from the interactive prompt. L is > initally [[100],['NA']]. Ideally limit would be 10^7. > > Here is my program: > > import random > n=100 > > def evolve(L,limit): > global n > while n evnt = event() > if evnt!="None": > ind = chooseInd(L,n) > action(evnt,L,ind) > > def chooseInd(L,n): > choiceSum=0 > index=0 > choice = random.randint(1,n) > while choiceSum < choice: > choiceSum+=L[index][0] > index +=1 > return (index-1) > > def event(): > choice = random.random() > if choice <= .3: > event='b' > elif choice <= .4: > event ='d' > elif choice<= .5: > event = 'm' > else: > event = 'None' > return event > > def action(event, L, index): > global n > if event == 'b': > L[index][0]+=1 > n +=1 > elif event == 'd': > L[index][0]-=1 > n -=1 > elif event == 'm': > L.append([1,index]) > n +=1 Hi Elsa, I didn't follow the earlier discussion on this, so pardon me if I'm repeating what others have said. First, have you profiled the code? That will help you find bottlenecks. I haven't profiled it myself so I am shooting from the hip, but a couple of obvious speedups in event() jump out at me. I'd rewrite it like this: def event(): choice = random.random() if choice <= .3: return 'b' elif choice <= .4: return 'd' elif choice <= .5: return 'm' return None Returning immediately when you've found your choice will save evaluating a couple of elifs -- not a big help, but it is a little. It comes at the cost having multiple exits to the function which is something I prefer to avoid, but this function is small enough that having multiple exits won't get confusing. Second, returning the Python object None rather than the string "None" allows you to change evolve() from this string comparison: if evnt!="None": to this comparison to the Python singleton None: if evnt != None: or this, which is even simpler (although it may not be faster): if evnt: Hope this helps Philip From john at castleamber.com Sat Jan 30 18:38:52 2010 From: john at castleamber.com (John Bokma) Date: Sat, 30 Jan 2010 17:38:52 -0600 Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> Message-ID: <87iqaj9n8z.fsf@castleamber.com> MRAB writes: > The PEP has a .pyr directory for each .py file: > > foo.py > foo.pyr/ > f2b30a0d.pyc # Python 2.5 > f2d10a0d.pyc # Python 2.6 > f2d10a0d.pyo # Python 2.6 -O > f2d20a0d.pyc # Python 2.6 -U > 0c4f0a0d.pyc # Python 3.1 wow: so much for human readable file names :-( -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From alfps at start.no Sat Jan 30 18:46:03 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 31 Jan 2010 00:46:03 +0100 Subject: PEP 3147 - new .pyc format In-Reply-To: <87iqaj9n8z.fsf@castleamber.com> References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <87iqaj9n8z.fsf@castleamber.com> Message-ID: * John Bokma: > MRAB writes: > >> The PEP has a .pyr directory for each .py file: >> >> foo.py >> foo.pyr/ >> f2b30a0d.pyc # Python 2.5 >> f2d10a0d.pyc # Python 2.6 >> f2d10a0d.pyo # Python 2.6 -O >> f2d20a0d.pyc # Python 2.6 -U >> 0c4f0a0d.pyc # Python 3.1 > > wow: so much for human readable file names :-( I agree. Human readable filenames would be much better. Cheers, - Alf From pavlovevidence at gmail.com Sat Jan 30 19:14:53 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Sat, 30 Jan 2010 16:14:53 -0800 (PST) Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> Message-ID: On Jan 30, 2:14?pm, John Roth wrote: > PEP 3147 has just been posted, proposing that, beginning in release > 3.2 (and possibly 2.7) compiled .pyc and .pyo files be placed in a > directory with a .pyr extension. The reason is so that compiled > versions of a program can coexist, which isn't possible now. > > Frankly, I think this is a really good idea, although I've got a few > comments. -1 I think it's a terrible, drastic approach to a minor problem. I'm not sure why the simple approach of just appending a number (perhaps the major-minor version, or a serial number) to the filename wouldn't work, like this: foo.pyc25 All I can think of is they are concerned with the typically minor expense of listing the directory (to see if there's already .pyc?? file present). This operation can be reasonably cached; when scanning a directory listing it need only record all occurrencs of .pyc?? and mark those modules as subject to version-specific .pyc files. Anyway, I'd expect the proposed -R switch would only be used in special cases (like installation time) when a minor inefficiency would be tolerable. > 1. Apple's MAC OS X should be mentioned, since 10.5 (and presumably > 10.6) ship with both Python release 2.3 and 2.5 installed. > > 2. I think the proposed logic is too complex. If this is installed in > 3.2, then that release should simply store its .pyc file in the .pyr > directory, without the need for either a command line switch or an > environment variable (both are mentioned in the PEP.) This is utterly unacceptable. Versioned *.pyc files should only be optionally requested by people who have to deal multiple versions, such as distro package maintainers. For my projects I don't give a flying F about versioned *.pyc and I don't want my project directory cluttered with a million subdirectories. (It would be a bit more tolerable if my directory was merely cluttered with *.pyc?? files, but I'd still rather Python didn't do that unless asked.) > 3. Tool support. There are tools that look for the .pyc files; these > need to be upgraded somehow. The ones that ship with Python should, of > course, be fixed with the PEP, but there are others. How will this affect tools like Py2exe? Now you have a bunch of identically-named *.pyc files. > 4. I'm in favor of putting the source in the .pyr directory as well, > but that's got a couple more issues. One is tool support, which is > likely to be worse for source, and the other is some kind of algorithm > for identifying which source goes with which object. Now this just too much. I didn't like the suggestion that I should be forced to put up with dozens of subdirectories, now you want me to force me to put the source files into the subdirectories as well? That would be a deal-breaker. Thankfully it is too ridiculous to ever happen. > Summary: I like it, but I think it needs a bit more work. I hope it's replaced with something less drastic. Carl Banks From python at mrabarnett.plus.com Sat Jan 30 19:19:50 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 31 Jan 2010 00:19:50 +0000 Subject: PEP 3147 - new .pyc format In-Reply-To: References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <87iqaj9n8z.fsf@castleamber.com> Message-ID: <4B64CCA6.9050607@mrabarnett.plus.com> Alf P. Steinbach wrote: > * John Bokma: >> MRAB writes: >> >>> The PEP has a .pyr directory for each .py file: >>> >>> foo.py >>> foo.pyr/ >>> f2b30a0d.pyc # Python 2.5 >>> f2d10a0d.pyc # Python 2.6 >>> f2d10a0d.pyo # Python 2.6 -O >>> f2d20a0d.pyc # Python 2.6 -U >>> 0c4f0a0d.pyc # Python 3.1 >> >> wow: so much for human readable file names :-( > > I agree. > > Human readable filenames would be much better. > The names are the magic numbers. It's all in the PEP. From jjposner at optimum.net Sat Jan 30 19:27:46 2010 From: jjposner at optimum.net (John Posner) Date: Sat, 30 Jan 2010 19:27:46 -0500 Subject: Still too slow In-Reply-To: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> References: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> Message-ID: <4B64CE82.1040109@optimum.net> On 1/30/2010 6:08 PM, elsa wrote: > Hello again, > > Thanks for the tips r.e random.ranint(). This improved matters > somewhat, however my program is still too slow. If anyone has any > further tips on how to speed it up, they would be much appreciated! > > So, I'm calling evolve(L,limit) from the interactive prompt. L is > initally [[100],['NA']]. Ideally limit would be 10^7. > > Here is my program: > > import random > n=100 > > def evolve(L,limit): > global n > while n evnt = event() > if evnt!="None": > ind = chooseInd(L,n) > action(evnt,L,ind) > > def chooseInd(L,n): > choiceSum=0 > index=0 > choice = random.randint(1,n) > while choiceSum< choice: > choiceSum+=L[index][0] > index +=1 > return (index-1) > > def event(): > choice = random.random() > if choice<= .3: > event='b' > elif choice<= .4: > event ='d' > elif choice<= .5: > event = 'm' > else: > event = 'None' > return event > > def action(event, L, index): > global n > if event == 'b': > L[index][0]+=1 > n +=1 > elif event == 'd': > L[index][0]-=1 > n -=1 > elif event == 'm': > L.append([1,index]) > n +=1 > > > thanks in advance, > > Elsa. Elsa, 1. You changed the subject line from "For loop searching takes too long!" to "Still too slow". This causes newsreader programs to start a new discussion thread, which makes life difficult for people who need to refer back to previous messages. Please don't change the subject line any more. 2. You provided a very clear description of your original question: Now, what I need to do is randomly choose one myList[i], however the distribution of my random choice needs to be proportional to the values of myList[i][0]. This description should be the doc string for the chooseInd() function -- for example: def chooseInd(L,n): """ randomly choose one L[i], so that the distribution of choices is proportional to the values of L[i][0] """ It is not clear (to me, anyway) what the other functions are trying to accomplish. So please add a doc string to each of these functions, with descriptions of similar clarity. This will help us a lot. And it will help you, too, if you return to the code after a few days/weeks/months of directing your attention elsewhere. 3. Please provide a *complete* transcript of an interactive session that exercises this code. Tx, John From john at castleamber.com Sat Jan 30 19:51:31 2010 From: john at castleamber.com (John Bokma) Date: Sat, 30 Jan 2010 18:51:31 -0600 Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <87iqaj9n8z.fsf@castleamber.com> Message-ID: <877hqz2j1o.fsf@castleamber.com> MRAB writes: > Alf P. Steinbach wrote: >> * John Bokma: >>> MRAB writes: >>> >>>> The PEP has a .pyr directory for each .py file: >>>> >>>> foo.py >>>> foo.pyr/ >>>> f2b30a0d.pyc # Python 2.5 >>>> f2d10a0d.pyc # Python 2.6 >>>> f2d10a0d.pyo # Python 2.6 -O >>>> f2d20a0d.pyc # Python 2.6 -U >>>> 0c4f0a0d.pyc # Python 3.1 >>> >>> wow: so much for human readable file names :-( >> >> I agree. >> >> Human readable filenames would be much better. >> > The names are the magic numbers. It's all in the PEP. Naming files using magic numbers is really beyond me. The fact that the above needs comments to explain what's what already shows to me that there's a problem with this naming scheme. What if for one reason or another I want to delete all pyc files for Python 2.5? Where do I look up the magic number? -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From nyamatongwe+thunder at gmail.com Sat Jan 30 20:09:07 2010 From: nyamatongwe+thunder at gmail.com (Neil Hodgson) Date: Sun, 31 Jan 2010 01:09:07 GMT Subject: PEP 3147 - new .pyc format In-Reply-To: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> Message-ID: John Roth: > 4. I'm in favor of putting the source in the .pyr directory as well, > but that's got a couple more issues. One is tool support, which is > likely to be worse for source, and the other is some kind of algorithm > for identifying which source goes with which object. Many tools work recursively except for hidden directories so would return both the source in the repository as well as the original source. If you want to do this then the repository directory should be hidden by starting with ".". Neil From python at mrabarnett.plus.com Sat Jan 30 20:16:54 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 31 Jan 2010 01:16:54 +0000 Subject: PEP 3147 - new .pyc format In-Reply-To: <877hqz2j1o.fsf@castleamber.com> References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <87iqaj9n8z.fsf@castleamber.com> <877hqz2j1o.fsf@castleamber.com> Message-ID: <4B64DA06.7020200@mrabarnett.plus.com> John Bokma wrote: > MRAB writes: > >> Alf P. Steinbach wrote: >>> * John Bokma: >>>> MRAB writes: >>>> >>>>> The PEP has a .pyr directory for each .py file: >>>>> >>>>> foo.py >>>>> foo.pyr/ >>>>> f2b30a0d.pyc # Python 2.5 >>>>> f2d10a0d.pyc # Python 2.6 >>>>> f2d10a0d.pyo # Python 2.6 -O >>>>> f2d20a0d.pyc # Python 2.6 -U >>>>> 0c4f0a0d.pyc # Python 3.1 >>>> wow: so much for human readable file names :-( >>> I agree. >>> >>> Human readable filenames would be much better. >>> >> The names are the magic numbers. It's all in the PEP. > > Naming files using magic numbers is really beyond me. The fact that the > above needs comments to explain what's what already shows to me that > there's a problem with this naming scheme. What if for one reason or > another I want to delete all pyc files for Python 2.5? Where do I look > up the magic number? > True. You might also want to note that "Python 2.6 -U" appears to have a different magic number from "Python 2.6" and "Python 2.6 -O". I don't know whether they always change for each new version. From steve at REMOVE-THIS-cybersource.com.au Sat Jan 30 20:18:52 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 31 Jan 2010 01:18:52 GMT Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> Message-ID: <0374c6d6$0$1309$c3e8da3@news.astraweb.com> On Sat, 30 Jan 2010 14:14:54 -0800, John Roth wrote: > PEP 3147 has just been posted, proposing that, beginning in release 3.2 > (and possibly 2.7) compiled .pyc and .pyo files be placed in a directory > with a .pyr extension. The reason is so that compiled versions of a > program can coexist, which isn't possible now. http://www.python.org/dev/peps/pep-3147/ Reading through the PEP, I went from an instinctive "oh no, that sounds horrible" reaction to "hmmm, well, that doesn't sound too bad". I don't think I need this, but I could live with it. Firstly, it does sound like there is a genuine need for a solution to the problem of multiple Python versions. This is not the first PEP trying to solve it, so even if you personally don't see the need, we can assume that others do. Secondly, the current behaviour will remain unchanged. Python will compile spam.py to spam.pyc (or spam.pyo with the -O switch) by default. If you don't need to support multiple versions, you don't need to care about this PEP. I like this aspect of the PEP very much. I would argue that any solution MUST support the status quo for those who don't care about multiple versions. To get the new behaviour, you have to explicitly ask for it. You ask for it by calling python with the -R switch, by setting an environment variable, or explicitly providing the extra spam/.pyc files. Thirdly, the magic file names aren't quite as magic as they appear at first glance. They represent the hexified magic number of the version of Python. More about the magic number here: http://nedbatchelder.com/blog/200804/the_structure_of_pyc_files.html Unfortunately the magic number doesn't seem to be documented anywhere I can find other than in the source code (import.c). The PEP gives some examples: f2b30a0d.pyc # Python 2.5 f2d10a0d.pyc # Python 2.6 f2d10a0d.pyo # Python 2.6 -O f2d20a0d.pyc # Python 2.6 -U 0c4f0a0d.pyc # Python 3.1 but how can one map magic numbers to versions, short of reading import.c? I propose that sys grow an object sys.magic which is the hexlified magic number. > 2. I think the proposed logic is too complex. If this is installed in > 3.2, then that release should simply store its .pyc file in the .pyr > directory, without the need for either a command line switch or an > environment variable (both are mentioned in the PEP.) I disagree. Making the new behaviour optional is an advantage, even if it leads to extra complexity. It is pointless forcing .pyc files to be in a subdirectory if you don't need multiple versions. > 3. Tool support. There are tools that look for the .pyc files; these > need to be upgraded somehow. The ones that ship with Python should, of > course, be fixed with the PEP, but there are others. Third party tools will be the responsibility of the third parties. > 4. I'm in favor of putting the source in the .pyr directory as well, but > that's got a couple more issues. One is tool support, which is likely to > be worse for source, and the other is some kind of algorithm for > identifying which source goes with which object. It certain does. What's the advantage of forcing .py files to live inside a directory with the same name? Modules: mymodule.py => mymodule/mymodule.py Packages: mypackage/__init__.py => mypackage/__init__/__init__.py mypackage/spam.py => mypackage/spam/spam.py Seems like a pointless and annoying extra layer to me. -- Steven From alfps at start.no Sat Jan 30 20:34:24 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 31 Jan 2010 02:34:24 +0100 Subject: Still too slow In-Reply-To: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> References: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> Message-ID: * elsa: > > Thanks for the tips r.e random.ranint(). This improved matters > somewhat, however my program is still too slow. If anyone has any > further tips on how to speed it up, they would be much appreciated! > > So, I'm calling evolve(L,limit) from the interactive prompt. L is > initally [[100],['NA']]. This would produce an unhandled exception since your code tries to compute the sum of the values in the list. You can't add strings and numbers. > Ideally limit would be 10^7. > > Here is my program: > > import random > n=100 > > def evolve(L,limit): > global n > while n evnt = event() > if evnt!="None": > ind = chooseInd(L,n) > action(evnt,L,ind) > > def chooseInd(L,n): > choiceSum=0 > index=0 > choice = random.randint(1,n) > while choiceSum < choice: > choiceSum+=L[index][0] > index +=1 > return (index-1) This is the problematic part of your program, because it uses time linear in the length of L. Consider if you know the sum s_low of the lower half of the array, and the sum s_high of the higher half of the array. Then you can choose the lower half with probability s_low/(s_low+s_high), otherwise the higher half. Then repeat this recursively for the half chosen, until you get down to a single array element. This requires keeping track of the sums of halfs of the array, e.g. in a tree like structure or a set of arrays of diminishing lengths. Generally it will use some constant factor times twice the storage that you're now using. But it reduces the time for choosing an index from O(n) to O(log n). For example, if you have a logical structure like * / \ * 1 / \ 98 1 then at the top level * you choose the left branch with probability 99/(99+1) = 99/100 = 0.99. At the second level * you choose the right branch with probability 1/(98+1) = 1/99. The combined probability of choosing that lower 1 is then 0.99*(1/99) = 0.01, as it should be. > def event(): > choice = random.random() > if choice <= .3: > event='b' > elif choice <= .4: > event ='d' > elif choice<= .5: > event = 'm' > else: > event = 'None' > return event Note that you can double the speed of your program by removing the 'None' value. Adjust the limits you're checking against so as to retain the same probabilities of the choices. Further constant factor speedup is possible by simply returning a function to Do Stuff instead of returning a character saying what to do. > def action(event, L, index): > global n > if event == 'b': > L[index][0]+=1 > n +=1 > elif event == 'd': > L[index][0]-=1 > n -=1 > elif event == 'm': > L.append([1,index]) > n +=1 > > > thanks in advance, An observation about design: you have a global n (the current total sum) and an array L that you're passing around everywhere, so that it's effectively also a global. This indicates that they should ideally instead be attributes of an object, with your routines above as methods. However in Python this may cause some overhead, so, perhaps first make it Work (and then make a Copy). :-) Cheers & hth., - Alf From alfps at start.no Sat Jan 30 20:48:10 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 31 Jan 2010 02:48:10 +0100 Subject: Still too slow In-Reply-To: References: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> Message-ID: * Alf P. Steinbach: > * elsa: >> >> Thanks for the tips r.e random.ranint(). This improved matters >> somewhat, however my program is still too slow. If anyone has any >> further tips on how to speed it up, they would be much appreciated! >> >> So, I'm calling evolve(L,limit) from the interactive prompt. L is >> initally [[100],['NA']]. > > This would produce an unhandled exception since your code tries to > compute the sum of the values in the list. You can't add strings and > numbers. > > >> Ideally limit would be 10^7. >> >> Here is my program: >> >> import random >> n=100 >> >> def evolve(L,limit): >> global n >> while n> evnt = event() >> if evnt!="None": >> ind = chooseInd(L,n) >> action(evnt,L,ind) >> >> def chooseInd(L,n): >> choiceSum=0 >> index=0 >> choice = random.randint(1,n) >> while choiceSum < choice: >> choiceSum+=L[index][0] >> index +=1 >> return (index-1) > > This is the problematic part of your program, because it uses time > linear in the length of L. > > Consider if you know the sum s_low of the lower half of the array, and > the sum s_high of the higher half of the array. Then you can choose the > lower half with probability s_low/(s_low+s_high), otherwise the higher > half. Then repeat this recursively for the half chosen, until you get > down to a single array element. > > This requires keeping track of the sums of halfs of the array, e.g. in a > tree like structure or a set of arrays of diminishing lengths. Generally > it will use some constant factor times twice the storage that you're now > using. But it reduces the time for choosing an index from O(n) to O(log n). > > For example, if you have a logical structure like > > * > / \ > * 1 > / \ > 98 1 > > then at the top level * you choose the left branch with probability > 99/(99+1) = 99/100 = 0.99. At the second level * you choose the right > branch with probability 1/(98+1) = 1/99. The combined probability of > choosing that lower 1 is then 0.99*(1/99) = 0.01, as it should be. Oh, forgot to add: since your array's first item will be the one that's far most often chosen a "blind" sort of literal implementation of the above will perhaps slow down the program instead of speeding it up, because it only speeds up those searches that progress beyond the first item. But anyway, I'm guessing that checking the first item first, as a special case, will really improve things. Only when it turns out that it's not the first item, apply binary search. >> def event(): >> choice = random.random() >> if choice <= .3: >> event='b' >> elif choice <= .4: >> event ='d' >> elif choice<= .5: >> event = 'm' >> else: >> event = 'None' >> return event > > Note that you can double the speed of your program by removing the /s/double/increase/g > 'None' value. Adjust the limits you're checking against so as to retain > the same probabilities of the choices. Further constant factor speedup > is possible by simply returning a function to Do Stuff instead of > returning a character saying what to do. > > > >> def action(event, L, index): >> global n >> if event == 'b': >> L[index][0]+=1 >> n +=1 >> elif event == 'd': >> L[index][0]-=1 >> n -=1 >> elif event == 'm': >> L.append([1,index]) >> n +=1 >> >> >> thanks in advance, > > An observation about design: you have a global n (the current total sum) > and an array L that you're passing around everywhere, so that it's > effectively also a global. This indicates that they should ideally > instead be attributes of an object, with your routines above as methods. > However in Python this may cause some overhead, so, perhaps first make > it Work (and then make a Copy). :-) Cheers & hth., - Alf From kerensaelise at hotmail.com Sat Jan 30 20:58:57 2010 From: kerensaelise at hotmail.com (elsa) Date: Sat, 30 Jan 2010 17:58:57 -0800 (PST) Subject: Still too slow References: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> <4B64CE82.1040109@optimum.net> Message-ID: Hi John and others, sorry about my etiquette errors. As you can tell I'm a newbie, and appreciate all the help I can get. I'm trying to master this thing with only the net and a couple of books as tutors. Here is what I'm running at the interactive prompt: >>> import myBDM >>> L=[[100,'NA']] >>> myBDM.evolve(L,100000) Ideally, I'd like to bump it up to myBDM.evolve(L,10000000). L keeps track of the number of subpopulations, how many individuals are in each subpopulation, and the parent subpopulation each subpopulation arose from. Here is my code again: import random n=100 def evolve(L,limit): """ evolves the population until the population size reaches limit, by choosing an individual of a particular subpopulation type, then randomly performing a birth, death, or mutation on this individual """ global n while n References: <85ee4831-3b9c-4096-b32a-387e59d6e7c1@r6g2000yqn.googlegroups.com> Message-ID: <4B64E9DD.7050609@ieee.org> peskar.miha at hotmail.com wrote: > 21 days has passed and still noone is willing to help :-( > > ch /willing/able/ I wouldn't say no-one, even then, since there were at least 10 messages in the thread on the 19th and 20th. Presumably they weren't all from you. If you were doing this to text files, I would have been happy to help. But interacting with MS Word isn't easy, and I don't know how. If you're a beginner as you stated, you need to pick an easier first task. If you're determined to do it, best advice I could give is to head over to the python-Win32 mailing list. At least they're used to dealing with COM, and other MS-specific stuff. DaveA From ben+python at benfinney.id.au Sat Jan 30 21:39:49 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 31 Jan 2010 13:39:49 +1100 Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <0374c6d6$0$1309$c3e8da3@news.astraweb.com> Message-ID: <871vh7t2tm.fsf@benfinney.id.au> Steven D'Aprano writes: > Unfortunately the magic number doesn't seem to be documented anywhere > I can find other than in the source code (import.c). The PEP gives > some examples: > > f2b30a0d.pyc # Python 2.5 > f2d10a0d.pyc # Python 2.6 > f2d10a0d.pyo # Python 2.6 -O > f2d20a0d.pyc # Python 2.6 -U > 0c4f0a0d.pyc # Python 3.1 > > but how can one map magic numbers to versions, short of reading > import.c? Mapping magic numbers to versions is infeasible and will be incomplete: Any mapping that exists in (say) Python 3.1 can't know in advance what the magic number will be for Python 4.5. The more important mapping is from version to magic number. > I propose that sys grow an object sys.magic which is the hexlified magic > number. The ?imp? module already has this:: >>> import sys >>> import imp >>> sys.version '2.5.4 (r254:67916, Jan 24 2010, 16:09:54) \n[GCC 4.4.2]' >>> imp.get_magic().encode('hex') 'b3f20d0a' Unfortunately, I think the examples in the PEP have mangled the magic number into little-endian byte ordering. -- \ ?I got up the other day, and everything in my apartment has | `\ been stolen and replaced with an exact replica.? ?Steven Wright | _o__) | Ben Finney From no.email at nospam.invalid Sat Jan 30 21:40:42 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 30 Jan 2010 18:40:42 -0800 Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <0374c6d6$0$1309$c3e8da3@news.astraweb.com> <871vh7t2tm.fsf@benfinney.id.au> Message-ID: <7xljffngid.fsf@ruckus.brouhaha.com> Ben Finney writes: >> 0c4f0a0d.pyc # Python 3.1 > Mapping magic numbers to versions is infeasible and will be incomplete: > Any mapping that exists in (say) Python 3.1 can't know in advance what > the magic number will be for Python 4.5. But why do the filenames have magic numbers instead of version numbers? From python at mrabarnett.plus.com Sat Jan 30 21:45:24 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 31 Jan 2010 02:45:24 +0000 Subject: Still too slow In-Reply-To: References: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> <4B64CE82.1040109@optimum.net> Message-ID: <4B64EEC4.7010004@mrabarnett.plus.com> elsa wrote: > Hi John and others, > > sorry about my etiquette errors. As you can tell I'm a newbie, and > appreciate all the help I can get. I'm trying to master this thing > with only the net and a couple of books as tutors. > > Here is what I'm running at the interactive prompt: > [snip] > > def event(): > """ > randomly chooses a birth (prob .3), death (prob .1) or mutation > (prob .1) event > """ > choice = random.random() > if choice <= .3: > event='b' > elif choice <= .4: > event ='d' > elif choice<= .5: > event = 'm' > else: > event = 'None' > return event > [snip] Here's my line of thought: Currently, 0 <= choice < 1.0. If you multiplied the random number by 10 then the probability thresholds could all be ints, and if you're using ints then 'choice' could be an index into a list: event_choices = ['b', 'b', 'b', 'd', 'm'] + ['None'] * 5 which then suggests using random.choice() instead: def event(): """ randomly chooses a birth (prob .3), death (prob .1) or mutation (prob .1) event """ return random.choice(event_choices) and you could put the event code inline instead of in a function, which saves the cost of a function call. Incidentally, for a probability of 0.3 the test should be "choice < 0.3", etc, not "choice <= 0.3", etc, although I suspect that you won't notice any difference in the results! :-) From steve at REMOVE-THIS-cybersource.com.au Sat Jan 30 22:09:25 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 31 Jan 2010 03:09:25 GMT Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <0374c6d6$0$1309$c3e8da3@news.astraweb.com> <871vh7t2tm.fsf@benfinney.id.au> <7xljffngid.fsf@ruckus.brouhaha.com> Message-ID: <0374e0be$0$1309$c3e8da3@news.astraweb.com> On Sat, 30 Jan 2010 18:40:42 -0800, Paul Rubin wrote: > Ben Finney writes: >>> 0c4f0a0d.pyc # Python 3.1 >> Mapping magic numbers to versions is infeasible and will be incomplete: >> Any mapping that exists in (say) Python 3.1 can't know in advance what >> the magic number will be for Python 4.5. > > But why do the filenames have magic numbers instead of version numbers? The magic number changes with each incompatible change in the byte code format, which is not the same as each release. Selected values taken from import.c: Python 2.5a0: 62071 Python 2.5a0: 62081 (ast-branch) Python 2.5a0: 62091 (with) Python 2.5a0: 62092 (changed WITH_CLEANUP opcode) Python 2.5b3: 62101 (fix wrong code: for x, in ...) Python 2.5b3: 62111 (fix wrong code: x += yield) Python 2.5c1: 62121 (fix wrong lnotab with for loops and storing constants that should have been removed) Python 2.5c2: 62131 (fix wrong code: for x, in ... in listcomp/genexp) http://svn.python.org/view/python/trunk/Python/import.c?view=markup The relationship between byte code magic number and release version number is not one-to-one. We could have, for the sake of the argument, releases 3.2.3 through 3.5.0 (say) all having the same byte codes. What version number should the .pyc file show? -- Steven From davea at ieee.org Sat Jan 30 22:42:21 2010 From: davea at ieee.org (Dave Angel) Date: Sat, 30 Jan 2010 22:42:21 -0500 Subject: Still too slow In-Reply-To: <4B64CE82.1040109@optimum.net> References: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> <4B64CE82.1040109@optimum.net> Message-ID: <4B64FC1D.9040200@ieee.org> John Posner wrote: >
On > 1/30/2010 6:08 PM, elsa wrote: >> Hello again, >> >> Thanks for the tips r.e random.ranint(). This improved matters >> somewhat, however my program is still too slow. If anyone has any >> further tips on how to speed it up, they would be much appreciated! >> >> So, I'm calling evolve(L,limit) from the interactive prompt. L is >> initally [[100],['NA']]. Ideally limit would be 10^7. >> >> Here is my program: >> >> import random >> n=100 >> >> def evolve(L,limit): >> global n >> while n> evnt = event() >> if evnt!="None": >> ind = chooseInd(L,n) >> action(evnt,L,ind) >> >> def chooseInd(L,n): >> choiceSum=0 >> index=0 >> choice = random.randint(1,n) >> while choiceSum< choice: >> choiceSum+=L[index][0] >> index +=1 >> return (index-1) >> >> def event(): >> choice = random.random() >> if choice<= .3: >> event='b' >> elif choice<= .4: >> event ='d' >> elif choice<= .5: >> event = 'm' >> else: >> event = 'None' >> return event >> >> def action(event, L, index): >> global n >> if event == 'b': >> L[index][0]+=1 >> n +=1 >> elif event == 'd': >> L[index][0]-=1 >> n -=1 >> elif event == 'm': >> L.append([1,index]) >> n +=1 >> >> >> thanks in advance, >> >> Elsa. > > Elsa, > > 1. You changed the subject line from "For loop searching takes too > long!" to "Still too slow". This causes newsreader programs to start a > new discussion thread, which makes life difficult for people who need > to refer back to previous messages. Please don't change the subject > line any more. > > 2. You provided a very clear description of your original question: > > Now, what I need to do is randomly choose one myList[i], however the > distribution of my random choice needs to be proportional to the > values of myList[i][0]. > > This description should be the doc string for the chooseInd() function > -- for example: > > def chooseInd(L,n): > """ > randomly choose one L[i], so that the distribution > of choices is proportional to the values of L[i][0] > """ > > It is not clear (to me, anyway) what the other functions are trying to > accomplish. So please add a doc string to each of these functions, > with descriptions of similar clarity. This will help us a lot. And it > will help you, too, if you return to the code after a few > days/weeks/months of directing your attention elsewhere. > > 3. Please provide a *complete* transcript of an interactive session > that exercises this code. > > Tx, > John > > Remarks aimed at elsa, not John. You used tabs in your source code, which don't survive mail very well. So indentation is different in my quoted copy than presumably in your original. No bug there, but I much prefer having my editor expand all tabs into spaces (4 colums) You have a typo in the initialization of L. Try pasting it from a working session, instead of retyping it. Presumably you wanted L= [[100,'NA']]. rather than [[100],['NA']]. The latter value throws an exception in the code. You could start simply by changing the values in the event() function. No point in returning None half the time when that'll accomplish nothing. So just double the fractions you're checking against, or scale the random value. The pattern produced is interesting. The only count that gets very big is L[0][0]. ("Them what has, gets") Lots of the other counts reach zero, and once they do, they won't ever change. The algorithm is O(N*N), and it already takes a long time for 10**4. So 10**7 would be enormous. If the improvement needed were minor, you should profile it, figure where all the time is spent, and optimize that. But I think that's hopeless with this data structure. At a guess, I'd think you should build a single list that grows to size 'limit', where you start with 100 items of value "NA". No counts needed, because they're all implicitly count of 1. Then write your code to manipulate that list. When you finish, construct the list you really want, by using something like the groupby() function. DaveA From steve at REMOVE-THIS-cybersource.com.au Sat Jan 30 22:43:56 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 31 Jan 2010 03:43:56 GMT Subject: Function name unchanged in error message References: <50697b2c1001290610t1a2ac5c8hcd5f9856aa85392b@mail.gmail.com> <5303e556-e9df-4ef3-bd4d-fa4928399c6e@r19g2000yqb.googlegroups.com> Message-ID: <0374e8d5$0$1309$c3e8da3@news.astraweb.com> On Sat, 30 Jan 2010 14:26:43 -0800, andrew cooke wrote: > On Jan 29, 11:50?am, exar... at twistedmatrix.com wrote: >> new.function and new.code will let you construct new objects with >> different values (and copying over whichever existing attributes you >> want to preserve). > > unfortunately new is deprecated and dropped from 3. i can't see how the > same functionality is available in the types module for 3 - am i missing > something obvious? You have to get the constructor from an existing object. type(obj) will always return the type object, which you can use as a constructor. >>> newfunction = type(lambda: None) >>> newcode = type((lambda: None).__code__) Possibly even easier: >>> import types >>> types.FunctionType is newfunction True >>> types.CodeType is newcode True So that's two ways to get a constructor. Now all we need is to learn how they work: >>> help(newfunction) Help on class function in module builtins: class function(object) | function(code, globals[, name[, argdefs[, closure]]]) | | Create a function object from a code object and a dictionary. [...] >>> help(newcode) Help on class code in module builtins: class code(object) | code(argcount, kwonlyargcount, nlocals, stacksize, flags, codestring, | constants, names, varnames, filename, name, firstlineno, | lnotab[, freevars[, cellvars]]) | | Create a code object. Not for the faint of heart. [...] Not for the faint of heart indeed! The best way of using this is to copy the parameters from an existing code object, one created by using def. Look for attributes of f.__code__ starting with "co_". Don't forget compile as well, which may help. -- Steven From alfps at start.no Sat Jan 30 22:44:18 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 31 Jan 2010 04:44:18 +0100 Subject: PEP 3147 - new .pyc format In-Reply-To: <0374e0be$0$1309$c3e8da3@news.astraweb.com> References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <0374c6d6$0$1309$c3e8da3@news.astraweb.com> <871vh7t2tm.fsf@benfinney.id.au> <7xljffngid.fsf@ruckus.brouhaha.com> <0374e0be$0$1309$c3e8da3@news.astraweb.com> Message-ID: * Steven D'Aprano: > On Sat, 30 Jan 2010 18:40:42 -0800, Paul Rubin wrote: > >> Ben Finney writes: >>>> 0c4f0a0d.pyc # Python 3.1 >>> Mapping magic numbers to versions is infeasible and will be incomplete: >>> Any mapping that exists in (say) Python 3.1 can't know in advance what >>> the magic number will be for Python 4.5. >> But why do the filenames have magic numbers instead of version numbers? > > The magic number changes with each incompatible change in the byte code > format, which is not the same as each release. Selected values taken from > import.c: > > Python 2.5a0: 62071 > Python 2.5a0: 62081 (ast-branch) > Python 2.5a0: 62091 (with) > Python 2.5a0: 62092 (changed WITH_CLEANUP opcode) > Python 2.5b3: 62101 (fix wrong code: for x, in ...) > Python 2.5b3: 62111 (fix wrong code: x += yield) > Python 2.5c1: 62121 (fix wrong lnotab with for loops and > storing constants that should have been > removed) > Python 2.5c2: 62131 (fix wrong code: for x, in ... in > listcomp/genexp) > > > http://svn.python.org/view/python/trunk/Python/import.c?view=markup > > The relationship between byte code magic number and release version > number is not one-to-one. We could have, for the sake of the argument, > releases 3.2.3 through 3.5.0 (say) all having the same byte codes. What > version number should the .pyc file show? I don't know enough about Python yet to comment on your question, but, just an idea: how about a human readable filename /with/ some bytecode version id (that added id could be the magic number)? I think that combo could serve both the human and machine needs, so to speak. :-) Cheers, - Alf From steve at REMOVE-THIS-cybersource.com.au Sun Jan 31 00:03:35 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 31 Jan 2010 05:03:35 GMT Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <0374c6d6$0$1309$c3e8da3@news.astraweb.com> <871vh7t2tm.fsf@benfinney.id.au> <7xljffngid.fsf@ruckus.brouhaha.com> <0374e0be$0$1309$c3e8da3@news.astraweb.com> Message-ID: <0374fb80$0$1309$c3e8da3@news.astraweb.com> On Sun, 31 Jan 2010 04:44:18 +0100, Alf P. Steinbach wrote: >> The relationship between byte code magic number and release version >> number is not one-to-one. We could have, for the sake of the argument, >> releases 3.2.3 through 3.5.0 (say) all having the same byte codes. What >> version number should the .pyc file show? > > I don't know enough about Python yet to comment on your question, but, > just an idea: how about a human readable filename /with/ some bytecode > version id (that added id could be the magic number)? Sorry, that still doesn't work. Consider the hypothetical given above. For simplicity, I'm going to drop the micro point versions, so let's say that releases 3.2 through 3.5 all use the same byte-code. (In reality, you'll be likely looking at version numbers like 3.2.1 rather than just 3.2.) Now suppose you have 3.2 and 3.5 both installed, and you look inside your $PYTHONPATH and see this: spam.py spam.pyr/ 3.2-f2e70a0d.pyc It would be fairly easy to have the import machinery clever enough to ignore the version number prefix, so that Python 3.5 correctly uses 3.2-f2e70a0d.pyc. That part is easy. (I trust nobody is going to suggest that Python should create multiple redundant, identical, copies of the .pyc files. That would be just lame.) But now consider the human reader. You want human-readable file names for the benefit of the human reader. How is the human reader supposed to know that Python 3.5 is going to use 3.2-f2e70a0d.pyc? Suppose I'm running Python 3.5, and have a troubling bug, and I think "I know, maybe there's some sort of problem with the compiled byte code, I should delete it". I go looking for spam.pyr/3.5-*.pyc and don't find anything. Now I have two obvious choices: (1) Start worrying about why Python 3.5 isn't writing .pyc files. Is my installation broken? Have I set the PYTHONDONTWRITEBYTECODE environment variable? Do I not have write permission to the folder? WTF is going on? Confusion will reign. (2) Learn that the 3.2- prefix is meaningless, and ignore it. Since you have to ignore the version number prefix anyway, let's not lay a trap for people by putting it there. You need to know the magic number to do anything sensible with the .pyc files other than delete them, so let's not pretend otherwise. If you don't wish to spend time looking up the magic number, the solution is simple: hit it with a sledgehammer. Why do you care *which* specific .pyc file is being used anyway? rm -r spam.pyr/ or for the paranoid: rm spam.pyr/*.py[co] (Obviously there will be people who care about the specific .pyc file being deleted. Those people can't use a sledgehammer, they need one of those little tack-hammers with the narrow head. But 99% of the time, a sledgehammer is fine.) Frankly, unless you're a core developer or you're hacking byte-code, you nearly always won't care about the .pyc files. You want the compiler to manage them. And for the few times you do care, it isn't hard to find out: >>> import binascii, imp >>> binascii.hexlify(imp.get_magic()) b'4f0c0d0a' Stick that in your "snippets" folder (you do have a snippets folder, don't you?) and, as they say Down Under, "She'll be right mate". -- Steven From a.abushkevich at gmail.com Sun Jan 31 03:17:21 2010 From: a.abushkevich at gmail.com (Aliaksandr Abushkevich) Date: Sun, 31 Jan 2010 10:17:21 +0200 Subject: Slow down while creating a big list and iterating over it In-Reply-To: <2fdd4e1b-db58-45d7-9b7d-73dcdca6384a@c29g2000yqd.googlegroups.com> References: <2fdd4e1b-db58-45d7-9b7d-73dcdca6384a@c29g2000yqd.googlegroups.com> Message-ID: Maybe it is a good idea to use Disco (http://discoproject.org/) to process your data. Yours faithfully, Alexander Abushkevich On Sat, Jan 30, 2010 at 10:36 PM, marc magrans de abril wrote: > Dear colleagues, > > I was doing a small program to classify log files for a cluster of > PCs, I just wanted to simplify a quite repetitive task in order to > find errors and so. > > My first naive implementation was something like: > ? ?patterns = [] > ? ?while(logs): > ? ? ? ?pattern = logs[0] > ? ? ? ?new_logs = [l for l in logs if dist(pattern,l)>THERESHOLD] > ? ? ? ?entry = (len(logs)-len(new_logs),pattern) > ? ? ? ?patterns.append(entry) > ? ? ? ?logs = new_logs > > Where dist(...) is the levenshtein distance (i.e. edit distance) and > logs is something like 1.5M logs (700 MB file). I thought that python > will be an easy choice although not really fast.. > > I was not surprised when the first iteration of the while loop was > taking ~10min. I thought "not bad, let's how much it takes". However, > it seemed that the second iteration never finished. > > My surprise was big when I added a print instead of the list > comprehension: > new_logs=[] > for count,l in enumerate(logs): > ? print count > ? if dist(pattern,l)>THERESHOLD: > ? ? ?new_logs.append(l) > > The surprise was that the displayed counter was running ~10 times > slower on the second iteration of the while loop. > > I am a little lost. Anyone knows the reson of this behavior? ?How > should I write a program that deals with large data sets in python? > > Thanks a lot! > marc magrans de abril > -- > http://mail.python.org/mailman/listinfo/python-list > From buzzilo at gmail.com Sun Jan 31 04:47:37 2010 From: buzzilo at gmail.com (buzz) Date: Sun, 31 Jan 2010 01:47:37 -0800 (PST) Subject: pyjon: pythonic javascript interpreter Message-ID: <2758e72f-c818-4827-8dda-34cd19db0ef6@c4g2000yqa.googlegroups.com> hi, Here is: http://code.google.com/p/pyjon/ Although the code is far from stable and completed, it's a good place to start play with. -- Evgen From dickinsm at gmail.com Sun Jan 31 05:02:01 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Sun, 31 Jan 2010 02:02:01 -0800 (PST) Subject: unencountered error in FFT python References: <4b64800a$0$6735$9b4e6d93@newsspool2.arcor-online.net> <4b6486d6$0$6724$9b4e6d93@newsspool2.arcor-online.net> Message-ID: On Jan 30, 8:20?pm, uche wrote: > Another issue: > ? ? x[a], x[b] = x[(a)] + W[(n % N)] * x[(b)], x[(a)] - W[(n % N)] * x > [(b)] > TypeError: can't multiply sequence by non-int of type 'complex' With your original code, the elements of array2 are strings, and here Python is refusing to multiply the string x[a] by the complex number W [n % N]. Presumably you want the elements of array2 to be integers or floats instead? (Your comments suggest integers, though I think floats would be more natural here.) MRAB's earlier reply suggests how to fix this: array2 = [] for a in range(len(array)): array2.append(int(array[a])) # note the extra int! That works, but the code is cleaner if you iterate over the elements of the array directly, instead of over their indices: array2 = [] for x in array: array2.append(int(x)) There are other ways, too: a seasoned Python user would probably write this either as a list comprehension: array2 = [int(x) for x in array] ... or using the built-in map function: array2 = map(int, array) -- Mark From dickinsm at gmail.com Sun Jan 31 05:07:11 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Sun, 31 Jan 2010 02:07:11 -0800 (PST) Subject: unencountered error in FFT python References: <4b64800a$0$6735$9b4e6d93@newsspool2.arcor-online.net> <4b6486d6$0$6724$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <1546ea7e-6e22-4185-8aaf-0ce52caf4ed7@a5g2000yqi.googlegroups.com> On Jan 31, 10:02?am, Mark Dickinson wrote: > Python is refusing to multiply the string x[a] by the complex number W > [n % N]. Whoops, that should have been "x[b]", not "x[a]". Why is it that a post-submission proofread turns up errors so much more often than a pre-submission proofread? -- Mark From martin at v.loewis.de Sun Jan 31 05:13:06 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Sun, 31 Jan 2010 11:13:06 +0100 Subject: PEP 3147 - new .pyc format In-Reply-To: <877hqz2j1o.fsf@castleamber.com> References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <87iqaj9n8z.fsf@castleamber.com> <877hqz2j1o.fsf@castleamber.com> Message-ID: <4B6557B2.5040805@v.loewis.de> > Naming files using magic numbers is really beyond me. The fact that the > above needs comments to explain what's what already shows to me that > there's a problem with this naming scheme. What if for one reason or > another I want to delete all pyc files for Python 2.5? Where do I look > up the magic number? py> import imp, binascii py> binascii.hexlify(imp.get_magic()) 'b3f20d0a' (note: this is on a little-endian system) Regards, Martin From martin at v.loewis.de Sun Jan 31 05:17:19 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Sun, 31 Jan 2010 11:17:19 +0100 Subject: PEP 3147 - new .pyc format In-Reply-To: References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <87iqaj9n8z.fsf@castleamber.com> <877hqz2j1o.fsf@castleamber.com> Message-ID: <4B6558AF.1090009@v.loewis.de> > True. You might also want to note that "Python 2.6 -U" appears to have a > different magic number from "Python 2.6" and "Python 2.6 -O". > > I don't know whether they always change for each new version. Here is a recent list of magic numbers: Python 2.6a0: 62151 (peephole optimizations and STORE_MAP opcode) Python 2.6a1: 62161 (WITH_CLEANUP optimization) Python 2.7a0: 62171 (optimize list comprehensions/change LIST_APPEND) Python 2.7a0: 62181 (optimize conditional branches: introduce POP_JUMP_IF_FALSE and POP_JUMP_IF_TRUE) Python 2.7a0 62191 (introduce SETUP_WITH) Python 2.7a0 62201 (introduce BUILD_SET) Python 2.7a0 62211 (introduce MAP_ADD and SET_ADD) #define MAGIC (62211 | ((long)'\r'<<16) | ((long)'\n'<<24)) Regards, Martin From stefan_ml at behnel.de Sun Jan 31 05:35:43 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 31 Jan 2010 11:35:43 +0100 Subject: unencountered error in FFT python In-Reply-To: <1546ea7e-6e22-4185-8aaf-0ce52caf4ed7@a5g2000yqi.googlegroups.com> References: <4b64800a$0$6735$9b4e6d93@newsspool2.arcor-online.net> <4b6486d6$0$6724$9b4e6d93@newsspool2.arcor-online.net> <1546ea7e-6e22-4185-8aaf-0ce52caf4ed7@a5g2000yqi.googlegroups.com> Message-ID: <4b655cff$0$7617$9b4e6d93@newsspool1.arcor-online.net> Mark Dickinson, 31.01.2010 11:07: > On Jan 31, 10:02 am, Mark Dickinson wrote: >> Python is refusing to multiply the string x[a] by the complex number W >> [n % N]. > > Whoops, that should have been "x[b]", not "x[a]". Why is it that a > post-submission proofread turns up errors so much more often than a > pre-submission proofread? Because it's easier to find mistakes in other people's text than in your own ones. Once the message is sent off to the Aether, when it comes back, you read it like someone else has written it, so you immediately find all those obvious bugs in it. Fits into the same corner as rubber-duck debugging. Stefan From rantingrick at gmail.com Sun Jan 31 06:01:51 2010 From: rantingrick at gmail.com (rantingrick) Date: Sun, 31 Jan 2010 03:01:51 -0800 (PST) Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> Message-ID: <7536f9c0-a36a-479d-ba95-9405b13793fd@e25g2000yqh.googlegroups.com> On Jan 30, 10:43?am, Nobody wrote: > That's also true for most functional languages, e.g. Haskell and ML, as > well as e.g. Tcl and most shells. Why require "f(x)" or "(f x)" if "f x" > will suffice? yuck! wrapping the arg list with parenthesis (python way) makes the most sense. Its to easy to misread somthing like this onetwothree four five six onetwothree(four, five, six) #ahhh... plain english. From steve at REMOVE-THIS-cybersource.com.au Sun Jan 31 06:35:10 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 31 Jan 2010 11:35:10 GMT Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> <7536f9c0-a36a-479d-ba95-9405b13793fd@e25g2000yqh.googlegroups.com> Message-ID: <03755745$0$1309$c3e8da3@news.astraweb.com> On Sun, 31 Jan 2010 03:01:51 -0800, rantingrick wrote: > On Jan 30, 10:43?am, Nobody wrote: > >> That's also true for most functional languages, e.g. Haskell and ML, as >> well as e.g. Tcl and most shells. Why require "f(x)" or "(f x)" if "f >> x" will suffice? > > yuck! wrapping the arg list with parenthesis (python way) makes the most > sense. Its to easy to misread somthing like this > > onetwothree four five six > > onetwothree(four, five, six) #ahhh... plain english. I think the readability factor is mostly down to what you're familiar with. But consistency is also important: in Python, you always refer to an object the same way. Given an object called x, you ALWAYS refer to the object itself as x. In languages that don't use parentheses, x refers to the object, unless the object is a function, in which case x refers to the result of calling the object with no arguments. Other languages need special syntax to get access to the function object itself. Because it's hard to do, people don't do it often. But in Python, getting the function object is easy, and so treating functions as first- class objects is easy. -- Steven From mr.spoon21 at gmail.com Sun Jan 31 06:51:46 2010 From: mr.spoon21 at gmail.com (Mr.SpOOn) Date: Sun, 31 Jan 2010 11:51:46 +0000 Subject: Keyboard input In-Reply-To: References: <8f67b6f81001290853i39bab1acs49d02ac5be948d13@mail.gmail.com> Message-ID: <8f67b6f81001310351m3485fd40o8944c11af8a4161f@mail.gmail.com> 2010/1/29 Gabriel Genellina : > > That's strange. If you're using Linux, make sure you have the readline > package installed. I'm using Linux. Ubuntu. I checked on synaptic and I have readline-common. Do you mean something else? Anyway I tried running the program in the iPython shell and it works great. I mean, I can do backspace and other stuff. > If readline capabilities are not enough, you could try Urwid, a library for > writing console programs: http://excess.org/urwid/ - but it's a big move. > wxPython would let you write a graphical interface (and it's a bigger move!) Yes, I know about Urwid and wxPython and they're just too much. I was thinking about curses too, as suggested by others here, but that's too much too. From affdfsdfdsfsd at b.com Sun Jan 31 07:07:55 2010 From: affdfsdfdsfsd at b.com (Tracubik) Date: 31 Jan 2010 12:07:55 GMT Subject: create a string of variable lenght Message-ID: <4b65729b$0$1126$4fafbaef@reader3.news.tin.it> Hi all, i want to print on linux console (terminal) a message like this one: ******************************** error message of variable lenght ******************************** to print the asterisks line i do this: def StringOfAsterisks(myString): asterisksString = "*" for i in range(1,len(myString): asterisksString += "*" print asterisksString so i create the asterisksString of the lenght i need There is a better way of creating asterisksString? the code seem to work for me, but it doesn't work properly if in errorMsg there is some "esotic" char like euro char (?): >>> euro = "?" >>> len(euro) 3 how i can solve this? Thank you all in advance and sorry for my bad english Nico From e_d_k at yahoo.com Sun Jan 31 07:28:41 2010 From: e_d_k at yahoo.com (Ed Keith) Date: Sun, 31 Jan 2010 04:28:41 -0800 (PST) Subject: Python and Ruby In-Reply-To: <03755745$0$1309$c3e8da3@news.astraweb.com> Message-ID: <681892.91862.qm@web58903.mail.re1.yahoo.com> --- On Sun, 1/31/10, Steven D'Aprano wrote: > From: Steven D'Aprano > Subject: Re: Python and Ruby > To: python-list at python.org > Date: Sunday, January 31, 2010, 6:35 AM > On Sun, 31 Jan 2010 03:01:51 -0800, > rantingrick wrote: > > > On Jan 30, 10:43?am, Nobody > wrote: > > > >> That's also true for most functional languages, > e.g. Haskell and ML, as > >> well as e.g. Tcl and most shells. Why require > "f(x)" or "(f x)" if "f > >> x" will suffice? > > > > yuck! wrapping the arg list with parenthesis (python > way) makes the most > > sense. Its to easy to misread somthing like this > > > > onetwothree four five six > > > > onetwothree(four, five, six) #ahhh... plain english. > > I think the readability factor is mostly down to what > you're familiar > with. But consistency is also important: in Python, you > always refer to > an object the same way. Given an object called x, you > ALWAYS refer to the > object itself as x. In languages that don't use > parentheses, x refers to > the object, unless the object is a function, in which case > x refers to > the result of calling the object with no arguments. > > Other languages need special syntax to get access to the > function object > itself. Because it's hard to do, people don't do it often. > But in Python, > getting the function object is easy, and so treating > functions as first- > class objects is easy. > In most functional languages you just name a function to access it and you do it ALL the time. for example, in if you have a function 'f' which takes two parameters to call the function and get the result you use: f 2 3 If you want the function itself you use: f The reason no parentheses are used is to support Currying (http://en.wikipedia.org/wiki/Currying). To get a new function which is equivalent to f with the first parameter set to a constant 2 you use: f 2 this give you a function which take only one parameter. Using parenthesis make currying more complicated, so most functional languages do not use them. It did take me a LONG time to get used to this, but it is only syntax, I do not let syntax bother me. Semantics on the other hand, are a big deal. -EdK Ed Keith e_d_k at yahoo.com Blog: edkeith.blogspot.com > From python.list at tim.thechases.com Sun Jan 31 07:35:31 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Sun, 31 Jan 2010 06:35:31 -0600 Subject: create a string of variable lenght In-Reply-To: <4b65729b$0$1126$4fafbaef@reader3.news.tin.it> References: <4b65729b$0$1126$4fafbaef@reader3.news.tin.it> Message-ID: <4B657913.5010303@tim.thechases.com> Tracubik wrote: > ******************************** > error message of variable lenght > ******************************** > > to print the asterisks line i do this: > > def StringOfAsterisks(myString): > asterisksString = "*" > for i in range(1,len(myString): > asterisksString += "*" > print asterisksString > > so i create the asterisksString of the lenght i need > > There is a better way of creating asterisksString? well, to make it more pythonic (ignoring camel-case variable-names for now), I'd just use print '*' * len(myString) print myString print '*' * len(myString) possibly stashing the resulting asterisksString once: asterisks_string = '*' * len(my_string) print asterisks_string print my_string print asterisks_string If I used it in multiple places, I might wrap it in a function and/or define a "DIVIDER_CHARACTER" constant, something like DIVIDER_CHARACTER = '*' def surround(s, divider=DIVIDER_CHARACTER): d = divider[0] print d * len(s) print s print d * len(s) surround('hello') surround('world', '-') surround('foo', 'xo') depending on the sort of behavior you want > the code seem to work for me, but it doesn't work properly if in errorMsg > there is some "esotic" char like euro char (?): > >>>> euro = "?" >>>> len(euro) > 3 I suspect you're seeing the byte-representation of your string with a particular-yet-undisclosed encoding. If it was a unicode string (a good idea to specify the encoding at the top of your file), the length should be accurate, so what happens if you >>> euro = u"?" >>> len(euro) ? (I don't have ready access to a terminal where I can enter unicode characters, and you don't display the representation of the string with print repr(euro) so I can steal the byte values to recreate it; but I suspect the result will correctly be 1). -tkc From __peter__ at web.de Sun Jan 31 07:41:04 2010 From: __peter__ at web.de (Peter Otten) Date: Sun, 31 Jan 2010 13:41:04 +0100 Subject: create a string of variable lenght References: <4b65729b$0$1126$4fafbaef@reader3.news.tin.it> Message-ID: Tracubik wrote: > Hi all, > > i want to print on linux console (terminal) a message like this one: > > ******************************** > error message of variable lenght > ******************************** > > to print the asterisks line i do this: > > def StringOfAsterisks(myString): > asterisksString = "*" > for i in range(1,len(myString): > asterisksString += "*" > print asterisksString > > so i create the asterisksString of the lenght i need > > There is a better way of creating asterisksString? > >>> "*" * 10 '**********' > the code seem to work for me, but it doesn't work properly if in errorMsg > there is some "esotic" char like euro char (?): > >>>> euro = "?" >>>> len(euro) > 3 > > how i can solve this? This is less likely to fail when you use unicode strings: >>> def print_message(s): ... print "*" * len(s) ... print s ... print "*" * len(s) ... >>> print_message(u"You are leaving the ? zone") ************************** You are leaving the ? zone ************************** Peter From gd_usenet at spamfence.net Sun Jan 31 07:46:16 2010 From: gd_usenet at spamfence.net (=?UTF-8?Q?G=C3=BCnther?= Dietrich) Date: Sun, 31 Jan 2010 13:46:16 +0100 Subject: create a string of variable lenght References: <4b65729b$0$1126$4fafbaef@reader3.news.tin.it> Message-ID: Tracubik wrote: >i want to print on linux console (terminal) a message like this one: > >******************************** >error message of variable lenght >******************************** > >to print the asterisks line i do this: > >def StringOfAsterisks(myString): > asterisksString = "*" > for i in range(1,len(myString): > asterisksString += "*" > print asterisksString def StringOfAsterisks(myString): print(''.ljust(len(myString), '*')) You might use string methods .rjust() or .center() instead of method .ljust(). >the code seem to work for me, but it doesn't work properly if in errorMsg >there is some "esotic" char like euro char (?): > >>>> euro = "?" >>>> len(euro) >3 Maybe you might solve this if you decode your string to unicode. Example: |>>> euro = "?" |>>> len(euro) |3 |>>> u_euro = euro.decode('utf_8') |>>> len(u_euro) |1 Adapt the encoding ('utf_8' in my example) to whatever you use. Or create the unicode string directly: |>>> u_euro = u'?' |>>> len(u_euro) |1 Best regards, G?nther From vicente.soler at gmail.com Sun Jan 31 09:03:14 2010 From: vicente.soler at gmail.com (vsoler) Date: Sun, 31 Jan 2010 06:03:14 -0800 (PST) Subject: Unable to install numpy References: <3a772f49-aee6-4450-b60e-8f05daec136a@21g2000yqj.googlegroups.com> Message-ID: On Jan 18, 9:08?pm, Robert Kern wrote: > On 2010-01-18 14:02 PM, vsoler wrote: > > > Hi all, > > > I just download Numpy, and tried to install it using "numpy-1.4.0- > > win32-superpack-python2.6.exe" > > > I get an error: ?"Python version 2.6 required, which was not found in > > the Registry" > > > However, I am using Python 2.6 every day. I'm running Windows 7. > > > What can I do? > > Please ask numpy installation questions on the numpy mailing list. > > ? ?http://www.scipy.org/Mailing_Lists > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless enigma > ? that is made terrible by our own mad attempt to interpret it as though it had > ? an underlying truth." > ? ?-- Umberto Eco Thank you Robert. I'm going to direct my questions towards scipy.org. However, since I do not find my questions already answered, could you tell me where to post it? I beg your pardon for still asking this question outside of scipy.org, but I've been unable to move on. Regards From andrej.mitrovich at gmail.com Sun Jan 31 09:13:44 2010 From: andrej.mitrovich at gmail.com (Andrej Mitrovic) Date: Sun, 31 Jan 2010 06:13:44 -0800 (PST) Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <87iqaj9n8z.fsf@castleamber.com> <877hqz2j1o.fsf@castleamber.com> <4B6558AF.1090009@v.loewis.de> Message-ID: Leave magic to the witches of Perl. :) From marcmagransdeabril at gmail.com Sun Jan 31 09:53:15 2010 From: marcmagransdeabril at gmail.com (marc magrans de abril) Date: Sun, 31 Jan 2010 06:53:15 -0800 (PST) Subject: Slow down while creating a big list and iterating over it References: <2fdd4e1b-db58-45d7-9b7d-73dcdca6384a@c29g2000yqd.googlegroups.com> Message-ID: <29a69a7c-a8a2-4c53-9521-760e13665cd3@f12g2000yqn.googlegroups.com> >Find out which pattern is being used on the second iteration and then try it on the first iteration. Is it just as slow? You were right, the second pattern was 1891 bytes but the first was just 142 :P I will need to put more thought than I expect in the "small script". From john at castleamber.com Sun Jan 31 10:06:18 2010 From: john at castleamber.com (John Bokma) Date: Sun, 31 Jan 2010 09:06:18 -0600 Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <0374c6d6$0$1309$c3e8da3@news.astraweb.com> <871vh7t2tm.fsf@benfinney.id.au> <7xljffngid.fsf@ruckus.brouhaha.com> <0374e0be$0$1309$c3e8da3@news.astraweb.com> <0374fb80$0$1309$c3e8da3@news.astraweb.com> Message-ID: <878wbenwk5.fsf@castleamber.com> Steven D'Aprano writes: > On Sun, 31 Jan 2010 04:44:18 +0100, Alf P. Steinbach wrote: > >>> The relationship between byte code magic number and release version >>> number is not one-to-one. We could have, for the sake of the argument, >>> releases 3.2.3 through 3.5.0 (say) all having the same byte codes. What >>> version number should the .pyc file show? >> >> I don't know enough about Python yet to comment on your question, but, >> just an idea: how about a human readable filename /with/ some bytecode >> version id (that added id could be the magic number)? > > Sorry, that still doesn't work. Consider the hypothetical given above. > For simplicity, I'm going to drop the micro point versions, so let's say > that releases 3.2 through 3.5 all use the same byte-code. (In reality, Based on the magic numbers I've seen so far it looks like that not an option. They increment with every minor change. So to me, at this moment (and maybe it's my ignorance) it looks like a made up example to justify what to me still looks like a bad decision. -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From skip at pobox.com Sun Jan 31 10:13:19 2010 From: skip at pobox.com (skip at pobox.com) Date: Sun, 31 Jan 2010 09:13:19 -0600 Subject: How to get UTC offset for non-standard time zone names? Message-ID: <19301.40463.29743.197241@montanaro.dyndns.org> I have the pytz package but it doesn't know about non-standard timezone names like "CDT5" or "CST6". I can obviously infer that they are either five or six hours behind UTC. Are they constructed in some standard way so that I can assume that if a timezone name is not known to pytz I can assume the trailing number represents the number of hours behind UTC? Is there some standard definition of the format of such semi-numeric timezone names? Thx, -- Skip Montanaro - skip at pobox.com - http://www.smontanaro.net/ From contact at xavierho.com Sun Jan 31 10:23:29 2010 From: contact at xavierho.com (Xavier Ho) Date: Mon, 1 Feb 2010 01:23:29 +1000 Subject: How to get UTC offset for non-standard time zone names? In-Reply-To: <19301.40463.29743.197241@montanaro.dyndns.org> References: <19301.40463.29743.197241@montanaro.dyndns.org> Message-ID: <2d56febf1001310723n1010a4f6t58d760c9e47ad7ce@mail.gmail.com> This may not answer your question directly, but have you thought about ingoring the number at the end of these non-standard timezones? CDT is Central Daylight-saving Timezone, while CST is Central Standard Timezone. And you are correct they are -5 and -6 hours respectively. Does pytz know about CDT and CST? Cheers, -Xav On Mon, Feb 1, 2010 at 1:13 AM, wrote: > > I have the pytz package but it doesn't know about non-standard timezone > names like "CDT5" or "CST6". I can obviously infer that they are either > five or six hours behind UTC. Are they constructed in some standard way so > that I can assume that if a timezone name is not known to pytz I can assume > the trailing number represents the number of hours behind UTC? Is there > some standard definition of the format of such semi-numeric timezone names? > > Thx, > > -- > Skip Montanaro - skip at pobox.com - http://www.smontanaro.net/ > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From no.email at please.post Sun Jan 31 10:38:04 2010 From: no.email at please.post (kj) Date: Sun, 31 Jan 2010 15:38:04 +0000 (UTC) Subject: How to set default encoding for print? Message-ID: It gets tedious to have to append .encode('utf-8') to all my unicode strings when I print them, as in: print foobar.encode('utf-8') I want to tell python to apply this encoding automatically to anything argument passed to print. How can I do this? TIA! K PS: BTW, sys.setdefaultencoding is *not* the answer, since this method gets removed by site.py... From no.email at please.post Sun Jan 31 10:52:28 2010 From: no.email at please.post (kj) Date: Sun, 31 Jan 2010 15:52:28 +0000 (UTC) Subject: How to pass Chinese characters as command-line arguments? Message-ID: I want to pass Chinese characters as command-line arguments to a Python script. My terminal has no problem displaying these characters, and passing them to the script, but I can't get Python to understand them properly. E.g. if I pass one such character to the simple script import sys print sys.argv[1] print type(sys.argv[1]) the first line of the output looks fine (identical to the input), but the second line says "". If I add the line arg = unicode(sys.argv[1]) I get the error Traceback (most recent call last): File "kgrep.py", line 4, in arg = unicode(sys.argv[1]) UnicodeDecodeError: 'ascii' codec can't decode byte 0xe8 in position 0: ordinal not in range(128) What must I do to get Python to recognize command-line arguments as utf-8 Unicode? FWIW, my os is Darwin, my shell (zsh) runs on Terminal, and none of my locale (LC_*) variables is set. TIA! kynn From deets at nospam.web.de Sun Jan 31 10:54:23 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Sun, 31 Jan 2010 16:54:23 +0100 Subject: How to set default encoding for print? In-Reply-To: References: Message-ID: <7slndhFnomU1@mid.uni-berlin.de> Am 31.01.10 16:38, schrieb kj: > It gets tedious to have to append .encode('utf-8') to all my unicode > strings when I print them, as in: > > print foobar.encode('utf-8') > > I want to tell python to apply this encoding automatically to > anything argument passed to print. > > How can I do this? > > TIA! > > K > > PS: BTW, sys.setdefaultencoding is *not* the answer, since this > method gets removed by site.py... Just wrap sys.stdout in a codecs-module based file-wrapper. import codecs import sys sys.stdout = codecs.getwriter("utf-8")(sys.stdout) print u"?" Diez From skip at pobox.com Sun Jan 31 10:57:31 2010 From: skip at pobox.com (Skip Montanaro) Date: Sun, 31 Jan 2010 15:57:31 +0000 (UTC) Subject: How to get UTC offset for non-standard time zone names? References: <19301.40463.29743.197241@montanaro.dyndns.org> <2d56febf1001310723n1010a4f6t58d760c9e47ad7ce@mail.gmail.com> Message-ID: > Does pytz know about CDT and CST? Nope... Skip From kyp at stsci.edu Sun Jan 31 11:22:42 2010 From: kyp at stsci.edu (Kyp) Date: Sun, 31 Jan 2010 08:22:42 -0800 (PST) Subject: iglob performance no better than glob Message-ID: <5ebd4965-32fc-4a03-94dd-c61ccdc9d630@f5g2000vbm.googlegroups.com> I have a dir with a large # of files that I need to perform operations on, but only needing to access a subset of the files, i.e. the first 100 files. Using glob is very slow, so I ran across iglob, which returns an iterator, which seemed just like what I wanted. I could iterate over the files that I wanted, not having to read the entire dir. So the iglob was faster, but accessing the first file took about the same time as glob.glob. Here's some code to compare glob vs. iglob performance, it outputs the time before/after a glob.iglob('*.*') files.next() sequence and a glob.glob('*.*') sequence. #!/usr/bin/env python import glob,time print '\nTest of glob.iglob' print 'before iglob:', time.asctime() files = glob.iglob('*.*') print 'after iglob:',time.asctime() print files.next() print 'after files.next():', time.asctime() print '\nTest of glob.glob' print 'before glob:', time.asctime() files = glob.glob('*.*') print 'after glob:',time.asctime() Here are the results: Test of glob.iglob before iglob: Sun Jan 31 11:09:08 2010 after iglob: Sun Jan 31 11:09:08 2010 foo.bar after files.next(): Sun Jan 31 11:09:59 2010 Test of glob.glob before glob: Sun Jan 31 11:09:59 2010 after glob: Sun Jan 31 11:10:51 2010 The results are about the same for the 2 approaches, both took about 51 seconds. Am I doing something wrong with iglob? Is there a way to get the first X # of files from a dir with lots of files, that does not take a long time to run? thanx, mark From no.email at please.post Sun Jan 31 11:28:55 2010 From: no.email at please.post (kj) Date: Sun, 31 Jan 2010 16:28:55 +0000 (UTC) Subject: How to set default encoding for print? References: <7slndhFnomU1@mid.uni-berlin.de> Message-ID: In <7slndhFnomU1 at mid.uni-berlin.de> "Diez B. Roggisch" writes: >Am 31.01.10 16:38, schrieb kj: >> It gets tedious to have to append .encode('utf-8') to all my unicode >> strings when I print them, as in: >> >> print foobar.encode('utf-8') >> >> I want to tell python to apply this encoding automatically to >> anything argument passed to print. >> >> How can I do this? >> >> TIA! >> >> K >> >> PS: BTW, sys.setdefaultencoding is *not* the answer, since this >> method gets removed by site.py... >Just wrap sys.stdout in a codecs-module based file-wrapper. >import codecs >import sys >sys.stdout = codecs.getwriter("utf-8")(sys.stdout) >print u"??" That's exactly what I was looking for. Thanks! kynn From contact at xavierho.com Sun Jan 31 11:38:16 2010 From: contact at xavierho.com (Xavier Ho) Date: Mon, 1 Feb 2010 02:38:16 +1000 Subject: How to get UTC offset for non-standard time zone names? In-Reply-To: References: <19301.40463.29743.197241@montanaro.dyndns.org> <2d56febf1001310723n1010a4f6t58d760c9e47ad7ce@mail.gmail.com> Message-ID: <2d56febf1001310838ub9bc0e5n4d290b34626d2740@mail.gmail.com> Would it hurt if you put in some extra information? http://www.timeanddate.com/library/abbreviations/timezones/ HTH, -Xav P.S: You, sir, have an awesome first name. On Mon, Feb 1, 2010 at 1:57 AM, Skip Montanaro wrote: > > Does pytz know about CDT and CST? > > Nope... > > Skip > > > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From deets at nospam.web.de Sun Jan 31 11:58:25 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Sun, 31 Jan 2010 17:58:25 +0100 Subject: How to pass Chinese characters as command-line arguments? In-Reply-To: References: Message-ID: <7slr5iFe66U1@mid.uni-berlin.de> Am 31.01.10 16:52, schrieb kj: > I want to pass Chinese characters as command-line arguments to a > Python script. My terminal has no problem displaying these > characters, and passing them to the script, but I can't get Python > to understand them properly. > > E.g. if I pass one such character to the simple script > > import sys > print sys.argv[1] > print type(sys.argv[1]) > > the first line of the output looks fine (identical to the input), > but the second line says "". If I add the line > > arg = unicode(sys.argv[1]) > > I get the error > > Traceback (most recent call last): > File "kgrep.py", line 4, in > arg = unicode(sys.argv[1]) > UnicodeDecodeError: 'ascii' codec can't decode byte 0xe8 in position 0: ordinal not in range(128) > > What must I do to get Python to recognize command-line arguments > as utf-8 Unicode? The last sentence reveals your problem: utf-8 is *not* unicode. It's an encoding of unicode, which is a crucial difference. From the outside you get byte-streams, and if these happen to be encoded in utf-8, you can simply decode them: arg = unicode(sys.argv[1], "utf-8") Diez From skip at pobox.com Sun Jan 31 12:29:57 2010 From: skip at pobox.com (Skip Montanaro) Date: Sun, 31 Jan 2010 17:29:57 +0000 (UTC) Subject: How to get UTC offset for non-standard time zone names? References: <19301.40463.29743.197241@montanaro.dyndns.org> <2d56febf1001310723n1010a4f6t58d760c9e47ad7ce@mail.gmail.com> <2d56febf1001310838ub9bc0e5n4d290b34626d2740@mail.gmail.com> Message-ID: > Would it hurt if you put in some extra information? > http://www.timeanddate.com/library/abbreviations/timezones/ In theory, no. At work we still use the ancient Rogue Wave C++ libraries in a number of applications. It has hard-coded timezone info so when the US changed the start and end of daylight savings a few years ago we adapted by switching from US/Central to either CDT5 or CST6 for a week around each entry/exit. For those couple weeks our TZ environment variable is set to one of those two values. I could hard-code a similar exception in my Python code, but it also has to run in London and New York (at least). In the past we've had offices in Hawaii and Asia, so I'd really like it if my code was location-independent. Skip P.S. Thanks for the compliment on my nickname. My first name is actually George. Still impressed? ;-) From skip at pobox.com Sun Jan 31 12:59:49 2010 From: skip at pobox.com (Skip Montanaro) Date: Sun, 31 Jan 2010 17:59:49 +0000 (UTC) Subject: iglob performance no better than glob References: <5ebd4965-32fc-4a03-94dd-c61ccdc9d630@f5g2000vbm.googlegroups.com> Message-ID: > So the iglob was faster, but accessing the first file took about the > same time as glob.glob. I'll wager most of the time required to access the first file is due to filesystem overhead, not any inherent limitation in Python. Skip Montanaro From john at castleamber.com Sun Jan 31 13:06:29 2010 From: john at castleamber.com (John Bokma) Date: Sun, 31 Jan 2010 12:06:29 -0600 Subject: iglob performance no better than glob References: <5ebd4965-32fc-4a03-94dd-c61ccdc9d630@f5g2000vbm.googlegroups.com> Message-ID: <87k4uyw3mi.fsf@castleamber.com> Kyp writes: > Is there a way to get the first X # of files from a dir with lots of > files, that does not take a long time to run? Assuming Linux: what does time ls thedir | head give? with thedir the name of the actual dir Also how many is many files? -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From wolftracks at invalid.com Sun Jan 31 13:10:53 2010 From: wolftracks at invalid.com (W. eWatson) Date: Sun, 31 Jan 2010 10:10:53 -0800 Subject: What's the Scoop on \\ for Paths? (Win) Message-ID: I'm sure that \\ is used in some way for paths in Win Python, but I have not found anything after quite a search. I even have a six page pdf on a file tutorial. Nothing. Two books. Nothing. When I try to open a file along do I need, for example, "Events\\record\\year\\today"? Are paths like, ".\\Events" allowed, or am I mixing up my Linux memory on this? From tinnews at isbd.co.uk Sun Jan 31 13:15:32 2010 From: tinnews at isbd.co.uk (tinnews at isbd.co.uk) Date: Sun, 31 Jan 2010 18:15:32 +0000 Subject: Can't get sys.stdin.readlines() to work Message-ID: <4vtg37-n89.ln1@chris.isbd.net> I'm trying to read some data from standard input, what I'm actually trying to do is process some date pasted in using the mouse cut and paste on a Linux box (xubuntu 9.10) in a terminal window. First attempts failed so I'm now trying the trivial:- import sys data = sys.stdin.readlines() print "Counted", len(data), "lines." When I run this and try to paste something into the terminal window I get the following errors:- /home/chris/bin/m2r.py: line 2: syntax error near unexpected token `(' /home/chris/bin/m2r.py: line 2: `data = sys.stdin.readlines()' It does exactly the same if I try:- cat | m2r.py and then paste something into the window. So - what on earth am I doing wrong? -- Chris Green From alfps at start.no Sun Jan 31 13:23:35 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 31 Jan 2010 19:23:35 +0100 Subject: What's the Scoop on \\ for Paths? (Win) In-Reply-To: References: Message-ID: * W. eWatson: > I'm sure that \\ is used in some way for paths in Win Python, but I have > not found anything after quite a search. I even have a six page pdf on a > file tutorial. Nothing. Two books. Nothing. When I try to open a file > along do I need, for example, "Events\\record\\year\\today"? Are paths > like, ".\\Events" allowed, or am I mixing up my Linux memory on this? The Python issue with \\ is that in a literal string \\ denotes a single \ character, like >>> print( "back\\slash" ) back\slash >>> _ This is just like in other languages with syntax inherited from C. Look up "escape sequences". It has nothing to do with files and paths per se, but means that you cannot write e.g. "c:\windows\system32", but must write something like "c:\\windows\\system32" (try to print that string), or, since Windows handles forward slashes as well, you can write "c:/windows/system32" :-). The Window issue with \\ is that \\ as a path prefix denotes an UNC (Universal Naming Convention) path. Usually that would be a LAN or WAN network path, but it can also denote a printer or a pipe or a mailslot or just about anything. Using UNC paths opens the door to creating files and directories that other programs won't be able to handle, so Just Say No(TM), if you can. Cheers & hth., - Alf From steve at holdenweb.com Sun Jan 31 13:26:16 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 31 Jan 2010 13:26:16 -0500 Subject: What's the Scoop on \\ for Paths? (Win) In-Reply-To: References: Message-ID: <4B65CB48.1040409@holdenweb.com> W. eWatson wrote: > I'm sure that \\ is used in some way for paths in Win Python, but I have > not found anything after quite a search. I even have a six page pdf on a > file tutorial. Nothing. Two books. Nothing. When I try to open a file > along do I need, for example, "Events\\record\\year\\today"? Are paths > like, ".\\Events" allowed, or am I mixing up my Linux memory on this? You need to read up on string literals is all. "\\" is simply the literal representation of a string containing a single backslash. This comes about because string literals are allowed to contain special "escape sequences" which are introduced by a backslash; since this gives the backslash a special meaning in string literals we also have to use an escape sequence ("\\") to represent a backslash. In practice you will find that a) Many Windows APIs (but not the command line) are just as happy with a forward slash as a backslash to separate file path components; and b) The best practice is to build filenames using the routines provided in the os.path module, which guarantees to give results correct for the current platform. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Sun Jan 31 13:26:16 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 31 Jan 2010 13:26:16 -0500 Subject: What's the Scoop on \\ for Paths? (Win) In-Reply-To: References: Message-ID: <4B65CB48.1040409@holdenweb.com> W. eWatson wrote: > I'm sure that \\ is used in some way for paths in Win Python, but I have > not found anything after quite a search. I even have a six page pdf on a > file tutorial. Nothing. Two books. Nothing. When I try to open a file > along do I need, for example, "Events\\record\\year\\today"? Are paths > like, ".\\Events" allowed, or am I mixing up my Linux memory on this? You need to read up on string literals is all. "\\" is simply the literal representation of a string containing a single backslash. This comes about because string literals are allowed to contain special "escape sequences" which are introduced by a backslash; since this gives the backslash a special meaning in string literals we also have to use an escape sequence ("\\") to represent a backslash. In practice you will find that a) Many Windows APIs (but not the command line) are just as happy with a forward slash as a backslash to separate file path components; and b) The best practice is to build filenames using the routines provided in the os.path module, which guarantees to give results correct for the current platform. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From chardster at gmail.com Sun Jan 31 13:26:26 2010 From: chardster at gmail.com (Richard Thomas) Date: Sun, 31 Jan 2010 10:26:26 -0800 (PST) Subject: Can't get sys.stdin.readlines() to work References: <4vtg37-n89.ln1@chris.isbd.net> Message-ID: <3a3c2ba0-7221-4402-9ed6-0b74e101f95b@m16g2000yqc.googlegroups.com> On Jan 31, 6:15?pm, tinn... at isbd.co.uk wrote: > I'm trying to read some data from standard input, what I'm actually > trying to do is process some date pasted in using the mouse cut and > paste on a Linux box (xubuntu 9.10) in a terminal window. > > First attempts failed so I'm now trying the trivial:- > > ? ? import sys > ? ? data = sys.stdin.readlines() > ? ? print "Counted", len(data), "lines." > > When I run this and try to paste something into the terminal window I > get the following errors:- > > ? ? /home/chris/bin/m2r.py: line 2: syntax error near unexpected token `(' > ? ? /home/chris/bin/m2r.py: line 2: `data = sys.stdin.readlines()' > > It does exactly the same if I try:- > > ? ? cat | m2r.py > > and then paste something into the window. > > So - what on earth am I doing wrong? > > -- > Chris Green You haven't put a shebang line at the top. #!/usr/bin/env python import sys data = sys.stdin.readlines() ... From steve at holdenweb.com Sun Jan 31 13:36:55 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 31 Jan 2010 13:36:55 -0500 Subject: Can't get sys.stdin.readlines() to work In-Reply-To: <4vtg37-n89.ln1@chris.isbd.net> References: <4vtg37-n89.ln1@chris.isbd.net> Message-ID: <4B65CDC7.5080305@holdenweb.com> tinnews at isbd.co.uk wrote: > I'm trying to read some data from standard input, what I'm actually > trying to do is process some date pasted in using the mouse cut and > paste on a Linux box (xubuntu 9.10) in a terminal window. > > First attempts failed so I'm now trying the trivial:- > > import sys > data = sys.stdin.readlines() > print "Counted", len(data), "lines." > > > When I run this and try to paste something into the terminal window I > get the following errors:- > > /home/chris/bin/m2r.py: line 2: syntax error near unexpected token `(' > /home/chris/bin/m2r.py: line 2: `data = sys.stdin.readlines()' > > > It does exactly the same if I try:- > > cat | m2r.py > > and then paste something into the window. > > > So - what on earth am I doing wrong? > > Trying to run Python scripts with /bin/sh, probably. First of all, does you PATH include the current directory (.)? It looks like it does, because your computer seems to be trying to execute the m2r.py script. However, because that script does not start with a "shebang" line - the one you need is probably either #!/usr/bin/python or #!/usr/bin/env python the shell is trying to run the script using the wrong interpreter. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Sun Jan 31 13:36:55 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 31 Jan 2010 13:36:55 -0500 Subject: Can't get sys.stdin.readlines() to work In-Reply-To: <4vtg37-n89.ln1@chris.isbd.net> References: <4vtg37-n89.ln1@chris.isbd.net> Message-ID: <4B65CDC7.5080305@holdenweb.com> tinnews at isbd.co.uk wrote: > I'm trying to read some data from standard input, what I'm actually > trying to do is process some date pasted in using the mouse cut and > paste on a Linux box (xubuntu 9.10) in a terminal window. > > First attempts failed so I'm now trying the trivial:- > > import sys > data = sys.stdin.readlines() > print "Counted", len(data), "lines." > > > When I run this and try to paste something into the terminal window I > get the following errors:- > > /home/chris/bin/m2r.py: line 2: syntax error near unexpected token `(' > /home/chris/bin/m2r.py: line 2: `data = sys.stdin.readlines()' > > > It does exactly the same if I try:- > > cat | m2r.py > > and then paste something into the window. > > > So - what on earth am I doing wrong? > > Trying to run Python scripts with /bin/sh, probably. First of all, does you PATH include the current directory (.)? It looks like it does, because your computer seems to be trying to execute the m2r.py script. However, because that script does not start with a "shebang" line - the one you need is probably either #!/usr/bin/python or #!/usr/bin/env python the shell is trying to run the script using the wrong interpreter. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From wolftracks at invalid.com Sun Jan 31 13:42:25 2010 From: wolftracks at invalid.com (W. eWatson) Date: Sun, 31 Jan 2010 10:42:25 -0800 Subject: What's the Scoop on \\ for Paths? (Win) In-Reply-To: References: Message-ID: Alf P. Steinbach wrote: > * W. eWatson: >> I'm sure that \\ is used in some way for paths in Win Python, but I >> have not found anything after quite a search. I even have a six page >> pdf on a file tutorial. Nothing. Two books. Nothing. When I try to >> open a file along do I need, for example, >> "Events\\record\\year\\today"? Are paths like, ".\\Events" allowed, or >> am I mixing up my Linux memory on this? > > The Python issue with \\ is that in a literal string \\ denotes a single > \ character, like > > >>> print( "back\\slash" ) > back\slash > >>> _ > > This is just like in other languages with syntax inherited from C. Look > up "escape sequences". It has nothing to do with files and paths per se, > but means that you cannot write e.g. "c:\windows\system32", but must > write something like "c:\\windows\\system32" (try to print that string), > or, since Windows handles forward slashes as well, you can write > "c:/windows/system32" :-). > > The Window issue with \\ is that \\ as a path prefix denotes an UNC > (Universal Naming Convention) path. Usually that would be a LAN or WAN > network path, but it can also denote a printer or a pipe or a mailslot > or just about anything. Using UNC paths opens the door to creating files > and directories that other programs won't be able to handle, so Just Say > No(TM), if you can. > > > Cheers & hth., > > - Alf Ah, yes. Thanks for the memory jog. From invalid at invalid.invalid Sun Jan 31 13:50:56 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Sun, 31 Jan 2010 18:50:56 +0000 (UTC) Subject: Can't get sys.stdin.readlines() to work References: <4vtg37-n89.ln1@chris.isbd.net> Message-ID: On 2010-01-31, Steve Holden wrote: > tinnews at isbd.co.uk wrote: >> I'm trying to read some data from standard input, what I'm actually >> trying to do is process some date pasted in using the mouse cut and >> paste on a Linux box (xubuntu 9.10) in a terminal window. >> >> First attempts failed so I'm now trying the trivial:- >> >> import sys >> data = sys.stdin.readlines() >> print "Counted", len(data), "lines." >> >> >> When I run this and try to paste something into the terminal window I >> get the following errors:- >> >> /home/chris/bin/m2r.py: line 2: syntax error near unexpected token `(' >> /home/chris/bin/m2r.py: line 2: `data = sys.stdin.readlines()' >> >> >> It does exactly the same if I try:- >> >> cat | m2r.py >> >> and then paste something into the window. >> >> >> So - what on earth am I doing wrong? >> >> > Trying to run Python scripts with /bin/sh, probably. Yup. I can't count the number of times I've run ImageMagick's "import" program unintentionally... -- Grant Edwards grante Yow! The SAME WAVE keeps at coming in and COLLAPSING visi.com like a rayon MUU-MUU ... From no.email at please.post Sun Jan 31 14:35:51 2010 From: no.email at please.post (kj) Date: Sun, 31 Jan 2010 19:35:51 +0000 (UTC) Subject: How to pass Chinese characters as command-line arguments? References: <7slr5iFe66U1@mid.uni-berlin.de> Message-ID: In <7slr5iFe66U1 at mid.uni-berlin.de> "Diez B. Roggisch" writes: >Am 31.01.10 16:52, schrieb kj: >> I want to pass Chinese characters as command-line arguments to a >> Python script. My terminal has no problem displaying these >> characters, and passing them to the script, but I can't get Python >> to understand them properly. >> >> E.g. if I pass one such character to the simple script >> >> import sys >> print sys.argv[1] >> print type(sys.argv[1]) >> >> the first line of the output looks fine (identical to the input), >> but the second line says "". If I add the line >> >> arg = unicode(sys.argv[1]) >> >> I get the error >> >> Traceback (most recent call last): >> File "kgrep.py", line 4, in >> arg = unicode(sys.argv[1]) >> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe8 in position 0: ordinal not in range(128) >> >> What must I do to get Python to recognize command-line arguments >> as utf-8 Unicode? >The last sentence reveals your problem: utf-8 is *not* unicode. It's an >encoding of unicode, which is a crucial difference. > From the outside you get byte-streams, and if these happen to be >encoded in utf-8, you can simply decode them: >arg = unicode(sys.argv[1], "utf-8") Thanks! kynn From python.list at tim.thechases.com Sun Jan 31 14:41:55 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Sun, 31 Jan 2010 13:41:55 -0600 Subject: What's the Scoop on \\ for Paths? (Win) In-Reply-To: References: Message-ID: <4B65DD03.50706@tim.thechases.com> Alf P. Steinbach wrote: > that you cannot write e.g. "c:\windows\system32", but must > write something like "c:\\windows\\system32" (try to print > that string), or, since Windows handles forward slashes as > well, you can write "c:/windows/system32" :-). Forward slashes work for some relative paths for some commands but not for others like absolute non-drive-specified paths: Microsoft Windows XP [Version 5.1.2600] C:\>md abc C:\>md abc\123 C:\>md abc\234 C:\>cd abc C:\abc>tree /f /a Folder PATH listing Volume serial number is 940C-3F80 C:. +---123 | \---234 \---234 C:\abc>cd 123 C:\abc\123>cd ../234 C:\abc\234>type ../123/234/hello.txt The syntax of the command is incorrect. C:\abc\234>cd ../123 C:\abc\123>cd /abc/123 The system cannot find the path specified. C:\abc>x: X:\>type c:/abc/123/234/hello.txt The syntax of the command is incorrect. ##### The previous absolute-path fails in cmd.exe for a variety of apps because the "/" is treated as a parameter/switch to the various programs. Fortunately, the Python path-handling sub-system is smart enough to do the right thing, even when Win32's internal handling is too dumb to behave: C:\abc\123>echo hello > 234/hello.txt C:\abc\123>cd .. C:\abc>tree /f /a Folder PATH listing Volume serial number is 940C-3F80 C:. +---123 | \---234 | hello.txt | \---234 C:\abc>python Python 2.4.3 (#69, Mar 29 2006, 17:35:34) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> file('/abc/123/234/hello.txt').read() 'hello \n' So as long as you stick within Python's insulation, forward slashes are a nice solution. But if you have to interact with external programs, use the \\ notation or raw strings: pth = r"c:\windows\system32" [mutters under breath about Win32 headaches] -tkc From wolftracks at invalid.com Sun Jan 31 14:43:15 2010 From: wolftracks at invalid.com (W. eWatson) Date: Sun, 31 Jan 2010 11:43:15 -0800 Subject: What's the Scoop on \\ for Paths? (Win) In-Reply-To: References: Message-ID: Steve Holden wrote: > You need to read up on string literals is all. "\\" is simply the > literal representation of a string containing a single backslash. This > comes about because string literals are allowed to contain special > "escape sequences" which are introduced by a backslash; since this gives > the backslash a special meaning in string literals we also have to use > an escape sequence ("\\") to represent a backslash. > > In practice you will find that > > a) Many Windows APIs (but not the command line) are just as happy with a > forward slash as a backslash to separate file path components; and > > b) The best practice is to build filenames using the routines provided > in the os.path module, which guarantees to give results correct for the > current platform. > > regards > Steve Basic sys functions brought out the \ separator for paths. What am I missing here? Looks OK to me. >>> abc=r'xyz\\' >>> abc 'xyz\\\\' >>> print abc xyz\\ >>> abc.replace(r'\',r'z') SyntaxError: invalid syntax >>> abc 'xyz\\\\' From __peter__ at web.de Sun Jan 31 14:44:22 2010 From: __peter__ at web.de (Peter Otten) Date: Sun, 31 Jan 2010 20:44:22 +0100 Subject: iglob performance no better than glob References: <5ebd4965-32fc-4a03-94dd-c61ccdc9d630@f5g2000vbm.googlegroups.com> Message-ID: Kyp wrote: > I have a dir with a large # of files that I need to perform operations > on, but only needing to access a subset of the files, i.e. the first > 100 files. > > Using glob is very slow, so I ran across iglob, which returns an > iterator, which seemed just like what I wanted. I could iterate over > the files that I wanted, not having to read the entire dir. > > So the iglob was faster, but accessing the first file took about the > same time as glob.glob. > > Here's some code to compare glob vs. iglob performance, it outputs > the time before/after a glob.iglob('*.*') files.next() sequence and a > glob.glob('*.*') sequence. > > #!/usr/bin/env python > > import glob,time > print '\nTest of glob.iglob' > print 'before iglob:', time.asctime() > files = glob.iglob('*.*') > print 'after iglob:',time.asctime() > print files.next() > print 'after files.next():', time.asctime() > > print '\nTest of glob.glob' > print 'before glob:', time.asctime() > files = glob.glob('*.*') > print 'after glob:',time.asctime() > > > Here are the results: > > Test of glob.iglob > before iglob: Sun Jan 31 11:09:08 2010 > after iglob: Sun Jan 31 11:09:08 2010 > foo.bar > after files.next(): Sun Jan 31 11:09:59 2010 > > Test of glob.glob > before glob: Sun Jan 31 11:09:59 2010 > after glob: Sun Jan 31 11:10:51 2010 > > The results are about the same for the 2 approaches, both took about > 51 seconds. Am I doing something wrong with iglob? No, but iglob() being lazy is pointless in your case because it uses os.listdir() and fnmatch.filter() underneath which both read the whole directory before returning anything. > Is there a way to get the first X # of files from a dir with lots of > files, that does not take a long time to run? Here's my attempt. It turned out to be more work than expected, so I cut a few corners. It's Linux-only "works on my machine" code, but may give you some hints on how to proceed. from ctypes import * import fnmatch import glob import os import re from itertools import ifilter, imap class dirent(Structure): "works on my machine ;)" _fields_ = [ ("d_ino", c_long), ("d_off", c_long), ("d_reclen", c_ushort), ("d_type", c_ubyte), ("d_name", c_char*256)] direntp = POINTER(dirent) LIBC = "libc.so.6" cdll.LoadLibrary(LIBC) libc = CDLL(LIBC) libc.readdir.restype = direntp def diriter(dir): "lazy partial replacement for os.listdir()" # errors? what errors? dirp = libc.opendir(dir) if not dirp: return try: while True: ep = libc.readdir(dirp) if not ep: break yield ep.contents.d_name finally: libc.closedir(dirp) def filter(names, pattern): "lazy partial replacement for fnmatch.filter()" import posixpath pattern = os.path.normcase(pattern) r = fnmatch.translate(pattern) r = re.compile(r) if os.path is not posixpath: names = imap(os.path.normcase, names) return ifilter(r.match, names) def globiter(path): "lazy partial replacement for glob.glob()" dir, filename = os.path.split(path) if glob.has_magic(dir): raise ValueError("wildcards in directory not supported") return filter(diriter(dir), filename) if __name__ == "__main__": import sys [pattern] = sys.argv[1:] for name in globiter(pattern): print name Peter From steve at holdenweb.com Sun Jan 31 14:54:45 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 31 Jan 2010 14:54:45 -0500 Subject: What's the Scoop on \\ for Paths? (Win) In-Reply-To: References: Message-ID: W. eWatson wrote: > Steve Holden wrote: > >> You need to read up on string literals is all. "\\" is simply the >> literal representation of a string containing a single backslash. This >> comes about because string literals are allowed to contain special >> "escape sequences" which are introduced by a backslash; since this gives >> the backslash a special meaning in string literals we also have to use >> an escape sequence ("\\") to represent a backslash. >> >> In practice you will find that >> >> a) Many Windows APIs (but not the command line) are just as happy with a >> forward slash as a backslash to separate file path components; and >> >> b) The best practice is to build filenames using the routines provided >> in the os.path module, which guarantees to give results correct for the >> current platform. >> >> regards >> Steve > Basic sys functions brought out the \ separator for paths. > > What am I missing here? Looks OK to me. > > >>>> abc=r'xyz\\' >>>> abc > 'xyz\\\\' >>>> print abc > xyz\\ >>>> abc.replace(r'\',r'z') > > SyntaxError: invalid syntax >>>> abc > 'xyz\\\\' The SyntaxError is thrown because although backslashes lose their special "escape sequence" meaning, for some reason I have never really understood they retain the requirement that they be followed by another character. abc.replace("\\", 'z') would have worked fine. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From no-spam at non-existing.invalid Sun Jan 31 14:57:31 2010 From: no-spam at non-existing.invalid (Robert) Date: Sun, 31 Jan 2010 20:57:31 +0100 Subject: HTML Parser which allows low-keyed local changes? Message-ID: I tried lxml, but after walking and making changes in the element tree, I'm forced to do a full serialization of the whole document (etree.tostring(tree)) - which destroys the "human edited" format of the original HTML code. makes it rather unreadable. is there an existing HTML parser which supports tracking/writing back particular changes in a cautious way by just making local changes? or a least tracks the tag start/end positions in the file? Robert From python.list at tim.thechases.com Sun Jan 31 14:59:43 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Sun, 31 Jan 2010 13:59:43 -0600 Subject: What's the Scoop on \\ for Paths? (Win) In-Reply-To: References: Message-ID: <4B65E12F.3070606@tim.thechases.com> W. eWatson wrote: > What am I missing here? Looks OK to me. > >>> abc.replace(r'\',r'z') > > SyntaxError: invalid syntax A raw string can't end in a single backslash (something that occasionally annoys me, but I've learned to deal with it). >>> s=r'\' File "", line 1 s=r'\' ^ SyntaxError: EOL while scanning single-quoted string My guess is that the parser sees the first call as abc.replace( r"r\'r" <-- how your single-quoted is interpreted z' <-- the leftovers that cause the syntax error ) -tkc From python at mrabarnett.plus.com Sun Jan 31 15:02:00 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 31 Jan 2010 20:02:00 +0000 Subject: What's the Scoop on \\ for Paths? (Win) In-Reply-To: References: Message-ID: <4B65E1B8.8070905@mrabarnett.plus.com> W. eWatson wrote: > Steve Holden wrote: > >> You need to read up on string literals is all. "\\" is simply the >> literal representation of a string containing a single backslash. This >> comes about because string literals are allowed to contain special >> "escape sequences" which are introduced by a backslash; since this gives >> the backslash a special meaning in string literals we also have to use >> an escape sequence ("\\") to represent a backslash. >> >> In practice you will find that >> >> a) Many Windows APIs (but not the command line) are just as happy with a >> forward slash as a backslash to separate file path components; and >> >> b) The best practice is to build filenames using the routines provided >> in the os.path module, which guarantees to give results correct for the >> current platform. >> >> regards >> Steve > Basic sys functions brought out the \ separator for paths. > > What am I missing here? Looks OK to me. > > > >>> abc=r'xyz\\' > >>> abc > 'xyz\\\\' > >>> print abc > xyz\\ > >>> abc.replace(r'\',r'z') > > SyntaxError: invalid syntax > >>> abc > 'xyz\\\\' It's not possible to end a raw string with a single backslash, and if you end it with a double backslash then you'll get a double backslash. Very annoying. From half.italian at gmail.com Sun Jan 31 15:04:36 2010 From: half.italian at gmail.com (Sean DiZazzo) Date: Sun, 31 Jan 2010 12:04:36 -0800 (PST) Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <87iqaj9n8z.fsf@castleamber.com> <877hqz2j1o.fsf@castleamber.com> <4B6558AF.1090009@v.loewis.de> Message-ID: > Here is a recent list of magic numbers: > > ? ? ? ?Python 2.6a0: 62151 (peephole optimizations and STORE_MAP opcode) > ? ? ? ?Python 2.6a1: 62161 (WITH_CLEANUP optimization) > ? ? ? ?Python 2.7a0: 62171 (optimize list comprehensions/change LIST_APPEND) > ? ? ? ?Python 2.7a0: 62181 (optimize conditional branches: > ? ? ? ? ? ? ? ? ? ? ? ? ? ? introduce POP_JUMP_IF_FALSE and POP_JUMP_IF_TRUE) > ? ? ? ?Python 2.7a0 ?62191 (introduce SETUP_WITH) > ? ? ? ?Python 2.7a0 ?62201 (introduce BUILD_SET) > ? ? ? ?Python 2.7a0 ?62211 (introduce MAP_ADD and SET_ADD) > > #define MAGIC (62211 | ((long)'\r'<<16) | ((long)'\n'<<24)) > > Regards, > Martin Does "magic" really need to be used? Why not just use the revision number? From marcmagransdeabril at gmail.com Sun Jan 31 15:08:19 2010 From: marcmagransdeabril at gmail.com (marc magrans de abril) Date: Sun, 31 Jan 2010 12:08:19 -0800 (PST) Subject: Slow down while creating a big list and iterating over it References: <2fdd4e1b-db58-45d7-9b7d-73dcdca6384a@c29g2000yqd.googlegroups.com> <29a69a7c-a8a2-4c53-9521-760e13665cd3@f12g2000yqn.googlegroups.com> Message-ID: <4b484bc6-f903-40a0-a8dc-9a00c8fc002a@k5g2000yqf.googlegroups.com> Hi! ...I have found a good enough solution, although it only works if the number of patterns (clusters) is not very big: def classify(f): THERESHOLD=0.1 patterns={} for l in enumerate(f): found = False for p,c in patterns.items(): if dist(l,p) < THERESHOLD: found=True patterns[p] = c +1 if not found: patterns[l] = 1 return patterns This algorithm is O(n*np*m^2). Where n is the number of logs, np the number of patterns, and m is the log length (i.e. m^2 is the distance cost). So it's way better O(n^2*m^2) and I can run it for some hours to get back the results. I wonder if there is a single threaded/process clustering algorithm than runs in O(n)? Cheers, marc From nobody at nowhere.com Sun Jan 31 15:14:47 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 31 Jan 2010 20:14:47 +0000 Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> Message-ID: On Sat, 30 Jan 2010 16:58:34 +0000, tanix wrote: >>I'm not familiar with Ruby, but most languages are cleaner than Python >>once you get beyond the "10-minute introduction" stage. > > I'd have to agree. The only ones that beat Python in that department are > Javascript and PHP. Plus CSS and HTML if you can call those languages. > > The very idea of using a number of blanks to identify your block level is > as insane as it gets. I don't have a problem with layout (indentation level as a syntax element). > First of all, combinations of blanks and tabs, > depending on how your ide is setup to expand tabs, may get you bugs, you'd > never imagine in your wild dreams. If you IDE places tab stops other than every 8 columns, it's broken. Configurable tab stops in a text editor is one of those "features" that differentiates a "coder" from a software engineer. A coder implements it because it's easy to implement, without giving a moment's thought to the wider context (such as: how to communicate the non-standard tab stops to any other program which needs to read the file). Even so, it's quite simple to implement layout in a way which doesn't care about tab widths: 1. If the current line begins with exactly the same sequence of whitespace characters as the last non-blank line, it's part of the same block. 2. If the sequence of whitespace characters at the beginning of the current line is an extension of that for the last non-blank line (i.e. it starts with the previous sequence, then adds some more whitespace), then it's the first line of a inner block. 3. If the current line begins with a left substring of the sequence for the last non-blank line, then it belongs to an outer block (whichever one has that sequence). 4. If none of the above are true, it's a syntax error. > Braces is the most reliable way to identify blocks. No they aren't. Indentation is much easier for humans to process than scanning the file for braces (and not missing any). If you have any indentation in the code, humans will interpret it as indicating the nesting, even if the compiler doesn't. E.g.: if(x) { if(y) foo(); else bar(); } See the problem? The compiler will match the else with if(y), but a human will tend to match it with if(x). Layout ensures that a human sees what the compiler "sees". Given that any sane program uses indentation to reflect the program's structure, braces (or "begin", or "end" (or endif/endwhile/etc)) are redundant. From mrholtsr at sbcglobal.net Sun Jan 31 15:25:57 2010 From: mrholtsr at sbcglobal.net (Ray Holt) Date: Sun, 31 Jan 2010 15:25:57 -0500 Subject: Why this error message Message-ID: <6B218CF5831243C08A438194D7F4E43E@ray> Why am I getting the error that test is not defined. Thanks, Ray class SpecialFile: def __init__(self, fileName): self.__file = open(fileName, 'W') self.__file.write('***** Start Special File *****\n\n') def write(self, str): self.__file.write(str) def writelines(self, strList): self.__file.writelines(strList) def __del__(self): # Destructor method, __del__. print "entered __del__" self.close() def close(self): # Clean up method if self.__file: self.__file.write('\\n***** End Special File *****') self.__file.close() self.__file = None def test(): f = SpecialFile('testfile') f.write('111111\n') f.close() test() Traceback (most recent call last): File "C:\Python26\Classes_and_OOP_Programing.py", line 145, in test() NameError: name 'test' is not defined -------------- next part -------------- An HTML attachment was scrubbed... URL: From alfps at start.no Sun Jan 31 15:36:57 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 31 Jan 2010 21:36:57 +0100 Subject: What's the Scoop on \\ for Paths? (Win) In-Reply-To: References: Message-ID: * Tim Chase: > Alf P. Steinbach wrote: >> that you cannot write e.g. "c:\windows\system32", but must >> write something like "c:\\windows\\system32" (try to print >> that string), or, since Windows handles forward slashes as >> well, you can write "c:/windows/system32" :-). > > Forward slashes work for some relative paths for some commands but not > for others like absolute non-drive-specified paths: > > Microsoft Windows XP [Version 5.1.2600] > C:\>md abc > C:\>md abc\123 > C:\>md abc\234 > C:\>cd abc > C:\abc>tree /f /a > Folder PATH listing > Volume serial number is 940C-3F80 > C:. > +---123 > | \---234 > \---234 > > C:\abc>cd 123 > C:\abc\123>cd ../234 > C:\abc\234>type ../123/234/hello.txt > The syntax of the command is incorrect. > C:\abc\234>cd ../123 > C:\abc\123>cd /abc/123 > The system cannot find the path specified. > C:\abc>x: > X:\>type c:/abc/123/234/hello.txt > The syntax of the command is incorrect. > > ##### > The previous absolute-path fails in cmd.exe for a variety of apps > because the "/" is treated as a parameter/switch to the various > programs. Yes, that's a valid concern when invoking external programs. > Fortunately, the Python path-handling sub-system is smart > enough to do the right thing, even when Win32's internal handling is too > dumb to behave: > > C:\abc\123>echo hello > 234/hello.txt > C:\abc\123>cd .. > C:\abc>tree /f /a > Folder PATH listing > Volume serial number is 940C-3F80 > C:. > +---123 > | \---234 > | hello.txt > | > \---234 > > C:\abc>python > Python 2.4.3 (#69, Mar 29 2006, 17:35:34) [MSC v.1310 32 bit (Intel)] on > win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> file('/abc/123/234/hello.txt').read() > 'hello \n' Oops, this is a dangerous misconception, because it can lead to Wrong Solutions to the perceived problems. It's not the "Win32's internal handling" that you're up against above, but the generally unpredictable quirks of syntax and semantics of commands in the Windows command interpreter: C:\test> echo bah >foo.txt C:\test> type ../test/foo.txt The syntax of the command is incorrect. C:\test> type "../test/foo.txt" bah C:\test> _ Generally the details of that command interpreter are undocumented. > So as long as you stick within Python's insulation, forward slashes are > a nice solution. No, it's not Python's insulation that makes things work (although perhaps it redundantly tries to help), it's simply the Windows API, which generally accepts forward or backward slashes. It's the same in other programming languages. In particular, it's not a good idea to carry the idea of using backslashes over to #include directives in C or C++, thinking that there's no Python insulation there (at least one compiler accepts and Windows compilers used to accept it at one time, but it's invalid, and non-portable, whereas forward slashes work). > But if you have to interact with external programs, > use the \\ notation or raw strings: > > pth = r"c:\windows\system32" > > [mutters under breath about Win32 headaches] He he. Yes. But anyways, the thing to remember is that when paths are passed to /programs/ in Windows, they generally need to be /quoted/. That's particularly important for paths containing spaces. For internal file handling in a program, including using libraries, forward slashes work fine -- unless a library is particularly ill-behaved and requires backslashes -- and provide some (but not complete) measure of portability. For example, >>> f = open( "c:/foo.txt", "w" ) >>> f.write( "works\n" ) 6 >>> f.close() >>> f = open( "c:/foo.txt", "r" ) >>> f.readlines() ['works\n'] >>> f.close() >>> _ In retrospect, I should have been more clear about that distinction between invoking programs and invoking routines such as Python 'open', yes. And as Steve Holden remarked else-thread, best practice for Python is to build paths using the os.path functionality. E.g., >>> import os.path >>> os.path.normpath( "c:/foo.txt" ) 'c:\\foo.txt' >>> _ Cheers & hth., - Alf From nobody at nowhere.com Sun Jan 31 15:43:03 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 31 Jan 2010 20:43:03 +0000 Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> <7536f9c0-a36a-479d-ba95-9405b13793fd@e25g2000yqh.googlegroups.com> Message-ID: On Sun, 31 Jan 2010 03:01:51 -0800, rantingrick wrote: >> That's also true for most functional languages, e.g. Haskell and ML, as >> well as e.g. Tcl and most shells. Why require "f(x)" or "(f x)" if "f x" >> will suffice? > > yuck! wrapping the arg list with parenthesis (python way) makes the most > sense. Its to easy to misread somthing like this > > onetwothree four five six > > onetwothree(four, five, six) #ahhh... plain english. Note: Functional languages allow: f (a,b,c) but that means something else, i.e. calling a function with a tuple as its argument (in functional languages, a function always has exactly one argument). The syntax: f a b c is equivalent to the Python expression: f(a)(b)(c) i.e. each argument is applied in turn, with all applications except the last yielding a function. Defining f as: f a b c = is shorthand for: f = \a -> (\b -> (\c -> )) or, in Python syntax: f = lambda a: (lambda b: (lambda c: )) This style (known as Currying, or a Curried function, after the mathematician Haskell Curry) is common in functional languages, as it allows you to partially apply functions, e.g.: map (f a b) someList whereas an uncurried function would require a lambda expression: map (\c -> f (a,b,c)) someList IOW, while the uncurried form is allowed, it has no advantages and one disadvantage, so it's seldom used (and where it is used, it's normally a case of a function whose sole argument is a naturally-occurring tuple, rather than one which is constructed simply to satisfy the mechanics of a function call). Partial application is common enough that Haskell supports it for infix operators as well, e.g. map (/ 2) someList -- (/ 2) => \x -> x / 2, i.e. halve map (1 /) someList -- (1 /) => \x -> 1 / x, i.e. reciprocal If it was common-place to use Curried functions and partial application in Python, you'd probably prefer "f a b c" to "f(a)(b)(c)" as well. From daniel at stutzbachenterprises.com Sun Jan 31 15:45:31 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Sun, 31 Jan 2010 14:45:31 -0600 Subject: ANN: blist 1.1.1 - now with sortedlist, sortedset, and sorteddict Message-ID: blist 1.1.1 is now available: http://pypi.python.org/pypi/blist/ What is blist? -------------- The blist is a drop-in replacement for the Python list the provides better performance when modifying large lists. Python's built-in list is a dynamically-sized array; to insert or removal an item from the beginning or middle of the list, it has to move most of the list in memory, i.e., O(n) operations. The blist uses a flexible, hybrid array/tree structure and only needs to move a small portion of items in memory, specifically using O(log n) operations. For small lists, the blist and the built-in list have virtually identical performance. What's new? ----------- blist 1.1 introduces other data structures based on the blist: - sortedlist - sortedset - weaksortedlist - weaksorteset - sorteddict - btuple These additional data structures are only available in Python 2.6 or higher, as they make use of Abstract Base Classes. The sortedlist is a list that's always sorted. It's iterable and indexable like a Python list, but to modify a sortedlist the same methods you would use on a Python set (add, discard, or remove). >>> from blist import sortedlist >>> my_list = sortedlist([3,7,2,1]) >>> my_list sortedlist([1, 2, 3, 7]) >>> my_list.add(5) >>> my_list[3] 5 >>> The sortedlist constructor takes an optional "key" argument, which may be used to change the sort order just like the sorted() function. >>> from blist import sortedlist >>> my_list = sortedlist([3,7,2,1], key=lambda i: -i) sortedlist([7, 3, 2, 1] >>> The sortedset is a set that's always sorted. It's iterable and indexable like a Python list, but modified like a set. Essentially, it's just like a sortedlist except that duplicates are ignored. >>> from blist import sortedset >>> my_set = sortedset([3,7,2,2]) sortedset([2, 3, 7] >>> The weaksortedlist and weaksortedset are weakref variations of the sortedlist and sortedset. The sorteddict works just like a regular dict, except the keys are always sorted. The sorteddict should not be confused with Python 2.7's OrderedDict type, which remembers the insertion order of the keys. >>> from blist import sorteddict >>> my_dict = sorteddict({1: 5, 6: 8, -5: 9}) >>> my_dict.keys() [-5, 1, 6] >>> The btuple is a drop-in replacement for the built-in tuple. Compared to the built-in tuple, the btuple offers the following advantages: - Constructing a btuple from a blist takes O(1) time. - Taking a slice of a btuple takes O(n) time, where n is the size of the original tuple. The size of the slice does not matter. >>> from blist import blist, btuple >>> x = blist([0]) # x is a blist with one element >>> x *= 2**29 # x is a blist with > 500 million elements >>> y = btuple(x) # y is a btuple with > 500 million elements Feedback -------- We're eager to hear about your experiences with the blist. You can email me at daniel at stutzbachenterprises.com. Alternately, bug reports and feature requests may be reported on our bug tracker at: http://github.com/DanielStutzbach/blist/issues -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrholtsr at sbcglobal.net Sun Jan 31 15:46:38 2010 From: mrholtsr at sbcglobal.net (Ray Holt) Date: Sun, 31 Jan 2010 15:46:38 -0500 Subject: Why this error message Message-ID: Guys, I apologize for the last email asking for help I am going to have to remember to check my indentation. Ray -------------- next part -------------- An HTML attachment was scrubbed... URL: From john at castleamber.com Sun Jan 31 15:47:08 2010 From: john at castleamber.com (John Bokma) Date: Sun, 31 Jan 2010 14:47:08 -0600 Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> Message-ID: <87y6jef1df.fsf@castleamber.com> Nobody writes: > Configurable tab stops in a text editor is one of those "features" that > differentiates a "coder" from a software engineer. A coder implements it > because it's easy to implement, without giving a moment's thought to the > wider context (such as: how to communicate the non-standard tab stops to > any other program which needs to read the file). [..] > if(x) { > if(y) > foo(); > else > bar(); > } > > See the problem? Nope, because a good editor will format this correctly. One written by software engineers ;-) > Given that any sane program uses indentation to reflect the program's > structure, braces (or "begin", or "end" (or endif/endwhile/etc)) are > redundant. An editor can correct the indenting of the braces example but can't with this one. if x: if y: foo() else: bar() While braces might be considered redundant they are not when for one reason or another formatting is lost or done incorrectly. FWIW: I have no problem with how Python doesn't use braces nor on how other languages do insist on braces or other structure markers. -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From ryan at rfk.id.au Sun Jan 31 15:53:00 2010 From: ryan at rfk.id.au (Ryan Kelly) Date: Mon, 01 Feb 2010 07:53:00 +1100 Subject: Why this error message In-Reply-To: <6B218CF5831243C08A438194D7F4E43E@ray> References: <6B218CF5831243C08A438194D7F4E43E@ray> Message-ID: <1264971180.2900.0.camel@durian> On Sun, 2010-01-31 at 15:25 -0500, Ray Holt wrote: > Why am I getting the error that test is not defined. Thanks, Ray > class SpecialFile: > def __init__(self, fileName): > self.__file = open(fileName, 'W') > self.__file.write('***** Start Special File *****\n\n') > def write(self, str): > self.__file.write(str) > def writelines(self, strList): > self.__file.writelines(strList) > > def __del__(self): # Destructor method, __del__. > print "entered __del__" > self.close() > > def close(self): # Clean up method > if self.__file: > self.__file.write('\\n***** End Special File *****') > self.__file.close() > self.__file = None > > def test(): > f = SpecialFile('testfile') > f.write('111111\n') > f.close() > test() > Traceback (most recent call last): > File "C:\Python26\Classes_and_OOP_Programing.py", line 145, in > > test() > NameError: name 'test' is not defined Based on the indentation, it looks like you've defined "test" as a method on the SpecialFile class. Try dedenting it to begin in the first column. Ryan -- Ryan Kelly http://www.rfk.id.au | This message is digitally signed. Please visit ryan at rfk.id.au | http://www.rfk.id.au/ramblings/gpg/ for details -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: From nobody at nowhere.com Sun Jan 31 15:57:44 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 31 Jan 2010 20:57:44 +0000 Subject: Utility to screenscrape sites using javascript ? References: <4c4298c0-b93d-4691-bceb-e982373f87e3@f17g2000prh.googlegroups.com> <3df16722-7726-4b05-b8c3-d1b6a9f7e98b@k6g2000prg.googlegroups.com> Message-ID: On Sat, 30 Jan 2010 11:28:47 -0800, KB wrote: >> > I have a service I subscribe to that uses javascript to stream news. > >> There's a Python interface to SpiderMonkey (Mozilla's JavaScript >> interpreter): >> >> http://pypi.python.org/pypi/python-spidermonkey > > Thanks! I don't see a documentation page, but how would one use this? > Would you download the HTML using urllib2/mechanize, then parse for the > .js script and use spider-monkey to "execute" the script and the output is > passed back to python? Something like that. The "homepage" link: http://github.com/davisp/python-spidermonkey has some examples further down. The first one starts with: >>> import spidermonkey >>> rt = spidermonkey.Runtime() >>> cx = rt.new_context() >>> cx.execute("var x = 3; x *= 4; x;") 12 It goes on to mention the .add_global(name, object) method, using it to name a Python object such that its variables and methods can be accessed from JS. For scraping web pages, I suspect that you'll at least need to create an object and name it "document", so that "document.location" etc works. How much you'll need to implement will depend upon what the web page uses, although you can probably use .__getattr__() to serve up dummy handlers for calls which can be ignored. Futher documentation is probably a case of "read the spidermonkey documentation, then read the python-spidermonkey source if it isn't clear how the wrapper relates to spidermonkey itself". From victorsubervi at gmail.com Sun Jan 31 16:15:34 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sun, 31 Jan 2010 13:15:34 -0800 Subject: OT: Instant Messenger Clients Message-ID: <4dc0cfea1001311315u3494ad6bk792327d05447c96d@mail.gmail.com> Hi; I need to record my IM conversations. I'm using Gmal's IM client and I can't figure out how to do it, nor do I find any help googling it. Is it possible with Gmail? If so, how? If not, is there a good IM client that will allow me to do this? TIA, beno -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From wolfgang.lipp at gmail.com Sun Jan 31 16:17:04 2010 From: wolfgang.lipp at gmail.com (_wolf) Date: Sun, 31 Jan 2010 13:17:04 -0800 (PST) Subject: whassup? builtins? python3000? Naah can't be right? Message-ID: <799ea994-d00f-485c-9164-ed4b66e750a5@k41g2000yqm.googlegroups.com> dear pythoneers, i would be very gladly accept any commentaries about what this sentence, gleaned from http://celabs.com/python-3.1/reference/executionmodel.html, is meant to mean, or why gods have decided this is the way to go. i anticipate this guy named Kay Schluehr will have a say on that, or maybe even the BDFL will care to pronounce ``__builtins__`` the correct way to his fallovers, followers, and fellownerds:: The built-in namespace associated with the execution of a code block is actually found by looking up the name __builtins__ in its global namespace; this should be a dictionary or a module (in the latter case the module?s dictionary is used). By default, when in the __main__ module, __builtins__ is the built-in module builtins; when in any other module, __builtins__ is an alias for the dictionary of the builtins module itself. __builtins__ can be set to a user-created dictionary to create a weak form of restricted execution. it used to be the case that there were at least two distinct terms, ?builtin? (in the singular) and ?builtins? (in the plural), some of which existed both in module and in dict form (?just guessing?). now there is only ?builtins?, so fortunately the ambivalence between singular and plural has gone?good riddance. but why does ``__builtins__`` change its meaning depending on whether this is the scope of the ?script? (i.e. the module whose name was present, when calling ``python foobar.py``) or whether this is the scope of a secondary module (imported or executed, directly or indirectly, by ``foobar.py``)? i cannot understand the reasoning behind this and find it highly confusing. rationale: why do i care??i want to be able to ?export names to the global namespace that were not marked private (by an underscore prefix) in a python module that i execute via ``exec( compile( get ( locator ), locator, 'exec' ), R )`` where ``R`` is supposed to going to hold the private names of said module?. it *is* a little arcane but the basic exercise is to by-pass python?s import system and get similr results... it is all about injecting names into the all-global and the module-global namespaces. still i get trapped by the above wordings in tzhe docs, and i have a weird case of a vanishing names, so maybe some people will care to share their thoughts. love & ~flow thank From burslem2001 at yahoo.com Sun Jan 31 16:27:46 2010 From: burslem2001 at yahoo.com (gazza) Date: Sun, 31 Jan 2010 13:27:46 -0800 (PST) Subject: gmtime Message-ID: <62938b5b-7eda-426c-b91d-3c7061840b2d@l26g2000yqd.googlegroups.com> Hi, I am trying to discover how to obtain the correct time of say CST/ America and EST/America in python? Any help on this would be appreciated. Thanks, Garyc From awilliam at opengroupware.us Sun Jan 31 16:29:07 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Sun, 31 Jan 2010 16:29:07 -0500 Subject: OT: Instant Messenger Clients In-Reply-To: <4dc0cfea1001311315u3494ad6bk792327d05447c96d@mail.gmail.com> References: <4dc0cfea1001311315u3494ad6bk792327d05447c96d@mail.gmail.com> Message-ID: <1264973347.19333.0.camel@linux-m3mt> On Sun, 2010-01-31 at 13:15 -0800, Victor Subervi wrote: > Hi; > I need to record my IM conversations. I'm using Gmal's IM client and I > can't figure out how to do it, nor do I find any help googling it. Is > it possible with Gmail? If so, how? If not, is there a good IM client > that will allow me to do this? "OT" is an understatement. Why not ask somewhere appropriate? From benjamin at python.org Sun Jan 31 16:30:34 2010 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 31 Jan 2010 21:30:34 +0000 (UTC) Subject: iglob performance no better than glob References: <5ebd4965-32fc-4a03-94dd-c61ccdc9d630@f5g2000vbm.googlegroups.com> Message-ID: Kyp stsci.edu> writes: > So the iglob was faster, but accessing the first file took about the > same time as glob.glob. That would be because glob is implemented in terms of iglob. From benjamin at python.org Sun Jan 31 16:32:42 2010 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 31 Jan 2010 21:32:42 +0000 (UTC) Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <87iqaj9n8z.fsf@castleamber.com> <877hqz2j1o.fsf@castleamber.com> <4B6558AF.1090009@v.loewis.de> Message-ID: Sean DiZazzo gmail.com> writes: > Does "magic" really need to be used? Why not just use the revision > number? Because magic is easier and otherwise CPython developers would have to rebuild their pycs everytime their working copy was updated. From nobody at nowhere.com Sun Jan 31 16:52:26 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 31 Jan 2010 21:52:26 +0000 Subject: What's the Scoop on \\ for Paths? (Win) References: Message-ID: On Sun, 31 Jan 2010 13:41:55 -0600, Tim Chase wrote: > The previous absolute-path fails in cmd.exe for a variety of apps because > the "/" is treated as a parameter/switch to the various programs. > Fortunately, the Python path-handling sub-system is smart enough to do the > right thing, even when Win32's internal handling is too dumb to behave: Let's not conflate multiple issues: 1. Windows API functions almost invariably accept either \ or /. 2. Python functions just pass filename strings verbatim; they won't try to "fix" them in cases where a specific separator is required (e.g. system(), which calls cmd.exe or %COMSPEC%, which has its own requirements). 3. Command-line programs may treat / as indicating a switch, even in what would normally be considered the middle of an argument. 4. Windows doesn't enforce the "char **argv" convention. Programs get passed the literal command line as a single string, and are free to parse it however they wish. 4a. Programs compiled with MSVC and using a main(argc, argv) interface will have the command-line parsed as documented in: http://msdn.microsoft.com/en-us/library/17w5ykft.aspx Programs built with other compilers, programs using WinMain(), and programs which use the "raw" command line may behave differently, and certain Windows command-line programs (most notably those inherited from DOS) *do* behave differently. 5. Python functions which accept command arguments as a list of strings (e.g. subprocess.call()) will assemble a command string according to the above specification. If you're invoking a program which doesn't follow that specification, you'll need to pass the command-line as a string rather than as a list of arguments. From newssw at gmail.com Sun Jan 31 17:07:44 2010 From: newssw at gmail.com (Mik0b0) Date: Sun, 31 Jan 2010 14:07:44 -0800 (PST) Subject: ftp.storlines error Message-ID: <24ffaab0-f319-4019-97cf-56a7531e3eef@n7g2000yqb.googlegroups.com> Good day/night/etc. I am rather a newb in Python (learning Python 3). I am trying to create a small script for FTP file uploads on my home network. The script looks like this: from ftplib import FTP ftp=FTP('10.0.0.1') ftp.login('mike','*****') directory='/var/www/blabla/' ftp.cwd(directory) ftp.retrlines('LIST') print('<- - - - - - - - - >') file_to_change='test' file=1 file=open(file_to_change,'w') text='test' file.write(text) ftp.storlines('STOR ' + file_to_change,file) ftp.retrlines('LIST') file.close() The output is like this: Traceback (most recent call last): File "ftp.py", line 13, in ftp.storlines('STOR ' + file_to_change,i) File "/usr/lib/python3.1/ftplib.py", line 474, in storlines buf = fp.readline() IOError: not readable What is wrong? From affdfsdfdsfsd at b.com Sun Jan 31 17:12:38 2010 From: affdfsdfdsfsd at b.com (Tracubik) Date: 31 Jan 2010 22:12:38 GMT Subject: create a string of variable lenght References: <4b65729b$0$1126$4fafbaef@reader3.news.tin.it> Message-ID: <4b660056$0$1131$4fafbaef@reader3.news.tin.it> Il Sun, 31 Jan 2010 13:46:16 +0100, G?nther Dietrich ha scritto: > Maybe you might solve this if you decode your string to unicode. > Example: > > |>>> euro = "?" > |>>> len(euro) > |3 > |>>> u_euro = euro.decode('utf_8') > |>>> len(u_euro) > |1 > > Adapt the encoding ('utf_8' in my example) to whatever you use. > > Or create the unicode string directly: > > |>>> u_euro = u'?' > |>>> len(u_euro) > |1 > > > > Best regards, > > G?nther thank you, your two solution is really interesting. is there a possible to set unicode encoding by default for my python scripts? i've tried inserting # -*- coding: utf-8 -*- at the beginning of my script but doesn't solve the problem From clp2 at rebertia.com Sun Jan 31 17:19:49 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 31 Jan 2010 14:19:49 -0800 Subject: ftp.storlines error In-Reply-To: <24ffaab0-f319-4019-97cf-56a7531e3eef@n7g2000yqb.googlegroups.com> References: <24ffaab0-f319-4019-97cf-56a7531e3eef@n7g2000yqb.googlegroups.com> Message-ID: <50697b2c1001311419p68b42c0dmf0ab855e452b326d@mail.gmail.com> On Sun, Jan 31, 2010 at 2:07 PM, Mik0b0 wrote: > Good day/night/etc. > I am rather a newb in Python (learning Python 3). I am trying to > create a small script for FTP file uploads ?on my home network. The > script looks like this: > > from ftplib import FTP > ftp=FTP('10.0.0.1') > ftp.login('mike','*****') > directory='/var/www/blabla/' > ftp.cwd(directory) > ftp.retrlines('LIST') > print('<- - - - - - - - - >') > file_to_change='test' > file=1 > file=open(file_to_change,'w') Here you open the file in (w)rite mode. Also, don't call it `file` as that shadows the name of the built-in type. > text='test' > file.write(text) And indeed, here you've written something to it. > ftp.storlines('STOR ' + file_to_change,file) storlines() needs a file opened in (r)ead mode however, hence the error. Obviously, it needs to read the contents of the file in order to send it over the network. Either do the writing separately, close the file, and then open it again in read mode, or open it in one of the modes that allows for both reading and writing [see help(open) for details]. Cheers, Chris -- http://blog.rebertia.com > ftp.retrlines('LIST') > file.close() > > The output is like this: > Traceback (most recent call last): > ?File "ftp.py", line 13, in > ? ?ftp.storlines('STOR ' + file_to_change,i) > ?File "/usr/lib/python3.1/ftplib.py", line 474, in storlines > ? ?buf = fp.readline() > IOError: not readable > > What is wrong? From tinnews at isbd.co.uk Sun Jan 31 17:24:13 2010 From: tinnews at isbd.co.uk (tinnews at isbd.co.uk) Date: Sun, 31 Jan 2010 22:24:13 +0000 Subject: Can't get sys.stdin.readlines() to work References: <4vtg37-n89.ln1@chris.isbd.net> <3a3c2ba0-7221-4402-9ed6-0b74e101f95b@m16g2000yqc.googlegroups.com> Message-ID: Richard Thomas wrote: > On Jan 31, 6:15?pm, tinn... at isbd.co.uk wrote: > > I'm trying to read some data from standard input, what I'm actually > > trying to do is process some date pasted in using the mouse cut and > > paste on a Linux box (xubuntu 9.10) in a terminal window. > > > > First attempts failed so I'm now trying the trivial:- > > > > ? ? import sys > > ? ? data = sys.stdin.readlines() > > ? ? print "Counted", len(data), "lines." > > > > When I run this and try to paste something into the terminal window I > > get the following errors:- > > > > ? ? /home/chris/bin/m2r.py: line 2: syntax error near unexpected token `(' > > ? ? /home/chris/bin/m2r.py: line 2: `data = sys.stdin.readlines()' > > > > It does exactly the same if I try:- > > > > ? ? cat | m2r.py > > > > and then paste something into the window. > > > > So - what on earth am I doing wrong? > > > > -- > > Chris Green > > You haven't put a shebang line at the top. > > #!/usr/bin/env python > import sys > data = sys.stdin.readlines() > ... Urk!!! Silly me. That catches me out every few months! :-) -- Chris Green From andrewdalke at gmail.com Sun Jan 31 17:29:09 2010 From: andrewdalke at gmail.com (Andrew Dalke) Date: Sun, 31 Jan 2010 14:29:09 -0800 (PST) Subject: recv_into(bytearray) complains about a "pinned buffer" Message-ID: <8dff1c70-c141-4f0e-aebb-3a92733b272b@o28g2000yqh.googlegroups.com> In Python 2.6 I can't socket.recv_into(a byte array instance). I get a TypeError which complains about a "pinned buffer". I have only an inkling of what that means. Since an array.array("b") works there, and since it works in Python 3.1.1, and since I thought the point of a bytearray was to make things like recv_into easier, I think this exception is a bug in Python 2.6. I want to double check before posting it to the tracker. Here's my reproducibles: Python 2.6.1 (r261:67515, Jul 7 2009, 23:51:51) [GCC 4.2.1 (Apple Inc. build 5646)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import socket >>> sock = socket.socket() >>> sock.connect( ("python.org", 80) ) >>> sock.send(b"GET / HTTP/1.0\r\n\r\n") 18 >>> buf = bytearray(b" " * 10) >>> sock.recv_into(buf) Traceback (most recent call last): File "", line 1, in TypeError: recv_into() argument 1 must be pinned buffer, not bytearray >>> I expected a bytearray to work there. In fact, I thought the point of bytearray was to allow this to work. By comparison, an array of bytes does work: >>> import array >>> arr = array.array("b") >>> arr.extend(map(ord, "This is a test")) >>> len(arr) 14 >>> sock.recv_into(arr) 14 >>> arr array('b', [72, 84, 84, 80, 47, 49, 46, 49, 32, 51, 48, 50, 32, 70]) >>> "".join(map(chr, arr)) 'HTTP/1.1 302 F' I don't even know what a "pinned buffer" means, and searching python.org isn't helpful. Using a bytearray in Python 3.1.1 *does* work: Python 3.1.1 (r311:74480, Jan 31 2010, 23:07:16) [GCC 4.2.1 (Apple Inc. build 5646) (dot 1)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import socket >>> sock = socket.socket() >>> sock.connect( ("python.org", 80) ) >>> sock.send(b"GET / HTTP/1.0\r\n\r\n") 18 >>> buf = bytearray(b" " * 10) >>> sock.recv_into(buf) 10 >>> buf bytearray(b'HTTP/1.1 3') >>> Is this a bug in Python 2.6 or a deliberate choice regarding implementation concerns I don't know about? If it's a bug, I'll add it to the tracker. Andrew Dalke dalke at dalkescientific.com From steve at REMOVE-THIS-cybersource.com.au Sun Jan 31 17:36:32 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 31 Jan 2010 22:36:32 GMT Subject: Python and Ruby References: Message-ID: <0375f245$0$1309$c3e8da3@news.astraweb.com> On Sun, 31 Jan 2010 04:28:41 -0800, Ed Keith wrote: > In most functional languages you just name a function to access it and > you do it ALL the time. > > for example, in if you have a function 'f' which takes two parameters to > call the function and get the result you use: > > f 2 3 > > If you want the function itself you use: > > f How do you call a function of no arguments? -- Steven From steve at REMOVE-THIS-cybersource.com.au Sun Jan 31 17:37:04 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 31 Jan 2010 22:37:04 GMT Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <0374c6d6$0$1309$c3e8da3@news.astraweb.com> <871vh7t2tm.fsf@benfinney.id.au> <7xljffngid.fsf@ruckus.brouhaha.com> <0374e0be$0$1309$c3e8da3@news.astraweb.com> <0374fb80$0$1309$c3e8da3@news.astraweb.com> <878wbenwk5.fsf@castleamber.com> Message-ID: <0375f265$0$1309$c3e8da3@news.astraweb.com> On Sun, 31 Jan 2010 09:06:18 -0600, John Bokma wrote: > Based on the magic numbers I've seen so far it looks like that not an > option. They increment with every minor change. They increment with every *incompatible* change to the marshal format, not every change to the compiler. > So to me, at this moment > (and maybe it's my ignorance) it looks like a made up example to justify > what to me still looks like a bad decision. Of course it's a made-up example. But with Python now entering a period where there is a moratorium on changes to the language, is it really so difficult to imagine that the marshal format will settle down for a while even as the standard library goes through upgrades? -- Steven From steve at REMOVE-THIS-cybersource.com.au Sun Jan 31 17:37:25 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 31 Jan 2010 22:37:25 GMT Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> <87y6jef1df.fsf@castleamber.com> Message-ID: <0375f27a$0$1309$c3e8da3@news.astraweb.com> On Sun, 31 Jan 2010 14:47:08 -0600, John Bokma wrote: > An editor can correct the indenting of the braces example but can't with > this one. > > if x: > if y: > foo() > else: > bar() > > While braces might be considered redundant they are not when for one > reason or another formatting is lost or done incorrectly. I've heard this argument before, and I don't buy it. Why should we expect the editor to correct malformed code? Would you expect your editor to correct this malformed code? result = sin(x+)y Why should broken indentation be held to a higher standard than any other breakage in code? -- Steven From steve at REMOVE-THIS-cybersource.com.au Sun Jan 31 17:40:03 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 31 Jan 2010 22:40:03 GMT Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> Message-ID: <0375f318$0$1309$c3e8da3@news.astraweb.com> On Sun, 31 Jan 2010 14:10:34 -0800, Dennis Lee Bieber wrote: > Ugh... That would mean that for an application using, say 20 > files, > one now has 20 subdirectories for what, in a lot of cases, will contain > just one file each (and since I doubt older Python's will be modified to > support this scheme, it will only be applicable to 3.x, and maybe a > 2.7?) If you only use one version of Python, then don't run it with the -R switch. Have you read the PEP? It is quite explicit that the default behaviour of .pyc files will remain unchanged, that to get the proposed behaviour you have to specifically ask for it. -- Steven From steve at REMOVE-THIS-cybersource.com.au Sun Jan 31 17:40:55 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 31 Jan 2010 22:40:55 GMT Subject: Keyboard input References: <8f67b6f81001290853i39bab1acs49d02ac5be948d13@mail.gmail.com> <8f67b6f81001310351m3485fd40o8944c11af8a4161f@mail.gmail.com> Message-ID: <0375f34c$0$1309$c3e8da3@news.astraweb.com> On Sun, 31 Jan 2010 14:10:34 -0800, Dennis Lee Bieber wrote: > On Sun, 31 Jan 2010 11:51:46 +0000, "Mr.SpOOn" > declaimed the following in gmane.comp.python.general: > >> 2010/1/29 Gabriel Genellina : >> > >> > That's strange. If you're using Linux, make sure you have the >> > readline package installed. >> >> I'm using Linux. Ubuntu. I checked on synaptic and I have >> readline-common. Do you mean something else? >> > But was your Python /built/ using readline? How do you ensure that it is? -- Steven From python at mrabarnett.plus.com Sun Jan 31 17:42:52 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 31 Jan 2010 22:42:52 +0000 Subject: Slow down while creating a big list and iterating over it In-Reply-To: <4b484bc6-f903-40a0-a8dc-9a00c8fc002a@k5g2000yqf.googlegroups.com> References: <2fdd4e1b-db58-45d7-9b7d-73dcdca6384a@c29g2000yqd.googlegroups.com> <29a69a7c-a8a2-4c53-9521-760e13665cd3@f12g2000yqn.googlegroups.com> <4b484bc6-f903-40a0-a8dc-9a00c8fc002a@k5g2000yqf.googlegroups.com> Message-ID: <4B66076C.4050102@mrabarnett.plus.com> marc magrans de abril wrote: > Hi! > > ...I have found a good enough solution, although it only works if the > number of patterns (clusters) is not very big: > def classify(f): > THERESHOLD=0.1 > > patterns={} > for l in enumerate(f): > found = False > for p,c in patterns.items(): > if dist(l,p) < THERESHOLD: > found=True > patterns[p] = c +1 > > if not found: > patterns[l] = 1 > > return patterns > > This algorithm is O(n*np*m^2). Where n is the number of logs, np the > number of patterns, and m is the log length (i.e. m^2 is the distance > cost). So it's way better O(n^2*m^2) and I can run it for some hours > to get back the results. > > I wonder if there is a single threaded/process clustering algorithm > than runs in O(n)? > Your original code used the first entry in the remaining logs for each pattern, but your new code stores the patterns in a dict, which is unordered, so you might get different results. But that doesn't matter, because your new code increments the count when it has found a match, and then continues looking, so it might match and increment more than once. Finally, your original code treated it as a match if distance <= threshold but your new code treats it as a match if distance < threshold. patterns = [] for x in logs: for index, (pat, count) in enumerate(patterns): if dist(pat, x) <= THRESHOLD: patterns[index] = pat, count + 1 break else: # Didn't break out of the loop, therefore no match. patterns.append((x, 1)) From python at mrabarnett.plus.com Sun Jan 31 17:49:56 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 31 Jan 2010 22:49:56 +0000 Subject: create a string of variable lenght In-Reply-To: <4b660056$0$1131$4fafbaef@reader3.news.tin.it> References: <4b65729b$0$1126$4fafbaef@reader3.news.tin.it> <4b660056$0$1131$4fafbaef@reader3.news.tin.it> Message-ID: <4B660914.1040409@mrabarnett.plus.com> Tracubik wrote: > Il Sun, 31 Jan 2010 13:46:16 +0100, G?nther Dietrich ha > scritto: > >> Maybe you might solve this if you decode your string to unicode. >> Example: >> >> |>>> euro = "?" >> |>>> len(euro) >> |3 >> |>>> u_euro = euro.decode('utf_8') >> |>>> len(u_euro) >> |1 >> >> Adapt the encoding ('utf_8' in my example) to whatever you use. >> >> Or create the unicode string directly: >> >> |>>> u_euro = u'?' >> |>>> len(u_euro) >> |1 >> >> >> >> Best regards, >> >> G?nther > > thank you, your two solution is really interesting. > is there a possible to set unicode encoding by default for my python > scripts? > i've tried inserting > # -*- coding: utf-8 -*- > > at the beginning of my script but doesn't solve the problem That tells Python which encoding the file is using, but you still need to save the file in that encoding. From newssw at gmail.com Sun Jan 31 18:06:25 2010 From: newssw at gmail.com (Mik0b0) Date: Sun, 31 Jan 2010 15:06:25 -0800 (PST) Subject: ftp.storlines error References: <24ffaab0-f319-4019-97cf-56a7531e3eef@n7g2000yqb.googlegroups.com> Message-ID: On Feb 1, 12:19?am, Chris Rebert wrote: > On Sun, Jan 31, 2010 at 2:07 PM, Mik0b0 wrote: > > Good day/night/etc. > > I am rather a newb in Python (learning Python 3). I am trying to > > create a small script for FTP file uploads ?on my home network. The > > script looks like this: > > > from ftplib import FTP > > ftp=FTP('10.0.0.1') > > ftp.login('mike','*****') > > directory='/var/www/blabla/' > > ftp.cwd(directory) > > ftp.retrlines('LIST') > > print('<- - - - - - - - - >') > > file_to_change='test' > > file=1 > > file=open(file_to_change,'w') > > Here you open the file in (w)rite mode. Also, don't call it `file` as > that shadows the name of the built-in type. > > > text='test' > > file.write(text) > > And indeed, here you've written something to it. > > > ftp.storlines('STOR ' + file_to_change,file) > > storlines() needs a file opened in (r)ead mode however, hence the > error. Obviously, it needs to read the contents of the file in order > to send it over the network. > Either do the writing separately, close the file, and then open it > again in read mode, or open it in one of the modes that allows for > both reading and writing [see help(open) for details]. > > Cheers, > Chris > --http://blog.rebertia.com > > > ftp.retrlines('LIST') > > file.close() > > > The output is like this: > > Traceback (most recent call last): > > ?File "ftp.py", line 13, in > > ? ?ftp.storlines('STOR ' + file_to_change,i) > > ?File "/usr/lib/python3.1/ftplib.py", line 474, in storlines > > ? ?buf = fp.readline() > > IOError: not readable > > > What is wrong? Thanks Chris, the final version looks like this and it works: from ftplib import FTP ftp=FTP('10.0.0.1') ftp.login('mike','******') directory='/var/www/blabla/' ftp.cwd(directory) ftp.retrlines('LIST') print('<- - - - - - - - - >') file_to_change='test.php' i=1 i=open(file_to_change,'w') text='test' i.write(text) i.close() i=open(file_to_change,'rb') ftp.storbinary('STOR ' + file_to_change,i) ftp.retrlines('LIST') i.close() From clp2 at rebertia.com Sun Jan 31 18:40:36 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 31 Jan 2010 15:40:36 -0800 Subject: Python and Ruby In-Reply-To: <0375f245$0$1309$c3e8da3@news.astraweb.com> References: <0375f245$0$1309$c3e8da3@news.astraweb.com> Message-ID: <50697b2c1001311540m1810839ic985f3071f283558@mail.gmail.com> On Sun, Jan 31, 2010 at 2:36 PM, Steven D'Aprano wrote: > On Sun, 31 Jan 2010 04:28:41 -0800, Ed Keith wrote: >> In most functional languages you just name a function to access it and >> you do it ALL the time. >> >> for example, in if you have a function 'f' which takes two parameters to >> call the function and get the result you use: >> >> ?f 2 3 >> >> If you want the function itself you use: >> >> ? ?f > > How do you call a function of no arguments? It's not really a function in that case, it's just a named constant. (Recall that functions don't/can't have side-effects.) Cheers, Chris -- http://blog.rebertia.com From burslem2001 at yahoo.com Sun Jan 31 19:01:43 2010 From: burslem2001 at yahoo.com (gazza) Date: Sun, 31 Jan 2010 16:01:43 -0800 (PST) Subject: gmtime References: <62938b5b-7eda-426c-b91d-3c7061840b2d@l26g2000yqd.googlegroups.com> Message-ID: <5c1b6f40-6620-4c0e-8e93-e04a9c65d922@l19g2000yqb.googlegroups.com> On Jan 31, 3:27?pm, gazza wrote: > Hi, > > I am trying to discover how to obtain the correct time of say CST/ > America and EST/America in python? > > Any help on this would be appreciated. > > Thanks, > Garyc I found some information. Someone suggested I use the pytz library? Cheers, Garyc From solipsis at pitrou.net Sun Jan 31 19:04:57 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 1 Feb 2010 00:04:57 +0000 (UTC) Subject: recv_into(bytearray) complains about a "pinned buffer" References: <8dff1c70-c141-4f0e-aebb-3a92733b272b@o28g2000yqh.googlegroups.com> Message-ID: Hello Andrew, > I don't even know what a "pinned buffer" means, and searching python.org > isn't helpful. > > Using a bytearray in Python 3.1.1 *does* work: > [...] Agreed, the error message is cryptic. The problem is that socket.recv_into() in 2.6 doesn't recognize the new buffer API which is needed to accept bytearray objects. (it does in 3.1, because the old buffer API doesn't exist anymore there) You could open an issue on the bug tracker for this. Thank you Antoine. From arnodel at googlemail.com Sun Jan 31 19:14:52 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 01 Feb 2010 00:14:52 +0000 Subject: Python and Ruby References: <0375f245$0$1309$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano writes: > On Sun, 31 Jan 2010 04:28:41 -0800, Ed Keith wrote: > >> In most functional languages you just name a function to access it and >> you do it ALL the time. >> >> for example, in if you have a function 'f' which takes two parameters to >> call the function and get the result you use: >> >> f 2 3 >> >> If you want the function itself you use: >> >> f > > How do you call a function of no arguments? In a functional language, a function of no arguments will always return the same value. So, from a non-functional point of vue, f is both the function and its value. -- Arnaud From kyp at stsci.edu Sun Jan 31 19:22:32 2010 From: kyp at stsci.edu (Kyp) Date: Sun, 31 Jan 2010 16:22:32 -0800 (PST) Subject: iglob performance no better than glob References: <5ebd4965-32fc-4a03-94dd-c61ccdc9d630@f5g2000vbm.googlegroups.com> <87k4uyw3mi.fsf@castleamber.com> Message-ID: On Jan 31, 1:06?pm, John Bokma wrote: > Kyp writes: > > Is there a way to get the first X # of files from a dir with lots of > > files, that does not take a long time to run? > > Assuming Linux: what does time > > ?ls thedir | head > > give? > > with thedir the name of the actual dir about 3 seconds. 3.086u 0.201s 0:03.32 98.7% 0+0k 0+0io 0pf+0w > > Also how many is many files? over 100K (I know I should not do that, but it's a temp dir holding files to be transferred) thanx, mark From kyp at stsci.edu Sun Jan 31 19:23:05 2010 From: kyp at stsci.edu (Kyp) Date: Sun, 31 Jan 2010 16:23:05 -0800 (PST) Subject: iglob performance no better than glob References: <5ebd4965-32fc-4a03-94dd-c61ccdc9d630@f5g2000vbm.googlegroups.com> Message-ID: <449a19de-42f7-4ae3-8099-99c27572adc8@m16g2000yqc.googlegroups.com> On Jan 31, 2:44?pm, Peter Otten <__pete... at web.de> wrote: > Kyp wrote: > > I have a dir with a large # of files that I need to perform operations > > on, but only needing to access a subset of the files, i.e. the first > > 100 files. > > > Using glob is very slow, so I ran across iglob, which returns an > > iterator, which seemed just like what I wanted. I could iterate over > > the files that I wanted, not having to read the entire dir. > > > So the iglob was faster, but accessing the first file took about the > > same time as glob.glob. > > > Here's some code to compare glob vs. iglob performance, ?it outputs > > the time before/after a glob.iglob('*.*') files.next() sequence and a > > glob.glob('*.*') sequence. > > > #!/usr/bin/env python > > > import glob,time > > print '\nTest of glob.iglob' > > print 'before ? ? ? iglob:', time.asctime() > > files = glob.iglob('*.*') > > print 'after ? ? ? ?iglob:',time.asctime() > > print files.next() > > print 'after files.next():', time.asctime() > > > print '\nTest of glob.glob' > > print 'before ? ? ? ?glob:', time.asctime() > > files = glob.glob('*.*') > > print 'after ? ? ? ? glob:',time.asctime() > > > Here are the results: > > > Test of glob.iglob > > before ? ? ? iglob: Sun Jan 31 11:09:08 2010 > > after ? ? ? ?iglob: Sun Jan 31 11:09:08 2010 > > foo.bar > > after files.next(): Sun Jan 31 11:09:59 2010 > > > Test of glob.glob > > before ? ? ? ?glob: Sun Jan 31 11:09:59 2010 > > after ? ? ? ? glob: Sun Jan 31 11:10:51 2010 > > > The results are about the same for the 2 approaches, both took about > > 51 seconds. Am I doing something wrong with iglob? > > No, but iglob() being lazy is pointless in your case because it uses > os.listdir() and fnmatch.filter() underneath which both read the whole > directory before returning anything. > > > Is there a way to get the first X # of files from a dir with lots of > > files, that does not take a long time to run? > > Here's my attempt. It turned out to be more work than expected, so I cut a > few corners. It's Linux-only "works on my machine" code, but may give you > some hints on how to proceed. > > from ctypes import * > import fnmatch > import glob > import os > import re > from itertools import ifilter, imap > > class dirent(Structure): > ? ? "works on my machine ;)" > ? ? _fields_ = [ > ? ? ? ? ("d_ino", c_long), > ? ? ? ? ("d_off", c_long), > ? ? ? ? ("d_reclen", c_ushort), > ? ? ? ? ("d_type", c_ubyte), > ? ? ? ? ("d_name", c_char*256)] > > direntp = POINTER(dirent) > > LIBC = "libc.so.6" > cdll.LoadLibrary(LIBC) > libc = CDLL(LIBC) > libc.readdir.restype = direntp > > def diriter(dir): > ? ? "lazy partial replacement for os.listdir()" > ? ? # errors? what errors? > ? ? dirp = libc.opendir(dir) > ? ? if not dirp: > ? ? ? ? return > ? ? try: > ? ? ? ? while True: > ? ? ? ? ? ? ep = libc.readdir(dirp) > ? ? ? ? ? ? if not ep: > ? ? ? ? ? ? ? ? break > ? ? ? ? ? ? yield ep.contents.d_name > ? ? finally: > ? ? ? ? libc.closedir(dirp) > > def filter(names, pattern): > ? ? "lazy partial replacement for fnmatch.filter()" > ? ? import posixpath > > ? ? pattern = os.path.normcase(pattern) > ? ? r = fnmatch.translate(pattern) > ? ? r = re.compile(r) > > ? ? if os.path is not posixpath: > ? ? ? ? names = imap(os.path.normcase, names) > > ? ? return ifilter(r.match, names) > > def globiter(path): > ? ? "lazy partial replacement for glob.glob()" > ? ? dir, filename = os.path.split(path) > ? ? if glob.has_magic(dir): > ? ? ? ? raise ValueError("wildcards in directory not supported") > ? ? return filter(diriter(dir), filename) > > if __name__ == "__main__": > ? ? import sys > ? ? [pattern] = sys.argv[1:] > ? ? for name in globiter(pattern): > ? ? ? ? print name > > Peter I'll give it a try, thanx for the reply. mark From steven at REMOVE.THIS.cybersource.com.au Sun Jan 31 19:25:44 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 01 Feb 2010 00:25:44 GMT Subject: Python and Ruby References: <0375f245$0$1309$c3e8da3@news.astraweb.com> Message-ID: On Sun, 31 Jan 2010 15:40:36 -0800, Chris Rebert wrote: > On Sun, Jan 31, 2010 at 2:36 PM, Steven D'Aprano > wrote: >> On Sun, 31 Jan 2010 04:28:41 -0800, Ed Keith wrote: >>> In most functional languages you just name a function to access it and >>> you do it ALL the time. >>> >>> for example, in if you have a function 'f' which takes two parameters >>> to call the function and get the result you use: >>> >>> ?f 2 3 >>> >>> If you want the function itself you use: >>> >>> ? ?f >> >> How do you call a function of no arguments? > > It's not really a function in that case, it's just a named constant. > (Recall that functions don't/can't have side-effects.) >>> time.time(), random.random() (1264983502.7505889, 0.29974255140479633) >>> time.time(), random.random() (1264983505.9283719, 0.74207867411026329) They don't look terribly constant to me. There is a difference between a function that does "give me whatever value is specified by a fixed description" and a function that does "give me a fixed value". -- Steven From e_d_k at yahoo.com Sun Jan 31 19:27:09 2010 From: e_d_k at yahoo.com (Ed Keith) Date: Sun, 31 Jan 2010 16:27:09 -0800 (PST) Subject: Python and Ruby In-Reply-To: <0375f245$0$1309$c3e8da3@news.astraweb.com> Message-ID: <553505.49272.qm@web58905.mail.re1.yahoo.com> --- On Sun, 1/31/10, Steven D'Aprano wrote: > From: Steven D'Aprano > Subject: Re: Python and Ruby > To: python-list at python.org > Date: Sunday, January 31, 2010, 5:36 PM > On Sun, 31 Jan 2010 04:28:41 -0800, > Ed Keith wrote: > > > In most functional languages you just name a function > to access it and > > you do it ALL the time. > > > > for example, in if you have a function 'f' which takes > two parameters to > > call the function and get the result you use: > > > >? f 2 3 > > > > If you want the function itself you use: > > > >? ? f > > How do you call a function of no arguments? > > In a 'pure' functional language a function with no arguments is, by definition, a constant. This is because a 'pure' function will always return the same result whenever given the same arguments. so if it has no argument it always returns a constant value. -EdK Ed Keith e_d_k at yahoo.com Blog: edkeith.blogspot.com From bblais at bryant.edu Sun Jan 31 19:27:39 2010 From: bblais at bryant.edu (Brian Blais) Date: Sun, 31 Jan 2010 19:27:39 -0500 Subject: odd drawing problem with turtle.py Message-ID: I'm on Python 2.5, but using the updated turtle.py Version 1.0.1 - 24. 9. 2009. The following script draws 5 circles, which it is supposed to, but then doesn't draw the second turtle which is supposed to simply move forward. Any ideas? from turtle import * from numpy.random import randint resetscreen() class Circle(object): def __init__(self,x,y,r,color): self.x=x self.y=y self.r=r self.color=color self.turtle=Turtle(visible=False) self.turtle.tracer(False) self.draw() def draw(self): self.turtle.penup() self.turtle.setposition(self.x,self.y) self.turtle.setheading(0) self.turtle.backward(self.r) self.turtle.pendown() self.turtle.fill(True) self.turtle.pencolor("black") self.turtle.fillcolor(self.color) self.turtle.circle(self.r) self.turtle.fill(False) self.turtle.penup() for i in range(5): c=Circle(randint(-350,350),randint(-250,250),10,"red") T=Turtle() T.forward(100) T.forward(100) thanks, bb -- Brian Blais bblais at bryant.edu http://web.bryant.edu/~bblais http://bblais.blogspot.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From john at castleamber.com Sun Jan 31 19:47:42 2010 From: john at castleamber.com (John Bokma) Date: Sun, 31 Jan 2010 18:47:42 -0600 Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> <87y6jef1df.fsf@castleamber.com> <0375f27a$0$1309$c3e8da3@news.astraweb.com> Message-ID: <878wbdye6p.fsf@castleamber.com> Steven D'Aprano writes: > On Sun, 31 Jan 2010 14:47:08 -0600, John Bokma wrote: > >> An editor can correct the indenting of the braces example but can't with >> this one. >> >> if x: >> if y: >> foo() >> else: >> bar() >> >> While braces might be considered redundant they are not when for one >> reason or another formatting is lost or done incorrectly. > > I've heard this argument before, and I don't buy it. Why should we expect > the editor to correct malformed code? Or a prettyfier. It doesn't matter. The point is that with braces there *is* redundancy that be used to fix the code. > Would you expect your editor to correct this malformed code? > > result = sin(x+)y Nice straw man. Let me repeat again: I am ok with how Python works. To be honest I think it's cleaner compared to using {}. But in there are real life examples in which Python code will break where code with braces will survive. -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From clp2 at rebertia.com Sun Jan 31 19:50:50 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 31 Jan 2010 16:50:50 -0800 Subject: Python and Ruby In-Reply-To: References: <0375f245$0$1309$c3e8da3@news.astraweb.com> Message-ID: <50697b2c1001311650qb7b1025p1e5b5b2d0ef21c8c@mail.gmail.com> On Sun, Jan 31, 2010 at 4:25 PM, Steven D'Aprano wrote: > On Sun, 31 Jan 2010 15:40:36 -0800, Chris Rebert wrote: >> On Sun, Jan 31, 2010 at 2:36 PM, Steven D'Aprano >> wrote: >>> On Sun, 31 Jan 2010 04:28:41 -0800, Ed Keith wrote: >>>> In most functional languages you just name a function to access it and >>>> you do it ALL the time. >>>> >>>> for example, in if you have a function 'f' which takes two parameters >>>> to call the function and get the result you use: >>>> >>>> ?f 2 3 >>>> >>>> If you want the function itself you use: >>>> >>>> ? ?f >>> >>> How do you call a function of no arguments? >> >> It's not really a function in that case, it's just a named constant. >> (Recall that functions don't/can't have side-effects.) > > >>>> time.time(), random.random() > (1264983502.7505889, 0.29974255140479633) >>>> time.time(), random.random() > (1264983505.9283719, 0.74207867411026329) > > > They don't look terribly constant to me. Those aren't functions in the pure functional programming sense; which is unsurprising since Python isn't a [pure] functional language. They both involve side-effects. time() does I/O to the clock chip to see what time it is, and random() uses and changes a global seed value variable (which, in a double-whammy, takes its initial value from time()). Pure functions must be referentially transparent [http://en.wikipedia.org/wiki/Referential_transparency_(computer_science)], and as you've demonstrated, neither of those Python functions qualify. Cheers, Chris -- http://blog.rebertia.com From john at castleamber.com Sun Jan 31 19:53:16 2010 From: john at castleamber.com (John Bokma) Date: Sun, 31 Jan 2010 18:53:16 -0600 Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> <87y6jef1df.fsf@castleamber.com> <0375f27a$0$1309$c3e8da3@news.astraweb.com> Message-ID: <874om1ydxf.fsf@castleamber.com> Steven D'Aprano writes: > On Sun, 31 Jan 2010 14:47:08 -0600, John Bokma wrote: > >> An editor can correct the indenting of the braces example but can't with >> this one. >> >> if x: >> if y: >> foo() >> else: >> bar() >> >> While braces might be considered redundant they are not when for one >> reason or another formatting is lost or done incorrectly. > > I've heard this argument before, and I don't buy it. Why should we expect > the editor to correct malformed code? I do expect my editor to assist me in coding. In Emacs I have to do some effort to enter the broken C code in the earlier post, and when I reformat the code, it will be lined out correctly. I can't do that with the above example, because it's correctly formatted. You don't have to buy my argument, I am not selling it. While in the past I wrote that an editor can't make you that more productive I want to take that back, on the record. Since I've switched to Emacs the editor has saved me several times from minor issues. Either because it refused to indent correctly thanks to a missing closing }, ), ] or other error. With the correct mode in Emacs one gets, in my experience, immediate feedback when making mistakes one otherwise find during the run/compiling phase. Note that I am also not selling Emacs. It's free after all. -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From benjamin.kaplan at case.edu Sun Jan 31 19:54:17 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Sun, 31 Jan 2010 19:54:17 -0500 Subject: create a string of variable lenght In-Reply-To: <4b660056$0$1131$4fafbaef@reader3.news.tin.it> References: <4b65729b$0$1126$4fafbaef@reader3.news.tin.it> <4b660056$0$1131$4fafbaef@reader3.news.tin.it> Message-ID: On Sun, Jan 31, 2010 at 5:12 PM, Tracubik wrote: > Il Sun, 31 Jan 2010 13:46:16 +0100, G?nther Dietrich ha > scritto: > >> Maybe you might solve this if you decode your string to unicode. >> Example: >> >> |>>> euro = "?" >> |>>> len(euro) >> |3 >> |>>> u_euro = euro.decode('utf_8') >> |>>> len(u_euro) >> |1 >> >> Adapt the encoding ('utf_8' in my example) to whatever you use. >> >> Or create the unicode string directly: >> >> |>>> u_euro = u'?' >> |>>> len(u_euro) >> |1 >> >> >> >> Best regards, >> >> G?nther > > thank you, your two solution is really interesting. > is there a possible to set unicode encoding by default for my python > scripts? > i've tried inserting > # -*- coding: utf-8 -*- > > at the beginning of my script but doesn't solve the problem First of all, if you haven't read this before, please do. It will make this much clearer. http://www.joelonsoftware.com/articles/Unicode.html To reiterate: UTF-8 IS NOT UNICODE!!!! In Python 2, '*' signifies a byte string. It is read as a sequence of bytes and interpreted as a sequence of bytes When Python encounters the sequence 0x27 0xe2 0x82 0xac 0x27 in the code (the UTF-8 bytes for '?') it interprets it as 3 bytes between the two quotes. It doesn't care about characters or anything like that. u'*' signifies a Unicode string. Python will attempt to convert the sequence of bytes into a sequence of characters. It can use any encoding for that: cp1252, utf-8, MacRoman, ISO-8859-15. UTF-8 isn't special, it's just one of the few encodings capable of storing all of the possible Unicode characters. What the line at the top says is that the file should be read using UTF-8. Byte strings are still just sequences of bytes- this doesn't affect them. But any Unicode string will be decoded using UTF-8. IF python looks at the above sequence of bytes as a Unicode string, it views the 3 bytes as a single character. When you ask for it's length, it returns the number of characters. Solution to your problem: in addition to keeping the #-*- coding ... line, go with G?nther's advice and use Unicode strings. > -- > http://mail.python.org/mailman/listinfo/python-list > From purui.wang at gmail.com Sun Jan 31 20:19:41 2010 From: purui.wang at gmail.com (pograph) Date: Sun, 31 Jan 2010 17:19:41 -0800 (PST) Subject: gmtime References: <62938b5b-7eda-426c-b91d-3c7061840b2d@l26g2000yqd.googlegroups.com> <5c1b6f40-6620-4c0e-8e93-e04a9c65d922@l19g2000yqb.googlegroups.com> Message-ID: <04b2f529-a66f-431f-8e76-3c773a183653@b7g2000pro.googlegroups.com> On Jan 31, 4:01?pm, gazza wrote: > On Jan 31, 3:27?pm, gazza wrote: > > > Hi, > > > I am trying to discover how to obtain the correct time of say CST/ > > America and EST/America in python? > > > Any help on this would be appreciated. > > > Thanks, > > Garyc > > I found some information. Someone suggested I use the pytz library? > > Cheers, > Garyc tz = pytz.timezone('US/Pacific') t = datetime.datetime.now(tz) From steven at REMOVE.THIS.cybersource.com.au Sun Jan 31 20:22:23 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 01 Feb 2010 01:22:23 GMT Subject: Python and Ruby References: <0375f245$0$1309$c3e8da3@news.astraweb.com> Message-ID: On Sun, 31 Jan 2010 16:50:50 -0800, Chris Rebert wrote: >>>> How do you call a function of no arguments? >>> >>> It's not really a function in that case, it's just a named constant. >>> (Recall that functions don't/can't have side-effects.) >> >> >>>>> time.time(), random.random() >> (1264983502.7505889, 0.29974255140479633) >>>>> time.time(), random.random() >> (1264983505.9283719, 0.74207867411026329) >> >> >> They don't look terribly constant to me. > > Those aren't functions in the pure functional programming sense; which > is unsurprising since Python isn't a [pure] functional language. They > both involve side-effects. time() does I/O to the clock chip to see what > time it is, and random() uses and changes a global seed value variable > (which, in a double-whammy, takes its initial value from time()). Yes, but these tasks -- get the time, get a (pseudo) random number -- are not unique to Python. Surely even Lisp and Haskell code will sometimes need to know the time. Whether they are "pure functions" (functions in the mathematical sense) or impure, they're still functions in some sense. How do you deal with such impure functions? -- Steven From e_d_k at yahoo.com Sun Jan 31 20:31:05 2010 From: e_d_k at yahoo.com (Ed Keith) Date: Sun, 31 Jan 2010 17:31:05 -0800 (PST) Subject: Python and Ruby In-Reply-To: Message-ID: <934786.78871.qm@web58905.mail.re1.yahoo.com> --- On Sun, 1/31/10, Steven D'Aprano wrote: > From: Steven D'Aprano > Subject: Re: Python and Ruby > To: python-list at python.org > Date: Sunday, January 31, 2010, 8:22 PM > On Sun, 31 Jan 2010 16:50:50 -0800, > Chris Rebert wrote: > > >>>> How do you call a function of no > arguments? > >>> > >>> It's not really a function in that case, it's > just a named constant. > >>> (Recall that functions don't/can't have > side-effects.) > >> > >> > >>>>> time.time(), random.random() > >> (1264983502.7505889, 0.29974255140479633) > >>>>> time.time(), random.random() > >> (1264983505.9283719, 0.74207867411026329) > >> > >> > >> They don't look terribly constant to me. > > > > Those aren't functions in the pure functional > programming sense; which > > is unsurprising since Python isn't a [pure] functional > language. They > > both involve side-effects. time() does I/O to the > clock chip to see what > > time it is, and random() uses and changes a global > seed value variable > > (which, in a double-whammy, takes its initial value > from time()). > > Yes, but these tasks -- get the time, get a (pseudo) random > number -- are > not unique to Python. Surely even Lisp and Haskell code > will sometimes > need to know the time. Whether they are "pure functions" > (functions in > the mathematical sense) or impure, they're still functions > in some sense. > How do you deal with such impure functions? > > > You pass it a monad (http://en.wikipedia.org/wiki/Monad_(functional_programming)). -EdK Ed Keith e_d_k at yahoo.com Blog: edkeith.blogspot.com From steven at REMOVE.THIS.cybersource.com.au Sun Jan 31 20:33:26 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 01 Feb 2010 01:33:26 GMT Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> <87y6jef1df.fsf@castleamber.com> <0375f27a$0$1309$c3e8da3@news.astraweb.com> <878wbdye6p.fsf@castleamber.com> Message-ID: On Sun, 31 Jan 2010 18:47:42 -0600, John Bokma wrote: > Steven D'Aprano writes: > >> On Sun, 31 Jan 2010 14:47:08 -0600, John Bokma wrote: >> >>> An editor can correct the indenting of the braces example but can't >>> with this one. >>> >>> if x: >>> if y: >>> foo() >>> else: >>> bar() >>> >>> While braces might be considered redundant they are not when for one >>> reason or another formatting is lost or done incorrectly. >> >> I've heard this argument before, and I don't buy it. Why should we >> expect the editor to correct malformed code? > > Or a prettyfier. It doesn't matter. The point is that with braces there > *is* redundancy that be used to fix the code. Prettyfiers are significant in languages that allow braces (or begin/end tokens) and indentation to go out of sync. Since that can't happen with Python, it's not a problem that needs solving. Prettyfiers exist to work around a limitation of languages where indentation is not significant. Arguing that an advantage of braces is that prettyfiers can work with them easily is a silly argument. That's like saying an advantage of horse- drawn buggies over cars is that it will go faster when you hit the horse with a whip. That's true, but you only need the whip because of the lack of accelerator pedal. A much better argument would be that an advantage of significant indentation is that you no longer need a prettyfier, and a second advantage is a major reduction in flame wars over coding styles. >> Would you expect your editor to correct this malformed code? >> >> result = sin(x+)y > > Nice straw man. It's not a straw man. It's a serious argument. There is an infinite number of problems with malformed code that your editor can't fix, and your prettifier can't deal with. Why should we care if indentation is one more? There are tools out there (such as some web forum software) that corrupt whitespace. Those tools are broken, and if you (generic you), as a developer, are relying on those tools to code with, then shame on you. And if you're the creator of such broken tools, then shame on you more. You wouldn't arbitrarily decide to remove leading "E"s from the user's text, or trailing full stops, so why do you arbitrarily remove whitespace? > Let me repeat again: I am ok with how Python works. To be honest I think > it's cleaner compared to using {}. But in there are real life examples > in which Python code will break where code with braces will survive. Yes. So what? They are rare and insignificant in practice. For every line of code that you get via a webforum that mangles indentation, you get ten thousand lines of code from some place that doesn't. So long as code is written by and for human beings, the benefit of significant indentation is 100% practical, and the practical benefit of braces will remain insignificant. -- Steven From clp2 at rebertia.com Sun Jan 31 20:33:51 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 31 Jan 2010 17:33:51 -0800 Subject: Python and Ruby In-Reply-To: References: <0375f245$0$1309$c3e8da3@news.astraweb.com> Message-ID: <50697b2c1001311733r5cc6181fy1c71c88e035a5efd@mail.gmail.com> On Sun, Jan 31, 2010 at 5:22 PM, Steven D'Aprano wrote: > On Sun, 31 Jan 2010 16:50:50 -0800, Chris Rebert wrote: >>>>> How do you call a function of no arguments? >>>> >>>> It's not really a function in that case, it's just a named constant. >>>> (Recall that functions don't/can't have side-effects.) >>> >>> >>>>>> time.time(), random.random() >>> (1264983502.7505889, 0.29974255140479633) >>>>>> time.time(), random.random() >>> (1264983505.9283719, 0.74207867411026329) >>> >>> >>> They don't look terribly constant to me. >> >> Those aren't functions in the pure functional programming sense; which >> is unsurprising since Python isn't a [pure] functional language. They >> both involve side-effects. time() does I/O to the clock chip to see what >> time it is, and random() uses and changes a global seed value variable >> (which, in a double-whammy, takes its initial value from time()). > > Yes, but these tasks -- get the time, get a (pseudo) random number -- are > not unique to Python. Surely even Lisp and Haskell code will sometimes > need to know the time. Whether they are "pure functions" (functions in > the mathematical sense) or impure, they're still functions in some sense. > How do you deal with such impure functions? Make the state explicit, i.e. monads or uniqueness typing. In the case of random(), you have it take the seed explicitly as an argument and have it return a new seed along with the "randomly" generated number. In the case of time(), you pass around an argument to represent the state of the outside world. I don't quite grok monads, but http://en.wikipedia.org/wiki/Uniqueness_type has a decent example of the general uniqueness type approach. Cheers, Chris -- http://blog.rebertia.com From andrewdalke at gmail.com Sun Jan 31 20:56:32 2010 From: andrewdalke at gmail.com (Andrew Dalke) Date: Sun, 31 Jan 2010 17:56:32 -0800 (PST) Subject: recv_into(bytearray) complains about a "pinned buffer" References: <8dff1c70-c141-4f0e-aebb-3a92733b272b@o28g2000yqh.googlegroups.com> Message-ID: <58ef4ced-bd16-44c3-8c44-678b671c4f9a@z26g2000yqm.googlegroups.com> On Feb 1, 1:04?am, Antoine Pitrou wrote: > The problem is that socket.recv_into() in 2.6 doesn't recognize the new > buffer API which is needed to accept bytearray objects. > (it does in 3.1, because the old buffer API doesn't exist anymore there) That's about what I thought it was, but I don't know if this was a deliberate choice or accidental. BTW, 2.7 (freshly built from version control) also has the same exception. > You could open an issue on the bug tracker for this. I've done that. It's http://bugs.python.org/issue7827 . Cheers! Andrew dalke at dalkescientific.com From andrej.mitrovich at gmail.com Sun Jan 31 20:59:41 2010 From: andrej.mitrovich at gmail.com (Andrej Mitrovic) Date: Sun, 31 Jan 2010 17:59:41 -0800 (PST) Subject: Python distutils build problems with MinGW Message-ID: <67168350-0f64-454c-a89b-767cba82e439@f12g2000yqn.googlegroups.com> Hi, I've made a similar post on the Cython mailing list, however I think this is more python-specific. I'm having trouble setting up distutils to use MinGW instead of Visual Studio when building a module. Even tho I've just uninstalled VS, and cleared out any leftover VS environment variables, distutils keeps wanting to use it. The steps I took: Fresh installation of Python 3.1.1 Successfully installed MinGW, added to the path variable (gcc in command prompt works) Successfully installed Cython, imports from Cython in Python work. Added a distutils.cfg file in \Python31\Lib\distutils\ directory with: [build] compiler=mingw32 (also tried adding [build_ext] compiler=mingw32) There's a demo setup.py module that came with Cython, I tried the following commands: ---------------------------------------------------- > python setup.py build_ext --inplace error: Unable to find vcvarsall.bat > python setup.py build error: Unable to find vcvarsall.bat ---------------------------------------------------- I'm having the exact same issue with trying to build the Polygon library via MinGW. In fact, the reason I had installed Visual Studio in the first place was to be able to build the Polygon library, since I was having these errors. What do I need to do to make distutils/python use MinGW? From kirby.urner at gmail.com Sun Jan 31 21:11:34 2010 From: kirby.urner at gmail.com (kirby urner) Date: Sun, 31 Jan 2010 18:11:34 -0800 Subject: [Edu-sig] odd drawing problem with turtle.py In-Reply-To: References: Message-ID: I don't see where you've defined a Turtle class to instantiate sir. Perhaps rename Circle to Turtle and rewrite the circle-drawing expression as: > c=Turtle(randint(-350,350),randint(-250,250),10,"red") You are making progress with a wrapper class for the Standard Library turtle. That's a well-known design pattern and a good way to get extra features sometimes. Kirby On Sun, Jan 31, 2010 at 4:27 PM, Brian Blais wrote: > I'm on Python 2.5, but using the updated turtle.py?Version 1.0.1 - 24. 9. > 2009. ?The following script draws 5 circles, which it is supposed to, but > then doesn't draw the second turtle which is supposed to simply move > forward. ?Any ideas? > from turtle import * > from numpy.random import randint > resetscreen() > class Circle(object): > ?? ?def __init__(self,x,y,r,color): > ?? ? ? ?self.x=x > ?? ? ? ?self.y=y > ?? ? ? ?self.r=r > ?? ? ? ?self.color=color > > ?? ? ? ?self.turtle=Turtle(visible=False) > ?? ? ? ?self.turtle.tracer(False) > ?? ? ? ?self.draw() > > ?? ?def draw(self): > ?? ? ? ?self.turtle.penup() > ?? ? ? ?self.turtle.setposition(self.x,self.y) > ?? ? ? ?self.turtle.setheading(0) > ?? ? ? ?self.turtle.backward(self.r) > ?? ? ? ?self.turtle.pendown() > ?? ? ? ?self.turtle.fill(True) > ?? ? ? ?self.turtle.pencolor("black") > ?? ? ? ?self.turtle.fillcolor(self.color) > ?? ? ? ?self.turtle.circle(self.r) > ?? ? ? ?self.turtle.fill(False) > ?? ? ? ?self.turtle.penup() > > for i in range(5): > ?? ?c=Circle(randint(-350,350),randint(-250,250),10,"red") > > > T=Turtle() > T.forward(100) > T.forward(100) > > > > > > > > thanks, > > bb > -- > Brian Blais > bblais at bryant.edu > http://web.bryant.edu/~bblais > http://bblais.blogspot.com/ > > > > _______________________________________________ > Edu-sig mailing list > Edu-sig at python.org > http://mail.python.org/mailman/listinfo/edu-sig > > From martin at v.loewis.de Sun Jan 31 21:30:56 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Mon, 01 Feb 2010 03:30:56 +0100 Subject: recv_into(bytearray) complains about a "pinned buffer" In-Reply-To: <8dff1c70-c141-4f0e-aebb-3a92733b272b@o28g2000yqh.googlegroups.com> References: <8dff1c70-c141-4f0e-aebb-3a92733b272b@o28g2000yqh.googlegroups.com> Message-ID: <4B663CE0.4030302@v.loewis.de> > In Python 2.6 I can't socket.recv_into(a byte array instance). I get a > TypeError which complains about a "pinned buffer". I have only an > inkling of what that means. A pinned buffer is one that cannot move in memory, even if another thread tries to behind your back. Typically, resizable containers are not inherently pinned, and "a user" (i.e. the API function) must explicitly pin it, which recv_into fails to do. > Is this a bug in Python 2.6 or a deliberate choice regarding > implementation concerns I don't know about? It's actually a bug also that you pass an array; doing so *should* give the very same error. It may be that the bugs surrounding buffers will never get fully resolved in the lifetime of Python 2.x, so I would probably just ignore recv_into. Regards, Martin From andrej.mitrovich at gmail.com Sun Jan 31 22:03:27 2010 From: andrej.mitrovich at gmail.com (Andrej Mitrovic) Date: Sun, 31 Jan 2010 19:03:27 -0800 (PST) Subject: Python distutils build problems with MinGW References: <67168350-0f64-454c-a89b-767cba82e439@f12g2000yqn.googlegroups.com> Message-ID: On Feb 1, 2:59?am, Andrej Mitrovic wrote: > Hi, > > I've made a similar post on the Cython mailing list, however I think > this is more python-specific. I'm having trouble setting up distutils > to use MinGW instead of Visual Studio when building a module. Even tho > I've just uninstalled VS, and cleared out any leftover VS environment > variables, distutils keeps wanting to use it. > > The steps I took: > > Fresh installation of Python 3.1.1 > Successfully installed MinGW, added to the path variable (gcc in > command prompt works) > Successfully installed Cython, imports from Cython in Python work. > Added a distutils.cfg file in \Python31\Lib\distutils\ directory with: > > [build] > compiler=mingw32 > > (also tried adding [build_ext] compiler=mingw32) > > There's a demo setup.py module that came with Cython, I tried the > following commands: > > ---------------------------------------------------- > > > python setup.py build_ext --inplace > > error: Unable to find vcvarsall.bat > > > python setup.py build > > error: Unable to find vcvarsall.bat > ---------------------------------------------------- > > I'm having the exact same issue with trying to build the Polygon > library via MinGW. In fact, the reason I had installed Visual Studio > in the first place was to be able to build the Polygon library, since > I was having these errors. > > What do I need to do to make distutils/python use MinGW? Update: I installed and tried building with Python 2.6, it calls MinGW when I have the distutils.cfg file configured properly (same configuration as the Python 3.1.1 one) But why doesn't it work on a fresh Python 3.1.1 installation as well? Is this a bug? From tjreedy at udel.edu Sun Jan 31 22:20:52 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 31 Jan 2010 22:20:52 -0500 Subject: whassup? builtins? python3000? Naah can't be right? In-Reply-To: <799ea994-d00f-485c-9164-ed4b66e750a5@k41g2000yqm.googlegroups.com> References: <799ea994-d00f-485c-9164-ed4b66e750a5@k41g2000yqm.googlegroups.com> Message-ID: On 1/31/2010 4:17 PM, _wolf wrote: > but why does ``__builtins__`` change its meaning depending on whether > this is the scope of the ?script? (i.e. the module whose name was > present, when calling ``python foobar.py``) or whether this is the > scope of a secondary module (imported or executed, directly or > indirectly, by ``foobar.py``)? There is a technical reason, which I believe Guido once gave in a post to the now-closed py3k list as part of the thread on the change to a single name. I do not remember, but I presume it has something to do with a difference between executing the main module and imported modules. Rest assured that it is intentional and has a reason. You can, of course, check if __name__ == '__main__': to know how to access. Terry Jan Reedy From john at castleamber.com Sun Jan 31 22:30:15 2010 From: john at castleamber.com (John Bokma) Date: Sun, 31 Jan 2010 21:30:15 -0600 Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> <87y6jef1df.fsf@castleamber.com> <0375f27a$0$1309$c3e8da3@news.astraweb.com> <878wbdye6p.fsf@castleamber.com> Message-ID: <87tyu1ws3c.fsf@castleamber.com> Steven D'Aprano writes: > On Sun, 31 Jan 2010 18:47:42 -0600, John Bokma wrote: > >> Steven D'Aprano writes: >> >>> On Sun, 31 Jan 2010 14:47:08 -0600, John Bokma wrote: >>> >>>> An editor can correct the indenting of the braces example but can't >>>> with this one. >>>> >>>> if x: >>>> if y: >>>> foo() >>>> else: >>>> bar() >>>> >>>> While braces might be considered redundant they are not when for one >>>> reason or another formatting is lost or done incorrectly. >>> >>> I've heard this argument before, and I don't buy it. Why should we >>> expect the editor to correct malformed code? >> >> Or a prettyfier. It doesn't matter. The point is that with braces there >> *is* redundancy that be used to fix the code. > > Prettyfiers are significant in languages that allow braces (or begin/end > tokens) and indentation to go out of sync. Since that can't happen with > Python, Yes it can. I *have* seen Python with broken indentation on web pages, and good luck sorting it out. Blaming it on "broken tools" is just another straw man. It happens, and if you're learning Python and interested in that code you have a problem. Snipped the rest, because you start to sound like a zealot. I should've know better. -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From tjreedy at udel.edu Sun Jan 31 22:48:33 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 31 Jan 2010 22:48:33 -0500 Subject: Python and Ruby In-Reply-To: References: <0375f245$0$1309$c3e8da3@news.astraweb.com> Message-ID: On 1/31/2010 7:25 PM, Steven D'Aprano wrote: > On Sun, 31 Jan 2010 15:40:36 -0800, Chris Rebert wrote: > >> On Sun, Jan 31, 2010 at 2:36 PM, Steven D'Aprano >> wrote: >>> On Sun, 31 Jan 2010 04:28:41 -0800, Ed Keith wrote: >>>> In most functional languages you just name a function to access it and >>>> you do it ALL the time. >>>> >>>> for example, in if you have a function 'f' which takes two parameters >>>> to call the function and get the result you use: >>>> >>>> f 2 3 >>>> >>>> If you want the function itself you use: >>>> >>>> f >>> >>> How do you call a function of no arguments? >> >> It's not really a function in that case, it's just a named constant. >> (Recall that functions don't/can't have side-effects.) Three of you gave essentially identical answers, but I still do not see how given something like def f(): return 1 I differentiate between 'function object at address xxx' and 'int 1' objects. >>>> time.time(), random.random() > (1264983502.7505889, 0.29974255140479633) >>>> time.time(), random.random() > (1264983505.9283719, 0.74207867411026329) > > They don't look terribly constant to me. I believe these are not functions in a functional language sense. Neither are input and output 'functions'. So all of these are either missing or some sort of special function-like something. I personally take a broader view of functions and include the relevant environment in their input and output, so that there are no 'side effects'. The concept of 'side-effect' is somewhat arbitrary. In fields other than computing, like pharmacology and politics, its meaning is somewhat different and a bit corrupt. > There is a difference between a function that does "give me whatever > value is specified by a fixed description" and a function that does "give > me a fixed value". Terry Jan Reedy From andrej.mitrovich at gmail.com Sun Jan 31 23:01:57 2010 From: andrej.mitrovich at gmail.com (Andrej Mitrovic) Date: Sun, 31 Jan 2010 20:01:57 -0800 (PST) Subject: whassup? builtins? python3000? Naah can't be right? References: <799ea994-d00f-485c-9164-ed4b66e750a5@k41g2000yqm.googlegroups.com> Message-ID: Hey, it's really simple! Just like the excerpt from the Learning Python book says: "Really, the built-in scope is just a built-in module called builtins, but you have to import builtins to query built-ins because the name builtins is not itself built-in...." :) From jjposner at optimum.net Sun Jan 31 23:05:24 2010 From: jjposner at optimum.net (John Posner) Date: Sun, 31 Jan 2010 23:05:24 -0500 Subject: odd drawing problem with turtle.py In-Reply-To: References: Message-ID: <4B665304.8040703@optimum.net> > I'm on Python 2.5, but using the updated turtle.py Version 1.0.1 - 24. 9. 2009. > The following script draws 5 circles, which it is supposed to, but then > doesn't draw the second turtle which is supposed to simply move forward. > Any ideas? Try commenting out this statement: self.turtle.tracer(False) That helps on Python 2.6.4. -John From no.email at nospam.invalid Sun Jan 31 23:22:36 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 31 Jan 2010 20:22:36 -0800 Subject: Python and Ruby References: <0375f245$0$1309$c3e8da3@news.astraweb.com> Message-ID: <7x1vh5shyr.fsf@ruckus.brouhaha.com> Terry Reedy writes: > Three of you gave essentially identical answers, but I still do not > see how given something like > > def f(): return 1 > > I differentiate between 'function object at address xxx' and 'int 1' > objects. In the languages they are talking about, there is no such thing as a function with no args. A function is closer to a mathematical function, i.e. a mapping from one type to another, so every function has an arg. Haskell and ML do have a "Unit" type, written "()", which is something like Python's "None". So in Haskell you could write f () = 1 which means the function f, applied to the Unit value (), results in the value 1. The parens are not function-calling syntax. () is simply a value. For example, you could say y = () z = f y which would mean that z = 1. From alfps at start.no Sun Jan 31 23:33:36 2010 From: alfps at start.no (Alf P. Steinbach) Date: Mon, 01 Feb 2010 05:33:36 +0100 Subject: odd drawing problem with turtle.py In-Reply-To: References: Message-ID: * John Posner: > > I'm on Python 2.5, but using the updated turtle.py Version 1.0.1 - > 24. 9. 2009. > > The following script draws 5 circles, which it is supposed to, but then > > doesn't draw the second turtle which is supposed to simply move forward. > > Any ideas? > > Try commenting out this statement: > > self.turtle.tracer(False) > > That helps on Python 2.6.4. I can't see the original message, but when turning off animation via .tracer there should be a final .update call to flush things, so to speak. Cheers & hth., - Alf From kirby.urner at gmail.com Sun Jan 31 23:44:35 2010 From: kirby.urner at gmail.com (kirby urner) Date: Sun, 31 Jan 2010 20:44:35 -0800 Subject: [Edu-sig] odd drawing problem with turtle.py In-Reply-To: <4B665390.4080602@canterburyschool.org> References: <4B665390.4080602@canterburyschool.org> Message-ID: On Sun, Jan 31, 2010 at 8:07 PM, Vern Ceder wrote: > kirby urner wrote: >> >> I don't see where you've defined a Turtle class to instantiate sir. > > The Turtle class is part of the turtle library, so that's not an issue. > Hey, good point Vern, not firing on all cylinders over here. So I just commented out the Circle loop, went straight to the Turtle part at the end. It works OK for me, even without the visible=True. I'm on Python 2.6 on Win7 and I took out the numpy randint in favor of random randint. turtle.py in lib_tk: # Version 1.0.1 - 24. 9. 2009 I'm maybe not the right beta tester though, as after I uncomment the circle loop, I just get an hour-glass on my Tk canvas and no drawing, no error message either, even with visible=True in the init. Not sure what that's about yet... Hey, I just found out about piratepad.net, another shared white board solution. Was helping a dude in Indonesia debug a VPython script just a moment ago, with variable names in Indonesian (Latin-1 spellings at least). I've appended what we're working on. [ on second thought, I'll spare ya ] Kirby >>> ? ? ? ?self.turtle=Turtle(visible=False) Does visibility *and* pen need to be controlled? Sorry for the basic question, but seems if pen is up... From keakon at gmail.com Sun Jan 31 23:58:50 2010 From: keakon at gmail.com (keakon) Date: Sun, 31 Jan 2010 20:58:50 -0800 (PST) Subject: A performance issue when using default value Message-ID: <4754e6b7-2d79-4995-8f3f-193b646b524e@z10g2000prh.googlegroups.com> I've found strange performance issue when using default value, the test code is list below: from timeit import Timer def f(x): y = x y.append(1) return y def g(x=[]): y = [] y.append(1) return y def h(x=[]): y = x y.append(1) return y def f2(x): y = x y.append(1) return y + [] def g2(x=[]): y = [] y.append(1) return y + [] def h2(x=[]): y = x y.append(1) return y + [] TIMES = 10000 print Timer('f([])','from __main__ import f, g, h').timeit(TIMES) print Timer('g()','from __main__ import f, g, h').timeit(TIMES) print Timer('h([])','from __main__ import f, g, h').timeit(TIMES) print Timer('h()','from __main__ import f, g, h').timeit(TIMES) print Timer('f2([])','from __main__ import f2, g2, h2').timeit(TIMES) print Timer('g2()','from __main__ import f2, g2, h2').timeit(TIMES) print Timer('h2([])','from __main__ import f2, g2, h2').timeit(TIMES) print Timer('h2()','from __main__ import f2, g2, h2').timeit(TIMES) I tested it with Python 2.5.4, 2.6.4 and 3.1.1 on Windows XP, and get almost the same result: 0.00449247041174 0.00439608944712 0.00455867994396 0.00327471787615 0.00791581052899 0.00684919452053 0.00734311204357 0.30974942346 h2() is about 42 times slower than h2([]), but h() is a litter faster than h([]). If change TIMES to 20000, other results are 2 times than before, but h2 () is 4 times(about 1.2 sec) than before. Is there any tricks in it? From clp2 at rebertia.com Fri Jan 1 00:24:56 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Thu, 31 Dec 2009 21:24:56 -0800 Subject: Exception as the primary error handling mechanism? In-Reply-To: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <50697b2c0912312124n7f989568t13e5a1dd8d883e9c@mail.gmail.com> On Thu, Dec 31, 2009 at 8:47 PM, Peng Yu wrote: > I observe that python library primarily use exception for error > handling rather than use error code. > > In the article API Design Matters by Michi Henning > > Communications of the ACM > Vol. 52 No. 5, Pages 46-56 > 10.1145/1506409.1506424 > http://cacm.acm.org/magazines/2009/5/24646-api-design-matters/fulltext > > It says "Another popular design flaw?namely, throwing exceptions for > expected outcomes?also causes inefficiencies because catching and > handling exceptions is almost always slower than testing a return > value." > > My observation is contradicted to the above statement by Henning. If > my observation is wrong, please just ignore my question below. > > Otherwise, could some python expert explain to me why exception is > widely used for error handling in python? Is it because the efficiency > is not the primary goal of python? Correct; programmer efficiency is a more important goal for Python instead. Python is ~60-100x slower than C;[1] if someone is worried by the inefficiency caused by exceptions, then they're using completely the wrong language. Cheers, Chris -- http://blog.rebertia.com [1] http://shootout.alioth.debian.org/u64/which-programming-languages-are-fastest.php?gcc=on&python=on&calc=chart From benjamin.kaplan at case.edu Fri Jan 1 00:26:09 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Fri, 1 Jan 2010 00:26:09 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: On Thu, Dec 31, 2009 at 11:47 PM, Peng Yu wrote: > I observe that python library primarily use exception for error > handling rather than use error code. > > In the article API Design Matters by Michi Henning > > Communications of the ACM > Vol. 52 No. 5, Pages 46-56 > 10.1145/1506409.1506424 > http://cacm.acm.org/magazines/2009/5/24646-api-design-matters/fulltext > > It says "Another popular design flaw?namely, throwing exceptions for > expected outcomes?also causes inefficiencies because catching and > handling exceptions is almost always slower than testing a return > value." > > My observation is contradicted to the above statement by Henning. If > my observation is wrong, please just ignore my question below. > > Otherwise, could some python expert explain to me why exception is > widely used for error handling in python? Is it because the efficiency > is not the primary goal of python? > -- > http://mail.python.org/mailman/listinfo/python-list > Read the quote again "Another popular design flaw?namely, throwing exceptions *for expected outcomes*" In Python, throwing exceptions for expected outcomes is considered very bad form (well, except for StopIteration but that should almost never be handled directly by the programmer). To answer why people recommend using "Easier to Ask Forgiveness than Permission" as opposed to "Look Before You Leap" : Because of the way it's implemented, Python works quite differently from most languages. An attribute look-up is rather expensive because it's a hash table look-up at run time. Wrapping a small piece of code in a try block however, isn't (relatively) very expensive at all in Python. It's only catching the exception that's expensive, but if you're catching the exception, something has gone wrong anyway and performance isn't your biggest issue. From wolftracks at invalid.com Fri Jan 1 00:49:28 2010 From: wolftracks at invalid.com (W. eWatson) Date: Thu, 31 Dec 2009 21:49:28 -0800 Subject: mod, modulo and % under 2.4 and 2.5 In-Reply-To: <87iqbm1qwz.fsf@benfinney.id.au> References: <034d421c$0$1277$c3e8da3@news.astraweb.com> <87iqbm1qwz.fsf@benfinney.id.au> Message-ID: Ben Finney wrote: > "W. eWatson" writes: > >> Steven D'Aprano wrote: >>> NameError: name 'mod' is not defined > >> So where is it? Here are the choices. >> import sys, os, glob >> import string >> from numpy import * > > If you use ?from foo import *? you forfeit any way of saying where a > name in your code gets bound. > > Hence, don't do that. > Good idea! From yoavglazner at gmail.com Fri Jan 1 02:26:19 2010 From: yoavglazner at gmail.com (Glazner) Date: Thu, 31 Dec 2009 23:26:19 -0800 (PST) Subject: multiprocessing module References: <79854c42-b2af-4adb-8967-3dc5e4ac0d2a@l13g2000yqb.googlegroups.com> Message-ID: <57c4f9dd-7a01-439a-b538-d604df908345@k23g2000yqa.googlegroups.com> On Dec 15 2009, 10:56?am, makobu wrote: > I have a function that makes two subprocess.Popen() calls on a file. > > I have 8 cores. I need 8 instances of that function running in > parallel at any given time till all the files are worked on. > Can the multiprocessing module do this? If so, whats the best method? > > A technical overview of how the multiprocessing module actually works > would also be really helpful. > > regards, > mak. I guess you need the Map-Reduce pattern. It appears like multiprocessing.Pool will do the trick. def doSomething(f): pass pool = Pool(8)# 8 processes 1 per core files = ['pop.txt','looper.txt','foo.bar'] results = pool.map(doSomething,files) #this does all the job got it? From victorsubervi at gmail.com Fri Jan 1 02:47:02 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 1 Jan 2010 02:47:02 -0500 Subject: Not Incrementing In-Reply-To: <4B3D0DB3.3000101@mrabarnett.plus.com> References: <4dc0cfea0912310851u6f61a93frad30042c9bb1b96a@mail.gmail.com> <4B3CE70D.5020109@ieee.org> <4dc0cfea0912311137w45f00e69h3625d83d7166d12e@mail.gmail.com> <4B3D02FC.8090403@mrabarnett.plus.com> <4dc0cfea0912311206q23fc3f3dh7b783916543ff744@mail.gmail.com> <4B3D0DB3.3000101@mrabarnett.plus.com> Message-ID: <4dc0cfea0912312347i40a29719x1f6f79c398aa022f@mail.gmail.com> On Thu, Dec 31, 2009 at 3:46 PM, MRAB wrote: > Victor Subervi wrote: > >> On Thu, Dec 31, 2009 at 3:01 PM, MRAB > python at mrabarnett.plus.com>> wrote: >> >> Victor Subervi wrote: >> >> On Thu, Dec 31, 2009 at 1:01 PM, Dave Angel > > >> >> wrote: >> >> [snip] >> >> >> Incidentally, good naming would help to make the code easier to >> debug. This finalTrees and trees variables don't look to me >> like >> you intended them to be trees at all, but lists of >> tab-indented data. >> >> No kidding. Many people have commented on my poor choices of >> names. That's one of the things on my agenda to change (with >> some tools that I understand are out there) once I get this >> program finished and working for clients. >> >> Your clients will get it _with_ its faults? You'll fix it with some >> tools that you _understand_ are out there, ie you haven't looked >> for/found them yet? Yikes! :-) >> >> >> How many espressos do you drink a day? Do you mainline the stuff? Relax, >> dude. Rome wasn't made in a day. I'll get there. Chill out, dude. >> >> Programming is about attention to detail. It's not like poetry, with its > "poetic licence". Has a mistake in poetry ever caused a rocket to fail > (Ariane 5, flight 501)? :-) Heavens no! And thank goodness! Yes, as I like to say, "The devil is in the details...and well employed in software development!" LOL! beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Fri Jan 1 02:48:12 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 1 Jan 2010 02:48:12 -0500 Subject: Cookies In-Reply-To: <4dc0cfea0912310829n3028c740w167978dc3bc637ae@mail.gmail.com> References: <4dc0cfea0912291142u3712fb5axe8eee6acc781640e@mail.gmail.com> <4dc0cfea0912300727uf4743b1s72e7bdd0f044341b@mail.gmail.com> <4dc0cfea0912300826td1a0a8eo14c16584270f36ba@mail.gmail.com> <4dc0cfea0912300919k23a71fb8td4233d6c5653b346@mail.gmail.com> <4dc0cfea0912301936q88a4996t810fd88818a85e3e@mail.gmail.com> <4dc0cfea0912310829n3028c740w167978dc3bc637ae@mail.gmail.com> Message-ID: <4dc0cfea0912312348i35b90b37u3777ddba1f7aa710@mail.gmail.com> I'm curious. Are there other instances where code needs to be inserted into the header as in the print cookie to get it to be baked on the user's PC? TIA, beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at REMOVE-THIS-cybersource.com.au Fri Jan 1 03:26:16 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 01 Jan 2010 08:26:16 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <034d9ead$0$1277$c3e8da3@news.astraweb.com> On Thu, 31 Dec 2009 20:47:49 -0800, Peng Yu wrote: > I observe that python library primarily use exception for error handling > rather than use error code. > > In the article API Design Matters by Michi Henning > > Communications of the ACM > Vol. 52 No. 5, Pages 46-56 > 10.1145/1506409.1506424 > http://cacm.acm.org/magazines/2009/5/24646-api-design-matters/fulltext > > It says "Another popular design flaw?namely, throwing exceptions for > expected outcomes?also causes inefficiencies because catching and > handling exceptions is almost always slower than testing a return > value." This is very, very wrong. Firstly, notice that the author doesn't compare the same thing. He compares "catching AND HANDLING" the exception (emphasis added) with *only* testing a return value. Of course it is faster to test a value and do nothing, than it is to catch an exception and then handle the exception. That's an unfair comparison, and that alone shows that the author is biased against exceptions. But it's also wrong. If you call a function one million times, and catch an exception ONCE (because exceptions are rare) that is likely to be much, much faster than testing a return code one million times. Before you can talk about which strategy is faster, you need to understand your problem. When exceptions are rare (in CPython, about one in ten or rarer) then try...except is faster than testing each time. The exact cut-off depends on how expensive the test is, and how much work gets done before the exception is raised. Using exceptions is only slow if they are common. But the most important reason for preferring exceptions is that the alternatives are error-prone! Testing error codes is the anti-pattern, not catching exceptions. See, for example: http://c2.com/cgi/wiki?UseExceptionsInsteadOfErrorValues http://c2.com/cgi/wiki?ExceptionsAreOurFriends http://c2.com/cgi/wiki?AvoidExceptionsWheneverPossible Despite the title of that last page, it has many excellent arguments for why exceptions are better than the alternatives. (Be warned: the c2 wiki is filled with Java and C++ programmers who mistake the work-arounds for quirks of their language as general design principles. For example, because exceptions in Java are evcen more expensive and slow than in Python, you will find lots of Java coders saying "don't use exceptions" instead of "don't use exceptions IN JAVA".) There are many problems with using error codes: * They complicate your code. Instead of returning the result you care about, you have to return a status code and the return result you care about. Even worse is to have a single global variable to hold the status of the last function call! * Nobody can agree whether the status code means the function call failed, or the function call succeeded. * If the function call failed, what do you return as the result code? * You can't be sure that the caller will remember to check the status code. In fact, you can be sure that the caller WILL forget sometimes! (This is human nature.) This leads to the frequent problem that by the time a caller checks the status code, the original error has been lost and the program is working with garbage. * Even if you remember to check the status code, it complicates the code, makes it less readable, confuses the intent of the code, and often leads to the Arrow Anti-pattern: http://c2.com/cgi/wiki?ArrowAntiPattern That last argument is critical. Exceptions exist to make writing correct code easier to write, understand and maintain. Python uses special result codes in at least two places: str.find(s) returns -1 if s is not in the string re.match() returns None is the regular expression fails Both of these are error-prone. Consider a naive way of getting the fractional part of a float string: >>> s = "234.567" >>> print s[s.find('.')+1:] 567 But see: >>> s = "234" >>> print s[s.find('.')+1:] 234 You need something like: p = s.find('.') if p == -1: print '' else: print s[p+1:] Similarly, we cannot safely do this in Python: >>> re.match(r'\d+', '123abcd').group() '123' >>> re.match(r'\d+', 'abcd').group() Traceback (most recent call last): File "", line 1, in AttributeError: 'NoneType' object has no attribute 'group' You need to do this: mo = re.match(r'\d+', '123abcd') if mo is not None: # or just `if mo` will work mo.group() Exceptions are about making it easier to have correct code. They're also about making it easier to have readable code. Which is easier to read, easier to understand and easier to debug? x = function(1, 2, 3) if x != -1: y = function(x, 1, 2) if y != -1: z = function(y, x, 1) if z != -1: print "result is", z else: print "an error occurred" else: print "an error occurred" else: print "an error occurred" versus: try: x = function(1, 2, 3) y = function(x, 1, 2) print "result is", function(y, x, 1) except ValueError: print "an error occurred" In Python, setting up the try...except block is very fast, about as fast as a plain "pass" statement, but actually catching the exception is quite slow. So let's compare string.find (which returns an error result) and string.index (which raises an exception): >>> from timeit import Timer >>> setup = "source = 'abcd'*100 + 'e'" >>> min(Timer("p = source.index('e')", setup).repeat()) 1.1308379173278809 >>> min(Timer("p = source.find('e')", setup).repeat()) 1.2237567901611328 There's hardly any difference at all, and in fact index is slightly faster. But what about if there's an exceptional case? >>> min(Timer(""" ... try: ... p = source.index('z') ... except ValueError: ... pass ... """, setup).repeat()) 3.5699808597564697 >>> min(Timer(""" ... p = source.find('z') ... if p == -1: ... pass ... """, setup).repeat()) 1.7874350070953369 So in Python, catching the exception is slower, in this case about twice as slow. But remember that the "if p == -1" test is not free. It might be cheap, but it does take time. If you call find() enough times, and every single time you then test the result returned, that extra cost may be more expensive than catching a rare exception. The general rule in Python is: * if the exceptional event is rare (say, on average, less than about one time in ten) then use a try...except and catch the exception; * but if it is very common (more than one time in ten) then it is faster to do a test. > My observation is contradicted to the above statement by Henning. If my > observation is wrong, please just ignore my question below. > > Otherwise, could some python expert explain to me why exception is > widely used for error handling in python? Is it because the efficiency > is not the primary goal of python? Yes. Python's aim is to be fast *enough*, without necessarily being as fast as possible. Python aims to be readable, and to be easy to write correct, bug-free code. -- Steven From victorsubervi at gmail.com Fri Jan 1 03:30:57 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 1 Jan 2010 03:30:57 -0500 Subject: Not Incrementing In-Reply-To: <4dc0cfea0912312347i40a29719x1f6f79c398aa022f@mail.gmail.com> References: <4dc0cfea0912310851u6f61a93frad30042c9bb1b96a@mail.gmail.com> <4B3CE70D.5020109@ieee.org> <4dc0cfea0912311137w45f00e69h3625d83d7166d12e@mail.gmail.com> <4B3D02FC.8090403@mrabarnett.plus.com> <4dc0cfea0912311206q23fc3f3dh7b783916543ff744@mail.gmail.com> <4B3D0DB3.3000101@mrabarnett.plus.com> <4dc0cfea0912312347i40a29719x1f6f79c398aa022f@mail.gmail.com> Message-ID: <4dc0cfea1001010030g28bca209o4e578fa6ad83b79@mail.gmail.com> On Fri, Jan 1, 2010 at 2:47 AM, Victor Subervi wrote: > On Thu, Dec 31, 2009 at 3:46 PM, MRAB wrote: > >> Victor Subervi wrote: >> >>> On Thu, Dec 31, 2009 at 3:01 PM, MRAB >> python at mrabarnett.plus.com>> wrote: >>> >>> Victor Subervi wrote: >>> >>> On Thu, Dec 31, 2009 at 1:01 PM, Dave Angel >> >> >>> >> wrote: >>> >>> [snip] >>> >>> >>> Incidentally, good naming would help to make the code easier to >>> debug. This finalTrees and trees variables don't look to me >>> like >>> you intended them to be trees at all, but lists of >>> tab-indented data. >>> >>> No kidding. Many people have commented on my poor choices of >>> names. That's one of the things on my agenda to change (with >>> some tools that I understand are out there) once I get this >>> program finished and working for clients. >>> >>> Your clients will get it _with_ its faults? You'll fix it with some >>> tools that you _understand_ are out there, ie you haven't looked >>> for/found them yet? Yikes! :-) >>> >>> >>> How many espressos do you drink a day? Do you mainline the stuff? Relax, >>> dude. Rome wasn't made in a day. I'll get there. Chill out, dude. >>> >>> Programming is about attention to detail. It's not like poetry, with its >> "poetic licence". Has a mistake in poetry ever caused a rocket to fail >> (Ariane 5, flight 501)? :-) > > > Heavens no! And thank goodness! Yes, as I like to say, "The devil is in the > details...and well employed in software development!" LOL! > beno > PS Poetry is the art of writing what goes without saying. beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From aahz at pythoncraft.com Fri Jan 1 03:43:48 2010 From: aahz at pythoncraft.com (Aahz) Date: 1 Jan 2010 00:43:48 -0800 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: In article , Benjamin Kaplan wrote: > >In Python, throwing exceptions for expected outcomes is considered >very bad form [...] Who says that? I certainly don't. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From ijeiseditor at gmail.com Fri Jan 1 04:58:00 2010 From: ijeiseditor at gmail.com (ijeiseditor) Date: Fri, 1 Jan 2010 01:58:00 -0800 (PST) Subject: International Journal of Electronics, Information and Systems (IJEIS) Call for Paper Message-ID: <2fd974c7-973f-4285-8a30-038efe9757a8@k19g2000yqc.googlegroups.com> The International Journal of Electronics, Information and Systems (IJEIS) publish original papers on all subjects relevant to electronics, computer science, communication network, and information systems. The highest priority will be given to those contributions concerned with a discussion of the background of a practical problem, the establishment of an appropriate model, the determination of a solution, approximate or exact, analytical or numerical, and a discussion of the relevance of the results when applied to the real- life problem. Paper submissions are invited in the area of electronics and computer science, in particular the technological advances and research results in the fields of theoretical, experimental, and applied electronics, computer science and Information technology. Topics of interest include but are not limited to the following: Electronics: Analogue and digital circuit design, Microwave circuits and systems, Optoelectronic circuits, Semiconductor devices, Sensor technology, Transport in electronic materials, VLSI technology and device processing, Analogue Circuits and Signal Processing, Digital Circuits and Signal Processing, RF and Wireless Circuits & Systems, Biomedical Circuits & Systems, System Architectures and Applications, System Integration, SoC and Mixed-Signal disign, Design Automation of Electronics & Systems, Assembly and Packaging, Sensing and Sensor Networks, Photonic and Optoelectronic Circuits, Test and Reliability, Neural Network Circuits & Systems, Heterogeneous Structures, Advanced Technologies, Telecommunications and Multimedia, Electronic Materials, Electronic Devices, Integrated Electronic-Circuits, Optoelectronics &Quantum Electronics, Biomedical Engineering, Electronics Applications, Sensing Systems, Communication and Networks, Electronic Commerce? All topics related Electronics Computer Science: Parallel Processing and Distributed Computing, Foundations of High- performance Computing, Graph Theory and Analysis of Algorithms, Artificial Intelligences and Pattern/Image Recognitions, Neural Networks and Biomedical Simulations, Virtual Visions and Virtual Simulations, Data Mining, Web Image Mining and Applications, Data Base Management and Information Retrievals Adaptive Systems, Bifurcation, Biocybernetics, Bioinformatics, Bio-Inspired Networks, Blind Systems, Cellular Neural Networks, Chaos, Chaos Neural Networks, Circuit Simulation, Circuit Theory, Communication Systems, Complex Systems, Control Systems, Cryptosystems, Data Compression, Data Mining, Detection, Echo Canceller, Estimation, Evolutional Computation, Filter, Filter Bank, Fractal, Fuzzy Systems, Image Processing, Image Recognition, Internet Security, Medical Applications, Modeling, Optimization, Speech Processing, Speech Synthesis, Speech Recognition, Synchronization, Video Signal Processing, Watermarking, Wavelet Transform, All topics related Computer Science Communication Network: Ad-Hoc, Mobile, Wireless Networks and Mobile Computing, Quantum Computing, Coding, and Error Controls Agent Computing and Multi-Agents Systems, Defining Spectrum Rights and Open Spectrum Solutions, Quality of Services and Communication Protocols, Satellite and Optical Communication Systems, 3G/4G Network Evolutions, CDMA/GSM Communication Protocols, Mobile Computing for e- Commerce,Transmission/Switching/Distribution technologies,Communication Theory,Signal Processing for Communications,Wireless Communications,Wireless & Mobile Networking,Optical Networks and Systems,Next-Generation Networking and Internet,Communication QoS, Reliability and Modelling,Ad-hoc, Sensor and Mesh Networking,Multimedia Services, Communication Software and Services,Communication and Information System Security, System control, network/service management, Network and Internet protocols and standards,Client-server, distributed and Web-based communication systems, Broadband and multimedia systems and applications, Trials of advanced systems and services, Any topics related Communication Network Information and Systems: Cryptography and Foundation of Computer Security, Authentication/ Authorization Issues, IDS/Firewall, Anti-Spam mail, Anti-virus issues, Wireless Access Security, Network Security Applications, Biometric authentication and algorithms, Fingerprint /Hand/Biometrics Recognitions and Technologies, IC-card Security, OTP and Key Management Issues, E-commerce, Ubiquitous and RFID Applications, Metadata and Meta Modeling, XML and Data Management, Knowledge Management, Web Security and Privacy, Cyber Threats, Web Services and Web Engineering, Web Intelligence, Wireless Applications, Protocols and Standards, Network Systems, Proxies and Servers, Multimedia Applications using Web Services, Ontology and the Semantic Web, B2B, B2C and C2C, e-Business System Design and Development, e-Payment, Portal Strategies, Social Networks and Information Systems, Social and Legal Issues, Digital Ecology, e-Governance, e-Learning and Virtual Classrooms, e-Entertainment, e-Journalism, Any topics related Information systems International Journal of Electronics, Information and Systems (IJEIS) ISSN: 1884-7609 Website: https://sites.google.com/site/ijeishp2/ Manuscript submission to: ijeiseditor at gmail.com All submitted papers will be judged based on their quality by the technical committee and reviewers. Papers that describe research and experimentation are encouraged. All paper submissions will be handled electronically and detailed instructions on submission procedure are available on IJEIS web pages (https://sites.google.com/site/ ijeishp2/). Researchers and authors are invited to participate in the peer-review process of IJEIS papers if your research interest matches with the themes of Call for Papers. For other information, please contact IJEIS Managing Editor, (ijeiseditor at gmail.com) From martin at v.loewis.de Fri Jan 1 05:34:19 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Fri, 01 Jan 2010 11:34:19 +0100 Subject: Exception as the primary error handling mechanism? In-Reply-To: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <4B3DCFAB.3030909@v.loewis.de> Peng Yu wrote: > I observe that python library primarily use exception for error > handling rather than use error code. [...] > It says "Another popular design flaw?namely, throwing exceptions for > expected outcomes?also causes inefficiencies because catching and > handling exceptions is almost always slower than testing a return > value." > > My observation is contradicted to the above statement by Henning. If > my observation is wrong, please just ignore my question below. Your observation is not wrong, but, as Benjamin already explained, you are misinterpreting Michi Henning's statement. He doesn't condemn exception handling per se, but only for the handling of *expected* outcomes. He would consider using exceptions fine for *exceptional* output, and that is exactly the way they are used in the Python API. Notice that in cases where the failure may be expected, Python also offers variants that avoid the exception: - if you look into a dictionary, expecting that a key may not be there, a regular access, d[k], may give a KeyError. However, alternatively, you can use d.get(k, default) which raises no exception, and you can test "k in d" in advance. - if you open a file, not knowing whether it will be there, you get an IOError. However, you can use os.path.exists in advance to determine whether the file is present (and create it if it's not). So, in these cases, it is a choice of the user to determine whether the error case is exceptional or not. Regards, Martin From jgardner at jonathangardner.net Fri Jan 1 05:43:21 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Fri, 1 Jan 2010 02:43:21 -0800 (PST) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <2109bc4b-834c-45a1-8287-57936c60a7bf@s31g2000yqs.googlegroups.com> On Jan 1, 12:43?am, a... at pythoncraft.com (Aahz) wrote: > In article , > Benjamin Kaplan ? wrote: > >In Python, throwing exceptions for expected outcomes is considered > >very bad form [...] > > Who says that? ?I certainly don't. Agreed. int("asdf") is supposed to return what, exactly? Any language that tries to return an int is horribly broken. From jpeyret at gmail.com Fri Jan 1 06:25:13 2010 From: jpeyret at gmail.com (J Peyret) Date: Fri, 1 Jan 2010 03:25:13 -0800 (PST) Subject: twenty years ago Guido created Python References: Message-ID: On Dec 31 2009, 2:06?pm, Steve Howell wrote: > FYI: > > http://twitter.com/gvanrossum > > Python is a truly awesome programming language. ?Not only is Guido a > genius language designer, but he is also a great project leader. ?What > an accomplishment. ?Congratulations to everybody who has contributed > to Python in the last two decades! Notwithstanding all of the above, which are all true, having met Guido, I say he is a genuinely nice human being. Go BSD derivatives. From simon at brunningonline.net Fri Jan 1 06:36:32 2010 From: simon at brunningonline.net (Simon Brunning) Date: Fri, 1 Jan 2010 11:36:32 +0000 Subject: pywinauto to show the dialog , menu, etc In-Reply-To: References: Message-ID: <8c7f10c61001010336s19c6f016l3d7c83554a282ad9@mail.gmail.com> 2009/12/31 Hari : > Hi > I am using pywinauto to automate an custom program to startup and load > process , execute etc. But cannot determine menuselect. Is there a way or > tool which can run against the exe to show the menu, dialog box, list box > which are contained within it. Winspector might be worth a try: http://www.windows-spy.com/ -- Cheers, Simon B. From hackingkk at gmail.com Fri Jan 1 07:04:44 2010 From: hackingkk at gmail.com (Krishnakant) Date: Fri, 01 Jan 2010 17:34:44 +0530 Subject: twenty years ago Guido created Python In-Reply-To: References: Message-ID: <1262347484.3458.3.camel@krishna-laptop> On Fri, 2010-01-01 at 03:25 -0800, J Peyret wrote: > On Dec 31 2009, 2:06 pm, Steve Howell wrote: > > FYI: > > > > http://twitter.com/gvanrossum > > > > Python is a truly awesome programming language. Not only is Guido a > > genius language designer, but he is also a great project leader. What > > an accomplishment. Congratulations to everybody who has contributed > > to Python in the last two decades! > > Notwithstanding all of the above, which are all true, having met > Guido, I say he is a genuinely nice human being. Go BSD derivatives. Indeed, python is a great programming language. May be it was not marketted as much as languages like java or not as much as platforms like rails. But who cares? I must mention here that I am totally blind and the talking software (screen reader ) I use on GNU/Linux called Orca is also programmed using python. And I make web applications and off late started to use pylons ( again centered around python). I see that python is for 2 kind of programmers. one which are absolute beginners to programming itself who want to learn the scientific art of programming and the second is the extreme experts who need the power and performance nicely balanced. Great work! happy hfacking. Krishnakant. From icanbob at gmail.com Fri Jan 1 07:39:47 2010 From: icanbob at gmail.com (bobicanprogram) Date: Fri, 1 Jan 2010 04:39:47 -0800 (PST) Subject: adding python engine into c++ application References: <3451d586-11b6-4183-88a6-d353bac44d31@q2g2000yqd.googlegroups.com> Message-ID: <9f36654c-1b8a-47e8-a12d-4c86ebf5f257@e27g2000yqd.googlegroups.com> On Dec 29 2009, 6:25 am, griwes wrote: > Hello, I am going to write an application in C++, application which > should be easy to extend by scripts. I chose python for script > language in this application. I'd like to have own API for python. And > here the question arises: how do I implement a python engine within an > application written in C++? > > Sory for any mistakes, english isn't my native language ;) If you want to keep your C++ and Python as separate but interacting modules you might want to check out the SIMPL project's (http:// www.icanprogram.com/SIMPL) Python hooks. There is some tutorial level code here: http://www.icanprogram.com/06py/main.html bob From lie.1296 at gmail.com Fri Jan 1 08:03:32 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Sat, 02 Jan 2010 00:03:32 +1100 Subject: multivariable assignment In-Reply-To: References: Message-ID: <4b3df2a3$1@dnews.tpgi.com.au> On 1/1/2010 3:13 AM, davidj411 wrote: > I am not sure why this behavior is this way. > at beginning of script, i want to create a bunch of empty lists and > use each one for its own purpose. > however, updating one list seems to update the others. > >>>> a = b = c = [] >>>> a.append('1') >>>> a.append('1') >>>> a.append('1') >>>> c > ['1', '1', '1'] >>>> a > ['1', '1', '1'] >>>> b > ['1', '1', '1'] Every time people get confused because of how python object model works, I always refer them to this article: http://effbot.org/zone/call-by-object.htm From wolfram.hinderer at googlemail.com Fri Jan 1 08:19:02 2010 From: wolfram.hinderer at googlemail.com (Wolfram Hinderer) Date: Fri, 1 Jan 2010 05:19:02 -0800 (PST) Subject: Dangerous behavior of list(generator) References: <7p356kF750U1@mid.individual.net> <034bd6f1$0$1277$c3e8da3@news.astraweb.com> <034d412d$0$1277$c3e8da3@news.astraweb.com> Message-ID: <6efeed07-e4df-480c-8591-e4f6e6a2bb58@j24g2000yqa.googlegroups.com> On 1 Jan., 02:47, Steven D'Aprano wrote: > On Thu, 31 Dec 2009 11:34:39 -0800, Tom Machinski wrote: > > On Wed, Dec 30, 2009 at 4:01 PM, Steven D'Aprano > > wrote: > >> On Wed, 30 Dec 2009 15:18:11 -0800, Tom Machinski wrote: > >>> Bottom line, I'm going to have to remove this pattern from my code: > > >>> ? foo = (foo for foo in foos if foo.bar).next() > > >> I don't see why. What's wrong with it? Unless you embed it in a call to > >> list, or similar, it will explicitly raise StopIteration as expected. > > > Exactly; this seems innocuous, but if some caller of this code uses it > > in a list() constructor, a very subtle and dangerous bug is introduced - > > see OP. This is the entire point of this post. > > Then don't use it in a list() constructor. > > That's a glib answer, of course. A better answer is to point out that the > problem is not with the above expression, but with letting StopIteration > bubble up as an error exception instead of dealing with it immediately. > That's not what it's for, and you can't trust it not to be captured by > something. If StopIteration represents an error condition, you need to > deal with it immediately and convert it to an exception which isn't > likely to disappear. > > > In a large, non-trivial application, you simply cannot afford the > > assumption that no caller will ever do that. Even if you have perfect > > memory, some of your other developers or library users may not. > > You shouldn't put the responsibility of dealing with the StopIteration on > the caller, because StopIteraction is a signal not an error condition, > and you can't tell when that signal will disappear. The responsibility > lies on the writer of the function containing the line (that is, the > Original Poster of this thread). > > So you need something like this: > > def my_function(): > ? ? try: > ? ? ? ? foo = (foo for foo in foos if foo.bar).next() > ? ? except StopIteration: > ? ? ? ? handle_empty_foos() > ? ? else: > ? ? ? ? handle_at_least_one_foo() > > handle_empty_foos may be as simple as raising a new exception and letting > that bubble up to whatever layer of the application is expected to deal > with it. > > > As for what's wrong with the "if not any" solution, Benjamin Kaplan's > > post hits the nail on its head. This is a bioinformatics application, so > > the iterable "foos" tends to be very large, so saving half the runtime > > makes a big difference. > > Possibly you haven't seen my reply to Benjamin, so I'll paraphrase: > that's incorrect, because any() is lazy and will return as soon as it > hits a non-false item. Tom's point is that if not any(foo for foo in foos if foo.bar): foo = (foo for foo in foos if foo.bar).next() iterates twice over (the same first few elements of) foos, which should take about twice as long as iterating once. The lazyness of "any" does not seem to matter here. Of course, you're right that the iteration might or might not be the bottleneck. On the other hand, foos might not even be reiterable. > If the foo items are arbitrary objects which have an equal chance of > being considered true or false, then on average it will have to look at > half the list, By which definition of chance? :-) Wolfram From iwaki at iwakihidekazu.net Fri Jan 1 08:46:36 2010 From: iwaki at iwakihidekazu.net (Hidekazu IWAKI) Date: Fri, 01 Jan 2010 22:46:36 +0900 Subject: Where is "urllib2" module in windows python3.1.1? Message-ID: <231220100101134636iwaki@iwakihidekazu.net> Hi; I'd like to import "urllib2" in windows python 3.1.1, but I'm not able to do it. So, I researched the library directory; the result is following: .... .... 2009/06/07 19:11 61,749 unittest.py 2010/01/01 22:18 urllib 2007/12/06 09:48 6,318 uu.py 2008/11/30 20:40 21,534 uuid.py .... .... Although it has "urllib", it doesn't have "urllib2". I guess to download and install the module. But I can't find "urllib2" module for windows with google. I wonder if it would be possible for me to give the location? From duncan.booth at invalid.invalid Fri Jan 1 08:57:03 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 1 Jan 2010 13:57:03 GMT Subject: Where is "urllib2" module in windows python3.1.1? References: Message-ID: Hidekazu IWAKI wrote: > Hi; > I'd like to import "urllib2" in windows python 3.1.1, but I'm not able to > do it. Python 3 doesn't have a urllib2 module; use the urllib package instead. See http://www.python.org/dev/peps/pep-3108/#urllib-package From usenot at geekmail.INVALID Fri Jan 1 09:01:32 2010 From: usenot at geekmail.INVALID (Andreas Waldenburger) Date: Fri, 1 Jan 2010 15:01:32 +0100 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4B3DCFAB.3030909@v.loewis.de> Message-ID: <20100101150132.0ddb72d6@geekmail.INVALID> On Fri, 01 Jan 2010 11:34:19 +0100 "Martin v. Loewis" wrote: > Your observation is not wrong, but, as Benjamin already explained, > you are misinterpreting Michi Henning's statement. He doesn't condemn > exception handling per se, but only for the handling of *expected* > outcomes. He would consider using exceptions fine for *exceptional* > output, and that is exactly the way they are used in the Python API. May I point out at this point that "exceptional" does not mean "unexpected"? You catch exceptions, not unexpectations. An exception is rare, but not surprising. Case in point: StopIteration. To put it differently: When you write "catch DeadParrot", you certainly expect to get a DeadParrot once in a while -- why else would you get it in your head to try and catch it? An unexpected exception is the one that crashes your program. /W -- INVALID? DE! From iwaki at iwakihidekazu.net Fri Jan 1 09:19:57 2010 From: iwaki at iwakihidekazu.net (IWAKI, Hidekazu) Date: Fri, 1 Jan 2010 23:19:57 +0900 Subject: Where is "urllib2" module in windows python3.1.1? In-Reply-To: References: Message-ID: >Python 3 doesn't have a urllib2 module; use the urllib package instead. Thank you for your answer and the link. Oh, sorry. It was one of the changes from .2.x to 3.x. I didn't know. There are really important and a lot of changes. Thank you! On Fri, Jan 1, 2010 at 10:57 PM, Duncan Booth wrote: > Hidekazu IWAKI wrote: > > > Hi; > > I'd like to import "urllib2" in windows python 3.1.1, but I'm not able to > > do it. > > Python 3 doesn't have a urllib2 module; use the urllib package instead. > > See http://www.python.org/dev/peps/pep-3108/#urllib-package > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at REMOVE-THIS-cybersource.com.au Fri Jan 1 09:24:50 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 01 Jan 2010 14:24:50 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <2109bc4b-834c-45a1-8287-57936c60a7bf@s31g2000yqs.googlegroups.com> Message-ID: <034df2b6$0$1277$c3e8da3@news.astraweb.com> On Fri, 01 Jan 2010 02:43:21 -0800, Jonathan Gardner wrote: > On Jan 1, 12:43?am, a... at pythoncraft.com (Aahz) wrote: >> In article , >> Benjamin Kaplan ? wrote: >> >In Python, throwing exceptions for expected outcomes is considered >> >very bad form [...] >> >> Who says that? ?I certainly don't. > > Agreed. > > int("asdf") is supposed to return what, exactly? Any language that tries > to return an int is horribly broken. [sarcasm] No no, the right way to deal with that is have int("asdf") return some arbitrary bit pattern, and expect the user to check a global variable to see whether the function returned a valid result or not. That's much better than catching an exception! [/sarcasm] -- Steven From lie.1296 at gmail.com Fri Jan 1 09:35:14 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Sat, 02 Jan 2010 01:35:14 +1100 Subject: Where is "urllib2" module in windows python3.1.1? In-Reply-To: References: Message-ID: <4b3e0821$1@dnews.tpgi.com.au> On 1/2/2010 12:46 AM, Hidekazu IWAKI wrote: > Hi; > I'd like to import "urllib2" in windows python 3.1.1, but I'm not able to > do it. > So, I researched the library directory; the result is following: > > ..... > ..... > 2009/06/07 19:11 61,749 unittest.py > 2010/01/01 22:18 urllib > 2007/12/06 09:48 6,318 uu.py > 2008/11/30 20:40 21,534 uuid.py > ..... > ..... > > Although it has "urllib", it doesn't have "urllib2". I guess to download > and install the module. But I can't find "urllib2" module for windows > with google. > > I wonder if it would be possible for me to give the location? urllib2 is merged into urllib in python3. From steve at REMOVE-THIS-cybersource.com.au Fri Jan 1 09:42:29 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 01 Jan 2010 14:42:29 GMT Subject: Dangerous behavior of list(generator) References: <7p356kF750U1@mid.individual.net> <034bd6f1$0$1277$c3e8da3@news.astraweb.com> <034d412d$0$1277$c3e8da3@news.astraweb.com> <6efeed07-e4df-480c-8591-e4f6e6a2bb58@j24g2000yqa.googlegroups.com> Message-ID: <034df6d9$0$1277$c3e8da3@news.astraweb.com> On Fri, 01 Jan 2010 05:19:02 -0800, Wolfram Hinderer wrote: > On 1 Jan., 02:47, Steven D'Aprano cybersource.com.au> wrote: >> On Thu, 31 Dec 2009 11:34:39 -0800, Tom Machinski wrote: [...] >> > As for what's wrong with the "if not any" solution, Benjamin Kaplan's >> > post hits the nail on its head. This is a bioinformatics application, >> > so the iterable "foos" tends to be very large, so saving half the >> > runtime makes a big difference. >> >> Possibly you haven't seen my reply to Benjamin, so I'll paraphrase: >> that's incorrect, because any() is lazy and will return as soon as it >> hits a non-false item. > > Tom's point is that > if not any(foo for foo in foos if foo.bar): > foo = (foo for foo in foos if foo.bar).next() > iterates twice over (the same first few elements of) foos, which should > take about twice as long as iterating once. The lazyness of "any" does > not seem to matter here. That's no different from any "Look Before You Leap" idiom. If you do this: if key in dict: x = dict[key] you search the dict twice, once to see if the key is there, and the second time to fetch the value. Whether that is better or faster than the alternative: try: x = dict[key] except KeyError: pass depends on how often you expect the lookup to fail. In any case, I would claim that Tom's argument is a classic example of premature optimization: by his own admission: 'the iterable "foos" tends to be very large' which implies that whatever happens to the foos after this test, it will probably be very time consuming. If it takes (for the sake of the argument) 10 milliseconds to process the entire iterable, who cares whether it takes 0.01 or 0.02 ms to check that the iterable is valid? > Of course, you're right that the iteration might or might not be the > bottleneck. On the other hand, foos might not even be reiterable. If that's the case, then the existing solution potentially throws away the first value of foos every time the caller tests to see if it is empty. Dealing with non-reiterable iterators can be a nuisance. In such a case, it may be best to avoid Look Before You Leap altogether: empty = True for foo in foos: if foo.bar: empty = False process(foo) if empty: handle_error_condition() -- Steven From steve at REMOVE-THIS-cybersource.com.au Fri Jan 1 09:49:02 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 01 Jan 2010 14:49:02 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <034df862$0$1277$c3e8da3@news.astraweb.com> On Fri, 01 Jan 2010 00:26:09 -0500, Benjamin Kaplan wrote: > On Thu, Dec 31, 2009 at 11:47 PM, Peng Yu wrote: >> I observe that python library primarily use exception for error >> handling rather than use error code. >> >> In the article API Design Matters by Michi Henning >> >> Communications of the ACM >> Vol. 52 No. 5, Pages 46-56 >> 10.1145/1506409.1506424 >> http://cacm.acm.org/magazines/2009/5/24646-api-design-matters/fulltext >> >> It says "Another popular design flaw?namely, throwing exceptions for >> expected outcomes?also causes inefficiencies because catching and >> handling exceptions is almost always slower than testing a return >> value." >> >> My observation is contradicted to the above statement by Henning. If my >> observation is wrong, please just ignore my question below. >> >> Otherwise, could some python expert explain to me why exception is >> widely used for error handling in python? Is it because the efficiency >> is not the primary goal of python? >> -- >> http://mail.python.org/mailman/listinfo/python-list >> >> > Read the quote again "Another popular design flaw?namely, throwing > exceptions *for expected outcomes*" > In Python, throwing exceptions for expected outcomes is considered very > bad form (well, except for StopIteration but that should almost never be > handled directly by the programmer). Exceptions are *exceptional*, not "errors" or "unexpected". They are exceptional because they aren't the "normal" case, but that doesn't mean they are surprising or unexpected. Are you surprised that your "for x in range(1000)" loop comes to an end? Of course you are not -- it is completely expected, even though less than 1% of the iterations are the last loop. The end of the sequence is EXCEPTIONAL but not UNEXPECTED. If you program without expecting that keys can sometimes be missing from dictionaries (KeyError), or that you might sometimes have to deal with a list index that is out of range (IndexError), or that the denominator in a division might be zero (ZeroDivisionError), then you must be writing really buggy code. None of these things are unexpected, but they are all exceptional. The urllib2 module defines a HTTPError class, which does double-duty as both an exception and a valid HTTP response. If you're doing any HTTP programming, you better expect to deal with HTTP 301, 302 etc. codes, or at least trust that the library you use will transparently handle them for you. > To answer why people recommend using "Easier to Ask Forgiveness than > Permission" as opposed to "Look Before You Leap" : Because of the way > it's implemented, Python works quite differently from most languages. An > attribute look-up is rather expensive because it's a hash table look-up > at run time. Wrapping a small piece of code in a try block however, > isn't (relatively) very expensive at all in Python. It's not just relatively inexpensive, it's absolutely inexpensive: it costs about as much as a pass statement in CPython, which is pretty much as cheap as it gets. (If anyone can demonstrate a cheaper operation available from pure Python, I'd love to see it.) > It's only catching the exception that's expensive, True. > but if you're catching the exception, > something has gone wrong anyway and performance isn't your biggest > issue. The second try...except clause in the urllib2 module reads: try: kind = int(kind) except ValueError: pass In this case, the error is simply ignored. Nothing has gone wrong. Here's an example from my own code: I have an API where you pass a mapping (either a dict or a list of (key, value) tuples) to a function. So I do this: try: it = obj.iteritems() except AttributeError: it = obj for key, value in it: do_stuff() There's nothing wrong with catching exceptions. -- Steven From phlip2005 at gmail.com Fri Jan 1 10:10:51 2010 From: phlip2005 at gmail.com (Phlip) Date: Fri, 1 Jan 2010 07:10:51 -0800 (PST) Subject: change an exception's message and re-raise it References: <97c21727-ed6d-4be4-8c22-24b7beccd7bb@e27g2000yqd.googlegroups.com> <034d2f31$0$1277$c3e8da3@news.astraweb.com> Message-ID: On Dec 31 2009, 4:30?pm, Steven D'Aprano wrote: > ... ? ? 1/0 > ... except ZeroDivisionError, e: > ... ? ? e.args = e.args + ('fe', 'fi', 'fo', 'fum') > ... ? ? raise When I added print e.args it showed the old args. Maybe I was trying too hard - this is why I said e seemed locked or something. This started working: new_exception = self.format_fault(e.args[0]) e.args = (new_exception,) + (e.args[1:]) raise May I ask if args always has more than one entry? I need to bypass the silly "'tuple' object does not support item assignment" issue... From phlip2005 at gmail.com Fri Jan 1 10:21:13 2010 From: phlip2005 at gmail.com (Phlip) Date: Fri, 1 Jan 2010 07:21:13 -0800 (PST) Subject: change an exception's message and re-raise it References: <97c21727-ed6d-4be4-8c22-24b7beccd7bb@e27g2000yqd.googlegroups.com> <034d2f31$0$1277$c3e8da3@news.astraweb.com> Message-ID: <910aeda1-9852-46e1-960c-06cc9b1357dd@22g2000yqr.googlegroups.com> On Dec 31 2009, 4:30?pm, Steven D'Aprano wrote: > For the record you can get the exception type from type(e): > > raise type(e)("whatever you want") > > but that creates a new exception, not re-raising the old one. Except if a type constructs with some other number of arguments, apparently... From lie.1296 at gmail.com Fri Jan 1 10:24:41 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Sat, 02 Jan 2010 02:24:41 +1100 Subject: Exception as the primary error handling mechanism? In-Reply-To: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <4b3e13bb@dnews.tpgi.com.au> On 1/1/2010 3:47 PM, Peng Yu wrote: > I observe that python library primarily use exception for error > handling rather than use error code. > > In the article API Design Matters by Michi Henning > > Communications of the ACM > Vol. 52 No. 5, Pages 46-56 > 10.1145/1506409.1506424 > http://cacm.acm.org/magazines/2009/5/24646-api-design-matters/fulltext > > It says "Another popular design flaw?namely, throwing exceptions for > expected outcomes?also causes inefficiencies because catching and > handling exceptions is almost always slower than testing a return > value." > > My observation is contradicted to the above statement by Henning. If > my observation is wrong, please just ignore my question below. > > Otherwise, could some python expert explain to me why exception is > widely used for error handling in python? Simple, when an exception is thrown and I don't catch it, the exception terminates the program immediately and I got a traceback showing the point of failure. When I return error value and I don't check for it, I passed passed errors silently and gets a traceback forty-two lines later when trying to use the resources I failed to acquire forty-two lines prior. > Is it because the efficiency > is not the primary goal of python? Efficiency is not primary goal of python, but since python encourages EAFP (Easier to Ask Forgiveness than Permission); the design decisions chosen makes setting up a try-block much cheaper than a language designed over LBYL (Look Before You Leap) and return codes. From jkpeck at gmail.com Fri Jan 1 10:41:30 2010 From: jkpeck at gmail.com (JKPeck) Date: Fri, 1 Jan 2010 07:41:30 -0800 (PST) Subject: How to Suppress Interactive Assignment to "_" Message-ID: <910af7d0-8aa7-43fa-b2c9-32e12dc2fbc9@m16g2000yqc.googlegroups.com> The gettext module uses the convention of defining a function named "_" that maps text into its translation. This conflicts with the automatic interactive interpreter assignment of expressions to a variable with that same name. While if you are careful, you can avoid that assignment while debugging, and you can choose a different function for gettext, this conflict is a nuisance. I am looking for a way to suppress the expression assignment to _ or to change the name of the variable assigned to. Is this possible? Using Python 2.6. TIA, Jon Peck From benjamin.kaplan at case.edu Fri Jan 1 11:02:28 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Fri, 1 Jan 2010 11:02:28 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <034df862$0$1277$c3e8da3@news.astraweb.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <034df862$0$1277$c3e8da3@news.astraweb.com> Message-ID: On Fri, Jan 1, 2010 at 9:49 AM, Steven D'Aprano wrote: > > Exceptions are *exceptional*, not "errors" or "unexpected". They are > exceptional because they aren't the "normal" case, but that doesn't mean > they are surprising or unexpected. Are you surprised that your "for x in > range(1000)" loop comes to an end? Of course you are not -- it is > completely expected, even though less than 1% of the iterations are the > last loop. The end of the sequence is EXCEPTIONAL but not UNEXPECTED. > Sorry if my word choice was confusing- I was trying to point out that in Python, you don't test errors for your typical conditions, but for ones that you know still exist but don't plan on occurring often. From mwilson at the-wire.com Fri Jan 1 11:06:21 2010 From: mwilson at the-wire.com (Mel) Date: Fri, 01 Jan 2010 11:06:21 -0500 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <2109bc4b-834c-45a1-8287-57936c60a7bf@s31g2000yqs.googlegroups.com> <034df2b6$0$1277$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > On Fri, 01 Jan 2010 02:43:21 -0800, Jonathan Gardner wrote: > >> On Jan 1, 12:43 am, a... at pythoncraft.com (Aahz) wrote: >>> In article , >>> Benjamin Kaplan wrote: >>> >In Python, throwing exceptions for expected outcomes is considered >>> >very bad form [...] >>> >>> Who says that? I certainly don't. >> >> Agreed. >> >> int("asdf") is supposed to return what, exactly? Any language that tries >> to return an int is horribly broken. > > > [sarcasm] > No no, the right way to deal with that is have int("asdf") return some > arbitrary bit pattern, and expect the user to check a global variable to > see whether the function returned a valid result or not. That's much > better than catching an exception! > [/sarcasm] Or the other way around, as in C (I suspect the original ACM article assumed C.) Look at the legion of C library subroutines that return only 0 for good or -1 for bad, and do all their real work in side-effects (through pointers as function arguments.) Python is a big improvement: use the function return values for the payload, and push the out-of-band "omyghod" response into an Exception. Mel. From steve at REMOVE-THIS-cybersource.com.au Fri Jan 1 11:26:59 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 01 Jan 2010 16:26:59 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <034df862$0$1277$c3e8da3@news.astraweb.com> Message-ID: <034e0f57$0$1277$c3e8da3@news.astraweb.com> On Fri, 01 Jan 2010 11:02:28 -0500, Benjamin Kaplan wrote: > I was trying to point out that in > Python, you don't test errors for your typical conditions, but for ones > that you know still exist but don't plan on occurring often. I'm sorry, but that makes no sense to me at all. I don't understand what you are trying to say. You do understand that exceptions aren't just for errors? They are raised under specific circumstances. Whether that circumstance is an error or not is entirely up to the caller. try: n = mylist.index('fault') except ValueError: print "All is good, no fault detected" else: print "Your data contains a fault in position", n People get hung up on the idea that exceptions == errors, but they aren't. They *may* be errors, and that is one common use, but that depends entirely on the caller. -- Steven From victorsubervi at gmail.com Fri Jan 1 11:52:17 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 1 Jan 2010 11:52:17 -0500 Subject: MySQL Error Message-ID: <4dc0cfea1001010852u1101627avf2677af1553fd484@mail.gmail.com> Hi; I'm trying to avoid the mortal sin of blank excepts. I intentionally threw this error: Traceback (most recent call last): File "/var/www/html/angrynates.com/cart/createAssociations2.py", line 137, in ? createAssociations2() File "/var/www/html/angrynates.com/cart/createAssociations2.py", line 108, in createAssociations2 cursor.execute(sql) File "/usr/lib64/python2.4/site-packages/MySQLdb/cursors.py", line 163, in execute self.errorhandler(self, exc, value) File "/usr/lib64/python2.4/site-packages/MySQLdb/connections.py", line 35, in defaulterrorhandler raise errorclass, errorvalue ProgrammingError: (1146, "Table 'test.productsAssociations' doesn't exist") However, "ProgrammingError" is not an error. How do I discover the real error, so I can write the appropriate except statement? TIA, beno -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Fri Jan 1 12:03:21 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 01 Jan 2010 17:03:21 +0000 Subject: twenty years ago Guido created Python In-Reply-To: References: Message-ID: <4B3E2AD9.5040901@mrabarnett.plus.com> J Peyret wrote: > On Dec 31 2009, 2:06 pm, Steve Howell wrote: >> FYI: >> >> http://twitter.com/gvanrossum >> >> Python is a truly awesome programming language. Not only is Guido a >> genius language designer, but he is also a great project leader. What >> an accomplishment. Congratulations to everybody who has contributed >> to Python in the last two decades! > > Notwithstanding all of the above, which are all true, having met > Guido, I say he is a genuinely nice human being. > And anyone who says he isn't will be put into the comfy chair! :-) From carsten.haese at gmail.com Fri Jan 1 12:04:10 2010 From: carsten.haese at gmail.com (Carsten Haese) Date: Fri, 01 Jan 2010 12:04:10 -0500 Subject: MySQL Error In-Reply-To: <4dc0cfea1001010852u1101627avf2677af1553fd484@mail.gmail.com> References: <4dc0cfea1001010852u1101627avf2677af1553fd484@mail.gmail.com> Message-ID: Victor Subervi wrote: > However, "ProgrammingError" is not an error. How do I discover the real > error, so I can write the appropriate except statement? You're not making any sense. How did you determine that ProgrammingError is not an error or that it's not the "real error"? Show us the code you ran, the output you expected, and the output it produced instead. Blind guess: You're using "except ProgrammingError" when you should be using "except MySQLdb.ProgrammingError". If this guess is incorrect, see above. -- Carsten Haese http://informixdb.sourceforge.net From __peter__ at web.de Fri Jan 1 12:06:18 2010 From: __peter__ at web.de (Peter Otten) Date: Fri, 01 Jan 2010 18:06:18 +0100 Subject: How to Suppress Interactive Assignment to "_" References: <910af7d0-8aa7-43fa-b2c9-32e12dc2fbc9@m16g2000yqc.googlegroups.com> Message-ID: JKPeck wrote: > The gettext module uses the convention of defining a function named > "_" that maps text into its translation. > This conflicts with the automatic interactive interpreter assignment > of expressions to a variable with that same name. > > While if you are careful, you can avoid that assignment while > debugging, and you can choose a different function for gettext, this > conflict is a nuisance. I am looking for a way to suppress the > expression assignment to _ or to change the name of the variable > assigned to. Is this possible? Using Python 2.6. $ cat displayhook.py import sys import __builtin__ as builtin def displayhook(obj): if obj is not None: builtin._last_result = obj print repr(obj) sys.displayhook = displayhook $ python -i displayhook.py >>> 42 42 >>> _ Traceback (most recent call last): File "", line 1, in NameError: name '_' is not defined >>> _last_result 42 From python at mrabarnett.plus.com Fri Jan 1 12:10:02 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 01 Jan 2010 17:10:02 +0000 Subject: MySQL Error In-Reply-To: <4dc0cfea1001010852u1101627avf2677af1553fd484@mail.gmail.com> References: <4dc0cfea1001010852u1101627avf2677af1553fd484@mail.gmail.com> Message-ID: <4B3E2C6A.4070506@mrabarnett.plus.com> Victor Subervi wrote: > Hi; > I'm trying to avoid the mortal sin of blank excepts. I intentionally > threw this error: > > Traceback (most recent call last): > File "/var/www/html/angrynates.com/cart/createAssociations2.py > ", line 137, in ? > createAssociations2() > File "/var/www/html/angrynates.com/cart/createAssociations2.py > ", line 108, in > createAssociations2 > cursor.execute(sql) > File "/usr/lib64/python2.4/site-packages/MySQLdb/cursors.py", line > 163, in execute > self.errorhandler(self, exc, value) > File "/usr/lib64/python2.4/site-packages/MySQLdb/connections.py", line > 35, in defaulterrorhandler > raise errorclass, errorvalue > ProgrammingError: (1146, "Table 'test.productsAssociations' doesn't exist") > > However, "ProgrammingError" is not an error. How do I discover the real > error, so I can write the appropriate except statement? > What makes you think it isn't? As the traceback says, "Table 'test.productsAssociations' doesn't exist", and that's the cause. From martin at v.loewis.de Fri Jan 1 12:18:32 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Fri, 01 Jan 2010 18:18:32 +0100 Subject: Exception as the primary error handling mechanism? In-Reply-To: <034e0f57$0$1277$c3e8da3@news.astraweb.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <034df862$0$1277$c3e8da3@news.astraweb.com> <034e0f57$0$1277$c3e8da3@news.astraweb.com> Message-ID: > You do understand that exceptions aren't just for errors? They are raised > under specific circumstances. Whether that circumstance is an error or > not is entirely up to the caller. I think that's a fairly narrow definition of the word error, and probably also the source of confusion in this thread. ISTM that there is a long tradition of giving different meaning to the word "error" in computing. For example, the Unix man pages list various conditions as "errors" purely by their outcome, and completely ignoring on whether the caller would consider the result erroneous - ISTM that a system call reports an "error" iff it is "unsuccessful". By that (common) usage of "error", it is a condition determined by the callee, not the caller (i.e. callee could not successfully complete the operation). In that sense, it is indeed equivalent to Python's usage of exceptions, which are also determined by the callee, and typically also in cases where successful completion is not possible. Whether these cases are "exceptional" in the word sense (i.e. deviating from the norm) would have to be decided by the application, again (which would set the norm). Regards, Martin From victorsubervi at gmail.com Fri Jan 1 13:32:20 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 1 Jan 2010 13:32:20 -0500 Subject: MySQL Error In-Reply-To: <4B3E2C6A.4070506@mrabarnett.plus.com> References: <4dc0cfea1001010852u1101627avf2677af1553fd484@mail.gmail.com> <4B3E2C6A.4070506@mrabarnett.plus.com> Message-ID: <4dc0cfea1001011032r3020992bs295c19af74ad61d4@mail.gmail.com> On Fri, Jan 1, 2010 at 12:10 PM, MRAB wrote: > Victor Subervi wrote: > >> Hi; >> I'm trying to avoid the mortal sin of blank excepts. I intentionally threw >> this error: >> >> Traceback (most recent call last): >> File "/var/www/html/angrynates.com/cart/createAssociations2.py < >> http://angrynates.com/cart/createAssociations2.py>", line 137, in ? >> createAssociations2() >> File "/var/www/html/angrynates.com/cart/createAssociations2.py < >> http://angrynates.com/cart/createAssociations2.py>", line 108, in >> createAssociations2 >> >> cursor.execute(sql) >> File "/usr/lib64/python2.4/site-packages/MySQLdb/cursors.py", line 163, >> in execute >> self.errorhandler(self, exc, value) >> File "/usr/lib64/python2.4/site-packages/MySQLdb/connections.py", line >> 35, in defaulterrorhandler >> raise errorclass, errorvalue >> ProgrammingError: (1146, "Table 'test.productsAssociations' doesn't >> exist") >> >> However, "ProgrammingError" is not an error. How do I discover the real >> error, so I can write the appropriate except statement? >> >> What makes you think it isn't? > Because I've tried except ProgrammingError: pass before and *that* threw an error. So I didnt' try again. beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From showell30 at yahoo.com Fri Jan 1 13:52:30 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 1 Jan 2010 10:52:30 -0800 (PST) Subject: twenty years ago Guido created Python References: Message-ID: <0003342a-9738-476d-b83d-f99921e18649@26g2000yqo.googlegroups.com> On Jan 1, 9:03?am, MRAB wrote: > J Peyret wrote: > > On Dec 31 2009, 2:06 pm, Steve Howell wrote: > >> FYI: > > >>http://twitter.com/gvanrossum > > >> Python is a truly awesome programming language. ?Not only is Guido a > >> genius language designer, but he is also a great project leader. ?What > >> an accomplishment. ?Congratulations to everybody who has contributed > >> to Python in the last two decades! > > > Notwithstanding all of the above, which are all true, having met > > Guido, I say he is a genuinely nice human being. > > And anyone who says he isn't will be put into the comfy chair! :-) And poked with the soft cushions! From jkpeck at gmail.com Fri Jan 1 13:53:31 2010 From: jkpeck at gmail.com (JKPeck) Date: Fri, 1 Jan 2010 10:53:31 -0800 (PST) Subject: How to Suppress Interactive Assignment to "_" References: <910af7d0-8aa7-43fa-b2c9-32e12dc2fbc9@m16g2000yqc.googlegroups.com> Message-ID: On Jan 1, 10:06?am, Peter Otten <__pete... at web.de> wrote: > JKPeck wrote: > > The gettext module uses the convention of defining a function named > > "_" that maps text into its translation. > > This conflicts with the automatic interactive interpreter assignment > > of expressions to a variable with that same name. > > > While if you are careful, you can avoid that assignment while > > debugging, and you can choose a different function for gettext, this > > conflict is a nuisance. ?I am looking for a way to suppress the > > expression assignment to _ or to change the name of the variable > > assigned to. ?Is this possible? ?Using Python 2.6. > > $ cat displayhook.py > import sys > import __builtin__ as builtin > > def displayhook(obj): > ? ? if obj is not None: > ? ? ? ? builtin._last_result = obj > ? ? ? ? print repr(obj) > > sys.displayhook = displayhook > $ python -i displayhook.py>>> 42 > 42 > >>> _ > > Traceback (most recent call last): > ? File "", line 1, in > NameError: name '_' is not defined>>> _last_result > > 42 Thanks. It's just what I needed. -Jon Peck From benjamin.kaplan at case.edu Fri Jan 1 14:04:34 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Fri, 1 Jan 2010 14:04:34 -0500 Subject: MySQL Error In-Reply-To: <4dc0cfea1001011032r3020992bs295c19af74ad61d4@mail.gmail.com> References: <4dc0cfea1001010852u1101627avf2677af1553fd484@mail.gmail.com> <4B3E2C6A.4070506@mrabarnett.plus.com> <4dc0cfea1001011032r3020992bs295c19af74ad61d4@mail.gmail.com> Message-ID: On Fri, Jan 1, 2010 at 1:32 PM, Victor Subervi wrote: > On Fri, Jan 1, 2010 at 12:10 PM, MRAB wrote: >> >> Victor Subervi wrote: >>> >>> Hi; >>> I'm trying to avoid the mortal sin of blank excepts. I intentionally >>> threw this error: >>> >>> Traceback (most recent call last): >>> ?File "/var/www/html/angrynates.com/cart/createAssociations2.py >>> ", line 137, in ? >>> ? ?createAssociations2() >>> ?File "/var/www/html/angrynates.com/cart/createAssociations2.py >>> ", line 108, in >>> createAssociations2 >>> ? ?cursor.execute(sql) >>> ?File "/usr/lib64/python2.4/site-packages/MySQLdb/cursors.py", line 163, >>> in execute >>> ? ?self.errorhandler(self, exc, value) >>> ?File "/usr/lib64/python2.4/site-packages/MySQLdb/connections.py", line >>> 35, in defaulterrorhandler >>> ? ?raise errorclass, errorvalue >>> ProgrammingError: (1146, "Table 'test.productsAssociations' doesn't >>> exist") >>> >>> However, "ProgrammingError" is not an error. How do I discover the real >>> error, so I can write the appropriate except statement? >>> >> What makes you think it isn't? > > Because I've tried > except ProgrammingError: > ? pass > before and *that* threw an error. So I didnt' try again. > beno Just because it's not in the current namespace doesn't mean it's not an error. From PEP 249 (the db api v2) : ProgrammingError Exception raised for programming errors, e.g. table not found or already exists, syntax error in the SQL statement, wrong number of parameters specified, etc. It must be a subclass of DatabaseError. > > -- > http://mail.python.org/mailman/listinfo/python-list > > From python at mrabarnett.plus.com Fri Jan 1 15:24:09 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 01 Jan 2010 20:24:09 +0000 Subject: MySQL Error In-Reply-To: References: <4dc0cfea1001010852u1101627avf2677af1553fd484@mail.gmail.com> <4B3E2C6A.4070506@mrabarnett.plus.com> <4dc0cfea1001011032r3020992bs295c19af74ad61d4@mail.gmail.com> Message-ID: <4B3E59E9.9020401@mrabarnett.plus.com> Benjamin Kaplan wrote: > On Fri, Jan 1, 2010 at 1:32 PM, Victor Subervi wrote: >> On Fri, Jan 1, 2010 at 12:10 PM, MRAB wrote: >>> Victor Subervi wrote: >>>> Hi; >>>> I'm trying to avoid the mortal sin of blank excepts. I intentionally >>>> threw this error: >>>> >>>> Traceback (most recent call last): >>>> File "/var/www/html/angrynates.com/cart/createAssociations2.py >>>> ", line 137, in ? >>>> createAssociations2() >>>> File "/var/www/html/angrynates.com/cart/createAssociations2.py >>>> ", line 108, in >>>> createAssociations2 >>>> cursor.execute(sql) >>>> File "/usr/lib64/python2.4/site-packages/MySQLdb/cursors.py", line 163, >>>> in execute >>>> self.errorhandler(self, exc, value) >>>> File "/usr/lib64/python2.4/site-packages/MySQLdb/connections.py", line >>>> 35, in defaulterrorhandler >>>> raise errorclass, errorvalue >>>> ProgrammingError: (1146, "Table 'test.productsAssociations' doesn't >>>> exist") >>>> >>>> However, "ProgrammingError" is not an error. How do I discover the real >>>> error, so I can write the appropriate except statement? >>>> >>> What makes you think it isn't? >> Because I've tried >> except ProgrammingError: >> pass >> before and *that* threw an error. So I didnt' try again. >> beno > > Just because it's not in the current namespace doesn't mean it's not > an error. From PEP 249 (the db api v2) : > > ProgrammingError > > Exception raised for programming errors, e.g. table not > found or already exists, syntax error in the SQL > statement, wrong number of parameters specified, etc. It > must be a subclass of DatabaseError. > You shouldn't just that it threw an error and give up, you should look at what the error was, in this case a NameError, which means that it doesn't know that name. Why? Because it's defined in the MySQL module that you imported, but you didn't import the name ProgrammingError. From python at bdurham.com Fri Jan 1 16:24:36 2010 From: python at bdurham.com (python at bdurham.com) Date: Fri, 01 Jan 2010 16:24:36 -0500 Subject: Which version of MSVC?90.DLL's to distribute with Python 2.6 based Py2exe executables? In-Reply-To: References: <05bd2f84-4f09-4a76-8e17-5868450ad393@c3g2000yqd.googlegroups.com> <753bb7c9-14bf-44e3-9679-0693635f2cd9@j14g2000yqm.googlegroups.com> <573025f1-0e5d-4d82-a86b-36ea33f85cad@c3g2000yqd.googlegroups.com> Message-ID: <1262381076.11442.1352533759@webmail.messagingengine.com> Waldemar, Thank your for sharing your technique - works great with 32-bit Python 2.6.4. Has anyone tried this with a 64-bit version of Python? Malcolm From python at bdurham.com Fri Jan 1 16:54:12 2010 From: python at bdurham.com (python at bdurham.com) Date: Fri, 01 Jan 2010 16:54:12 -0500 Subject: Safe file I/O to shared file (or SQLite) from multi-threaded web server Message-ID: <1262382852.16237.1352535253@webmail.messagingengine.com> I'm looking for the best practice way for a multi-threaded python web server application to read/write to a shared file or a SQLite database. What do I need to do (if anything) to make sure my writes to a regular file on disk or to a SQLite database are atomic in nature when multiple clients post data to my application simultaneously? Do I need to use a Queue type data structure and then run a background thread that monitors my Queue for data which it (and it alone) removes and copies to the destination file or SQLite datatbase? Note: In my specific case, the web server will be based on CherryPy 3.1 but I think this type of question is relevant across other Python based web server frameworks as well. Thank you, Malcolm From drsalists at gmail.com Fri Jan 1 17:02:36 2010 From: drsalists at gmail.com (Dan Stromberg) Date: Fri, 01 Jan 2010 14:02:36 -0800 Subject: Significant whitespace Message-ID: <4B3E70FC.1010702@gmail.com> I put together a page about significant whitespace (and the lack thereof). You're invited to check it out: http://stromberg.dnsalias.org/~dstromberg/significant-whitespace.html From clp2 at rebertia.com Fri Jan 1 17:19:28 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Fri, 1 Jan 2010 14:19:28 -0800 Subject: Significant whitespace In-Reply-To: <4B3E70FC.1010702@gmail.com> References: <4B3E70FC.1010702@gmail.com> Message-ID: <50697b2c1001011419q4f055bc7jec668e890aae974c@mail.gmail.com> On Fri, Jan 1, 2010 at 2:02 PM, Dan Stromberg wrote: > I put together a page about significant whitespace (and the lack thereof). > > You're invited to check it out: > > http://stromberg.dnsalias.org/~dstromberg/significant-whitespace.html For those of us who weren't around during the heyday of FORTRAN, can anyone describe this apparently much-reviled significant whitespace feature that continues to make some programmers unjustly fearful about Python's use of indentation? Thanks in advance. Cheers, Chris -- http://blog.rebertia.com From deets at nospam.web.de Fri Jan 1 17:20:19 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Fri, 01 Jan 2010 23:20:19 +0100 Subject: Safe file I/O to shared file (or SQLite) from multi-threaded web server In-Reply-To: References: Message-ID: <7q7ap3F3u36s0U1@mid.uni-berlin.de> python at bdurham.com schrieb: > I'm looking for the best practice way for a multi-threaded python web > server application to read/write to a shared file or a SQLite database. > > What do I need to do (if anything) to make sure my writes to a regular > file on disk or to a SQLite database are atomic in nature when multiple > clients post data to my application simultaneously? > > Do I need to use a Queue type data structure and then run a background > thread that monitors my Queue for data which it (and it alone) removes > and copies to the destination file or SQLite datatbase? > > Note: In my specific case, the web server will be based on CherryPy 3.1 > but I think this type of question is relevant across other Python based > web server frameworks as well. AFAIK, sqlite ensures process-serialization via locking, and threads synchronize themselves as well. So you shouldn't need to worry at all. Diez From bluemangroupie at gmail.com Fri Jan 1 17:51:25 2010 From: bluemangroupie at gmail.com (dennis) Date: Fri, 1 Jan 2010 14:51:25 -0800 (PST) Subject: twenty years ago Guido created Python References: Message-ID: <91141e23-6691-4f44-a73f-efd4292d9ffa@h9g2000yqa.googlegroups.com> On Jan 1, 6:04?am, Krishnakant wrote: > On Fri, 2010-01-01 at 03:25 -0800, J Peyret wrote: > > On Dec 31 2009, 2:06 pm, Steve Howell wrote: > > > FYI: > > > >http://twitter.com/gvanrossum > > > > Python is a truly awesome programming language. ?Not only is Guido a > > > genius language designer, but he is also a great project leader. ?What > > > an accomplishment. ?Congratulations to everybody who has contributed > > > to Python in the last two decades! > > > Notwithstanding all of the above, which are all true, having met > > Guido, I say he is a genuinely nice human being. ?Go BSD derivatives. > > Indeed, > python is a great programming language. > May be it was not marketted as much as languages like java or not as > much as platforms like rails. > But who cares? ?I must mention here that I am totally blind and the > talking software (screen reader ) I use on GNU/Linux called Orca is also > programmed using python. > > And I make web applications and off late started to use pylons ( ?again > centered around python). > I see that python is for 2 kind of programmers. > one which are absolute beginners to programming itself who want to learn > the scientific art of programming and the second is the extreme experts > who need the power and performance nicely balanced. > > Great work! > > happy hfacking. > Krishnakant. That's incredible. From ktenney at gmail.com Fri Jan 1 17:55:45 2010 From: ktenney at gmail.com (Kent Tenney) Date: Fri, 1 Jan 2010 16:55:45 -0600 Subject: Trying to run a sudo command from script Message-ID: Howdy, A script running as a regular user sometimes wants to run sudo commands. It gets the password with getpass. pw = getpass.getpass() I've fiddled a bunch with stuff like proc = subprocess.Popen('sudo touch /etc/foo'.split(), stdin=subprocess.PIPE) proc.communicate(input=pw) getting assorted errors with all variations I try. Googling says use pexpect, but I'd prefer a stdlib solution. Any help appreciated. Thanks, Kent From deets at nospam.web.de Fri Jan 1 18:08:58 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Sat, 02 Jan 2010 00:08:58 +0100 Subject: Trying to run a sudo command from script In-Reply-To: References: Message-ID: <7q7dkaF3t8r0rU1@mid.uni-berlin.de> Kent Tenney schrieb: > Howdy, > > A script running as a regular user sometimes wants > to run sudo commands. > > It gets the password with getpass. > pw = getpass.getpass() > > I've fiddled a bunch with stuff like > proc = subprocess.Popen('sudo touch /etc/foo'.split(), stdin=subprocess.PIPE) > proc.communicate(input=pw) > > getting assorted errors with all variations I try. > > Googling says use pexpect, but I'd prefer a stdlib solution. pexpect is pure python, and it's needed. There is no easy way around the issue, so if you insist on not using pexpect, you re-invent the wheel and write the exact same code - just more error-prone, because of wheel-reinvention.... Diez From mydimle at gmail.com Fri Jan 1 18:27:57 2010 From: mydimle at gmail.com (myle) Date: Fri, 1 Jan 2010 15:27:57 -0800 (PST) Subject: Bare Excepts References: <4dc0cfea0912300209x15e5bc96nd6e6224d89df0638@mail.gmail.com> <4B3B4FE5.8070605@sequans.com> <4dc0cfea0912300509v7f9bb367jf33bb63736407ed@mail.gmail.com> <4dc0cfea0912300512h43a4fdb1oa585a36391de341e@mail.gmail.com> <6944e440-c797-439d-9bf9-b61a05a3e26c@k23g2000yqa.googlegroups.com> <878wck2ojd.fsf@benfinney.id.au> <4dc0cfea0912301927q16445c59r1fb8594ee0f04d24@mail.gmail.com> Message-ID: Why we should prefer ``if: ...'' over a ``try: ... except something: pass'' block? In http://wiki.python.org/moin/PythonSpeed/PerformanceTips#InitializingDictionaryElements it is stated that a try catch block is faster if more often no exception occurs. Happy new year, Dimitris Leventeas From roy at panix.com Fri Jan 1 18:29:24 2010 From: roy at panix.com (Roy Smith) Date: Fri, 01 Jan 2010 18:29:24 -0500 Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> Message-ID: In article , Chris Rebert wrote: > On Fri, Jan 1, 2010 at 2:02 PM, Dan Stromberg wrote: > > I put together a page about significant whitespace (and the lack thereof). > > > > You're invited to check it out: > > > > http://stromberg.dnsalias.org/~dstromberg/significant-whitespace.html > > For those of us who weren't around during the heyday of FORTRAN, can > anyone describe this apparently much-reviled significant whitespace > feature that continues to make some programmers unjustly fearful about > Python's use of indentation? I know it's bad form to cite yourself, but I think I said it pretty well (oh my) 12 years ago: http://www.python.org/doc/humor/#bad-habits To address your question more directly, here's a couple of ways Fortran treated whitespace which would surprise the current crop of Java/PHP/Python/Ruby programmers: 1) Line numbers (i.e. the things you could GOTO to) were in column 2-7 (column 1 was reserved for a comment indicator). This is not quite significant whitespace, it's more like significant indentation. 2) Whitespace was not required in many places. For example, the following two statements (this will only make sense in a fixed-width font) are identical: DO 10 I = 1, 10 DO10I=1,10 People make fun of Fortran these days, but it was a staggeringly important advance in software development compared to what came before (i.e. assembler). The move from assembler to Fortran was a far more significant advance than the move from, say, Perl to Python. From mensanator at aol.com Fri Jan 1 18:29:46 2010 From: mensanator at aol.com (Mensanator) Date: Fri, 1 Jan 2010 15:29:46 -0800 (PST) Subject: Significant whitespace References: Message-ID: <5b7a43b4-c53e-49d5-a1ef-ee19533d851c@o35g2000vbl.googlegroups.com> On Jan 1, 4:02?pm, Dan Stromberg wrote: > I put together a page about significant whitespace (and the lack thereof). The real problem is your use of proportional spaced fonts. > > You're invited to check it out: > > http://stromberg.dnsalias.org/~dstromberg/significant-whitespace.html From tartley at tartley.com Fri Jan 1 18:51:00 2010 From: tartley at tartley.com (Jonathan Hartley) Date: Fri, 1 Jan 2010 15:51:00 -0800 (PST) Subject: Which version of MSVC?90.DLL's to distribute with Python 2.6 based Py2exe executables? References: <05bd2f84-4f09-4a76-8e17-5868450ad393@c3g2000yqd.googlegroups.com> <753bb7c9-14bf-44e3-9679-0693635f2cd9@j14g2000yqm.googlegroups.com> <573025f1-0e5d-4d82-a86b-36ea33f85cad@c3g2000yqd.googlegroups.com> Message-ID: <87e07145-4d35-47ff-9203-5c021a841f06@j24g2000yqa.googlegroups.com> > the method involves editing python26.dll in order to remove > dependency references and then dropping msvcr90.dll in the same > directory as the py2exe produced executable. Clever idea Waldemar, thanks for that, but for the moment, using the dll as a win32 assembly (ie. with a manifest file, as described by others earlier in this thread) seems to work just as well. >> > You'll need to include Microsoft.VC90.CRT.manifest and msvcr90.dll. So, for the record, I have included these two files in a directory called 'Microsoft.VC90.CRT', which is in the same dir as my executable, and that makes everything work, even on my 'bare bones' windows XP virtual machine. The end result is a small game some friends and I put together for the PyWeek competition a few months ago. If anyone wants to see exactly how the resulting dll and manifest look, you can download the Windows binary under the 'featured downloads' from here: http://code.google.com/p/brokenspell/ I appreciate David Bolen's comment that an installer isn't that hard to put together: Acknowledged, but for the moment I still seem to be getting away without needing one. Best regards, Jonathan From steve at REMOVE-THIS-cybersource.com.au Fri Jan 1 20:05:51 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 02 Jan 2010 01:05:51 GMT Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> Message-ID: <034e88f2$0$1277$c3e8da3@news.astraweb.com> On Fri, 01 Jan 2010 14:19:28 -0800, Chris Rebert wrote: > For those of us who weren't around during the heyday of FORTRAN, can > anyone describe this apparently much-reviled significant whitespace > feature that continues to make some programmers unjustly fearful about > Python's use of indentation? I'm not a Fortran expert, but I understand that whitespace was optional within lines unless needed to make something unambiguous. In Python terms, imagine if we could write foriinrange(10): instead of the usual for i in range(10): Since the colon makes it unambiguous that it is some sort of block construct, and it starts with "for", it must be a for loop. Pretty horrible, yes? -- Steven From sjmachin at lexicon.net Fri Jan 1 20:27:03 2010 From: sjmachin at lexicon.net (John Machin) Date: Fri, 1 Jan 2010 17:27:03 -0800 (PST) Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> Message-ID: <3db95947-1e35-4bd1-bd4c-37df646f9ed5@v25g2000yqk.googlegroups.com> On Jan 2, 10:29?am, Roy Smith wrote: > > To address your question more directly, here's a couple of ways Fortran > treated whitespace which would surprise the current crop of > Java/PHP/Python/Ruby programmers: > > 1) Line numbers (i.e. the things you could GOTO to) were in column 2-7 > (column 1 was reserved for a comment indicator). ?This is not quite > significant whitespace, it's more like significant indentation. That would also surprise former FORTRAN programmers (who rarely referred to the language as "Fortran"). A comment was signified by a C in col 1. Otherwise cols 1-5 were used for statement labels (the things you could GOTO), col 6 for a statement continuation indicator, cols 7-72 for statement text, and cols 73-80 for card sequence numbers. From pengyu.ut at gmail.com Fri Jan 1 20:36:24 2010 From: pengyu.ut at gmail.com (Peng Yu) Date: Fri, 1 Jan 2010 19:36:24 -0600 Subject: Exception as the primary error handling mechanism? In-Reply-To: <50697b2c0912312124n7f989568t13e5a1dd8d883e9c@mail.gmail.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <50697b2c0912312124n7f989568t13e5a1dd8d883e9c@mail.gmail.com> Message-ID: <366c6f341001011736u3d5adb1ek9357af29ce060d7c@mail.gmail.com> On Thu, Dec 31, 2009 at 11:24 PM, Chris Rebert wrote: > On Thu, Dec 31, 2009 at 8:47 PM, Peng Yu wrote: >> I observe that python library primarily use exception for error >> handling rather than use error code. >> >> In the article API Design Matters by Michi Henning >> >> Communications of the ACM >> Vol. 52 No. 5, Pages 46-56 >> 10.1145/1506409.1506424 >> http://cacm.acm.org/magazines/2009/5/24646-api-design-matters/fulltext >> >> It says "Another popular design flaw?namely, throwing exceptions for >> expected outcomes?also causes inefficiencies because catching and >> handling exceptions is almost always slower than testing a return >> value." >> >> My observation is contradicted to the above statement by Henning. If >> my observation is wrong, please just ignore my question below. >> >> Otherwise, could some python expert explain to me why exception is >> widely used for error handling in python? Is it because the efficiency >> is not the primary goal of python? > > Correct; programmer efficiency is a more important goal for Python instead. > Python is ~60-100x slower than C;[1] if someone is worried by the > inefficiency caused by exceptions, then they're using completely the > wrong language. Could somebody let me know how the python calls and exceptions are dispatched? Is there a reference for it? From wolftracks at invalid.com Fri Jan 1 20:37:40 2010 From: wolftracks at invalid.com (W. eWatson) Date: Fri, 01 Jan 2010 17:37:40 -0800 Subject: Potential Conflicts by Installing Two Versions of Python (Windows)? Message-ID: I suspect that if one installs v2.4 and 2.5, or any two versions, that one will dominate, or there will be a conflict. I suppose it would not be possible to choose which one should be used. Comments? From wuwei23 at gmail.com Fri Jan 1 20:54:36 2010 From: wuwei23 at gmail.com (alex23) Date: Fri, 1 Jan 2010 17:54:36 -0800 (PST) Subject: Potential Conflicts by Installing Two Versions of Python (Windows)? References: Message-ID: On Jan 2, 11:37?am, "W. eWatson" wrote: > I suspect that if one installs v2.4 and 2.5, or any two versions, that > one will dominate, or there will be a conflict. ?I suppose it would not > be possible to choose which one should be used. Comments? I suspect that you're not the first person to ask this question. I suppose it would not be possible to search the group and find previous advice given on this subject. http://u.nu/4ape4 Is phrasing it as an implicit shortcoming of Python supposed to spur others to comment? Perhaps that's the problem you're having searching: try keywords that represent what you're looking for, not what you're naively expecting. From benjamin.kaplan at case.edu Fri Jan 1 20:57:13 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Fri, 1 Jan 2010 20:57:13 -0500 Subject: Potential Conflicts by Installing Two Versions of Python (Windows)? In-Reply-To: References: Message-ID: The convention (more used among Unix variants but I guess the same thing applies to Windows if you're setting the system path) is that running "python" from the command line will give you the most recently installed one. If you want to specify a version, it would be "python24" or "python25". Each version of Python is installed in it's own directory and maintains its own path so there aren't any problems with stdlib conflicts. As far as Windows file associations go, I suppose the most recent install would dominate though you could always reset the associations yourself. On Fri, Jan 1, 2010 at 8:37 PM, W. eWatson wrote: > I suspect that if one installs v2.4 and 2.5, or any two versions, that one > will dominate, or there will be a conflict. ?I suppose it would not be > possible to choose which one should be used. Comments? > -- > http://mail.python.org/mailman/listinfo/python-list > From benjamin.kaplan at case.edu Fri Jan 1 20:57:34 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Fri, 1 Jan 2010 20:57:34 -0500 Subject: Potential Conflicts by Installing Two Versions of Python (Windows)? In-Reply-To: References: Message-ID: On Fri, Jan 1, 2010 at 8:37 PM, W. eWatson wrote: > I suspect that if one installs v2.4 and 2.5, or any two versions, that one > will dominate, or there will be a conflict. ?I suppose it would not be > possible to choose which one should be used. Comments? > -- > http://mail.python.org/mailman/listinfo/python-list > The convention (more used among Unix variants but I guess the same thing applies to Windows if you're setting the system path) is that running "python" from the command line will give you the most recently installed one. If you want to specify a version, it would be "python24" or "python25". Each version of Python is installed in it's own directory and maintains its own path so there aren't any problems with stdlib conflicts. As far as Windows file associations go, I suppose the most recent install would dominate though you could always reset the associations yourself. From cousinstanley at gmail.com Fri Jan 1 21:01:04 2010 From: cousinstanley at gmail.com (Cousin Stanley) Date: Sat, 2 Jan 2010 02:01:04 +0000 (UTC) Subject: Dynamic text color References: Message-ID: > John Posner wrote .... > .... > I've posted a complete solution .... > > http://cl1p.net/jjp_dynamic_text_color/. John .... Thanks for posting your solution to Dave McCormick's query about colorizing text .... I was not familiar with the re.finditer method for searching strings or with using tags in Tkinter Text widgets to change text attributes .... Instead of binding the Text widget to events to call the colorizer on each key-stroke, I used a button callback .... http://cl1p.net/cs_static_text_color This provides a static one-shot colorization after all of the text is entered and should save a few cpu cycles although it's not as nifty as the dynamic process .... I used a dictionary instead of a list for position data and also added a { term : color } dictionary so that words other than color names can be hi-lighted .... -- Stanley C. Kitching Human Being Phoenix, Arizona From python at mrabarnett.plus.com Fri Jan 1 21:47:52 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 02 Jan 2010 02:47:52 +0000 Subject: Significant whitespace In-Reply-To: <4B3E70FC.1010702@gmail.com> References: <4B3E70FC.1010702@gmail.com> Message-ID: <4B3EB3D8.6030603@mrabarnett.plus.com> Dan Stromberg wrote: > > I put together a page about significant whitespace (and the lack thereof). > > You're invited to check it out: > > http://stromberg.dnsalias.org/~dstromberg/significant-whitespace.html > You might also want to mention that programmers tend to indent anyway for clarity. From roy at panix.com Fri Jan 1 22:56:16 2010 From: roy at panix.com (Roy Smith) Date: Fri, 01 Jan 2010 22:56:16 -0500 Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> <3db95947-1e35-4bd1-bd4c-37df646f9ed5@v25g2000yqk.googlegroups.com> Message-ID: In article <3db95947-1e35-4bd1-bd4c-37df646f9ed5 at v25g2000yqk.googlegroups.com>, John Machin wrote: > On Jan 2, 10:29?am, Roy Smith wrote: > > > > > To address your question more directly, here's a couple of ways Fortran > > treated whitespace which would surprise the current crop of > > Java/PHP/Python/Ruby programmers: > > > > 1) Line numbers (i.e. the things you could GOTO to) were in column 2-7 > > (column 1 was reserved for a comment indicator). ?This is not quite > > significant whitespace, it's more like significant indentation. > > That would also surprise former FORTRAN programmers (who rarely > referred to the language as "Fortran"). A comment was signified by a C > in col 1. Otherwise cols 1-5 were used for statement labels (the > things you could GOTO), col 6 for a statement continuation indicator, > cols 7-72 for statement text, and cols 73-80 for card sequence numbers. My apologies, you are correct. The long disused neural pathways have started to degrade. Trust me, I really did write FORTRAN. On punch cards even. From rogerb at rogerbinns.com Sat Jan 2 00:20:52 2010 From: rogerb at rogerbinns.com (Roger Binns) Date: Fri, 01 Jan 2010 21:20:52 -0800 Subject: Safe file I/O to shared file (or SQLite) from multi-threaded web server In-Reply-To: <7q7ap3F3u36s0U1@mid.uni-berlin.de> References: <7q7ap3F3u36s0U1@mid.uni-berlin.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Diez B. Roggisch wrote: > AFAIK, sqlite ensures process-serialization via locking, and threads > synchronize themselves as well. SQLite versions prior to 3.5 did not support using the same connection or cursors in different threads. (You needed to allocate, use, and close all in the same thread.) Since then SQLite objects can be used in any thread you want at any time. The SQLite error handling API is not threadsafe and requires a lock to be held otherwise you can get incorrect errors or worst case program crashes. The sqlite3/pysqlite code does not hold that lock (API introduced in SQLite 3.6.5) so you are only safe if you continue to only use objects in the same thread. If you use APSW then you can use any SQLite object at any time in any thread (it holds the lock amongst other things). > So you shouldn't need to worry at all. The main gotcha is that SQLite uses file locking and the default behaviour when unable to get a lock is to immediately return an error. SQLite does have an API to specify how long it should wait to acquire the lock (it keeps retrying until the time expires). sqlite3/pysqlite only lets you specify this maximum time when opening the connection and defaults to 5 seconds. On a busy server this may be too short so you'll end up getting busy errors. (Remember that writes require at least two disk syncs and that the default behaviour for Linux is to flush all outstanding writes not just for the file requested.) If you use APSW then you get default SQLite behaviour and two APIs - one lets you set/change the timeout period and the other lets you install your own busy handler which can do whatever it wants in order to prod things along. (Disclosure: I am the author of APSW.) Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAks+17QACgkQmOOfHg372QSiCwCgpr6fSOr6UcUUZqTDoFA4RBcK zb8An21zZCr30AQ7VGP/Q/CsQ3z+2EVs =55MC -----END PGP SIGNATURE----- From donn.ingle at gmail.com Sat Jan 2 00:56:15 2010 From: donn.ingle at gmail.com (Donn) Date: Sat, 2 Jan 2010 07:56:15 +0200 Subject: Significant whitespace In-Reply-To: <4B3E70FC.1010702@gmail.com> References: <4B3E70FC.1010702@gmail.com> Message-ID: <201001020756.15385.donn.ingle@gmail.com> On Saturday 02 January 2010 00:02:36 Dan Stromberg wrote: > I put together a page about significant whitespace (and the lack thereof). The only thing about Python's style that worries me is that it can't be compressed like javascript can*, and perhaps that will prevent it becoming a browser-side language one day. How I'd love to code PyQuery instead of JQuery! * Or am I wrong? \d From clp2 at rebertia.com Sat Jan 2 01:53:10 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Fri, 1 Jan 2010 22:53:10 -0800 Subject: Significant whitespace In-Reply-To: <201001020756.15385.donn.ingle@gmail.com> References: <4B3E70FC.1010702@gmail.com> <201001020756.15385.donn.ingle@gmail.com> Message-ID: <50697b2c1001012253l3f3a166fiab496e348b221f34@mail.gmail.com> On Fri, Jan 1, 2010 at 9:56 PM, Donn wrote: > On Saturday 02 January 2010 00:02:36 Dan Stromberg wrote: >> I put together a page about significant whitespace (and the lack thereof). > The only thing about Python's style that worries me is that it can't be > compressed like javascript can*, and perhaps that will prevent it becoming a > browser-side language one day. How I'd love to code PyQuery instead of JQuery! > > * Or am I wrong? Not to my knowledge, but CPython's bytecode format could be used to similar effect. However, the larger and much more daunting obstacle to client-side Python would be that it's not sandboxed or secured like JavaScript is, although there are some people working on it. Cheers, Chris -- http://blog.rebertia.com From aahz at pythoncraft.com Sat Jan 2 01:56:46 2010 From: aahz at pythoncraft.com (Aahz) Date: 1 Jan 2010 22:56:46 -0800 Subject: Raw string substitution problem References: <931375.44828.qm@web58902.mail.re1.yahoo.com> <7p0q09F6qiU1@mid.individual.net> <7p2juvFu8tU1@mid.individual.net> Message-ID: In article <7p2juvFu8tU1 at mid.individual.net>, Gregory Ewing wrote: >MRAB wrote: >> >> In simple cases you might be replacing with the same string every time, >> but other cases you might want the replacement to contain substrings >> captured by the regex. > >But you can give it a function that has access to the match object and >can produce whatever replacement string it wants. Assuming I remember correctly, the function capability came after the replacement capability. I think that breaking replacement would be a Bad Idea. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From aahz at pythoncraft.com Sat Jan 2 02:02:50 2010 From: aahz at pythoncraft.com (Aahz) Date: 1 Jan 2010 23:02:50 -0800 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4B3DCFAB.3030909@v.loewis.de> Message-ID: In article <4B3DCFAB.3030909 at v.loewis.de>, Martin v. Loewis wrote: > >Notice that in cases where the failure may be expected, Python >also offers variants that avoid the exception: >- if you look into a dictionary, expecting that a key may not > be there, a regular access, d[k], may give a KeyError. However, > alternatively, you can use d.get(k, default) which raises no > exception, and you can test "k in d" in advance. >- if you open a file, not knowing whether it will be there, > you get an IOError. However, you can use os.path.exists in > advance to determine whether the file is present (and create > it if it's not). But you *still* need to catch IOError: someone might delete the file after the test. Figuring out how to deal with race conditions is one of the main reasons Alex Martelli advocates EAFP over LBYL. Of course, in the real world, you often end up wanting to do it both ways. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From r.grimm at science-computing.de Sat Jan 2 02:18:38 2010 From: r.grimm at science-computing.de (Rainer Grimm) Date: Fri, 1 Jan 2010 23:18:38 -0800 (PST) Subject: multivariable assignment References: Message-ID: <347e6dcf-1437-4d1f-a563-c9d865fc9eb9@j4g2000yqe.googlegroups.com> On Dec 31 2009, 5:13?pm, davidj411 wrote: > I am not sure why this behavior is this way. > at beginning of script, i want to create a bunch of empty lists and > use each one for its own purpose. > however, updating one list seems to update the others. > > >>> a = b = c = [] > >>> a.append('1') > >>> a.append('1') > >>> a.append('1') > >>> c > ['1', '1', '1'] > >>> a > ['1', '1', '1'] > >>> b > > ['1', '1', '1'] a, b and c are the same objects. You can check the object identity by >>> a = b = c = [] >>> print id(a),id(b),id(c) 3083044140 3083044140 3083044140 >>> a is b is c True Greetings From tjreedy at udel.edu Sat Jan 2 02:22:37 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 02 Jan 2010 02:22:37 -0500 Subject: Potential Conflicts by Installing Two Versions of Python (Windows)? In-Reply-To: References: Message-ID: On 1/1/2010 8:57 PM, Benjamin Kaplan wrote: > As far as Windows file associations go, I suppose the most recent > install would dominate though you could always reset the associations > yourself. The Windows installer asks whether one wants the about-to-be-installed version to capture the associations or not. From apt.shansen at gmail.com Sat Jan 2 03:05:56 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Sat, 2 Jan 2010 00:05:56 -0800 Subject: Exception as the primary error handling mechanism? In-Reply-To: <366c6f341001011736u3d5adb1ek9357af29ce060d7c@mail.gmail.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <50697b2c0912312124n7f989568t13e5a1dd8d883e9c@mail.gmail.com> <366c6f341001011736u3d5adb1ek9357af29ce060d7c@mail.gmail.com> Message-ID: <7a9c25c21001020005l1e1feff5head73ee0d7789705@mail.gmail.com> On Fri, Jan 1, 2010 at 5:36 PM, Peng Yu wrote: > >> Otherwise, could some python expert explain to me why exception is > >> widely used for error handling in python? Is it because the efficiency > >> is not the primary goal of python? > > > > Correct; programmer efficiency is a more important goal for Python > instead. > > Python is ~60-100x slower than C;[1] if someone is worried by the > > inefficiency caused by exceptions, then they're using completely the > > wrong language. > > Could somebody let me know how the python calls and exceptions are > dispatched? Is there a reference for it? > I don't quite understand what you're asking here, but it sounds almost like you're looking at the question from an incorrect POV. "Exceptions" are a general sort of concept in computer science and various computer programming languages, but they are not at all equal from one language to another. The document you referenced was speaking to a particular implementation of the concept, and speaking to particular characteristics of that language's implementation. Even though its not talking just about say, C, C#, Java, or anything -- its speaking from a certain POV of a certain classes of languages. In Python, setting up an exception -- the 'try' clause -- costs virtually nothing. Its about equivalent to having a 'pass' statement in there. If you do a test for every iteration of some activity, you're incurring a non-negligable cost each time. If you're performing an action and "usually" (to varying definitions of 'usually'), it's going to succeed-- then that test will result in far more cost in time then using a try/except clause in Python. Because in the implementation of exceptions in Python, you only pay a more expensive cost /if/ that exception is thrown and handled. If its very likely that in a situation an exception would be thrown, then yes-- then you should probably test first... if that exception-catch is so expensive as to be important to your. In most cases, its not. In the vast majority of cases, this is premature optimization and often adds additional weight to the test as you have to protect against race conditions. (As an aside, in many cases using exceptions actually helps you in a wider problem of preventing race conditions. Its not a cure-all by any means, but it helps) If someone specifies a file, the chances are-- the file is there. Its cheaper for you to just try to open it then to test if its there first, because if you do the test? Then the likely circumstance (the file exists) is running code to test that case /every time/. Whereas if you just try to open it, in the likely circumstance -- it works. In the exceptional circumstance, it might cost more then if you had tested first... but that's an exceptional circumstance and therefore more rare. --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From xahlee at gmail.com Sat Jan 2 04:14:12 2010 From: xahlee at gmail.com (Xah Lee) Date: Sat, 2 Jan 2010 01:14:12 -0800 (PST) Subject: Xah's Edu Corner: Teach Ourself Programing In Ten Years? Message-ID: <1035ce5d-914e-42c0-8e9c-a349a6d55e55@v25g2000yqk.googlegroups.com> On Dec 25 2009, 12:44 am, r... at rpw3.org (Rob Warnock) wrote: > jos... at corporate-world.lisp.de wrote: > > +--------------- > | p... at informatimago.com (Pascal J. Bourguignon) wrote: > | > LOL Yeah right! Give gavino ten years of rest to let > | > his unconscious mind work on it! > | > | Norvig's 'Teach Yourself Programming in Ten Years' ( > |http://norvig.com/21-days.html > | ) gets a new meaning... > +--------------- See: ? The Condition of Industrial Programers http://xahlee.org/UnixResource_dir/writ/it_programers.html plain text version follows. ------------------------------ The Condition of Industrial Programers Xah Lee, 2006-05 Before i stepped into the computing industry, my first industrial programing experience was at Wolfram Research Inc as a intern in 1995. (Wolfram Research is famously known for their highly successful flagship product Mathematica) I thought, that the programers at Wolfram are the world's top mathematicians, gathered together to research and decide and write a extremely advanced technology. But i realized it is not so. Not at all. In fact, we might say it's just a bunch of Ph Ds (or equivalent experience). The people there, are not unlike average white-collar Joes. Each working individually. And, fights and bouts of arguments between co-workers are not uncommon. Sometimes downright ugly in emails. Almost nothing is as i naively imagined, as if some world's top mathematicians are gathered together there, daily to confer and solve the world's top problems as in some top secret government agency depicted in movies. Well, that was my introduction to the industry. The bulk of my surprise is due to my naiveness and inexperience of the industry, of any industry, as i was just a intern and this is my first experience seeing how the real world works. After Wolfram, after a couple of years i went into the web programing industry in 1998, using unix, Perl, Apache, Java, database technologies, in the center of world's software technology the Silicon Valley. My evaluation of industrial programers and how software are written is a precipitous fall from my observations at Wolfram. In the so-called Info Tech industry, the vast majority of programers are poorly qualified. I learned this from my colleagues, and in dealing with programers from other companies, service providers, data centers, sys admins, API gateways, and duties of field tutoring. I didn't think i have very qualified expertise in what i do, but the reality i realized is that most are far lesser than me, and that is the common situation. That they have no understanding of basic mathematics such as trigonometry or calculus. Most have no interest in math whatsoever, and would be hard pressed for them to explain what is a ?algorithm?. I have always thought, that programing X software of field Y usually means that the programers are thoroughly fluent in languages, protocols, tools of X, and also being a top expert in field of Y. But to my great surprise, the fact is that that is almost never the case. In fact, most of the time the programers simply just had to learn a language, protocol, software tool, right at the moment as he is trying to implement a software for a field he never had experience in. I myself had to do jobs half of the time i've never done before. Constantly I'm learning new languages, protocols, systems, tools, APIs, other rising practices and technologies, reading semi-written or delve into non-existent docs. It is the norm in the IT industry, that most products are really produces of learning experiences. Extremely hurried grasping of new technologies in competition with deadlines. There is in fact little actual learning going on, as there are immense pressure to simply ?get it to (demonstrably) work? and ship it. Thinking back, in fact the Wolfram people are the most knowledgeable and inquisitive people i've met as colleagues, by far. What prompted me to write this essay is after reading the essay Teach Yourself Programming in Ten Years by Peter Norvig, 2001, at http://www.norvig.com/21-days.html. In which, the LISP dignitary Peter Norvig derides the widely popular computing books in the name of Teaching Yourself X In (Fast) Days. Although i agree with his sentiment that a language or technology takes time to master and use well, that these books form somewhat of a damaging fad and subtly multiply ignorance, but he fails to address the main point, that is: the cause of the popularity of such books, and how to remedy the situation. When you work in the industry, and are given a responsibility of coding in some new language the company decided to use, or emerging protocol (such as voice-chat protocols or cellphone internet), or your engineering group adopted a new team coding/reviewing process, you are not going to tell you boss ?nah, i want to do a good job so i'll study the issue a few months before i contribute?. Chances are, you are going to run out and buy a copy of ?XYZ in 7 days?, and complete the job in a way satisfactorily to your company, as well feeling proud of your abilities in acquiring new material. To see this in a different context, suppose you need to pass a important Math XYZ exam or review in your career or get a certificate, but you don't remember your Math XYZ. You will likely, run out and get a ?Math XYZ for Dummies?. Chances are, the book will indeed help you, and you will pass your exam or interview, and actually have learned something about XYZ, but never looked at Math XYZ squarely again. These books are the bedrock of the industry. It is not because people are impatient, or that they wish to hurry, but rather, it is the condition of the IT industry, in the same way modern society drives people to live certain life styles. No amount of patience or proselytization can right this, except that we change the industry's practice of quickly churning out bug-ridden software products to beat competitors. Companies do that due to market forces, and the market forces is a result of how people and organizations actually choose to purchase software. In my opinion, a solution to this is by installing the concept of responsible licenses. Please see this essay Responsible Software Licensing and spread the word. Xah ? http://xahlee.org/ ? From sanneh27 at hotmail.com Sat Jan 2 04:52:45 2010 From: sanneh27 at hotmail.com (baboucarr sanneh) Date: Sat, 2 Jan 2010 09:52:45 +0000 Subject: HAppy New Year Message-ID: Hi guys, Jus want to wish you a happy new year to u all.. 1.Copy the content below and Paste it on a Notepad.2 . Use replace all (Ctrl + H)3. Type in ?6? in the ?find what? column 4. Type in _ (underscore) in the ?replace with? column 5. Click on replace all. 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666996666699669966999999996699666669966666669966666699669999999966996666996666666666666696666666666666666666666 666996666699669966999999996699666669966666666996666996669999999966996666996666666666666999666666666666666666666 666996666699669966996666666699666669966666666699669966669966669966996666996666666666669969966666666666666666666 666996696699669966999999996699999999966666666669999666669966669966996666996666666666699666996666666666666666666 666996999699669966999999996699999999966666666666996666669966669966996666996666666666999999999666666666666666666 666999969999669966666666996699666669966666666666996666669966669966996666996666666669999999999966666666666666666 666999666999669966999999996699666669966666666666996666669999999966996666996666666699666666666996666666666666666 666996666699669966999999996699666669966666666666996666669999999966999999996666666996666666666699666666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 669966666996666666669666666666999999996699999999669966666699666666669966666996699999999669966666996666666666666 669966666996666666699966666666999999996699999999666996666996666666669996666996699999999669966666996666666666666 669966666996666666996996666666996666996699666699666699669966666666669999666996699666666669966666996666666666666 669999999996666669966699666666999999996699999999666669999666666666669969966996699999999669966966996666666666666 669999999996666699999999966666999999996699999999666666996666666666669966996996699999999669969996996666666666666 669966666996666999999999996666999666666699666666666666996666666666669966699996699666666669999699996666666666666 669966666996669966666666699666996666666699666666666666996666666666669966669996699999999669996669996666666666666 669966666996699666666666669966996666666699666666666666996666666666669966666996699999999669966666996666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666996666669966999999996666666669666666666999999996666666699999996699999999666999666699999999666666666666666666 666699666699666999999996666666699966666666999999996666666699999999699666699666699666699666699666666666666666666 666669966996666996666666666666996996666666996666996666666666666699699966699666699666699966699666666666666666666 666666999966666999999996666669966699666666999999996666666666666996699696699666699666699696699666666666666666666 666666699666666999999996666699999999966666999999996666666666669966699669699666699666699669699666666666666666666 666666699666666996666666666999999999996666996996666666666666996666699666999666699666699666999666666666666666666 666666699666666999999996669966666666699666996699666666666699999999699666699666699666699666699666666666666666666 666666699666666999999996699666666666669966996666996666666699999999699999999666999966699999999666666666666666666 66666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 Regards, Baboucarr. _________________________________________________________________ Windows Live: Friends get your Flickr, Yelp, and Digg updates when they e-mail you. http://www.microsoft.com/middleeast/windows/windowslive/see-it-in-action/social-network-basics.aspx?ocid=PID23461::T:WLMTAGL:ON:WL:en-xm:SI_SB_3:092010 -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul at subsignal.org Sat Jan 2 05:13:51 2010 From: paul at subsignal.org (=?ISO-8859-1?Q?Paul_K=F6lle?=) Date: Sat, 02 Jan 2010 11:13:51 +0100 Subject: Trying to run a sudo command from script In-Reply-To: References: Message-ID: Am 01.01.2010 23:55, schrieb Kent Tenney: > Howdy, Hi Kent, > A script running as a regular user sometimes wants > to run sudo commands. > > It gets the password with getpass. > pw = getpass.getpass() > > I've fiddled a bunch with stuff like > proc = subprocess.Popen('sudo touch /etc/foo'.split(), stdin=subprocess.PIPE) > proc.communicate(input=pw) If you don't use shell=True you have to provide the full path to commands (and split command and parameters as you do). So eather of this works for me: p = Popen('/usr/bin/sudo /usr/bin/touch /tmp/foo.txt'.split(), stdin=PIPE, stdout=PIPE) p = Popen('/usr/bin/sudo /usr/bin/touch /tmp/foo2.txt', stdin=PIPE, stdout=PIPE, shell=True) The bad news is: It this gives me a password promt inside the interactive interpreter. Seems you can't catch stdout this way. hth Paul > > getting assorted errors with all variations I try. > > Googling says use pexpect, but I'd prefer a stdlib solution. > > Any help appreciated. > > Thanks, > Kent From steve at REMOVE-THIS-cybersource.com.au Sat Jan 2 05:42:17 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 02 Jan 2010 10:42:17 GMT Subject: Bare Excepts References: <4dc0cfea0912300209x15e5bc96nd6e6224d89df0638@mail.gmail.com> <4B3B4FE5.8070605@sequans.com> <4dc0cfea0912300509v7f9bb367jf33bb63736407ed@mail.gmail.com> <4dc0cfea0912300512h43a4fdb1oa585a36391de341e@mail.gmail.com> <6944e440-c797-439d-9bf9-b61a05a3e26c@k23g2000yqa.googlegroups.com> <878wck2ojd.fsf@benfinney.id.au> <4dc0cfea0912301927q16445c59r1fb8594ee0f04d24@mail.gmail.com> Message-ID: <034f1009$0$1277$c3e8da3@news.astraweb.com> On Fri, 01 Jan 2010 15:27:57 -0800, myle wrote: > Why we should prefer ``if: ...'' over a ``try: ... except something: > pass'' block? We shouldn't, not in general. Often, the if test is just as expensive as actually doing it. E.g.: if x in mylist: position = mylist.index(x) else: do_something() has to search the list twice, first to see if x is there, then search it again to return the index. That does double the work needed. A better way is: try: position = mylist.index(x) except ValueError: do_something() unless you expect that x is often missing: setting up the try block is much faster than the test, but catching the exception is much, much slower. So long as exceptions are rare, it is better to use the try version (also known as "Better to ask forgiveness than permission"). But if exceptions are common, it may be better to use the if version ("Look before you leap"). The try version is also better whenever there could be a race condition. For example, when opening a file, you might be tempted to do this: if os.path.exists(filename): f = open(filename, 'r') text = f.read() else: print "file missing" but that is dangerous in a multi-processing computer system (which nearly all computers are these days). The problem is that in the fraction of a second after you test that the file exists, some other process might come along and delete it and the open call will then fail. So to be safe, you have to do this: if os.path.exists(filename): try: f = open(filename, 'r') except IOError: print "file missing" else: text = f.read() else: print "file missing" But of course now the test is redundant, and all you really need is the try...except block: try: f = open(filename, 'r') except IOError: print "file missing" else: text = f.read() (Note: the above is still not entirely robust. For example, if you try to read a file that you don't have permission to read, it wrongly reports it is missing instead of a permissions error. To make this more robust, you need to inspect the exception and check the error code.) -- Steven From duncan.booth at invalid.invalid Sat Jan 2 05:42:39 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 2 Jan 2010 10:42:39 GMT Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> Message-ID: Donn wrote: > On Saturday 02 January 2010 00:02:36 Dan Stromberg wrote: >> I put together a page about significant whitespace (and the lack >> thereof). > The only thing about Python's style that worries me is that it can't > be compressed like javascript can*, and perhaps that will prevent it > becoming a browser-side language one day. How I'd love to code PyQuery > instead of JQuery! You can do simple white-space removal as easily in Python as in Javascript, you just need to maintain a minimal indentation and as you don't need Javascript's curly braces there probably isn't must difference especially once the code has been gzipped for transfer to the browser. You can already run Python client side in certain situations (e.g. Silverlight), but if you want to write Python for the browser portably today have a look at Pyjamas http://code.google.com/p/pyjamas/ . Google's Chrome also supports native code (currently just for x86 machines though they have said they intend to support ARM), so there should be some potential there to compile a Python interpreter that runs client-side in Google's sandbox: see http://plash.beasts.org/wiki/NativeClient for an early port. From victorsubervi at gmail.com Sat Jan 2 05:53:26 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 2 Jan 2010 05:53:26 -0500 Subject: Append Problem Message-ID: <4dc0cfea1001020253u6f2bf8a9i7f16c088d4ad78f7@mail.gmail.com> Hi; I have the following code snippet: print 'Original: ', catChains, '
' while i < MAXLEVEL: flag = 0 j = 0 while j < len(parents): for chain in catChains: if parents[j] == chain[len(chain)-1]: chain.append(children[j]) print '1: ', catChains, '
' catChains.append(chain) print '2: ', catChains, '
' flag = 1 j += 1 i += 1 if flag == 0: break print 'Final: ', catChains which prints this: Original: [['prodCat1'], ['prodCat2']] 1: [['prodCat1', 'prodCat3'], ['prodCat2']] 2: [['prodCat1', 'prodCat3'], ['prodCat2'], ['prodCat1', 'prodCat3']] 1: [['prodCat1', 'prodCat3', 'prodCat5'], ['prodCat2'], ['prodCat1', 'prodCat3', 'prodCat5']] 2: [['prodCat1', 'prodCat3', 'prodCat5'], ['prodCat2'], ['prodCat1', 'prodCat3', 'prodCat5'], ['prodCat1', 'prodCat3', 'prodCat5']] Final: [['prodCat1', 'prodCat3', 'prodCat5'], ['prodCat2'], ['prodCat1', 'prodCat3', 'prodCat5'], ['prodCat1', 'prodCat3', 'prodCat5']] Why is it that my append statement *deletes* elements of my tuple? The entire code follows: #! /usr/bin/python import string import cgitb; cgitb.enable() import MySQLdb import cgi import sys,os sys.path.append(os.getcwd()) from login import login import datetime, Cookie, random from particulars import title from templateFrame import top, bottom from particulars import myCookie import time import fpformat from sets import Set def makeNav(): print 'Content-type: text/html\r\n' print '\n' user, passwd, db, host = login() db = MySQLdb.connect(host, user, passwd, db) cursor= db.cursor() form = cgi.FieldStorage() store = form.getfirst('store') sql = 'select Category, Parent from categories%s;' % (store[0].upper() + store[1:]) cursor.execute(sql) children = [itm[0] for itm in cursor] parents = [itm[1] for itm in cursor] catChains = [] i = 0 while i < len(parents): if parents[i] is None: child = [] child.append(children[i]) catChains.append(child) i += 1 i = 0 MAXLEVEL = 10 print 'Original: ', catChains, '
' while i < MAXLEVEL: flag = 0 j = 0 while j < len(parents): for chain in catChains: if parents[j] == chain[len(chain)-1]: chain.append(children[j]) print '1: ', catChains, '
' catChains.append(chain) print '2: ', catChains, '
' flag = 1 j += 1 i += 1 if flag == 0: break print 'Final: ', catChains lastChain = [] lastChain.append('root') print "' makeNav() PS (Mainly, I believe, for Carsten): While programming is difficult for me, I am writing (and have pretty much finished) this (almost) fully automated shopping cart. I don't intend to write any other serious programming project. However, a shopping cart is vital to my Web design business. I will hereafter outsource my programming. But, as any good supervisor knows, one has to know how to supervise! If I can't read code, then I'm at the mercy of my programmer(s). If I can't afford to hire some top gun who can supervise, then it's my responsibility. Plus, if my programmer(s) quit, I need to step in and take over. I'm an ok businessman, and absolutely gifted in sales. And in poetry ;) beno -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From doomster at knuut.de Sat Jan 2 07:01:28 2010 From: doomster at knuut.de (Ulrich Eckhardt) Date: Sat, 02 Jan 2010 13:01:28 +0100 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <50697b2c0912312124n7f989568t13e5a1dd8d883e9c@mail.gmail.com> Message-ID: <7q8qspF3u95vnU1@mid.uni-berlin.de> Peng Yu wrote: > Could somebody let me know how the python calls and exceptions are > dispatched? Is there a reference for it? I'm not a Python expert, but I have read some parts of the implementation. Hopefully someone steps up if I misrepresent things here... In order to understand Python exception handling, take a look at various C function implementations. You will see that they commonly return a pointer to a Python object (PyObject*), even if it is a pointer to the 'None' singleton. So, a function in Python _always_ returns something, even if it is 'None'. If, at the C level, a function doesn't return anything (i.e. a C NULL pointer) that means that the function raised an exception. Checking this pointer is pretty easy, typically you check that, clean up and return NULL yourself. Further functions for manipulating the exception stack and declarations of exception types and singletons are found in pyerrors.h (in Python 2.5, at least). I mentioned an "exception stack" above, though I'm not 100% sure if that is the proper term. I think that exceptions can be stacked upon each other (e.g. an HTTPD throwing a high-level RequestError when it encounters a low- level IOError) and that that is also how the backtrace is implemented, but I'm not sure about that. Hopefully someone can confirm or correct me here and that it helped you. Cheers! Uli From steve at REMOVE-THIS-cybersource.com.au Sat Jan 2 07:03:28 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 02 Jan 2010 12:03:28 GMT Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> Message-ID: <034f2310$0$1277$c3e8da3@news.astraweb.com> On Sat, 02 Jan 2010 10:42:39 +0000, Duncan Booth wrote: > Donn wrote: > >> On Saturday 02 January 2010 00:02:36 Dan Stromberg wrote: >>> I put together a page about significant whitespace (and the lack >>> thereof). >> The only thing about Python's style that worries me is that it can't be >> compressed like javascript can*, and perhaps that will prevent it >> becoming a browser-side language one day. How I'd love to code PyQuery >> instead of JQuery! > > You can do simple white-space removal as easily in Python as in > Javascript, you just need to maintain a minimal indentation and as you > don't need Javascript's curly braces there probably isn't must > difference especially once the code has been gzipped for transfer to the > browser. I don't understand why you would bother removing whitespace, particularly if you're going to gzip the file anyway. I've just picked one of my modules at random: about 47K in size, of which about 15K is whitespace. Compressed to .bz, it is 12K, and if I strip the whitespace from it first and then compress it, it is 11K. So, even though whitespace (spaces and newlines) make up about 30% of the original source, once compressed they represent only about 8%. Given the extra complexity of stripping whitespace, I don't know that it's worthwhile. It's not like you're going to reduce the download time from three hours to one; it's more like 2 seconds to 1.5. -- Steven From deets at nospam.web.de Sat Jan 2 07:05:11 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Sat, 02 Jan 2010 13:05:11 +0100 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <50697b2c0912312124n7f989568t13e5a1dd8d883e9c@mail.gmail.com> Message-ID: <7q8r3nF3u6mmdU1@mid.uni-berlin.de> Peng Yu schrieb: > On Thu, Dec 31, 2009 at 11:24 PM, Chris Rebert wrote: >> On Thu, Dec 31, 2009 at 8:47 PM, Peng Yu wrote: >>> I observe that python library primarily use exception for error >>> handling rather than use error code. >>> >>> In the article API Design Matters by Michi Henning >>> >>> Communications of the ACM >>> Vol. 52 No. 5, Pages 46-56 >>> 10.1145/1506409.1506424 >>> http://cacm.acm.org/magazines/2009/5/24646-api-design-matters/fulltext >>> >>> It says "Another popular design flaw?namely, throwing exceptions for >>> expected outcomes?also causes inefficiencies because catching and >>> handling exceptions is almost always slower than testing a return >>> value." >>> >>> My observation is contradicted to the above statement by Henning. If >>> my observation is wrong, please just ignore my question below. >>> >>> Otherwise, could some python expert explain to me why exception is >>> widely used for error handling in python? Is it because the efficiency >>> is not the primary goal of python? >> Correct; programmer efficiency is a more important goal for Python instead. >> Python is ~60-100x slower than C;[1] if someone is worried by the >> inefficiency caused by exceptions, then they're using completely the >> wrong language. > > Could somebody let me know how the python calls and exceptions are > dispatched? Is there a reference for it? The source? http://python.org/ftp/python/2.6.4/Python-2.6.4.tgz These are really deep internals that - if they really concern you - need intensive studies, not casual reading of introductionary documents. IMHO you shouldn't worry, but then, there's a lot things you seem to care I wouldn't... :) Diez From vicente.soler at gmail.com Sat Jan 2 07:50:32 2010 From: vicente.soler at gmail.com (vsoler) Date: Sat, 2 Jan 2010 04:50:32 -0800 (PST) Subject: Endless loop Message-ID: <7e1289b1-4d6d-42e0-9325-ee4066f3e5b0@r5g2000yqb.googlegroups.com> hello, I'm learning Python and OOP, and I am confronted with a rather theoretical problem. If I run the following script: class stepper: def __getitem__(self, i): return self.data[i] X=stepper() X.data="Spam" for item in X: print item, ... what I get is S p a m which seems logical to me since the loop stops after the 4th character. However if I run class stepper: def __getitem__(self, i): return i X=stepper() X.data="Spam" for item in X: print item, ... what I get is an endless loop, starting at zero: 0 1 2 3 4 5 6 7 8 9 10 11 and so on. My question is: why does this second script not stop after printing number 3? what made the first one stop while the second one will not? Thanks for your advise Vicente Soler From martin at v.loewis.de Sat Jan 2 07:52:10 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Sat, 02 Jan 2010 13:52:10 +0100 Subject: Exception as the primary error handling mechanism? In-Reply-To: <7q8qspF3u95vnU1@mid.uni-berlin.de> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <50697b2c0912312124n7f989568t13e5a1dd8d883e9c@mail.gmail.com> <7q8qspF3u95vnU1@mid.uni-berlin.de> Message-ID: <4B3F417A.6080109@v.loewis.de> > I mentioned an "exception stack" above, though I'm not 100% sure if that is > the proper term. I think that exceptions can be stacked upon each other > (e.g. an HTTPD throwing a high-level RequestError when it encounters a low- > level IOError) and that that is also how the backtrace is implemented, but > I'm not sure about that. Not exactly. In this scenario, the IOError exception gets caught, its entire traceback discarded, and an entirely new exception RequestError gets raised (that has no connection to the original IOError anymore, unless the httpd code explicitly links the two). Instead, the traceback objects are created for a single exception. They are essentially the same as the call stack, just in reverse order (so that you get the "most recent call last" traceback output). Each traceback links to a frame object, and a next traceback object. Regards, Martin From steve at holdenweb.com Sat Jan 2 08:08:12 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 02 Jan 2010 08:08:12 -0500 Subject: Trying to run a sudo command from script In-Reply-To: References: Message-ID: Paul K?lle wrote: > Am 01.01.2010 23:55, schrieb Kent Tenney: >> Howdy, > Hi Kent, > >> A script running as a regular user sometimes wants >> to run sudo commands. >> >> It gets the password with getpass. >> pw = getpass.getpass() >> >> I've fiddled a bunch with stuff like >> proc = subprocess.Popen('sudo touch /etc/foo'.split(), >> stdin=subprocess.PIPE) >> proc.communicate(input=pw) > If you don't use shell=True you have to provide the full path to > commands (and split command and parameters as you do). So eather of this > works for me: > p = Popen('/usr/bin/sudo /usr/bin/touch /tmp/foo.txt'.split(), > stdin=PIPE, stdout=PIPE) > > p = Popen('/usr/bin/sudo /usr/bin/touch /tmp/foo2.txt', stdin=PIPE, > stdout=PIPE, shell=True) > > The bad news is: It this gives me a password promt inside the > interactive interpreter. Seems you can't catch stdout this way. > [please put your answers *after* the questions!] I don't think it so much that you can't catch stdout. Rather, sudo ensures it is talking to the user by explicitly reading the password from /dev/tty. Consequently there is no way to send the password value over the process's stdin, hence the need to use pexpect. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From taradov at gmail.com Sat Jan 2 08:18:13 2010 From: taradov at gmail.com (alexru) Date: Sat, 2 Jan 2010 05:18:13 -0800 (PST) Subject: Endless loop References: <7e1289b1-4d6d-42e0-9325-ee4066f3e5b0@r5g2000yqb.googlegroups.com> Message-ID: <0b4c8c2b-36e5-4962-bf69-7af9b175a27b@22g2000yqr.googlegroups.com> On Jan 2, 3:50?pm, vsoler wrote: > My question is: why does this second script not stop after printing > number 3? ?what made the first one stop while the second one will not? First one will raise IndexError when string is over, second one won't. From doomster at knuut.de Sat Jan 2 08:21:27 2010 From: doomster at knuut.de (Ulrich Eckhardt) Date: Sat, 02 Jan 2010 14:21:27 +0100 Subject: Endless loop References: <7e1289b1-4d6d-42e0-9325-ee4066f3e5b0@r5g2000yqb.googlegroups.com> Message-ID: <7q8virF3v0lcqU1@mid.uni-berlin.de> vsoler wrote: > class stepper: > def __getitem__(self, i): > return self.data[i] > > X=stepper() > X.data="Spam" > for item in X: > print item, > > ... what I get is S p a m which seems logical to me since the > loop stops after the 4th character. I think you're mistaking the cause and effect here, see below. > class stepper: > def __getitem__(self, i): > return i > > X=stepper() > X.data="Spam" > for item in X: > print item, > > ... what I get is an endless loop, starting at zero: 0 1 2 3 4 5 6 > 7 8 9 10 11 and so on. > > My question is: why does this second script not stop after printing > number 3? what made the first one stop while the second one will not? First thing to observe in the second case is that X's data field is not used anywhere. The field is set and then not used any further. In particular, it is not used to somehow get the length of the sequence to return. Now, "spam"[4] will raise an IndexError, thus terminating the iteration. The second implementation which only returns the index never will do that, so the iteration never stops. Uli From steve at holdenweb.com Sat Jan 2 08:24:33 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 02 Jan 2010 08:24:33 -0500 Subject: Append Problem In-Reply-To: <4dc0cfea1001020253u6f2bf8a9i7f16c088d4ad78f7@mail.gmail.com> References: <4dc0cfea1001020253u6f2bf8a9i7f16c088d4ad78f7@mail.gmail.com> Message-ID: Victor Subervi wrote: > Hi; > I have the following code snippet: > > print 'Original: ', catChains, '
' > while i < MAXLEVEL: > flag = 0 > j = 0 > while j < len(parents): > for chain in catChains: > if parents[j] == chain[len(chain)-1]: > chain.append(children[j]) > print '1: ', catChains, '
' > catChains.append(chain) > print '2: ', catChains, '
' > flag = 1 > j += 1 > i += 1 > if flag == 0: > break > print 'Final: ', catChains > > which prints this: > > Original: [['prodCat1'], ['prodCat2']] > 1: [['prodCat1', 'prodCat3'], ['prodCat2']] > 2: [['prodCat1', 'prodCat3'], ['prodCat2'], ['prodCat1', 'prodCat3']] > 1: [['prodCat1', 'prodCat3', 'prodCat5'], ['prodCat2'], ['prodCat1', > 'prodCat3', 'prodCat5']] > 2: [['prodCat1', 'prodCat3', 'prodCat5'], ['prodCat2'], ['prodCat1', > 'prodCat3', 'prodCat5'], ['prodCat1', 'prodCat3', 'prodCat5']] > Final: [['prodCat1', 'prodCat3', 'prodCat5'], ['prodCat2'], ['prodCat1', > 'prodCat3', 'prodCat5'], ['prodCat1', 'prodCat3', 'prodCat5']] > > Why is it that my append statement *deletes* elements of my tuple? The > entire code follows: > There isn't a tuple is sight there - you are dealing with lists, and you are modifying the very list you are iterating over. Try rewriting the code to create a new list from the old one (i.e. iterate over catChains and have your code append to an initially empty list called, for example, newCatChains, then finally throw the old list away with catChains = newCatChains or some such). [code snipped] > > PS (Mainly, I believe, for Carsten): While programming is difficult for > me, I am writing (and have pretty much finished) this (almost) fully > automated shopping cart. I don't intend to write any other serious > programming project. However, a shopping cart is vital to my Web design > business. I will hereafter outsource my programming. But, as any good > supervisor knows, one has to know how to supervise! If I can't read > code, then I'm at the mercy of my programmer(s). If I can't afford to > hire some top gun who can supervise, then it's my responsibility. Plus, > if my programmer(s) quit, I need to step in and take over. I'm an ok > businessman, and absolutely gifted in sales. And in poetry ;) > beno > Given that there are umpteen thousand hosting services who will let you create a web site with shopping cart included I have to question whether you are making the best use of your time here. Of course it's your time, and your decision, so please don't think I am dictating to you. While the learning you have performed in getting this system to work (for some rather questionable value of "work", I can't help feeling, but we'll overlook the horrors induced by your lack of programming experience) is a testament to your persistence (and this list's tolerance) I can't help feeling that it leaves you no better qualified to supervise programmers. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From vicente.soler at gmail.com Sat Jan 2 08:45:18 2010 From: vicente.soler at gmail.com (vsoler) Date: Sat, 2 Jan 2010 05:45:18 -0800 (PST) Subject: Endless loop References: <7e1289b1-4d6d-42e0-9325-ee4066f3e5b0@r5g2000yqb.googlegroups.com> <7q8virF3v0lcqU1@mid.uni-berlin.de> Message-ID: On 2 ene, 14:21, Ulrich Eckhardt wrote: > vsoler wrote: > > class stepper: > > ? ? def __getitem__(self, i): > > ? ? ? ? return self.data[i] > > > X=stepper() > > X.data="Spam" > > for item in X: > > ? ? print item, > > > ... what I get is ? ? S p a m ? ? which seems logical to me since the > > loop stops after the 4th character. > > I think you're mistaking the cause and effect here, see below. > > > class stepper: > > ? ? def __getitem__(self, i): > > ? ? ? ? return i > > > X=stepper() > > X.data="Spam" > > for item in X: > > ? ? print item, > > > ... what I get is an endless loop, starting at zero: ? ?0 1 2 3 4 5 6 > > 7 8 9 10 11 ?and so on. > > > My question is: why does this second script not stop after printing > > number 3? ?what made the first one stop while the second one will not? > > First thing to observe in the second case is that X's data field is not used > anywhere. The field is set and then not used any further. In particular, it > is not used to somehow get the length of the sequence to return. > > Now, "spam"[4] will raise an IndexError, thus terminating the iteration. The > second implementation which only returns the index never will do that, so > the iteration never stops. > > Uli It's clear, I now understand. Thank you From lie.1296 at gmail.com Sat Jan 2 09:10:51 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Sun, 03 Jan 2010 01:10:51 +1100 Subject: Bare Excepts In-Reply-To: <034f1009$0$1277$c3e8da3@news.astraweb.com> References: <4dc0cfea0912300209x15e5bc96nd6e6224d89df0638@mail.gmail.com> <4B3B4FE5.8070605@sequans.com> <4dc0cfea0912300509v7f9bb367jf33bb63736407ed@mail.gmail.com> <4dc0cfea0912300512h43a4fdb1oa585a36391de341e@mail.gmail.com> <6944e440-c797-439d-9bf9-b61a05a3e26c@k23g2000yqa.googlegroups.com> <878wck2ojd.fsf@benfinney.id.au> <4dc0cfea0912301927q16445c59r1fb8594ee0f04d24@mail.gmail.com> <034f1009$0$1277$c3e8da3@news.astraweb.com> Message-ID: <4b3f53ec$1@dnews.tpgi.com.au> On 1/2/2010 9:42 PM, Steven D'Aprano wrote: > On Fri, 01 Jan 2010 15:27:57 -0800, myle wrote: > >> Why we should prefer ``if: ...'' over a ``try: ... except something: >> pass'' block? > > We shouldn't, not in general. One exception (pun intended) is if the try-block have a side effect that is difficult to cleanup cleanly. From admin at daffitt.com Sat Jan 2 09:15:26 2010 From: admin at daffitt.com (David M Covey Sr.) Date: Sat, 2 Jan 2010 09:15:26 -0500 Subject: Windows 7 : any problems installing or running Python ? Message-ID: <000001ca8bb6$08391d70$18ab5850$@com> Hello Skippy, In response to your message "Windows 7 : any problems installing or running Python ?" I found posted at (http://mail.python.org/pipermail/python-list/2009-August/1215524.html), I've got to say that I can't seem to get any version of Python to work on my computer. I have a Toshiba Satellite laptop running Windows 7 Home Premium with an AMD Turion II Dual-Core processor. I've tried all of the versions listed at http://python.org/download with no success. They install fine but when I try to run the IDLE (Python GUI), it does nothing at all. Do you have any suggestions that might help me out here? I would really appreciate your input. Thank you, David M Covey Sr. admin at daffitt.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From pengyu.ut at gmail.com Sat Jan 2 10:04:24 2010 From: pengyu.ut at gmail.com (Peng Yu) Date: Sat, 2 Jan 2010 09:04:24 -0600 Subject: Exception as the primary error handling mechanism? In-Reply-To: <7q8r3nF3u6mmdU1@mid.uni-berlin.de> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <50697b2c0912312124n7f989568t13e5a1dd8d883e9c@mail.gmail.com> <7q8r3nF3u6mmdU1@mid.uni-berlin.de> Message-ID: <366c6f341001020704m6811a2d0yeca281dda62d4c3e@mail.gmail.com> On Sat, Jan 2, 2010 at 6:05 AM, Diez B. Roggisch wrote: > Peng Yu schrieb: >> >> On Thu, Dec 31, 2009 at 11:24 PM, Chris Rebert wrote: >>> >>> On Thu, Dec 31, 2009 at 8:47 PM, Peng Yu wrote: >>>> >>>> I observe that python library primarily use exception for error >>>> handling rather than use error code. >>>> >>>> In the article API Design Matters by Michi Henning >>>> >>>> Communications of the ACM >>>> Vol. 52 No. 5, Pages 46-56 >>>> 10.1145/1506409.1506424 >>>> http://cacm.acm.org/magazines/2009/5/24646-api-design-matters/fulltext >>>> >>>> It says "Another popular design flaw?namely, throwing exceptions for >>>> expected outcomes?also causes inefficiencies because catching and >>>> handling exceptions is almost always slower than testing a return >>>> value." >>>> >>>> My observation is contradicted to the above statement by Henning. If >>>> my observation is wrong, please just ignore my question below. >>>> >>>> Otherwise, could some python expert explain to me why exception is >>>> widely used for error handling in python? Is it because the efficiency >>>> is not the primary goal of python? >>> >>> Correct; programmer efficiency is a more important goal for Python >>> instead. >>> Python is ~60-100x slower than C;[1] if someone is worried by the >>> inefficiency caused by exceptions, then they're using completely the >>> wrong language. >> >> Could somebody let me know how the python calls and exceptions are >> dispatched? Is there a reference for it? > > The source? > > http://python.org/ftp/python/2.6.4/Python-2.6.4.tgz > > These are really deep internals that - if they really concern you - need > intensive studies, not casual reading of introductionary documents. IMHO you > shouldn't worry, but then, there's a lot things you seem to care I > wouldn't... :) For my own interest, I want understand the run time behavior of python and what details causes it much slower. Although people choose python for its programming efficiency, but sometimes the runtime still matters. This is an important aspect of the language. I'm wondering this is not even documented. Why everybody has to go to the source code to understand it? Are you sure that there is no document that describes how python is working internally (including exceptions)? From martin at v.loewis.de Sat Jan 2 10:17:53 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Sat, 02 Jan 2010 16:17:53 +0100 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <50697b2c0912312124n7f989568t13e5a1dd8d883e9c@mail.gmail.com> <7q8r3nF3u6mmdU1@mid.uni-berlin.de> Message-ID: <4B3F63A1.1060201@v.loewis.de> > For my own interest, I want understand the run time behavior of python > and what details causes it much slower. Although people choose python > for its programming efficiency, but sometimes the runtime still > matters. This is an important aspect of the language. I'm wondering > this is not even documented. Why everybody has to go to the source > code to understand it? There are two answers to this question: a) Because the source is the most precise and most complete way of documenting it. Any higher-level documentation would necessarily be incomplete. b) Because nobody has contributed documentation. The two causes correlate: because writing documentation of VM internals takes a lot of effort and is of questionable use, nobody has written any. > Are you sure that there is no document that describes how python is > working internally (including exceptions)? Such documents certainly exist, but not as part of the Python distribution. See http://wiki.python.org/moin/CPythonVmInternals for one such document. Regards, Martin From victorsubervi at gmail.com Sat Jan 2 10:26:15 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 2 Jan 2010 10:26:15 -0500 Subject: Append Problem In-Reply-To: References: <4dc0cfea1001020253u6f2bf8a9i7f16c088d4ad78f7@mail.gmail.com> Message-ID: <4dc0cfea1001020726v50be38d4s5096c53d7e4c4152@mail.gmail.com> There isn't a tuple is sight there - you are dealing with lists, and you > are modifying the very list you are iterating over. > > Try rewriting the code to create a new list from the old one (i.e. > iterate over catChains and have your code append to an initially empty > list called, for example, newCatChains, then finally throw the old list > away with > > catChains = newCatChains > > or some such). > Thanks! > > PS (Mainly, I believe, for Carsten): While programming is difficult for > > me, I am writing (and have pretty much finished) this (almost) fully > > automated shopping cart. I don't intend to write any other serious > > programming project. However, a shopping cart is vital to my Web design > > business. I will hereafter outsource my programming. But, as any good > > supervisor knows, one has to know how to supervise! If I can't read > > code, then I'm at the mercy of my programmer(s). If I can't afford to > > hire some top gun who can supervise, then it's my responsibility. Plus, > > if my programmer(s) quit, I need to step in and take over. I'm an ok > > businessman, and absolutely gifted in sales. And in poetry ;) > > beno > > > Given that there are umpteen thousand hosting services who will let you > create a web site with shopping cart included I have to question whether > you are making the best use of your time here. Of course it's your time, > and your decision, so please don't think I am dictating to you. > It doesn't give one the control upon which I insist. No. That's not an option. > > While the learning you have performed in getting this system to work > (for some rather questionable value of "work", I can't help feeling, but > we'll overlook the horrors induced by your lack of programming > experience) is a testament to your persistence (and this list's > tolerance) I can't help feeling that it leaves you no better qualified > to supervise programmers. > Ah, but you aren't a businessman, are you? You wouldn't know... beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at holdenweb.com Sat Jan 2 10:46:43 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 02 Jan 2010 10:46:43 -0500 Subject: Append Problem In-Reply-To: <4dc0cfea1001020726v50be38d4s5096c53d7e4c4152@mail.gmail.com> References: <4dc0cfea1001020253u6f2bf8a9i7f16c088d4ad78f7@mail.gmail.com> <4dc0cfea1001020726v50be38d4s5096c53d7e4c4152@mail.gmail.com> Message-ID: <4B3F6A63.2080509@holdenweb.com> Victor Subervi wrote: [...] > While the learning you have performed in getting this system to work > (for some rather questionable value of "work", I can't help feeling, but > we'll overlook the horrors induced by your lack of programming > experience) is a testament to your persistence (and this list's > tolerance) I can't help feeling that it leaves you no better qualified > to supervise programmers. > > > Ah, but you aren't a businessman, are you? You wouldn't know... > beno See the .sig. Of course I'm a businessman. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From nobody at nowhere.com Sat Jan 2 10:59:14 2010 From: nobody at nowhere.com (Nobody) Date: Sat, 02 Jan 2010 15:59:14 +0000 Subject: Potential Conflicts by Installing Two Versions of Python (Windows)? References: Message-ID: On Fri, 01 Jan 2010 17:37:40 -0800, W. eWatson wrote: > I suspect that if one installs v2.4 and 2.5, or any two versions, that > one will dominate, or there will be a conflict. I suppose it would not > be possible to choose which one should be used. Comments? The only inherent conflict is that you can only associate the .py extension with one version for any particular user. This only matters if you need to be able to run Python files as if they were executables, e.g. by double-clicking them in Explorer. If you invoke Python files by explicitly specifying the Python interpreter, there isn't a problem. From victorsubervi at gmail.com Sat Jan 2 10:59:47 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 2 Jan 2010 10:59:47 -0500 Subject: Append Problem In-Reply-To: <4B3F6A63.2080509@holdenweb.com> References: <4dc0cfea1001020253u6f2bf8a9i7f16c088d4ad78f7@mail.gmail.com> <4dc0cfea1001020726v50be38d4s5096c53d7e4c4152@mail.gmail.com> <4B3F6A63.2080509@holdenweb.com> Message-ID: <4dc0cfea1001020759x475a4dccu2312e6ef1eac80a2@mail.gmail.com> On Sat, Jan 2, 2010 at 10:46 AM, Steve Holden wrote: > > See the .sig. Of course I'm a businessman. > Nice site! In that case, respectfully I disagree! beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From vnewel at invalid.invalid Sat Jan 2 11:27:48 2010 From: vnewel at invalid.invalid (VanceE) Date: Sat, 2 Jan 2010 11:27:48 -0500 Subject: assert type([]) == type(()) Message-ID: Just curious. type([]) == type(()) is False as expected and assert type([]) == type(()) throws an AssertionError as expected. However the following is not an error for x in []: assert type(x) == type(()) I expected an AssertionError but get no errors at all. Any explaination? BTW I'm using Python2.5 and tried type(None) == type(()) which is False as expected. TIA From jrh at joshh.co.uk Sat Jan 2 11:40:14 2010 From: jrh at joshh.co.uk (Josh Holland) Date: Sat, 2 Jan 2010 16:40:14 +0000 (UTC) Subject: assert type([]) == type(()) References: Message-ID: On 2010-01-02, VanceE wrote: > for x in []: > assert type(x) == type(()) > > I expected an AssertionError but get no errors at all. > Any explaination? That loop never runs. It immediately raises a StopIteration and the body is never executed. cf. for x in []: print "In loop body" You won't see anything printed. OTOH, look at this one: for x in [[]]: # a list containing an empty list assert type(x) == type(()) That will raise the AssertionError as expected. -- Josh "dutchie" Holland http://joshh.co.uk http://twitter.com/jshholland http://identi.ca/jshholland From wojciech_mula at poczta.null.onet.pl.invalid Sat Jan 2 11:41:20 2010 From: wojciech_mula at poczta.null.onet.pl.invalid (Wojciech =?ISO-8859-2?Q?Mu=B3a?=) Date: Sat, 2 Jan 2010 17:41:20 +0100 Subject: assert type([]) == type(()) References: Message-ID: <20100102174120.050865c9.wojciech_mula@poczta.null.onet.pl.invalid> VanceE wrote: > for x in []: > assert type(x) == type(()) > > I expected an AssertionError but get no errors at all. > Any explaination? [] is an empty sequence, so your loop executes exactly 0 times. :) for x in [None]: assert ... w. From python.list at tim.thechases.com Sat Jan 2 11:45:16 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Sat, 02 Jan 2010 10:45:16 -0600 Subject: assert type([]) == type(()) In-Reply-To: References: Message-ID: <4B3F781C.4040602@tim.thechases.com> > However the following is not an error > > for x in []: > assert type(x) == type(()) > > I expected an AssertionError but get no errors at all. > Any explaination? number_of_times_through_the_loop = 0 for x in []: assert type(x) == type(()) number_of_times_through_the_loop += 1 print number_of_times_through_the_loop make sense? :) -tkc From zuo at chopin.edu.pl Sat Jan 2 11:51:58 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sat, 02 Jan 2010 17:51:58 +0100 Subject: assert type([]) == type(()) In-Reply-To: References: Message-ID: > However the following is not an error > > for x in []: > assert type(x) == type(()) Trying to iterate over an empty sequence or iterator causes 0 (zero) steps of iteration -- so above assert statement is never run. Cheers, *j -- Jan Kaliszewski (zuo) From aahz at pythoncraft.com Sat Jan 2 12:40:44 2010 From: aahz at pythoncraft.com (Aahz) Date: 2 Jan 2010 09:40:44 -0800 Subject: Bare Excepts References: <4dc0cfea0912300209x15e5bc96nd6e6224d89df0638@mail.gmail.com> Message-ID: In article <034f1009$0$1277$c3e8da3 at news.astraweb.com>, Steven D'Aprano wrote: > >The try version is also better whenever there could be a race condition. >For example, when opening a file, you might be tempted to do this: > >if os.path.exists(filename): > f = open(filename, 'r') > text = f.read() >else: > print "file missing" > >but that is dangerous in a multi-processing computer system (which nearly >all computers are these days). The problem is that in the fraction of a >second after you test that the file exists, some other process might come >along and delete it and the open call will then fail. So to be safe, you >have to do this: > >if os.path.exists(filename): > try: > f = open(filename, 'r') > except IOError: > print "file missing" > else: > text = f.read() >else: > print "file missing" > >But of course now the test is redundant, and all you really need is the >try...except block: > >try: > f = open(filename, 'r') >except IOError: > print "file missing" >else: > text = f.read() OTOH, if you want to do something different depending on whether the file exists, you need to use both approaches: if os.path.exists(fname): try: f = open(fname, 'rb') data = f.read() f.close() return data except IOError: logger.error("Can't read: %s", fname) return '' else: try: f = open(fname, 'wb') f.write(data) f.close() except IOError: logger.error("Can't write: %s", fname) return None (This is a somewhat stupid example strictly for illustration. A better and more-elaborate example would be something like trying to copy a file to fname and rename an existing file to '.bak' if it exists. The tricky part would be trying to rename the '.bak' to fname if the copy fails. And yes, that's exactly what some code I wrote a few days ago does.) -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From mike at clove.com Sat Jan 2 13:21:11 2010 From: mike at clove.com (Mike Howard) Date: Sat, 02 Jan 2010 11:21:11 -0700 Subject: Python Concurrency Workshop, Jan 14-15, 2010 In-Reply-To: <12C497C8-47D0-4DFA-B8FD-5FDB18D5A951@dabeaz.com> References: <12C497C8-47D0-4DFA-B8FD-5FDB18D5A951@dabeaz.com> Message-ID: <4B3F8E97.90508@clove.com> Hi Dave, Chicago in January? How about moving it to Denver - it's a nice town and I live close by. Mike David Beazley wrote: > Python Concurrency Workshop, v2.0 > January 14-15, 2010 > Chicago, Illinois > http://www.dabeaz.com/chicago/concurrent.html > > *** Last Two Weeks to Register *** > > Join David Beazley, author of the Python Essential Reference, for an > in-depth workshop on concurrent programming techniques and > idioms. This workshop, designed for more experienced Python > programmers, covers threads, synchronization, message passing, > multiprocessing, distributed computing, coroutines, asynchronous I/O > and other related topics with an eye towards writing programs that can > run on multiple CPU cores, clusters, or distributed systems. A major > theme of the workshop is to explore and understand different > programming techniques, their associated performance properties, and > other tradeoffs. You'll definitely walk away with new insight and a > better understanding of how different parts of Python work under the > covers. > > Workshop attendance is strictly limited to six people. More information, > including a detailed topic index, is available at: > > http://www.dabeaz.com/chicago/concurrent.html > > Hopefully I'll see you in a few weeks! > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: mike.vcf Type: text/x-vcard Size: 242 bytes Desc: not available URL: From mackrackit at gmail.com Sat Jan 2 13:47:24 2010 From: mackrackit at gmail.com (Dave McCormick) Date: Sat, 02 Jan 2010 11:47:24 -0700 Subject: Dynamic text color In-Reply-To: References: Message-ID: <4B3F94BC.4030904@gmail.com> WooHoo!!! I got it!!! Yup, I am sure it can be optimized but it works!!! John, I have to admit that I spent several hours working on this before I looked at your example, then I spent another several hours getting this far. Would never have gotten it with out you help. Thanks!!! Also reading the thread "Significant whitespace" helped. I had an indent problem in a FOR loop. This is a whole lot different than the MCU programming I do in ASM or BASIC, starting to make sense though. Thanks again for all the help and putting up with folks like me, Dave Here is what I can up with. ####### from Tkinter import * import re redList = "red dog".split() blueList = "blue ball".split() greenList = "green grass".split() def get_position(event): complete = Tbox.get("1.0", END) ####RED######## for word in redList: new_Rword(complete, word) def new_Rword(complete, word): Tbox.tag_remove(word, "1.0", END) for matchobj in re.finditer(word, complete): start,end = matchobj.span() Tbox.tag_add("red", "1.0 + %d chars" % start,"1.0 + %d chars" % end) Tbox.tag_config("red", foreground="red") ####BLUE####### for word in blueList: new_Bword(complete, word) def new_Bword(complete, word): Tbox.tag_remove(word, "1.0", END) for matchobj in re.finditer(word, complete): start,end = matchobj.span() Tbox.tag_add("blue", "1.0 + %d chars" % start,"1.0 + %d chars" % end) Tbox.tag_config("blue", foreground="blue") ####GREEN####### for word in greenList: new_Gword(complete, word) def new_Gword(complete, word): Tbox.tag_remove(word, "1.0", END) for matchobj in re.finditer(word, complete): start,end = matchobj.span() Tbox.tag_add("green", "1.0 + %d chars" % start,"1.0 + %d chars" % end) Tbox.tag_config("green", foreground="green") root = Tk() Tbox = Text(root, width=40, height=15, wrap=CHAR, font="Times 14 bold", bg="#dddddd") Tbox.pack() Tbox.bind("", get_position) Tbox.focus() root.mainloop() John Posner wrote: > On Thu, 31 Dec 2009 10:24:44 -0500, Dave McCormick > wrote: > >> John, >> >> Thank you for the tips. >> I was changing the line-column index to a FLOAT because the search >> would return the starting position (pos) of the string, then by >> making it a FLOAT and adding the string length I was able to get the >> end position. >> If "red" was on line 1 column 0.. >> Tbox.tag_add("red", pos, float(pos)+.03) >> = >> Tbox.tag_add("red", 1.0, 1.3) >> It was all I could come up with. > > Yup, Dave, I've dug this kind of hole for myself many times! > >> >> You have convinced me about the re.finditer for this, I think... >> Still in the prototyping mode: >> >> def get_position(event): >> pos = Tbox.get(1.0, END) >> match = [ matchobj.span() for matchobj in >> re.finditer("red", pos) ] >> print "match ",match #debug to shell > > Notes: > > * Variable "pos" should be "text" or "complete_text" or something > similar. > > * The first argument to the get() function must be a string: > > wrong ... complete_text = Tbox.get(1.0, END) > right ... complete_text = Tbox.get("1.0", END) > > But there's a more important problem. Is this function supposed to > handle *one* word to be colored red, or *all the words* to be colored > red? Here's what you want to do on each user keystroke: > > 1. Use get() to place the entire contents of the Text widget in a > variable, say "complete_text". > > 2. Use re.finditer() to generate START,END pairs for the > substrings to > be colored red. You might find it easier to create a list from the > iterator, though it isn't really necessary: > > start_end_pairs = list(re.finditer("red", complete_text)) > > 3. Loop over the START,END pairs in this list. In each loop, use > tag_add() to tag one of the substrings. > > [OOPS: I apologize if my suggestion in the previous post misled you. I > described the output of finditer() as "a list of (start,end) pairs for > an invocation of Text.tag_add()". I should have said "a list of > (start,end) pairs, *WHICH CAN BE LOOPED OVER, FOR A SERIES OF > INVOCATIONS* of Text.tag_add()".] > > Note that the 3 steps above only handle the color red. So you want to > place these steps in a function, then call the function for each color: > > insert_color_markup(text, "red") > insert_color_markup(text, "green") > insert_color_markup(text, "blue") > > For each call, pass in the contents of complete_text as the first > argument. > > So the function-call hierarchy would be: > > get_position() <--- invoked on each keystroke > insert_color_markup() <--- called once for each color > get() then finditer() then loop-driven calls to tag_add() > > I hope that makes sense to you! > > >> Gives all of START,END pairs just fine. It is the last hint about >> line-column indexes that I am have problems with. All of the >> documentation I can find about "text.tag_add()" uses line-column for >> coordinates. > > Lie Ryan has already pointed you to clarifying documentation. Be sure > to bookmark http://infohost.nmt.edu/tcc/help/pubs/tkinter/ in your Web > browser! > >> If I count characters from the beginning how do I know what line the >> text is on? Would you mind making your last hint a bit stronger... > > The point is that *you don't need to*. Handling the text as a simple > sequence of characters is much simpler than worrying about line/column > pairs. (If you were using a different shade of red on different lines, > you *would* need to worry about line/column pairs.) > > One more suggestion, which I forgot to make in the previous round. > Your code includes this: > > Tbox.grid(column=0, row=0, sticky=(N+W+E+S)) > root.grid_columnconfigure(0, weight=1) > root.grid_rowconfigure(0, weight=1) > > You're working too hard here. This is sufficient: > > Tbox.pack() > > Keep at it, Dave. I've posted a complete solution at > http://cl1p.net/jjp_dynamic_text_color/. But I suggest that you put in > a couple of more hours of coding before peeking at it. > > Best, > John From mwilson at the-wire.com Sat Jan 2 14:00:18 2010 From: mwilson at the-wire.com (Mel) Date: Sat, 02 Jan 2010 14:00:18 -0500 Subject: assert type([]) == type(()) References: Message-ID: VanceE wrote: > Just curious. > > type([]) == type(()) > is False as expected > > and > > assert type([]) == type(()) > throws an AssertionError as expected. > > However the following is not an error > > for x in []: > assert type(x) == type(()) > > I expected an AssertionError but get no errors at all. > Any explaination? Others have said what happens. Bit of a mind bender -- here's the smallest change to do what you expected: Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41) [GCC 4.3.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> for x in [],: ... assert type(x) == type (()) ... Traceback (most recent call last): File "", line 2, in AssertionError >>> Because of the comma, this creates a tuple containing the empty list, and iterates over the tuple. Mel. From vnewel at invalid.invalid Sat Jan 2 15:06:06 2010 From: vnewel at invalid.invalid (VanceE) Date: Sat, 2 Jan 2010 15:06:06 -0500 Subject: Thx (Was: assert type([]) == type(()) ) References: Message-ID: A big Thank You to all for the answer (and the hints). That sure explains a lot. Again, Thank you very much. Cheers, Vance From martin at v.loewis.de Sat Jan 2 15:17:59 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Sat, 02 Jan 2010 21:17:59 +0100 Subject: Dangerous behavior of list(generator) In-Reply-To: References: <7p356kF750U1@mid.individual.net> Message-ID: <4B3FA9F7.2080304@v.loewis.de> >> Bottom line, I'm going to have to remove this pattern from my code: >> >> foo = (foo for foo in foos if foo.bar).next() I recommend to rewrite this like so: def first(gen): try: return gen.next() except StopIteration: raise ValueError, "No first value" foo = first(foo for foo in foos if foo.bar) As others have said: don't let StopIteration appear unexpectedly; IOW, consume generators right away in a loop construct (where this first function is a loop construct as well). A different way of writing it would be def first(gen): for value in gen: return value raise ValueError, "empty collection" Regards, Martin From martin at v.loewis.de Sat Jan 2 15:27:54 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Sat, 02 Jan 2010 21:27:54 +0100 Subject: Dangerous behavior of list(generator) In-Reply-To: References: <20091213143521.2549.845539319.divmod.xquotient.731@localhost.localdomain> <4B2652AB.7080501@egenix.com> Message-ID: <4B3FAC4A.4090306@v.loewis.de> > I'm asking about why the behavior of a StopIteration exception being > handled from the `expression` of a generator expression to mean "stop > the loop" is accepted by "the devs" as acceptable. I may be late to this discussion, but the answer is "most definitely yes". *Any* exception leads to termination of the iterator, and StopIteration is no different: py> def stop(e): ... def f(): ... raise e ... return f ... py> g = (f() for f in (lambda:1,stop(StopIteration),lambda:2)) py> g.next py> g.next() 1 py> g.next() Traceback (most recent call last): File "", line 1, in File "", line 1, in File "", line 3, in f StopIteration py> g.next() Traceback (most recent call last): File "", line 1, in StopIteration py> g = (f() for f in (lambda:1,stop(ValueError),lambda:2)) py> g.next() 1 py> g.next() Traceback (most recent call last): File "", line 1, in File "", line 1, in File "", line 3, in f ValueError py> g.next() Traceback (most recent call last): File "", line 1, in StopIteration Regards, Martin From pydecker at gmail.com Sat Jan 2 16:04:03 2010 From: pydecker at gmail.com (Peter Decker) Date: Sat, 2 Jan 2010 16:04:03 -0500 Subject: whoops: create a splash window in python In-Reply-To: <4B3C374C.6010307@depauw.edu> References: <4b3c246d$1@dnews.tpgi.com.au> <4B3C3584.8070500@depauw.edu> <4B3C374C.6010307@depauw.edu> Message-ID: On Thu, Dec 31, 2009 at 12:31 AM, Ron Croonenberg wrote: > is there a way, in python, to create a splash window and when the program > has completed disappears by sending a msg to it? (I tried creating two gtk > windows but gtk_main doesn't seem to return unless it gets closed.) It's really simple to do this in Dabo (which uses wxPython under the hood, but is _so_ much more elegant!). When you create your app, add the following two parameters: app = dabo.dApp(showSplashScreen=True, splashImage="/path/to/splash.png") That's it! -- # p.d. From tjreedy at udel.edu Sat Jan 2 16:48:35 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 02 Jan 2010 16:48:35 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <366c6f341001020704m6811a2d0yeca281dda62d4c3e@mail.gmail.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <50697b2c0912312124n7f989568t13e5a1dd8d883e9c@mail.gmail.com> <7q8r3nF3u6mmdU1@mid.uni-berlin.de> <366c6f341001020704m6811a2d0yeca281dda62d4c3e@mail.gmail.com> Message-ID: On 1/2/2010 10:04 AM, Peng Yu wrote: > For my own interest, I want understand the run time behavior of python That depends on the implementation. > and what details causes it much slower. A language feature that slows all implementation is the dynamic name/slot binding and resolution. Any implementation can be made faster by restricting the dynamism (which makes the imlementaion one of a subset of Python). > Although people choose python > for its programming efficiency, but sometimes the runtime still > matters. There is no 'the' runtime. Whether or not there even *is* a runtime, as usually understoold, is a matter of the implementation. > This is an important aspect of the language. It is an aspect of each implementation, of which there are now more than one. Terry Jan Reedy From Dr.Alain.Picard at gmail.com Sat Jan 2 16:48:57 2010 From: Dr.Alain.Picard at gmail.com (Alain Picard) Date: Sun, 03 Jan 2010 08:48:57 +1100 Subject: Xah's Edu Corner: Teach Ourself Programing In Ten Years? References: <1035ce5d-914e-42c0-8e9c-a349a6d55e55@v25g2000yqk.googlegroups.com> Message-ID: <87my0w89fa.fsf@gmail.com> [Aplogies about the wild cross-post follow-up --- I guess the topic really is relevant to most programming communities.] Xah Lee writes: > To see this in a different context, suppose you need to pass a > important Math XYZ exam or review in your career or get a certificate, > but you don't remember your Math XYZ. There's a difference between cramming to remember XYZ, and never having understood (or even been exposed to!) XYZ previously. The former is reasonable; the latter bound to fail. > [SNIP] In my opinion, a solution to this is by installing > the concept of responsible licenses. Please see this essay Responsible > Software Licensing IOW, you think IT will have to become like engineering and "grow up". I.e. when your manager says "do XYZ in 7 days", you just laugh at them, and your risk of being fired are minimal, because all other certified engineers who could replace you also will laugh at him because your duty of care to your profession and certification is greater than the one you owe your bozo manager. I'm, personally, not holding my breath. If you want to change the world, you start by changing yourself. Cheers, --ap From john at castleamber.com Sat Jan 2 17:15:52 2010 From: john at castleamber.com (John Bokma) Date: Sat, 02 Jan 2010 16:15:52 -0600 Subject: Xah's Edu Corner: Teach Ourself Programing In Ten Years? References: <1035ce5d-914e-42c0-8e9c-a349a6d55e55@v25g2000yqk.googlegroups.com> <87my0w89fa.fsf@gmail.com> Message-ID: <87y6kgrw4n.fsf@castleamber.com> Alain Picard writes: > If you want to change the world, you start by changing yourself. Like for starters setting a follow-up to header, especially if you spam 4 groups. But Xah is Xah. -- John Bokma Read my blog: http://johnbokma.com/ Hire me (Perl/Python): http://castleamber.com/ From emile at fenx.com Sat Jan 2 17:19:17 2010 From: emile at fenx.com (Emile van Sebille) Date: Sat, 02 Jan 2010 14:19:17 -0800 Subject: Significant whitespace In-Reply-To: <034e88f2$0$1277$c3e8da3@news.astraweb.com> References: <4B3E70FC.1010702@gmail.com> <034e88f2$0$1277$c3e8da3@news.astraweb.com> Message-ID: On 1/1/2010 5:05 PM Steven D'Aprano said... > In Python terms, imagine if we could write > > foriinrange(10): > > instead of the usual > > for i in range(10): > > Since the colon makes it unambiguous that it is some sort of block > construct, and it starts with "for", it must be a for loop. Pretty > horrible, yes? Yes -- early on I got stung by something similar. My first programming job (1977?) was working in a basic dialect called Buisness Basic Level II (BBII). To delete a record in the file you used a remove statement. White space on the line was optional, allowing you to write lines like '100 remove(1,key=k$)' and '100 fori=1to10'. REM was how a remark statement began, which allowed statements like "100 REM comment". I was sent on-site to a customer running the prior version (BBI). I didn't realize the REMOVE command changed between versions from REKEY, and it was only after a more experienced programmer co-wroker pointed out that my REMOVE statement was being interpreted as a REM that I finally found the bug I'd created. Manuals and documentation, while often lacking, are as often underrated. Emile From emile at fenx.com Sat Jan 2 17:42:35 2010 From: emile at fenx.com (Emile van Sebille) Date: Sat, 02 Jan 2010 14:42:35 -0800 Subject: Xah's Edu Corner: Teach Ourself Programing In Ten Years? In-Reply-To: <1035ce5d-914e-42c0-8e9c-a349a6d55e55@v25g2000yqk.googlegroups.com> References: <1035ce5d-914e-42c0-8e9c-a349a6d55e55@v25g2000yqk.googlegroups.com> Message-ID: On 1/2/2010 1:14 AM Xah Lee said... > These books are the bedrock of the industry. It is not because people > are impatient, or that they wish to hurry, but rather, it is the > condition of the IT industry, in the same way modern society drives > people to live certain life styles. Turing complete. Once you've learned to program, learning dialects is a different study. I hated Word Perfect early on, but if I were faced with it today I expect that everything I know about word processing would kelp to deploy it effectively with minimal in-depth knowledge of it's peculiarities. Likewise, I don't work in perl, but I don't hesitate to deploy and modify perl projects to my needs, and mostly having only brushed up on the semantics that need to change. > No amount of patience or > proselytization can right this, except that we change the industry's > practice of quickly churning out bug-ridden software products to beat > competitors. Companies do that due to market forces, and the market > forces is a result of how people and organizations actually choose to > purchase software. In my opinion, a solution to this is by installing > the concept of responsible licenses. Please see this essay Responsible > Software Licensing and spread the word. Licensing programmers isn't likely to turn Microsoft around. Enforcement doesn't carry much weight with them. Let's work to strengthen that, then there's a functioning mechanism in place to work with. Not-holding-my-breath-ly y'rs, Emile From iwaki at iwakihidekazu.net Sat Jan 2 17:55:54 2010 From: iwaki at iwakihidekazu.net (Hidekazu IWAKI) Date: Sun, 03 Jan 2010 07:55:54 +0900 Subject: Where is "urllib2" module in windows python3.1.1? In-Reply-To: References: Message-ID: <243620100102225554iwaki@iwakihidekazu.net> Hi. Thank you for the answers. Oh, sorry. It was one of the changes from .2.x to 3.x. I didn't know. There are really important and a lot of changes. Thank you! At 01 Jan 2010 13:57:03 +0000 Duncan Booth wrote: > Hidekazu IWAKI wrote: > > > Hi; > > I'd like to import "urllib2" in windows python 3.1.1, but I'm not able to > > do it. > > Python 3 doesn't have a urllib2 module; use the urllib package instead. > > See http://www.python.org/dev/peps/pep-3108/#urllib-package > -- > http://mail.python.org/mailman/listinfo/python-list From bblais at bryant.edu Sat Jan 2 18:17:52 2010 From: bblais at bryant.edu (Brian Blais) Date: Sat, 02 Jan 2010 18:17:52 -0500 Subject: Python Concurrency Workshop, Jan 14-15, 2010 In-Reply-To: <4B3F8E97.90508@clove.com> References: <12C497C8-47D0-4DFA-B8FD-5FDB18D5A951@dabeaz.com> <4B3F8E97.90508@clove.com> Message-ID: <22DCF99A-C64E-46E1-8AA7-C8DF383153E2@bryant.edu> On Jan 2, 2010, at 13:21 , Mike Howard wrote: > Hi Dave, > > Chicago in January? > > How about moving it to Denver - it's a nice town and I live close by. > > Mike > > David Beazley wrote: >> Python Concurrency Workshop, v2.0 I think, in the spirit of the topic, they should hold it at both places at the same time. bb -- Brian Blais bblais at bryant.edu http://web.bryant.edu/~bblais http://bblais.blogspot.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From fpm at u.washington.edu Sat Jan 2 18:21:45 2010 From: fpm at u.washington.edu (cassiope) Date: Sat, 2 Jan 2010 15:21:45 -0800 (PST) Subject: IOError - cannot create file (linux daemon-invoked script) Message-ID: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> I have a daemon on a Linux system that supports a number of Windows clients. Among the functions is to send e-mails, which is sufficiently complicated that I fork() a separate process which gets setuid to a lesser user, and calls a python script which does the actual formatting and emailing (the daemon is written in C). I want to save a copy of the email in a particular directory which is accessible to the Windows clients via samba. The strange thing is that even with the right user-id, I cannot seem to write to the directory, getting an IOError exception. Changing the directory to world-writable fixes this. I can confirm the uid and gid for the script by having the script print these values just before trying to create/write the file. Becoming the same lesser user, I have no problem writing a file to the same directory. Is there anything that I can do to diagnose why this script is failing? For various reasons I don't want to make the directory world- writable. This is on a Debian "squeeze" system, with python 2.5. Thanks for any insights! From steve at holdenweb.com Sat Jan 2 18:46:22 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 02 Jan 2010 18:46:22 -0500 Subject: IOError - cannot create file (linux daemon-invoked script) In-Reply-To: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> References: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> Message-ID: cassiope wrote: > I have a daemon on a Linux system that supports a number of Windows > clients. Among the functions is to send e-mails, which is > sufficiently complicated that I fork() a separate process which gets > setuid to a lesser user, and calls a python script which does the > actual formatting and emailing (the daemon is written in C). I want > to save a copy of the email in a particular directory which is > accessible to the Windows clients via samba. > > The strange thing is that even with the right user-id, I cannot seem > to write to the directory, getting an IOError exception. Changing the > directory to world-writable fixes this. I can confirm the uid and gid > for the script by having the script print these values just before > trying to create/write the file. Becoming the same lesser user, I > have no problem writing a file to the same directory. > Have you looked at the IOError's errno attribute to find out exactly why the Python subprocess is unable to write to the directory? > Is there anything that I can do to diagnose why this script is > failing? For various reasons I don't want to make the directory world- > writable. > I'd concur on that decision. > This is on a Debian "squeeze" system, with python 2.5. > > Thanks for any insights! Take a closer look at the exception, that might stimulate a thought or two. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From llcawthorne at archlinux.us Sat Jan 2 18:47:37 2010 From: llcawthorne at archlinux.us (Lewis Cawthorne) Date: Sat, 02 Jan 2010 18:47:37 -0500 Subject: Windows 7 : any problems installing or running Python ? In-Reply-To: <000001ca8bb6$08391d70$18ab5850$@com> References: <000001ca8bb6$08391d70$18ab5850$@com> Message-ID: <4B3FDB19.5080405@archlinux.us> I'm not Skippy, but I do have my wife's Toshiba Satellite running Windows 7 Home Premium laying around. I figured that since I am fairly well set to duplicate all the information you gave in your problem, I would take a look. The bad news is that Python works great on it, so your problem needs more troubleshooting.. I have IDLE open right now with no issues. Python version: 3.1.1, Idle version: 3.1.1, Tk version: 8.5. Have you tried launching Python from a command line in the directory that you installed it to perhaps? Maybe it would spit out an error message (or the interpreter might just work and maybe you have a Tk problem... who knows). Lewis Cawthorne On 01/02/2010 09:15 AM, David M Covey Sr. wrote: > > Hello Skippy, > > In response to your message "Windows 7 : any problems installing or > running Python ?" I found posted at > (http://mail.python.org/pipermail/python-list/2009-August/1215524.html), > I've got to say that I can't seem to get any version of Python to work > on my computer. I have a Toshiba Satellite laptop running Windows 7 > Home Premium with an AMD Turion II Dual-Core processor. I've tried all > of the versions listed at http://python.org/download with no success. > They install fine but when I try to run the IDLE (Python GUI), it does > nothing at all. > > Do you have any suggestions that might help me out here? I would > really appreciate your input. > > Thank you, > > David M Covey Sr. > > admin at daffitt.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sysadmin at example.com Sat Jan 2 19:01:50 2010 From: sysadmin at example.com (Wanna-Be Sys Admin) Date: Sat, 02 Jan 2010 16:01:50 -0800 Subject: Xah's Edu Corner: Teach Ourself Programing In Ten Years? References: <1035ce5d-914e-42c0-8e9c-a349a6d55e55@v25g2000yqk.googlegroups.com> <87my0w89fa.fsf@gmail.com> Message-ID: Alain Picard wrote: > [Aplogies about the wild cross-post follow-up --- I guess the topic > [really > is?relevant?to?most?programming?communities.] No, it's not relevant. Xah Lee is a self spammer, in that he spams about himself, tries to get people hyped up about him and thinks he's impressing people (he's not). He continually cross posts to the language groups he thinks he knows about (often to Perl, where there's no mention of Perl, other than he's used it once or so). He knows very little about any of these languages, but likes to talk a whole lot, as in talking up about himself and how he thinks himself a genius. If you could please not cross post his self-spamming replies to the Python and Perl groups, at least, I'd appreciate it. Most people know about him and have filtered his posts by now. Thanks. -- Not really a wanna-be, but I don't know everything. From steve at REMOVE-THIS-cybersource.com.au Sat Jan 2 19:30:03 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 03 Jan 2010 00:30:03 GMT Subject: Bare Excepts References: <4dc0cfea0912300209x15e5bc96nd6e6224d89df0638@mail.gmail.com> <034f1009$0$1277$c3e8da3@news.astraweb.com> Message-ID: <034fd208$0$1277$c3e8da3@news.astraweb.com> On Sat, 02 Jan 2010 09:40:44 -0800, Aahz wrote: > OTOH, if you want to do something different depending on whether the > file exists, you need to use both approaches: > > if os.path.exists(fname): > try: > f = open(fname, 'rb') > data = f.read() > f.close() > return data > except IOError: > logger.error("Can't read: %s", fname) return '' > else: > try: > f = open(fname, 'wb') > f.write(data) > f.close() > except IOError: > logger.error("Can't write: %s", fname) > return None Unfortunately, this is still vulnerable to the same sort of race condition I spoke about. Even more unfortunately, I don't know that there is any fool-proof way of avoiding such race conditions in general. Particularly the problem of "open this file for writing only if it doesn't already exist". > (This is a somewhat stupid example strictly for illustration. A better > and more-elaborate example would be something like trying to copy a file > to fname and rename an existing file to '.bak' if it exists. The tricky > part would be trying to rename the '.bak' to fname if the copy fails. > And yes, that's exactly what some code I wrote a few days ago does.) Sounds interesting and useful. Would you care to share it with us, or to publish it as a recipe? -- Steven From steve at REMOVE-THIS-cybersource.com.au Sat Jan 2 19:30:17 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 03 Jan 2010 00:30:17 GMT Subject: Bare Excepts References: <4dc0cfea0912300209x15e5bc96nd6e6224d89df0638@mail.gmail.com> <4B3B4FE5.8070605@sequans.com> <4dc0cfea0912300509v7f9bb367jf33bb63736407ed@mail.gmail.com> <4dc0cfea0912300512h43a4fdb1oa585a36391de341e@mail.gmail.com> <6944e440-c797-439d-9bf9-b61a05a3e26c@k23g2000yqa.googlegroups.com> <878wck2ojd.fsf@benfinney.id.au> <4dc0cfea0912301927q16445c59r1fb8594ee0f04d24@mail.gmail.com> <034f1009$0$1277$c3e8da3@news.astraweb.com> <4b3f53ec$1@dnews.tpgi.com.au> Message-ID: <034fd216$0$1277$c3e8da3@news.astraweb.com> On Sun, 03 Jan 2010 01:10:51 +1100, Lie Ryan wrote: > On 1/2/2010 9:42 PM, Steven D'Aprano wrote: >> On Fri, 01 Jan 2010 15:27:57 -0800, myle wrote: >> >>> Why we should prefer ``if: ...'' over a ``try: ... except something: >>> pass'' block? >> >> We shouldn't, not in general. > > One exception (pun intended) is if the try-block have a side effect that > is difficult to cleanup cleanly. What if the "if ..." test has a side-effect? This is an argument against side-effects, not against using exceptions. But of course you are right, there are lots of good reasons for choosing to use an if test rather than a try block, or vice versa. I would argue that the try block is generally more Pythonic, and should be preferred, but that doesn't mean that there aren't reasons for sometimes choosing other strategies instead. -- Steven From jjposner at optimum.net Sat Jan 2 20:38:58 2010 From: jjposner at optimum.net (John Posner) Date: Sat, 02 Jan 2010 20:38:58 -0500 (EST) Subject: Dynamic text color Message-ID: <10564900.973857.1262482738487.JavaMail.jjposner@mail.srv.hcvlny.cv.net> On Sat, Jan 2, 2010 at 1:47 PM, Dave McCormick wrote: > WooHoo!!! > I got it!!! Yup, I am sure it can be optimized but it works!!! Hmmm ... it doesn't work for me ... ####RED######## > for word in redList: > new_Rword(complete, word) def new_Rword(complete, word): > Tbox.tag_remove(word, "1.0", END) > for matchobj in re.finditer(word, complete): > start,end = matchobj.span() Tbox.tag_add("red", > "1.0 + %d chars" % start,"1.0 + %d chars" % end) > Tbox.tag_config("red", foreground="red") How *could* this work, Dave, since you call function new_Rword() before you define it? I rearranged the statements in your program, and it now works for me: #--------------------------------------------- from Tkinter import * import re redList = "red dog".split() blueList = "blue ball".split() greenList = "green grass".split() def get_position(event): complete = Tbox.get("1.0", END) for word in redList: new_Rword(complete, word) for word in blueList: new_Bword(complete, word) for word in greenList: new_Gword(complete, word) def new_Rword(complete, word): Tbox.tag_remove(word, "1.0", END) for matchobj in re.finditer(word, complete): start,end = matchobj.span() Tbox.tag_add("red", "1.0 + %d chars" % start,"1.0 + %d chars" % end) def new_Bword(complete, word): Tbox.tag_remove(word, "1.0", END) for matchobj in re.finditer(word, complete): start,end = matchobj.span() Tbox.tag_add("blue", "1.0 + %d chars" % start,"1.0 + %d chars" % end) def new_Gword(complete, word): Tbox.tag_remove(word, "1.0", END) for matchobj in re.finditer(word, complete): start,end = matchobj.span() Tbox.tag_add("green", "1.0 + %d chars" % start,"1.0 + %d chars" % end) root = Tk() Tbox = Text(root, width=40, height=15, wrap=CHAR, font="Times 14 bold", bg="#dddddd") Tbox.tag_config("red", foreground="red") Tbox.tag_config("blue", foreground="blue") Tbox.tag_config("green", foreground="green") Tbox.pack() Tbox.bind("", get_position) Tbox.focus() root.mainloop() #--------------------------------------------- This version also has the advantage of defining each function just once, instead of multiple times on each keystroke! Still to-do: * rename get_position() to get_complete_text() * replace new_Rword(), new_Bword(), and new_Gword() with a single function that has an extra parameter, "color". Keep at it! -John From jjposner at optimum.net Sat Jan 2 20:52:33 2010 From: jjposner at optimum.net (John Posner) Date: Sat, 02 Jan 2010 20:52:33 -0500 (EST) Subject: Dynamic text color Message-ID: <21336368.974257.1262483554337.JavaMail.jjposner@mail.srv.hcvlny.cv.net> On Sat, Jan 2, 2010 at 1:47 PM, Dave McCormick wrote: > WooHoo!!! > I got it!!! Yup, I am sure it can be optimized but it works!!! Dave, please ignore a couple of my bogus complaints in the previous message: ... you call function new_Rword() before you define it ... this version also has the advantage of defining each function just once, instead of multiple times on each keystroke But I stand by my overall statement that the program didn't work for me, and that rearranging the lines produces a working program. -John From mensanator at aol.com Sat Jan 2 20:52:55 2010 From: mensanator at aol.com (Mensanator) Date: Sat, 2 Jan 2010 17:52:55 -0800 (PST) Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> <034e88f2$0$1277$c3e8da3@news.astraweb.com> Message-ID: <4c436757-cd85-4dea-ab5e-df00f826b9bd@r24g2000yqd.googlegroups.com> On Jan 2, 4:19?pm, Emile van Sebille wrote: > On 1/1/2010 5:05 PM Steven D'Aprano said... > > > In Python terms, imagine if we could write > > > ? ? ?foriinrange(10): > > > instead of the usual > > > ? ? ?for i in range(10): > > > Since the colon makes it unambiguous that it is some sort of block > > construct, and it starts with "for", it must be a for loop. Pretty > > horrible, yes? > > Yes -- early on I got stung by something similar. ?My first programming > job (1977?) was working in a basic dialect called Buisness Basic Level > II (BBII). ?To delete a record in the file you used a remove statement. > ? White space on the line was optional, allowing you to write lines like > '100 remove(1,key=k$)' and '100 fori=1to10'. ?REM was how a remark > statement began, which allowed statements like "100 REM comment". ?I was > sent on-site to a customer running the prior version (BBI). ?I didn't > realize the REMOVE command changed between versions from REKEY, and it > was only after a more experienced programmer co-wroker pointed out that > my REMOVE statement was being interpreted as a REM that I finally found > the bug I'd created. Better than the other way 'round, eh? > > Manuals and documentation, while often lacking, are as often underrated. > > Emile From robert.kern at gmail.com Sat Jan 2 21:27:09 2010 From: robert.kern at gmail.com (Robert Kern) Date: Sat, 02 Jan 2010 20:27:09 -0600 Subject: Python Concurrency Workshop, Jan 14-15, 2010 In-Reply-To: <22DCF99A-C64E-46E1-8AA7-C8DF383153E2@bryant.edu> References: <12C497C8-47D0-4DFA-B8FD-5FDB18D5A951@dabeaz.com> <4B3F8E97.90508@clove.com> <22DCF99A-C64E-46E1-8AA7-C8DF383153E2@bryant.edu> Message-ID: On 2010-01-02 17:17 , Brian Blais wrote: > On Jan 2, 2010, at 13:21 , Mike Howard wrote: > >> Hi Dave, >> >> Chicago in January? >> >> How about moving it to Denver - it's a nice town and I live close by. >> >> Mike >> >> David Beazley wrote: >>> Python Concurrency Workshop, v2.0 > > I think, in the spirit of the topic, they should hold it at both places > at the same time. .... Well played. +1 QOTW -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From davea at ieee.org Sat Jan 2 21:35:48 2010 From: davea at ieee.org (Dave Angel) Date: Sat, 02 Jan 2010 21:35:48 -0500 Subject: Bare Excepts In-Reply-To: <034fd208$0$1277$c3e8da3@news.astraweb.com> References: <4dc0cfea0912300209x15e5bc96nd6e6224d89df0638@mail.gmail.com> <034f1009$0$1277$c3e8da3@news.astraweb.com> <034fd208$0$1277$c3e8da3@news.astraweb.com> Message-ID: <4B400284.1060909@ieee.org> Steven D'Aprano wrote: > On Sat, 02 Jan 2010 09:40:44 -0800, Aahz wrote: > > >> OTOH, if you want to do something different depending on whether the >> file exists, you need to use both approaches: >> >> if os.path.exists(fname): >> try: >> f = open(fname, 'rb') >> data = f.read() >> f.close() >> return data >> except IOError: >> logger.error("Can't read: %s", fname) return '' >> else: >> try: >> f = open(fname, 'wb') >> f.write(data) >> f.close() >> except IOError: >> logger.error("Can't write: %s", fname) >> return None >> > > Unfortunately, this is still vulnerable to the same sort of race > condition I spoke about. > > Even more unfortunately, I don't know that there is any fool-proof way of > avoiding such race conditions in general. Particularly the problem of > "open this file for writing only if it doesn't already exist". > > In Windows, there is a way to do it. It's just not exposed to the Python built-in function open(). You use the CreateFile() function, with /dwCreationDisposition/ of CREATE_NEW. It's atomic, and fails politely if the file already exists. No idea if Unix has a similar functionality. DaveA From bthate at gmail.com Sat Jan 2 21:40:01 2010 From: bthate at gmail.com (Bart Thate) Date: Sat, 2 Jan 2010 18:40:01 -0800 (PST) Subject: CMNDBOT 0.1 released Message-ID: new in this release: * updated the repository to GZRBOT code * a outputcache and poller gadget is now available to support writing to waves (right now the poller polls every minute) * RSS plugin looks stable todo: * make gozernet work .. this lets GZRBOT bots communicate with each other by using json over xmpp * use this to implement wave <-> IRC relaying * port monitoring of bot output * port karma plugin * port quote plugin demo: http://cmndbot.appspot.com wave/xmpp: cmndbot at appspot.com about CMNDBOT: CMNDBOT is a port of GOZERBOT to the Google Application Engine. It supports wave, web and xmpp. It supports a plugin structure that lets you add commands or register callbacks for events. License is BSD From lists at cheimes.de Sat Jan 2 21:40:17 2010 From: lists at cheimes.de (Christian Heimes) Date: Sun, 03 Jan 2010 03:40:17 +0100 Subject: IOError - cannot create file (linux daemon-invoked script) In-Reply-To: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> References: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> Message-ID: cassiope wrote: > The strange thing is that even with the right user-id, I cannot seem > to write to the directory, getting an IOError exception. Changing the > directory to world-writable fixes this. I can confirm the uid and gid > for the script by having the script print these values just before > trying to create/write the file. Becoming the same lesser user, I > have no problem writing a file to the same directory. Are you able to write to the directory with the user id when you tried to create a file manually? How are you changing the uid and gid of your script? IIRC you have to set the effective user id with os.seteuid() and os.setegid(). Christian From n00m at narod.ru Sat Jan 2 22:17:38 2010 From: n00m at narod.ru (n00m) Date: Sat, 2 Jan 2010 19:17:38 -0800 (PST) Subject: Any Swisses here? Message-ID: <145e6acd-4260-495b-b562-da634fccfa1a@j19g2000yqk.googlegroups.com> Congrats! Your choice -- to ban building of muslim mosques -- is the only choice to save our civililazation. This yellow plague (incl. Chinese etc) must be eliminated from our Planet, They are very cunning critters, they can play on strings of compassion, but its riffraffs. What do you know about Russia? Who are yuo against Russia? Nothing. Dust. We are simply tired people. From zhushazang at yahoo.com.br Sat Jan 2 22:26:59 2010 From: zhushazang at yahoo.com.br (Zhu Sha Zang) Date: Sun, 03 Jan 2010 01:26:59 -0200 Subject: Any Swisses here? In-Reply-To: <145e6acd-4260-495b-b562-da634fccfa1a@j19g2000yqk.googlegroups.com> References: <145e6acd-4260-495b-b562-da634fccfa1a@j19g2000yqk.googlegroups.com> Message-ID: <4B400E83.1080305@yahoo.com.br> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Em 03-01-2010 01:17, n00m escreveu: > Congrats! > Your choice -- to ban building of muslim mosques -- is the only choice > to save our civililazation. > This yellow plague (incl. Chinese etc) must be eliminated from our > Planet, > They are very cunning critters, they can play on strings of > compassion, > but its riffraffs. What do you know about Russia? Who are yuo against > Russia? > Nothing. Dust. We are simply tired people. WTF? -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAktADoIACgkQ35zeJy7JhCgkOACfZG9y57J6WpmPRcxBLT9lHi4D AjYAn059bwVoN+D2/WZHwsGPVVP/I3wF =msDn -----END PGP SIGNATURE----- From steve at holdenweb.com Sat Jan 2 22:30:27 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 02 Jan 2010 22:30:27 -0500 Subject: Any Swisses here? In-Reply-To: <4B400E83.1080305@yahoo.com.br> References: <145e6acd-4260-495b-b562-da634fccfa1a@j19g2000yqk.googlegroups.com> <4B400E83.1080305@yahoo.com.br> Message-ID: Zhu Sha Zang wrote: >> [stuff and nonsense from a third party] > WTF? We do get the occasional bigot dropping in from time to time. Best to ignore them 'til they go away. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From n00m at narod.ru Sat Jan 2 22:38:30 2010 From: n00m at narod.ru (n00m) Date: Sat, 2 Jan 2010 19:38:30 -0800 (PST) Subject: Any Swisses here? References: <145e6acd-4260-495b-b562-da634fccfa1a@j19g2000yqk.googlegroups.com> <4B400E83.1080305@yahoo.com.br> Message-ID: On Jan 3, 5:30?am, Steve Holden wrote: > Zhu Sha Zang wrote: > >> [stuff and nonsense from a third party] > > WTF? > > We do get the occasional bigot dropping in from time to time. Best to > ignore them 'til they go away. > > regards > ?Steve > -- > Steve Holden ? ? ? ? ? +1 571 484 6266 ? +1 800 494 3119 > PyCon is coming! Atlanta, Feb 2010 ?http://us.pycon.org/ > Holden Web LLC ? ? ? ? ? ? ? ?http://www.holdenweb.com/ > UPCOMING EVENTS: ? ? ? ?http://holdenweb.eventbrite.com/ And who are YOU? What you achieved in the life? For me you is nothing I recall only Kipling, Dickens, Newton its your background. My backgroung is the best rockets From sisson.j at gmail.com Sat Jan 2 22:47:42 2010 From: sisson.j at gmail.com (J Sisson) Date: Sat, 2 Jan 2010 21:47:42 -0600 Subject: Any Swisses here? In-Reply-To: References: <145e6acd-4260-495b-b562-da634fccfa1a@j19g2000yqk.googlegroups.com> <4B400E83.1080305@yahoo.com.br> Message-ID: <4297a9021001021947y78b6fa4cy28f0ff53d59930a2@mail.gmail.com> On Sat, Jan 2, 2010 at 9:38 PM, n00m wrote: > What you achieved in the life? English mastery, for starters... -------------- next part -------------- An HTML attachment was scrubbed... URL: From cs at zip.com.au Sat Jan 2 23:02:38 2010 From: cs at zip.com.au (Cameron Simpson) Date: Sun, 3 Jan 2010 15:02:38 +1100 Subject: IOError - cannot create file (linux daemon-invoked script) In-Reply-To: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> References: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> Message-ID: <20100103040238.GA15992@cskk.homeip.net> On 02Jan2010 15:21, cassiope wrote: | [...] I want | to save a copy of the email in a particular directory which is | accessible to the Windows clients via samba. | | The strange thing is that even with the right user-id, I cannot seem | to write to the directory, getting an IOError exception. Changing the | directory to world-writable fixes this. I can confirm the uid and gid | for the script by having the script print these values just before | trying to create/write the file. Becoming the same lesser user, I | have no problem writing a file to the same directory. Can you show us: - the directory user and group ownership and permissions - the daemon's user and group values? You can also strace your daemon: strace -f -e trace=file your-daemon your-daemon-args... 2>strace.out and then examine the log for the precise UNIX-level failure. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Money won't buy happiness, but it will pay the salary of a large research staff to study the problem. - Bill Vaughan From nagle at animats.com Sun Jan 3 01:40:31 2010 From: nagle at animats.com (John Nagle) Date: Sat, 02 Jan 2010 22:40:31 -0800 Subject: Safe file I/O to shared file (or SQLite) from multi-threaded web server In-Reply-To: References: Message-ID: <4b403899$0$1646$742ec2ed@news.sonic.net> python at bdurham.com wrote: > I'm looking for the best practice way for a multi-threaded python web > server application to read/write to a shared file or a SQLite database. > > What do I need to do (if anything) to make sure my writes to a regular > file on disk or to a SQLite database are atomic in nature when multiple > clients post data to my application simultaneously? SQLite can do that correctly, but SQLite isn't intended for use as a database engine for a busy database being used by many concurrent operations. Especially if those operations involve updates. Any update in SQLite locks all tables involved for the duration of the operation. When SQLite hits a lock, it returns an error code, and the caller should retry after a delay. If this occurs frequently in your application, you've hit the limits of SQLite. Then it's time to move up to MySQL. If you have enough traffic that you need a multi-threaded web server, it's probably time to move up. John Nagle From mackrackit at gmail.com Sun Jan 3 03:57:11 2010 From: mackrackit at gmail.com (Dave McCormick) Date: Sun, 03 Jan 2010 01:57:11 -0700 Subject: Dynamic text color In-Reply-To: <21336368.974257.1262483554337.JavaMail.jjposner@mail.srv.hcvlny.cv.net> References: <21336368.974257.1262483554337.JavaMail.jjposner@mail.srv.hcvlny.cv.net> Message-ID: <4B405BE7.4060202@gmail.com> John, Interesting and confusing... I tested my code on Ubuntu 8.04, Win XP and 7, and WinMob before I called it "good". And of course your modifications work on all too. """ * rename get_position() to get_complete_text() * replace new_Rword(), new_Bword(), and new_Gword() with a single function that has an extra parameter, "color". """ The first easy, the second is going to take some thinking.... Keep at it I will. I am hooked!!! Being able to write once and run on all of the above platforms is amazing!!! Thanks again!!! Dave John Posner wrote: > On Sat, Jan 2, 2010 at 1:47 PM, Dave McCormick wrote: > >> WooHoo!!! >> I got it!!! Yup, I am sure it can be optimized but it works!!! > > Dave, please ignore a couple of my bogus complaints in the previous > message: > > ... you call function new_Rword() before you define it > > ... this version also has the advantage of defining each function > just once, instead of multiple times on each keystroke > > > But I stand by my overall statement that the program didn't work for > me, and that rearranging the lines produces a working program. > > -John From vicente.soler at gmail.com Sun Jan 3 06:27:46 2010 From: vicente.soler at gmail.com (vsoler) Date: Sun, 3 Jan 2010 03:27:46 -0800 (PST) Subject: What is the best data structure for a very simple spreadsheet? Message-ID: Hi, Not sure this is the best group to post, but I cannot think of any other. My application would contain a limited set of "cells" represented by the instances of a Cell class: class Cell: ... A1=Cell(7) A2=Cell(2*A1) A3=Cell(3*A1+A2) A4=Cell(A3*4) Of course, A1 = 7, A2 = 14, A3 = 35 and A4 = 140 Now, I somehow want to be able to show a dependency tree 1 level dependency trees A1: None A2: A1 A3: A1, A2 A4: A3 All levels dependency trees A1: None A2: A1 A3: A1, A2 A4: A3, A2, A1 Leaf + values dependency trees: A1: 7 A2: A1=7, 2 A3: 3, A1=7, 2 A4: 3, A1=7, 2, 4 What I'd like to know is: 1) what are, in your opinion, the basic elements of the Cell class? 2) Do I need a parser to evaluate the formulas like ?3*A1+A2?? Can you recommend one library that already contains one? 3) Do I need a tree data structure to represent my data? would the tree be an attribute of the class instance? I imagine a lot can be said on these questions. What I am looking for is some hints that help me get out of where I am now. Any help is highly appreciated. Vicente Soler From python at bdurham.com Sun Jan 3 07:09:52 2010 From: python at bdurham.com (python at bdurham.com) Date: Sun, 03 Jan 2010 07:09:52 -0500 Subject: Windows 7 : any problems installing or running Python ? In-Reply-To: <000001ca8bb6$08391d70$18ab5850$@com> References: <000001ca8bb6$08391d70$18ab5850$@com> Message-ID: <1262520592.29221.1352686197@webmail.messagingengine.com> David, Try disabling your firewall software momentarily to see if that makes a difference. IDLE uses a local port for inter-process communication - you may need to configure your firewall to allow IDLE's port usage. To test whether Python itself has been properly installed, open up a cmd prompt, change your current directory to the folder where you installed Python, create the short, 1 line script that follows, and then run it. If this works, then Python has been successfully installed on your workstation and the problem you're having is specific to IDLE. 1 line script (place in hello.py) print "Hello world" Command line to run above script: python hello.py You might also try installing an alternate Python build, eg. if you installed from python.org, then try one of the setups from ActiveState. Or vice-versa. We're running Python 2.6.4 on Windows XP (32-bit), Windows Vista (32 and 64-bit), and Windows 7 (32 and 64-bit) without problem. Good luck! Malcolm -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at REMOVE-THIS-cybersource.com.au Sun Jan 3 07:28:40 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 03 Jan 2010 12:28:40 GMT Subject: What is the best data structure for a very simple spreadsheet? References: Message-ID: <03507a71$0$1277$c3e8da3@news.astraweb.com> On Sun, 03 Jan 2010 03:27:46 -0800, vsoler wrote: > My application would contain a limited set of "cells" represented by the > instances of a Cell class: > > class Cell: > ... > > A1=Cell(7) > A2=Cell(2*A1) > A3=Cell(3*A1+A2) > A4=Cell(A3*4) > > Of course, A1 = 7, A2 = 14, A3 = 35 and A4 = 140 > > Now, I somehow want to be able to show a dependency tree > > 1 level dependency trees > A1: None > A2: A1 > A3: A1, A2 > A4: A3 > > All levels dependency trees > > A1: None > A2: A1 > A3: A1, A2 > A4: A3, A2, A1 > > Leaf + values dependency trees: > > A1: 7 > A2: A1=7, 2 > A3: 3, A1=7, 2 > A4: 3, A1=7, 2, 4 > > What I'd like to know is: > > 1) what are, in your opinion, the basic elements of the Cell class? def Cell(object): def __init__(self, payload): self.payload = payload def __str__(self): return str(self.payload) def __float__(self): return float(self.payload) def dependency(self): try: return self.payload.dependency() except AttributeError: return ['None'] Cells can contain one of three things: a number, a string, or a formula. The first two can be supported by providing a built-in Python object (float or str) as payload. You can support formulae two ways that I can think of: (1) Provide a formula as a string, with a leading '='. Then, whenever you want to evaluate such a cell, you fetch the string from the cell, parse it, generate an arithmetic expression, and calculate it. (2) Instead of parsing the formula on every single spreadsheet refresh, use a couple of helper classes: class Indirect(object): def __init__(self, ref, sheet=None): if sheet is None: self.sheet = default_sheet() else: self.sheet = sheet self.ref = ref def __str__(self): return str(self.sheet[self.ref]) def float(self): return float(self.sheet[self.ref]) def dependency(self): return [self.ref] class Formula(object): def __init__(self, x, y, op): self.x = x self.y = y self.op = op def eval(self): return self.op(float(x), float(y)) def dependency(self): return self.x.dependency(level) + self.y.dependency(level) Then do something like this: sheet = {} sheet['A1'] = Cell(7) sheet['A2'] = Cell(Formula(2, Indirect('A1'), operator.mul)) sheet['A3'] = Cell( Formula( Formula(3, Indirect('A1'), operator.mul), Indirect('A2'), operator.add )) sheet['A4'] = Cell(Formula(Indirect('A3'), 4, operator.mul)) Then you only need to parse each human-readable formula like '=2*A1' once. > 2) Do I need a parser to evaluate the formulas like ?3*A1+A2?? Yes. > Can you recommend one library that already contains one? Try PyParsing. > 3) Do I need a tree > data structure to represent my data? would the tree be an attribute of > the class instance? I suspect a dict will be faster. To get the dependencies of each cell: for key, value in sheet.items(): print key, value.dependency() Keep in mind I haven't tested ANY of this -- it is entirely stream of consciousness. I've never actually done this, so I have no idea whether it is a good approach or not, but it seems to me that it might be. -- Steven From vicente.soler at gmail.com Sun Jan 3 07:38:28 2010 From: vicente.soler at gmail.com (vsoler) Date: Sun, 3 Jan 2010 04:38:28 -0800 (PST) Subject: What is the best data structure for a very simple spreadsheet? References: <03507a71$0$1277$c3e8da3@news.astraweb.com> Message-ID: <339dfbd9-d6d9-4f53-b998-46092f85d260@m25g2000yqc.googlegroups.com> On Jan 3, 1:28?pm, Steven D'Aprano wrote: > On Sun, 03 Jan 2010 03:27:46 -0800, vsoler wrote: > > My application would contain a limited set of "cells" represented by the > > instances of a Cell class: > > > class Cell: > > ... > > > A1=Cell(7) > > A2=Cell(2*A1) > > A3=Cell(3*A1+A2) > > A4=Cell(A3*4) > > > Of course, A1 = 7, A2 = 14, A3 = 35 and A4 = 140 > > > Now, I somehow want to be able to show a dependency tree > > > 1 level dependency trees > > ? A1: None > > ? A2: A1 > > ? A3: A1, A2 > > ? A4: A3 > > > All levels dependency trees > > > ? A1: None > > ? A2: A1 > > ? A3: A1, A2 > > ? A4: A3, A2, A1 > > > Leaf + values dependency trees: > > > ? A1: 7 > > ? A2: A1=7, 2 > > ? A3: 3, A1=7, 2 > > ? A4: 3, A1=7, 2, 4 > > > What I'd like to know is: > > > 1) what are, in your opinion, the basic elements of the Cell class? > > def Cell(object): > ? ? def __init__(self, payload): > ? ? ? ? self.payload = payload > ? ? def __str__(self): > ? ? ? ? return str(self.payload) > ? ? def __float__(self): > ? ? ? ? return float(self.payload) > ? ? def dependency(self): > ? ? ? ? try: > ? ? ? ? ? ? return self.payload.dependency() > ? ? ? ? except AttributeError: > ? ? ? ? ? ? return ['None'] > > Cells can contain one of three things: a number, a string, or a formula. > The first two can be supported by providing a built-in Python object > (float or str) as payload. You can support formulae two ways that I can > think of: > > (1) Provide a formula as a string, with a leading '='. Then, whenever you > want to evaluate such a cell, you fetch the string from the cell, parse > it, generate an arithmetic expression, and calculate it. > > (2) Instead of parsing the formula on every single spreadsheet refresh, > use a couple of helper classes: > > class Indirect(object): > ? ? def __init__(self, ref, sheet=None): > ? ? ? ? if sheet is None: > ? ? ? ? ? ? self.sheet = default_sheet() > ? ? ? ? else: > ? ? ? ? ? ? self.sheet = sheet > ? ? ? ? self.ref = ref > ? ? def __str__(self): > ? ? ? ? return str(self.sheet[self.ref]) > ? ? def float(self): > ? ? ? ? return float(self.sheet[self.ref]) > ? ? def dependency(self): > ? ? ? ? return [self.ref] > > class Formula(object): > ? ? def __init__(self, x, y, op): > ? ? ? ? self.x = x > ? ? ? ? self.y = y > ? ? ? ? self.op = op > ? ? def eval(self): > ? ? ? ? return self.op(float(x), float(y)) > ? ? def dependency(self): > ? ? ? ? return self.x.dependency(level) + self.y.dependency(level) > > Then do something like this: > > sheet = {} > sheet['A1'] = Cell(7) > sheet['A2'] = Cell(Formula(2, Indirect('A1'), operator.mul)) > sheet['A3'] = Cell( > ? Formula( > ? ? Formula(3, Indirect('A1'), operator.mul), > ? ? Indirect('A2'), > ? ? operator.add > ? ? )) > sheet['A4'] = Cell(Formula(Indirect('A3'), 4, operator.mul)) > > Then you only need to parse each human-readable formula like '=2*A1' once. > > > 2) Do I need a parser to evaluate the formulas like ?3*A1+A2?? > > Yes. > > > Can you recommend one library that already contains one? > > Try PyParsing. > > > 3) Do I need a tree > > data structure to represent my data? would the tree be an attribute of > > the class instance? > > I suspect a dict will be faster. > > To get the dependencies of each cell: > > for key, value in sheet.items(): > ? ? print key, value.dependency() > > Keep in mind I haven't tested ANY of this -- it is entirely stream of > consciousness. I've never actually done this, so I have no idea whether > it is a good approach or not, but it seems to me that it might be. > > -- > Steven WOW!!! After lunch I am going to read your post thoroughly, but I can already see that you've put into it a lot of time and expertise. Thank you From ktenney at gmail.com Sun Jan 3 09:05:44 2010 From: ktenney at gmail.com (Kent Tenney) Date: Sun, 3 Jan 2010 08:05:44 -0600 Subject: Trying to run a sudo command from script In-Reply-To: <7q7dkaF3t8r0rU1@mid.uni-berlin.de> References: <7q7dkaF3t8r0rU1@mid.uni-berlin.de> Message-ID: On Fri, Jan 1, 2010 at 5:08 PM, Diez B. Roggisch wrote: > Kent Tenney schrieb: >> >> Howdy, >> >> A script running as a regular user sometimes wants >> to run sudo commands. >> >> It gets the password with getpass. >> pw = getpass.getpass() >> >> I've fiddled a bunch with stuff like >> proc = subprocess.Popen('sudo touch /etc/foo'.split(), >> stdin=subprocess.PIPE) >> proc.communicate(input=pw) >> >> getting assorted errors with all variations I try. >> >> Googling says use pexpect, but I'd prefer a stdlib solution. > > pexpect is pure python, and it's needed. There is no easy way around the > issue, so if you insist on not using pexpect, you re-invent the wheel and > write the exact same code - just more error-prone, because of > wheel-reinvention.... Indeed, the requirements of this are way more complex than I guessed. The following seems to work well, took some fiddling with EOF. def sudo(command, password=None, prompt="Enter password "): import pexpect if not password: import getpass password = getpass.getpass(prompt) command = "sudo " + command child = pexpect.spawn(command) child.expect(['ssword', pexpect.EOF]) child.sendline(password) child.expect(pexpect.EOF) # is this necessary? child.close() Thanks, Kent > > Diez > -- > http://mail.python.org/mailman/listinfo/python-list > From steve at holdenweb.com Sun Jan 3 09:31:57 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 03 Jan 2010 09:31:57 -0500 Subject: Safe file I/O to shared file (or SQLite) from multi-threaded web server In-Reply-To: <4b403899$0$1646$742ec2ed@news.sonic.net> References: <4b403899$0$1646$742ec2ed@news.sonic.net> Message-ID: <4B40AA5D.1090507@holdenweb.com> John Nagle wrote: > python at bdurham.com wrote: >> I'm looking for the best practice way for a multi-threaded python web >> server application to read/write to a shared file or a SQLite database. >> >> What do I need to do (if anything) to make sure my writes to a regular >> file on disk or to a SQLite database are atomic in nature when multiple >> clients post data to my application simultaneously? > > SQLite can do that correctly, but SQLite isn't intended for use > as a database engine for a busy database being used by many > concurrent operations. Especially if those operations involve > updates. Any update in SQLite locks all tables involved for the duration > of the operation. When SQLite hits a lock, it returns an error code, and > the caller should retry after a delay. If this occurs frequently in > your application, you've hit the limits of SQLite. Then it's > time to move up to MySQL. > Or PostgreSQL, which has superior SQL standards conformance and excellent high-volume data performance. > If you have enough traffic that you need a multi-threaded web server, > it's probably time to move up. > Yes, but not to MySQL, please. Particularly since there is a sword of Damocles hanging over its head while the Oracle takeover of Sun is pending. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Sun Jan 3 09:31:57 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 03 Jan 2010 09:31:57 -0500 Subject: Safe file I/O to shared file (or SQLite) from multi-threaded web server In-Reply-To: <4b403899$0$1646$742ec2ed@news.sonic.net> References: <4b403899$0$1646$742ec2ed@news.sonic.net> Message-ID: <4B40AA5D.1090507@holdenweb.com> John Nagle wrote: > python at bdurham.com wrote: >> I'm looking for the best practice way for a multi-threaded python web >> server application to read/write to a shared file or a SQLite database. >> >> What do I need to do (if anything) to make sure my writes to a regular >> file on disk or to a SQLite database are atomic in nature when multiple >> clients post data to my application simultaneously? > > SQLite can do that correctly, but SQLite isn't intended for use > as a database engine for a busy database being used by many > concurrent operations. Especially if those operations involve > updates. Any update in SQLite locks all tables involved for the duration > of the operation. When SQLite hits a lock, it returns an error code, and > the caller should retry after a delay. If this occurs frequently in > your application, you've hit the limits of SQLite. Then it's > time to move up to MySQL. > Or PostgreSQL, which has superior SQL standards conformance and excellent high-volume data performance. > If you have enough traffic that you need a multi-threaded web server, > it's probably time to move up. > Yes, but not to MySQL, please. Particularly since there is a sword of Damocles hanging over its head while the Oracle takeover of Sun is pending. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From lie.1296 at gmail.com Sun Jan 3 09:58:05 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Mon, 04 Jan 2010 01:58:05 +1100 Subject: What is the best data structure for a very simple spreadsheet? In-Reply-To: References: Message-ID: <4b40b07e@dnews.tpgi.com.au> On 1/3/2010 10:27 PM, vsoler wrote: > 1) what are, in your opinion, the basic elements of the Cell class? The "user-entered formula" and "effective value". A Cell containing a formula "abc" has a value of "abc"; a cell containing the formula "=1+5" has a value of "6". You could use the 'property' decorator for the "effective value" attribute. > 2) Do I need a parser to evaluate the formulas like ?3*A1+A2?? Can you > recommend one library that already contains one? Yes you do. PyParsing, regex, write your own parser; choose your own poison depending on the expected complexity of the formulaes. > 3) Do I need a tree data structure to represent my data? There are two main options for your "primary data structure": - table (2x2 list) -- if you expect most of your cells to be filled, this is simple, but uses lots of memory - sparse table (list/dicts of cells) -- if you have a large table that is mostly empty cells The Dependency Tree is the "side data structure"; a "Cell" is a "Node"; a Cell's "dependencies" is the Node's "children"; a Cell's "dependant" is the Node's "parent". Cells maintains a list of dependencies (or dynamically generates them when requested) by analyzing the "formula" attribute for references to other cells. PS: Strictly speaking, this "side data structure" is not a "Tree" as there are multiple root nodes. I think it's called "Directed Acyclic Graph". However, it's true that, given a cell as a root, the cell's dependencies is a proper tree. > would the > tree be an attribute of the class instance? you could use 'property' decorator and make the dependency tree a "computed attribute" of the cell; many people recommended against a 'property' that is heavy to compute though and calculating dependencies may be quite heavy depending on the complexity of the spreadsheet. > I imagine a lot can be said on these questions. What I am looking for > is some hints that help me get out of where I am now. > > Any help is highly appreciated. > > Vicente Soler From joncle at googlemail.com Sun Jan 3 10:25:49 2010 From: joncle at googlemail.com (Jon Clements) Date: Sun, 3 Jan 2010 07:25:49 -0800 (PST) Subject: What is the best data structure for a very simple spreadsheet? References: <4b40b07e@dnews.tpgi.com.au> Message-ID: <6aad5bae-e50a-425c-b9c4-bc0c2d595670@o28g2000yqh.googlegroups.com> On Jan 3, 2:58?pm, Lie Ryan wrote: > On 1/3/2010 10:27 PM, vsoler wrote: > > > 1) what are, in your opinion, the basic elements of the Cell class? > > The "user-entered formula" and "effective value". A Cell containing a > formula "abc" has a value of "abc"; a cell containing the formula "=1+5" > has a value of "6". You could use the 'property' decorator for the > "effective value" attribute. > > > 2) Do I need a parser to evaluate the formulas like 3*A1+A2 ? Can you > > recommend one library that already contains one? > > Yes you do. PyParsing, regex, write your own parser; choose your own > poison depending on the expected complexity of the formulaes. > > > 3) Do I need a tree data structure to represent my data? > > There are two main options for your "primary data structure": > - table (2x2 list) -- if you expect most of your cells to be filled, > this is simple, but uses lots of memory > - sparse table (list/dicts of cells) -- if you have a large table that > is mostly empty cells > > The Dependency Tree is the "side data structure"; a "Cell" is a "Node"; > a Cell's "dependencies" is the Node's "children"; a Cell's "dependant" > is the Node's "parent". Cells maintains a list of dependencies (or > dynamically generates them when requested) by analyzing the "formula" > attribute for references to other cells. > > PS: Strictly speaking, this "side data structure" is not a "Tree" as > there are multiple root nodes. I think it's called "Directed Acyclic > Graph". However, it's true that, given a cell as a root, the cell's > dependencies is a proper tree. > > > would the > > tree be an attribute of the class instance? > > you could use 'property' decorator and make the dependency tree a > "computed attribute" of the cell; many people recommended against a > 'property' that is heavy to compute though and calculating dependencies > may be quite heavy depending on the complexity of the spreadsheet. > > > I imagine a lot can be said on these questions. What I am looking for > > is some hints that help me get out of where I am now. > > > Any help is highly appreciated. > > > Vicente Soler > > As well as what Steven & Lie have mentioned, maybe you could get inspiration from http://pyspread.sourceforge.net/ Although I've not looked at it, my gut feeling is the author would've had to address these issues (take that as a disclaimer of some sort :) ) >From not having to had a need for this sort of thing, I'd probably go for a sparse representation as a graph. But again, depends why you're segmenting the "spreadsheet" from "need to tell dependencies". Jon. From laiwei.ustc at gmail.com Sun Jan 3 11:01:22 2010 From: laiwei.ustc at gmail.com (laiwei.ustc) Date: Mon, 4 Jan 2010 00:01:22 +0800 Subject: HAppy New Year References: Message-ID: <201001040001187704615@gmail.com> HI Baboucarr good work beautiful thanks 2010-01-03 laiwei.ustc ???? baboucarr sanneh ????? 2010-01-03 22:32:29 ???? python-list at python.org ??? ??? HAppy New Year Hi guys, Jus want to wish you a happy new year to u all.. 1.Copy the content below and Paste it on a Notepad. 2 . Use replace all (Ctrl + H) 3. Type in ?6? in the ?find what? column 4. Type in _ (underscore) in the ?replace with? column 5. Click on replace all. 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666996666699669966999999996699666669966666669966666699669999999966996666996666666666666696666666666666666666666 666996666699669966999999996699666669966666666996666996669999999966996666996666666666666999666666666666666666666 666996666699669966996666666699666669966666666699669966669966669966996666996666666666669969966666666666666666666 666996696699669966999999996699999999966666666669999666669966669966996666996666666666699666996666666666666666666 666996999699669966999999996699999999966666666666996666669966669966996666996666666666999999999666666666666666666 666999969999669966666666996699666669966666666666996666669966669966996666996666666669999999999966666666666666666 666999666999669966999999996699666669966666666666996666669999999966996666996666666699666666666996666666666666666 666996666699669966999999996699666669966666666666996666669999999966999999996666666996666666666699666666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 669966666996666666669666666666999999996699999999669966666699666666669966666996699999999669966666996666666666666 669966666996666666699966666666999999996699999999666996666996666666669996666996699999999669966666996666666666666 669966666996666666996996666666996666996699666699666699669966666666669999666996699666666669966666996666666666666 669999999996666669966699666666999999996699999999666669999666666666669969966996699999999669966966996666666666666 669999999996666699999999966666999999996699999999666666996666666666669966996996699999999669969996996666666666666 669966666996666999999999996666999666666699666666666666996666666666669966699996699666666669999699996666666666666 669966666996669966666666699666996666666699666666666666996666666666669966669996699999999669996669996666666666666 669966666996699666666666669966996666666699666666666666996666666666669966666996699999999669966666996666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 666996666669966999999996666666669666666666999999996666666699999996699999999666999666699999999666666666666666666 666699666699666999999996666666699966666666999999996666666699999999699666699666699666699666699666666666666666666 666669966996666996666666666666996996666666996666996666666666666699699966699666699666699966699666666666666666666 666666999966666999999996666669966699666666999999996666666666666996699696699666699666699696699666666666666666666 666666699666666999999996666699999999966666999999996666666666669966699669699666699666699669699666666666666666666 666666699666666996666666666999999999996666996996666666666666996666699666999666699666699666999666666666666666666 666666699666666999999996669966666666699666996699666666666699999999699666699666699666699666699666666666666666666 666666699666666999999996699666666666669966996666996666666699999999699999999666999966699999999666666666666666666 66666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666 Regards, Baboucarr. Windows Live: Friends get your Flickr, Yelp, and Digg updates when they e-mail you. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: laiwei.ustc at gmail.com.jpg Type: image/jpeg Size: 7941 bytes Desc: not available URL: From jackdied at gmail.com Sun Jan 3 11:07:07 2010 From: jackdied at gmail.com (Jack Diederich) Date: Sun, 3 Jan 2010 11:07:07 -0500 Subject: Any Swisses here? In-Reply-To: References: <145e6acd-4260-495b-b562-da634fccfa1a@j19g2000yqk.googlegroups.com> <4B400E83.1080305@yahoo.com.br> Message-ID: On Sat, Jan 2, 2010 at 10:38 PM, n00m wrote: > On Jan 3, 5:30?am, Steve Holden wrote: >> Zhu Sha Zang wrote: >> >> [stuff and nonsense from a third party] >> > WTF? >> >> We do get the occasional bigot dropping in from time to time. Best to >> ignore them 'til they go away. >> > And who are YOU? > What you achieved in the life? > For me you is nothing I recall only Kipling, Dickens, Newton > its your background. My backgroung is the best rockets Don't be so dismissive Steve. What we have here is clearly a struggling young AI trying to extrapolate the average content of an alt.group to the comp.lang community. The fact that it passed the gmail spam filters is a sign of progress. So hello new friend, and your backgroung is the best rockets! -Jack From cjns1989 at gmail.com Sun Jan 3 12:20:26 2010 From: cjns1989 at gmail.com (Chris Jones) Date: Sun, 03 Jan 2010 12:20:26 -0500 Subject: twenty years ago Guido created Python In-Reply-To: References: Message-ID: <20100103172026.GO3125@turki.gavron.org> On Thu, Dec 31, 2009 at 05:06:24PM EST, Steve Howell wrote: > FYI: > > http://twitter.com/gvanrossum > > Python is a truly awesome programming language. Not only is Guido a > genius language designer, but he is also a great project leader. What > an accomplishment. Congratulations to everybody who has contributed > to Python in the last two decades! Interesting to note that Guido's achievements prompt much less response, and get much less coverage, than Xah Lee's :-) I guess Guido's not one to care much about beauty contests anyway. HNY, CJ From python at mrabarnett.plus.com Sun Jan 3 14:13:27 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 03 Jan 2010 19:13:27 +0000 Subject: What is the best data structure for a very simple spreadsheet? In-Reply-To: References: Message-ID: <4B40EC57.7000401@mrabarnett.plus.com> vsoler wrote: > Hi, > > Not sure this is the best group to post, but I cannot think of any > other. > > My application would contain a limited set of "cells" represented by > the instances of a Cell class: > > class Cell: > ... > > A1=Cell(7) > A2=Cell(2*A1) > A3=Cell(3*A1+A2) > A4=Cell(A3*4) > > Of course, A1 = 7, A2 = 14, A3 = 35 and A4 = 140 > > Now, I somehow want to be able to show a dependency tree > > 1 level dependency trees > A1: None > A2: A1 > A3: A1, A2 > A4: A3 > > All levels dependency trees > > A1: None > A2: A1 > A3: A1, A2 > A4: A3, A2, A1 > > Leaf + values dependency trees: > > A1: 7 > A2: A1=7, 2 > A3: 3, A1=7, 2 > A4: 3, A1=7, 2, 4 > > What I'd like to know is: > > 1) what are, in your opinion, the basic elements of the Cell class? > 2) Do I need a parser to evaluate the formulas like ?3*A1+A2?? Can you > recommend one library that already contains one? > 3) Do I need a tree data structure to represent my data? would the > tree be an attribute of the class instance? > > I imagine a lot can be said on these questions. What I am looking for > is some hints that help me get out of where I am now. > > Any help is highly appreciated. > As well as considering the other replies, you don't necessarily need to parse the expressions yourself. If A1 is a cell then 2*A1 tries to call the __rmul__ method of A1 (and A1*2 tries to call the __mul__ method of A1), so you could have it return a formula object that will multiply and return the numeric value of self (A1) by 2 when its own numeric value is requested. This means that a cell could be initialised with either a number or a formula. From dotancohen at gmail.com Sun Jan 3 14:18:38 2010 From: dotancohen at gmail.com (Dotan Cohen) Date: Sun, 3 Jan 2010 21:18:38 +0200 Subject: HAppy New Year In-Reply-To: References: Message-ID: <880dece01001031118h4e6e8a7nac7f8a6b80ca8b5d@mail.gmail.com> 2010/1/2 baboucarr sanneh : > Hi guys, > > Jus want to wish you a happy new year to u all.. > > 1.Copy the content below and Paste it on a Notepad. > 2 . Use replace all (Ctrl + H) > 3. Type in ?6? in the ?find what? column > > 4.?? Type in _ (underscore) in the ?replace with? column > > 5.?? Click on replace all. > Wow, I just love Windows users. Sending this to the Python list? Paste "on a notepad"?!? The "Replace All" function in "a notepad" is Ctrl-H?!?? What does H stand for?!?!? Explicit instructions how to :%s/6/_/g in "a notepad"?!?!? Happy new year! -- Dotan Cohen http://what-is-what.com http://gibberish.co.il From steve at holdenweb.com Sun Jan 3 14:54:14 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 03 Jan 2010 14:54:14 -0500 Subject: HAppy New Year In-Reply-To: <880dece01001031118h4e6e8a7nac7f8a6b80ca8b5d@mail.gmail.com> References: <880dece01001031118h4e6e8a7nac7f8a6b80ca8b5d@mail.gmail.com> Message-ID: Dotan Cohen wrote: > 2010/1/2 baboucarr sanneh : >> Hi guys, >> >> Jus want to wish you a happy new year to u all.. >> >> 1.Copy the content below and Paste it on a Notepad. >> 2 . Use replace all (Ctrl + H) >> 3. Type in ?6? in the ?find what? column >> >> 4. Type in _ (underscore) in the ?replace with? column >> >> 5. Click on replace all. >> > > Wow, I just love Windows users. Sending this to the Python list? Paste > "on a notepad"?!? The "Replace All" function in "a notepad" is > Ctrl-H?!?? What does H stand for?!?!? Explicit instructions how to > :%s/6/_/g in "a notepad"?!?!? > > Happy new year! > What I want to know is why didn't he just write """\ ... original long string ... """.replace("6", "_") But watch that snarkiness ... I'm a Windows user too! regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From dotancohen at gmail.com Sun Jan 3 15:04:40 2010 From: dotancohen at gmail.com (Dotan Cohen) Date: Sun, 3 Jan 2010 22:04:40 +0200 Subject: HAppy New Year In-Reply-To: References: <880dece01001031118h4e6e8a7nac7f8a6b80ca8b5d@mail.gmail.com> Message-ID: <880dece01001031204v1f9f1552je08ca9e98970a732@mail.gmail.com> > What I want to know is why didn't he just write > > """\ > ... original long string ... > """.replace("6", "_") > Uf! Why didn't I think of that?! > But watch that snarkiness ... I'm a Windows user too! > That's fine, I know that fresh air has to get into the basement somehow ! -- Dotan Cohen http://what-is-what.com http://gibberish.co.il From dteslenko at gmail.com Sun Jan 3 15:07:17 2010 From: dteslenko at gmail.com (Dmitry Teslenko) Date: Sun, 3 Jan 2010 23:07:17 +0300 Subject: [gtk+thread] Why worker thread never wakes from time.sleep()? Message-ID: <91325fec1001031207s6bb8cf3am3b3d999cd94c1b99@mail.gmail.com> Hello! I have simple gui gtk app. It has worker thread that populates list with strings and gtk window with main loop which pops strings from this list and shows them in TreeView. Thread runs get_data_from_pcap to populate list with strings. Gtk app calls update_store() with gobject.timeout_add every second. If I comment time.sleep() in update_store(), worker thread never wakes up after his time.sleep(). Why? Here's runnable example: #!/usr/bin/python # -*- coding: utf-8 -*- import pygtk pygtk.require('2.0') import gtk import gobject import pcap import sys import threading import time CONSOLE_ENCODING = 'utf-8' if sys.platform == 'win32': CONSOLE_ENCODING = 'cp866' global_pcap_queue = [] global_pcap_lock = threading.Lock() global_pcap_stop_event = threading.Event() class CityGame: def __init__(self): window = gtk.Window(gtk.WINDOW_TOPLEVEL) window.connect('delete_event', self.delete_event) window.connect('destroy', self.destroy) store = gtk.ListStore(gobject.TYPE_STRING) view = gtk.TreeView(store) #name col = gtk.TreeViewColumn('Data') cell = gtk.CellRendererText() view.append_column(col) col.pack_start(cell, True) col.add_attribute(cell, 'text', 0) vb = gtk.VBox() vb.pack_start(view) window.add(vb) window.set_size_request(400, 300) window.show_all() self.__store = store def main(self): gobject.timeout_add(1000, self.update_store) gtk.main() def update_store(self): data = None global_pcap_lock.acquire() if len(global_pcap_queue): data = global_pcap_queue.pop(0) print 'Update' global_pcap_lock.release() time.sleep(0.01) if data: self.__store.append([data]) return True def delete_event(self, widget, event, data = None): dlg = gtk.MessageDialog(flags = gtk.DIALOG_MODAL, type = gtk.MESSAGE_QUESTION, buttons = gtk.BUTTONS_YES_NO, message_format = 'Are you sure you want to quit?') dlg.set_title('CityGame') result = dlg.run() dlg.destroy() return (result != gtk.RESPONSE_YES) def destroy(self, widget, data = None): gtk.main_quit() def main(args): cg = CityGame() cg.main() def get_data_from_pcap(): #while True: while not global_pcap_stop_event.isSet(): global_pcap_lock.acquire() global_pcap_queue.append(str(time.time())) print 'Thread' global_pcap_lock.release() time.sleep(0.01) return if __name__ == '__main__': global_pcap_stop_event.clear() pcap_thread = threading.Thread(target = get_data_from_pcap) pcap_thread.start() main(sys.argv[1:]) global_pcap_stop_event.set() pcap_thread.join() -- A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing in e-mail? From peloko45 at gmail.com Sun Jan 3 16:09:49 2010 From: peloko45 at gmail.com (Joan Miller) Date: Sun, 3 Jan 2010 13:09:49 -0800 (PST) Subject: Set variables based on dictionary Message-ID: <49caee31-b697-4aa2-8e83-8b43316a0d67@j5g2000yqm.googlegroups.com> How to set local variables based on dictionary contents? From ben+python at benfinney.id.au Sun Jan 3 16:19:02 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 04 Jan 2010 08:19:02 +1100 Subject: twenty years ago Guido created Python References: Message-ID: <871vi628ft.fsf@benfinney.id.au> Chris Jones writes: > Interesting to note that Guido's achievements prompt much less > response, and get much less coverage [?] The entirety of ?comp.lang.python? is an ongoing response to Guido van Rossum's achievements (and all the others that make Python great). Mere week- or month-long threads are paltry by comparison. -- \ ?If this is your first visit to the USSR, you are welcome to | `\ it.? ?hotel room, Moscow | _o__) | Ben Finney From massimodipierro71 at gmail.com Sun Jan 3 16:40:19 2010 From: massimodipierro71 at gmail.com (mdipierro) Date: Sun, 3 Jan 2010 13:40:19 -0800 (PST) Subject: What is the best data structure for a very simple spreadsheet? References: Message-ID: <82c5a9f1-ed36-40c4-83d6-2d2d33ee40bc@j24g2000yqa.googlegroups.com> Perhaps this can be useful: http://www.web2py.com/examples/spreadsheet The code is in a single file with not dependencies and it does not require web2py to run: http://code.google.com/p/web2py/source/browse/gluon/contrib/spreadsheet.py Here is a sample controller that shows you how to embed the spreadsheet in web page: http://code.google.com/p/web2py/source/browse/applications/examples/controllers/spreadsheet.py Massimo On Jan 3, 5:27?am, vsoler wrote: > Hi, > > Not sure this is the best group to post, but I cannot think of any > other. > > My application would contain a limited set of "cells" represented by > the instances of a Cell class: > > class Cell: > ... > > A1=Cell(7) > A2=Cell(2*A1) > A3=Cell(3*A1+A2) > A4=Cell(A3*4) > > Of course, A1 = 7, A2 = 14, A3 = 35 and A4 = 140 > > Now, I somehow want to be able to show a dependency tree > > 1 level dependency trees > ? A1: None > ? A2: A1 > ? A3: A1, A2 > ? A4: A3 > > All levels dependency trees > > ? A1: None > ? A2: A1 > ? A3: A1, A2 > ? A4: A3, A2, A1 > > Leaf + values dependency trees: > > ? A1: 7 > ? A2: A1=7, 2 > ? A3: 3, A1=7, 2 > ? A4: 3, A1=7, 2, 4 > > What I'd like to know is: > > 1) what are, in your opinion, the basic elements of the Cell class? > 2) Do I need a parser to evaluate the formulas like ?3*A1+A2?? Can you > recommend one library that already contains one? > 3) Do I need a tree data structure to represent my data? would the > tree be an attribute of the class instance? > > I imagine a lot can be said on these questions. What I am looking for > is some hints that help me get out of where I am now. > > Any help is highly appreciated. > > Vicente Soler From michi at triodia.com Sun Jan 3 16:44:29 2010 From: michi at triodia.com (Michi) Date: Sun, 3 Jan 2010 13:44:29 -0800 (PST) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> On Jan 1, 2:47?pm, Peng Yu wrote: > > In the article API Design Matters by Michi Henning > > Communications of the ACM > Vol. 52 No. 5, Pages 46-56 > 10.1145/1506409.1506424http://cacm.acm.org/magazines/2009/5/24646-api-design-matters/fulltext > > It says "Another popular design flaw?namely, throwing exceptions for > expected outcomes?also causes inefficiencies because catching and > handling exceptions is almost always slower than testing a return > value." > > My observation is contradicted to the above statement by Henning. If > my observation is wrong, please just ignore my question below. Seeing that quite a few people have put their own interpretation on what I wrote, I figured I'll post a clarification. The quoted sentence appears in a section of the article that deals with efficiency. I point out in that section that bad APIs often have a price not just in terms of usability and defect rate, but that they are often inefficient as well. (For example, wrapper APIs often require additional memory allocations and/or data copies.) Incorrect use of exceptions also incurs an efficiency penalty. In many language implementations, exception handling is expensive; significantly more expensive than testing a return value. Consider the following: int x; try { x = func(); } catch (SomeException) { doSomething(); return; } doSomethingElse(); Here is the alternative without exceptions. (func() returns SpecialValue instead of throwing.) int x; x = func(); if (x == SpecialValue) { doSomething(); return; } doSomethingElse(); In many language implementations, the second version is considerably faster, especially when the exception may be thrown from deep in the bowels of func(), possibly many frames down the call tree. If func() throws an exception for something that routinely occurs in the normal use of the API, the extra cost can be noticeable. Note that I am not advocating not to use exceptions. I *am* advocating to not throw exceptions for conditions that are not exceptional. The classic example of this are lookup functions that, for example, retrieve the value of an environment variable, do a table lookup, or similar. Many such APIs throw an exception when the lookup fails because the key isn't the table. However, very often, looking for something that isn't there is a common case, such as when looking for a value and, if the value isn't present already, adding it. Here is an example of this: KeyType k = ...; ValueType v; try { v = collection.lookup(k); } catch (NotFoundException) { collection.add(k, defaultValue); v = defaultValue; } doSomethingWithValue(v); The same code if collection doesn't throw when I look up something that isn't there: KeyType k = ...; ValueType v; v = collection.lookup(k); if (v == null) { collection.add(k, defaultValue); v = defaultValue; } doSomethingWithValue(v); The problem is that, if I do something like this in a loop, and the loop is performance-critical, the exception version can cause a significant penalty. As the API designer, when I make the choice between returning a special value to indicate some condition, or throwing an exception, I should consider the following questions: * Is the special condition such that, under most conceivable circumstances, the caller will treat the condition as an unexpected error? * Is it appropriate to force the caller to deal with the condition in a catch-handler? * If the caller fails to explicitly deal with the condition, is it appropriate to terminate the program? Only if the answer to these questions is "yes" is it appropriate to throw an exception. Note the third question, which is often forgotten. By throwing an exception, I not only force the caller to handle the exception with a catch-handler (as opposed to leaving the choice to the caller), I also force the caller to *always* handle the exception: if the caller wants to ignore the condition, he/she still has to write a catch-handler and failure to do so terminates the program. Apart from the potential performance penalty, throwing exceptions for expected outcomes is bad also because it forces a try-catch block on the caller. One example of this is the .NET socket API: if I do non- blocking I/O on a socket, I get an exception if no data is ready for reading (which is the common and expected case), and I get a zero return value if the connection was lost (which is the uncommon and unexpected case). In other words, the .NET API gets this completely the wrong way round. Code that needs to do non-blocking reads from a socket turns into a proper mess as a result because the outcome of a read() call is tri- state: * Data was available and returned: no exception * No data available: exception * Connection lost: no exception Because such code normally lives in a loop that decrements a byte count until the expected number of bytes have been read, the control flow because really awkward because the successful case must be dealt with in both the try block and the catch handler, and the error condition must be dealt with in the try block as well. If the API did what it should, namely, throw an exception when the connection is lost, and not throw when I do a read (whether data was ready or not), the code would be far simpler and far more maintainable. At no point did I ever advocate not to use exception handling. Exceptions are the correct mechanism to handle errors. However, what is considered an error is very much in the eye of the beholder. As the API creator, if I indicate errors with exceptions, I make a policy decision about what is an error and what is not. It behooves me to be conservative in that policy: I should throw exceptions only for conditions that are unlikely to arise during routine and normal use of the API. Cheers, Michi. From darcy at druid.net Sun Jan 3 16:45:00 2010 From: darcy at druid.net (D'Arcy J.M. Cain) Date: Sun, 3 Jan 2010 16:45:00 -0500 Subject: HAppy New Year In-Reply-To: <880dece01001031118h4e6e8a7nac7f8a6b80ca8b5d@mail.gmail.com> References: <880dece01001031118h4e6e8a7nac7f8a6b80ca8b5d@mail.gmail.com> Message-ID: <20100103164500.3277112c.darcy@druid.net> On Sun, 3 Jan 2010 21:18:38 +0200 Dotan Cohen wrote: > Wow, I just love Windows users. Sending this to the Python list? Paste I was going to ask him if his mommy and daddy knew that he was playing with the grown-ups' computer. -- D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner. From fpm at u.washington.edu Sun Jan 3 16:54:16 2010 From: fpm at u.washington.edu (cassiope) Date: Sun, 3 Jan 2010 13:54:16 -0800 (PST) Subject: IOError - cannot create file (linux daemon-invoked script) References: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> Message-ID: On Jan 2, 3:46?pm, Steve Holden wrote: > cassiope wrote: > > I have a daemon on a Linux system that supports a number of Windows > > clients. ?Among the functions is to send e-mails, which is > > sufficiently complicated that I fork() a separate process which gets > > setuid to a lesser user, and calls a python script which does the > > actual formatting and emailing (the daemon is written in C). ?I want > > to save a copy of the email in a particular directory which is > > accessible to the Windows clients via samba. > > > The strange thing is that even with the right user-id, I cannot seem > > to write to the directory, getting an IOError exception. ?Changing the > > directory to world-writable fixes this. ?I can confirm the uid and gid > > for the script by having the script print these values just before > > trying to create/write the file. ?Becoming the same lesser user, I > > have no problem writing a file to the same directory. > > Have you looked at the IOError's errno attribute to find out exactly why > the Python subprocess is unable to write to the directory? It's errno=13 ... "permission denied". > > Is there anything that I can do to diagnose why this script is > > failing? ?For various reasons I don't want to make the directory world- > > writable. > > I'd concur on that decision. > > > This is on a Debian "squeeze" system, with python 2.5. > > > Thanks for any insights! > > Take a closer look at the exception, that might stimulate a thought or two. > > regards > ?Steve > -- > Steve Holden ? ? ? ? ? +1 571 484 6266 ? +1 800 494 3119 > PyCon is coming! Atlanta, Feb 2010 ?http://us.pycon.org/ > Holden Web LLC ? ? ? ? ? ? ? ?http://www.holdenweb.com/ > UPCOMING EVENTS: ? ? ? ?http://holdenweb.eventbrite.com/ From fpm at u.washington.edu Sun Jan 3 16:56:24 2010 From: fpm at u.washington.edu (cassiope) Date: Sun, 3 Jan 2010 13:56:24 -0800 (PST) Subject: IOError - cannot create file (linux daemon-invoked script) References: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> Message-ID: <40b81356-f0c1-4249-a536-916356377afa@u41g2000yqe.googlegroups.com> On Jan 2, 6:40?pm, Christian Heimes wrote: > cassiope wrote: > > The strange thing is that even with the right user-id, I cannot seem > > to write to the directory, getting an IOError exception. ?Changing the > > directory to world-writable fixes this. ?I can confirm the uid and gid > > for the script by having the script print these values just before > > trying to create/write the file. ?Becoming the same lesser user, I > > have no problem writing a file to the same directory. > > Are you able to write to the directory with the user id when you tried > to create a file manually? Yes. Sorry that wasn't clear. > How are you changing the uid and gid of your script? IIRC you have to > set the effective user id with os.seteuid() and os.setegid(). I'm changing the uid and gid in the daemon (which runs with root permissions until the fork and uid/gid change). The uid and gid are confirmed by printing os.getuid() and os.getgid() in the script. > Christian From __peter__ at web.de Sun Jan 3 16:57:09 2010 From: __peter__ at web.de (Peter Otten) Date: Sun, 03 Jan 2010 22:57:09 +0100 Subject: Set variables based on dictionary References: <49caee31-b697-4aa2-8e83-8b43316a0d67@j5g2000yqm.googlegroups.com> Message-ID: Joan Miller wrote: > How to set local variables based on dictionary contents? >>> def f(**d): ... for k, v in d.iteritems(): exec "%s = v" % k ... return locals() ... >>> f(a=42, b="yadda") {'a': 42, 'k': 'b', 'b': 'yadda', 'd': {'a': 42, 'b': 'yadda'}, 'v': 'yadda'} >>> f(**{"print 'warning: this can execute arbitrary code'; a":42}) warning: this can execute arbitrary code {'a': 42, 'k': "print 'warning: this can execute arbitrary code'; a", 'd': {"print 'warning: this can execute arbitrary code'; a": 42}, 'v': 42} Peter From steve at holdenweb.com Sun Jan 3 17:13:32 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 03 Jan 2010 17:13:32 -0500 Subject: IOError - cannot create file (linux daemon-invoked script) In-Reply-To: <40b81356-f0c1-4249-a536-916356377afa@u41g2000yqe.googlegroups.com> References: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> <40b81356-f0c1-4249-a536-916356377afa@u41g2000yqe.googlegroups.com> Message-ID: cassiope wrote: > On Jan 2, 6:40 pm, Christian Heimes wrote: >> cassiope wrote: >>> The strange thing is that even with the right user-id, I cannot seem >>> to write to the directory, getting an IOError exception. Changing the >>> directory to world-writable fixes this. I can confirm the uid and gid >>> for the script by having the script print these values just before >>> trying to create/write the file. Becoming the same lesser user, I >>> have no problem writing a file to the same directory. >> Are you able to write to the directory with the user id when you tried >> to create a file manually? > > Yes. Sorry that wasn't clear. > >> How are you changing the uid and gid of your script? IIRC you have to >> set the effective user id with os.seteuid() and os.setegid(). > > I'm changing the uid and gid in the daemon (which runs with root > permissions > until the fork and uid/gid change). The uid and gid are confirmed by > printing os.getuid() and os.getgid() in the script. > And what do os.geteuid() and os.getegid() report? I suppose it's possible there's some bizarre difference between the effective and actual process parameters. IS the filestore a local file system, or an NFS mount? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From fpm at u.washington.edu Sun Jan 3 17:20:19 2010 From: fpm at u.washington.edu (cassiope) Date: Sun, 3 Jan 2010 14:20:19 -0800 (PST) Subject: IOError - cannot create file (linux daemon-invoked script) References: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> Message-ID: <8617a8f1-7567-4728-9568-9d03245df1fd@j24g2000yqa.googlegroups.com> On Jan 2, 8:02?pm, Cameron Simpson wrote: > On 02Jan2010 15:21, cassiope wrote: > | [...] ?I want > | to save a copy of the email in a particular directory which is > | accessible to the Windows clients via samba. > | > | The strange thing is that even with the right user-id, I cannot seem > | to write to the directory, getting an IOError exception. ?Changing the > | directory to world-writable fixes this. ?I can confirm the uid and gid > | for the script by having the script print these values just before > | trying to create/write the file. ?Becoming the same lesser user, I > | have no problem writing a file to the same directory. > > Can you show us: > ? - the directory user and group ownership and permissions > ? - the daemon's user and group values? Directory permissions: 774 Directory ownership: "lesser user", "special group" where /etc/group has "special group" members including the "lesser user", as well as those who are expected to use the daemon, but not root. Script ownership: "lesser user"; permissions 755 Daemon ownership: root; permissions: 755 (always started by root). The script also has to connect to a postgresql database for part of its work - that part works, > You can also strace your daemon: > > ? strace -f -e trace=file your-daemon your-daemon-args... 2>strace.out > > and then examine the log for the precise UNIX-level failure. > > Cheers, > -- > Cameron Simpson DoD#743http://www.cskk.ezoshosting.com/cs/ > > Money won't buy happiness, but it will pay the salary of a large research > staff to study the problem. - Bill Vaughan Thanks, Cameron (and Steve and Christian). My first shot with strace (it's been awhile since I've used that - I think your syntax may be a tiny bit off - but it's probably the tool I need to use. Will explore further... From cjns1989 at gmail.com Sun Jan 3 17:20:21 2010 From: cjns1989 at gmail.com (Chris Jones) Date: Sun, 03 Jan 2010 17:20:21 -0500 Subject: twenty years ago Guido created Python In-Reply-To: <871vi628ft.fsf@benfinney.id.au> References: <871vi628ft.fsf@benfinney.id.au> Message-ID: <20100103222021.GU3125@turki.gavron.org> On Sun, Jan 03, 2010 at 04:19:02PM EST, Ben Finney wrote: > Chris Jones writes: > > Interesting to note that Guido's achievements prompt much less > > response, and get much less coverage [?] > > The entirety of ?comp.lang.python? is an ongoing response to Guido van > Rossum's achievements (and all the others that make Python great). > Mere week- or month-long threads are paltry by comparison. That I understand. But my remark was not really about Guido.. but rather in a roundabout way, more of a poke at Xah Lee, hopefully avoiding the troll-feeding trap. Sorry you missed my point. Come to think of it, since I have nothing much to say about python, but nevertheless enjoy reading most of the material on this list, this was mostly an excuse to wish everyone here a Happy New Year. CJ From cs at zip.com.au Sun Jan 3 18:00:28 2010 From: cs at zip.com.au (Cameron Simpson) Date: Mon, 4 Jan 2010 10:00:28 +1100 Subject: IOError - cannot create file (linux daemon-invoked script) In-Reply-To: <8617a8f1-7567-4728-9568-9d03245df1fd@j24g2000yqa.googlegroups.com> References: <8617a8f1-7567-4728-9568-9d03245df1fd@j24g2000yqa.googlegroups.com> Message-ID: <20100103230028.GA11635@cskk.homeip.net> On 03Jan2010 14:20, cassiope wrote: | On Jan 2, 8:02?pm, Cameron Simpson wrote: | > Can you show us: | > ? - the directory user and group ownership and permissions | > ? - the daemon's user and group values? | | Directory permissions: 774 That's unusual - why the "4"? Directories with read but no search (1) are of limited use. (Not none - it's only unusual, not insane). | Directory ownership: "lesser user", "special group" where /etc/group | has "special group" members including the "lesser user", as well as | those who are expected to use the daemon, but not root. | Script ownership: "lesser user"; permissions 755 | Daemon ownership: root; permissions: 755 (always started by root). And the script/daemon _runs_ as the "lesser user"? If so, superficially the permissions look like they should work. -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ I couldn't think of anything else to do with it, so I put it on the web. From python at mrabarnett.plus.com Sun Jan 3 18:28:27 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 03 Jan 2010 23:28:27 +0000 Subject: Exception as the primary error handling mechanism? In-Reply-To: <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> Message-ID: <4B41281B.1080707@mrabarnett.plus.com> Michi wrote: > On Jan 1, 2:47 pm, Peng Yu wrote: >> In the article API Design Matters by Michi Henning >> >> Communications of the ACM >> Vol. 52 No. 5, Pages 46-56 >> 10.1145/1506409.1506424http://cacm.acm.org/magazines/2009/5/24646-api-design-matters/fulltext >> >> It says "Another popular design flaw?namely, throwing exceptions for >> expected outcomes?also causes inefficiencies because catching and >> handling exceptions is almost always slower than testing a return >> value." >> >> My observation is contradicted to the above statement by Henning. If >> my observation is wrong, please just ignore my question below. > > Seeing that quite a few people have put their own interpretation on > what I wrote, I figured I'll post a clarification. > > The quoted sentence appears in a section of the article that deals > with efficiency. I point out in that section that bad APIs often have > a price not just in terms of usability and defect rate, but that they > are often inefficient as well. (For example, wrapper APIs often > require additional memory allocations and/or data copies.) Incorrect > use of exceptions also incurs an efficiency penalty. > > In many language implementations, exception handling is expensive; > significantly more expensive than testing a return value. Consider the > following: > > int x; > try { > x = func(); > } catch (SomeException) { > doSomething(); > return; > } > doSomethingElse(); > > Here is the alternative without exceptions. (func() returns > SpecialValue instead of throwing.) > > int x; > x = func(); > if (x == SpecialValue) { > doSomething(); > return; > } > doSomethingElse(); > > In many language implementations, the second version is considerably > faster, especially when the exception may be thrown from deep in the > bowels of func(), possibly many frames down the call tree. > > If func() throws an exception for something that routinely occurs in > the normal use of the API, the extra cost can be noticeable. Note that > I am not advocating not to use exceptions. I *am* advocating to not > throw exceptions for conditions that are not exceptional. > > The classic example of this are lookup functions that, for example, > retrieve the value of an environment variable, do a table lookup, or > similar. Many such APIs throw an exception when the lookup fails > because the key isn't the table. However, very often, looking for > something that isn't there is a common case, such as when looking for > a value and, if the value isn't present already, adding it. Here is an > example of this: > > KeyType k = ...; > ValueType v; > > try { > v = collection.lookup(k); > } catch (NotFoundException) { > collection.add(k, defaultValue); > v = defaultValue; > } > doSomethingWithValue(v); > > The same code if collection doesn't throw when I look up something > that isn't there: > > KeyType k = ...; > ValueType v; > > v = collection.lookup(k); > if (v == null) { > collection.add(k, defaultValue); > v = defaultValue; > } > doSomethingWithValue(v); > > The problem is that, if I do something like this in a loop, and the > loop is performance-critical, the exception version can cause a > significant penalty. > In Python, of course, there's a method for this: setdefault. > As the API designer, when I make the choice between returning a > special value to indicate some condition, or throwing an exception, I > should consider the following questions: > > * Is the special condition such that, under most conceivable > circumstances, the caller will treat the condition as an unexpected > error? > > * Is it appropriate to force the caller to deal with the condition in > a catch-handler? > > * If the caller fails to explicitly deal with the condition, is it > appropriate to terminate the program? > > Only if the answer to these questions is "yes" is it appropriate to > throw an exception. Note the third question, which is often forgotten. > By throwing an exception, I not only force the caller to handle the > exception with a catch-handler (as opposed to leaving the choice to > the caller), I also force the caller to *always* handle the exception: > if the caller wants to ignore the condition, he/she still has to write > a catch-handler and failure to do so terminates the program. > > Apart from the potential performance penalty, throwing exceptions for > expected outcomes is bad also because it forces a try-catch block on > the caller. One example of this is the .NET socket API: if I do non- > blocking I/O on a socket, I get an exception if no data is ready for > reading (which is the common and expected case), and I get a zero > return value if the connection was lost (which is the uncommon and > unexpected case). > > In other words, the .NET API gets this completely the wrong way round. > Code that needs to do non-blocking reads from a socket turns into a > proper mess as a result because the outcome of a read() call is tri- > state: > > * Data was available and returned: no exception > > * No data available: exception > > * Connection lost: no exception > > Because such code normally lives in a loop that decrements a byte > count until the expected number of bytes have been read, the control > flow because really awkward because the successful case must be dealt > with in both the try block and the catch handler, and the error > condition must be dealt with in the try block as well. > > If the API did what it should, namely, throw an exception when the > connection is lost, and not throw when I do a read (whether data was > ready or not), the code would be far simpler and far more > maintainable. > > At no point did I ever advocate not to use exception handling. > Exceptions are the correct mechanism to handle errors. However, what > is considered an error is very much in the eye of the beholder. As the > API creator, if I indicate errors with exceptions, I make a policy > decision about what is an error and what is not. It behooves me to be > conservative in that policy: I should throw exceptions only for > conditions that are unlikely to arise during routine and normal use of > the API. > In another area, string slicing in C# uses the Substring method, where you provide the start position and number of characters. If the start index is out of bounds (it must be >= 0 and < length) or the string is too short, then it throws an exception. In practice I find Python's behaviour easier to use (and the code is shorter too!). C# also misses Python's trick (in Python 2.6 and above) of giving string instances a format method, instead making it a class method, so you need to write: string.format(format_string, ...) instead of Python's: format_string.format(...) On the other hand, C#'s equivalent of raw strings treat backslashes always as a normal character. I think it's the only feature of C#'s string handling that I prefer to Python's. From fpm at u.washington.edu Sun Jan 3 18:56:46 2010 From: fpm at u.washington.edu (cassiope) Date: Sun, 3 Jan 2010 15:56:46 -0800 (PST) Subject: IOError - cannot create file (linux daemon-invoked script) References: <8617a8f1-7567-4728-9568-9d03245df1fd@j24g2000yqa.googlegroups.com> Message-ID: On Jan 3, 3:00?pm, Cameron Simpson wrote: > On 03Jan2010 14:20, cassiope wrote: > | On Jan 2, 8:02?pm, Cameron Simpson wrote: > | > Can you show us: > | > ? - the directory user and group ownership and permissions > | > ? - the daemon's user and group values? > | > | Directory permissions: 774 > > That's unusual - why the "4"? Directories with read but no search (1) > are of limited use. (Not none - it's only unusual, not insane). > > | Directory ownership: "lesser user", "special group" where /etc/group > | has "special group" members including the "lesser user", as well as > | those who are expected to use the daemon, but not root. > | Script ownership: "lesser user"; permissions 755 > | Daemon ownership: root; permissions: 755 (always started by root). > > And the script/daemon _runs_ as the "lesser user"? > > If so, superficially the permissions look like they should work. > -- > Cameron Simpson DoD#743http://www.cskk.ezoshosting.com/cs/ > > I couldn't think of anything else to do with it, so I put it on the web. Strace confirms the uid and gid == "lesser user". Changing the directory permissions to 775 changes nothing. Clearly get EACCES error on the attempted file creation. The only other thing is that as part of the python interpreter call, I provide a "reduced environment", just UID,GID,TMP,PWD,USER, and HOME. Is anything else needed? Thanks again, Cameron! From cs at zip.com.au Sun Jan 3 19:36:39 2010 From: cs at zip.com.au (Cameron Simpson) Date: Mon, 4 Jan 2010 11:36:39 +1100 Subject: IOError - cannot create file (linux daemon-invoked script) In-Reply-To: References: Message-ID: <20100104003639.GA6459@cskk.homeip.net> On 03Jan2010 15:56, cassiope wrote: | Strace confirms the uid and gid == "lesser user". Changing the | directory | permissions to 775 changes nothing. Clearly get EACCES error on the | attempted | file creation. | | The only other thing is that as part of the python interpreter call, I | provide | a "reduced environment", just UID,GID,TMP,PWD,USER, and HOME. Is | anything | else needed? Should be irrelevant. Ok: does the file to be created already exist? If so, what are its permissions? If the file exists and isn't writable you may get this error. Also, did you eyeball the actual open() call to ensure the file pathname is correct, and doesn't use a bogus (non-existent) directory name? -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ "GOD IS MY SOURCE" - Bumper sticker, Chapel Hill, NC I'll have to remember that one for the next code review meeting. - ajvander at ingr.com (Alain van der Heide) From timr at probo.com Sun Jan 3 20:09:46 2010 From: timr at probo.com (Tim Roberts) Date: Sun, 03 Jan 2010 17:09:46 -0800 Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> Message-ID: Roy Smith wrote: > >2) Whitespace was not required in many places. For example, the following >two statements (this will only make sense in a fixed-width font) are >identical: > > DO 10 I = 1, 10 > DO10I=1,10 More than "not required", it was "not relevant". This led to one of the most infamous programming blunders in the early days of the space program, when one programmer accidentially typed a period instead of a comma resulting in the loss of a satellite: DO 10 I = 1. 10 What this actually does is store the floating point value "1.1" in a new variable called "DO10I". -- Tim Roberts, timr at probo.com Providenza & Boekelheide, Inc. From ldo at geek-central.gen.new_zealand Sun Jan 3 20:23:45 2010 From: ldo at geek-central.gen.new_zealand (Lawrence D'Oliveiro) Date: Mon, 04 Jan 2010 14:23:45 +1300 Subject: DST and datetime References: <874on82oan.fsf@benfinney.id.au> Message-ID: In message <874on82oan.fsf at benfinney.id.au>, Ben Finney wrote: > Or you could use the ready-made wheel maintained by others: > > tzinfo Objects > But that?s only an abstract base class, which means it doesn?t actually implement any reading of actual timezone info. > World timezone definitions, modern and historical > Shame that they maintain their own duplicate of the Olson database, instead of reading the original directly from /usr/share/zoneinfo . From ldo at geek-central.gen.new_zealand Sun Jan 3 20:31:11 2010 From: ldo at geek-central.gen.new_zealand (Lawrence D'Oliveiro) Date: Mon, 04 Jan 2010 14:31:11 +1300 Subject: Safe file I/O to shared file (or SQLite) from multi-threaded web server References: <4b403899$0$1646$742ec2ed@news.sonic.net> Message-ID: In message , Steve Holden wrote: > Yes, but not to MySQL, please. Particularly since there is a sword of > Damocles hanging over its head while the Oracle takeover of Sun is > pending. Ah, I see the FUDsters are crawling out of the woodwork here, as well. I?ve got news for you: MySQL is an open-source product. And you can?t kill Open Source. So go crawling back to your proprietary world, if that?s the only world you understand. From mensanator at aol.com Sun Jan 3 20:42:39 2010 From: mensanator at aol.com (Mensanator) Date: Sun, 3 Jan 2010 17:42:39 -0800 (PST) Subject: How do you configure IDLE on a Mac, seeing as there's no Configure Option... Message-ID: ...because there's no [Options] menu on the shell window? Or at least give me a clue to how to use Courier New font? For some inscrutable reason, depite the plethora of formatting tools, someone decided that proportional spaced fonts ought to be the default for IDLE. From steve at holdenweb.com Sun Jan 3 20:45:28 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 03 Jan 2010 20:45:28 -0500 Subject: Safe file I/O to shared file (or SQLite) from multi-threaded web server In-Reply-To: References: <4b403899$0$1646$742ec2ed@news.sonic.net> Message-ID: Lawrence D'Oliveiro wrote: > In message , Steve > Holden wrote: > >> Yes, but not to MySQL, please. Particularly since there is a sword of >> Damocles hanging over its head while the Oracle takeover of Sun is >> pending. > > Ah, I see the FUDsters are crawling out of the woodwork here, as well. I?ve > got news for you: MySQL is an open-source product. And you can?t kill Open > Source. So go crawling back to your proprietary world, if that?s the only > world you understand. I have no objection to you attempting to inform me about things I already understand, but I would appreciate at least some attempt on your part to maintain civility in your discourse. There is no need to be so obnoxious, or so ill-informed: I didn't get to chair the Python Software Foundation by "crawling [around in a] proprietary world", so kindly mind your manners. MySQL has always been technically inferior to other choices of open source database. The current state of affairs was entirely predictable, and appears to be more to do with Monty Widenius' wish to continue exploiting a brand that he sold toSun two years ago than it has to do with technical issues, as I pointed out yesterday. http://holdenweb.blogspot.com/2010/01/wht-save-mysql-now.html regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From news123 at free.fr Sun Jan 3 21:12:09 2010 From: news123 at free.fr (News123) Date: Mon, 04 Jan 2010 03:12:09 +0100 Subject: python xmlrpc client with ssl client certificates and standard modules Message-ID: <4b414e79$0$23321$426a34cc@news.free.fr> Hi, I was googling fot quite some time and was not really succesfull. I found one solution, which I will try soon. It is http://www.cs.technion.ac.il/~danken/xmlrpc-ssl.html (found in http://hamakor.org.il/pipermail/python-il/2008-February/000029.html ) This will probably work, but it requires the module M2Crypto. In order to avoid installing M2Crypto an all hosts that want to run the script I wondered, whether there is no other solution. I can do xmlrpc over ssl WITHOUT certificates with following code: import xmlrpclib server_url = 'https://myserver' server = xmlrpclib.Server(server_url); and I can perform a https get request WITH certificates with below snippet: import httplib conn = httplib.HTTPSConnection( HOSTNAME, key_file = KEYFILE, cert_file = CERTFILE ) conn.putrequest('GET', '/') conn.endheaders() response = conn.getresponse() print response.read() I'm just lost of how to 'combine' both. Thanks in advance for any suggestions / hints N From prakash.stack at gmail.com Sun Jan 3 21:45:24 2010 From: prakash.stack at gmail.com (prakash jp) Date: Mon, 4 Jan 2010 08:15:24 +0530 Subject: Which version of MSVC?90.DLL's to distribute with Python 2.6 based Py2exe executables? In-Reply-To: References: <05bd2f84-4f09-4a76-8e17-5868450ad393@c3g2000yqd.googlegroups.com> <753bb7c9-14bf-44e3-9679-0693635f2cd9@j14g2000yqm.googlegroups.com> <573025f1-0e5d-4d82-a86b-36ea33f85cad@c3g2000yqd.googlegroups.com> Message-ID: <805f59d51001031845k6a9b241bo9ed60df646b4f88e@mail.gmail.com> *#How to use setup.py file with py2exe:* ** python daniesetup.py py2exe --bundle 1 *#Also the data files have to taken care off in the options* list *#Here is a sample setup.py:* *#----------------------------------------------------* from distutils.core import setup import py2exe import sys # no arguments if len(sys.argv) == 1: sys.argv.append("py2exe") # creates a standalone .exe file, no zip files setup( options = {"py2exe": {"compressed": 1, "optimize": 2, "ascii": 1, "bundle_files": 1}}, zipfile = None, # replace test.py with your own code filename here ... console = [{"script": 'test.py'}] ) Regards Prakash -------------- next part -------------- An HTML attachment was scrubbed... URL: From roy at panix.com Sun Jan 3 21:59:02 2010 From: roy at panix.com (Roy Smith) Date: Sun, 03 Jan 2010 21:59:02 -0500 Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> Message-ID: In article , Tim Roberts wrote: > Roy Smith wrote: > > > >2) Whitespace was not required in many places. For example, the following > >two statements (this will only make sense in a fixed-width font) are > >identical: > > > > DO 10 I = 1, 10 > > DO10I=1,10 > > More than "not required", it was "not relevant". This led to one of the > most infamous programming blunders in the early days of the space program, > when one programmer accidentially typed a period instead of a comma > resulting in the loss of a satellite: > > DO 10 I = 1. 10 > > What this actually does is store the floating point value "1.1" in a new > variable called "DO10I". 1.1 or 1.01? There were some places where a blank was equivalent to a zero in a number. I don't remember if this was one of them. From n00m at narod.ru Sun Jan 3 22:05:13 2010 From: n00m at narod.ru (n00m) Date: Sun, 3 Jan 2010 19:05:13 -0800 (PST) Subject: twenty years ago Guido created Python References: <871vi628ft.fsf@benfinney.id.au> Message-ID: i'm sobered up Yes of course Guido and his lang is both superb things From ben+python at benfinney.id.au Sun Jan 3 22:18:18 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 04 Jan 2010 14:18:18 +1100 Subject: twenty years ago Guido created Python References: <871vi628ft.fsf@benfinney.id.au> Message-ID: <871vi6zhfp.fsf@benfinney.id.au> Chris Jones writes: > On Sun, Jan 03, 2010 at 04:19:02PM EST, Ben Finney wrote: > > The entirety of ?comp.lang.python? is an ongoing response to Guido > > van Rossum's achievements (and all the others that make Python > > great). Mere week- or month-long threads are paltry by comparison. > > That I understand. But my remark was not really about Guido.. but > rather in a roundabout way, more of a poke at [a troll], hopefully > avoiding the troll-feeding trap. You didn't avoid it, since you irrelevantly raised (twice, now) discussion of a troll in a thread originally unrelated to that person. > Sorry you missed my point. I didn't, since I attempted to bring it back on topic by expunging the off-topic part of your message. Sorry I failed to salvage the thread. -- \ ?If sharing a thing in no way diminishes it, it is not rightly | `\ owned if it is not shared.? ?Augustine of Hippo (354?430 CE) | _o__) | Ben Finney From n00m at narod.ru Sun Jan 3 22:23:47 2010 From: n00m at narod.ru (n00m) Date: Sun, 3 Jan 2010 19:23:47 -0800 (PST) Subject: Any Swisses here? References: <145e6acd-4260-495b-b562-da634fccfa1a@j19g2000yqk.googlegroups.com> <4B400E83.1080305@yahoo.com.br> <0e89f58c-a001-42c9-a851-6f0ce6570873@h9g2000yqa.googlegroups.com> Message-ID: What Imeant by what waht try to solve e.g. http://acm.sgu.ru/problem.php?contest=0&problem=482 99% people here are 0 in this. They know this they know that, but they can't nothing, they simply are sitting on this group and apraised themselves But you can nothing From steven at REMOVE.THIS.cybersource.com.au Sun Jan 3 22:30:42 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 04 Jan 2010 03:30:42 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> Message-ID: On Sun, 03 Jan 2010 13:44:29 -0800, Michi wrote: > The quoted sentence appears in a section of the article that deals with > efficiency. I point out in that section that bad APIs often have a price > not just in terms of usability and defect rate, but that they are often > inefficient as well. This is very true, but good APIs often trade-off increased usability and reduced defect rate against machine efficiency too. In fact, I would argue that this is a general design principle of programming languages: since correctness and programmer productivity are almost always more important than machine efficiency, the long-term trend across virtually all languages is to increase correctness and productivity even if doing so costs some extra CPU cycles. > (For example, wrapper APIs often require additional > memory allocations and/or data copies.) Incorrect use of exceptions also > incurs an efficiency penalty. And? *Correct* use of exceptions also incur a penalty. So does the use of functions. Does this imply that putting code in functions is a poor API? Certainly not. > In many language implementations, exception handling is expensive; > significantly more expensive than testing a return value. And in some it is less expensive. But no matter how much more expensive, there will always be a cut-off point where it is cheaper on average to suffer the cost of handling an exception than it is to make unnecessary tests. In Python, for dictionary key access, that cut-off is approximately at one failure per ten or twenty attempts. So unless you expect more than one in ten attempts to lead to a failure, testing first is actually a pessimation, not an optimization. > Consider the following: > > int x; > try { > x = func(); > } catch (SomeException) { > doSomething(); > return; > } > doSomethingElse(); > > Here is the alternative without exceptions. (func() returns SpecialValue > instead of throwing.) > > int x; > x = func(); > if (x == SpecialValue) { > doSomething(); > return; > } > doSomethingElse(); In some, limited, cases you might be able to use the magic return value strategy, but this invariably leads to lost programmer productivity, more complex code, lowered readability and usability, and more defects, because programmers will invariably neglect to test for the special value: int x; x = func(); doSomething(x); return; Or worse, they will write doSomething() so that it too needs to know about SpecialValue, and so do all the functions it calls. Instead of dealing with the failure in one place, you can end up having to deal with it in a dozen places. But even worse is common case that SpecialValue is a legal value when passed to doSomething, and you end up with the error propagating deep into the application before being found. Or even worse, it is never found at all, and the application simply does the wrong thing. > In many language implementations, the second version is considerably > faster, especially when the exception may be thrown from deep in the > bowels of func(), possibly many frames down the call tree. This is a classic example of premature optimization. Unless such inefficiency can be demonstrated to actually matter, then you do nobody any favours by preferring the API that leads to more defects on the basis of *assumed* efficiency. If your test for a special value is 100 times faster than handling the exception, and exceptions occur only one time in 1000, then using a strategy of testing for a special value is actually ten times slower on average than catching an exception. > If func() throws an exception for something that routinely occurs in the > normal use of the API, the extra cost can be noticeable. "Can be". But it also might not be noticeable at all. [...] > Here is an example of this: > > KeyType k = ...; > ValueType v; > > try { > v = collection.lookup(k); > } catch (NotFoundException) { > collection.add(k, defaultValue); > v = defaultValue; > } > doSomethingWithValue(v); > > The same code if collection doesn't throw when I look up something that > isn't there: > > KeyType k = ...; > ValueType v; > > v = collection.lookup(k); > if (v == null) { > collection.add(k, defaultValue); > v = defaultValue; > } > doSomethingWithValue(v); > > The problem is that, if I do something like this in a loop, and the loop > is performance-critical, the exception version can cause a significant > penalty. No, the real problems are: (1) The caller has to remember to check the return result for the magic value. Failure to do so leads to bugs, in some cases, serious and hard-to- find bugs. (2) If missing keys are rare enough, the cost of all those unnecessary tests will out-weigh the saving of avoiding catching the exception. "Rare enough" may still be very common: in the case of Python, the cross-over point is approximately 1 time in 15. (3) Your collection now cannot use the magic value as a legitimate value. This last one can be *very* problematic. In the early 1990s, I was programming using a callback API that could only return an integer. The standard way of indicating an error was to return -1. But what happens if -1 is a legitimate return value, e.g. for a maths function? The solution used was to have the function create a global variable holding a flag: result = function(args) if result == -1: if globalErrorState == -1: print "An error occurred" exit doSomething(result) That is simply horrible. > As the API designer, when I make the choice between returning a special > value to indicate some condition, or throwing an exception, I should > consider the following questions: > > * Is the special condition such that, under most conceivable > circumstances, the caller will treat the condition as an unexpected > error? Wrong. It doesn't matter whether it is an error or not. They are called EXCEPTIONS, not ERRORS. What matters is that it is an exceptional case. Whether that exceptional case is an error condition or not is dependent on the application. > * Is it appropriate to force the caller to deal with the condition in > a catch-handler? > > * If the caller fails to explicitly deal with the condition, is it > appropriate to terminate the program? > > Only if the answer to these questions is "yes" is it appropriate to > throw an exception. Note the third question, which is often forgotten. > By throwing an exception, I not only force the caller to handle the > exception with a catch-handler (as opposed to leaving the choice to the > caller), I also force the caller to *always* handle the exception: if > the caller wants to ignore the condition, he/she still has to write a > catch-handler and failure to do so terminates the program. That's a feature of exceptions, not a problem. > Apart from the potential performance penalty, throwing exceptions for > expected outcomes is bad also because it forces a try-catch block on the > caller. But it's okay to force a `if (result==MagicValue)` test instead? Look, the caller has to deal with exceptional cases (which may include error conditions) one way or the other. If you don't deal with them at all, your code will core dump, or behave incorrectly, or something. If the caller fails to deal with the exceptional case, it is better to cause an exception that terminates the application immediately than it is to allow the application to generate incorrect results. > One example of this is the .NET socket API: if I do non- > blocking I/O on a socket, I get an exception if no data is ready for > reading (which is the common and expected case), and I get a zero return > value if the connection was lost (which is the uncommon and unexpected > case). > > In other words, the .NET API gets this completely the wrong way round. Well we can agree on that! > If the API did what it should, namely, throw an exception when the > connection is lost, and not throw when I do a read (whether data was > ready or not), the code would be far simpler and far more maintainable. > > At no point did I ever advocate not to use exception handling. > Exceptions are the correct mechanism to handle errors. However, what is > considered an error is very much in the eye of the beholder. As the API > creator, if I indicate errors with exceptions, I make a policy decision > about what is an error and what is not. It behooves me to be > conservative in that policy: I should throw exceptions only for > conditions that are unlikely to arise during routine and normal use of > the API. But lost connections *are* routine and normal. Hopefully they are rare. -- Steven From drobinow at gmail.com Sun Jan 3 22:33:57 2010 From: drobinow at gmail.com (David Robinow) Date: Sun, 3 Jan 2010 22:33:57 -0500 Subject: Significant whitespace In-Reply-To: References: <4B3E70FC.1010702@gmail.com> Message-ID: <4eb0089f1001031933o5b70edeo4d8fd57784747113@mail.gmail.com> On Sun, Jan 3, 2010 at 8:09 PM, Tim Roberts wrote: > More than "not required", it was "not relevant". ?This led to one of the > most infamous programming blunders in the early days of the space program, > when one programmer accidentially typed a period instead of a comma > resulting in the loss of a satellite: Interesting story. Did you make it up? From roy at panix.com Sun Jan 3 22:36:44 2010 From: roy at panix.com (Roy Smith) Date: Sun, 03 Jan 2010 22:36:44 -0500 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> Message-ID: In article , Steven D'Aprano wrote: > This last one can be *very* problematic. In the early 1990s, I was > programming using a callback API that could only return an integer. The > standard way of indicating an error was to return -1. But what happens if > -1 is a legitimate return value, e.g. for a maths function? One of the truly nice features of Python is the universally distinguished value, None. From roy at panix.com Sun Jan 3 22:42:16 2010 From: roy at panix.com (Roy Smith) Date: Sun, 03 Jan 2010 22:42:16 -0500 Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> Message-ID: In article , David Robinow wrote: > On Sun, Jan 3, 2010 at 8:09 PM, Tim Roberts wrote: > > More than "not required", it was "not relevant". ?This led to one of the > > most infamous programming blunders in the early days of the space program, > > when one programmer accidentially typed a period instead of a comma > > resulting in the loss of a satellite: > Interesting story. Did you make it up? It's a fairly well known story. http://catless.ncl.ac.uk/Risks/5.64.html#subj4.2 From drobinow at gmail.com Sun Jan 3 23:42:44 2010 From: drobinow at gmail.com (David Robinow) Date: Sun, 3 Jan 2010 23:42:44 -0500 Subject: Significant whitespace In-Reply-To: References: <4B3E70FC.1010702@gmail.com> Message-ID: <4eb0089f1001032042v2aaabbdasf3405ffaf571545@mail.gmail.com> On Sun, Jan 3, 2010 at 10:42 PM, Roy Smith wrote: > In article , > ?David Robinow wrote: > >> On Sun, Jan 3, 2010 at 8:09 PM, Tim Roberts wrote: >> > More than "not required", it was "not relevant". ?This led to one of the >> > most infamous programming blunders in the early days of the space program, >> > when one programmer accidentially typed a period instead of a comma >> > resulting in the loss of a satellite: >> Interesting story. Did you make it up? > > It's a fairly well known story. > > http://catless.ncl.ac.uk/Risks/5.64.html#subj4.2 Sure. But the question is, "Who made it up?" http://en.wikiquote.org/wiki/Fortran Computer folklore has incorrectly attributed the loss of the Mariner 1 space probe to a syntax error in a Fortran program. For example, "Recall the first American space probe to Venus, reportedly lost because Fortran cannot recognize a missing comma in a DO statement?"[ From n00m at narod.ru Sun Jan 3 23:44:38 2010 From: n00m at narod.ru (n00m) Date: Sun, 3 Jan 2010 20:44:38 -0800 (PST) Subject: twenty years ago Guido created Python References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> Message-ID: <04dfc0f0-bee3-4df3-bc50-e8e9309b45d1@e37g2000yqn.googlegroups.com> Now tell me: what ahat this mobs of yellow, africans, asiats, are doing in Stockholm? Do you think your govs let me in just to vist England, Norway? Never. Its all conditionally. But the fact is. IS. You roll our world into abyss of stupidity From robertoedwins at gmail.com Mon Jan 4 00:50:06 2010 From: robertoedwins at gmail.com (rieh25) Date: Sun, 3 Jan 2010 21:50:06 -0800 (PST) Subject: Question about an application Message-ID: <27009118.post@talk.nabble.com> I am thinking of installing a python webserver I coded in every computer at my work. This would allow me to run specific tasks in them, like creating backups, installing things, etc. Is there another way to run programs in remote computers? Thanks for your opinions. -- View this message in context: http://old.nabble.com/Question-about-an-application-tp27009118p27009118.html Sent from the Python - python-list mailing list archive at Nabble.com. From rodrick.brown at gmail.com Mon Jan 4 00:54:22 2010 From: rodrick.brown at gmail.com (rodrick brown) Date: Mon, 4 Jan 2010 00:54:22 -0500 Subject: Question about an application In-Reply-To: <27009118.post@talk.nabble.com> References: <27009118.post@talk.nabble.com> Message-ID: <9A7F3B4C-5621-44CF-ABCA-D440E7ADA01E@gmail.com> Take a look at ssh Sent from my iPhone 3GS. On Jan 4, 2010, at 12:50 AM, rieh25 wrote: > > I am thinking of installing a python webserver I coded in every > computer at > my work. This would allow me to run specific tasks in them, like > creating > backups, installing things, etc. Is there another way to run > programs in > remote computers? Thanks for your opinions. > -- > View this message in context: http://old.nabble.com/Question-about-an-application-tp27009118p27009118.html > Sent from the Python - python-list mailing list archive at Nabble.com. > > -- > http://mail.python.org/mailman/listinfo/python-list From aioe.org at technicalbloke.com Mon Jan 4 01:11:35 2010 From: aioe.org at technicalbloke.com (r0g) Date: Mon, 04 Jan 2010 06:11:35 +0000 Subject: twenty years ago Guido created Python References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> Message-ID: n00m wrote: > Ok, let me evolve a bit my thought. > Guido is Eropean Culture. > Now say what was invented by Japanese, by Chinese? > Nothing. > Barbaric cultures, they are animals inborn animals. > Russian Sikorsky built choppers for Eisenhower. > Look at letter "S" on them -- its his name > What you know about his destiny? > You know knothing. Yuou don't know that only the > greatest Russian composer Rakhmaninov helped him, in America. > Literally -- by money, the composer gave him all his fortune ~$1500 > Be not this help -- no choppers in America Announcement: You can download the latest version of this cutting edge bot from here... http://www.illiteratexenophobicrantingracistsimulator.ru/usenet_spambot.py Plugins available here... http://www.illiteratexenophobicrantingracistsimulator.ru/terrible_spelling.py http://www.illiteratexenophobicrantingracistsimulator.ru/inconsequential_facts.py http://www.illiteratexenophobicrantingracistsimulator.ru/rabid_nationalism.py Happy new year everybody! Roger. From robertoedwins at gmail.com Mon Jan 4 01:20:38 2010 From: robertoedwins at gmail.com (rieh25) Date: Sun, 3 Jan 2010 22:20:38 -0800 (PST) Subject: Question about an application In-Reply-To: <9A7F3B4C-5621-44CF-ABCA-D440E7ADA01E@gmail.com> References: <27009118.post@talk.nabble.com> <9A7F3B4C-5621-44CF-ABCA-D440E7ADA01E@gmail.com> Message-ID: <27009252.post@talk.nabble.com> Thanks, I had forgotten about it. I'll investigate if there are ways to automate using it to run programs in several computers at the same time. Rodrick Brown wrote: > > Take a look at ssh > > Sent from my iPhone 3GS. > > On Jan 4, 2010, at 12:50 AM, rieh25 wrote: > >> >> I am thinking of installing a python webserver I coded in every >> computer at >> my work. This would allow me to run specific tasks in them, like >> creating >> backups, installing things, etc. Is there another way to run >> programs in >> remote computers? Thanks for your opinions. >> -- >> View this message in context: >> http://old.nabble.com/Question-about-an-application-tp27009118p27009118.html >> Sent from the Python - python-list mailing list archive at Nabble.com. >> >> -- >> http://mail.python.org/mailman/listinfo/python-list > -- > http://mail.python.org/mailman/listinfo/python-list > > -- View this message in context: http://old.nabble.com/Question-about-an-application-tp27009118p27009252.html Sent from the Python - python-list mailing list archive at Nabble.com. From cjns1989 at gmail.com Mon Jan 4 01:25:33 2010 From: cjns1989 at gmail.com (Chris Jones) Date: Mon, 04 Jan 2010 01:25:33 -0500 Subject: twenty years ago Guido created Python In-Reply-To: <871vi6zhfp.fsf@benfinney.id.au> References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> Message-ID: <20100104062533.GB3125@turki.gavron.org> On Sun, Jan 03, 2010 at 10:18:18PM EST, Ben Finney wrote: > Chris Jones writes: [..] > > Sorry you missed my point. > > I didn't, since I attempted to bring it back on topic by expunging the > off-topic part of your message. Er.. what's 'off-topic' about mentioning that a couple of trolls on this list often seem to attract far more interest than a genuine testimonial to Guido's achievement? > Sorry I failed to salvage the thread. Man with a Mission, hehe.. CJ From aioe.org at technicalbloke.com Mon Jan 4 01:54:05 2010 From: aioe.org at technicalbloke.com (r0g) Date: Mon, 04 Jan 2010 06:54:05 +0000 Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> Message-ID: David Robinow wrote: > On Sun, Jan 3, 2010 at 10:42 PM, Roy Smith wrote: >> In article , >> David Robinow wrote: >> >>> On Sun, Jan 3, 2010 at 8:09 PM, Tim Roberts wrote: >>>> More than "not required", it was "not relevant". This led to one of the >>>> most infamous programming blunders in the early days of the space program, >>>> when one programmer accidentially typed a period instead of a comma >>>> resulting in the loss of a satellite: >>> Interesting story. Did you make it up? >> It's a fairly well known story. >> >> http://catless.ncl.ac.uk/Risks/5.64.html#subj4.2 > Sure. But the question is, "Who made it up?" > http://en.wikiquote.org/wiki/Fortran > > Computer folklore has incorrectly attributed the loss of the Mariner 1 > space probe to a syntax error in a Fortran program. For example, > "Recall the first American space probe to Venus, reportedly lost > because Fortran cannot recognize a missing comma in a DO statement?"[ Wow you're totally right! It was a missing UNDERSCORE not a DOT being typed instead of a COMMA! We've been lied to ALL ALONG! What kind of sickos would do this? Let's find this Tim character, flay him alive and then go crown you the KING OF EVERYTHING for rescuing us from that deadly nugget of misinformation! Honestly I might have believed those dot/comma LIES if it wasn't for your noble selfless efforts to uncover the SHOCKING TRUTH. Imagine what might have happened! The character in question being an UNDERSCORE COMPLETELY CHANGES the meaning and point of the story. As for it being LEFT OUT rather than TRANSPOSED, well, anyone can see how that TURNS THE WHOLE STORY UPSIDE DOWN. Of course there are the naysayers who might claim absolute accuracy is no big deal in stories that are essentially parables but they do not understand the HARM that can come from not remembering really tiny, almost inconsequential details. For what it's worth I DO NOT consider you a tiresome pedant and I think your taunting, patronising style is completely appropriate and proportional to Tim's sickening crimes against fact. Roger. From steven at REMOVE.THIS.cybersource.com.au Mon Jan 4 01:55:01 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 04 Jan 2010 06:55:01 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> Message-ID: On Sun, 03 Jan 2010 22:36:44 -0500, Roy Smith wrote: > In article , > Steven D'Aprano wrote: > >> This last one can be *very* problematic. In the early 1990s, I was >> programming using a callback API that could only return an integer. The >> standard way of indicating an error was to return -1. But what happens >> if -1 is a legitimate return value, e.g. for a maths function? > > One of the truly nice features of Python is the universally > distinguished value, None. What happens if you need to return None as a legitimate value? Here's a good example: iterating over a list. Python generates an exception when you hit the end of the list. If instead, Python returned None when the index is out of bounds, you couldn't store None in a list without breaking code. So we produce a special sentinel object EndOfSequence. Now we can't do this: for obj in ["", 12, None, EndOfSequence, [], {}]: print dir(obj) # or some other useful operation The fundamental flaw of using magic values is that, there will always be some application where you want to use the magic value as a non-special value, and then you're screwed. This is why, for instance, it's difficult for C strings to contain a null byte, and there are problems with text files on DOS and CP/M (and Windows under some circumstances) that contain a ^Z byte. -- Steven From steven at REMOVE.THIS.cybersource.com.au Mon Jan 4 01:56:03 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 04 Jan 2010 06:56:03 GMT Subject: Significant whitespace References: <4B3E70FC.1010702@gmail.com> Message-ID: On Sun, 03 Jan 2010 23:42:44 -0500, David Robinow wrote: > On Sun, Jan 3, 2010 at 10:42 PM, Roy Smith wrote: >> In article , >> ?David Robinow wrote: >> >>> On Sun, Jan 3, 2010 at 8:09 PM, Tim Roberts wrote: >>> > More than "not required", it was "not relevant". ?This led to one of >>> > the most infamous programming blunders in the early days of the >>> > space program, when one programmer accidentially typed a period >>> > instead of a comma resulting in the loss of a satellite: >>> Interesting story. Did you make it up? >> >> It's a fairly well known story. >> >> http://catless.ncl.ac.uk/Risks/5.64.html#subj4.2 > Sure. But the question is, "Who made it up?" > http://en.wikiquote.org/wiki/Fortran > > Computer folklore has incorrectly attributed the loss of the Mariner 1 > space probe to a syntax error in a Fortran program. For example, "Recall > the first American space probe to Venus, reportedly lost because Fortran > cannot recognize a missing comma in a DO statement?"[ What makes you believe it is incorrect? -- Steven From aioe.org at technicalbloke.com Mon Jan 4 02:05:23 2010 From: aioe.org at technicalbloke.com (r0g) Date: Mon, 04 Jan 2010 07:05:23 +0000 Subject: Question about an application References: <27009118.post@talk.nabble.com> <9A7F3B4C-5621-44CF-ABCA-D440E7ADA01E@gmail.com> Message-ID: rieh25 wrote: > Thanks, I had forgotten about it. I'll investigate if there are ways to > automate using it to run programs in several computers at the same time. > > > Rodrick Brown wrote: >> Take a look at ssh >> There are. Take a look at paramiko if you want to interface with SSH within python. Depending on what you need to do it may well be easier to use bash scripts / batch files or simply the CLI instead e.g. ssh account_name at 192.168.x.x 'ls /' > root_folder_listing.txt This line... starts ssh logs into account on machine 192.168.x.x executes ls pipes the data back to a file on your local machine Easy peasy :) Roger. From highcar at gmail.com Mon Jan 4 02:42:36 2010 From: highcar at gmail.com (elca) Date: Sun, 3 Jan 2010 23:42:36 -0800 (PST) Subject: python mechanize proxy support question Message-ID: <27009696.post@talk.nabble.com> Hello All Happy New Year! i have some question about python mechanize 's proxy support. im making some web client script, and i would like to insert proxy support function into my script. for example ,if i have such like following some script. how can i add proxy support into my mechanize script? i was look for some reference , but not so much good hint from google. params = urllib.urlencode({'id':id, 'passwd':pw}) rq = mechanize.Request("http://www.example.com" params) rs = mechanize.urlopen(rq) whenever i open this 'www.example.com' website , i would like open go through proxy. Thanks in advance! -- View this message in context: http://old.nabble.com/python-mechanize-proxy-support-question-tp27009696p27009696.html Sent from the Python - python-list mailing list archive at Nabble.com. From gagsl-py2 at yahoo.com.ar Mon Jan 4 02:51:40 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 04 Jan 2010 04:51:40 -0300 Subject: TypeError: can only concatenate list (not "tuple") to list Message-ID: This py> [1,2,3] + (4,5) Traceback (most recent call last): File "", line 1, in TypeError: can only concatenate list (not "tuple") to list -- Gabriel Genellina From clp2 at rebertia.com Mon Jan 4 02:58:54 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 3 Jan 2010 23:58:54 -0800 Subject: TypeError: can only concatenate list (not "tuple") to list In-Reply-To: References: Message-ID: <50697b2c1001032358j59a2f08epfb01e6ec73615dc7@mail.gmail.com> On Sun, Jan 3, 2010 at 11:51 PM, Gabriel Genellina wrote: > This > py> [1,2,3] + (4,5) > Traceback (most recent call last): > ?File "", line 1, in > TypeError: can only concatenate list (not "tuple") to list Given that tuples are sometimes used as a poor man's object (i.e. collection of data fields), whereas lists are not typically used that way, I'd say it's probably a good thing an explicit type conversion is required here. Cheers, Chris -- http://blog.rebertia.com From gagsl-py2 at yahoo.com.ar Mon Jan 4 02:59:02 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 04 Jan 2010 04:59:02 -0300 Subject: TypeError: can only concatenate list (not "tuple") to list Message-ID: Is there any reason for this error? Apart from "nobody cared to write the code" py> [1,2,3] + (4,5) Traceback (most recent call last): File "", line 1, in TypeError: can only concatenate list (not "tuple") to list In-place addition += does work: py> a = [1,2,3] py> a += (4,5) py> a [1, 2, 3, 4, 5] -- Gabriel Genellina From david at bibliolabs.com Mon Jan 4 03:20:44 2010 From: david at bibliolabs.com (David Williams) Date: Mon, 4 Jan 2010 02:20:44 -0600 (CST) Subject: TypeError: can only concatenate list (not "tuple") to list In-Reply-To: References: Message-ID: <1984.68.59.29.80.1262593244.squirrel@www.bibliobazaar.com> > Is there any reason for this error? Apart from "nobody cared to write the > code" > > py> [1,2,3] + (4,5) > Traceback (most recent call last): > File "", line 1, in > TypeError: can only concatenate list (not "tuple") to list > > In-place addition += does work: > > py> a = [1,2,3] > py> a += (4,5) > py> a > [1, 2, 3, 4, 5] > > -- > Gabriel Genellina > > -- > http://mail.python.org/mailman/listinfo/python-list > They are different types. From steven at REMOVE.THIS.cybersource.com.au Mon Jan 4 03:22:44 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 04 Jan 2010 08:22:44 GMT Subject: TypeError: can only concatenate list (not "tuple") to list References: Message-ID: On Mon, 04 Jan 2010 04:59:02 -0300, Gabriel Genellina wrote: > Is there any reason for this error? Apart from "nobody cared to write > the code" Yes, because such implicit conversions would be a bad idea. > py> [1,2,3] + (4,5) What result are you expecting? A list or a tuple? > Traceback (most recent call last): > File "", line 1, in > TypeError: can only concatenate list (not "tuple") to list Apart from the different error message, this is essentially the same error as this: >>> 2 + "2" Traceback (most recent call last): File "", line 1, in TypeError: unsupported operand type(s) for +: 'int' and 'str' > In-place addition += does work: > > py> a = [1,2,3] > py> a += (4,5) > py> a > [1, 2, 3, 4, 5] I call that an impressive gotcha. I believe that is because in-place addition of lists is implemented as functionally equivalent to the extend method: >>> a += "abc" # same as a.extend("abc") >>> a [1, 2, 3, 4, 5, 'a', 'b', 'c'] >>> a += {None: -1} >>> a [1, 2, 3, 4, 5, 'a', 'b', 'c', None] -- Steven From david at bibliolabs.com Mon Jan 4 03:24:56 2010 From: david at bibliolabs.com (David Williams) Date: Mon, 4 Jan 2010 02:24:56 -0600 (CST) Subject: TypeError: can only concatenate list (not "tuple") to list In-Reply-To: References: Message-ID: <1992.68.59.29.80.1262593496.squirrel@www.bibliobazaar.com> > Is there any reason for this error? Apart from "nobody cared to write the > code" > > py> [1,2,3] + (4,5) > Traceback (most recent call last): > File "", line 1, in > TypeError: can only concatenate list (not "tuple") to list > > In-place addition += does work: > > py> a = [1,2,3] > py> a += (4,5) > py> a > [1, 2, 3, 4, 5] > > -- > Gabriel Genellina > > -- > http://mail.python.org/mailman/listinfo/python-list > I guess to expand a bit more on what I said... What should the result be? A list or a tuple? The reason += works is because the end result is clear; a list. But it is ambiguous in the case of concatenation: did you want a tuple or a list? From gagsl-py2 at yahoo.com.ar Mon Jan 4 03:27:59 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 04 Jan 2010 05:27:59 -0300 Subject: TypeError: can only concatenate list (not "tuple") to list References: Message-ID: En Mon, 04 Jan 2010 04:58:54 -0300, Chris Rebert escribi?: > On Sun, Jan 3, 2010 at 11:51 PM, Gabriel Genellina > wrote: >> py> [1,2,3] + (4,5) >> Traceback (most recent call last): >> File "", line 1, in >> TypeError: can only concatenate list (not "tuple") to list Sorry, I inadvertedly posted an incomplete message. Note the last part: >> In-place addition += does work: >> >> py> a = [1,2,3] >> py> a += (4,5) >> py> a >> [1, 2, 3, 4, 5] > Given that tuples are sometimes used as a poor man's object (i.e. > collection of data fields), whereas lists are not typically used that > way, I'd say it's probably a good thing an explicit type conversion is > required here. In that case += should not be allowed either... -- Gabriel Genellina From sanneh27 at hotmail.com Mon Jan 4 03:30:04 2010 From: sanneh27 at hotmail.com (baboucarr sanneh) Date: Mon, 4 Jan 2010 08:30:04 +0000 Subject: HAppy New Year In-Reply-To: <880dece01001031204v1f9f1552je08ca9e98970a732@mail.gmail.com> References: , <880dece01001031118h4e6e8a7nac7f8a6b80ca8b5d@mail.gmail.com>, , <880dece01001031204v1f9f1552je08ca9e98970a732@mail.gmail.com> Message-ID: Oops ya rite.. Y didn't i think of that ?..neway thnx guys guess y we ave this mailing list to share ideas rite... Regards, Baboucarr. > Date: Sun, 3 Jan 2010 22:04:40 +0200 > Subject: Re: HAppy New Year > From: dotancohen at gmail.com > To: steve at holdenweb.com > CC: python-list at python.org > > > What I want to know is why didn't he just write > > > > """\ > > ... original long string ... > > """.replace("6", "_") > > > > Uf! Why didn't I think of that?! > > > > But watch that snarkiness ... I'm a Windows user too! > > > > That's fine, I know that fresh air has to get into the basement somehow ! > > -- > Dotan Cohen > > http://what-is-what.com > http://gibberish.co.il > -- > http://mail.python.org/mailman/listinfo/python-list _________________________________________________________________ Windows Live: Make it easier for your friends to see what you?re up to on Facebook. http://www.microsoft.com/middleeast/windows/windowslive/see-it-in-action/social-network-basics.aspx?ocid=PID23461::T:WLMTAGL:ON:WL:en-xm:SI_SB_2:092009 -------------- next part -------------- An HTML attachment was scrubbed... URL: From wuwei23 at gmail.com Mon Jan 4 03:34:14 2010 From: wuwei23 at gmail.com (alex23) Date: Mon, 4 Jan 2010 00:34:14 -0800 (PST) Subject: python mechanize proxy support question References: Message-ID: <4ca2b83a-3b82-4d20-85a3-b6304f25fb81@21g2000yqj.googlegroups.com> On Jan 4, 5:42?pm, elca wrote: > how can i add proxy support into my mechanize script? > i was look for some reference , but not so much good hint from google. There are examples on using proxies with mechanize on the module's home page: http://wwwsearch.sourceforge.net/mechanize/ From fetchinson at googlemail.com Mon Jan 4 03:50:27 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Mon, 4 Jan 2010 09:50:27 +0100 Subject: twenty years ago Guido created Python In-Reply-To: <871vi6zhfp.fsf@benfinney.id.au> References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> Message-ID: > Sorry I failed to salvage the thread. Go, quick, these threads are all around us! They are popping up everywhere! Go, go, quick! There is another troll thread right there! They are encircling us! They keep coming! At least one troll thread started on usenet just since I started typing! Quick, quick, do something! Do something! Save us! -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From gagsl-py2 at yahoo.com.ar Mon Jan 4 03:51:06 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 04 Jan 2010 05:51:06 -0300 Subject: How to Suppress Interactive Assignment to "_" References: <910af7d0-8aa7-43fa-b2c9-32e12dc2fbc9@m16g2000yqc.googlegroups.com> Message-ID: En Fri, 01 Jan 2010 15:53:31 -0300, JKPeck escribi?: > On Jan 1, 10:06 am, Peter Otten <__pete... at web.de> wrote: >> JKPeck wrote: >> > The gettext module uses the convention of defining a function named >> > "_" that maps text into its translation. >> > This conflicts with the automatic interactive interpreter assignment >> > of expressions to a variable with that same name. >> >> $ cat displayhook.py >> [...] > > Thanks. It's just what I needed. In case you didn't know, look at sitecustomize.py and the PYTHONSTARTUP [2] environment variable. They allow for the above code to be automatically executed. [1] http://docs.python.org/library/site.html#index-549 [2] http://docs.python.org/using/cmdline.html#envvar-PYTHONSTARTUP -- Gabriel Genellina From massimodipierro71 at gmail.com Mon Jan 4 03:59:39 2010 From: massimodipierro71 at gmail.com (mdipierro) Date: Mon, 4 Jan 2010 00:59:39 -0800 (PST) Subject: python xmlrpc client with ssl client certificates and standard modules References: <4b414e79$0$23321$426a34cc@news.free.fr> Message-ID: xmlrpc acts at the application layer and ssl at the transport layer so they can inter operate easily as long as you do not use the certificate to authenticate the client but only validate the server and encrypt data (which you can also do but it is more complicated) One option for you is to use web2py which include an xmlrpc server that uses a wsgi ssl enabled web server. Here is how: 1) Install web2py 2) Visit http://127.0.0.1:8000/admin and create a new application from the web based IDE 3) create your web service for example, in a controller default.py @service.xmlrpc def add(a,b): return int(a)+int(b) 4) Restart web2py with python web2py.py -a ADMIN_PASSWD -c SSL_CERTIFICATE -k SSL_PRIVATE_KEY -i 0.0.0.0 -p 443 5) You can now access the service from any Python program: >>> import xmlrpclib >>> server_url = 'https://myserver:443/yourapp/default/call/ xmlrpc' >>> server = xmlrpclib.Server(server_url) >>> print server.add(3,4) 7 Hope this helps. On Jan 3, 8:12?pm, News123 wrote: > Hi, > > I was googling fot quite some time and was not really succesfull. > > I found one solution, which I will try soon. > It ishttp://www.cs.technion.ac.il/~danken/xmlrpc-ssl.html > (found inhttp://hamakor.org.il/pipermail/python-il/2008-February/000029.html) > > This will probably work, but it requires the module M2Crypto. > > In order to avoid installing M2Crypto an all hosts that want to run the > script I wondered, whether there is no other solution. > > I can do xmlrpc over ssl WITHOUT certificates with following code: > > import xmlrpclib > server_url = 'https://myserver' > server = xmlrpclib.Server(server_url); > > and I can perform a https get request WITH certificates with below snippet: > > import httplib > conn = httplib.HTTPSConnection( > ? ? ? ? HOSTNAME, > ? ? ? ? key_file = KEYFILE, > ? ? ? ? cert_file = CERTFILE > ) > conn.putrequest('GET', '/') > conn.endheaders() > response = conn.getresponse() > print response.read() > > I'm just lost of how to 'combine' both. > > Thanks in advance for any suggestions / hints > > N From gagsl-py2 at yahoo.com.ar Mon Jan 4 04:27:48 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 04 Jan 2010 06:27:48 -0300 Subject: TypeError: can only concatenate list (not "tuple") to list References: <1992.68.59.29.80.1262593496.squirrel@www.bibliobazaar.com> Message-ID: En Mon, 04 Jan 2010 05:24:56 -0300, David Williams escribi?: >> py> [1,2,3] + (4,5) >> Traceback (most recent call last): >> File "", line 1, in >> TypeError: can only concatenate list (not "tuple") to list >> >> In-place addition += does work: >> >> py> a = [1,2,3] >> py> a += (4,5) >> py> a >> [1, 2, 3, 4, 5] > > I guess to expand a bit more on what I said... What should the result > be? > A list or a tuple? The reason += works is because the end result is > clear; a list. But it is ambiguous in the case of concatenation: did you > want a tuple or a list? Uhm... it seems "obvious" to me that [1,2,3] + (4,5) should be [1,2,3,4,5]. A list. That's what I would expect, although I cannot explain why is it *so* obvious to me. Given that 2 + 3.5, and 'abc' + u'def' both return an instance of their right operand's type, I should probably revise my preconceptions... -- Gabriel Genellina From news123 at free.fr Mon Jan 4 04:38:58 2010 From: news123 at free.fr (News123) Date: Mon, 04 Jan 2010 10:38:58 +0100 Subject: python xmlrpc client with ssl client certificates and standard modules In-Reply-To: References: <4b414e79$0$23321$426a34cc@news.free.fr> Message-ID: <4b41b733$0$13831$426a74cc@news.free.fr> Thanks for your answer. I'll look at web2py. However web2py seems to address the xmlrpc server (at least in your example). The xmlrpc server application exists alerady and requires a client certificate. The client example doesn't seem to be using a certificate. So I'll be reading a little into web2py. bye N mdipierro wrote: > xmlrpc acts at the application layer and ssl at the transport layer so > they can inter operate easily as long as you do not use the > certificate to authenticate the client but only validate the server > and encrypt data (which you can also do but it is more complicated) > > One option for you is to use web2py which include an xmlrpc server > that uses a wsgi ssl enabled web server. > > Here is how: > > 1) Install web2py > 2) Visit http://127.0.0.1:8000/admin and create a new application from > the web based IDE > 3) create your web service for example, in a controller default.py > > @service.xmlrpc > def add(a,b): return int(a)+int(b) > > 4) Restart web2py with > > python web2py.py -a ADMIN_PASSWD -c SSL_CERTIFICATE -k > SSL_PRIVATE_KEY -i 0.0.0.0 -p 443 > > 5) You can now access the service from any Python program: > > >>> import xmlrpclib > >>> server_url = 'https://myserver:443/yourapp/default/call/ > xmlrpc' > >>> server = xmlrpclib.Server(server_url) > >>> print server.add(3,4) > 7 > > Hope this helps. > > > > On Jan 3, 8:12 pm, News123 wrote: >> Hi, >> >> I was googling fot quite some time and was not really succesfull. >> >> I found one solution, which I will try soon. >> It ishttp://www.cs.technion.ac.il/~danken/xmlrpc-ssl.html >> (found inhttp://hamakor.org.il/pipermail/python-il/2008-February/000029.html) >> >> This will probably work, but it requires the module M2Crypto. >> >> In order to avoid installing M2Crypto an all hosts that want to run the >> script I wondered, whether there is no other solution. >> >> I can do xmlrpc over ssl WITHOUT certificates with following code: >> >> import xmlrpclib >> server_url = 'https://myserver' >> server = xmlrpclib.Server(server_url); >> >> and I can perform a https get request WITH certificates with below snippet: >> >> import httplib >> conn = httplib.HTTPSConnection( >> HOSTNAME, >> key_file = KEYFILE, >> cert_file = CERTFILE >> ) >> conn.putrequest('GET', '/') >> conn.endheaders() >> response = conn.getresponse() >> print response.read() >> >> I'm just lost of how to 'combine' both. >> >> Thanks in advance for any suggestions / hints >> >> N > From gagsl-py2 at yahoo.com.ar Mon Jan 4 04:39:45 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 04 Jan 2010 06:39:45 -0300 Subject: TypeError: can only concatenate list (not "tuple") to list References: Message-ID: En Mon, 04 Jan 2010 05:22:44 -0300, Steven D'Aprano escribi?: > On Mon, 04 Jan 2010 04:59:02 -0300, Gabriel Genellina wrote: > >> Is there any reason for this error? Apart from "nobody cared to write >> the code" > > Yes, because such implicit conversions would be a bad idea. I'm slowly convincing myself that it was actually a bad idea... >> In-place addition += does work: >> >> py> a = [1,2,3] >> py> a += (4,5) >> py> a >> [1, 2, 3, 4, 5] > > I call that an impressive gotcha. I believe that is because in-place > addition of lists is implemented as functionally equivalent to the extend > method: > >>>> a += "abc" # same as a.extend("abc") >>>> a > [1, 2, 3, 4, 5, 'a', 'b', 'c'] >>>> a += {None: -1} >>>> a > [1, 2, 3, 4, 5, 'a', 'b', 'c', None] So += and extend are completely permissive - they slurp whatever comes from iterating their right operand. Totally unexpected in some cases, as in your examples above... -- Gabriel Genellina From steven at REMOVE.THIS.cybersource.com.au Mon Jan 4 04:45:46 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 04 Jan 2010 09:45:46 GMT Subject: TypeError: can only concatenate list (not "tuple") to list References: <1992.68.59.29.80.1262593496.squirrel@www.bibliobazaar.com> Message-ID: On Mon, 04 Jan 2010 06:27:48 -0300, Gabriel Genellina wrote: > En Mon, 04 Jan 2010 05:24:56 -0300, David Williams > escribi?: > >>> py> [1,2,3] + (4,5) >>> Traceback (most recent call last): >>> File "", line 1, in >>> TypeError: can only concatenate list (not "tuple") to list >>> >>> In-place addition += does work: >>> >>> py> a = [1,2,3] >>> py> a += (4,5) >>> py> a >>> [1, 2, 3, 4, 5] >> >> I guess to expand a bit more on what I said... What should the result >> be? >> A list or a tuple? The reason += works is because the end result is >> clear; a list. But it is ambiguous in the case of concatenation: did >> you want a tuple or a list? > > Uhm... it seems "obvious" to me that [1,2,3] + (4,5) should be > [1,2,3,4,5]. A list. That's what I would expect, although I cannot > explain why is it *so* obvious to me. If you can't explain it, it's probably a bad idea. Why should lists be privileged over tuples? How does it work? Do tuples automatically turn into lists, or does the left hand value over-ride the right hand value? In other words, would you expect: "1" + 1 = "11" 1 + "1" = 2 or would you expect: "1" + 1 = "11" 1 + "1" = "11" > Given that 2 + 3.5, and 'abc' + u'def' both return an instance of their > right operand's type, I should probably revise my preconceptions... Yes, you should be a little more careful in your tests. >>> 2 + 3.5 5.5 >>> 3.5 + 2 5.5 Nothing to do with left versus right. -- Steven From ben+python at benfinney.id.au Mon Jan 4 04:48:37 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 04 Jan 2010 20:48:37 +1100 Subject: twenty years ago Guido created Python References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> Message-ID: <87oclaxksq.fsf@benfinney.id.au> Daniel Fetchinson writes: > Go, quick, these threads are all around us! They are popping up > everywhere! Go, go, quick! There is another troll thread right there! I wouldn't characterise this thread as a ?troll thread?. If you think it is, that's fine I suppose. I'll continue to try improving the signal to noise ratio (modulo responses like this, sadly) by trying to emit more signal and less noise. -- \ ?Never do anything against conscience even if the state demands | `\ it.? ?Albert Einstein | _o__) | Ben Finney From tim.wintle at teamrubber.com Mon Jan 4 05:43:06 2010 From: tim.wintle at teamrubber.com (Tim Wintle) Date: Mon, 04 Jan 2010 10:43:06 +0000 Subject: ANN: Pymazon 0.1beta released. In-Reply-To: <7f014ea60912291044i58f9490amddfc6cf24bdb2a18@mail.gmail.com> References: <7f014ea60912291044i58f9490amddfc6cf24bdb2a18@mail.gmail.com> Message-ID: <1262601786.8156.2.camel@localhost> On Tue, 2009-12-29 at 19:44 +0100, Chris Colbert wrote: > I'm happy to announce the first beta release of Pymazon: a Python > implemented alternative to the Amazon mp3 downloader. > > Pymazon was created specifically to alleviate the issues surrounding > the Linux version of the Amazon mp3 downloader (though it should run > just fine in Windows too). Thanks! I've been complaining to them for not providing 64-bit binaries (or source) for ages! (Although hats off to them for even providing the number of linux variants they do support) From larry at hastings.org Mon Jan 4 06:09:32 2010 From: larry at hastings.org (Larry Hastings) Date: Mon, 04 Jan 2010 03:09:32 -0800 Subject: Best practices for simultaneously installed versioned packages? Message-ID: <4B41CC6C.3020808@hastings.org> I'm writing a package for Python 3--let's call it "spacegoblin". I fear someday I may need multiple versions installed and available simultaneously, even within one version of Python. So I want to plan ahead for that possibility. What would be the best way to allow this? Right now I install the package with the version in the name, like "spacegoblin_1_0" and "spacegoblin_1_1". But I have an uneasy sense that I'm doing something stupid... mainly because I've never seen anyone do this before. A coworker said they did it like "spacegoblin.1_0" and "spacegoblin.1_1" at a previous employer. That seems like an improvement, though not the "yes that's obviously right" answer I'm holding out for. Your thoughts? Thanks, /larry/ p.s. Before you ask: no, I don't want to use "virtualenv" for this. -------------- next part -------------- An HTML attachment was scrubbed... URL: From highcar at gmail.com Mon Jan 4 06:25:29 2010 From: highcar at gmail.com (elca) Date: Mon, 4 Jan 2010 03:25:29 -0800 (PST) Subject: python mechanize proxy support question In-Reply-To: <4ca2b83a-3b82-4d20-85a3-b6304f25fb81@21g2000yqj.googlegroups.com> References: <27009696.post@talk.nabble.com> <4ca2b83a-3b82-4d20-85a3-b6304f25fb81@21g2000yqj.googlegroups.com> Message-ID: <27011661.post@talk.nabble.com> alex23 wrote: > > On Jan 4, 5:42?pm, elca wrote: >> how can i add proxy support into my mechanize script? >> i was look for some reference , but not so much good hint from google. > > There are examples on using proxies with mechanize on the module's > home page: > http://wwwsearch.sourceforge.net/mechanize/ > -- > http://mail.python.org/mailman/listinfo/python-list > > HI.. that is only support mechanize.browser module.. actually im looking mechanize.urlopen method. thanks -- View this message in context: http://old.nabble.com/python-mechanize-proxy-support-question-tp27009696p27011661.html Sent from the Python - python-list mailing list archive at Nabble.com. From steve at holdenweb.com Mon Jan 4 06:49:57 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 04 Jan 2010 06:49:57 -0500 Subject: twenty years ago Guido created Python In-Reply-To: <87oclaxksq.fsf@benfinney.id.au> References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> <87oclaxksq.fsf@benfinney.id.au> Message-ID: Ben Finney wrote: > Daniel Fetchinson writes: > >> Go, quick, these threads are all around us! They are popping up >> everywhere! Go, go, quick! There is another troll thread right there! > > I wouldn't characterise this thread as a ?troll thread?. If you think it > is, that's fine I suppose. > > I'll continue to try improving the signal to noise ratio (modulo > responses like this, sadly) by trying to emit more signal and less > noise. > And I, for one, offer you my thanks for your efforts. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Mon Jan 4 06:54:37 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 04 Jan 2010 06:54:37 -0500 Subject: Significant whitespace In-Reply-To: References: <4B3E70FC.1010702@gmail.com> Message-ID: r0g wrote: > David Robinow wrote: >> On Sun, Jan 3, 2010 at 10:42 PM, Roy Smith wrote: >>> In article , >>> David Robinow wrote: >>> >>>> On Sun, Jan 3, 2010 at 8:09 PM, Tim Roberts wrote: >>>>> More than "not required", it was "not relevant". This led to one of the >>>>> most infamous programming blunders in the early days of the space program, >>>>> when one programmer accidentially typed a period instead of a comma >>>>> resulting in the loss of a satellite: >>>> Interesting story. Did you make it up? >>> It's a fairly well known story. >>> >>> http://catless.ncl.ac.uk/Risks/5.64.html#subj4.2 >> Sure. But the question is, "Who made it up?" >> http://en.wikiquote.org/wiki/Fortran >> >> Computer folklore has incorrectly attributed the loss of the Mariner 1 >> space probe to a syntax error in a Fortran program. For example, >> "Recall the first American space probe to Venus, reportedly lost >> because Fortran cannot recognize a missing comma in a DO statement?"[ > > > > Wow you're totally right! It was a missing UNDERSCORE not a DOT being > typed instead of a COMMA! We've been lied to ALL ALONG! What kind of > sickos would do this? Let's find this Tim character, flay him alive and > then go crown you the KING OF EVERYTHING for rescuing us from that > deadly nugget of misinformation! > > Honestly I might have believed those dot/comma LIES if it wasn't for > your noble selfless efforts to uncover the SHOCKING TRUTH. Imagine what > might have happened! The character in question being an UNDERSCORE > COMPLETELY CHANGES the meaning and point of the story. As for it being > LEFT OUT rather than TRANSPOSED, well, anyone can see how that TURNS THE > WHOLE STORY UPSIDE DOWN. > > Of course there are the naysayers who might claim absolute accuracy is > no big deal in stories that are essentially parables but they do not > understand the HARM that can come from not remembering really tiny, > almost inconsequential details. For what it's worth I DO NOT consider > you a tiresome pedant and I think your taunting, patronising style is > completely appropriate and proportional to Tim's sickening crimes > against fact. > > Roger. Personally I think the overall tone of this group would have been marginally improved if you had summoned the self-restraint to simply not make this posting. Fortunately it's still an unmoderated group, so you can continue to be as hostile and snarky as you like. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Mon Jan 4 06:57:05 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 04 Jan 2010 06:57:05 -0500 Subject: Question about an application In-Reply-To: <27009252.post@talk.nabble.com> References: <27009118.post@talk.nabble.com> <9A7F3B4C-5621-44CF-ABCA-D440E7ADA01E@gmail.com> <27009252.post@talk.nabble.com> Message-ID: rieh25 wrote: [top-posting corrected] >> On Jan 4, 2010, at 12:50 AM, rieh25 wrote: >> >>> I am thinking of installing a python webserver I coded in every >>> computer at >>> my work. This would allow me to run specific tasks in them, like >>> creating >>> backups, installing things, etc. Is there another way to run >>> programs in >>> remote computers? Thanks for your opinions. > > Rodrick Brown wrote: >> Take a look at ssh >> >> Sent from my iPhone 3GS. > Thanks, I had forgotten about it. I'll investigate if there are ways to > automate using it to run programs in several computers at the same time. > You might also want to investigate Fabric, which was put together for such purposes. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From jeanmichel at sequans.com Mon Jan 4 08:36:25 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 04 Jan 2010 14:36:25 +0100 Subject: TypeError: can only concatenate list (not "tuple") to list In-Reply-To: References: <1992.68.59.29.80.1262593496.squirrel@www.bibliobazaar.com> Message-ID: <4B41EED9.4040506@sequans.com> Gabriel Genellina wrote: > En Mon, 04 Jan 2010 05:24:56 -0300, David Williams > escribi?: > >>> py> [1,2,3] + (4,5) >>> Traceback (most recent call last): >>> File "", line 1, in >>> TypeError: can only concatenate list (not "tuple") to list >>> >>> In-place addition += does work: >>> >>> py> a = [1,2,3] >>> py> a += (4,5) >>> py> a >>> [1, 2, 3, 4, 5] >> >> I guess to expand a bit more on what I said... What should the >> result be? >> A list or a tuple? The reason += works is because the end result is >> clear; a list. But it is ambiguous in the case of concatenation: did >> you >> want a tuple or a list? > > Uhm... it seems "obvious" to me that [1,2,3] + (4,5) should be > [1,2,3,4,5]. A list. That's what I would expect, although I cannot > explain why is it *so* obvious to me. > Given that 2 + 3.5, and 'abc' + u'def' both return an instance of > their right operand's type, I should probably revise my preconceptions... > Haa... The well known 'obviousness theorem' :o) by which everything that I say becomes true. Best theorem ever. I'm really surprised that s = [1,2,3] ; s += (4,5) works fine. Semantically, it is adding 2 operands of different type, this is not very smart. As variales have no predefined type and can change over statements, it would be unwise to assume that s += (4,5) should produce a list. JM From fetchinson at googlemail.com Mon Jan 4 08:51:42 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Mon, 4 Jan 2010 14:51:42 +0100 Subject: twenty years ago Guido created Python In-Reply-To: References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> <87oclaxksq.fsf@benfinney.id.au> Message-ID: >>> Go, quick, these threads are all around us! They are popping up >>> everywhere! Go, go, quick! There is another troll thread right there! >> >> I wouldn't characterise this thread as a ?troll thread?. If you think it >> is, that's fine I suppose. >> >> I'll continue to try improving the signal to noise ratio (modulo >> responses like this, sadly) by trying to emit more signal and less >> noise. >> > And I, for one, offer you my thanks for your efforts. Me too. But Ben will not get the Best Sense of Humor Award of 2010 for sure :) Although who knows, there is a full year ahead and things can change ........ Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From sccolbert at gmail.com Mon Jan 4 08:52:16 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Mon, 4 Jan 2010 14:52:16 +0100 Subject: ANN: Pymazon 0.1beta released. In-Reply-To: <1262601786.8156.2.camel@localhost> References: <7f014ea60912291044i58f9490amddfc6cf24bdb2a18@mail.gmail.com> <1262601786.8156.2.camel@localhost> Message-ID: <7f014ea61001040552x5c7ae3fbg844386245e2c1707@mail.gmail.com> On Mon, Jan 4, 2010 at 11:43 AM, Tim Wintle wrote: > On Tue, 2009-12-29 at 19:44 +0100, Chris Colbert wrote: > > I'm happy to announce the first beta release of Pymazon: a Python > > implemented alternative to the Amazon mp3 downloader. > > > > Pymazon was created specifically to alleviate the issues surrounding > > the Linux version of the Amazon mp3 downloader (though it should run > > just fine in Windows too). > > Thanks! > > I've been complaining to them for not providing 64-bit binaries (or > source) for ages! (Although hats off to them for even providing the > number of linux variants they do support) > Youre completely right, and my language on the google code site was a little harsh towards Amazon in that respect. I've fixed that now. Cheers! -------------- next part -------------- An HTML attachment was scrubbed... URL: From massimodipierro71 at gmail.com Mon Jan 4 09:05:32 2010 From: massimodipierro71 at gmail.com (mdipierro) Date: Mon, 4 Jan 2010 06:05:32 -0800 (PST) Subject: python xmlrpc client with ssl client certificates and standard modules References: <4b414e79$0$23321$426a34cc@news.free.fr> <4b41b733$0$13831$426a74cc@news.free.fr> Message-ID: If it is a client problem than web2py will be on help. If your server is written already you may be able to use it with the ssl cherrypy wsgi server (the one that web2py uses) and you do not need web2py at all. Massimo On Jan 4, 3:38?am, News123 wrote: > Thanks for your answer. > > I'll look at web2py. > > However web2py seems to address the xmlrpc server (at least in your > example). The xmlrpc server application exists alerady and requires a > client certificate. > > The client example doesn't seem to be using a certificate. > > So I'll be reading a little into web2py. > > bye > > N > > mdipierro wrote: > > xmlrpc acts at the application layer and ssl at the transport layer so > > they can inter operate easily as long as you do not use the > > certificate to authenticate the client but only validate the server > > and encrypt data (which you can also do but it is more complicated) > > > One option for you is to use web2py which include an xmlrpc server > > that uses a wsgi ssl enabled web server. > > > Here is how: > > > 1) Install web2py > > 2) Visithttp://127.0.0.1:8000/adminand create a new application from > > the web based IDE > > 3) create your web service for example, in a controller default.py > > > ? ? ?@service.xmlrpc > > ? ? ?def add(a,b): return int(a)+int(b) > > > 4) Restart web2py with > > > ? ? ?python web2py.py -a ADMIN_PASSWD -c SSL_CERTIFICATE -k > > SSL_PRIVATE_KEY -i 0.0.0.0 -p 443 > > > 5) You can now access the service from any Python program: > > > ? ? ?>>> import xmlrpclib > > ? ? ?>>> server_url = 'https://myserver:443/yourapp/default/call/ > > xmlrpc' > > ? ? ?>>> server = xmlrpclib.Server(server_url) > > ? ? ?>>> print server.add(3,4) > > ? ? ?7 > > > Hope this helps. > > > On Jan 3, 8:12 pm, News123 wrote: > >> Hi, > > >> I was googling fot quite some time and was not really succesfull. > > >> I found one solution, which I will try soon. > >> It ishttp://www.cs.technion.ac.il/~danken/xmlrpc-ssl.html > >> (found inhttp://hamakor.org.il/pipermail/python-il/2008-February/000029.html) > > >> This will probably work, but it requires the module M2Crypto. > > >> In order to avoid installing M2Crypto an all hosts that want to run the > >> script I wondered, whether there is no other solution. > > >> I can do xmlrpc over ssl WITHOUT certificates with following code: > > >> import xmlrpclib > >> server_url = 'https://myserver' > >> server = xmlrpclib.Server(server_url); > > >> and I can perform a https get request WITH certificates with below snippet: > > >> import httplib > >> conn = httplib.HTTPSConnection( > >> ? ? ? ? HOSTNAME, > >> ? ? ? ? key_file = KEYFILE, > >> ? ? ? ? cert_file = CERTFILE > >> ) > >> conn.putrequest('GET', '/') > >> conn.endheaders() > >> response = conn.getresponse() > >> print response.read() > > >> I'm just lost of how to 'combine' both. > > >> Thanks in advance for any suggestions / hints > > >> N > > From alfps at start.no Mon Jan 4 10:17:12 2010 From: alfps at start.no (Alf P. Steinbach) Date: Mon, 04 Jan 2010 16:17:12 +0100 Subject: How do you configure IDLE on a Mac, seeing as there's no Configure Option... In-Reply-To: References: Message-ID: * Mensanator: > ...because there's no [Options] menu on the shell window? > > Or at least give me a clue to how to use Courier New font? > > For some inscrutable reason, depite the plethora of formatting tools, > someone decided that proportional spaced fonts ought to be the > default for IDLE. Why not just use the Python interpreter, a reasonable programmer's editor and, if you need it, some debugger. The source code level debugging in IDLE is nothing to write home about. In Windows XP it is just silly: it marks the current line by ordinary text selection, which (depending on your config, but I think this is default) is invisible when you're looking at the debugger window for single stepping. You can see it by switching back to the source code window but that's annoying, not very practical. Invisible highlighting of the current line, he he... Cheers & hth., - Alf From nobody at nowhere.com Mon Jan 4 10:46:57 2010 From: nobody at nowhere.com (Nobody) Date: Mon, 04 Jan 2010 15:46:57 +0000 Subject: IOError - cannot create file (linux daemon-invoked script) References: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> <40b81356-f0c1-4249-a536-916356377afa@u41g2000yqe.googlegroups.com> Message-ID: On Sun, 03 Jan 2010 13:56:24 -0800, cassiope wrote: > I'm changing the uid and gid in the daemon (which runs with root > permissions > until the fork and uid/gid change). The uid and gid are confirmed by > printing os.getuid() and os.getgid() in the script. Those tell you the *real* UID/GID. Filesystem checks are performed using the *effective* UID/GID. You need to set them with seteuid/setegid and check them with geteuid/getegid. From benjamin.kaplan at case.edu Mon Jan 4 11:05:00 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Mon, 4 Jan 2010 11:05:00 -0500 Subject: How do you configure IDLE on a Mac, seeing as there's no Configure Option... In-Reply-To: References: Message-ID: On Sun, Jan 3, 2010 at 8:42 PM, Mensanator wrote: > ...because there's no [Options] menu on the shell window? > > Or at least give me a clue to how to use Courier New font? > > For some inscrutable reason, depite the plethora of formatting tools, > someone decided that proportional spaced fonts ought to be the > default for IDLE. > -- > http://mail.python.org/mailman/listinfo/python-list > Mac Applications don't have "options" menus. They have a "preferences" pane which is the second item in the application menu (the one that takes the name of the active program) and has a keyboard shortcut of Cmd-, Apple's user-interface guidelines are pretty specific, so you'll find that this works for all native OS X (i.e. not X11) applications. From briandenzer at gmail.com Mon Jan 4 11:09:56 2010 From: briandenzer at gmail.com (Brian D) Date: Mon, 4 Jan 2010 08:09:56 -0800 (PST) Subject: fsync() doesn't work as advertised? Message-ID: If I'm running a process in a loop that runs for a long time, I occasionally would like to look at a log to see how it's going. I know about the logging module, and may yet decide to use that. Still, I'm troubled by how fsync() doesn't seem to work as advertised: http://docs.python.org/library/os.html "If you?re starting with a Python file object f, first do f.flush(), and then do os.fsync(f.fileno())" Has anyone else experienced and overcome this issue? What I've seen is that flush() alone produces a complete log when the loop finishes. When I used fsync(), I lost all of the write entries except the first, along with odd error trap and the last entry. From benjamin.kaplan at case.edu Mon Jan 4 11:16:39 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Mon, 4 Jan 2010 11:16:39 -0500 Subject: Best practices for simultaneously installed versioned packages? In-Reply-To: <4B41CC6C.3020808@hastings.org> References: <4B41CC6C.3020808@hastings.org> Message-ID: On Mon, Jan 4, 2010 at 6:09 AM, Larry Hastings wrote: > > I'm writing a package for Python 3--let's call it "spacegoblin". I fear > someday I may need multiple versions installed and available simultaneously, > even within one version of Python. So I want to plan ahead for that > possibility. What would be the best way to allow this? Right now I install > the package with the version in the name, like "spacegoblin_1_0" and > "spacegoblin_1_1". But I have an uneasy sense that I'm doing something > stupid... mainly because I've never seen anyone do this before. > > A coworker said they did it like "spacegoblin.1_0" and "spacegoblin.1_1" at > a previous employer. That seems like an improvement, though not the "yes > that's obviously right" answer I'm holding out for. Your thoughts? > > Thanks, > > > /larry/ > > p.s. Before you ask: no, I don't want to use "virtualenv" for this. > wxpython installs a "wxversion" module which has functions like getInstalled(), ensureMinimal(version), and select(version). You can call wxversion.select before importing wx and it will make sure that the correct version is imported. You might want to look up what they did. From me4 at privacy.net Mon Jan 4 11:22:42 2010 From: me4 at privacy.net (=?iso-8859-1?Q?Jens_M=FCller?=) Date: Mon, 4 Jan 2010 17:22:42 +0100 Subject: Speeding up network access: threading? Message-ID: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> Hello, what would be best practise for speeding up a larger number of http-get requests done via urllib? Until now they are made in sequence, each request taking up to one second. The results must be merged into a list, while the original sequence needs not to be kept. I think speed could be improved by parallizing. One could use multiple threads. Are there any python best practises, or even existing modules, for creating and handling a task queue with a fixed number of concurrent threads? Thanks and regards! From solipsis at pitrou.net Mon Jan 4 11:29:45 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 4 Jan 2010 16:29:45 +0000 (UTC) Subject: fsync() doesn't work as advertised? References: Message-ID: Le Mon, 04 Jan 2010 08:09:56 -0800, Brian D a ?crit?: > > What I've seen is that flush() alone produces a complete log when the > loop finishes. When I used fsync(), I lost all of the write entries > except the first, along with odd error trap and the last entry. Perhaps you are writing to the file from several threads or processes at once? By the way, you shouldn't need fsync() if you merely want to look at the log files, because your OS will have an up-to-date view of the file contents anyway. fsync() is useful if you want to be sure the data has been written to the hard disk drive, rather than just kept in the operating system's filesystem cache. From steve at holdenweb.com Mon Jan 4 11:48:18 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 04 Jan 2010 11:48:18 -0500 Subject: How do you configure IDLE on a Mac, seeing as there's no Configure Option... In-Reply-To: References: Message-ID: Alf P. Steinbach wrote: > * Mensanator: >> ...because there's no [Options] menu on the shell window? >> >> Or at least give me a clue to how to use Courier New font? >> >> For some inscrutable reason, depite the plethora of formatting tools, >> someone decided that proportional spaced fonts ought to be the >> default for IDLE. Yeah, that was a smart move, wasn't it? > > Why not just use the Python interpreter, a reasonable programmer's > editor and, if you need it, some debugger. > > The source code level debugging in IDLE is nothing to write home about. > I quite agree. > In Windows XP it is just silly: it marks the current line by ordinary > text selection, which (depending on your config, but I think this is > default) is invisible when you're looking at the debugger window for > single stepping. You can see it by switching back to the source code > window but that's annoying, not very practical. Invisible highlighting > of the current line, he he... > I remember being extremely surprised to read that a Python developer I greatly admire uses IDLE as his main development interface. Frankly I have always found IDLE extremely non-intuitive, and I am constantly frustrated at having to explain about its shortcomings and oddities to newcomers in my "Introduction to Python" classes. Who actually maintains IDLE nowadays? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From jjposner at optimum.net Mon Jan 4 11:49:15 2010 From: jjposner at optimum.net (John Posner) Date: Mon, 04 Jan 2010 11:49:15 -0500 Subject: Dynamic text color References: Message-ID: On Fri, 01 Jan 2010 21:01:04 -0500, Cousin Stanley wrote: > > I was not familiar with the re.finditer method > for searching strings ... Stanley and Dave -- So far, we've just been using finditer() to perform standard-string searches (e.g. on the word "red"). Since Dave now wants to color multiple words the same color (e.g. the words in redList), we can use a single regular-expression search to locate *all* the words in a list. This eliminates the need to use a "for" loop to handle the list. Here's what I mean: >>> import re >>> s = "it is neither red nor crimson, but scarlet, you see" ########## individual searches >>> [matchobj.span() for matchobj in re.finditer("red", s)] [(14, 17)] >>> [matchobj.span() for matchobj in re.finditer("crimson", s)] [(22, 29)] >>> [matchobj.span() for matchobj in re.finditer("scarlet", s)] [(35, 42)] ########## one "swell foop" >>> redList = "red crimson scarlet".split() >>> redList_regexp = "|".join(redList) >>> redList_regexp 'red|crimson|scarlet' >>> [matchobj.span() for matchobj in re.finditer(redList_regexp, s)] [(14, 17), (22, 29), (35, 42)] -John From exarkun at twistedmatrix.com Mon Jan 4 11:52:28 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Mon, 04 Jan 2010 16:52:28 -0000 Subject: Speeding up network access: threading? In-Reply-To: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> References: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <20100104165228.15596.233351734.divmod.xquotient.816@localhost.localdomain> On 04:22 pm, me4 at privacy.net wrote: >Hello, > >what would be best practise for speeding up a larger number of http-get >requests done via urllib? Until now they are made in sequence, each >request taking up to one second. The results must be merged into a >list, while the original sequence needs not to be kept. > >I think speed could be improved by parallizing. One could use multiple >threads. >Are there any python best practises, or even existing modules, for >creating and handling a task queue with a fixed number of concurrent >threads? Using multiple threads is one approach. There are a few thread pool implementations lying about; one is part of Twisted, . Another approach is to use non-blocking or asynchronous I/O to make multiple requests without using multiple threads. Twisted can help you out with this, too. There's two async HTTP client APIs available. The older one: http://twistedmatrix.com/documents/current/api/twisted.web.client.getPage.html http://twistedmatrix.com/documents/current/api/twisted.web.client.HTTPClientFactory.html And the newer one, introduced in 9.0: http://twistedmatrix.com/documents/current/api/twisted.web.client.Agent.html Jean-Paul From fpm at u.washington.edu Mon Jan 4 12:16:52 2010 From: fpm at u.washington.edu (cassiope) Date: Mon, 4 Jan 2010 09:16:52 -0800 (PST) Subject: IOError - cannot create file (linux daemon-invoked script) References: <7804e895-e543-4ac7-b3f3-2c06c716394d@j4g2000yqe.googlegroups.com> <40b81356-f0c1-4249-a536-916356377afa@u41g2000yqe.googlegroups.com> Message-ID: On Jan 4, 7:46?am, Nobody wrote: > On Sun, 03 Jan 2010 13:56:24 -0800, cassiope wrote: > > I'm changing the uid and gid in the daemon (which runs with root > > permissions > > until the fork and uid/gid change). ?The uid and gid are confirmed by > > printing os.getuid() and os.getgid() in the script. > > Those tell you the *real* UID/GID. Filesystem checks are performed using > the *effective* UID/GID. > > You need to set them with seteuid/setegid and check them with > geteuid/getegid. To Cameron: the file doesn't (yet) exist; and it has the correct full path. To "Nobody" : hey, this seems interesting. First test, invoking seteuid() and setegid() didn't help - but strange thing was these calls didn't show up in the strace, so perhaps I wasn't testing what I thought I was. I'll have to check this later as people are now back to work and needing the daemon for its other functions... Thanks for your ideas! I'll be checking it more later... From tjreedy at udel.edu Mon Jan 4 12:17:46 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 04 Jan 2010 12:17:46 -0500 Subject: Speeding up network access: threading? In-Reply-To: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> References: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> Message-ID: On 1/4/2010 11:22 AM, Jens M?ller wrote: > Hello, > > what would be best practise for speeding up a larger number of http-get > requests done via urllib? Until now they are made in sequence, each > request taking up to one second. The results must be merged into a list, > while the original sequence needs not to be kept. > > I think speed could be improved by parallizing. One could use multiple > threads. > Are there any python best practises, or even existing modules, for > creating and handling a task queue with a fixed number of concurrent > threads? I believe code of this type has been published here in various threads. The fairly obvious thing to do is use a queue.queue for tasks and another for results and a pool of threads that read, fetch, and write. From louisrom1 at gmail.com Mon Jan 4 12:24:22 2010 From: louisrom1 at gmail.com (louisJ) Date: Mon, 4 Jan 2010 09:24:22 -0800 (PST) Subject: No module named _socket, on windows Message-ID: Hi I installed python 2.6 (from python.org) for windows XP, and then Pylab. When I type "import pylab" in a python shell it shows the error: ImportError: No module named _socket Any idea what can I do to make this work? thank you From victorsubervi at gmail.com Mon Jan 4 12:28:13 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Mon, 4 Jan 2010 13:28:13 -0400 Subject: Can't Add Variable Message-ID: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> Hi; I have this code snippet: sql '''create table if not exists %sCustomerData ( ID tinyint(8) unsigned primary key auto_increment, Email varchar(120) not null, PhoneNumber varchar(20) not null, BillingName varchar(80) not null, BillingAddress1 varchar(100) not null, BillingAddress2 varchar(100) null, BillingCity varchar(50) not null, BillingState varchar(2) not null, BillingPostalCode varchar(8) not null, ShippingName varchar(80) not null, ShippingAddress1 varchar(100) not null, ShippingAddress2 varchar(100) null, ShippingCity varchar(50) not null, ShippingState varchar(2) not null, ShippingPostalCode varchar(8) not null, TypeOfCard set("VISA", "MasterCard", "AmEx", "Discover", "PayPal") null, CreditCardNumber tinyint(8) unsigned null, CreditCardExpMonth tinyint(2) unsigned null, CreditCardExpYear tinyint(4) unsigned null );''' % store It throws this error: [Mon Jan 04 09:23:54 2010] [error] [client 66.82.9.61] File "/var/www/html/angrynates.com/cart/cart2.py", line 62, referer: http://angrynates.com/cart/cart.py [Mon Jan 04 09:23:54 2010] [error] [client 66.82.9.61] );''' % store, referer: http://angrynates.com/cart/cart.py [Mon Jan 04 09:23:54 2010] [error] [client 66.82.9.61] ^, referer: http://angrynates.com/cart/cart.py [Mon Jan 04 09:23:54 2010] [error] [client 66.82.9.61] SyntaxError: invalid syntax, referer: http://angrynates.com/cart/cart.py [Mon Jan 04 09:23:54 2010] [error] [client 66.82.9.61] Premature end of script headers: cart2.py, referer: http://angrynates.com/cart/cart.py Variable "store" is defined. How is my syntax invalid? Total code follows. TIA, beno #! /usr/bin/python import string import cgitb; cgitb.enable() import MySQLdb import cgi import sys,os sys.path.append(os.getcwd()) from login import login import datetime, Cookie, random from particulars import title, myCookie, ourOptions, whereToShip, paymentOptionsPickup, paymentOptionsFax, paymentGateways, VISA, masterCard, amEx, discover, payPal from templateFrame import top, bottom import time import fpformat from sets import Set ourURL = string.split(__file__, 'html/') ourURL = string.split(ourFile[1], '/')[0] def commitSale(): user, passwd, db, host = login() database = MySQLdb.connect(host, user, passwd, db) cursor= database.cursor() ourEmail = email() form = cgi.FieldStorage() which = form.getfirst('which') store = form.getfirst('store') patientID = form.getfirst('patientID') customerLoginEmail = form.getfirst('customerEmail') # This is for customers who lost their login info customerLoginFirstName = form.getfirst('customerLoginFirstName') customerLoginLastName = form.getfirst('customerLoginLastName') customerLoginData = [patientID, customerLoginEmail, customerLoginFirstName, customerLoginLastName] if (store == 'prescriptions') and ((customerLoginEmail is not None) or (customerLoginFirstName is not None)): myMail(which, store, '', ourURL, '', '', customerLoginData) elif which == 'order': form = cgi.FieldStorage() tmpTable = form.getfirst('tmpTable') howPay = form.getfirst('howPay') ccNumber = form.getfirst('creditCardNumber') ccFirstHalf = ccNumber[0:8] ccSecondHalf = ccNumber[8:] sql '''create table if not exists %sCustomerData ( ID tinyint(8) unsigned primary key auto_increment, Email varchar(120) not null, PhoneNumber varchar(20) not null, BillingName varchar(80) not null, BillingAddress1 varchar(100) not null, BillingAddress2 varchar(100) null, BillingCity varchar(50) not null, BillingState varchar(2) not null, BillingPostalCode varchar(8) not null, ShippingName varchar(80) not null, ShippingAddress1 varchar(100) not null, ShippingAddress2 varchar(100) null, ShippingCity varchar(50) not null, ShippingState varchar(2) not null, ShippingPostalCode varchar(8) not null, TypeOfCard set("VISA", "MasterCard", "AmEx", "Discover", "PayPal") null, CreditCardNumber tinyint(8) unsigned null, CreditCardExpMonth tinyint(2) unsigned null, CreditCardExpYear tinyint(4) unsigned null );''' % store print sql # cursor.execute(sql) sql = '''insert into %sCustomerData (Email, PhoneNumber, BillingName, BillingAddress1, BillingAddress2, BillingCity, BillingState, BillingPostalCode, ShippingName, ShippingAddress1, ShippingAddress2, ShippingCity, ShippingState, ShippingPostalCode, TypeOfCard, CreditCardNumber, CreditCardExpMonth, CreditCardExpYear) value ("%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s", "%s");' % (store, form.getfirst('customerEmail'), form.getfirst('customerPhoneNumber'), form.getfirst('customerBillingName'), form.getfirst('customerBillingAddr1'), form.getfirst('customerBillingAddr2'), form.getfirst('customerBillingCity'), form.getfirst('customerBillingState'), form.getfirst('customerBillingPostalCode'), form.getfirst('customerShippingName'), form.getfirst('shippingSameAsShipping'), form.getfirst('customerShippingAddr1'), form.getfirst('customerShippingAddr2'), form.getfirst('customerShippingCity'), form.getfirst('customerShippingState'), form.getfirst('customerShippingPostalCode'), form.getfirst('typeOfCard'), ccFirstHalf, form.getfirst('creditCartCVC')) print sql # cursor.execute(sql) db.commit() ccFirstHalf = None cursor.execute('select max(ID) from %sCustomerData;' % store) custID = cursor.fetchone()[0] myMail(which, store, BillingName, ourURL, tmpTable, ccSecondHalf) cursor.execute('describe %s;' % tmpTable) fields = [itm[0] for itm in cursor] cursor.execute('select * from %s;' % tmpTable) order = cursor.fetchall() allValues = [] for item in order: itemValues = [] i = 0 while i < len(fields): itemValues(item[i]) i += 1 allValues.append(itemValues) order = '' for itemValues in allValues: i = 0 for item in itemValues: order += '%s: %s\t' % (fields[i], item) i += 1 order += '\n' order += 'Total: %s\n' % form.getfirst('total') def cart2(): user, passwd, db, host = login() db = MySQLdb.connect(host, user, passwd, db) cursor= db.cursor() print '''Content-Type: text/html\r\n ''' head = '' form = cgi.FieldStorage() numberOfProducts = int(form.getfirst('numberOfProducts')) total = form.getfirst('total') store = form.getfirst('store') i = 0 html = '' deleteThese = [] editThis = 0 quantity = 0 tmpTable = form.getfirst('tmpTable') while i < numberOfProducts: id = int(form.getfirst('id%d' % i)) if form.getfirst('delete%d' % i) is not None: deleteThese.append(form.getfirst('delete%d' % i)) if form.getfirst('edit%d' % i) is not None: editThis = int(form.getfirst('edit%d' % i)) quantity = form.getfirst('quantity%d' % i) id = form.getfirst('quantityID%d' % i) i += 1 i = 0 while i < len(deleteThese): sql = 'delete from %s where ProdID=%s;' % (tmpTable, deleteThese[i]) html += sql cursor.execute(sql) head = "" % store i += 1 if editThis != 0: sql = 'update %s set Quantity=%s where ProdID=%s;' % (tmpTable, quantity, id) html += sql cursor.execute(sql) head = "" % (store, quantity) if head == '': paymentMethods = [] cardTypes = [] for paymentStore, paymentType in paymentOptionsPickup().iteritems(): if (paymentStore == store) and (paymentType == 'on'): paymentMethods.append('pickup') for paymentStore, paymentType in paymentOptionsFax().iteritems(): if (paymentStore == store) and (paymentType == 'on'): paymentMethods.append('fax') for cardStore, cardType in VISA().iteritems(): if (cardStore == store) and (cardType == 'on'): cardTypes.append('VISA') for cardStore, cardType in masterCard().iteritems(): if (cardStore == store) and (cardType == 'on'): cardTypes.append('MasterCard') for cardStore, cardType in amEx().iteritems(): if (cardStore == store) and (cardType == 'on'): cardTypes.append('American Express') for cardStore, cardType in discover().iteritems(): if (cardStore == store) and (cardType == 'on'): cardTypes.append('Discover') for cardStore, cardType in payPal().iteritems(): if (cardStore == store) and (cardType == 'on'): cardTypes.append('PayPal') print '' print "" % store print "" % total print "" print "" % tmpTable print '''
''' if store != 'prescriptions': print ''' ''' else: patientID = form.getfirst('patientID') sql = 'select FirstName, LastName from patientsPersonalData where ID=%s;' % patientID cursor.execute(sql) print "%s, please tell us how you would like to pay for this order?" % (cursor.fetchone()[0] + ' ' + cursor.fetchone()[1]) PRINT PAYMENT OPTION STUFF HERE print "If you would like to pay by credit card, please fill in the following:" print ''' PRINT CC INFO HERE
Billing Information
Name:
Address 1:
Address 2:
City:
State:
Postal Code
Email Address
Phone Number
Shipping Information Same as billing information?
Shipping Information
Name:
Address 1:
Address 2:
City:
State:
Postal Code
Credit Card Information
Type of Card:
Credit Card Number:
CVC Code:
Expiration Date: Month: Year:
Method of Payment
Pay by Credit Card: ''' for method in paymentMethods: print "   Pay by %s: " % (method[0].upper() + method[1:], method) print '''
''' commitSale() else: print head print '' print '' # print html print '' cart2() -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From goon12 at gmail.com Mon Jan 4 12:40:13 2010 From: goon12 at gmail.com (Joe Riopel) Date: Mon, 4 Jan 2010 12:40:13 -0500 Subject: Can't Add Variable In-Reply-To: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> References: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> Message-ID: <6a2ccd191001040940v69ffc4bav153971cd39febc01@mail.gmail.com> On Mon, Jan 4, 2010 at 12:28 PM, Victor Subervi wrote: > Hi; > I have this code snippet: > > ??? sql '''create table if not exists %sCustomerData ( I think you may have forgotten the "=" after sql. From apt.shansen at gmail.com Mon Jan 4 12:40:18 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Mon, 4 Jan 2010 09:40:18 -0800 Subject: Can't Add Variable In-Reply-To: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> References: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> Message-ID: <7a9c25c21001040940s6e1b09dcg38d4f02c0871c361@mail.gmail.com> On Mon, Jan 4, 2010 at 9:28 AM, Victor Subervi wrote: > Hi; > I have this code snippet: > > sql '''create table if not exists %sCustomerData ( > You left out the actual assignment operator. --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From clp2 at rebertia.com Mon Jan 4 12:42:26 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Mon, 4 Jan 2010 09:42:26 -0800 Subject: Can't Add Variable In-Reply-To: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> References: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> Message-ID: <50697b2c1001040942y4f20e7f8xac6a29e195d535c@mail.gmail.com> On Mon, Jan 4, 2010 at 9:28 AM, Victor Subervi wrote: > Hi; > I have this code snippet: > > ??? sql '''create table if not exists %sCustomerData ( You're missing an equal sign there to start with (i.e. sql = '''). Cheers, Chris -- http://blog.rebertia.com From clp2 at rebertia.com Mon Jan 4 12:47:06 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Mon, 4 Jan 2010 09:47:06 -0800 Subject: How to validate the __init__ parameters In-Reply-To: References: <803c4db7-3813-448a-b428-0ec11c57f098@t42g2000yqd.googlegroups.com> <00b98b9d$0$15623$c3e8da3@news.astraweb.com> Message-ID: <50697b2c1001040947w342466d6rdc0d1c72c22e7919@mail.gmail.com> On Mon, Jan 4, 2010 at 10:17 AM, Albert van der Horst wrote: > This triggers a question: I can see the traceback, but it > would be much more valuable, if I could see the arguments > passed to the functions. Is there a tool? print(locals()) #this actually gives the bindings for the entire local namespace #but is less work than writing something more precise by hand. It is amazing how handy the humble print() function/statement is when debugging. Cheers, Chris -- http://blog.rebertia.com From python at mrabarnett.plus.com Mon Jan 4 12:55:09 2010 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 04 Jan 2010 17:55:09 +0000 Subject: Can't Add Variable In-Reply-To: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> References: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> Message-ID: <4B422B7D.7010001@mrabarnett.plus.com> Victor Subervi wrote: > Hi; > I have this code snippet: > > sql '''create table if not exists %sCustomerData ( > ID tinyint(8) unsigned primary key auto_increment, > Email varchar(120) not null, > PhoneNumber varchar(20) not null, > BillingName varchar(80) not null, > BillingAddress1 varchar(100) not null, > BillingAddress2 varchar(100) null, > BillingCity varchar(50) not null, > BillingState varchar(2) not null, > BillingPostalCode varchar(8) not null, > ShippingName varchar(80) not null, > ShippingAddress1 varchar(100) not null, > ShippingAddress2 varchar(100) null, > ShippingCity varchar(50) not null, > ShippingState varchar(2) not null, > ShippingPostalCode varchar(8) not null, > TypeOfCard set("VISA", "MasterCard", "AmEx", "Discover", "PayPal") > null, > CreditCardNumber tinyint(8) unsigned null, > CreditCardExpMonth tinyint(2) unsigned null, > CreditCardExpYear tinyint(4) unsigned null > );''' % store > > It throws this error: > > [Mon Jan 04 09:23:54 2010] [error] [client 66.82.9.61] File > "/var/www/html/angrynates.com/cart/cart2.py > ", line 62, referer: > http://angrynates.com/cart/cart.py > [Mon Jan 04 09:23:54 2010] [error] [client 66.82.9.61] );''' % > store, referer: http://angrynates.com/cart/cart.py > [Mon Jan 04 09:23:54 2010] [error] [client 66.82.9.61] ^, > referer: http://angrynates.com/cart/cart.py > [Mon Jan 04 09:23:54 2010] [error] [client 66.82.9.61] SyntaxError: > invalid syntax, referer: http://angrynates.com/cart/cart.py > [Mon Jan 04 09:23:54 2010] [error] [client 66.82.9.61] Premature end of > script headers: cart2.py, referer: http://angrynates.com/cart/cart.py > > Variable "store" is defined. How is my syntax invalid? Total code follows. > If it was complaining that "store" wasn't defined, the exception would have been NameError. It's a SyntaxError because of the missing "=". From george.trojan at noaa.gov Mon Jan 4 12:58:31 2010 From: george.trojan at noaa.gov (George Trojan) Date: Mon, 04 Jan 2010 17:58:31 +0000 Subject: asyncore based port splitter code questions Message-ID: The following code is a attempt at port splitter: I want to forward data coming on tcp connection to several host/port addresses. It sort of works, but I am not happy with it. asyncore based code is supposed to be simple, but I need while loops and a lot of try/except clauses. Also, I had to add suspend/activate_channel methods in the Writer class that use variables with leading underscores. Otherwise the handle_write() method is called in a tight loop. I designed the code by looking at Python 2.3 source for asyncore and originally wanted to use add_channel() and del_channel() methods. However in Python 2.6 del_channel() closes the socket in addition to deleting it from the map. I do not want to have one connection per message, the traffic may be high and there are no message delimiters. The purpose of this exercise is to split incoming operational data so I can test a new version of software. Comments please - I have cognitive dissonance about the code, my little yellow rubber duck is of no help here. The code is run as: python2.6 afwdport.py 50002 localhost 50003 catbert 50001 where 50002 is the localhost incoming data port, (localhost, 50003) and (catbert, 50001) are destinations. George import asyncore, os, socket, sys, time TMOUT = 10 #---------------------------------------------------------------------- def log_msg(msg): print >> sys.stderr, '%s: %s' % (time.ctime(), msg) #---------------------------------------------------------------------- class Reader(asyncore.dispatcher): def __init__(self, sock, writers): asyncore.dispatcher.__init__(self, sock) self.writers = writers def handle_read(self): data = self.recv(1024) for writer in self.writers: writer.add_data(data) def handle_expt(self): self.handle_close() def handle_close(self): log_msg('closing reader connection') self.close() def writable(self): return False #---------------------------------------------------------------------- class Writer(asyncore.dispatcher): def __init__(self, address): asyncore.dispatcher.__init__(self) self.address = address self.data = '' self.mksocket() def suspend_channel(self, map=None): fd = self._fileno if map is None: map = self._map if fd in map: del map[fd] def activate_channel(self): if self._fileno not in self._map: self._map[self._fileno] = self def mksocket(self): self.create_socket(socket.AF_INET, socket.SOCK_STREAM) self.set_reuse_addr() self.connect(self.address) log_msg('connected to %s' % str(self.address)) def add_data(self, data): self.data += data self.activate_channel() def handle_write(self): while self.data: log_msg('sending data to %s' % str(self.address)) sent = self.send(self.data) self.data = self.data[sent:] self.suspend_channel() def handle_expt(self): err = self.socket.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR) log_msg(asyncore._strerror(err)) self.handle_close() def handle_close(self): log_msg('closing writer connection') self.close() # try to reconnect time.sleep(TMOUT) self.mksocket() def readable(self): return False #---------------------------------------------------------------------- class Dispatcher(asyncore.dispatcher): def __init__(self, port, destinations): asyncore.dispatcher.__init__(self) self.address = socket.gethostbyname(socket.gethostname()), port self.writers = [Writer(_) for _ in destinations] self.reader = None self.handle_connect() def handle_connect(self): self.create_socket(socket.AF_INET, socket.SOCK_STREAM) self.bind(self.address) self.listen(1) log_msg('listening on %s' % str(self.address)) def handle_accept(self): conn, addr = self.accept() log_msg('connection from %s' % str(addr)) # current read connection not closed for some reason if self.reader: self.reader.close() self.reader = Reader(conn, self.writers) def cleanup(self): try: if self.reader: self.reader.close() except socket.error, e: log_msg('error closing reader connection %s' % e) # writer might have unwatched connections for w in self.writers: try: w.close() except socket.error, e: log_msg('error closing writer connection %s' % e) #---------------------------------------------------------------------- def main(port, destinations): disp = None try: # asyncore.loop() exits when input connection closes while True: try: disp = Dispatcher(port, destinations) asyncore.loop(timeout=TMOUT, use_poll=True) except socket.error, (errno, e): if errno == 98: log_msg('sleeping %d s: %s', (30, e)) time.sleep(30) except BaseException, e: log_msg('terminating - uncaught exception: %s' % e) raise SystemExit finally: if disp: disp.cleanup() #---------------------------------------------------------------------- if __name__ == '__main__': nargs = len(sys.argv) try: assert nargs > 3 and nargs % 2 == 0 port = int(sys.argv[1]) destinations = [(sys.argv[n], int(sys.argv[n+1])) \ for n in range(2, nargs-1, 2)] main(port, destinations) except (AssertionError, ValueError), e: print 'Error: %s' % e print 'Usage: python %s local-port host port ...' % sys.argv[0] raise SystemExit(1) From victorsubervi at gmail.com Mon Jan 4 13:00:48 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Mon, 4 Jan 2010 14:00:48 -0400 Subject: Can't Add Variable In-Reply-To: <50697b2c1001040942y4f20e7f8xac6a29e195d535c@mail.gmail.com> References: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> <50697b2c1001040942y4f20e7f8xac6a29e195d535c@mail.gmail.com> Message-ID: <4dc0cfea1001041000rdf417a1g22d12917196a2192@mail.gmail.com> On Mon, Jan 4, 2010 at 1:42 PM, Chris Rebert wrote: > On Mon, Jan 4, 2010 at 9:28 AM, Victor Subervi > wrote: > > Hi; > > I have this code snippet: > > > > sql '''create table if not exists %sCustomerData ( > > You're missing an equal sign there to start with (i.e. sql = '''). > LOL. I think it's time to put a gun to my head ;/ Thanks. beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From briandenzer at gmail.com Mon Jan 4 13:09:16 2010 From: briandenzer at gmail.com (Brian D) Date: Mon, 4 Jan 2010 10:09:16 -0800 (PST) Subject: fsync() doesn't work as advertised? References: Message-ID: On Jan 4, 10:29?am, Antoine Pitrou wrote: > Le Mon, 04 Jan 2010 08:09:56 -0800, Brian D a ?crit?: > > > > > What I've seen is that flush() alone produces a complete log when the > > loop finishes. When I used fsync(), I lost all of the write entries > > except the first, along with odd error trap and the last entry. > > Perhaps you are writing to the file from several threads or processes at > once? > > By the way, you shouldn't need fsync() if you merely want to look at the > log files, because your OS will have an up-to-date view of the file > contents anyway. fsync() is useful if you want to be sure the data has > been written to the hard disk drive, rather than just kept in the > operating system's filesystem cache. Sure -- I hadn't considered how threads might affect the write process. That's a good lead to perhaps fixing the problem. Thanks for your help, Antoine. From albert at spenarnc.xs4all.nl Mon Jan 4 13:17:04 2010 From: albert at spenarnc.xs4all.nl (Albert van der Horst) Date: 04 Jan 2010 18:17:04 GMT Subject: How to validate the __init__ parameters References: <803c4db7-3813-448a-b428-0ec11c57f098@t42g2000yqd.googlegroups.com> <00b98b9d$0$15623$c3e8da3@news.astraweb.com> Message-ID: In article , Steve Holden wrote: > >What's the exact reason for requiring that a creator argument be of a >specific type? So operations on the instances don't go wrong? Well, why >not just admit that we don't have control over everything, and just *let >things go wrong* when the wrong type is passed? > >What will then happen? Why, an exception will be raised! > >You might argue that the user, seeing the traceback from the exception, >won't know what to make of it. In response to which I would assert (pun >intended) that neither would the traceback produced by the assertion >failure. > >Given that somebody who knows what they are talking about has to cast >their eye over a traceback to understand what's wrong with the program, >it's not much extra effort for that person realise that a method has >been called with an invalid argument type. I accept that there are >legitimate arguments in opposition to this point of view, but it has a >certain engineering efficiency (as long as we are prepared to work >inside frameworks like Django that will encapsulate any error tracebacks >so that you see all the information that you need, and the user gets >some other inexplicable error message like "A programmer probably >screwed up). I can agree to a large extent with this reasoning. However, let us consider an example. I just solved the euler problem 269, something with cutting a piece of paper at exact boundaries. At some point I have reasoned that at least one of 4 arguments must be greater than one, such that an infinite recursion is prevented. Now I greatly prefer to have an assert there and then, instead of the infinite recursion, if my reasoning is wrong. You know it is not the only point where I did a similar assumption, and the problem was complicated enough as it is (for me). This triggers a question: I can see the traceback, but it would be much more valuable, if I could see the arguments passed to the functions. Is there a tool? > >In other words, "be prepared to fix your program when it goes wrong" is >a valid alternative to trying to anticipate every single last thing that >might go wrong. This philosophy might not be appropriate for >extra-terrestrial exploration, but most Python programmers aren't doing >that. Funny that you make this distinction. There is less of a distinction than you may think. Even while in embedded programming we anticipate as much as possible. there is also the watch dog timer, that resets the whole system for things unforeseen, hardware failures included. You bet space crafts have those on board. > >regards > Steve Groetjes Albert -- -- Albert van der Horst, UTRECHT,THE NETHERLANDS Economic growth -- being exponential -- ultimately falters. albert at spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst From phlip2005 at gmail.com Mon Jan 4 13:53:43 2010 From: phlip2005 at gmail.com (Phlip) Date: Mon, 04 Jan 2010 10:53:43 -0800 Subject: How to validate the __init__ parameters In-Reply-To: References: <803c4db7-3813-448a-b428-0ec11c57f098@t42g2000yqd.googlegroups.com> <00b98b9d$0$15623$c3e8da3@news.astraweb.com> Message-ID: Steve Holden wrote: > What's the exact reason for requiring that a creator argument be of a > specific type? So operations on the instances don't go wrong? Well, why > not just admit that we don't have control over everything, and just *let > things go wrong* when the wrong type is passed? Because some interfaces are "public", meaning visible within their own little arena of modules, and some interfaces are "published", meaning visible to other packages. Encapsulation is hierarchical - this is just a "small things go inside big things" situation. The higher an interface's level, the more "published" it is. Published interfaces should fail as early and clearly as possible if they are going to fail. The obvious example here is an interface that fails and prints a message telling a newbie how to call the package the right way. But that newbie could be you! (And don't forget the wall-to-wall unit tests, too;) -- Phlip http://c2.com/cgi/wiki?MoreliaViridis From gagsl-py2 at yahoo.com.ar Mon Jan 4 14:19:21 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 04 Jan 2010 16:19:21 -0300 Subject: No module named _socket, on windows References: Message-ID: En Mon, 04 Jan 2010 14:24:22 -0300, louisJ escribi?: > I installed python 2.6 (from python.org) for windows XP, and then > Pylab. > When I type "import pylab" in a python shell it shows the error: > > ImportError: No module named _socket Open the Python command line, type the following lines, and tell us what happens: Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import socket >>> socket._socket -- Gabriel Genellina From gagsl-py2 at yahoo.com.ar Mon Jan 4 14:51:53 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 04 Jan 2010 16:51:53 -0300 Subject: How to validate the __init__ parameters References: <803c4db7-3813-448a-b428-0ec11c57f098@t42g2000yqd.googlegroups.com> <00b98b9d$0$15623$c3e8da3@news.astraweb.com> Message-ID: En Mon, 04 Jan 2010 15:17:04 -0300, Albert van der Horst escribi?: > This triggers a question: I can see the traceback, but it > would be much more valuable, if I could see the arguments > passed to the functions. Is there a tool? Yes, the cgitb module [1]. Despite its name it's a general purpose module. This code: import cgitb cgitb.enable(format="text") spam = [] def a(x, y): """This is function a""" z = x+y return b(z) def b(z, n=3): """This is function b""" return c(foo=z*n) def c(foo=0, bar=1): """This is function c""" baz = foo+bar spam.somenamethatdoesnotexist(foo+bar) a(10, 20) generates this error message: A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurr ed. d:\temp\test_traceback.py in () 19 baz = foo+bar 20 spam.somenamethatdoesnotexist(foo+bar) 21 22 23 a(10, 20) a = d:\temp\test_traceback.py in a(x=10, y=20) 7 """This is function a""" 8 z = x+y 9 return b(z) 10 11 global b = z = 30 d:\temp\test_traceback.py in b(z=30, n=3) 12 def b(z, n=3): 13 """This is function b""" 14 return c(foo=z*n) 15 16 global c = foo undefined z = 30 n = 3 d:\temp\test_traceback.py in c(foo=90, bar=1) 18 """This is function c""" 19 baz = foo+bar 20 spam.somenamethatdoesnotexist(foo+bar) 21 22 global spam = [] spam.somenamethatdoesnotexist undefined foo = 90 bar = 1 : 'list' object has no attribu te 'somenamethatdoesnotexist' __class__ = __dict__ = {} __doc__ = 'Attribute not found.' ... more exception attributes ... The above is a description of an error in a Python program. Her e is the original traceback: Traceback (most recent call last): File "d:\temp\test_traceback.py", line 23, in a(10, 20) File "d:\temp\test_traceback.py", line 9, in a return b(z) File "d:\temp\test_traceback.py", line 14, in b return c(foo=z*n) File "d:\temp\test_traceback.py", line 20, in c spam.somenamethatdoesnotexist(foo+bar) AttributeError: 'list' object has no attribute 'somenamethatdoes notexist' [1] http://docs.python.org/library/cgitb.html -- Gabriel Genellina From phlip2005 at gmail.com Mon Jan 4 15:01:46 2010 From: phlip2005 at gmail.com (Phlip) Date: Mon, 4 Jan 2010 12:01:46 -0800 (PST) Subject: Where's a DOM builder that uses the Builder Pattern to ... build DOMs? Message-ID: <04523152-a0ff-432c-a48f-2c653577e7b7@m25g2000yqc.googlegroups.com> Not Hyp: I hope I'm wrong, but seems that DOMBuilder, found among the various xml.dom packages, cannot build DOM like this: var html = DomBuilder.apply(); var form = html.FORM( html.DIV( html.INPUT({type : 'text', name : 'email'}), html.INPUT({type : 'text', name : 'password'}), html.INPUT({type : 'submit'}), ) ); Do anyone know any good DOM builder packages that do build DOM good like a DOM builder should? -- Phlip http://zeekland.zeroplayer.com/Uncle_Wiggilys_Travels/1 From nad at acm.org Mon Jan 4 15:25:01 2010 From: nad at acm.org (Ned Deily) Date: Mon, 04 Jan 2010 12:25:01 -0800 Subject: How do you configure IDLE on a Mac, seeing as there's no Configure Option... References: Message-ID: In article , Benjamin Kaplan wrote: > On Sun, Jan 3, 2010 at 8:42 PM, Mensanator wrote: > > ...because there's no [Options] menu on the shell window? > > > > Or at least give me a clue to how to use Courier New font? > > > > For some inscrutable reason, depite the plethora of formatting tools, > > someone decided that proportional spaced fonts ought to be the > > default for IDLE. > Mac Applications don't have "options" menus. They have a "preferences" > pane which is the second item in the application menu (the one that > takes the name of the active program) and has a keyboard shortcut of > Cmd-, > > Apple's user-interface guidelines are pretty specific, so you'll find > that this works for all native OS X (i.e. not X11) applications. But be aware that IDLE on OS X has had various problems with missing and/or duplicate menu items. IIRC, the current python.org OS X IDLEs (2.6.4 and 3.1.1) should be OK and you can modify the font via the Preferences menu (and currently changed preferences apply to all versions of IDLE). Note also that there are two different ways to launch the OS X IDLE. python.org installers put a double-clickable IDLE.app in /Applications/Python n.x. There is also a command-line "idlen.n" in /Library/Frameworks/Python.framework/Versions/n.n/bin which, optionally, has a symlink from /usr/local/bin. -- Ned Deily, nad at acm.org From nad at acm.org Mon Jan 4 15:31:10 2010 From: nad at acm.org (Ned Deily) Date: Mon, 04 Jan 2010 12:31:10 -0800 Subject: Best practices for simultaneously installed versioned packages? References: <4B41CC6C.3020808@hastings.org> Message-ID: In article , Benjamin Kaplan wrote: > On Mon, Jan 4, 2010 at 6:09 AM, Larry Hastings wrote: > > I'm writing a package for Python 3--let's call it "spacegoblin". I fear > > someday I may need multiple versions installed and available simultaneously, > > even within one version of Python. So I want to plan ahead for that > > possibility. What would be the best way to allow this? Right now I install > > the package with the version in the name, like "spacegoblin_1_0" and > > "spacegoblin_1_1". But I have an uneasy sense that I'm doing something > > stupid... mainly because I've never seen anyone do this before. > > > > A coworker said they did it like "spacegoblin.1_0" and "spacegoblin.1_1" at > > a previous employer. That seems like an improvement, though not the "yes > > that's obviously right" answer I'm holding out for. Your thoughts?> > wxpython installs a "wxversion" module which has functions like > getInstalled(), ensureMinimal(version), and select(version). You can > call wxversion.select before importing wx and it will make sure that > the correct version is imported. You might want to look up what they > did. Also, setuptools (and, its successor, distribute, which supports Python 3) allow the installation and management of multiple versions of a package within one python site-library instance. -- Ned Deily, nad at acm.org From victorsubervi at gmail.com Mon Jan 4 16:00:12 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Mon, 4 Jan 2010 17:00:12 -0400 Subject: Premature End Of Script Headers Message-ID: <4dc0cfea1001041300g57801a2fp1945a5ad628eea0f@mail.gmail.com> Hi; Here's my entire test script: #!/usr/bin/python def myMail(): print 'Content-type: text/html' print print ''' ''' Here's the error: [Mon Jan 04 12:59:15 2010] [error] [client 66.82.9.61] Premature end of script headers: mail.py, referer: http://angrynates.com/cart/cart2.py [root at 13gems simplemail]# ls -al total 60 drwxr-xr-x 2 beno beno 4096 Jan 4 12:58 . drwxrwxrwx 6 beno beno 4096 Jan 4 12:53 .. -rwxr-xr-x 1 beno beno 252 Jan 4 06:18 mailer_test.py -rwxr-xr-x 1 beno beno 807 Jan 4 06:18 mail_old.py -rwxr-xr-x 1 beno beno 2988 Jan 4 12:58 mail.py -rwxr-xr-x 1 beno beno 17918 Jan 4 06:18 simplemail.py -rwxr-xr-x 1 beno beno 16230 Jan 4 06:18 template.py -rwxr-xr-x 1 beno beno 263 Jan 4 06:18 testmail.py What gives? TIA, beno -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben+python at benfinney.id.au Mon Jan 4 16:06:08 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 05 Jan 2010 08:06:08 +1100 Subject: twenty years ago Guido created Python References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> <87oclaxksq.fsf@benfinney.id.au> Message-ID: <877hrxy3zz.fsf@benfinney.id.au> Daniel Fetchinson writes: > But Ben will not get the Best Sense of Humor Award of 2010 for sure :) Sometimes the failing of humour is not with the recipient. -- \ ?Are you pondering what I'm pondering, Pinky?? ?Sure, Brain, | `\ but how are we going to find chaps our size?? ?_Pinky and The | _o__) Brain_ | Ben Finney From navjotmusic at gmail.com Mon Jan 4 16:22:20 2010 From: navjotmusic at gmail.com (Nav) Date: Mon, 4 Jan 2010 13:22:20 -0800 (PST) Subject: Newbie help- Can multiple instances with multiple names automatically created. Message-ID: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> I have a class of let's say empty bottle which can have a mix of two items. I want to create let's say 30 of these objects which will have names based on the 2 attributes (apple juice, beer, grape juice, beer, etc) that I provide from a list. All the objects are a mix of (1 of three alcohols) and (1 of 10 juices), so I don't want to go through typing in the names of all the objects (which would be totally stupid). I get problems if I try such as I can't assign to literal etc. If I make a list of names using the attributes. then equate names to objects. the list gets populated by the objects and the names disappear. I want the ability to be able to call up any object by its name and manipulate it and yet not have to assign the name manually. How can this be done? From news123 at free.fr Mon Jan 4 16:22:40 2010 From: news123 at free.fr (News123) Date: Mon, 04 Jan 2010 22:22:40 +0100 Subject: python xmlrpc client with ssl client certificates and standard modules In-Reply-To: References: <4b414e79$0$23321$426a34cc@news.free.fr> <4b41b733$0$13831$426a74cc@news.free.fr> Message-ID: <4b425c20$0$27951$426a34cc@news.free.fr> Hi Massimo, I'm still a litle confused: My setup: server host: ------------ apache, php with an xmlrpc server interface. no python installed. multiple client hosts (linux / windows only default python installed) ----------------------------------------------------------------------- an existing python script performing an xmlrpc call to the server host. The current working (without certificates) code snippet is: import xmlrpclib server_url = 'https://myserver' server = xmlrpclib.Server(server_url); result = server.myfunction(args) The whole setup is working as long as no client certificates are imposed by the server. The whole setup is not working as soon as the server is configured to accept only a given set of SSL-client certificates. My question is how to change above four line code snippet, such, that a client certificate will be sent to the xmlrpc server asuming the variables CLIENT_KEY_FILE and CLIENT_CRT_FILE are defined and pointing to the client certificate files. I hope to have more time tomorrow to check out the option, that I found and the option, that you suggest. option 1: http://www.cs.technion.ac.il/~danken/xmlrpc-ssl.html ( with non standard py module M2Crypto ) option 2: or web2py, which at first (so far no second) glance seems more to be targeted at implementing server side application with ajax or the client side. thanks again and bye N mdipierro wrote: > If it is a client problem than web2py will be on help. > > If your server is written already you may be able to use it with the > ssl cherrypy wsgi server (the one that web2py uses) and you do not > need web2py at all. > > Massimo > > On Jan 4, 3:38 am, News123 wrote: >> Thanks for your answer. >> >> I'll look at web2py. >> >> However web2py seems to address the xmlrpc server (at least in your >> example). The xmlrpc server application exists alerady and requires a >> client certificate. >> >> The client example doesn't seem to be using a certificate. >> >> So I'll be reading a little into web2py. >> >> bye >> >> N >> >> mdipierro wrote: >>> xmlrpc acts at the application layer and ssl at the transport layer so >>> they can inter operate easily as long as you do not use the >>> certificate to authenticate the client but only validate the server >>> and encrypt data (which you can also do but it is more complicated) >>> One option for you is to use web2py which include an xmlrpc server >>> that uses a wsgi ssl enabled web server. >>> Here is how: >>> 1) Install web2py >>> 2) Visithttp://127.0.0.1:8000/adminand create a new application from >>> the web based IDE >>> 3) create your web service for example, in a controller default.py >>> @service.xmlrpc >>> def add(a,b): return int(a)+int(b) >>> 4) Restart web2py with >>> python web2py.py -a ADMIN_PASSWD -c SSL_CERTIFICATE -k >>> SSL_PRIVATE_KEY -i 0.0.0.0 -p 443 >>> 5) You can now access the service from any Python program: >>> >>> import xmlrpclib >>> >>> server_url = 'https://myserver:443/yourapp/default/call/ >>> xmlrpc' >>> >>> server = xmlrpclib.Server(server_url) >>> >>> print server.add(3,4) >>> 7 >>> Hope this helps. >>> On Jan 3, 8:12 pm, News123 wrote: >>>> Hi, >>>> I was googling fot quite some time and was not really succesfull. >>>> I found one solution, which I will try soon. >>>> It ishttp://www.cs.technion.ac.il/~danken/xmlrpc-ssl.html >>>> (found inhttp://hamakor.org.il/pipermail/python-il/2008-February/000029.html) >>>> This will probably work, but it requires the module M2Crypto. >>>> In order to avoid installing M2Crypto an all hosts that want to run the >>>> script I wondered, whether there is no other solution. >>>> I can do xmlrpc over ssl WITHOUT certificates with following code: >>>> import xmlrpclib >>>> server_url = 'https://myserver' >>>> server = xmlrpclib.Server(server_url); >>>> and I can perform a https get request WITH certificates with below snippet: >>>> import httplib >>>> conn = httplib.HTTPSConnection( >>>> HOSTNAME, >>>> key_file = KEYFILE, >>>> cert_file = CERTFILE >>>> ) >>>> conn.putrequest('GET', '/') >>>> conn.endheaders() >>>> response = conn.getresponse() >>>> print response.read() >>>> I'm just lost of how to 'combine' both. >>>> Thanks in advance for any suggestions / hints >>>> N >> > From shawn at milochik.com Mon Jan 4 16:32:22 2010 From: shawn at milochik.com (Shawn Milochik) Date: Mon, 4 Jan 2010 16:32:22 -0500 Subject: Newbie help- Can multiple instances with multiple names automatically created. In-Reply-To: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> Message-ID: <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> You could put them in a dictionary with the key being the name, instead of a list. Shawn From michi at triodia.com Mon Jan 4 16:34:34 2010 From: michi at triodia.com (Michi) Date: Mon, 4 Jan 2010 13:34:34 -0800 (PST) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> Message-ID: <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> On Jan 4, 1:30?pm, Steven D'Aprano wrote: > > This is very true, but good APIs often trade-off increased usability and > reduced defect rate against machine efficiency too. In fact, I would > argue that this is a general design principle of programming languages: > since correctness and programmer productivity are almost always more > important than machine efficiency, the long-term trend across virtually > all languages is to increase correctness and productivity even if doing > so costs some extra CPU cycles. Yes, I agree with that in general. Correctness and productivity are more important, as a rule, and should be given priority. > > (For example, wrapper APIs often require additional > > memory allocations and/or data copies.) Incorrect use of exceptions also > > incurs an efficiency penalty. > > And? *Correct* use of exceptions also incur a penalty. So does the use of > functions. Does this imply that putting code in functions is a poor API? > Certainly not. It does imply that incorrect use of exceptions incurs an unnecessary performance penalty, no more, no less, just as incorrect use of wrappers incurs an unnecessary performance penalty. > But no matter how much more expensive, there will always be a cut-off > point where it is cheaper on average to suffer the cost of handling an > exception than it is to make unnecessary tests. > > In Python, for dictionary key access, that cut-off is approximately at > one failure per ten or twenty attempts. So unless you expect more than > one in ten attempts to lead to a failure, testing first is actually a > pessimation, not an optimization. What this really comes down to is how frequently or infrequently a particular condition arises before that condition should be considered an exceptional condition rather than a normal one. It also relates to how the set of conditions partitions into "normal" conditions and "abnormal" conditions. The difficulty for the API designer is to make these choices correctly. > In some, limited, cases you might be able to use the magic return value > strategy, but this invariably leads to lost programmer productivity, more > complex code, lowered readability and usability, and more defects, > because programmers will invariably neglect to test for the special value: I disagree here, to the extent that, whether something is an error or not can very much depend on the circumstances in which the API is used. The collection case is a very typical example. Whether failing to locate a value in a collection is an error very much depends on what the collection is used for. In some cases, it's a hard error (because it might, for example, imply that internal program state has been corrupted); in other cases, not finding a value is perfectly normal. For the API designer, the problem is that an API that throws an exception when it should not sucks just as much as an API that doesn't throw an exception when it should. For general-purpose APIs, such as a collection API, as the designer, I usually cannot know. As I said elsewhere in the article, general-purpose APIs should be policy-free, and special-purpose APIs should be policy-rich. As the designer, the more I know about the circumstances in which the API will be used, the more fascist I can be in the design and bolt down the API more in terms of static and run-time safety. Wanting to ignore a return value from a function is perfectly normal and legitimate in many cases. However, if a function throws instead of returning a value, ignoring that value becomes more difficult for the caller and can extract a performance penalty that may be unacceptable to the caller. The problem really is that, at the time the API is designed, there often is no way to tell whether this will actually be the case; in turn, no matter whether I choose to throw an exception or return an error code, it will be wrong for some people some of the time. > This is a classic example of premature optimization. Unless such > inefficiency can be demonstrated to actually matter, then you do nobody > any favours by preferring the API that leads to more defects on the basis > of *assumed* efficiency. I agree with the concern about premature optimisation. However, I don't agree with a blanket statement that special return values always and unconditionally lead to more defects. Returning to the .NET non- blocking I/O example, the fact that the API throws an exception when it shouldn't very much complicates the code and introduces a lot of extra control logic that is much more likely to be wrong than a simple if-then-else statement. As I said, throwing an exception when none should be thrown can be just as harmful as the opposite case. > It doesn't matter whether it is an error or not. They are called > EXCEPTIONS, not ERRORS. What matters is that it is an exceptional case. > Whether that exceptional case is an error condition or not is dependent > on the application. Exactly. To me, that implies that making something an exception that, to the caller, shouldn't be is just as inconvenient as the other way around. > > ?* Is it appropriate to force the caller to deal with the condition in > > a catch-handler? > > > ?* If the caller fails to explicitly deal with the condition, is it > > appropriate to terminate the program? > > > Only if the answer to these questions is "yes" is it appropriate to > > throw an exception. Note the third question, which is often forgotten. > > By throwing an exception, I not only force the caller to handle the > > exception with a catch-handler (as opposed to leaving the choice to the > > caller), I also force the caller to *always* handle the exception: if > > the caller wants to ignore the condition, he/she still has to write a > > catch-handler and failure to do so terminates the program. > > That's a feature of exceptions, not a problem. Yes, and didn't say that it is a problem. However, making the wrong choice for the use of the feature is a problem, just as making the wrong choice for not using the feature is. > > Apart from the potential performance penalty, throwing exceptions for > > expected outcomes is bad also because it forces a try-catch block on the > > caller. > > But it's okay to force a `if (result==MagicValue)` test instead? Yes, in some cases it is. For example: int numBytes; int fd = open(...); while ((numBytes = read(fd, ?)) > 0) { // process data... } Would you prefer to see EOF indicated by an exception rather than a zero return value? I wouldn't. > Look, the caller has to deal with exceptional cases (which may include > error conditions) one way or the other. If you don't deal with them at > all, your code will core dump, or behave incorrectly, or something. If > the caller fails to deal with the exceptional case, it is better to cause > an exception that terminates the application immediately than it is to > allow the application to generate incorrect results. I agree that failing to deal with exceptional cases causes problems. I also agree that exceptions, in general, are better than error codes because they are less likely to go unnoticed. But, as I said, it really depends on the caller whether something should be an exception or not. The core problem isn't whether exceptions are good or bad in a particular case, but that most APIs make this an either-or choice. For example, if I had an API that allowed me to choose at run time whether an exception will be thrown for a particular condition, I could adapt that API to my needs, instead of being stuck with whatever the designer came up with. There are many ways this could be done. For example, I could have a find() operation on a collection that throws if a value isn't found, and I could have findNoThrow() if I want a sentinel value returned. Or, the API could offer a callback hook that decides at run time whether to throw or not. (There are many other possible ways to do this, such as setting the behaviour at construction time, or by having different collection types with different behaviours.) The point is that a more flexible API is likely to be more useful than one that sets a single exception policy for everyone. > > As the API > > creator, if I indicate errors with exceptions, I make a policy decision > > about what is an error and what is not. It behooves me to be > > conservative in that policy: I should throw exceptions only for > > conditions that are unlikely to arise during routine and normal use of > > the API. > > But lost connections *are* routine and normal. Hopefully they are rare. In the context of my example, they are not. The range of behaviours naturally falls into these categories: * No data ready * Data ready * EOF * Socket error The first three cases are the "normal" ones; they operate on the same program state and they are completely expected: while reading a message off the wire, the program will almost certainly encounter the first two conditions and, if there is no error, it will always encounter the EOF condition. The fourth case is the unexpected one, in the sense that this case will often not arise at all. That's not to say that lost connections aren't routine; they are. But, when a connection is lost, the program has to do different things and operate on different state than when the connection stays up. This strongly suggests that the first three conditions should be dealt with by return values and/or out parameters, and the fourth condition should be dealt with as an exception. Cheers, Michi. From louisrom1 at gmail.com Mon Jan 4 16:41:20 2010 From: louisrom1 at gmail.com (louisJ) Date: Mon, 4 Jan 2010 13:41:20 -0800 (PST) Subject: No module named _socket, on windows References: Message-ID: I don't know what happened or what I did in the meantime but it works now...no more errors. Gabriel, for information now I have: >>> import socket >>> socket._socket Thank you anyway. From clp2 at rebertia.com Mon Jan 4 16:54:41 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Mon, 4 Jan 2010 13:54:41 -0800 Subject: Newbie help- Can multiple instances with multiple names automatically created. In-Reply-To: <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> Message-ID: <50697b2c1001041354x588aee56xd984740d12d2e6d5@mail.gmail.com> On Mon, Jan 4, 2010 at 1:32 PM, Shawn Milochik wrote: > You could put them in a dictionary with the key being the name, instead of a list. To illustrate that for the OP: name2drink = {} for booze in liquors: for juice in juices: name = juice +" "+booze # or however you're naming them drink = Bottle(booze, juice) name2drink[name] = drink #example use favorite = name2drink["apple wine"] favorite.rating = 9/10 Cheers, Chris -- http://blog.rebertia.com From albert at spenarnc.xs4all.nl Mon Jan 4 17:00:22 2010 From: albert at spenarnc.xs4all.nl (Albert van der Horst) Date: 04 Jan 2010 22:00:22 GMT Subject: How to validate the __init__ parameters References: <803c4db7-3813-448a-b428-0ec11c57f098@t42g2000yqd.googlegroups.com> Message-ID: In article , Chris Rebert wrote: >On Mon, Jan 4, 2010 at 10:17 AM, Albert van der Horst > wrote: > >> This triggers a question: I can see the traceback, but it >> would be much more valuable, if I could see the arguments >> passed to the functions. Is there a tool? > >print(locals()) #this actually gives the bindings for the entire local namespace >#but is less work than writing something more precise by hand. > >It is amazing how handy the humble print() function/statement is >when debugging. It is also amazing how much output is generated in this way in a typical Euler program (that require seconds, if not minutes solid computer time on Giga Hz machines). I have to routinely clean out my 30 Gbyte home directory. Not so long ago generating so much debug output wasn't an option! What I hoped for is something along the line of what you can do with a coredump in a classical debugger. Especially if recursion runs dozens of levels deep, it is difficult to keep track. But locals() is nice function, thanks. > >Cheers, >Chris Groetjes Albert -- -- Albert van der Horst, UTRECHT,THE NETHERLANDS Economic growth -- being exponential -- ultimately falters. albert at spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst From richardbp at gmail.com Mon Jan 4 17:04:12 2010 From: richardbp at gmail.com (Richard) Date: Mon, 4 Jan 2010 14:04:12 -0800 (PST) Subject: Efficiently determine where documents differ Message-ID: Hello, I have been using the difflib library to find where 2 large HTML documents differ. The Differ().compare() method does this, but it is very slow - atleast 100x slower than the unix diff command. How can I efficiently determine where 2 documents differ in Python? (Ideally I am after the positions rather the actual text, which is what SequenceMatcher().get_opcodes() returns.) Richard From martin at v.loewis.de Mon Jan 4 17:12:43 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Mon, 04 Jan 2010 23:12:43 +0100 Subject: python xmlrpc client with ssl client certificates and standard modules In-Reply-To: <4b414e79$0$23321$426a34cc@news.free.fr> References: <4b414e79$0$23321$426a34cc@news.free.fr> Message-ID: <4B4267DB.3080409@v.loewis.de> > I can do xmlrpc over ssl WITHOUT certificates with following code: > > import xmlrpclib > server_url = 'https://myserver' > server = xmlrpclib.Server(server_url); > > > and I can perform a https get request WITH certificates with below snippet: > > import httplib > conn = httplib.HTTPSConnection( > HOSTNAME, > key_file = KEYFILE, > cert_file = CERTFILE > ) > conn.putrequest('GET', '/') > conn.endheaders() > response = conn.getresponse() > print response.read() > > > I'm just lost of how to 'combine' both. In this case, read through the source of xmlrpclib: a) SafeTransport receives x509 parameters from get_host_info b) get_host_info supports a case where host is a tuple host, x509 So, without testing: server = xmlrpclib.Server((server_url, {'key_file': KEYFILE, 'cert_file': CERTFILE})) Please do read the code before trying this out. HTH, Martin From n00m at narod.ru Mon Jan 4 17:20:24 2010 From: n00m at narod.ru (n00m) Date: Mon, 4 Jan 2010 14:20:24 -0800 (PST) Subject: twenty years ago Guido created Python References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> <87oclaxksq.fsf@benfinney.id.au> <877hrxy3zz.fsf@benfinney.id.au> Message-ID: <02d9bd25-5ae0-4de3-910d-dc36bec0fc11@m3g2000yqf.googlegroups.com> Ben, go away from here. With all your stupids sigs. Do you think are you original? You are a stupid animal. Guido, Tim Peters, Raymond Hettinger are geniuis. I don't know exactly Python mob. Maybe forgot someone. You is only a source of depspise for them You get your everymonth fucking 1000 e sitting here whom are you teaching? This crowds of animals? They and you you cant nothing From gtu2003 at alice.it Mon Jan 4 17:35:02 2010 From: gtu2003 at alice.it (wiso) Date: Mon, 04 Jan 2010 23:35:02 +0100 Subject: FileInput too slow Message-ID: <4b426d17$0$1113$4fafbaef@reader4.news.tin.it> I'm trying the fileinput module, and I like it, but I don't understand why it's so slow... look: from time import time from fileinput import FileInput file = ['r1_200907.log', 'r1_200908.log', 'r1_200909.log', 'r1_200910.log', 'r1_200911.log'] def f1(): n = 0 for f in file: print "new file: %s" % f ff = open(f) for line in ff: n += 1 ff.close() return n def f2(): f = FileInput(file) for line in f: if f.isfirstline(): print "new file: %s" % f.filename() return f.lineno() def f3(): # f2 simpler f = FileInput(file) for line in f: pass return f.lineno() t = time(); f1(); print time()-t # 1.0 t = time(); f2(); print time()-t # 7.0 !!! t = time(); f3(); print time()-t # 5.5 I'm using text files, there are 2563150 lines in total. From gagsl-py2 at yahoo.com.ar Mon Jan 4 17:46:13 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 04 Jan 2010 19:46:13 -0300 Subject: Efficiently determine where documents differ References: Message-ID: En Mon, 04 Jan 2010 19:04:12 -0300, Richard escribi?: > I have been using the difflib library to find where 2 large HTML > documents differ. The Differ().compare() method does this, but it is > very slow - atleast 100x slower than the unix diff command. Differ compares sequences of lines *and* lines as sequences of characters to provide intra-line differences. The diff command only processes lines. If you aren't interested in intra-line differences, use a SequenceMatcher instead. Or, invoke the diff command using subprocess.Popen + communicate. -- Gabriel Genellina From steve at holdenweb.com Mon Jan 4 17:48:41 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 04 Jan 2010 17:48:41 -0500 Subject: Premature End Of Script Headers In-Reply-To: <4dc0cfea1001041300g57801a2fp1945a5ad628eea0f@mail.gmail.com> References: <4dc0cfea1001041300g57801a2fp1945a5ad628eea0f@mail.gmail.com> Message-ID: Victor Subervi wrote: > Hi; > > Here's my entire test script: > > #!/usr/bin/python > > def myMail(): > print 'Content-type: text/html' > print > print ''' > > > > > > > ''' > > Here's the error: > > [Mon Jan 04 12:59:15 2010] [error] [client 66.82.9.61] Premature end of > script headers: mail.py, referer: http://angrynates.com/cart/cart2.py > > [root at 13gems simplemail]# ls -al > total 60 > drwxr-xr-x 2 beno beno 4096 Jan 4 12:58 . > drwxrwxrwx 6 beno beno 4096 Jan 4 12:53 .. > -rwxr-xr-x 1 beno beno 252 Jan 4 06:18 mailer_test.py > -rwxr-xr-x 1 beno beno 807 Jan 4 06:18 mail_old.py > -rwxr-xr-x 1 beno beno 2988 Jan 4 12:58 mail.py > -rwxr-xr-x 1 beno beno 17918 Jan 4 06:18 simplemail.py > -rwxr-xr-x 1 beno beno 16230 Jan 4 06:18 template.py > -rwxr-xr-x 1 beno beno 263 Jan 4 06:18 testmail.py > What is this directory listing supposed to tell us? > What gives? Perhaps, having defined that nice function, you shoudl call it to produce some output? The Apache server is complaining that there's nothing there! regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From zuo at chopin.edu.pl Mon Jan 4 17:56:57 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Mon, 04 Jan 2010 23:56:57 +0100 Subject: Newbie help- Can multiple instances with multiple names automatically created. In-Reply-To: <50697b2c1001041354x588aee56xd984740d12d2e6d5@mail.gmail.com> References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <50697b2c1001041354x588aee56xd984740d12d2e6d5@mail.gmail.com> Message-ID: 2010-01-04, 22:54:41 Chris Rebert wrote: > name2drink = {} > for booze in liquors: > for juice in juices: > name = juice +" "+booze # or however you're naming them > drink = Bottle(booze, juice) > name2drink[name] = drink @Nav: ...and if you really desire to have those objects in global (module's) namespace, you can do that: global_namespace = globals() for booze in liquors: for juice in juices: name = '{0}_with_{1}_juice'.format(booze, juice) drink = Bottle(booze, juice) global_namespace[name] = drink # then you have them in module's namespace print(wine_with_apple) wine_with_apple.rating = 0.9 Though in most cases it'd be not necessary. Cheers, *j PS. Another way to express practically the same: from itertools import product globals().update(('{0}_with_{1}_juice'.format(booze, juice), Bottle(booze, juice)) for booze, juice in product(liquors, juices)) -- Jan Kaliszewski (zuo) From navjotmusic at gmail.com Mon Jan 4 17:59:28 2010 From: navjotmusic at gmail.com (Nav) Date: Mon, 4 Jan 2010 14:59:28 -0800 (PST) Subject: Newbie help- Can multiple instances with multiple names automatically created. References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> Message-ID: <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> On Jan 4, 4:54?pm, Chris Rebert wrote: > On Mon, Jan 4, 2010 at 1:32 PM, Shawn Milochik wrote: > > You could put them in a dictionary with the key being the name, instead of a list. > > To illustrate that for the OP: > > name2drink = {} > for booze in liquors: > ? ? for juice in juices: > ? ? ? ? name = juice +" "+booze # or however you're naming them > ? ? ? ? drink = Bottle(booze, juice) > ? ? ? ? name2drink[name] = drink > > #example use > favorite = name2drink["apple wine"] > favorite.rating = 9/10 typing favorite = such and such is what I am trying to avoid. I want to be able to use the name 'apple_wine' as the variable which has the object apple wine but not have to do this manually as you did with favorite. From shawn at milochik.com Mon Jan 4 18:06:57 2010 From: shawn at milochik.com (Shawn Milochik) Date: Mon, 4 Jan 2010 18:06:57 -0500 Subject: Newbie help- Can multiple instances with multiple names automatically created. In-Reply-To: <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> Message-ID: <7545C537-E886-4254-8770-C636717703FB@milochik.com> On Jan 4, 2010, at 5:59 PM, Nav wrote: > On Jan 4, 4:54 pm, Chris Rebert wrote: >> On Mon, Jan 4, 2010 at 1:32 PM, Shawn Milochik wrote: >>> You could put them in a dictionary with the key being the name, instead of a list. >> >> To illustrate that for the OP: >> >> name2drink = {} >> for booze in liquors: >> for juice in juices: >> name = juice +" "+booze # or however you're naming them >> drink = Bottle(booze, juice) >> name2drink[name] = drink >> >> #example use >> favorite = name2drink["apple wine"] >> favorite.rating = 9/10 > > typing > favorite = such and such is what I am trying to avoid. > > I want to be able to use the name 'apple_wine' as the variable which > has the object apple wine but not have to do this manually as you did > with favorite. > > > > > Then don't assign a variable to favorite, and just use name2drink["apple wine"]. From steve at holdenweb.com Mon Jan 4 18:12:03 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 04 Jan 2010 18:12:03 -0500 Subject: Newbie help- Can multiple instances with multiple names automatically created. In-Reply-To: <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> Message-ID: Nav wrote: > On Jan 4, 4:54 pm, Chris Rebert wrote: >> On Mon, Jan 4, 2010 at 1:32 PM, Shawn Milochik wrote: >>> You could put them in a dictionary with the key being the name, instead of a list. >> To illustrate that for the OP: >> >> name2drink = {} >> for booze in liquors: >> for juice in juices: >> name = juice +" "+booze # or however you're naming them >> drink = Bottle(booze, juice) >> name2drink[name] = drink >> >> #example use >> favorite = name2drink["apple wine"] >> favorite.rating = 9/10 > > typing > favorite = such and such is what I am trying to avoid. > > I want to be able to use the name 'apple_wine' as the variable which > has the object apple wine but not have to do this manually as you did > with favorite. > Why? The example is trying to show you that it's much more sensible (i.e. better controlled, easier to manage, less likely to cause problems) if instead of looking up the names in the global namespace you instead looked them up in a dictionary. This question arises so frequently it should really be a FAQ, but the closest I can come is http://techblog.ironfroggy.com/2007/06/dynamic-hell.html which does at least exercise the necessary arguments. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From python at mrabarnett.plus.com Mon Jan 4 18:18:22 2010 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 04 Jan 2010 23:18:22 +0000 Subject: Exception as the primary error handling mechanism? In-Reply-To: <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> Message-ID: <4B42773E.7050806@mrabarnett.plus.com> Michi wrote: > On Jan 4, 1:30 pm, Steven D'Aprano > wrote: [snip] >>> * Is it appropriate to force the caller to deal with the condition in >>> a catch-handler? >>> * If the caller fails to explicitly deal with the condition, is it >>> appropriate to terminate the program? >>> Only if the answer to these questions is "yes" is it appropriate to >>> throw an exception. Note the third question, which is often forgotten. >>> By throwing an exception, I not only force the caller to handle the >>> exception with a catch-handler (as opposed to leaving the choice to the >>> caller), I also force the caller to *always* handle the exception: if >>> the caller wants to ignore the condition, he/she still has to write a >>> catch-handler and failure to do so terminates the program. >> That's a feature of exceptions, not a problem. > > Yes, and didn't say that it is a problem. However, making the wrong > choice for the use of the feature is a problem, just as making the > wrong choice for not using the feature is. > >>> Apart from the potential performance penalty, throwing exceptions for >>> expected outcomes is bad also because it forces a try-catch block on the >>> caller. >> But it's okay to force a `if (result==MagicValue)` test instead? > > Yes, in some cases it is. For example: > > int numBytes; > int fd = open(...); > while ((numBytes = read(fd, ?)) > 0) > { > // process data... > } > > Would you prefer to see EOF indicated by an exception rather than a > zero return value? I wouldn't. > I wouldn't consider zero to be a magic value in this case. Returning a negative number if an error occurred would be magic. A better comparison might be str.find vs str.index, the former returning a magic value -1. Which is used more often? >> Look, the caller has to deal with exceptional cases (which may include >> error conditions) one way or the other. If you don't deal with them at >> all, your code will core dump, or behave incorrectly, or something. If >> the caller fails to deal with the exceptional case, it is better to cause >> an exception that terminates the application immediately than it is to >> allow the application to generate incorrect results. > > I agree that failing to deal with exceptional cases causes problems. I > also agree that exceptions, in general, are better than error codes > because they are less likely to go unnoticed. But, as I said, it > really depends on the caller whether something should be an exception > or not. > > The core problem isn't whether exceptions are good or bad in a > particular case, but that most APIs make this an either-or choice. For > example, if I had an API that allowed me to choose at run time whether > an exception will be thrown for a particular condition, I could adapt > that API to my needs, instead of being stuck with whatever the > designer came up with. > > There are many ways this could be done. For example, I could have a > find() operation on a collection that throws if a value isn't found, > and I could have findNoThrow() if I want a sentinel value returned. > Or, the API could offer a callback hook that decides at run time > whether to throw or not. (There are many other possible ways to do > this, such as setting the behaviour at construction time, or by having > different collection types with different behaviours.) > Or find() could have an extra keyword argument, eg. string.find(substring, default=-1), although that should probably be string.index(substring, default=-1) as a replacement for string.find(substring). From navjotmusic at gmail.com Mon Jan 4 18:21:15 2010 From: navjotmusic at gmail.com (Nav) Date: Mon, 4 Jan 2010 15:21:15 -0800 (PST) Subject: Newbie help- Can multiple instances with multiple names automatically created. References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <50697b2c1001041354x588aee56xd984740d12d2e6d5@mail.gmail.com> Message-ID: Thanks Jan, You read my mind. That is exactly what I needed. Thanks for showing the product function from itertools as well. It seems easier to grasp than the nested loops, I had been using. I noticed chopin.edu.pl. Are you a musician? Nav From navjotmusic at gmail.com Mon Jan 4 18:33:25 2010 From: navjotmusic at gmail.com (Nav) Date: Mon, 4 Jan 2010 15:33:25 -0800 (PST) Subject: Newbie help- Can multiple instances with multiple names automatically created. References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> Message-ID: Thanks for pointing it out Steve. The blog post doesn't explain it very well. I understand the risk of exec or eval(input). but what are the risks of globalnamespace use and what are the benefits? From deets at nospam.web.de Mon Jan 4 18:35:28 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Tue, 05 Jan 2010 00:35:28 +0100 Subject: Safe file I/O to shared file (or SQLite) from multi-threaded web server In-Reply-To: References: <4b403899$0$1646$742ec2ed@news.sonic.net> Message-ID: <7qfca0FncdU1@mid.uni-berlin.de> Lawrence D'Oliveiro schrieb: > In message , Steve > Holden wrote: > >> Yes, but not to MySQL, please. Particularly since there is a sword of >> Damocles hanging over its head while the Oracle takeover of Sun is >> pending. > > Ah, I see the FUDsters are crawling out of the woodwork here, as well. I?ve > got news for you: MySQL is an open-source product. And you can?t kill Open > Source. So go crawling back to your proprietary world, if that?s the only > world you understand. Since when is suggesting Postgres a sign of a proprietary world crawler? And while I don't wish MySQL anything bad - open source *can* die, and will, if leadership changes for the worst - which can happen. And is certainly an immediate threat here. Diez From mensanator at aol.com Mon Jan 4 18:37:29 2010 From: mensanator at aol.com (Mensanator) Date: Mon, 4 Jan 2010 15:37:29 -0800 (PST) Subject: twenty years ago Guido created Python References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> <87oclaxksq.fsf@benfinney.id.au> <877hrxy3zz.fsf@benfinney.id.au> <02d9bd25-5ae0-4de3-910d-dc36bec0fc11@m3g2000yqf.googlegroups.com> Message-ID: On Jan 4, 4:20?pm, n00m wrote: > Ben, go away from here. With all your stupids sigs. > Do you think are you original? > You are a stupid animal. > Guido, Tim Peters, Raymond Hettinger are geniuis. > I don't know exactly Python mob. Maybe forgot someone. > You is only a source of depspise for them > You get your everymonth fucking 1000 e sitting here > whom are you teaching? This crowds of animals? > They and you you cant nothing Rants are so much funnier when the speaker can't speak English. From steve at REMOVE-THIS-cybersource.com.au Mon Jan 4 18:44:09 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 04 Jan 2010 23:44:09 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> Message-ID: <03526a3a$0$1336$c3e8da3@news.astraweb.com> On Mon, 04 Jan 2010 13:34:34 -0800, Michi wrote: > On Jan 4, 1:30?pm, Steven D'Aprano > wrote: >> >> This is very true, but good APIs often trade-off increased usability >> and reduced defect rate against machine efficiency too. In fact, I >> would argue that this is a general design principle of programming >> languages: since correctness and programmer productivity are almost >> always more important than machine efficiency, the long-term trend >> across virtually all languages is to increase correctness and >> productivity even if doing so costs some extra CPU cycles. > > Yes, I agree with that in general. Correctness and productivity are more > important, as a rule, and should be given priority. I'm glad we agree on that, but I wonder why you previously emphasised machine efficiency so much, and correctness almost not at all, in your previous post? >> > (For example, wrapper APIs often require additional memory >> > allocations and/or data copies.) Incorrect use of exceptions also >> > incurs an efficiency penalty. >> >> And? *Correct* use of exceptions also incur a penalty. So does the use >> of functions. Does this imply that putting code in functions is a poor >> API? Certainly not. > > It does imply that incorrect use of exceptions incurs an unnecessary > performance penalty, no more, no less, just as incorrect use of wrappers > incurs an unnecessary performance penalty. If all you're argument is that we shouldn't write crappy APIs, then I agree with you completely. The .NET example you gave previously is a good example of an API that is simply poor: using exceptions isn't a panacea that magically makes code better. So I can't disagree that using exceptions badly incurs an unnecessary performance penalty, but it also incurs an unnecessary penalty against correctness and programmer productivity. >> But no matter how much more expensive, there will always be a cut-off >> point where it is cheaper on average to suffer the cost of handling an >> exception than it is to make unnecessary tests. >> >> In Python, for dictionary key access, that cut-off is approximately at >> one failure per ten or twenty attempts. So unless you expect more than >> one in ten attempts to lead to a failure, testing first is actually a >> pessimation, not an optimization. > > What this really comes down to is how frequently or infrequently a > particular condition arises before that condition should be considered > an exceptional condition rather than a normal one. It also relates to > how the set of conditions partitions into "normal" conditions and > "abnormal" conditions. The difficulty for the API designer is to make > these choices correctly. The first case is impossible for the API designer to predict, although she may be able to make some educated estimates based on experience. For instance I know that when I search a string for a substring, "on average" I expect to find the substring present more often than not. I've put "on average" in scare-quotes because it's not a statistical average at all, but a human expectation -- a prejudice in fact. I *expect* to have searching succeed more often than fail, not because I actually know how many searches succeed and fail, but because I think of searching for an item to "naturally" find the item. But if I actually profiled my code in use on real data, who knows what ratio of success/failure I would find? In the second case, the decision of what counts as "ordinary" and what counts as "exceptional" should, in general, be rather obvious. (That's not to discount the possibility of unobvious cases, but that's probably a case that the function is too complex and tries to do too much.) Take the simplest description of what the function is supposed to do: (e.g. "find the offset of a substring in a source string"). That's the ordinary case, and should be returned. Is there anything else that the function may do? (E.g. fail to find the substring because it isn't there.) Then that's an exceptional case. (There may be other exceptional cases, which is another reason to prefer exceptions to magic return values. In general, it's much easier to deal with multiple exception types than it is to test for multiple magic return values. Consider a function that returns a pointer. You can return null to indicate an error. What if you want to distinguish between two different error states? What about ten error states?) I argue that as designers, we should default to raising an exception and only choose otherwise if there is a good reason not to. As we agreed earlier, exceptions (in general) are better for correctness and productivity, which in turn are (in general) more important than machine efficiency. The implication of this is that in general, we should prefer exceptions, and only avoid them when necessary. Your argument seems to be that we should avoid exceptions by default, and only use them if unavoidable. I think that is backwards. >> In some, limited, cases you might be able to use the magic return value >> strategy, but this invariably leads to lost programmer productivity, >> more complex code, lowered readability and usability, and more defects, >> because programmers will invariably neglect to test for the special >> value: > > I disagree here, to the extent that, whether something is an error or > not can very much depend on the circumstances in which the API is used. That's certainly true: a missing key (for example) may be an error, or a present key may be an error, or neither may be an error, just different branches of an algorithm. That's an application-specific decision. But I don't see how that relates to my claim that magic return values are less robust and usable than exceptions. Whether it is an error or not, it still needs to be handled. If the caller neglects to handle the special case, an exception-based strategy will almost certainly lead to the application halting (hopefully leading to a harmless bug report rather than the crash of a billion-dollar space probe), but a magic return value will very often lead to the application silently generating invalid results. [...] > Wanting to ignore a return value from a function is perfectly normal and > legitimate in many cases. I wouldn't say that's normal. If you don't care about the function's result, why are you calling it? For the side-effects? In languages that support procedures, such mutator functions should be written as procedures that don't return anything. For languages that don't, like Python, they should be written as de-facto procedures, always return None, and allow the user to pretend that nothing was returned. That is to say, ignoring the return value is acceptable as a work-around for the lack of true procedures. But even there, procedures necessarily operate by side-effect, and side-effects should be avoided as much as possible. So I would say, ideally, wanting to ignore the return value should be exceptionally rare. > However, if a function throws instead of > returning a value, ignoring that value becomes more difficult for the > caller and can extract a performance penalty that may be unacceptable to > the caller. There's that premature micro-optimization again. > The problem really is that, at the time the API is designed, > there often is no way to tell whether this will actually be the case; in > turn, no matter whether I choose to throw an exception or return an > error code, it will be wrong for some people some of the time. I've been wondering when you would reach the conclusion that an API should offer both forms. For example, Python offers both key-lookup that raises exceptions (dict[key]) and key-lookup that doesn't (dict.get(key)). The danger of this is that it complicates the API, leads to a more complex implementation, and may result in duplicated code (if the two functions have independent implementations). But if you don't duplicate the code, then the assumed performance benefit of magic return values over exceptions might very well be completely negated: def get(self, key): # This is not the real Python dict.get implementation! # This is merely an illustration of how it *could* be. try: return self[key] except KeyError: return None This just emphasises the importance of not optimising code by assumption. If you haven't *measured* the speed of a function you don't know whether it will be faster or slower than catching an exception. You will note that the above has nothing to do with the API, but is entirely an implementation decision. This to me demonstrates that the question of machine efficiency is irrelevant to API design. >> This is a classic example of premature optimization. Unless such >> inefficiency can be demonstrated to actually matter, then you do nobody >> any favours by preferring the API that leads to more defects on the >> basis of *assumed* efficiency. > > I agree with the concern about premature optimisation. However, I don't > agree with a blanket statement that special return values always and > unconditionally lead to more defects. I can't say that they *always* lead to more defects, since that also depends on the competence of the caller, but I will say that as a general principle, they should be *expected* to lead to more defects. > Returning to the .NET non- > blocking I/O example, the fact that the API throws an exception when it > shouldn't very much complicates the code and introduces a lot of extra > control logic that is much more likely to be wrong than a simple > if-then-else statement. As I said, throwing an exception when none > should be thrown can be just as harmful as the opposite case. In this case, it's worse than that -- they use a special return value when there should be an exception, and an exception when there should be an ordinary, non-special value (an empty string, if I recall correctly). >> It doesn't matter whether it is an error or not. They are called >> EXCEPTIONS, not ERRORS. What matters is that it is an exceptional case. >> Whether that exceptional case is an error condition or not is dependent >> on the application. > > Exactly. To me, that implies that making something an exception that, to > the caller, shouldn't be is just as inconvenient as the other way > around. Well, obviously I agree that you should only make things be an exception if they actually should be an exception. I don't quite see where the implication is -- I find myself in the curious position of agreeing with your conclusion while questioning your reasoning, as if you had said something like: All cats have four legs, therefore cats are mammals. >> > Apart from the potential performance penalty, throwing exceptions for >> > expected outcomes is bad also because it forces a try-catch block on >> > the caller. >> >> But it's okay to force a `if (result==MagicValue)` test instead? > > Yes, in some cases it is. For example: > > int numBytes; > int fd = open(...); > while ((numBytes = read(fd, ?)) > 0) { > // process data... > } > > Would you prefer to see EOF indicated by an exception rather than a zero > return value? I wouldn't. Why not? Assuming this is a blocking read, once you hit EOF you will never recover from it. Is this about the micro-optimisation again? Disc IO is almost certainly a thousand times slower than any exception you could catch here. In Python, we *do* use exceptions for file reads. An explicit read returns an empty string, and we might write: f = open(filename) while 1: block = f.read(buffersize) if not block: f.close() break process(block) This would arguably be easier to write and read, and demonstrates the intent of the while loop better: f = open(filename) try: while 1: process(f.read(buffersize)) except EOFError: f.close() (But the above doesn't work, because an explicit read doesn't raise an exception.) However, there's another idiom for reading a file which does use an exception: line-by-line reading. f = open(filename) for line in f: process(line) f.close() Because iterating over the file generates a StopIteration when EOF is reached, the for loop automatically breaks. If you wanted to handle that by hand, something like this should work (but is unnecessary, because Python already does it for you): f = open(filename) try: while 1: process(f.next()) except StopIteration: f.close() [...] > The core problem isn't whether exceptions are good or bad in a > particular case, but that most APIs make this an either-or choice. For > example, if I had an API that allowed me to choose at run time whether > an exception will be thrown for a particular condition, I could adapt > that API to my needs, instead of being stuck with whatever the designer > came up with. > > There are many ways this could be done. For example, I could have a > find() operation on a collection that throws if a value isn't found, and > I could have findNoThrow() if I want a sentinel value returned. Or, the > API could offer a callback hook that decides at run time whether to > throw or not. (There are many other possible ways to do this, such as > setting the behaviour at construction time, or by having different > collection types with different behaviours.) > > The point is that a more flexible API is likely to be more useful than > one that sets a single exception policy for everyone. This has costs of its own. The costs of developer education -- learning about, memorising, and deciding between such multiple APIs does not come for free. The costs of developing and maintaining the multiple functions. The risks of duplicated code in the implementation. The cost of writing documentation. A bloated API is not free of costs. >> > As the API >> > creator, if I indicate errors with exceptions, I make a policy >> > decision about what is an error and what is not. It behooves me to be >> > conservative in that policy: I should throw exceptions only for >> > conditions that are unlikely to arise during routine and normal use >> > of the API. >> >> But lost connections *are* routine and normal. Hopefully they are rare. > > In the context of my example, they are not. The range of behaviours > naturally falls into these categories: > > * No data ready > * Data ready > * EOF > * Socket error Right -- that fourth example is one of the NATURAL categories that any half-way decent developer needs to be aware of. When you say something isn't natural, and then immediately contradict yourself, that's a sign you need to think about what you really mean :) > The first three cases are the "normal" ones; they operate on the same > program state and they are completely expected: while reading a message > off the wire, the program will almost certainly encounter the first two > conditions and, if there is no error, it will always encounter the EOF > condition. I would call these the ordinary cases, as opposed to the exceptional cases. > The fourth case is the unexpected one, in the sense that this > case will often not arise at all. But it is still expected -- you have to expect that you might get a socket error, and code accordingly. > That's not to say that lost connections aren't routine; they are. Right -- we actually agree on this, we just disagree on the terminology. I believe that talking about "normal" and "errors" is misleading. Better is to talk about "ordinary" and "exceptional". > But, when a connection is lost, > the program has to do different things and operate on different state > than when the connection stays up. This strongly suggests that the first > three conditions should be dealt with by return values and/or out > parameters, and the fourth condition should be dealt with as an > exception. Agreed. -- Steven From keesvanschaik at gmail.com Mon Jan 4 18:50:16 2010 From: keesvanschaik at gmail.com (KvS) Date: Mon, 4 Jan 2010 15:50:16 -0800 (PST) Subject: Printing plain text with exact positioning on Windows Message-ID: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> Dear all, for a python program running on Win XP that is used to produce invoices I need to be able to have the printer print a page with just plain text, but positioned (almost) exactly at prescribed places (due to the use of 'pre formatted' invoice forms). In particular as close to the left and right edges of the page as possible. So far I've been using a combination of HTML & CSS, IE automation and fiddling in the registry to set left and right printing margins of IE to 0 and back to defaults after the printing is done. This worked ok, but recently some problem occured that I wasn't able to solve so far (seemingly random crashes, probably due to an update to IE 8, I've posted about this problem before: http://groups.google.com/group/comp.lang.python/browse_thread/thread/3922fdfa4db31ffe/ba8dc24c2b85252c?lnk=gst&q=KvS#ba8dc24c2b85252c but unfortunately no answers). So now I'm looking for other ways to get this printing job done. I know of Tim Goldens page about Windows printing: http://timgolden.me.uk/python/win32_how_do_i/print.html and have been googling a lot, but I'm still not convinced what the best way to go is. E.g. I can't get clear whether the combination reportlab & Acrobat Reader allows to print very close to the left and right edge of the paper (without a user having to change something in a Print dialog etc.). You would expect this is a relatively typical problem and others have ran into it before me. Any thoughts on this? Thanks in advance, Kees From aleksandar27 at BRISIOVOnet.hr Mon Jan 4 18:58:26 2010 From: aleksandar27 at BRISIOVOnet.hr (alejandro) Date: Tue, 5 Jan 2010 00:58:26 +0100 Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> Message-ID: I think the easyest way for printing text is with ReportLab. Just few lines of code and you have a nice pdf... From keesvanschaik at gmail.com Mon Jan 4 19:15:18 2010 From: keesvanschaik at gmail.com (KvS) Date: Mon, 4 Jan 2010 16:15:18 -0800 (PST) Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> Message-ID: <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> On Jan 4, 11:58?pm, "alejandro" wrote: > I think the easyest way for printing text is with ReportLab. Just few lines > of code and you have a nice pdf... Ok, thanks. In that scenario I would also need to be able to programatically adjust the printing margins in Acrobat reader, i.e. automate it to some extent. Is that also possible? From cs at zip.com.au Mon Jan 4 19:23:34 2010 From: cs at zip.com.au (Cameron Simpson) Date: Tue, 5 Jan 2010 11:23:34 +1100 Subject: IOError - cannot create file (linux daemon-invoked script) In-Reply-To: References: Message-ID: <20100105002334.GA10849@cskk.homeip.net> On 04Jan2010 09:16, cassiope wrote: | To Cameron: the file doesn't (yet) exist; and it has the correct full | path. Can you show us the strace output of the failing open() call? | To "Nobody" : hey, this seems interesting. First test, invoking | seteuid() | and setegid() didn't help - but strange thing was these calls didn't | show | up in the strace, so perhaps I wasn't testing what I thought I was. If you're using the "-e trace=file" option it won't. That constrains the output to file operations to make the log easier to read. Discard the -e option to get everything. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Rimmer: It will be happened; it shall be going to be happening; it will be was an event that could will have been taken place in the future. - Red Dwarf, _Future Echoes_ From aioe.org at technicalbloke.com Mon Jan 4 19:52:56 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 00:52:56 +0000 Subject: TypeError: can only concatenate list (not "tuple") to list References: Message-ID: Gabriel Genellina wrote: > En Mon, 04 Jan 2010 04:58:54 -0300, Chris Rebert > escribi?: >> On Sun, Jan 3, 2010 at 11:51 PM, Gabriel Genellina >> wrote: > >>> py> [1,2,3] + (4,5) >>> Traceback (most recent call last): >>> File "", line 1, in >>> TypeError: can only concatenate list (not "tuple") to list > > Sorry, I inadvertedly posted an incomplete message. Note the last part: > >>> In-place addition += does work: >>> >>> py> a = [1,2,3] >>> py> a += (4,5) >>> py> a >>> [1, 2, 3, 4, 5] > >> Given that tuples are sometimes used as a poor man's object (i.e. >> collection of data fields), whereas lists are not typically used that >> way, I'd say it's probably a good thing an explicit type conversion is >> required here. > > In that case += should not be allowed either... > I'd be strongly inclined to think the result would be the sequence on the left with the data from the second sequence appended to it. What's wrong with a little duck typing here eh? Roger. From aleksandar27 at BRISIOVOnet.hr Mon Jan 4 20:01:33 2010 From: aleksandar27 at BRISIOVOnet.hr (alejandro) Date: Tue, 5 Jan 2010 02:01:33 +0100 Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> Message-ID: Don't remember much.. but maybe you could play with canvas? I think i don't undertand(i am a croatian so english is not my mother language). If you have the need that the pdf and the printer setup have lets say the same paper size, sorry but can't help. If you want lets say different printer margins for different documents/pages just calculate them. A4 is 210x297mm and you have need for a margin of 10mm.. width = 210 - margin height = 297 - margin now for positioning use the variables instead the absolute positioning.. i think that reportlab has some methods that alow you to position text/photos from the left top corner instead the normal positioning. Don't know if I helped you in any way... Good luck! "KvS" wrote in message news:54eb4786-5db3-453d-971a-a4b359fc2de8 at f5g2000yqh.googlegroups.com... On Jan 4, 11:58 pm, "alejandro" wrote: > I think the easyest way for printing text is with ReportLab. Just few > lines > of code and you have a nice pdf... Ok, thanks. In that scenario I would also need to be able to programatically adjust the printing margins in Acrobat reader, i.e. automate it to some extent. Is that also possible? From gagsl-py2 at yahoo.com.ar Mon Jan 4 20:01:39 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 04 Jan 2010 22:01:39 -0300 Subject: FileInput too slow References: <4b426d17$0$1113$4fafbaef@reader4.news.tin.it> Message-ID: En Mon, 04 Jan 2010 19:35:02 -0300, wiso escribi?: > I'm trying the fileinput module, and I like it, but I don't understand > why > it's so slow... look: > > from time import time > from fileinput import FileInput > > file = ['r1_200907.log', 'r1_200908.log', 'r1_200909.log', > 'r1_200910.log', > 'r1_200911.log'] > > def f1(): > n = 0 > for f in file: > print "new file: %s" % f > ff = open(f) > for line in ff: > n += 1 > ff.close() > return n > > def f2(): > f = FileInput(file) > for line in f: > if f.isfirstline(): print "new file: %s" % f.filename() > return f.lineno() > > def f3(): # f2 simpler > f = FileInput(file) > for line in f: > pass > return f.lineno() Yes, the fileinput module is A LOT slower than normal file processing. You may use itertools.chain instead: def f4(): f = itertools.chain.from_iterable(open(fn) for fn in file) n = 0 for line in f: n += 1 return n I get similar timings as f1() above. Known major issues of this "poor man's" implementation: - no lineno/filelineno/isfirstline attributes - close() is implicit - only for reading; inplace and backup don't work -- Gabriel Genellina From aleksandar27 at BRISIOVOnet.hr Mon Jan 4 20:05:22 2010 From: aleksandar27 at BRISIOVOnet.hr (alejandro) Date: Tue, 5 Jan 2010 02:05:22 +0100 Subject: mechanize module problem Message-ID: When I inport it wia python command line it all looks fine but if i try to run it from eclipse it gives me an error that there is no package mechanize. Have anybody had this situation? win XP python26 From chris.gonnerman at newcenturycomputers.net Mon Jan 4 20:15:29 2010 From: chris.gonnerman at newcenturycomputers.net (Chris Gonnerman) Date: Mon, 04 Jan 2010 19:15:29 -0600 Subject: [Python] Printing plain text with exact positioning on Windows In-Reply-To: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> Message-ID: <4B4292B1.9030602@newcenturycomputers.net> KvS wrote: > So now I'm looking for other ways to get this printing job done. I > know of Tim Goldens page about Windows printing: > http://timgolden.me.uk/python/win32_how_do_i/print.html and have been > googling a lot, but I'm still not convinced what the best way to go > is. E.g. I can't get clear whether the combination reportlab & Acrobat > Reader allows to print very close to the left and right edge of the > paper (without a user having to change something in a Print dialog > etc.). > I have a page, and a module, for Windows printing which gives a lot of control. http://newcenturycomputers.net/projects/pythonicwindowsprinting.html But in your case, I think I'd consider ReportLab. Also, it seems to me that I've seen a simpler PDF generator module for Python. Dunno where, though. -- Chris. From aleksandar27 at BRISIOVOnet.hr Mon Jan 4 20:26:15 2010 From: aleksandar27 at BRISIOVOnet.hr (alejandro) Date: Tue, 5 Jan 2010 02:26:15 +0100 Subject: mechanize module problem References: Message-ID: resolved! don't know why or how but it is... just restared eclipse for the fourth time "alejandro" wrote in message news:hhu38i$h21$1 at ss408.t-com.hr... > When I inport it wia python command line it all looks fine but if i try to > run it from eclipse it gives me an error that there is no package > mechanize. Have anybody had this situation? > win XP > python26 > > From wuwei23 at gmail.com Mon Jan 4 20:34:17 2010 From: wuwei23 at gmail.com (alex23) Date: Mon, 4 Jan 2010 17:34:17 -0800 (PST) Subject: python mechanize proxy support question References: <27009696.post@talk.nabble.com> <4ca2b83a-3b82-4d20-85a3-b6304f25fb81@21g2000yqj.googlegroups.com> Message-ID: <60cd73ea-1bb6-4fc8-9992-27115cc8ff09@j4g2000yqe.googlegroups.com> On Jan 4, 9:25?pm, elca wrote: > that is only support mechanize.browser module.. actually > im looking mechanize.urlopen method. >From the docs: In these examples, the workings are hidden inside the mechanize.urlopen () function, which is an extension of urllib2.urlopen(). Redirects, proxies and cookies are handled automatically by this function (note that you may need a bit of configuration to get your proxies correctly set up: see urllib2 documentation). http://wwwsearch.sourceforge.net/mechanize/doc.html From aahz at pythoncraft.com Mon Jan 4 20:35:24 2010 From: aahz at pythoncraft.com (Aahz) Date: 4 Jan 2010 17:35:24 -0800 Subject: Design question about pretree classifier References: <7351dfbf-90a2-4303-b3d8-2377dcd2698e@o28g2000yqh.googlegroups.com> Message-ID: In article , Steve Holden wrote: >Julian wrote: >> >> But: >> >> - none classification: return an exception or None? I think None is >> better, hence its not an exception that there is no classification but >> a defined state. What do you think? >> - many classifications: what to do? retun a sequence of strings? raise >> an exception and implement another method wich returns than the >> classifications? what should I do here? > >Always return a list or tuple. For no classifications it should be >empty, for one classification it should have one element, ... , for N >classifications it should have N elements. Why not a set? If you're only going list classifications, a tuple or list would be fine, but if you're going to probe classifications, you should use something that isn't O(N). -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From aahz at pythoncraft.com Mon Jan 4 20:36:52 2010 From: aahz at pythoncraft.com (Aahz) Date: 4 Jan 2010 17:36:52 -0800 Subject: Using PyImport_ExtendInittab with package References: Message-ID: In article , Julien Danjou wrote: > >I'm trying to embed Python and therefore use PyImport_ExtendInittab() to >register modules. >My current problem is that, if it works well with a simple module >"hello", naming a module "hello.foobar" in the inittab struct does not >seems to work. >imp.find_module("hello.foobar") returns correctly that the module is >found, but import hello.foobar fails badly. > >Is . notation supported in inittab? Am I doing something wrong? This is probably wrong in some detail, but I think you need to import each level of a package separately. I.e. you need to import 'hello' before you can import 'hello.foobar'. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From wuwei23 at gmail.com Mon Jan 4 20:41:36 2010 From: wuwei23 at gmail.com (alex23) Date: Mon, 4 Jan 2010 17:41:36 -0800 (PST) Subject: Newbie help- Can multiple instances with multiple names automatically created. References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> Message-ID: <14370388-6f6e-451c-b8a6-4ce2f6ec32fc@l30g2000yqb.googlegroups.com> On Jan 5, 9:33?am, Nav wrote: > what are the risks of globalnamespace use You're unnecessarily tying your code to the implementation. > and what are the benefits? Absolutely none that using a dictionary doesn't also give you. From keesvanschaik at gmail.com Mon Jan 4 20:50:13 2010 From: keesvanschaik at gmail.com (KvS) Date: Mon, 4 Jan 2010 17:50:13 -0800 (PST) Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> Message-ID: <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> On Jan 5, 1:01?am, "alejandro" wrote: > Don't remember much.. but maybe you could play with canvas? > I think i don't undertand(i am a croatian so english is not my mother > language). If you have the need that the pdf and the printer setup have lets > say the same paper size, sorry but can't help. > If you want lets say different printer margins for different documents/pages > just calculate them. > A4 is 210x297mm and you have need for a margin of 10mm.. > width = 210 - margin > height = 297 - margin > now for positioning use the variables instead the absolute positioning.. > > i think that reportlab has some methods that alow you to position > text/photos from the left top corner instead the normal positioning. > > Don't know if I helped you in any way... > > Good luck! > > "KvS" wrote in message > > news:54eb4786-5db3-453d-971a-a4b359fc2de8 at f5g2000yqh.googlegroups.com... > On Jan 4, 11:58 pm, "alejandro" wrote: > > > I think the easyest way for printing text is with ReportLab. Just few > > lines > > of code and you have a nice pdf... > > Ok, thanks. In that scenario I would also need to be able to > programatically adjust the printing margins in Acrobat reader, i.e. > automate it to some extent. Is that also possible? I trust I can do that in reportlab. But then if you print the pdf, you would still get some margin between the edge of the paper and the text due to the printing settings of Acrobat Reader basically. I need the text to be as close as possible to the actual edge of the paper, so I need to somehow make those margins as small as possible as well... From aioe.org at technicalbloke.com Mon Jan 4 20:58:09 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 01:58:09 +0000 Subject: Off Topic ( was Re: Significant whitespace) References: <4B3E70FC.1010702@gmail.com> Message-ID: Steve Holden wrote: > r0g wrote: >> David Robinow wrote: >>> On Sun, Jan 3, 2010 at 10:42 PM, Roy Smith wrote: >>>> In article , >>>> David Robinow wrote: >>>> >>>>> On Sun, Jan 3, 2010 at 8:09 PM, Tim Roberts wrote: >>>>>> More than "not required", it was "not relevant". This led to one of the >>>>>> most infamous programming blunders in the early days of the space program, >>>>>> when one programmer accidentially typed a period instead of a comma >>>>>> resulting in the loss of a satellite: >>>>> Interesting story. Did you make it up? >>>> It's a fairly well known story. >>>> >>>> http://catless.ncl.ac.uk/Risks/5.64.html#subj4.2 >>> Sure. But the question is, "Who made it up?" >>> http://en.wikiquote.org/wiki/Fortran >> completely appropriate and proportional to Tim's sickening crimes >> against fact. >> >> Roger. > > Personally I think the overall tone of this group would have been > marginally improved if you had summoned the self-restraint to simply not > make this posting. > > Fortunately it's still an unmoderated group, so you can continue to be > as hostile and snarky as you like. > > regards > Steve Well I could say the same thing Steve: you could also not have posted and thereby spared the group even more off topic noise, I'd have quite happily left it there. Of course our emotional urges trump rationality almost every time so I understand your urge to address what has upset you, much as I hope you understand my subsequent urge to respond. I appreciate this is a professional programming forum and that my post was crass in the extreme but I get really fed up with the needless pedantry of computer geeks sometimes; not to mention the insufferably sanctimonious tone they adopted whilst engaging in it. It ranges from semi-conscious passive aggression to out and out dick swinging and I think the "overall tone of this group" would be "marginally improved" by doing away with it. There are perfectly good way's of putting things that don't require taunting and smugness. Contrast the clearly snarky personal challenge... "Interesting story. Did you make it up?" ...with the far more convivial correction... "Interestingly it was a missing underscore character ;) see http://example.com/rahrarhrah"... Both could satisfy the inner pedant's demands for attention, why choose the first? Personally I think the first respondent should have "summoned the self restraint" to just let it drop too, but they didn't and I feel they did the OP an injustice by that. My apologies for the noise everybody, I shall say no more on the subject. Cheers, Roger. From wuwei23 at gmail.com Mon Jan 4 21:14:17 2010 From: wuwei23 at gmail.com (alex23) Date: Mon, 4 Jan 2010 18:14:17 -0800 (PST) Subject: Can't Add Variable References: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> Message-ID: <5b5fd3f1-c5b5-4b65-b1bd-f7f64a9ba079@m25g2000yqc.googlegroups.com> On Jan 5, 3:40?am, Joe Riopel wrote: > On Mon, Jan 4, 2010 at 12:28 PM, Victor Subervi wrote: > > Hi; > > I have this code snippet: > > > ??? sql '''create table if not exists %sCustomerData ( > > I think you may have forgotten the "=" after sql. Jesus wept. How long is this list going to continue to finish Victor's project for him? These questions _clearly_ belong on python-tutor. From python at mrabarnett.plus.com Mon Jan 4 21:25:55 2010 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 05 Jan 2010 02:25:55 +0000 Subject: Can't Add Variable In-Reply-To: <5b5fd3f1-c5b5-4b65-b1bd-f7f64a9ba079@m25g2000yqc.googlegroups.com> References: <4dc0cfea1001040928o75929377g380713b6c187f5c9@mail.gmail.com> <5b5fd3f1-c5b5-4b65-b1bd-f7f64a9ba079@m25g2000yqc.googlegroups.com> Message-ID: <4B42A333.7060003@mrabarnett.plus.com> alex23 wrote: > On Jan 5, 3:40 am, Joe Riopel wrote: >> On Mon, Jan 4, 2010 at 12:28 PM, Victor Subervi wrote: >>> Hi; >>> I have this code snippet: >>> sql '''create table if not exists %sCustomerData ( >> I think you may have forgotten the "=" after sql. > > Jesus wept. How long is this list going to continue to finish Victor's > project for him? > > These questions _clearly_ belong on python-tutor. Who is this "Jesus" person? You should remember that this is a predominantly Guido-worshipping list! :-) From aioe.org at technicalbloke.com Mon Jan 4 21:31:34 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 02:31:34 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> Message-ID: Michi wrote: > On Jan 4, 1:30 pm, Steven D'Aprano > wrote: >> In some, limited, cases you might be able to use the magic return value >> strategy, but this invariably leads to lost programmer productivity, more >> complex code, lowered readability and usability, and more defects, >> because programmers will invariably neglect to test for the special value: > > I disagree here, to the extent that, whether something is an error or > not can very much depend on the circumstances in which the API is > used. The collection case is a very typical example. Whether failing > to locate a value in a collection is an error very much depends on > what the collection is used for. In some cases, it's a hard error > (because it might, for example, imply that internal program state has > been corrupted); in other cases, not finding a value is perfectly > normal. A pattern I have used a few times is that of returning an explicit success/failure code alongside whatever the function normally returns. While subsequent programmers might not intuit the need to test for (implicit) "magic" return values they ought to notice if they start getting tuples back where they expected scalars... def foo(x) if x>0: return True, x*x else: return False, "Bad value of x in foo:",str(x) ok, value = foo(-1) if ok: print "foo of x is", value else: print "ERROR:", value Roger. From steven at REMOVE.THIS.cybersource.com.au Mon Jan 4 21:35:03 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 05 Jan 2010 02:35:03 GMT Subject: TypeError: can only concatenate list (not "tuple") to list References: Message-ID: On Tue, 05 Jan 2010 00:52:56 +0000, r0g wrote: > I'd be strongly inclined to think the result would be the sequence on > the left with the data from the second sequence appended to it. What's > wrong with a little duck typing here eh? That's not the existing behaviour. List concatenation doesn't mutate the left hand list, it creates a new list: >>> L = [1, 2, 3] >>> L2 = L + [4, 5, 6] >>> L [1, 2, 3] >>> L2 [1, 2, 3, 4, 5, 6] But if you insist on in-place modification, why do you prefer appending the right hand sequence to the left instead of prepending the left hand sequence to the right? -- Steven From aioe.org at technicalbloke.com Mon Jan 4 22:01:52 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 03:01:52 +0000 Subject: TypeError: can only concatenate list (not "tuple") to list References: Message-ID: Steven D'Aprano wrote: > On Tue, 05 Jan 2010 00:52:56 +0000, r0g wrote: > >> I'd be strongly inclined to think the result would be the sequence on >> the left with the data from the second sequence appended to it. What's >> wrong with a little duck typing here eh? OK, I hadn't read all the other responses when I posted, some of which make fair points why this wouldn't be wise. Fair enough. > > That's not the existing behaviour. List concatenation doesn't mutate the > left hand list, it creates a new list: > I would expect it to append. That's my prejudice though, as I do that far more often :/ > >>>> L = [1, 2, 3] >>>> L2 = L + [4, 5, 6] >>>> L > [1, 2, 3] >>>> L2 > [1, 2, 3, 4, 5, 6] > > > But if you insist on in-place modification, why do you prefer appending > the right hand sequence to the left instead of prepending the left hand > sequence to the right? > > > In-place seems more natural for a mutable type. I admit the left right thing is my prejudice though, western cultural bias I suppose. Its not entirely unprecedented though, the parser reads left to right and the leftmost terms take precedent in lazy logic evaluation. Still, the responses to this have convinced me the + operator shouldn't make assumptions, I'm more open to how += works though as it implies in-place and the left over right precedent quite nicely. Roger. From navjotmusic at gmail.com Mon Jan 4 22:12:53 2010 From: navjotmusic at gmail.com (Nav) Date: Mon, 4 Jan 2010 19:12:53 -0800 (PST) Subject: Newbie help- Can multiple instances with multiple names automatically created. References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> <14370388-6f6e-451c-b8a6-4ce2f6ec32fc@l30g2000yqb.googlegroups.com> Message-ID: <1df4f606-8e46-4ff1-9470-583e35612b25@r5g2000yqb.googlegroups.com> Okay, let me ask another question: When we create instances of objects by doing x = className () are we using globalnamespace? if yes then: if using globalnamespace is bad then why does every book or tutorial about python classes give the above style of assignment as an example? Second why do we use a dictionary to create something like this? I know how it works, but what is wrong with simply creating instances automatically? Once created the data for the instances is automatically saved in their own space? Why have a round about way using dictionaries? Is there an advantage or does it conflict with something else? Why not have for i in specialList: #I presume it would have to be special, because it can't be a problematic type i = className(whatever) > > > what are the risks of globalnamespace use > > You're unnecessarily tying your code to the implementation. > > > and what are the benefits? > > Absolutely none that using a dictionary doesn't also give you. From steven at REMOVE.THIS.cybersource.com.au Mon Jan 4 22:27:36 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 05 Jan 2010 03:27:36 GMT Subject: FileInput too slow References: <4b426d17$0$1113$4fafbaef@reader4.news.tin.it> Message-ID: On Mon, 04 Jan 2010 23:35:02 +0100, wiso wrote: > I'm trying the fileinput module, and I like it, but I don't understand > why it's so slow... Because it is written for convenience, not speed. From the source code: "Performance: this module is unfortunately one of the slower ways of processing large numbers of input lines." > look: > > from time import time > from fileinput import FileInput > > file = ['r1_200907.log', 'r1_200908.log', 'r1_200909.log', > 'r1_200910.log', 'r1_200911.log'] > > def f1(): > n = 0 > for f in file: > print "new file: %s" % f > ff = open(f) > for line in ff: > n += 1 > ff.close() > return n > > def f2(): > f = FileInput(file) > for line in f: > if f.isfirstline(): print "new file: %s" % f.filename() > return f.lineno() > > def f3(): # f2 simpler > f = FileInput(file) > for line in f: > pass > return f.lineno() > > > t = time(); f1(); print time()-t # 1.0 > t = time(); f2(); print time()-t # 7.0 !!! > t = time(); f3(); print time()-t # 5.5 > > I'm using text files, there are 2563150 lines in total. The extra second and a half in f2() is probably due to the time it takes to call f.isfirstline() 2563150 times. -- Steven From tjreedy at udel.edu Mon Jan 4 22:30:52 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 04 Jan 2010 22:30:52 -0500 Subject: FileInput too slow In-Reply-To: <4b426d17$0$1113$4fafbaef@reader4.news.tin.it> References: <4b426d17$0$1113$4fafbaef@reader4.news.tin.it> Message-ID: On 1/4/2010 5:35 PM, wiso wrote: > I'm trying the fileinput module, and I like it, but I don't understand why > it's so slow... look: > > from time import time > from fileinput import FileInput > > file = ['r1_200907.log', 'r1_200908.log', 'r1_200909.log', 'r1_200910.log', > 'r1_200911.log'] > > def f1(): > n = 0 > for f in file: > print "new file: %s" % f > ff = open(f) > for line in ff: > n += 1 > ff.close() > return n > > def f2(): > f = FileInput(file) > for line in f: > if f.isfirstline(): print "new file: %s" % f.filename() > return f.lineno() > > def f3(): # f2 simpler > f = FileInput(file) > for line in f: > pass > return f.lineno() > > > t = time(); f1(); print time()-t # 1.0 > t = time(); f2(); print time()-t # 7.0 !!! > t = time(); f3(); print time()-t # 5.5 > > > I'm using text files, there are 2563150 lines in total. 1. Timings should include platform and Python version. 2. fileinput executes a lot of Python code on top of the underlying file methods. Your n += 1 is inadequate as compensation. Fileinput does at least the following for each line : try: line = self._buffer[self._bufindex] except IndexError: pass else: self._bufindex += 1 self._lineno += 1 self._filelineno += 1 That is 5 attribute accesses, an indexing, and 3 additions 3. You are welcome to read the Python source in .../pythonxy/Lib/fileinput.py 4. Doc string for 3.1 version says "Performance: this module is unfortunately one of the slower ways of processing large numbers of input lines. Nevertheless, a significant speed-up has been obtained by using readlines(bufsize) instead of readline(). A new keyword argument, bufsize=N, is present on the input() function and the FileInput() class to override the default buffer size." If your version has bufsize, try something larger than the default of 8*1024, say 1024*1024. Terry Jan Reedy From larry at hastings.org Mon Jan 4 22:39:10 2010 From: larry at hastings.org (Larry Hastings) Date: Mon, 04 Jan 2010 19:39:10 -0800 Subject: Best practices for simultaneously installed versioned packages? In-Reply-To: References: <4B41CC6C.3020808@hastings.org> Message-ID: <4B42B45E.9070900@hastings.org> > In article > , > Benjamin Kaplan wrote: > >> wxpython installs a "wxversion" module which has functions like >> getInstalled(), ensureMinimal(version), and select(version). You can >> call wxversion.select before importing wx and it will make sure that >> the correct version is imported. You might want to look up what they >> did. >> I can imagine what they did: wxversion.select() explicitly imports the desired wxwindows module using __import__(), then assigns it to sys.modules["wxwindows"]. A good idea! Ned Deily wrote: > Also, setuptools (and, its successor, distribute, which supports Python > 3) allow the installation and management of multiple versions of a > package within one python site-library instance. > That sounds lovely too, but I can't figure out what facility setuptools / distribute permits that. Could you be more specific? A link to the documentation for this feature would be much appreciated. Thanks, /larry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven at REMOVE.THIS.cybersource.com.au Mon Jan 4 22:52:02 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 05 Jan 2010 03:52:02 GMT Subject: Newbie help- Can multiple instances with multiple names automatically created. References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> <14370388-6f6e-451c-b8a6-4ce2f6ec32fc@l30g2000yqb.googlegroups.com> <1df4f606-8e46-4ff1-9470-583e35612b25@r5g2000yqb.googlegroups.com> Message-ID: On Mon, 04 Jan 2010 19:12:53 -0800, Nav wrote: > Okay, let me ask another question: > > When we create instances of objects by doing > x = className() > are we using globalnamespace? That depends on whether you are doing x = className() inside a function (or class), or in the top level of the program. If I do this: x = 1234 def function(): y = 4567 then x is defined in the global namespace and y is defined in the namespace which is local to function(). > if yes then: > if using globalnamespace is bad then why does every book or tutorial > about python classes give the above style of assignment as an example? No, you're confused -- the problem isn't with using the global namespace. The problem is that you don't know what names you want to use ahead of time. You use assignment like: x = something() when you know the name x when you are writing the code. That way you can write x in the code, and all is good: x = something() print x.method() mydict = {x: -1} assert mydict.keys() == [x] Now, imagine that you didn't know what names you have to use. Say, for example, that you need a variable number of Somethings: a = Something() b = Something() c = Something() d = Something() # I never know when to stop... z = Something() # ... keep going? too far? who knows??? HELP! process(a) process(b) process(c) # when do I stop??? process(x) process(y) # ... That's the wrong way to deal with it. So instead you use a list: mylist = [] # define ONE NAME in the global namespace for i in range(some_number): mylist.append(Something()) # later... for x in mylist: # again, we use ONE name, `x` process(x) A list implicitly maps numbers (the position) to values. If you want to map strings (names) to values, use a dict. Here is an example. Notice we don't know how many players there are, or what their names are: print "Welcome to the game." print "Please enter the name of each player," print "or the word 'STOP' when there are no more players." players = {} i = 1 while True: # loop forever name = raw_input("Name of player %d? " % i) name = name.strip() # get rid of extra whitespace if name.upper() == 'STOP': # we're done break players[name] = NewPlayer() # much later... for name, player in players.items(): print "Welcome, player %s" % name play_game(player) The only downside is that dicts are unordered, so the order of the players is not the same as the order they were entered in. -- Steven From wuwei23 at gmail.com Mon Jan 4 22:56:51 2010 From: wuwei23 at gmail.com (alex23) Date: Mon, 4 Jan 2010 19:56:51 -0800 (PST) Subject: Newbie help- Can multiple instances with multiple names automatically created. References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> <14370388-6f6e-451c-b8a6-4ce2f6ec32fc@l30g2000yqb.googlegroups.com> <1df4f606-8e46-4ff1-9470-583e35612b25@r5g2000yqb.googlegroups.com> Message-ID: <09b79018-e226-4f7e-8bf6-78ada2ad3c77@22g2000yqr.googlegroups.com> On Jan 5, 1:12?pm, Nav wrote: > When we create instances of objects by doing > x = className () > are we using globalnamespace? Well, you're using a namespace, which namespaces its in would depend on the scope in which that assignment occurred. And there's not really a global namespace, only module ones. > if yes then: > ? if using globalnamespace is bad then why does every book or tutorial > about python classes give the above ?style of assignment as an > example? That's a basic assignment example. It's not a direct manipulation of globals(), like the solution given by Jan, which you seem to feel is the answer. And unless it happens outside of a nested scope like a class or function definition, it's not 'global'. > Second why do we use a dictionary to create something like this? I > know how it works, but what is wrong with simply creating instances > automatically? If I came across code like this in a joint project I was working on: def somefunc(*someparams): magical_instance_injection() for elem in magically_appearing_collection: .... We'd have serious words about the readability & maintainability of such code. If the values are important, make a clear set up that places them in a mapped collection, and make it obvious that the collection is how those values are used. Also, you've removed the set up from the place where the values are actually used, meaning I'd have to look at two sets of code to identify the values' names, as well as to confirm that 'some_random_label' _is_ actually one of the instances and not just a value you're using without instantiating. > Once created the data for the instances is > automatically saved in their own space? Why have a round about way > using dictionaries? Because using a dictionary removes the dependency your function has on the injection code, which in turns makes it more flexible for re-use. Let's assume you write your injection function, and call it within every function that uses those objects: def inject_values(): # here be global() manipulations def do_that_thang(): inject_values() ... What if you want to do_that_thang to a separate set of values? With this approach, you'd have to extend like so: def inject_values2(): ... def do_that_thang2(): inject_values2() ... You could, of course, have one inject_values function that accepts a flag and returns the appropriate set of values, but now you're having to modify that function every time the values change. With a dictionary, it's just so much simpler: values1 = dict( a = SomeObject(1), b = AnotherObject(1)... ) values2 = dict( a = SomeObject(2), b = AnotherObject(2)... ) def do_that_thang(values): c = values['a'].thanger(values['b']) ... It's clean, readable and far more easily extensible. It's also far more obvious what you're doing, which you'll really appreciate when you return to the code in a year's time :) From mensanator at aol.com Mon Jan 4 23:44:01 2010 From: mensanator at aol.com (Mensanator) Date: Mon, 4 Jan 2010 20:44:01 -0800 (PST) Subject: How do you configure IDLE on a Mac, seeing as there's no Configure Option... References: Message-ID: <54d9a742-d7fb-44b0-9cf6-ffa49ca5f2f4@u41g2000yqe.googlegroups.com> On Jan 4, 10:05?am, Benjamin Kaplan wrote: > On Sun, Jan 3, 2010 at 8:42 PM, Mensanator wrote: > > ...because there's no [Options] menu on the shell window? > > > Or at least give me a clue to how to use Courier New font? > > > For some inscrutable reason, depite the plethora of formatting tools, > > someone decided that proportional spaced fonts ought to be the > > default for IDLE. > > -- > >http://mail.python.org/mailman/listinfo/python-list > > Mac Applications don't have "options" menus. They have a "preferences" > pane which is the second item in the application menu Sometimes it's the second item, sometimes it's not > (the one that > takes the name of the active program) and has a keyboard shortcut of > Cmd-, > > Apple's user-interface guidelines are pretty specific, so you'll find > that this works for all native OS X (i.e. not X11) applications. Right, it doesn't apply to IDLE which uses X11 (there is a preferences window, but it doesn't have any font control, unlike stuff like Safari). I assume there must be a configuration file. Any idea what that file is called? From fpm at u.washington.edu Tue Jan 5 00:17:58 2010 From: fpm at u.washington.edu (cassiope) Date: Mon, 4 Jan 2010 21:17:58 -0800 (PST) Subject: IOError - cannot create file (linux daemon-invoked script) References: Message-ID: On Jan 4, 4:23?pm, Cameron Simpson wrote: > On 04Jan2010 09:16, cassiope wrote: > | To Cameron: the file doesn't (yet) exist; and it has the correct full > | path. > > Can you show us the strace output of the failing open() call? Ah...presumably you mean: [pid 1976] open("/var/tmp/share/lvrq-Robert.Smith", O_WRONLY| O_CREAT|O_TRUNC, 0666) = -1 EACCES (Permission denied) > | To "Nobody" : hey, this seems interesting. ?First test, invoking > | seteuid() > | and setegid() didn't help - but strange thing was these calls didn't > | show > | up in the strace, so perhaps I wasn't testing what I thought I was. > > If you're using the "-e trace=file" option it won't. That constrains the > output to file operations to make the log easier to read. Discard the -e > option to get everything. > > Cheers, > -- > Cameron Simpson DoD#743http://www.cskk.ezoshosting.com/cs/ Wasn't using the -e option. It turns out that the compiler was changing the code to use the linux functions setresgid() and setresuid(). That's why I didn't see it previously. If I only use the seteuid/setegid, it "works" - it is able to write the file. Unfortunately this may be partly due to its incompletely dropping priviledges - the file has root ownership, not "lesser user" ownership. Using setuid/setgid or setresuid/setresgid where just real OR both real and effective identities are set to "lesser user" - it still doesn't work - no file is written. Again, "lesser user" has no problem writing a file into this directory. I remain mystified :( Thanks for your valiant efforts! From mensanator at aol.com Tue Jan 5 00:24:55 2010 From: mensanator at aol.com (Mensanator) Date: Mon, 4 Jan 2010 21:24:55 -0800 (PST) Subject: How do you configure IDLE on a Mac, seeing as there's no Configure Option... References: Message-ID: <0d70cb54-3d77-4176-b621-e764ecf613b7@26g2000yqo.googlegroups.com> On Jan 4, 2:25?pm, Ned Deily wrote: > In article > , > ?Benjamin Kaplan wrote: > > > On Sun, Jan 3, 2010 at 8:42 PM, Mensanator wrote: > > > ...because there's no [Options] menu on the shell window? > > > > Or at least give me a clue to how to use Courier New font? > > > > For some inscrutable reason, depite the plethora of formatting tools, > > > someone decided that proportional spaced fonts ought to be the > > > default for IDLE. > > Mac Applications don't have "options" menus. They have a "preferences" > > pane which is the second item in the application menu (the one that > > takes the name of the active program) and has a keyboard shortcut of > > Cmd-, > > > Apple's user-interface guidelines are pretty specific, so you'll find > > that this works for all native OS X (i.e. not X11) applications. > > But be aware that IDLE on OS X has had various problems with missing > and/or duplicate menu items. ?IIRC, the current python.org OS X IDLEs > (2.6.4 and 3.1.1) should be OK and you can modify the font via the > Preferences menu No, I already checked. Both system preferences and X11 preferences. > (and currently changed preferences apply to all > versions of IDLE). ?Note also that there are two different ways to > launch the OS X IDLE. ?python.org installers put a double-clickable > IDLE.app in /Applications/Python n.x. Yeah, I've still got that one installed, but it has the broken distutils from python.org. >?There is also a command-line > "idlen.n" in /Library/Frameworks/Python.framework/Versions/n.n/bin > which, optionally, has a symlink from /usr/local/bin. AARRGGHH!!! I'm pulling my hair out! I haven't been using the IDLE from the broken python.org disk image. I assume I've been using the IDLE from macports. From the command prompt I've been typing "idle". This launches a "shell" window which appears to have an X11 parent application for which there are no "preferences" applicable to fonts. However, if I use the quick launcher from the python.org, I get a "shell" whose parent is named "IDLE"! And that one has a completely different preferences, one similar the the Windows Configure which allows me to set the font! Now, if I close this shell and start IDLE from the command line again, I still get a "shell" with an X11 parent, but, lo and behold, the font has changed to what I had previously set with the IDLE parent. Course, I can't import gmpy, cause the python.org version can't compile it, so I still have to use the macports install of 3.1, but that's ok, once I use the IDLE application to set the preferences, I can switch back to the X11 version and the preferences will follow. Wow! That was fun. Thanks for the help everyone. Don't ask me to explain what I just did, sometimes you just have to let art flow over you. > > -- > ?Ned Deily, > ?n... at acm.org From navjotmusic at gmail.com Tue Jan 5 00:27:12 2010 From: navjotmusic at gmail.com (Nav) Date: Mon, 4 Jan 2010 21:27:12 -0800 (PST) Subject: Newbie help- Can multiple instances with multiple names automatically created. References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> <14370388-6f6e-451c-b8a6-4ce2f6ec32fc@l30g2000yqb.googlegroups.com> <1df4f606-8e46-4ff1-9470-583e35612b25@r5g2000yqb.googlegroups.com> <09b79018-e226-4f7e-8bf6-78ada2ad3c77@22g2000yqr.googlegroups.com> Message-ID: <6f1cf0c7-eafc-472e-bfff-f86a8ae33658@r24g2000yqd.googlegroups.com> "> if yes then: > if using globalnamespace is bad then why does every book or tutorial > about python classes give the above style of assignment as an > example? That's a basic assignment example. It's not a direct manipulation of globals(), like the solution given by Jan, which you seem to feel is the answer. And unless it happens outside of a nested scope like a class or function definition, it's not 'global'. " Thanks for clearing that up Alex. The reason I liked Jan's solution at first glance was that it allowed me to manipulate the objects directly after I created the class. But I am using Shawn's advice now because it lets me abstract, but still allows me to directly manipulate the object if I need to. @ Steven.... "No, you're confused -- the problem isn't with using the global namespace. The problem is that you don't know what names you want to use ahead of time. " Actually I know what the names would be and how I want to use them. If you look at my example in the beginning you will notice that it creates unique objects with unique names, which anyone who looks at what is being mixed can easily figure out and of course the number and names of objects is directly dependent on the mixed elements. (although perhaps bottle is not the best analogy because the contents can change, which will not change once the object is created.) Having a stable and descriptive naming strategy was important, especially to use the unique object directly, and to not have to go searching where they might be located at any moment, e.g. if they are being moved about as part of other data structures. If anyone has any other thoughts regarding this, I would appreciate them. - Thanks everyone. From davea at ieee.org Tue Jan 5 00:28:58 2010 From: davea at ieee.org (Dave Angel) Date: Tue, 05 Jan 2010 00:28:58 -0500 Subject: Newbie help- Can multiple instances with multiple names automatically created. In-Reply-To: <1df4f606-8e46-4ff1-9470-583e35612b25@r5g2000yqb.googlegroups.com> References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> <14370388-6f6e-451c-b8a6-4ce2f6ec32fc@l30g2000yqb.googlegroups.com> <1df4f606-8e46-4ff1-9470-583e35612b25@r5g2000yqb.googlegroups.com> Message-ID: <4B42CE1A.2050801@ieee.org> (You top-posted. It's polite on most newsgroups, and specifically on this one to add your comments *following* the quoted earlier text) Nav wrote: > Okay, let me ask another question: > > When we create instances of objects by doing > x = className () > are we using globalnamespace? > > If that line appears at top-level, it uses the global namespace for the symbol x. If it's inside a class, or inside a function, then it uses some other namespace. The risk for globals is mainly that the namespace gets "polluted", or full of unrelated symbols, which sooner or later are going to collide. And this is made much worse if someone is creating such names indirectly, by monkeying with the non-portable constructs such as global(), or by doing from mymodule import * There are other risks, but this is a short message. As far as I know, there's very little performance cost for having lots of stuff in globals. It's just bad practice because it frequently leads to bugs. > if yes then: > if using globalnamespace is bad then why does every book or tutorial > about python classes give the above style of assignment as an > example? > > Beginner books keep things simple. Besides the same statement is great if it's in a function. And it's usually a small step for the beginner to start moving stuff from the top-level of the module into named functions. And eventually to functions that are small enough to be maintainable. > Second why do we use a dictionary to create something like this? I > know how it works, but what is wrong with simply creating instances > automatically? Once created the data for the instances is > automatically saved in their own space? Why have a round about way > using dictionaries? > Is there an advantage or does it conflict with something else? > > What? Automatically where? What's roundabout about dictionaries? > Why not have > for i in specialList: #I presume it would have to be special, because > it can't be a problematic type > i = className(whatever) > > > Once you bind a new value to i, it no longer has any relationship to the value which was in specialList. Perhaps you want something like: for i in xrange(len(specialList)): specialList[i] = className(...) which will replace the present values in specialList with instances of className. > > >>> what are the risks of globalnamespace use >>> >> You're unnecessarily tying your code to the implementation. >> >> >>> and what are the benefits? >>> >> Absolutely none that using a dictionary doesn't also give you. >> > > > From fpm at u.washington.edu Tue Jan 5 00:30:31 2010 From: fpm at u.washington.edu (cassiope) Date: Mon, 4 Jan 2010 21:30:31 -0800 (PST) Subject: IOError - cannot create file (linux daemon-invoked script) References: Message-ID: One more tidbit observed: my last note, that it works when using seteuid/setegid? Well - that only applies if the daemon is running under strace (!). It fails if started directly by root, or if the strace session has ended, leaving the main body of the daemon running in its normal headless manner. I wonder if running under "strace -f" - might setegid/seteuid be prevented from having their normal effect? Sigh. From mensanator at aol.com Tue Jan 5 00:32:13 2010 From: mensanator at aol.com (Mensanator) Date: Mon, 4 Jan 2010 21:32:13 -0800 (PST) Subject: How do you configure IDLE on a Mac, seeing as there's no Configure Option... References: <54d9a742-d7fb-44b0-9cf6-ffa49ca5f2f4@u41g2000yqe.googlegroups.com> Message-ID: <30ebdab2-4dc8-411e-a3e3-482f99cfd2ce@k17g2000yqh.googlegroups.com> On Jan 4, 10:44?pm, Mensanator wrote: > On Jan 4, 10:05?am, Benjamin Kaplan wrote: > > > On Sun, Jan 3, 2010 at 8:42 PM, Mensanator wrote: > > > ...because there's no [Options] menu on the shell window? > > > > Or at least give me a clue to how to use Courier New font? > > > > For some inscrutable reason, depite the plethora of formatting tools, > > > someone decided that proportional spaced fonts ought to be the > > > default for IDLE. > > > -- > > >http://mail.python.org/mailman/listinfo/python-list > > > Mac Applications don't have "options" menus. They have a "preferences" > > pane which is the second item in the application menu > > Sometimes it's the second item, sometimes it's not > > > (the one that > > takes the name of the active program) and has a keyboard shortcut of > > Cmd-, > > > Apple's user-interface guidelines are pretty specific, so you'll find > > that this works for all native OS X (i.e. not X11) applications. > > Right, it doesn't apply to IDLE which uses X11 Depending on how you launch it. If I type "idle" at a command prompt, I get a shell window whose parent is X11. Here you will get X11 preferences which can't be used to change fonts. If I click the IDLE icon, then I get a shell whose parent is IDLE, not X11 and here you'll find a preferences window similar to the OPTIONS/Configure in Windows. > (there is a preferences > window, > but it doesn't have any font control, unlike stuff like Safari). > > I assume there must be a configuration file. Any idea what that file > is called? I still don't know but apparently as long as I use the IDLE application instead of X11, I can set the font. Thanks for your time. From mensanator at aol.com Tue Jan 5 00:39:38 2010 From: mensanator at aol.com (Mensanator) Date: Mon, 4 Jan 2010 21:39:38 -0800 (PST) Subject: How do you configure IDLE on a Mac, seeing as there's no Configure Option... References: Message-ID: <12cf7e45-b0f7-4e61-88c0-6cf44ddf4e09@a15g2000yqm.googlegroups.com> On Jan 4, 9:17?am, "Alf P. Steinbach" wrote: > * Mensanator: > > > ...because there's no [Options] menu on the shell window? > > > Or at least give me a clue to how to use Courier New font? > > > For some inscrutable reason, depite the plethora of formatting tools, > > someone decided that proportional spaced fonts ought to be the > > default for IDLE. > > Why not just use the Python interpreter, a reasonable programmer's editor and, > if you need it, some debugger. I think I've solved my problem. I'll just say here that at least IDLE is cross-platform AND is part of the "batteries included". I'll be using it on both my new Mac and my desktop PCs running Windows. > > The source code level debugging in IDLE is nothing to write home about. I've got 20 years worth of programs and web pages written with Notepad, so don't let me hear any dissing of IDLE, ok? :-) > > In Windows XP it is just silly: it marks the current line by ordinary text > selection, which (depending on your config, but I think this is default) is > invisible when you're looking at the debugger window for single stepping. You > can see it by switching back to the source code window but that's annoying, not > very practical. Invisible highlighting of the current line, he he... > > Cheers & hth., > > - Alf From steven at REMOVE.THIS.cybersource.com.au Tue Jan 5 01:03:28 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 05 Jan 2010 06:03:28 GMT Subject: Newbie help- Can multiple instances with multiple names automatically created. References: <92742e15-17e9-458e-ad9c-e40a7f4f9d79@a21g2000yqc.googlegroups.com> <0D45750E-5C6F-4ABD-BEFE-2B2767D0E9AB@milochik.com> <3bb3b608-2644-448a-aef4-33ddac8d0eb0@35g2000yqa.googlegroups.com> <14370388-6f6e-451c-b8a6-4ce2f6ec32fc@l30g2000yqb.googlegroups.com> <1df4f606-8e46-4ff1-9470-583e35612b25@r5g2000yqb.googlegroups.com> <09b79018-e226-4f7e-8bf6-78ada2ad3c77@22g2000yqr.googlegroups.com> <6f1cf0c7-eafc-472e-bfff-f86a8ae33658@r24g2000yqd.googlegroups.com> Message-ID: On Mon, 04 Jan 2010 21:27:12 -0800, Nav wrote: > @ Steven.... > "No, you're confused -- the problem isn't with using the global > namespace. > The problem is that you don't know what names you want to use ahead of > time. " > > Actually I know what the names would be and how I want to use them. You said earlier: "I have a class of let's say empty bottle which can have a mix of two items. I want to create let's say 30 of these objects which will have names based on the 2 attributes (apple juice, beer, grape juice, beer, etc) that I provide from a list." Your description is confusing to me. What on earth is an empty bottle which has a mix of two items in it? Surely that means it's not empty any more? But putting that aside: "All the objects are a mix of (1 of three alcohols) and (1 of 10 juices), so I don't want to go through typing in the names of all the objects (which would be totally stupid)." Right... so your problem isn't that you don't know what the variable names is, but there are too many to comfortably enumerate in the source code. The answer is, again, avoid named variables. Instead of (say): gin_apple_strawberry = Bottle('gin', 'apple') gin_apple_orange = Bottle('gin', 'orange') # etc. again you should use a list or dict: bottles = [] for alcohol in ('gin', 'beer', 'wine'): for fruit in ('apple', 'banana', 'blueberry', 'strawberry', 'orange', 'peach'): bottles.append(Bottle(alcohol, fruit)) for bottle in bottles: process(bottle) -- Steven From nad at acm.org Tue Jan 5 01:13:10 2010 From: nad at acm.org (Ned Deily) Date: Mon, 04 Jan 2010 22:13:10 -0800 Subject: Best practices for simultaneously installed versioned packages? References: <4B41CC6C.3020808@hastings.org> <4B42B45E.9070900@hastings.org> Message-ID: In article <4B42B45E.9070900 at hastings.org>, Larry Hastings wrote: > Ned Deily wrote: > > Also, setuptools (and, its successor, distribute, which supports Python > > 3) allow the installation and management of multiple versions of a > > package within one python site-library instance. > That sounds lovely too, but I can't figure out what facility setuptools > / distribute permits that. Could you be more specific? A link to the > documentation for this feature would be much appreciated. By default, easy_install from setuptools/distribute uses version information from a package's setup.py to qualify the package name and manipulate python's sys.path to ensure the most recent version is the one found by default. There are options to easy_install which control this. See in particular the easy_install "--multi-version" command-line option (the links here are to those for the Distribute fork of setuptools): http://packages.python.org/distribute/easy_install.html If you need to declare specific version dependencies for other packages, you can do so in setup.py. See the Declaring Dependencies section of the setuptools API documentation: http://packages.python.org/distribute/setuptools.html Much of the magic is handled by setuptools version of pkg_resources: http://packages.python.org/distribute/pkg_resources.html -- Ned Deily, nad at acm.org From nad at acm.org Tue Jan 5 01:32:47 2010 From: nad at acm.org (Ned Deily) Date: Mon, 04 Jan 2010 22:32:47 -0800 Subject: How do you configure IDLE on a Mac, seeing as there's no Configure Option... References: <0d70cb54-3d77-4176-b621-e764ecf613b7@26g2000yqo.googlegroups.com> Message-ID: In article <0d70cb54-3d77-4176-b621-e764ecf613b7 at 26g2000yqo.googlegroups.com>, Mensanator wrote: > I assume I've been using the IDLE from macports. From the command > prompt I've > been typing "idle". This launches a "shell" window which appears to > have an X11 > parent application for which there are no "preferences" applicable to > fonts. > > However, if I use the quick launcher from the python.org, I get a > "shell" whose > parent is named "IDLE"! And that one has a completely different > preferences, > one similar the the Windows Configure which allows me to set the font! > > Now, if I close this shell and start IDLE from the command line again, > I still > get a "shell" with an X11 parent, but, lo and behold, the font has > changed to > what I had previously set with the IDLE parent. > > Course, I can't import gmpy, cause the python.org version can't > compile it, so I > still have to use the macports install of 3.1, but that's ok, once I > use > the IDLE application to set the preferences, I can switch back to the > X11 version and the preferences will follow. The prefs follow because all versions of IDLE use the same (unversioned) directory for configuration files, ~/.idlerc/. In particular, the configuration file ~/.idlerc/config-main.cfg contains, among other things, any changes to the default font. So, if you're successful at changing it in one version of IDLE, it will likely affect all versions you have. Note the file is a simple ini format: [EditorWindow] font = monaco so you can edit it by hand. BTW, the python.org IDLEs and the Apple-supplied IDLEs use the system-supplied Aqua (aka Quartz) Tk not the X11 one that MacPorts builds by default. The MacPorts Tk port does have a "quartz" variant but that doesn't yet work in 64-bit mode. -- Ned Deily, nad at acm.org From lie.1296 at gmail.com Tue Jan 5 02:02:37 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Tue, 05 Jan 2010 18:02:37 +1100 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> Message-ID: <4b42e40c$1@dnews.tpgi.com.au> On 1/5/2010 1:31 PM, r0g wrote: > Michi wrote: >> On Jan 4, 1:30 pm, Steven D'Aprano >> wrote: > >>> In some, limited, cases you might be able to use the magic return value >>> strategy, but this invariably leads to lost programmer productivity, more >>> complex code, lowered readability and usability, and more defects, >>> because programmers will invariably neglect to test for the special value: >> >> I disagree here, to the extent that, whether something is an error or >> not can very much depend on the circumstances in which the API is >> used. The collection case is a very typical example. Whether failing >> to locate a value in a collection is an error very much depends on >> what the collection is used for. In some cases, it's a hard error >> (because it might, for example, imply that internal program state has >> been corrupted); in other cases, not finding a value is perfectly >> normal. > > > > A pattern I have used a few times is that of returning an explicit > success/failure code alongside whatever the function normally returns. > While subsequent programmers might not intuit the need to test for > (implicit) "magic" return values they ought to notice if they start > getting tuples back where they expected scalars... > > def foo(x) > if x>0: > return True, x*x > else: > return False, "Bad value of x in foo:",str(x) > > ok, value = foo(-1) > if ok: > print "foo of x is", value > else: > print "ERROR:", value Except that that is a reinvention of try-wheel: def foo(x): if x > 0: return x*x else: raise MathError("Bad value of x in foo: %s" % x) try: print foo(-1) except MathError, e: print "ERROR: System integrity is doubted" or rather; that is perhaps a good example of when to use 'assert'. If the domain of foo() is positive integers, calling -1 on foo is a bug in the caller, not foo(). I have been looking at Haskell recently and the way the pure functional language handled exceptions and I/O gives me a new distinct "insight" that exceptions can be thought of as a special return value that is implicitly wrapped and unwrapped up the call stack until it is explicitly handled. From stefan_ml at behnel.de Tue Jan 5 03:16:17 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 05 Jan 2010 09:16:17 +0100 Subject: Where's a DOM builder that uses the Builder Pattern to ... build DOMs? In-Reply-To: <04523152-a0ff-432c-a48f-2c653577e7b7@m25g2000yqc.googlegroups.com> References: <04523152-a0ff-432c-a48f-2c653577e7b7@m25g2000yqc.googlegroups.com> Message-ID: <4b42f551$0$7625$9b4e6d93@newsspool1.arcor-online.net> Phlip, 04.01.2010 21:01: > Not Hyp: > > I hope I'm wrong, but seems that DOMBuilder, found among the various > xml.dom packages, cannot build DOM like this: > > var html = DomBuilder.apply(); > > var form = html.FORM( > html.DIV( > html.INPUT({type : 'text', name : 'email'}), > html.INPUT({type : 'text', name : 'password'}), > html.INPUT({type : 'submit'}), > ) > ); > > Do anyone know any good DOM builder packages that do build DOM good > like a DOM builder should? You might be looking for something like this: http://codespeak.net/lxml/lxmlhtml.html#creating-html-with-the-e-factory Note that there are tons of ways to generate HTML with Python. A quick web search (or a quick read on PyPI or the Python Wiki) should get you started. Stefan From steven at REMOVE.THIS.cybersource.com.au Tue Jan 5 03:26:10 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 05 Jan 2010 08:26:10 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> Message-ID: On Tue, 05 Jan 2010 02:31:34 +0000, r0g wrote: > A pattern I have used a few times is that of returning an explicit > success/failure code alongside whatever the function normally returns. That doesn't work for languages that can only return a single result, e.g. C or Pascal. You can fake it by creating a struct that contains a flag and the result you want, but that means doubling the number of data types you deal with. > While subsequent programmers might not intuit the need to test for > (implicit) "magic" return values they ought to notice if they start > getting tuples back where they expected scalars... What if they're expecting tuples as the result? > def foo(x) > if x>0: > return True, x*x > else: > return False, "Bad value of x in foo:",str(x) > > ok, value = foo(-1) Oops, that gives: ValueError: too many values to unpack because you've returned three items instead of two. When an idiom is easy to get wrong, it's time to think hard about it. > if ok: > print "foo of x is", value > else: > print "ERROR:", value Whenever I come across a function that returns a flag and a result, I never know whether the flag comes first or second. Should I write: flag, result = foo(x) or result, flag = foo(x) I've seen APIs that do both. And I never know if the flag should be interpreted as a success or a failure. Should I write: ok, result = foo(x) if ok: process(result) else: fail() or err, result = foo(x) if err: fail() else: process(result) Again, I've seen APIs that do both. And if the flag indicates failure, what should go into result? An error code? An error message? That's impossible for statically-typed languages, unless they have variant records or the function normally returns a string. And even if you dismiss all those concerns, it still hurts readability by obfuscating the code. Consider somebody who wants to do this: result = foo(bar(x)) but instead has to do this: flag, result = bar(x) if flag: # I think this means success flag, result = foo(x) # oops, I meant result Again, it's error-prone and messy. Imagine writing: flag, a = sin(x) if flag: flag, b = sqrt(x) if flag: flag, c = cos(b) if flag: flag, d = exp(a + c) if flag: flag, e = log(x) if flag: # Finally, the result we want!!! flag, y = d/e if not flag: fail(y) else: fail(e) else: fail(d) else: fail(c) else: fail(b) else: fail(a) Compare that to the way with exceptions: y = exp(sin(x) + cos(sqrt(x)))/log(x) Which would you prefer? -- Steven From g.bogle at auckland.no.spam.ac.nz Tue Jan 5 03:36:59 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Tue, 05 Jan 2010 21:36:59 +1300 Subject: A null program - what is it doing? Message-ID: No doubt a dumb question from a noob: The following program (a cut down version of some test code) uses no CPU, and does not terminate: import sys from PyQt4.QtCore import * if __name__=="__main__": app = QCoreApplication(sys.argv) sys.exit(app.exec_()) What is the program doing? What is the correct way to terminate the execution? Thanks in advance for educating me. From news123 at free.fr Tue Jan 5 04:03:56 2010 From: news123 at free.fr (News123) Date: Tue, 05 Jan 2010 10:03:56 +0100 Subject: python xmlrpc client with ssl client certificates and standard modules In-Reply-To: <4B4267DB.3080409@v.loewis.de> References: <4b414e79$0$23321$426a34cc@news.free.fr> <4B4267DB.3080409@v.loewis.de> Message-ID: <4b43007c$0$11218$426a74cc@news.free.fr> Hi Martin, Thanks a lot for your reply. It helped me to find the correct solution. Unfortunaltely xmlrpclib.ServerProxy does not allow a host tuple, but just a uri. So the simplest solution, that I found is to create a custom transport import xmlrpclib class SafeTransportWithCert(xmlrpclib.SafeTransport): __cert_file = DFLT_CERTFILE __key_file = DFLT_KEYFILE def make_connection(self,host): host_with_cert = (host, { 'key_file' : self.__key_file, 'cert_file' : self.__cert_file } ) return \ xmlrpclib.SafeTransport.make_connection( self,host_with_cert) transport = SafeTransportWithCert() server = xmlrpclib.ServerProxy(server_url, transport = transport) rslt = server.mymethod(args) Perfect. Now the server can ensure, that only certified clients connect. My next task is how to find out at the client side, that the server certificate is a properly signed one. bye N Martin v. Loewis wrote: >> I can do xmlrpc over ssl WITHOUT certificates with following code: >> >> import xmlrpclib >> server_url = 'https://myserver' >> server = xmlrpclib.Server(server_url); >> >> >> and I can perform a https get request WITH certificates with below snippet: >> >> import httplib >> conn = httplib.HTTPSConnection( >> HOSTNAME, >> key_file = KEYFILE, >> cert_file = CERTFILE >> ) >> conn.putrequest('GET', '/') >> conn.endheaders() >> response = conn.getresponse() >> print response.read() >> >> >> I'm just lost of how to 'combine' both. > > In this case, read through the source of xmlrpclib: > > a) SafeTransport receives x509 parameters from get_host_info > b) get_host_info supports a case where host is a tuple host, x509 > > So, without testing: > > server = xmlrpclib.Server((server_url, {'key_file': KEYFILE, > 'cert_file': CERTFILE})) > > Please do read the code before trying this out. > > HTH, > Martin From aioe.org at technicalbloke.com Tue Jan 5 04:07:20 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 09:07:20 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> Message-ID: Lie Ryan wrote: > On 1/5/2010 1:31 PM, r0g wrote: >> Michi wrote: >>> On Jan 4, 1:30 pm, Steven D'Aprano >>> wrote: >> A pattern I have used a few times is that of returning an explicit >> success/failure code alongside whatever the function normally returns. >> While subsequent programmers might not intuit the need to test for >> (implicit) "magic" return values they ought to notice if they start >> getting tuples back where they expected scalars... >> >> def foo(x) >> if x>0: >> return True, x*x >> else: >> return False, "Bad value of x in foo:",str(x) >> >> ok, value = foo(-1) >> if ok: >> print "foo of x is", value >> else: >> print "ERROR:", value > > Except that that is a reinvention of try-wheel: > True, but there's more than one way to skin a cat! Mine's faster if you expect a high rate of failures (over 15%). > def foo(x): > if x > 0: > return x*x > else: > raise MathError("Bad value of x in foo: %s" % x) > > try: > print foo(-1) > except MathError, e: > print "ERROR: System integrity is doubted" > > or rather; that is perhaps a good example of when to use 'assert'. If > the domain of foo() is positive integers, calling -1 on foo is a bug in > the caller, not foo(). Maybe, although I recently learned on here that one can't rely on assert statements in production code, their intended use is to aid debugging and testing really. Besides, that was just a toy example. > > I have been looking at Haskell recently and the way the pure functional > language handled exceptions and I/O gives me a new distinct "insight" > that exceptions can be thought of as a special return value that is > implicitly wrapped and unwrapped up the call stack until it is > explicitly handled. Yes there's some very interesting paradigms coming out of functional programming but, unless you're a maths major, functional languages are a long way off being productivity tools! Elegant: yes, provable: maybe, practical for everyday coding: not by a long shot! Roger. From clp2 at rebertia.com Tue Jan 5 04:19:00 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Tue, 5 Jan 2010 01:19:00 -0800 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> Message-ID: <50697b2c1001050119g7ae3b41fme7b5f8824396a99d@mail.gmail.com> On Tue, Jan 5, 2010 at 1:07 AM, r0g wrote: > Lie Ryan wrote: >> I have been looking at Haskell recently and the way the pure functional >> language handled exceptions and I/O gives me a new distinct "insight" >> that exceptions can be thought of as a special return value that is >> implicitly wrapped and unwrapped up the call stack until it is >> explicitly handled. > > Yes there's some very interesting paradigms coming out of functional > programming but, unless you're a maths major, functional languages are a > long way off being productivity tools! Elegant: yes, provable: maybe, > practical for everyday coding: not by a long shot! Methinks the authors of Real World Haskell (excellent read btw) have a bone to pick with you. Cheers, Chris -- http://blog.rebertia.com From aleksandar27 at BRISIOVOnet.hr Tue Jan 5 04:21:29 2010 From: aleksandar27 at BRISIOVOnet.hr (alejandro) Date: Tue, 5 Jan 2010 10:21:29 +0100 Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> Message-ID: Did you mean borderless printing? Every printer needs his margins, some more some less. Some printers have the ability to do borderless printing but usualy they can do it only on special or photo paper. So you can adjust the pdf as you wish, even with no margins, and then try to find under printer options "borderless printing". That is why I didn't understand :-)) it is a printer thing not pdf! From ashish.vyas at motorola.com Tue Jan 5 04:38:50 2010 From: ashish.vyas at motorola.com (VYAS ASHISH M-NTB837) Date: Tue, 5 Jan 2010 17:38:50 +0800 Subject: lxml 2.2.4 on python3.1, Windows XP gives importerror Message-ID: <7C57DB58C81FB64CA979EC6C4DB73E7A03E7039F@ZMY16EXM67.ds.mot.com> Dear All I have Python 3.1 installed on Windows XP and Works nice. I downloaded lxml 2.2.4 (lxml-2.2.4.win32-py3.1.exe) from pypi. When I try: from lxml import etree I get: ImportError: DLL load failed: This application has failed to start because the application configuration is incorrect. Reinstalling the application may fix this problem. For information: 'import lxml' works fine. After reinstalling python3.1 also the error message is the same. Any help is appreciated! Regards, Ashish Vyas -------------- next part -------------- An HTML attachment was scrubbed... URL: From richardbp at gmail.com Tue Jan 5 04:43:12 2010 From: richardbp at gmail.com (Richard) Date: Tue, 5 Jan 2010 01:43:12 -0800 (PST) Subject: Efficiently determine where documents differ References: Message-ID: On Jan 5, 9:46?am, "Gabriel Genellina" wrote: > En Mon, 04 Jan 2010 19:04:12 -0300, Richard escribi?: > > > I have been using the difflib library to find where 2 large HTML > > documents differ. The Differ().compare() method does this, but it is > > very slow - atleast 100x slower than the unix diff command. > > Differ compares sequences of lines *and* lines as sequences of characters ? > to provide intra-line differences. The diff command only processes lines. > If you aren't interested in intra-line differences, use a SequenceMatcher ? > instead. Or, invoke the diff command using ? subprocess.Popen + ? > communicate. > > -- > Gabriel Genellina thank you very much Gabriel! Passing a list of the document lines makes the efficiency comparable to the diff command. Richard From aioe.org at technicalbloke.com Tue Jan 5 04:52:32 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 09:52:32 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> Message-ID: Steven D'Aprano wrote: > On Tue, 05 Jan 2010 02:31:34 +0000, r0g wrote: > >> A pattern I have used a few times is that of returning an explicit >> success/failure code alongside whatever the function normally returns. > > That doesn't work for languages that can only return a single result, > e.g. C or Pascal. You can fake it by creating a struct that contains a > flag and the result you want, but that means doubling the number of data > types you deal with. No, but that's why I try not to use languages where you can only return a single result, I always found that an arbitrary and annoying constraint to have. I leads to ugly practices like "magic" return values in C or explicitly packing things into hashtables like PHP, yuk! > > >> While subsequent programmers might not intuit the need to test for >> (implicit) "magic" return values they ought to notice if they start >> getting tuples back where they expected scalars... > > What if they're expecting tuples as the result? > > > >> def foo(x) >> if x>0: >> return True, x*x >> else: >> return False, "Bad value of x in foo:",str(x) >> >> ok, value = foo(-1) > > Oops, that gives: > > ValueError: too many values to unpack > > > because you've returned three items instead of two. When an idiom is easy > to get wrong, it's time to think hard about it. > That seems pretty clear to me, "too many values to unpack", either I've not given it enough variables to unpack the result into or I've returned too many things. That would take a couple of seconds to notice and fix. In fact I was trying to make the point that it would be quite noticable if a function returned more things than the programmer was expecting, this illustrates that quite well :) > > >> if ok: >> print "foo of x is", value >> else: >> print "ERROR:", value > > > Whenever I come across a function that returns a flag and a result, I > never know whether the flag comes first or second. Should I write: > Flag then result, isn't it obvious? The whole point of returning a flag AND a result is so you can test the flag so you know what to do with the result so that implies a natural order. Of course it doesn't matter technically which way you do it, make a convention and stick to it. If you get perpetually confused as to the order of parameters then you'd better avoid this kind of thing, can't say as I've ever had a problem with it though. > And I never know if the flag should be interpreted as a success or a > failure. Should I write: > > ok, result = foo(x) > if ok: process(result) > else: fail() Yes. That would be my strong preference anyway. Naturally you can do it the other way round if you like, as long as you document it properly in your API. As you say different APIs do it differently... Unix has a convention of returning 0 on no-error but unix has to encapsulate a lot in that "error code" which is a bit of an anachronism these days. I'd argue in favour of remaining positive and using names like ok or success, this is closer to the familiar paradigm of checking a result does not evaluate to false before using it... name = "" if name: print name > And if the flag indicates failure, what should go into result? An error > code? An error message? That's impossible for statically-typed languages, > unless they have variant records or the function normally returns a > string. Yeah, in my example it's an error message. Maybe I shouldn't have used the word "pattern" above though as it has overtones of "universally applicable" which it clearly isn't. > Again, it's error-prone and messy. Imagine writing: > > > flag, a = sin(x) > if flag: > flag, b = sqrt(x) > if flag: > Compare that to the way with exceptions: > > y = exp(sin(x) + cos(sqrt(x)))/log(x) > > > Which would you prefer? > LOL, straw man is straw! You know full well I'm not suggesting every function return a flag, that would be silly. There's no reason returning flag and a value shouldn't be quite readable and there may be times when it's preferable to raising an exception. I use exceptions a lot as they're often the right tool for the job and they seem pleasingly pythonic but from time to time they can be too slow or verbose, where's the sense in forcing yourself to use them then? Roger. From paul.nospam at rudin.co.uk Tue Jan 5 04:54:42 2010 From: paul.nospam at rudin.co.uk (Paul Rudin) Date: Tue, 05 Jan 2010 09:54:42 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> Message-ID: <87eim4j2ql.fsf@rudin.co.uk> r0g writes: > Steven D'Aprano wrote: >> On Tue, 05 Jan 2010 02:31:34 +0000, r0g wrote: >> >>> A pattern I have used a few times is that of returning an explicit >>> success/failure code alongside whatever the function normally returns. >> >> That doesn't work for languages that can only return a single result, >> e.g. C or Pascal. You can fake it by creating a struct that contains a >> flag and the result you want, but that means doubling the number of data >> types you deal with. > > > No, but that's why I try not to use languages where you can only return > a single result, I always found that an arbitrary and annoying > constraint to have. I leads to ugly practices like "magic" return values > in C or explicitly packing things into hashtables like PHP, yuk! Doesn't python just return a single result? (I know it can be a tuple and assignment statements will unpack a tuple for you.) From aioe.org at technicalbloke.com Tue Jan 5 04:58:56 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 09:58:56 +0000 Subject: A null program - what is it doing? References: Message-ID: Gib Bogle wrote: > No doubt a dumb question from a noob: > > The following program (a cut down version of some test code) uses no > CPU, and does not terminate: > > import sys > from PyQt4.QtCore import * > > if __name__=="__main__": > app = QCoreApplication(sys.argv) > sys.exit(app.exec_()) > > What is the program doing? What is the correct way to terminate the > execution? > > Thanks in advance for educating me. I've never used QT but other graphical toolkits I have used all start their own "main loop" which is a loop that cycles round receiving, queuing and dispatching "events". You probably need to call the QCoreApplication's quit method to break out of this e.g. app.exit() or something similar, have a look at some complete PyQt4 examples or google for PyQt4 mainloop. Roger. From almar.klein at gmail.com Tue Jan 5 05:18:33 2010 From: almar.klein at gmail.com (Almar Klein) Date: Tue, 5 Jan 2010 11:18:33 +0100 Subject: A null program - what is it doing? In-Reply-To: References: Message-ID: 2010/1/5 r0g > Gib Bogle wrote: > > No doubt a dumb question from a noob: > > > > The following program (a cut down version of some test code) uses no > > CPU, and does not terminate: > > > > import sys > > from PyQt4.QtCore import * > > > > if __name__=="__main__": > > app = QCoreApplication(sys.argv) > > sys.exit(app.exec_()) > > > > What is the program doing? What is the correct way to terminate the > > execution? > > > > Thanks in advance for educating me. > > > > I've never used QT but other graphical toolkits I have used all start > their own "main loop" which is a loop that cycles round receiving, > queuing and dispatching "events". You probably need to call the > QCoreApplication's quit method to break out of this e.g. > > app.exit() or something similar, have a look at some complete PyQt4 > examples or google for PyQt4 mainloop. > app.exec_() starts the QT mainloop. If you would have created a window or app before starting the loop, you app would now be responsive and wait for you to do things with it. When you would then close the app, the function returns the error code with which the app closed, and then subsequently the Python process exits with that same error code. But since no widgets were created before starting the mainloop, I don't think you can stop the process in any way other than killing it. By the way, QT has excellent documentation: http://doc.trolltech.com/4.4/qapplication.html#exec Cheers, Almar -------------- next part -------------- An HTML attachment was scrubbed... URL: From aioe.org at technicalbloke.com Tue Jan 5 05:22:23 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 10:22:23 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> Message-ID: Chris Rebert wrote: > > On Tue, Jan 5, 2010 at 1:07 AM, r0g wrote: >> Lie Ryan wrote: >>> I have been looking at Haskell recently and the way the pure functional >>> language handled exceptions and I/O gives me a new distinct "insight" >>> that exceptions can be thought of as a special return value that is >>> implicitly wrapped and unwrapped up the call stack until it is >>> explicitly handled. >> Yes there's some very interesting paradigms coming out of functional >> programming but, unless you're a maths major, functional languages are a >> long way off being productivity tools! Elegant: yes, provable: maybe, >> practical for everyday coding: not by a long shot! > > Methinks the authors of Real World Haskell (excellent read btw) have a > bone to pick with you. > > Cheers, > Chris > -- > http://blog.rebertia.com LOL, it seems things have come a long way since ML! I'm impressed how many useful libraries Haskell has, and that they've included IF-THEN-ELSE in the syntax! :) For all its advantages I still think you need to be fundamentally cleverer to write the same programs in a functional language than an old fashioned "English like" language. Maybe I'm just mistrusting of the new school though and you'll see me on comp.lang.haskell in a few years having to eat my own monads! Roger. From aioe.org at technicalbloke.com Tue Jan 5 05:31:03 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 10:31:03 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> Message-ID: Paul Rudin wrote: > r0g writes: > >> Steven D'Aprano wrote: >>> On Tue, 05 Jan 2010 02:31:34 +0000, r0g wrote: >>> >>>> A pattern I have used a few times is that of returning an explicit >>>> success/failure code alongside whatever the function normally returns. >>> That doesn't work for languages that can only return a single result, >>> e.g. C or Pascal. You can fake it by creating a struct that contains a >>> flag and the result you want, but that means doubling the number of data >>> types you deal with. >> >> No, but that's why I try not to use languages where you can only return >> a single result, I always found that an arbitrary and annoying >> constraint to have. I leads to ugly practices like "magic" return values >> in C or explicitly packing things into hashtables like PHP, yuk! > > Doesn't python just return a single result? (I know it can be a tuple and > assignment statements will unpack a tuple for you.) Yes, it returns a tuple if you return more than one value, it just has a lovely syntax for it. In static languages you'd need to manually create an new array or struct, pack your return vars into it and unpack them on the other side. That's something I'd be happy never to see again, sadly I have to write in PHP sometimes :( Roger. From coert at holmes.nl Tue Jan 5 06:19:00 2010 From: coert at holmes.nl (Coert Klaver (DT)) Date: Tue, 5 Jan 2010 12:19:00 +0100 Subject: ctypes: How to call unexported functions in a dll Message-ID: Hi, I am using ctypes in python 3 on a WXP machine Loading a dll and using its exported functions works fine. Now I want to use a function in the dll that is not exported. In C this can be done by just casting the address in the dll of that function to an apropriate function pointer and call it (you need to be sure about the address). Can a similar thing be done directly with ctypes? A work around I see is writing in wrapper dll in c that loads the main dll, exports a function that calls the unexported function in the main dll, but I don't find that an elegant solution. Thanks for any hint in the right direction. BR Coert From davea at ieee.org Tue Jan 5 06:59:59 2010 From: davea at ieee.org (Dave Angel) Date: Tue, 05 Jan 2010 06:59:59 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> Message-ID: <4B4329BF.6080704@ieee.org> r0g wrote: > > > Maybe, although I recently learned on here that one can't rely on assert > statements in production code, their intended use is to aid debugging > and testing really. > > Hopefully, what you learned is that you can't use assert() in production code to validate user data. It's fine to use it to validate program logic, because that shouldn't still need testing in production. DaveA From davea at ieee.org Tue Jan 5 07:12:40 2010 From: davea at ieee.org (Dave Angel) Date: Tue, 05 Jan 2010 07:12:40 -0500 Subject: A null program - what is it doing? In-Reply-To: References: Message-ID: <4B432CB8.8040500@ieee.org> r0g wrote: > Gib Bogle wrote: > >> No doubt a dumb question from a noob: >> >> The following program (a cut down version of some test code) uses no >> CPU, and does not terminate: >> >> import sys >> from PyQt4.QtCore import * >> >> if __name__=="__main__": >> app = QCoreApplication(sys.argv) >> sys.exit(app.exec_()) >> >> What is the program doing? What is the correct way to terminate the >> execution? >> >> Thanks in advance for educating me. >> > > > > I've never used QT but other graphical toolkits I have used all start > their own "main loop" which is a loop that cycles round receiving, > queuing and dispatching "events". You probably need to call the > QCoreApplication's quit method to break out of this e.g. > > app.exit() or something similar, have a look at some complete PyQt4 > examples or google for PyQt4 mainloop. > > > Roger. > > Likewise, I haven't used QT, but have used others. Gib: It looks like app.exec() is the mainloop for QT, and it won't terminate till it gets a terminate event. That might be the user clicking the X in the corner, or it might be a special keystroke like Alt-F4. Or it might be some other event for which your event handler makes an explicit call to terminate. Once one of these things happens, the app.exec() will return, and the sys.exit() will execute. DaveA From kk720801 at yahoo.com Tue Jan 5 07:33:52 2010 From: kk720801 at yahoo.com (Krzysztof Kobus) Date: Tue, 5 Jan 2010 04:33:52 -0800 (PST) Subject: embedded python on mac - linking problem Message-ID: <477262.24163.qm@web53508.mail.re2.yahoo.com> Hi, I have a problem with linking python module with my application on mac in order to make the module available in "embedded python". My python module is contained in j3kmodule.cxx file and module initialization function is exported in j3kmodule.h j3kmodule.h: ------------ PyMODINIT_FUNC PyInit_j3k(void); j3kmodule.cxx: -------------- PyMODINIT_FUNC PyInit_j3k(void) { PyObject *m = NULL; if ((m = PyModule_Create(&j3k_module)) == NULL) return NULL; return m; } Then in my application in KkPython.cxx file I have: KkPython.cxx: ------------- #include "j3kmodule.h" /* Add a builtin module, before Py_Initialize */ PyImport_AppendInittab("j3k", PyInit_j3k); I link my application with the module and get following linking error on mac although on open suse linux exactly the same procedure works fine: g++ -headerpad_max_install_names -o ../../bin/render.app/Contents/MacOS/render main.o KkPython.o -lkkbase -L/Users/kk/dev/J3K/trunk/j3ksrc/examples/render/../../lib/ -lQtSolutions_PropertyBrowser-2.5 -lpython3.1 /Users/kk/dev/J3K/trunk/j3ksrc/python/modules/build/lib.macosx-10.3-i386-3.1/j3k.so -framework OpenGL -framework AGL ld: warning in /Users/kk/dev/J3K/trunk/j3ksrc/python/modules/build/lib.macosx-10.3-i386-3.1/j3k.so, file is not of required architecture Undefined symbols: "_PyInit_j3k", referenced from: _PyInit_j3k$non_lazy_ptr in KkPython.o ld: symbol(s) not found collect2: ld returned 1 exit status make: *** [../../bin/render.app/Contents/MacOS/render] Error 1 I appreciate any hints, best greetings, Krzysztof Kobus From keesvanschaik at gmail.com Tue Jan 5 07:40:14 2010 From: keesvanschaik at gmail.com (KvS) Date: Tue, 5 Jan 2010 04:40:14 -0800 (PST) Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> Message-ID: <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> On Jan 5, 9:21?am, "alejandro" wrote: > Did you mean borderless printing? > Every printer needs his margins, some more some less. Some printers have the > ability to do borderless printing but usualy they can do it only on special > or photo paper. So you can adjust the pdf as you wish, even with no margins, > and then try to find under printer options "borderless printing". That is > why I didn't understand :-)) it is a printer thing not pdf! As much as possible "borderless", yes. Of course the printer will still apply some small margin, but that's ok. A margin of say <0.5 cm. is fine. So it's not a printer thing, I accept the (physical) limitations of the printer, but I want to avoid any extra margins due to software settings. So I need the create a pdf without/less as possible margins, and then I need Acrobat Reader to (silently) print it without any/less as possible extra margins added. I believe the first step is possible, but I am wondering if the second step is possible as well, i.e. can I adjust the options normally appearing in the Printing Dialog through Python? From ben+python at benfinney.id.au Tue Jan 5 07:52:46 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 05 Jan 2010 23:52:46 +1100 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> Message-ID: <87tyv0ww69.fsf@benfinney.id.au> r0g writes: > Paul Rudin wrote: > > Doesn't python just return a single result? (I know it can be a > > tuple and assignment statements will unpack a tuple for you.) > > Yes, it returns a tuple if you return more than one value, it just has > a lovely syntax for it. No, there is nothing inherent to the ?return? statement for dealing with multiple values. The ?return? statement *always* returns a single value: whatever value you specify as the argument to the statement (or the value ?None? if no argument is specified. If you specify a tuple ? and Python has a nice syntax for creating a literal tuple ? that's the single value the statement will use. -- \ ?There's no excuse to be bored. Sad, yes. Angry, yes. | `\ Depressed, yes. Crazy, yes. But there's no excuse for boredom, | _o__) ever.? ?Viggo Mortensen | Ben Finney From chris.gonnerman at newcenturycomputers.net Tue Jan 5 07:56:02 2010 From: chris.gonnerman at newcenturycomputers.net (Chris Gonnerman) Date: Tue, 05 Jan 2010 06:56:02 -0600 Subject: [Python] Re: Printing plain text with exact positioning on Windows In-Reply-To: <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> Message-ID: <4B4336E2.9090908@newcenturycomputers.net> KvS wrote: > ... can I adjust the options normally appearing in > the Printing Dialog through Python? > Yes, if you use my method or my module, as I gave in my previous post. If you use Adobe Reader to print, I'm not sure how to automate the print settings. From aioe.org at technicalbloke.com Tue Jan 5 08:06:20 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 13:06:20 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> Message-ID: Dave Angel wrote: > > > r0g wrote: >> >> >> Maybe, although I recently learned on here that one can't rely on assert >> statements in production code, their intended use is to aid debugging >> and testing really. >> >> > Hopefully, what you learned is that you can't use assert() in production > code to validate user data. It's fine to use it to validate program > logic, because that shouldn't still need testing in production. > > > > DaveA Well maybe I didn't quite get it then, could you explain a bit further? My understanding was that asserts aren't executed at all if python is started with the -O or -OO option, or run through an optimizer. If that's the case how can you expect it to validate anything at all in production? Do you mean for debugging in situ or something? Could you maybe give me an example scenario to illustrate your point? Cheers, Roger. From roy at panix.com Tue Jan 5 08:15:55 2010 From: roy at panix.com (Roy Smith) Date: Tue, 05 Jan 2010 08:15:55 -0500 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> Message-ID: In article , r0g wrote: > No, but that's why I try not to use languages where you can only return > a single result, I always found that an arbitrary and annoying > constraint to have. I leads to ugly practices like "magic" return values > in C or explicitly packing things into hashtables like PHP, yuk! Python only lets you return a single result, just like C or C++. The difference is that in Python it's trivial to build tuples on the fly and return them. About the closest you get to that in C++ is std::pair, and that's about 5 bucks short and a week late. From steve at REMOVE-THIS-cybersource.com.au Tue Jan 5 09:02:50 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 05 Jan 2010 14:02:50 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> Message-ID: <03533377$0$1336$c3e8da3@news.astraweb.com> On Tue, 05 Jan 2010 13:06:20 +0000, r0g wrote: > Dave Angel wrote: >> >> >> r0g wrote: >>> >>> >>> Maybe, although I recently learned on here that one can't rely on >>> assert >>> statements in production code, their intended use is to aid debugging >>> and testing really. >>> >>> >> Hopefully, what you learned is that you can't use assert() in >> production code to validate user data. It's fine to use it to validate >> program logic, because that shouldn't still need testing in production. >> >> >> >> DaveA > > > > Well maybe I didn't quite get it then, could you explain a bit further? > > My understanding was that asserts aren't executed at all if python is > started with the -O or -OO option, Correct. > or run through an optimizer. I don't know what you mean by that. > If > that's the case how can you expect it to validate anything at all in > production? The asserts still operate so long as you don't use the -O switch. > Do you mean for debugging in situ or something? Could you > maybe give me an example scenario to illustrate your point? There are at least two sorts of validation that you will generally need to perform: validating user data, and validating your program logic. You *always* need to validate user data (contents of user-editable config files, command line arguments, data files, text they type into fields, etc.) because you have no control over what they put into that. So you shouldn't use assert for validating user data except for quick-and-dirty scripts you intend to use once and throw away. Program logic, on the other hand, theoretically shouldn't need to be validated at all, because we, the programmers, are very clever and naturally never make mistakes. Since we never make mistakes, any logic validation we do is pointless and a waste of time, and therefore we should be able to optimise it away to save time. *cough* Since in reality we're not that clever and do make mistakes, we actually do want to do some such program validation, but with the option to optimise it away. Hence the assert statement. So, a totally made-up example: def function(x, y): if x < 0: raise ValueError("x must be zero or positive") if y > 0: raise ValueError("y must be zero or negative") z = x*y assert z < 0, "expected product of +ve and -ve number to be -ve" return 1.0/(z-1) This example cunningly demonstrates: (1) Using explicit test-and-raise for ensuring that user-supplied arguments are always validated; (2) Using an assertion to test your program logic; (3) That the assertion in fact will catch an error in the program logic, since if you pass x or y equal to zero, the assertion will fail. Any time you are tempted to write a comment saying "This can't happen, but we check for it just in case", that is a perfect candidate for an assertion. Since it can't happen, it doesn't matter if it goes away with the -O flag; but since we're imperfect, and we want to cover ourselves just in case it does happen, we perform the test when not optimized. >From my own code, I have a global constant: UNICODE_NUMERALS = u'\uff10\uff11\uff12\uff13\uff14\uff15\uff16\uff17 \uff18\uff19' And then to make sure I haven't missed any: assert len(UNICODE_NUMERALS) == 10 In another function, I validate a mapping {key:value} to ensure that all the values are unique: seen_values = set() for k,v in mapping.items(): if v in seen_values: raise ValueError('duplicate value %s' % k) seen_values.add(v) # If we get here without error, then the mapping contains no # duplicate values. assert len(seen_values) == len(mapping) The assertion acts as a double-check on my logic, not the data. If my logic is wrong (perhaps there is a way to get past the for-loop while there is a duplicate?) then the assertion will catch it. -- Steven From me4 at privacy.net Tue Jan 5 09:04:56 2010 From: me4 at privacy.net (=?iso-8859-1?Q?Jens_M=FCller?=) Date: Tue, 5 Jan 2010 15:04:56 +0100 Subject: Speeding up network access: threading? In-Reply-To: References: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <4b4365b2$0$6568$9b4e6d93@newsspool4.arcor-online.net> Hello, > The fairly obvious thing to do is use a queue.queue for tasks and another > for results and a pool of threads that read, fetch, and write. Thanks, indeed. Is a list thrad-safe or do I need to lock when adding the results of my worker threads to a list? The order of the elements in the list does not matter. Jens From starglider.dev at gmail.com Tue Jan 5 09:09:43 2010 From: starglider.dev at gmail.com (starglider develop) Date: Tue, 5 Jan 2010 14:09:43 +0000 Subject: Talking with ebay using easyBay Message-ID: Hi, I made an application to manage auctions with easyBay my problem is that easybay site is down and there is any other source of information in the net regarding the packadge. Any one has the docs? or know how to make a search? Thank you in advance. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eight32 at gmail.com Tue Jan 5 09:15:52 2010 From: eight32 at gmail.com (Stuart Murray-Smith) Date: Tue, 5 Jan 2010 16:15:52 +0200 Subject: Python books, literature etc Message-ID: Greetings list I can code in Python (strong beginner), and would like to read more books and/or online resources. Could someone please point out any good books, websites, tutorials etc to help me get to the next level. Your help && insight highly appreciated :) Stuart From darcy at druid.net Tue Jan 5 09:19:59 2010 From: darcy at druid.net (D'Arcy J.M. Cain) Date: Tue, 5 Jan 2010 09:19:59 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <03533377$0$1336$c3e8da3@news.astraweb.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> <03533377$0$1336$c3e8da3@news.astraweb.com> Message-ID: <20100105091959.ce6bf039.darcy@druid.net> On 05 Jan 2010 14:02:50 GMT Steven D'Aprano wrote: > shouldn't use assert for validating user data except for quick-and-dirty > scripts you intend to use once and throw away. A mythcial beast that has yet to be spotted in the wild. -- D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner. From aioe.org at technicalbloke.com Tue Jan 5 09:21:05 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 14:21:05 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> Message-ID: Ben Finney wrote: > r0g writes: > >> Paul Rudin wrote: >>> Doesn't python just return a single result? (I know it can be a >>> tuple and assignment statements will unpack a tuple for you.) >> Yes, it returns a tuple if you return more than one value, it just has >> a lovely syntax for it. > > No, there is nothing inherent to the ?return? statement for dealing with > multiple values. > > The ?return? statement *always* returns a single value: whatever value > you specify as the argument to the statement (or the value ?None? if no > argument is specified. If you specify a tuple ? and Python has a nice > syntax for creating a literal tuple ? that's the single value the > statement will use. > That's what I said Ben... >>> Doesn't python just return a single result? >> Yes, See how I agree that "The ?return? statement *always* returns a single value"? >> it returns a tuple if you return more than one value, it just has >> a lovely syntax for it. You're confusing literal and conceptual returns. You can tell there are two senses of return at play because the two returns have different possessives within the same sentence. Look... "Yes, IT returns a tuple" - Speaking literally about Python... return 1, 2, 3 # Python returned one value "YOU return more than one value" - Speaking conceptually from a programmers perspective. return 1, 2, 3 # I returned three values Both are valid in their own context. The second (conceptual) statement MIGHT be ambiguous if it weren't for the first literal one. Both cannot be literally true and the first one clearly IS a literal factual statement about Python so the second cannot also be interpreted as literal. So let's disgard it... >> it returns a tuple, it just has a lovely syntax for it. The first two "it"s are Python, the third could either be... (a) The construction of tuples or at a stretch... (b) The act of returning tuples i.e. some special syntax for returning tuples. You seem to think I meant (b) - I actually meant (a) Maybe I could have made that clearer but I don't think people want to read legalese and I think it takes a particular pedantic, nitpicking disposition to even spot such small ambiguities. Of course I'm now guilty of pedantry too :/ I might have let it slip had you not started your reply with the word "No", that just p***** me off. Having said that I find the mental image of you slamming your fist on the table and shouting it out loud whenever you read something you disagree with on usenet quite amusing! Roger. From n00m at narod.ru Tue Jan 5 09:22:10 2010 From: n00m at narod.ru (n00m) Date: Tue, 5 Jan 2010 06:22:10 -0800 (PST) Subject: twenty years ago Guido created Python References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> <87oclaxksq.fsf@benfinney.id.au> <877hrxy3zz.fsf@benfinney.id.au> <02d9bd25-5ae0-4de3-910d-dc36bec0fc11@m3g2000yqf.googlegroups.com> Message-ID: Stick your English into your ass From aioe.org at technicalbloke.com Tue Jan 5 09:48:46 2010 From: aioe.org at technicalbloke.com (r0g) Date: Tue, 05 Jan 2010 14:48:46 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> <03533377$0$1336$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > On Tue, 05 Jan 2010 13:06:20 +0000, r0g wrote: >> Well maybe I didn't quite get it then, could you explain a bit further? >> >> My understanding was that asserts aren't executed at all if python is >> started with the -O or -OO option, > > Correct. > > >> or run through an optimizer. > > I don't know what you mean by that. I've never used them but I heard there are optimizers for python (psycho?). I assumed these would do everythin -O does and more, including losing the asserts. > >> If >> that's the case how can you expect it to validate anything at all in >> production? > > The asserts still operate so long as you don't use the -O switch. > >> Do you mean for debugging in situ or something? Could you >> maybe give me an example scenario to illustrate your point? > > > There are at least two sorts of validation that you will generally need > to perform: validating user data, and validating your program logic. > Cool, that's what I thought i.e. you can't rely on asserts being there so don't use them for anything critical but it's still a good idea to use them for logic/consistency checking in production code as, should you be running your production code unoptimised, it might catch something you'd otherwise miss. Thanks for responding is such detail :) Roger. From theller at python.net Tue Jan 5 10:02:59 2010 From: theller at python.net (Thomas Heller) Date: Tue, 05 Jan 2010 16:02:59 +0100 Subject: ctypes: How to call unexported functions in a dll In-Reply-To: References: Message-ID: <7qh2l3Fka8U1@mid.individual.net> Am 05.01.2010 12:19, schrieb Coert Klaver (DT): > Hi, > > I am using ctypes in python 3 on a WXP machine > > Loading a dll and using its exported functions works fine. > > Now I want to use a function in the dll that is not exported. > > In C this can be done by just casting the address in the dll of that > function to an apropriate function pointer and call it (you need to be > sure about the address). Can a similar thing be done directly with > ctypes? > > A work around I see is writing in wrapper dll in c that loads the main > dll, exports a function that calls the unexported function in the main > dll, but I don't find that an elegant solution. No need for a workaround. One solution is to first create a prototype for the function by calling WINFUNCTYPE or CFUNCTYPE, depending on the calling convention: stdcall or cdecl, then call the prototype with the address of the dll function which will return a Python callable. Demonstrated by the following script using GetProcAddress to get the address of the GetModuleHandleA win32 api function: >>> from ctypes import * >>> dll = windll.kernel32 >>> addr = dll.GetProcAddress(dll._handle, "GetModuleHandleA") >>> print hex(addr) 0x7c80b741 >>> proto = WINFUNCTYPE(c_int, c_char_p) >>> func = proto(addr) >>> func >>> func(None) 486539264 >>> hex(func(None)) '0x1d000000' >>> hex(func("python24.dll")) '0x1e000000' >>> hex(func("python.exe")) '0x1d000000' >>> Thomas From mackrackit at gmail.com Tue Jan 5 10:31:09 2010 From: mackrackit at gmail.com (Dave McCormick) Date: Tue, 05 Jan 2010 08:31:09 -0700 Subject: Dynamic text color In-Reply-To: References: Message-ID: <4B435B3D.3040405@gmail.com> John Posner wrote: > On Fri, 01 Jan 2010 21:01:04 -0500, Cousin Stanley > wrote: > > >> >> I was not familiar with the re.finditer method >> for searching strings ... > > Stanley and Dave -- > > So far, we've just been using finditer() to perform standard-string > searches (e.g. on the word "red"). Since Dave now wants to color > multiple words the same color (e.g. the words in redList), we can use > a single regular-expression search to locate *all* the words in a > list. This eliminates the need to use a "for" loop to handle the list. > Here's what I mean: > > >>> import re > >>> s = "it is neither red nor crimson, but scarlet, you see" > > ########## individual searches > > >>> [matchobj.span() for matchobj in re.finditer("red", s)] > [(14, 17)] > >>> [matchobj.span() for matchobj in re.finditer("crimson", s)] > [(22, 29)] > >>> [matchobj.span() for matchobj in re.finditer("scarlet", s)] > [(35, 42)] > > ########## one "swell foop" > > >>> redList = "red crimson scarlet".split() > >>> redList_regexp = "|".join(redList) > >>> redList_regexp > 'red|crimson|scarlet' > >>> [matchobj.span() for matchobj in re.finditer(redList_regexp, s)] > [(14, 17), (22, 29), (35, 42)] > > -John Thanks again John, This is fun!!! I made a "red.text" file to hold the "red words", they are separated by a space in the file. Still need to add the extra parameter "color" someplace. But this is what I have so far. ############## file = 'red.txt' file = open("red.txt","r") rList = file.readlines() file.close() redList = str(rList).split() blueList = "blue ball".split() greenList = "green grass".split() def get_complete_text(event): complete_text = Tbox.get("1.0", END) Tbox.tag_remove("red", "1.0", END) Tbox.tag_remove("blue", "1.0", END) Tbox.tag_remove("green", "1.0", END) ####RED######## redList_regexp = "|".join(redList) for matchobj in re.finditer(redList_regexp, complete_text): start,end = matchobj.span() Tbox.tag_add("red", "1.0 + %d chars" % start,"1.0 + %d chars" % end) Tbox.tag_config("red", foreground="red") ####BLUE####### blueList_regexp = "|".join(blueList) for matchobj in re.finditer(blueList_regexp, complete_text): start,end = matchobj.span() Tbox.tag_add("blue", "1.0 + %d chars" % start,"1.0 + %d chars" % end) Tbox.tag_config("blue", foreground="blue") ####GREEN####### greenList_regexp = "|".join(greenList) for matchobj in re.finditer(greenList_regexp, complete_text): start,end = matchobj.span() Tbox.tag_add("green", "1.0 + %d chars" % start,"1.0 + %d chars" % end) Tbox.tag_config("green", foreground="green") From davea at ieee.org Tue Jan 5 10:34:56 2010 From: davea at ieee.org (Dave Angel) Date: Tue, 05 Jan 2010 10:34:56 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> Message-ID: <4B435C20.6050105@ieee.org> r0g wrote: > Dave Angel wrote: > >> r0g wrote: >> >>> >>> >>> Maybe, although I recently learned on here that one can't rely on assert >>> statements in production code, their intended use is to aid debugging >>> and testing really. >>> >>> >>> >> Hopefully, what you learned is that you can't use assert() in production >> code to validate user data. It's fine to use it to validate program >> logic, because that shouldn't still need testing in production. >> >> >> >> DaveA >> > > > > Well maybe I didn't quite get it then, could you explain a bit further? > > My understanding was that asserts aren't executed at all if python is > started with the -O or -OO option, or run through an optimizer. If > that's the case how can you expect it to validate anything at all in > production? Do you mean for debugging in situ or something? Could you > maybe give me an example scenario to illustrate your point? > > Cheers, > > Roger. > > You understand the -O and -OO options fine. But the point is that you should not use assert() for anything that will be properly debugged before going to the user. You use if statements, and throw's to catch the error, and print to stderr, or GUI dialog boxes, or whatever mechanism you use to tell your user. But those errors are ones caused by his data, not by your buggy code. And the message tells him what's wrong with his data, not that you encountered a negative value for some low level function. I agree with Steve's pessimistic view of the state of most released software. But if you view a particular internal check as useful for production, then it should be coded in another mechanism, not in assert. Go ahead and write one, with a UI that's appropriate for your particular application. But it should do a lot more than assert does, including telling the user your contact information to call for support. def production_assert(expression, message): if not expression: dialog_box("Serious internal bug, call NNN-NNN-NNNN immediately", message) For an overly simplified example showing a user validation, and an assert : import sys def main(): try: text = raw_input("Enter your age, between 1 and 22 ") age = int(text) except ValueError, e: age = -1 if not 1 <= age <= 22: #not an assert print "Age must be between 1 and 22" print "Run program again" sys.exit(2) grade = calc_grade(age) print "Your grade is probably", grade table = [0, 0, 0, 0, 0, "K", "First", "2nd", 3] def calc_grade(age): """ calculate a probable grade value, given an i2nteger age between 1 and 22, inclusive """ assert(1 <= age <= len(table)) grade = table[age] #assume I have a fixed-length table for this return grade main() Note a few things. One I have a bug, in that the table isn't as big as the limit I'm checking for. With defensive coding, I'd have another assert for that, or even have the table size be available as a global constant (all uppers) so that everyone's in synch on the upper limit. But in any case, the test suite would be checking to make sure the code worked for 1, for 22, for a couple of values in between, and that a proper error response happened when a non-integer was entered, or one outside of the range. That all happens in separate code, not something in this file. And the test suite is run after every change to the sources, and certainly before release to production. Next, see the docstring. It establishes a precondition for the function. Since the function is called only by me (not the user), any preconditions can be checked with an assert. An assert without a supporting comment (or docstring) is almost worthless. And finally, notice that I check the user's input *before* passing it on to any uncontrolled code. So any asserts after that cannot fire, unless I have a bug which was not caught during testing. All opinions my own, of course. DaveA From benjamin.kaplan at case.edu Tue Jan 5 10:48:45 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Tue, 5 Jan 2010 10:48:45 -0500 Subject: embedded python on mac - linking problem In-Reply-To: <477262.24163.qm@web53508.mail.re2.yahoo.com> References: <477262.24163.qm@web53508.mail.re2.yahoo.com> Message-ID: On Tue, Jan 5, 2010 at 7:33 AM, Krzysztof Kobus wrote: > Hi, > > I have a problem with linking python module with my application on mac in order to make the module available in "embedded python". > > My python module is contained in ?j3kmodule.cxx file and module initialization function is exported in j3kmodule.h > > j3kmodule.h: > ------------ > PyMODINIT_FUNC PyInit_j3k(void); > > > j3kmodule.cxx: > -------------- > PyMODINIT_FUNC > PyInit_j3k(void) > ?{ > ?PyObject *m = NULL; > > ?if ((m = PyModule_Create(&j3k_module)) == NULL) > ? ?return NULL; > > ?return m; > ?} > > > Then in my application in KkPython.cxx file I have: > > KkPython.cxx: > ------------- > > #include "j3kmodule.h" > > /* Add a builtin module, before Py_Initialize */ > PyImport_AppendInittab("j3k", PyInit_j3k); > > > I link my application with the module and get following linking error on mac although on open suse linux exactly the same procedure works fine: > > g++ -headerpad_max_install_names -o ../../bin/render.app/Contents/MacOS/render main.o KkPython.o -lkkbase -L/Users/kk/dev/J3K/trunk/j3ksrc/examples/render/../../lib/ -lQtSolutions_PropertyBrowser-2.5 -lpython3.1 /Users/kk/dev/J3K/trunk/j3ksrc/python/modules/build/lib.macosx-10.3-i386-3.1/j3k.so -framework OpenGL -framework AGL > ld: warning in /Users/kk/dev/J3K/trunk/j3ksrc/python/modules/build/lib.macosx-10.3-i386-3.1/j3k.so, file is not of required architecture > Undefined symbols: > ?"_PyInit_j3k", referenced from: > ? ? ?_PyInit_j3k$non_lazy_ptr in KkPython.o > ld: symbol(s) not found > collect2: ld returned 1 exit status > make: *** [../../bin/render.app/Contents/MacOS/render] Error 1 > > > I appreciate any hints, > > best greetings, > > Krzysztof Kobus > Well, it seems that one of your files is a different architecture than the others. Based on the location, I'd say it's i386 while the rest of it would be PowerPC. You can cross-compile but you can't link an i386 library to a PowerPC library. From me4 at privacy.net Tue Jan 5 11:15:45 2010 From: me4 at privacy.net (=?iso-8859-1?Q?Jens_M=FCller?=) Date: Tue, 5 Jan 2010 17:15:45 +0100 Subject: Speeding up network access: threading? In-Reply-To: References: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <4b4365b2$1$6568$9b4e6d93@newsspool4.arcor-online.net> Hello, > The fairly obvious thing to do is use a queue.queue for tasks and another > for results and a pool of threads that read, fetch, and write. Thanks, indeed. Is a list thrad-safe or do I need to lock when adding the results of my worker threads to a list? The order of the elements in the list does not matter. Jens From keesvanschaik at gmail.com Tue Jan 5 11:22:53 2010 From: keesvanschaik at gmail.com (KvS) Date: Tue, 5 Jan 2010 08:22:53 -0800 (PST) Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> Message-ID: <38940c20-d714-4267-969a-4d9db1b4d5aa@j4g2000yqe.googlegroups.com> On Jan 5, 12:56?pm, Chris Gonnerman wrote: > KvS wrote: > > ... can I adjust the options normally appearing in > > the Printing Dialog through Python? > > Yes, if you use my method or my module, as I gave in my previous post. ? > If you use Adobe Reader to print, I'm not sure how to automate the print > settings. Thanks Chris, I'll go on and have a look. From keesvanschaik at gmail.com Tue Jan 5 11:34:20 2010 From: keesvanschaik at gmail.com (KvS) Date: Tue, 5 Jan 2010 08:34:20 -0800 (PST) Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> Message-ID: <263f9c76-17fb-4398-b65c-5b1b0007682b@j5g2000yqm.googlegroups.com> On Jan 5, 12:56?pm, Chris Gonnerman wrote: > KvS wrote: > > ... can I adjust the options normally appearing in > > the Printing Dialog through Python? > > Yes, if you use my method or my module, as I gave in my previous post. ? > If you use Adobe Reader to print, I'm not sure how to automate the print > settings. Ok, actually I quite like being able to print straightforward through your code, i.e. without any extra modules installed. I understand that sending text to the printer is in principle as simple as dc.TextOut(scale_factor * 72, -1 * scale_factor * 72, "Testing...") I didn't see you do anything with adjusting margins in the code. Does that mean that if I would e.g. do dc.TextOut(0, 0, "Testing...") the printout would appear in the upper left corner of the paper, as close to the edges as the printer is capable of? (Sorry, but I only have Ubuntu available at the moment, no Windows). From jjposner at optimum.net Tue Jan 5 11:35:22 2010 From: jjposner at optimum.net (John Posner) Date: Tue, 05 Jan 2010 11:35:22 -0500 Subject: A null program - what is it doing? In-Reply-To: References: Message-ID: <4B436A4A.1020005@optimum.net> > > No doubt a dumb question from a noob: > > The following program (a cut down version of some test code) uses no > CPU, and does not terminate: > > import sys > from PyQt4.QtCore import * > > if __name__=="__main__": > app = QCoreApplication(sys.argv) > sys.exit(app.exec_()) > > What is the program doing? What is the correct way to terminate the > execution? > Are you trying to understand QCoreApplication()? I can't help you there, since I've never used it. If you're just trying to get started with PyQt, use QApplication() instead: import sys from PyQt4.QtCore import * from PyQt4.QtGui import * if __name__=="__main__": app = QApplication(sys.argv) window = QLabel("I'm a PyQt window") window.show() sys.exit(app.exec_()) To terminate execution, just close the window by clicking the "X" in the window banner. HTH, John From phlip2005 at gmail.com Tue Jan 5 12:00:01 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 5 Jan 2010 09:00:01 -0800 (PST) Subject: Where's a DOM builder that uses the Builder Pattern to ... build DOMs? References: <04523152-a0ff-432c-a48f-2c653577e7b7@m25g2000yqc.googlegroups.com> <4b42f551$0$7625$9b4e6d93@newsspool1.arcor-online.net> Message-ID: <026df5db-bcef-424d-8cd5-56dacebf01f8@c3g2000yqd.googlegroups.com> On Jan 5, 12:16?am, Stefan Behnel wrote: > Note that there are tons of ways to generate HTML with Python. Forgot to note - I'm generating schematic XML, and I'm trying to find a way better than the Django template I started with! From python at mrabarnett.plus.com Tue Jan 5 12:02:29 2010 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 05 Jan 2010 17:02:29 +0000 Subject: Speeding up network access: threading? In-Reply-To: <4b4365b2$1$6568$9b4e6d93@newsspool4.arcor-online.net> References: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> <4b4365b2$1$6568$9b4e6d93@newsspool4.arcor-online.net> Message-ID: <4B4370A5.8060205@mrabarnett.plus.com> Jens M?ller wrote: > Hello, > >> The fairly obvious thing to do is use a queue.queue for tasks and another >> for results and a pool of threads that read, fetch, and write. > > Thanks, indeed. > > Is a list thrad-safe or do I need to lock when adding the results of my > worker threads to a list? The order of the elements in the list does not > matter. > Terry said "queue". not "list". Use the Queue class (it's thread-safe) in the "Queue" module (assuming you're using Python 2.x; in Python 3.x it's called the "queue" module). From lie.1296 at gmail.com Tue Jan 5 12:10:27 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Wed, 06 Jan 2010 04:10:27 +1100 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> <03533377$0$1336$c3e8da3@news.astraweb.com> Message-ID: <4b437282$1@dnews.tpgi.com.au> On 1/6/2010 1:48 AM, r0g wrote: > Steven D'Aprano wrote: >> On Tue, 05 Jan 2010 13:06:20 +0000, r0g wrote: >>> If >>> that's the case how can you expect it to validate anything at all in >>> production? >> >> The asserts still operate so long as you don't use the -O switch. >> >>> Do you mean for debugging in situ or something? Could you >>> maybe give me an example scenario to illustrate your point? >> >> >> There are at least two sorts of validation that you will generally need >> to perform: validating user data, and validating your program logic. >> > > > > Cool, that's what I thought i.e. you can't rely on asserts being there > so don't use them for anything critical but it's still a good idea to > use them for logic/consistency checking in production code as, should > you be running your production code unoptimised, it might catch > something you'd otherwise miss. Steven described the traditional approach to using assertions; another approach to when to use assertion is the one inspired by Design-by-Contract paradigm. DbC extends the traditional approach by focusing on writing a contract (instead of writing assertions) and generating assertions[1] to validate the contract. Just like assertions, these contracts are meant to be removed in production releases. In Design-by-Contract, only codes that interacts with the outer-world (e.g. getting user/file/network input, etc) need to do any sort of validations. Codes that doesn't interact directly with outside world only need to have a "contract" and simplified by *not* needing argument checking, since the function relies on the caller obeying the contract[2] and never calling it with an invalid input. DbC uses assertions[1] spuriously, unlike the traditional approach which is much more conservative when using assertions. [1] or explicit language support which is just syntax sugar for assertions [2] of course, on a debug release, the contract validation code will still be enforced to catch logic/consistency bugs that causes the violation From solipsis at pitrou.net Tue Jan 5 12:43:06 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 5 Jan 2010 17:43:06 +0000 (UTC) Subject: Speeding up network access: threading? References: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> <4b4365b2$0$6568$9b4e6d93@newsspool4.arcor-online.net> Message-ID: Le Tue, 05 Jan 2010 15:04:56 +0100, Jens M?ller a ?crit?: > > Is a list thrad-safe or do I need to lock when adding the results of my > worker threads to a list? The order of the elements in the list does not > matter. The built-in list type is thread-safe, but is doesn't provide the waiting features that queue.Queue provides. Regards Antoine. From steve at holdenweb.com Tue Jan 5 12:44:40 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 05 Jan 2010 12:44:40 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <20100105091959.ce6bf039.darcy@druid.net> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> <03533377$0$1336$c3e8da3@news.astraweb.com> <20100105091959.ce6bf039.darcy@druid.net> Message-ID: <4B437A88.1070604@holdenweb.com> D'Arcy J.M. Cain wrote: > On 05 Jan 2010 14:02:50 GMT > Steven D'Aprano wrote: >> shouldn't use assert for validating user data except for quick-and-dirty >> scripts you intend to use once and throw away. > > A mythcial beast that has yet to be spotted in the wild. > Not true (he wrote, picking nits). Such programs are written all the time. The fact that they invariably get used more often than intended doesn't negate the intentions of the author. ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Tue Jan 5 12:44:40 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 05 Jan 2010 12:44:40 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <20100105091959.ce6bf039.darcy@druid.net> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> <03533377$0$1336$c3e8da3@news.astraweb.com> <20100105091959.ce6bf039.darcy@druid.net> Message-ID: <4B437A88.1070604@holdenweb.com> D'Arcy J.M. Cain wrote: > On 05 Jan 2010 14:02:50 GMT > Steven D'Aprano wrote: >> shouldn't use assert for validating user data except for quick-and-dirty >> scripts you intend to use once and throw away. > > A mythcial beast that has yet to be spotted in the wild. > Not true (he wrote, picking nits). Such programs are written all the time. The fact that they invariably get used more often than intended doesn't negate the intentions of the author. ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From jjposner at optimum.net Tue Jan 5 12:53:01 2010 From: jjposner at optimum.net (John Posner) Date: Tue, 05 Jan 2010 12:53:01 -0500 Subject: Dynamic text color References: Message-ID: On Tue, 05 Jan 2010 10:31:09 -0500, Dave McCormick wrote: > ... But this is what I have so far. > ############## > file = 'red.txt' > file = open("red.txt","r") > rList = file.readlines() > file.close() > redList = str(rList).split() Dave, you're doing exactly the right thing: gradually expanding your program, to provide more functionality and to learn more about the available programming tools. It's also very good that you take care to close() the file after processing it. Now for the bad news ... 1. Don't use "file" as a variable name -- it's a built-in object type. (Some people don't like the fact that Python allows you to redefine such "reserved words".) 2. It's probably not the best idea to use a single variable (you use "file") to do double-duty: to hold the name of a file, and to hold the open-file object returned by the open() function. It's perfectly legal, but it hides information that might be useful when you're debugging a program. This is better: fname = 'red.txt' inpf = open(fname, "r") 3. It might be better to use read() rather than readlines() to process the "red.txt" file. It depends on what that file is supposed to contain. For example, if you expect "red.txt" to contain exactly one line, which has one or more words, you can process the open-file object like this: file_contents = inpf.read() redList = file_contents.split() ... or ... redList = inpf.read().split() It's certainly a mistake to use the expression "str(rList).split()". Using str() to convert the list "rList" into a string creates a mess that includes square-bracket characters. Did this actually work for you? Best, John From vicente.soler at gmail.com Tue Jan 5 13:12:00 2010 From: vicente.soler at gmail.com (vsoler) Date: Tue, 5 Jan 2010 10:12:00 -0800 (PST) Subject: parsing an Excel formula with the re module Message-ID: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> Hello, I am acessing an Excel file by means of Win 32 COM technology. For a given cell, I am able to read its formula. I want to make a map of how cells reference one another, how different sheets reference one another, how workbooks reference one another, etc. Hence, I need to parse Excel formulas. Can I do it by means only of re (regular expressions)? I know that for simple formulas such as "=3*A7+5" it is indeed possible. What about complex for formulas that include functions, sheet names and possibly other *.xls files? For example "=Book1!A5+8" should be parsed into ["=","Book1", "!", "A5","+","8"] Can anybody help? Any suggestions? Vicente Soler From keesvanschaik at gmail.com Tue Jan 5 13:19:09 2010 From: keesvanschaik at gmail.com (KvS) Date: Tue, 5 Jan 2010 10:19:09 -0800 (PST) Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> Message-ID: On Jan 5, 12:56?pm, Chris Gonnerman wrote: > KvS wrote: > > ... can I adjust the options normally appearing in > > the Printing Dialog through Python? > > Yes, if you use my method or my module, as I gave in my previous post. ? > If you use Adobe Reader to print, I'm not sure how to automate the print > settings. Sorry, one more. I completely forgot it's not exactly plain text, but occasionally also a limited number of non-ASCII characters (accents in names etc.). Would this be possible through your method? From mensanator at aol.com Tue Jan 5 13:24:39 2010 From: mensanator at aol.com (Mensanator) Date: Tue, 5 Jan 2010 10:24:39 -0800 (PST) Subject: it gets worse (was: How do you configure IDLE on a Mac...) References: <0d70cb54-3d77-4176-b621-e764ecf613b7@26g2000yqo.googlegroups.com> Message-ID: <6672dad2-26ba-458b-8075-21bac6506179@e37g2000yqn.googlegroups.com> On Jan 5, 12:32?am, Ned Deily wrote: > In article > <0d70cb54-3d77-4176-b621-e764ecf61... at 26g2000yqo.googlegroups.com>, > > > > > > ?Mensanator wrote: > > I assume I've been using the IDLE from macports. From the command > > prompt I've > > been typing "idle". This launches a "shell" window which appears to > > have an X11 > > parent application for which there are no "preferences" applicable to > > fonts. > > > However, if I use the quick launcher from the python.org, I get a > > "shell" whose > > parent is named "IDLE"! And that one has a completely different > > preferences, > > one similar the the Windows Configure which allows me to set the font! > > > Now, if I close this shell and start IDLE from the command line again, > > I still > > get a "shell" with an X11 parent, but, lo and behold, the font has > > changed to > > what I had previously set with the IDLE parent. > > > Course, I can't import gmpy, cause the python.org version can't > > compile it, so I > > still have to use the macports install of 3.1, but that's ok, once I > > use > > the IDLE application to set the preferences, I can switch back to the > > X11 version and the preferences will follow. > > The prefs follow because all versions of IDLE use the same (unversioned) > directory for configuration files, ~/.idlerc/. ?In particular, the > configuration file ~/.idlerc/config-main.cfg contains, among other > things, any changes to the default font. ?So, if you're successful at > changing it in one version of IDLE, it will likely affect all versions > you have. ?Note the file is a simple ini format: > > [EditorWindow] > font = monaco > > so you can edit it by hand. Good to know. But, as the subject says... > > BTW, the python.org IDLEs and the Apple-supplied IDLEs use the > system-supplied Aqua (aka Quartz) Tk not the X11 one that MacPorts > builds by default. ?The MacPorts Tk port does have a "quartz" variant > but that doesn't yet work in 64-bit mode. So, for all practical purposes, the macports install is broken also. IDLE simply does not work in an X11 window (you think someone would have noticed that). The missing preferences is just the beginning. Apparently NONE of the menu item shortcuts work. For example, the Cut, Copy, Paste shortcuts are given as Command-X, Command-C and Command-V. But that doesn't work in an X11 window, apperently only in an Aqua Tk (parent application appears as IDLE). Of course, I can do Control-X, Control-C and Control-V to do Cut, Copy and Paste. Don't know if this works for all shortcuts, but I suppose I could just pick them from the menu (and I can bang my head against the wall while I'm at it). What do you think, suppose I copy the gmpy built with the macports install over to the directory where the python.org version is? Would it import? If that'll work, I can switch back to using the python.org install and use it's version of IDLE. I certainly won't be needing distutils once I have a working version of gmpy. > > -- > ?Ned Deily, > ?n... at acm.org- Hide quoted text - > > - Show quoted text - From python at mrabarnett.plus.com Tue Jan 5 13:35:02 2010 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 05 Jan 2010 18:35:02 +0000 Subject: parsing an Excel formula with the re module In-Reply-To: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> Message-ID: <4B438656.7080601@mrabarnett.plus.com> vsoler wrote: > Hello, > > I am acessing an Excel file by means of Win 32 COM technology. > For a given cell, I am able to read its formula. I want to make a map > of how cells reference one another, how different sheets reference one > another, how workbooks reference one another, etc. > > Hence, I need to parse Excel formulas. Can I do it by means only of re > (regular expressions)? > > I know that for simple formulas such as "=3*A7+5" it is indeed > possible. What about complex for formulas that include functions, > sheet names and possibly other *.xls files? > > For example "=Book1!A5+8" should be parsed into ["=","Book1", "!", > "A5","+","8"] > > Can anybody help? Any suggestions? > Do you mean "how" or do you really mean "whether", ie, get a list of the other cells that are referred to by a certain cell, for example, "=3*A7+5" should give ["A7"] and "=Book1!A5+8" should give ["Book1!A5"]? From me4 at privacy.net Tue Jan 5 13:45:50 2010 From: me4 at privacy.net (=?iso-8859-1?Q?Jens_M=FCller?=) Date: Tue, 5 Jan 2010 19:45:50 +0100 Subject: Speeding up network access: threading? In-Reply-To: References: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> <4b4365b2$1$6568$9b4e6d93@newsspool4.arcor-online.net> Message-ID: <4b4388dd$0$7616$9b4e6d93@newsspool1.arcor-online.net> Hi and sorry for double posting - had mailer problems, > Terry said "queue". not "list". Use the Queue class (it's thread-safe) > in the "Queue" module (assuming you're using Python 2.x; in Python 3.x > it's called the "queue" module). Yes yes, I know. I use a queue to realize the thread pool queue, that works all right. But each worker thread calculates a result and needs to make it avaialable to the application in the main thread again. Therefore, it appends its result to a common list. This seems works as well, but I was thinking of possible conflict situations that maybe could happen when two threads append their results to that same result list at the same moment. Regards, Jens From nobody at nowhere.com Tue Jan 5 13:58:23 2010 From: nobody at nowhere.com (Nobody) Date: Tue, 05 Jan 2010 18:58:23 +0000 Subject: IOError - cannot create file (linux daemon-invoked script) References: Message-ID: On Mon, 04 Jan 2010 21:30:31 -0800, cassiope wrote: > One more tidbit observed: my last note, that it works when using > seteuid/setegid? > Well - that only applies if the daemon is running under strace (!). > It fails > if started directly by root, or if the strace session has ended, > leaving the > main body of the daemon running in its normal headless manner. > > I wonder if running under "strace -f" - might setegid/seteuid be > prevented from > having their normal effect? Possibly. The ptrace() syscall on which strace depends will fail if you try to trace a "privileged" process and you aren't root, so it's possible that a ptrace()d process will refuse to become privileged. Here, "privileged" includes a process which has changed any of its UIDs or GIDs (this prevents a normal user from tracing, killing, etc an otherwise privileged process which has switched to the user's UID for the time being). From phlip2005 at gmail.com Tue Jan 5 14:00:24 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 5 Jan 2010 11:00:24 -0800 (PST) Subject: twenty years ago Guido created Python References: Message-ID: <51564fb3-a367-4f11-b97f-5518deedbe48@a21g2000yqc.googlegroups.com> On Dec 31 2009, 2:06?pm, Steve Howell wrote: > Python is a truly awesome programming language. ?Not only is Guido a > genius language designer, but he is also a great project leader. ?What > an accomplishment. ?Congratulations to everybody who has contributed > to Python in the last two decades! The more languages you learn before getting to Smalltalk, the more awesome Smalltalk will be for you. -- Phlip From vicente.soler at gmail.com Tue Jan 5 14:00:43 2010 From: vicente.soler at gmail.com (vsoler) Date: Tue, 5 Jan 2010 11:00:43 -0800 (PST) Subject: parsing an Excel formula with the re module References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> Message-ID: <11cdf1c8-51e4-45ec-9ac4-1fd77c5e018a@e37g2000yqn.googlegroups.com> On 5 ene, 19:35, MRAB wrote: > vsoler wrote: > > Hello, > > > I am acessing an Excel file by means of Win 32 COM technology. > > For a given cell, I am able to read its formula. I want to make a map > > of how cells reference one another, how different sheets reference one > > another, how workbooks reference one another, etc. > > > Hence, I need to parse Excel formulas. Can I do it by means only of re > > (regular expressions)? > > > I know that for simple formulas such as "=3*A7+5" it is indeed > > possible. What about complex for formulas that include functions, > > sheet names and possibly other *.xls files? > > > For example ? ?"=Book1!A5+8" should be parsed into ["=","Book1", "!", > > "A5","+","8"] > > > Can anybody help? Any suggestions? > > Do you mean "how" or do you really mean "whether", ie, get a list of the > other cells that are referred to by a certain cell, for example, > "=3*A7+5" should give ["A7"] and "=Book1!A5+8" should give ["Book1!A5"]? I'd like to know how to do it, should it be possible. Vicente From jjposner at optimum.net Tue Jan 5 14:04:05 2010 From: jjposner at optimum.net (John Posner) Date: Tue, 05 Jan 2010 14:04:05 -0500 Subject: parsing an Excel formula with the re module References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> Message-ID: On Tue, 05 Jan 2010 13:12:00 -0500, vsoler wrote: > Hello, > > I am acessing an Excel file by means of Win 32 COM technology. > For a given cell, I am able to read its formula. I want to make a map > of how cells reference one another, how different sheets reference one > another, how workbooks reference one another, etc. > > Hence, I need to parse Excel formulas. Can I do it by means only of re > (regular expressions)? > > I know that for simple formulas such as "=3*A7+5" it is indeed > possible. What about complex for formulas that include functions, > sheet names and possibly other *.xls files? > > For example "=Book1!A5+8" should be parsed into ["=","Book1", "!", > "A5","+","8"] > > Can anybody help? Any suggestions? It seems like you want to recreate data structures that Excel, itself, must maintain in order to recalculate cells in the correct order. As long as you're using COM, you might be able to tap into those data structures. My 15-year-old (!) "Using Excel Visual Basic for Applications" book wasn't any help. :-( After a short Google session, I came up with one possible lead: http://www.decisionmodels.com/ Good luck! John From mensanator at aol.com Tue Jan 5 14:05:32 2010 From: mensanator at aol.com (Mensanator) Date: Tue, 5 Jan 2010 11:05:32 -0800 (PST) Subject: parsing an Excel formula with the re module References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> Message-ID: <5ef3575d-eb1d-4a15-90e0-87dd85cdc2e6@35g2000yqa.googlegroups.com> On Jan 5, 12:35?pm, MRAB wrote: > vsoler wrote: > > Hello, > > > I am acessing an Excel file by means of Win 32 COM technology. > > For a given cell, I am able to read its formula. I want to make a map > > of how cells reference one another, how different sheets reference one > > another, how workbooks reference one another, etc. > > > Hence, I need to parse Excel formulas. Can I do it by means only of re > > (regular expressions)? > > > I know that for simple formulas such as "=3*A7+5" it is indeed > > possible. What about complex for formulas that include functions, > > sheet names and possibly other *.xls files? > > > For example ? ?"=Book1!A5+8" should be parsed into ["=","Book1", "!", > > "A5","+","8"] > > > Can anybody help? Any suggestions? > > Do you mean "how" or do you really mean "whether", ie, get a list of the > other cells that are referred to by a certain cell, for example, > "=3*A7+5" should give ["A7"] and "=Book1!A5+8" should give ["Book1!A5] Ok, although "Book1" would be the default name of a workbook, with default worksheets labeled "Sheet1". "Sheet2", etc. If I had a worksheet named "Sheety" that wanted to reference a cell on "Sheetx" OF THE SAME WORKBOOK, it would be =Sheet2!A7. If the reference was to a completely different workbook (say Book1 with worksheets labeled "Sheet1", "Sheet2") then the cell might have =[Book1]Sheet1!A7. And don't forget the $'s! You may see =[Book1]Sheet1!$A$7. From neilc at norwich.edu Tue Jan 5 14:07:26 2010 From: neilc at norwich.edu (Neil Cerutti) Date: 5 Jan 2010 19:07:26 GMT Subject: Dynamic text color References: Message-ID: <7qhgveFe0fU1@mid.individual.net> On 2010-01-05, John Posner wrote: > 2. It's probably not the best idea to use a single variable > (you use "file") to do double-duty: to hold the name of a > file, and to hold the open-file object returned by the open() > function. It's perfectly legal, but it hides information that > might be useful when you're debugging a program. This is > better: > > fname = 'red.txt' > inpf = open(fname, "r") Alternatively: >>> infile = open("red.txt", "r") >>> infile.name 'red.txt' -- Neil Cerutti From nobody at nowhere.com Tue Jan 5 14:08:02 2010 From: nobody at nowhere.com (Nobody) Date: Tue, 05 Jan 2010 19:08:02 +0000 Subject: fsync() doesn't work as advertised? References: Message-ID: On Mon, 04 Jan 2010 08:09:56 -0800, Brian D wrote: > If I'm running a process in a loop that runs for a long time, I > occasionally would like to look at a log to see how it's going. > > I know about the logging module, and may yet decide to use that. > > Still, I'm troubled by how fsync() doesn't seem to work as advertised: > > http://docs.python.org/library/os.html > > "If you?re starting with a Python file object f, first do f.flush(), > and then do os.fsync(f.fileno())" The .flush() method (and the C fflush() function) causes the contents of application buffers to be sent to the OS, which basically copies the data into the OS-level buffers. fsync() causes the OS-level buffers to be written to the physical drive. File operations normally use the OS-level buffers; e.g. if one process write()s to a file and another process read()s it, the latter will see what the former has written regardless of whether the data has been written to the drive. The main reason for using fsync() is to prevent important data from being lost in the event of an unexpected reboot or power-cycle (an expected reboot via the "shutdown" or "halt" commands will flush all OS-level buffers to the drive first). Other than that, fsync() is almost invisible (I say "almost", as there are mechanisms to bypass the OS-level buffers, e.g. the O_DIRECT open() flag). From vicente.soler at gmail.com Tue Jan 5 14:09:09 2010 From: vicente.soler at gmail.com (vsoler) Date: Tue, 5 Jan 2010 11:09:09 -0800 (PST) Subject: What is the best data structure for a very simple spreadsheet? References: <82c5a9f1-ed36-40c4-83d6-2d2d33ee40bc@j24g2000yqa.googlegroups.com> Message-ID: <3b7d5cf5-2a2d-46a9-9f1b-490fac3da54b@v25g2000yqk.googlegroups.com> On 3 ene, 22:40, mdipierro wrote: > Perhaps this can be useful:http://www.web2py.com/examples/spreadsheet > > The code is in a single file with not dependencies and it does not > require web2py to run:http://code.google.com/p/web2py/source/browse/gluon/contrib/spreadshe... > > Here is a sample controller that shows you how to embed the > spreadsheet in web page:http://code.google.com/p/web2py/source/browse/applications/examples/c... > > Massimo > > On Jan 3, 5:27?am, vsoler wrote: > > > Hi, > > > Not sure this is the best group to post, but I cannot think of any > > other. > > > My application would contain a limited set of "cells" represented by > > the instances of a Cell class: > > > class Cell: > > ... > > > A1=Cell(7) > > A2=Cell(2*A1) > > A3=Cell(3*A1+A2) > > A4=Cell(A3*4) > > > Of course, A1 = 7, A2 = 14, A3 = 35 and A4 = 140 > > > Now, I somehow want to be able to show a dependency tree > > > 1 level dependency trees > > ? A1: None > > ? A2: A1 > > ? A3: A1, A2 > > ? A4: A3 > > > All levels dependency trees > > > ? A1: None > > ? A2: A1 > > ? A3: A1, A2 > > ? A4: A3, A2, A1 > > > Leaf + values dependency trees: > > > ? A1: 7 > > ? A2: A1=7, 2 > > ? A3: 3, A1=7, 2 > > ? A4: 3, A1=7, 2, 4 > > > What I'd like to know is: > > > 1) what are, in your opinion, the basic elements of the Cell class? > > 2) Do I need a parser to evaluate the formulas like ?3*A1+A2?? Can you > > recommend one library that already contains one? > > 3) Do I need a tree data structure to represent my data? would the > > tree be an attribute of the class instance? > > > I imagine a lot can be said on these questions. What I am looking for > > is some hints that help me get out of where I am now. > > > Any help is highly appreciated. > > > Vicente Soler > > There is something that I appreciate in this group, and it is the high degree of knowledge of all the participants that are helping with their answers. After studying your suggestions, I'll come back to you. Thank you very much. Vicente Soler From steve at holdenweb.com Tue Jan 5 14:11:05 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 05 Jan 2010 14:11:05 -0500 Subject: Speeding up network access: threading? In-Reply-To: <4b4388dd$0$7616$9b4e6d93@newsspool1.arcor-online.net> References: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> <4b4365b2$1$6568$9b4e6d93@newsspool4.arcor-online.net> <4b4388dd$0$7616$9b4e6d93@newsspool1.arcor-online.net> Message-ID: <4B438EC9.3010100@holdenweb.com> Jens M?ller wrote: > Hi and sorry for double posting - had mailer problems, > >> Terry said "queue". not "list". Use the Queue class (it's thread-safe) >> in the "Queue" module (assuming you're using Python 2.x; in Python 3.x >> it's called the "queue" module). > > Yes yes, I know. I use a queue to realize the thread pool queue, that > works all right. > > But each worker thread calculates a result and needs to make it > avaialable to the application in the main thread again. Therefore, it > appends its result to a common list. This seems works as well, but I was > thinking of possible conflict situations that maybe could happen when > two threads append their results to that same result list at the same > moment. > If you don't need to take anything off the list ever, just create a separate thread that reads items from an output Queue and appends them to the list. If you *do* take them off, then use a Queue. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Tue Jan 5 14:11:05 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 05 Jan 2010 14:11:05 -0500 Subject: Speeding up network access: threading? In-Reply-To: <4b4388dd$0$7616$9b4e6d93@newsspool1.arcor-online.net> References: <4b4215cc$0$6716$9b4e6d93@newsspool2.arcor-online.net> <4b4365b2$1$6568$9b4e6d93@newsspool4.arcor-online.net> <4b4388dd$0$7616$9b4e6d93@newsspool1.arcor-online.net> Message-ID: <4B438EC9.3010100@holdenweb.com> Jens M?ller wrote: > Hi and sorry for double posting - had mailer problems, > >> Terry said "queue". not "list". Use the Queue class (it's thread-safe) >> in the "Queue" module (assuming you're using Python 2.x; in Python 3.x >> it's called the "queue" module). > > Yes yes, I know. I use a queue to realize the thread pool queue, that > works all right. > > But each worker thread calculates a result and needs to make it > avaialable to the application in the main thread again. Therefore, it > appends its result to a common list. This seems works as well, but I was > thinking of possible conflict situations that maybe could happen when > two threads append their results to that same result list at the same > moment. > If you don't need to take anything off the list ever, just create a separate thread that reads items from an output Queue and appends them to the list. If you *do* take them off, then use a Queue. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Tue Jan 5 14:15:02 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 05 Jan 2010 14:15:02 -0500 Subject: twenty years ago Guido created Python In-Reply-To: <51564fb3-a367-4f11-b97f-5518deedbe48@a21g2000yqc.googlegroups.com> References: <51564fb3-a367-4f11-b97f-5518deedbe48@a21g2000yqc.googlegroups.com> Message-ID: <4B438FB6.9050003@holdenweb.com> Phlip wrote: > On Dec 31 2009, 2:06 pm, Steve Howell wrote: > >> Python is a truly awesome programming language. Not only is Guido a >> genius language designer, but he is also a great project leader. What >> an accomplishment. Congratulations to everybody who has contributed >> to Python in the last two decades! > > The more languages you learn before getting to Smalltalk, the more > awesome Smalltalk will be for you. > After implementing SmallTalk I more or less gave up OO programming for ten years, resuming it only after I met Python. SmallTalk didn't seem that awesome to me. Though for its time it was an incredible system, its insistence on a SmallTalk-only VM environment seemed a little solipsistic. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From nobody at nowhere.com Tue Jan 5 14:16:32 2010 From: nobody at nowhere.com (Nobody) Date: Tue, 05 Jan 2010 19:16:32 +0000 Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> Message-ID: On Tue, 05 Jan 2010 04:40:14 -0800, KvS wrote: >> Did you mean borderless printing? >> Every printer needs his margins, some more some less. Some printers have the >> ability to do borderless printing but usualy they can do it only on special >> or photo paper. So you can adjust the pdf as you wish, even with no margins, >> and then try to find under printer options "borderless printing". That is >> why I didn't understand :-)) it is a printer thing not pdf! > > As much as possible "borderless", yes. Of course the printer will > still apply some small margin, but that's ok. A margin of say <0.5 cm. > is fine. So it's not a printer thing, I accept the (physical) > limitations of the printer, but I want to avoid any extra margins due > to software settings. "Hardcopy" document formats such as PostScript and PDF use positions relative to the edges of the page, not the margins. From python at mrabarnett.plus.com Tue Jan 5 14:20:09 2010 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 05 Jan 2010 19:20:09 +0000 Subject: parsing an Excel formula with the re module In-Reply-To: <11cdf1c8-51e4-45ec-9ac4-1fd77c5e018a@e37g2000yqn.googlegroups.com> References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> <11cdf1c8-51e4-45ec-9ac4-1fd77c5e018a@e37g2000yqn.googlegroups.com> Message-ID: <4B4390E9.1070900@mrabarnett.plus.com> vsoler wrote: > On 5 ene, 19:35, MRAB wrote: >> vsoler wrote: >>> Hello, >>> I am acessing an Excel file by means of Win 32 COM technology. >>> For a given cell, I am able to read its formula. I want to make a map >>> of how cells reference one another, how different sheets reference one >>> another, how workbooks reference one another, etc. >>> Hence, I need to parse Excel formulas. Can I do it by means only of re >>> (regular expressions)? >>> I know that for simple formulas such as "=3*A7+5" it is indeed >>> possible. What about complex for formulas that include functions, >>> sheet names and possibly other *.xls files? >>> For example "=Book1!A5+8" should be parsed into ["=","Book1", "!", >>> "A5","+","8"] >>> Can anybody help? Any suggestions? >> Do you mean "how" or do you really mean "whether", ie, get a list of the >> other cells that are referred to by a certain cell, for example, >> "=3*A7+5" should give ["A7"] and "=Book1!A5+8" should give ["Book1!A5"]? > > I'd like to know how to do it, should it be possible. > Something like this should work: references = re.findall(r"\b((?:\w+!)?[A-Za-z]+\d+)\b", formula) From vicente.soler at gmail.com Tue Jan 5 14:21:50 2010 From: vicente.soler at gmail.com (vsoler) Date: Tue, 5 Jan 2010 11:21:50 -0800 (PST) Subject: parsing an Excel formula with the re module References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> <5ef3575d-eb1d-4a15-90e0-87dd85cdc2e6@35g2000yqa.googlegroups.com> Message-ID: <6473e930-c5f4-423f-b78b-aae63f2aa794@k17g2000yqh.googlegroups.com> On 5 ene, 20:05, Mensanator wrote: > On Jan 5, 12:35?pm, MRAB wrote: > > > > > vsoler wrote: > > > Hello, > > > > I am acessing an Excel file by means of Win 32 COM technology. > > > For a given cell, I am able to read its formula. I want to make a map > > > of how cells reference one another, how different sheets reference one > > > another, how workbooks reference one another, etc. > > > > Hence, I need to parse Excel formulas. Can I do it by means only of re > > > (regular expressions)? > > > > I know that for simple formulas such as "=3*A7+5" it is indeed > > > possible. What about complex for formulas that include functions, > > > sheet names and possibly other *.xls files? > > > > For example ? ?"=Book1!A5+8" should be parsed into ["=","Book1", "!", > > > "A5","+","8"] > > > > Can anybody help? Any suggestions? > > > Do you mean "how" or do you really mean "whether", ie, get a list of the > > other cells that are referred to by a certain cell, for example, > > "=3*A7+5" should give ["A7"] and "=Book1!A5+8" should give ["Book1!A5] > > Ok, although "Book1" would be the default name of a workbook, with > default > worksheets labeled "Sheet1". "Sheet2", etc. > > If I had a worksheet named "Sheety" that wanted to reference a cell on > "Sheetx" > OF THE SAME WORKBOOK, it would be =Sheet2!A7. If the reference was to > a completely > different workbook (say Book1 with worksheets labeled "Sheet1", > "Sheet2") then > the cell might have =[Book1]Sheet1!A7. > > And don't forget the $'s! You may see =[Book1]Sheet1!$A$7. Yes, Mensanator, but... what re should I use? I'm looking for the re statement. No doubt you can help! Thank you. From aps.it2000 at gmail.com Tue Jan 5 14:31:47 2010 From: aps.it2000 at gmail.com (aung paing Soe) Date: Tue, 5 Jan 2010 11:31:47 -0800 Subject: Fwd: I would like to install Python on my 64 bit Win 7 In-Reply-To: References: Message-ID: ---------- Forwarded message ---------- From: aung paing Soe Date: Tue, Jan 5, 2010 at 11:27 AM Subject: I would like to install Python on my 64 bit Win 7 To: webmaster at python.org Hello , I would like to study about Python Programming . So I want to install Python . But my laptop is Window 7 64-bit home basic . So please give me a advice how to install Python in my 64 bit computer. I really want to study python programming . I am looking forward your reply. Thank you very much Yours, Beginner -------------- next part -------------- An HTML attachment was scrubbed... URL: From keesvanschaik at gmail.com Tue Jan 5 14:40:25 2010 From: keesvanschaik at gmail.com (KvS) Date: Tue, 5 Jan 2010 11:40:25 -0800 (PST) Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> Message-ID: <96ea39f2-1fb6-4b03-b206-d4c1445ea021@v25g2000yqk.googlegroups.com> On Jan 5, 7:16?pm, Nobody wrote: > On Tue, 05 Jan 2010 04:40:14 -0800, KvS wrote: > >> Did you mean borderless printing? > >> Every printer needs his margins, some more some less. Some printers have the > >> ability to do borderless printing but usualy they can do it only on special > >> or photo paper. So you can adjust the pdf as you wish, even with no margins, > >> and then try to find under printer options "borderless printing". That is > >> why I didn't understand :-)) it is a printer thing not pdf! > > > As much as possible "borderless", yes. Of course the printer will > > still apply some small margin, but that's ok. A margin of say <0.5 cm. > > is fine. So it's not a printer thing, I accept the (physical) > > limitations of the printer, but I want to avoid any extra margins due > > to software settings. > > "Hardcopy" document formats such as PostScript and PDF use positions > relative to the edges of the page, not the margins. Right. Still, Acrobat Reader by default scales the contents to fit on a page and creates some margins by doing so, no? So if my text is close to the left and right edges, as I want, it will get scaled and extra margins will occur. Avoiding this still requires me to be able to turn off this scaling in the printing preferences somehow programmatically, so it doesn't seem to make the problem easier? From python.list at tim.thechases.com Tue Jan 5 14:49:13 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Tue, 05 Jan 2010 13:49:13 -0600 Subject: parsing an Excel formula with the re module In-Reply-To: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> Message-ID: <4B4397B9.7000804@tim.thechases.com> vsoler wrote: > Hence, I need to parse Excel formulas. Can I do it by means only of re > (regular expressions)? > > I know that for simple formulas such as "=3*A7+5" it is indeed > possible. What about complex for formulas that include functions, > sheet names and possibly other *.xls files? Where things start getting ugly is when you have nested function calls, such as =if(Sum(A1:A25)>42,Min(B1:B25), if(Sum(C1:C25)>3.14, (Min(C1:C25)+3)*18,Max(B1:B25))) Regular expressions don't do well with nested parens (especially arbitrarily-nesting-depth such as are possible), so I'd suggest going for a full-blown parsing solution like pyparsing. If you have fair control over what can be contained in the formulas and you know they won't contain nested parens/functions, you might be able to formulate some sort of "kinda, sorta, maybe parses some forms of formulas" regexp. -tkc From vicente.soler at gmail.com Tue Jan 5 14:54:00 2010 From: vicente.soler at gmail.com (vsoler) Date: Tue, 5 Jan 2010 11:54:00 -0800 (PST) Subject: parsing an Excel formula with the re module References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> <5ef3575d-eb1d-4a15-90e0-87dd85cdc2e6@35g2000yqa.googlegroups.com> <6473e930-c5f4-423f-b78b-aae63f2aa794@k17g2000yqh.googlegroups.com> Message-ID: On 5 ene, 20:21, vsoler wrote: > On 5 ene, 20:05, Mensanator wrote: > > > > > On Jan 5, 12:35?pm, MRAB wrote: > > > > vsoler wrote: > > > > Hello, > > > > > I am acessing an Excel file by means of Win 32 COM technology. > > > > For a given cell, I am able to read its formula. I want to make a map > > > > of how cells reference one another, how different sheets reference one > > > > another, how workbooks reference one another, etc. > > > > > Hence, I need to parse Excel formulas. Can I do it by means only of re > > > > (regular expressions)? > > > > > I know that for simple formulas such as "=3*A7+5" it is indeed > > > > possible. What about complex for formulas that include functions, > > > > sheet names and possibly other *.xls files? > > > > > For example ? ?"=Book1!A5+8" should be parsed into ["=","Book1", "!", > > > > "A5","+","8"] > > > > > Can anybody help? Any suggestions? > > > > Do you mean "how" or do you really mean "whether", ie, get a list of the > > > other cells that are referred to by a certain cell, for example, > > > "=3*A7+5" should give ["A7"] and "=Book1!A5+8" should give ["Book1!A5] > > > Ok, although "Book1" would be the default name of a workbook, with > > default > > worksheets labeled "Sheet1". "Sheet2", etc. > > > If I had a worksheet named "Sheety" that wanted to reference a cell on > > "Sheetx" > > OF THE SAME WORKBOOK, it would be =Sheet2!A7. If the reference was to > > a completely > > different workbook (say Book1 with worksheets labeled "Sheet1", > > "Sheet2") then > > the cell might have =[Book1]Sheet1!A7. > > > And don't forget the $'s! You may see =[Book1]Sheet1!$A$7. > > Yes, Mensanator, but... ?what re should I use? I'm looking for the re > statement. No doubt you can help! > > Thank you. Let me give you an example: >>> import re >>> re.split("([^0-9])", "123+456*/") [?123?, ?+?, ?456?, ?*?, ??, ?/?, ??] I find it excellent that one single statement is able to do a lexical analysis of an expression! If the expression contains variables, such as A12 or B9, I can try another re expression. Which one should I use? And if my expression contains parenthesis? And the sin() function? Vicente Soler From phlip2005 at gmail.com Tue Jan 5 14:58:36 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 5 Jan 2010 11:58:36 -0800 (PST) Subject: please help shrink this each_with_index() implementation Message-ID: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> Hypo Nt: def each_with_index(seq): index = 0 result = [] for item in seq: result.append([item, index]) index += 1 return result My Pythonic sequencing skills are obviously feeble. Can anything think of a way to write that in fewer lines? -- Phlip http://c2.com/cgi/wiki?MoreliaViridis From mackrackit at gmail.com Tue Jan 5 15:03:13 2010 From: mackrackit at gmail.com (Dave McCormick) Date: Tue, 05 Jan 2010 13:03:13 -0700 Subject: Fwd: I would like to install Python on my 64 bit Win 7 In-Reply-To: References: Message-ID: <4B439B01.50407@gmail.com> An HTML attachment was scrubbed... URL: From mensanator at aol.com Tue Jan 5 15:05:06 2010 From: mensanator at aol.com (Mensanator) Date: Tue, 5 Jan 2010 12:05:06 -0800 (PST) Subject: twenty years ago Guido created Python References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> <87oclaxksq.fsf@benfinney.id.au> <877hrxy3zz.fsf@benfinney.id.au> <02d9bd25-5ae0-4de3-910d-dc36bec0fc11@m3g2000yqf.googlegroups.com> Message-ID: <29699581-f804-4ae0-9721-1d415ce1065b@h9g2000yqa.googlegroups.com> On Jan 5, 8:22?am, n00m wrote: > Stick your English into your ass Most people would say "up your ass". And use a period at the end of the sentence. Got any more funny insults? From mackrackit at gmail.com Tue Jan 5 15:08:04 2010 From: mackrackit at gmail.com (Dave McCormick) Date: Tue, 05 Jan 2010 13:08:04 -0700 Subject: Dynamic text color In-Reply-To: References: Message-ID: <4B439C24.9050205@gmail.com> John Posner wrote: > > Dave, you're doing exactly the right thing: gradually expanding your > program, to provide more functionality and to learn more about the > available programming tools. It's also very good that you take care to > close() the file after processing it. Now for the bad news ... Seems like there is always bad news :) > > 1. Don't use "file" as a variable name -- it's a built-in object type. > (Some people don't like the fact that Python allows you to redefine > such "reserved words".) > > 2. It's probably not the best idea to use a single variable (you use > "file") to do double-duty: to hold the name of a file, and to hold the > open-file object returned by the open() function. It's perfectly > legal, but it hides information that might be useful when you're > debugging a program. This is better: > > 3. It might be better to use read() rather than readlines() to process > the "red.txt" file. It depends on what that file is supposed to > contain. For example, if you expect "red.txt" to contain exactly one > line, which has one or more words, you can process the open-file > object like this: All noted and fixed. > > It's certainly a mistake to use the expression "str(rList).split()". > Using str() to convert the list "rList" into a string creates a mess > that includes square-bracket characters. Did this actually work for you? It sort of worked. With one color file it seemed fine but after I posted I added another color file and things fell apart. Now with the above fixes it works with three colors from three files. When the list are printed to the shell the list look like this: redList ['red', 'dog', 'apple', '#'] blueList ['blue', 'ball', 'berry'] greenList ['green', 'grass', 'do'] But another problem is noticed. It does not matter if the list is built in code or from a file. If dog is entered, "do" will be green with the "g" being red. Back to the drawing board..... Here is the complete code" ###### from Tkinter import * import re RFfile = 'red.txt' inpRF = open(RFfile,"r") rList = inpRF.read() inpRF.close() BFfile = 'blue.txt' inpBF = open(BFfile,"r") bList = inpBF.read() inpBF.close() GFfile = 'green.txt' inpGF = open(GFfile,"r") gList = inpGF.read() inpGF.close() def get_complete_text(event): complete_text = Tbox.get("1.0", END) redList = str(rList).split() blueList = str(bList).split() greenList = str(gList).split() print "redList",redList print "blueList",blueList print "greenList",greenList Tbox.tag_remove("red", "1.0", END) Tbox.tag_remove("blue", "1.0", END) Tbox.tag_remove("green", "1.0", END) ####RED######## redList_regexp = "|".join(redList) for matchobj in re.finditer(redList_regexp, complete_text): start,end = matchobj.span() Tbox.tag_add("red", "1.0 + %d chars" % start,"1.0 + %d chars" % end) Tbox.tag_config("red", foreground="red") ####BLUE####### blueList_regexp = "|".join(blueList) for matchobj in re.finditer(blueList_regexp, complete_text): start,end = matchobj.span() Tbox.tag_add("blue", "1.0 + %d chars" % start,"1.0 + %d chars" % end) Tbox.tag_config("blue", foreground="blue") ####GREEN####### greenList_regexp = "|".join(greenList) for matchobj in re.finditer(greenList_regexp, complete_text): start,end = matchobj.span() Tbox.tag_add("green", "1.0 + %d chars" % start,"1.0 + %d chars" % end) Tbox.tag_config("green", foreground="green") root = Tk() Tbox = Text(root, width=40, height=15, wrap=CHAR, font="Times 14 bold", bg="#dddddd") Tbox.pack() Tbox.bind("", get_complete_text) Tbox.focus() root.mainloop() #### Thanks again, Dave From benjamin.kaplan at case.edu Tue Jan 5 15:13:24 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Tue, 5 Jan 2010 15:13:24 -0500 Subject: Fwd: I would like to install Python on my 64 bit Win 7 In-Reply-To: <4B439B01.50407@gmail.com> References: <4B439B01.50407@gmail.com> Message-ID: On Tue, Jan 5, 2010 at 3:03 PM, Dave McCormick wrote: > > > aung paing Soe wrote: > > ---------- Forwarded message ---------- > From: aung paing Soe > Date: Tue, Jan 5, 2010 at 11:27 AM > Subject: I would like to install Python on my 64 bit Win 7 > To: webmaster at python.org > > > Hello , > ????????? I would like to study about Python Programming . So I want to > install Python . > But my laptop is Window 7 64-bit home basic? . > So please give me a advice how to install Python in my 64 bit computer. > I really want to study python programming . > I am looking forward your reply. > Thank you very much > > Yours, > Beginner > > I am using WIN7 64 bit also. > Go to python.org and look for > http://www.python.org/download/releases/2.5.4/ > python-2.5.4.amd64.msi > > Dave Unless you need a package that requires Python 2.5, it's a good idea to use Python 2.6 instead. 2.5 isn't getting any more bug fixes and 2.6 includes quite a few new features. http://python.org/download/releases/2.6.4/ > > -- > http://mail.python.org/mailman/listinfo/python-list > > From akean at clear.net.nz Tue Jan 5 15:17:11 2010 From: akean at clear.net.nz (akean) Date: Tue, 5 Jan 2010 12:17:11 -0800 (PST) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> Message-ID: On Jan 6, 8:58?am, Phlip wrote: > Hypo Nt: > > def each_with_index(seq): > ? ? index = 0 > ? ? result = [] > > ? ? for item in seq: > ? ? ? result.append([item, index]) > ? ? ? index += 1 > > ? ? return result > > My Pythonic sequencing skills are obviously feeble. Can anything think > of a way to write that in fewer lines? > > -- > ? Phlip > ?http://c2.com/cgi/wiki?MoreliaViridis y = [[seq[i],i] for i in range(len(seq))] gives the same result. The index is accessible without assigning it to another variable. -- Anita From nawijn at gmail.com Tue Jan 5 15:20:58 2010 From: nawijn at gmail.com (Marco Nawijn) Date: Tue, 5 Jan 2010 12:20:58 -0800 (PST) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> Message-ID: On Jan 5, 8:58?pm, Phlip wrote: > Hypo Nt: > > def each_with_index(seq): > ? ? index = 0 > ? ? result = [] > > ? ? for item in seq: > ? ? ? result.append([item, index]) > ? ? ? index += 1 > > ? ? return result > > My Pythonic sequencing skills are obviously feeble. Can anything think > of a way to write that in fewer lines? > > -- > ? Phlip > ?http://c2.com/cgi/wiki?MoreliaViridis You could use the build-in function enumerate inside a list comprehension. >>> seq = range(5) >>> [ (i,s) for i,s in enumerate(seq) ] [(0, 0), (1, 1), (2, 2), (3, 3), (4, 4)] This will reduce the function to a one-liner. Regards, Marco From carsten.haese at gmail.com Tue Jan 5 15:22:51 2010 From: carsten.haese at gmail.com (Carsten Haese) Date: Tue, 05 Jan 2010 15:22:51 -0500 Subject: please help shrink this each_with_index() implementation In-Reply-To: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> Message-ID: Phlip wrote: > Hypo Nt: > > def each_with_index(seq): > index = 0 > result = [] > > for item in seq: > result.append([item, index]) > index += 1 > > return result > > My Pythonic sequencing skills are obviously feeble. Can anything think > of a way to write that in fewer lines? Couldn't you just use the built-in enumerate() to replace the whole thing? -- Carsten Haese http://informixdb.sourceforge.net From jjposner at optimum.net Tue Jan 5 15:34:14 2010 From: jjposner at optimum.net (John Posner) Date: Tue, 05 Jan 2010 15:34:14 -0500 Subject: Dynamic text color References: Message-ID: On Tue, 05 Jan 2010 15:08:04 -0500, Dave McCormick wrote: >> >> It's certainly a mistake to use the expression "str(rList).split()". >> Using str() to convert the list "rList" into a string creates a mess >> that includes square-bracket characters. Did this actually work for you? > It sort of worked. With one color file it seemed fine but after I > posted I added another color file and things fell apart. > Now with the above fixes it works with three colors from three files. > When the list are printed to the shell the list look like this: > redList ['red', 'dog', 'apple', '#'] > blueList ['blue', 'ball', 'berry'] > greenList ['green', 'grass', 'do'] > > But another problem is noticed. It does not matter if the list is built > in code or from a file. > If dog is entered, "do" will be green with the "g" being red. Back to > the drawing board..... It sounds like the program is doing exactly what you TOLD it to do (which might not be what you WANT it to do): 1. In an earlier pass on the text, color the string "dog" red. 2. In a later pass, color the string "do" green. You need to decide what you WANT to happen if one word to be colored is a substring of another word to be colored differently. Or maybe you want to outlaw such situations. After making that decision, you can start to think about how to write the appropriate code. Best, John From phlip2005 at gmail.com Tue Jan 5 15:36:07 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 5 Jan 2010 12:36:07 -0800 (PST) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> Message-ID: > > My Pythonic sequencing skills are obviously feeble. Can anything think > > of a way to write that in fewer lines? Thanks, all! > Couldn't you just use the built-in enumerate() to replace the whole thing? Because that would involve, like, reading an entire Python book just to locate that method? GMAB I'm too busy writing high-end Django via TDD & BDD! C-: -- Phlip http://zeekland.zeroplayer.com/Pigleg_Too/1 From solipsis at pitrou.net Tue Jan 5 16:10:00 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 5 Jan 2010 21:10:00 +0000 (UTC) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> Message-ID: >> Couldn't you just use the built-in enumerate() to replace the whole >> thing? > > Because that would involve, like, reading an entire Python book just to > locate that method? Actually, no. It just involves reading one of the most important pages in the documentation, the page which describes the built-in functions: http://docs.python.org/library/functions.html Don't forget that the Python documentation is rich and structured. And good luck. From g.bogle at auckland.no.spam.ac.nz Tue Jan 5 16:16:01 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Wed, 06 Jan 2010 10:16:01 +1300 Subject: A null program - what is it doing? References: Message-ID: r0g wrote: > Gib Bogle wrote: >> No doubt a dumb question from a noob: >> >> The following program (a cut down version of some test code) uses no >> CPU, and does not terminate: >> >> import sys >> from PyQt4.QtCore import * >> >> if __name__=="__main__": >> app = QCoreApplication(sys.argv) >> sys.exit(app.exec_()) >> >> What is the program doing? What is the correct way to terminate the >> execution? >> >> Thanks in advance for educating me. > > > > I've never used QT but other graphical toolkits I have used all start > their own "main loop" which is a loop that cycles round receiving, > queuing and dispatching "events". You probably need to call the > QCoreApplication's quit method to break out of this e.g. > > app.exit() or something similar, have a look at some complete PyQt4 > examples or google for PyQt4 mainloop. > > > Roger. Thanks. I've realized that QCoreApplication (or QApplication) manages the event loop, and normally when it is executed you are showing a widget of some kind, closing which ends the application. Alt-F4 acts like Ctrl-C to terminate from the keyboard. From tn.pablo at gmail.com Tue Jan 5 16:28:01 2010 From: tn.pablo at gmail.com (Pablo Torres N.) Date: Tue, 5 Jan 2010 13:28:01 -0800 (PST) Subject: Commands for a breakpoint in .pdbrc Message-ID: Hi all, I'd like to save the commands for a breakpoint in a .pdbrc, something like: b 81 commands 1 pp foo.attr1 pp foo.attr2 end b 108 commands 2 pp bar.attr1 pp bar.attr2 end This would automate setting the environment for the debugging session. However, this does not work with 'python -m pdb script.py', because at the line 'commands 1', the pdb prompt starts and asks me for the commands for the first breakpoint, ignoring what I wrote in .pdbrc; further, it raises a NameError after I type 'end' at the pdb prompt, because of 'foo.attr1', 'foo.attr2' and even 'end'. The same happens for the rest of the breakpoints, so I end up with them set but not their commands. What would be the correct way to do this? Is it even possible? Thanks a lot, Pablo Torres N. From deets at nospam.web.de Tue Jan 5 16:39:05 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Tue, 05 Jan 2010 22:39:05 +0100 Subject: embedded python on mac - linking problem In-Reply-To: References: Message-ID: <7qhprpF3qeU1@mid.uni-berlin.de> Krzysztof Kobus schrieb: > Hi, > > I have a problem with linking python module with my application on mac in order to make the module available in "embedded python". > > My python module is contained in j3kmodule.cxx file and module initialization function is exported in j3kmodule.h > > j3kmodule.h: > ------------ > PyMODINIT_FUNC PyInit_j3k(void); > > > j3kmodule.cxx: > -------------- > PyMODINIT_FUNC > PyInit_j3k(void) > { > PyObject *m = NULL; > > if ((m = PyModule_Create(&j3k_module)) == NULL) > return NULL; > > return m; > } > > > Then in my application in KkPython.cxx file I have: > > KkPython.cxx: > ------------- > > #include "j3kmodule.h" > > /* Add a builtin module, before Py_Initialize */ > PyImport_AppendInittab("j3k", PyInit_j3k); > > > I link my application with the module and get following linking error on mac although on open suse linux exactly the same procedure works fine: > > g++ -headerpad_max_install_names -o ../../bin/render.app/Contents/MacOS/render main.o KkPython.o -lkkbase -L/Users/kk/dev/J3K/trunk/j3ksrc/examples/render/../../lib/ -lQtSolutions_PropertyBrowser-2.5 -lpython3.1 /Users/kk/dev/J3K/trunk/j3ksrc/python/modules/build/lib.macosx-10.3-i386-3.1/j3k.so -framework OpenGL -framework AGL > ld: warning in /Users/kk/dev/J3K/trunk/j3ksrc/python/modules/build/lib.macosx-10.3-i386-3.1/j3k.so, file is not of required architecture > Undefined symbols: > "_PyInit_j3k", referenced from: > _PyInit_j3k$non_lazy_ptr in KkPython.o > ld: symbol(s) not found > collect2: ld returned 1 exit status > make: *** [../../bin/render.app/Contents/MacOS/render] Error 1 > > > I appreciate any hints, The missing symbol looks like a C++-symbol - but Python is C. Do you maybe miss the extern "C" declaration. Diez From tosters at gmail.com Tue Jan 5 16:52:18 2010 From: tosters at gmail.com (t0ster) Date: Tue, 5 Jan 2010 13:52:18 -0800 (PST) Subject: Python multiprocessing: Permission denied Message-ID: <1c85295e-11ee-471c-bd2f-420e0f2fc3b6@j4g2000yqe.googlegroups.com> Hi guys, I'm getting an error when trying to execute python program that uses multiprocessing package: File "/usr/local/lib/python2.6/multiprocessing/__init__.py", line 178, in RLock return RLock() File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line 142, in __init__ SemLock.__init__(self, RECURSIVE_MUTEX, 1, 1) File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line 49, in __init__ sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue) OSError: [Errno 13] Permission denied It looks like the user don't have permission to access shared memory. When executing with root privileges it works fine. Is there any solution to run it as normal user(not root)? Python version 2.6.2 , OS is Linux 2.6.18 (CentOS release 5.4) and it's VPS machine. From mackrackit at gmail.com Tue Jan 5 16:54:44 2010 From: mackrackit at gmail.com (Dave McCormick) Date: Tue, 05 Jan 2010 14:54:44 -0700 Subject: Dynamic text color In-Reply-To: References: Message-ID: <4B43B524.9090805@gmail.com> John Posner wrote: > On Tue, 05 Jan 2010 15:08:04 -0500, Dave McCormick > wrote: > > It sounds like the program is doing exactly what you TOLD it to do > (which might not be what you WANT it to do): > > 1. In an earlier pass on the text, color the string "dog" red. > 2. In a later pass, color the string "do" green. > > You need to decide what you WANT to happen if one word to be colored > is a substring of another word to be colored differently. Or maybe you > want to outlaw such situations. After making that decision, you can > start to think about how to write the appropriate code. > > Best, > John Darn thing doing what I told it to do... Guess that means I did something right :) But it is not what I am wanting. I first thought to make it look for a space but that would not work when a single character like "#" is to be colored if there is a "string" of them. Or if all of the characters between quotes are to be colored. I always did like puzzles! Dave From nad at acm.org Tue Jan 5 17:03:45 2010 From: nad at acm.org (Ned Deily) Date: Tue, 05 Jan 2010 14:03:45 -0800 Subject: it gets worse (was: How do you configure IDLE on a Mac...) References: <0d70cb54-3d77-4176-b621-e764ecf613b7@26g2000yqo.googlegroups.com> <6672dad2-26ba-458b-8075-21bac6506179@e37g2000yqn.googlegroups.com> Message-ID: In article <6672dad2-26ba-458b-8075-21bac6506179 at e37g2000yqn.googlegroups.com>, Mensanator wrote: [...] > So, for all practical purposes, the macports install is broken also. > > IDLE simply does not work in an X11 window (you think someone would > have noticed that). The missing preferences is just the beginning. > Apparently NONE of the menu item shortcuts work. > > For example, the Cut, Copy, Paste shortcuts are given as Command-X, > Command-C and Command-V. But that doesn't work in an X11 window, > apperently only in an Aqua Tk (parent application appears as IDLE). > > Of course, I can do Control-X, Control-C and Control-V to do Cut, > Copy and Paste. Don't know if this works for all shortcuts, but > I suppose I could just pick them from the menu (and I can bang > my head against the wall while I'm at it). > > What do you think, suppose I copy the gmpy built with the macports > install over to the directory where the python.org version is? Would > it > import? If that'll work, I can switch back to using the python.org > install and use it's version of IDLE. I certainly won't be needing > distutils once I have a working version of gmpy. Let's go back to your original problem, which, if I understand correctly, was trying to get going with Python 3 and gmpy on OS X 10.6. (Sorry I was away over the holidays and didn't get a chance to respond to your original postings at the time.) I believe the problems you originally encountered with installing gmpy were all due to a couple of problems with building C extension modules on 10.6 when using the current 3.1.1 OS X python.org. Unfortunately, 3.1.1 was released before 10.6 was so there are a couple of important fixes that haven't yet been released for 3.1 (but are in the 2.6.4 installer which was released after 10.6 came out). Fortunately, though, there are simple workarounds for the problems. Keep in mind, though, that, at the moment, the python.org installers for OS X are 32-bit only; that will change in the future but if you do need a 64-bit Python 3 you'll need to stick to other solutions like MacPorts for the time being. First, make sure the gmp library you've installed has 32-bit support. If you installed it using MacPorts, check with the file command: $ file /opt/local/lib/libgmp.dylib /opt/local/lib/libgmp.dylib: Mach-O universal binary with 2 architectures /opt/local/lib/libgmp.dylib (for architecture i386): Mach-O dynamically linked shared library i386 /opt/local/lib/libgmp.dylib (for architecture x86_64): Mach-O 64-bit dynamically linked shared library x86_64 If it doesn't have an i386 variant, reinstall the gmp library from MacPorts: $ sudo port selfupdate # make sure MacPorts is up-to-date $ sudo port clean gmp $ sudo port install gmp +universal # install 32-/64-bit variants Second, you need to install the MacOSX10.4u SDK because the current python.org pythons are built with it. That SDK is included in the Snow Leopard Xcode installer package but it is not installed by default. There should be an Xcode.mpkg somewhere, perhaps on your hard disk if your system came with Snow Leopard factory-installed or perhaps on a restore DVD. If not, it's on the retail Snow Leopard DVD and can be downloaded from the Apple Developer site. After launching the Xcode installer, just select and install the "Mac OS 10.4 Support" package from the Custom Install menu. Third, you need to tell Distutils to use the older gcc-4.0 instead of the gcc-4.2 which is now the default on 10.6. $ cd /path/to/gmpy-1.11rc1 $ export CC=/usr/bin/gcc-4.0 $ /usr/local/bin/python3.1 setup.py install ... $ /usr/local/bin/python3.1 test3/gmpy_test.py Unit tests for gmpy 1.11 on Python 3.1.1 (r311:74543, Aug 24 2009, 18:44:04) [GCC 4.0.1 (Apple Inc. build 5493)] Testing gmpy 1.11 (GMP 4.3.1), default caching (100, 128) ... 1500 tests in 42 items. 1500 passed and 0 failed. -- Ned Deily, nad at acm.org From python at mrabarnett.plus.com Tue Jan 5 17:37:08 2010 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 05 Jan 2010 22:37:08 +0000 Subject: parsing an Excel formula with the re module In-Reply-To: <5ef3575d-eb1d-4a15-90e0-87dd85cdc2e6@35g2000yqa.googlegroups.com> References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> <5ef3575d-eb1d-4a15-90e0-87dd85cdc2e6@35g2000yqa.googlegroups.com> Message-ID: <4B43BF14.7000102@mrabarnett.plus.com> Mensanator wrote: > On Jan 5, 12:35 pm, MRAB wrote: >> vsoler wrote: >>> Hello, >>> I am acessing an Excel file by means of Win 32 COM technology. >>> For a given cell, I am able to read its formula. I want to make a map >>> of how cells reference one another, how different sheets reference one >>> another, how workbooks reference one another, etc. >>> Hence, I need to parse Excel formulas. Can I do it by means only of re >>> (regular expressions)? >>> I know that for simple formulas such as "=3*A7+5" it is indeed >>> possible. What about complex for formulas that include functions, >>> sheet names and possibly other *.xls files? >>> For example "=Book1!A5+8" should be parsed into ["=","Book1", "!", >>> "A5","+","8"] >>> Can anybody help? Any suggestions? >> Do you mean "how" or do you really mean "whether", ie, get a list of the >> other cells that are referred to by a certain cell, for example, >> "=3*A7+5" should give ["A7"] and "=Book1!A5+8" should give ["Book1!A5] > > Ok, although "Book1" would be the default name of a workbook, with > default > worksheets labeled "Sheet1". "Sheet2", etc. > > If I had a worksheet named "Sheety" that wanted to reference a cell on > "Sheetx" > OF THE SAME WORKBOOK, it would be =Sheet2!A7. If the reference was to > a completely > different workbook (say Book1 with worksheets labeled "Sheet1", > "Sheet2") then > the cell might have =[Book1]Sheet1!A7. > > And don't forget the $'s! You may see =[Book1]Sheet1!$A$7. I forgot about the dollars! In that case, the regex is: references = re.findall(r"\b((?:\w+!)?\$?[A-Za-z]+\$?\d+)\b", formula) From ben+python at benfinney.id.au Tue Jan 5 17:39:08 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 06 Jan 2010 09:39:08 +1100 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> Message-ID: <87pr5ow50z.fsf@benfinney.id.au> r0g writes: > Of course I'm now guilty of pedantry too :/ I might have let it slip > had you not started your reply with the word "No", that just p***** me > off. Well, if being told ?no? is going to piss you off, I think you're in for a rough time. > Having said that I find the mental image of you slamming your fist on > the table and shouting it out loud whenever you read something you > disagree with on usenet quite amusing! In return, I find myself quite amused that you would get such an image. To reduce the stress you describe above, you might want to read what is written, rather than inventing fanciful emotion where it wasn't in the message to begin with. -- \ ?One time I went to a drive-in in a cab. The movie cost me | `\ ninety-five dollars.? ?Steven Wright | _o__) | Ben Finney From phlip2005 at gmail.com Tue Jan 5 17:40:49 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 5 Jan 2010 14:40:49 -0800 (PST) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> Message-ID: <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> On Jan 5, 1:10?pm, Antoine Pitrou wrote: > http://docs.python.org/library/functions.html > > Don't forget that the Python documentation is rich and structured. > And good luck. Does it say how to convert a string containing either an integer representation, or something alphabetic, into an integer, or a zero, in like 1 method call? (No except: ?) Nothing personal, but I'm finding the super-hard stuff very facile & tractable, and the easy stuff absurdly hard around here... From steve at REMOVE-THIS-cybersource.com.au Tue Jan 5 18:20:49 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 05 Jan 2010 23:20:49 GMT Subject: Python multiprocessing: Permission denied References: <1c85295e-11ee-471c-bd2f-420e0f2fc3b6@j4g2000yqe.googlegroups.com> Message-ID: <0353b63c$0$1336$c3e8da3@news.astraweb.com> On Tue, 05 Jan 2010 13:52:18 -0800, t0ster wrote: > It looks like the user don't have permission to access shared memory. > When executing with root privileges it works fine. > > Is there any solution to run it as normal user(not root)? Then give the user permission to access shared memory. Why do you expect that Python would be able to over-ride the operating system's security? This problem is no different from saying "It looks like the user doesn't have permission to access this file". -- Steven From steve at REMOVE-THIS-cybersource.com.au Tue Jan 5 18:22:27 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 05 Jan 2010 23:22:27 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> Message-ID: <0353b69e$0$1336$c3e8da3@news.astraweb.com> On Wed, 06 Jan 2010 09:39:08 +1100, Ben Finney wrote: > r0g writes: > >> Of course I'm now guilty of pedantry too :/ I might have let it slip >> had you not started your reply with the word "No", that just p***** me >> off. > > Well, if being told ?no? is going to piss you off, I think you're in for > a rough time. Oh, you're in trouble now! If you think he gets upset at being told no, you should see how upset he gets at being told he's in for a rough time!!! *wink* -- Steven From steve at REMOVE-THIS-cybersource.com.au Tue Jan 5 18:26:26 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 05 Jan 2010 23:26:26 GMT Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> Message-ID: <0353b78e$0$1336$c3e8da3@news.astraweb.com> On Tue, 05 Jan 2010 14:40:49 -0800, Phlip wrote: > On Jan 5, 1:10?pm, Antoine Pitrou wrote: > >> http://docs.python.org/library/functions.html >> >> Don't forget that the Python documentation is rich and structured. And >> good luck. > > Does it say how to convert a string containing either an integer > representation, or something alphabetic, into an integer, or a zero, in > like 1 method call? (No except: ?) If you mean something like this: >>> int('153') 153 then yes it does. But if you mean something like this: >>> some_mysterious_function('one hundred and fifty-three') 153 then no, it doesn't. > Nothing personal, but I'm finding the super-hard stuff very facile & > tractable, and the easy stuff absurdly hard around here... Then perhaps you should work through the tutorial to learn the basics. -- Steven From phlip2005 at gmail.com Tue Jan 5 18:30:09 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 5 Jan 2010 15:30:09 -0800 (PST) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> Message-ID: > > Does it say how to convert a string containing either an integer > > representation, or something alphabetic, into an integer, or a zero, in > > like 1 method call? (No except: ?) > > If you mean something like this: > > >>> int('153') > > 153 The point: int('') or int('something') both throw an error. In general, this is hand-holding, but in specific I don't think the "rich and structured" documentation will cover how to beat a 0 out of it in less than 3 lines. So I will persist in my idiotic questions here! > Then perhaps you should work through the tutorial to learn the basics. They will tell me how to use except: (which is a good example why a program should not use exceptions for its normal control flow if at all possible). Please, please, please save your newbie admonitions for those who qualify! From skwslide at gmail.com Tue Jan 5 18:50:39 2010 From: skwslide at gmail.com (Steven K. Wong) Date: Tue, 5 Jan 2010 15:50:39 -0800 (PST) Subject: subprocess.Popen does not close pipe in an error case Message-ID: <07641018-9581-44af-bb20-8f9b96a44c4f@p8g2000yqb.googlegroups.com> Below, I have a Python script that launches 2 child programs, prog1 and prog2, with prog1's stdout connected to prog2's stdin via a pipe. (It's like executing "prog1 | prog2" in the shell.) If both child programs exit with 0, then the script runs to completion. But if prog2 exits with non-0, prog1 does not exit and the script hangs (i.e. prog1.poll() always returns None) -- unless I uncomment the 2 lines marked by XXX to close prog1.stdout. I was expecting that I don't have to explicitly close prog1.stdout, whether prog2 succeeds or fails. Is the current behavior a bug in the subprocess module or is it expected? Or am I doing something wrong? Thanks. import subprocess import time # prog1: a program that writes lots of data to the pipe cmd = ['zcat', '--force', 'a_large_file'] prog1 = subprocess.Popen(cmd, bufsize=-1, stdout=subprocess.PIPE) # prog2: a program that fails without reading much data from the pipe cmd = ['python', '-c', 'import time; time.sleep(10); asdf'] prog2 = subprocess.Popen(cmd, bufsize=-1, stdin=prog1.stdout, stdout=open('popen.out', 'w')) print 'waiting for a while' retCodeProg2 = prog2.wait() print 'prog2 returns', retCodeProg2 # XXX # if retCodeProg2 != 0: # prog1.stdout.close() while prog1.poll() is None: print 'sleep a bit' time.sleep(1) retCodeProg1 = prog1.poll() print 'prog1 returns', retCodeProg1 From phlip2005 at gmail.com Tue Jan 5 18:51:29 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 5 Jan 2010 15:51:29 -0800 (PST) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: Peng Yu wrote: > Otherwise, could some python expert explain to me why exception is > widely used for error handling in python? Is it because the efficiency > is not the primary goal of python? It's not about efficiency, it's about making assumptions for the programmer about what kind of rigor they need. Why can't int('nonnumeric') return None? Why can't a Django Record.objects.get(pk=-1) return a None? That's what it's for. (A related question - why can't I just go 'if record = method(): use (record)'. Why extra lines just to trap and assign the variable before using it?) There are workarounds that sometimes benefit the code. In the case of collections, like recordsets, you might be better off using for ... all (): Then your controlled block efficiently does not happen if it saw no records. "Efficiently" in terms of programmer complexity - the number and meaning of lines that a programmer must comprehend. And why can't Record.objects.get(pk='nonnumeric') return None? Because, of course, deep inside it calls int(). I can't simplify the calling code, and rely on garbage-in-None-out, because Python decided which simplifications I should avoid with self-righteous indignation. The Samurai Principle (return victorious, or not at all) is very useful, sometimes. But other times it just prematurely depletes your supply of Samurai... -- Phlip http://zeekland.zeroplayer.com/ From skwslide at gmail.com Tue Jan 5 18:53:15 2010 From: skwslide at gmail.com (Steven K. Wong) Date: Tue, 5 Jan 2010 15:53:15 -0800 (PST) Subject: subprocess.Popen does not close pipe in an error case References: <07641018-9581-44af-bb20-8f9b96a44c4f@p8g2000yqb.googlegroups.com> Message-ID: <15549117-c3f3-40ab-85de-c8fb85193d1d@f5g2000yqh.googlegroups.com> BTW, I'm using Python 2.6.2 on Linux. From haggardii at gmail.com Tue Jan 5 19:14:49 2010 From: haggardii at gmail.com (Matt Haggard) Date: Tue, 5 Jan 2010 16:14:49 -0800 (PST) Subject: unittest inconsistent Message-ID: Can anyone tell me why this test fails? http://pastebin.com/f20039b17 This is a minimal example of a much more complex thing I'm trying to do. I'm trying to hijack a function and inspect the args passed to it by another function. The reason the 'Tester' object has no attribute 'arg1' is because "self" still refers to the object made for testA. From mackrackit at gmail.com Tue Jan 5 19:26:38 2010 From: mackrackit at gmail.com (Mackrackit) Date: Tue, 5 Jan 2010 17:26:38 -0700 Subject: please help shrink this each_with_index() implementation In-Reply-To: References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> Message-ID: On Jan 5, 2010, at 4:30 PM, Phlip wrote: >> >> > The point: int('') or int('something') both throw an error. In > general, this is hand-holding, but in specific I don't think the "rich > and structured" documentation will cover how to beat a 0 out of it in > less than 3 lines. So I will persist in my idiotic questions here! >> What does an extra two or three lines of code matter? Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From aioe.org at technicalbloke.com Tue Jan 5 19:58:58 2010 From: aioe.org at technicalbloke.com (r0g) Date: Wed, 06 Jan 2010 00:58:58 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > On Wed, 06 Jan 2010 09:39:08 +1100, Ben Finney wrote: > >> r0g writes: >> >>> Of course I'm now guilty of pedantry too :/ I might have let it slip >>> had you not started your reply with the word "No", that just p***** me >>> off. >> Well, if being told ?no? is going to piss you off, I think you're in for >> a rough time. > > Oh, you're in trouble now! If you think he gets upset at being told no, > you should see how upset he gets at being told he's in for a rough time!!! > > *wink* > > > NO! It's a rude way to start a sentence don't you think? Just because you're correcting someone doesn't mean you have to be combative and try and make them feel small. Unless they've adopted a hostile or wilfully ignorant tone there's no reason to be so brusqe with people. You can be both nice AND terse you know. I can't imagine why I expect good manners on usenet though, AFAICT it's never been like that (well not since I got on it anyway). Roger. From clp2 at rebertia.com Tue Jan 5 20:01:02 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Tue, 5 Jan 2010 17:01:02 -0800 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <50697b2c1001051701v6b635deeh1a02426f10836d24@mail.gmail.com> On Tue, Jan 5, 2010 at 3:51 PM, Phlip wrote: > Peng Yu wrote: >> Otherwise, could some python expert explain to me why exception is >> widely used for error handling in python? Is it because the efficiency >> is not the primary goal of python? > > It's not about efficiency, it's about making assumptions for the > programmer about what kind of rigor they need. > > Why can't int('nonnumeric') return None? Errors should never pass silently. Unless explicitly silenced. -- The Zen of Python (http://www.python.org/dev/peps/pep-0020/) Better to throw an exception and ensure the case is specifically dealt with one way or another than to silently return an error flag result which may only delay the error until later in the program, making it harder to debug. Is it that much of a burden to write and use the small function that does what you want? def int_or_None(string): try: return int(string) except ValueError: return None Heck, you can even write it inline and dispense with the function if you want: try: foo = int(bar) except ValueError: foo = None Quibbling over a mere one more line of code (or writing one short function) seems a bit petty. > (A related question - why can't I just go 'if record = method(): ?use > (record)'. Why extra lines just to trap and assign the variable before > using it?) I believe that's disallowed so as to prevent the subtle bugs seen in C code which result from when someone makes a typo and omits the second "=" in their `if foo == bar():` test. Cheers, Chris -- http://blog.rebertia.com From aioe.org at technicalbloke.com Tue Jan 5 20:03:12 2010 From: aioe.org at technicalbloke.com (r0g) Date: Wed, 06 Jan 2010 01:03:12 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <4b42e40c$1@dnews.tpgi.com.au> <03533377$0$1336$c3e8da3@news.astraweb.com> <4b437282$1@dnews.tpgi.com.au> Message-ID: Lie Ryan wrote: > On 1/6/2010 1:48 AM, r0g wrote: >> Steven D'Aprano wrote: >>> On Tue, 05 Jan 2010 13:06:20 +0000, r0g wrote: >>>> If >>>> that's the case how can you expect it to validate anything at all in >>>> production? >>> >>> The asserts still operate so long as you don't use the -O switch. >>> > checking, since the function relies on the caller obeying the > contract[2] and never calling it with an invalid input. > > DbC uses assertions[1] spuriously, unlike the traditional approach which > is much more conservative when using assertions. > > [1] or explicit language support which is just syntax sugar for assertions > [2] of course, on a debug release, the contract validation code will > still be enforced to catch logic/consistency bugs that causes the violation Thanks for the responses Steven/Dave/Lie, that's some really insightful stuff :) Roger. From phlip2005 at gmail.com Tue Jan 5 20:45:58 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 5 Jan 2010 17:45:58 -0800 (PST) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> On Jan 5, 5:01?pm, Chris Rebert wrote: > > Why can't int('nonnumeric') return None? > > Errors should never pass silently. You are saying I, as the programmer, cannot decide what is an error and what is a pass-thru. The decision is made for me. (Yes yes I can write int_or_None(), etc...) Here's a super easy example: { 42: 'forty two' }.get(41, None) Because I can supply a default, I can decide what is an error and what is . Now the equivalent in a language that does not enjoy this false "Zen": { 42: 'forty two' }[41] # returns None { 42: 'forty two' }.fetch(41, None) # ibid { 42: 'forty two' }.fetch(41) # raises an exception The quicky validation is available if I _request_ it. > Quibbling over a mere one more line of code (or writing one short > function) seems a bit petty. Because that "Zen of Python" is an empty sophistry that forces me to add a "mere one more line of code" over and over again... > > (A related question - why can't I just go 'if record = method(): ?use > > (record)'. Why extra lines just to trap and assign the variable before > > using it?) > > I believe that's disallowed so as to prevent the subtle bugs seen in C > code which result from when someone makes a typo and omits the second > "=" in their `if foo == bar():` test. Don't prevent me from using a technique just because others had trouble with it. And if bar() == foo is the superior technique anyway, because the == happens in chronological and lexical order after the bar() call. From phlip2005 at gmail.com Tue Jan 5 20:50:00 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 5 Jan 2010 17:50:00 -0800 (PST) Subject: unittest inconsistent References: Message-ID: On Jan 5, 4:14?pm, Matt Haggard wrote: > Can anyone tell me why this test fails? > > http://pastebin.com/f20039b17 > > This is a minimal example of a much more complex thing I'm trying to > do. ?I'm trying to hijack a function and inspect the args passed to it > by another function. > > The reason the 'Tester' object has no attribute 'arg1' is because > "self" still refers to the object made for testA. I hope someone else can spot the low-level reason... ...but why aren't you using http://pypi.python.org/pypi/mock/ ? Look up its patch_object facility... From fetchinson at googlemail.com Tue Jan 5 20:50:30 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Tue, 5 Jan 2010 17:50:30 -0800 Subject: twenty years ago Guido created Python In-Reply-To: <4B438FB6.9050003@holdenweb.com> References: <51564fb3-a367-4f11-b97f-5518deedbe48@a21g2000yqc.googlegroups.com> <4B438FB6.9050003@holdenweb.com> Message-ID: >>> Python is a truly awesome programming language. Not only is Guido a >>> genius language designer, but he is also a great project leader. What >>> an accomplishment. Congratulations to everybody who has contributed >>> to Python in the last two decades! >> >> The more languages you learn before getting to Smalltalk, the more >> awesome Smalltalk will be for you. >> > After implementing SmallTalk I more or less gave up OO programming for > ten years, But why? What was so frightening about the OO model of SmallTalk? Cheers, Daniel > resuming it only after I met Python. SmallTalk didn't seem > that awesome to me. > > Though for its time it was an incredible system, its insistence on a > SmallTalk-only VM environment seemed a little solipsistic. -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From phlip2005 at gmail.com Tue Jan 5 20:56:48 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 5 Jan 2010 17:56:48 -0800 (PST) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <09b70061-c411-45a9-8546-c9113c127ced@p8g2000yqb.googlegroups.com> > Errors should never pass silently. > Unless explicitly silenced. > -- The Zen of Python (http://www.python.org/dev/peps/pep-0020/) "The person who says it cannot be done should never interrupt the person doing it" From andre.roberge at gmail.com Tue Jan 5 20:57:53 2010 From: andre.roberge at gmail.com (=?ISO-8859-1?Q?Andr=E9?=) Date: Tue, 5 Jan 2010 17:57:53 -0800 (PST) Subject: unittest inconsistent References: Message-ID: <3dede91a-ab3c-42c0-af45-adde220ededd@z2g2000vbf.googlegroups.com> On Jan 5, 8:14?pm, Matt Haggard wrote: > Can anyone tell me why this test fails? > > http://pastebin.com/f20039b17 > > This is a minimal example of a much more complex thing I'm trying to > do. ?I'm trying to hijack a function and inspect the args passed to it > by another function. > > The reason the 'Tester' object has no attribute 'arg1' is because > "self" still refers to the object made for testA. Quick answer: change faketest.py as follows: #-------------------------------------------------- # faketest.py #-------------------------------------------------- #from importme import render import importme def run(somearg): return importme.render(somearg) ========= A long answer, with explanation, will cost you twice as much ;-) (but will have to wait) Andr? From steve at holdenweb.com Tue Jan 5 21:06:01 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 05 Jan 2010 21:06:01 -0500 Subject: parsing an Excel formula with the re module In-Reply-To: <4B4397B9.7000804@tim.thechases.com> References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> <4B4397B9.7000804@tim.thechases.com> Message-ID: Tim Chase wrote: > vsoler wrote: >> Hence, I need to parse Excel formulas. Can I do it by means only of re >> (regular expressions)? >> >> I know that for simple formulas such as "=3*A7+5" it is indeed >> possible. What about complex for formulas that include functions, >> sheet names and possibly other *.xls files? > > Where things start getting ugly is when you have nested function calls, > such as > > =if(Sum(A1:A25)>42,Min(B1:B25), if(Sum(C1:C25)>3.14, > (Min(C1:C25)+3)*18,Max(B1:B25))) > > Regular expressions don't do well with nested parens (especially > arbitrarily-nesting-depth such as are possible), so I'd suggest going > for a full-blown parsing solution like pyparsing. > > If you have fair control over what can be contained in the formulas and > you know they won't contain nested parens/functions, you might be able > to formulate some sort of "kinda, sorta, maybe parses some forms of > formulas" regexp. > And don't forget about named ranges, which can reference cells without using anything but a plain identifier ... regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From sridharr at activestate.com Tue Jan 5 21:25:47 2010 From: sridharr at activestate.com (Sridhar Ratnakumar) Date: Tue, 05 Jan 2010 18:25:47 -0800 Subject: Fwd: I would like to install Python on my 64 bit Win 7 In-Reply-To: References: Message-ID: <4B43F4AB.6070308@activestate.com> On 1/5/2010 11:31 AM, aung paing Soe wrote: > > Hello , > I would like to study about Python Programming . So I want to > install Python . > But my laptop is Window 7 64-bit home basic . > So please give me a advice how to install Python in my 64 bit computer. > I really want to study python programming . > I am looking forward your reply. > Thank you very much Even though your laptop runs a 64-bit operating system, you do not have to install 64-bit Python build. You may also install a 32-bit build. (Some Python packages may not even build on 64-bit Python) You can install Python from one of the two places: 1. http://python.org/download/releases/2.6.4/ (AMD64 == 64-bit) 2. http://www.activestate.com/activepython/ (Includes PyWin32, a package manager and extra documentation and tutorials). -srid From solipsis at pitrou.net Tue Jan 5 21:28:08 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 6 Jan 2010 02:28:08 +0000 (UTC) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> Message-ID: > The point: int('') or int('something') both throw an error. In general, > this is hand-holding, but in specific I don't think the "rich and > structured" documentation will cover how to beat a 0 out of it in less > than 3 lines. Because it's a bad idea to do so and Python doesn't encourage such sloppy behaviour. If you like it, though, you might prefer PHP. By the way: error reporting is definitely *not* hand-holding. It is simply not good to let errors pass silently by default. Again, if you think the contrary, PHP is your friend ;-) From aahz at pythoncraft.com Tue Jan 5 21:38:11 2010 From: aahz at pythoncraft.com (Aahz) Date: 5 Jan 2010 18:38:11 -0800 Subject: Minor bug in multiprocessing? References: Message-ID: [p&e] In article , Frank Millman wrote: > >Is this worth reporting, if it has not been reported already? Defiitely report it. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From ethan at stoneleaf.us Tue Jan 5 21:41:40 2010 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 05 Jan 2010 18:41:40 -0800 Subject: Object Relational Mappers are evil (a meditation) In-Reply-To: <85hbr9ergw.fsf@agentultra.com> References: <06532563-3f2b-4cca-802f-34cb326e6d9f@v20g2000vbs.googlegroups.com> <16a44521-1f07-454a-ace7-9ba5d5db4b9c@y28g2000prd.googlegroups.com> <50f98a4c0910060810k3a4fa910vabc5bf51381d00f9@mail.gmail.com> <00ac7cf3$0$15654$c3e8da3@news.astraweb.com> <85hbrrxaf9.fsf@agentultra.com> <85d42eyag8.fsf@agentultra.com> <7osvd9F3r79lcU1@mid.individual.net> <858wd2xmi4.fsf@agentultra.com> <4b2c77f1$1@dnews.tpgi.com.au> <854onkxonm.fsf@agentultra.com> <00b98caf$0$15623$c3e8da3@news.astraweb.com> <85zl59d6s8.fsf@agentultra.com> <00bc18b1$0$15637$c3e8da3@news.astraweb.com> <85hbr9ergw.fsf@agentultra.com> Message-ID: <4B43F864.8080009@stoneleaf.us> J Kenneth King wrote: > > In many contexts I'm sure there is reason to use Perl instead of Python > just as there are situations where C is more appropriate than either. > > However, the mark of a poor programmer in my line of reasoning is one > who cannot recognize such distinctions. > > One must be aware of the benefits and short-comings of their tools. If > your tools influence the way you think then you are being ignorant of > this principle. And I would suggest that makes you a poor programmer. Perhaps "influence the way you think" is not the right way to phrase it... how about "be the tool" ;) We have all seen the struggles that newcomers to a language go through as they try (or don't try) to adjust their thinking to the tool at hand -- programming Java, BASIC, FORTRAN, or xyz in Python. Even now Phlip is raging against exceptions and the very Zen of Python. Converting FoxPro to Python is an interesting excercise for me -- version 6 at least doesn't have many of the cool things that Python does, and consequently thinking in Python while writing FoxPro (when I have to) is extremely frustrating; going the other way is a bit of a challenge also, although much more rewarding. For a more concrete example, take sail-boats and speed-boats: you get used to the speed boat, it's quick handling and sharp turns... then you take a sail boat out for a cruise -- if you don't adjust your thinking from speed to sail, you could very well end up in the rocks. To sum up: I agree with your "poor programmer" line of reasoning in the second paragraph above, but not with the "tools influencing the way we think" line of reasoning -- while I am cognizant of Python's shortcomings, I am very much enjoying the changes in my thinking the more I use it. ~Ethan~ From clp2 at rebertia.com Tue Jan 5 21:48:55 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Tue, 5 Jan 2010 18:48:55 -0800 Subject: Exception as the primary error handling mechanism? In-Reply-To: <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> Message-ID: <50697b2c1001051848ye0793eep953a890ca4973a20@mail.gmail.com> On Tue, Jan 5, 2010 at 5:45 PM, Phlip wrote: > On Jan 5, 5:01?pm, Chris Rebert wrote: >> > Why can't int('nonnumeric') return None? >> >> Errors should never pass silently. > > You are saying I, as the programmer, cannot decide what is an error > and what is a pass-thru. The decision is made for me. (Yes yes I can > write int_or_None(), etc...) No, you can certainly decide yourself. You merely need be explicit about it; you're leaving out the other tandem half of the couplet: "*Unless* explicitly silenced." Throwing exceptions is only the default because it tends to lead to more reliable code in that you're forced to deal with the error condition by either catching an exception or preventing one from being thrown (by, say, using a different method in the API that just returns a default value instead). As an aside, I would guess the built-in types don't have default value parameters for conversions partly because it'd be a bit less elegant to implement; since None is a commonly used default value, they'd have to use a different sentinel as said parameter's default value to indicate that the caller wanted an exception raised, and having a hierarchy of nil values in the core language detracts from the language's elegance. At any rate, if you like dict.get(), I don't see why, say, my_int('nonnumeric', None) should be such a problem. > Here's a super easy example: > > ?{ 42: 'forty two' }.get(41, None) > > Because I can supply a default, I can decide what is an error and what > is . Exactly, that's explicitly silencing the error; no one ever said an `except` clause was the only silencing mechanism. You're making your intention clear by using .get() and passing in a desired default. I agree this is a fine thing. > Now the equivalent in a language that does not enjoy this false "Zen": > > ?{ 42: 'forty two' }[41] ?# returns None > ?{ 42: 'forty two' }.fetch(41, None) ?# ibid > ?{ 42: 'forty two' }.fetch(41) ?# raises an exception > > The quicky validation is available if I _request_ it. Seems like rather "fast-and-loose" programming if you don't care about validation enough of the time to merit it being default. If your programming is indeed so fast-and-loose, I refer you to the recent comment about PHP (and other less "exception-happy" languages). Anyway, I do totally agree that you should, if feasible, be provided an easy way to designate a common error-handling strategy (for example, in this case, by using a default value via .fetch()). However, go too loose on error handling and exceptions and one ends up with something like JavaScript with its infamous `undefined` value which can make debugging a nightmare (whoever came up with the idea of JS's `undefined` should be slapped upside the head). Cheers, Chris -- http://blog.rebertia.com From mensanator at aol.com Tue Jan 5 21:51:22 2010 From: mensanator at aol.com (Mensanator) Date: Tue, 5 Jan 2010 18:51:22 -0800 (PST) Subject: it gets worse (was: How do you configure IDLE on a Mac...) References: <0d70cb54-3d77-4176-b621-e764ecf613b7@26g2000yqo.googlegroups.com> <6672dad2-26ba-458b-8075-21bac6506179@e37g2000yqn.googlegroups.com> Message-ID: On Jan 5, 4:03?pm, Ned Deily wrote: > In article > <6672dad2-26ba-458b-8075-21bac6506... at e37g2000yqn.googlegroups.com>,?Mensanator wrote: > > [...] > > > > > > > So, for all practical purposes, the macports install is broken also. > > > IDLE simply does not work in an X11 window (you think someone would > > have noticed that). The missing preferences is just the beginning. > > Apparently NONE of the menu item shortcuts work. > > > For example, the Cut, Copy, Paste shortcuts are given as Command-X, > > Command-C and Command-V. But that doesn't work in an X11 window, > > apperently only in an Aqua Tk (parent application appears as IDLE). > > > Of course, I can do Control-X, Control-C and Control-V to do Cut, > > Copy and Paste. Don't know if this works for all shortcuts, but > > I suppose I could just pick them from the menu (and I can bang > > my head against the wall while I'm at it). > > > What do you think, suppose I copy the gmpy built with the macports > > install over to the directory where the python.org version is? Would > > it > > import? If that'll work, I can switch back to using the python.org > > install and use it's version of IDLE. I certainly won't be needing > > distutils once I have a working version of gmpy. > > Let's go back to your original problem, which, if I understand > correctly, was trying to get going with Python 3 and gmpy on OS X 10.6. ? Right. > (Sorry I was away over the holidays and didn't get a chance to respond > to your original postings at the time.) ? Hey, no problem. I bought this dingus with the idea I would spend my the holiday time to get it to work. Boy, was I right. > I believe the problems you > originally encountered with installing gmpy were all due to a couple of > problems with building C extension modules on 10.6 when using the > current 3.1.1 OS X python.org. ? Yeah, I used the Mac disk image for Python 3.1. > Unfortunately, 3.1.1 was released before > 10.6 was so there are a couple of important fixes that haven't yet been > released for 3.1 (but are in the 2.6.4 installer which was released > after 10.6 came out). ?Fortunately, though, there are simple workarounds > for the problems. ?Keep in mind, though, that, at the moment, the > python.org installers for OS X are 32-bit only; I just checked, I was told to check sys.maxint but that doesn't exist, I assume it's now sys.maxsize. On the python.org disk image, that returns >>> hex(sys.maxsize) '0x7fffffff' looks like 32 bits. > that will change in the > future but if you do need a 64-bit Python 3 you'll need to stick to > other solutions like MacPorts for the time being. The macports install of 3.1 gives: >>> hex(sys.maxsize) '07x7fffffffffffffff' so the macports must be 64 bits. > > First, make sure the gmp library you've installed has 32-bit support. ? Uh, why would I want that? If it comes down to a choice between IDLE and 64 bits, I'll live without IDLE. > If you installed it using MacPorts, check with the file command: > > $ file /opt/local/lib/libgmp.dylib > /opt/local/lib/libgmp.dylib: Mach-O universal binary with 2 architectures > /opt/local/lib/libgmp.dylib (for architecture i386): ?Mach-O dynamically > linked shared library i386 > /opt/local/lib/libgmp.dylib (for architecture x86_64): ? Mach-O 64-bit > dynamically linked shared library x86_64 I get /opt/local/lib/libgmpdylib: Mach-0 64-bit dynamically linked shared library x86_64 > > If it doesn't have an i386 variant, reinstall the gmp library from > MacPorts: But I only need that if I want to run the 32 bit version of Python from python.org. If I'm willing to use the 64 bit version from macports, I don't care, right? > > $ sudo port selfupdate ? ?# make sure MacPorts is up-to-date > $ sudo port clean gmp > $ sudo port install gmp +universal ? ? # install 32-/64-bit variants > > Second, you need to install the MacOSX10.4u SDK because the current > python.org pythons are built with it. Then I'll just not use the download from python.org. Is there a way to uninstall that disk image? >?That SDK is included in the Snow > Leopard Xcode installer package but it is not installed by default. ? > There should be an Xcode.mpkg somewhere, perhaps on your hard disk if > your system came with Snow Leopard factory-installed or perhaps on a > restore DVD. ?If not, it's on the retail Snow Leopard DVD and can be > downloaded from the Apple Developer site. ?After launching the Xcode > installer, just select and install the "Mac OS 10.4 Support" package > from the Custom Install menu. > > Third, you need to tell Distutils to use the older gcc-4.0 instead of > the gcc-4.2 which is now the default on 10.6. > > $ cd /path/to/gmpy-1.11rc1 > $ export CC=/usr/bin/gcc-4.0 > $ /usr/local/bin/python3.1 setup.py install > ... > $ /usr/local/bin/python3.1 test3/gmpy_test.py > Unit tests for gmpy 1.11 > ? ? on Python 3.1.1 (r311:74543, Aug 24 2009, 18:44:04) > [GCC 4.0.1 (Apple Inc. build 5493)] > Testing gmpy 1.11 (GMP 4.3.1), default caching (100, 128) > ... > 1500 tests in 42 items. > 1500 passed and 0 failed. It would appear that all the above is unnecessary, as I have 64 bit Python from macports working with the 64 bit version of gmpy compiled by macports. Not only does it pass the unit test, but since it's 64 bits, I can reach beyond the "outrageous exponent error" that I've seen in 32 bit Windows versions. I'm told that gmp can't do more than 48 billion decimal digits, so I now hit that restriction before I reach the next level of "outrageous exponent". I just won't be able to properly run IDLE, but as I said, that's expendable. Anyway, thanks for pointing that out, didn't realize the python.org disk image was 32 bits. There was no point in downloading it at all. As I pointed out in the thread "the need for 64 bits", once I've seen Paris, there's no going back. > > -- > ?Ned Deily, > ?n... at acm.org From python at mrabarnett.plus.com Tue Jan 5 21:59:50 2010 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 06 Jan 2010 02:59:50 +0000 Subject: please help shrink this each_with_index() implementation In-Reply-To: References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> Message-ID: <4B43FCA6.8040101@mrabarnett.plus.com> Antoine Pitrou wrote: >> The point: int('') or int('something') both throw an error. In general, >> this is hand-holding, but in specific I don't think the "rich and >> structured" documentation will cover how to beat a 0 out of it in less >> than 3 lines. > > Because it's a bad idea to do so and Python doesn't encourage such sloppy > behaviour. If you like it, though, you might prefer PHP. > > By the way: error reporting is definitely *not* hand-holding. It is > simply not good to let errors pass silently by default. Again, if you > think the contrary, PHP is your friend ;-) > Someone wrote about a port of a Perl script to Python. The script would parse the output from a program it would call, but the Python script sometimes raised a ValueError when it tried to convert a certain field to a number. It turned out that sometimes the field would contain "ERROR" instead of a number. The Perl script had been silently 'converting' any such "ERROR" to 0! From wuwei23 at gmail.com Tue Jan 5 22:46:01 2010 From: wuwei23 at gmail.com (alex23) Date: Tue, 5 Jan 2010 19:46:01 -0800 (PST) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> Message-ID: <1715bea7-1ff1-4294-9dad-672873e14530@v25g2000yqk.googlegroups.com> Phlip wrote: > They will tell me how to use except: (which is a good example why a > program should not use exceptions for its normal control flow if at > all possible). Really? Magic functions that coerce and eat errors are a better coding technique than exceptions and explicit handling? What kool-aid have you been drinking? From adityashukla1983 at gmail.com Tue Jan 5 23:26:39 2010 From: adityashukla1983 at gmail.com (aditya shukla) Date: Tue, 5 Jan 2010 22:26:39 -0600 Subject: Do I have to use threads? Message-ID: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> Hello people, I have 5 directories corresponding 5 different urls .I want to download images from those urls and place them in the respective directories.I have to extract the contents and download them simultaneously.I can extract the contents and do then one by one. My questions is for doing it simultaneously do I have to use threads? Please point me in the right direction. Thanks Aditya -------------- next part -------------- An HTML attachment was scrubbed... URL: From philip at semanchuk.com Tue Jan 5 23:36:19 2010 From: philip at semanchuk.com (Philip Semanchuk) Date: Tue, 5 Jan 2010 23:36:19 -0500 Subject: Do I have to use threads? In-Reply-To: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> Message-ID: On Jan 5, 2010, at 11:26 PM, aditya shukla wrote: > Hello people, > > I have 5 directories corresponding 5 different urls .I want to > download > images from those urls and place them in the respective > directories.I have > to extract the contents and download them simultaneously.I can > extract the > contents and do then one by one. My questions is for doing it > simultaneously > do I have to use threads? No. You could spawn 5 copies of wget (or curl or a Python program that you've written). Whether or not that will perform better or be easier to code, debug and maintain depends on the other aspects of your program(s). bye Philip From steven at REMOVE.THIS.cybersource.com.au Tue Jan 5 23:48:30 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 06 Jan 2010 04:48:30 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: On Wed, 06 Jan 2010 00:58:58 +0000, r0g wrote: > Steven D'Aprano wrote: >> On Wed, 06 Jan 2010 09:39:08 +1100, Ben Finney wrote: >> >>> r0g writes: >>> >>>> Of course I'm now guilty of pedantry too :/ I might have let it slip >>>> had you not started your reply with the word "No", that just p***** >>>> me off. >>> Well, if being told ?no? is going to piss you off, I think you're in >>> for a rough time. >> >> Oh, you're in trouble now! If you think he gets upset at being told no, >> you should see how upset he gets at being told he's in for a rough >> time!!! >> >> *wink* >> >> >> >> > > NO! It's a rude way to start a sentence don't you think? No. I'm not Japanese, I don't feel any social prohibition at saying No in this context. I'm also happy to start a sentence with "You're wrong", and occasionally I give in to temptation to start it with "Are you on crack?". > Just because > you're correcting someone doesn't mean you have to be combative and try > and make them feel small. Unless they've adopted a hostile or wilfully > ignorant tone there's no reason to be so brusqe with people. You can be > both nice AND terse you know. Now I feel hurt that you're publicly rebuking me and making me feel as if I've done something wrong by trying to lighten the mood with a small joke... Nah, just kidding. -- Steven From steven at REMOVE.THIS.cybersource.com.au Tue Jan 5 23:49:52 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 06 Jan 2010 04:49:52 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: On Tue, 05 Jan 2010 15:51:29 -0800, Phlip wrote: > Why can't int('nonnumeric') return None? It could do that, but it shouldn't, because returning magic values instead of raising exceptions is usually a bad, bad idea. > (A related question - why can't I just go 'if record = method(): use > (record)'. Why extra lines just to trap and assign the variable before > using it?) Because that idiom is responsible for probably the most common error in C of all, at least one of the most common errors. Thank goodness Python forbids such a dangerous construct. -- Steven From steven at REMOVE.THIS.cybersource.com.au Tue Jan 5 23:52:12 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 06 Jan 2010 04:52:12 GMT Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> Message-ID: On Tue, 05 Jan 2010 15:30:09 -0800, Phlip wrote: >> > Does it say how to convert a string containing either an integer >> > representation, or something alphabetic, into an integer, or a zero, >> > in like 1 method call? (No except: ?) >> >> If you mean something like this: >> >> >>> int('153') >> >> 153 > > The point: int('') or int('something') both throw an error. Well, that's a bug. Obviously they should return 42. Or is that 23? I forget. But seriously... of course they do. What did you expect them to do? Any Python function will raise an exception if you pass invalid data to it. So the solutions are, don't pass invalid data, or wrap the function in a try block. If you don't want to use try, then make sure that your data is good. > In general, > this is hand-holding, but in specific I don't think the "rich and > structured" documentation will cover how to beat a 0 out of it in less > than 3 lines. So I will persist in my idiotic questions here! > >> Then perhaps you should work through the tutorial to learn the basics. > > They will tell me how to use except: (which is a good example why a > program should not use exceptions for its normal control flow if at all > possible). Huh? So because YOU don't know how to use except, programs shouldn't use exceptions for flow control? I reject that, and I will continue using exceptions for flow control when I think it is appropriate. > Please, please, please save your newbie admonitions for those who > qualify! Oh please, get off your high horse. You're asking newbie questions, no matter how many years of using Python you may or may not have, your experience is obviously low. Stick around and you might learn something, but if you bite every time somebody tries to teach you, you'll soon run out of people willing to help you. -- Steven From chris.gonnerman at newcenturycomputers.net Wed Jan 6 00:08:42 2010 From: chris.gonnerman at newcenturycomputers.net (Chris Gonnerman) Date: Tue, 05 Jan 2010 23:08:42 -0600 Subject: [Python] Re: Printing plain text with exact positioning on Windows In-Reply-To: <263f9c76-17fb-4398-b65c-5b1b0007682b@j5g2000yqm.googlegroups.com> References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> <263f9c76-17fb-4398-b65c-5b1b0007682b@j5g2000yqm.googlegroups.com> Message-ID: <4B441ADA.9090301@newcenturycomputers.net> KvS wrote: > Ok, actually I quite like being able to print straightforward through > your code, i.e. without any extra modules installed. I understand that > sending text to the printer is in principle as simple as > > dc.TextOut(scale_factor * 72, > -1 * scale_factor * 72, > "Testing...") > > I didn't see you do anything with adjusting margins in the code. Does > that mean that if I would e.g. do > > dc.TextOut(0, > 0, > "Testing...") > > the printout would appear in the upper left corner of the paper, as > close to the edges as the printer is capable of? (Sorry, but I only > have Ubuntu available at the moment, no Windows). > Actually, as I understand it (and I'll admit my understanding is a bit flawed in some areas), when you tell MSWinPrint.py to print at (0, 0), you are telling it to print at the actual paper edge; if that's outside the printable area (and it probably is for most printers), your printing will be cut off. Best thing to do is to try it. From chris.gonnerman at newcenturycomputers.net Wed Jan 6 00:10:04 2010 From: chris.gonnerman at newcenturycomputers.net (Chris Gonnerman) Date: Tue, 05 Jan 2010 23:10:04 -0600 Subject: [Python] Re: Printing plain text with exact positioning on Windows In-Reply-To: References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> Message-ID: <4B441B2C.9000702@newcenturycomputers.net> KvS wrote: > Sorry, one more. I completely forgot it's not exactly plain text, but > occasionally also a limited number of non-ASCII characters (accents in > names etc.). Would this be possible through your method? If Windows can print it, then MSWinPrint.py should be able to also. But I haven't tested that extensively. YMMV. From steven at REMOVE.THIS.cybersource.com.au Wed Jan 6 00:10:28 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 06 Jan 2010 05:10:28 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> Message-ID: On Tue, 05 Jan 2010 17:45:58 -0800, Phlip wrote: > On Jan 5, 5:01?pm, Chris Rebert wrote: > >> > Why can't int('nonnumeric') return None? >> >> Errors should never pass silently. > > You are saying I, as the programmer, cannot decide what is an error and > what is a pass-thru. The decision is made for me. Every function decides for you what is an error and what isn't. >>> max() Traceback (most recent call last): File "", line 1, in TypeError: max expected 1 arguments, got 0 But I wanted it to return (sys.maxint - 7). How DARE the creator of the language decide that it should be an error instead of returning the arbitrary result I choose! Not. > (Yes yes I can write int_or_None(), etc...) That's right. As a programmer, your job is to program. If a language doesn't provide a function you want, write your own using the primitives available to you. > Here's a super easy example: > > { 42: 'forty two' }.get(41, None) > > Because I can supply a default, I can decide what is an error and what > is . You can ALWAYS decide what is an error. d = {42: 'forty two'} try: d[41] except KeyError: print "All is good, no problems, dict does not contain 41" except: print "WARNING WARNING WARNING!!!" print "FATAL ERROR: dict contains 41!!!" sys.exit() You do so by programming. As a programmer, that's your job. > Now the equivalent in a language that does not enjoy this false "Zen": > > { 42: 'forty two' }[41] # returns None Suppose you have a dict d supplied from somewhere else. You don't know what's in it. You do this: d[41] and you get a result None. Does this mean that the dict looks like this? d = {} or like this? d = {41: None} > { 42: 'forty two' }.fetch(41, None) # ibid In Python, "fetch" is spelled "get". > { 42: 'forty two' }.fetch(41) # raises an exception In Python, that would be spelled {42: 'forty two'}[41] > The quicky validation is available if I _request_ it. When you write d[41] you are requesting it. If you don't want it, use get instead. >> Quibbling over a mere one more line of code (or writing one short >> function) seems a bit petty. > > Because that "Zen of Python" is an empty sophistry that forces me to add > a "mere one more line of code" over and over again... If you're writing that one line of code over and over again, that's a good sign that you're doing it wrong and should rethink your strategy. >> > (A related question - why can't I just go 'if record = method(): ?use >> > (record)'. Why extra lines just to trap and assign the variable >> > before using it?) >> >> I believe that's disallowed so as to prevent the subtle bugs seen in C >> code which result from when someone makes a typo and omits the second >> "=" in their `if foo == bar():` test. > > Don't prevent me from using a technique just because others had trouble > with it. Any language allows and prevents certain techniques, simply by the very nature of the language. Every language has it's own syntax: you can't write weakly-typed stack-based concatenative code (like Forth) in Java, or strongly-typed dynamic object-oriented Python code in Pascal. Every language forces the programmer to use some features and avoid others. > And if bar() == foo is the superior technique anyway, because the == > happens in chronological and lexical order after the bar() call. That makes no sense. -- Steven From rodrick.brown at gmail.com Wed Jan 6 00:40:39 2010 From: rodrick.brown at gmail.com (Rodrick Brown) Date: Wed, 6 Jan 2010 00:40:39 -0500 Subject: Do I have to use threads? In-Reply-To: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> Message-ID: On Tue, Jan 5, 2010 at 11:26 PM, aditya shukla wrote: > Hello people, > > I have 5 directories corresponding 5 different urls .I want to download > images from those urls and place them in the respective directories.I have > to extract the contents and download them simultaneously.I can extract the > contents and do then one by one. My questions is for doing it simultaneously > do I have to use threads? > > Please point me in the right direction. > > Threads in python are very easy to work with but not very efficient and for most cases slower than running multiple processes. Look at using multiple processes instead of going with threads performance will be much better. > Thanks > > Aditya > > -- > http://mail.python.org/mailman/listinfo/python-list > > -- [ Rodrick R. Brown ] http://www.rodrickbrown.com http://www.linkedin.com/in/rodrickbrown -------------- next part -------------- An HTML attachment was scrubbed... URL: From Brian.Mingus at Colorado.EDU Wed Jan 6 00:45:14 2010 From: Brian.Mingus at Colorado.EDU (Brian J Mingus) Date: Tue, 5 Jan 2010 22:45:14 -0700 Subject: Do I have to use threads? In-Reply-To: References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> Message-ID: <9839a05c1001052145j663f9f73nfac6b3cdeec411fe@mail.gmail.com> On Tue, Jan 5, 2010 at 9:36 PM, Philip Semanchuk wrote: > > On Jan 5, 2010, at 11:26 PM, aditya shukla wrote: > > Hello people, >> >> I have 5 directories corresponding 5 different urls .I want to download >> images from those urls and place them in the respective directories.I have >> to extract the contents and download them simultaneously.I can extract the >> contents and do then one by one. My questions is for doing it >> simultaneously >> do I have to use threads? >> > > No. You could spawn 5 copies of wget (or curl or a Python program that > you've written). Whether or not that will perform better or be easier to > code, debug and maintain depends on the other aspects of your program(s). > > bye > Philip Obviously, spawning 5 copies of wget is equivalent to starting 5 threads. The answer is 'yes'. -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank at chagford.com Wed Jan 6 00:57:08 2010 From: frank at chagford.com (Frank Millman) Date: Wed, 6 Jan 2010 07:57:08 +0200 Subject: Minor bug in multiprocessing? References: Message-ID: "Aahz" wrote: > Frank Millman wrote: >> >>Is this worth reporting, if it has not been reported already? > > Defiitely report it. Thanks, Aahz. I took the lack of responses to indicate that there was no reason *not* to report it, so I reported it on 24th December (issue 7571), and it was fixed on the same day (r77038). Frank From victorsubervi at gmail.com Wed Jan 6 01:21:47 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 6 Jan 2010 01:21:47 -0500 Subject: chown'ing by script Message-ID: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> Hi; I have a script that is called via the web. This script writes another script that is also called by the web, which in turn needs to have execution privileges. The problem is that the programmatically created file is owned by apache.apache and thus doesn't have execution privileges. I've tried os.chown(...) but this throws an OSError. I understand that chown'ing programmatically opens a big security hole. However, setting the gid to give apache execution privileges isn't any better. What do you suggest? TIA, beno -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From adityashukla1983 at gmail.com Wed Jan 6 01:21:58 2010 From: adityashukla1983 at gmail.com (aditya shukla) Date: Wed, 6 Jan 2010 00:21:58 -0600 Subject: Do I have to use threads? In-Reply-To: References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> Message-ID: <73045cca1001052221r18a17b3by28b650039d720fec@mail.gmail.com> Thanks.i will look into multiprocessing. Aditya -------------- next part -------------- An HTML attachment was scrubbed... URL: From cs at zip.com.au Wed Jan 6 01:29:10 2010 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 6 Jan 2010 17:29:10 +1100 Subject: chown'ing by script In-Reply-To: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> References: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> Message-ID: <20100106062910.GA18453@cskk.homeip.net> On 06Jan2010 01:21, Victor Subervi wrote: | I have a script that is called via the web. This script writes another | script that is also called by the web, which in turn needs to have execution | privileges. The problem is that the programmatically created file is owned | by apache.apache and thus doesn't have execution privileges. I've tried | os.chown(...) but this throws an OSError. I understand that chown'ing | programmatically opens a big security hole. However, setting the gid to give | apache execution privileges isn't any better. What do you suggest? Are you sure you don't want chmod? -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Winter is gods' way of telling us to polish. - Peter Harper From carsten.haese at gmail.com Wed Jan 6 01:41:57 2010 From: carsten.haese at gmail.com (Carsten Haese) Date: Wed, 06 Jan 2010 01:41:57 -0500 Subject: chown'ing by script In-Reply-To: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> References: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> Message-ID: Victor Subervi wrote: > Hi; > I have a script that is called via the web. This script writes another > script that is also called by the web, which in turn needs to have > execution privileges. The problem is that the programmatically created > file is owned by apache.apache and thus doesn't have execution > privileges. I've tried os.chown(...) but this throws an OSError. I > understand that chown'ing programmatically opens a big security hole. > However, setting the gid to give apache execution privileges isn't any > better. What do you suggest? I suggest you find a way to achieve whatever it is you are trying to achieve without having to execute programmatically created scripts. What is the underlying problem you're trying to solve with this approach? -Carsten From wuwei23 at gmail.com Wed Jan 6 01:48:59 2010 From: wuwei23 at gmail.com (alex23) Date: Tue, 5 Jan 2010 22:48:59 -0800 (PST) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> Message-ID: <51fbdc79-f093-497b-9a98-ce4636cb5610@j14g2000yqm.googlegroups.com> Steven D'Aprano wrote: > Stick around and you might learn something, > but if you bite every time somebody tries to teach you, you'll soon run > out of people willing to help you. The ongoing crowdsourced development by this group of "Victor Subervi's" project would seem to indicate that this isn't the case. Or that if it is, that it will be a good handful of months at the very soonest before it becomes true. From half.italian at gmail.com Wed Jan 6 01:50:10 2010 From: half.italian at gmail.com (Sean DiZazzo) Date: Tue, 5 Jan 2010 22:50:10 -0800 (PST) Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> <96ea39f2-1fb6-4b03-b206-d4c1445ea021@v25g2000yqk.googlegroups.com> Message-ID: On Jan 5, 11:40?am, KvS wrote: > On Jan 5, 7:16?pm, Nobody wrote: > > > > > On Tue, 05 Jan 2010 04:40:14 -0800, KvS wrote: > > >> Did you mean borderless printing? > > >> Every printer needs his margins, some more some less. Some printers have the > > >> ability to do borderless printing but usualy they can do it only on special > > >> or photo paper. So you can adjust the pdf as you wish, even with no margins, > > >> and then try to find under printer options "borderless printing". That is > > >> why I didn't understand :-)) it is a printer thing not pdf! > > > > As much as possible "borderless", yes. Of course the printer will > > > still apply some small margin, but that's ok. A margin of say <0.5 cm. > > > is fine. So it's not a printer thing, I accept the (physical) > > > limitations of the printer, but I want to avoid any extra margins due > > > to software settings. > > > "Hardcopy" document formats such as PostScript and PDF use positions > > relative to the edges of the page, not the margins. > > Right. Still, Acrobat Reader by default scales the contents to fit on > a page and creates some margins by doing so, no? So if my text is > close to the left and right edges, as I want, it will get scaled and > extra margins will occur. Avoiding this still requires me to be able > to turn off this scaling in the printing preferences somehow > programmatically, so it doesn't seem to make the problem easier? Maybe you could have the user print your data on a larger sheet of paper;one that is sure to include all of the data, and include crop marks on the printout. The user then cuts along the crop marks to leave a perfectly sized, marginless page. This is how printers do bleeds. From benjamin.kaplan at case.edu Wed Jan 6 01:54:17 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Wed, 6 Jan 2010 01:54:17 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> Message-ID: On Tue, Jan 5, 2010 at 8:45 PM, Phlip wrote: > > Here's a super easy example: > > ?{ 42: 'forty two' }.get(41, None) > > Because I can supply a default, I can decide what is an error and what > is . > > Now the equivalent in a language that does not enjoy this false "Zen": > > ?{ 42: 'forty two' }[41] ?# returns None > ?{ 42: 'forty two' }.fetch(41, None) ?# ibid > ?{ 42: 'forty two' }.fetch(41) ?# raises an exception Here's another super easy example. Should {42 : 'forty two'}[41] be treated the same as {41: None}[41] ? In cases where None is a valid result, you can't use it to signal failure. From victorsubervi at gmail.com Wed Jan 6 02:11:31 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 6 Jan 2010 02:11:31 -0500 Subject: TypeError Message-ID: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> Hi; I get this error: /var/www/html/angrynates.com/christians/cart/simplemail/mail.py 153 154 ''' 155 commitSale() 156 myMail() 157 print ''' commitSale = /var/www/html/angrynates.com/christians/cart/simplemail/mail.py in commitSale() 98 cursor.execute('select max(ID) from %sCustomerData;' % store) 99 custID = cursor.fetchone()[0] 100 customerData(store, tmpTable, custID, patientID) 101 102 def myMail(): global customerData = , global store = 'products', tmpTable = 'tem12627568064', custID = 1, global patientID = 'None' /var/www/html/angrynates.com/christians/cart/customerData.py in customerData(store='products', tmpTable='tem12627568064', custID=1, patientID='None') 39 40 """ 41 print """ 42 print '

%s Customer Data

' % (store[0].upper() + store[1:]) 43 cursor.execute('describe %sCustomerData' % store) store = 'products' ValueError: unsupported format character '(' (0x28) at index 54 args = ("unsupported format character '(' (0x28) at index 54",) Apparently that character is a "file separator", which I presume is an invisible character. I tried retyping the area in question, but with no avail (threw same error). Please advise. Complete code follows. TIA, beno #!/usr/bin/python import sys,os sys.path.append(os.getcwd()) import MySQLdb from login import login import re, string def customerData(store, tmpTable, custID, patientID): user, passwd, db, host = login() db = MySQLdb.connect(host, user, passwd, db) cursor = db.cursor() page= """#!/usr/bin/python import sys,os sys.path.append(os.getcwd()) import MySQLdb from login import login import re, string from particulars import ourStores, ourOptions import fpformat from sets import Set from processOrder import processOrder def %sCustomerData(): print 'Content-Type: text/html' print print ''' """ print """ print '

%s Customer Data

' % (store[0].upper() + store[1:]) cursor.execute('describe %sCustomerData' % store) cols = [itm[0] for itm in cursor] cursor.execute('select * from %sCustomerData where ID=%s' % (store, custID)) ourCustomerData = cursor.fetchone() ourColsAndCustomerDataDict = dict(zip(cols, ourCustomerData)) for col, data in ourColsAndCustomerDataDict.iteritems(): print '%s: %s
' % (col, data) print "

Customer Order

" print "\n " numberShippingFields, totalTotal, html = processOrder(patientID, store, tmpTable) print html print " " if store != 'prescriptions': numberCols = 12 + numberShippingFields else: numberCols = 8 + numberShippingFields print ' \n \n \n ' % (numberCols - 1, str(fpformat.fix(round(int(totalTotal * 100))/100,2))) print "
TOTAL$%s
" cursor.close() print ''' ''' %sCustomerData() """ % (store, store) os.chdir('..') try: os.delete('%s/%sCustomerData.py' % (os.getcwd(), store)) except AttributeError: pass file = '%sCustomerData.py' % store f = open(file, 'w') f.write(page) f.close() # os.chown('%s/%s' % (os.getcwd(), file), 500, 500) os.chmod('%s/%s' % (os.getcwd(), file), 0755) -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From wuwei23 at gmail.com Wed Jan 6 02:12:00 2010 From: wuwei23 at gmail.com (alex23) Date: Tue, 5 Jan 2010 23:12:00 -0800 (PST) Subject: chown'ing by script References: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> Message-ID: Carsten Haese wrote: > What is the underlying problem you're trying to solve with this > approach? To be paid for developing a web site shopping cart without actually having to learn Python. From victorsubervi at gmail.com Wed Jan 6 02:14:11 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 6 Jan 2010 02:14:11 -0500 Subject: chown'ing by script In-Reply-To: References: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> Message-ID: <4dc0cfea1001052314s77f23c84w3b331bfdaaf71d89@mail.gmail.com> On Wed, Jan 6, 2010 at 1:41 AM, Carsten Haese wrote: > Victor Subervi wrote: > > Hi; > > I have a script that is called via the web. This script writes another > > script that is also called by the web, which in turn needs to have > > execution privileges. The problem is that the programmatically created > > file is owned by apache.apache and thus doesn't have execution > > privileges. I've tried os.chown(...) but this throws an OSError. I > > understand that chown'ing programmatically opens a big security hole. > > However, setting the gid to give apache execution privileges isn't any > > better. What do you suggest? > > I suggest you find a way to achieve whatever it is you are trying to > achieve without having to execute programmatically created scripts. What > is the underlying problem you're trying to solve with this approach? > As I was writing to respond, I re-thought the problem out and now I don't need any more advice...at least on this issue ;) Thanks! beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From gherron at islandtraining.com Wed Jan 6 02:29:20 2010 From: gherron at islandtraining.com (Gary Herron) Date: Tue, 05 Jan 2010 23:29:20 -0800 Subject: Do I have to use threads? In-Reply-To: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> Message-ID: <4B443BD0.8020704@islandtraining.com> aditya shukla wrote: > Hello people, > > I have 5 directories corresponding 5 different urls .I want to > download images from those urls and place them in the respective > directories.I have to extract the contents and download them > simultaneously.I can extract the contents and do then one by one. My > questions is for doing it simultaneously do I have to use threads? > > Please point me in the right direction. > > > Thanks > > Aditya You've been given some bad advice here. First -- threads are lighter-weight than processes, so threads are probably *more* efficient. However, with only five thread/processes, the difference is probably not noticeable. (If the prejudice against threads comes from concerns over the GIL -- that also is a misplaced concern in this instance. Since you only have network connection, you will receive only one packet at a time, so only one thread will be active at a time. If the extraction process uses a significant enough amount of CPU time so that the extractions are all running at the same time *AND* if you are running on a machine with separate CPU/cores *AND* you would like the extractions to be running truly in parallel on those separate cores, *THEN*, and only then, will processes be more efficient than threads.) Second, running 5 wgets is equivalent to 5 processes not 5 threads. And third -- you don't have to use either threads *or* processes. There is another possibility which is much more light-weight: asynchronous I/O, available through the low level select module, or more usefully via the higher-level asyncore module. (Although the learning curve might trip you up, and some people find the programming model for asyncore hard to fathom, I find it more intuitive in this case than threads/processes.) In fact, the asyncore manual page has a ~20 line class which implements a web page retrieval. You could replace that example's single call to http_client with five calls, one for each of your ULRs. Then when you enter the last line (that is the asyncore.loop() call) the five will be downloading simultaneously. See http://docs.python.org/library/asyncore.html Gary Herron From ashish.vyas at motorola.com Wed Jan 6 03:51:46 2010 From: ashish.vyas at motorola.com (VYAS ASHISH M-NTB837) Date: Wed, 6 Jan 2010 16:51:46 +0800 Subject: lxml 2.2.4 on python3.1, Windows XP gives importerror In-Reply-To: <7C57DB58C81FB64CA979EC6C4DB73E7A03E7039F@ZMY16EXM67.ds.mot.com> Message-ID: <7C57DB58C81FB64CA979EC6C4DB73E7A03E7053F@ZMY16EXM67.ds.mot.com> Posting again as I did not get any response: Dear All I have Python 3.1 installed on Windows XP and Works nice. I downloaded lxml 2.2.4 (lxml-2.2.4.win32-py3.1.exe) from pypi. When I try: from lxml import etree I get: ImportError: DLL load failed: This application has failed to start because the application configuration is incorrect. Reinstalling the application may fix this problem. For information: 'import lxml' works fine. After reinstalling python3.1 also the error message is the same. Any help is appreciated! Regards, Ashish Vyas -------------- next part -------------- An HTML attachment was scrubbed... URL: From __peter__ at web.de Wed Jan 6 04:01:42 2010 From: __peter__ at web.de (Peter Otten) Date: Wed, 06 Jan 2010 10:01:42 +0100 Subject: unittest inconsistent References: <3dede91a-ab3c-42c0-af45-adde220ededd@z2g2000vbf.googlegroups.com> Message-ID: Andr? wrote: > On Jan 5, 8:14 pm, Matt Haggard wrote: >> Can anyone tell me why this test fails? >> >> http://pastebin.com/f20039b17 >> >> This is a minimal example of a much more complex thing I'm trying to >> do. I'm trying to hijack a function and inspect the args passed to it >> by another function. >> >> The reason the 'Tester' object has no attribute 'arg1' is because >> "self" still refers to the object made for testA. > > Quick answer: change faketest.py as follows: > > #-------------------------------------------------- > # faketest.py > #-------------------------------------------------- > > #from importme import render > import importme > > def run(somearg): > return importme.render(somearg) > > ========= > A long answer, with explanation, will cost you twice as much ;-) > (but will have to wait) > > Andr? Or you figure it out yourself staring at >>> import os >>> from os import rename >>> os.rename = 42 >>> rename >>> os.rename 42 from module import name binds the object referred to by module.name to the name variable in the current module. You can think of it as a shortcut for import module name = module.name del module When you later rebind import module module.name = something_else the reference in the current module isn't magically updated to point to something_else. Peter From ben+python at benfinney.id.au Wed Jan 6 04:15:33 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 06 Jan 2010 20:15:33 +1100 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: <87ljgbwq4q.fsf@benfinney.id.au> r0g writes: > NO! It's a rude way to start a sentence don't you think? Shouting is usually rude, yes. > Just because you're correcting someone doesn't mean you have to be > combative and try and make them feel small. Again, you're reading something that isn't there. I utterly deny the motives you're imputing. > I can't imagine why I expect good manners on usenet though, AFAICT > it's never been like that (well not since I got on it anyway). I'll reiterate that you can address this by paying more attention to what is actually written, and endeavouring to avoid finding emotions or motives that are likely not in the message. -- \ ?The greatest tragedy in mankind's entire history may be the | `\ hijacking of morality by religion.? ?Arthur C. Clarke, 1991 | _o__) | Ben Finney From victorsubervi at gmail.com Wed Jan 6 04:33:25 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 6 Jan 2010 04:33:25 -0500 Subject: chown'ing by script In-Reply-To: References: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> Message-ID: <4dc0cfea1001060133q35dd0e4dk4a905e6957ff86a2@mail.gmail.com> On Wed, Jan 6, 2010 at 2:12 AM, alex23 wrote: > Carsten Haese wrote: > > What is the underlying problem you're trying to solve with this > > approach? > > To be paid for developing a web site shopping cart without actually > having to learn Python. > LOL! I've written about 12,000 lines of code in this, BTW. Crawl back into your cave :) beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsagert at gmail.com Wed Jan 6 05:11:49 2010 From: bsagert at gmail.com (Bill) Date: Wed, 6 Jan 2010 02:11:49 -0800 (PST) Subject: Need help to pass self.count to other classes. Message-ID: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> After a year with Python 2.5 on my Windows box, I still have trouble understanding classes. Below, see the batch file and the configuration script for my Python interactive prompt. The widths of the secondary prompts increase when the self.count of SysPrompt1 exceeds 99. I am using a global variable "zxc" to share self.count, which is not Pythonic. How can I pass in self.count without a global? I did RTFM, aka Google, but to no avail. echo py.bat set PYTHONSTARTUP=c:\scripts\startup.py python ^Z # startup.py # inspired by: # http://www.doughellmann.com/PyMOTW/sys/interpreter.html import sys class SysPrompt1(object): def __init__(self): self.count = 0 def __str__(self): self.count += 1 global zxc zxc = self.count return '[%2d]> ' % self.count class SysPrompt2(object): def __str__(self): global zxc if zxc > 99: return '...... ' else: return '..... ' class DisplayHook(object): def __call__(self, value): if value is None: return global zxc if zxc > 99: print '[ out]', value, '\n' else: print '[out]', value, '\n' class ExceptHook(object): def __call__(self, type, value, trace): global zxc if zxc > 99: print '[ err]', value, '\n' else: print '[err]', value, '\n' sys.ps1 = SysPrompt1() sys.ps2 = SysPrompt2() sys.displayhook = DisplayHook() sys.excepthook = ExceptHook() From bieffe62 at gmail.com Wed Jan 6 05:43:06 2010 From: bieffe62 at gmail.com (Francesco Bochicchio) Date: Wed, 6 Jan 2010 02:43:06 -0800 (PST) Subject: Need help to pass self.count to other classes. References: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> Message-ID: <987471cd-2bf1-4eb7-af77-e52abae82f16@o28g2000yqh.googlegroups.com> On 6 Gen, 11:11, Bill wrote: > After a year with Python 2.5 on my Windows box, I still have trouble > understanding classes. > > Below, see the batch file and the configuration script for > my Python interactive prompt. > > The widths of the secondary prompts increase when ?the self.count of > SysPrompt1 exceeds 99. > > I am using a global variable "zxc" to share self.count, which is not > Pythonic. > > How can I pass in self.count without a global? > I did RTFM, aka Google, but to no avail. > > echo py.bat > set PYTHONSTARTUP=c:\scripts\startup.py > python > ^Z > > # startup.py > # inspired by: > #http://www.doughellmann.com/PyMOTW/sys/interpreter.html > > import sys > > class SysPrompt1(object): > ? ? def __init__(self): > ? ? ? ? self.count = 0 > ? ? def __str__(self): > ? ? ? ? self.count += 1 > ? ? ? ? global zxc > ? ? ? ? zxc = self.count > ? ? ? ? return '[%2d]> ' % self.count > > class SysPrompt2(object): > ? ? def __str__(self): > ? ? ? ? global zxc > ? ? ? ? if zxc > 99: return '...... ' > ? ? ? ? else: return '..... ' > > class DisplayHook(object): > ? ? def __call__(self, value): > ? ? ? ? if value is None: return > ? ? ? ? global zxc > ? ? ? ? if zxc > 99: print '[ out]', value, '\n' > ? ? ? ? else: print '[out]', value, '\n' > > class ExceptHook(object): > ? ? def __call__(self, type, value, trace): > ? ? ? ? global zxc > ? ? ? ? if zxc > 99: print '[ err]', value, '\n' > ? ? ? ? else: print '[err]', value, '\n' > > sys.ps1 = SysPrompt1() > sys.ps2 = SysPrompt2() > sys.displayhook = DisplayHook() > sys.excepthook = ExceptHook() First of all, you shouldn't do OOP if you don't feel it. Python, unlike Java and like C++, supports also procedural programming, so you can write your scripts without writing classes (but still using objects, since all in python is an object). If you have classes with no data and a single __call_ method, then they are no classes, they are functions (or methods) in disguise. So, one solution could be to use plain functions and use global as you already do. 'global' is pythonic if you are not doing OOP, although I don't like it. If you want to stick to OOP, then I suggest to have a make display_hook and except_hook methods of your class SysPrompt1; This way all your code can access to self.count without needing globals. As for your class SysPrompt2, I don't understand enough your code to know what you are trying to do. maybe make it a sunclass of SysPrompt1 ? HTH Ciao ---- FB From gtu2003 at alice.it Wed Jan 6 06:03:36 2010 From: gtu2003 at alice.it (wiso) Date: Wed, 06 Jan 2010 12:03:36 +0100 Subject: Convert month name to month number faster Message-ID: <4b446e09$0$1107$4fafbaef@reader4.news.tin.it> I'm optimizing the inner most loop of my script. I need to convert month name to month number. I'm using python 2.6 on linux x64. month_dict = {"Jan":1,"Feb":2,"Mar":3,"Apr":4, "May":5, "Jun":6, "Jul":7,"Aug":8,"Sep":9,"Oct":10,"Nov":11,"Dec":12} def to_dict(name): return month_dict[name] def to_if(name): if name == "Jan": return 1 elif name == "Feb": return 2 elif name == "Mar": return 3 elif name == "Apr": return 4 elif name == "May": return 5 elif name == "Jun": return 6 elif name == "Jul": return 7 elif name == "Aug": return 8 elif name == "Sep": return 9 elif name == "Oct": return 10 elif name == "Nov": return 11 elif name == "Dec": return 12 else: raise ValueError import random l = [random.choice(month_dict.keys()) for _ in range(1000000)] from time import time t = time(); xxx=map(to_dict,l); print time() - t # 0.5 t = time(); xxx=map(to_if,l); print time() - t # 1.0 is there a faster solution? Maybe something with str.translate? The problem is a little different because I don't read random data, but sorted data. For example: l = [x for x in ("Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec") for _ in range(1000)] # ["Jan","Jan", ..., "Feb", "Feb", ...] so maybe the to_if approach will be faster if I write the case in the best order. Look: l = ["Jan"] * 1000000 # to_if is in the best order for "Jan" t = time(); xxx=map(to_dict,l); print time() - t # 0.5 t = time(); xxx=map(to_if,l); print time() - t # 0.5 From ashish.vyas at motorola.com Wed Jan 6 06:14:04 2010 From: ashish.vyas at motorola.com (VYAS ASHISH M-NTB837) Date: Wed, 6 Jan 2010 19:14:04 +0800 Subject: Convert month name to month number faster In-Reply-To: <4b446e09$0$1107$4fafbaef@reader4.news.tin.it> Message-ID: <7C57DB58C81FB64CA979EC6C4DB73E7A03E70581@ZMY16EXM67.ds.mot.com> How about using list.index() and storing month names in a list? You may want to measure performance your self and conclude. Regards, Ashish Vyas -----Original Message----- From: python-list-bounces+ntb837=motorola.com at python.org [mailto:python-list-bounces+ntb837=motorola.com at python.org] On Behalf Of wiso Sent: Wednesday, January 06, 2010 4:34 PM To: python-list at python.org Subject: Convert month name to month number faster I'm optimizing the inner most loop of my script. I need to convert month name to month number. I'm using python 2.6 on linux x64. month_dict = {"Jan":1,"Feb":2,"Mar":3,"Apr":4, "May":5, "Jun":6, "Jul":7,"Aug":8,"Sep":9,"Oct":10,"Nov":11,"Dec":12} def to_dict(name): return month_dict[name] def to_if(name): if name == "Jan": return 1 elif name == "Feb": return 2 elif name == "Mar": return 3 elif name == "Apr": return 4 elif name == "May": return 5 elif name == "Jun": return 6 elif name == "Jul": return 7 elif name == "Aug": return 8 elif name == "Sep": return 9 elif name == "Oct": return 10 elif name == "Nov": return 11 elif name == "Dec": return 12 else: raise ValueError import random l = [random.choice(month_dict.keys()) for _ in range(1000000)] from time import time t = time(); xxx=map(to_dict,l); print time() - t # 0.5 t = time(); xxx=map(to_if,l); print time() - t # 1.0 is there a faster solution? Maybe something with str.translate? The problem is a little different because I don't read random data, but sorted data. For example: l = [x for x in ("Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec" ) for _ in range(1000)] # ["Jan","Jan", ..., "Feb", "Feb", ...] so maybe the to_if approach will be faster if I write the case in the best order. Look: l = ["Jan"] * 1000000 # to_if is in the best order for "Jan" t = time(); xxx=map(to_dict,l); print time() - t # 0.5 t = time(); xxx=map(to_if,l); print time() - t # 0.5 -- http://mail.python.org/mailman/listinfo/python-list From solipsis at pitrou.net Wed Jan 6 06:53:46 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 6 Jan 2010 11:53:46 +0000 (UTC) Subject: Convert month name to month number faster References: <4b446e09$0$1107$4fafbaef@reader4.news.tin.it> Message-ID: Le Wed, 06 Jan 2010 12:03:36 +0100, wiso a ?crit?: > from time import time > t = time(); xxx=map(to_dict,l); print time() - t # 0.5 t = time(); > xxx=map(to_if,l); print time() - t # 1.0 Don't define your own function just for attribute access. Instead just write: xxx = map(month_dict.__getitem__, l) From gerard.blais at gmail.com Wed Jan 6 06:54:17 2010 From: gerard.blais at gmail.com (Gerry) Date: Wed, 6 Jan 2010 03:54:17 -0800 (PST) Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> <96ea39f2-1fb6-4b03-b206-d4c1445ea021@v25g2000yqk.googlegroups.com> Message-ID: <4a4634ea-95b0-4b6e-ba35-bc12cd8e7cbe@21g2000vbh.googlegroups.com> If this is, by any chance, an HP printer, the printer may support PCL 5 (or a similar language). I've written PCL scripts (in Pascal, so a while ago) to precisely print points at the printer resolution (i.e., I picked which six-hundredth of an inch in height and width dimensions to print a dot. PCL 5 absolutely would let you put text anywhere you wanted, including the absolute limits on margins that the hardware supports. (PCL is printer control language). Gerry From kk720801 at yahoo.com Wed Jan 6 06:57:52 2010 From: kk720801 at yahoo.com (Krzysztof Kobus) Date: Wed, 6 Jan 2010 03:57:52 -0800 (PST) Subject: No subject Message-ID: <433599.23145.qm@web53502.mail.re2.yahoo.com> Hi, > Well, it seems that one of your files is a different architecture than > the others. Based on the location, I'd say it's i386 while the rest of > it would be PowerPC. You can cross-compile but you can't link an i386 > library to a PowerPC library. Thank you for the hint. I have checked with "file" command the format of the libraries on mac and got following result: one of my application's libraries: libkkbase.dylib: Mach-O dynamically linked shared library i386 my python module that has been built using standard python distutils: j3k.so: Mach-O bundle i386 I am not actually sure what's the difference between "bundle" and "dynamically linked shared library" on mac and if they can be linked together? Perhaps I should configure distutils on mac somehow specially to make the module ready for embedding? Thanks, KK On Tue, Jan 5, 2010 at 7:33 AM, Krzysztof Kobus wrote: > Hi, > > I have a problem with linking python module with my application on mac in order to make the module available in "embedded python". > > My python module is contained in j3kmodule.cxx file and module initialization function is exported in j3kmodule.h > > j3kmodule.h: > ------------ > PyMODINIT_FUNC PyInit_j3k(void); > > > j3kmodule.cxx: > -------------- > PyMODINIT_FUNC > PyInit_j3k(void) > { > PyObject *m = NULL; > > if ((m = PyModule_Create(&j3k_module)) == NULL) > return NULL; > > return m; > } > > > Then in my application in KkPython.cxx file I have: > > KkPython.cxx: > ------------- > > #include "j3kmodule.h" > > /* Add a builtin module, before Py_Initialize */ > PyImport_AppendInittab("j3k", PyInit_j3k); > > > I link my application with the module and get following linking error on mac although on open suse linux exactly the same procedure works fine: > > g++ -headerpad_max_install_names -o ../../bin/render.app/Contents/MacOS/render main.o KkPython.o -lkkbase -L/Users/kk/dev/J3K/trunk/j3ksrc/examples/render/../../lib/ -lQtSolutions_PropertyBrowser-2.5 -lpython3.1 /Users/kk/dev/J3K/trunk/j3ksrc/python/modules/build/lib.macosx-10.3-i386-3.1/j3k.so -framework OpenGL -framework AGL > ld: warning in /Users/kk/dev/J3K/trunk/j3ksrc/python/modules/build/lib.macosx-10.3-i386-3.1/j3k.so, file is not of required architecture > Undefined symbols: > "_PyInit_j3k", referenced from: > _PyInit_j3k$non_lazy_ptr in KkPython.o > ld: symbol(s) not found > collect2: ld returned 1 exit status > make: *** [../../bin/render.app/Contents/MacOS/render] Error 1 > > > I appreciate any hints, > > best greetings, > > Krzysztof Kobus > From sccolbert at gmail.com Wed Jan 6 06:57:58 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Wed, 6 Jan 2010 12:57:58 +0100 Subject: how to change when the logging module creates the log file? Message-ID: <7f014ea61001060357r62567020wb82aa92940aa5763@mail.gmail.com> I have an application the writes to a log file when specific exceptions are handled. However, if no exceptions are encountered, I don't want to create a log at all. The problem I am running into is that the stdlib logging module creates the log file immediately upon logger instantiation. Thus: >>> logger = logging.basicConifg('testlog.txt') already creates the file 'testlog.txt'. So, at program close, I am reading the size of the log file, and if it is empty I remove it with os.remove(). This works fine on Linux, but throws a permission denied exception on Windows. There has to be a better way to do this than using a hack like that. Is there a way to make the logging module hold-off on file creation until the first log is generated? I could do it by wrapping logger in a class, but that would remove the beauty of having any module import logging from the stdlib and being able to write to the log. Thanks for any pointers! Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: From wuwei23 at gmail.com Wed Jan 6 06:58:39 2010 From: wuwei23 at gmail.com (alex23) Date: Wed, 6 Jan 2010 03:58:39 -0800 (PST) Subject: Convert month name to month number faster References: <4b446e09$0$1107$4fafbaef@reader4.news.tin.it> Message-ID: <126bd455-52f2-4e92-8427-1982b4de925e@b2g2000yqi.googlegroups.com> On Jan 6, 9:03?pm, wiso wrote: > I'm optimizing the inner most loop of my script. I need to convert month > name to month number. I'm using python 2.6 on linux x64. > > month_dict = {"Jan":1,"Feb":2,"Mar":3,"Apr":4, "May":5, "Jun":6, > ? ? ? ? ? ?"Jul":7,"Aug":8,"Sep":9,"Oct":10,"Nov":11,"Dec":12} > > def to_dict(name): > ? return month_dict[name] Try replacing the to_dict function with: to_dict = month_dict.get That removes one extra function call per lookup. On my computer, this reduces the time for your test from 0.26 to 0.09. From kk720801 at yahoo.com Wed Jan 6 06:59:50 2010 From: kk720801 at yahoo.com (Krzysztof Kobus) Date: Wed, 6 Jan 2010 03:59:50 -0800 (PST) Subject: embedded python on mac - linking problem Message-ID: <921541.24413.qm@web53506.mail.re2.yahoo.com> Hi, > > Well, it seems that one of your files is a different architecture than > > the others. Based on the location, I'd say it's i386 while the rest of > > it would be PowerPC. You can cross-compile but you can't link an i386 > > library to a PowerPC library. Thank you for the hint. I have checked with "file" command the format of the libraries on mac and got following result: one of my application's libraries: libkkbase.dylib: Mach-O dynamically linked shared library i386 my python module that has been built using standard python distutils: j3k.so: Mach-O bundle i386 I am not actually sure what's the difference between "bundle" and "dynamically linked shared library" on mac and if they can be linked together? Perhaps I should configure distutils on mac somehow specially to make the module ready for embedding? Thanks, KK > On Tue, Jan 5, 2010 at 7:33 AM, Krzysztof Kobus wrote: > > Hi, > > > > > I have a problem with linking python module with my application on mac > in order to make the module available in "embedded python". > > > > My python module is contained in j3kmodule.cxx file and module initialization > function is exported in j3kmodule.h > > > > j3kmodule.h: > > ------------ > > PyMODINIT_FUNC PyInit_j3k(void); > > > > > > j3kmodule.cxx: > > -------------- > > PyMODINIT_FUNC > > PyInit_j3k(void) > > { > > PyObject *m = NULL; > > > > if ((m = PyModule_Create(&j3k_module)) == NULL) > > return NULL; > > > > return m; > > } > > > > > > Then in my application in KkPython.cxx file I have: > > > > KkPython.cxx: > > ------------- > > > > #include "j3kmodule.h" > > > > /* Add a builtin module, before Py_Initialize */ > > PyImport_AppendInittab("j3k", PyInit_j3k); > > > > > > > I link my application with the module and get following linking error > on mac although on open suse linux exactly the same procedure works > fine: > > > > g++ -headerpad_max_install_names -o > ../../bin/render.app/Contents/MacOS/render main.o KkPython.o -lkkbase > -L/Users/kk/dev/J3K/trunk/j3ksrc/examples/render/../../lib/ > -lQtSolutions_PropertyBrowser-2.5 -lpython3.1 > /Users/kk/dev/J3K/trunk/j3ksrc/python/modules/build/lib.macosx-10.3-i386-3.1/j3k.so > -framework OpenGL -framework AGL > > ld: warning in > /Users/kk/dev/J3K/trunk/j3ksrc/python/modules/build/lib.macosx-10.3-i386-3.1/j3k.so, > file is not of required architecture > > Undefined symbols: > > "_PyInit_j3k", referenced from: > > _PyInit_j3k$non_lazy_ptr in KkPython.o > > ld: symbol(s) not found > > collect2: ld returned 1 exit status > > make: *** [../../bin/render.app/Contents/MacOS/render] Error 1 > > > > > > I appreciate any hints, > > > > best greetings, > > > > Krzysztof Kobus > > From gtu2003 at alice.it Wed Jan 6 07:03:38 2010 From: gtu2003 at alice.it (wiso) Date: Wed, 06 Jan 2010 13:03:38 +0100 Subject: Convert month name to month number faster References: <4b446e09$0$1107$4fafbaef@reader4.news.tin.it> Message-ID: <4b447c1b$0$1115$4fafbaef@reader4.news.tin.it> Antoine Pitrou wrote: > Le Wed, 06 Jan 2010 12:03:36 +0100, wiso a ?crit : > > >> from time import time >> t = time(); xxx=map(to_dict,l); print time() - t # 0.5 t = time(); >> xxx=map(to_if,l); print time() - t # 1.0 > > Don't define your own function just for attribute access. Instead just > write: > > xxx = map(month_dict.__getitem__, l) t = time(); xxx=map(month_dict.__getitem__,l); print time() - t # 0.2 month_list = ("","Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec") t = time(); xxx=map(month_list.index,l); time() - t # 0.6 From peter.bienstman at gmail.com Wed Jan 6 07:26:02 2010 From: peter.bienstman at gmail.com (pbienst) Date: Wed, 6 Jan 2010 04:26:02 -0800 (PST) Subject: creating tar file and streaming it over HTTP? Message-ID: <5222d923-7aef-47cf-a0d6-f37f09a66593@35g2000yqa.googlegroups.com> I would like to bundle up a number of files in a tar file and send it over a HTTP connection, but I would like to do this without creating the tar file on disk first. I know I can get tarfile to output to a stream by doing something like tar_pipe = tarfile.open(mode="w|", fileobj=my_file_obj) However, I can't figure out which file object to use to send this over a HTTP connection. I tried conn = httplib.HTTPConnection(hostname, port) conn.putrequest("PUT", "/client/files") conn.endheaders() tar_pipe = tarfile.open(mode="w|", fileobj=conn.sock.makefile()) for filename in filenames: tar_pipe.add(filename) tar_pipe.close() conn.getresponse() but that does not send any data... From steve at REMOVE-THIS-cybersource.com.au Wed Jan 6 07:48:32 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 06 Jan 2010 12:48:32 GMT Subject: Convert month name to month number faster References: <4b446e09$0$1107$4fafbaef@reader4.news.tin.it> Message-ID: <03547388$0$1336$c3e8da3@news.astraweb.com> On Wed, 06 Jan 2010 12:03:36 +0100, wiso wrote: > I'm optimizing the inner most loop of my script. I need to convert month > name to month number. I'm using python 2.6 on linux x64. According to your own figures below, it takes less than a nanosecond per lookup, at worst, even using a remarkably inefficient technique. Are you trying to tell us that this is the bottleneck in your script? I'm sorry, I find that implausible. I think you're wasting your time trying to optimise something that doesn't need optimizing. Even if you halve the time, and deal with a million data points each time you run your script, you will only save half a second per run. I can see from the times you posted that you've spent at least an hour trying to optimise this. To make up for that one hour, you will need to run your script 7200 times, before you see *any* time savings at all. > month_dict = {"Jan":1,"Feb":2,"Mar":3,"Apr":4, "May":5, "Jun":6, > "Jul":7,"Aug":8,"Sep":9,"Oct":10,"Nov":11,"Dec":12} > > def to_dict(name): > return month_dict[name] This leads to a pointless function call. Just call month_dict[name] instead of calling a function that calls it. > def to_if(name): > if name == "Jan": return 1 > elif name == "Feb": return 2 > elif name == "Mar": return 3 > elif name == "Apr": return 4 > elif name == "May": return 5 > elif name == "Jun": return 6 > elif name == "Jul": return 7 > elif name == "Aug": return 8 > elif name == "Sep": return 9 > elif name == "Oct": return 10 > elif name == "Nov": return 11 > elif name == "Dec": return 12 > else: raise ValueError That is remarkably awful. > import random > l = [random.choice(month_dict.keys()) for _ in range(1000000)] > > from time import time > t = time(); xxx=map(to_dict,l); print time() - t # 0.5 > t = time(); xxx=map(to_if,l); print time() - t # 1.0 This is not a reliable way to do timings. You should use the timeit module. > is there a faster solution? Maybe something with str.translate? What makes you think str.translate is even remotely useful for this? -- Steven From athena.kurukshetra at gmail.com Wed Jan 6 08:08:20 2010 From: athena.kurukshetra at gmail.com (dhashrath govindarajan) Date: Wed, 6 Jan 2010 05:08:20 -0800 (PST) Subject: Online math-coding contest Message-ID: Hi , we gladly invite you to take part in Athena - the Online Math Coding Contest of Kurukshetra 2010 , the International Techo-Management Fest organised by College Of Engineering Guindy , India under the patronage of UNESCO . Here's your chance to lock horns against the best minds across the globe and showcase your algorithmic prowess and mathematical acumen !! Participants from 20+ countries already registered!! catch the action at www.athena.kurukshetra.org.in Exciting prizes to be won !! For queries please contact athena at kurukshetra.org.in Dates : Practice contest : 6th January 5.00 P.M Indian Standard Time Main contest : 8th January - 15th January awaiting your presence , Team Athena From briandenzer at gmail.com Wed Jan 6 08:10:56 2010 From: briandenzer at gmail.com (Brian D) Date: Wed, 6 Jan 2010 05:10:56 -0800 (PST) Subject: fsync() doesn't work as advertised? References: Message-ID: On Jan 5, 1:08?pm, Nobody wrote: > On Mon, 04 Jan 2010 08:09:56 -0800, Brian D wrote: > > If I'm running a process in a loop that runs for a long time, I > > occasionally would like to look at a log to see how it's going. > > > I know about the logging module, and may yet decide to use that. > > > Still, I'm troubled by howfsync() doesn't seem to work as advertised: > > >http://docs.python.org/library/os.html > > > "If you?re starting with a Python file object f, first do f.flush(), > > and then do os.fsync(f.fileno())" > > The .flush() method (and the C fflush() function) causes the > contents of application buffers to be sent to the OS, which basically > copies the data into the OS-level buffers. > > fsync() causes the OS-level buffers to be written to the physical drive. > > File operations normally use the OS-level buffers; e.g. if one process > write()s to a file and another process read()s it, the latter will see > what the former has written regardless of whether the data has been > written to the drive. > > The main reason for usingfsync() is to prevent important data from being > lost in the event of an unexpected reboot or power-cycle (an expected > reboot via the "shutdown" or "halt" commands will flush all OS-level > buffers to the drive first). Other than that,fsync() is almost invisible > (I say "almost", as there are mechanisms to bypass the OS-level buffers, > e.g. the O_DIRECT open() flag). An excellent explanation of the process. I've seen this in other programming environments, so I could visualize something to that effect, but couldn't have verbalized it. I certainly have a better idea what's happening. Thanks for the contribution. From bruno.42.desthuilliers at websiteburo.invalid Wed Jan 6 08:13:25 2010 From: bruno.42.desthuilliers at websiteburo.invalid (Bruno Desthuilliers) Date: Wed, 06 Jan 2010 14:13:25 +0100 Subject: Need help to pass self.count to other classes. In-Reply-To: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> References: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> Message-ID: <4b448c69$0$6291$426a74cc@news.free.fr> Bill a ?crit : > After a year with Python 2.5 on my Windows box, I still have trouble > understanding classes. > > Below, see the batch file and the configuration script for > my Python interactive prompt. > > The widths of the secondary prompts increase when the self.count of > SysPrompt1 exceeds 99. > > I am using a global variable "zxc" to share self.count, which is not > Pythonic. It's not "unpythonic" - but it's not good OO style neither !-) > How can I pass in self.count without a global? (snip) > sys.ps1 = SysPrompt1() > sys.ps2 = SysPrompt2() > sys.displayhook = DisplayHook() > sys.excepthook = ExceptHook() > hint #1: a bound method is also a callable. hint #2: an object can be an attribute of another object. Possible OO solution: class Counter(object): def __init__(self): self._count = 0 def inc(self): self._count += 1 value = property(fget=lambda s: s._count) class Prompt1(object): def __init__(self, counter): self._counter = counter def _str_(self): self._counter.inc() return '[%2d]> ' % self._counter.value class Prompt2(object): def __init__(self, counter): self._counter = counter def _str_(self): if self._counter.value > 99: # XXX magic number return '...... ' else: return '..... ' class MySysHookHandler(object): def __init__(self): self._counter = Counter() self.ps1 = Prompt1(self._counter) self.ps2 = Prompt2(self._counter) def displayhook(self, value): if value is None: return if self._counter.value > 99: # XXX magic number print '[ out]', value, '\n' else: print '[out]', value, '\n' def excepthook(self, type, value, trace): if self._counter.value > 99: # XXX magic number print '[ err]', value, '\n' else: print '[err]', value, '\n' handler = MySysHook() sys.ps1 = handler.ps1 sys.ps2 = handler.ps2 sys.displayhook = handler.displayhook sys.excepthook = handler.excepthook HTH From philip at semanchuk.com Wed Jan 6 08:24:54 2010 From: philip at semanchuk.com (Philip Semanchuk) Date: Wed, 6 Jan 2010 08:24:54 -0500 Subject: Do I have to use threads? In-Reply-To: <9839a05c1001052145j663f9f73nfac6b3cdeec411fe@mail.gmail.com> References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> <9839a05c1001052145j663f9f73nfac6b3cdeec411fe@mail.gmail.com> Message-ID: On Jan 6, 2010, at 12:45 AM, Brian J Mingus wrote: > On Tue, Jan 5, 2010 at 9:36 PM, Philip Semanchuk > wrote: > >> >> On Jan 5, 2010, at 11:26 PM, aditya shukla wrote: >> >> Hello people, >>> >>> I have 5 directories corresponding 5 different urls .I want to >>> download >>> images from those urls and place them in the respective >>> directories.I have >>> to extract the contents and download them simultaneously.I can >>> extract the >>> contents and do then one by one. My questions is for doing it >>> simultaneously >>> do I have to use threads? >>> >> >> No. You could spawn 5 copies of wget (or curl or a Python program >> that >> you've written). Whether or not that will perform better or be >> easier to >> code, debug and maintain depends on the other aspects of your >> program(s). >> >> bye >> Philip > > > Obviously, spawning 5 copies of wget is equivalent to starting 5 > threads. > The answer is 'yes'. ??? Process != thread From steve at holdenweb.com Wed Jan 6 08:27:27 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 08:27:27 -0500 Subject: creating tar file and streaming it over HTTP? In-Reply-To: <5222d923-7aef-47cf-a0d6-f37f09a66593@35g2000yqa.googlegroups.com> References: <5222d923-7aef-47cf-a0d6-f37f09a66593@35g2000yqa.googlegroups.com> Message-ID: pbienst wrote: > I would like to bundle up a number of files in a tar file and send it > over a HTTP connection, but I would like to do this without creating > the tar file on disk first. > > I know I can get tarfile to output to a stream by doing something like > > tar_pipe = tarfile.open(mode="w|", fileobj=my_file_obj) > > However, I can't figure out which file object to use to send this over > a HTTP connection. > > I tried > > conn = httplib.HTTPConnection(hostname, port) > conn.putrequest("PUT", "/client/files") > conn.endheaders() > tar_pipe = tarfile.open(mode="w|", fileobj=conn.sock.makefile()) > for filename in filenames: > tar_pipe.add(filename) > tar_pipe.close() > conn.getresponse() > > but that does not send any data... > I haven't used socket.makefile() in a blue age so this is untested, but I'm guessing you need a *write* file (remember, sockets have two directions). So try .... fileobj=conn.sock.makefile("w")... and let me know if that helps. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From Brian.Mingus at Colorado.EDU Wed Jan 6 08:52:27 2010 From: Brian.Mingus at Colorado.EDU (Brian J Mingus) Date: Wed, 6 Jan 2010 06:52:27 -0700 Subject: Do I have to use threads? In-Reply-To: References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> <9839a05c1001052145j663f9f73nfac6b3cdeec411fe@mail.gmail.com> Message-ID: <9839a05c1001060552g5a5acb67na5cebe50191a26f4@mail.gmail.com> On Wed, Jan 6, 2010 at 6:24 AM, Philip Semanchuk wrote: > > On Jan 6, 2010, at 12:45 AM, Brian J Mingus wrote: > > On Tue, Jan 5, 2010 at 9:36 PM, Philip Semanchuk > >wrote: >> >> >>> On Jan 5, 2010, at 11:26 PM, aditya shukla wrote: >>> >>> Hello people, >>> >>>> >>>> I have 5 directories corresponding 5 different urls .I want to download >>>> images from those urls and place them in the respective directories.I >>>> have >>>> to extract the contents and download them simultaneously.I can extract >>>> the >>>> contents and do then one by one. My questions is for doing it >>>> simultaneously >>>> do I have to use threads? >>>> >>>> >>> No. You could spawn 5 copies of wget (or curl or a Python program that >>> you've written). Whether or not that will perform better or be easier to >>> code, debug and maintain depends on the other aspects of your program(s). >>> >>> bye >>> Philip >>> >> >> >> Obviously, spawning 5 copies of wget is equivalent to starting 5 threads. >> The answer is 'yes'. >> > > ??? > > Process != thread Just like the other nitpicker it is up to you to explain why the differences, and not he similarities, are relevant to this problem. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at holdenweb.com Wed Jan 6 08:56:23 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 08:56:23 -0500 Subject: Need help to pass self.count to other classes. In-Reply-To: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> References: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> Message-ID: Bill wrote: > After a year with Python 2.5 on my Windows box, I still have trouble > understanding classes. > > Below, see the batch file and the configuration script for > my Python interactive prompt. > > The widths of the secondary prompts increase when the self.count of > SysPrompt1 exceeds 99. > > I am using a global variable "zxc" to share self.count, which is not > Pythonic. > > How can I pass in self.count without a global? > I did RTFM, aka Google, but to no avail. > > echo py.bat > set PYTHONSTARTUP=c:\scripts\startup.py > python > ^Z > > # startup.py > # inspired by: > # http://www.doughellmann.com/PyMOTW/sys/interpreter.html > > import sys > > class SysPrompt1(object): > def __init__(self): > self.count = 0 > def __str__(self): > self.count += 1 > global zxc > zxc = self.count > return '[%2d]> ' % self.count > > class SysPrompt2(object): > def __str__(self): > global zxc > if zxc > 99: return '...... ' > else: return '..... ' > > class DisplayHook(object): > def __call__(self, value): > if value is None: return > global zxc > if zxc > 99: print '[ out]', value, '\n' > else: print '[out]', value, '\n' > > class ExceptHook(object): > def __call__(self, type, value, trace): > global zxc > if zxc > 99: print '[ err]', value, '\n' > else: print '[err]', value, '\n' > > sys.ps1 = SysPrompt1() > sys.ps2 = SysPrompt2() > sys.displayhook = DisplayHook() > sys.excepthook = ExceptHook() > As Francesco points out, OO programming isn't *mandatory* in Python. However, if you used a single class instead of multiples then you could use an instance variable for zxc, and have each of the functions be bound methods (i.e. methods of the instance, not the class). This is untested code (some days I don't seem to write any other kind ...) but it should give you the flavor: class kbInterface(object): def __init__(self): self.zxc = 0 def prompt1(self): self.count += 1 return "[%d]> " def prompt2(self): l = len(str(self.count))+1 return "%s " % "."*l def dhook(self, value): print "[%d out]" % self.count def ehook(self, type, value, trace): print "[%d err]\n" % value kbi = kbInterface() sys.ps1 = kbi.prompt1 sys.ps2 = kbi.prompt2 sys.displayhook = kbi.dhook sys.excepthook = kbi.ehook Do you get the idea? Now the count is shared between all the methods, and it's available in the instance's namespace. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Wed Jan 6 08:58:13 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 08:58:13 -0500 Subject: chown'ing by script In-Reply-To: <4dc0cfea1001060133q35dd0e4dk4a905e6957ff86a2@mail.gmail.com> References: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> <4dc0cfea1001060133q35dd0e4dk4a905e6957ff86a2@mail.gmail.com> Message-ID: Victor Subervi wrote: > On Wed, Jan 6, 2010 at 2:12 AM, alex23 > wrote: > > Carsten Haese > wrote: > > What is the underlying problem you're trying to solve with this > > approach? > > To be paid for developing a web site shopping cart without actually > having to learn Python. > > > LOL! I've written about 12,000 lines of code in this, BTW. Crawl back > into your cave :) > beno > Which, I don't doubt, could have been 2,000 lines had you bothered to take the excellent advice you've been offered on learning Python. Still, "the better the advice the worse it's wasted" ... regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From exarkun at twistedmatrix.com Wed Jan 6 09:11:34 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Wed, 06 Jan 2010 14:11:34 -0000 Subject: Do I have to use threads? In-Reply-To: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> Message-ID: <20100106141134.2771.1806839751.divmod.xquotient.41@localhost.localdomain> On 04:26 am, adityashukla1983 at gmail.com wrote: >Hello people, > >I have 5 directories corresponding 5 different urls .I want to >download >images from those urls and place them in the respective directories.I >have >to extract the contents and download them simultaneously.I can extract >the >contents and do then one by one. My questions is for doing it >simultaneously >do I have to use threads? > >Please point me in the right direction. See Twisted, http://twistedmatrix.com/ in particular, Twisted Web's asynchronous HTTP client, http://twistedmatrix.com/documents/current/web/howto/client.html http://twistedmatrix.com/documents/current/api/twisted.web.client.html Jean-Paul From shawn at milochik.com Wed Jan 6 09:35:44 2010 From: shawn at milochik.com (Shawn Milochik) Date: Wed, 6 Jan 2010 09:35:44 -0500 Subject: Python books, literature etc In-Reply-To: References: Message-ID: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> Search Google. You'll find it all. Search this list's archives. This kind of thing has been discussed a thousand times. It also wouldn't hurt to brush up on this: http://catb.org/~esr/faqs/smart-questions.html From jfabiani at yolo.com Wed Jan 6 09:43:29 2010 From: jfabiani at yolo.com (jfabiani at yolo.com) Date: Wed, 06 Jan 2010 06:43:29 -0800 Subject: Convert month name to month number faster References: Message-ID: VYAS ASHISH M-NTB837 wrote: > > How about using list.index() and storing month names in a list? You may > want to measure performance your self and conclude. > > Regards, > Ashish Vyas > > -----Original Message----- > From: python-list-bounces+ntb837=motorola.com at python.org > [mailto:python-list-bounces+ntb837=motorola.com at python.org] On Behalf Of > wiso > Sent: Wednesday, January 06, 2010 4:34 PM > To: python-list at python.org > Subject: Convert month name to month number faster > > I'm optimizing the inner most loop of my script. I need to convert month > name to month number. I'm using python 2.6 on linux x64. > > > month_dict = {"Jan":1,"Feb":2,"Mar":3,"Apr":4, "May":5, "Jun":6, > "Jul":7,"Aug":8,"Sep":9,"Oct":10,"Nov":11,"Dec":12} > > def to_dict(name): > return month_dict[name] > > def to_if(name): > if name == "Jan": return 1 > elif name == "Feb": return 2 > elif name == "Mar": return 3 > elif name == "Apr": return 4 > elif name == "May": return 5 > elif name == "Jun": return 6 > elif name == "Jul": return 7 > elif name == "Aug": return 8 > elif name == "Sep": return 9 > elif name == "Oct": return 10 > elif name == "Nov": return 11 > elif name == "Dec": return 12 > else: raise ValueError > > import random > l = [random.choice(month_dict.keys()) for _ in range(1000000)] > > from time import time > t = time(); xxx=map(to_dict,l); print time() - t # 0.5 > t = time(); xxx=map(to_if,l); print time() - t # 1.0 > > > is there a faster solution? Maybe something with str.translate? > > The problem is a little different because I don't read random data, but > sorted data. For example: > > l = [x for x in > ("Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec" > ) > for _ in range(1000)] # ["Jan","Jan", ..., "Feb", "Feb", ...] > > so maybe the to_if approach will be faster if I write the case in the > best > order. Look: > > l = ["Jan"] * 1000000 # to_if is in the best order for "Jan" > t = time(); xxx=map(to_dict,l); print time() - t # 0.5 > t = time(); xxx=map(to_if,l); print time() - t # 0.5 > > what just using get(key[, default])? Return the value for key if key is in the dictionary, else default. If default is not given, it defaults to None, so that this method never raises a KeyError. From darcy at druid.net Wed Jan 6 09:46:33 2010 From: darcy at druid.net (D'Arcy J.M. Cain) Date: Wed, 6 Jan 2010 09:46:33 -0500 Subject: chown'ing by script In-Reply-To: References: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> <4dc0cfea1001060133q35dd0e4dk4a905e6957ff86a2@mail.gmail.com> Message-ID: <20100106094633.b820725b.darcy@druid.net> On Wed, 06 Jan 2010 08:58:13 -0500 Steve Holden wrote: > Victor Subervi wrote: [Usual nonsense removed] > Which, I don't doubt, could have been 2,000 lines had you bothered to Steve - any chance that you could stop replying to this idiot or at least do it privately. There's not much point to kill-filing someone if someone else is just going to repeat his posts. -- D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner. From kk720801 at yahoo.com Wed Jan 6 09:48:28 2010 From: kk720801 at yahoo.com (Krzysztof Kobus) Date: Wed, 6 Jan 2010 06:48:28 -0800 (PST) Subject: embedded python on mac - linking problem Message-ID: <782094.10621.qm@web53506.mail.re2.yahoo.com> Hi, >The missing symbol looks like a C++-symbol - but Python is C. Do you maybe miss the > extern "C" > > declaration. I have not specified extern "C" as I assume it is a part of PyMODINIT_FUNC define. At least documentation says so: "Note that PyMODINIT_FUNC declares the function as PyObject * return type, declares any special linkage declarations required by the platform, and for C++ declares the function as extern "C"." Regards, KK From martinskou2 at gmail.com Wed Jan 6 09:53:40 2010 From: martinskou2 at gmail.com (msj@infoserv.dk) Date: Wed, 6 Jan 2010 06:53:40 -0800 (PST) Subject: Introspection Message-ID: <0d782b7a-11fb-4933-84b7-29a3653ec847@p8g2000yqb.googlegroups.com> I'm looking for a way to make a list of string literals in a class. Example: class A: def method(self): print 'A','BC' >>> ExtractLiterals(A) ['A','BC'] Is this possible? Can anyone point me in the right direction? Thanks. /Martin From steve at holdenweb.com Wed Jan 6 10:07:59 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 10:07:59 -0500 Subject: chown'ing by script In-Reply-To: <20100106094633.b820725b.darcy@druid.net> References: <4dc0cfea1001052221u4952708bu3724734389ab680b@mail.gmail.com> <4dc0cfea1001060133q35dd0e4dk4a905e6957ff86a2@mail.gmail.com> <20100106094633.b820725b.darcy@druid.net> Message-ID: <4B44A74F.8090400@holdenweb.com> D'Arcy J.M. Cain wrote: > On Wed, 06 Jan 2010 08:58:13 -0500 > Steve Holden wrote: >> Victor Subervi wrote: > [Usual nonsense removed] > >> Which, I don't doubt, could have been 2,000 lines had you bothered to > > Steve - any chance that you could stop replying to this idiot or at > least do it privately. There's not much point to kill-filing someone > if someone else is just going to repeat his posts. > I think idiot's a *bit* strong, but I have to confess that even my tireless wish to evangelize Python has in Victor's case become a bit tired, so I will be happy to oblige. Victor, please refer to python-tutor for further (and more appropriate) assistance. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From phlip2005 at gmail.com Wed Jan 6 10:10:43 2010 From: phlip2005 at gmail.com (Phlip) Date: Wed, 6 Jan 2010 07:10:43 -0800 (PST) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: On Jan 5, 8:49?pm, Steven D'Aprano wrote: > > (A related question - why can't I just go 'if record = method(): ?use > > (record)'. Why extra lines just to trap and assign the variable before > > using it?) > > Because that idiom is responsible for probably the most common error in C > of all, at least one of the most common errors. Thank goodness Python > forbids such a dangerous construct. switching = for == is the "most common error in C"? I can't tell if you are joking. From phlip2005 at gmail.com Wed Jan 6 10:12:12 2010 From: phlip2005 at gmail.com (Phlip) Date: Wed, 6 Jan 2010 07:12:12 -0800 (PST) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> Message-ID: On Jan 5, 10:54?pm, Benjamin Kaplan wrote: > {41: None}[41] ? > > In cases where None is a valid result, you can't use it to signal failure. Asked and answered. You change the "sentinel" in .fetch to something else. But y'all keep on defending the language making your programming decisions for you! From dreadpiratejeff at gmail.com Wed Jan 6 10:17:29 2010 From: dreadpiratejeff at gmail.com (J) Date: Wed, 6 Jan 2010 10:17:29 -0500 Subject: Python books, literature etc In-Reply-To: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> Message-ID: <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> On Wed, Jan 6, 2010 at 09:35, Shawn Milochik wrote: > Search Google. You'll find it all. > > Search this list's archives. This kind of thing has been discussed a thousand times. > > It also wouldn't hurt to brush up on this: > http://catb.org/~esr/faqs/smart-questions.html Heh... I've seen that link mentioned many many times on some other lists I belong to :) But I agree. HOWEVER, that is a valid question. BUT, the answer is really up to the person asking it. For example, I too am a relative beginner with Python. Luckily I DO have at least an educational background in OOP, and at least a professional background in basic coding (some perl, BASH, etc). So I do tend to rely a lot on google when I run into a snag, THEN if that doesn't work, I come here with specific questions. FWIW, my Google searches always look something like "python " and 99% of the time, that gives me the answers I seek. That being said, however, I also am somewhat old school and prefer to have hard copy at hand too. Sometimes, it's just that much more satisfying to have a physical book handy to look things up in. Maybe it's a comfort thing, I don't know... So in that vein, while I can't suggest any specific books, I can say this... right now, I have a copy of O'Reilly's Learning Python that I use as a reference, as well as a much older copy of the Python 2.1 Bible that I picked up almost a decade ago now the first time I messed around with Python. My personal preference, while searching Google and asking on lists like this is quick and invaluable for the "real world" knowledge that is shared, is to also have A: something along the lines of the Learning Foo books from O'Reilly because they tend to have lengthy explanations that I usually can understand on my own, and B: some sort of Cookbook on the topic at hand. I haven't found a Python cookbook that I like enough to buy yet, but I'm still looking. Those come in handy for simple snippets and examples when doing specific tasks. So yeah, Google and this list are certainly invaluable, but some of us actually like having the heft of a lofty tome to peruse for the answers to the mysteries of the universe. Cheers Jeff -- Charles de Gaulle - "The better I get to know men, the more I find myself loving dogs." - http://www.brainyquote.com/quotes/authors/c/charles_de_gaulle.html From steve at holdenweb.com Wed Jan 6 10:21:34 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 10:21:34 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> Message-ID: <4B44AA7E.9070903@holdenweb.com> Phlip wrote: [...] > Don't prevent me from using a technique just because others had > trouble with it. > I presume you also campaign against anti-lock braking systems (or at least don't use cars which have them - after all, anyone who knows how to drive should be able to brake properly, right? And even if they don't, *you* can, so you shouldn't have to pay the extra. Of course, if someone else happens to hit you because they *couldn't* drive safely you will happily give up your life). > And if bar() == foo is the superior technique anyway, because the == > happens in chronological and lexical order after the bar() call. That's about the most specious piece of hogwash I've seen this week. Python doesn't make any guarantees about the evaluation order of =='s operands (unlike "and" and "or"). That's cargo cult programming right there. Readability is surely the only reason to decide how to write that "if", and I suspect most people would choose "if foo == bar()" for that reason, though the difference is slight. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Wed Jan 6 10:21:34 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 10:21:34 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> Message-ID: <4B44AA7E.9070903@holdenweb.com> Phlip wrote: [...] > Don't prevent me from using a technique just because others had > trouble with it. > I presume you also campaign against anti-lock braking systems (or at least don't use cars which have them - after all, anyone who knows how to drive should be able to brake properly, right? And even if they don't, *you* can, so you shouldn't have to pay the extra. Of course, if someone else happens to hit you because they *couldn't* drive safely you will happily give up your life). > And if bar() == foo is the superior technique anyway, because the == > happens in chronological and lexical order after the bar() call. That's about the most specious piece of hogwash I've seen this week. Python doesn't make any guarantees about the evaluation order of =='s operands (unlike "and" and "or"). That's cargo cult programming right there. Readability is surely the only reason to decide how to write that "if", and I suspect most people would choose "if foo == bar()" for that reason, though the difference is slight. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From invalid at invalid.invalid Wed Jan 6 10:34:54 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Wed, 6 Jan 2010 15:34:54 +0000 (UTC) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: On 2010-01-06, r0g wrote: > NO! It's a rude way to start a sentence don't you think? No. When somebody asks a yes/no question, answering yes or no seems quite polite to me. Following the yes/no answer with an explanation of the answer is always nice, and I've little doubt that's what happened. > Just because you're correcting someone doesn't mean you have > to be combative and try and make them feel small. Answering a yes/no question with "no" doesn't seem to me to be combative if the correct answer is indeed "no". But I've lost track of the post you found objectionable... -- Grant Edwards grante Yow! And then we could sit at on the hoods of cars at visi.com stop lights! From bruno.42.desthuilliers at websiteburo.invalid Wed Jan 6 10:41:36 2010 From: bruno.42.desthuilliers at websiteburo.invalid (Bruno Desthuilliers) Date: Wed, 06 Jan 2010 16:41:36 +0100 Subject: Need help to pass self.count to other classes. In-Reply-To: References: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> Message-ID: <4b44af22$0$13292$426a74cc@news.free.fr> Steve Holden a ?crit : (snip) > This is untested code indeed !-) > class kbInterface(object): > def __init__(self): > self.zxc = 0 > def prompt1(self): > self.count += 1 Ahem... (snip) From miki.tebeka at gmail.com Wed Jan 6 10:44:31 2010 From: miki.tebeka at gmail.com (Miki) Date: Wed, 6 Jan 2010 07:44:31 -0800 (PST) Subject: Introspection References: <0d782b7a-11fb-4933-84b7-29a3653ec847@p8g2000yqb.googlegroups.com> Message-ID: <92011e74-c6d3-4a97-a78e-20f062a97e40@r24g2000yqd.googlegroups.com> Hello Martin, > I'm looking for a way to make a list of string literals in a class. from inspect import getsourcelines from tokenize import generate_tokens, STRING, NUMBER def is_literal(t): return t[0] in (STRING, NUMBER) def get_lieterals(obj): lines, _ = getsourcelines(obj) readline = iter(lines).next return [t[1] for t in generate_tokens(readline) if is_literal(t)] if __name__ == "__main__": class A: def f(self): print "A", "B" print get_lieterals(A) HTH, -- Miki From steve at holdenweb.com Wed Jan 6 10:47:05 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 10:47:05 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <4B44B079.5090008@holdenweb.com> Phlip wrote: > On Jan 5, 8:49 pm, Steven D'Aprano > wrote: > >>> (A related question - why can't I just go 'if record = method(): use >>> (record)'. Why extra lines just to trap and assign the variable before >>> using it?) >> Because that idiom is responsible for probably the most common error in C >> of all, at least one of the most common errors. Thank goodness Python >> forbids such a dangerous construct. > > switching = for == is the "most common error in C"? > > I can't tell if you are joking. Well, there's also the "indenting code without surrounding it by braces" error, but y'all just keep defending the approach to programming that *you* think is best. Just don't expect others to agree. The FAQ clearly documents why Python is a statement-based, and not an expression-based, language. Me, I think Guido is a pretty good language designer, and while we've had our differences I think that Python is a *much* better language than C and most others. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From bruno.42.desthuilliers at websiteburo.invalid Wed Jan 6 10:50:29 2010 From: bruno.42.desthuilliers at websiteburo.invalid (Bruno Desthuilliers) Date: Wed, 06 Jan 2010 16:50:29 +0100 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <4b44b138$0$11237$426a74cc@news.free.fr> Phlip a ?crit : > On Jan 5, 8:49 pm, Steven D'Aprano > wrote: > >>> (A related question - why can't I just go 'if record = method(): use >>> (record)'. Why extra lines just to trap and assign the variable before >>> using it?) >> Because that idiom is responsible for probably the most common error in C >> of all, at least one of the most common errors. Thank goodness Python >> forbids such a dangerous construct. > > switching = for == is the "most common error in C"? > > I can't tell if you are joking. It's at least a _very_ common error in all languages that allow this construct. In C, it's common enough to gave birth to the "BestPractice" you described, ie swapping operand orders in equality test to have the compiler detect the problem - at least when one of the operand is a function call expression or constant (it obviously won't 'work' when both operands are variables). Anyway: in Python, assignment is not an expression, and this isn't going to change anytime soon. From jjposner at optimum.net Wed Jan 6 11:18:35 2010 From: jjposner at optimum.net (John Posner) Date: Wed, 06 Jan 2010 11:18:35 -0500 Subject: Dynamic text color References: Message-ID: On Tue, 05 Jan 2010 16:54:44 -0500, Dave McCormick wrote: > But it is not what I am wanting. I first thought to make it look for a > space but that would not work when a single character like "#" is to be > colored if there is a "string" of them. Or if all of the characters > between quotes are to be colored. Regular expressions are good at handling searches like: * all the characters between quotes * the two-character string "do", but only if it's a complete word -John From bsk16 at case.edu Wed Jan 6 11:24:39 2010 From: bsk16 at case.edu (Ben Kaplan) Date: Wed, 06 Jan 2010 11:24:39 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> Message-ID: <4B44B947.1040102@case.edu> On 1/6/10 10:12 AM, Phlip wrote: > On Jan 5, 10:54 pm, Benjamin Kaplan wrote: > > >> {41: None}[41] ? >> >> In cases where None is a valid result, you can't use it to signal failure. >> > Asked and answered. You change the "sentinel" in .fetch to something > else. > When did I specify a sentinel value? You're saying that if None is a valid value in the dict, you can't use slice syntax for it. > But y'all keep on defending the language making your programming > decisions for you! > Your example is the one where the language makes programming decisions for you. Your language is deciding that if I use slice syntax, I want the sentinel to be None. Python is deciding that unless I give it a sentinel, there is no sentinel. From python at mrabarnett.plus.com Wed Jan 6 11:29:04 2010 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 06 Jan 2010 16:29:04 +0000 Subject: TypeError In-Reply-To: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> References: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> Message-ID: <4B44BA50.1060203@mrabarnett.plus.com> Victor Subervi wrote: > Hi; > I get this error: > > /var/www/html/angrynates.com/christians/cart/simplemail/mail.py > > 153 > 154 ''' > 155 commitSale() > 156 myMail() > 157 print ''' > commitSale = > /var/www/html/angrynates.com/christians/cart/simplemail/mail.py > in commitSale() > 98 cursor.execute('select max(ID) from %sCustomerData;' % store) > 99 custID = cursor.fetchone()[0] > 100 customerData(store, tmpTable, custID, patientID) > 101 > 102 def myMail(): > global customerData = , global store = > 'products', tmpTable = 'tem12627568064', custID = 1, global patientID = > 'None' > /var/www/html/angrynates.com/christians/cart/customerData.py > in > customerData(store='products', tmpTable='tem12627568064', custID=1, > patientID='None') > 39 > 40 """ > 41 print """ > 42 print '

%s Customer Data

' % > (store[0].upper() + store[1:]) > 43 cursor.execute('describe %sCustomerData' % store) > store = 'products' > > ValueError: unsupported format character '(' (0x28) at index 54 > args = ("unsupported format character '(' (0x28) at index 54",) > > Apparently that character is a "file separator", which I presume is an > invisible character. I tried retyping the area in question, but with no > avail (threw same error). Please advise. Complete code follows. > Please provide the actual code (and, preferably, an easier-to-understand traceback, like what CPython does!). What you supplied isn't runnable. From fpm at u.washington.edu Wed Jan 6 11:29:19 2010 From: fpm at u.washington.edu (cassiope) Date: Wed, 6 Jan 2010 08:29:19 -0800 (PST) Subject: IOError - cannot create file (linux daemon-invoked script) References: Message-ID: <31a53b2f-45cc-45a2-8ae8-c7658e8a3949@j19g2000yqk.googlegroups.com> On Jan 5, 10:58?am, Nobody wrote: > On Mon, 04 Jan 2010 21:30:31 -0800, cassiope wrote: > > One more tidbit observed: my last note, that it works when using > > seteuid/setegid? > > Well - that only applies if the daemon is running under strace (!). > > It fails > > if started directly by root, or if the strace session has ended, > > leaving the > > main body of the daemon running in its normal headless manner. > > > I wonder if running under "strace -f" - might setegid/seteuid be > > prevented from > > having their normal effect? > > Possibly. The ptrace() syscall on which strace depends will fail if you > try to trace a "privileged" process and you aren't root, so it's possible > that a ptrace()d process will refuse to become privileged. > > Here, "privileged" includes a process which has changed any of its UIDs or > GIDs (this prevents a normal user from tracing, killing, etc an otherwise > privileged process which has switched to the user's UID for the time being). Thanks. I guess it's time to trim this beast to a simpler state to isolate what's going wrong. This may take awhile amidst everything else I have to be doing... From steve at REMOVE-THIS-cybersource.com.au Wed Jan 6 11:38:35 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 06 Jan 2010 16:38:35 GMT Subject: Introspection References: <0d782b7a-11fb-4933-84b7-29a3653ec847@p8g2000yqb.googlegroups.com> Message-ID: <0354a972$0$1336$c3e8da3@news.astraweb.com> On Wed, 06 Jan 2010 06:53:40 -0800, msj at infoserv.dk wrote: > I'm looking for a way to make a list of string literals in a class. > > Example: > > class A: > def method(self): > print 'A','BC' > >>>> ExtractLiterals(A) > ['A','BC'] > > Is this possible? Can anyone point me in the right direction? class A: def extract_literals(self): return "A BC".split() def method(self): print self.extract_literals() a = A() a.extract_literals() -- Steven From peter.bienstman at gmail.com Wed Jan 6 11:39:08 2010 From: peter.bienstman at gmail.com (pbienst) Date: Wed, 6 Jan 2010 08:39:08 -0800 (PST) Subject: creating tar file and streaming it over HTTP? References: <5222d923-7aef-47cf-a0d6-f37f09a66593@35g2000yqa.googlegroups.com> Message-ID: Thanks for the tip! It doesn't change anything, though, so I've debugged this a little bit further. The problem seems to be that the receiving end (wsgi server) does not see the end of the data: socket = environ["wsgi.input"] while True: sys.stderr.write("before") chunk = socket.read(4096) sys.stderr.write("after") if not chunk: sys.stderr.write("done") break sys.stderr.write(chunk) There is data from the tar file being printed, but in the end it hangs in the 'before' statement, and never gets to 'done'. I tried flushing the fileobj created by makefile(), but that doesn't seem to help. There is also no environ["CONTENT_LENGTH"] that I can use to detect the end of the stream. I'm probably missing something basic here... From victorsubervi at gmail.com Wed Jan 6 11:40:35 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 6 Jan 2010 12:40:35 -0400 Subject: TypeError In-Reply-To: <4B44BA50.1060203@mrabarnett.plus.com> References: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> <4B44BA50.1060203@mrabarnett.plus.com> Message-ID: <4dc0cfea1001060840r4f507f3bg1f71dfa97e903517@mail.gmail.com> On Wed, Jan 6, 2010 at 12:29 PM, MRAB wrote: > Victor Subervi wrote: > >> Hi; >> I get this error: >> >> /var/www/html/angrynates.com/christians/cart/simplemail/mail.py < >> http://angrynates.com/christians/cart/simplemail/mail.py> >> >> 153 >> 154 ''' >> 155 commitSale() >> 156 myMail() >> 157 print ''' >> commitSale = >> /var/www/html/angrynates.com/christians/cart/simplemail/mail.py < >> http://angrynates.com/christians/cart/simplemail/mail.py> in commitSale() >> >> 98 cursor.execute('select max(ID) from %sCustomerData;' % store) >> 99 custID = cursor.fetchone()[0] >> 100 customerData(store, tmpTable, custID, patientID) >> 101 >> 102 def myMail(): >> global customerData = , global store = 'products', >> tmpTable = 'tem12627568064', custID = 1, global patientID = 'None' >> /var/www/html/angrynates.com/christians/cart/customerData.py < >> http://angrynates.com/christians/cart/customerData.py> in >> customerData(store='products', tmpTable='tem12627568064', custID=1, >> patientID='None') >> >> 39 >> 40 """ >> 41 print """ >> 42 print '

%s Customer Data

' % >> (store[0].upper() + store[1:]) >> 43 cursor.execute('describe %sCustomerData' % store) >> store = 'products' >> >> ValueError: unsupported format character '(' (0x28) at index 54 >> args = ("unsupported format character '(' (0x28) at index 54",) >> >> Apparently that character is a "file separator", which I presume is an >> invisible character. I tried retyping the area in question, but with no >> avail (threw same error). Please advise. Complete code follows. >> >> Please provide the actual code (and, preferably, an easier-to-understand > traceback, like what CPython does!). What you supplied isn't runnable. > I ended up working around this problem another way. Thanks. beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From phlip2005 at gmail.com Wed Jan 6 11:41:29 2010 From: phlip2005 at gmail.com (Phlip) Date: Wed, 06 Jan 2010 08:41:29 -0800 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: Steve Holden wrote: > y'all just keep defending the approach to programming that > *you* think is best. Speak for yourself... From steve at holdenweb.com Wed Jan 6 11:42:21 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 11:42:21 -0500 Subject: Need help to pass self.count to other classes. In-Reply-To: <4b44af22$0$13292$426a74cc@news.free.fr> References: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> <4b44af22$0$13292$426a74cc@news.free.fr> Message-ID: Bruno Desthuilliers wrote: > Steve Holden a ?crit : > (snip) >> This is untested code > > indeed !-) > >> class kbInterface(object): >> def __init__(self): >> self.zxc = 0 >> def prompt1(self): >> self.count += 1 > > Ahem... > > (snip) > > Caveat emptor ... this code is worth what you paid for it :) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From sridharr at activestate.com Wed Jan 6 11:44:58 2010 From: sridharr at activestate.com (Sridhar Ratnakumar) Date: Wed, 06 Jan 2010 08:44:58 -0800 Subject: lxml 2.2.4 on python3.1, Windows XP gives importerror In-Reply-To: <7C57DB58C81FB64CA979EC6C4DB73E7A03E7039F@ZMY16EXM67.ds.mot.com> References: <7C57DB58C81FB64CA979EC6C4DB73E7A03E7039F@ZMY16EXM67.ds.mot.com> Message-ID: <4B44BE0A.7060508@activestate.com> On 1/5/2010 1:38 AM, VYAS ASHISH M-NTB837 wrote: > Dear All > I have Python 3.1 installed on Windows XP and Works nice. > I downloaded lxml 2.2.4 (lxml-2.2.4.win32-py3.1.exe) from pypi. > When I try: > from lxml import etree > I get: > ImportError: DLL load failed: This application has failed to start > because the application configuration is incorrect. Reinstalling the > application may fix this problem. Are you by any chance running 64-bit Python (whereas the lxml binary you have installed is 32-bit)? C:> python -c "import platform; print platform.architecture()" -srid From sccolbert at gmail.com Wed Jan 6 11:47:20 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Wed, 6 Jan 2010 17:47:20 +0100 Subject: how to change when the logging module creates the log file? In-Reply-To: <7f014ea61001060357r62567020wb82aa92940aa5763@mail.gmail.com> References: <7f014ea61001060357r62567020wb82aa92940aa5763@mail.gmail.com> Message-ID: <7f014ea61001060847j16fe8298xd1377a5caf4df0ad@mail.gmail.com> i was able to fix the exception by calling logging.shutdown() before the call to os.remove(). However, I still think there is probably a more elegant solution. On Wed, Jan 6, 2010 at 12:57 PM, Chris Colbert wrote: > I have an application the writes to a log file when specific exceptions are > handled. However, if no exceptions are encountered, I don't want to create a > log at all. > > The problem I am running into is that the stdlib logging module creates the > log file immediately upon logger instantiation. > > Thus: > >>> logger = logging.basicConifg('testlog.txt') > > already creates the file 'testlog.txt'. > > So, at program close, I am reading the size of the log file, and if it is > empty I remove it with os.remove(). This works fine on Linux, but throws a > permission denied exception on Windows. > > There has to be a better way to do this than using a hack like that. Is > there a way to make the logging module hold-off on file creation until the > first log is generated? I could do it by wrapping logger in a class, but > that would remove the beauty of having any module import logging from the > stdlib and being able to write to the log. > > Thanks for any pointers! > > Chris > -------------- next part -------------- An HTML attachment was scrubbed... URL: From james.b.looney at ulalaunch.com Wed Jan 6 11:48:44 2010 From: james.b.looney at ulalaunch.com (Looney, James B) Date: Wed, 6 Jan 2010 09:48:44 -0700 Subject: Python tk Listbox: -listvariable Message-ID: <8306B8561B5E1F4A97129B09E06D2473023C3F4F37@BROWN1.ad.ulalaunch.com> Yesterday, I searched all over trying to figure out how to properly use the "listvariable" argument with tk's Listbox class. Unfortunately, no amount of searching (online) could come up with anything more useful than telling me the variable needed to be a list, and nothing built-in exists. I finally came across a way of using it that I consider fairly simple, and wanted to toss it out there for everyone else. One post I saw mentioned using StringVar. That just didn't work well in my head, so instead I used its parent class: Variable. Apparently, the trick is to use tuples. If I use a list, string, etc - the Listbox seems to split the content based on spaces. Not the behavior I was looking for as I have spaces in the strings I wish to display. If anyone has better suggestions, I'd love to hear them. (I thought about a ListVar class, but haven't made it much beyond the thought). Regardless, I hope the following code helps others avoid the confusion I went through. Sample code: import Tkinter, tkSimpleDialog from Tkconstants import * class ListboxTest( tkSimpleDialog.Dialog ): ############################################################################# def __init__( self, master, tupleItems = () ): self.myVar = Tkinter.Variable() self.myVar.set( tupleItems ) tkSimpleDialog.Dialog.__init__( self, master, "Listbox testing" ) ############################################################################# def body( self, master ): lbox = Tkinter.Listbox( master, listvariable = self.myVar ) lbox.grid( row = 0, column = 0, sticky = N + W ) self.myVar.set( self.myVar.get() + tuple( [ "***** Final string being added *****" ] ) ) print type( self.myVar.get() ), self.myVar.get() if( "__main__" == __name__ ): tk = Tkinter.Tk() lt = ListboxTest( tk, tuple( [ "String 1 - with some spaces", "String 2 - with more spaces" ] ) ) -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Wed Jan 6 11:49:29 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 6 Jan 2010 12:49:29 -0400 Subject: getfirst and re Message-ID: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> Hi; I need to do something like the following: pat = re.compile('edit[0-9]*:[0-9]*') check = form.getfirst(pat) (to check things like 'edit0:1') How do I do this? TIA, beno -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From python.list at tim.thechases.com Wed Jan 6 12:27:50 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Wed, 06 Jan 2010 11:27:50 -0600 Subject: getfirst and re In-Reply-To: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> Message-ID: <4B44C816.2090201@tim.thechases.com> > I need to do something like the following: > > pat = re.compile('edit[0-9]*:[0-9]*') > check = form.getfirst(pat) > > (to check things like 'edit0:1') How do I do this? Well, you can do it either as check = pat.search(string_to_search) which is pretty plainly detailed in the help for the "re" module, both under the search() function and under the "8.2.2 Matching vs. Searching" section. Alternatively, if you plan to get the others too, you can use fi = pat.finditer(string_to_search) check = fi.next().group(0) # beware this may throw # a StopIteration if there are no more matches. which would usually be done inside a loop where the StopIteration does the Right Thing(tm). But if you're using it on HTML form text, regexps are usually the wrong tool, and you should be using an HTML parser (such as BeautifulSoup) that knows how to handle odd text and escapings better and more robustly than regexps will. -tkc From mail.to.daniel.platz at googlemail.com Wed Jan 6 12:28:45 2010 From: mail.to.daniel.platz at googlemail.com (Daniel Platz) Date: Wed, 6 Jan 2010 09:28:45 -0800 (PST) Subject: Pass multidimensional array (matrix) to c function using ctypes Message-ID: <63ac1a01-8491-4885-bae7-cb884abb582f@34g2000yqp.googlegroups.com> Hello, I would like to pass a two dimensional array to C function in a dll. I use ctypes to call the function. I compile the dll with visual studio 2008 express and my C source code looks like this. #include #ifdef __cplusplus extern "C" { // only need to export C interface if // used by C++ source code using namespace std; #endif __declspec(dllexport) int print(double** ptr, int ny, int nx) { int i, j; for(i=0; i Furthermore, I am wondering if there is a fast way to use a numpy 2D array instead or alternatively to cast the ctypes array into a numpy array. Has someone an idea to help me? Thank you very much Daniel From victorsubervi at gmail.com Wed Jan 6 12:31:41 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 6 Jan 2010 13:31:41 -0400 Subject: getfirst and re In-Reply-To: <4B44C816.2090201@tim.thechases.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> Message-ID: <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> On Wed, Jan 6, 2010 at 1:27 PM, Tim Chase wrote: > But if you're using it on HTML form text, regexps are usually the wrong > tool, and you should be using an HTML parser (such as BeautifulSoup) that > knows how to handle odd text and escapings better and more robustly than > regexps will. > I have an automatically generated HTML form from which I need to extract data to the script which this form calls (to which the information is sent). I believe BeautifulSoup is geared to scraping pages that exist permanently on the web. By the time BeautifulSoup was called, this page would be gone. Any other ideas? TIA beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From nobody at nowhere.com Wed Jan 6 12:56:24 2010 From: nobody at nowhere.com (Nobody) Date: Wed, 06 Jan 2010 17:56:24 +0000 Subject: Printing plain text with exact positioning on Windows References: <725ca37c-e975-470e-b7cc-84e3bbc1d988@e27g2000yqd.googlegroups.com> <54eb4786-5db3-453d-971a-a4b359fc2de8@f5g2000yqh.googlegroups.com> <1f38f91e-c5f2-408f-81ae-f4ef9b877e0d@o28g2000yqh.googlegroups.com> <1b6c0a22-e786-4203-a744-7fb0575a8add@s3g2000yqs.googlegroups.com> <96ea39f2-1fb6-4b03-b206-d4c1445ea021@v25g2000yqk.googlegroups.com> Message-ID: On Tue, 05 Jan 2010 11:40:25 -0800, KvS wrote: >> "Hardcopy" document formats such as PostScript and PDF use positions >> relative to the edges of the page, not the margins. > > Right. Still, Acrobat Reader by default scales the contents to fit on > a page and creates some margins by doing so, no? So if my text is > close to the left and right edges, as I want, it will get scaled and > extra margins will occur. Avoiding this still requires me to be able > to turn off this scaling in the printing preferences somehow > programmatically, so it doesn't seem to make the problem easier? If the document is the same size as the physical page, it will be transferred directly without any scaling or offset. The document will not be scaled to fit the printable area; if the document contains any marks which lie within the printer's margins, those marks won't appear on the printed page. PostScript and PDF documents don't have "margins". There might be an area around the edge of the page which doesn't contain any marks, but that's irrelevant; the area is still part of the page. From nobody at nowhere.com Wed Jan 6 12:59:40 2010 From: nobody at nowhere.com (Nobody) Date: Wed, 06 Jan 2010 17:59:40 +0000 Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> Message-ID: On Tue, 05 Jan 2010 12:20:58 -0800, Marco Nawijn wrote: > You could use the build-in function enumerate inside a list > comprehension. > >>>> seq = range(5) >>>> [ (i,s) for i,s in enumerate(seq) ] > [(0, 0), (1, 1), (2, 2), (3, 3), (4, 4)] Just use list(), i.e. "list(enumerate(seq))". From python.list at tim.thechases.com Wed Jan 6 12:59:52 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Wed, 06 Jan 2010 11:59:52 -0600 Subject: getfirst and re In-Reply-To: <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> Message-ID: <4B44CF98.2090702@tim.thechases.com> Victor Subervi wrote: > On Wed, Jan 6, 2010 at 1:27 PM, Tim Chase wrote: > >> But if you're using it on HTML form text, regexps are usually the wrong >> tool, and you should be using an HTML parser (such as BeautifulSoup) that >> knows how to handle odd text and escapings better and more robustly than >> regexps will > > I have an automatically generated HTML form from which I need to extract > data to the script which this form calls (to which the information is sent). > I believe BeautifulSoup is geared to scraping pages that exist permanently > on the web. By the time BeautifulSoup was called, this page would be gone. BeautifulSoup takes string data fed to it, and builds a structure that can be neatly navigated. That string data can come from a web page, from a disk, or even a serial port, a random-character-generator, or just from HTML that's built up in memory and never sees a network or a disk. It's worth reading its documentation[1] and trying its examples to get familiar with it. -tkc [1] http://www.crummy.com/software/BeautifulSoup/documentation.html From thimxx at gmail.com Wed Jan 6 13:00:25 2010 From: thimxx at gmail.com (Valentin de Pablo Fouce) Date: Wed, 6 Jan 2010 10:00:25 -0800 (PST) Subject: File transfer with python Message-ID: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> Hi there, I hope this is the rigth place, if not please, tell me which is the right dicussion place. I apologize in such case. Ok, I am trying to do a very quick application (is "home based" so is not a big deal...). My intention is to transfer files from one computer to another. I am using several OS (Linux and Windows basicly, but potentially in the future Mac too), and the computers won't be potentially in the same network (ok, the most times will... but there will be exceptions). My intention is to be able to transfer files from one computer to another in this environment. Looking (and surfing) at internet the only suggestion given is to use low level sockets for this file transfer. Is there another way to do it, is there any top level library that helps you to do that? That's a lot! From mudit.tuli at gmail.com Wed Jan 6 13:01:46 2010 From: mudit.tuli at gmail.com (mudit tuli) Date: Wed, 6 Jan 2010 23:31:46 +0530 Subject: 3 byte network ordered int, How To ? Message-ID: <3d4c0f161001061001ma900949j1fdea2789cff7367@mail.gmail.com> For a single byte, struct.pack(') For two bytes, struct.pack(') what if I want three bytes ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Wed Jan 6 13:04:41 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 6 Jan 2010 14:04:41 -0400 Subject: getfirst and re In-Reply-To: <4B44CF98.2090702@tim.thechases.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4B44CF98.2090702@tim.thechases.com> Message-ID: <4dc0cfea1001061004u63ba8c17xca03c682b2d7b7fc@mail.gmail.com> On Wed, Jan 6, 2010 at 1:59 PM, Tim Chase wrote: > Victor Subervi wrote: > >> On Wed, Jan 6, 2010 at 1:27 PM, Tim Chase > >wrote: >> >> But if you're using it on HTML form text, regexps are usually the wrong >>> tool, and you should be using an HTML parser (such as BeautifulSoup) that >>> knows how to handle odd text and escapings better and more robustly than >>> regexps will >>> >> >> I have an automatically generated HTML form from which I need to extract >> data to the script which this form calls (to which the information is >> sent). >> I believe BeautifulSoup is geared to scraping pages that exist permanently >> on the web. By the time BeautifulSoup was called, this page would be gone. >> > > BeautifulSoup takes string data fed to it, and builds a structure that can > be neatly navigated. That string data can come from a web page, from a > disk, or even a serial port, a random-character-generator, or just from HTML > that's built up in memory and never sees a network or a disk. It's worth > reading its documentation[1] and trying its examples to get familiar with > it. > k. Thanks. beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From pavlovevidence at gmail.com Wed Jan 6 13:08:21 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Wed, 6 Jan 2010 10:08:21 -0800 (PST) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> Message-ID: <0be1d648-07ac-4662-950f-7feb24ec713c@c3g2000yqd.googlegroups.com> On Jan 5, 2:40?pm, Phlip wrote: > On Jan 5, 1:10?pm, Antoine Pitrou wrote: > > >http://docs.python.org/library/functions.html > > > Don't forget that the Python documentation is rich and structured. > > And good luck. > > Does it say how to convert a string containing either an integer > representation, or something alphabetic, into an integer, or a zero, > in like 1 method call? No, as he said, it only describes the most important functions. Carl Banks From nobody at nowhere.com Wed Jan 6 13:10:36 2010 From: nobody at nowhere.com (Nobody) Date: Wed, 06 Jan 2010 18:10:36 +0000 Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> <1715bea7-1ff1-4294-9dad-672873e14530@v25g2000yqk.googlegroups.com> Message-ID: On Tue, 05 Jan 2010 19:46:01 -0800, alex23 wrote: >> They will tell me how to use except: (which is a good example why a >> program should not use exceptions for its normal control flow if at >> all possible). > > Really? Magic functions that coerce and eat errors are a better coding > technique than exceptions and explicit handling? > > What kool-aid have you been drinking? Maybe he's doing it for a living? Contract programming seems to work on the basis that the ultimate requirement is for the client to hand over the money. If, at that point, the program is still full of bugs, you get to charge extra for "upgrades". Writing robust software from the outset puts you at a competitive disadvantage to those who understand how the system works. From shawn at milochik.com Wed Jan 6 13:15:31 2010 From: shawn at milochik.com (Shawn Milochik) Date: Wed, 6 Jan 2010 13:15:31 -0500 Subject: File transfer with python In-Reply-To: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> References: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> Message-ID: <90C09BF4-48D9-4245-A967-5DFABE2A64BD@milochik.com> Have a look at Paramiko. It lets you do secure transfers easily (scp/sftp) http://www.lag.net/paramiko/ Shawn From lie.1296 at gmail.com Wed Jan 6 13:20:41 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Thu, 07 Jan 2010 05:20:41 +1100 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> Message-ID: <4b44d479$1@dnews.tpgi.com.au> On 1/7/2010 2:12 AM, Phlip wrote: > On Jan 5, 10:54 pm, Benjamin Kaplan wrote: > >> {41: None}[41] ? >> >> In cases where None is a valid result, you can't use it to signal failure.. > > Asked and answered. You change the "sentinel" in .fetch to something > else. I believe Ben Kaplan's point is that if dict slicing returns sentinel for missing keys, the slicing syntax would be unusable to differentiate {41: sentinel} and {}; if the default sentinel had been None, that would make it too easy to forget to reset the sentinel and wrote a buggy code. > But y'all keep on defending the language making your programming > decisions for you! When designing a language, there are two different ways to signal a missing key in a dict: 1) raise an exception or 2) return a sentinel. Both are useful for different purpose. One or the other must be the default behavior, and the other must give way. Python decided that the default behavior should be raising exception and sentinel have to use the dict.get() method. Simple and clear. The other possible behavior (i.e. slicing returns a sentinel while dict.get() raises an exception) is arguably just as simple and just as clear; but python doesn't do it that way. Why? Because it's part of what makes python Python[1]. > But y'all keep on defending the language making your programming > decisions for you! Indeed, the language makes decisions for you; why would you want to use a language that nags you for every possible different behavior? I want a reasonable default and I want a reasonably easy way to access the non-default behaviors. [1] the essence of which is emboldened as the Zen, which is the broad design guideline for both python's syntax and codes written in python. Specifically, "Errors should never pass silently" because missing key indicates a possible error and "Simple is better than complex" because returning sentinel for missing keys implies the need for an interface to change the sentinel[2]. [2] or become like Java, whose HashMap cannot reliably store null and doesn't allow you to specify the sentinel. From lie.1296 at gmail.com Wed Jan 6 13:23:04 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Thu, 07 Jan 2010 05:23:04 +1100 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> Message-ID: <4b44d507$1@dnews.tpgi.com.au> On 1/7/2010 3:41 AM, Phlip wrote: > Steve Holden wrote: > >> y'all just keep defending the approach to programming that >> *you* think is best. > > Speak for yourself... Everyone speaks for themselves, is that a problem? From nobody at nowhere.com Wed Jan 6 13:30:03 2010 From: nobody at nowhere.com (Nobody) Date: Wed, 06 Jan 2010 18:30:03 +0000 Subject: subprocess.Popen does not close pipe in an error case References: <07641018-9581-44af-bb20-8f9b96a44c4f@p8g2000yqb.googlegroups.com> Message-ID: On Tue, 05 Jan 2010 15:50:39 -0800, Steven K. Wong wrote: > Below, I have a Python script that launches 2 child programs, prog1 > and prog2, with prog1's stdout connected to prog2's stdin via a pipe. > (It's like executing "prog1 | prog2" in the shell.) > > If both child programs exit with 0, then the script runs to > completion. But if prog2 exits with non-0, prog1 does not exit and the > script hangs (i.e. prog1.poll() always returns None) -- unless I > uncomment the 2 lines marked by XXX to close prog1.stdout. > > I was expecting that I don't have to explicitly close prog1.stdout, > whether prog2 succeeds or fails. Is the current behavior a bug in the > subprocess module or is it expected? Or am I doing something wrong? > > Thanks. > > import subprocess > import time > > # prog1: a program that writes lots of data to the pipe > cmd = ['zcat', '--force', 'a_large_file'] > prog1 = subprocess.Popen(cmd, bufsize=-1, stdout=subprocess.PIPE) > > # prog2: a program that fails without reading much data from the pipe > cmd = ['python', '-c', 'import time; time.sleep(10); asdf'] > prog2 = subprocess.Popen(cmd, bufsize=-1, stdin=prog1.stdout, > stdout=open('popen.out', 'w')) I think that you should close prog1.stdout here. Otherwise, there will be two readers on the pipe (the calling process and prog2). Even if one of them dies, there's always the possibility that the caller might eventually decide to read prog1.stdout itself. If you close it in the caller, when prog2 terminates there will be no readers, and prog1 will get SIGPIPE (or write() will fail with EPIPE if SIGPIPE is handled). From no.i.dont at want.mail.from.spammers.com Wed Jan 6 13:33:15 2010 From: no.i.dont at want.mail.from.spammers.com (Fencer) Date: Wed, 06 Jan 2010 19:33:15 +0100 Subject: suds problem Message-ID: <7qk3bcFnbjU1@mid.individual.net> Hello, I just started using suds to use web services. First I tried suds with a very simple web service I had written and was running myself. That worked fine. Then I tried to use the web services provided by KEGG: http://soap.genome.jp/KEGG.wsdl But I get a SAXParseException due to a supposed mis-matched tag when I try to consume that wsdl with suds. I checked the wsdl in oxygene and it claims it's valid. What is the problem here? My test program is below and it's very simple: from suds.client import Client url = 'http://soap.genome.jp/KEGG.wsdl' client = Client(url) print client - Fencer From mrabarnett at mrabarnett.plus.com Wed Jan 6 13:33:19 2010 From: mrabarnett at mrabarnett.plus.com (Matthew Barnett) Date: Wed, 06 Jan 2010 18:33:19 +0000 Subject: 3 byte network ordered int, How To ? In-Reply-To: <3d4c0f161001061001ma900949j1fdea2789cff7367@mail.gmail.com> References: <3d4c0f161001061001ma900949j1fdea2789cff7367@mail.gmail.com> Message-ID: <4B44D76F.3010200@mrabarnett.plus.com> mudit tuli wrote: > For a single byte, struct.pack(') > For two bytes, struct.pack(') > what if I want three bytes ? > Four bytes and then discard the most-significant byte: struct.pack(')[ : -1] From hjtoi-better-remove-before-reply at comcast.net Wed Jan 6 13:34:11 2010 From: hjtoi-better-remove-before-reply at comcast.net (Heikki Toivonen) Date: Wed, 06 Jan 2010 10:34:11 -0800 Subject: python xmlrpc client with ssl client certificates and standard modules References: <4b414e79$0$23321$426a34cc@news.free.fr> Message-ID: News123 wrote: > This will probably work, but it requires the module M2Crypto. > > In order to avoid installing M2Crypto an all hosts that want to run the > script I wondered, whether there is no other solution. > > I can do xmlrpc over ssl WITHOUT certificates with following code: [...] Please note that if you just use the stdlib it is not secure out of the box. With Python 2.6 and the ssl module you can make it so, but it requires some work on your part. See for example http://www.heikkitoivonen.net/blog/2008/10/14/ssl-in-python-26/ -- Heikki Toivonen - http://heikkitoivonen.net From lie.1296 at gmail.com Wed Jan 6 13:48:04 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Thu, 07 Jan 2010 05:48:04 +1100 Subject: File transfer with python In-Reply-To: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> References: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> Message-ID: <4b44dae4$1@dnews.tpgi.com.au> On 1/7/2010 5:00 AM, Valentin de Pablo Fouce wrote: > My intention is to be able to transfer files from one computer to > another in this environment. Do you have a USB flashdrive? > Looking (and surfing) at internet the only suggestion given is to use > low level sockets for this file transfer. Is there another way to do > it, is there any top level library that helps you to do that? Networked file system (e.g. Samba/SMB), File Transfer Protocol (FTP), whatever works for you. pysamba is available from sourceforge and there is ftplib in the standard library. For most purposes though, you could just setup a network shared directory and drag and drop files from the file manager. From dreadpiratejeff at gmail.com Wed Jan 6 14:01:29 2010 From: dreadpiratejeff at gmail.com (J) Date: Wed, 6 Jan 2010 14:01:29 -0500 Subject: Python books, literature etc In-Reply-To: <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> Message-ID: <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> A good point was brought up to me privately, and I agree completely, that the OP should re-state the request with a bit more specifics... Since the OP says he is at least familiar with Python, does he need info on beginner level books that are general purpose, or is he interested in resources that are more specific (e.g. geared toward web programming, mathematical analysis, data modeling, etc).... My suggestions were meant just as an example of what I use in the course of learning something at the basic and intermediate level, once it goes beyond that, it's useful to know WHAT you intend to do so you can find the right resources to go in that direction. Just want to make sure I am not misunderstood or anything :-) For what it's worth, I also tend to collect technical books for some reason... My wife is just barely tolerant of my bookshelf full of things on various computer topics, astronomy, photography, radio and antenna theory and so forth ;-) I just let her keep her shoe collection, and we have a quid pro quo. Cheers Jeff -- Mike Ditka - "If God had wanted man to play soccer, he wouldn't have given us arms." - http://www.brainyquote.com/quotes/authors/m/mike_ditka.html From no.i.dont at want.mail.from.spammers.com Wed Jan 6 14:02:58 2010 From: no.i.dont at want.mail.from.spammers.com (Fencer) Date: Wed, 06 Jan 2010 20:02:58 +0100 Subject: suds problem In-Reply-To: <7qk3bcFnbjU1@mid.individual.net> References: <7qk3bcFnbjU1@mid.individual.net> Message-ID: <7qk532F25jU1@mid.individual.net> On 2010-01-06 19:33, Fencer wrote: > Hello, I just started using suds to use web services. First I tried suds > with a very simple web service I had written and was running myself. > That worked fine. Then I tried to use the web services provided by KEGG: > http://soap.genome.jp/KEGG.wsdl > But I get a SAXParseException due to a supposed mis-matched tag when I > try to consume that wsdl with suds. I checked the wsdl in oxygene and it > claims it's valid. What is the problem here? My test program is below > and it's very simple: > from suds.client import Client > > url = 'http://soap.genome.jp/KEGG.wsdl' > > client = Client(url) > > print client > > - Fencer Same problem with reactome: http://www.reactome.org:8080/caBIOWebApp/services/caBIOService?wsdl I suppose I'm mis-using suds, maybe some setting that wasn't needed for my simple web service but is needed for kegg and reactome. - Fencer From carsten.haese at gmail.com Wed Jan 6 14:09:38 2010 From: carsten.haese at gmail.com (Carsten Haese) Date: Wed, 06 Jan 2010 14:09:38 -0500 Subject: getfirst and re In-Reply-To: <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> Message-ID: Victor Subervi wrote: > I have an automatically generated HTML form from which I need to extract > data to the script which this form calls (to which the information is > sent). Ideally, the script that receives the submitted fields should know how the form was generated, so it knows what fields to expect. Failing that, you should realize that getfirst() is not the only way to access the contents of a cgi.FieldStorage object. If you need to know the values of all fields whose names obey a certain pattern, I recommend you start with a list of all field names in the FieldStorage object and pick out the ones whose names obey that pattern. To get the list of field names, you should consider using the .keys() method of the FieldStorage object. -Carsten From victorsubervi at gmail.com Wed Jan 6 14:19:40 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 6 Jan 2010 15:19:40 -0400 Subject: getfirst and re In-Reply-To: References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> Message-ID: <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> On Wed, Jan 6, 2010 at 3:09 PM, Carsten Haese wrote: > Victor Subervi wrote: > > I have an automatically generated HTML form from which I need to extract > > data to the script which this form calls (to which the information is > > sent). > > Ideally, the script that receives the submitted fields should know how > the form was generated, so it knows what fields to expect. Failing that, > you should realize that getfirst() is not the only way to access the > contents of a cgi.FieldStorage object. > > If you need to know the values of all fields whose names obey a certain > pattern, I recommend you start with a list of all field names in the > FieldStorage object and pick out the ones whose names obey that pattern. > To get the list of field names, you should consider using the .keys() > method of the FieldStorage object. > Cool. Thanks! beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From wolftracks at invalid.com Wed Jan 6 14:25:40 2010 From: wolftracks at invalid.com (W. eWatson) Date: Wed, 06 Jan 2010 11:25:40 -0800 Subject: Astronomy--Programs to Compute Siderial Time? Message-ID: Is there a smallish Python library of basic astronomical functions? There are a number of large such libraries that are crammed with excessive functions not needed for common calculations. From jcb at iteris.com Wed Jan 6 14:26:41 2010 From: jcb at iteris.com (Metalone) Date: Wed, 6 Jan 2010 11:26:41 -0800 (PST) Subject: Need help with multiprocessing.manager and passing the manager a multiprocessing.Connection Message-ID: <350adac0-c6ba-4861-8757-e1717ad8e636@c34g2000yqn.googlegroups.com> The following code snippet is taken from the Python 2.6 multiprocessing documentation with a simple change and this change does not work. I would like to know how to make it work or something similar. I want to pass a Connection object to the MathsClass. I get the following error on Windows: Traceback (most recent call last): File "c:\apps\python26\lib\multiprocessing\managers.py", line 214, in serve_cl ient request = recv() TypeError: Required argument 'handle' (pos 1) not found from multiprocessing.managers import BaseManager from multiprocessing import Pipe # I added this class MathsClass(object): def set(self, conn): # I added the set() function self.conn = conn def add(self, x, y): return x + y def mul(self, x, y): return x * y class MyManager(BaseManager): pass MyManager.register('Maths', MathsClass) if __name__ == '__main__': parent_conn, child_conn = Pipe() # I added this manager = MyManager() manager.start() maths = manager.Maths() print maths.add(4, 3) # prints 7 print maths.mul(7, 8) # prints 56 maths.set(child_conn) # I added this, error is thrown here From skwslide at gmail.com Wed Jan 6 14:39:37 2010 From: skwslide at gmail.com (Steven K. Wong) Date: Wed, 6 Jan 2010 11:39:37 -0800 (PST) Subject: subprocess.Popen does not close pipe in an error case References: <07641018-9581-44af-bb20-8f9b96a44c4f@p8g2000yqb.googlegroups.com> Message-ID: On Jan 6, 10:30 am, Nobody wrote: > I think that you should close prog1.stdout here. Otherwise, there will > be two readers on the pipe (the calling process and prog2). Even if one of > them dies, there's always the possibility that the caller might eventually > decide to read prog1.stdout itself. If you close it in the caller, when > prog2 terminates there will be no readers, and prog1 will get SIGPIPE (or > write() will fail with EPIPE if SIGPIPE is handled). Thanks for raising a great point, that prog1.stdout is also readable by the calling process, not just by prog2. Therefore, I agree it makes sense to explicitly call prog1.stdout.close() in the given code (say right after the creation of prog2). Suppose now all the prog1.poll() calls/loop are replaced by a single prog1.wait(). Without the explicit prog1.stdout.close(), prog1.wait() will not return, so the calling process still hangs. Because calling prog1.wait() means that the calling process will naturally never read prog1.stdout, I would argue that prog1.wait() should close the pipe before actually waiting for prog1 to exit. Makes sense? From invalid at invalid.invalid Wed Jan 6 14:51:38 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Wed, 6 Jan 2010 19:51:38 +0000 (UTC) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4b44d507$1@dnews.tpgi.com.au> Message-ID: On 2010-01-06, Lie Ryan wrote: > On 1/7/2010 3:41 AM, Phlip wrote: >> Steve Holden wrote: >> >>> y'all just keep defending the approach to programming that >>> *you* think is best. >> >> Speak for yourself... > > Everyone speaks for themselves, [...] Except for the Lorax. He speaks for the trees. -- Grant Edwards grante Yow! The SAME WAVE keeps at coming in and COLLAPSING visi.com like a rayon MUU-MUU ... From van.lindberg at gmail.com Wed Jan 6 15:01:10 2010 From: van.lindberg at gmail.com (VanL) Date: Wed, 06 Jan 2010 14:01:10 -0600 Subject: The END (of PyCon early bird registration) is NEAR! Message-ID: Today is the last day of registration for PyCon 2010 at the early bird rate. Registration at the early bird rate is still good as long as it is January 6 somewhere in the world. Register now! - https://us.pycon.org/2010/register/ From phlip2005 at gmail.com Wed Jan 6 15:12:08 2010 From: phlip2005 at gmail.com (Phlip) Date: Wed, 06 Jan 2010 12:12:08 -0800 Subject: please help shrink this each_with_index() implementation In-Reply-To: References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> <1715bea7-1ff1-4294-9dad-672873e14530@v25g2000yqk.googlegroups.com> Message-ID: Nobody wrote: > On Tue, 05 Jan 2010 19:46:01 -0800, alex23 wrote: > >>> They will tell me how to use except: (which is a good example why a >>> program should not use exceptions for its normal control flow if at >>> all possible). >> Really? Magic functions that coerce and eat errors are a better coding >> technique than exceptions and explicit handling? >> >> What kool-aid have you been drinking? > > Maybe he's doing it for a living? > > Contract programming seems to work on the basis that the ultimate > requirement is for the client to hand over the money. If, at that point, > the program is still full of bugs, you get to charge extra for "upgrades". Uh, no, right now I'm working with the long-term goal of creating useful modules that sustain us equitably. > Writing robust software from the outset puts you at a competitive > disadvantage to those who understand how the system works. And I, not my language, should pick and chose how to be rigorous. The language should not make the decision for me. From yoavglazner at gmail.com Wed Jan 6 15:15:45 2010 From: yoavglazner at gmail.com (Glazner) Date: Wed, 6 Jan 2010 12:15:45 -0800 (PST) Subject: an't start a thread Pool from another thread Message-ID: <47ceb43c-a219-4c5d-ad45-6ee1a96094b6@v25g2000yqk.googlegroups.com> Hi all, I hope someone can help me with this issue I see that i can't start a thread Pool from another thread, why? running python 2.6.4 windowsXP >>> import multiprocessing.dummy as threads >>> def makePool(): threads.Pool(3) >>> makePool() >>> import thread >>> thread.start_new(makePool,()) Unhandled exception in thread started by 6960 From phlip2005 at gmail.com Wed Jan 6 15:39:36 2010 From: phlip2005 at gmail.com (Phlip) Date: Wed, 6 Jan 2010 12:39:36 -0800 (PST) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4b44d507$1@dnews.tpgi.com.au> Message-ID: <8f6af7ac-35f6-4e22-b69b-7c293694be26@j14g2000yqm.googlegroups.com> On Jan 6, 10:23?am, Lie Ryan wrote: > On 1/7/2010 3:41 AM, Phlip wrote: > > > Steve Holden wrote: > > >> y'all just keep defending the approach to programming that > >> *you* think is best. > > > Speak for yourself... > > Everyone speaks for themselves, is that a problem? Of course not. I was pointing out that Steve is telling me not to force my programming opinions on everyone... ...while defending Python enforcing programming opinions on everyone. And now, if everyone will excuse me, I have to get back to writing a unit-test-to-code ratio of 2:1. Have fun being rigorous, everyone! From adityashukla1983 at gmail.com Wed Jan 6 15:44:52 2010 From: adityashukla1983 at gmail.com (aditya shukla) Date: Wed, 6 Jan 2010 14:44:52 -0600 Subject: Mencoder and creating videos Message-ID: <73045cca1001061244y61cc020fj5ea2c633df98d03f@mail.gmail.com> Hello Guys, I have a multiprocessing script which downloads images from 5 urls to 5 directories(usinf multiprocess in python 2.6).The download is for 5 mins.My aim is to create a video for every minute for each directory and dump the images as the video is created. My question are , should i use *mencoder.exe * to create the videos? and how should i create the videos without interrupting the download. Thanks Aditya -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Wed Jan 6 15:49:37 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 06 Jan 2010 15:49:37 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <4b44d479$1@dnews.tpgi.com.au> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <6bb7d00c-b4b3-4c25-b943-4360b850cd68@l30g2000yqb.googlegroups.com> <4b44d479$1@dnews.tpgi.com.au> Message-ID: On 1/6/2010 1:20 PM, Lie Ryan wrote: > Python decided that the default behavior should be raising exception and > sentinel have to use the dict.get() method. Simple and clear. The other > possible behavior (i.e. slicing returns a sentinel while dict.get() > raises an exception) is arguably just as simple and just as clear; but > python doesn't do it that way. Why? Because it's part of what makes > python Python[1]. The altermatives are not quite equivalent. The current way lets one specify the sentinel whereas the alternative does not. There is hardly any reason to specify the exception. tjr From steve at REMOVE-THIS-cybersource.com.au Wed Jan 6 16:14:14 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 06 Jan 2010 21:14:14 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4b44d507$1@dnews.tpgi.com.au> <8f6af7ac-35f6-4e22-b69b-7c293694be26@j14g2000yqm.googlegroups.com> Message-ID: <0354ea0c$0$1336$c3e8da3@news.astraweb.com> On Wed, 06 Jan 2010 12:39:36 -0800, Phlip wrote: > And now, if everyone will excuse me, I have to get back to writing a > unit-test-to-code ratio of 2:1. In my experience, that's about half as many unit-tests as needed for full code coverage for even a simple class. If you're trying to impress us, you have failed. > Have fun being rigorous, everyone! You say that as if writing correct code was a bad thing. -- Steven From jason.scheirer at gmail.com Wed Jan 6 16:17:49 2010 From: jason.scheirer at gmail.com (Jason Scheirer) Date: Wed, 6 Jan 2010 13:17:49 -0800 (PST) Subject: Introspection References: <0d782b7a-11fb-4933-84b7-29a3653ec847@p8g2000yqb.googlegroups.com> <0354a972$0$1336$c3e8da3@news.astraweb.com> Message-ID: <9af6ffd9-a450-4eef-90ed-430aeea9f77c@j24g2000yqa.googlegroups.com> On Jan 6, 8:38?am, Steven D'Aprano wrote: > On Wed, 06 Jan 2010 06:53:40 -0800, m... at infoserv.dk wrote: > > I'm looking for a way to make a list of string literals in a class. > > > Example: > > > class A: > > ? ?def method(self): > > ? ? ? ?print 'A','BC' > > >>>> ExtractLiterals(A) > > ['A','BC'] > > > Is this possible? Can anyone point me in the right direction? > > class A: > ? ? def extract_literals(self): > ? ? ? ? return "A BC".split() > ? ? def method(self): > ? ? ? ? print self.extract_literals() > > a = A() > a.extract_literals() > > -- > Steven Slightly more robust than Miki's solution insofar as it doesn't require the source to exist in a .py file: import types def extract_literals(klass): for attr in (getattr(klass, item) for item in dir(klass)): if isinstance(attr, types.MethodType): for literal in attr.im_func.func_code.co_consts: if isinstance(literal, basestring): yield literal class full_of_strings(object): def a(self): return "a", "b", "c" def b(self): "y", "z" print list(extract_literals(full_of_strings)) ['a', 'b', 'c', 'y', 'z'] print list(extract_literals(full_of_strings())) ['a', 'b', 'c', 'y', 'z'] Note that this is evil and should be avoided. From steve at REMOVE-THIS-cybersource.com.au Wed Jan 6 16:20:40 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 06 Jan 2010 21:20:40 GMT Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> <1715bea7-1ff1-4294-9dad-672873e14530@v25g2000yqk.googlegroups.com> Message-ID: <0354eb8e$0$1336$c3e8da3@news.astraweb.com> On Wed, 06 Jan 2010 12:12:08 -0800, Phlip wrote: > And I, not my language, should pick and chose how to be rigorous. The > language should not make the decision for me. All languages make that decision for you by making some thing possible and other things not. The language designer, not the programmer, decides what syntactic features to allow and disallow, what programming styles to include, and what functionality to provide as primitives. -- Steven From zabin.farishta at gmail.com Wed Jan 6 16:23:39 2010 From: zabin.farishta at gmail.com (Zabin) Date: Wed, 6 Jan 2010 13:23:39 -0800 (PST) Subject: QDoubleValidator Message-ID: Hey! I am new PyQt programmer and want to restrict users to allow only numeric values into a table and lineedit boxes. I found the QDoubleValidator class but am unsure as to how to implement it. (I am a little shaky on the concept of parent and how to define them). Any help would be much appreciated! From steve at holdenweb.com Wed Jan 6 16:33:54 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 16:33:54 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: <8f6af7ac-35f6-4e22-b69b-7c293694be26@j14g2000yqm.googlegroups.com> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4b44d507$1@dnews.tpgi.com.au> <8f6af7ac-35f6-4e22-b69b-7c293694be26@j14g2000yqm.googlegroups.com> Message-ID: Phlip wrote: > On Jan 6, 10:23 am, Lie Ryan wrote: >> On 1/7/2010 3:41 AM, Phlip wrote: >> >>> Steve Holden wrote: >>>> y'all just keep defending the approach to programming that >>>> *you* think is best. >>> Speak for yourself... >> Everyone speaks for themselves, is that a problem? > > Of course not. I was pointing out that Steve is telling me not to > force my programming opinions on everyone... > > ...while defending Python enforcing programming opinions on everyone. > That's because Python is not a person, and Guido is a better language designer than both of us put together. No languagecan be all things to all programmers, and Python represents a set of pragmatic and useful choices. > And now, if everyone will excuse me, I have to get back to writing a > unit-test-to-code ratio of 2:1. Have fun being rigorous, everyone! Consider yourself excused ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From zuo at chopin.edu.pl Wed Jan 6 16:42:21 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Wed, 06 Jan 2010 22:42:21 +0100 Subject: File transfer with python In-Reply-To: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> References: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> Message-ID: Valentin de Pablo Fouce wrote: > Ok, I am trying to do a very quick application (is "home based" so is > not a big deal...). My intention is to transfer files from one > computer to another. > My intention is to be able to transfer files from one computer to > another in this environment. > > Looking (and surfing) at internet the only suggestion given is to use > low level sockets for this file transfer. Is there another way to do > it, is there any top level library that helps you to do that? Python standard library offers tools for HTTP communication (rather easy to use) -- see: http://docs.python.org/library/simplehttpserver.html http://docs.python.org/library/cgihttpserver.html http://docs.python.org/library/basehttpserver.html -- as well as: http://docs.python.org/library/urllib.html http://docs.python.org/library/urllib2.html Cheers, *j -- Jan Kaliszewski (zuo) From aioe.org at technicalbloke.com Wed Jan 6 16:52:52 2010 From: aioe.org at technicalbloke.com (r0g) Date: Wed, 06 Jan 2010 21:52:52 +0000 Subject: creating tar file and streaming it over HTTP? References: <5222d923-7aef-47cf-a0d6-f37f09a66593@35g2000yqa.googlegroups.com> Message-ID: pbienst wrote: > I would like to bundle up a number of files in a tar file and send it > over a HTTP connection, but I would like to do this without creating > the tar file on disk first. > Stringio lets you treat a strings as a files... http://docs.python.org/library/stringio.html Roger. From trzewiczek at trzewiczek.info Wed Jan 6 16:53:09 2010 From: trzewiczek at trzewiczek.info (trzewiczek at trzewiczek.info) Date: Wed, 06 Jan 2010 22:53:09 +0100 Subject: GUI for multiplatform multimedia project Message-ID: Hi everyone, I posted that question on a python-forum, but got answer, so I ask here. I'm working on an artistic project and I'm looking for the best cross-platform GUI solution. The problem is that it's gonna be a tool that will have to be double-click installable/runnable and pre-installation of any libraries for end-users is very much like an evil. It really has to be double-click tool My first thought was PyQt, because it's a real framework with a lot of stuff inside (including Phonon) and I know some cross-platform media software written in C++ QT (like VLC). But on the other hand I've heard that it's not that easy to make it "double-clicky" multi-platform. Is that true? Another thing that matters for me is ease of integration with libraries like OpenCV. I will be VERY thankful for any help. I'm SO tired googling the problem (it's like weeks now!!) Best from Poland, trzewiczek From emile at fenx.com Wed Jan 6 16:56:07 2010 From: emile at fenx.com (Emile van Sebille) Date: Wed, 06 Jan 2010 13:56:07 -0800 Subject: Mencoder and creating videos In-Reply-To: <73045cca1001061244y61cc020fj5ea2c633df98d03f@mail.gmail.com> References: <73045cca1001061244y61cc020fj5ea2c633df98d03f@mail.gmail.com> Message-ID: On 1/6/2010 12:44 PM aditya shukla said... > Hello Guys, > > I have a multiprocessing script which downloads images from 5 urls to 5 > directories(usinf multiprocess in python 2.6).The download is for 5 > mins.My aim is to create a video for every minute for each directory and > dump the images as the video is created. My question are , should i use > _*mencoder.exe*_ to create the videos? I did something similar but on linux and used mencoder -- I'm guessing you can do the same on windows. > and how should i create the > videos without interrupting the download. In my situation I'd set up an ftp server and had webcams sending images once a second there. I'd also set up a cron job (scheduled task on windows systems) to create the video and archive the images. Emile From emile at fenx.com Wed Jan 6 16:58:17 2010 From: emile at fenx.com (Emile van Sebille) Date: Wed, 06 Jan 2010 13:58:17 -0800 Subject: File transfer with python In-Reply-To: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> References: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> Message-ID: On 1/6/2010 10:00 AM Valentin de Pablo Fouce said... > Hi there, > > I hope this is the rigth place, if not please, tell me which is the > right dicussion place. I apologize in such case. > > Ok, I am trying to do a very quick application (is "home based" so is > not a big deal...). My intention is to transfer files from one > computer to another. > > I am using several OS (Linux and Windows basicly, but potentially in > the future Mac too), and the computers won't be potentially in the > same network (ok, the most times will... but there will be > exceptions). cygwin + scp or rsync are other options. Emile > > My intention is to be able to transfer files from one computer to > another in this environment. > > Looking (and surfing) at internet the only suggestion given is to use > low level sockets for this file transfer. Is there another way to do > it, is there any top level library that helps you to do that? > > That's a lot! From steve at holdenweb.com Wed Jan 6 17:03:56 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 17:03:56 -0500 Subject: creating tar file and streaming it over HTTP? In-Reply-To: References: <5222d923-7aef-47cf-a0d6-f37f09a66593@35g2000yqa.googlegroups.com> Message-ID: <4B4508CC.6080508@holdenweb.com> r0g wrote: > pbienst wrote: >> I would like to bundle up a number of files in a tar file and send it >> over a HTTP connection, but I would like to do this without creating >> the tar file on disk first. >> > > Stringio lets you treat a strings as a files... > > http://docs.python.org/library/stringio.html > > Roger. ... though that does mean that the whole tar file has to be created before it can be sent, I suspect. If it will comfortably fit into memory that shouldn't matter, of course. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Wed Jan 6 17:03:56 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 17:03:56 -0500 Subject: creating tar file and streaming it over HTTP? In-Reply-To: References: <5222d923-7aef-47cf-a0d6-f37f09a66593@35g2000yqa.googlegroups.com> Message-ID: <4B4508CC.6080508@holdenweb.com> r0g wrote: > pbienst wrote: >> I would like to bundle up a number of files in a tar file and send it >> over a HTTP connection, but I would like to do this without creating >> the tar file on disk first. >> > > Stringio lets you treat a strings as a files... > > http://docs.python.org/library/stringio.html > > Roger. ... though that does mean that the whole tar file has to be created before it can be sent, I suspect. If it will comfortably fit into memory that shouldn't matter, of course. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From michi at triodia.com Wed Jan 6 17:15:55 2010 From: michi at triodia.com (Michi) Date: Wed, 6 Jan 2010 14:15:55 -0800 (PST) Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <03526a3a$0$1336$c3e8da3@news.astraweb.com> Message-ID: On Jan 5, 9:44?am, Steven D'Aprano wrote: > > I'm glad we agree on that, but I wonder why you previously emphasised > machine efficiency so much, and correctness almost not at all, in your > previous post? Uh? Because the original poster quoted one small paragraph out of a large article and that paragraph happened to deal with this particular (and minor) point of API design? > If all you're argument is that we shouldn't write crappy APIs, then I > agree with you completely. Well, yes: the article was precisely about that. And the point about exception efficiency was a minor side remark in that article. > Your argument seems to be > that we should avoid exceptions by default, and only use them if > unavoidable. I think that is backwards. I never made that argument. After 25 years as a software engineer, I well and truly have come to appreciate exceptions as a superior form of error handling. I simply stated that throwing an exception when none should be thrown is a pain and often inefficient on top of that. That's all, really. > I wouldn't say that's normal. If you don't care about the function's > result, why are you calling it? For the side-effects? printf returns a value that is almost always ignored. And, yes, a function such as printf is inevitable called for its side effects. We could argue that printf is misdesigned (I would): the return value is not useful, otherwise it would be used more. And if printf threw an exception when something didn't work, that would be appropriate because it fails so rarely that, if it does fail, I probably want to know. > > However, if a function throws instead of > > returning a value, ignoring that value becomes more difficult for the > > caller and can extract a performance penalty that may be unacceptable to > > the caller. > > There's that premature micro-optimization again. Let's be clear here: the entire discussion is about *inappropriate* use of exceptions. This isn't a premature optimisation. It's about deciding when an exception is appropriate and when not. If I throw an exception when I shouldn't, I make the API harder to use *and* less efficient. The real crime isn't the loss of efficiency though, it's the inappropriate exception. > I've been wondering when you would reach the conclusion that an API > should offer both forms. For example, Python offers both key-lookup that > raises exceptions (dict[key]) and key-lookup that doesn't (dict.get(key)). > > The danger of this is that it complicates the API, leads to a more > complex implementation, and may result in duplicated code (if the two > functions have independent implementations). Offering a choice in some form can be appropriate for some APIs. I'm not advocating it as a panacea, and I'm aware of the down-side in increased complexity, learning curve, etc. (BTW, the article discusses this issue in some detail.) > Well, obviously I agree that you should only make things be an exception > if they actually should be an exception. I don't quite see where the > implication is In the context of the original article, I argued that throwing exceptions that are inappropriate is one of the many things that API designers get wrong. To many people, that's stating the obvious. The number of APIs that still make exactly this mistake suggests that the point is worth making though. Anyway, some of the early posts implied that I was arguing against exception handling per-se because exceptions can be less efficient. I responded to correct that misconception. What the article really said is that throwing an exception when none should be thrown is bad API design, and inefficient to boot. I stand by that statement. Cheers, Michi. From aioe.org at technicalbloke.com Wed Jan 6 17:36:41 2010 From: aioe.org at technicalbloke.com (r0g) Date: Wed, 06 Jan 2010 22:36:41 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> <87ljgbwq4q.fsf@benfinney.id.au> Message-ID: Ben Finney wrote: > r0g writes: > >> NO! It's a rude way to start a sentence don't you think? > > Shouting is usually rude, yes. > >> Just because you're correcting someone doesn't mean you have to be >> combative and try and make them feel small. > > Again, you're reading something that isn't there. I utterly deny the > motives you're imputing. > >> I can't imagine why I expect good manners on usenet though, AFAICT >> it's never been like that (well not since I got on it anyway). > > I'll reiterate that you can address this by paying more attention to > what is actually written, and endeavouring to avoid finding emotions or > motives that are likely not in the message. > Both you and Steve are routinely harsh is all I'm saying, although he's worse than you. I'm sure you're both very good programmers who don't have the time to tread carefully around those who come in, often under prepared, seeking advice. Fair enough you don't tolerate fools lightly, I can respect that (within reason) and you contribute a lot to the group so you're perhaps owed a bit more slack than your average Joe but... There's a difference between that and repeatedly seeking out reasons to stick your boot in - and to my mind you seem to do that. Maybe it's just me, but have you considered that maybe it's not?... Most of the time when I see an extremely blunt, overly harsh response to a question on here I know if I glance up to the name field I'll find one of your names there (as I say, probably Steve more often that yourself). I'm sure you think I'm making this up but many passive aggressive don't realise how they're acting - can't see the wood for the trees and all that. Anyway I'm going to butt out now, I'm sure everyone is getting thoroughly bored of the OT noise. Roger From pdlemper at earthlink.net Wed Jan 6 17:40:09 2010 From: pdlemper at earthlink.net (pdlemper at earthlink.net) Date: Wed, 06 Jan 2010 16:40:09 -0600 Subject: Python interactive terminal in Ubuntu Linux : some keys fouled up Message-ID: Have recently moved from XP to Ubuntu Linux. Successfully installed Python 3.1.1 in the Ubuntu 9.04 release on my desktop. Problem is the python interactive terminal >>> . Many of the keys now do not work. eg pressing left-arrow yields ^[[D right-arrow ^[[C Home ^[OH Del ^[[3~ up-arrow ^[[A Frustrating as I use all these , esp up-arrow to repeat recent lines. Found the same thing on my sons MacBook. This is not mentioned in two recent Python books or one on Ubuntu. Nor could I found help on the www. Is there any work-around ? Should I just forget the python prompt >>> ? Thanks, Dave pdlemper at earthlink.net From aioe.org at technicalbloke.com Wed Jan 6 17:46:33 2010 From: aioe.org at technicalbloke.com (r0g) Date: Wed, 06 Jan 2010 22:46:33 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: Grant Edwards wrote: > On 2010-01-06, r0g wrote: > >> NO! It's a rude way to start a sentence don't you think? > > No. When somebody asks a yes/no question, answering yes or no > seems quite polite to me. Following the yes/no answer with an > explanation of the answer is always nice, and I've little doubt > that's what happened. > Well actually I hadn't asked a question and I hadn't been talking to him, he just butted in with it. Otherwise yes I agree completely. Roger. From zabin.farishta at gmail.com Wed Jan 6 17:47:37 2010 From: zabin.farishta at gmail.com (Zabin) Date: Wed, 6 Jan 2010 14:47:37 -0800 (PST) Subject: QDoubleValidator References: Message-ID: On Jan 7, 10:23?am, Zabin wrote: > Hey! > > I am new PyQt programmer and want to restrict users to allow only > numeric values into a table and lineedit boxes. I found the > QDoubleValidator class but am unsure as to how to implement it. (I am > a little shaky on the concept of parent and how to define them). Any > help would be much appreciated! I managed to get it going for the line edit- but am stuck with checking a cell in a table as cells do not have the set validator attribute- here's the code i had for the line edit: self.ui.Thickness_datum.setValidator(QtGui.QDoubleValidator(-999.0, 999.0, 2, self.ui.Thickness_datum)) for the table i thought it would be: self.ui.table_process.item(row,2).setValidator(QtGui.QDoubleValidator (-999.0, 999.0, 2, self.ui.table_process.item(row,2))) Any help would be appreciated From jcb at iteris.com Wed Jan 6 17:50:28 2010 From: jcb at iteris.com (Metalone) Date: Wed, 6 Jan 2010 14:50:28 -0800 (PST) Subject: Problem with multiprocessing managers Message-ID: <7f1f21c0-7679-4a8d-9980-715a25841619@a6g2000yqm.googlegroups.com> >From the documentation for Using a remote manager there is the following example code: from multiprocessing.managers import BaseManager import Queue queue = Queue.Queue() class QueueManager(BaseManager): pass QueueManager.register('get_queue', callable=lambda:queue) m = QueueManager(address=('', 50000), authkey='abracadabra') s = m.get_server() s.serve_forever() I don't know how to stop the server. The documentation states that to call shutdown(), start() must be used instead of server_forever(). If I use m.start() instead of m.get_server().server_forever() I receive a PicklingError. From aioe.org at technicalbloke.com Wed Jan 6 17:55:27 2010 From: aioe.org at technicalbloke.com (r0g) Date: Wed, 06 Jan 2010 22:55:27 +0000 Subject: File transfer with python References: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> Message-ID: Valentin de Pablo Fouce wrote: > Hi there, > > I hope this is the rigth place, if not please, tell me which is the > right dicussion place. I apologize in such case. > > Ok, I am trying to do a very quick application (is "home based" so is > not a big deal...). My intention is to transfer files from one > computer to another. > > I am using several OS (Linux and Windows basicly, but potentially in > the future Mac too), and the computers won't be potentially in the > same network (ok, the most times will... but there will be > exceptions). > > My intention is to be able to transfer files from one computer to > another in this environment. > > Looking (and surfing) at internet the only suggestion given is to use > low level sockets for this file transfer. Is there another way to do > it, is there any top level library that helps you to do that? > > That's a lot! Well there's several file transfer protocols you could use, TFTP, FTP, SFTP and you can do SCP and Rysnc over SSH. AFAIK there's python libs that handle all of those. Unless you need very high performance I'd suggest using SFTP (Secure file transfer protocol). Lookup the Paramiko library, it does this and other SSH based stuff. Roger. From aioe.org at technicalbloke.com Wed Jan 6 18:24:52 2010 From: aioe.org at technicalbloke.com (r0g) Date: Wed, 06 Jan 2010 23:24:52 +0000 Subject: TypeError References: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> Message-ID: MRAB wrote: > Victor Subervi wrote: >> Hi; >> I get this error: >> >> /var/www/html/angrynates.com/christians/cart/simplemail/mail.py >> >> 153 >> 154 ''' >> 155 commitSale() >> 156 myMail() >> 157 print ''' >> commitSale = >> /var/www/html/angrynates.com/christians/cart/simplemail/mail.py >> in >> commitSale() >> 98 cursor.execute('select max(ID) from %sCustomerData;' % store) >> 99 custID = cursor.fetchone()[0] >> 100 customerData(store, tmpTable, custID, patientID) >> 101 >> 102 def myMail(): >> global customerData = , global store = >> 'products', tmpTable = 'tem12627568064', custID = 1, global patientID >> = 'None' >> /var/www/html/angrynates.com/christians/cart/customerData.py >> in >> customerData(store='products', tmpTable='tem12627568064', custID=1, >> patientID='None') >> 39 >> 40 """ >> 41 print """ >> 42 print '

%s Customer Data

' % >> (store[0].upper() + store[1:]) >> 43 cursor.execute('describe %sCustomerData' % store) >> store = 'products' >> >> ValueError: unsupported format character '(' (0x28) at index 54 >> args = ("unsupported format character '(' (0x28) at index 54",) >> >> Apparently that character is a "file separator", which I presume is an >> invisible character. I tried retyping the area in question, but with >> no avail (threw same error). Please advise. Complete code follows. >> > Please provide the actual code (and, preferably, an easier-to-understand > traceback, like what CPython does!). What you supplied isn't runnable. Oh and, it's probably not a good idea to post things that identify the site you're working on, especially if you're fairly new to programming and you're doing a shopping cart! Hackers have been known to trawl google looking for mention of novice coders websites, giving them domain names, internal paths, table names etc might well help them hack you! Cheers & good luck, Roger. From aioe.org at technicalbloke.com Wed Jan 6 18:44:48 2010 From: aioe.org at technicalbloke.com (r0g) Date: Wed, 06 Jan 2010 23:44:48 +0000 Subject: Mencoder and creating videos References: <73045cca1001061244y61cc020fj5ea2c633df98d03f@mail.gmail.com> Message-ID: Emile van Sebille wrote: > On 1/6/2010 12:44 PM aditya shukla said... >> Hello Guys, >> >> I have a multiprocessing script which downloads images from 5 urls to 5 >> directories(usinf multiprocess in python 2.6).The download is for 5 >> mins.My aim is to create a video for every minute for each directory and >> dump the images as the video is created. My question are , should i use >> _*mencoder.exe*_ to create the videos? > Ffmpeg will do multiprocessor encoding too if the compression codec supports it. It will also compile stills into video as long as you are careful to number the file names sequentially. There's no "official" windows build of it but there's various 3rd party builds floating around the web, or you can build it yourself. Roger, From solipsis at pitrou.net Wed Jan 6 18:45:31 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 6 Jan 2010 23:45:31 +0000 (UTC) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> <1715bea7-1ff1-4294-9dad-672873e14530@v25g2000yqk.googlegroups.com> Message-ID: Le Wed, 06 Jan 2010 12:12:08 -0800, Phlip a ?crit?: > > And I, not my language, should pick and chose how to be rigorous. The > language should not make the decision for me. And that's why there is the "try: ... except: ..." construct. Your rant is getting tiring. From steven at REMOVE.THIS.cybersource.com.au Wed Jan 6 18:45:40 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 06 Jan 2010 23:45:40 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: On Wed, 06 Jan 2010 22:46:33 +0000, r0g wrote: > Grant Edwards wrote: >> On 2010-01-06, r0g wrote: >> >>> NO! It's a rude way to start a sentence don't you think? >> >> No. When somebody asks a yes/no question, answering yes or no seems >> quite polite to me. Following the yes/no answer with an explanation of >> the answer is always nice, and I've little doubt that's what happened. >> >> > Well actually I hadn't asked a question and I hadn't been talking to > him, he just butted in with it. Otherwise yes I agree completely. It's a public forum. You're talking to the whole world. -- Steven From martin at v.loewis.de Wed Jan 6 18:55:12 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Thu, 07 Jan 2010 00:55:12 +0100 Subject: python xmlrpc client with ssl client certificates and standard modules In-Reply-To: <4b43007c$0$11218$426a74cc@news.free.fr> References: <4b414e79$0$23321$426a34cc@news.free.fr> <4B4267DB.3080409@v.loewis.de> <4b43007c$0$11218$426a74cc@news.free.fr> Message-ID: <4B4522E0.4030206@v.loewis.de> > My next task is how to find out at the client side, that the server > certificate is a properly signed one. As Heikki says, you'll need Python 2.6 for that. You'll probably need to extend your transport implementation. Regards, Martin From ben+python at benfinney.id.au Wed Jan 6 18:55:58 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 07 Jan 2010 10:55:58 +1100 Subject: Assertions, challenges, and polite discourse (was: Exception as the primary error handling mechanism?) References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: <87hbqywzxt.fsf_-_@benfinney.id.au> Grant Edwards writes: > Answering a yes/no question with "no" doesn't seem to me to be > combative if the correct answer is indeed "no". But I've lost > track of the post you found objectionable... In fairness, the ?No? was in response, not to an explicit question, but to an assertion. Every assertion expressed, though, implies the question ?is this assertion true??. It was that question that was answered ?No? (followed by an explanation of why the assertion was not true). People sometimes get upset ? on an immediate, irrational level ? when their assertions are challenged. There's no denying that emotions entangle our discourse, and our interpretation of the discourse of others. That's not something I'd ever want to eradicate. I ask only that, rather than decrying that assertions be challenged per se, the challenge be assessed to see whether it's valid. -- \ ?Everything you read in newspapers is absolutely true, except | `\ for that rare story of which you happen to have first-hand | _o__) knowledge.? ?Erwin Knoll | Ben Finney From aioe.org at technicalbloke.com Wed Jan 6 18:58:21 2010 From: aioe.org at technicalbloke.com (r0g) Date: Wed, 06 Jan 2010 23:58:21 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > On Wed, 06 Jan 2010 22:46:33 +0000, r0g wrote: > >> Grant Edwards wrote: >>> On 2010-01-06, r0g wrote: >>> >>>> NO! It's a rude way to start a sentence don't you think? >>> No. When somebody asks a yes/no question, answering yes or no seems >>> quite polite to me. Following the yes/no answer with an explanation of >>> the answer is always nice, and I've little doubt that's what happened. >>> >>> >> Well actually I hadn't asked a question and I hadn't been talking to >> him, he just butted in with it. Otherwise yes I agree completely. > > > It's a public forum. You're talking to the whole world. > > > > See? Spoiling for an argument even now! I never said you weren't allowed to butt in, just that you did. Butting in is fine, that's half the point of public groups after all but it's also besides the point. I was merely explaining to Grant that I hadn't posed a yes/no question to anyone, let alone you. Roger. From casevh at gmail.com Wed Jan 6 19:09:08 2010 From: casevh at gmail.com (casevh) Date: Wed, 6 Jan 2010 16:09:08 -0800 (PST) Subject: Python interactive terminal in Ubuntu Linux : some keys fouled up References: Message-ID: On Jan 6, 2:40?pm, pdlem... at earthlink.net wrote: > Have recently moved from XP to Ubuntu Linux. > Successfully installed Python 3.1.1 in the Ubuntu 9.04 > release on my desktop. > Problem is the python interactive terminal ?>>> . > Many of the keys now do not work. > eg pressing left-arrow yields ?^[[D > ? ? ? ? ? ? ? ? ? ? right-arrow ? ? ? ? ^[[C > ? ? ? ? ? ? ? ? ? ? Home ? ? ? ? ? ? ? ? ?^[OH > ? ? ? ? ? ? ? ? ? ? Del ? ? ? ? ? ? ? ? ? ? ?^[[3~ > ? ? ? ? ? ? ? ? ? ? up-arrow ? ? ? ? ? ?^[[A ? ? ? ? ? ? ? ? ? > > Frustrating as I use all these , esp ?up-arrow to > repeat recent lines. ?Found the same thing on > my sons MacBook. > > This is not mentioned in two recent Python books > or one on Ubuntu. Nor could I found help on the www. > > Is there any work-around ?? ?Should I just forget > the python prompt >>> ? ? ? ? ? ? ? ? ? ? ? > > Thanks, ? ? ? ? ? ? Dave ? ? ? ?pdlem... at earthlink.net Assuming you compiled the source code, you will also need to install "libreadline5-dev" via Synaptic or apt-get. casevh From no.i.dont at want.mail.from.spammers.com Wed Jan 6 19:10:24 2010 From: no.i.dont at want.mail.from.spammers.com (Fencer) Date: Thu, 07 Jan 2010 01:10:24 +0100 Subject: suds problem In-Reply-To: <7qk532F25jU1@mid.individual.net> References: <7qk3bcFnbjU1@mid.individual.net> <7qk532F25jU1@mid.individual.net> Message-ID: <7qkn3hFe6nU1@mid.individual.net> On 2010-01-06 20:02, Fencer wrote: > On 2010-01-06 19:33, Fencer wrote: >> Hello, I just started using suds to use web services. First I tried suds >> with a very simple web service I had written and was running myself. >> That worked fine. Then I tried to use the web services provided by KEGG: >> http://soap.genome.jp/KEGG.wsdl >> But I get a SAXParseException due to a supposed mis-matched tag when I >> try to consume that wsdl with suds. I checked the wsdl in oxygene and it >> claims it's valid. What is the problem here? My test program is below >> and it's very simple: >> from suds.client import Client >> >> url = 'http://soap.genome.jp/KEGG.wsdl' >> >> client = Client(url) >> >> print client >> >> - Fencer > > Same problem with reactome: > http://www.reactome.org:8080/caBIOWebApp/services/caBIOService?wsdl > > I suppose I'm mis-using suds, maybe some setting that wasn't needed for > my simple web service but is needed for kegg and reactome. > > - Fencer Seems to be a known problem regarding suds. :( A work-around was suggested to remove some cache files but that didn't solve it for me. Appreciate other ideas! - Fencer From sjmachin at lexicon.net Wed Jan 6 19:14:41 2010 From: sjmachin at lexicon.net (John Machin) Date: Wed, 6 Jan 2010 16:14:41 -0800 (PST) Subject: TypeError References: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> Message-ID: On Jan 7, 3:29?am, MRAB wrote: > Victor Subervi wrote: > > ValueError: unsupported format character '(' (0x28) at index 54 > > ? ? ? args = ("unsupported format character '(' (0x28) at index 54",) > > > Apparently that character is a "file separator", which I presume is an > > invisible character. I tried retyping the area in question, but with no > > avail (threw same error). Please advise. Complete code follows. > OP is barking up the wrong tree. "file separator" has ordinal 28 DECIMAL. Correct tree contains '(' (left parenthesis, ordinal 0x28 (HEX)) as the error message says. From sjmachin at lexicon.net Wed Jan 6 19:17:57 2010 From: sjmachin at lexicon.net (John Machin) Date: Wed, 6 Jan 2010 16:17:57 -0800 (PST) Subject: 3 byte network ordered int, How To ? References: <3d4c0f161001061001ma900949j1fdea2789cff7367@mail.gmail.com> Message-ID: <02bc346a-81dc-4071-8914-2f7ba0dc2cb2@q4g2000yqm.googlegroups.com> On Jan 7, 5:33?am, Matthew Barnett wrote: > mudit tuli wrote: > > For a single byte, struct.pack(') > > For two bytes, struct.pack(') > > what if I want three bytes ? > > Four bytes and then discard the most-significant byte: > > struct.pack(')[ : -1] AARRGGHH! network ordering is BIGendian, struct.pack('<..... is LITTLEendian From wuwei23 at gmail.com Wed Jan 6 19:33:58 2010 From: wuwei23 at gmail.com (alex23) Date: Wed, 6 Jan 2010 16:33:58 -0800 (PST) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> <1715bea7-1ff1-4294-9dad-672873e14530@v25g2000yqk.googlegroups.com> Message-ID: Phlip wrote: > And I, not my language, should pick and chose how to be rigorous. The language > should not make the decision for me. Since you seem unwilling to put the minimal effort into producing the support code you'd need to work with Python the way you want, perhaps Perl might be more to your liking? From aioe.org at technicalbloke.com Wed Jan 6 19:35:42 2010 From: aioe.org at technicalbloke.com (r0g) Date: Thu, 07 Jan 2010 00:35:42 +0000 Subject: Assertions, challenges, and polite discourse References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> <87hbqywzxt.fsf_-_@benfinney.id.au> Message-ID: Ben Finney wrote: > In fairness, the ?No? was in response, not to an explicit question, but > to an assertion. > > Every assertion expressed, though, implies the question ?is this > assertion true??. It was that question that was answered ?No? (followed > by an explanation of why the assertion was not true). > That's a fair point I had never really considered. I'd wouldn't have been nearly as upset if you'd started your sentence with AssertionError :) > People sometimes get upset ? on an immediate, irrational level ? when > their assertions are challenged. There's no denying that emotions > entangle our discourse, and our interpretation of the discourse of > others. That's truer than most people appreciate, to the extent that it's a good idea to tread very lightly when correcting strangers if you want rational discourse to continue. Even small amounts of negativity commonly provoke large threat responses in people which in turn inhibit rational thinking... Have a watch of this Google Tech Talk if you have time, it's really quite enlightening... http://www.youtube.com/watch?v=XeJSXfXep4M > > That's not something I'd ever want to eradicate. I ask only that, rather > than decrying that assertions be challenged per se, the challenge be > assessed to see whether it's valid. > Well I think sometimes, for the sake of expediency and overall pleasantness, it's better to let the smaller things go: and if you just can't let them go then at least try and issue corrections in a friendly manner rather than a cold or pious one. As this thread is demonstrating endless checking of every detail of every assertion leads to very long execution times and on balance I'm not sure we benefit from these extra digits of precision. Roger. From zabin.farishta at gmail.com Wed Jan 6 19:37:05 2010 From: zabin.farishta at gmail.com (Zabin) Date: Wed, 6 Jan 2010 16:37:05 -0800 (PST) Subject: Validating cells of a table PyQt Message-ID: <2d6c6f77-30cc-4787-919a-f2781026dd66@a21g2000yqc.googlegroups.com> Hey! I am new PyQT programmer. I am trying to create a table in which cells only take in numeric data. I am unable to use setValidator on table cells. Looking around i found some material on the editor attribute but I dont know how to apply this. Any help will be appreciated Cheers! From wolftracks at invalid.com Wed Jan 6 19:40:31 2010 From: wolftracks at invalid.com (W. eWatson) Date: Wed, 06 Jan 2010 16:40:31 -0800 Subject: Astronomy--Programs to Compute Siderial Time? In-Reply-To: References: Message-ID: W. eWatson wrote: > Is there a smallish Python library of basic astronomical functions? > There are a number of large such libraries that are crammed with > excessive functions not needed for common calculations. It looks like I've entered a new era in my knowledge of Python. I found a module somewhat like I want, siderial.py. You can see an intro to it at . It appears that I can get the code for it through section 1.2, near the bottom. I scooped it siderial.py up, and placed it in a corresponding file of the same name and type via NotePad. However, there is a xml file below it. I know little about it. I thought maybe I could do the same, but Notepad didn't like some characters in it. As I understand Python doc files are useful. So how do I get this done, and where do I put the files? From wuwei23 at gmail.com Wed Jan 6 19:40:46 2010 From: wuwei23 at gmail.com (alex23) Date: Wed, 6 Jan 2010 16:40:46 -0800 (PST) Subject: File transfer with python References: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> Message-ID: <0856a3ed-40bf-4027-89e0-ffcb6d9e1d04@35g2000yqa.googlegroups.com> Valentin de Pablo Fouce wrote: > My intention is to be able to transfer files from one computer to > another in this environment. > > Looking (and surfing) at internet the only suggestion given is to use > low level sockets for this file transfer. Is there another way to do > it, is there any top level library that helps you to do that? If your computers are distributed across many networks, and they all have internet access, Richard Jones' GmailFS[1] might give you some ideas. It uses the Python bindings for FUSE to create a local filestore that's a wrapper around the Gmail API, stashing the actual data on Google's servers. It's out of date now, but it shouldn't be much of an effort to update to the latest Gmail API, or to replace with any other backend data store you like. Google App Engine[2] provides a hefty amount of free storage to developers, so that's another option. 1: http://richard.jones.name/google-hacks/gmail-filesystem/gmail-filesystem.html 2: http://code.google.com/appengine/ From pavlovevidence at gmail.com Wed Jan 6 19:52:03 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Wed, 6 Jan 2010 16:52:03 -0800 (PST) Subject: please help shrink this each_with_index() implementation References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> <1715bea7-1ff1-4294-9dad-672873e14530@v25g2000yqk.googlegroups.com> Message-ID: <58a5e5bf-ca89-4fd6-be57-c187975a7391@s19g2000vbm.googlegroups.com> On Jan 6, 12:12?pm, Phlip wrote: > Nobody wrote: > > Writing robust software from the outset puts you at a competitive > > disadvantage to those who understand how the system works. > > And I, not my language, should pick and chose how to be rigorous. The language > should not make the decision for me. Oh well, your language does. Deal with it, or pick another language. Carl Banks P.S. Actually, Python has ways to request no expection when it's actually useful, e.g. the get method of dicts, but not when it's ridiculous and stupid like int() returning zero on error. From ben+python at benfinney.id.au Wed Jan 6 19:53:24 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 07 Jan 2010 11:53:24 +1100 Subject: Assertions, challenges, and polite discourse References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> <87hbqywzxt.fsf_-_@benfinney.id.au> Message-ID: <873a2iwxa3.fsf@benfinney.id.au> r0g writes: > Ben Finney wrote: > > People sometimes get upset ? on an immediate, irrational level ? > > when their assertions are challenged. There's no denying that > > emotions entangle our discourse, and our interpretation of the > > discourse of others. > > That's truer than most people appreciate, to the extent that it's a > good idea to tread very lightly when correcting strangers if you want > rational discourse to continue. Even small amounts of negativity > commonly provoke large threat responses in people which in turn > inhibit rational thinking... I prefer the strategy of acknowledging and desensitising this irrational response, by making it obvious that every assertion expressed is inevitably an exposure of that assertion to challenge and criticism. Challenging assertions and criticising reasoning are both healthy and in insufficient supply, and I want them to be normal and routine. I try to act accordingly. -- \ ?Nothing is more sacred than the facts.? ?Sam Harris, _The End | `\ of Faith_, 2004 | _o__) | Ben Finney From sjmachin at lexicon.net Wed Jan 6 19:53:54 2010 From: sjmachin at lexicon.net (John Machin) Date: Wed, 6 Jan 2010 16:53:54 -0800 (PST) Subject: parsing an Excel formula with the re module References: <6e7190d6-420f-4df4-8d03-2c75190602cf@a6g2000yqm.googlegroups.com> <5ef3575d-eb1d-4a15-90e0-87dd85cdc2e6@35g2000yqa.googlegroups.com> <6473e930-c5f4-423f-b78b-aae63f2aa794@k17g2000yqh.googlegroups.com> Message-ID: <8cff3674-f113-4b80-8284-56faa1bc53da@c3g2000yqd.googlegroups.com> On Jan 6, 6:54?am, vsoler wrote: > On 5 ene, 20:21, vsoler wrote: > > > > > On 5 ene, 20:05, Mensanator wrote: > > > > On Jan 5, 12:35?pm, MRAB wrote: > > > > > vsoler wrote: > > > > > Hello, > > > > > > I am acessing an Excel file by means of Win 32 COM technology. > > > > > For a given cell, I am able to read its formula. I want to make a map > > > > > of how cells reference one another, how different sheets reference one > > > > > another, how workbooks reference one another, etc. > > > > > > Hence, I need to parse Excel formulas. Can I do it by means only of re > > > > > (regular expressions)? > > > > > > I know that for simple formulas such as "=3*A7+5" it is indeed > > > > > possible. What about complex for formulas that include functions, > > > > > sheet names and possibly other *.xls files? > > > > > > For example ? ?"=Book1!A5+8" should be parsed into ["=","Book1", "!", > > > > > "A5","+","8"] > > > > > > Can anybody help? Any suggestions? > > > > > Do you mean "how" or do you really mean "whether", ie, get a list of the > > > > other cells that are referred to by a certain cell, for example, > > > > "=3*A7+5" should give ["A7"] and "=Book1!A5+8" should give ["Book1!A5] > > > > Ok, although "Book1" would be the default name of a workbook, with > > > default > > > worksheets labeled "Sheet1". "Sheet2", etc. > > > > If I had a worksheet named "Sheety" that wanted to reference a cell on > > > "Sheetx" > > > OF THE SAME WORKBOOK, it would be =Sheet2!A7. If the reference was to > > > a completely > > > different workbook (say Book1 with worksheets labeled "Sheet1", > > > "Sheet2") then > > > the cell might have =[Book1]Sheet1!A7. > > > > And don't forget the $'s! You may see =[Book1]Sheet1!$A$7. > > > Yes, Mensanator, but... ?what re should I use? I'm looking for the re > > statement. No doubt you can help! > > > Thank you. > > Let me give you an example: > > >>> import re > >>> re.split("([^0-9])", "123+456*/") > > [?123?, ?+?, ?456?, ?*?, ??, ?/?, ??] > > I find it excellent that one single statement is able to do a lexical > analysis of an expression! That is NOT lexical analysis. > > If the expression contains variables, such as A12 or B9, I can try > another re expression. Which one should I use? > > And if my expression contains parenthesis? ? And the sin() function? You need a proper lexical analysis, followed by a parser. What you are trying to do can NOT be accomplished in any generality with a single regex. The Excel formula syntax has several tricky bits. E.g. IIRC whether TAX09 is a (macro) name or a cell reference depends on what version of Excel you are targetting but if it appears like TAX09! A1:B2 then it's a sheet name. The xlwt package (of which I am the maintainer) has a lexer and parser for a largish subset of the syntax ... see http://pypi.python.org/pypi/xlwt From roy at panix.com Wed Jan 6 20:22:48 2010 From: roy at panix.com (Roy Smith) Date: Wed, 06 Jan 2010 20:22:48 -0500 Subject: Astronomy--Programs to Compute Siderial Time? References: Message-ID: In article , "W. eWatson" wrote: > Is there a smallish Python library of basic astronomical functions? > There are a number of large such libraries that are crammed with > excessive functions not needed for common calculations. FWIW, if you have any interest in this kind of stuff, you must read the classic book on the subject: http://www.amazon.com/Astronomical-Formulae-Calculators-Jean-Meeus/dp/094339 6220 This is not some textbook which takes graduate level physics to understand. It's a straight-forward primer for people who want to use calculators to compute phase of the moon and stuff like that. From sjmachin at lexicon.net Wed Jan 6 20:31:16 2010 From: sjmachin at lexicon.net (John Machin) Date: Wed, 6 Jan 2010 17:31:16 -0800 (PST) Subject: TypeError References: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> Message-ID: <4c29f4d1-a4d2-4877-91fd-09b346f1a5f2@k19g2000yqc.googlegroups.com> On Jan 7, 11:14?am, John Machin wrote: > On Jan 7, 3:29?am, MRAB wrote: > > > Victor Subervi wrote: > > > ValueError: unsupported format character '(' (0x28) at index 54 > > > ? ? ? args = ("unsupported format character '(' (0x28) at index 54",) > > > > Apparently that character is a "file separator", which I presume is an > > > invisible character. I tried retyping the area in question, but with no > > > avail (threw same error). Please advise. Complete code follows. > > OP is barking up the wrong tree. "file separator" has ordinal 28 > DECIMAL. Correct tree contains '(' (left parenthesis, ordinal 0x28 > (HEX)) as the error message says. It took a bit of mucking about to get an example of that error message (without reading the Python source code): |>>> anything = object() \|>>> "foo%(" % anything Traceback (most recent call last): File "", line 1, in TypeError: format requires a mapping |>>> "foo%(" % {} Traceback (most recent call last): File "", line 1, in ValueError: incomplete format key |>>> "foo%2(" % anything Traceback (most recent call last): File "", line 1, in ValueError: unsupported format character '(' (0x28) at index 5 FWIW, the OP's message subject is "TypeError" but the reported message contains ValueError ... possibly indicative of code that first builds a format string (incorrectly) and then uses it with error messages that can vary from run to run depending on exactly what was stuffed into the format string. I note that in the code shown there are examples of building an SQL query where the table name is concocted at runtime via the % operator ... key phrases: "bad database design" (one table per store!), "SQL injection attack" A proper traceback would be very nice ... at this stage it's not certain what was the line of source that triggers the exception. From steven at REMOVE.THIS.cybersource.com.au Wed Jan 6 20:31:38 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 07 Jan 2010 01:31:38 GMT Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: On Wed, 06 Jan 2010 23:58:21 +0000, r0g wrote: > Steven D'Aprano wrote: >> On Wed, 06 Jan 2010 22:46:33 +0000, r0g wrote: >> >>> Grant Edwards wrote: >>>> On 2010-01-06, r0g wrote: >>>> >>>>> NO! It's a rude way to start a sentence don't you think? >>>> No. When somebody asks a yes/no question, answering yes or no seems >>>> quite polite to me. Following the yes/no answer with an explanation >>>> of the answer is always nice, and I've little doubt that's what >>>> happened. >>>> >>>> >>> Well actually I hadn't asked a question and I hadn't been talking to >>> him, he just butted in with it. Otherwise yes I agree completely. >> >> >> It's a public forum. You're talking to the whole world. > > > See? Spoiling for an argument even now! I never said you weren't allowed > to butt in, just that you did. Butting in is fine, that's half the point > of public groups after all but it's also besides the point. I was merely > explaining to Grant that I hadn't posed a yes/no question to anyone, let > alone you. I quote from your very next post: [quote] Well I think sometimes, for the sake of expediency and overall pleasantness, it's better to let the smaller things go: and if you just can't let them go then at least try and issue corrections in a friendly manner rather than a cold or pious one. [end quote] Perhaps you should consider taking your own advice instead of lecturing us in an unpleasant, aggressive manner about how horrible we are for answering your questions with things you don't want to hear? BTW, you were actually talking about Ben butting in -- it was Ben's answer that started with "No" that triggered this series of complaints from you: [quote] I might have let it slip had you not started your reply with the word "No", that just p***** me off. [end quote] And the offending, dastardly comment from Ben? [r0g] Yes, it returns a tuple if you return more than one value, it just has a lovely syntax for it. [ben] No, there is nothing inherent to the ?return? statement for dealing with multiple values. The word "No" was clearly and obviously a response to the previous sentence that started "Yes". If this is all it takes to put you in a temper ("p***** me off") then I suggest you need to look at your own behaviour and stop blaming others. -- Steven From aioe.org at technicalbloke.com Wed Jan 6 20:35:33 2010 From: aioe.org at technicalbloke.com (r0g) Date: Thu, 07 Jan 2010 01:35:33 +0000 Subject: Assertions, challenges, and polite discourse References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> <87hbqywzxt.fsf_-_@benfinney.id.au> <873a2iwxa3.fsf@benfinney.id.au> Message-ID: Ben Finney wrote: > r0g writes: > >> Ben Finney wrote: >>> People sometimes get upset ? on an immediate, irrational level ? >>> when their assertions are challenged. There's no denying that >>> emotions entangle our discourse, and our interpretation of the >>> discourse of others. >> That's truer than most people appreciate, to the extent that it's a >> good idea to tread very lightly when correcting strangers if you want >> rational discourse to continue. Even small amounts of negativity >> commonly provoke large threat responses in people which in turn >> inhibit rational thinking... > > I prefer the strategy of acknowledging and desensitising this irrational > response, by making it obvious that every assertion expressed is > inevitably an exposure of that assertion to challenge and criticism. I see what you're saying but the "tough love" approach has been shown to be ineffective in many of the situations it has been tried in, it's something people instinctively think ought to work but rarely does in the modern world. Anyway, it's only an irrational response in the sense that it is likely to provoke irrationality in your counterpart which isn't a good outcome for either of you. To our brains it is a very natural and powerful response. That it's emotional doesn't necessarily make it irrational. Bear in mind that evolution is the distilled rationality of countless generations and to fight it is to engage in a very one sided battle. Seriously, watch that video I recommended, it's very interesting and delves into the neurology of this exact subject. > > Challenging assertions and criticising reasoning are both healthy and in > insufficient supply, and I want them to be normal and routine. I try to > act accordingly. > OK that's fair enough as a general principle but I disagree we are insufficiently supplied with either here on usenet. I reckon we have too much if anything. I do understand that a balance must be struck and the bar ought to err on the high side after, all this group isn't yahoo answers but neither is it a peer reviewed journal or a legal proceeding. And anyway, my main point was concerning the tone used when challenging (perceived) falsehoods rather than the rationale behind challenging (perceived) falsehoods. Naturally it's right to correct non-trivial technical falsehoods in the context of this group. Roger. From wolftracks at invalid.com Wed Jan 6 20:43:10 2010 From: wolftracks at invalid.com (W. eWatson) Date: Wed, 06 Jan 2010 17:43:10 -0800 Subject: Astronomy--Programs to Compute Siderial Time? In-Reply-To: References: Message-ID: Roy Smith wrote: > In article , > "W. eWatson" wrote: > >> Is there a smallish Python library of basic astronomical functions? >> There are a number of large such libraries that are crammed with >> excessive functions not needed for common calculations. > > FWIW, if you have any interest in this kind of stuff, you must read the > classic book on the subject: > > http://www.amazon.com/Astronomical-Formulae-Calculators-Jean-Meeus/dp/094339 > 6220 > > This is not some textbook which takes graduate level physics to understand. > It's a straight-forward primer for people who want to use calculators to > compute phase of the moon and stuff like that. Thanks, but I'm quite familiar with it. My copy is some 10-15 years old. I suspect he never wrote a newer edition for any computer language. I do have access to a C++ library, but I don't think that's going to do me much good with Python. My problem at the moment is finding either someone who has implemented it or another similar module. Note I posted just before you that I have found a siderial.py module, but am not familiar with how one installs them or the doc mechanism. From sjmachin at lexicon.net Wed Jan 6 20:49:41 2010 From: sjmachin at lexicon.net (John Machin) Date: Wed, 6 Jan 2010 17:49:41 -0800 (PST) Subject: Astronomy--Programs to Compute Siderial Time? References: Message-ID: On Jan 7, 11:40?am, "W. eWatson" wrote: > W. eWatson wrote: > > Is there a smallish Python library of basic astronomical functions? > > There are a number of large such libraries that are crammed with > > excessive functions not needed for common calculations. > > It looks like I've entered a new era in my knowledge of Python. Mild curiosity: this would be a wonderful outcome, but what makes it look so? > I found > a module somewhat like I want, siderial.py. You can see an intro to it > at . > It appears that I can get the code for it through section 1.2, near the > bottom. I scooped it siderial.py up, and placed it in a corresponding > file of the same name and type via NotePad. However, there is a xml file > below it. I know little about it. I thought maybe I could do the same, > but Notepad didn't like some characters in it. As I understand Python > doc files are useful. So how do I get this done, and where do I put the > files? The file you need is sidereal.py, not your twice-mentioned siderial.py (the existence of which on the referenced website is doubtful). What you have been reading is the "Internal maintenance specification" (large font, near the top of the page) for the module. The xml file is the source of the docs, not meant to be user-legible. A very tiny amount of googling "sidereal.py" (quotes included) leads to the user documentation at http://infohost.nmt.edu/tcc/help/lang/python/examples/sidereal/ Where do you put the files? Well, we're now down to only one file, sidereal.py, and you put it wherever you'd put any other module that you'd like to call ... if there's only going to be one caller, put it in the same directory as that caller's code. More generally, drop it in /Lib/site-packages From aioe.org at technicalbloke.com Wed Jan 6 20:54:25 2010 From: aioe.org at technicalbloke.com (r0g) Date: Thu, 07 Jan 2010 01:54:25 +0000 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > On Wed, 06 Jan 2010 23:58:21 +0000, r0g wrote: > >> Steven D'Aprano wrote: >>> On Wed, 06 Jan 2010 22:46:33 +0000, r0g wrote: >>> >>>> Grant Edwards wrote: >>>>> On 2010-01-06, r0g wrote: >> See? Spoiling for an argument even now! I never said you weren't allowed >> to butt in, just that you did. Butting in is fine, that's half the point >> of public groups after all but it's also besides the point. I was merely >> explaining to Grant that I hadn't posed a yes/no question to anyone, let >> alone you. > > I quote from your very next post: > > > [quote] > Well I think sometimes, for the sake of expediency and overall > pleasantness, it's better to let the smaller things go: and if you just > can't let them go then at least try and issue corrections in a friendly > manner rather than a cold or pious one. > [end quote] > > I did answer HIM in a friendly manner, to you I'm merely responding in kind. Anyway I got upset at Ben's comments for several reasons which I explained at length in that very post (and several subsequent ones) so I won't rehash them any further here. The "No" in question was merely the straw that broke the camels back and triggered my admittedly undignified outburst. I have just seen too many supercilious replies beginning "No, " in this forum and so yes, it p***** me off - I can happily admit it. Of course clearly you're too cool to ever get p***** off yourself, I detect no background seething at all here. Roger. From steve at holdenweb.com Wed Jan 6 21:38:23 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 21:38:23 -0500 Subject: TypeError In-Reply-To: <4c29f4d1-a4d2-4877-91fd-09b346f1a5f2@k19g2000yqc.googlegroups.com> References: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> <4c29f4d1-a4d2-4877-91fd-09b346f1a5f2@k19g2000yqc.googlegroups.com> Message-ID: John Machin wrote: [...] > I note that in the code shown there are examples of building an SQL > query where the table name is concocted at runtime via the % > operator ... key phrases: "bad database design" (one table per > store!), "SQL injection attack" > I'm not trying to defend the code overall, but most databases won't let you parameterize the table or column names, just the data values. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Wed Jan 6 21:43:29 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 21:43:29 -0500 Subject: TypeError In-Reply-To: References: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> <4c29f4d1-a4d2-4877-91fd-09b346f1a5f2@k19g2000yqc.googlegroups.com> Message-ID: Steve Holden wrote: > John Machin wrote: > [...] >> I note that in the code shown there are examples of building an SQL >> query where the table name is concocted at runtime via the % >> operator ... key phrases: "bad database design" (one table per >> store!), "SQL injection attack" >> > I'm not trying to defend the code overall, but most databases won't let > you parameterize the table or column names, just the data values. > And, apropos of nothing in particular, here's a completely gratuitous additional chance to tell me off again for spamming the list about a conference: http://holdenweb.blogspot.com/2010/01/register-for-pycon-or-kitten-gets-it.html regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From wuwei23 at gmail.com Wed Jan 6 21:58:29 2010 From: wuwei23 at gmail.com (alex23) Date: Wed, 6 Jan 2010 18:58:29 -0800 (PST) Subject: Assertions, challenges, and polite discourse References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> <87hbqywzxt.fsf_-_@benfinney.id.au> Message-ID: <487fa8b4-5e27-4e61-99a3-4c57370154e6@c3g2000yqd.googlegroups.com> r0g wrote: > Well I think sometimes, for the sake of expediency and overall > pleasantness, it's better to let the smaller things go: and if you just > can't let them go then at least try and issue corrections in a friendly > manner rather than a cold or pious one. The irony, it is too rich... From skwslide at gmail.com Wed Jan 6 22:05:40 2010 From: skwslide at gmail.com (Steven K. Wong) Date: Wed, 6 Jan 2010 19:05:40 -0800 (PST) Subject: subprocess.Popen does not close pipe in an error case References: <07641018-9581-44af-bb20-8f9b96a44c4f@p8g2000yqb.googlegroups.com> Message-ID: <40a2adab-941a-4844-9279-10667bf2ba5e@v25g2000yqk.googlegroups.com> Well, the example code at http://www.python.org/doc/2.6.2/library/subprocess.html#replacing-shell-pipeline has the same issue: output=`dmesg | grep hda` ==> p1 = Popen(["dmesg"], stdout=PIPE) p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE) output = p2.communicate()[0] After communicate() returns, if you wait for p1 to finish (by calling p1.poll() repeatedly or p1.wait()), you can hang if the conditions described in the original post are true, i.e. p1 wrote lots of data to the pipe and p2 failed without reading much data from the pipe. Perhaps the doc can be improved to remind folks to close p1.stdout if the calling process doesn't need it, unless wait() is changed to close it and p1.wait() is called. Am I making any sense here? From sjmachin at lexicon.net Wed Jan 6 22:28:49 2010 From: sjmachin at lexicon.net (John Machin) Date: Wed, 6 Jan 2010 19:28:49 -0800 (PST) Subject: TypeError References: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> <4c29f4d1-a4d2-4877-91fd-09b346f1a5f2@k19g2000yqc.googlegroups.com> Message-ID: On Jan 7, 1:38?pm, Steve Holden wrote: > John Machin wrote: > > [...]> I note that in the code shown there are examples of building an SQL > > query where the table name is concocted at runtime via the % > > operator ... key phrases: "bad database design" (one table per > > store!), "SQL injection attack" > > I'm not trying to defend the code overall, but most databases won't let > you parameterize the table or column names, just the data values. That's correct, and that's presumably why the OP is constructing whole SQL statements on the fly e.g. cursor.execute('select max(ID) from %sCustomerData;' % store) What is the reason for "but" in "but most databases won't ..."? What are you rebutting? Let me try again: One table per store is bad design. The implementation of that bad design may use: cursor.execute('select max(ID) from %sCustomerData;' % store) or (if available) cursor.execute('select max(ID) from ?CustomerData;', (store, )) but the implementation means is irrelevant. From jjposner at optimum.net Wed Jan 6 22:31:42 2010 From: jjposner at optimum.net (John Posner) Date: Wed, 06 Jan 2010 22:31:42 -0500 Subject: QDoubleValidator References: Message-ID: On Wed, 06 Jan 2010 17:47:37 -0500, Zabin wrote: > On Jan 7, 10:23 am, Zabin wrote: >> Hey! >> >> I am new PyQt programmer and want to restrict users to allow only >> numeric values into a table and lineedit boxes. I found the >> QDoubleValidator class but am unsure as to how to implement it. (I am >> a little shaky on the concept of parent and how to define them). Any >> help would be much appreciated! You'll probably get more help in these Qt-specific forums: http://lists.trolltech.com/mailman/listinfo/qt-interest http://www.riverbankcomputing.com/mailman/listinfo/pyqt Good luck, John From wolftracks at invalid.com Wed Jan 6 22:40:52 2010 From: wolftracks at invalid.com (W. eWatson) Date: Wed, 06 Jan 2010 19:40:52 -0800 Subject: Astronomy--Programs to Compute Siderial Time? In-Reply-To: References: Message-ID: John Machin wrote: > On Jan 7, 11:40 am, "W. eWatson" wrote: >> W. eWatson wrote: >>> Is there a smallish Python library of basic astronomical functions? >>> There are a number of large such libraries that are crammed with >>> excessive functions not needed for common calculations. >> It looks like I've entered a new era in my knowledge of Python. > > Mild curiosity: this would be a wonderful outcome, but what makes it > look so? I actually need to learn how to make a module that can be imported, which in the short interlude I have done. Also looked into docstrings and docs, which I now have a decent grasp of. Never really used either doc info or writing my own module before. Easy. > >> I found >> a module somewhat like I want, siderial.py. You can see an intro to it >> at . >> It appears that I can get the code for it through section 1.2, near the >> bottom. I scooped it siderial.py up, and placed it in a corresponding >> file of the same name and type via NotePad. However, there is a xml file >> below it. I know little about it. I thought maybe I could do the same, >> but Notepad didn't like some characters in it. As I understand Python >> doc files are useful. So how do I get this done, and where do I put the >> files? > > The file you need is sidereal.py, not your twice-mentioned siderial.py > (the existence of which on the referenced website is doubtful). How right you are. I misspelled it twice, and quickly found that out when I tried to use the [side][real] (easy mnemonic, two words) module. sidereal. > > What you have been reading is the "Internal maintenance > specification" (large font, near the top of the page) for the module. > The xml file is the source of the docs, not meant to be user-legible. What is it used for? Do I need it? > A very tiny amount of googling "sidereal.py" (quotes included) leads > to the user documentation at http://infohost.nmt.edu/tcc/help/lang/python/examples/sidereal/ Found that too as I cruised around. > > Where do you put the files? Well, we're now down to only one file, > sidereal.py, and you put it wherever you'd put any other module that > you'd like to call ... if there's only going to be one caller, put it > in the same directory as that caller's code. More generally, drop it > in /Lib/site-packages Again in my "learning about modules", discovered that too. I think I'm on my way. Thanks. From steve at holdenweb.com Wed Jan 6 22:51:38 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 06 Jan 2010 22:51:38 -0500 Subject: TypeError In-Reply-To: References: <4dc0cfea1001052311ra3d5671o92bc9dea0b794e8e@mail.gmail.com> <4c29f4d1-a4d2-4877-91fd-09b346f1a5f2@k19g2000yqc.googlegroups.com> Message-ID: <4B455A4A.2090702@holdenweb.com> John Machin wrote: > On Jan 7, 1:38 pm, Steve Holden wrote: >> John Machin wrote: >> >> [...]> I note that in the code shown there are examples of building an SQL >>> query where the table name is concocted at runtime via the % >>> operator ... key phrases: "bad database design" (one table per >>> store!), "SQL injection attack" >> I'm not trying to defend the code overall, but most databases won't let >> you parameterize the table or column names, just the data values. > > That's correct, and that's presumably why the OP is constructing whole > SQL statements on the fly e.g. > > cursor.execute('select max(ID) from %sCustomerData;' % store) > Well yes, but that is just a symptom of the real disease, which is that he has very little idea what he is doing. > What is the reason for "but" in "but most databases won't ..."? What > are you rebutting? > I was simply pointing out that the OP had chosen the only available way of generating variable table names. The bad database design has been discussed,if not ad infinitum then certainly ad nauseam. All advice has been ignored. > Let me try again: One table per store is bad design. The > implementation of that bad design may use: > > cursor.execute('select max(ID) from %sCustomerData;' % store) > or (if available) > cursor.execute('select max(ID) from ?CustomerData;', (store, )) > but the implementation means is irrelevant. [Do you know any database on which the latter technique will work? I realise I said "most", but I suspect I should have said "all" - at least I can't think of a counterexample now I have put myself on the spot]. I pointed this out to the OP some time ago. It won't make any difference. In the particular code you mention the tables didn't seem to be constructed from user input, thereby removing much of the danger of SQL injection exploits, but I could be wrong - reading it made me feel squeamish so I didn't analyze it thoroughly. The whole thing is a hodge-podge of Python and HTML produced by an individual who appears to feel it isn't necessary to understand either HTTP or HTML in order to produce dynamic web sites, whose grasp of Python itself is slight and whose response to constructive criticism is to defer acting on it until it is no longer going to be helpful. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From philip at semanchuk.com Wed Jan 6 22:53:01 2010 From: philip at semanchuk.com (Philip Semanchuk) Date: Wed, 6 Jan 2010 22:53:01 -0500 Subject: GUI for multiplatform multimedia project In-Reply-To: References: Message-ID: On Jan 6, 2010, at 4:53 PM, wrote: > Hi everyone, > > I posted that question on a python-forum, but got answer, so I ask > here. > > I'm working on an artistic project and I'm looking for the best > cross-platform GUI solution. The problem is that it's gonna be a > tool that > will have to be double-click installable/runnable and pre- > installation of > any libraries for end-users is very much like an evil. It really has > to be > double-click tool > > My first thought was PyQt, because it's a real framework with a lot of > stuff inside (including Phonon) and I know some cross-platform media > software written in C++ QT (like VLC). But on the other hand I've > heard > that it's not that easy to make it "double-clicky" multi-platform. > Is that > true? > > Another thing that matters for me is ease of integration with > libraries > like OpenCV. > > I will be VERY thankful for any help. I'm SO tired googling the > problem > (it's like weeks now!!) Cze?? trzewiczek, I'm not well qualified to answer your question but since no one else has I'll make a stab at it. I'm interested because I'll have to solve the same problem in some months time with a suite of wxPython-based apps that rely on a lot of libraries. When you say "cross platform" I assume you mean OS X, Linux and Windows. All three of those require a different technique to create a double-clickable app, so in a way you have three problems to solve, not just one. For instance, py2exe is a popular solution for building monolithic application blobs for Windows, while py2app does the same for the Mac. It might be true that PyQt isn't all that easy to cram into a "double- clicky" application blob. But I don't know that e.g. wxPython will be any easier. Bundling a Python GUI app is non-trivial. I think that's exactly why you've been able to spend weeks googling -- there's no easy, obvious, this-is-it, canonical solution. Different people use different techniques. So to get back to your original question, although I don't know if one GUI toolkit is easier to bundle than another, I suspect that none of them are easy. I would pick the GUI toolkit based on what suits your technical & licensing needs best and worry about distribution later. That's one man's opinion. I hope someone else with more practical experience in the matter can offer you some advice. Good luck Philip From aioe.org at technicalbloke.com Wed Jan 6 23:25:30 2010 From: aioe.org at technicalbloke.com (r0g) Date: Thu, 07 Jan 2010 04:25:30 +0000 Subject: GUI for multiplatform multimedia project References: Message-ID: Philip Semanchuk wrote: > > On Jan 6, 2010, at 4:53 PM, > wrote: > >> Hi everyone, >> >> I posted that question on a python-forum, but got answer, so I ask here. >> >> I'm working on an artistic project and I'm looking for the best >> cross-platform GUI solution. The problem is that it's gonna be a tool >> that >> will have to be double-click installable/runnable and pre-installation of >> any libraries for end-users is very much like an evil. It really has >> to be >> double-click tool >> >> My first thought was PyQt, because it's a real framework with a lot of >> stuff inside (including Phonon) and I know some cross-platform media >> software written in C++ QT (like VLC). But on the other hand I've heard >> that it's not that easy to make it "double-clicky" multi-platform. Is >> that >> true? >> >> Another thing that matters for me is ease of integration with libraries >> like OpenCV. >> >> I will be VERY thankful for any help. I'm SO tired googling the problem >> (it's like weeks now!!) > > Cze?? trzewiczek, > I'm not well qualified to answer your question but since no one else has > I'll make a stab at it. I'm interested because I'll have to solve the > same problem in some months time with a suite of wxPython-based apps > that rely on a lot of libraries. > Actually the one I wouldn't recommend for the OP is wxPython. It's very good and I use it almost exclusively but as the OP has an "artistic project" I'm not sure it would be ideal. wxPython works by leveraging the runtime platform's native toolkit so your apps are very portable and fit in nicely with the look and feel of whatever system they're on but... Sometimes that can make it hard to do very precise layouts that look the same across platforms. Sometimes it doesn't manifest and the times where it does wont be a problem for the vast majority of apps but if you require very granular control you may be better off picking one toolkit and sticking to it. I'd pick QT over GTK if I had to make a choice and also I agree with the last poster that packaging any of them (for windows at least) will require a fair degree of faff. Roger. From aioe.org at technicalbloke.com Wed Jan 6 23:38:41 2010 From: aioe.org at technicalbloke.com (r0g) Date: Thu, 07 Jan 2010 04:38:41 +0000 Subject: Assertions, challenges, and polite discourse References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> <87hbqywzxt.fsf_-_@benfinney.id.au> <487fa8b4-5e27-4e61-99a3-4c57370154e6@c3g2000yqd.googlegroups.com> Message-ID: alex23 wrote: > r0g wrote: >> Well I think sometimes, for the sake of expediency and overall >> pleasantness, it's better to let the smaller things go: and if you just >> can't let them go then at least try and issue corrections in a friendly >> manner rather than a cold or pious one. > > The irony, it is too rich... Hi Alex, I didn't think anyone else would still be reading this thread! The above was in the context of people who have done nothing to warrant coldness and piety. Sadly we seem to have gotten way beyond that. You are right though, in retrospect I should have let this slip, clearly no minds have been changed by not doing :/ Roger. From ashish.vyas at motorola.com Wed Jan 6 23:48:58 2010 From: ashish.vyas at motorola.com (VYAS ASHISH M-NTB837) Date: Thu, 7 Jan 2010 12:48:58 +0800 Subject: lxml 2.2.4 on python3.1, Windows XP gives importerror In-Reply-To: <4B44BE0A.7060508@activestate.com> Message-ID: <7C57DB58C81FB64CA979EC6C4DB73E7A03E70654@ZMY16EXM67.ds.mot.com> Processor is Intel Pentium 32 bit. import platform print (platform.architecture()) gives -> ('32bit', 'WindowsPE') Regards, Ashish Vyas -----Original Message----- From: Sridhar Ratnakumar [mailto:sridharr at activestate.com] Sent: Wednesday, January 06, 2010 10:15 PM To: VYAS ASHISH M-NTB837 Cc: python-list at python.org Subject: Re: lxml 2.2.4 on python3.1, Windows XP gives importerror On 1/5/2010 1:38 AM, VYAS ASHISH M-NTB837 wrote: > Dear All > I have Python 3.1 installed on Windows XP and Works nice. > I downloaded lxml 2.2.4 (lxml-2.2.4.win32-py3.1.exe) from pypi. > When I try: > from lxml import etree > I get: > ImportError: DLL load failed: This application has failed to start > because the application configuration is incorrect. Reinstalling the > application may fix this problem. Are you by any chance running 64-bit Python (whereas the lxml binary you have installed is 32-bit)? C:> python -c "import platform; print platform.architecture()" -srid From jkordani at intlogsys.com Thu Jan 7 00:07:31 2010 From: jkordani at intlogsys.com (Joshua Kordani) Date: Thu, 07 Jan 2010 00:07:31 -0500 Subject: from datetime.datetime import today not working. python2.6.4 on windows Message-ID: <4B456C13.7060005@intlogsys.com> Greetings all! So I'm reading through the manual and I get to the point where it talks about packages and how to import them. namely section 6.4 in the tutorial. I wont repeat the section here, but I want to understand whats going on in the following (as typed on my computer). Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import datetime >>> dir(datetime) ['MAXYEAR', 'MINYEAR', '__doc__', '__name__', '__package__', 'date', 'datetime', 'datetime_CAPI', 'time', 'timedelta', 'tzinfo'] >>> dir(datetime.datetime) ['__add__', '__class__', '__delattr__', '__doc__', '__eq__', '__format__', '__ge __', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', ' __ne__', '__new__', '__radd__', '__reduce__', '__reduce_ex__', '__repr__', '__rs ub__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', 'a stimezone', 'combine', 'ctime', 'date', 'day', 'dst', 'fromordinal', 'fromtimest amp', 'hour', 'isocalendar', 'isoformat', 'isoweekday', 'max', 'microsecond', 'm in', 'minute', 'month', 'now', 'replace', 'resolution', 'second', 'strftime', 's trptime', 'time', 'timetuple', 'timetz', 'today', 'toordinal', 'tzinfo', 'tzname ', 'utcfromtimestamp', 'utcnow', 'utcoffset', 'utctimetuple', 'weekday', 'year'] >>> from datetime.datetime import today Traceback (most recent call last): File "", line 1, in ImportError: No module named datetime >>> so dir on datetime shows symbols date, time, datetime,etc dir on datetime shows today, now, etc lets say for arguments sake that I want to just import the today function, according to the documentation, the line should be: from datetime.datetime import today. as you can see, that didn't work. why not? Josh From benjamin.kaplan at case.edu Thu Jan 7 00:39:50 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Thu, 7 Jan 2010 00:39:50 -0500 Subject: from datetime.datetime import today not working. python2.6.4 on windows In-Reply-To: <4B456C13.7060005@intlogsys.com> References: <4B456C13.7060005@intlogsys.com> Message-ID: On Thu, Jan 7, 2010 at 12:07 AM, Joshua Kordani wrote: > Greetings all! > > So I'm reading through the manual and I get to the point where it talks > about packages and how to import them. ?namely section 6.4 in the tutorial. > ?I wont repeat the section here, but I want to understand whats going on in > the following (as typed on my computer). > > Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)] > on > win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> import datetime >>>> dir(datetime) > ['MAXYEAR', 'MINYEAR', '__doc__', '__name__', '__package__', 'date', > 'datetime', > ?'datetime_CAPI', 'time', 'timedelta', 'tzinfo'] >>>> dir(datetime.datetime) > ['__add__', '__class__', '__delattr__', '__doc__', '__eq__', '__format__', > '__ge > __', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', > '__lt__', ' > __ne__', '__new__', '__radd__', '__reduce__', '__reduce_ex__', '__repr__', > '__rs > ub__', '__setattr__', '__sizeof__', '__str__', '__sub__', > '__subclasshook__', 'a > stimezone', 'combine', 'ctime', 'date', 'day', 'dst', 'fromordinal', > 'fromtimest > amp', 'hour', 'isocalendar', 'isoformat', 'isoweekday', 'max', > 'microsecond', 'm > in', 'minute', 'month', 'now', 'replace', 'resolution', 'second', > 'strftime', 's > trptime', 'time', 'timetuple', 'timetz', 'today', 'toordinal', 'tzinfo', > 'tzname > ', 'utcfromtimestamp', 'utcnow', 'utcoffset', 'utctimetuple', 'weekday', > 'year'] > >>>> from datetime.datetime import today > Traceback (most recent call last): > ?File "", line 1, in > ImportError: No module named datetime >>>> > > so dir on datetime shows symbols date, time, datetime,etc > dir on datetime shows today, now, etc > > lets say for arguments sake that I want to just import the today function, > according to the documentation, the line should be: > from datetime.datetime import today. > > as you can see, that didn't work. ?why not? > >>> import datetime >>> type(datetime.datetime) datetime.datetime is a type (so it's a class), not a module and today is an attribute of that type. You can't import from a class. From steve at holdenweb.com Thu Jan 7 01:12:21 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 07 Jan 2010 01:12:21 -0500 Subject: from datetime.datetime import today not working. python2.6.4 on windows In-Reply-To: <4B456C13.7060005@intlogsys.com> References: <4B456C13.7060005@intlogsys.com> Message-ID: Joshua Kordani wrote: > Greetings all! > > So I'm reading through the manual and I get to the point where it talks > about packages and how to import them. namely section 6.4 in the > tutorial. I wont repeat the section here, but I want to understand > whats going on in the following (as typed on my computer). > > Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit > (Intel)] on > win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> import datetime >>>> dir(datetime) > ['MAXYEAR', 'MINYEAR', '__doc__', '__name__', '__package__', 'date', > 'datetime', > 'datetime_CAPI', 'time', 'timedelta', 'tzinfo'] >>>> dir(datetime.datetime) > ['__add__', '__class__', '__delattr__', '__doc__', '__eq__', > '__format__', '__ge > __', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', > '__lt__', ' > __ne__', '__new__', '__radd__', '__reduce__', '__reduce_ex__', > '__repr__', '__rs > ub__', '__setattr__', '__sizeof__', '__str__', '__sub__', > '__subclasshook__', 'a > stimezone', 'combine', 'ctime', 'date', 'day', 'dst', 'fromordinal', > 'fromtimest > amp', 'hour', 'isocalendar', 'isoformat', 'isoweekday', 'max', > 'microsecond', 'm > in', 'minute', 'month', 'now', 'replace', 'resolution', 'second', > 'strftime', 's > trptime', 'time', 'timetuple', 'timetz', 'today', 'toordinal', 'tzinfo', > 'tzname > ', 'utcfromtimestamp', 'utcnow', 'utcoffset', 'utctimetuple', 'weekday', > 'year'] > >>>> from datetime.datetime import today > Traceback (most recent call last): > File "", line 1, in > ImportError: No module named datetime >>>> > > so dir on datetime shows symbols date, time, datetime,etc > dir on datetime shows today, now, etc > > lets say for arguments sake that I want to just import the today > function, according to the documentation, the line should be: > from datetime.datetime import today. > > as you can see, that didn't work. why not? > Because datetime is a module, but datetime.datetime is a class. You can import individual names from a module, but a class is a monolithic all-or-nothing chunk. There's the potential for confusion because Python also has "packages", which are like modules but more structured: you can import a module from a package: >>> from xml import etree or import the package module directly: >>> import xml.etree) You can also import a submodule from the module: >>> from xml.etree import cElementTree or import the submodule directly >>> import xml.etree.cElementTree You can even import a class from the submodule: >>> from xml.etree.cElementTree import Element but what you *can't* do is import the submodule's class directly: >>> import xml.etree.cElementTree.Element Traceback (most recent call last): File "", line 1, in ImportError: No module named Element However, what you did was import the datetime module (which is not a package, and does not therefore contains submodules), and reference the datetime class within that module (datetime.datetime), which is all OK. But datetime.datetime is a class, not a module, so you can't import anything from it. Neither can you import it directly: >>> import datetime.datetime Traceback (most recent call last): File "", line 1, in ImportError: No module named datetime (in the last line, the name "datetime" refers to the class, not themodule. Hope this helps. You may learn a bit more by actually looking at the source of the module, which probably lives in C:\Python26\Lib\datetime.py regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From gherron at islandtraining.com Thu Jan 7 01:21:56 2010 From: gherron at islandtraining.com (Gary Herron) Date: Wed, 06 Jan 2010 22:21:56 -0800 Subject: from datetime.datetime import today not working. python2.6.4 on windows In-Reply-To: <4B456C13.7060005@intlogsys.com> References: <4B456C13.7060005@intlogsys.com> Message-ID: <4B457D84.4050207@islandtraining.com> Joshua Kordani wrote: > Greetings all! > > So I'm reading through the manual and I get to the point where it > talks about packages and how to import them. namely section 6.4 in > the tutorial. I wont repeat the section here, but I want to > understand whats going on in the following (as typed on my computer). > > Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit > (Intel)] on > win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> import datetime > >>> dir(datetime) > ['MAXYEAR', 'MINYEAR', '__doc__', '__name__', '__package__', 'date', > 'datetime', > 'datetime_CAPI', 'time', 'timedelta', 'tzinfo'] > >>> dir(datetime.datetime) > ['__add__', '__class__', '__delattr__', '__doc__', '__eq__', > '__format__', '__ge > __', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', > '__lt__', ' > __ne__', '__new__', '__radd__', '__reduce__', '__reduce_ex__', > '__repr__', '__rs > ub__', '__setattr__', '__sizeof__', '__str__', '__sub__', > '__subclasshook__', 'a > stimezone', 'combine', 'ctime', 'date', 'day', 'dst', 'fromordinal', > 'fromtimest > amp', 'hour', 'isocalendar', 'isoformat', 'isoweekday', 'max', > 'microsecond', 'm > in', 'minute', 'month', 'now', 'replace', 'resolution', 'second', > 'strftime', 's > trptime', 'time', 'timetuple', 'timetz', 'today', 'toordinal', > 'tzinfo', 'tzname > ', 'utcfromtimestamp', 'utcnow', 'utcoffset', 'utctimetuple', > 'weekday', 'year'] > > >>> from datetime.datetime import today > Traceback (most recent call last): > File "", line 1, in > ImportError: No module named datetime > >>> > > so dir on datetime shows symbols date, time, datetime,etc > dir on datetime shows today, now, etc > > lets say for arguments sake that I want to just import the today > function, according to the documentation, the line should be: > from datetime.datetime import today. > > as you can see, that didn't work. why not? Just a little confusion on your part. datetime is a module, so you can import it (as you did) and objects from it datetime.datetime is a type (like a class is a type) not a module so you cannot import from it. So you can import datetime and you can from datetime import but that's all the further you can go with imports. Some of the confusion may be from having a class (well actually it's a type) with the same name as its module. This is now considered bad form. You might achieve the effect you were you were trying for like this: import datetime today = datetime.datetime.today or from datetime import datetime today = datetime.today Then you'll be able to call today() at any time. Gary Herron > > Josh From metolone+gmane at gmail.com Thu Jan 7 01:25:40 2010 From: metolone+gmane at gmail.com (Mark Tolonen) Date: Wed, 6 Jan 2010 22:25:40 -0800 Subject: Pass multidimensional array (matrix) to c function using ctypes References: <63ac1a01-8491-4885-bae7-cb884abb582f@34g2000yqp.googlegroups.com> Message-ID: "Daniel Platz" wrote in message news:63ac1a01-8491-4885-bae7-cb884abb582f at 34g2000yqp.googlegroups.com... > Hello, > > I would like to pass a two dimensional array to C function in a dll. I > use ctypes to call the function. > > I compile the dll with visual studio 2008 express and my C source code > looks like this. > > #include > #ifdef __cplusplus > extern "C" { // only need to export C interface if > // used by C++ source code > using namespace std; > #endif > > > __declspec(dllexport) int print(double** ptr, int ny, int nx) > { > int i, j; > for(i=0; i { > for(j=0; j { > printf("%.3f \t", *(*(ptr+i)+j)); > } > printf("\n"); > } > return 0; > } > > #ifdef __cplusplus > } > #endif > > As you can see the function expects a doube** variable. I tried to > call this function from Python with this code. > > import ctypes as ct > > matrix = ct.cdll.LoadLibrary('matrix.dll') > getattr(matrix, 'print_matrix') > > ptr_type = ct.c_double * 5 * 4 > ptr = ptr_type() > matrix.print_matrix(ptr, ct.c_int(4), ct.c_int(5)) > > I expected to see a matrix showing only zeros since I can address the > single entries in Python by ptr[i][j]. Instead the interpreter returns > with the error message > > WindowsError: exception: access violation reading 0x000000 > WARNING: Failure executing file: > > Furthermore, I am wondering if there is a fast way to use a numpy 2D > array instead or alternatively to cast the ctypes array into a numpy > array. > > Has someone an idea to help me? A two-dimentional array is not equivalent to a double**, so depending on what your requirements are there are two solutions. 1. To work with your original matrix.c code above, some Python code that works is: ------------------------------------- import ctypes as ct # I like C types in caps DOUBLE = ct.c_double PDOUBLE = ct.POINTER(DOUBLE) PPDOUBLE = ct.POINTER(PDOUBLE) INT = ct.c_int matrix = ct.cdll.LoadLibrary('matrix.dll') print_matrix = getattr(matrix,'print') # An array of doubles can be passed to a function that takes double*. DBL5ARR = DOUBLE * 5 # An array of double* can be passed to your function as double**. PDBL4ARR = PDOUBLE * 4 # Declare double* array. ptr = PDBL4ARR() for i in range(4): # fill out each pointer with an array of doubles. ptr[i] = DBL5ARR() for j in range(5): ptr[i][j] = i + j # just to initialize the actual doubles. print_matrix(ptr,4,5) ----------------------------------------------- 2. In C, multidimensional arrays are really just single dimentional arrays where the compiler does the math for you, so if you declare your matrix print function like this: ----- matrix2.c --------------------------- #include #ifdef __cplusplus extern "C" { // only need to export C interface if // used by C++ source code #endif __declspec(dllexport) int print(double* ptr, int ny, int nx) { int i, j; for(i=0; i <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: r0g wrote: > Steven D'Aprano wrote: >> On Wed, 06 Jan 2010 23:58:21 +0000, r0g wrote: >> >>> Steven D'Aprano wrote: >>>> On Wed, 06 Jan 2010 22:46:33 +0000, r0g wrote: >>>> >>>>> Grant Edwards wrote: >>>>>> On 2010-01-06, r0g wrote: >>> See? Spoiling for an argument even now! I never said you weren't allowed >>> to butt in, just that you did. Butting in is fine, that's half the point > detect no background seething at all here. > > Roger. > Actually guys I've just reread this thread from the start and I clearly did overreact, sorry about that. In fact it reminded me of this comic... http://xkcd.com/481/ Time to smoke a joint and get a half decent nights sleep I think! Roger. From QBX634 at motorola.com Thu Jan 7 02:16:32 2010 From: QBX634 at motorola.com (Mishra Gopal-QBX634) Date: Thu, 7 Jan 2010 15:16:32 +0800 Subject: How to reduce the memory size of python Message-ID: <6AFAAA2E71101642861E945A057CE7CD023A882B@ZMY16EXM70.ds.mot.com> Hi, When i write following pyhon program and execute it in linux machine, if __name__=='__main__': while True: pass When i check the VmRSS size, it shows 2956 KB in size. Is there any way to reduce the memory size taken by python. I am working in flash memory devices. Any suggession is highly helpfull to me. Thanks, Gopal -------------- next part -------------- An HTML attachment was scrubbed... URL: From trzewiczek at trzewiczek.info Thu Jan 7 02:19:45 2010 From: trzewiczek at trzewiczek.info (t r z e w i c z e k) Date: Thu, 07 Jan 2010 08:19:45 +0100 Subject: Validating cells of a table PyQt In-Reply-To: <2d6c6f77-30cc-4787-919a-f2781026dd66@a21g2000yqc.googlegroups.com> References: <2d6c6f77-30cc-4787-919a-f2781026dd66@a21g2000yqc.googlegroups.com> Message-ID: On Thu, 07 Jan 2010 01:37:05 +0100, Zabin wrote: > Hey! > > I am new PyQT programmer. I am trying to create a table in which cells > only take in numeric data. I am unable to use setValidator on table > cells. Looking around i found some material on the editor attribute > but I dont know how to apply this. Any help will be appreciated > > Cheers! Hi, I'm not an expert but could you write, why you can't use validators on the cells? There is this free ebook of first edition of "C++ GUI Programming with Qt 4" that has an excel like application as an example and I remember there was a part with validators that looked pretty nice. You can easly and legally download this book from various sites. Best from Poland, trzewiczek -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From trzewiczek at trzewiczek.info Thu Jan 7 02:22:53 2010 From: trzewiczek at trzewiczek.info (t r z e w i c z e k) Date: Thu, 07 Jan 2010 08:22:53 +0100 Subject: Python interactive terminal in Ubuntu Linux : some keys fouled up In-Reply-To: References: Message-ID: On Thu, 07 Jan 2010 01:09:08 +0100, casevh wrote: > On Jan 6, 2:40 pm, pdlem... at earthlink.net wrote: >> Have recently moved from XP to Ubuntu Linux. >> Successfully installed Python 3.1.1 in the Ubuntu 9.04 >> release on my desktop. >> Problem is the python interactive terminal >>> . >> Many of the keys now do not work. >> eg pressing left-arrow yields ^[[D >> right-arrow ^[[C >> Home ^[OH >> Del ^[[3~ >> up-arrow ^[[A >> Frustrating as I use all these , esp up-arrow to >> repeat recent lines. Found the same thing on >> my sons MacBook. >> >> This is not mentioned in two recent Python books >> or one on Ubuntu. Nor could I found help on the www. >> >> Is there any work-around ? Should I just forget >> the python prompt >>> ? >> Thanks, Dave pdlem... at earthlink.net > > Assuming you compiled the source code, you will also need to install > "libreadline5-dev" via Synaptic or apt-get. > > casevh Or you can use Idle instead of terminal. It works kind of different. sudo apt-get install idle-python3.1 will do the thing! cheers, trzewiczek -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From steve at holdenweb.com Thu Jan 7 02:29:50 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 07 Jan 2010 02:29:50 -0500 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4ff510b3-12c0-4a7c-8607-16e45ab24315@j14g2000yqm.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: <4B458D6E.5020204@holdenweb.com> r0g wrote: > r0g wrote: >> Steven D'Aprano wrote: >>> On Wed, 06 Jan 2010 23:58:21 +0000, r0g wrote: >>> >>>> Steven D'Aprano wrote: >>>>> On Wed, 06 Jan 2010 22:46:33 +0000, r0g wrote: >>>>> >>>>>> Grant Edwards wrote: >>>>>>> On 2010-01-06, r0g wrote: >>>> See? Spoiling for an argument even now! I never said you weren't allowed >>>> to butt in, just that you did. Butting in is fine, that's half the point > >> detect no background seething at all here. >> >> Roger. >> > > > Actually guys I've just reread this thread from the start and I clearly > did overreact, sorry about that. In fact it reminded me of this comic... > > http://xkcd.com/481/ > > Time to smoke a joint and get a half decent nights sleep I think! > > Roger. Brilliant. It takes a real whole human being to make an admission like that (or even to bother to question their own behavior sufficiently to bother re-reading the thread). I think a lot more of you for the admission. Not that you really need care one way or the other what *I* think ... We all have off-days. About three years ago I went completely apeshit at someone in a fit of tiredness-induced pique, as was at one point amply evidenced by http://mail.python.org/pipermail/python-list/2007-August/thread.html#454510 Strangely the universe conspired to stamp on that thread, and it appears to have been lost to the python.org archives. A Google search for "Steve Holden" and "not a fucking computer", however, reveals that my shame has not been totally expunged. [If only I could rickroll those links ... :)] regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Thu Jan 7 02:49:39 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 07 Jan 2010 02:49:39 -0500 Subject: How to reduce the memory size of python In-Reply-To: <6AFAAA2E71101642861E945A057CE7CD023A882B@ZMY16EXM70.ds.mot.com> References: <6AFAAA2E71101642861E945A057CE7CD023A882B@ZMY16EXM70.ds.mot.com> Message-ID: Mishra Gopal-QBX634 wrote: > Hi, > > When i write following pyhon program and execute it in linux machine, > > if __name__=='__main__': > while True: > pass > > When i check the VmRSS size, it shows 2956 KB in size. > > Is there any way to reduce the memory size taken by python. > > I am working in flash memory devices. > > Any suggession is highly helpfull to me. > It would not be easy to reduce the size of the standard interpreter, but there are various implementations for embedded devices. You may get some help from http://wiki.python.org/moin/Tiny%20Python and http://wiki.python.org/moin/Tiny%20Python A company called Synapse has a working cut-down Python implementation that they embed in their 802.15 wireless mesh devices, which IIRC occupies less than 128K. There is a lot in the standard interpreter that you won't need - many embedded systems don't need floating point arithmetic, for example. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From jkordani at intlogsys.com Thu Jan 7 02:51:07 2010 From: jkordani at intlogsys.com (Joshua Kordani) Date: Thu, 07 Jan 2010 02:51:07 -0500 Subject: from datetime.datetime import today not working. python2.6.4 on windows In-Reply-To: <4B457D84.4050207@islandtraining.com> References: <4B456C13.7060005@intlogsys.com> <4B457D84.4050207@islandtraining.com> Message-ID: <4B45926B.1040409@intlogsys.com> Gary Herron wrote: > Joshua Kordani wrote: >> Greetings all! >> >> So I'm reading through the manual and I get to the point where it >> talks about packages and how to import them. namely section 6.4 in >> the tutorial. I wont repeat the section here, but I want to >> understand whats going on in the following (as typed on my computer). >> >> Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit >> (Intel)] on >> win32 >> Type "help", "copyright", "credits" or "license" for more information. >> >>> import datetime >> >>> dir(datetime) >> ['MAXYEAR', 'MINYEAR', '__doc__', '__name__', '__package__', 'date', >> 'datetime', >> 'datetime_CAPI', 'time', 'timedelta', 'tzinfo'] >> >>> dir(datetime.datetime) >> ['__add__', '__class__', '__delattr__', '__doc__', '__eq__', >> '__format__', '__ge >> __', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', >> '__lt__', ' >> __ne__', '__new__', '__radd__', '__reduce__', '__reduce_ex__', >> '__repr__', '__rs >> ub__', '__setattr__', '__sizeof__', '__str__', '__sub__', >> '__subclasshook__', 'a >> stimezone', 'combine', 'ctime', 'date', 'day', 'dst', 'fromordinal', >> 'fromtimest >> amp', 'hour', 'isocalendar', 'isoformat', 'isoweekday', 'max', >> 'microsecond', 'm >> in', 'minute', 'month', 'now', 'replace', 'resolution', 'second', >> 'strftime', 's >> trptime', 'time', 'timetuple', 'timetz', 'today', 'toordinal', >> 'tzinfo', 'tzname >> ', 'utcfromtimestamp', 'utcnow', 'utcoffset', 'utctimetuple', >> 'weekday', 'year'] >> >> >>> from datetime.datetime import today >> Traceback (most recent call last): >> File "", line 1, in >> ImportError: No module named datetime >> >>> >> >> so dir on datetime shows symbols date, time, datetime,etc >> dir on datetime shows today, now, etc >> >> lets say for arguments sake that I want to just import the today >> function, according to the documentation, the line should be: >> from datetime.datetime import today. >> >> as you can see, that didn't work. why not? > > Just a little confusion on your part. > > datetime is a module, so you can import it (as you did) and objects from it > > datetime.datetime is a type (like a class is a type) not a module so you > cannot import from it. > > > So you can > import datetime > and you can > from datetime import > but that's all the further you can go with imports. > > Some of the confusion may be from having a class (well actually it's a > type) with the same name as its module. This is now considered bad form. > > You might achieve the effect you were you were trying for like this: > import datetime > today = datetime.datetime.today > or > from datetime import datetime > today = datetime.today > Then you'll be able to call > today() > at any time. > > Gary Herron > > >> >> Josh > @ All thanks for the responses! That was the key bit I was missing. Josh From marco.salden at gmail.com Thu Jan 7 03:00:05 2010 From: marco.salden at gmail.com (Marco Salden) Date: Thu, 7 Jan 2010 00:00:05 -0800 (PST) Subject: Do I have to use threads? References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> Message-ID: <432961e5-627e-452d-93db-9562951551d2@e27g2000yqd.googlegroups.com> On Jan 6, 5:36?am, Philip Semanchuk wrote: > On Jan 5, 2010, at 11:26 PM, aditya shukla wrote: > > > Hello people, > > > I have 5 directories corresponding 5 ?different urls .I want to ? > > download > > images from those urls and place them in the respective ? > > directories.I have > > to extract the contents and download them simultaneously.I can ? > > extract the > > contents and do then one by one. My questions is for doing it ? > > simultaneously > > do I have to use threads? > > No. You could spawn 5 copies of wget (or curl or a Python program that ? > you've written). Whether or not that will perform better or be easier ? > to code, debug and maintain depends on the other aspects of your ? > program(s). > > bye > Philip Yep, the more easier and straightforward the approach, the better: threads are always (programmers')-error-prone by nature. But my question would be: does it REALLY need to be simultaneously: the CPU/OS only has more overhead doing this in parallel with processess. Measuring sequential processing and then trying to optimize (e.g. for user response or whatever) would be my prefered way to go. Less=More. regards, Marco From andrew.gillanders at uqconnect.edu.au Thu Jan 7 03:19:47 2010 From: andrew.gillanders at uqconnect.edu.au (Andrew Gillanders) Date: Thu, 7 Jan 2010 18:19:47 +1000 Subject: buffer interface problem Message-ID: <3C5B8086-5D3D-444B-B9D2-C9152EBF5DA3@uqconnect.edu.au> I have run into a problem running a Python script that is part of the TerraGear suite for building scenery for FlightGear. I am using Mac OS X 10.4, running Python (version 3.0.1) in a Unix terminal. The purpose of the script is to walk a directory tree, unzipping files, and passing the contents to an executable C programme. The problem occurs here: gzin = GzipFile(fname, 'rb') data = gzin.readline() min_x,min_y = map(atoi,data.split()[:2]) The input file, when uncompressed, is an ASCII file with a line with two numbers, then a line of four numbers, then many long lines of numbers. I can see what the last is trying to do: split the string into two words, convert them to integers, and assign them to min_x and min_y. At the third line, I get the message "expected an object with the buffer interface". Which object is it referring to? Have some functions been changed to pass buffer objects instead of strings? How can I fix the source code to make it run? Any help appreciated Andrew (Python newbie) From QBX634 at motorola.com Thu Jan 7 03:34:23 2010 From: QBX634 at motorola.com (Mishra Gopal-QBX634) Date: Thu, 7 Jan 2010 16:34:23 +0800 Subject: How to reduce the memory size of python In-Reply-To: References: <6AFAAA2E71101642861E945A057CE7CD023A882B@ZMY16EXM70.ds.mot.com> Message-ID: <6AFAAA2E71101642861E945A057CE7CD023A884D@ZMY16EXM70.ds.mot.com> Hi, I use twisted framework too to handle the xmlrpc request. It takes around 3-4MB of memory while importing itself. Is there any python coding standard I should follow to save the memory. Like import logging takes 1MB of memory. We only use on function getLogger by 'from logging import getLogger' But it still take the same 1 MB memory. Instead of loading whole logging module only load the getLogger function. I there any way to save the memory with taking care of small things in code.. Thanks, Gopal -----Original Message----- From: python-list-bounces+qbx634=motorola.com at python.org [mailto:python-list-bounces+qbx634=motorola.com at python.org] On Behalf Of Steve Holden Sent: Thursday, January 07, 2010 1:20 PM To: python-list at python.org Subject: Re: How to reduce the memory size of python Mishra Gopal-QBX634 wrote: > Hi, > > When i write following pyhon program and execute it in linux machine, > > if __name__=='__main__': > while True: > pass > > When i check the VmRSS size, it shows 2956 KB in size. > > Is there any way to reduce the memory size taken by python. > > I am working in flash memory devices. > > Any suggession is highly helpfull to me. > It would not be easy to reduce the size of the standard interpreter, but there are various implementations for embedded devices. You may get some help from http://wiki.python.org/moin/Tiny%20Python and http://wiki.python.org/moin/Tiny%20Python A company called Synapse has a working cut-down Python implementation that they embed in their 802.15 wireless mesh devices, which IIRC occupies less than 128K. There is a lot in the standard interpreter that you won't need - many embedded systems don't need floating point arithmetic, for example. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ -- http://mail.python.org/mailman/listinfo/python-list From rajat.dudeja at gmail.com Thu Jan 7 04:12:23 2010 From: rajat.dudeja at gmail.com (Rajat) Date: Thu, 7 Jan 2010 01:12:23 -0800 (PST) Subject: How to execute a script from another script and other script does not do busy wait. Message-ID: <76bff0b6-70cb-4960-9b4d-3af7a9a1e0e7@z41g2000yqz.googlegroups.com> I want to run a python script( aka script2) from another python script (aka script1). While script1 executes script2 it waits for script2 to complete and in doing so it also does some other useful work.(does not do a busy wait). My intention is to update a third party through script1 that script2 is going to take longer. Please suggest how should I go about implementing it. I'm currently executing it as: import main from script2 ret_code = main() return ret_code which surely is not going to achieve me what I intend. Thanks, Rajat. From clp2 at rebertia.com Thu Jan 7 04:13:50 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Thu, 7 Jan 2010 01:13:50 -0800 Subject: buffer interface problem In-Reply-To: <3C5B8086-5D3D-444B-B9D2-C9152EBF5DA3@uqconnect.edu.au> References: <3C5B8086-5D3D-444B-B9D2-C9152EBF5DA3@uqconnect.edu.au> Message-ID: <50697b2c1001070113i3a8ce2aay50dac69ef8a25a83@mail.gmail.com> On Thu, Jan 7, 2010 at 12:19 AM, Andrew Gillanders wrote: > I have run into a problem running a Python script that is part of the > TerraGear suite for building scenery for FlightGear. I am using Mac OS X > 10.4, running Python (version 3.0.1) in a Unix terminal. > > The purpose of the script is to walk a directory tree, unzipping files, and > passing the contents to an executable C program. The problem occurs here: > > ? ?gzin = GzipFile(fname, 'rb') > ? ?data = gzin.readline() > ? ?min_x,min_y = map(atoi,data.split()[:2]) > > The input file, when uncompressed, is an ASCII file with a line with two > numbers, then a line of four numbers, then many long lines of numbers. I can > see what the last is trying to do: split the string into two words, convert > them to integers, and assign them to min_x and min_y. > > At the third line, I get the message "expected an object with the buffer > interface". Which object is it referring to? The elements of the list produced by `data.split()[:2]`, which are either Unicode strings or bytestrings, neither of which are buffers. > Have some functions been > changed to pass buffer objects instead of strings? How can I fix the source > code to make it run? The error is being raised by the atoi() function (in the future, please post the full Traceback, not just the final error message). What module/library does your atoi() function come from (look for an `import` statement mentioning it)? The only functions by that name in the Python standard library both operate on strings, not buffers, and thus can't be the same one your code is using. In any case, replacing `atoi` with `int` in your code will likely solve the problem. The built-in int() function* can convert strings to integers. Cheers, Chris -- http://blog.rebertia.com *Not really a function, but close enough for newbie explanatory purposes. From ashish.vyas at motorola.com Thu Jan 7 04:21:20 2010 From: ashish.vyas at motorola.com (VYAS ASHISH M-NTB837) Date: Thu, 7 Jan 2010 17:21:20 +0800 Subject: How to execute a script from another script and other script does notdo busy wait. In-Reply-To: <76bff0b6-70cb-4960-9b4d-3af7a9a1e0e7@z41g2000yqz.googlegroups.com> Message-ID: <7C57DB58C81FB64CA979EC6C4DB73E7A03E7071F@ZMY16EXM67.ds.mot.com> Use threads Regards, Ashish Vyas -----Original Message----- From: python-list-bounces+ntb837=motorola.com at python.org [mailto:python-list-bounces+ntb837=motorola.com at python.org] On Behalf Of Rajat Sent: Thursday, January 07, 2010 2:42 PM To: python-list at python.org Subject: How to execute a script from another script and other script does notdo busy wait. I want to run a python script( aka script2) from another python script (aka script1). While script1 executes script2 it waits for script2 to complete and in doing so it also does some other useful work.(does not do a busy wait). My intention is to update a third party through script1 that script2 is going to take longer. From sjmachin at lexicon.net Thu Jan 7 04:43:59 2010 From: sjmachin at lexicon.net (John Machin) Date: Thu, 7 Jan 2010 01:43:59 -0800 (PST) Subject: Astronomy--Programs to Compute Siderial Time? References: Message-ID: On Jan 7, 2:40?pm, "W. eWatson" wrote: > John Machin wrote: > > > What you have been reading is the "Internal maintenance > > specification" (large font, near the top of the page) for the module. > > The xml file is the source of the docs, not meant to be user-legible. > > What is it used for? The maintainer of the module processes the xml file with some script or other to create the user-legible docs. > Do I need it? No. From eight32 at gmail.com Thu Jan 7 04:46:38 2010 From: eight32 at gmail.com (Stuart Murray-Smith) Date: Thu, 7 Jan 2010 11:46:38 +0200 Subject: Python books, literature etc In-Reply-To: <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: 2010/1/6 J : > A good point was brought up to me privately, and I agree completely, > that the OP should re-state the request with a bit more specifics... > > Since the OP says he is at least familiar with Python, does he need > info on beginner level books that are general purpose, or is he > interested in resources that are more specific (e.g. geared toward web > programming, mathematical analysis, data modeling, etc).... > > My suggestions were meant just as an example of what I use in the > course of learning something at the basic and intermediate level, once > it goes beyond that, it's useful to know WHAT you intend to do so you > can find the right resources to go in that direction. > > Just want to make sure I am not misunderstood or anything :-) > > For what it's worth, I also tend to collect technical books for some > reason... My wife is just barely tolerant of my bookshelf full of > things on various computer topics, astronomy, photography, radio and > antenna theory and so forth ;-) ?I just let her keep her shoe > collection, and we have a quid pro quo. Thanks J for your reply, much appreciated :) Oops, vague OP, my bad. Agreed, Google turns up myriad of topical books, and ESR's guide to smart questions [1] helps set the pace of list culture. I subscribe to various list servers, on one of them we gracefully accept that a question like my OP is looking for opinion on a matter, which I now understand would be a list's sub-culture? Anyways, to rephrase, could someone kindly mention any of their preferred Python books, websites, tutorials etc to help me get to an intermediate/advanced level? Something that would help me add functionality to Ubiquity, say. Have a great day! Stu@ [1] http://catb.org/~esr/faqs/smart-questions.html From gagsl-py2 at yahoo.com.ar Thu Jan 7 05:08:21 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Thu, 7 Jan 2010 02:08:21 -0800 (PST) Subject: Python books, literature etc References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: <304854.43355.qm@smtp139.mail.mud.yahoo.com> Stuart Murray-Smith wrote in news:aadebb9f1001070146n70f5be7bw2e515f9d4afed93e at mail.gmail.com: > Anyways, to rephrase, could someone kindly mention any of their > preferred Python books, websites, tutorials etc to help me get > to an intermediate/advanced level? Something that would help me > add functionality to Ubiquity, say. Have a look at the Getting Started section of the wiki: http://wiki.python.org/moin/ specially the PythonBooks section -- Gabriel Genellina From gagsl-py2 at yahoo.com.ar Thu Jan 7 05:08:21 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Thu, 7 Jan 2010 10:08:21 +0000 (UTC) Subject: Python books, literature etc References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: Stuart Murray-Smith wrote in news:aadebb9f1001070146n70f5be7bw2e515f9d4afed93e at mail.gmail.com: > Anyways, to rephrase, could someone kindly mention any of their > preferred Python books, websites, tutorials etc to help me get > to an intermediate/advanced level? Something that would help me > add functionality to Ubiquity, say. Have a look at the Getting Started section of the wiki: http://wiki.python.org/moin/ specially the PythonBooks section -- Gabriel Genellina From bhavik.patel at gmail.com Thu Jan 7 05:20:32 2010 From: bhavik.patel at gmail.com (Bhavik) Date: Thu, 7 Jan 2010 02:20:32 -0800 (PST) Subject: Call a DLL function with argument type as unsigned char * Message-ID: Hello, I am a newbie to the python language, and I need to call a DLL function from the python program. The DLL function has following prototype: unsigned int DLLFunction(unsigned char *, unsigned int); Now, I need to declare an array of 6 bytes in the python, and pass that array as the first argument to the DLL function. I tried some methods mentioned on the Internet, but it gives error like "can not convert argument 1". Can someone tell me how to do this? Thanks for the help. From bhavik.patel at gmail.com Thu Jan 7 05:28:33 2010 From: bhavik.patel at gmail.com (Bhavik) Date: Thu, 7 Jan 2010 02:28:33 -0800 (PST) Subject: Call a DLL function with argument type as unsigned char * References: Message-ID: Just to clarify, I am using Python 2.5.1 From gagsl-py2 at yahoo.com.ar Thu Jan 7 06:09:53 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Thu, 07 Jan 2010 08:09:53 -0300 Subject: creating tar file and streaming it over HTTP? References: <5222d923-7aef-47cf-a0d6-f37f09a66593@35g2000yqa.googlegroups.com> Message-ID: En Wed, 06 Jan 2010 13:39:08 -0300, pbienst escribi?: > The problem seems to be that the > receiving end (wsgi server) does not see the end of the data: > > socket = environ["wsgi.input"] > while True: > sys.stderr.write("before") > chunk = socket.read(4096) > sys.stderr.write("after") > if not chunk: > sys.stderr.write("done") > break > sys.stderr.write(chunk) > > There is data from the tar file being printed, but in the end it hangs > in the 'before' statement, and never gets to 'done'. I tried flushing > the fileobj created by makefile(), but that doesn't seem to help. > There is also no environ["CONTENT_LENGTH"] that I can use to detect > the end of the stream. > > I'm probably missing something basic here... Either send a "Connection: Close" header (and close the connection at the end), or a "Transfer-Encoding: chunked" header (see http://en.wikipedia.org/wiki/Chunked_transfer_encoding ) -- Gabriel Genellina From rajat.dudeja at gmail.com Thu Jan 7 06:19:19 2010 From: rajat.dudeja at gmail.com (Rajat) Date: Thu, 7 Jan 2010 03:19:19 -0800 (PST) Subject: How to execute a script from another script and other script does notdo busy wait. References: Message-ID: <016d2fad-a925-47f4-8b76-5602a8532651@r5g2000yqb.googlegroups.com> On Jan 7, 2:21?pm, "VYAS ASHISH M-NTB837" wrote: > Use threads > > Regards, > Ashish Vyas > > > > -----Original Message----- > From: python-list-bounces+ntb837=motorola.... at python.org > > [mailto:python-list-bounces+ntb837=motorola.... at python.org] On Behalf Of > Rajat > Sent: Thursday, January 07, 2010 2:42 PM > To: python-l... at python.org > Subject: How to execute a script from another script and other script > does notdo busy wait. > > I want to run a python script( aka script2) from another python script > (aka script1). While script1 executes script2 it waits for script2 to > complete and in doing so it also does some other useful work.(does not > do a busy wait). > > My intention is to update a third party through script1 that script2 is > going to take longer.- Hide quoted text - > > - Show quoted text - Thanks Ashish. I've single CPU machine. I've a feeling that the thread created, which would run script2, would eat up all of the CPU if I do not use sleep() in script2. That way, script1 would still be waiting for script2 to finish. Thus, my program is no way different from the sample program I posted earlier. Is there any other way out? From ashish.vyas at motorola.com Thu Jan 7 06:33:43 2010 From: ashish.vyas at motorola.com (VYAS ASHISH M-NTB837) Date: Thu, 7 Jan 2010 19:33:43 +0800 Subject: How to execute a script from another script and other script doesnotdo busy wait. In-Reply-To: <016d2fad-a925-47f4-8b76-5602a8532651@r5g2000yqb.googlegroups.com> Message-ID: <7C57DB58C81FB64CA979EC6C4DB73E7A03E70781@ZMY16EXM67.ds.mot.com> Did you try? Thanks Ashish. I've single CPU machine. I've a feeling that the thread created, which would run script2, would eat up all of the CPU if I do not use sleep() in script2. That way, script1 would still be waiting for script2 to finish. Thus, my program is no way different from the sample program I posted earlier. Is there any other way out? From ben+python at benfinney.id.au Thu Jan 7 06:38:58 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 07 Jan 2010 22:38:58 +1100 Subject: Exception as the primary error handling mechanism? References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4b5c45af-e571-4fbd-8c99-979fbcf29d10@c3g2000yqd.googlegroups.com> <87eim4j2ql.fsf@rudin.co.uk> <87tyv0ww69.fsf@benfinney.id.au> <87pr5ow50z.fsf@benfinney.id.au> <0353b69e$0$1336$c3e8da3@news.astraweb.com> Message-ID: <87tyuyuotp.fsf@benfinney.id.au> Steve Holden writes: > Brilliant. It takes a real whole human being to make an admission like > that (or even to bother to question their own behavior sufficiently to > bother re-reading the thread). I think a lot more of you for the > admission. Seconded. -- \ ?bash awk grep perl sed, df du, du-du du-du, vi troff su fsck | `\ rm * halt LART LART LART!? ?The Swedish BOFH, | _o__) alt.sysadmin.recovery | Ben Finney From rschroev_nospam_ml at fastmail.fm Thu Jan 7 06:43:14 2010 From: rschroev_nospam_ml at fastmail.fm (Roel Schroeven) Date: Thu, 07 Jan 2010 12:43:14 +0100 Subject: Exception as the primary error handling mechanism? In-Reply-To: <4b44b138$0$11237$426a74cc@news.free.fr> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4b44b138$0$11237$426a74cc@news.free.fr> Message-ID: Bruno Desthuilliers schreef: > Phlip a ?crit : >> On Jan 5, 8:49 pm, Steven D'Aprano >> wrote: >> >>>> (A related question - why can't I just go 'if record = method(): use >>>> (record)'. Why extra lines just to trap and assign the variable before >>>> using it?) >>> Because that idiom is responsible for probably the most common error in C >>> of all, at least one of the most common errors. Thank goodness Python >>> forbids such a dangerous construct. >> switching = for == is the "most common error in C"? >> >> I can't tell if you are joking. > > It's at least a _very_ common error in all languages that allow this > construct. Maybe it is, maybe it's not. All I know is my own experience; in all the years I've been doing C and C++ (1998 - now) I've made that mistake only twice. And in both cases I found the mistake very rapidly. > In C, it's common enough to gave birth to the "BestPractice" > you described, ie swapping operand orders in equality test to have the > compiler detect the problem - at least when one of the operand is a > function call expression or constant (it obviously won't 'work' when > both operands are variables). I've never liked that practice, for the following reasons: - As you say, it doesn't work when both operands are variables. In my experience, in many cases both operands are variables. - I tend to think that not following that practice trains me to be careful in all cases, whereas I'm afraid that following the practice will make me careless, which is dangerous in all the cases where the practice won't protect me. > Anyway: in Python, assignment is not an expression, and this isn't going > to change anytime soon. To be fully clear: I'm not advocating to change the current behavior in Python, I'm just stating my experience in other languages. -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven From steve at holdenweb.com Thu Jan 7 06:48:12 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 07 Jan 2010 06:48:12 -0500 Subject: How to reduce the memory size of python In-Reply-To: <6AFAAA2E71101642861E945A057CE7CD023A884D@ZMY16EXM70.ds.mot.com> References: <6AFAAA2E71101642861E945A057CE7CD023A882B@ZMY16EXM70.ds.mot.com> <6AFAAA2E71101642861E945A057CE7CD023A884D@ZMY16EXM70.ds.mot.com> Message-ID: Mishra Gopal-QBX634 wrote: > > Hi, > > I use twisted framework too to handle the xmlrpc request. It takes > around 3-4MB of memory while importing itself. > Is there any python coding standard I should follow to save the memory. > > Like import logging takes 1MB of memory. > We only use on function getLogger by 'from logging import getLogger' > > But it still take the same 1 MB memory. > > Instead of loading whole logging module only load the getLogger > function. > > I there any way to save the memory with taking care of small things in > code.. > No. You are seeing the size of the interpreter. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Thu Jan 7 06:50:51 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 07 Jan 2010 06:50:51 -0500 Subject: Call a DLL function with argument type as unsigned char * In-Reply-To: References: Message-ID: Bhavik wrote: > Just to clarify, I am using Python 2.5.1 > Take a look at the ctypes module, which allows you to do such things (at the risk of segmentation faults and the like if you get your calls wrong). regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From kk720801 at yahoo.com Thu Jan 7 07:22:21 2010 From: kk720801 at yahoo.com (Krzysztof Kobus) Date: Thu, 7 Jan 2010 04:22:21 -0800 (PST) Subject: embedded python on mac - linking problem Message-ID: <537150.77892.qm@web53503.mail.re2.yahoo.com> Hi, I have been able to solve the problem finally: Initially I was trying (wrongly) to link distutils-made module with my application and that has failed. Solution was to (instead of linking the module) compile the source files making up the module and link corresponding objects as any other sources of the application. Thank you for your help, KK From andrew.gillanders at uqconnect.edu.au Thu Jan 7 07:47:40 2010 From: andrew.gillanders at uqconnect.edu.au (Andrew Gillanders) Date: Thu, 7 Jan 2010 22:47:40 +1000 Subject: buffer interface problem In-Reply-To: <50697b2c1001070113i3a8ce2aay50dac69ef8a25a83@mail.gmail.com> References: <3C5B8086-5D3D-444B-B9D2-C9152EBF5DA3@uqconnect.edu.au> <50697b2c1001070113i3a8ce2aay50dac69ef8a25a83@mail.gmail.com> Message-ID: <3AC7891B-5B65-4C5E-99D6-EDF1BB37B1A9@uqconnect.edu.au> Thanks Chris. The atoi function was coming from the locale library (from locale import atoi). I changed it to int and now it works. The next hurdle is this: gzin = GzipFile(fname, 'rb') data = gzin.readline() # min_x,min_y = map(atoi,data.split()[:2]) min_x,min_y = map(int,data.split()[:2]) data = gzin.readline() # span_x,step_x,span_y,step_y = map(atoi,data.split()[:4]) span_x,step_x,span_y,step_y = map(int,data.split()[:4]) data = gzin.read().split('\n') The last line is a problem, giving me this message: Type str doesn't support the buffer API (I am guessing a conflict between split and read?) Sorry, I am new to Python, so how do I get a Traceback? Thanks Andrew On 07/01/2010, at 7:13 PM, Chris Rebert wrote: > On Thu, Jan 7, 2010 at 12:19 AM, Andrew Gillanders > wrote: >> I have run into a problem running a Python script that is part of the >> TerraGear suite for building scenery for FlightGear. I am using >> Mac OS X >> 10.4, running Python (version 3.0.1) in a Unix terminal. >> >> The purpose of the script is to walk a directory tree, unzipping >> files, and >> passing the contents to an executable C program. The problem >> occurs here: >> >> gzin = GzipFile(fname, 'rb') >> data = gzin.readline() >> min_x,min_y = map(atoi,data.split()[:2]) >> >> The input file, when uncompressed, is an ASCII file with a line >> with two >> numbers, then a line of four numbers, then many long lines of >> numbers. I can >> see what the last is trying to do: split the string into two >> words, convert >> them to integers, and assign them to min_x and min_y. >> >> At the third line, I get the message "expected an object with the >> buffer >> interface". Which object is it referring to? > > The elements of the list produced by `data.split()[:2]`, which are > either Unicode strings or bytestrings, neither of which are buffers. > >> Have some functions been >> changed to pass buffer objects instead of strings? How can I fix >> the source >> code to make it run? > > The error is being raised by the atoi() function (in the future, > please post the full Traceback, not just the final error message). > What module/library does your atoi() function come from (look for an > `import` statement mentioning it)? > The only functions by that name in the Python standard library both > operate on strings, not buffers, and thus can't be the same one your > code is using. > > In any case, replacing `atoi` with `int` in your code will likely > solve the problem. The built-in int() function* can convert strings to > integers. > > Cheers, > Chris > -- > http://blog.rebertia.com > > *Not really a function, but close enough for newbie explanatory > purposes. From crebert at ucsd.edu Thu Jan 7 08:03:56 2010 From: crebert at ucsd.edu (Chris Rebert) Date: Thu, 7 Jan 2010 05:03:56 -0800 Subject: buffer interface problem In-Reply-To: <3AC7891B-5B65-4C5E-99D6-EDF1BB37B1A9@uqconnect.edu.au> References: <3C5B8086-5D3D-444B-B9D2-C9152EBF5DA3@uqconnect.edu.au> <50697b2c1001070113i3a8ce2aay50dac69ef8a25a83@mail.gmail.com> <3AC7891B-5B65-4C5E-99D6-EDF1BB37B1A9@uqconnect.edu.au> Message-ID: <50697b2c1001070503q7f8aa4fase7a6f6627ac784e0@mail.gmail.com> On Thu, Jan 7, 2010 at 4:47 AM, Andrew Gillanders wrote: > On 07/01/2010, at 7:13 PM, Chris Rebert wrote: >> On Thu, Jan 7, 2010 at 12:19 AM, Andrew Gillanders >> wrote: >>> >>> I have run into a problem running a Python script that is part of the >>> TerraGear suite for building scenery for FlightGear. I am using Mac OS X >>> 10.4, running Python (version 3.0.1) in a Unix terminal. >>> >>> The purpose of the script is to walk a directory tree, unzipping files, >>> and >>> passing the contents to an executable C program. The problem occurs here: >>> >>> ? gzin = GzipFile(fname, 'rb') >>> ? data = gzin.readline() >>> ? min_x,min_y = map(atoi,data.split()[:2]) >>> >>> The input file, when uncompressed, is an ASCII file with a line with two >>> numbers, then a line of four numbers, then many long lines of numbers. I >>> can >>> see what the last is trying to do: split the string into two words, >>> convert >>> them to integers, and assign them to min_x and min_y. >>> >>> At the third line, I get the message "expected an object with the buffer >>> interface". Which object is it referring to? >> >> The elements of the list produced by `data.split()[:2]`, which are >> either Unicode strings or bytestrings, neither of which are buffers. >> >>> Have some functions been >>> changed to pass buffer objects instead of strings? How can I fix the >>> source >>> code to make it run? >> >> The error is being raised by the atoi() function (in the future, >> please post the full Traceback, not just the final error message). >> What module/library does your atoi() function come from (look for an >> `import` statement mentioning it)? >> The only functions by that name in the Python standard library both >> operate on strings, not buffers, and thus can't be the same one your >> code is using. >> >> In any case, replacing `atoi` with `int` in your code will likely >> solve the problem. The built-in int() function* can convert strings to >> integers. > Thanks Chris. The atoi function was coming from the locale library (from > locale import atoi). I changed it to int and now it works. Hm, that's odd since it was one of the 2 functions in the std lib which the docs say operates on strings... > The next hurdle is this: > gzin = GzipFile(fname, 'rb') > > data = gzin.readline() > # min_x,min_y = map(atoi,data.split()[:2]) > min_x,min_y = map(int,data.split()[:2]) > > data = gzin.readline() > # span_x,step_x,span_y,step_y = map(atoi,data.split()[:4]) > span_x,step_x,span_y,step_y = map(int,data.split()[:4]) > > data = gzin.read().split('\n') > > The last line is a problem, giving me this message: Type str doesn't support > the buffer API (I am guessing a conflict between split and read?) Ah, looking at the 3.0 docs on buffers, I'd surmise gzin.read() returns bytes (http://docs.python.org/3.1/library/functions.html#bytes) rather than a string. You'll want to decode the bytes into characters first, and then you can operate on the resulting string normally. Try: data = gzin.read().decode('ascii').split('\n') > Sorry, I am new to Python, so how do I get a Traceback? You should get one by default. Are you running the script in some environment other than the command line? Here's what a traceback looks like: Traceback (most recent call last): File "foo", line 161, in main() File "foo.py", line 157, in main bot.run() File "foo.py", line 68, in bar self.baz("Enter number: ") File "foo.py", line 112, in baz choice = int(raw_input(prompt))-1 ValueError: invalid literal for int() with base 10: 'y' Cheers, Chris -- http://blog.rebertia.com From mail.to.daniel.platz at googlemail.com Thu Jan 7 08:23:42 2010 From: mail.to.daniel.platz at googlemail.com (Daniel Platz) Date: Thu, 7 Jan 2010 05:23:42 -0800 (PST) Subject: Pass multidimensional array (matrix) to c function using ctypes References: <63ac1a01-8491-4885-bae7-cb884abb582f@34g2000yqp.googlegroups.com> Message-ID: <8c514dac-ed22-4ea1-b117-d5b98f099df9@m16g2000yqc.googlegroups.com> Thanks a lot. This solves my problem and I understand now much better what is going on. Best regards, Daniel From stefan_ml at behnel.de Thu Jan 7 08:36:49 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 07 Jan 2010 14:36:49 +0100 Subject: Where's a DOM builder that uses the Builder Pattern to ... build DOMs? In-Reply-To: <026df5db-bcef-424d-8cd5-56dacebf01f8@c3g2000yqd.googlegroups.com> References: <04523152-a0ff-432c-a48f-2c653577e7b7@m25g2000yqc.googlegroups.com> <4b42f551$0$7625$9b4e6d93@newsspool1.arcor-online.net> <026df5db-bcef-424d-8cd5-56dacebf01f8@c3g2000yqd.googlegroups.com> Message-ID: <4b45e370$0$6736$9b4e6d93@newsspool2.arcor-online.net> Phlip, 05.01.2010 18:00: > On Jan 5, 12:16 am, Stefan Behnel wrote: > >> Note that there are tons of ways to generate HTML with Python. > > Forgot to note - I'm generating schematic XML, and I'm trying to find > a way better than the Django template I started with! Well, then note that there are tons of ways to generate XML with Python, including the one I pointed you to. Stefan From rschroev_nospam_ml at fastmail.fm Thu Jan 7 08:37:07 2010 From: rschroev_nospam_ml at fastmail.fm (Roel Schroeven) Date: Thu, 07 Jan 2010 14:37:07 +0100 Subject: please help shrink this each_with_index() implementation In-Reply-To: References: <3bba1855-efd4-43e0-b132-64a6674ec10b@b2g2000yqi.googlegroups.com> <2b751caa-d4c7-4f2b-80cc-22580772b0e7@u41g2000yqe.googlegroups.com> <0353b78e$0$1336$c3e8da3@news.astraweb.com> <1715bea7-1ff1-4294-9dad-672873e14530@v25g2000yqk.googlegroups.com> Message-ID: <7sl1n.7163$BK3.5872@newsfe16.ams2> Phlip schreef: > Nobody wrote: >> Writing robust software from the outset puts you at a competitive >> disadvantage to those who understand how the system works. > > And I, not my language, should pick and chose how to be rigorous. The language > should not make the decision for me. You can always easily make your program less rigorous than the language, but the reverse is generally very difficult. So a rigorous language gives you the choice, why a non-rigorous language does not. -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven From eight32 at gmail.com Thu Jan 7 09:17:26 2010 From: eight32 at gmail.com (Stuart Murray-Smith) Date: Thu, 7 Jan 2010 16:17:26 +0200 Subject: Python books, literature etc In-Reply-To: <304854.43355.qm@smtp139.mail.mud.yahoo.com> References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> <304854.43355.qm@smtp139.mail.mud.yahoo.com> Message-ID: > Have a look at the Getting Started section of the wiki: > > http://wiki.python.org/moin/ > > specially the PythonBooks section Perfect! Exactly what I'm looking for :) Thanks Gabriel! From lie.1296 at gmail.com Thu Jan 7 09:37:08 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Fri, 08 Jan 2010 01:37:08 +1100 Subject: Exception as the primary error handling mechanism? In-Reply-To: References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4b44b138$0$11237$426a74cc@news.free.fr> Message-ID: <4b45f195$1@dnews.tpgi.com.au> On 1/7/2010 10:43 PM, Roel Schroeven wrote: > - I tend to think that not following that practice trains me to be > careful in all cases, whereas I'm afraid that following the practice > will make me careless, which is dangerous in all the cases where the > practice won't protect me. > That's a sign of a gotcha... a well-designed language makes you think about your problem at hand and less about the language's syntax. From mackrackit at gmail.com Thu Jan 7 09:56:35 2010 From: mackrackit at gmail.com (Dave McCormick) Date: Thu, 07 Jan 2010 07:56:35 -0700 Subject: Exception as the primary error handling mechanism? In-Reply-To: <4b45f195$1@dnews.tpgi.com.au> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4b44b138$0$11237$426a74cc@news.free.fr> <4b45f195$1@dnews.tpgi.com.au> Message-ID: <4B45F623.7060303@gmail.com> Lie Ryan wrote: > That's a sign of a gotcha... a well-designed language makes you think > about your problem at hand and less about the language's syntax. Not until you learn the language that is. From a Python newbee.... ;-) From cmpython at gmail.com Thu Jan 7 10:26:18 2010 From: cmpython at gmail.com (CM) Date: Thu, 7 Jan 2010 07:26:18 -0800 (PST) Subject: GUI for multiplatform multimedia project References: Message-ID: On Jan 6, 4:53?pm, wrote: > Hi everyone, > > I posted that question on a python-forum, but got answer, so I ask here. > > I'm working on an artistic project and I'm looking for the best > cross-platform GUI solution. The problem is that it's gonna be a tool that > will have to be double-click installable/runnable and pre-installation of > any libraries for end-users is very much like an evil. It really has to be > double-click tool > > My first thought was PyQt, because it's a real framework with a lot of > stuff inside (including Phonon) and I know some cross-platform media > software written in C++ QT (like VLC). But on the other hand I've heard > that it's not that easy to make it "double-clicky" multi-platform. Is that > true? > > Another thing that matters for me is ease of integration with libraries > like OpenCV. > > I will be VERY thankful for any help. I'm SO tired googling the problem > (it's like weeks now!!) > > Best from Poland, > trzewiczek I don't know this for sure, but I would be surprised if any of the widget toolkits gave you much more trouble than any other when making your app into a bundled executable. I have made wxPython apps into a Windows .exe file easily using GUI2Exe, which is an excellent GUI interface (written in wxPython by Andrea Gavana) to a number of the executable bundlers: py2exe, PyInstaller, py2app, cx_Freeze, bbFreeze. Some of these are for Windows, some for Mac, some for Linux. wxPython apparently works with OpenCV: http://opencv.willowgarage.com/wiki/wxpython While you're Googling you might want to be aware of any legal concerns with py2exe and distributing dll files (if there are any). From vmail at mycircuit.org Thu Jan 7 11:10:05 2010 From: vmail at mycircuit.org (Peter) Date: Thu, 07 Jan 2010 17:10:05 +0100 Subject: Recommended "new" way for config files Message-ID: <4B46075D.8080606@mycircuit.org> Hi There seems to be several strategies to enhance the old ini-style config files with real python code, for example: 1) the documentation tool sphinx uses a python file conf.py that is exefile(d) , but execfile is suppressed in Python 3 2) there is a module cfgparse on sourceforge that supports a hybrid style 3) modern tools like ipython seems to favor a new style based on python code config files but also support a hybrid style mixing .ini files and python code files. 4) I could use __import__ to import modules based on some command line options Is there a strategy that should be prefered for new projects ? thanks peter From grahn+nntp at snipabacken.se Thu Jan 7 11:18:32 2010 From: grahn+nntp at snipabacken.se (Jorgen Grahn) Date: 7 Jan 2010 16:18:32 GMT Subject: How to execute a script from another script and other script does not do busy wait. References: <76bff0b6-70cb-4960-9b4d-3af7a9a1e0e7@z41g2000yqz.googlegroups.com> Message-ID: On Thu, 2010-01-07, Rajat wrote: > I want to run a python script( aka script2) from another python script > (aka script1). While script1 executes script2 it waits for script2 to > complete and in doing so it also does some other useful work.(does not > do a busy wait). > > My intention is to update a third party through script1 that script2 > is going to take longer. I do not understand that sentence. What are you trying to do, more exactly? The best solution can be threads, os.popen, os.system or something different -- depending on the details of what you want to do. > Please suggest how should I go about implementing it. > > I'm currently executing it as: > > import main from script2 > ret_code = main() > return ret_code > > which surely is not going to achieve me what I intend. > > > Thanks, > Rajat. /Jorgen -- // Jorgen Grahn O o . From lie.1296 at gmail.com Thu Jan 7 11:30:45 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Fri, 08 Jan 2010 03:30:45 +1100 Subject: Recommended "new" way for config files In-Reply-To: References: Message-ID: <4b460c36$1@dnews.tpgi.com.au> On 1/8/2010 3:10 AM, Peter wrote: > Is there a strategy that should be prefered for new projects ? The answer is, it depends. From jeanmichel at sequans.com Thu Jan 7 11:32:32 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Thu, 07 Jan 2010 17:32:32 +0100 Subject: Recommended "new" way for config files In-Reply-To: <4B46075D.8080606@mycircuit.org> References: <4B46075D.8080606@mycircuit.org> Message-ID: <4B460CA0.20800@sequans.com> Peter wrote: > Hi > There seems to be several strategies to enhance the old ini-style > config files with real python code, for example: > > 1) the documentation tool sphinx uses a python file conf.py that is > exefile(d) , but execfile is suppressed in Python 3 > 2) there is a module cfgparse on sourceforge that supports a hybrid style > 3) modern tools like ipython seems to favor a new style based on > python code config files but also support a hybrid style mixing .ini > files and python code files. > 4) I could use __import__ to import modules based on some command line > options > > > Is there a strategy that should be prefered for new projects ? > > thanks > peter I would add the standard module ConfigParser http://docs.python.org/library/configparser.html to your list. I don't know exactly what you intend to do with point 4/, but I would exclude it if any other point may fit. Imports can become tricky when used out of the common way. Anyway, hacking the import statement for managing configuration files does not sound very appropriate. The .ini file is the simpliest solution, at least from the user point of view, no need to learn any python syntax. However, speeking for myself, I am using python coded configuration files, but: we all worship python in the team and thus are familiar with it. JM From grahn+nntp at snipabacken.se Thu Jan 7 11:32:58 2010 From: grahn+nntp at snipabacken.se (Jorgen Grahn) Date: 7 Jan 2010 16:32:58 GMT Subject: Do I have to use threads? References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> <432961e5-627e-452d-93db-9562951551d2@e27g2000yqd.googlegroups.com> Message-ID: On Thu, 2010-01-07, Marco Salden wrote: > On Jan 6, 5:36?am, Philip Semanchuk wrote: >> On Jan 5, 2010, at 11:26 PM, aditya shukla wrote: >> >> > Hello people, >> >> > I have 5 directories corresponding 5 ?different urls .I want to ? >> > download >> > images from those urls and place them in the respective ? >> > directories.I have >> > to extract the contents and download them simultaneously.I can ? >> > extract the >> > contents and do then one by one. My questions is for doing it ? >> > simultaneously >> > do I have to use threads? >> >> No. You could spawn 5 copies of wget (or curl or a Python program that ? >> you've written). Whether or not that will perform better or be easier ? >> to code, debug and maintain depends on the other aspects of your ? >> program(s). >> >> bye >> Philip > > Yep, the more easier and straightforward the approach, the better: > threads are always (programmers')-error-prone by nature. > But my question would be: does it REALLY need to be simultaneously: > the CPU/OS only has more overhead doing this in parallel with > processess. Measuring sequential processing and then trying to > optimize (e.g. for user response or whatever) would be my prefered way > to go. Less=More. Normally when you do HTTP in parallell over several TCP sockets, it has nothing to do with CPU overhead. You just don't want every GET to be delayed just because the server(s) are lazy responding to the first few ones; or you might want to read the text of a web page and the CSS before a few huge pictures have been downloaded. His "I have to [do them] simultaneously" makes me want to ask "Why?". If he's expecting *many* pictures, I doubt that the parallel download will buy him much. Reusing the same TCP socket for all of them is more likely to help, especially if the pictures aren't tiny. One long-lived TCP connection is much more efficient than dozens of short-lived ones. Personally, I'd popen() wget and let it do the job for me. /Jorgen -- // Jorgen Grahn O o . From robert.kern at gmail.com Thu Jan 7 11:38:06 2010 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 07 Jan 2010 10:38:06 -0600 Subject: Recommended "new" way for config files In-Reply-To: <4B46075D.8080606@mycircuit.org> References: <4B46075D.8080606@mycircuit.org> Message-ID: On 2010-01-07 10:10 AM, Peter wrote: > Hi > There seems to be several strategies to enhance the old ini-style config > files with real python code, for example: > > 1) the documentation tool sphinx uses a python file conf.py that is > exefile(d) , but execfile is suppressed in Python 3 Only because it is redundant, not because it is a discouraged approach. You can still read the file and exec() the resulting string. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From phlip2005 at gmail.com Thu Jan 7 11:44:25 2010 From: phlip2005 at gmail.com (Phlip) Date: Thu, 7 Jan 2010 08:44:25 -0800 (PST) Subject: Where's a DOM builder that uses the Builder Pattern to ... build DOMs? References: <04523152-a0ff-432c-a48f-2c653577e7b7@m25g2000yqc.googlegroups.com> <4b42f551$0$7625$9b4e6d93@newsspool1.arcor-online.net> <026df5db-bcef-424d-8cd5-56dacebf01f8@c3g2000yqd.googlegroups.com> <4b45e370$0$6736$9b4e6d93@newsspool2.arcor-online.net> Message-ID: On Jan 7, 5:36?am, Stefan Behnel wrote: > Well, then note that there are tons of ways to generate XML with Python, > including the one I pointed you to. from lxml.html import builder as E xml = E.foo() All I want is "", but I get "AttributeError: 'module' object has no attribute 'foo'". A peek at dir(E) shows it only has HTML tags, all hard coded. So how to get it to generate any random XML tag my clients think of? I will write this myself with __getattr__ etc, if I can't find it, because the permissive & expressive builder pattern I'm after would be very ... permissive & expressive. All I want is a library that reads my mind!!! Is that too much to ask??? (Unless if the library insists on throwing a NullMind exception, on principle...) -- Phlip http://twitter.com/Pen_Bird From grahn+nntp at snipabacken.se Thu Jan 7 12:22:04 2010 From: grahn+nntp at snipabacken.se (Jorgen Grahn) Date: 7 Jan 2010 17:22:04 GMT Subject: Python books, literature etc References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: On Thu, 2010-01-07, Stuart Murray-Smith wrote: ... > [...] ESR's guide to > smart questions [1] helps set the pace of list culture. It's good, if you can ignore the "These People Are Very Important Hacker Gods, Not Mere Mortals" subtext. ... > Anyways, to rephrase, could someone kindly mention any of their > preferred Python books, websites, tutorials etc to help me get to an > intermediate/advanced level? Something that would help me add > functionality to Ubiquity, say. I may be alone in this, but Alex Martelli's book ("Python in a nutshell"?) on Python 2.2 and a bit of 2.3, plus the official documentation, plus this group, is all I think I need. But I had a lot of Unix, C, C++ and Perl experience to help me. /Jorgen -- // Jorgen Grahn O o . From python at mrabarnett.plus.com Thu Jan 7 12:38:46 2010 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 07 Jan 2010 17:38:46 +0000 Subject: Do I have to use threads? In-Reply-To: References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> <432961e5-627e-452d-93db-9562951551d2@e27g2000yqd.googlegroups.com> Message-ID: <4B461C26.6030209@mrabarnett.plus.com> Jorgen Grahn wrote: > On Thu, 2010-01-07, Marco Salden wrote: >> On Jan 6, 5:36 am, Philip Semanchuk wrote: >>> On Jan 5, 2010, at 11:26 PM, aditya shukla wrote: >>> >>>> Hello people, >>>> I have 5 directories corresponding 5 different urls .I want to >>>> download >>>> images from those urls and place them in the respective >>>> directories.I have >>>> to extract the contents and download them simultaneously.I can >>>> extract the >>>> contents and do then one by one. My questions is for doing it >>>> simultaneously >>>> do I have to use threads? >>> No. You could spawn 5 copies of wget (or curl or a Python program that >>> you've written). Whether or not that will perform better or be easier >>> to code, debug and maintain depends on the other aspects of your >>> program(s). >>> >>> bye >>> Philip >> Yep, the more easier and straightforward the approach, the better: >> threads are always (programmers')-error-prone by nature. >> But my question would be: does it REALLY need to be simultaneously: >> the CPU/OS only has more overhead doing this in parallel with >> processess. Measuring sequential processing and then trying to >> optimize (e.g. for user response or whatever) would be my prefered way >> to go. Less=More. > > Normally when you do HTTP in parallell over several TCP sockets, it > has nothing to do with CPU overhead. You just don't want every GET to > be delayed just because the server(s) are lazy responding to the first > few ones; or you might want to read the text of a web page and the CSS > before a few huge pictures have been downloaded. > > His "I have to [do them] simultaneously" makes me want to ask "Why?". > > If he's expecting *many* pictures, I doubt that the parallel download > will buy him much. Reusing the same TCP socket for all of them is > more likely to help, especially if the pictures aren't tiny. One > long-lived TCP connection is much more efficient than dozens of > short-lived ones. > > Personally, I'd popen() wget and let it do the job for me. > From my own experience: I wanted to download a number of webpages. I noticed that there was a significant delay before it would reply, and an especially long delay for one of them, so I used a number of threads, each one reading a URL from a queue, performing the download, and then reading the next URL, until there were none left (actually, until it read the sentinel None, which it put back for the other threads). The result? Shorter total download time because it could be downloading one webpage while waiting for another to reply. (Of course, I had to make sure that I didn't have too many threads, because that might've put too many demands on the website, not a nice thing to do!) From philip at semanchuk.com Thu Jan 7 12:53:39 2010 From: philip at semanchuk.com (Philip Semanchuk) Date: Thu, 7 Jan 2010 12:53:39 -0500 Subject: Do I have to use threads? In-Reply-To: References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> <432961e5-627e-452d-93db-9562951551d2@e27g2000yqd.googlegroups.com> Message-ID: <0CFE9D52-F77E-4158-A461-AD1F56DD2B04@semanchuk.com> On Jan 7, 2010, at 11:32 AM, Jorgen Grahn wrote: > On Thu, 2010-01-07, Marco Salden wrote: >> On Jan 6, 5:36 am, Philip Semanchuk wrote: >>> On Jan 5, 2010, at 11:26 PM, aditya shukla wrote: >>> >>>> Hello people, >>> >>>> I have 5 directories corresponding 5 different urls .I want to >>>> download >>>> images from those urls and place them in the respective >>>> directories.I have >>>> to extract the contents and download them simultaneously.I can >>>> extract the >>>> contents and do then one by one. My questions is for doing it >>>> simultaneously >>>> do I have to use threads? >>> >>> No. You could spawn 5 copies of wget (or curl or a Python program >>> that >>> you've written). Whether or not that will perform better or be >>> easier >>> to code, debug and maintain depends on the other aspects of your >>> program(s). >>> >>> bye >>> Philip >> >> Yep, the more easier and straightforward the approach, the better: >> threads are always (programmers')-error-prone by nature. >> But my question would be: does it REALLY need to be simultaneously: >> the CPU/OS only has more overhead doing this in parallel with >> processess. Measuring sequential processing and then trying to >> optimize (e.g. for user response or whatever) would be my prefered >> way >> to go. Less=More. > > Normally when you do HTTP in parallell over several TCP sockets, it > has nothing to do with CPU overhead. You just don't want every GET to > be delayed just because the server(s) are lazy responding to the first > few ones; or you might want to read the text of a web page and the CSS > before a few huge pictures have been downloaded. > > His "I have to [do them] simultaneously" makes me want to ask "Why?". Exactly what I was thinking. He's surely doing something more complicated than his post suggests, and without that detail it's impossible to say whether threads, processes, asynch or voodoo is the best approach. bye P From sccolbert at gmail.com Thu Jan 7 13:13:39 2010 From: sccolbert at gmail.com (S. Chris Colbert) Date: Thu, 7 Jan 2010 19:13:39 +0100 Subject: ANN: Pymazon 0.1.0 released! Message-ID: <201001071913.39467.sccolbert@gmail.com> Hello, I'm happy to announce the first non-beta release of Pymazon: a python implemented downloader for the Amazon mp3 store. Improvements from the beta: - Running download status indicator - Various fixes for Windows - Some code cleanup Pymazon was created to be a simple and easy alternative for the Linux version of the Amazon downloader, and alleviate the pain of getting it to work with 64bit Linux. You can read about Pymazon at http://pymazon.googlecode.com You can download from googlecode or the cheeseshop: $ pip install pymazon or $ easy_install pymazon It also works on Windows. Dependencies: PyCrypto (it's in the ubuntu repos and the cheeseshop) PyQt4 >= 4.5 (optional, only needed for GUI) GPLv3 License Cheers! SCC From vmail at mycircuit.org Thu Jan 7 13:19:40 2010 From: vmail at mycircuit.org (Peter) Date: Thu, 07 Jan 2010 19:19:40 +0100 Subject: Recommended "new" way for config files In-Reply-To: <4B460CA0.20800@sequans.com> References: <4B46075D.8080606@mycircuit.org> <4B460CA0.20800@sequans.com> Message-ID: <4B4625BC.6090201@mycircuit.org> Thanks for your answer, let me be more precise: > I would add the standard module ConfigParser > http://docs.python.org/library/configparser.html to your list. of course, that was the implicit starting point of my request, when talking about .ini files. > I don't know exactly what you intend to do with point 4/, It would allow me to select different conf.py files with command line switches, like for example a -c option. > but I would exclude it if any other point may fit. Imports can become > tricky when used out of the common way. Anyway, hacking the import > statement for managing configuration files does not sound very > appropriate. > Would this be considered a hack ? #!/usr/bin/env python import sys # parse command line options here if option='standard': const = __import__('consts') else: const = __import__('alternative_consts') > The .ini file is the simpliest solution, at least from the user point > of view, no need to learn any python syntax. I am speaking from the point of view of a python programmer, and I find the .ini restrictions not necessarily simple, for example when dealing with structured data (I suppose it is trivial to specify a dictionnary or a list for the purpose of my request) For example, configuration files for the logging module get unwieldy when you specify several loggers , handlers, formatters etc, because you have to break down structured data ( objects ) to name,value pairs. > However, speeking for myself, I am using python coded configuration > files, but: we all worship python in the team and thus are familiar > with it. > so do I. > JM > > So what is the "worshipped" approach, when you need more than name=value pairs ? Peter From Lee at JamToday.com Thu Jan 7 13:22:57 2010 From: Lee at JamToday.com (Lee) Date: Thu, 07 Jan 2010 13:22:57 -0500 Subject: Dictionary used to build a Triple Store Message-ID: Definitely a newbie question, so please bear with me. I'm reading "Programming the Semantic Web" by Segaran, Evans, and Tayor. It's about the Semantic Web BUT it uses python to build a "toy" triple store claimed to have good performance in the "tens of thousands" of triples. Just in case anybody doesnt know what an RDF triple is (not that it matters for my question) think of it as an ordered 3 tuple representing a Subject, a Predicate, and an Object eg: (John, loves, Mary) (Mary, has-a, lamb) {theSky, has-color,blue} To build the triple store entirely in Python, the authors recommend using the Python hash. Three hashes actually (I get that. You want to have a hash with the major index being the Subject in one hash, the Predicate in another hash, or the Object for the third hash) He creates a class SimpleGraph which initializes itself by setting the three hashes names _spo, _pos, and _osp thus class SimpleGraph; def __init__(self); self._spo={}; self._pos=(); self._osp={}; So far so good. I get the convention with the double underbars for the initializer but Q1: Not the main question but while I'm here....I'm a little fuzzy on the convention about the use of the single underbar in the definition of the hashes. Id the idea to "underbar" all objects and methods that belong to the class? Why do that? But now the good stuff: Our authors define the hashes thus: (showing only one of the three hashes because they're all the same idea) self._pos = {predicate:{object:set( [subject] ) }} Q2: Wha? Two surprises ... 1) Why not {predicate:{object:subject}} i.e. pos[predicate][object]=subject....why the set( [object] ) construct? putting the object into a list and turning the list into a set to be the "value" part of a name:value pair. Why not just use the naked subject for the value? 2) Why not something like pos[predicate][object][subject] = 1 .....or any constant. The idea being to create the set of three indexes. If the triple exists in the hash, its "in" your tripple store. If not, then there's no such triple. From clp2 at rebertia.com Thu Jan 7 13:23:58 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Thu, 7 Jan 2010 10:23:58 -0800 Subject: Recommended "new" way for config files In-Reply-To: <4B4625BC.6090201@mycircuit.org> References: <4B46075D.8080606@mycircuit.org> <4B460CA0.20800@sequans.com> <4B4625BC.6090201@mycircuit.org> Message-ID: <50697b2c1001071023m28e9a242uef20c6e65414c371@mail.gmail.com> On Thu, Jan 7, 2010 at 10:19 AM, Peter wrote: >> The .ini file is the simpliest solution, at least from the user point of >> view, no need to learn any python syntax. > > I am speaking from the point of view of a python programmer, and I find the > .ini restrictions not necessarily simple, for example when dealing with > structured data (I suppose it is trivial to specify a dictionnary or a list > for the purpose of my request) For example, configuration files for the > logging module get unwieldy when you specify several loggers , handlers, > formatters etc, because you have to break down structured data ( objects ) > to name,value pairs. > So what is the "worshipped" approach, when you need more than name=value > pairs ? JSON is one option: http://docs.python.org/library/json.html Cheers, Chris -- http://blog.rebertia.com From tjreedy at udel.edu Thu Jan 7 13:31:48 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 07 Jan 2010 13:31:48 -0500 Subject: How to reduce the memory size of python In-Reply-To: <6AFAAA2E71101642861E945A057CE7CD023A884D@ZMY16EXM70.ds.mot.com> References: <6AFAAA2E71101642861E945A057CE7CD023A882B@ZMY16EXM70.ds.mot.com> <6AFAAA2E71101642861E945A057CE7CD023A884D@ZMY16EXM70.ds.mot.com> Message-ID: On 1/7/2010 3:34 AM, Mishra Gopal-QBX634 wrote: > > Like import logging takes 1MB of memory. > We only use on function getLogger by 'from logging import getLogger' > > But it still take the same 1 MB memory. > > Instead of loading whole logging module only load the getLogger > function. from x import y causes creation of module x and binding of the module to sys.modules'x]. It then binds name 'y' in the current namespace to the corresponding object in x. Functions in general need a reference to the module namespace to resolve module-level variables. To save anything, you must cut the function out of the module and verify that it works in isolation. But I presume 'getLogger' refers to other stuff in the logging module and would not work in isolation. Terry Jan Reedy From vmail at mycircuit.org Thu Jan 7 13:34:18 2010 From: vmail at mycircuit.org (Peter) Date: Thu, 07 Jan 2010 19:34:18 +0100 Subject: Python books, literature etc In-Reply-To: References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: <4B46292A.4020308@mycircuit.org> >> Anyways, to rephrase, could someone kindly mention any of their >> preferred Python books, websites, tutorials etc to help me get to an >> intermediate/advanced level? Something that would help me add >> functionality to Ubiquity, say. >> > I may be alone in this, but Alex Martelli's book ("Python in a > nutshell"?) on Python 2.2 and a bit of 2.3, plus the official > documentation, plus this group, is all I think I need. > But I had a lot of Unix, C, C++ and Perl experience to help me. > > /Jorgen > > I find Alex Martellis "Python Cookbook" excellent/invaluable and ( and also his Nutshell book mentioned above ) and depending on your application domain, I liked: 1) Hans Petter Langtangen: Python Scripting for Computational Science A truly excellent book, not only with respect to Python Scripting , but also on how to avoid paying license fees by using opensource tools as an engineer ( plotting, graphing, gui dev etc ). Very good , pratical introduction to Python with careful and non-trivial examples and exercises. 2) There is a book at Apress on using Python and matplotlib ( amongst other ) "Beginning Python Visualization" which is not as comprehensive as reference 1) but useful , especially for beginners who wants to visualize data from an engineers background 3) "Programming for the semantic web" Oreilly is a very pratical and interesting guide to things like OWL, triplestore, logic, reasoning, data mining and it is amongst the very few books on these topics I have seen that has working code examples 4) "Natural language priocessing with Python " Oreilly is also a pratical book with lots of working code if you are interested in data mining, text searching and natural language tasks. It is based on a rather large opensource library for natural language processing ( sorry forgot the exact name,but easy to find on the net) All these book make you feel warm and confortable if you have ever tried to do these things in Perl, C++ or Java Peter From sccolbert at gmail.com Thu Jan 7 13:39:42 2010 From: sccolbert at gmail.com (S. Chris Colbert) Date: Thu, 7 Jan 2010 19:39:42 +0100 Subject: Dictionary used to build a Triple Store In-Reply-To: References: Message-ID: <201001071939.42847.sccolbert@gmail.com> > Definitely a newbie question, so please bear with me. > > I'm reading "Programming the Semantic Web" by Segaran, Evans, and Tayor. > > It's about the Semantic Web BUT it uses python to build a "toy" triple > store claimed to have good performance in the "tens of thousands" of > triples. > > Just in case anybody doesnt know what an RDF triple is (not that it > matters for my question) think of it as an ordered 3 tuple representing > a Subject, a Predicate, and an Object eg: (John, loves, Mary) (Mary, > has-a, lamb) {theSky, has-color,blue} > > To build the triple store entirely in Python, the authors recommend > using the Python hash. Three hashes actually (I get that. You > want to have a hash with the major index being the Subject in one hash, > the Predicate in another hash, or the Object for the third hash) > > He creates a class SimpleGraph which initializes itself by setting the > three hashes names _spo, _pos, and _osp thus > > class SimpleGraph; > def __init__(self); > self._spo={}; > self._pos=(); > self._osp={}; > > So far so good. I get the convention with the double underbars for the > initializer but > > Q1: Not the main question but while I'm here....I'm a little fuzzy on > the convention about the use of the single underbar in the definition of > the hashes. Id the idea to "underbar" all objects and methods that > belong to the class? Why do that? > > But now the good stuff: > > Our authors define the hashes thus: (showing only one of the three > hashes because they're all the same idea) > > self._pos = {predicate:{object:set( [subject] ) }} > > Q2: Wha? Two surprises ... > 1) Why not {predicate:{object:subject}} i.e. > pos[predicate][object]=subject....why the set( [object] ) construct? > putting the object into a list and turning the list into a set to be the > "value" part of a name:value pair. Why not just use the naked subject > for the value? > because the argument has to be iterable. In [1]: set(1) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) /home/brucewayne/ in () TypeError: 'int' object is not iterable > 2) Why not something like pos[predicate][object][subject] = 1 > .....or any constant. The idea being to create the set of three indexes. > If the triple exists in the hash, its "in" your tripple store. If not, > then there's no such triple. > I can't really answer that, I imagine there is a better way to code what is trying to be accomplished. But I'm no Steven D'Aprano and I'm already a few beers in ;) From steve at holdenweb.com Thu Jan 7 13:54:05 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 07 Jan 2010 13:54:05 -0500 Subject: Dictionary used to build a Triple Store In-Reply-To: References: Message-ID: Lee wrote: > Definitely a newbie question, so please bear with me. > > I'm reading "Programming the Semantic Web" by Segaran, Evans, and Tayor. > > It's about the Semantic Web BUT it uses python to build a "toy" triple > store claimed to have good performance in the "tens of thousands" of > triples. > > Just in case anybody doesnt know what an RDF triple is (not that it > matters for my question) think of it as an ordered 3 tuple representing > a Subject, a Predicate, and an Object eg: (John, loves, Mary) (Mary, > has-a, lamb) {theSky, has-color,blue} > > To build the triple store entirely in Python, the authors recommend > using the Python hash. Three hashes actually (I get that. You > want to have a hash with the major index being the Subject in one hash, > the Predicate in another hash, or the Object for the third hash) > > He creates a class SimpleGraph which initializes itself by setting the > three hashes names _spo, _pos, and _osp thus > > class SimpleGraph; > def __init__(self); > self._spo={}; > self._pos=(); > self._osp={}; > > So far so good. I get the convention with the double underbars for the > initializer but > > Q1: Not the main question but while I'm here....I'm a little fuzzy on > the convention about the use of the single underbar in the definition of > the hashes. Id the idea to "underbar" all objects and methods that > belong to the class? Why do that? > > But now the good stuff: > > Our authors define the hashes thus: (showing only one of the three > hashes because they're all the same idea) > > self._pos = {predicate:{object:set( [subject] ) }} > > Q2: Wha? Two surprises ... > 1) Why not {predicate:{object:subject}} i.e. > pos[predicate][object]=subject....why the set( [object] ) construct? > putting the object into a list and turning the list into a set to be the > "value" part of a name:value pair. Why not just use the naked subject > for the value? > Because for a given predicate there can be many objects, and you need to be able to look up the subjects associated with the same object and predicate. (I am assuming this is initialization code: to add another subject with the same object to the predicate you would use self._pos[predicate][object].add(subject) > 2) Why not something like pos[predicate][object][subject] = 1 .....or > any constant. The idea being to create the set of three indexes. If the > triple exists in the hash, its "in" your tripple store. If not, then > there's no such triple. > Because it's less efficient. Since there will only ever be one unique occurrence of each (predicate, object, subject) triple using a dict would be unnecessarily wasteful. Containment checks for sets are just as fast as for dicts, asn you don't need to store all those references to 1. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From rschroev_nospam_ml at fastmail.fm Thu Jan 7 14:30:37 2010 From: rschroev_nospam_ml at fastmail.fm (Roel Schroeven) Date: Thu, 07 Jan 2010 20:30:37 +0100 Subject: Exception as the primary error handling mechanism? In-Reply-To: <4b45f195$1@dnews.tpgi.com.au> References: <765ec66b-5f11-4f77-ac21-26931ffba4ee@r24g2000yqd.googlegroups.com> <4b44b138$0$11237$426a74cc@news.free.fr> <4b45f195$1@dnews.tpgi.com.au> Message-ID: Lie Ryan schreef: > On 1/7/2010 10:43 PM, Roel Schroeven wrote: >> - I tend to think that not following that practice trains me to be >> careful in all cases, whereas I'm afraid that following the practice >> will make me careless, which is dangerous in all the cases where the >> practice won't protect me. >> > > That's a sign of a gotcha... a well-designed language makes you think > about your problem at hand and less about the language's syntax. It's not a big deal to me, but you're right, it's a gotcha. I don't think there's a single language without gotcha's; one of the things I like about Python is that is has many less than the other languages I know. -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven From vardan.pogosyan at gmail.com Thu Jan 7 14:32:07 2010 From: vardan.pogosyan at gmail.com (h0uk) Date: Thu, 7 Jan 2010 11:32:07 -0800 (PST) Subject: PyQt QThreadPool error Message-ID: <7dc7a9f3-cff9-4911-bfde-6086db471b41@k17g2000yqh.googlegroups.com> Hello. I have the following code: #workers = {} QtCore.QThreadPool.globalInstance().setExpiryTimeout (300000) QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) for i in range(1, int(userscnt) + 1): work = wk.Worker(i) # connect signals work.mmShowCaptcha.connect(self.show_captcha_dlg) work.log.connect(self.handle_log) self.captcha_answer.connect(work.mmCaptchaAnswer) work.setAutoDelete(True) QtCore.QThreadPool.globalInstance().start(work) On last line of code ( QtCore.QThreadPool.globalInstance().start (work) ) i get an error: SystemError: error return without exception set What is wrong in my code??? Any advise??? Thanks From danmcleran at yahoo.com Thu Jan 7 14:48:45 2010 From: danmcleran at yahoo.com (danmcleran at yahoo.com) Date: Thu, 7 Jan 2010 11:48:45 -0800 (PST) Subject: How to execute a script from another script and other script does not do busy wait. References: <76bff0b6-70cb-4960-9b4d-3af7a9a1e0e7@z41g2000yqz.googlegroups.com> Message-ID: <342e58d5-ecd5-4cc5-ad4c-cd1ea169f9e0@c34g2000yqn.googlegroups.com> On Jan 7, 9:18?am, Jorgen Grahn wrote: > On Thu, 2010-01-07, Rajat wrote: > > I want to run a python script( aka script2) from another python script > > (aka script1). While script1 executes script2 it waits for script2 to > > complete and in doing so it also does some other useful work.(does not > > do a busy wait). > > > My intention is to update a third party through script1 that script2 > > is going to take longer. > > I do not understand that sentence. > What are you trying to do, more exactly? ?The best solution can be > threads, os.popen, os.system or something different -- depending on > the details of what you want to do. > > > Please suggest how should I go about implementing it. > > > I'm currently executing it as: > > > import main from script2 > > ret_code ?= main() > > return ret_code > > > which surely is not going to achieve me what I intend. > > > Thanks, > > Rajat. > > /Jorgen > > -- > ? // Jorgen Grahn \X/ ? ? snipabacken.se> ? O ?o ? . I personally use subprocess. Once you launch via subprocess you can wait or not. p = subprocess.Popen(...) p.wait() #or not. See subprocess docs. From deets at nospam.web.de Thu Jan 7 14:56:22 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Thu, 07 Jan 2010 20:56:22 +0100 Subject: GUI for multiplatform multimedia project In-Reply-To: References: Message-ID: <7qmsj8Fc9tU1@mid.uni-berlin.de> trzewiczek at trzewiczek.info schrieb: > Hi everyone, > > I posted that question on a python-forum, but got answer, so I ask here. > > I'm working on an artistic project and I'm looking for the best > cross-platform GUI solution. The problem is that it's gonna be a tool that > will have to be double-click installable/runnable and pre-installation of > any libraries for end-users is very much like an evil. It really has to be > double-click tool > > My first thought was PyQt, because it's a real framework with a lot of > stuff inside (including Phonon) and I know some cross-platform media > software written in C++ QT (like VLC). But on the other hand I've heard > that it's not that easy to make it "double-clicky" multi-platform. Is that > true? I don't know exactly what you mean with that, but I doubt it's easier with anything else. Another option might be pygame + simple OpenGL though, if you are planning on heavy use of Canvas-like things, that might be good enough & less heavyweight. > > Another thing that matters for me is ease of integration with libraries > like OpenCV. That has little todo with the toolkit. You need some image-converting-code, I've written such for Cocoa, to convert OpenCV's RGBA to OSX ARGB. But the last resort would be to save the images from OpenCV and read them using Qt (or whatever toolkit you use in th eend. Diez From deets at nospam.web.de Thu Jan 7 15:02:25 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Thu, 07 Jan 2010 21:02:25 +0100 Subject: PyQt QThreadPool error In-Reply-To: <7dc7a9f3-cff9-4911-bfde-6086db471b41@k17g2000yqh.googlegroups.com> References: <7dc7a9f3-cff9-4911-bfde-6086db471b41@k17g2000yqh.googlegroups.com> Message-ID: <7qmsujFc9tU2@mid.uni-berlin.de> h0uk schrieb: > Hello. > > I have the following code: > > #workers = {} > QtCore.QThreadPool.globalInstance().setExpiryTimeout > (300000) > QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) > for i in range(1, int(userscnt) + 1): > work = wk.Worker(i) > # connect signals > work.mmShowCaptcha.connect(self.show_captcha_dlg) > work.log.connect(self.handle_log) > self.captcha_answer.connect(work.mmCaptchaAnswer) > work.setAutoDelete(True) > QtCore.QThreadPool.globalInstance().start(work) > > > > On last line of code ( QtCore.QThreadPool.globalInstance().start > (work) ) i get an error: > > SystemError: error return without exception set > > What is wrong in my code??? Any advise??? The error is on C-level. AFAIK it occurs when a Python-C-function returns "NULL" without setting an exception. It's hard to say where it really occurs. I'd use a debug-build of PyQt, and possibly Python, and then investigate using gdb. Alternatively, what happens when you do some "dummy"-work that doesn't use signals and no other libraries? Diez From vmail at mycircuit.org Thu Jan 7 15:12:54 2010 From: vmail at mycircuit.org (Peter) Date: Thu, 07 Jan 2010 21:12:54 +0100 Subject: Recommended "new" way for config files In-Reply-To: <50697b2c1001071023m28e9a242uef20c6e65414c371@mail.gmail.com> References: <4B46075D.8080606@mycircuit.org> <4B460CA0.20800@sequans.com> <4B4625BC.6090201@mycircuit.org> <50697b2c1001071023m28e9a242uef20c6e65414c371@mail.gmail.com> Message-ID: <4B464046.6050505@mycircuit.org> > > >> So what is the "worshipped" approach, when you need more than name=value >> pairs ? >> > JSON is one option: http://docs.python.org/library/json.html > > > Thanks, didn't think about that, although most of the apps I know don't seem to use this approach for improved conf file handling ( ipython, pylons, etc ). To me , the ipython way ( hybrid: ipy_user_conf.py and *.ini files ) seems to be the most comprehensive way amongst the larger apps I know of, since it let you have a python coded file for what ever you might want to do during initialization and have additional .ini files, ,possibily several in different locations, for simpler options in name,value format. Has anybody experiences with other tools that use this approach ? Peter From zuo at chopin.edu.pl Thu Jan 7 15:18:27 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Thu, 07 Jan 2010 21:18:27 +0100 Subject: How to execute a script from another script and other script does notdo busy wait. In-Reply-To: <016d2fad-a925-47f4-8b76-5602a8532651@r5g2000yqb.googlegroups.com> References: <016d2fad-a925-47f4-8b76-5602a8532651@r5g2000yqb.googlegroups.com> Message-ID: Rajat wrote: > I've single CPU machine. I've a feeling that the thread created, which > would run script2, would eat up all of the CPU if I do not use sleep() > in script2. > That way, script1 would still be waiting for script2 to finish. Single CPU is not a problem for threads (in fact it's even better). It'll work. Try it. Another possibility is to run script2 in a separate process (e.g. using subprocess module). Cheers, *j From zuo at chopin.edu.pl Thu Jan 7 15:27:52 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Thu, 07 Jan 2010 21:27:52 +0100 Subject: File transfer with python In-Reply-To: References: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> Message-ID: Valentin de Pablo Fouce wrote: > On 6 ene, 22:42, "Jan Kaliszewski" wrote: >> Valentin de Pablo Fouce wrote: >> >> > Ok, I am trying to do a very quick application (is "home based" so is >> > not a big deal...). My intention is to transfer files from one >> > computer to another. >> > My intention is to be able to transfer files from one computer to >> > another in this environment. >> >> > Looking (and surfing) at internet the only suggestion given is to use >> > low level sockets for this file transfer. Is there another way to do >> > it, is there any top level library that helps you to do that? >> >> Python standard library offers tools for HTTP communication (rather >> easy to use) -- > Your solution looks quite nice...but one question, just by looking to > it I think I will need to create an HTTP server on a "file" server PC, > isn't it? Yes, in case of such communication protocols like HTTP always one side have a role of a serwer, and the other -- of a client. However, please note that files can be both downloaded *from* serwer and uploaded *to* serwer. You can also run *each* side both as a client *and* as a serwer (though it's rather unnecessary...). Cheers, *j From aahz at pythoncraft.com Thu Jan 7 15:42:56 2010 From: aahz at pythoncraft.com (Aahz) Date: 7 Jan 2010 12:42:56 -0800 Subject: Regex help needed! References: <19de1d6e-5ba9-42b5-9221-ed7246e39b4a@u36g2000prn.googlegroups.com> Message-ID: In article <19de1d6e-5ba9-42b5-9221-ed7246e39b4a at u36g2000prn.googlegroups.com>, Oltmans wrote: > >I've written this regex that's kind of working >re.findall("\w+\s*\W+amazon_(\d+)",str) > >but I was just wondering that there might be a better RegEx to do that >same thing. Can you kindly suggest a better/improved Regex. Thank you >in advance. 'Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems.' --Jamie Zawinski Take the advice other people gave you and use BeautifulSoup. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." --Red Adair From vardan.pogosyan at gmail.com Thu Jan 7 16:03:24 2010 From: vardan.pogosyan at gmail.com (h0uk) Date: Thu, 7 Jan 2010 13:03:24 -0800 (PST) Subject: PyQt QThreadPool error References: <7dc7a9f3-cff9-4911-bfde-6086db471b41@k17g2000yqh.googlegroups.com> <7qmsujFc9tU2@mid.uni-berlin.de> Message-ID: <7c781f4c-45c4-4e15-ac21-c4ca174a9a89@l30g2000yqb.googlegroups.com> On 8 ???, 01:02, "Diez B. Roggisch" wrote: > h0uk schrieb: > > > > > Hello. > > > I have the following code: > > > ? ? ? ? ? ? #workers = {} > > ? ? ? ? ? ? QtCore.QThreadPool.globalInstance().setExpiryTimeout > > (300000) > > ? ? ? ? ? ? QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) > > ? ? ? ? ? ? for i in range(1, int(userscnt) + 1): > > ? ? ? ? ? ? ? ? work = wk.Worker(i) > > ? ? ? ? ? ? ? ? # connect signals > > ? ? ? ? ? ? ? ? work.mmShowCaptcha.connect(self.show_captcha_dlg) > > ? ? ? ? ? ? ? ? work.log.connect(self.handle_log) > > ? ? ? ? ? ? ? ? self.captcha_answer.connect(work.mmCaptchaAnswer) > > ? ? ? ? ? ? ? ? work.setAutoDelete(True) > > ? ? ? ? ? ? ? ? QtCore.QThreadPool.globalInstance().start(work) > > > On last line of code ( QtCore.QThreadPool.globalInstance().start > > (work) ) i get an error: > > > SystemError: error return without exception set > > > What is wrong in my code??? Any advise??? > > The error is on C-level. AFAIK it occurs when a Python-C-function > returns "NULL" without setting an exception. > > It's hard to say where it really occurs. I'd use a debug-build of PyQt, > and possibly Python, and then investigate using gdb. > > Alternatively, what happens when you do some "dummy"-work that doesn't > use signals and no other libraries? > > Diez About some "dummy" code: #!/usr/bin/env python # -*- coding: utf-8 -*- import sys import os import time from PyQt4 import QtCore, QtGui class Job(QtCore.QRunnable): def __init__(self, name): QtCore.QRunnable.__init__(self) self._name = name def run(self): time.sleep(10) print self._name def autoDelete(self): return self._auto def setAutoDelete(self, auto): self._auto = auto if __name__ == "__main__": app = QtGui.QApplication(sys.argv) QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) j = Job("Job-1") j.setAutoDelete(True) QtCore.QThreadPool.globalInstance().start(j) Even this code not work. On the last line of code ( QtCore.QThreadPool.globalInstance().start(j) ) i get the error: >> An unhandled win32 exception occured in python.exe From mailist at sokol-web.de Thu Jan 7 16:19:38 2010 From: mailist at sokol-web.de (Kamill Sokol) Date: Thu, 7 Jan 2010 13:19:38 -0800 (PST) Subject: File transfer with python References: <4592f37d-4bc6-4a45-92bd-392745a97875@u7g2000yqm.googlegroups.com> Message-ID: <3171c09a-a72b-4485-9b96-31b8b6f9e032@c34g2000yqn.googlegroups.com> On Jan 6, 7:00?pm, Valentin de Pablo Fouce wrote: > Hi there, > > My intention is to transfer files from one > computer to another. > Hi Valentin, take a look at dropbox! Clients are available for mac, linux and windows. Up and running in just 2 minutes. http://dropbox.com Regards Kamill From deets at nospam.web.de Thu Jan 7 16:25:33 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Thu, 07 Jan 2010 22:25:33 +0100 Subject: PyQt QThreadPool error In-Reply-To: <7c781f4c-45c4-4e15-ac21-c4ca174a9a89@l30g2000yqb.googlegroups.com> References: <7dc7a9f3-cff9-4911-bfde-6086db471b41@k17g2000yqh.googlegroups.com> <7qmsujFc9tU2@mid.uni-berlin.de> <7c781f4c-45c4-4e15-ac21-c4ca174a9a89@l30g2000yqb.googlegroups.com> Message-ID: <7qn1qdFcqvU1@mid.uni-berlin.de> h0uk schrieb: > On 8 ???, 01:02, "Diez B. Roggisch" wrote: >> h0uk schrieb: >> >> >> >>> Hello. >>> I have the following code: >>> #workers = {} >>> QtCore.QThreadPool.globalInstance().setExpiryTimeout >>> (300000) >>> QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) >>> for i in range(1, int(userscnt) + 1): >>> work = wk.Worker(i) >>> # connect signals >>> work.mmShowCaptcha.connect(self.show_captcha_dlg) >>> work.log.connect(self.handle_log) >>> self.captcha_answer.connect(work.mmCaptchaAnswer) >>> work.setAutoDelete(True) >>> QtCore.QThreadPool.globalInstance().start(work) >>> On last line of code ( QtCore.QThreadPool.globalInstance().start >>> (work) ) i get an error: >>> SystemError: error return without exception set >>> What is wrong in my code??? Any advise??? >> The error is on C-level. AFAIK it occurs when a Python-C-function >> returns "NULL" without setting an exception. >> >> It's hard to say where it really occurs. I'd use a debug-build of PyQt, >> and possibly Python, and then investigate using gdb. >> >> Alternatively, what happens when you do some "dummy"-work that doesn't >> use signals and no other libraries? >> >> Diez > > About some "dummy" code: > > #!/usr/bin/env python > # -*- coding: utf-8 -*- > > import sys > import os > import time > > from PyQt4 import QtCore, QtGui > > class Job(QtCore.QRunnable): > def __init__(self, name): > QtCore.QRunnable.__init__(self) > self._name = name > > def run(self): > time.sleep(10) > print self._name > > def autoDelete(self): > return self._auto > > def setAutoDelete(self, auto): > self._auto = auto > > > > if __name__ == "__main__": > > app = QtGui.QApplication(sys.argv) > > QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) > > j = Job("Job-1") > j.setAutoDelete(True) > QtCore.QThreadPool.globalInstance().start(j) > > > Even this code not work. On the last line of code > ( QtCore.QThreadPool.globalInstance().start(j) ) i get the error: > >>> An unhandled win32 exception occured in python.exe Hm. I suggest you take this to the PyQt mailinglist. Phil Thompson is very responsive. Diez From vardan.pogosyan at gmail.com Thu Jan 7 16:33:24 2010 From: vardan.pogosyan at gmail.com (h0uk) Date: Thu, 7 Jan 2010 13:33:24 -0800 (PST) Subject: PyQt QThreadPool error References: <7dc7a9f3-cff9-4911-bfde-6086db471b41@k17g2000yqh.googlegroups.com> <7qmsujFc9tU2@mid.uni-berlin.de> <7c781f4c-45c4-4e15-ac21-c4ca174a9a89@l30g2000yqb.googlegroups.com> <7qn1qdFcqvU1@mid.uni-berlin.de> Message-ID: <343c51a8-e352-4a9c-bb4d-bdc2b82ecc47@j14g2000yqm.googlegroups.com> On 8 ???, 02:25, "Diez B. Roggisch" wrote: > h0uk schrieb: > > > > > On 8 ???, 01:02, "Diez B. Roggisch" wrote: > >> h0uk schrieb: > > >>> Hello. > >>> I have the following code: > >>> ? ? ? ? ? ? #workers = {} > >>> ? ? ? ? ? ? QtCore.QThreadPool.globalInstance().setExpiryTimeout > >>> (300000) > >>> ? ? ? ? ? ? QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) > >>> ? ? ? ? ? ? for i in range(1, int(userscnt) + 1): > >>> ? ? ? ? ? ? ? ? work = wk.Worker(i) > >>> ? ? ? ? ? ? ? ? # connect signals > >>> ? ? ? ? ? ? ? ? work.mmShowCaptcha.connect(self.show_captcha_dlg) > >>> ? ? ? ? ? ? ? ? work.log.connect(self.handle_log) > >>> ? ? ? ? ? ? ? ? self.captcha_answer.connect(work.mmCaptchaAnswer) > >>> ? ? ? ? ? ? ? ? work.setAutoDelete(True) > >>> ? ? ? ? ? ? ? ? QtCore.QThreadPool.globalInstance().start(work) > >>> On last line of code ( QtCore.QThreadPool.globalInstance().start > >>> (work) ) i get an error: > >>> SystemError: error return without exception set > >>> What is wrong in my code??? Any advise??? > >> The error is on C-level. AFAIK it occurs when a Python-C-function > >> returns "NULL" without setting an exception. > > >> It's hard to say where it really occurs. I'd use a debug-build of PyQt, > >> and possibly Python, and then investigate using gdb. > > >> Alternatively, what happens when you do some "dummy"-work that doesn't > >> use signals and no other libraries? > > >> Diez > > > About some "dummy" code: > > > #!/usr/bin/env python > > # -*- coding: utf-8 -*- > > > import sys > > import os > > import time > > > from PyQt4 import QtCore, QtGui > > > class Job(QtCore.QRunnable): > > ? ?def __init__(self, name): > > ? ? ? ? ? ?QtCore.QRunnable.__init__(self) > > ? ? ? ? ? ?self._name = name > > > ? ?def run(self): > > ? ? ? ? ? ?time.sleep(10) > > ? ? ? ? ? ?print self._name > > > ? ?def autoDelete(self): > > ? ? ? ? ? ?return self._auto > > > ? ?def setAutoDelete(self, auto): > > ? ? ? ? ? ?self._auto = auto > > > if __name__ == "__main__": > > > ? ?app = QtGui.QApplication(sys.argv) > > > ? ?QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) > > > ? ?j = Job("Job-1") > > ? ?j.setAutoDelete(True) > > ? ?QtCore.QThreadPool.globalInstance().start(j) > > > Even this code not work. On the last line of code > > ( QtCore.QThreadPool.globalInstance().start(j) ) i get the error: > > >>> An unhandled win32 exception occured in python.exe > > Hm. I suggest you take this to the PyQt mailinglist. Phil Thompson is > very responsive. > > Diez Thanks you, Diez. Thanks for your time. I already wrote to PyQt mailing list, but yet no get answer. I will be waiting and try to write Phil Thompson. From phil at riverbankcomputing.com Thu Jan 7 17:02:56 2010 From: phil at riverbankcomputing.com (Phil Thompson) Date: Thu, 07 Jan 2010 22:02:56 +0000 Subject: PyQt QThreadPool error In-Reply-To: <7c781f4c-45c4-4e15-ac21-c4ca174a9a89@l30g2000yqb.googlegroups.com> References: <7dc7a9f3-cff9-4911-bfde-6086db471b41@k17g2000yqh.googlegroups.com> <7qmsujFc9tU2@mid.uni-berlin.de> <7c781f4c-45c4-4e15-ac21-c4ca174a9a89@l30g2000yqb.googlegroups.com> Message-ID: <30264a260cf2f1faafd319e080de1522@localhost> On Thu, 7 Jan 2010 13:03:24 -0800 (PST), h0uk wrote: > On 8 ???, 01:02, "Diez B. Roggisch" wrote: >> h0uk schrieb: >> >> >> >> > Hello. >> >> > I have the following code: >> >> > ? ? ? ? ? ? #workers = {} >> > ? ? ? ? ? ? QtCore.QThreadPool.globalInstance().setExpiryTimeout >> > (300000) >> > ? ? ? ? ? ? >> > QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) >> > ? ? ? ? ? ? for i in range(1, int(userscnt) + 1): >> > ? ? ? ? ? ? ? ? work = wk.Worker(i) >> > ? ? ? ? ? ? ? ? # connect signals >> > ? ? ? ? ? ? ? ? >> > work.mmShowCaptcha.connect(self.show_captcha_dlg) >> > ? ? ? ? ? ? ? ? work.log.connect(self.handle_log) >> > ? ? ? ? ? ? ? ? >> > self.captcha_answer.connect(work.mmCaptchaAnswer) >> > ? ? ? ? ? ? ? ? work.setAutoDelete(True) >> > ? ? ? ? ? ? ? ? QtCore.QThreadPool.globalInstance().start(work) >> >> > On last line of code ( QtCore.QThreadPool.globalInstance().start >> > (work) ) i get an error: >> >> > SystemError: error return without exception set >> >> > What is wrong in my code??? Any advise??? >> >> The error is on C-level. AFAIK it occurs when a Python-C-function >> returns "NULL" without setting an exception. >> >> It's hard to say where it really occurs. I'd use a debug-build of PyQt, >> and possibly Python, and then investigate using gdb. >> >> Alternatively, what happens when you do some "dummy"-work that doesn't >> use signals and no other libraries? >> >> Diez > > About some "dummy" code: > > #!/usr/bin/env python > # -*- coding: utf-8 -*- > > import sys > import os > import time > > from PyQt4 import QtCore, QtGui > > class Job(QtCore.QRunnable): > def __init__(self, name): > QtCore.QRunnable.__init__(self) > self._name = name > > def run(self): > time.sleep(10) > print self._name > > def autoDelete(self): > return self._auto > > def setAutoDelete(self, auto): > self._auto = auto > > > > if __name__ == "__main__": > > app = QtGui.QApplication(sys.argv) > > QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) > > j = Job("Job-1") > j.setAutoDelete(True) > QtCore.QThreadPool.globalInstance().start(j) > > > Even this code not work. On the last line of code > ( QtCore.QThreadPool.globalInstance().start(j) ) i get the error: > >>> An unhandled win32 exception occured in python.exe You aren't letting the thread run before exiting the program. Try adding... app.exec_() ...after you call start(). Also, I'm not sure what you are trying to achieve with your implementations of autoDelete() and setAutoDelete(). Phil From klaussfreire at gmail.com Thu Jan 7 17:18:34 2010 From: klaussfreire at gmail.com (Klauss) Date: Thu, 7 Jan 2010 14:18:34 -0800 (PST) Subject: a huge shared read-only data in parallel accesses -- How? multithreading? multiprocessing? References: <6fdd9088-7418-488a-9868-abe6446cb388@r5g2000yqb.googlegroups.com> <33b66d1e-d674-4008-88f6-45823aa74485@s3g2000yqs.googlegroups.com> Message-ID: On Dec 31 2009, 6:36?pm, garyrob wrote: > One thing I'm not clear on regarding Klauss' patch. He says it's > applicable where the data is primarily non-numeric. In trying to > understand why that would be the case, I'm thinking that the increased > per-object memory overhead for reference-counting would outweigh the > space gains from the shared memory. > > Klauss's test code stores a large number of dictionaries which each > contain just 3 items. The stored items are strings, but short ones... > it looks like they take up less space than double floats(?). > > So my understanding is that the point is that the overhead for the > dictionaries is big enough that the patch is very helpful even though > the stored items are small. And that the patch would be less and less > effective as the number of items stored in each dictionary became > greater and greater, until eventually the patch might do more use more > space for reference counting than it saved by shared memory. Not really. The real difference is that numbers (ints and floats) are allocated out of small contiguous pools. So even if a great percentage of those objects would remain read-only, there's probably holes in those pools left by the irregular access pattern during initialization, and those holes would be written to eventually as the pool gets used. In essence, those pools aren't read-only for other reasons than reference counting. Dictionaries, tuples and lists (and many other types) don't exhibit that behavior. From g.bogle at auckland.no.spam.ac.nz Thu Jan 7 17:30:53 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Fri, 08 Jan 2010 11:30:53 +1300 Subject: Threading change, 2.5.4 -> 2.6.1 Message-ID: The code below runs with Python 2.5.4, but gives the following error messages with Python 2.6.1. What needs to be done to make it work? Thanks. C:\Summer09\Tutorials>python url_queue.pyw Traceback (most recent call last): File "url_queue.pyw", line 3, in import threading File "C:\Summer09\Tutorials\threading.py", line 9, in class ProcessingThread(threading.Thread, QtCore.QObject): AttributeError: 'module' object has no attribute 'Thread' url_queue.py ------------ #!/usr/bin/env python import Queue import threading import urllib2 import time hosts = ["http://yahoo.com", "http://google.com", "http://amazon.com", "http://ibm.com", "http://apple.com"] queue = Queue.Queue() class ThreadUrl(threading.Thread): #"""Threaded Url Grab""" def __init__(self, queue,i): threading.Thread.__init__(self) self.queue = queue self.num = i print "Thread: ",self.num def run(self): while True: #grabs host from queue host = self.queue.get() print "num, host: ",self.num,host #grabs urls of hosts and prints first 1024 bytes of page url = urllib2.urlopen(host) print url.read(1024) #signals to queue job is done self.queue.task_done() start = time.time() def main(): #spawn a pool of threads, and pass them queue instance for i in range(5): t = ThreadUrl(queue,i) t.setDaemon(True) t.start() #populate queue with data for host in hosts: queue.put(host) #wait on the queue until everything has been processed queue.join() main() print "Elapsed Time: %s" % (time.time() - start) From darkrho at gmail.com Thu Jan 7 17:38:57 2010 From: darkrho at gmail.com (Rolando Espinoza La Fuente) Date: Thu, 7 Jan 2010 18:38:57 -0400 Subject: Regex help needed! In-Reply-To: References: <19de1d6e-5ba9-42b5-9221-ed7246e39b4a@u36g2000prn.googlegroups.com> Message-ID: <4eca3f41001071438vb9656ffk13d93255e333466a@mail.gmail.com> # http://gist.github.com/271661 import lxml.html import re src = """ lksjdfls
kdjff lsdfs
sdjfls
sdfsd
welcome
hello, my age is 86 years old and I was born in 1945. Do you know that PI is roughly 3.1443534534534534534 """ regex = re.compile('amazon_(\d+)') doc = lxml.html.document_fromstring(src) for div in doc.xpath('//div[starts-with(@id, "amazon_")]'): match = regex.match(div.get('id')) if match: print match.groups()[0] On Thu, Jan 7, 2010 at 4:42 PM, Aahz wrote: > In article <19de1d6e-5ba9-42b5-9221-ed7246e39b4a at u36g2000prn.googlegroups.com>, > Oltmans ? wrote: >> >>I've written this regex that's kind of working >>re.findall("\w+\s*\W+amazon_(\d+)",str) >> >>but I was just wondering that there might be a better RegEx to do that >>same thing. Can you kindly suggest a better/improved Regex. Thank you >>in advance. > > 'Some people, when confronted with a problem, think "I know, I'll use > regular expressions." ?Now they have two problems.' > --Jamie Zawinski > > Take the advice other people gave you and use BeautifulSoup. > -- > Aahz (aahz at pythoncraft.com) ? ? ? ? ? <*> ? ? ? ? http://www.pythoncraft.com/ > > "If you think it's expensive to hire a professional to do the job, wait > until you hire an amateur." ?--Red Adair > -- > http://mail.python.org/mailman/listinfo/python-list > -- Rolando Espinoza La fuente www.rolandoespinoza.info From MLMDev at Comcast.net Thu Jan 7 17:47:13 2010 From: MLMDev at Comcast.net (Mitchell L Model) Date: Thu, 7 Jan 2010 17:47:13 -0500 Subject: One function calling another defined in the same file being exec'd Message-ID: <67CD7A9A-2446-4CE3-AAF6-8F737FA5A68D@Comcast.net> [Python 3.1] I thought I thoroughly understood eval, exec, globals, and locals, but I encountered something bewildering today. I have some short files I want to exec. (Users of my application write them, and the application gives them a command that opens a file dialog box and execs the chosen file. Users are expected to be able to write simple Python scripts, including function definitions. Neither security nor errors are relevant for the purposes of this discussion, though I do deal with them in my actual code.) Here is a short piece of code to exec a file and report its result. (The file being exec'd must assign 'result'.) def dofile(filename): ldict = {'result': None} with open(filename) as file: exec(file.read(), globals(), ldict) print('Result for {}: {}'.format(filename, ldict['result'])) First I call dofile() on a file containing the following: ################################ def fn(arg): return sum(range(arg)) result = fn(5) ################################ The results are as expected. Next I call dofile() on a slightly more complex file, in which one function calls another function defined earlier in the same file. ################################ def fn1(val): return sum(range(val)) def fn2(arg): return fn1(arg) result = fn2(5) ################################ This produces a surprise: NameError: global name 'fn1' is not defined [1] How is it that fn2 can be called from the top-level of the script but fn1 cannot be called from fn2? [2] Is this correct behavior or is there something wrong with Python here? [3] How should I write a file to be exec'd that defines several functions that call each other, as in the trivial fn1-fn2 example above? From anon at blank.com Thu Jan 7 17:55:40 2010 From: anon at blank.com (anon) Date: Thu, 07 Jan 2010 22:55:40 GMT Subject: Threading change, 2.5.4 -> 2.6.1 In-Reply-To: References: Message-ID: Gib Bogle wrote: > The code below runs with Python 2.5.4, but gives the following error > messages with Python 2.6.1. What needs to be done to make it work? > Thanks. > > C:\Summer09\Tutorials>python url_queue.pyw > Traceback (most recent call last): > File "url_queue.pyw", line 3, in > import threading > File "C:\Summer09\Tutorials\threading.py", line 9, in > class ProcessingThread(threading.Thread, QtCore.QObject): > AttributeError: 'module' object has no attribute 'Thread' > Nothing to do with the version of python You have a file called threading.py in C:\Summer09\Tutorials (the same folder as url_queue.pyw). The line 'import threading' is finding this module before python's threading module. Rename this file and you should be fine. From Lee at JamToday.com Thu Jan 7 17:58:37 2010 From: Lee at JamToday.com (Lee) Date: Thu, 07 Jan 2010 17:58:37 -0500 Subject: Dictionary used to build a Triple Store In-Reply-To: References: Message-ID: Lee wrote: > Definitely a newbie question, so please bear with me. > > I'm reading "Programming the Semantic Web" by Segaran, Evans, and Tayor. > > It's about the Semantic Web BUT it uses python to build a "toy" triple > store claimed to have good performance in the "tens of thousands" of > triples. > > Just in case anybody doesnt know what an RDF triple is (not that it > matters for my question) think of it as an ordered 3 tuple representing > a Subject, a Predicate, and an Object eg: (John, loves, Mary) (Mary, > has-a, lamb) {theSky, has-color,blue} > > To build the triple store entirely in Python, the authors recommend > using the Python hash. Three hashes actually (I get that. You > want to have a hash with the major index being the Subject in one hash, > the Predicate in another hash, or the Object for the third hash) > > He creates a class SimpleGraph which initializes itself by setting the > three hashes names _spo, _pos, and _osp thus > > class SimpleGraph; > def __init__(self); > self._spo={}; > self._pos=(); > self._osp={}; > > So far so good. I get the convention with the double underbars for the > initializer but > > Q1: Not the main question but while I'm here....I'm a little fuzzy on > the convention about the use of the single underbar in the definition of > the hashes. Id the idea to "underbar" all objects and methods that > belong to the class? Why do that? > > But now the good stuff: > > Our authors define the hashes thus: (showing only one of the three > hashes because they're all the same idea) > > self._pos = {predicate:{object:set( [subject] ) }} > > Q2: Wha? Two surprises ... > 1) Why not {predicate:{object:subject}} i.e. > pos[predicate][object]=subject....why the set( [object] ) construct? > putting the object into a list and turning the list into a set to be the > "value" part of a name:value pair. Why not just use the naked subject > for the value? > > 2) Why not something like pos[predicate][object][subject] = 1 .....or > any constant. The idea being to create the set of three indexes. If the > triple exists in the hash, its "in" your tripple store. If not, then > there's no such triple. > > OK, Thanx. That cleares things up. I had forgotton that once you have a "Cell" in a 2D matrix to represent the first two components of the 3 tuple, you then want multiple values IN the cell for the possibly multi valued third component. From python at mrabarnett.plus.com Thu Jan 7 18:01:52 2010 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 07 Jan 2010 23:01:52 +0000 Subject: Threading change, 2.5.4 -> 2.6.1 In-Reply-To: References: Message-ID: <4B4667E0.8070101@mrabarnett.plus.com> Gib Bogle wrote: > The code below runs with Python 2.5.4, but gives the following error > messages with Python 2.6.1. What needs to be done to make it work? > Thanks. > > C:\Summer09\Tutorials>python url_queue.pyw > Traceback (most recent call last): > File "url_queue.pyw", line 3, in > import threading > File "C:\Summer09\Tutorials\threading.py", line 9, in > class ProcessingThread(threading.Thread, QtCore.QObject): > AttributeError: 'module' object has no attribute 'Thread' > [snip] url_queue.pyw is trying to import Python's "threading" module, but it's finding the threading.py script in your "Tutorials" folder first. Renaming your tutorial script to something other than "threading.py". From vardan.pogosyan at gmail.com Thu Jan 7 18:07:10 2010 From: vardan.pogosyan at gmail.com (h0uk) Date: Thu, 7 Jan 2010 15:07:10 -0800 (PST) Subject: PyQt QThreadPool error References: <7dc7a9f3-cff9-4911-bfde-6086db471b41@k17g2000yqh.googlegroups.com> <7qmsujFc9tU2@mid.uni-berlin.de> <7c781f4c-45c4-4e15-ac21-c4ca174a9a89@l30g2000yqb.googlegroups.com> Message-ID: On 8 ???, 03:02, Phil Thompson wrote: > On Thu, 7 Jan 2010 13:03:24 -0800 (PST), h0uk > wrote: > > > On 8 ???, 01:02, "Diez B. Roggisch" wrote: > >> h0uk schrieb: > > >> > Hello. > > >> > I have the following code: > > >> > ? ? ? ? ? ? #workers = {} > >> > ? ? ? ? ? ? QtCore.QThreadPool.globalInstance().setExpiryTimeout > >> > (300000) > >> > ? ? ? ? ? ? > >> > QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) > >> > ? ? ? ? ? ? for i in range(1, int(userscnt) + 1): > >> > ? ? ? ? ? ? ? ? work = wk.Worker(i) > >> > ? ? ? ? ? ? ? ? # connect signals > >> > ? ? ? ? ? ? ? ? > >> > work.mmShowCaptcha.connect(self.show_captcha_dlg) > >> > ? ? ? ? ? ? ? ? work.log.connect(self.handle_log) > >> > ? ? ? ? ? ? ? ? > >> > self.captcha_answer.connect(work.mmCaptchaAnswer) > >> > ? ? ? ? ? ? ? ? work.setAutoDelete(True) > >> > ? ? ? ? ? ? ? ? > > QtCore.QThreadPool.globalInstance().start(work) > > > > > > >> > On last line of code ( QtCore.QThreadPool.globalInstance().start > >> > (work) ) i get an error: > > >> > SystemError: error return without exception set > > >> > What is wrong in my code??? Any advise??? > > >> The error is on C-level. AFAIK it occurs when a Python-C-function > >> returns "NULL" without setting an exception. > > >> It's hard to say where it really occurs. I'd use a debug-build of PyQt, > >> and possibly Python, and then investigate using gdb. > > >> Alternatively, what happens when you do some "dummy"-work that doesn't > >> use signals and no other libraries? > > >> Diez > > > About some "dummy" code: > > > #!/usr/bin/env python > > # -*- coding: utf-8 -*- > > > import sys > > import os > > import time > > > from PyQt4 import QtCore, QtGui > > > class Job(QtCore.QRunnable): > > ? ?def __init__(self, name): > > ? ? ? ? ? ?QtCore.QRunnable.__init__(self) > > ? ? ? ? ? ?self._name = name > > > ? ?def run(self): > > ? ? ? ? ? ?time.sleep(10) > > ? ? ? ? ? ?print self._name > > > ? ?def autoDelete(self): > > ? ? ? ? ? ?return self._auto > > > ? ?def setAutoDelete(self, auto): > > ? ? ? ? ? ?self._auto = auto > > > if __name__ == "__main__": > > > ? ?app = QtGui.QApplication(sys.argv) > > > ? ?QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) > > > ? ?j = Job("Job-1") > > ? ?j.setAutoDelete(True) > > ? ?QtCore.QThreadPool.globalInstance().start(j) > > > Even this code not work. On the last line of code > > ( QtCore.QThreadPool.globalInstance().start(j) ) i get the error: > > >>> An unhandled win32 exception occured in python.exe > > You aren't letting the thread run before exiting the program. Try adding... > > ? ? app.exec_() > > ...after you call start(). > > Also, I'm not sure what you are trying to achieve with your implementations > of autoDelete() and setAutoDelete(). > > Phil Phil you right about app.exec_(). But situation is sligthly different. I want to have more than one Job. I add these Jobs into QThreadPool trough cycle. And I also want these Jobs to run sequentially. The following code illustrate what I mean: #!/usr/bin/env python # -*- coding: utf-8 -*- import sys import os import time from PyQt4 import QtCore, QtGui class Job(QtCore.QRunnable): def __init__(self, name): QtCore.QRunnable.__init__(self) self._name = name def run(self): time.sleep(3) print self._name if __name__ == "__main__": app = QtGui.QApplication(sys.argv) QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) for i in range(5): j = Job("Job-" + str(i)) j.setAutoDelete(True) QtCore.QThreadPool.globalInstance().start(j, i) app.exec_() After 5 cycle I get the same error: An unhandled win32 exception occured in python.exe. How I can do it?? To run my Jobs sequentially??? Vardan. From vmail at mycircuit.org Thu Jan 7 18:13:16 2010 From: vmail at mycircuit.org (Peter) Date: Fri, 08 Jan 2010 00:13:16 +0100 Subject: restructuredText editor ? Message-ID: <4B466A8C.2020609@mycircuit.org> What editor do people out there use to edit .rst files for sphinx-python documentation ? Peter From mailtome200420032002 at gmail.com Thu Jan 7 18:23:48 2010 From: mailtome200420032002 at gmail.com (aj) Date: Thu, 7 Jan 2010 15:23:48 -0800 (PST) Subject: Accessing python from a network share in windows 7 Message-ID: I access python from a network share. This works fine on XP but on windows 7 it throws the following error: Python 2.6.1 (r261:67517, Dec 4 2008, 16:51:00) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import random Traceback (most recent call last): File "", line 1, in File "t:\win32\python-2.6.1\lib\random.py", line 871, in _inst = Random() File "t:\win32\python-2.6.1\lib\random.py", line 96, in __init__ self.seed(x) File "t:\win32\python-2.6.1\lib\random.py", line 110, in seed a = long(_hexlify(_urandom(16)), 16) WindowsError: [Error 127] The specified procedure could not be found Is there some security policy that I need to enable/disable to use python from a network on windows 7? From MLMDev at Comcast.net Thu Jan 7 18:34:57 2010 From: MLMDev at Comcast.net (Mitchell L Model) Date: Thu, 7 Jan 2010 18:34:57 -0500 Subject: One function calling another defined in the same file being exec'd Message-ID: I forgot to offer one answer for question [3] in what I just posted: I can define all the secondary functions inside one main one and just call the main one. That provides a separate local scope within the main function, with the secondary functions defined inside it when (each time) the main function is called. Not too bad, but will freak out my users and it doesn't seem as if it should be necessary to resort to this. From anon at blank.com Thu Jan 7 18:39:45 2010 From: anon at blank.com (anon) Date: Thu, 07 Jan 2010 23:39:45 GMT Subject: One function calling another defined in the same file being exec'd In-Reply-To: References: Message-ID: <5hu1n.23389$Ym4.557@text.news.virginmedia.com> Rather than exec the files, why not import them? I can get both your examples to work using the 'imp' module. http://docs.python.org/3.1/library/imp.html#module-imp I used python 2.6.4. Note that 3.1 also has 'importlib' module. import imp # the name of the python file written by a user name = 'test1' fp, pathname, description = imp.find_module(name) test1 = imp.load_module(name, fp, pathname, description) print test1.result # remember to close file (see docs) fp.close() From python at mrabarnett.plus.com Thu Jan 7 18:51:50 2010 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 07 Jan 2010 23:51:50 +0000 Subject: Accessing python from a network share in windows 7 In-Reply-To: References: Message-ID: <4B467396.3080009@mrabarnett.plus.com> aj wrote: > I access python from a network share. This works fine on XP but on > windows 7 it throws the following error: > > Python 2.6.1 (r261:67517, Dec 4 2008, 16:51:00) [MSC v.1500 32 bit > (Intel)] on > win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> import random > Traceback (most recent call last): > File "", line 1, in > File "t:\win32\python-2.6.1\lib\random.py", line 871, in > _inst = Random() > File "t:\win32\python-2.6.1\lib\random.py", line 96, in __init__ > self.seed(x) > File "t:\win32\python-2.6.1\lib\random.py", line 110, in seed > a = long(_hexlify(_urandom(16)), 16) > WindowsError: [Error 127] The specified procedure could not be found > > Is there some security policy that I need to enable/disable to use > python from a network on windows 7? Is it a problem with the share or with Windows 7? Does it work with Windows 7 when running a local copy of Python? From mailtome200420032002 at gmail.com Thu Jan 7 19:01:05 2010 From: mailtome200420032002 at gmail.com (aj) Date: Thu, 7 Jan 2010 16:01:05 -0800 (PST) Subject: Accessing python from a network share in windows 7 References: Message-ID: <08ac1875-8747-4225-bf78-21972e959386@k9g2000vbl.googlegroups.com> On Jan 7, 3:51?pm, MRAB wrote: > aj wrote: > > I access python from a network share. This works fine on XP but on > > windows 7 it throws the following error: > > > Python 2.6.1 (r261:67517, Dec ?4 2008, 16:51:00) [MSC v.1500 32 bit > > (Intel)] on > > win32 > > Type "help", "copyright", "credits" or "license" for more information. > >>>> import random > > Traceback (most recent call last): > > ? File "", line 1, in > > ? File "t:\win32\python-2.6.1\lib\random.py", line 871, in > > ? ? _inst = Random() > > ? File "t:\win32\python-2.6.1\lib\random.py", line 96, in __init__ > > ? ? self.seed(x) > > ? File "t:\win32\python-2.6.1\lib\random.py", line 110, in seed > > ? ? a = long(_hexlify(_urandom(16)), 16) > > WindowsError: [Error 127] The specified procedure could not be found > > > Is there some security policy that I need to enable/disable to use > > python from a network on windows 7? > > Is it a problem with the share or with Windows 7? Does it work with > Windows 7 when running a local copy of Python? It works without any issue on win7 if I copy python to my local drive. Also, accessing python from the same network share works fine on win XP. So I am suspecting some security policy of win7 that is causing problem while accessing it over a network share. From suresh.amritapuri at gmail.com Thu Jan 7 19:14:45 2010 From: suresh.amritapuri at gmail.com (suresh.amritapuri) Date: Thu, 7 Jan 2010 16:14:45 -0800 (PST) Subject: PIL show() not working for 2nd pic Message-ID: Hi I am using PIL for image processing in ubuntu 9.04. When i give two im.show() commands for two different images, the second image is not displayed (eye of gnome is the display program). It says no such file or directory. Any ideas? thanks suresh From g.bogle at auckland.no.spam.ac.nz Thu Jan 7 20:14:59 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Fri, 08 Jan 2010 14:14:59 +1300 Subject: Threading change, 2.5.4 -> 2.6.1 References: Message-ID: MRAB wrote: > Gib Bogle wrote: >> The code below runs with Python 2.5.4, but gives the following error >> messages with Python 2.6.1. What needs to be done to make it work? >> Thanks. >> >> C:\Summer09\Tutorials>python url_queue.pyw >> Traceback (most recent call last): >> File "url_queue.pyw", line 3, in >> import threading >> File "C:\Summer09\Tutorials\threading.py", line 9, in >> class ProcessingThread(threading.Thread, QtCore.QObject): >> AttributeError: 'module' object has no attribute 'Thread' >> > [snip] > url_queue.pyw is trying to import Python's "threading" module, but it's > finding the threading.py script in your "Tutorials" folder first. > > Renaming your tutorial script to something other than "threading.py". Thanks very much to you both. It's actually not in my folder, it is (presumably) in my student's folder (I'm the one using 2.5.4, she has 2.6.1 and thought this could be the problem.) From no.i.dont at want.mail.from.spammers.com Thu Jan 7 20:21:01 2010 From: no.i.dont at want.mail.from.spammers.com (Fencer) Date: Fri, 08 Jan 2010 02:21:01 +0100 Subject: How do I access what's in this module? Message-ID: <7qnfk0Fr6kU1@mid.individual.net> Hello, look at this lxml documentation page: http://codespeak.net/lxml/api/index.html How do I access the functions and variables listed? I tried from lxml.etree import ElementTree and the import itself seems to pass without complaint by the python interpreter but I can't seem to access anything in ElementTree, not the functions or variables. What is the proper way to import that module? For example: >>> from lxml.etree import ElementTree >>> ElementTree.dump(None) Traceback (most recent call last): File "", line 1, in Also, can I access those items that begin with an underscore if I get the import sorted? - Fencer From apt.shansen at gmail.com Thu Jan 7 20:39:30 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Thu, 7 Jan 2010 17:39:30 -0800 Subject: Where's a DOM builder that uses the Builder Pattern to ... build DOMs? In-Reply-To: References: <04523152-a0ff-432c-a48f-2c653577e7b7@m25g2000yqc.googlegroups.com> <4b42f551$0$7625$9b4e6d93@newsspool1.arcor-online.net> <026df5db-bcef-424d-8cd5-56dacebf01f8@c3g2000yqd.googlegroups.com> <4b45e370$0$6736$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <7a9c25c21001071739t5e2f2689vaf9452ec67784964@mail.gmail.com> On Thu, Jan 7, 2010 at 8:44 AM, Phlip wrote: > On Jan 7, 5:36 am, Stefan Behnel wrote: > > > Well, then note that there are tons of ways to generate XML with Python, > > including the one I pointed you to. > > from lxml.html import builder as E > xml = E.foo() > > All I want is "", but I get "AttributeError: 'module' object has > no attribute 'foo'". > > A peek at dir(E) shows it only has HTML tags, all hard coded. > > So how to get it to generate any random XML tag my clients think of? > If you want to generate random XML, don't use the HTML sub-module of lxml. Its a specific sub-set of functionality for HTML only documents. >>> from lxml.builder import ElementMaker >>> from lxml import etree >>> E = ElementMaker() >>> html = E.html( ... E.form( ... E.input(type="text", name="email"), ... E.input(type="text", name="blah") ... ) ... ) >>> etree.tostring(html) '
' HTH, --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at REMOVE-THIS-cybersource.com.au Thu Jan 7 20:57:53 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 08 Jan 2010 01:57:53 GMT Subject: One function calling another defined in the same file being exec'd References: Message-ID: <03567e01$0$1336$c3e8da3@news.astraweb.com> On Thu, 07 Jan 2010 17:47:13 -0500, Mitchell L Model wrote: > Next I call dofile() on a slightly more complex file, in which one > function calls another function defined earlier in the same file. > > ################################ > def fn1(val): > return sum(range(val)) > > def fn2(arg): > return fn1(arg) > > result = fn2(5) > ################################ > > This produces a surprise: > > NameError: global name 'fn1' is not defined > > [1] How is it that fn2 can be called from the top-level of the script > but fn1 cannot be called from fn2? This might help you to see what's going on. Define your own cut-down version of the global namespace, and a local namespace, and a string to execute: myglobals = {'__builtins__': None, 'globals': globals, 'locals': locals, 'print': print} mylocals = {'result': None} s = """def f(): print("Globals inside f:", globals()) print("Locals inside f:", locals()) print("Globals at the top level:", globals()) print("Locals at the top level:", locals()) f() """ exec(s, myglobals, mylocals) And this is what you should see: Globals at the top level: {'__builtins__': None, 'print': , 'globals': , 'locals': } Locals at the top level: {'result': None, 'f': } Globals inside f: {'__builtins__': None, 'print': , 'globals': , 'locals': } Locals inside f: {} Does that clarify what's going on? > [2] Is this correct behavior or is there something wrong with Python > here? This certainly surprised me too. I don't know if it is correct or not, but it goes back to at least Python 2.5. > [3] How should I write a file to be exec'd that defines several > functions that call each other, as in the trivial fn1-fn2 example above? My preference would be to say, don't use exec, just import the module. Put responsibility on the user to ensure that they set a global "result", and then just do this: mod = __import__('user_supplied_file_name') result = mod.result But if that's unworkable for you, then try simulating the namespace setup at the top level of a module. The thing to remember is that in the top level of a module: >>> globals() is locals() True so let's simulate that: myglobals = {'result': None} # You probably also want __builtins__ s = """def f(): return g() + 1 def g(): return 2 result = f() """ exec(s, myglobals, myglobals) myglobals['result'] This works for me. -- Steven From sjmachin at lexicon.net Thu Jan 7 22:40:26 2010 From: sjmachin at lexicon.net (John Machin) Date: Thu, 7 Jan 2010 19:40:26 -0800 (PST) Subject: How do I access what's in this module? References: <7qnfk0Fr6kU1@mid.individual.net> Message-ID: <29a775c4-223f-4e74-b739-12d633e5739e@a21g2000yqc.googlegroups.com> On Jan 8, 12:21?pm, Fencer wrote: > Hello, look at this lxml documentation page:http://codespeak.net/lxml/api/index.html That's for getting details about an object once you know what object you need to use to do what. In the meantime, consider reading the tutorial and executing some of the examples: http://codespeak.net/lxml/tutorial.html > How do I access the functions and variables listed? > > I tried from lxml.etree import ElementTree and the import itself seems > to pass without complaint by the python interpreter but I can't seem to > access anything in ElementTree, not the functions or variables. What is > the proper way to import that module? > > For example: > ?>>> from lxml.etree import ElementTree > ?>>> ElementTree.dump(None) > Traceback (most recent call last): > ? ?File "", line 1, in lxml.etree is a module. ElementTree is effectively a class. The error message that you omitted to show us might have given you a clue. To save keystrokes you may like to try from lxml import etree as ET and thereafter refer to the module as "ET" | >>> from lxml import etree as ET | >>> type(ET) | | >>> type(ET.ElementTree) | | >>> help(ET.ElementTree) | Help on built-in function ElementTree in module lxml.etree: | | ElementTree(...) | ElementTree(element=None, file=None, parser=None) | | ElementTree wrapper class. > Also, can I access those items that begin with an underscore if I get > the import sorted? Using pommy slang like "sorted" in an IT context has the potential to confuse your transatlantic correspondents :-) Can access? Yes. Should access? The usual Python convention is that an object whose name begins with an underscore should be accessed only via a documented interface (or, at your own risk, if you think you know what you are doing). HTH, John From WaterLin at ymail.invalid Thu Jan 7 22:44:48 2010 From: WaterLin at ymail.invalid (Water Lin) Date: Fri, 08 Jan 2010 11:44:48 +0800 Subject: Ask how to use HTMLParser Message-ID: <87vdfdtg3z.fsf@ymail.invalid> I am a new guy to use Python, but I want to parse a html page now. I tried to use HTMLParse. Here is my sample code: ---------------------- from HTMLParser import HTMLParser from urllib2 import urlopen class MyParser(HTMLParser): title = "" is_title = "" def __init__(self, url): HTMLParser.__init__(self) req = urlopen(url) self.feed(req.read()) def handle_starttag(self, tag, attrs): if tag == 'div' and attrs[0][1] == 'articleTitle': print "Found link => %s" % attrs[0][1] self.is_title = 1 def handle_data(self, data): if self.is_title: print "here" self.title = data print self.title self.is_title = 0 ----------------------- For the tag -------
open article title
------- I use my code to parse it. I can locate the div tag but I don't know how to get the text for the tag which is "open article title" in my example. How can I get the html content? What's wrong in my handle_data function? Thanks Water Lin -- Water Lin's notes and pencils: http://en.waterlin.org Email: WaterLin at ymail.com From no.i.dont at want.mail.from.spammers.com Thu Jan 7 22:45:19 2010 From: no.i.dont at want.mail.from.spammers.com (Fencer) Date: Fri, 08 Jan 2010 04:45:19 +0100 Subject: How do I access what's in this module? In-Reply-To: <29a775c4-223f-4e74-b739-12d633e5739e@a21g2000yqc.googlegroups.com> References: <7qnfk0Fr6kU1@mid.individual.net> <29a775c4-223f-4e74-b739-12d633e5739e@a21g2000yqc.googlegroups.com> Message-ID: <7qno2iF1b7U1@mid.individual.net> On 2010-01-08 04:40, John Machin wrote: >> >> For example: >> >>> from lxml.etree import ElementTree >> >>> ElementTree.dump(None) >> Traceback (most recent call last): >> File "", line 1, in > > lxml.etree is a module. ElementTree is effectively a class. The error > message that you omitted to show us might have given you a clue. But I did show the error message? It's just above what you just wrote. I try to include all relevant information in my posts. > > Using pommy slang like "sorted" in an IT context has the potential to > confuse your transatlantic correspondents :-) Ah, of course! :-) > > Can access? Yes. Should access? The usual Python convention is that an > object whose name begins with an underscore should be accessed only > via a documented interface (or, at your own risk, if you think you > know what you are doing). It turns out I no longer want to access anything in there but I thank you for your information nontheless. > > HTH, > John - Fencer From sjmachin at lexicon.net Thu Jan 7 23:01:00 2010 From: sjmachin at lexicon.net (John Machin) Date: Thu, 7 Jan 2010 20:01:00 -0800 (PST) Subject: How do I access what's in this module? References: <7qnfk0Fr6kU1@mid.individual.net> <29a775c4-223f-4e74-b739-12d633e5739e@a21g2000yqc.googlegroups.com> <7qno2iF1b7U1@mid.individual.net> Message-ID: On Jan 8, 2:45?pm, Fencer wrote: > On 2010-01-08 04:40, John Machin wrote: > > > > >> For example: > >> ? >>> ?from lxml.etree import ElementTree > >> ? >>> ?ElementTree.dump(None) > >> Traceback (most recent call last): > >> ? ? File "", line 1, in > > > lxml.etree is a module. ElementTree is effectively a class. The error > > message that you omitted to show us might have given you a clue. > > But I did show the error message? It's just above what you just wrote. I > try to include all relevant information in my posts. Traceback (most recent call last): File "", line 1, in Also, can I access those items ... Error message should appear after line starting with "File". Above excerpt taken from google groups; identical to what shows in http://news.gmane.org/gmane.comp.python.general ... what are you looking at? With Windows XP and Python 2.5.4 I get: Traceback (most recent call last): File "", line 1, in AttributeError: 'builtin_function_or_method' object has no attribute 'dump' > It turns out I no longer want to access anything in there but I thank > you for your information nontheless. You're welcome -- the advice on _methods is portable :-) From vardan.pogosyan at gmail.com Thu Jan 7 23:17:17 2010 From: vardan.pogosyan at gmail.com (h0uk) Date: Thu, 7 Jan 2010 20:17:17 -0800 (PST) Subject: Ask how to use HTMLParser References: <87vdfdtg3z.fsf@ymail.invalid> Message-ID: <395c651a-65f5-4ceb-a220-58eabc067c71@21g2000yqj.googlegroups.com> On 8 ???, 08:44, Water Lin wrote: > I am a new guy to use Python, but I want to parse a html page now. I > tried to use HTMLParse. Here is my sample code: > ---------------------- > from HTMLParser import HTMLParser > from urllib2 import urlopen > > class MyParser(HTMLParser): > ? ? title = "" > ? ? is_title = "" > ? ? def __init__(self, url): > ? ? ? ? HTMLParser.__init__(self) > ? ? ? ? req = urlopen(url) > ? ? ? ? self.feed(req.read()) > > ? ? def handle_starttag(self, tag, attrs): > ? ? ? ? if tag == 'div' and attrs[0][1] == 'articleTitle': > ? ? ? ? ? ? print "Found link => %s" % attrs[0][1] > ? ? ? ? ? ? self.is_title = 1 > > ? ? def handle_data(self, data): > ? ? ? ? if self.is_title: > ? ? ? ? ? ? print "here" > ? ? ? ? ? ? self.title = data > ? ? ? ? ? ? print self.title > ? ? ? ? ? ? self.is_title = 0 > ----------------------- > > For the tag > ------- >
open article title
> ------- > > I use my code to parse it. I can locate the div tag but I don't know how > to get the text for the tag which is "open article title" in my example. > > How can I get the html content? What's wrong in my handle_data function? > > Thanks > > Water Lin > > -- > Water Lin's notes and pencils:http://en.waterlin.org > Email: Water... at ymail.com Hi. Have you get errors or anything else??? What is wrong?? Vardan. From vardan.pogosyan at gmail.com Thu Jan 7 23:42:58 2010 From: vardan.pogosyan at gmail.com (h0uk) Date: Thu, 7 Jan 2010 20:42:58 -0800 (PST) Subject: Ask how to use HTMLParser References: <87vdfdtg3z.fsf@ymail.invalid> Message-ID: On 8 ???, 08:44, Water Lin wrote: > I am a new guy to use Python, but I want to parse a html page now. I > tried to use HTMLParse. Here is my sample code: > ---------------------- > from HTMLParser import HTMLParser > from urllib2 import urlopen > > class MyParser(HTMLParser): > ? ? title = "" > ? ? is_title = "" > ? ? def __init__(self, url): > ? ? ? ? HTMLParser.__init__(self) > ? ? ? ? req = urlopen(url) > ? ? ? ? self.feed(req.read()) > > ? ? def handle_starttag(self, tag, attrs): > ? ? ? ? if tag == 'div' and attrs[0][1] == 'articleTitle': > ? ? ? ? ? ? print "Found link => %s" % attrs[0][1] > ? ? ? ? ? ? self.is_title = 1 > > ? ? def handle_data(self, data): > ? ? ? ? if self.is_title: > ? ? ? ? ? ? print "here" > ? ? ? ? ? ? self.title = data > ? ? ? ? ? ? print self.title > ? ? ? ? ? ? self.is_title = 0 > ----------------------- > > For the tag > ------- >
open article title
> ------- > > I use my code to parse it. I can locate the div tag but I don't know how > to get the text for the tag which is "open article title" in my example. > > How can I get the html content? What's wrong in my handle_data function? > > Thanks > > Water Lin > > -- > Water Lin's notes and pencils:http://en.waterlin.org > Email: Water... at ymail.com I want to say your code works well From steve at REMOVE-THIS-cybersource.com.au Thu Jan 7 23:56:18 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 08 Jan 2010 04:56:18 GMT Subject: Need help to pass self.count to other classes. References: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> Message-ID: <0356a7d1$0$1336$c3e8da3@news.astraweb.com> On Wed, 06 Jan 2010 08:56:23 -0500, Steve Holden wrote: > This is untested code (some days I don't seem to write any other kind > ...) but it should give you the flavor: > > class kbInterface(object): > def __init__(self): > self.zxc = 0 > def prompt1(self): > self.count += 1 > return "[%d]> " > def prompt2(self): > l = len(str(self.count))+1 > return "%s " % "."*l > def dhook(self, value): > print "[%d out]" % self.count > def ehook(self, type, value, trace): > print "[%d err]\n" % value > > kbi = kbInterface() > sys.ps1 = kbi.prompt1 > sys.ps2 = kbi.prompt2 > sys.displayhook = kbi.dhook > sys.excepthook = kbi.ehook Unfortunately this won't do what you expect, because sys.ps1 and ps2 should be either strings, or objects with a __str__ method. They aren't called to generate the prompt. (After fixing the typo with self.count vs self.zxc) >>> kbi = kbInterface() >>> sys.ps1 = kbi.prompt1 >print "Hello" Hello > -- Steven From pete at shinners.org Fri Jan 8 01:19:44 2010 From: pete at shinners.org (peteshinners) Date: Thu, 7 Jan 2010 22:19:44 -0800 (PST) Subject: Clarifications on compiling for Windows Message-ID: <1f0b409b-f943-4b1a-a806-87b834337367@p8g2000yqb.googlegroups.com> My presentation for Pycon is coming together, but I need to make sure my information about compiling Python and Python extensions for Windows is correct. I'm really only experienced with this on the Linux side of things. First of all, is the Windows FAQ fairly up to date? Should people be referring to section 6 if they are going to build an application with an embedded Python interpreter? http://www.python.org/doc/faq/windows/#how-can-i-embed-python-into-a-windows-application If I understand correctly, compiled extensions for Python on Windows should match the compiler that was used to build the interpreter itself? Is there a list somewhere that shows which version of msvc was used to compile the recent Python binaries? Thank you for feedback. I definitely want to make sure I have this correct before telling anybody else? From WaterLin at ymail.invalid Fri Jan 8 01:44:16 2010 From: WaterLin at ymail.invalid (Water Lin) Date: Fri, 08 Jan 2010 14:44:16 +0800 Subject: Ask how to use HTMLParser References: <87vdfdtg3z.fsf@ymail.invalid> Message-ID: <87k4vtrt8f.fsf@ymail.invalid> h0uk writes: > On 8 ???, 08:44, Water Lin wrote: >> I am a new guy to use Python, but I want to parse a html page now. I >> tried to use HTMLParse. Here is my sample code: >> ---------------------- >> from HTMLParser import HTMLParser >> from urllib2 import urlopen >> >> class MyParser(HTMLParser): >> ? ? title = "" >> ? ? is_title = "" >> ? ? def __init__(self, url): >> ? ? ? ? HTMLParser.__init__(self) >> ? ? ? ? req = urlopen(url) >> ? ? ? ? self.feed(req.read()) >> >> ? ? def handle_starttag(self, tag, attrs): >> ? ? ? ? if tag == 'div' and attrs[0][1] == 'articleTitle': >> ? ? ? ? ? ? print "Found link => %s" % attrs[0][1] >> ? ? ? ? ? ? self.is_title = 1 >> >> ? ? def handle_data(self, data): >> ? ? ? ? if self.is_title: >> ? ? ? ? ? ? print "here" >> ? ? ? ? ? ? self.title = data >> ? ? ? ? ? ? print self.title >> ? ? ? ? ? ? self.is_title = 0 >> ----------------------- >> >> For the tag >> ------- >>
open article title
>> ------- >> >> I use my code to parse it. I can locate the div tag but I don't know how >> to get the text for the tag which is "open article title" in my example. >> >> How can I get the html content? What's wrong in my handle_data function? >> >> Thanks >> >> Water Lin >> >> -- >> Water Lin's notes and pencils:http://en.waterlin.org >> Email: Water... at ymail.com > > I want to say your code works well But in handle_data I can't print self.title. I don't why I can't set the self.title in handle_data. Thanks Water Lin -- Water Lin's notes and pencils: http://en.waterlin.org Email: WaterLin at ymail.com From mensanator at aol.com Fri Jan 8 01:51:26 2010 From: mensanator at aol.com (Mensanator) Date: Thu, 7 Jan 2010 22:51:26 -0800 (PST) Subject: Clarifications on compiling for Windows References: <1f0b409b-f943-4b1a-a806-87b834337367@p8g2000yqb.googlegroups.com> Message-ID: <6170f903-4e83-4716-9b1d-cc40bba31a65@j5g2000yqm.googlegroups.com> On Jan 8, 12:19?am, peteshinners wrote: > My presentation for Pycon is coming together, but I need to make sure > my information about compiling Python and Python extensions for > Windows is correct. I'm really only experienced with this on the Linux > side of things. > > First of all, is the Windows FAQ fairly up to date? Should people be > referring to section 6 if they are going to build an application with > an embedded Python interpreter?http://www.python.org/doc/faq/windows/#how-can-i-embed-python-into-a-... > > If I understand correctly, compiled extensions for Python on Windows > should match the compiler that was used to build the interpreter > itself? Is there a list somewhere that shows which version of msvc was > used to compile the recent Python binaries? > > Thank you for feedback. I definitely want to make sure I have this > correct before telling anybody else? You aren't going to try it? From casevh at gmail.com Fri Jan 8 02:55:32 2010 From: casevh at gmail.com (casevh) Date: Thu, 7 Jan 2010 23:55:32 -0800 (PST) Subject: Caching objects in a C extension Message-ID: <1bd0de37-1a32-44f4-91fa-d3126e3aaeda@u7g2000yqm.googlegroups.com> I'm working with a C extension that needs to rapidly create and delete objects. I came up with an approach to cache objects that are being deleted and resurrect them instead of creating new objects. It appears to work well but I'm afraid I may be missing something (besides heeding the warning in the documentation that _Py_NewReference is for internal interpreter use only). Below is a simplified version of the approach I'm using: MyType_dealloc(MyTypeObject *self) { if(I_want_to_save_MyType(self)) { // Save the object pointer in a cache save_it(self); } else { PyObject_Del(self); } } MyType_new(void) { MyTypeObject *self; if(there_is_an_object_in_the_cache) { self = get_object_from_cache; _Py_NewReference((PyObject*)self); } else { if(!(self = PyObjectNew(MyTypeObject, &MyType)) return NULL; initialize_the_new_object(self); } return self; } The objects referenced in the cache have a reference count of 0 and I don't increment the reference count until I need to resurrect the object. Could these objects be clobbered by the garbage collector? Would it be safer to create the new reference before stuffing the object into the cache (even though it will look like there is a memory leak when running under a debug build)? Thanks in advance for any comments, casevh From vardan.pogosyan at gmail.com Fri Jan 8 03:24:00 2010 From: vardan.pogosyan at gmail.com (h0uk) Date: Fri, 8 Jan 2010 00:24:00 -0800 (PST) Subject: Ask how to use HTMLParser References: <87vdfdtg3z.fsf@ymail.invalid> <87k4vtrt8f.fsf@ymail.invalid> Message-ID: On 8 ???, 11:44, Water Lin wrote: > h0uk writes: > > On 8 ???, 08:44, Water Lin wrote: > >> I am a new guy to use Python, but I want to parse a html page now. I > >> tried to use HTMLParse. Here is my sample code: > >> ---------------------- > >> from HTMLParser import HTMLParser > >> from urllib2 import urlopen > > >> class MyParser(HTMLParser): > >> ? ? title = "" > >> ? ? is_title = "" > >> ? ? def __init__(self, url): > >> ? ? ? ? HTMLParser.__init__(self) > >> ? ? ? ? req = urlopen(url) > >> ? ? ? ? self.feed(req.read()) > > >> ? ? def handle_starttag(self, tag, attrs): > >> ? ? ? ? if tag == 'div' and attrs[0][1] == 'articleTitle': > >> ? ? ? ? ? ? print "Found link => %s" % attrs[0][1] > >> ? ? ? ? ? ? self.is_title = 1 > > >> ? ? def handle_data(self, data): > >> ? ? ? ? if self.is_title: > >> ? ? ? ? ? ? print "here" > >> ? ? ? ? ? ? self.title = data > >> ? ? ? ? ? ? print self.title > >> ? ? ? ? ? ? self.is_title = 0 > >> ----------------------- > > >> For the tag > >> ------- > >>
open article title
> >> ------- > > >> I use my code to parse it. I can locate the div tag but I don't know how > >> to get the text for the tag which is "open article title" in my example. > > >> How can I get the html content? What's wrong in my handle_data function? > > >> Thanks > > >> Water Lin > > >> -- > >> Water Lin's notes and pencils:http://en.waterlin.org > >> Email: Water... at ymail.com > > > I want to say your code works well > > But in handle_data I can't print self.title. I don't why I can't set the > self.title in handle_data. > > Thanks > > Water Lin > > -- > Water Lin's notes and pencils:http://en.waterlin.org > Email: Water... at ymail.com I have tested your code as : #!/usr/bin/env python # -*- conding: utf-8 -*- from HTMLParser import HTMLParser class MyParser(HTMLParser): title = "" is_title = "" def __init__(self, data): HTMLParser.__init__(self) self.feed(data) def handle_starttag(self, tag, attrs): if tag == 'div' and attrs[0][1] == 'articleTitle': print "Found link => %s" % attrs[0][1] self.is_title = 1 def handle_data(self, data): if self.is_title: print "here" self.title = data print self.title self.is_title = 0 if __name__ == "__main__": m = MyParser("""
open article title
Ask how to use HTMLParser
Parametrs
""") All stuff printed and handled fine. Also, the 'print self.title' statement works fine. Try run my code. Vardan. From nobody at nowhere.org Fri Jan 8 03:39:50 2010 From: nobody at nowhere.org (Georg) Date: Fri, 8 Jan 2010 09:39:50 +0100 Subject: C Structure rebuild with ctypes References: <7p6ksnFkg1U1@mid.individual.net> Message-ID: <7qo9avFivmU1@mid.individual.net> Hi Mark, > Are you passing in these values, or are they being returned? To me the > depth of the pointer references implies numVars, varNames, and varTypes > are out parameters. I'll assume that for now. If they are in/out > parameters let me know. If these parameters were in parameters. What would I have to do different? Best regards Georg From victorsubervi at gmail.com Fri Jan 8 04:23:00 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 8 Jan 2010 04:23:00 -0500 Subject: getfirst and re In-Reply-To: <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> Message-ID: <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> On Wed, Jan 6, 2010 at 2:19 PM, Victor Subervi wrote: > On Wed, Jan 6, 2010 at 3:09 PM, Carsten Haese wrote: > >> Victor Subervi wrote: >> > I have an automatically generated HTML form from which I need to extract >> > data to the script which this form calls (to which the information is >> > sent). >> >> Ideally, the script that receives the submitted fields should know how >> the form was generated, so it knows what fields to expect. Failing that, >> you should realize that getfirst() is not the only way to access the >> contents of a cgi.FieldStorage object. >> >> If you need to know the values of all fields whose names obey a certain >> pattern, I recommend you start with a list of all field names in the >> FieldStorage object and pick out the ones whose names obey that pattern. >> To get the list of field names, you should consider using the .keys() >> method of the FieldStorage object. >> > > Code snippet: def cgiFieldStorageToDict(fieldStorage): params = {} for key in fieldStorage.keys(): params[key] = cgi.FieldStorage[key].value return params def display(): top() dict = cgiFieldStorageToDict(cgi.FieldStorage()) print dict Error: /var/www/html/angrynates.com/christians/cart/display.py 163 print 'All products together usually cost: $%f

' % str(fpformat.fix(round(int(allPrices * 100))/100, 2)) 164 cursor.close() 165 bottom() 166 167 display() display = /var/www/html/angrynates.com/christians/cart/display.py in display() 120 def display(): 121 top() 122 dict = cgiFieldStorageToDict(cgi.FieldStorage()) 123 print dict 124 if store == 'prescriptions': builtin dict = , global cgiFieldStorageToDict = , global cgi = , cgi.FieldStorage = /var/www/html/angrynates.com/christians/cart/display.py in cgiFieldStorageToDict(fieldStorage=FieldStorage(None, None, [MiniFieldStorage('stor...products'), MiniFieldStorage('cat', 'prodCat1')])) 115 params = {} 116 for key in fieldStorage.keys(): 117 params[key] = cgi.FieldStorage[key].value 118 return params 119 params = {}, key = 'store', global cgi = , cgi.FieldStorage = , ].value undefined TypeError: unsubscriptable object args = ('unsubscriptable object',) What do? TIA, beno #! /usr/bin/python import cgitb; cgitb.enable() import cgi import MySQLdb import sys,os sys.path.append(os.getcwd()) from login import login from template import top, bottom from sets import Set import fpformat form = cgi.FieldStorage() store = form.getfirst('store') cat = form.getfirst('cat', '') user, passwd, db, host = login() db = MySQLdb.connect(host, user, passwd, db) cursor = db.cursor() def displayProducts(patientID=''): try: # These are stores with categories where ordering by price is important sql = 'select ID from %s where Category="%s" order by Price desc;' % (store, cat) cursor.execute(sql) except: # Stores, like prescriptions, where ordering by price is not important sql = 'select ID from %s;' % (store) cursor.execute(sql) ids = [itm[0] for itm in cursor] cursor.execute('describe %s;' % store) colFields, colFieldValues = [itm[0] for itm in cursor], [itm[1] for itm in cursor] i = 0 if len(ids) > 0: for id in ids: # print '\n' print '
' print "" % store print "" % id j = 0 for col in colFields: sql = 'select %s from %s where ID="%s";' % (col, store, str(id)) cursor.execute(sql) colValue = cursor.fetchone() if col == 'SKU': print "" % colValue[0] print '%s: %s
\n' % (col, colValue[0]) elif col == 'Category': print "" % colValue[0] print '%s: %s
\n' % (col, colValue[0]) elif col == 'OutOfStock': if colValue[0] == '1': # This product is out of stock outOfStockFlag = 'yes' else: outOfStockFlag = 'no' elif col[:3] != 'pic': notSet = 1 if isinstance(colValue[0], (str, int, float, long, complex, unicode, list, buffer, xrange, tuple)): pass else: try: html = "%s:
" print html except: pass if notSet == 1: if len(col) > 49: colX = col[:50] + '...' else: colX = col print '%s: %s
\n' % (colX, colValue[0]) elif col == 'pic1': try: if (colValue[0] != None): # if (colValue[0] != None) and (len(colValue[0] > 0)): print '
\n' % (store, col[3:], id, store, col[3:], id, store, col[3:], id) except TypeError: raise except: raise # i += 1 # try: # content = colValues[0][x].tostring() # pic = "tmp" + str(i) + ".jpg" # try: # os.remove(pic) # except: # pass # img = open(pic, "w") # img.write(content) # print '

' % pic # img.close() # except: # pass j += 1 if store != 'prescriptions': if outOfStockFlag == 'yes': print 'This item is currently out of stock.' else: print "" else: print "" % patientID print "" print '


' print '\n' print '\n' def cgiFieldStorageToDict(fieldStorage): params = {} for key in fieldStorage.keys(): params[key] = cgi.FieldStorage[key].value return params def display(): top() dict = cgiFieldStorageToDict(cgi.FieldStorage()) print dict if store == 'prescriptions': password = form.getfirst('password') email = form.getfirst('email') sql = 'select ID, FirstName, LastName from %s where PW="%s" and Email="%s";' % ('patientsPersonalData', password, email) try: cursor.execute(sql) patientID = [itm[0] for itm in cursor][0] print "

Welcome, %s %s!

" % ([itm[1] for itm in cursor][0], [itm[2] for itm in cursor][0]) displayProducts(patientID) except: print "We're sorry. The email address and password you entered do not correspond with what is recorded in our database. Please click the "back" button and try again.

" else: displayProducts() cursor.execute('show tables like "%sPackages";' % store) if cursor.fetchone() is not None: sql = 'select ID from categories%s where Category="%s";' % (store[0].upper() + store[1:], cat) cursor.execute(sql) categoryID = cursor.fetchone()[0] sql = 'select ID from %sPackages p join %sCategoriesPackages c where c.CategoryID=%s;' % (store, store, categoryID) cursor.execute(sql) packageIDs = [itm[0] for itm in cursor] for pid in packageIDs: print 'XXX' sql = 'select ProductID from %sProductsPackages where PackageID=%s;' % (store, pid) cursor.execute(sql) productIDs = [itm[0] for itm in cursor] sql = 'select Name, Price from %sPackages where ID=%s;' % (store, pid) name, price = cursor.fetchone() print 'Package Name: %s
' % name print 'Price: %s
' % price allPrices = 0.00 for pid in productIDs: sql = 'select Name, Price from %s;' % store # print sql cursor.execute(sql) pName, pPrice = cursor.fetchone() pPrice = float(pPrice) allPrices += pPrice print 'Product Name: %s
' % pName print 'All products together usually cost: $%f

' % str(fpformat.fix(round(int(allPrices * 100))/100, 2)) cursor.close() bottom() display() -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank at chagford.com Fri Jan 8 04:33:42 2010 From: frank at chagford.com (Frank Millman) Date: Fri, 8 Jan 2010 11:33:42 +0200 Subject: multiprocessing and remote objects Message-ID: Hi all I am experimenting with the multiprocessing module, to get a feel of what is possible and what is not. I have got a situation that does not behave as expected. I can work around it, but I would like to find out the underlying reason, to get a more solid understanding. I am trying to create 'remote objects' that can be shared between processes. I can create and return an instance of a class, with no problem. Now I want to add a method to the class which, when called remotely, creates and returns an instance of another class. I cannot get this to work. On the server side I have - tables = {} # a cache of table instances, keyed on name class Table(object): def __init__(self, name): self.name = name def get_table(self, tablename): # if table with this name does not exist, # create it and store it in cache if tablename not in tables: tables[tablename] = Table(tablename) # retrieve table instance from cache and return it return tables[tablename] class MyManager(BaseManager): pass MyManager.register('get_table', get_table, method_to_typeid={'get_table': 'Table') MyManager.register('Table', Table) if __name__ == '__main__': manager = MyManager(address=('127.0.0.1', 50000)) server = manager.get_server() server.serve_forever() On the client side I have - class MyManager(BaseManager): pass MyManager.register('get_table') MyManager.register('Table') if __name__ == '__main__': manager = MyManager(address=('127.0.0.1', 50000)) manager.connect() cust = manager.get_table('Customers') So far so good. Then I extend the server program as follows - class Table(object): def __init__(self, name): self.name = name + self.columns = {} + def add_column(self, colname): + if colname not in self.columns: + self.columns[colname] = Column(colname) + return self.columns[colname] + class Column(object): + def __init__(self, colname): + self.colname = colname MyManager.register('get_table', get_table, method_to_typeid={'get_table': 'Table') - MyManager.register('Table', Table) + MyManager.register('Table', Table, + method_to_typeid={'add_column': 'Column') + MyManager.register('Column', Column) and I extend the client program as follows - MyManager.register('get_table') MyManager.register('Table') + MyManager.register('Column') cust = manager.get_table('Customers') + acno = cust.add_column('Acno') The server appears to create and return the column ok, but on the client side I get the following traceback - Traceback (most recent call last): File "F:\junk\multiprocess\mp13b.py", line 29, in acno = cust.add_column('Acno') File "", line 2, in add_column File "C:\Python26\lib\multiprocessing\managers.py", line 726, in _callmethod kind, result = conn.recv() AttributeError: 'module' object has no attribute 'Column' Does anyone know if it is possible to do what I am attempting, and what the correct approach is? BTW, version is Python 2.6.2. Thanks Frank Millman From taradov at gmail.com Fri Jan 8 05:25:18 2010 From: taradov at gmail.com (alexru) Date: Fri, 8 Jan 2010 02:25:18 -0800 (PST) Subject: Standardized interpreter speed evaluation tool Message-ID: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> Is there any standardized interpreter speed evaluation tool? Say I made few changes in interpreter code and want to know if those changes made python any better, which test should I use? From davea at ieee.org Fri Jan 8 05:34:08 2010 From: davea at ieee.org (Dave Angel) Date: Fri, 08 Jan 2010 05:34:08 -0500 Subject: Ask how to use HTMLParser In-Reply-To: <87k4vtrt8f.fsf@ymail.invalid> References: <87vdfdtg3z.fsf@ymail.invalid> <87k4vtrt8f.fsf@ymail.invalid> Message-ID: <4B470A20.8060303@ieee.org> Water Lin wrote: > h0uk writes: > > >> On 8 ???, 08:44, Water Lin wrote: >> >>> I am a new guy to use Python, but I want to parse a html page now. I >>> tried to use HTMLParse. Here is my sample code: >>> ---------------------- >>> from HTMLParser import HTMLParser >>> from urllib2 import urlopen >>> >>> class MyParser(HTMLParser): >>> title = "" >>> is_title = "" >>> def __init__(self, url): >>> HTMLParser.__init__(self) >>> req = urlopen(url) >>> self.feed(req.read()) >>> >>> def handle_starttag(self, tag, attrs): >>> if tag == 'div' and attrs[0][1] == 'articleTitle': >>> print "Found link => %s" % attrs[0][1] >>> self.is_title = 1 >>> >>> def handle_data(self, data): >>> if self.is_title: >>> print "here" >>> self.title = data >>> print self.title >>> self.is_title = 0 >>> ----------------------- >>> >>> For the tag >>> ------- >>>
open article title
>>> ------- >>> >>> I use my code to parse it. I can locate the div tag but I don't know how >>> to get the text for the tag which is "open article title" in my example. >>> >>> How can I get the html content? What's wrong in my handle_data function? >>> >>> Thanks >>> >>> Water Lin >>> >>> -- >>> Water Lin's notes and pencils:http://en.waterlin.org >>> Email: Water... at ymail.com >>> >> I want to say your code works well >> > > But in handle_data I can't print self.title. I don't why I can't set the > self.title in handle_data. > > Thanks > > Water Lin > > I don't know HTMLParser, but I see a possible confusion point in your class definition. You have both class-attributes and instance-attributes of the same names (title and is_title). So if you have more than one instance of MyParser, then they won't see each other's changes. Normally, I'd move the initialization of such attributes into the __init__() method, so the behavior is clear. When an instance-attribute has the same name as a class-attribute, the instance-attribute takes precedence, and "hides" the class-attribute, for further processing in that same instance. So effectively, the class-attribute acts as a default value. From no.i.dont at want.mail.from.spammers.com Fri Jan 8 05:42:04 2010 From: no.i.dont at want.mail.from.spammers.com (Fencer) Date: Fri, 08 Jan 2010 11:42:04 +0100 Subject: How do I access what's in this module? In-Reply-To: References: <7qnfk0Fr6kU1@mid.individual.net> <29a775c4-223f-4e74-b739-12d633e5739e@a21g2000yqc.googlegroups.com> <7qno2iF1b7U1@mid.individual.net> Message-ID: <7qogftFu1aU1@mid.individual.net> On 2010-01-08 05:01, John Machin wrote: > > Error message should appear after line starting with "File". Above > excerpt taken from google groups; identical to what shows in > http://news.gmane.org/gmane.comp.python.general ... what are you > looking at? > > With Windows XP and Python 2.5.4 I get: > > Traceback (most recent call last): > File "", line 1, in > AttributeError: 'builtin_function_or_method' object has no attribute > 'dump' I'm sorry, I managed to leave out that last line by mistake! My bad. I didn't spot that in my first reply to you because I was under the impression that you hadn't seen the tiniest part of traceback. As you neatly pointed out earlier, it's easy to become confused when communicating. :) > >> It turns out I no longer want to access anything in there but I thank >> you for your information nontheless. > > You're welcome -- the advice on _methods is portable :-) I will look more closely at what other advice you write, I must confess I didn't actually do that because, as I mentioned, I no longer had any interest in accessing the module and I was busy (and still am) with another problem. Thanks John! - Fencer From deets at nospam.web.de Fri Jan 8 05:55:35 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Fri, 08 Jan 2010 11:55:35 +0100 Subject: Standardized interpreter speed evaluation tool In-Reply-To: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> References: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> Message-ID: <7qoh97F2mjU1@mid.uni-berlin.de> alexru schrieb: > Is there any standardized interpreter speed evaluation tool? Say I > made few changes in interpreter code and want to know if those changes > made python any better, which test should I use? Pybench I guess. Diez From deets at nospam.web.de Fri Jan 8 05:59:32 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Fri, 08 Jan 2010 11:59:32 +0100 Subject: Caching objects in a C extension In-Reply-To: <1bd0de37-1a32-44f4-91fa-d3126e3aaeda@u7g2000yqm.googlegroups.com> References: <1bd0de37-1a32-44f4-91fa-d3126e3aaeda@u7g2000yqm.googlegroups.com> Message-ID: <7qohgkF2mjU2@mid.uni-berlin.de> casevh schrieb: > I'm working with a C extension that needs to rapidly create and delete > objects. I came up with an approach to cache objects that are being > deleted and resurrect them instead of creating new objects. It appears > to work well but I'm afraid I may be missing something (besides > heeding the warning in the documentation that _Py_NewReference is for > internal interpreter use only). > > Below is a simplified version of the approach I'm using: > > MyType_dealloc(MyTypeObject *self) > { > if(I_want_to_save_MyType(self)) { > // Save the object pointer in a cache > save_it(self); > } else { > PyObject_Del(self); > } > } > > MyType_new(void) > { > MyTypeObject *self; > if(there_is_an_object_in_the_cache) { > self = get_object_from_cache; > _Py_NewReference((PyObject*)self); > } else { > if(!(self = PyObjectNew(MyTypeObject, &MyType)) > return NULL; > initialize_the_new_object(self); > } > return self; > } > > The objects referenced in the cache have a reference count of 0 and I > don't increment the reference count until I need to resurrect the > object. Could these objects be clobbered by the garbage collector? > Would it be safer to create the new reference before stuffing the > object into the cache (even though it will look like there is a memory > leak when running under a debug build)? Deep out of my guts I'd say keeping a reference, and using you own LRU-scheme would be the safest without residing to use dark magic. Diez From davea at ieee.org Fri Jan 8 06:11:49 2010 From: davea at ieee.org (Dave Angel) Date: Fri, 08 Jan 2010 06:11:49 -0500 Subject: Standardized interpreter speed evaluation tool In-Reply-To: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> References: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> Message-ID: <4B4712F5.70205@ieee.org> alexru wrote: > Is there any standardized interpreter speed evaluation tool? Say I > made few changes in interpreter code and want to know if those changes > made python any better, which test should I use? > > Not trying to be a smart-aleck, but the test you use should reflect your definition of the phrase "any better." For example, suppose you decided that you could speed things up by pre-calculating a few dozen megabytes of data, and including that in the python.dll. This would change the memory footprint of Python, and possibly the interpreter's startup/load time, not just the runtime of whatever loop you are testing. But if I assume you know all that and just want to do timings. There are at least two stdlib functions that can help: import time time.time() and time.clock() will give you a wall-clock floating point number, and you can subtract two of these within the same program to see how long something takes. This approach ignores interpreter startup, and does nothing to compensate for other processes running on the system. But it can be very useful, and easy to run. The resolution on each function varies by OS, so you may have to experiment to see which one gives the most precision. import time start = time.time() dotest() print "Elapsed time", time.time() - start timeit module can be used within code for timing, or it may be used to load and run a test from the command line. In the latter version, it includes the startup time for the interpreter, which might be important. It also can execute the desired code repeatedly, so you can get some form of averaging, or amortizing. *** python -m timeit .... Note that due to system caching, doing multiple runs consecutively may give different results than ones that are separated by other programs running. And of course when you recompile and link., the system buffers will contain an unusual set of data. So there are ways (which I don't recall) of flushing the system buffers to let programs start on equal footing. DaveA From phil at riverbankcomputing.com Fri Jan 8 06:27:38 2010 From: phil at riverbankcomputing.com (Phil Thompson) Date: Fri, 08 Jan 2010 11:27:38 +0000 Subject: PyQt QThreadPool error In-Reply-To: References: <7dc7a9f3-cff9-4911-bfde-6086db471b41@k17g2000yqh.googlegroups.com> <7qmsujFc9tU2@mid.uni-berlin.de> <7c781f4c-45c4-4e15-ac21-c4ca174a9a89@l30g2000yqb.googlegroups.com> Message-ID: <4fe15ddf1fa181a42d42dae8e99a5997@localhost> On Thu, 7 Jan 2010 15:07:10 -0800 (PST), h0uk wrote: ... > Phil you right about app.exec_(). But situation is sligthly different. > > I want to have more than one Job. I add these Jobs into QThreadPool > trough cycle. And I also want these Jobs to run sequentially. > > The following code illustrate what I mean: > > #!/usr/bin/env python > # -*- coding: utf-8 -*- > > import sys > import os > import time > > from PyQt4 import QtCore, QtGui > > class Job(QtCore.QRunnable): > def __init__(self, name): > QtCore.QRunnable.__init__(self) > self._name = name > > def run(self): > time.sleep(3) > print self._name > > > if __name__ == "__main__": > > app = QtGui.QApplication(sys.argv) > > QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) > > for i in range(5): > j = Job("Job-" + str(i)) > j.setAutoDelete(True) > QtCore.QThreadPool.globalInstance().start(j, i) > app.exec_() > > After 5 cycle I get the same error: An unhandled win32 exception > occured in python.exe. > > How I can do it?? To run my Jobs sequentially??? It's a PyQt bug. The workaround is not to use setAutoDelete() and instead keep an explicit reference to your Job instances - for example in a list of jobs. Phil From tiago.b.almeida at gmail.com Fri Jan 8 06:39:06 2010 From: tiago.b.almeida at gmail.com (tiago almeida) Date: Fri, 8 Jan 2010 11:39:06 +0000 Subject: Transforming a List of elements into a List of lists of elements Message-ID: Hello all, I'd like to ask how you'd implement a function *f* that transforms a list of elements into a list of lists of elements by grouping contiguous elements together. Examples: a=[1,2,3,4,5] print( f(a, 2) ) # -> [ [1, 2], [3, 4], [5] ] print( f(a, 3) ) # -> [ [1, 2, 3], [4, 5] ] print( f(a, 1) ) # -> [ [1], [2], [3], [4], [5] ] I have done a small function that does this using a while loop but what I'm looking for is a more *pythonic* way to do this. Maybe some function in some standard Module does this and I don't know? Ps: I don't post the function here because I don't have it with me. Thanks a lot in advance! Tiago Almeida tiago.b.almeida at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From zuo at chopin.edu.pl Fri Jan 8 07:16:34 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Fri, 08 Jan 2010 13:16:34 +0100 Subject: Transforming a List of elements into a List of lists of elements In-Reply-To: References: Message-ID: 08-01-2010 tiago almeida wrote: > Hello all, > > I'd like to ask how you'd implement a function *f* that transforms a > list of elements into a list of lists of elements by grouping > contiguous elements together. > Examples: > > a=[1,2,3,4,5] > print( f(a, 2) ) # -> [ [1, 2], [3, 4], [5] ] > print( f(a, 3) ) # -> [ [1, 2, 3], [4, 5] ] > print( f(a, 1) ) # -> [ [1], [2], [3], [4], [5] ] > > > I have done a small function that does this using a while loop but > what I'm looking for is a more *pythonic* way to do this. Maybe > some function in some standard Module does this and I don't know? [In Python 2.x] >>> s = [1,2,3,4,5] >>> zip(*[iter(s)]*3) # zip truncates result [(1, 2, 3)] >>> zip(*[iter(s)]*2) [(1, 2), (3, 4)] >>> zip(*[iter(s)]*3) # or simpler: zip(s) :-) [(1,), (2,), (3,), (4,), (5,)] >>> map(None, *[iter(s)]*2) # map fills result with None [(1, 2), (3, 4), (5, None)] >>> map(None, *[iter(s)]*3) [(1, 2, 3), (4, 5, None)] Instead of map, you can use izip_longest from itertools module: >>> list(itertools.izip_longest(*[iter(s)]*3)) [(1, 2, 3), (4, 5, None)] >>> list(itertools.izip_longest(*[iter(s)]*2)) [(1, 2), (3, 4), (5, None)] See: http://docs.python.org/library/functions.html#zip http://docs.python.org/library/functions.html#map http://docs.python.org/library/itertools.html#itertools.izip_longest Cheers, *j -- Jan Kaliszewski (zuo) From jeanmichel at sequans.com Fri Jan 8 07:39:05 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Fri, 08 Jan 2010 13:39:05 +0100 Subject: Transforming a List of elements into a List of lists of elements In-Reply-To: References: Message-ID: <4B472769.5000201@sequans.com> tiago almeida wrote: > Hello all, > > I'd like to ask how you'd implement a function /f/ that transforms a > list of elements into a list of lists of elements by grouping > contiguous elements together. > Examples: > > a=[1,2,3,4,5] > print( f(a, 2) ) # -> [ [1, 2], [3, 4], [5] ] > print( f(a, 3) ) # -> [ [1, 2, 3], [4, 5] ] > print( f(a, 1) ) # -> [ [1], [2], [3], [4], [5] ] > > > I have done a small function that does this using a while loop but > what I'm looking for is a more /pythonic/ way to do this. Maybe some > function in some standard Module does this and I don't know? > > Ps: I don't post the function here because I don't have it with me. > > Thanks a lot in advance! > Tiago Almeida > tiago.b.almeida at gmail.com > using list comprehention (http://docs.python.org/tutorial/datastructures.html) # note: poor naming def func(l, slice): r = range(0, len(l), slice) result = [l[sl:sl+slice] for sl in r] print result >>> a=[1,2,3,4,5] >>> func(a,1) ; func(a,2) ; func(a,3) [[1], [2], [3], [4], [5]] [[1, 2], [3, 4], [5]] [[1, 2, 3], [4, 5]] I would add that I personally find list comprehension hard to read, so I'm not sure it is that pythonic. Your old school while loop could be less concise/elegant, but perfectly readable. Jean-Michel From zuo at chopin.edu.pl Fri Jan 8 07:39:45 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Fri, 08 Jan 2010 13:39:45 +0100 Subject: Transforming a List of elements into a List of lists of elements In-Reply-To: References: Message-ID: PS. Sorry, I wrote: > >>> zip(*[iter(s)]*3) # or simpler: zip(s) :-) But should be > >>> zip(*[iter(s)]*1) # or simpler: zip(s) :-) *j From phd at phd.pp.ru Fri Jan 8 07:43:36 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Fri, 8 Jan 2010 15:43:36 +0300 Subject: SQLObject 0.11.3 Message-ID: <20100108124336.GB10206@phd.pp.ru> Hello! I'm pleased to announce version 0.11.3, a minor bugfix release of 0.11 branch of SQLObject. What is SQLObject ================= SQLObject is an object-relational mapper. Your database tables are described as classes, and rows are instances of those classes. SQLObject is meant to be easy to use and quick to get started with. SQLObject supports a number of backends: MySQL, PostgreSQL, SQLite, Firebird, Sybase, MSSQL and MaxDB (also known as SAPDB). Where is SQLObject ================== Site: http://sqlobject.org Development: http://sqlobject.org/devel/ Mailing list: https://lists.sourceforge.net/mailman/listinfo/sqlobject-discuss Archives: http://news.gmane.org/gmane.comp.python.sqlobject Download: http://cheeseshop.python.org/pypi/SQLObject/0.11.3 News and changes: http://sqlobject.org/News.html What's New ========== News since 0.11.2 ----------------- * The cache culling algorithm was enhanced to eliminate memory leaks by removing references to dead objects; tested on a website that runs around 4 million requests a day. * Fixed a bug in col.py and dbconnection.py - if dbEncoding is None suppose it's 'ascii'. * Fixed a bug in FirebirdConnection. For a more complete list, please see the news: http://sqlobject.org/News.html Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From phd at phd.pp.ru Fri Jan 8 07:45:21 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Fri, 8 Jan 2010 15:45:21 +0300 Subject: SQLObject 0.12.1 Message-ID: <20100108124521.GF10206@phd.pp.ru> Hello! I'm pleased to announce version 0.12.1, a bugfix release of branch 0.12 of SQLObject. What is SQLObject ================= SQLObject is an object-relational mapper. Your database tables are described as classes, and rows are instances of those classes. SQLObject is meant to be easy to use and quick to get started with. SQLObject supports a number of backends: MySQL, PostgreSQL, SQLite, Firebird, Sybase, MSSQL and MaxDB (also known as SAPDB). Where is SQLObject ================== Site: http://sqlobject.org Development: http://sqlobject.org/devel/ Mailing list: https://lists.sourceforge.net/mailman/listinfo/sqlobject-discuss Archives: http://news.gmane.org/gmane.comp.python.sqlobject Download: http://cheeseshop.python.org/pypi/SQLObject/0.12.1 News and changes: http://sqlobject.org/News.html What's New ========== News since 0.12.0 ----------------- * The cache culling algorithm was enhanced to eliminate memory leaks by removing references to dead objects; tested on a website that runs around 4 million requests a day. * Fixed a bug in col.py and dbconnection.py - if dbEncoding is None suppose it's 'ascii'. * Fixed three bugs in PostgresConnection. * Fixed a bug in FirebirdConnection. For a more complete list, please see the news: http://sqlobject.org/News.html Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From Erman.CELEN at 3ds.com Fri Jan 8 08:48:00 2010 From: Erman.CELEN at 3ds.com (CELEN Erman) Date: Fri, 8 Jan 2010 08:48:00 -0500 Subject: C Module's '1.#INF' changes to 'inf' at Python Message-ID: Hi All, My problem is that I've noticed a strange behavior in Python while handling FPEs on Windows after switching compilers (msvc8 to msvc9) and I am trying to find out how Python handles INF values to figure out where the problem might be. The problem appeared when we noticed that behavior of Numeric (older version of NumPy) library has changed when dealing with Floating-Point Exceptions. We are building our own slightly modified version of Python in-house (including Numeric/NumPy) and when we switched from the msvc8 to msvc9 compiler, Numeric started to return '-inf' as a result of 'Numeric.log10(0.0)' instead of rasing an OverflowError. I know that Numeric is using umath instead of the math library and since math.log10(0.0) is still raising an Error (which is ValueError in 2.6 and OverflowError in 2.4, but still an error) I thought that it might have something to do with how umath or Numeric handles the input or return values of log functions. Before hunting blindly I wanted to see how Python interprets INF results. I built a simple module with only one function that causes an FPE with log10(0.0) using C's math library and prints the results with printf. Then I imported that module from python, called the function and printed the result in python too to see if there are any differences between two values (which I found out that there are) Here is the that part of the code from my C Python module: static PyObject *badNum(PyObject *self, PyObject *args) { int sts = 0; //Not used, just dummy double a = 0.0; double c = 0.0; if (!PyArg_ParseTuple(args, "i", &sts)) //Dummy check, no use return NULL; disableFPEs(); c = log10(a); //Since FPEs disabled, should return '-1.#INF' enableFPEs(); printf("C-Val: %f\n", c); return Py_BuildValue("d", c); } After I build the module and imported it into Python, I called the function and printed the return value in Python. The output was different in two versions of Python: Output1 (Module was built with msvc8 and run by python24) C-Val: -1.#INF Py-Val: -1.#INF Output2 (Module was built with msvc9 and run by python26) C-Val: -1.#INF Py-Val: -inf I now know that compiler change didn't cause any change in return value of log10(0.0) at C level (both compilations resulted in '-1.#INF') so I am thinking that somehow at somewhere in Python core the value is translated into '-inf' and that might be the reason why Numeric/NumPy is missing the problem and having issues raising OverflowError like it's supposed to. I started debugging Python.exe (with my module loaded) in Visual Studio. While tracing what's happening after the line "return Py_BuildValue("d", c);" I went down to the "floatobject.c " file (which I thought is highly relevant to floating-point value conversions) and in particular "format_float()" function but since I am not that familiar with Python internals, I am having trouble pinpointing the issue and the code where the "conversion" is happening and I can't even be sure if I am looking at the right place. I would appreciate if someone could help me out or at least could point me to the right direction so that I can figure out the rest. Thanks, Best Regards, Ali Erman CELEN Platform Specialists / Porting Dassault Syst?mes I www.3ds.com This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged. If you are not one of the named recipients or have received this email in error, (i) you should not read, disclose, or copy it, (ii) please notify sender of your receipt by reply email and delete this email and all attachments, (iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.For other languages, go to http://www.3ds.com/terms/email-disclaimer. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ashish.vyas at motorola.com Fri Jan 8 08:49:30 2010 From: ashish.vyas at motorola.com (VYAS ASHISH M-NTB837) Date: Fri, 8 Jan 2010 21:49:30 +0800 Subject: dtd validation on python 3.1 on Windows. Message-ID: <7C57DB58C81FB64CA979EC6C4DB73E7A03E709A1@ZMY16EXM67.ds.mot.com> What is the best way to validate xmls against my dtd in python? I dont see minodom doing this. Sax has dtd validation, but does not complain on dtd violations. (it does access the dtd, if it does not find dtd it complains me.) I am using python 3.1 on Win32 machine. I was planning to try lxml2.2.4 but it shows DLL importerror. I posted the problem on this forum a few days back, but there is no reply on that. Any help?! Regards Ashish -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at holdenweb.com Fri Jan 8 08:58:50 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 08 Jan 2010 08:58:50 -0500 Subject: Need help to pass self.count to other classes. In-Reply-To: <0356a7d1$0$1336$c3e8da3@news.astraweb.com> References: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> <0356a7d1$0$1336$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: [... points out my misapprehension ...] > >>>> kbi = kbInterface() >>>> sys.ps1 = kbi.prompt1 > 0xb7cbd52c>>print "Hello" > Hello > 0xb7cbd52c>> > Right, this is expert mode ... regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From grahn+nntp at snipabacken.se Fri Jan 8 09:05:52 2010 From: grahn+nntp at snipabacken.se (Jorgen Grahn) Date: 8 Jan 2010 14:05:52 GMT Subject: How to execute a script from another script and other script does not do busy wait. References: <76bff0b6-70cb-4960-9b4d-3af7a9a1e0e7@z41g2000yqz.googlegroups.com> <342e58d5-ecd5-4cc5-ad4c-cd1ea169f9e0@c34g2000yqn.googlegroups.com> Message-ID: On Thu, 2010-01-07, danmcleran at yahoo.com wrote: > On Jan 7, 9:18?am, Jorgen Grahn wrote: >> On Thu, 2010-01-07, Rajat wrote: >> > I want to run a python script( aka script2) from another python script >> > (aka script1). While script1 executes script2 it waits for script2 to >> > complete and in doing so it also does some other useful work.(does not >> > do a busy wait). >> >> > My intention is to update a third party through script1 that script2 >> > is going to take longer. >> >> I do not understand that sentence. >> What are you trying to do, more exactly? ?The best solution can be >> threads, os.popen, os.system or something different -- depending on >> the details of what you want to do. ... > I personally use subprocess. Once you launch via subprocess you can > wait or not. > > p = subprocess.Popen(...) > p.wait() #or not. > > See subprocess docs. Yes, that was included in "or something different" above. I have never used it myself, since I've needed to be compatible with Python < 2.4. Still, we need to know what he tries to do. /Jorgen -- // Jorgen Grahn O o . From carsten.haese at gmail.com Fri Jan 8 09:11:05 2010 From: carsten.haese at gmail.com (Carsten Haese) Date: Fri, 08 Jan 2010 09:11:05 -0500 Subject: getfirst and re In-Reply-To: <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> Message-ID: Victor Subervi wrote: > Code snippet: > [...] > > Error: > [...] > What do? After eliminating the pieces of your post that have been copied or quoted from elsewhere, I am left with a total of five words of your own to ask this question, which seems to be approximately the same amount of effort you have put into trying to figure out what's causing the error. If you put in a little bit more effort than that, you might figure out yourself what's causing the error, which should then logically lead you to how to fix the error. Good luck. -Carsten From grahn+nntp at snipabacken.se Fri Jan 8 09:21:38 2010 From: grahn+nntp at snipabacken.se (Jorgen Grahn) Date: 8 Jan 2010 14:21:38 GMT Subject: Do I have to use threads? References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> Message-ID: On Wed, 2010-01-06, Gary Herron wrote: > aditya shukla wrote: >> Hello people, >> >> I have 5 directories corresponding 5 different urls .I want to >> download images from those urls and place them in the respective >> directories.I have to extract the contents and download them >> simultaneously.I can extract the contents and do then one by one. My >> questions is for doing it simultaneously do I have to use threads? >> >> Please point me in the right direction. >> >> >> Thanks >> >> Aditya > > You've been given some bad advice here. > > First -- threads are lighter-weight than processes, so threads are > probably *more* efficient. However, with only five thread/processes, > the difference is probably not noticeable. (If the prejudice against > threads comes from concerns over the GIL -- that also is a misplaced > concern in this instance. Since you only have network connection, you > will receive only one packet at a time, so only one thread will be > active at a time. If the extraction process uses a significant enough > amount of CPU time I wonder what that "extraction" would be, by the way. Unless you ask for compression of the HTTP data, the images come as-is on the TCP stream. > so that the extractions are all running at the same > time *AND* if you are running on a machine with separate CPU/cores *AND* > you would like the extractions to be running truly in parallel on those > separate cores, *THEN*, and only then, will processes be more efficient > than threads.) I can't remember what the bad advice was, but here processes versus threads clearly doesn't matter performance-wise. I generally recommend processes, because how they work is well-known and they're not as vulnerable to weird synchronization bugs as threads. > Second, running 5 wgets is equivalent to 5 processes not 5 threads. > > And third -- you don't have to use either threads *or* processes. There > is another possibility which is much more light-weight: asynchronous > I/O, available through the low level select module, or more usefully > via the higher-level asyncore module. Yeah, that would be my first choice too for a problem which isn't clearly CPU-bound. Or my second choice -- the first would be calling on a utility like wget(1). /Jorgen -- // Jorgen Grahn O o . From victorsubervi at gmail.com Fri Jan 8 09:22:35 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 8 Jan 2010 10:22:35 -0400 Subject: getfirst and re In-Reply-To: References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> Message-ID: <4dc0cfea1001080622i39e8e92dpcbfce1cec472c2f8@mail.gmail.com> On Fri, Jan 8, 2010 at 10:11 AM, Carsten Haese wrote: > Victor Subervi wrote: > > Code snippet: > > [...] > > > > Error: > > [...] > > What do? > > After eliminating the pieces of your post that have been copied or > quoted from elsewhere, I am left with a total of five words of your own > to ask this question, which seems to be approximately the same amount of > effort you have put into trying to figure out what's causing the error. > First I get scolded for not including enough information. Now I get scolded for including too much. Seems I can't win. I *am* putting effort into understanding this. I'm sorry I'm not as sharp as you are in these matters. params = {}, key = 'store', global cgi = , cgi.FieldStorage = , ].value undefined TypeError: unsubscriptable object args = ('unsubscriptable object',) Why is the value undefined? What is an unsubscriptable object? TIA, beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From aioe.org at technicalbloke.com Fri Jan 8 09:31:21 2010 From: aioe.org at technicalbloke.com (r0g) Date: Fri, 08 Jan 2010 14:31:21 +0000 Subject: Do I have to use threads? References: <73045cca1001052026n3cc1e121k9875f54791934144@mail.gmail.com> <432961e5-627e-452d-93db-9562951551d2@e27g2000yqd.googlegroups.com> Message-ID: Marco Salden wrote: > On Jan 6, 5:36 am, Philip Semanchuk wrote: >> On Jan 5, 2010, at 11:26 PM, aditya shukla wrote: >> >>> Hello people, >>> I have 5 directories corresponding 5 different urls .I want to >>> download >>> images from those urls and place them in the respective >>> directories.I have >>> to extract the contents and download them simultaneously.I can >>> extract the >>> contents and do then one by one. My questions is for doing it >>> simultaneously >>> do I have to use threads? >> No. You could spawn 5 copies of wget (or curl or a Python program that >> you've written). Whether or not that will perform better or be easier >> to code, debug and maintain depends on the other aspects of your >> program(s). >> >> bye >> Philip > > Yep, the more easier and straightforward the approach, the better: > threads are always (programmers')-error-prone by nature. > But my question would be: does it REALLY need to be simultaneously: > the CPU/OS only has more overhead doing this in parallel with > processess. Measuring sequential processing and then trying to > optimize (e.g. for user response or whatever) would be my prefered way > to go. Less=More. > > regards, > Marco Threads aren't as hard a some people make out although it does depend on the problem. If your processes are effectively independent then threads are probably the right solution. You can turn any function into a thread quite easily, I posted a function for this a while back... http://groups.google.com/group/comp.lang.python/msg/3361a897db3834b4?dmode=source Also it's often a good idea to build in a flag that switches your app from multi threaded to single threaded as it's easier to debug the latter. Roger. From mccolgst at gmail.com Fri Jan 8 09:32:42 2010 From: mccolgst at gmail.com (McColgst) Date: Fri, 8 Jan 2010 06:32:42 -0800 (PST) Subject: PIL show() not working for 2nd pic References: Message-ID: Do you get any errors or warnings? Could we see the code you ran to get this problem? Thanks Sean From steve at holdenweb.com Fri Jan 8 09:34:42 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 08 Jan 2010 09:34:42 -0500 Subject: getfirst and re In-Reply-To: <4dc0cfea1001080622i39e8e92dpcbfce1cec472c2f8@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4dc0cfea1001080622i39e8e92dpcbfce1cec472c2f8@mail.gmail.com> Message-ID: Victor Subervi wrote: > On Fri, Jan 8, 2010 at 10:11 AM, Carsten Haese > wrote: > > Victor Subervi wrote: > > Code snippet: > > [...] > > > > Error: > > [...] > > What do? > > After eliminating the pieces of your post that have been copied or > quoted from elsewhere, I am left with a total of five words of your own > to ask this question, which seems to be approximately the same amount of > effort you have put into trying to figure out what's causing the error. > > > First I get scolded for not including enough information. Now I get > scolded for including too much. Seems I can't win. I *am* putting effort > into understanding this. I'm sorry I'm not as sharp as you are in these > matters. > > params = {}, key = 'store', global cgi = '/usr/lib64/python2.4/cgi.pyc' >>, cgi.FieldStorage = , ].value undefined > > TypeError: unsubscriptable object > args = ('unsubscriptable object',) > > Why is the value undefined? What is an unsubscriptable object? > TIA, > beno > What is "why"? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From grahn+nntp at snipabacken.se Fri Jan 8 09:37:07 2010 From: grahn+nntp at snipabacken.se (Jorgen Grahn) Date: 8 Jan 2010 14:37:07 GMT Subject: Scripting (was Re: Python books, literature etc) References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: On Thu, 2010-01-07, Peter wrote: > [...] depending on your > application domain, I liked: > > 1) Hans Petter Langtangen: Python Scripting for Computational Science > A truly excellent book, not only with respect to Python Scripting , but > also on how to avoid paying license fees by using opensource tools as > an engineer ( plotting, graphing, gui dev etc ). Very good , pratical > introduction to Python with careful and non-trivial examples and exercises. Sounds good. Regarding the book's title: is it just me, or are Python programmers in general put off when people call it "scripting"? I won't attempt a strict definition of the term "scripting language", but it seems like non-programmers use it to mean "less scary than what you might think of as programming", while programmers interpret it as "not useful as a general-purpose language". /Jorgen -- // Jorgen Grahn O o . From steve at holdenweb.com Fri Jan 8 09:40:27 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 08 Jan 2010 09:40:27 -0500 Subject: getfirst and re In-Reply-To: <4dc0cfea1001080622i39e8e92dpcbfce1cec472c2f8@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4dc0cfea1001080622i39e8e92dpcbfce1cec472c2f8@mail.gmail.com> Message-ID: Victor Subervi wrote: [...] > TypeError: unsubscriptable object > args = ('unsubscriptable object',) > > Why is the value undefined? What is an unsubscriptable object? > TIA, > beno > Sorry, that wasn't very helpful. Here's some more advice: Google is you friend. Try Googling for components of the error traceback. It's unlikely you are the first person to experience this problem. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From gagsl-py2 at yahoo.com.ar Fri Jan 8 09:43:33 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 08 Jan 2010 11:43:33 -0300 Subject: One function calling another defined in the same file being exec'd References: <67CD7A9A-2446-4CE3-AAF6-8F737FA5A68D@Comcast.net> Message-ID: En Thu, 07 Jan 2010 19:47:13 -0300, Mitchell L Model escribi?: > def dofile(filename): > ldict = {'result': None} > with open(filename) as file: > exec(file.read(), globals(), ldict) > print('Result for {}: {}'.format(filename, ldict['result'])) > > Next I call dofile() on a slightly more complex file, in which one > function > calls another function defined earlier in the same file. > > ################################ > def fn1(val): > return sum(range(val)) > > def fn2(arg): > return fn1(arg) > > result = fn2(5) > ################################ > > This produces a surprise: > > NameError: global name 'fn1' is not defined Ok - short answer or long answer? Short answer: Emulate how modules work. Make globals() same as locals(). (BTW, are you sure you want the file to run with the *same* globals as the caller? It sees the dofile() function and everything you have defined/imported there...). Simply use: exec(..., ldict, ldict) > [1] How is it that fn2 can be called from the top-level of the script > but fn1 > cannot be called from fn2? Long answer: First, add these lines before result=fn2(5): print("globals=", globals().keys()) print("locals=", locals().keys()) import dis dis.dis(fn2) and you'll get: globals()= dict_keys(['dofile', '__builtins__', '__file__', '__package__', '__name__', '__doc__']) locals()= dict_keys(['result', 'fn1', 'fn2']) So fn1 and fn2 are defined in the *local* namespace (as always happens in Python, unless you use the global statement). Now look at the code of fn2: 6 0 LOAD_GLOBAL 0 (fn1) 3 LOAD_FAST 0 (arg) 6 CALL_FUNCTION 1 9 RETURN_VALUE Again, the compiler knows that fn1 is not local to fn2, so it must be global (because there is no other enclosing scope) and emits a LOAD_GLOBAL instruction. But when the code is executed, 'fn1' is not in the global scope... Solution: make 'fn1' exist in the global scope. Since assignments (implied by the def statement) are always in the local scope, the only alternative is to make both scopes (global and local) the very same one. This shows that the identity "globals() is locals()" is essential for the module system to work. > [2] Is this correct behavior or is there something wrong with Python > here? It's perfectly logical once you get it... :) > [3] How should I write a file to be exec'd that defines several > functions that > call each other, as in the trivial fn1-fn2 example above? Use the same namespace for both locals and globals: exec(file.read(), ldict, ldict) -- Gabriel Genellina From grahn+nntp at snipabacken.se Fri Jan 8 09:52:32 2010 From: grahn+nntp at snipabacken.se (Jorgen Grahn) Date: 8 Jan 2010 14:52:32 GMT Subject: Recommended "new" way for config files References: <4B46075D.8080606@mycircuit.org> Message-ID: On Thu, 2010-01-07, Jean-Michel Pichavant wrote: > Peter wrote: >> Hi >> There seems to be several strategies to enhance the old ini-style >> config files with real python code, for example: ... >> Is there a strategy that should be prefered for new projects ? ... > The .ini file is the simpliest solution, at least from the user point of > view, no need to learn any python syntax. Yeah. Use whatever your users expect, and deal with it. The language you've happened to implement your stuff in should normally be irrelevant to the users. I wouldn't use .ini-style, but that's because I'm a Unix guy and they remind me of brief and painful experiments with Windows 3.1. Just remember to include support for commented-out lines. /Jorgen -- // Jorgen Grahn O o . From carsten.haese at gmail.com Fri Jan 8 09:55:17 2010 From: carsten.haese at gmail.com (Carsten Haese) Date: Fri, 08 Jan 2010 09:55:17 -0500 Subject: getfirst and re In-Reply-To: <4dc0cfea1001080622i39e8e92dpcbfce1cec472c2f8@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4dc0cfea1001080622i39e8e92dpcbfce1cec472c2f8@mail.gmail.com> Message-ID: Victor Subervi wrote: > First I get scolded for not including enough information. Now I get > scolded for including too much. Seems I can't win. I *am* putting effort > into understanding this. I'm sorry I'm not as sharp as you are in these > matters. I didn't scold you for including too much information. I scolded you for not demonstrating any effort on your part to try to figure out the problem. Maybe you did put effort into figuring it out, but you didn't show us any of it. You simply threw your entire code out there and asked an open-ended question that gave us no insight into your thought process. > params = {}, key = 'store', global cgi = '/usr/lib64/python2.4/cgi.pyc' >>, cgi.FieldStorage = , ].value undefined > > TypeError: unsubscriptable object > args = ('unsubscriptable object',) > > Why is the value undefined? What is an unsubscriptable object? At least those are specific questions. "value undefined" means that the cgitb traceback is unable to show a meaningful value for the object known as <>. Maybe you should ask yourself why that is and why you're operating on an object that doesn't have a meaningful value. An unsubscriptable object is an object that can't be subscripted. In other words, it's an object <> for which the expression foo[bar] is invalid and results in exactly the kind of exception you're struggling with right now. -Carsten From aioe.org at technicalbloke.com Fri Jan 8 09:57:20 2010 From: aioe.org at technicalbloke.com (r0g) Date: Fri, 08 Jan 2010 14:57:20 +0000 Subject: Recommended "new" way for config files References: <4B46075D.8080606@mycircuit.org> <4B460CA0.20800@sequans.com> <4B4625BC.6090201@mycircuit.org> Message-ID: Chris Rebert wrote: > On Thu, Jan 7, 2010 at 10:19 AM, Peter wrote: > >>> The .ini file is the simpliest solution, at least from the user point of >>> view, no need to learn any python syntax. >> I am speaking from the point of view of a python programmer, and I find the >> .ini restrictions not necessarily simple, for example when dealing with >> structured data (I suppose it is trivial to specify a dictionnary or a list >> for the purpose of my request) For example, configuration files for the >> logging module get unwieldy when you specify several loggers , handlers, >> formatters etc, because you have to break down structured data ( objects ) >> to name,value pairs. > >> So what is the "worshipped" approach, when you need more than name=value >> pairs ? > > JSON is one option: http://docs.python.org/library/json.html > > Cheers, > Chris > -- > http://blog.rebertia.com Yes, JSON is rapidly becoming a standard for stuff like this as it's widely portable and less bulky than XML. It's the native format for couchdb too which is nice if you want to replicate and share the contents of your documents. Roger. From dreadpiratejeff at gmail.com Fri Jan 8 09:57:56 2010 From: dreadpiratejeff at gmail.com (J) Date: Fri, 8 Jan 2010 09:57:56 -0500 Subject: Scripting (was Re: Python books, literature etc) In-Reply-To: References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: <36dec4ff1001080657m3c7a0de7u46cc576f72a701df@mail.gmail.com> On Fri, Jan 8, 2010 at 09:37, Jorgen Grahn wrote: > Regarding the book's title: is it just me, or are Python programmers > in general put off when people call it "scripting"? > > I won't attempt a strict definition of the term "scripting language", > but it seems like non-programmers use it to mean "less scary than what > you might think of as programming", while programmers interpret it as > "not useful as a general-purpose language". I dunno... I consider it programming when I'm writing bash scripts. Same with running python scripts. My personal take on it, so YMMV, is that scripting is just a synonym for programming an interpreted language, as opposed to programming (common parlance) which is usually meant writing code for a compiled language (C, C++, VB, etc...) Then again, I also tend to use scripting, coding and programming interchangeably too. And sometimes scripting = just writing a quick and dirty program to do a small task, programming = writing something much larger for long term use. Either way, I'm not offended by any of those terms as they all involve programming, regardless of whether or not someone actually calls it programming. For another analogy, what do they call Chinese food in China? Food. Cheers Jeff -- Ted Turner - "Sports is like a war without the killing." - http://www.brainyquote.com/quotes/authors/t/ted_turner.html From aioe.org at technicalbloke.com Fri Jan 8 10:09:07 2010 From: aioe.org at technicalbloke.com (r0g) Date: Fri, 08 Jan 2010 15:09:07 +0000 Subject: Accessing python from a network share in windows 7 References: <08ac1875-8747-4225-bf78-21972e959386@k9g2000vbl.googlegroups.com> Message-ID: aj wrote: > On Jan 7, 3:51 pm, MRAB wrote: >> aj wrote: > It works without any issue on win7 if I copy python to my local drive. > Also, accessing python from the same network share works fine on win > XP. So I am suspecting some security policy of win7 that is causing > problem while accessing it over a network share. You might also want to ask on Microsoft Answers... http://social.answers.microsoft.com/Forums/en-US/categories Good luck, Roger From robert.kern at gmail.com Fri Jan 8 10:36:44 2010 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 08 Jan 2010 09:36:44 -0600 Subject: C Module's '1.#INF' changes to 'inf' at Python In-Reply-To: References: Message-ID: On 2010-01-08 07:48 AM, CELEN Erman wrote: > Hi All, > > My problem is that I?ve noticed a strange behavior in Python while > handling FPEs on Windows after switching compilers (msvc8 to msvc9) and > I am trying to find out how Python handles INF values to figure out > where the problem might be. > > The problem appeared when we noticed that behavior of Numeric (older > version of NumPy) library has changed when dealing with Floating-Point > Exceptions. We are building our own slightly modified version of Python > in-house (including Numeric/NumPy) and when we switched from the msvc8 > to msvc9 compiler, Numeric started to return ?-inf? as a result of > ?Numeric.log10(0.0)? instead of rasing an OverflowError. I know that > Numeric is using umath instead of the math library and since > math.log10(0.0) is still raising an Error (which is ValueError in 2.6 > and OverflowError in 2.4, but still an error) I thought that it might > have something to do with how umath or Numeric handles the input or > return values of log functions. Numeric.log10() will check to see if the errno was set to ERANGE. It does not check if a floating point exception flag was set, which is tricky to do across platforms. The newer numpy can do it because we've finally managed to implement all of that platform-specific code, but the earlier Numeric does not. Presumably, the msvc8 C runtime's implementation of log10() sets errno and the msvc9 runtime's version does not. > Before hunting blindly I wanted to see how Python interprets INF > results. I built a simple module with only one function that causes an > FPE with log10(0.0) using C?s math library and prints the results with > printf. Then I imported that module from python, called the function and > printed the result in python too to see if there are any differences > between two values (which I found out that there are) Python 2.6 changed the string representations of the special floating point values inf and nan. Previously, the string representation was left up to the C runtime, so they differed between platforms. Python 2.6 normalized the string representation across all platforms. The underlying values are the same. What version of Python are you using? -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From mail.to.daniel.platz at googlemail.com Fri Jan 8 10:37:49 2010 From: mail.to.daniel.platz at googlemail.com (Daniel Platz) Date: Fri, 8 Jan 2010 07:37:49 -0800 (PST) Subject: Manipulating pointers in C using ctypes Message-ID: <08bda34e-9bee-44b1-b2de-80d647151786@a15g2000yqm.googlegroups.com> Hello! I have to ask a newbie question about manipulating pointers in C using ctypes. I have a C dll with two functions. The first one creates a pointer and returns it to python. The second one takes a pointer as an argument shows its address and the value at which it is pointing. This I have implemented using the following code ----------- -------- pointers.c ---------------------------- #include #ifdef __cplusplus extern "C" { // only need to export C interface if // used by C++ source code using namespace std; #endif __declspec(dllexport) void* create() { double number = 2.2; double* ptr = &number; printf("Pointer address \t %p \n", ptr); printf("Value at pointer \t %f \n", ptr[0]); return (void*) ptr; } __declspec(dllexport) int show(double* ptr) { printf("Pointer address \t %p \n", ptr); printf("Pointer value \t %f\n", *ptr); *ptr = 2.4; printf("New pointer value \t %f\n", *ptr); return 0; } #ifdef __cplusplus } #endif ------------------------------------------------------------------------ Please note that in the second function, the show function, I want to manipulate the value at which the pointer points. Now, I call this function from python with the following script. --------------------------- pointers.py -------------------------- import ctypes as ct # Load dll pointers = ct.cdll.LoadLibrary('pointers.dll') getattr(pointers, 'create') getattr(pointers, 'show') pointers.create.restype = ct.c_void_p # Create pointer in C ptr = pointers.create() # Show pointer address and value print 'Adress returned to python ' +hex(ptr) pointers.show(ct.c_void_p(ptr)) ------------------------------------------------------------------- Calling this script gives me the following output: Pointer address 0021E508 Value at pointer 2.200000 Adress returned to python 0x21e508 Pointer address 0021E508 Pointer value 0.000000 (2.20000 expected) New pointer value 2.400000 (2.40000 expected) But the script returns also an error. WindowsError: exception: access violation reading 0x40033333 WARNING: Failure executing file: Another thing that I find strange is that the return value of pointers.create is actually an integer instead of an ct.c_void_p object. Moreover, I also tried to directly manipulate the address of the pointer given as an argument to pointers.show. But when it returns to python the pointer points still at the same address as before the function call. Can someone help me with this problem? I would be very glad about an answer. With kind regards, Daniel From chander at otg-nc.com Fri Jan 8 10:39:44 2010 From: chander at otg-nc.com (Chander Ganesan) Date: Fri, 08 Jan 2010 10:39:44 -0500 Subject: Advanced Python Programming - March 8-12, 2010 Message-ID: <4B4751C0.6070705@otg-nc.com> Looking to make the most of Python's advanced features? Then this is the class for you! The Open Technology Group's Advanced Python Programming course focuses on topics of interest to experienced Python programmers, and teaches you how to leverage advanced features and functionality of Python, including: - Python objects, generators, and decorators - Writing and using regular expressions - Using sockets and networking - Improving performance using threads and the multiprocess module - Signal handling - Cooperative multitasking & networking with Twisted - Unit testing with the unittest module - XML document processing - and much more! This 5 day course is taught in a hands-on, instructor-led setting with classes limited to no more than 12 students. Hands on exercises allow students to put what they learn into practice immediately, and receive immediate instructor feedback. All-inclusive pricing includes round-trip airfare, hotel accommodation, and local shuttle services for just $2,595 (the course by itself is only $2,295). For more course details and to enroll, visit our web site at http://www.otg-nc.com/advanced-python-training , reply to this message, or contact us at 877-258-8987. Space is limited, so enroll today! -- Chander Ganesan Open Technology Group, Inc. One Copley Parkway, Suite 210 Morrisville, NC 27560 919-463-0999/877-258-8987 http://www.otg-nc.com From python at mrabarnett.plus.com Fri Jan 8 10:39:55 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 08 Jan 2010 15:39:55 +0000 Subject: getfirst and re In-Reply-To: <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> Message-ID: <4B4751CB.2050907@mrabarnett.plus.com> Victor Subervi wrote: [snip] > > Code snippet: > > def cgiFieldStorageToDict(fieldStorage): > params = {} > for key in fieldStorage.keys(): > params[key] = cgi.FieldStorage[key].value ^^^^^^^^^^^^^^^^^^^^^ This is your problem. > return params > [snip] From deets at nospam.web.de Fri Jan 8 11:24:12 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Fri, 08 Jan 2010 17:24:12 +0100 Subject: restructuredText editor ? In-Reply-To: References: Message-ID: <7qp4hcFsg3U1@mid.uni-berlin.de> Peter schrieb: > What editor do people out there use to edit .rst files for sphinx-python > documentation ? Emacs has a RST-mode (which unfortunately has some runtime-issues, the longer the text & the edit, the slower it gets), and together with flymake & rst2xml, you get nice error-notifications. Diez From python at mrabarnett.plus.com Fri Jan 8 11:24:51 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 08 Jan 2010 16:24:51 +0000 Subject: Scripting (was Re: Python books, literature etc) In-Reply-To: References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: <4B475C53.2040807@mrabarnett.plus.com> Jorgen Grahn wrote: > On Thu, 2010-01-07, Peter wrote: > >> [...] depending on your >> application domain, I liked: >> >> 1) Hans Petter Langtangen: Python Scripting for Computational Science >> A truly excellent book, not only with respect to Python Scripting , but >> also on how to avoid paying license fees by using opensource tools as >> an engineer ( plotting, graphing, gui dev etc ). Very good , pratical >> introduction to Python with careful and non-trivial examples and exercises. > > Sounds good. > > Regarding the book's title: is it just me, or are Python programmers > in general put off when people call it "scripting"? > > I won't attempt a strict definition of the term "scripting language", > but it seems like non-programmers use it to mean "less scary than what > you might think of as programming", while programmers interpret it as > "not useful as a general-purpose language". > I'd probably say that in "scripting", convenience is more important than speed. You don't need to create a project, just put the code into a file and then run it. From casevh at gmail.com Fri Jan 8 11:39:17 2010 From: casevh at gmail.com (casevh) Date: Fri, 8 Jan 2010 08:39:17 -0800 (PST) Subject: Caching objects in a C extension References: <1bd0de37-1a32-44f4-91fa-d3126e3aaeda@u7g2000yqm.googlegroups.com> <7qohgkF2mjU2@mid.uni-berlin.de> Message-ID: On Jan 8, 2:59?am, "Diez B. Roggisch" wrote: > casevh schrieb: > > > > > > > I'm working with a C extension that needs to rapidly create and delete > > objects. I came up with an approach to cache objects that are being > > deleted and resurrect them instead of creating new objects. It appears > > to work well but I'm afraid I may be missing something (besides > > heeding the warning in the documentation that _Py_NewReference is for > > internal interpreter use only). > > > Below is a simplified version of the approach I'm using: > > > MyType_dealloc(MyTypeObject *self) > > { > > ? ? if(I_want_to_save_MyType(self)) { > > ? ? ? ? // Save the object pointer in a cache > > ? ? ? ? save_it(self); > > ? ? } else { > > ? ? ? ? PyObject_Del(self); > > ? ? } > > } > > > MyType_new(void) > > { > > ? ? MyTypeObject *self; > > ? ? if(there_is_an_object_in_the_cache) { > > ? ? ? ? self = get_object_from_cache; > > ? ? ? ? _Py_NewReference((PyObject*)self); > > ? ? } else { > > ? ? ? ? if(!(self = PyObjectNew(MyTypeObject, &MyType)) > > ? ? ? ? ? ? return NULL; > > ? ? ? ? initialize_the_new_object(self); > > ? ? } > > ? ? return self; > > } > > > The objects referenced in the cache have a reference count of 0 and I > > don't increment the reference count until I need to resurrect the > > object. Could these objects be clobbered by the garbage collector? > > Would it be safer to create the new reference before stuffing the > > object into the cache (even though it will look like there is a memory > > leak when running under a debug build)? > > Deep out of my guts I'd say keeping a reference, and using you own > LRU-scheme would be the safest without residing to use dark magic. > > Diez- Hide quoted text - > > - Show quoted text - Thanks for the reply. I realized that I missed one detail. The objects are created by the extension but are deleted by Python. I don't know that an object is no longer needed until its tp_dealloc is called. At that point, its reference count is 0. casevh From tanix at mongo.net Fri Jan 8 11:50:49 2010 From: tanix at mongo.net (tanix) Date: Fri, 08 Jan 2010 16:50:49 GMT Subject: Python Goldmine has been updated: http://preciseinfo.org/Convert/index_Convert_Python.html Message-ID: Python Goldmine collection contains the extensive collection of articles going back several years. It includes thousands of code examples and expert discussions on all major topics. The information is organized by relevant topics, covered by the corresponding chapters. The information was filtered with sophisticated filters and vast majority of artilces with little relevance have been filtered out. If you have any specific requests for some new chapters to be added and it is of interest to others, please post your requests on this thread. If anyone feels he has above average level of competence, or can reccommend someone who posts on this group, you may request to be included in the expert chapters. The Python Goldmine is at: http://preciseinfo.org/Convert/index_Convert_Python.html -- Programmer's Goldmine collections: http://preciseinfo.org Tens of thousands of code examples and expert discussions on C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, organized by major topics of language, tools, methods, techniques. From solipsis at pitrou.net Fri Jan 8 11:56:33 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 8 Jan 2010 16:56:33 +0000 (UTC) Subject: Caching objects in a C extension References: <1bd0de37-1a32-44f4-91fa-d3126e3aaeda@u7g2000yqm.googlegroups.com> <7qohgkF2mjU2@mid.uni-berlin.de> Message-ID: Le Fri, 08 Jan 2010 08:39:17 -0800, casevh a ?crit?: > > Thanks for the reply. I realized that I missed one detail. The objects > are created by the extension but are deleted by Python. I don't know > that an object is no longer needed until its tp_dealloc is called. At > that point, its reference count is 0. tuple objects and others already have such a caching scheme, so you could download the Python source and look at e.g. Objects/tupleobject.c to see how it's done. Regards Antoine. From MLMDev at Comcast.net Fri Jan 8 12:02:30 2010 From: MLMDev at Comcast.net (Mitchell L Model) Date: Fri, 8 Jan 2010 12:02:30 -0500 Subject: One function calling another defined in the same file being exec'd In-Reply-To: References: Message-ID: On Jan 7, 2010, at 10:45 PM, Steven D'Aprano wrote an extensive answer to my questions about one function calling another in the same file being exec'd. His suggestion about printing out locals() and globals() in the various possible places provided the clues to explain what was going on. I would like to summarize what I have learned from this, because although I have known all the relevant pieces for many years I never put them together in a way that explains the odd behavior I observed. Statements that bind new names -- assignment, def, and class -- do so in the local scope. While exec'ing a file the local scope is determined by the arguments passed to exec; in my case, I passed an explicit local scope. It was particularly obtuse of me not to notice the effects of this because I was intentionally using it so that an assignment to 'result' in the exec'd script would enable the exec'ing code to retrieve the value of result. However, although the purity of Python with respect to the binding actions of def and class statements is wonderful and powerful, it is very difficult cognitively to view a def on a page and think "aha! that's just like an assignment of a newly created function to a name", even though that is precisely the documented behavior of def. So mentally I was making an incorrect distinction between what was getting bound locally and what was getting bound globally in the exec'd script. Moreover, the normal behavior of imported code, in which any function in the module can refer to any other function in the module, seduced me into this inappropriate distinction. To my eye I was just defining and using function definitions the way they are in modules. There is a key difference between module import and exec: as Steven pointed out, inside a module locals() is globals(). On further reflection, I will add that what appears to be happening is that during import both the global and local dictionaries are set to a copy of the globals() from the importing scope and that copy becomes the value of the module's __dict__ once import has completed successfully. Top-level statements bind names in locals(), as always, but because locals() and globals() are the same dictionary, they are also binding them in globals(), so that every function defined in the module uses the modified copy of globals -- the value of the module's __dict__ -- as its globals() when it executes. Because exec leaves locals() and globals() distinct, functions defined at the top level of a string being exec'd don't see other assignments and definitions that are also in the string. Another misleading detail is that top-level expressions in the exec can use other top-level names (assigned, def'd, etc.), which they will find in the exec string's local scope, but function bodies do not see the string's local scope. The problem I encountered arises because the function definitions need to access each other through the global scope, not the local scope. In fact, the problem would arise if one of the functions tried to call itself recursively, since its own name would not be in the global scope. So we have a combination of two distinctions: the different ways module import and exec use globals and locals and the difference between top-level statements finding other top-level names in locals but functions looking for them in globals. Sorry for the long post. These distinctions go deep into the semantics of Python namespaces, which though they are lean, pure, and beautiful, have some consequences that can be surprising -- more so the more familiar you are with other languages that do things differently. Oh, and as far as using import instead of exec for my scripts, I don't think that's appropriate, if only because I don't want my application's namespace polluted by what could be many of these pseudo- modules users might load during a session. (Yes, I could remove the name once the import is finished, but importing solely for side- effects rather than to use the imported module is offensive. Well, I would be using one module name -- result -- but that doesn't seem to justify the complexities of setting up the import and accessing the module when exec does in principle just what I need.) Finally, once all of this is really understood, there is a simple way to change an exec string's def's to bind globally instead of locally: simply begin the exec with a global declaration for any function called by one of the others. In my example, adding a "global fn1" at the beginning of the file fixes it so exec works. ################################ global fn1 # enable fn1 to be called from fn2! def fn1(val): return sum(range(val)) def fn2(arg): return fn1(arg) result = fn2(5) ################################ From vardan.pogosyan at gmail.com Fri Jan 8 12:11:10 2010 From: vardan.pogosyan at gmail.com (h0uk) Date: Fri, 8 Jan 2010 09:11:10 -0800 (PST) Subject: PyQt QThreadPool error References: <7dc7a9f3-cff9-4911-bfde-6086db471b41@k17g2000yqh.googlegroups.com> <7qmsujFc9tU2@mid.uni-berlin.de> <7c781f4c-45c4-4e15-ac21-c4ca174a9a89@l30g2000yqb.googlegroups.com> Message-ID: On 8 ???, 16:27, Phil Thompson wrote: > On Thu, 7 Jan 2010 15:07:10 -0800 (PST), h0uk > wrote: > > ... > > > > > Phil you right about app.exec_(). But situation is sligthly different. > > > I want to have more than one Job. I add these Jobs into QThreadPool > > trough cycle. And I also want these Jobs to run ?sequentially. > > > The following code illustrate what I mean: > > > #!/usr/bin/env python > > # -*- coding: utf-8 -*- > > > import sys > > import os > > import time > > > from PyQt4 import QtCore, QtGui > > > class Job(QtCore.QRunnable): > > ? ?def __init__(self, name): > > ? ? ? ? ? ?QtCore.QRunnable.__init__(self) > > ? ? ? ? ? ?self._name = name > > > ? ?def run(self): > > ? ? ? ? ? ?time.sleep(3) > > ? ? ? ? ? ?print self._name > > > if __name__ == "__main__": > > > ? ?app = QtGui.QApplication(sys.argv) > > > ? ?QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) > > > ? ?for i in range(5): > > ? ? ? ? ? ?j = Job("Job-" + str(i)) > > ? ? ? ? ? ?j.setAutoDelete(True) > > ? ? ? ? ? ?QtCore.QThreadPool.globalInstance().start(j, i) > > ? ?app.exec_() > > > After 5 cycle I get the same error: ?An unhandled win32 exception > > occured in python.exe. > > > How I can do it?? To run my Jobs sequentially??? > > It's a PyQt bug. The workaround is not to use setAutoDelete() and instead > keep an explicit reference to your Job instances - for example in a list of > jobs. > > Phil Thanks, Phil. From MLMDev at Comcast.net Fri Jan 8 12:13:34 2010 From: MLMDev at Comcast.net (Mitchell L Model) Date: Fri, 8 Jan 2010 12:13:34 -0500 Subject: One function calling another defined in the same file being exec'd In-Reply-To: References: Message-ID: <41676BD4-F23E-4F02-8845-3E3327644859@Comcast.net> On Jan 8, 2010, at 9:55 AM, "Gabriel Genellina" wrote: > > Ok - short answer or long answer? > > Short answer: Emulate how modules work. Make globals() same as > locals(). (BTW, are you sure you want the file to run with the > *same* globals as the caller? It sees the dofile() function and > everything you have defined/imported there...). Simply use: > exec(..., ldict, ldict) > >> [1] How is it that fn2 can be called from the top-level of the >> script but fn1 >> cannot be called from fn2? > > Long answer: First, add these lines before result=fn2(5): > > print("globals=", globals().keys()) > print("locals=", locals().keys()) > import dis > dis.dis(fn2) > > and you'll get: > > globals()= dict_keys(['dofile', '__builtins__', '__file__', > '__package__', '__name__', '__doc__']) > locals()= dict_keys(['result', 'fn1', 'fn2']) > > So fn1 and fn2 are defined in the *local* namespace (as always > happens in Python, unless you use the global statement). Now look at > the code of fn2: > > 6 0 LOAD_GLOBAL 0 (fn1) > 3 LOAD_FAST 0 (arg) > 6 CALL_FUNCTION 1 > 9 RETURN_VALUE > > Again, the compiler knows that fn1 is not local to fn2, so it must > be global (because there is no other enclosing scope) and emits a > LOAD_GLOBAL instruction. But when the code is executed, 'fn1' is not > in the global scope... > > Solution: make 'fn1' exist in the global scope. Since assignments > (implied by the def statement) are always in the local scope, the > only alternative is to make both scopes (global and local) the very > same one. This is very helpful additional information and clarification! Thanks. > > This shows that the identity "globals() is locals()" is essential > for the module system to work. Yes, though I doubt more than a few Python programmers would guess that identity. > >> [2] Is this correct behavior or is there something wrong with >> Python here? > > It's perfectly logical once you get it... :) I think I'm convinced. > >> [3] How should I write a file to be exec'd that defines several >> functions that >> call each other, as in the trivial fn1-fn2 example above? > > Use the same namespace for both locals and globals: > exec(file.read(), ldict, ldict) > I was going to say that this wouldn't work because the script couldn't use any built-in names, but the way exec works if the value passed for the globals argument doesn't contain an entry for '__builtins__' it adds one. I would have a further problem in that there are some names I want users to be able to use in their scripts, in particular classes that have been imported into the scope of the code doing the exec, but come to think of it I don't want to expose the entire globals() anyway. The solution is do use the same dictionary for both globals and locals, as you suggest, to emulate the behavior of module import, and explicitly add to it the names I want to make available (and since they are primarily classes, there are relatively few of those, as opposed to an API of hundreds of functions). Thanks for the help. From deets at nospam.web.de Fri Jan 8 12:19:21 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Fri, 08 Jan 2010 18:19:21 +0100 Subject: Caching objects in a C extension In-Reply-To: References: <1bd0de37-1a32-44f4-91fa-d3126e3aaeda@u7g2000yqm.googlegroups.com> <7qohgkF2mjU2@mid.uni-berlin.de> Message-ID: <7qp7opFglrU1@mid.uni-berlin.de> casevh schrieb: > On Jan 8, 2:59 am, "Diez B. Roggisch" wrote: >> casevh schrieb: >> >> >> >> >> >>> I'm working with a C extension that needs to rapidly create and delete >>> objects. I came up with an approach to cache objects that are being >>> deleted and resurrect them instead of creating new objects. It appears >>> to work well but I'm afraid I may be missing something (besides >>> heeding the warning in the documentation that _Py_NewReference is for >>> internal interpreter use only). >>> Below is a simplified version of the approach I'm using: >>> MyType_dealloc(MyTypeObject *self) >>> { >>> if(I_want_to_save_MyType(self)) { >>> // Save the object pointer in a cache >>> save_it(self); >>> } else { >>> PyObject_Del(self); >>> } >>> } >>> MyType_new(void) >>> { >>> MyTypeObject *self; >>> if(there_is_an_object_in_the_cache) { >>> self = get_object_from_cache; >>> _Py_NewReference((PyObject*)self); >>> } else { >>> if(!(self = PyObjectNew(MyTypeObject, &MyType)) >>> return NULL; >>> initialize_the_new_object(self); >>> } >>> return self; >>> } >>> The objects referenced in the cache have a reference count of 0 and I >>> don't increment the reference count until I need to resurrect the >>> object. Could these objects be clobbered by the garbage collector? >>> Would it be safer to create the new reference before stuffing the >>> object into the cache (even though it will look like there is a memory >>> leak when running under a debug build)? >> Deep out of my guts I'd say keeping a reference, and using you own >> LRU-scheme would be the safest without residing to use dark magic. >> >> Diez- Hide quoted text - >> >> - Show quoted text - > > Thanks for the reply. I realized that I missed one detail. The objects > are created by the extension but are deleted by Python. I don't know > that an object is no longer needed until its tp_dealloc is called. At > that point, its reference count is 0. I don't fully understand. Whoever creates these objects, you get a reference to them at some point. Then you increment (through the destined Macros) the ref-count. All objects in your pool with refcount 1 are canditates for removal. All you need to do is to keep a kind of timestamp together with them, since when they are released. If that's to old, fully release them. Diez From steve at holdenweb.com Fri Jan 8 12:26:31 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 08 Jan 2010 12:26:31 -0500 Subject: getfirst and re In-Reply-To: <4B4751CB.2050907@mrabarnett.plus.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4B4751CB.2050907@mrabarnett.plus.com> Message-ID: MRAB wrote: > Victor Subervi wrote: > [snip] >> >> Code snippet: >> >> def cgiFieldStorageToDict(fieldStorage): ^^^^^^^^^^^^ Further hint ... >> params = {} >> for key in fieldStorage.keys(): >> params[key] = cgi.FieldStorage[key].value > ^^^^^^^^^^^^^^^^^^^^^ > This is your problem. > >> return params >> > [snip] > regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From casevh at gmail.com Fri Jan 8 12:50:05 2010 From: casevh at gmail.com (casevh) Date: Fri, 8 Jan 2010 09:50:05 -0800 (PST) Subject: Caching objects in a C extension References: <1bd0de37-1a32-44f4-91fa-d3126e3aaeda@u7g2000yqm.googlegroups.com> <7qohgkF2mjU2@mid.uni-berlin.de> <7qp7opFglrU1@mid.uni-berlin.de> Message-ID: <4ff69845-b7c6-4b0c-a0c5-c4154551098c@m25g2000yqc.googlegroups.com> On Jan 8, 9:19?am, "Diez B. Roggisch" wrote: > casevh schrieb: > > > > > > > On Jan 8, 2:59 am, "Diez B. Roggisch" wrote: > >> casevh schrieb: > > >>> I'm working with a C extension that needs to rapidly create and delete > >>> objects. I came up with an approach to cache objects that are being > >>> deleted and resurrect them instead of creating new objects. It appears > >>> to work well but I'm afraid I may be missing something (besides > >>> heeding the warning in the documentation that _Py_NewReference is for > >>> internal interpreter use only). > >>> Below is a simplified version of the approach I'm using: > >>> MyType_dealloc(MyTypeObject *self) > >>> { > >>> ? ? if(I_want_to_save_MyType(self)) { > >>> ? ? ? ? // Save the object pointer in a cache > >>> ? ? ? ? save_it(self); > >>> ? ? } else { > >>> ? ? ? ? PyObject_Del(self); > >>> ? ? } > >>> } > >>> MyType_new(void) > >>> { > >>> ? ? MyTypeObject *self; > >>> ? ? if(there_is_an_object_in_the_cache) { > >>> ? ? ? ? self = get_object_from_cache; > >>> ? ? ? ? _Py_NewReference((PyObject*)self); > >>> ? ? } else { > >>> ? ? ? ? if(!(self = PyObjectNew(MyTypeObject, &MyType)) > >>> ? ? ? ? ? ? return NULL; > >>> ? ? ? ? initialize_the_new_object(self); > >>> ? ? } > >>> ? ? return self; > >>> } > >>> The objects referenced in the cache have a reference count of 0 and I > >>> don't increment the reference count until I need to resurrect the > >>> object. Could these objects be clobbered by the garbage collector? > >>> Would it be safer to create the new reference before stuffing the > >>> object into the cache (even though it will look like there is a memory > >>> leak when running under a debug build)? > >> Deep out of my guts I'd say keeping a reference, and using you own > >> LRU-scheme would be the safest without residing to use dark magic. > > >> Diez- Hide quoted text - > > >> - Show quoted text - > > > Thanks for the reply. I realized that I missed one detail. The objects > > are created by the extension but are deleted by Python. I don't know > > that an object is no longer needed until its tp_dealloc is called. At > > that point, its reference count is 0. > > I don't fully understand. Whoever creates these objects, you get a > reference to them at some point. Then you increment (through the > destined Macros) the ref-count. > > All objects in your pool with refcount 1 are canditates for removal. All > you need to do is to keep a kind of timestamp together with them, since > when they are released. If that's to old, fully release them. > > Diez- Hide quoted text - > > - Show quoted text - These are numeric objects created by gmpy. I'm trying to minimize the overhead for using mpz with small numbers. Objects are created and deleted very often by the interpreter as expressions are evaluated. I don't keep ownership of the objects. casevh From gtu2003 at alice.it Fri Jan 8 12:53:45 2010 From: gtu2003 at alice.it (wiso) Date: Fri, 08 Jan 2010 18:53:45 +0100 Subject: monitor reading file with thread Message-ID: <4b47712a$0$1115$4fafbaef@reader4.news.tin.it> I'm reading and processing a huge file, so during the execution I want to now the state of the processing: how many lines are already processed, and so on. The first approach is: f = open(filename) n = 0 for l in f: if n % 1000 = 0: print "Reading %d lines" %n do_something(l) but I want something diffent. I want to monitor the running of the computing every n seconds. It's the first time I use threading, and I think mine is not the best solution: import threading import time import Queue class Reader(): def __init__(self,filename): self.filename = filename self.lineno = 0 def __iter__(self): f = open(self.filename) for line in f: self.lineno += 1 time.sleep(0.01) # slow down yield line f.close() class Monitor(threading.Thread): def __init__(self,reader,stop_queue,interval=2): threading.Thread.__init__(self) self.interval = interval self.reader = reader self.stop_queue = stop_queue def run(self): while True: try: if self.stop_queue.get(timeout=self.interval) == "stop": break except Queue.Empty: pass print "MONITOR: ", reader.lineno reader = Reader("r1_200910.log") q = Queue.Queue() monitor = Monitor(reader,q) monitor.start() for line in reader: pass # do_somethinghard(line) q.put("stop") monitor.join() It't works, but I don't like how I'm stopping the thread. In general the Monitor class can be more complex, for example a progress bar. Using python 2.6 From marlowequart at hotmail.com Fri Jan 8 12:59:15 2010 From: marlowequart at hotmail.com (marlowe) Date: Fri, 8 Jan 2010 09:59:15 -0800 (PST) Subject: table from csv file Message-ID: <59653a09-c3f7-464f-a36f-475f0d759e46@e27g2000yqd.googlegroups.com> I am trying to create a table in python from a csv file where I input which columns I would like to see, and the table only shows those columns. I have attached an example of the csv file i am using, and some of the code I have written. I am having trouble converting variables between lists, dictionaries and tuples. Is there a name for what I am attempting to do? any help to get me on the right track with this is appreciated. test.csv Date Open High Low Close Volume Adj Close 12/14/09 110.01 110.7 109.5 110.24 16316000 110.24 12/11/09 110.6 110.82 108.72 109.32 28983100 109.32 12/10/09 110.66 111.03 110.04 110.82 20491700 110.82 12/9/09 111.6 112.48 109.38 110.84 37104900 110.84 12/8/09 112.76 113.06 110.21 110.95 37630800 110.95 12/7/09 111.51 114.22 111.44 113.11 42480500 113.11 12/4/09 117.15 117.26 112.41 113.75 79182600 113.75 12/3/09 118.57 119.54 118.03 118.7 28802100 118.7 12/2/09 118.8 119.27 118.3 119.18 30994600 119.18 12/1/09 117.3 117.93 116.78 117.38 27641000 117.38 11/30/09 114.48 115.89 114.27 115.64 16402300 115.64 11/27/09 113.08 115.81 113.02 115.06 21234400 115.06 11/25/09 115.69 116.88 115.53 116.62 24553300 116.62 11/24/09 114.73 114.81 113.97 114.73 22599700 114.73 11/23/09 114.67 115.12 113.99 114.29 24422700 114.29 11/20/09 111.74 112.94 111.54 112.94 17302500 112.94 11/19/09 111.85 112.4 110.76 112.3 21239800 112.3 11/18/09 112.69 113.09 111.8 112.25 22320600 112.25 11/17/09 111.09 111.99 110.9 111.97 19732900 111.97 11/16/09 110.7 112.16 110.65 111.63 25002300 111.63 11/13/09 108.32 109.8 108.14 109.74 17246000 109.74 11/12/09 109.16 109.56 108.12 108.21 17848300 108.21 11/11/09 109.49 109.71 109 109.6 17654100 109.6 11/10/09 108.03 108.78 107.7 108.39 15973300 108.39 11/9/09 108.69 108.75 107.91 108.19 18444800 108.19 11/6/09 107.38 108.04 107.06 107.43 14789000 107.43 11/5/09 106.81 107.2 106.6 106.98 10189000 106.98 11/4/09 107.11 107.68 106.43 107.1 27125500 107.1 table.py; import csv (D, O, H, L, C, V, A) = (11, 'open', 'high', 'low', 'close', 66, 77) d = {'high':H, 'low':L, 'close':C, 'open':O} spacing = '%-*s' w=raw_input('what do you want to see? (use commas to seperate values) ') y=w.lower() x=y.replace(' ','') print x p = x.split(',') #this takes string, converts to list print p num = len(p) format = num*spacing width = 12*num wi = 12 secwidth = width - wi bb = [] i=0 while i < num: #creates new list with variables ll = d[p[i]] bb.insert(i,ll) i+=1 print bb i = 0 while i < num: bb.insert(i*2, 12) #this works on list i+=1 print bb i = 0 while i < num: p.insert(i*2, 12) #this works on list i+=1 q = tuple (p) #takes list, converts to tuple reader = csv.reader(open('/prog/test.csv', "rb")) rownum = 0 for row in reader: if rownum == 0: print '=' * width print format % (q) print '-' * width print '' else: D, O, H, L, C, V, A = row [:7] o = tuple (bb) print format % (o) rownum += 1 From __peter__ at web.de Fri Jan 8 13:10:37 2010 From: __peter__ at web.de (Peter Otten) Date: Fri, 08 Jan 2010 19:10:37 +0100 Subject: Need help to pass self.count to other classes. References: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> <0356a7d1$0$1336$c3e8da3@news.astraweb.com> Message-ID: Steve Holden wrote: > Steven D'Aprano wrote: > [... points out my misapprehension ...] >> >>>>> kbi = kbInterface() >>>>> sys.ps1 = kbi.prompt1 >> > 0xb7cbd52c>>print "Hello" >> Hello >> > 0xb7cbd52c>> >> > Right, this is expert mode ... Here's a way to turn expert mode into something less advanced: >>> import sys >>> def expert_mode(): ... return "for newbies ;) " ... >>> sys.ps1 = expert_mode class Str: ... def __init__(self, f): self.f = f ... def __str__(self): return self.f() ... sys.ps1 = Str(expert_mode) for newbies ;) Peter From mackrackit at gmail.com Fri Jan 8 13:37:41 2010 From: mackrackit at gmail.com (Dave McCormick) Date: Fri, 8 Jan 2010 11:37:41 -0700 Subject: Dynamic text color In-Reply-To: References: Message-ID: <2a36ceaf1001081037n5ed5701ft9104c1396f36977c@mail.gmail.com> On Wed, Jan 6, 2010 at 9:18 AM, John Posner wrote: > On Tue, 05 Jan 2010 16:54:44 -0500, Dave McCormick > wrote: > > But it is not what I am wanting. I first thought to make it look for a >> space but that would not work when a single character like "#" is to be >> colored if there is a "string" of them. Or if all of the characters between >> quotes are to be colored. >> > > Regular expressions are good at handling searches like: > > * all the characters between quotes > * the two-character string "do", but only if it's a complete word > > -John > > -- > http://mail.python.org/mailman/listinfo/python-list > I need another hint... Been doing some reading and playing and it looks like r'\bxxx\b' is what I need. But I can not figure out how to pass a variable between \b___\b If the word in question is between the "\b \b" and in the list then it works like I want it to. The below does not work. greenList_regexp = "|".join(greenList) for matchobj in re.finditer(r'\bgreenList_regexp\b', complete_text): start,end = matchobj.span() Thanks, Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Fri Jan 8 13:43:03 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 8 Jan 2010 14:43:03 -0400 Subject: getfirst and re In-Reply-To: References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4B4751CB.2050907@mrabarnett.plus.com> Message-ID: <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden wrote: > MRAB wrote: > > Victor Subervi wrote: > > [snip] > >> > >> Code snippet: > >> > >> def cgiFieldStorageToDict(fieldStorage): > ^^^^^^^^^^^^ > Further hint ... > > >> params = {} > >> for key in fieldStorage.keys(): > >> params[key] = cgi.FieldStorage[key].value > > ^^^^^^^^^^^^^^^^^^^^^ > > This is your problem. > The problem is that I don't understand this code that I exactly copied from a Web page tutorial. Can you folks point me to tutorials where I can learn to comprehend this code? Specifically, the line in question. How is it that one can code "params[key]" (what does that mean?) and the other side, what does that mean -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinay_sajip at yahoo.co.uk Fri Jan 8 13:50:21 2010 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Fri, 8 Jan 2010 10:50:21 -0800 (PST) Subject: Recommended "new" way for config files References: <4B46075D.8080606@mycircuit.org> <4B460CA0.20800@sequans.com> <4B4625BC.6090201@mycircuit.org> <50697b2c1001071023m28e9a242uef20c6e65414c371@mail.gmail.com> Message-ID: <70ca1982-6f43-41d2-84eb-c1e226463bff@35g2000yqa.googlegroups.com> On Jan 7, 8:12?pm, Peter wrote: > > > > >> So what is the "worshipped" approach, when you need more than name=value > >> pairs ? > > > JSON is one option:http://docs.python.org/library/json.html > > Thanks, didn't think about that, although most of the apps I know don't > seem to use this approach for improved conf file handling ( ipython, > pylons, etc ). > > To me , the ipython way ( hybrid: ipy_user_conf.py and *.ini files ) > seems to be the most comprehensive way amongst the larger apps I know > of, since it let you have a python coded file for what ever you might > want to do during initialization and have additional .ini files, > ,possibily several in different locations, for simpler options in > name,value format. > > Has anybody experiences with other tools that use this approach ? > > Peter I don't know if you will find it relevant to your needs, but PEP 391 suggests an alternative method of configuring logging using dicts. The use of dicts as the basic format (rather than Python code) allows for some end-user flexibility and interoperability with other systems: e.g. JSON, YAML and Python can all produce Python dicts. Regards, Vinay Sajip From jeanmichel at sequans.com Fri Jan 8 13:52:10 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Fri, 08 Jan 2010 19:52:10 +0100 Subject: getfirst and re In-Reply-To: <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4B4751CB.2050907@mrabarnett.plus.com> <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> Message-ID: <4B477EDA.4000105@sequans.com> Victor Subervi wrote: > On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden > wrote: > > MRAB wrote: > > Victor Subervi wrote: > > [snip] > >> > >> Code snippet: > >> > >> def cgiFieldStorageToDict(fieldStorage): > ^^^^^^^^^^^^ > Further hint ... > > >> params = {} > >> for key in fieldStorage.keys(): > >> params[key] = cgi.FieldStorage[key].value > > ^^^^^^^^^^^^^^^^^^^^^ > > This is your problem. > > > The problem is that I don't understand this code that I exactly copied > from a Web page tutorial. Can you folks point me to tutorials where I > can learn to comprehend this code? Specifically, the line in question. > How is it that one can code "params[key]" (what does that mean?) and > the other side, what does that mean I think you are gathering more fans Victor :) http://docs.python.org/tutorial/datastructures.html#dictionaries JM From joncle at googlemail.com Fri Jan 8 13:55:55 2010 From: joncle at googlemail.com (Jon Clements) Date: Fri, 8 Jan 2010 10:55:55 -0800 (PST) Subject: table from csv file References: <59653a09-c3f7-464f-a36f-475f0d759e46@e27g2000yqd.googlegroups.com> Message-ID: <77b18520-c2c4-4e21-9147-1164d81b9f25@l30g2000yqb.googlegroups.com> On Jan 8, 5:59?pm, marlowe wrote: > I am trying to create a table in python from a csv file where I input > which columns I would like to see, and the table only shows those > columns. I have attached an example of the csv file i am using, and > some of the code I have written. I am having trouble converting > variables between lists, dictionaries and tuples. Is there a name for > what I am attempting to do? any help to get me on the right track with > this is appreciated. > > test.csv > Date ? ?Open ? ? High ? ?Low ? ?Close ? Volume ?Adj Close > 12/14/09 ? ? ? ?110.01 ?110.7 ? 109.5 ? 110.24 ?16316000 ? ? ? ?110.24 > 12/11/09 ? ? ? ?110.6 ? 110.82 ?108.72 ?109.32 ?28983100 ? ? ? ?109.32 > 12/10/09 ? ? ? ?110.66 ?111.03 ?110.04 ?110.82 ?20491700 ? ? ? ?110.82 > 12/9/09 111.6 ? 112.48 ?109.38 ?110.84 ?37104900 ? ? ? ?110.84 > 12/8/09 112.76 ?113.06 ?110.21 ?110.95 ?37630800 ? ? ? ?110.95 > 12/7/09 111.51 ?114.22 ?111.44 ?113.11 ?42480500 ? ? ? ?113.11 > 12/4/09 117.15 ?117.26 ?112.41 ?113.75 ?79182600 ? ? ? ?113.75 > 12/3/09 118.57 ?119.54 ?118.03 ?118.7 ? 28802100 ? ? ? ?118.7 > 12/2/09 118.8 ? 119.27 ?118.3 ? 119.18 ?30994600 ? ? ? ?119.18 > 12/1/09 117.3 ? 117.93 ?116.78 ?117.38 ?27641000 ? ? ? ?117.38 > 11/30/09 ? ? ? ?114.48 ?115.89 ?114.27 ?115.64 ?16402300 ? ? ? ?115.64 > 11/27/09 ? ? ? ?113.08 ?115.81 ?113.02 ?115.06 ?21234400 ? ? ? ?115.06 > 11/25/09 ? ? ? ?115.69 ?116.88 ?115.53 ?116.62 ?24553300 ? ? ? ?116.62 > 11/24/09 ? ? ? ?114.73 ?114.81 ?113.97 ?114.73 ?22599700 ? ? ? ?114.73 > 11/23/09 ? ? ? ?114.67 ?115.12 ?113.99 ?114.29 ?24422700 ? ? ? ?114.29 > 11/20/09 ? ? ? ?111.74 ?112.94 ?111.54 ?112.94 ?17302500 ? ? ? ?112.94 > 11/19/09 ? ? ? ?111.85 ?112.4 ? 110.76 ?112.3 ? 21239800 ? ? ? ?112.3 > 11/18/09 ? ? ? ?112.69 ?113.09 ?111.8 ? 112.25 ?22320600 ? ? ? ?112.25 > 11/17/09 ? ? ? ?111.09 ?111.99 ?110.9 ? 111.97 ?19732900 ? ? ? ?111.97 > 11/16/09 ? ? ? ?110.7 ? 112.16 ?110.65 ?111.63 ?25002300 ? ? ? ?111.63 > 11/13/09 ? ? ? ?108.32 ?109.8 ? 108.14 ?109.74 ?17246000 ? ? ? ?109.74 > 11/12/09 ? ? ? ?109.16 ?109.56 ?108.12 ?108.21 ?17848300 ? ? ? ?108.21 > 11/11/09 ? ? ? ?109.49 ?109.71 ?109 ? ? 109.6 ? 17654100 ? ? ? ?109.6 > 11/10/09 ? ? ? ?108.03 ?108.78 ?107.7 ? 108.39 ?15973300 ? ? ? ?108.39 > 11/9/09 108.69 ?108.75 ?107.91 ?108.19 ?18444800 ? ? ? ?108.19 > 11/6/09 107.38 ?108.04 ?107.06 ?107.43 ?14789000 ? ? ? ?107.43 > 11/5/09 106.81 ?107.2 ? 106.6 ? 106.98 ?10189000 ? ? ? ?106.98 > 11/4/09 107.11 ?107.68 ?106.43 ?107.1 ? 27125500 ? ? ? ?107.1 > > table.py; > > import csv > > (D, O, H, L, C, V, A) = (11, 'open', 'high', 'low', 'close', 66, 77) > > d = {'high':H, 'low':L, 'close':C, 'open':O} > > spacing = '%-*s' > > w=raw_input('what do you want to see? (use commas to seperate values) > ') > y=w.lower() > x=y.replace(' ','') > print x > > p = x.split(',') ? ? ? ? ? ? ? ? ? ?#this takes string, converts to > list > print p > num = len(p) > format = num*spacing > width = 12*num > wi = 12 > secwidth = width - wi > > bb = [] > i=0 > while i < num: ? ? ? ? ? ? ? ? ? ? ?#creates new list with variables > ? ? ll = d[p[i]] > ? ? bb.insert(i,ll) > ? ? i+=1 > print bb > i = 0 > while i < num: > ? ? bb.insert(i*2, 12) ? ? ? ? ? ? ? #this works on list > ? ? i+=1 > print bb > > i = 0 > while i < num: > ? ? p.insert(i*2, 12) ? ? ? ? ? ? ? #this works on list > ? ? i+=1 > > q = tuple (p) ? ? ? ? ? ? ? ? ? ? ? #takes list, converts to tuple > > reader = csv.reader(open('/prog/test.csv', "rb")) > > rownum = 0 > for row in reader: > ? ? if rownum == 0: > ? ? ? ? print '=' * width > ? ? ? ? print format % (q) > ? ? ? ? print '-' * width > ? ? ? ? print '' > > ? ? else: > ? ? ? ? D, O, H, L, C, V, A = row [:7] > ? ? ? ? o = tuple (bb) > ? ? ? ? print format % (o) > > ? ? rownum += 1 This might be a useful starting point (I'm guessing this is what you're after...) Let's assume your 'CSV' file is tab separated as it's certainly not comma separated :) import csv csvin = csv.reader(open('test.csv'), delimiter='\t') header = dict( (val.strip(),idx) for idx, val in enumerate(next (csvin)) ) We can use header as a column name->column index lookup eg header ['Open'] == 1 from operator import itemgetter wanted = ['Open', 'Close'] # Although you'll want to use raw_input and split on ',' getcols = itemgetter(*[header[col] for col in wanted]) getcols is a helper function that'll return a tuple of the columns in the requested order... for row in csvin: print getcols(row) Loop over the rest of the file and output the required columns. hth Jon. From victorsubervi at gmail.com Fri Jan 8 14:02:41 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 8 Jan 2010 15:02:41 -0400 Subject: getfirst and re In-Reply-To: <4B477EDA.4000105@sequans.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4B4751CB.2050907@mrabarnett.plus.com> <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> <4B477EDA.4000105@sequans.com> Message-ID: <4dc0cfea1001081102h15a16dbm7e9da66aa0e322@mail.gmail.com> On Fri, Jan 8, 2010 at 2:52 PM, Jean-Michel Pichavant < jeanmichel at sequans.com> wrote: > Victor Subervi wrote: > > On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden > steve at holdenweb.com>> wrote: >> >> MRAB wrote: >> > Victor Subervi wrote: >> > [snip] >> >> >> >> Code snippet: >> >> >> >> def cgiFieldStorageToDict(fieldStorage): >> ^^^^^^^^^^^^ >> Further hint ... >> >> >> params = {} >> >> for key in fieldStorage.keys(): >> >> params[key] = cgi.FieldStorage[key].value >> > ^^^^^^^^^^^^^^^^^^^^^ >> > This is your problem. >> >> >> The problem is that I don't understand this code that I exactly copied >> from a Web page tutorial. Can you folks point me to tutorials where I can >> learn to comprehend this code? Specifically, the line in question. How is it >> that one can code "params[key]" (what does that mean?) and the other side, >> what does that mean >> > I think you are gathering more fans Victor :) > Fans? You aught to hear me sing. I didn't think I'd gain fans here LOL! > > http://docs.python.org/tutorial/datastructures.html#dictionaries > Thanks, beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From jason.scheirer at gmail.com Fri Jan 8 14:02:50 2010 From: jason.scheirer at gmail.com (Jason Scheirer) Date: Fri, 8 Jan 2010 11:02:50 -0800 (PST) Subject: Manipulating pointers in C using ctypes References: <08bda34e-9bee-44b1-b2de-80d647151786@a15g2000yqm.googlegroups.com> Message-ID: <035f8cc8-c19c-4c91-8bd0-db9d74efe2ef@q4g2000yqm.googlegroups.com> On Jan 8, 7:37?am, Daniel Platz wrote: > Hello! > > I have to ask a newbie question about manipulating pointers in C using > ctypes. I have a C dll with two functions. The first one creates a > pointer and returns it to python. The second one takes a pointer as an > argument shows its address and the value at which it is pointing. This > I have implemented using the following code > > ----------- -------- pointers.c ---------------------------- > #include > #ifdef __cplusplus > extern "C" { ?// only need to export C interface if > ? ? ? ? ? ? ? // used by C++ source code > using namespace std; > #endif > > __declspec(dllexport) void* create() > { > ? ? ? ? double number = 2.2; > ? ? ? ? double* ptr = &number; > ? ? ? ? printf("Pointer address \t %p \n", ptr); > ? ? ? ? printf("Value at pointer \t %f \n", ptr[0]); > ? ? ? ? return (void*) ptr; > > } > > __declspec(dllexport) int show(double* ptr) > { > ? ? ? ? printf("Pointer address \t %p \n", ptr); > ? ? ? ? printf("Pointer value \t %f\n", *ptr); > ? ? ? ? *ptr = 2.4; > ? ? ? ? printf("New pointer value \t %f\n", *ptr); > ? ? ? ? return 0; > > } > > #ifdef __cplusplus} > > #endif > ------------------------------------------------------------------------ > > Please note that in the second function, the show function, I want to > manipulate the value at which the pointer points. > Now, I call this function from python with the following script. > > --------------------------- pointers.py -------------------------- > import ctypes as ct > > # Load dll > pointers = ct.cdll.LoadLibrary('pointers.dll') > getattr(pointers, 'create') > getattr(pointers, 'show') > pointers.create.restype = ct.c_void_p > > # Create pointer in C > ptr = pointers.create() > > # Show pointer address and value > print 'Adress returned to python ' +hex(ptr) > pointers.show(ct.c_void_p(ptr)) > ------------------------------------------------------------------- > > Calling this script gives me the following output: > > Pointer address ? ? ? ? ?0021E508 > Value at pointer ? ? ? ? 2.200000 > Adress returned to python 0x21e508 > Pointer address ? ? ? ? ?0021E508 > Pointer value ? ?0.000000 ? ?(2.20000 expected) > New pointer value ? ? ? ?2.400000 (2.40000 expected) > > But the script returns also an error. > > WindowsError: exception: access violation reading 0x40033333 > WARNING: Failure executing file: > > Another thing that I find strange is that the return value of > pointers.create is actually an integer instead of an ct.c_void_p > object. > > Moreover, I also tried to directly manipulate the address of the > pointer given as an argument to pointers.show. But when it returns to > python the pointer points still at the same address as before the > function call. > > Can someone help me with this problem? I would be very glad about an > answer. > > With kind regards, > > Daniel try this: __declspec(dllexport) void* create() { double* ptr = new double; *ptr = 2.2; printf("Pointer address \t %p \n", ptr); printf("Value at pointer \t %f \n", ptr[0]); return (void*) ptr; } Basically once you leave create() the address being used for number is no longer valid. The other implication for this is you are going to need a delete() function to free that memory you allocated for the double as well. From victorsubervi at gmail.com Fri Jan 8 14:09:04 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 8 Jan 2010 15:09:04 -0400 Subject: getfirst and re In-Reply-To: <4B477EDA.4000105@sequans.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4B4751CB.2050907@mrabarnett.plus.com> <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> <4B477EDA.4000105@sequans.com> Message-ID: <4dc0cfea1001081109n384ed968sf955e02c8971cf7@mail.gmail.com> On Fri, Jan 8, 2010 at 2:52 PM, Jean-Michel Pichavant < jeanmichel at sequans.com> wrote: > Victor Subervi wrote: > > On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden > steve at holdenweb.com>> wrote: >> >> MRAB wrote: >> > Victor Subervi wrote: >> > [snip] >> >> >> >> Code snippet: >> >> >> >> def cgiFieldStorageToDict(fieldStorage): >> ^^^^^^^^^^^^ >> Further hint ... >> >> >> params = {} >> >> for key in fieldStorage.keys(): >> >> params[key] = cgi.FieldStorage[key].value >> > ^^^^^^^^^^^^^^^^^^^^^ >> > This is your problem. >> >> >> The problem is that I don't understand this code that I exactly copied >> from a Web page tutorial. Can you folks point me to tutorials where I can >> learn to comprehend this code? Specifically, the line in question. How is it >> that one can code "params[key]" (what does that mean?) and the other side, >> what does that mean >> > I think you are gathering more fans Victor :) > > http://docs.python.org/tutorial/datastructures.html#dictionaries > This still isn't telling me what I need to know. Perhaps I'm missing the point, as in the recent case with putting the "print cookie" statement in the header. I am assuming (translation: making an a$$ out of you and me) that "params[key] automatically assigns the fieldStorage key to the newly created params key in the dict of the same, and assigning the value from the cgi.FieldStorage of that key to the params value. Is that correct? TIA, beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From adityashukla1983 at gmail.com Fri Jan 8 14:14:07 2010 From: adityashukla1983 at gmail.com (aditya shukla) Date: Fri, 8 Jan 2010 13:14:07 -0600 Subject: Mencoder not working from a python script Message-ID: <73045cca1001081114p1be22328k538a4512d7bc8cc0@mail.gmail.com> Hello guys, I am trying to create a python script which uses mencoder to create videos from a set of images.When I am using the command(windows 7) i am able to generate the video properly.But when used within a python script i am ggetting error.Below is the code and error.any help is appreciated. import *subprocess*,*sys*,os def *createVideo*(): mencoder = *"C:\\MPlayer-p4-svn-30075\\mencoder.exe"* path=*"mf://\"C:\\videos\\test\\*.jpg\"" * output=*"\"C:\\videos\\test\\xyz.avi\""* commande = *"%s %s -mf w=800:h=600:fps=25:type=jpg -ovc copy -oac copy -o %s"* os.system(commande % (mencoder, path,output)) if __name__==*"__main__"*: createVideo() --------------------------------error----------------- MEncoder Sherpya-SVN-r30075-4.2.5 (C) 2000-2009 MPlayer Team success: format: 16 data: 0x0 - 0x0 MF file format detected. [mf] search expr: C:\cameras\Perkins Road\*.jpg [mf] number of files: 301 (1204) VIDEO: [IJPG] 800x600 24bpp 25.000 fps 0.0 kbps ( 0.0 kbyte/s) [V] filefmt:16 fourcc:0x47504A49 size:800x600 fps:25.000 ftime:=0.0400 videocodec: framecopy (800x600 24bpp fourcc=47504a49) Writing index... Writing header... ODML: Aspect information not (yet?) available or unspecified, not writing vprp header. Video stream: nan kbit/s (-2147483648 B/s) size: 0 bytes 0.000 secs 0 frames Thanks Aditya -------------- next part -------------- An HTML attachment was scrubbed... URL: From fetchinson at googlemail.com Fri Jan 8 14:14:51 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Fri, 8 Jan 2010 20:14:51 +0100 Subject: lightweight encryption of text file Message-ID: I have a plain text file which I would like to protect in a very simple minded, yet for my purposes sufficient, way. I'd like to encrypt/convert it into a binary file in such a way that possession of a password allows anyone to convert it back into the original text file while not possessing the password one would only see the following with the standard linux utility 'file': [fetchinson at fetch ~]$ file encrypted.data encrypted.data: data and the effort required to convert the file back to the original text file without the password would be equivalent to guessing the password. I'm fully aware of the security implications of this loose specification, but for my purposes this would be a good solution. What would be the simplest way to achieve this using preferably stock python without 3rd party modules? If a not too complex 3rd party module made it really simple that would be acceptable too. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From victorsubervi at gmail.com Fri Jan 8 14:15:51 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 8 Jan 2010 15:15:51 -0400 Subject: getfirst and re In-Reply-To: <4dc0cfea1001081109n384ed968sf955e02c8971cf7@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4B4751CB.2050907@mrabarnett.plus.com> <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> <4B477EDA.4000105@sequans.com> <4dc0cfea1001081109n384ed968sf955e02c8971cf7@mail.gmail.com> Message-ID: <4dc0cfea1001081115kffe51cdpb3b31be683822db9@mail.gmail.com> On Fri, Jan 8, 2010 at 3:09 PM, Victor Subervi wrote: > > > On Fri, Jan 8, 2010 at 2:52 PM, Jean-Michel Pichavant < > jeanmichel at sequans.com> wrote: > >> Victor Subervi wrote: >> >> On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden >> steve at holdenweb.com>> wrote: >>> >>> MRAB wrote: >>> > Victor Subervi wrote: >>> > [snip] >>> >> >>> >> Code snippet: >>> >> >>> >> def cgiFieldStorageToDict(fieldStorage): >>> ^^^^^^^^^^^^ >>> Further hint ... >>> >>> >> params = {} >>> >> for key in fieldStorage.keys(): >>> >> params[key] = cgi.FieldStorage[key].value >>> > ^^^^^^^^^^^^^^^^^^^^^ >>> > This is your problem. >>> >>> >>> The problem is that I don't understand this code that I exactly copied >>> from a Web page tutorial. Can you folks point me to tutorials where I can >>> learn to comprehend this code? Specifically, the line in question. How is it >>> that one can code "params[key]" (what does that mean?) and the other side, >>> what does that mean >>> >> I think you are gathering more fans Victor :) >> >> http://docs.python.org/tutorial/datastructures.html#dictionaries >> > > This still isn't telling me what I need to know. Perhaps I'm missing the > point, as in the recent case with putting the "print cookie" statement in > the header. I am assuming (translation: making an a$$ out of you and me) > that "params[key] automatically assigns the fieldStorage key to the newly > created params key in the dict of the same, and assigning the value from the > cgi.FieldStorage of that key to the params value. Is that correct? > TIA, > beno > I may have answered my own question. I have this: def cgiFieldStorageToDict(fieldStorage): params = {} for key in fieldStorage.keys(): params[key] = fieldStorage[key].value return params dict = cgiFieldStorageToDict(cgi.FieldStorage()) print dict which gave me this: {'store': 'products', 'cat': 'prodCat1'} which looks about right. I would have written the code like this: keys = [] values = [] for key, value in fieldStorage.iteritems(): keys.append(key) values.append(value) params = dict(zip(keys, values)) which obviously isn't as elegant. But that's what I knew. Learned another trick, I guess ;) Thanks all. beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Fri Jan 8 14:21:42 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 08 Jan 2010 19:21:42 +0000 Subject: getfirst and re In-Reply-To: <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4B4751CB.2050907@mrabarnett.plus.com> <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> Message-ID: <4B4785C6.4010801@mrabarnett.plus.com> Victor Subervi wrote: > On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden > wrote: > > MRAB wrote: > > Victor Subervi wrote: > > [snip] > >> > >> Code snippet: > >> > >> def cgiFieldStorageToDict(fieldStorage): > ^^^^^^^^^^^^ > Further hint ... > > >> params = {} > >> for key in fieldStorage.keys(): > >> params[key] = cgi.FieldStorage[key].value > > ^^^^^^^^^^^^^^^^^^^^^ > > This is your problem. > > > The problem is that I don't understand this code that I exactly copied > from a Web page tutorial. Can you folks point me to tutorials where I > can learn to comprehend this code? Specifically, the line in question. > How is it that one can code "params[key]" (what does that mean?) and the > other side, what does that mean > If you got it from: Recipe 81547: Using a simple dictionary for CGI parameters http://code.activestate.com/recipes/81547/ then no, it wasn't "exactly copied". If you'd copy-and-pasted it then it would've been correct, as well as much quicker to do... From martinskou2 at gmail.com Fri Jan 8 14:26:25 2010 From: martinskou2 at gmail.com (msj@infoserv.dk) Date: Fri, 8 Jan 2010 11:26:25 -0800 (PST) Subject: Introspection References: <0d782b7a-11fb-4933-84b7-29a3653ec847@p8g2000yqb.googlegroups.com> <0354a972$0$1336$c3e8da3@news.astraweb.com> <9af6ffd9-a450-4eef-90ed-430aeea9f77c@j24g2000yqa.googlegroups.com> Message-ID: <1a190037-2326-4bdf-97fe-d0364e64d3e8@r5g2000yqb.googlegroups.com> Thanks Miki and Jason. I knew it could be done :-) From vmail at mycircuit.org Fri Jan 8 14:27:16 2010 From: vmail at mycircuit.org (Peter) Date: Fri, 08 Jan 2010 20:27:16 +0100 Subject: Scripting (was Re: Python books, literature etc) In-Reply-To: References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: <4B478714.9040104@mycircuit.org> > Sounds good. > > Regarding the book's title: is it just me, or are Python programmers > in general put off when people call it "scripting"? > > I won't attempt a strict definition of the term "scripting language", > but it seems like non-programmers use it to mean "less scary than what > you might think of as programming", while programmers interpret it as > "not useful as a general-purpose language". > > It took me a while to take "scripting" seriously. I grew up with Pascal and Eiffel and I found it difficult to appreciate dynamic typing and scripting. The author Langtangen is explaining in detail why he considers scripting useful, in particular he provides an automatic test suite to run different language versions ( perl, python, c, c++) of the same program to compare performance. The results are amazing, in that some of the examples run faster than the C++ version. I find Python extremly useful as a general purpose language ( its clearly now my prefered one ) and I find it equally useful to develop toy apps in C++, Haskell and Lisp, just to better appreciate the idea of "general purpose". For me, it has turned out that the point is not "scripting versus not scripting" or "static versus dynamic typing" but having automatic unittests or not having automatic unittests. My most important module is "nose" for running unittests the easy way. Peter From python at mrabarnett.plus.com Fri Jan 8 14:28:57 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 08 Jan 2010 19:28:57 +0000 Subject: Dynamic text color In-Reply-To: <2a36ceaf1001081037n5ed5701ft9104c1396f36977c@mail.gmail.com> References: <2a36ceaf1001081037n5ed5701ft9104c1396f36977c@mail.gmail.com> Message-ID: <4B478779.2080908@mrabarnett.plus.com> Dave McCormick wrote: > > > On Wed, Jan 6, 2010 at 9:18 AM, John Posner > wrote: > > On Tue, 05 Jan 2010 16:54:44 -0500, Dave McCormick > > wrote: > > But it is not what I am wanting. I first thought to make it look > for a space but that would not work when a single character like > "#" is to be colored if there is a "string" of them. Or if all > of the characters between quotes are to be colored. > > > Regular expressions are good at handling searches like: > > * all the characters between quotes > * the two-character string "do", but only if it's a complete word > > -John > > -- > http://mail.python.org/mailman/listinfo/python-list > > I need another hint... > > Been doing some reading and playing and it looks like > r'\bxxx\b' > is what I need. But I can not figure out how to pass a variable between > \b___\b > If the word in question is between the "\b \b" and in the list then it > works like I want it to. > The below does not work. > > greenList_regexp = "|".join(greenList) > for matchobj in re.finditer(r'\bgreenList_regexp\b', complete_text): > start,end = matchobj.span() > The regex r'\bgreenList_regexp\b' will match the string 'greenList_regexp' if it's a whole word. What you mean is "any of these words, provided that they're whole words". You'll need to group the alternatives within "(?:...)", like this: r'\b(?:' + greenList_regexp + ')\b' From vmail at mycircuit.org Fri Jan 8 14:29:35 2010 From: vmail at mycircuit.org (Peter) Date: Fri, 08 Jan 2010 20:29:35 +0100 Subject: Recommended "new" way for config files In-Reply-To: References: <4B46075D.8080606@mycircuit.org> <4B460CA0.20800@sequans.com> <4B4625BC.6090201@mycircuit.org> Message-ID: <4B47879F.7020709@mycircuit.org> On 01/08/2010 03:57 PM, r0g wrote: > Chris Rebert wrote: > >> On Thu, Jan 7, 2010 at 10:19 AM, Peter wrote: >> >> >>>> The .ini file is the simpliest solution, at least from the user point of >>>> view, no need to learn any python syntax. >>>> >>> I am speaking from the point of view of a python programmer, and I find the >>> .ini restrictions not necessarily simple, for example when dealing with >>> structured data (I suppose it is trivial to specify a dictionnary or a list >>> for the purpose of my request) For example, configuration files for the >>> logging module get unwieldy when you specify several loggers , handlers, >>> formatters etc, because you have to break down structured data ( objects ) >>> to name,value pairs. >>> >> >> >>> So what is the "worshipped" approach, when you need more than name=value >>> pairs ? >>> >> JSON is one option: http://docs.python.org/library/json.html >> >> Cheers, >> Chris >> -- >> http://blog.rebertia.com >> > > Yes, JSON is rapidly becoming a standard for stuff like this as it's > widely portable and less bulky than XML. It's the native format for > couchdb too which is nice if you want to replicate and share the > contents of your documents. > > Roger. > Excellent, thanks , have to check that out. Any experience with yaml ? Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Fri Jan 8 14:32:34 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 8 Jan 2010 15:32:34 -0400 Subject: Another Screwy Problem Message-ID: <4dc0cfea1001081132p191fe045p72a79744054fc520@mail.gmail.com> Hi; I have this line of code: sql = 'select Name, Price from %sPackages where ID=%s;' % (store, pid) which prints to this: select Name, Price from productsPackages where ID=1; which when I enter it into the MySQL interpreter gives me this: mysql> select Name, Price from productsPackages where ID=1; +------+--------+ | Name | Price | +------+--------+ | pkg | 123.45 | +------+--------+ 1 row in set (0.00 sec) exactly what I expect. However, in my script for some reason it returns this: ((1,),) Why would it do that? I guess there's some foolish thing I did in my code somewhere, but I'll be darned if I know where. Here's the whole script: #! /usr/bin/python import cgitb; cgitb.enable() import cgi import MySQLdb import sys,os sys.path.append(os.getcwd()) from login import login from template import top, bottom from sets import Set import fpformat form = cgi.FieldStorage() store = form.getfirst('store') cat = form.getfirst('cat', '') user, passwd, db, host = login() db = MySQLdb.connect(host, user, passwd, db) cursor = db.cursor() def displayProducts(patientID=''): try: # These are stores with categories where ordering by price is important sql = 'select ID from %s where Category="%s" order by Price desc;' % (store, cat) cursor.execute(sql) except: # Stores, like prescriptions, where ordering by price is not important sql = 'select ID from %s;' % (store) cursor.execute(sql) ids = [itm[0] for itm in cursor] cursor.execute('describe %s;' % store) colFields, colFieldValues = [itm[0] for itm in cursor], [itm[1] for itm in cursor] i = 0 if len(ids) > 0: for id in ids: # print '\n' print '
' print "" % store print "" % id j = 0 for col in colFields: sql = 'select %s from %s where ID="%s";' % (col, store, str(id)) cursor.execute(sql) colValue = cursor.fetchone() if col == 'SKU': print "" % colValue[0] print '%s: %s
\n' % (col, colValue[0]) elif col == 'Category': print "" % colValue[0] print '%s: %s
\n' % (col, colValue[0]) elif col == 'OutOfStock': if colValue[0] == '1': # This product is out of stock outOfStockFlag = 'yes' else: outOfStockFlag = 'no' elif col[:3] != 'pic': notSet = 1 if isinstance(colValue[0], (str, int, float, long, complex, unicode, list, buffer, xrange, tuple)): pass else: try: html = "%s:
" print html except: pass if notSet == 1: if len(col) > 49: colX = col[:50] + '...' else: colX = col print '%s: %s
\n' % (colX, colValue[0]) elif col == 'pic1': try: if (colValue[0] != None): # if (colValue[0] != None) and (len(colValue[0] > 0)): print '
\n' % (store, col[3:], id, store, col[3:], id, store, col[3:], id) except TypeError: raise except: raise # i += 1 # try: # content = colValues[0][x].tostring() # pic = "tmp" + str(i) + ".jpg" # try: # os.remove(pic) # except: # pass # img = open(pic, "w") # img.write(content) # print '

' % pic # img.close() # except: # pass j += 1 if store != 'prescriptions': if outOfStockFlag == 'yes': print 'This item is currently out of stock.' else: print "" else: print "" % patientID print "" print '


' print '\n' print '\n' def cgiFieldStorageToDict(fieldStorage): params = {} for key in fieldStorage.keys(): params[key] = fieldStorage[key].value return params def display(): top() dict = cgiFieldStorageToDict(cgi.FieldStorage()) print dict if store == 'prescriptions': password = form.getfirst('password') email = form.getfirst('email') sql = 'select ID, FirstName, LastName from %s where PW="%s" and Email="%s";' % ('patientsPersonalData', password, email) try: cursor.execute(sql) patientID = [itm[0] for itm in cursor][0] print "

Welcome, %s %s!

" % ([itm[1] for itm in cursor][0], [itm[2] for itm in cursor][0]) displayProducts(patientID) except: print "We're sorry. The email address and password you entered do not correspond with what is recorded in our database. Please click the "back" button and try again.

" else: displayProducts() cursor.execute('show tables like "%sPackages";' % store) if cursor.fetchone() is not None: sql = 'select ID from categories%s where Category="%s";' % (store[0].upper() + store[1:], cat) cursor.execute(sql) categoryID = cursor.fetchone()[0] sql = 'select p.ID from %sPackages p join %sCategoriesPackages c where c.CategoryID=%s;' % (store, store, categoryID) cursor.execute(sql) packageIDs = [itm[0] for itm in cursor] for pid in packageIDs: sql = 'select ProductID from %sProductsPackages where PackageID=%s;' % (store, pid) cursor.execute(sql) productIDs = [itm[0] for itm in cursor] sql = 'select Name, Price from %sPackages where ID=%s;' % (store, pid) # print sql print cursor.fetchall() name = [itm[0] for itm in cursor] price = [itm[1] for itm in cursor] print 'Package Name: %s
' % name print 'Price: %s
' % price allPrices = 0.00 for pid in productIDs: sql = 'select Name, Price from %s;' % store # print sql cursor.execute(sql) pName, pPrice = cursor.fetchone() pPrice = float(pPrice) allPrices += pPrice print 'Product Name: %s
' % pName print 'All products together usually cost: $%f

' % str(fpformat.fix(round(int(allPrices * 100))/100, 2)) cursor.close() bottom() display() TIA, beno -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Fri Jan 8 14:36:31 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 8 Jan 2010 15:36:31 -0400 Subject: getfirst and re In-Reply-To: <4B4785C6.4010801@mrabarnett.plus.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4B44C816.2090201@tim.thechases.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4B4751CB.2050907@mrabarnett.plus.com> <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> <4B4785C6.4010801@mrabarnett.plus.com> Message-ID: <4dc0cfea1001081136l3ea55531g5aafbe3a9125332a@mail.gmail.com> On Fri, Jan 8, 2010 at 3:21 PM, MRAB wrote: > Victor Subervi wrote: > > On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden > steve at holdenweb.com>> wrote: >> >> MRAB wrote: >> > Victor Subervi wrote: >> > [snip] >> >> >> >> Code snippet: >> >> >> >> def cgiFieldStorageToDict(fieldStorage): >> ^^^^^^^^^^^^ >> Further hint ... >> >> >> params = {} >> >> for key in fieldStorage.keys(): >> >> params[key] = cgi.FieldStorage[key].value >> > ^^^^^^^^^^^^^^^^^^^^^ >> > This is your problem. >> >> >> The problem is that I don't understand this code that I exactly copied >> from a Web page tutorial. Can you folks point me to tutorials where I can >> learn to comprehend this code? Specifically, the line in question. How is it >> that one can code "params[key]" (what does that mean?) and the other side, >> what does that mean >> >> If you got it from: > > Recipe 81547: Using a simple dictionary for CGI parameters > http://code.activestate.com/recipes/81547/ > > then no, it wasn't "exactly copied". If you'd copy-and-pasted it then it > would've been correct, as well as much quicker to do... I'm afraid to say you're right again :-} beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Fri Jan 8 14:37:49 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 08 Jan 2010 19:37:49 +0000 Subject: getfirst and re In-Reply-To: <4dc0cfea1001081115kffe51cdpb3b31be683822db9@mail.gmail.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4dc0cfea1001060931h4d8e35et84d6b7c23b69f6c2@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4B4751CB.2050907@mrabarnett.plus.com> <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> <4B477EDA.4000105@sequans.com> <4dc0cfea1001081109n384ed968sf955e02c8971cf7@mail.gmail.com> <4dc0cfea1001081115kffe51cdpb3b31be683822db9@mail.gmail.com> Message-ID: <4B47898D.1050807@mrabarnett.plus.com> Victor Subervi wrote: > On Fri, Jan 8, 2010 at 3:09 PM, Victor Subervi > wrote: > > > > On Fri, Jan 8, 2010 at 2:52 PM, Jean-Michel Pichavant > > wrote: > > Victor Subervi wrote: > > On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden > > >> > wrote: > > MRAB wrote: > > Victor Subervi wrote: > > [snip] > >> > >> Code snippet: > >> > >> def cgiFieldStorageToDict(fieldStorage): > ^^^^^^^^^^^^ > Further hint ... > > >> params = {} > >> for key in fieldStorage.keys(): > >> params[key] = cgi.FieldStorage[key].value > > ^^^^^^^^^^^^^^^^^^^^^ > > This is your problem. > > > The problem is that I don't understand this code that I > exactly copied from a Web page tutorial. Can you folks point > me to tutorials where I can learn to comprehend this code? > Specifically, the line in question. How is it that one can > code "params[key]" (what does that mean?) and the other > side, what does that mean > > I think you are gathering more fans Victor :) > > http://docs.python.org/tutorial/datastructures.html#dictionaries > > > This still isn't telling me what I need to know. Perhaps I'm missing > the point, as in the recent case with putting the "print cookie" > statement in the header. I am assuming (translation: making an a$$ > out of you and me) that "params[key] automatically assigns the > fieldStorage key to the newly created params key in the dict of the > same, and assigning the value from the cgi.FieldStorage of that key > to the params value. Is that correct? > TIA, > beno > > > I may have answered my own question. I have this: > > def cgiFieldStorageToDict(fieldStorage): > params = {} > for key in fieldStorage.keys(): > params[key] = fieldStorage[key].value > return params > Which is what Recipe 81547 actually says! > dict = cgiFieldStorageToDict(cgi.FieldStorage()) > print dict > > which gave me this: > > {'store': 'products', 'cat': 'prodCat1'} > > which looks about right. I would have written the code like this: > > keys = [] > values = [] > for key, value in fieldStorage.iteritems(): > keys.append(key) > values.append(value) > params = dict(zip(keys, values)) > > which obviously isn't as elegant. But that's what I knew. Learned > another trick, I guess ;) > If that works then so should this: params = dict(fieldStorage.iteritems()) From robert.kern at gmail.com Fri Jan 8 14:51:47 2010 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 08 Jan 2010 13:51:47 -0600 Subject: lightweight encryption of text file In-Reply-To: References: Message-ID: On 2010-01-08 13:14 PM, Daniel Fetchinson wrote: > I have a plain text file which I would like to protect in a very > simple minded, yet for my purposes sufficient, way. I'd like to > encrypt/convert it into a binary file in such a way that possession of > a password allows anyone to convert it back into the original text > file while not possessing the password one would only see the > following with the standard linux utility 'file': > > [fetchinson at fetch ~]$ file encrypted.data > encrypted.data: data > > and the effort required to convert the file back to the original text > file without the password would be equivalent to guessing the > password. > > I'm fully aware of the security implications of this loose > specification, but for my purposes this would be a good solution. > > What would be the simplest way to achieve this using preferably stock > python without 3rd party modules? If a not too complex 3rd party > module made it really simple that would be acceptable too. Paul Rubin's p3.py algorithm is probably the most straightforward way to meet these requirements. It's not a standard crypto algorithm by any means, but Paul knows his stuff and has devised it with these deployment restrictions in mind. http://www.nightsong.com/phr/crypto/p3.py -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From mackrackit at gmail.com Fri Jan 8 14:55:26 2010 From: mackrackit at gmail.com (Dave McCormick) Date: Fri, 8 Jan 2010 12:55:26 -0700 Subject: Dynamic text color In-Reply-To: <4B478779.2080908@mrabarnett.plus.com> References: <2a36ceaf1001081037n5ed5701ft9104c1396f36977c@mail.gmail.com> <4B478779.2080908@mrabarnett.plus.com> Message-ID: <2a36ceaf1001081155q72f771ebwf42fd00bf9d6e805@mail.gmail.com> On Fri, Jan 8, 2010 at 12:28 PM, MRAB wrote: > Dave McCormick wrote: > >> >> >> On Wed, Jan 6, 2010 at 9:18 AM, John Posner > jjposner at optimum.net>> wrote: >> >> On Tue, 05 Jan 2010 16:54:44 -0500, Dave McCormick >> > wrote: >> >> But it is not what I am wanting. I first thought to make it look >> for a space but that would not work when a single character like >> "#" is to be colored if there is a "string" of them. Or if all >> of the characters between quotes are to be colored. >> >> >> Regular expressions are good at handling searches like: >> >> * all the characters between quotes >> * the two-character string "do", but only if it's a complete word >> >> -John >> >> -- http://mail.python.org/mailman/listinfo/python-list >> >> I need another hint... >> >> Been doing some reading and playing and it looks like >> r'\bxxx\b' >> is what I need. But I can not figure out how to pass a variable between >> \b___\b >> If the word in question is between the "\b \b" and in the list then it >> works like I want it to. >> The below does not work. >> >> greenList_regexp = "|".join(greenList) >> for matchobj in re.finditer(r'\bgreenList_regexp\b', complete_text): >> start,end = matchobj.span() >> >> The regex r'\bgreenList_regexp\b' will match the string > 'greenList_regexp' if it's a whole word. > > What you mean is "any of these words, provided that they're whole > words". You'll need to group the alternatives within "(?:...)", like > this: > > r'\b(?:' + greenList_regexp + ')\b' > > > Thanks but that did not work either. Maybe I have something else wrong too? complete_text = Tbox.get("1.0", END) greenList = "green grass".split() greenList_regexp = "|".join(greenList) for matchobj in re.finditer(r'\b(?:' + greenList_regexp + ')\b', complete_text): start,end = matchobj.span() Tbox.tag_add("green", "1.0 + %d chars" % start,"1.0 + %d chars" % end) Tbox.tag_config("green", foreground="green") The words "green" and "grass" do not turn green. Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Fri Jan 8 15:01:55 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Fri, 8 Jan 2010 16:01:55 -0400 Subject: getfirst and re In-Reply-To: <4B47898D.1050807@mrabarnett.plus.com> References: <4dc0cfea1001060849x57fa77aei9dc7f7115e577c64@mail.gmail.com> <4dc0cfea1001061119raa34a56oe0daf56af1a16fb6@mail.gmail.com> <4dc0cfea1001080123g234fbf9fn372e9740f23abede@mail.gmail.com> <4B4751CB.2050907@mrabarnett.plus.com> <4dc0cfea1001081043k197f5d9u2b4807072b95ca9f@mail.gmail.com> <4B477EDA.4000105@sequans.com> <4dc0cfea1001081109n384ed968sf955e02c8971cf7@mail.gmail.com> <4dc0cfea1001081115kffe51cdpb3b31be683822db9@mail.gmail.com> <4B47898D.1050807@mrabarnett.plus.com> Message-ID: <4dc0cfea1001081201q2ec3d6f5t8b85cd15479053f2@mail.gmail.com> On Fri, Jan 8, 2010 at 3:37 PM, MRAB wrote: > Victor Subervi wrote: > >> On Fri, Jan 8, 2010 at 3:09 PM, Victor Subervi > victorsubervi at gmail.com>> wrote: >> >> >> >> On Fri, Jan 8, 2010 at 2:52 PM, Jean-Michel Pichavant >> > wrote: >> >> Victor Subervi wrote: >> >> On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden >> >> >> >> >> wrote: >> >> MRAB wrote: >> > Victor Subervi wrote: >> > [snip] >> >> >> >> Code snippet: >> >> >> >> def cgiFieldStorageToDict(fieldStorage): >> ^^^^^^^^^^^^ >> Further hint ... >> >> >> params = {} >> >> for key in fieldStorage.keys(): >> >> params[key] = cgi.FieldStorage[key].value >> > ^^^^^^^^^^^^^^^^^^^^^ >> > This is your problem. >> >> >> The problem is that I don't understand this code that I >> exactly copied from a Web page tutorial. Can you folks point >> me to tutorials where I can learn to comprehend this code? >> Specifically, the line in question. How is it that one can >> code "params[key]" (what does that mean?) and the other >> side, what does that mean >> >> I think you are gathering more fans Victor :) >> >> http://docs.python.org/tutorial/datastructures.html#dictionaries >> >> >> This still isn't telling me what I need to know. Perhaps I'm missing >> the point, as in the recent case with putting the "print cookie" >> statement in the header. I am assuming (translation: making an a$$ >> out of you and me) that "params[key] automatically assigns the >> fieldStorage key to the newly created params key in the dict of the >> same, and assigning the value from the cgi.FieldStorage of that key >> to the params value. Is that correct? >> TIA, >> beno >> >> >> I may have answered my own question. I have this: >> >> def cgiFieldStorageToDict(fieldStorage): >> params = {} >> for key in fieldStorage.keys(): >> params[key] = fieldStorage[key].value >> return params >> >> Which is what Recipe 81547 actually says! > > > dict = cgiFieldStorageToDict(cgi.FieldStorage()) >> print dict >> >> which gave me this: >> >> {'store': 'products', 'cat': 'prodCat1'} >> >> which looks about right. I would have written the code like this: >> >> keys = [] >> values = [] >> for key, value in fieldStorage.iteritems(): >> keys.append(key) >> values.append(value) >> params = dict(zip(keys, values)) >> >> which obviously isn't as elegant. But that's what I knew. Learned another >> trick, I guess ;) >> >> If that works then so should this: > > params = dict(fieldStorage.iteritems()) Yeah. Thanks! beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From gnewsg at gmail.com Fri Jan 8 15:14:02 2010 From: gnewsg at gmail.com (Giampaolo Rodola') Date: Fri, 8 Jan 2010 12:14:02 -0800 (PST) Subject: asyncore based port splitter code questions References: Message-ID: On 4 Gen, 18:58, George Trojan wrote: > asyncore based code is supposed to be simple, > but I need while loops and a lot of try/except clauses. First of all: you DON'T have to use while loops or anything which is blocking where by "blocking" I mean anything like time.sleep(). asyncore, just like Twisted, is an asynchrounous abstraction on top of select(). Anything you call must return *immediately* otherwise the whole thing will hang aka "stop serving any connected client or server". > I designed the code by looking at Python 2.3 > source for asyncore and originally wanted to use add_channel() and > del_channel() methods. However in Python 2.6 del_channel() closes the > socket in addition to deleting it from the map. Don't look at the 2.3 source. Use asyncore of Python 2.6 which is far more improved, bug-fixed and also *different*, where by that I mean that it might actually behaves differently. If you are forced to use Python 2.3 my advice is to get a copy of Python's asyncore.py and asynchat.py and include them in your code. Secondly, to temporarily "sleep" your connections *don't* remove anything from your map. The correct way of doing things here is to override readable() and writable() methods and make them return False as long as you want your connection to hang. Now I'm going to comment some parts of your code. > class Reader(asyncore.dispatcher): > ? ? ?def __init__(self, sock, writers): > ? ? ? ? ?asyncore.dispatcher.__init__(self, sock) > ? ? ? ? ?self.writers = writers > > ? ? ?def handle_read(self): > ? ? ? ? ?data = self.recv(1024) > ? ? ? ? ?for writer in self.writers: > ? ? ? ? ? ? ?writer.add_data(data) [...] > ? ? ?def handle_write(self): > ? ? ? ? ?while self.data: > ? ? ? ? ? ? ?log_msg('sending data to %s' % str(self.address)) > ? ? ? ? ? ? ?sent = self.send(self.data) > ? ? ? ? ? ? ?self.data = self.data[sent:] > ? ? ? ? ?self.suspend_channel() By looking at how you are appending data you want to send in a buffer, it looks like you might want to use asynchat.async_chat rather than asyncore.dispatcher. async_chat.push() and async_chat.push_with_producer() methods already take care of buffers logic and make sure that all the data gets sent to the other peer without going lost. Actually there's no reason to use asyncore.dispatcher class directly except for creating a socket which listens on an interface and then passes the connection to another class inheriting from asynchat.async_chat which will actually handle that session. So my recommendation is to use asynchat.async_chat whenever possible. > ? ? ?def suspend_channel(self, map=None): > ? ? ? ? ?fd = self._fileno > ? ? ? ? ?if map is None: > ? ? ? ? ? ? ?map = self._map > ? ? ? ? ?if fd in map: > ? ? ? ? ? ? ?del map[fd] > As I said this is unecessary. Override readable() and writable() methods instead. > ? ? ?def handle_close(self): > ? ? ? ? ?log_msg('closing writer connection') > ? ? ? ? ?self.close() > ? ? ? ? ?# try to reconnect > ? ? ? ? ?time.sleep(TMOUT) > ? ? ? ? ?self.mksocket() You really don't want to use time.sleep() there. It blocks everything. > ? ? ? ? ?while True: > ? ? ? ? ? ? ?try: > ? ? ? ? ? ? ? ? ?disp = Dispatcher(port, destinations) > ? ? ? ? ? ? ? ? asyncore.loop(timeout=TMOUT, use_poll=True) > ? ? ? ? ? ? ?except socket.error, (errno, e): > ? ? ? ? ? ? ? ? ?if errno == 98: > ? ? ? ? ? ? ? ? ? ? ?log_msg('sleeping %d s: %s', (30, e)) > ? ? ? ? ? ? ? ? ? ? ?time.sleep(30) Same as above. As a final note I would recommend to take a look at pyftpdlib code which uses asyncore/asynchat as part of its core: http://code.google.com/p/pyftpdlib It can be of some help to figure out how things should be done. --- Giampaolo http://code.google.com/p/pyftpdlib/ From suresh.amritapuri at gmail.com Fri Jan 8 15:17:03 2010 From: suresh.amritapuri at gmail.com (suresh.amritapuri) Date: Fri, 8 Jan 2010 12:17:03 -0800 (PST) Subject: PIL show() not working for 2nd pic References: Message-ID: <5d0f9d9b-b0fb-47bb-a125-d7305a01e7fd@j5g2000yqm.googlegroups.com> On Jan 8, 6:32?am, McColgst wrote: > Do you get any errors or warnings? > Could we see the code you ran to get this problem? > > Thanks > Sean I used to get no such file or directory showing some files in /tmp directory. But today I am getting a different type of message, which appeared yesterday also. (eog:8368): Gtk-CRITICAL **: gtk_tree_model_get_iter: assertion `path- >depth > 0' failed (eog:8368): Gtk-CRITICAL **: gtk_list_store_get_value: assertion `VALID_ITER (iter, list_store)' failed (eog:8368): GLib-GObject-WARNING **: /build/buildd/glib2.0-2.20.1/ gobject/gtype.c:3940: type id `0' is invalid (eog:8368): GLib-GObject-WARNING **: can't peek value table for type `' which is not currently referenced Segmentation fault -----------code---------------------- #!/usr/bin/python print "Aum Amriteshwaryai Namaha" import Image imagePath = "/home/suresh/EE241/book_images_3ed/ch03/" im34 = Image.open(imagePath + "breast_digital_Xray.tif") im35 = Image.open(imagePath + "DFT_no_log.tif") im35.show() def neg(x): return 255-1-x import math def logtr(x): y = math.log(1+x,10) print y return y*100 im34x = im34.point(neg) im34x.show() im35x = im35.point(logtr) im35x.show() ---------------------------------------end of code Any ideas? Thanks suresh From dreadpiratejeff at gmail.com Fri Jan 8 15:31:35 2010 From: dreadpiratejeff at gmail.com (J) Date: Fri, 8 Jan 2010 15:31:35 -0500 Subject: table from csv file In-Reply-To: <77b18520-c2c4-4e21-9147-1164d81b9f25@l30g2000yqb.googlegroups.com> References: <59653a09-c3f7-464f-a36f-475f0d759e46@e27g2000yqd.googlegroups.com> <77b18520-c2c4-4e21-9147-1164d81b9f25@l30g2000yqb.googlegroups.com> Message-ID: <36dec4ff1001081231k62029267tb8df7ff16ad60ddb@mail.gmail.com> On Fri, Jan 8, 2010 at 13:55, Jon Clements wrote: > On Jan 8, 5:59?pm, marlowe wrote: >> I am trying to create a table in python from a csv file where I input >> which columns I would like to see, and the table only shows those >> columns. I have attached an example of the csv file i am using, and >> some of the code I have written. I am having trouble converting >> variables between lists, dictionaries and tuples. Is there a name for >> what I am attempting to do? any help to get me on the right track with >> this is appreciated. >> >> test.csv I had to edit that and comma delimit it, because cut and paste gave me random numbers/types of whitespace... [code snipped] > This might be a useful starting point (I'm guessing this is what > you're after...) > > Let's assume your 'CSV' file is tab separated as it's certainly not > comma separated :) > > import csv > csvin = csv.reader(open('test.csv'), delimiter='\t') > header = dict( (val.strip(),idx) for idx, val in enumerate(next > (csvin)) ) > > We can use header as a column name->column index lookup eg header > ['Open'] == 1 > > from operator import itemgetter > wanted = ['Open', 'Close'] # Although you'll want to use raw_input and > split on ',' > getcols = itemgetter(*[header[col] for col in wanted]) > > getcols is a helper function that'll return a tuple of the columns in > the requested order... > > for row in csvin: > ? ?print getcols(row) > > Loop over the rest of the file and output the required columns. As someone who knows just enough to be dangerous... what about this: import csv reader = open('C:/test.txt','rb') data = csv.DictReader(reader,restval='000',restkey='Misc') print "Options are: Date, Open, Close, High, Low, Volume, Adj Close" column = raw_input('What do you want? (comma delimited)?') choice = column.split(',') f = 12 print ''.join([s.center(f) for s in choice]) for row in data: print ''.join([(row.get(s)).center(f) for s in choice]) -- Mike Ditka - "If God had wanted man to play soccer, he wouldn't have given us arms." - http://www.brainyquote.com/quotes/authors/m/mike_ditka.html From carsten.haese at gmail.com Fri Jan 8 16:06:55 2010 From: carsten.haese at gmail.com (Carsten Haese) Date: Fri, 08 Jan 2010 16:06:55 -0500 Subject: Another Screwy Problem In-Reply-To: <4dc0cfea1001081132p191fe045p72a79744054fc520@mail.gmail.com> References: <4dc0cfea1001081132p191fe045p72a79744054fc520@mail.gmail.com> Message-ID: Victor Subervi wrote: > Hi; > I have this line of code: > sql = 'select Name, Price from %sPackages where ID=%s;' % (store, pid) > which prints to this: > select Name, Price from productsPackages where ID=1; > which when I enter it into the MySQL interpreter gives me this: > mysql> select Name, Price from productsPackages where ID=1; > +------+--------+ > | Name | Price | > +------+--------+ > | pkg | 123.45 | > +------+--------+ > 1 row in set (0.00 sec) > > exactly what I expect. However, in my script for some reason it returns > this: > ((1,),) The only logical explanation is that this is not the output from executing the above-mentioned query. Maybe you should ask yourself what it actually is. -Carsten From rsomerville at sjgeophysics.com Fri Jan 8 16:12:30 2010 From: rsomerville at sjgeophysics.com (Robert Somerville) Date: Fri, 08 Jan 2010 13:12:30 -0800 Subject: 24 bit signed integer binary conversion help needed Message-ID: <4B479FBE.7040206@sjgeophysics.com> hi; I am trying to read 24bit signed WAV format (little endian) data from a WAV file and convert it to 32 bit little endian integer format ... can anybody please tell me how to do the conversion from 24 bit to 32 bit with a snippet of Python code ??? Thanks so much Robert Somerville From hansmu at xs4all.nl Fri Jan 8 16:13:50 2010 From: hansmu at xs4all.nl (Hans Mulder) Date: Fri, 08 Jan 2010 22:13:50 +0100 Subject: Threading change, 2.5.4 -> 2.6.1 In-Reply-To: References: Message-ID: <4b47a083$0$22937$e4fe514c@news.xs4all.nl> anon wrote: > Gib Bogle wrote: >> The code below runs with Python 2.5.4, but gives the following error >> messages with Python 2.6.1. What needs to be done to make it work? >> Thanks. >> >> C:\Summer09\Tutorials>python url_queue.pyw >> Traceback (most recent call last): >> File "url_queue.pyw", line 3, in >> import threading >> File "C:\Summer09\Tutorials\threading.py", line 9, in >> class ProcessingThread(threading.Thread, QtCore.QObject): >> AttributeError: 'module' object has no attribute 'Thread' >> > > Nothing to do with the version of python > > You have a file called threading.py in C:\Summer09\Tutorials (the same > folder as url_queue.pyw). The line 'import threading' is finding this > module before python's threading module. > > Rename this file and you should be fine. Keep in mind that if there's a threading.pyc in C:\Summer09\Tutorials, you have to rename (or delete) that one as well. Hope this helps, -- HansM From invalid at invalid.invalid Fri Jan 8 16:24:36 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 8 Jan 2010 21:24:36 +0000 (UTC) Subject: 24 bit signed integer binary conversion help needed References: Message-ID: On 2010-01-08, Robert Somerville wrote: > I am trying to read 24bit signed WAV format (little endian) data from a > WAV file and convert it to 32 bit little endian integer format ... can > anybody please tell me how to do the conversion from 24 bit to 32 bit > with a snippet of Python code ??? def sext24(d): if ord(d[2]) & 0x80: return d+'\xff' else: return d+'\x00' -- Grant Edwards grante Yow! Am I having fun yet? at visi.com From jjposner at optimum.net Fri Jan 8 16:27:02 2010 From: jjposner at optimum.net (John Posner) Date: Fri, 08 Jan 2010 16:27:02 -0500 Subject: Dynamic text color References: <2a36ceaf1001081037n5ed5701ft9104c1396f36977c@mail.gmail.com> Message-ID: On Fri, 08 Jan 2010 14:28:57 -0500, MRAB wrote: > Dave McCormick wrote: >> On Wed, Jan 6, 2010 at 9:18 AM, John Posner > > wrote: >> On Tue, 05 Jan 2010 16:54:44 -0500, Dave McCormick >> > wrote: >> But it is not what I am wanting. I first thought to make it >> look >> for a space but that would not work when a single character like >> "#" is to be colored if there is a "string" of them. Or if all >> of the characters between quotes are to be colored. >> Regular expressions are good at handling searches like: >> * all the characters between quotes >> * the two-character string "do", but only if it's a complete word >> -John >> -- http://mail.python.org/mailman/listinfo/python-list >> I need another hint... >> Been doing some reading and playing and it looks like >> r'\bxxx\b' >> is what I need. But I can not figure out how to pass a variable between >> \b___\b >> If the word in question is between the "\b \b" and in the list then it >> works like I want it to. >> The below does not work. >> greenList_regexp = "|".join(greenList) >> for matchobj in re.finditer(r'\bgreenList_regexp\b', complete_text): >> start,end = matchobj.span() >> > The regex r'\bgreenList_regexp\b' will match the string > 'greenList_regexp' if it's a whole word. > > What you mean is "any of these words, provided that they're whole > words". You'll need to group the alternatives within "(?:...)", like > this: > > r'\b(?:' + greenList_regexp + ')\b' Oops, MRAB, you forgot to make the last literal a RAW string -- it should be r')\b' Dave, we're already into some pretty heavy regular-expression work, huh?. Here's another approach -- not nearly as elegant as MRAB's: Given this list: greenList = ['green', 'grass', 'grump'] ... you currently are using join() to construct this regexp search string: 'green|grass|grump' ... but you've decided that you really want this similar regexp search string: r'\bgreen\b|\bgrass\b|\bgrump\b' You can achieve this by transforming each item on the list, then invoking join() on the transformed list to create the search string. Here are a couple of ways to transform the list: * List comprehension: whole_word_greenList = [ r'\b' + word + r'\b' for word in greenList] * map() and a user-defined function: def xform_to_wholeword_searchstring(word): return r'\b' + word + r'\b' whole_word_greenList = map(xform_to_wholeword_searchstring, greenList) HTH, John From invalid at invalid.invalid Fri Jan 8 16:33:03 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 8 Jan 2010 21:33:03 +0000 (UTC) Subject: 24 bit signed integer binary conversion help needed References: Message-ID: On 2010-01-08, Grant Edwards wrote: > On 2010-01-08, Robert Somerville wrote: > >> I am trying to read 24bit signed WAV format (little endian) data from a >> WAV file and convert it to 32 bit little endian integer format ... can >> anybody please tell me how to do the conversion from 24 bit to 32 bit >> with a snippet of Python code ??? > > def sext24(d): > if ord(d[2]) & 0x80: > return d+'\xff' > else: > return d+'\x00' I guess I assumed you knew how to read 3 bytes of data from a file: f = open('datafile','rb') d = f.read(3) -- Grant Edwards grante Yow! I need to discuss at BUY-BACK PROVISIONS visi.com with at least six studio SLEAZEBALLS!! From irmen-NOSPAM- at xs4all.nl Fri Jan 8 16:34:13 2010 From: irmen-NOSPAM- at xs4all.nl (Irmen de Jong) Date: Fri, 08 Jan 2010 22:34:13 +0100 Subject: 24 bit signed integer binary conversion help needed In-Reply-To: References: Message-ID: <4b47a483$0$22903$e4fe514c@news.xs4all.nl> On 8-1-2010 22:12, Robert Somerville wrote: > hi; > I am trying to read 24bit signed WAV format (little endian) data from a > WAV file and convert it to 32 bit little endian integer format ... can > anybody please tell me how to do the conversion from 24 bit to 32 bit > with a snippet of Python code ??? > > Thanks so much > Robert Somerville Are you using the standard wave module? I guess that will produce a string of 3-byte audio frames with readframes(). Won't it work to chop this up in individual 3-byte frames, then appending a '\0' char to every frame, and then writing the frames to the target wave ? -irmen From invalid at invalid.invalid Fri Jan 8 16:37:26 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 8 Jan 2010 21:37:26 +0000 (UTC) Subject: 24 bit signed integer binary conversion help needed References: <4b47a483$0$22903$e4fe514c@news.xs4all.nl> Message-ID: On 2010-01-08, Irmen de Jong wrote: > On 8-1-2010 22:12, Robert Somerville wrote: >> hi; >> I am trying to read 24bit signed WAV format (little endian) data from a >> WAV file and convert it to 32 bit little endian integer format ... can >> anybody please tell me how to do the conversion from 24 bit to 32 bit >> with a snippet of Python code ??? > > Are you using the standard wave module? I guess that will > produce a string of 3-byte audio frames with readframes(). > > Won't it work to chop this up in individual 3-byte frames, > then appending a '\0' char to every frame, and then writing > the frames to the target wave ? Not if the 3-byte values are signed two's compliment values. -- Grant Edwards grante Yow! I feel like I am at sharing a ``CORN-DOG'' visi.com with NIKITA KHRUSCHEV ... From fetchinson at googlemail.com Fri Jan 8 16:39:39 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Fri, 8 Jan 2010 22:39:39 +0100 Subject: lightweight encryption of text file In-Reply-To: References: Message-ID: >> I have a plain text file which I would like to protect in a very >> simple minded, yet for my purposes sufficient, way. I'd like to >> encrypt/convert it into a binary file in such a way that possession of >> a password allows anyone to convert it back into the original text >> file while not possessing the password one would only see the >> following with the standard linux utility 'file': >> >> [fetchinson at fetch ~]$ file encrypted.data >> encrypted.data: data >> >> and the effort required to convert the file back to the original text >> file without the password would be equivalent to guessing the >> password. >> >> I'm fully aware of the security implications of this loose >> specification, but for my purposes this would be a good solution. >> >> What would be the simplest way to achieve this using preferably stock >> python without 3rd party modules? If a not too complex 3rd party >> module made it really simple that would be acceptable too. > > Paul Rubin's p3.py algorithm is probably the most straightforward way to > meet > these requirements. It's not a standard crypto algorithm by any means, but > Paul > knows his stuff and has devised it with these deployment restrictions in > mind. > > http://www.nightsong.com/phr/crypto/p3.py Thanks a lot, currently I'm having trouble using this code on python 2.6 but probably some small tweaking will fix it. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From suresh.amritapuri at gmail.com Fri Jan 8 16:43:45 2010 From: suresh.amritapuri at gmail.com (suresh.amritapuri) Date: Fri, 8 Jan 2010 13:43:45 -0800 (PST) Subject: PIL how to display multiple images side by side Message-ID: <6d9fef57-7fdc-46ec-9b53-7aa824e785e9@s31g2000yqs.googlegroups.com> Hi, In PIL, how to display multiple images in say m rows and n colums when I have m*n images. suresh From jcd at sdf.lonestar.org Fri Jan 8 16:44:55 2010 From: jcd at sdf.lonestar.org (J. Clifford Dyer) Date: Fri, 8 Jan 2010 16:44:55 -0500 Subject: Another Screwy Problem In-Reply-To: References: <4dc0cfea1001081132p191fe045p72a79744054fc520@mail.gmail.com> Message-ID: <20100108214455.GA12839@sdf.lonestar.org> Victor Subervi wrote: > Hi; > I have this line of code: > sql = 'select Name, Price from %sPackages where ID=%s;' % (store, pid) > which prints to this: > select Name, Price from productsPackages where ID=1; > which when I enter it into the MySQL interpreter gives me this: > mysql> select Name, Price from productsPackages where ID=1; > +------+--------+ > | Name | Price | > +------+--------+ > | pkg | 123.45 | > +------+--------+ > 1 row in set (0.00 sec) > > exactly what I expect. However, in my script for some reason it returns > this: > ((1,),) First, never use string formatting to pass parameters to your database. Read the MySQLdb documentation (or sqlite, or psycopg2) documentation for reasons why, and how to do it right. Second, in the same documentation, look up anything having the word "fetch" in it. That should show you how to get the data you want Third, please be more specific in your questions. You say "In my script for some reason *it* returns this: ((1,),)," but you don't show us what "it" is. How did you get that from your script? So far we've got a variable called sql with your query in it. How do I get the same wrong result you got? I don't know. I could pass it to a function that looks like this: def botch_sql_query(sql): return ((1,),) I could furthermore "fix" it, so that it looks right, by doing this: def fixed_sql_query(sql): return(('pkg', 123.45),) But that probably doesn't help. Give us enough code to be clear, but not so much as to be overwhelming. There's an essay called "How to ask smart question" by Eric Raymond that should help. Cheers, Cliff From fetchinson at googlemail.com Fri Jan 8 16:47:26 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Fri, 8 Jan 2010 22:47:26 +0100 Subject: lightweight encryption of text file In-Reply-To: References: Message-ID: >>> I have a plain text file which I would like to protect in a very >>> simple minded, yet for my purposes sufficient, way. I'd like to >>> encrypt/convert it into a binary file in such a way that possession of >>> a password allows anyone to convert it back into the original text >>> file while not possessing the password one would only see the >>> following with the standard linux utility 'file': >>> >>> [fetchinson at fetch ~]$ file encrypted.data >>> encrypted.data: data >>> >>> and the effort required to convert the file back to the original text >>> file without the password would be equivalent to guessing the >>> password. >>> >>> I'm fully aware of the security implications of this loose >>> specification, but for my purposes this would be a good solution. >>> >>> What would be the simplest way to achieve this using preferably stock >>> python without 3rd party modules? If a not too complex 3rd party >>> module made it really simple that would be acceptable too. >> >> Paul Rubin's p3.py algorithm is probably the most straightforward way to >> meet >> these requirements. It's not a standard crypto algorithm by any means, >> but >> Paul >> knows his stuff and has devised it with these deployment restrictions in >> mind. >> >> http://www.nightsong.com/phr/crypto/p3.py > > Thanks a lot, currently I'm having trouble using this code on python > 2.6 but probably some small tweaking will fix it. Actually, it also doesn't work with python 2.5 and currently I don't have access to anything older. array.array raises a ValueError: string length not a multiple of item size Does anyone recall a change to array.array? The full traceback is Traceback (most recent call last): File "p3.py", line 163, in _test() File "p3.py", line 143, in _test c1 = e(plain,key) File "p3.py", line 69, in p3_encrypt xkey = _expand_key(k_enc, n+4) File "p3.py", line 41, in _expand_key return array ('L', j) ValueError: string length not a multiple of item size Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From irmen-NOSPAM- at xs4all.nl Fri Jan 8 16:55:44 2010 From: irmen-NOSPAM- at xs4all.nl (Irmen de Jong) Date: Fri, 08 Jan 2010 22:55:44 +0100 Subject: 24 bit signed integer binary conversion help needed In-Reply-To: References: <4b47a483$0$22903$e4fe514c@news.xs4all.nl> Message-ID: <4b47a98e$0$22903$e4fe514c@news.xs4all.nl> On 8-1-2010 22:37, Grant Edwards wrote: > On 2010-01-08, Irmen de Jong wrote: >> Are you using the standard wave module? I guess that will >> produce a string of 3-byte audio frames with readframes(). >> >> Won't it work to chop this up in individual 3-byte frames, >> then appending a '\0' char to every frame, and then writing >> the frames to the target wave ? > > Not if the 3-byte values are signed two's compliment values. > Meh. The wave module is pretty useless for format conversion then until it grows some utility methods. -irmen From jcd at sdf.lonestar.org Fri Jan 8 16:57:35 2010 From: jcd at sdf.lonestar.org (J. Clifford Dyer) Date: Fri, 8 Jan 2010 16:57:35 -0500 Subject: Another Screwy Problem In-Reply-To: <4dc0cfea1001081132p191fe045p72a79744054fc520@mail.gmail.com> References: <4dc0cfea1001081132p191fe045p72a79744054fc520@mail.gmail.com> Message-ID: <20100108215735.GA22400@sdf.lonestar.org> On Fri, Jan 08, 2010 at 03:32:34PM -0400, Victor Subervi wrote regarding Another Screwy Problem: > Date: Fri, 8 Jan 2010 15:32:34 -0400 > From: Victor Subervi > To: python-list > Subject: Another Screwy Problem > > Hi; > I have this line of code: > sql = 'select Name, Price from %sPackages where ID=%s;' % (store, pid) > which prints to this: > select Name, Price from productsPackages where ID=1; > which when I enter it into the MySQL interpreter gives me this: > mysql> select Name, Price from productsPackages where ID=1; > +------+--------+ > | Name | Price | > +------+--------+ > | pkg | 123.45 | > +------+--------+ > 1 row in set (0.00 sec) > exactly what I expect. However, in my script for some reason it returns > this: > ((1,),) > Why would it do that? I guess there's some foolish thing I did in my > code somewhere, but I'll be darned if I know where. Here's the whole > script: > #! /usr/bin/python > import cgitb; cgitb.enable() > import cgi > import MySQLdb > import sys,os > sys.path.append(os.getcwd()) > from login import login > from template import top, bottom > from sets import Set > import fpformat > form = cgi.FieldStorage() > store = form.getfirst('store') > cat = form.getfirst('cat', '') > user, passwd, db, host = login() > db = MySQLdb.connect(host, user, passwd, db) > cursor = db.cursor() > def displayProducts(patientID=''): > try: # These are stores with categories where ordering by price is > important > sql = 'select ID from %s where Category="%s" order by Price desc;' > % (store, cat) > cursor.execute(sql) > except: # Stores, like prescriptions, where ordering by price is not > important > sql = 'select ID from %s;' % (store) > cursor.execute(sql) > ids = [itm[0] for itm in cursor] > cursor.execute('describe %s;' % store) > colFields, colFieldValues = [itm[0] for itm in cursor], [itm[1] for > itm in cursor] > i = 0 > if len(ids) > 0: > for id in ids: > # print '\n' > print '
' > print "" % store > print "" % id > j = 0 > for col in colFields: > sql = 'select %s from %s where ID="%s";' % (col, store, > str(id)) > cursor.execute(sql) > colValue = cursor.fetchone() > if col == 'SKU': > print "" % > colValue[0] > print '%s: %s
\n' % (col, colValue[0]) > elif col == 'Category': > print "" % > colValue[0] > print '%s: %s
\n' % (col, colValue[0]) > elif col == 'OutOfStock': > if colValue[0] == '1': # This product is out of stock > outOfStockFlag = 'yes' > else: > outOfStockFlag = 'no' > elif col[:3] != 'pic': > notSet = 1 > if isinstance(colValue[0], (str, int, float, long, complex, > unicode, list, buffer, xrange, tuple)): > pass > else: > try: > html = "%s:
" > print html > except: > pass > if notSet == 1: > if len(col) > 49: > colX = col[:50] + '...' > else: > colX = col > print '%s: %s
\n' % (colX, colValue[0]) > elif col == 'pic1': > try: > if (colValue[0] != None): > # if (colValue[0] != None) and (len(colValue[0] > 0)): > print ' class="highslide" href="getpic.py?store=%s&pic=%s&id=%s" > onclick="return hs.expand(this)"> src="getpic.py?store=%s&pic=%s&id=%s" width="100" height="80" alt="" > align="left" border="0" title="Click to enlarge" style="border: > 0px">
\n' % (store, col[3:], id, store, col[3:], > id, store, col[3:], id) > except TypeError: > raise > except: > raise > # i += 1 > # try: > # content = colValues[0][x].tostring() > # pic = "tmp" + str(i) + ".jpg" > # try: > # os.remove(pic) > # except: > # pass > # img = open(pic, "w") > # img.write(content) > # print '

' % pic > # img.close() > # except: > # pass > j += 1 > if store != 'prescriptions': > if outOfStockFlag == 'yes': > print 'This item is currently out of > stock.' > else: > print "" > else: > print "" % > patientID > print "" > print '


' > print '\n' > print '\n' > def cgiFieldStorageToDict(fieldStorage): > params = {} > for key in fieldStorage.keys(): > params[key] = fieldStorage[key].value > return params > def display(): > top() > dict = cgiFieldStorageToDict(cgi.FieldStorage()) > print dict > if store == 'prescriptions': > password = form.getfirst('password') > email = form.getfirst('email') > sql = 'select ID, FirstName, LastName from %s where PW="%s" and > Email="%s";' % ('patientsPersonalData', password, email) > try: > cursor.execute(sql) > patientID = [itm[0] for itm in cursor][0] > print "

Welcome, %s %s!

" % ([itm[1] for itm in > cursor][0], [itm[2] for itm in cursor][0]) > displayProducts(patientID) > except: > print "We're sorry. The email address and password you entered do > not correspond with what is recorded in our database. Please click the > "back" button and try again.

" > else: > displayProducts() > cursor.execute('show tables like "%sPackages";' % store) > if cursor.fetchone() is not None: > sql = 'select ID from categories%s where Category="%s";' % > (store[0].upper() + store[1:], cat) > cursor.execute(sql) > categoryID = cursor.fetchone()[0] > sql = 'select p.ID from %sPackages p join %sCategoriesPackages c > where c.CategoryID=%s;' % (store, store, categoryID) > cursor.execute(sql) > packageIDs = [itm[0] for itm in cursor] > for pid in packageIDs: > sql = 'select ProductID from %sProductsPackages where > PackageID=%s;' % (store, pid) > cursor.execute(sql) > productIDs = [itm[0] for itm in cursor] > sql = 'select Name, Price from %sPackages where ID=%s;' % > (store, pid) > # print sql > print cursor.fetchall() > name = [itm[0] for itm in cursor] > price = [itm[1] for itm in cursor] > print 'Package Name: %s
' % name > print 'Price: %s
' % price > allPrices = 0.00 > for pid in productIDs: > sql = 'select Name, Price from %s;' % store > # print sql > cursor.execute(sql) > pName, pPrice = cursor.fetchone() > pPrice = float(pPrice) > allPrices += pPrice > print 'Product Name: %s
' % pName > print 'All products together usually cost: $%f

' % > str(fpformat.fix(round(int(allPrices * 100))/100, 2)) > cursor.close() > bottom() > display() > TIA, > beno > -- > The Logos has come to bear > [1]http://logos.13gems.com/ > > References > > 1. http://logos.13gems.com/ > -- > http://mail.python.org/mailman/listinfo/python-list Sigh. Ignore my last message. I was reading the part that Caersten quoted, not the original message. Well, ignore the part where I say you didn't show us your code. You obviously did. However, you showed us so much code that it's hard to find your problem. Before you ask questions, try to trim your code down to just enough to demonstrate the problem. In this case: >>> db = MySQLdb.connect(host, user, passwd, db) >>> cursor = db.cursor() >>> sql = 'select Name, Price from %sPackages where ID=%s;' % (store, pid) >>> print cursor.fetchall() ((1,),) Would have been sufficient. Then you could ask, "why does cursor.fetchall() return ((1,),) instead of the expected results. (The answer is that you never executed your query.) From irmen-NOSPAM- at xs4all.nl Fri Jan 8 17:02:00 2010 From: irmen-NOSPAM- at xs4all.nl (Irmen de Jong) Date: Fri, 08 Jan 2010 23:02:00 +0100 Subject: lightweight encryption of text file In-Reply-To: References: Message-ID: <4b47ab07$0$22919$e4fe514c@news.xs4all.nl> On 8-1-2010 22:39, Daniel Fetchinson wrote: >> >> http://www.nightsong.com/phr/crypto/p3.py > > Thanks a lot, currently I'm having trouble using this code on python > 2.6 but probably some small tweaking will fix it. If you keep having issues with this module, maybe you can try this: http://www.freenet.org.nz/ezPyCrypto/ It provides a very easy compact api on top of PyCrypto. So you'll need to install that as well to be able to use this. -irmen From robert.kern at gmail.com Fri Jan 8 17:09:27 2010 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 08 Jan 2010 16:09:27 -0600 Subject: lightweight encryption of text file In-Reply-To: References: Message-ID: On 2010-01-08 15:47 PM, Daniel Fetchinson wrote: >>>> I have a plain text file which I would like to protect in a very >>>> simple minded, yet for my purposes sufficient, way. I'd like to >>>> encrypt/convert it into a binary file in such a way that possession of >>>> a password allows anyone to convert it back into the original text >>>> file while not possessing the password one would only see the >>>> following with the standard linux utility 'file': >>>> >>>> [fetchinson at fetch ~]$ file encrypted.data >>>> encrypted.data: data >>>> >>>> and the effort required to convert the file back to the original text >>>> file without the password would be equivalent to guessing the >>>> password. >>>> >>>> I'm fully aware of the security implications of this loose >>>> specification, but for my purposes this would be a good solution. >>>> >>>> What would be the simplest way to achieve this using preferably stock >>>> python without 3rd party modules? If a not too complex 3rd party >>>> module made it really simple that would be acceptable too. >>> >>> Paul Rubin's p3.py algorithm is probably the most straightforward way to >>> meet >>> these requirements. It's not a standard crypto algorithm by any means, >>> but >>> Paul >>> knows his stuff and has devised it with these deployment restrictions in >>> mind. >>> >>> http://www.nightsong.com/phr/crypto/p3.py >> >> Thanks a lot, currently I'm having trouble using this code on python >> 2.6 but probably some small tweaking will fix it. > > Actually, it also doesn't work with python 2.5 and currently I don't > have access to anything older. array.array raises a > > ValueError: string length not a multiple of item size > > Does anyone recall a change to array.array? > > The full traceback is > > Traceback (most recent call last): > File "p3.py", line 163, in > _test() > File "p3.py", line 143, in _test > c1 = e(plain,key) > File "p3.py", line 69, in p3_encrypt > xkey = _expand_key(k_enc, n+4) > File "p3.py", line 41, in _expand_key > return array ('L', j) > ValueError: string length not a multiple of item size Are you on a 64-bit platform? Unfortunately, array's integer typecodes are platform-specific, but p3.py requires a 32-bit integer and was written on a 32-bit platform. It's reasonably straightforward to fix. Put this bit of (untested) code at the top of the file and replace occurrences of 'L' with uint32: # Find the typecode of a 32-bit unsigned integer. for typecode in 'IL': if len(array(typecode, [0]).tostring()) == 4: uint32 = typecode break else: raise RuntimeError("Neither 'I' nor 'L' are unsigned 32-bit integers.") -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From rsomerville at sjgeophysics.com Fri Jan 8 17:11:22 2010 From: rsomerville at sjgeophysics.com (Robert Somerville) Date: Fri, 08 Jan 2010 14:11:22 -0800 Subject: 24 bit signed integer binary conversion help needed Message-ID: <4B47AD8A.3040406@sjgeophysics.com> An HTML attachment was scrubbed... URL: From mackrackit at gmail.com Fri Jan 8 17:21:07 2010 From: mackrackit at gmail.com (Dave McCormick) Date: Fri, 08 Jan 2010 15:21:07 -0700 Subject: Dynamic text color In-Reply-To: References: <2a36ceaf1001081037n5ed5701ft9104c1396f36977c@mail.gmail.com> Message-ID: <4B47AFD3.8040507@gmail.com> John Posner wrote: > On Fri, 08 Jan 2010 14:28:57 -0500, MRAB > wrote: > >> The regex r'\bgreenList_regexp\b' will match the string >> 'greenList_regexp' if it's a whole word. >> >> What you mean is "any of these words, provided that they're whole >> words". You'll need to group the alternatives within "(?:...)", like >> this: >> >> r'\b(?:' + greenList_regexp + ')\b' > > Oops, MRAB, you forgot to make the last literal a RAW string -- it > should be r')\b' > > Dave, we're already into some pretty heavy regular-expression work, > huh?. Here's another approach -- not nearly as elegant as MRAB's: > > Given this list: > > greenList = ['green', 'grass', 'grump'] > > ... you currently are using join() to construct this regexp search > string: > > 'green|grass|grump' > > ... but you've decided that you really want this similar regexp search > string: > > r'\bgreen\b|\bgrass\b|\bgrump\b' > > You can achieve this by transforming each item on the list, then > invoking join() on the transformed list to create the search string. > Here are a couple of ways to transform the list: > > * List comprehension: > > whole_word_greenList = [ r'\b' + word + r'\b' for word in greenList] > > * map() and a user-defined function: > > def xform_to_wholeword_searchstring(word): > return r'\b' + word + r'\b' > > whole_word_greenList = map(xform_to_wholeword_searchstring, greenList) > > > HTH, > John John, That second "r" appears to do the trick. Yea, pretty heavy into it. I read someplace that regular-expressions were tricky, but I did not expect this :) Now to start working this into the rest of my app and study your second approach. Thanks again for the help!!! Dave From sccolbert at gmail.com Fri Jan 8 17:21:16 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Fri, 8 Jan 2010 23:21:16 +0100 Subject: lightweight encryption of text file In-Reply-To: <4b47ab07$0$22919$e4fe514c@news.xs4all.nl> References: <4b47ab07$0$22919$e4fe514c@news.xs4all.nl> Message-ID: <7f014ea61001081421w42e95ccob56f16ac1ead0fa0@mail.gmail.com> PyCrypto is already pretty easy to use by itself. I dont know why you want a wrapper on top of it. On Fri, Jan 8, 2010 at 11:02 PM, Irmen de Jong wrote: > On 8-1-2010 22:39, Daniel Fetchinson wrote: > >> >>> http://www.nightsong.com/phr/crypto/p3.py >>> >> >> Thanks a lot, currently I'm having trouble using this code on python >> 2.6 but probably some small tweaking will fix it. >> > > If you keep having issues with this module, maybe you can try this: > > http://www.freenet.org.nz/ezPyCrypto/ > > It provides a very easy compact api on top of PyCrypto. So you'll need to > install that as well to be able to use this. > > -irmen > > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsomerville at sjgeophysics.com Fri Jan 8 17:25:38 2010 From: rsomerville at sjgeophysics.com (Robert Somerville) Date: Fri, 08 Jan 2010 14:25:38 -0800 Subject: 24 bit signed integer binary conversion help needed Message-ID: <4B47B0E2.3040500@sjgeophysics.com> thanks Grant, your sext24 function does the trick (this Python newbie thanks you a lot ...) so i loop on d=wave.readframes(1) call dd=sext24(d) on the frame read , then ddd=struct.unpack("<%ul" % 1 ,dd) unpack(dd) the output of sext24 into a 32bit integer (seems to do the trick correctly for signed 24 bit integers ....) -thanks , bob From ben+python at benfinney.id.au Fri Jan 8 17:44:31 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 09 Jan 2010 09:44:31 +1100 Subject: Scripting References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: <87pr5kushc.fsf@benfinney.id.au> Jorgen Grahn writes: > Regarding the book's title: is it just me, or are Python programmers > in general put off when people call it "scripting"? The term ?script? for a Python program is part of the official terminology, so those who would be put off by that term have to at least contend with that fact. I refer to such things as ?programs?, whatever language they're written in, since ?script? does seem to have derogatory connotations. I don't get particularly off-put either way. -- \ ?He was the mildest-mannered man / That ever scuttled ship or | `\ cut a throat.? ??Lord? George Gordon Noel Byron, _Don Juan_ | _o__) | Ben Finney From ben+python at benfinney.id.au Fri Jan 8 17:54:30 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 09 Jan 2010 09:54:30 +1100 Subject: Recommended "new" way for config files References: <4B46075D.8080606@mycircuit.org> <4B460CA0.20800@sequans.com> <4B4625BC.6090201@mycircuit.org> Message-ID: <87iqbcus0p.fsf@benfinney.id.au> Chris Rebert writes: > JSON is one option: http://docs.python.org/library/json.html YAML is another contender. Compared to JSON, it is yet to gain as much mind-share, but even more human-friendly and no less expressive. Here are some discussions of YAML that can help you evaluate it: -- \ ?The Things to do are: the things that need doing, that you see | `\ need to be done, and that no one else seems to see need to be | _o__) done.? ?Richard Buckminster Fuller, 1970-02-16 | Ben Finney From tjreedy at udel.edu Fri Jan 8 18:48:43 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 08 Jan 2010 18:48:43 -0500 Subject: Caching objects in a C extension In-Reply-To: <4ff69845-b7c6-4b0c-a0c5-c4154551098c@m25g2000yqc.googlegroups.com> References: <1bd0de37-1a32-44f4-91fa-d3126e3aaeda@u7g2000yqm.googlegroups.com> <7qohgkF2mjU2@mid.uni-berlin.de> <7qp7opFglrU1@mid.uni-berlin.de> <4ff69845-b7c6-4b0c-a0c5-c4154551098c@m25g2000yqc.googlegroups.com> Message-ID: On 1/8/2010 12:50 PM, casevh wrote: > These are numeric objects created by gmpy. I'm trying to minimize the > overhead for using mpz with small numbers. Objects are created and > deleted very often by the interpreter as expressions are evaluated. I > don't keep ownership of the objects. CPython creates a hidden list of small ints on startup for the same reason. Currently about -10 to 256. int.__new__ checks that a requested int object would have a value in this range (if so, reuse) -- or not (create new object). From tjreedy at udel.edu Fri Jan 8 18:52:21 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 08 Jan 2010 18:52:21 -0500 Subject: Standardized interpreter speed evaluation tool In-Reply-To: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> References: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> Message-ID: On 1/8/2010 5:25 AM, alexru wrote: > Is there any standardized interpreter speed evaluation tool? Say I > made few changes in interpreter code and want to know if those changes > made python any better, which test should I use? The Unladen Swallow (sp?) project at code.google.com/????, which looks to change and speedup CPython, has a suite of benchmarks that are are perhaps better than PyBench. From george.trojan at noaa.gov Fri Jan 8 18:58:51 2010 From: george.trojan at noaa.gov (George Trojan) Date: Fri, 08 Jan 2010 23:58:51 +0000 Subject: asyncore based port splitter code questions In-Reply-To: References: Message-ID: Thanks for your help. Some comments below. George Giampaolo Rodola' wrote: > On 4 Gen, 18:58, George Trojan wrote: > > > Secondly, to temporarily "sleep" your connections *don't* remove > anything from your map. > The correct way of doing things here is to override readable() and > writable() methods and make them return False as long as you want your > connection to hang. > Good advice. > Now I'm going to comment some parts of your code. > >> class Reader(asyncore.dispatcher): >> def __init__(self, sock, writers): >> asyncore.dispatcher.__init__(self, sock) >> self.writers = writers >> >> def handle_read(self): >> data = self.recv(1024) >> for writer in self.writers: >> writer.add_data(data) > [...] >> def handle_write(self): >> while self.data: >> log_msg('sending data to %s' % str(self.address)) >> sent = self.send(self.data) >> self.data = self.data[sent:] >> self.suspend_channel() > > > By looking at how you are appending data you want to send in a buffer, > it looks like you might want to use asynchat.async_chat rather than > asyncore.dispatcher. > async_chat.push() and async_chat.push_with_producer() methods already > take care of buffers logic and make sure that all the data gets sent > to the other peer without going lost. > > Actually there's no reason to use asyncore.dispatcher class directly > except for creating a socket which listens on an interface and then > passes the connection to another class inheriting from > asynchat.async_chat which will actually handle that session. My understanding is that asynchat is used for bi-directional connection, I don't see how it applies to my case (forwarding data). However I rewrote the Writer class following some of asynchat code. > > So my recommendation is to use asynchat.async_chat whenever possible. > You really don't want to use time.sleep() there. > It blocks everything. > >> while True: >> try: >> disp = Dispatcher(port, destinations) >> asyncore.loop(timeout=TMOUT, use_poll=True) >> except socket.error, (errno, e): >> if errno == 98: >> log_msg('sleeping %d s: %s', (30, e)) >> time.sleep(30) > > Same as above. > I wanted to reconnect after the os cleans up half-closed sockets. Otherwise the program exits immediately with a message: terminating - uncaught exception: [Errno 98] Address already in use > > As a final note I would recommend to take a look at pyftpdlib code > which uses asyncore/asynchat as part of its core: > http://code.google.com/p/pyftpdlib > It can be of some help to figure out how things should be done. > Thanks for good example to study. > > > --- Giampaolo > http://code.google.com/p/pyftpdlib/ From clp2 at rebertia.com Fri Jan 8 19:02:46 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Fri, 8 Jan 2010 16:02:46 -0800 Subject: Standardized interpreter speed evaluation tool In-Reply-To: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> References: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> Message-ID: <50697b2c1001081602m7614b94esc4cd9241fafbdf49@mail.gmail.com> On Fri, Jan 8, 2010 at 2:25 AM, alexru wrote: > Is there any standardized interpreter speed evaluation tool? Say I > made few changes in interpreter code and want to know if those changes > made python any better, which test should I use? Although apparently undocumented, test.pystone is some sort of interpreter benchmark. Cheers, Chris -- http://blog.rebertia.com From tjreedy at udel.edu Fri Jan 8 19:35:39 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 08 Jan 2010 19:35:39 -0500 Subject: One function calling another defined in the same file being exec'd In-Reply-To: References: Message-ID: On 1/8/2010 12:02 PM, Mitchell L Model wrote: > On further reflection, I will add that > what appears to be happening is that during import both the global and > local dictionaries are set to a copy of the globals() from the importing > scope and that copy becomes the value of the module's __dict__ once > import has completed successfully. I have no idea why you think that. The module dict starts empty except for __name__, __file__, and perhaps a couple of other 'hidden' items. It is not a copy and has nothing to do with importing scopes. > and that copy becomes the value of the module's __dict__ once > import has completed successfully. That new dict becomes .... . > Because exec leaves locals() and globals() distinct, Not necessarily. In 3.x, at least, exec(s) executes s in the current scope. If this is top level, where locals is globals, then same should be true within exec. d = {} exec(s, d) In 3.x, at least, d will also be used as locals. exec(s, d, d) Again, globals and locals are not distinct. It would seem that in 3.x, the only way for exec to have distinct globals and locals is to call exec(s) where they are distinct or to pass distince globals and locals. Some of the issues of this thread are discussed in Language Reference 4.1, Naming and Binding. I suppose it could be clearer that it is, but the addition of nonlocal scope complicated things. Terry Jan Reedy From steve at REMOVE-THIS-cybersource.com.au Fri Jan 8 20:50:59 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 09 Jan 2010 01:50:59 GMT Subject: lightweight encryption of text file References: Message-ID: <0357cddd$0$1336$c3e8da3@news.astraweb.com> On Fri, 08 Jan 2010 20:14:51 +0100, Daniel Fetchinson wrote: > I have a plain text file which I would like to protect in a very simple > minded, yet for my purposes sufficient, way. I'd like to encrypt/convert > it into a binary file in such a way that possession of a password allows > anyone to convert it back into the original text file while not > possessing the password one would only see the following with the > standard linux utility 'file': > > [fetchinson at fetch ~]$ file encrypted.data > encrypted.data: data If that is your sole requirement, then the addition of a single non-text byte (say, a null) anywhere in the file would be sufficient to have file identify it as data. You say "encrypt/convert" -- does this mean that you don't care if people can read the text in a hex editor, so long as file identifies it as data? Would something like a binary Vigenere Cipher be sufficient? # Untested def encrypt(plaintext, password): cipher = [] for i, c in enumerate(plaintext): shift = password[i % len(password)] shift = ord(shift) cipher.append((ord(c) + shift) % 256) return ''.join([chr(n) for n in cipher]) def decrypt(ciphertext, password): plain = [] for i, c in enumerate(ciphertext): shift = password[i % len(password)] shift = ord(shift) plain.append((256 + ord(c) - shift) % 256) return ''.join([chr(n) for n in plain]) (How times have changed... once upon a time, the Vigenere Cipher was considered the gold-standard unbreakable encryption technology. Now it merely qualifies as obfuscation.) Is it acceptable if there is a chance (small, possibly vanishingly small) that file will identify it as text? As far as I know, even the heavyweight "serious" encryption algorithms don't *guarantee* that the cipher text will include non-text bytes. > and the effort required to convert the file back to the original text > file without the password would be equivalent to guessing the password. If you seriously mean that, then "lightweight encryption" won't do the job, because it is vulnerable to frequency analysis, which is easier than guessing the password. You need proper, heavy-weight encryption. > I'm fully aware of the security implications of this loose > specification, but for my purposes this would be a good solution. Can you explain what your objection to real encryption is? Are you concerned about the complex API? The memory requirements and processing power required? (Neither of which are particularly high for small text files on a modern PC, but perhaps you have to encrypt tens of millions of huge files on an underpowered device...) > What would be the simplest way to achieve this using preferably stock > python without 3rd party modules? If a not too complex 3rd party module > made it really simple that would be acceptable too. The problem is that, as I see it, you've assumed a solution rather than state what your requirements are. I'm *guessing* that you are more concerned of having to learn to use a complex API, rather than actually *requiring* a lightweight encryption algorithm. If that's the case, then something serious like blowfish or similar would be perfectly acceptable to you, so long as the API was simple. (On the other hand, perhaps vulnerability to frequency analysis is a feature, not a bug, in your use-case. If you forget the password, you have a chance of recovering the text.) -- Steven From diesch at spamfence.net Fri Jan 8 21:29:41 2010 From: diesch at spamfence.net (Florian Diesch) Date: Sat, 09 Jan 2010 03:29:41 +0100 Subject: Scripting (was Re: Python books, literature etc) References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: Jorgen Grahn writes: > Regarding the book's title: is it just me, or are Python programmers > in general put off when people call it "scripting"? > > I won't attempt a strict definition of the term "scripting language", > but it seems like non-programmers use it to mean "less scary than what > you might think of as programming", while programmers interpret it as > "not useful as a general-purpose language". For me "scripting" means something like "task automation within a given program or environment", in contrast to "wring a stand-alone program". Florian -- From diesch at spamfence.net Fri Jan 8 21:32:05 2010 From: diesch at spamfence.net (Florian Diesch) Date: Sat, 09 Jan 2010 03:32:05 +0100 Subject: restructuredText editor ? References: Message-ID: <5e6l17-28k.ln1@mid.florian-diesch.de> Peter writes: > What editor do people out there use to edit .rst files for > sphinx-python documentation ? Emacs with ReST mode and YASnippet Florian -- From half.italian at gmail.com Fri Jan 8 22:43:04 2010 From: half.italian at gmail.com (Sean DiZazzo) Date: Fri, 8 Jan 2010 19:43:04 -0800 (PST) Subject: PIL how to display multiple images side by side References: <6d9fef57-7fdc-46ec-9b53-7aa824e785e9@s31g2000yqs.googlegroups.com> Message-ID: <6057f3b6-00f9-4fd2-b82f-65844e30d96f@v25g2000yqk.googlegroups.com> On Jan 8, 1:43?pm, "suresh.amritapuri" wrote: > Hi, > > In PIL, how to display multiple images in say m rows and n colums when > I have m*n images. > > suresh Sounds like a good project to learn PIL with. ~Sean From tosters at gmail.com Fri Jan 8 22:56:48 2010 From: tosters at gmail.com (t0ster) Date: Fri, 8 Jan 2010 19:56:48 -0800 (PST) Subject: Python multiprocessing: Permission denied References: <1c85295e-11ee-471c-bd2f-420e0f2fc3b6@j4g2000yqe.googlegroups.com> <0353b63c$0$1336$c3e8da3@news.astraweb.com> Message-ID: <87ebeabb-7336-4acd-9196-cd6d180587c9@e27g2000yqd.googlegroups.com> On Jan 6, 1:20?am, Steven D'Aprano wrote: > On Tue, 05 Jan 2010 13:52:18 -0800,t0sterwrote: > > It looks like the user don't have permission to access shared memory. > > When executing with root privileges it works fine. > > > Is there any solution to run it as normal user(not root)? > > Then give the user permission to access shared memory. > > Why do you expect that Python would be able to over-ride the operating > system's security? This problem is no different from saying "It looks > like the user doesn't have permission to access this file". > > -- > Steven Yes, your are right. The problem was solved by chmoding /dev/shm From casevh at gmail.com Fri Jan 8 23:33:45 2010 From: casevh at gmail.com (casevh) Date: Fri, 8 Jan 2010 20:33:45 -0800 (PST) Subject: Caching objects in a C extension References: <1bd0de37-1a32-44f4-91fa-d3126e3aaeda@u7g2000yqm.googlegroups.com> <7qohgkF2mjU2@mid.uni-berlin.de> Message-ID: On Jan 8, 8:56?am, Antoine Pitrou wrote: > Le Fri, 08 Jan 2010 08:39:17 -0800, casevh a ?crit?: > > > > > Thanks for the reply. I realized that I missed one detail. The objects > > are created by the extension but are deleted by Python. I don't know > > that an object is no longer needed until its tp_dealloc is called. At > > that point, its reference count is 0. > > tuple objects and others already have such a caching scheme, so you could > download the Python source and look at e.g. Objects/tupleobject.c to see > how it's done. > > Regards > > Antoine. Thanks. That's exactly the information I was looking for. casevh From metolone+gmane at gmail.com Sat Jan 9 00:09:05 2010 From: metolone+gmane at gmail.com (Mark Tolonen) Date: Fri, 8 Jan 2010 21:09:05 -0800 Subject: Manipulating pointers in C using ctypes References: <08bda34e-9bee-44b1-b2de-80d647151786@a15g2000yqm.googlegroups.com> Message-ID: "Daniel Platz" wrote in message news:08bda34e-9bee-44b1-b2de-80d647151786 at a15g2000yqm.googlegroups.com... > Hello! > > I have to ask a newbie question about manipulating pointers in C using > ctypes. I have a C dll with two functions. The first one creates a > pointer and returns it to python. The second one takes a pointer as an > argument shows its address and the value at which it is pointing. This > I have implemented using the following code > > ----------- -------- pointers.c ---------------------------- > #include > #ifdef __cplusplus > extern "C" { // only need to export C interface if > // used by C++ source code > using namespace std; > #endif > > __declspec(dllexport) void* create() > { > double number = 2.2; > double* ptr = &number; > printf("Pointer address \t %p \n", ptr); > printf("Value at pointer \t %f \n", ptr[0]); > return (void*) ptr; > } > > __declspec(dllexport) int show(double* ptr) > { > printf("Pointer address \t %p \n", ptr); > printf("Pointer value \t %f\n", *ptr); > *ptr = 2.4; > printf("New pointer value \t %f\n", *ptr); > return 0; > } > > #ifdef __cplusplus > } > #endif > ------------------------------------------------------------------------ > > Please note that in the second function, the show function, I want to > manipulate the value at which the pointer points. > Now, I call this function from python with the following script. > > --------------------------- pointers.py -------------------------- > import ctypes as ct > > # Load dll > pointers = ct.cdll.LoadLibrary('pointers.dll') > getattr(pointers, 'create') > getattr(pointers, 'show') > pointers.create.restype = ct.c_void_p > > # Create pointer in C > ptr = pointers.create() > > # Show pointer address and value > print 'Adress returned to python ' +hex(ptr) > pointers.show(ct.c_void_p(ptr)) > ------------------------------------------------------------------- > > Calling this script gives me the following output: > > Pointer address 0021E508 > Value at pointer 2.200000 > Adress returned to python 0x21e508 > Pointer address 0021E508 > Pointer value 0.000000 (2.20000 expected) > New pointer value 2.400000 (2.40000 expected) > > But the script returns also an error. > > WindowsError: exception: access violation reading 0x40033333 > WARNING: Failure executing file: > > Another thing that I find strange is that the return value of > pointers.create is actually an integer instead of an ct.c_void_p > object. > > Moreover, I also tried to directly manipulate the address of the > pointer given as an argument to pointers.show. But when it returns to > python the pointer points still at the same address as before the > function call. > > Can someone help me with this problem? I would be very glad about an > answer. As Jason observed, your create() function creates a number on the local stack, which becomes invalid once the function returns. If you don't want to worry about freeing the object created in Jason's solution, you can create the double in python yourself and let create initialize it: -------------- pointers.py ---------------------- import ctypes as ct pointers = ct.CDLL('pointers.dll') pointers.create.restype = None pointers.show.restype = None dbl = ct.c_double() print ct.byref(dbl),dbl ptr = pointers.create(ct.byref(dbl)) # Show pointer address and value print 'Value returned to python: %f' % dbl.value pointers.show(ct.byref(dbl)) print 'Value returned to python: %f' % dbl.value -------------- pointers.c ------------------------ #include __declspec(dllexport) void create(double* ptr) { *ptr = 2.2; printf("Pointer address %p\n", ptr); printf("Value at pointer %f\n", *ptr); } __declspec(dllexport) void show(double* ptr) { printf("Pointer address %p\n", ptr); printf("Pointer value %f\n", *ptr); *ptr = 2.4; printf("New pointer value %f\n", *ptr); } -------------- OUTPUT ------------------------- c_double(0.0) Pointer address 00A05DC8 Value at pointer 2.200000 Value returned to python: 2.200000 Pointer address 00A05DC8 Pointer value 2.200000 New pointer value 2.400000 Value returned to python: 2.400000 ----------------------------------------------------- HTH, Mark From metolone+gmane at gmail.com Sat Jan 9 00:28:21 2010 From: metolone+gmane at gmail.com (Mark Tolonen) Date: Fri, 8 Jan 2010 21:28:21 -0800 Subject: C Structure rebuild with ctypes References: <7p6ksnFkg1U1@mid.individual.net> <7qo9avFivmU1@mid.individual.net> Message-ID: "Georg" wrote in message news:7qo9avFivmU1 at mid.individual.net... > Hi Mark, > >> Are you passing in these values, or are they being returned? To me the >> depth of the pointer references implies numVars, varNames, and varTypes >> are out parameters. I'll assume that for now. If they are in/out >> parameters let me know. > > If these parameters were in parameters. What would I have to do different? If you are passing in the values, you can remove a level of * referencing, since the parameters aren't being modified. Note the syntax to create an array: (type * length)(initializers...): --------------- func.c -------------------- #include __declspec(dllexport) void func(int numVars, char **varNames, int *varTypes) { int i; printf("numVars = %d\n",numVars); for(i = 0; i < numVars; i++) printf("%d: %s\n",varTypes[i],varNames[i]); } --------------- func.py ------------------ import ctypes as c # int func (int numVars, char **varNames, int *varTypes) INT = c.c_int PINT = c.POINTER(INT) PCHAR = c.c_char_p PPCHAR = c.POINTER(PCHAR) func = c.CDLL('func').func func.restype = None func.argtypes = [INT,PPCHAR,PINT] numVars = 3 varNames = (PCHAR * numVars)('abc','def','ghi') varTypes = (INT * numVars)(1,2,3) func(numVars,varNames,varTypes) --------------- OUTPUT --------------- numVars = 3 1: abc 2: def 3: ghi -------------------------------------------- -Mark From lie.1296 at gmail.com Sat Jan 9 01:14:31 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Sat, 09 Jan 2010 17:14:31 +1100 Subject: Need help to pass self.count to other classes. In-Reply-To: <0356a7d1$0$1336$c3e8da3@news.astraweb.com> References: <1d51a971-69ee-4b0e-9da6-9df1c37a2d10@34g2000yqp.googlegroups.com> <0356a7d1$0$1336$c3e8da3@news.astraweb.com> Message-ID: <4b481ec8$1@dnews.tpgi.com.au> On 1/8/2010 3:56 PM, Steven D'Aprano wrote: > Unfortunately this won't do what you expect, because sys.ps1 and ps2 > should be either strings, or objects with a __str__ method. They aren't > called to generate the prompt. but their __str__ does get called to generate the prompt. import sys class kbInterfaceHelper(object): def __init__(self, str_func): self.str_func = str_func def __str__(self): return self.str_func() class kbInterface(object): def __init__(self): self.count = 0 def prompt1(self): self.count += 1 return "[%d]> " % self.count def prompt2(self): l = len(str(self.count))+1 return "%s " % "."*l def dhook(self, value): print "[%d out]" % self.count def ehook(self, type, value, trace): print "[%d err]\n" % value kbi = kbInterface() sys.ps1 = kbInterfaceHelper(kbi.prompt1) sys.ps2 = kbInterfaceHelper(kbi.prompt2) sys.displayhook = kbi.dhook sys.excepthook = kbi.ehook From parul.pandey85 at gmail.com Sat Jan 9 03:30:06 2010 From: parul.pandey85 at gmail.com (pp) Date: Sat, 9 Jan 2010 00:30:06 -0800 (PST) Subject: Append to an Excel file Message-ID: Hi All, How do I add a line to an existing file. This should append to the existing data in the excel file, which was saved previously. Thanks, PP From jason.scheirer at gmail.com Sat Jan 9 03:47:21 2010 From: jason.scheirer at gmail.com (Jason Scheirer) Date: Sat, 9 Jan 2010 00:47:21 -0800 (PST) Subject: Append to an Excel file References: Message-ID: <313cb2b5-4100-4d4d-bb38-dbfe3ca71e17@j5g2000yqm.googlegroups.com> On Jan 9, 12:30?am, pp wrote: > Hi All, > > How do I add a line to an existing file. This should append to the > existing data in the excel file, which was saved previously. > > Thanks, > PP http://pypi.python.org/pypi/xlwt From jason.scheirer at gmail.com Sat Jan 9 03:56:06 2010 From: jason.scheirer at gmail.com (Jason Scheirer) Date: Sat, 9 Jan 2010 00:56:06 -0800 (PST) Subject: Clarifications on compiling for Windows References: <1f0b409b-f943-4b1a-a806-87b834337367@p8g2000yqb.googlegroups.com> <6170f903-4e83-4716-9b1d-cc40bba31a65@j5g2000yqm.googlegroups.com> Message-ID: <22445936-8c2b-4d43-af3e-ef103c8e9a93@o28g2000yqh.googlegroups.com> On Jan 7, 10:51?pm, Mensanator wrote: > On Jan 8, 12:19?am, peteshinners wrote: > > > > > > > > > My presentation for Pycon is coming together, but I need to make sure > > my information about compiling Python and Python extensions for > > Windows is correct. I'm really only experienced with this on the Linux > > side of things. > > > First of all, is the Windows FAQ fairly up to date? Should people be > > referring to section 6 if they are going to build an application with > > an embedded Python interpreter?http://www.python.org/doc/faq/windows/#how-can-i-embed-python-into-a-... > > > If I understand correctly, compiled extensions for Python on Windows > > should match the compiler that was used to build the interpreter > > itself? Is there a list somewhere that shows which version of msvc was > > used to compile the recent Python binaries? > > > Thank you for feedback. I definitely want to make sure I have this > > correct before telling anybody else? > > You aren't going to try it? At a high level: YES. Avoid FILE* and you are golden on Windows. Works like a charm. MSVC 2008 works for 2.6 with the least effort (I strongly recommend having it installed as that's what the build uses). If you have VS2008 you will have no problem whatsoever with setup.py install, even with C extensions. I'd like to verify the same with earlier versions of VS but I can't. MinGW works, too, but with slightly more effort: there are some command line arguments you have to issue setup.py to know how to use/where the MinGW compiler is. From parul.pandey85 at gmail.com Sat Jan 9 03:58:29 2010 From: parul.pandey85 at gmail.com (pp) Date: Sat, 9 Jan 2010 00:58:29 -0800 (PST) Subject: Append to an Excel file References: <313cb2b5-4100-4d4d-bb38-dbfe3ca71e17@j5g2000yqm.googlegroups.com> Message-ID: <74a95a2f-90b7-48e6-9b69-c7250b979978@l30g2000yqb.googlegroups.com> On Jan 9, 1:47?am, Jason Scheirer wrote: > On Jan 9, 12:30?am, pp wrote: > > > Hi All, > > > How do I add a line to an existing file. This should append to the > > existing data in the excel file, which was saved previously. > > > Thanks, > > PP > > http://pypi.python.org/pypi/xlwt Hi Jason and all, Thanks I have seen this.. my question is there a way to append to a excel file which has been closed. Any specific modes which can be added to the sheet so that it adds a line to the data which was return in some earlier running of the program. Thanks. From kzagradskiy at gmail.com Sat Jan 9 04:07:39 2010 From: kzagradskiy at gmail.com (kzagradskiy) Date: Sat, 9 Jan 2010 01:07:39 -0800 (PST) Subject: Link to module Stack Message-ID: Link to module Stack: http://groups.google.com/group/comp.lang.python/browse_thread/thread/e6a0668bb2be9a8e/64cb44a120baeca2?lnk=gst&q=stack+module#64cb44a120baeca2 Here's the stack module for py4th. nick ------------------------------- #!/usr/Util/bin/python # # @(#)stack.py 1.1 # # stack.py # generic stack class. class Stack: def __init__(self): self.__heap = [] def push (self, word): self.__heap.append (word) def pop (self): if len(self.__heap) == 0: raise InnerInterpreterError, "stack underflow" result = self.__heap[-1] del self.__heap[-1] return result def __repr__(self): return `self.__heap` def __str__(self): return `self.__heap` def flush (self): self.__heap = [] From dickinsm at gmail.com Sat Jan 9 04:22:29 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 9 Jan 2010 01:22:29 -0800 (PST) Subject: C Module's '1.#INF' changes to 'inf' at Python References: Message-ID: On Jan 8, 3:36?pm, Robert Kern wrote: > On 2010-01-08 07:48 AM, CELEN Erman wrote: > > My problem is that I?ve noticed a strange behavior in Python while > > handling FPEs on Windows after switching compilers (msvc8 to msvc9) and > > I am trying to find out how Python handles INF values to figure out > > where the problem might be. > > [...] > > Python 2.6 changed the string representations of the special floating point > values inf and nan. Previously, the string representation was left up to the C > runtime, so they differed between platforms. Python 2.6 normalized the string > representation across all platforms. The underlying values are the same. What > version of Python are you using? In addition to this, for good or ill Python 2.6 also standardized exceptional behaviour for the math module functions: log10(0.0) and sqrt(-1) used to produce different results across implementations, but now both should consistently produce ValueError regardless of the platform. This is achieved by dealing directly with special cases in the input before delegating to the relevant libm function; this of course has an associated performance cost. There are some notes on the (intended) current behaviour at the top of the Modules/ mathmodule.c file: http://svn.python.org/view/*checkout*/python/branches/release26-maint/Modules/mathmodule.c -- Mark From steve at REMOVE-THIS-cybersource.com.au Sat Jan 9 04:25:08 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 09 Jan 2010 09:25:08 GMT Subject: Link to module Stack References: Message-ID: <0358384b$0$1336$c3e8da3@news.astraweb.com> On Sat, 09 Jan 2010 01:07:39 -0800, kzagradskiy wrote: > class Stack: > def __init__(self): > self.__heap = [] A "heap" has a technical meaning in programming. To describe the internals of a stack as "heap" will be disconcerting and confusing to anyone who knows about stacks and heaps. > def push (self, word): > self.__heap.append (word) > def pop (self): > if len(self.__heap) == 0: > raise InnerInterpreterError, "stack underflow" "InnerInterpreterError" is the most inappropriate exception name I've ever seen. It has nothing to do with the interpreter, it's a stack error. > result = self.__heap[-1] > del self.__heap[-1] That is better written as result = self.__heap.pop(). -- Steven From anthra.norell at bluewin.ch Sat Jan 9 04:52:06 2010 From: anthra.norell at bluewin.ch (Anthra Norell) Date: Sat, 09 Jan 2010 10:52:06 +0100 Subject: lightweight encryption of text file In-Reply-To: References: Message-ID: <4B4851C6.1050009@bluewin.ch> Daniel Fetchinson wrote: > I have a plain text file which I would like to protect in a very > simple minded, yet for my purposes sufficient, way. I'd like to > encrypt/convert it into a binary file in such a way that possession of > a password allows anyone to convert it back into the original text > file while not possessing the password one would only see the > following with the standard linux utility 'file': > > [fetchinson at fetch ~]$ file encrypted.data > encrypted.data: data > > and the effort required to convert the file back to the original text > file without the password would be equivalent to guessing the > password. > > I'm fully aware of the security implications of this loose > specification, but for my purposes this would be a good solution. > > What would be the simplest way to achieve this using preferably stock > python without 3rd party modules? If a not too complex 3rd part > module made it really simple that would be acceptable too. Daniel, Here's what looks like another thread veering off into package-ology, leaving a stumped OP behind. "Don't use a random generator for encryption purposes!" warns the manual, of which fact I was reminded in no uncertain terms on this forum a few years ago when I proposed the following little routine in response to a post very similar to yours. One critic challenged me to encode my credit card data and post it. Which I did. Upon which another critic conjured up the horror vision of gigahertzes hacking my pathetic little effort to pieces as I was reading his message. Of the well-meaning kind, he urged me to put an immediate stop to this foolishness. I didn't. No unplanned expenditures ensued. Or to quote ... I forget who: Fools and innovators are people who don't care much about what one is not supposed to do. So, take or leave what follows for what it is worth or not worth, I am confident it works and would serve your purpose, which, as I understand, is not to write a text book on cryptology. Regards Frederic ############################## import random def crypt_string (string, key, floor = 0, size_of_set = 255): # key is a number. The sign of that number controls which way the process # goes. If the number is positive, the result is an encryption. A negative # number orders a decryption. if key == 0: return string # No processing import random MAX_CHUNK_SIZE = 32 MIN_CHUNK_SIZE = 16 def process_sequence (sequence): s = '' for c in sequence: r = random.randint (0, size_of_set - 1) s += chr (((ord (c) - floor + r * sign) % size_of_set) + floor) return s def shuffle_sequence (sequence): random.shuffle (sequence) return sequence sign = (key > 0) * 2 - 1 random.seed (key * sign) s = '' if sign > 0: # forward i = 0 while i < len (string): random_size_of_chunk = random.randint (MIN_CHUNK_SIZE, MAX_CHUNK_SIZE) clear_chunk_shuffled = shuffle_sequence (list (string [i:i+random_size_of_chunk])) code_chunk_shuffled = process_sequence (clear_chunk_shuffled) s += code_chunk_shuffled i += len (code_chunk_shuffled) else: # backward i = 0 while i < len (string): random_size_of_chunk = random.randint (MIN_CHUNK_SIZE, MAX_CHUNK_SIZE) code_chunk_shuffled = list (string [i:i+random_size_of_chunk]) real_size_of_chunk = len (code_chunk_shuffled) unshuffling_template = shuffle_sequence (range (real_size_of_chunk)) # 1. same ... clear_chunk_shuffled = process_sequence (code_chunk_shuffled) # 2. ... order clear_chunk = real_size_of_chunk * [None] for ii in range (real_size_of_chunk): clear_chunk [unshuffling_template[ii]] = clear_chunk_shuffled [ii] s += ''.join (clear_chunk) i += real_size_of_chunk return s def _crypt_file (in_file, out_file, key): BUFFER_SIZE = 1024 while 1: s = in_file.read (BUFFER_SIZE) if s == '': break out_file.write (crypt_string (s, key)) def crypt_file (in_file_name, out_file_name, key): in_file = open (in_file_name, 'rb') out_file = open (out_file_name, 'wb') _crypt_file (in_file, out_file, key) out_file.close () From parul.pandey85 at gmail.com Sat Jan 9 05:24:16 2010 From: parul.pandey85 at gmail.com (pp) Date: Sat, 9 Jan 2010 02:24:16 -0800 (PST) Subject: Porblem with xlutils/xlrd/xlwt Message-ID: <8e9245ec-05cb-48ed-ae32-f9fb0c46509f@j5g2000yqm.googlegroups.com> Whenever i run the code below I get the following error: AttributeError: 'Book' object has no attribute 'on_demand' WARNING: Failure executing file: Why is it so?? from xlrd import open_workbook from xlwt import easyxf from xlutils.copy import copy rb = open_workbook('source.xls',formatting_info=True) rs = rb.sheet_by_index(0) wb = copy(rb) ws = wb.get_sheet(0) plain = easyxf('') for i,cell in enumerate(rs.col(2)): if not i: continue ws.write(i,2,cell.value,plain) for i,cell in enumerate(rs.col(4)): if not i: continue ws.write(i,4,cell.value-1000) wb.save('output.xls') From martin at v.loewis.de Sat Jan 9 05:24:40 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Sat, 09 Jan 2010 11:24:40 +0100 Subject: Clarifications on compiling for Windows In-Reply-To: <1f0b409b-f943-4b1a-a806-87b834337367@p8g2000yqb.googlegroups.com> References: <1f0b409b-f943-4b1a-a806-87b834337367@p8g2000yqb.googlegroups.com> Message-ID: <4B485968.7020605@v.loewis.de> > First of all, is the Windows FAQ fairly up to date? Fairly, yes. > Should people be > referring to section 6 if they are going to build an application with > an embedded Python interpreter? I think that's very selective in its view of problems - why would I be using SWIG, for example? (yet there are three issues dedicated to SWIG in this section) pythonNN.dll is not in \windows\system, but in system32 or winsxs. pythonNN.lib is *not* a static library, but (as the text then notes) an import library. So even if you link with pythonNN.lib, you *still* need pythonNN.dll at run-time (what is discussed as a drawback of dynamic linking). Of course, it might be possible to build a static library out of Python (which then still might be called pythonNN.lib). IMO, it should be possible to link Python into the executable (at the expense of not supporting dynamic loading of extension modules anymore). > If I understand correctly, compiled extensions for Python on Windows > should match the compiler that was used to build the interpreter > itself? Is there a list somewhere that shows which version of msvc was > used to compile the recent Python binaries? See PCbuild/readme.txt of the respective Python release. 2.3: VC 6 2.4, 2.5: VC 7.1 / VS .NET 2003 2.6, 3.1: VC 9 / VS 2008 Regards, Martin From fetchinson at googlemail.com Sat Jan 9 05:26:38 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Sat, 9 Jan 2010 11:26:38 +0100 Subject: lightweight encryption of text file In-Reply-To: References: Message-ID: >>>>> I have a plain text file which I would like to protect in a very >>>>> simple minded, yet for my purposes sufficient, way. I'd like to >>>>> encrypt/convert it into a binary file in such a way that possession of >>>>> a password allows anyone to convert it back into the original text >>>>> file while not possessing the password one would only see the >>>>> following with the standard linux utility 'file': >>>>> >>>>> [fetchinson at fetch ~]$ file encrypted.data >>>>> encrypted.data: data >>>>> >>>>> and the effort required to convert the file back to the original text >>>>> file without the password would be equivalent to guessing the >>>>> password. >>>>> >>>>> I'm fully aware of the security implications of this loose >>>>> specification, but for my purposes this would be a good solution. >>>>> >>>>> What would be the simplest way to achieve this using preferably stock >>>>> python without 3rd party modules? If a not too complex 3rd party >>>>> module made it really simple that would be acceptable too. >>>> >>>> Paul Rubin's p3.py algorithm is probably the most straightforward way to >>>> meet >>>> these requirements. It's not a standard crypto algorithm by any means, >>>> but >>>> Paul >>>> knows his stuff and has devised it with these deployment restrictions in >>>> mind. >>>> >>>> http://www.nightsong.com/phr/crypto/p3.py >>> >>> Thanks a lot, currently I'm having trouble using this code on python >>> 2.6 but probably some small tweaking will fix it. >> >> Actually, it also doesn't work with python 2.5 and currently I don't >> have access to anything older. array.array raises a >> >> ValueError: string length not a multiple of item size >> >> Does anyone recall a change to array.array? >> >> The full traceback is >> >> Traceback (most recent call last): >> File "p3.py", line 163, in >> _test() >> File "p3.py", line 143, in _test >> c1 = e(plain,key) >> File "p3.py", line 69, in p3_encrypt >> xkey = _expand_key(k_enc, n+4) >> File "p3.py", line 41, in _expand_key >> return array ('L', j) >> ValueError: string length not a multiple of item size > > Are you on a 64-bit platform? Unfortunately, array's integer typecodes are > platform-specific, but p3.py requires a 32-bit integer and was written on a > 32-bit platform. It's reasonably straightforward to fix. Put this bit of > (untested) code at the top of the file and replace occurrences of 'L' with > uint32: > > # Find the typecode of a 32-bit unsigned integer. > for typecode in 'IL': > if len(array(typecode, [0]).tostring()) == 4: > uint32 = typecode > break > else: > raise RuntimeError("Neither 'I' nor 'L' are unsigned 32-bit integers.") Thanks! That was exactly the problem, I'm on a 64 bit machine and your fix seems to work indeed. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From joncle at googlemail.com Sat Jan 9 05:26:48 2010 From: joncle at googlemail.com (Jon Clements) Date: Sat, 9 Jan 2010 02:26:48 -0800 (PST) Subject: table from csv file References: <59653a09-c3f7-464f-a36f-475f0d759e46@e27g2000yqd.googlegroups.com> <77b18520-c2c4-4e21-9147-1164d81b9f25@l30g2000yqb.googlegroups.com> Message-ID: <8e1d52da-f6e8-4600-8afa-3cc9be7bee80@a21g2000yqc.googlegroups.com> On Jan 8, 8:31?pm, J wrote: > On Fri, Jan 8, 2010 at 13:55, Jon Clements wrote: > > On Jan 8, 5:59?pm, marlowe wrote: > >> I am trying to create a table in python from a csv file where I input > >> which columns I would like to see, and the table only shows those > >> columns. I have attached an example of the csv file i am using, and > >> some of the code I have written. I am having trouble converting > >> variables between lists, dictionaries and tuples. Is there a name for > >> what I am attempting to do? any help to get me on the right track with > >> this is appreciated. > > >> test.csv > > I had to edit that and comma delimit it, because cut and paste gave me > random numbers/types of whitespace... > > [code snipped] > > > > > This might be a useful starting point (I'm guessing this is what > > you're after...) > > > Let's assume your 'CSV' file is tab separated as it's certainly not > > comma separated :) > > > import csv > > csvin = csv.reader(open('test.csv'), delimiter='\t') > > header = dict( (val.strip(),idx) for idx, val in enumerate(next > > (csvin)) ) > > > We can use header as a column name->column index lookup eg header > > ['Open'] == 1 > > > from operator import itemgetter > > wanted = ['Open', 'Close'] # Although you'll want to use raw_input and > > split on ',' > > getcols = itemgetter(*[header[col] for col in wanted]) > > > getcols is a helper function that'll return a tuple of the columns in > > the requested order... > > > for row in csvin: > > ? ?print getcols(row) > > > Loop over the rest of the file and output the required columns. > > As someone who knows just enough to be dangerous... what about this: > > import csv > > reader = open('C:/test.txt','rb') > data = csv.DictReader(reader,restval='000',restkey='Misc') [snip] DictReader works, but what use to bug me was the fact you couldn't then output the cols in the 'correct' order afterwards, so you had to store the header row anyway to re-order the rows... (although admittedly this doesn't affect the OP's question). However, I see that 2.6+ offers .fieldnames on DictReader objects. Cheers, Jon. From joncle at googlemail.com Sat Jan 9 05:42:43 2010 From: joncle at googlemail.com (Jon Clements) Date: Sat, 9 Jan 2010 02:42:43 -0800 (PST) Subject: Porblem with xlutils/xlrd/xlwt References: <8e9245ec-05cb-48ed-ae32-f9fb0c46509f@j5g2000yqm.googlegroups.com> Message-ID: <972fdcfc-73a6-4e87-9752-3bc10da0dcad@s3g2000yqs.googlegroups.com> On Jan 9, 10:24?am, pp wrote: > Whenever i run the code below I get the following error: > > AttributeError: 'Book' object has no attribute 'on_demand' > WARNING: Failure executing file: > > Why is it so?? > > from xlrd import open_workbook > from xlwt import easyxf > from xlutils.copy import copy > rb = ?open_workbook('source.xls',formatting_info=True) > rs = ?rb.sheet_by_index(0) > wb = ?copy(rb) > ws = ?wb.get_sheet(0) > plain = easyxf('') > for i,cell in enumerate(rs.col(2)): > ? ? ?if not i: > ? ? ? ? ?continue > ? ? ?ws.write(i,2,cell.value,plain) > for i,cell in enumerate(rs.col(4)): > ? ? ?if not i: > ? ? ? ? ?continue > ? ? ?ws.write(i,4,cell.value-1000) > wb.save('output.xls') I suspect your version of xlrd is not up to date (although I thought on_demand was ages ago!). Make sure all the tools are the latest versions from http://www.python-excel.org There's also a dedicated Google Group for the xl* products listed on that page. hth Jon. From parul.pandey85 at gmail.com Sat Jan 9 05:44:28 2010 From: parul.pandey85 at gmail.com (pp) Date: Sat, 9 Jan 2010 02:44:28 -0800 (PST) Subject: Porblem with xlutils/xlrd/xlwt References: <8e9245ec-05cb-48ed-ae32-f9fb0c46509f@j5g2000yqm.googlegroups.com> <972fdcfc-73a6-4e87-9752-3bc10da0dcad@s3g2000yqs.googlegroups.com> Message-ID: <43a1e323-8ccd-48b7-a6a4-e540aca2b1cd@35g2000yqa.googlegroups.com> On Jan 9, 3:42?am, Jon Clements wrote: > On Jan 9, 10:24?am, pp wrote: > > > > > Whenever i run the code below I get the following error: > > > AttributeError: 'Book' object has no attribute 'on_demand' > > WARNING: Failure executing file: > > > Why is it so?? > > > from xlrd import open_workbook > > from xlwt import easyxf > > from xlutils.copy import copy > > rb = ?open_workbook('source.xls',formatting_info=True) > > rs = ?rb.sheet_by_index(0) > > wb = ?copy(rb) > > ws = ?wb.get_sheet(0) > > plain = easyxf('') > > for i,cell in enumerate(rs.col(2)): > > ? ? ?if not i: > > ? ? ? ? ?continue > > ? ? ?ws.write(i,2,cell.value,plain) > > for i,cell in enumerate(rs.col(4)): > > ? ? ?if not i: > > ? ? ? ? ?continue > > ? ? ?ws.write(i,4,cell.value-1000) > > wb.save('output.xls') > > I suspect your version of xlrd is not up to date (although I thought > on_demand was ages ago!). > Make sure all the tools are the latest versions fromhttp://www.python-excel.org > > There's also a dedicated Google Group for the xl* products listed on > that page. > > hth > Jon. yeah all my versions are latest from http://www.python-excel.org . just checked!! what could be the problem? From fetchinson at googlemail.com Sat Jan 9 05:45:14 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Sat, 9 Jan 2010 11:45:14 +0100 Subject: lightweight encryption of text file In-Reply-To: <0357cddd$0$1336$c3e8da3@news.astraweb.com> References: <0357cddd$0$1336$c3e8da3@news.astraweb.com> Message-ID: On 1/9/10, Steven D'Aprano wrote: > On Fri, 08 Jan 2010 20:14:51 +0100, Daniel Fetchinson wrote: > >> I have a plain text file which I would like to protect in a very simple >> minded, yet for my purposes sufficient, way. I'd like to encrypt/convert >> it into a binary file in such a way that possession of a password allows >> anyone to convert it back into the original text file while not >> possessing the password one would only see the following with the >> standard linux utility 'file': >> >> [fetchinson at fetch ~]$ file encrypted.data >> encrypted.data: data > > If that is your sole requirement, No, that was not my sole requirement, I also wrote: """ and the effort required to convert the file back to the original text file without the password would be equivalent to guessing the password. """ > then the addition of a single non-text > byte (say, a null) anywhere in the file would be sufficient to have file > identify it as data. Yes, but this would not satisfy the other requirement quoted above. One could read the file without an effort that is equivalent to guessing a random password. > You say "encrypt/convert" -- does this mean that you > don't care if people can read the text in a hex editor, so long as file > identifies it as data? I do care. See the quote above :) > Would something like a binary Vigenere Cipher be sufficient? > > > # Untested > def encrypt(plaintext, password): > cipher = [] > for i, c in enumerate(plaintext): > shift = password[i % len(password)] > shift = ord(shift) > cipher.append((ord(c) + shift) % 256) > return ''.join([chr(n) for n in cipher]) > > def decrypt(ciphertext, password): > plain = [] > for i, c in enumerate(ciphertext): > shift = password[i % len(password)] > shift = ord(shift) > plain.append((256 + ord(c) - shift) % 256) > return ''.join([chr(n) for n in plain]) Thanks, this looks simple enough and probably sufficient for my purposes! I'll see if I'll use this or Paul Rubin's p3.py. > Is it acceptable if there is a chance (small, possibly vanishingly small) > that file will identify it as text? As far as I know, even the > heavyweight "serious" encryption algorithms don't *guarantee* that the > cipher text will include non-text bytes. Hmmm, that's a good point, but actually it doesn't matter if 'file' identifies it as text with a very small probability. >> and the effort required to convert the file back to the original text >> file without the password would be equivalent to guessing the password. > > If you seriously mean that, then "lightweight encryption" won't do the > job, because it is vulnerable to frequency analysis, which is easier than > guessing the password. You need proper, heavy-weight encryption. Well, probably you are right and I should have been more clear. What typically people call obfuscation is sufficient for me, as long as the obfuscation involves a password, something that your solution seems to do. >> I'm fully aware of the security implications of this loose >> specification, but for my purposes this would be a good solution. > > Can you explain what your objection to real encryption is? Not much really, I simply don't want to overkill, that's all. First and foremost I wouldn't want to introduce a dependency on a 3rd party module and also wouldn't want to read documentation of a complex API when all I need are two functions: encrypt( text, password) and decrypt( text, password ). I really like your solution because that's all it does. > The problem is that, as I see it, you've assumed a solution rather than > state what your requirements are. I'm *guessing* that you are more > concerned of having to learn to use a complex API, Well, that's sort of true about learning a complex API :) But it's also true that I'm not storing anything really valuable in the file but still wouldn't want to leave it lying around in plain text. In case I lose the laptop with the file I seriously doubt anybody who finds it will go through each and every file and try to find what's in it, even though they look like data files and there is no hint what so ever that any one of them contains encrypted info. If they see a text file, well, that can give them ideas, so let's encrypt a little bit. So basically that's the story, granted, it's not a full specification or anything like that, it's a description of a vague situation but that's really all I have :) Cheers, Daniel From fetchinson at googlemail.com Sat Jan 9 05:49:38 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Sat, 9 Jan 2010 11:49:38 +0100 Subject: lightweight encryption of text file In-Reply-To: <4B4851C6.1050009@bluewin.ch> References: <4B4851C6.1050009@bluewin.ch> Message-ID: > > I have a plain text file which I would like to protect in a very > > simple minded, yet for my purposes sufficient, way. I'd like to > > encrypt/convert it into a binary file in such a way that possession of > > a password allows anyone to convert it back into the original text > > file while not possessing the password one would only see the > > following with the standard linux utility 'file': > > > > [fetchinson at fetch ~]$ file encrypted.data > > encrypted.data: data > > > > and the effort required to convert the file back to the original text > > file without the password would be equivalent to guessing the > > password. > > > > I'm fully aware of the security implications of this loose > > specification, but for my purposes this would be a good solution. > > > > What would be the simplest way to achieve this using preferably stock > > python without 3rd party modules? If a not too complex 3rd part > > module made it really simple that would be acceptable too. > > > > > Daniel, > > Here's what looks like another thread veering off into package-ology, > leaving a stumped OP behind. > > "Don't use a random generator for encryption purposes!" warns the > manual, of which fact I was reminded in no uncertain terms on this forum > a few years ago when I proposed the following little routine in response > to a post very similar to yours. One critic challenged me to encode my > credit card data and post it. Which I did. Upon which another critic > conjured up the horror vision of gigahertzes hacking my pathetic little > effort to pieces as I was reading his message. Of the well-meaning kind, > he urged me to put an immediate stop to this foolishness. I didn't. > > No unplanned expenditures ensued. > > Or to quote ... I forget who: Fools and innovators are people who don't > care much about what one is not supposed to do. > > So, take or leave what follows for what it is worth or not worth, I am > confident it works and would serve your purpose, which, as I understand, > is not to write a text book on cryptology. > > Regards > > Frederic > > > ############################## > > > import random > > > def crypt_string (string, key, floor = 0, size_of_set = 255): > > # key is a number. The sign of that number controls which way the > process > # goes. If the number is positive, the result is an encryption. A > negative > # number orders a decryption. > > if key == 0: return string # No processing > > import random > > MAX_CHUNK_SIZE = 32 > MIN_CHUNK_SIZE = 16 > > def process_sequence (sequence): > s = '' > for c in sequence: > r = random.randint (0, size_of_set - 1) > s += chr (((ord (c) - floor + r * sign) % size_of_set) + floor) > return s > > def shuffle_sequence (sequence): > random.shuffle (sequence) > return sequence > > sign = (key > 0) * 2 - 1 > random.seed (key * sign) > > s = '' > > if sign > 0: # forward > > i = 0 > while i < len (string): > random_size_of_chunk = random.randint (MIN_CHUNK_SIZE, > MAX_CHUNK_SIZE) > clear_chunk_shuffled = shuffle_sequence (list (string > [i:i+random_size_of_chunk])) > code_chunk_shuffled = process_sequence (clear_chunk_shuffled) > s += code_chunk_shuffled > i += len (code_chunk_shuffled) > > else: # backward > > i = 0 > while i < len (string): > random_size_of_chunk = random.randint (MIN_CHUNK_SIZE, > MAX_CHUNK_SIZE) > code_chunk_shuffled = list (string [i:i+random_size_of_chunk]) > real_size_of_chunk = len (code_chunk_shuffled) > unshuffling_template = shuffle_sequence (range > (real_size_of_chunk)) # 1. same ... > clear_chunk_shuffled = process_sequence > (code_chunk_shuffled) # 2. ... order > clear_chunk = real_size_of_chunk * [None] > for ii in range (real_size_of_chunk): > clear_chunk [unshuffling_template[ii]] = > clear_chunk_shuffled [ii] > s += ''.join (clear_chunk) > i += real_size_of_chunk > > return s > > > > def _crypt_file (in_file, out_file, key): > > BUFFER_SIZE = 1024 > while 1: > s = in_file.read (BUFFER_SIZE) > if s == '': break > out_file.write (crypt_string (s, key)) > > > def crypt_file (in_file_name, out_file_name, key): > > in_file = open (in_file_name, 'rb') > out_file = open (out_file_name, 'wb') > _crypt_file (in_file, out_file, key) > out_file.close () Thanks a lot! Your description is a good fit for my purposes, indeed I don't plan on writing a text book on encryption :) Also, I don't plan on encrypting credit card numbers either, all I need is that a file doesn't look obviously full of ascii characters but something that is generic data. And since it will not be an entire system, with lots of files of this type, only we are talking about a single file, there is no incentive to decipher my algorithm. I'll take a look at your code, thanks a lot, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From joncle at googlemail.com Sat Jan 9 05:52:56 2010 From: joncle at googlemail.com (Jon Clements) Date: Sat, 9 Jan 2010 02:52:56 -0800 (PST) Subject: Porblem with xlutils/xlrd/xlwt References: <8e9245ec-05cb-48ed-ae32-f9fb0c46509f@j5g2000yqm.googlegroups.com> <972fdcfc-73a6-4e87-9752-3bc10da0dcad@s3g2000yqs.googlegroups.com> <43a1e323-8ccd-48b7-a6a4-e540aca2b1cd@35g2000yqa.googlegroups.com> Message-ID: <5ac38519-9ad4-4a89-9b3f-ccd56fb18721@m3g2000yqf.googlegroups.com> On Jan 9, 10:44?am, pp wrote: > On Jan 9, 3:42?am, Jon Clements wrote: > > > > > On Jan 9, 10:24?am, pp wrote: > > > > Whenever i run the code below I get the following error: > > > > AttributeError: 'Book' object has no attribute 'on_demand' > > > WARNING: Failure executing file: > > > > Why is it so?? > > > > from xlrd import open_workbook > > > from xlwt import easyxf > > > from xlutils.copy import copy > > > rb = ?open_workbook('source.xls',formatting_info=True) > > > rs = ?rb.sheet_by_index(0) > > > wb = ?copy(rb) > > > ws = ?wb.get_sheet(0) > > > plain = easyxf('') > > > for i,cell in enumerate(rs.col(2)): > > > ? ? ?if not i: > > > ? ? ? ? ?continue > > > ? ? ?ws.write(i,2,cell.value,plain) > > > for i,cell in enumerate(rs.col(4)): > > > ? ? ?if not i: > > > ? ? ? ? ?continue > > > ? ? ?ws.write(i,4,cell.value-1000) > > > wb.save('output.xls') > > > I suspect your version of xlrd is not up to date (although I thought > > on_demand was ages ago!). > > Make sure all the tools are the latest versions fromhttp://www.python-excel.org > > > There's also a dedicated Google Group for the xl* products listed on > > that page. > > > hth > > Jon. > > yeah all my versions are latest fromhttp://www.python-excel.org. > just checked!! > what could be the problem? Does rb = xlrd.open_workbook('somesheet.xls', on_demand=True) work by itself? From parul.pandey85 at gmail.com Sat Jan 9 05:56:03 2010 From: parul.pandey85 at gmail.com (pp) Date: Sat, 9 Jan 2010 02:56:03 -0800 (PST) Subject: Porblem with xlutils/xlrd/xlwt References: <8e9245ec-05cb-48ed-ae32-f9fb0c46509f@j5g2000yqm.googlegroups.com> <972fdcfc-73a6-4e87-9752-3bc10da0dcad@s3g2000yqs.googlegroups.com> <43a1e323-8ccd-48b7-a6a4-e540aca2b1cd@35g2000yqa.googlegroups.com> <5ac38519-9ad4-4a89-9b3f-ccd56fb18721@m3g2000yqf.googlegroups.com> Message-ID: On Jan 9, 3:52?am, Jon Clements wrote: > On Jan 9, 10:44?am, pp wrote: > > > > > On Jan 9, 3:42?am, Jon Clements wrote: > > > > On Jan 9, 10:24?am, pp wrote: > > > > > Whenever i run the code below I get the following error: > > > > > AttributeError: 'Book' object has no attribute 'on_demand' > > > > WARNING: Failure executing file: > > > > > Why is it so?? > > > > > from xlrd import open_workbook > > > > from xlwt import easyxf > > > > from xlutils.copy import copy > > > > rb = ?open_workbook('source.xls',formatting_info=True) > > > > rs = ?rb.sheet_by_index(0) > > > > wb = ?copy(rb) > > > > ws = ?wb.get_sheet(0) > > > > plain = easyxf('') > > > > for i,cell in enumerate(rs.col(2)): > > > > ? ? ?if not i: > > > > ? ? ? ? ?continue > > > > ? ? ?ws.write(i,2,cell.value,plain) > > > > for i,cell in enumerate(rs.col(4)): > > > > ? ? ?if not i: > > > > ? ? ? ? ?continue > > > > ? ? ?ws.write(i,4,cell.value-1000) > > > > wb.save('output.xls') > > > > I suspect your version of xlrd is not up to date (although I thought > > > on_demand was ages ago!). > > > Make sure all the tools are the latest versions fromhttp://www.python-excel.org > > > > There's also a dedicated Google Group for the xl* products listed on > > > that page. > > > > hth > > > Jon. > > > yeah all my versions are latest fromhttp://www.python-excel.org. > > just checked!! > > what could be the problem? > > Does rb = xlrd.open_workbook('somesheet.xls', on_demand=True) work by > itself? Yes it does. The problem is with line: wb = copy(rb) here I am getting the error: AttributeError: 'Book' object has no attribute 'on_demand' Thanks .. From davea at ieee.org Sat Jan 9 05:56:36 2010 From: davea at ieee.org (Dave Angel) Date: Sat, 09 Jan 2010 05:56:36 -0500 Subject: Link to module Stack In-Reply-To: <0358384b$0$1336$c3e8da3@news.astraweb.com> References: <0358384b$0$1336$c3e8da3@news.astraweb.com> Message-ID: <4B4860E4.2090705@ieee.org> Steven D'Aprano wrote: > On Sat, 09 Jan 2010 01:07:39 -0800, kzagradskiy wrote: > > >> class Stack: >> def __init__(self): >> self.__heap = [] >> > > A "heap" has a technical meaning in programming. To describe the > internals of a stack as "heap" will be disconcerting and confusing to > anyone who knows about stacks and heaps. > > > >> def push (self, word): >> self.__heap.append (word) >> def pop (self): >> if len(self.__heap) == 0: >> raise InnerInterpreterError, "stack underflow" >> > > "InnerInterpreterError" is the most inappropriate exception name I've > ever seen. It has nothing to do with the interpreter, it's a stack error. > > It has everything to do with the (Forth) interpreter. Exceptions can readily be named according to their application -- it's not always about Python. Anyway, Forth has an inner-interpreter and an outer-interpreter, and the name will make sense to a Forth programmer. >> result = self.__heap[-1] >> del self.__heap[-1] >> > > That is better written as result = self.__heap.pop(). > > > or even better, without the extra local var: def pop (self): if len(self.__heap) == 0: raise InnerInterpreterError, "stack underflow" return self.__heap.pop(1) P.S. - I'm puzzled why the OP even put this message here. There's no question posted with it. DaveA From davea at ieee.org Sat Jan 9 06:05:05 2010 From: davea at ieee.org (Dave Angel) Date: Sat, 09 Jan 2010 06:05:05 -0500 Subject: lightweight encryption of text file In-Reply-To: <4B4851C6.1050009@bluewin.ch> References: <4B4851C6.1050009@bluewin.ch> Message-ID: <4B4862E1.6080406@ieee.org> Anthra Norell wrote: >
Daniel > Fetchinson wrote: > > I have a plain text file which I would like to protect in a very > > simple minded, yet for my purposes sufficient, way. I'd like to > > encrypt/convert it into a binary file in such a way that possession of > > a password allows anyone to convert it back into the original text > > file while not possessing the password one would only see the > > following with the standard linux utility 'file': > > > > [fetchinson at fetch ~]$ file encrypted.data > > encrypted.data: data > > > > and the effort required to convert the file back to the original text > > file without the password would be equivalent to guessing the > > password. > > > > I'm fully aware of the security implications of this loose > > specification, but for my purposes this would be a good solution. > > > > What would be the simplest way to achieve this using preferably stock > > python without 3rd party modules? If a not too complex 3rd part > > module made it really simple that would be acceptable too. > > > > > Daniel, > > Here's what looks like another thread veering off into package-ology, > leaving a stumped OP behind. > > "Don't use a random generator for encryption purposes!" warns the > manual, of which fact I was reminded in no uncertain terms on this > forum a few years ago when I proposed the following little routine in > response to a post very similar to yours. One critic challenged me to > encode my credit card data and post it. Which I did. Upon which > another critic conjured up the horror vision of gigahertzes hacking my > pathetic little effort to pieces as I was reading his message. Of the > well-meaning kind, he urged me to put an immediate stop to this > foolishness. I didn't. > > No unplanned expenditures ensued. > > Or to quote ... I forget who: Fools and innovators are people who > don't care much about what one is not supposed to do. > > So, take or leave what follows for what it is worth or not worth, I am > confident it works and would serve your purpose, which, as I > understand, is not to write a text book on cryptology. > > Regards > > Frederic > > The problem I'd have with this approach (not studied in detail), is that there's no reason that next year's Python must use the same random number generator, or the same shuffle algorithm. So in order to assure that "encrypted" archives will be recoverable, one should store with them the CPython implementation, in source form, just in case that's needed to reconstruct things. DaveA From rdmoores at gmail.com Sat Jan 9 06:31:49 2010 From: rdmoores at gmail.com (Richard D. Moores) Date: Sat, 9 Jan 2010 03:31:49 -0800 Subject: How to get many places of pi from Machin's Equation? Message-ID: Machin's Equation is 4 arctan (1/5) - arctan(1/239) = pi/4 Using Python 3.1 and the math module: >>> from math import atan, pi >>> pi 3.141592653589793 >>> (4*atan(.2) - atan(1/239))*4 3.1415926535897936 >>> (4*atan(.2) - atan(1/239))*4 == pi False >>> abs((4*atan(.2) - atan(1/239))*4) - pi < .000000000000000001 False >>> abs((4*atan(.2) - atan(1/239))*4) - pi < .0000000000000001 False >>> abs((4*atan(.2) - atan(1/239))*4) - pi < .000000000000001 True >>> Is there a way in Python 3.1 to calculate pi to greater accuracy using Machin's Equation? Even to an arbitrary number of places? Thanks, Dick Moores From duncan.booth at invalid.invalid Sat Jan 9 06:40:42 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 9 Jan 2010 11:40:42 GMT Subject: Link to module Stack References: <0358384b$0$1336$c3e8da3@news.astraweb.com> Message-ID: Dave Angel wrote: > or even better, without the extra local var: > > def pop (self): > if len(self.__heap) == 0: > raise InnerInterpreterError, "stack underflow" > return self.__heap.pop(1) pop(1)? Anyway if would be simpler and almost certainly faster to not bother checking before the pop: def pop(self): try: return self.__heap.pop() except IndexError: raise InnerInterpreterError, "stack underflow" and if performance mattered the OP might even consider pre-binding the pop method in __init__: self.__pop = self.__heap.pop but that's probably premature optimisation. > P.S. - I'm puzzled why the OP even put this message here. There's no > question posted with it. Me too. It's a repost of something from 2004. Bizarre. From as at sci.fi Sat Jan 9 06:55:30 2010 From: as at sci.fi (Anssi Saari) Date: Sat, 09 Jan 2010 13:55:30 +0200 Subject: Accessing python from a network share in windows 7 References: Message-ID: aj writes: > I access python from a network share. This works fine on XP but on > windows 7 it throws the following error: > > Python 2.6.1 (r261:67517, Dec 4 2008, 16:51:00) [MSC v.1500 32 bit > (Intel)] on > win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> import random > Traceback (most recent call last): > File "", line 1, in > File "t:\win32\python-2.6.1\lib\random.py", line 871, in > _inst = Random() > File "t:\win32\python-2.6.1\lib\random.py", line 96, in __init__ > self.seed(x) > File "t:\win32\python-2.6.1\lib\random.py", line 110, in seed > a = long(_hexlify(_urandom(16)), 16) > WindowsError: [Error 127] The specified procedure could not be found > > Is there some security policy that I need to enable/disable to use > python from a network on windows 7? Well, there was just a complaint about this sort of thing on a local newsgroup here. Specifically, someone was trying to execute a Windows program from a share and every time Windows 7 pops up a warning window saying that the program is maybe from the evil interwebby and are you really sure you actually want to run it. So probably that's the command line version of same. Solution is apparently specifying your server to be in the local intranet, in IE's security settings. Apparently there is a non-working autodetection for what is a local intranet, so specifying the server IP address by hand in the advanced settigns is the working solution. From steve at holdenweb.com Sat Jan 9 07:50:48 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 09 Jan 2010 07:50:48 -0500 Subject: Standardized interpreter speed evaluation tool In-Reply-To: <50697b2c1001081602m7614b94esc4cd9241fafbdf49@mail.gmail.com> References: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> <50697b2c1001081602m7614b94esc4cd9241fafbdf49@mail.gmail.com> Message-ID: <4B487BA8.8070108@holdenweb.com> Chris Rebert wrote: > On Fri, Jan 8, 2010 at 2:25 AM, alexru wrote: >> Is there any standardized interpreter speed evaluation tool? Say I >> made few changes in interpreter code and want to know if those changes >> made python any better, which test should I use? > > Although apparently undocumented, test.pystone is some sort of > interpreter benchmark. > It's undocumented because it's not considered a representative benchmark.Sure, you can use it to get *some* idea of relative performance, but a single program is a very poor tool for such a complex topic a comparing implementations of a dynamic language. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Sat Jan 9 07:50:48 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 09 Jan 2010 07:50:48 -0500 Subject: Standardized interpreter speed evaluation tool In-Reply-To: <50697b2c1001081602m7614b94esc4cd9241fafbdf49@mail.gmail.com> References: <1d79a508-e94e-413f-80b0-c904c2fa86ec@m16g2000yqc.googlegroups.com> <50697b2c1001081602m7614b94esc4cd9241fafbdf49@mail.gmail.com> Message-ID: <4B487BA8.8070108@holdenweb.com> Chris Rebert wrote: > On Fri, Jan 8, 2010 at 2:25 AM, alexru wrote: >> Is there any standardized interpreter speed evaluation tool? Say I >> made few changes in interpreter code and want to know if those changes >> made python any better, which test should I use? > > Although apparently undocumented, test.pystone is some sort of > interpreter benchmark. > It's undocumented because it's not considered a representative benchmark.Sure, you can use it to get *some* idea of relative performance, but a single program is a very poor tool for such a complex topic a comparing implementations of a dynamic language. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From arnodel at googlemail.com Sat Jan 9 07:54:11 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sat, 09 Jan 2010 12:54:11 +0000 Subject: lightweight encryption of text file References: Message-ID: Daniel Fetchinson writes: > I have a plain text file which I would like to protect in a very > simple minded, yet for my purposes sufficient, way. I'd like to > encrypt/convert it into a binary file in such a way that possession of > a password allows anyone to convert it back into the original text > file while not possessing the password one would only see the > following with the standard linux utility 'file': > > [fetchinson at fetch ~]$ file encrypted.data > encrypted.data: data [...] This is probably not what you want, but it is very simple and doesn't import any module:) I am not qualified to say how easy it is to discover the message without the password. def str2int(txt): return reduce(lambda n, c: n*255 + ord(c), txt, 0) def int2str(n): chars = [] while n: n, o = divmod(n, 255) chars.append(chr(o)) return ''.join(reversed(chars)) def encrypt(txt, pwd): return int2str(str2int(txt)*str2int(pwd)) def decrypt(txt, pwd): return int2str(str2int(txt)/str2int(pwd)) def test(txt, pwd): encrypted_txt = encrypt(txt, pwd) decrypted_txt = decrypt(encrypted_txt, pwd) print "text:%r" % txt print "encrypted:%r" % encrypted_txt print "decrypted:%r" % decrypted_txt >>> test("This encryption scheme is definitely unbreakable.", "secret") text:'This encryption scheme is definitely unbreakable.' encrypted:'&2\xa5\xd4\x17i+E\x01k\xfa\x94\xf80\xa8\x8f\xea.w\x128\xf1\xd9\x0f9\xf2t\xc9\r`\x90%\xd6\xf3~\x1f\x00%u&\x8a\xe4\xe0\xa7\xb8\xb0ec)S>\xcb\xf2>\xec' decrypted:'This encryption scheme is definitely unbreakable.' -- Arnaud From steve at holdenweb.com Sat Jan 9 07:59:01 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 09 Jan 2010 07:59:01 -0500 Subject: Append to an Excel file In-Reply-To: <74a95a2f-90b7-48e6-9b69-c7250b979978@l30g2000yqb.googlegroups.com> References: <313cb2b5-4100-4d4d-bb38-dbfe3ca71e17@j5g2000yqm.googlegroups.com> <74a95a2f-90b7-48e6-9b69-c7250b979978@l30g2000yqb.googlegroups.com> Message-ID: pp wrote: > On Jan 9, 1:47 am, Jason Scheirer wrote: >> On Jan 9, 12:30 am, pp wrote: >> >>> Hi All, >>> How do I add a line to an existing file. This should append to the >>> existing data in the excel file, which was saved previously. >>> Thanks, >>> PP >> http://pypi.python.org/pypi/xlwt > > Hi Jason and all, > > Thanks > > I have seen this.. my question is there a way to append to a excel > file which has been closed. Any specific modes which can be added to > the sheet so that it adds a line to the data which was return in some > earlier running of the program. > If you are talking about an XLS file and not a CSV then it's a highly structured object, and you can't just "stick bits on the end" with any expectation that Excel will know what to do with the new data. The most likely outcomes would be Excel either complaining the file format is invalid or ignoring the extra data. If it's a CSV file, then f = open(file, "a") should do it. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From peter.bienstman at gmail.com Sat Jan 9 08:19:00 2010 From: peter.bienstman at gmail.com (pbienst) Date: Sat, 9 Jan 2010 05:19:00 -0800 (PST) Subject: creating tar file and streaming it over HTTP? References: <5222d923-7aef-47cf-a0d6-f37f09a66593@35g2000yqa.googlegroups.com> Message-ID: <0995c098-8976-46ae-98f0-977fe5e09a39@22g2000yqr.googlegroups.com> OK, thanks to the feedback from everyone I got the PUT from a client to the WSGI server working. I'm now trying to go the other way around: use a tar stream in one of the functions in the WSGI server in order to send files to the client. Problem is that the WSGI specs expects an iterator as return value for streaming, whereas TarFile needs to write to a file obj. Is there any way I can get these two to work together? Peter From victorsubervi at gmail.com Sat Jan 9 08:23:06 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 9 Jan 2010 08:23:06 -0500 Subject: Something More Elegant Message-ID: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> Hi; The following code works fine. I would like you to suggest something more simple and elegant: sql = 'select p.ID from %sPackages p join %sCategoriesPackages c where c.CategoryID=%s;' % (store, store, categoryID) cursor.execute(sql) tmp = [itm[0] for itm in cursor] packageIDs = [] for t in tmp: if t not in packageIDs: packageIDs.append(t) TIA, beno -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From iurisilvio at gmail.com Sat Jan 9 08:30:17 2010 From: iurisilvio at gmail.com (Iuri) Date: Sat, 9 Jan 2010 11:30:17 -0200 Subject: Something More Elegant In-Reply-To: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> Message-ID: <789aac5a1001090530w7d5d2039h137402c0f9e0f339@mail.gmail.com> Your code select some ids from database and list distinct ids in packageIDs. You can use SELECT DISTINCT in your SQL statement. On Sat, Jan 9, 2010 at 11:23 AM, Victor Subervi wrote: > Hi; > The following code works fine. I would like you to suggest something more > simple and elegant: > > sql = 'select p.ID from %sPackages p join %sCategoriesPackages c > where c.CategoryID=%s;' % (store, store, categoryID) > cursor.execute(sql) > tmp = [itm[0] for itm in cursor] > packageIDs = [] > for t in tmp: > if t not in packageIDs: > packageIDs.append(t) > > TIA, > beno > > -- > The Logos has come to bear > http://logos.13gems.com/ > > -- > http://mail.python.org/mailman/listinfo/python-list > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From python.list at tim.thechases.com Sat Jan 9 08:39:11 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Sat, 09 Jan 2010 07:39:11 -0600 Subject: Something More Elegant In-Reply-To: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> Message-ID: <4B4886FF.2030503@tim.thechases.com> Victor Subervi wrote: > Hi; > The following code works fine. I would like you to suggest something more > simple and elegant: > > sql = 'select p.ID from %sPackages p join %sCategoriesPackages c where > c.CategoryID=%s;' % (store, store, categoryID) > cursor.execute(sql) > tmp = [itm[0] for itm in cursor] > packageIDs = [] > for t in tmp: > if t not in packageIDs: > packageIDs.append(t) You mean like sql = "select distinct p.ID from ..." % (...) # ^^^^^^^^ cursor.execute(sql) package_ids = [row[0] for row in cursor.fetchall()] It would also help if you didn't pass the categoryID as a string-formatted value, but as a proper parameter, something like sql = "... where c.categoryid=?" % (store, store) cursor.execute(sql, (category_id,)) This helps prevent SQL-injection attacks (assuming you have full control over the value of "store"...otherwise, as you've been advised, if the remote user has control over the value in "store", you're asking to be exploited). You'd have to check the place-holder character for your particular back-end: >>> import as db >>> print db.paramstyle should tell you whether to use "?", "%s", or some other notation. From victorsubervi at gmail.com Sat Jan 9 09:01:25 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 9 Jan 2010 09:01:25 -0500 Subject: Something More Elegant In-Reply-To: <4B4886FF.2030503@tim.thechases.com> References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> Message-ID: <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> On Sat, Jan 9, 2010 at 8:39 AM, Tim Chase wrote: > Victor Subervi wrote: > >> Hi; >> The following code works fine. I would like you to suggest something more >> simple and elegant: >> >> sql = 'select p.ID from %sPackages p join %sCategoriesPackages c >> where >> c.CategoryID=%s;' % (store, store, categoryID) >> cursor.execute(sql) >> tmp = [itm[0] for itm in cursor] >> packageIDs = [] >> for t in tmp: >> if t not in packageIDs: >> packageIDs.append(t) >> > > You mean like > > sql = "select distinct p.ID from ..." % (...) > Oh, that's good! > # ^^^^^^^^ > cursor.execute(sql) > package_ids = [row[0] for row in cursor.fetchall()] > > It would also help if you didn't pass the categoryID as a string-formatted > value, but as a proper parameter, something like > > sql = "... where c.categoryid=?" % (store, store) > cursor.execute(sql, (category_id,)) > I now have the following: sql = 'select distinct p.ID from %sPackages p join %sCategoriesPackages c where c.CategoryID=?;' % (store, store) cursor.execute(sql, (categoryID,)) packageIDs = [itm[0] for itm in cursor] It threw this error: /var/www/html/angrynates.com/christians/cart/display.py 141 print '\n' 142 cursor.close() 143 bottom() 144 145 display() display = /var/www/html/angrynates.com/christians/cart/display.py in display() 109 categoryID = cursor.fetchone()[0] 110 sql = 'select distinct p.ID from %sPackages p join %sCategoriesPackages c where c.CategoryID=?;' % (store, store) 111 cursor.execute(sql, (categoryID,)) 112 packageIDs = [itm[0] for itm in cursor] 113 for pid in packageIDs: global cursor = , cursor.execute = >, sql = 'select distinct p.ID from productsPackages p join productsCategoriesPackages c where c.CategoryID=?;', categoryID = 1L /usr/lib64/python2.4/site-packages/MySQLdb/cursors.py in execute(self=, query='select distinct p.ID from productsPackages p join productsCategoriesPackages c where c.CategoryID=?;', args=(1L,)) 146 query = query.encode(charset) 147 if args is not None: 148 query = query % db.literal(args) 149 try: 150 r = self._query(query) query = 'select distinct p.ID from productsPackages p join productsCategoriesPackages c where c.CategoryID=?;', db = , db.literal = >, args = (1L,) TypeError: not all arguments converted during string formatting args = ('not all arguments converted during string formatting',) > This helps prevent SQL-injection attacks (assuming you have full control > over the value of "store"...otherwise, as you've been advised, if the remote > user has control over the value in "store", you're asking to be exploited). > They have control over it. I pass it in the url. Please advise. > You'd have to check the place-holder character for your particular > back-end: > > >>> import as db > >>> print db.paramstyle > > Printed "format". What's that mean? I use MySQLdb TIA, beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcd at sdf.lonestar.org Sat Jan 9 09:07:02 2010 From: jcd at sdf.lonestar.org (J. Cliff Dyer) Date: Sat, 09 Jan 2010 09:07:02 -0500 Subject: Another Screwy Problem In-Reply-To: <4dc0cfea1001090459g618727b9j6ef90fadda6b4464@mail.gmail.com> References: <4dc0cfea1001081132p191fe045p72a79744054fc520@mail.gmail.com> <20100108214455.GA12839@sdf.lonestar.org> <4dc0cfea1001090459g618727b9j6ef90fadda6b4464@mail.gmail.com> Message-ID: <1263046022.2685.2.camel@webb> On Sat, 2010-01-09 at 07:59 -0500, Victor Subervi wrote: > On Fri, Jan 8, 2010 at 4:44 PM, J. Clifford Dyer > wrote: > Victor Subervi wrote: > > Hi; > > I have this line of code: > > sql = 'select Name, Price from %sPackages where ID=%s;' % > (store, pid) > > which prints to this: > > select Name, Price from productsPackages where ID=1; > > which when I enter it into the MySQL interpreter gives me > this: > > mysql> select Name, Price from productsPackages where ID=1; > > +------+--------+ > > | Name | Price | > > +------+--------+ > > | pkg | 123.45 | > > +------+--------+ > > 1 row in set (0.00 sec) > > > > exactly what I expect. However, in my script for some reason > it returns > > this: > > ((1,),) > > > > First, got your other email. I thought I had executed the statement. > Oops. Works fine now. Sorry. > > First, never use string formatting to pass parameters to your > database. Read the MySQLdb documentation (or sqlite, or > psycopg2) documentation for reasons why, and how to do it > right. > > The only thing I found, which collaborates with something someone else > taught me on this list about entering binary data, is that one must > pass the parameters in the execute statement. Is that what you mean? > If so, I find that for all purposes thus far other than binary data, > the way I've been doing it seems to work just fine. I would prefer to > keep doing it that way, because I find putting a print statement > between the sql= line and the execute statement gives me a good > opportunity to review the sql statement and catch errors. Is this not > good practice? > > > Thanks. > beno This is a horrendous practice. You leave yourself vulnerable not only to attacks, but to simple absent-mindedness as well. Using parameters in your execute statement will handle all necessary quoting for you, which eliminates the possibility of a bad query sneaking in. For more information, as I mentioned, look up SQL injection. Also, read this: http://xkcd.com/327/ Cheers, Cliff From sjmachin at lexicon.net Sat Jan 9 09:12:36 2010 From: sjmachin at lexicon.net (John Machin) Date: Sat, 9 Jan 2010 06:12:36 -0800 (PST) Subject: How to get many places of pi from Machin's Equation? References: Message-ID: <60828494-e7aa-4a99-9c33-7d3a525afe4b@22g2000yqr.googlegroups.com> On Jan 9, 10:31?pm, "Richard D. Moores" wrote: > Machin's Equation is > > 4 arctan (1/5) - arctan(1/239) = pi/4 > > Using Python 3.1 and the math module: > > > > >>> from math import atan, pi > >>> pi > 3.141592653589793 > >>> (4*atan(.2) - atan(1/239))*4 > 3.1415926535897936 > >>> (4*atan(.2) - atan(1/239))*4 == pi > False > >>> abs((4*atan(.2) - atan(1/239))*4) - pi < .000000000000000001 > False > >>> abs((4*atan(.2) - atan(1/239))*4) - pi < .0000000000000001 > False > >>> abs((4*atan(.2) - atan(1/239))*4) - pi < .000000000000001 > True > > Is there a way in Python 3.1 to calculate pi to greater accuracy using > Machin's Equation? Even to an arbitrary number of places? Considering that my namesake calculated pi to 100 decimal places with the computational equipment available in 1706 (i.e. not much), I'd bet you London to a brick that Python (any version from 0.1 onwards) could be used to simulate his calculations to any reasonable number of places. So my answers to your questions are yes and yes. Suggestion: search_the_fantastic_web("machin pi python") From victorsubervi at gmail.com Sat Jan 9 09:14:47 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 9 Jan 2010 09:14:47 -0500 Subject: Another Screwy Problem In-Reply-To: <1263046022.2685.2.camel@webb> References: <4dc0cfea1001081132p191fe045p72a79744054fc520@mail.gmail.com> <20100108214455.GA12839@sdf.lonestar.org> <4dc0cfea1001090459g618727b9j6ef90fadda6b4464@mail.gmail.com> <1263046022.2685.2.camel@webb> Message-ID: <4dc0cfea1001090614g27ec483cv21b718f8a1776c1b@mail.gmail.com> On Sat, Jan 9, 2010 at 9:07 AM, J. Cliff Dyer wrote: > On Sat, 2010-01-09 at 07:59 -0500, Victor Subervi wrote: > > On Fri, Jan 8, 2010 at 4:44 PM, J. Clifford Dyer > > wrote: > > Victor Subervi wrote: > > > Hi; > > > I have this line of code: > > > sql = 'select Name, Price from %sPackages where ID=%s;' % > > (store, pid) > > > which prints to this: > > > select Name, Price from productsPackages where ID=1; > > > which when I enter it into the MySQL interpreter gives me > > this: > > > mysql> select Name, Price from productsPackages where ID=1; > > > +------+--------+ > > > | Name | Price | > > > +------+--------+ > > > | pkg | 123.45 | > > > +------+--------+ > > > 1 row in set (0.00 sec) > > > > > > exactly what I expect. However, in my script for some reason > > it returns > > > this: > > > ((1,),) > > > > > > > > First, got your other email. I thought I had executed the statement. > > Oops. Works fine now. Sorry. > > > > First, never use string formatting to pass parameters to your > > database. Read the MySQLdb documentation (or sqlite, or > > psycopg2) documentation for reasons why, and how to do it > > right. > > > > The only thing I found, which collaborates with something someone else > > taught me on this list about entering binary data, is that one must > > pass the parameters in the execute statement. Is that what you mean? > > If so, I find that for all purposes thus far other than binary data, > > the way I've been doing it seems to work just fine. I would prefer to > > keep doing it that way, because I find putting a print statement > > between the sql= line and the execute statement gives me a good > > opportunity to review the sql statement and catch errors. Is this not > > good practice? > > > > > > Thanks. > > beno > > This is a horrendous practice. You leave yourself vulnerable not only > to attacks, but to simple absent-mindedness as well. Using parameters > in your execute statement will handle all necessary quoting for you, > which eliminates the possibility of a bad query sneaking in. For more > information, as I mentioned, look up SQL injection. Also, read this: > http://xkcd.com/327/ > > Thanks :) beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at holdenweb.com Sat Jan 9 09:35:38 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 09 Jan 2010 09:35:38 -0500 Subject: Something More Elegant In-Reply-To: <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> Message-ID: Victor Subervi wrote: > On Sat, Jan 9, 2010 at 8:39 AM, Tim Chase > wrote: > > Victor Subervi wrote: > > Hi; > The following code works fine. I would like you to suggest > something more > simple and elegant: > > sql = 'select p.ID from %sPackages p join > %sCategoriesPackages c where > c.CategoryID=%s;' % (store, store, categoryID) > cursor.execute(sql) > tmp = [itm[0] for itm in cursor] > packageIDs = [] > for t in tmp: > if t not in packageIDs: > packageIDs.append(t) > > > You mean like > > sql = "select distinct p.ID from ..." % (...) > > > Oh, that's good! > > > # ^^^^^^^^ > cursor.execute(sql) > package_ids = [row[0] for row in cursor.fetchall()] > > It would also help if you didn't pass the categoryID as a > string-formatted value, but as a proper parameter, something like > > sql = "... where c.categoryid=?" % (store, store) > cursor.execute(sql, (category_id,)) > > > I now have the following: > > sql = 'select distinct p.ID from %sPackages p join > %sCategoriesPackages c where c.CategoryID=?;' % (store, store) > cursor.execute(sql, (categoryID,)) > packageIDs = [itm[0] for itm in cursor] > > It threw this error: > > /var/www/html/angrynates.com/christians/cart/display.py > > 141 print '\n' > 142 cursor.close() > 143 bottom() > 144 > 145 display() > display = > /var/www/html/angrynates.com/christians/cart/display.py > in display() > 109 categoryID = cursor.fetchone()[0] > 110 sql = 'select distinct p.ID from %sPackages p join > %sCategoriesPackages c where c.CategoryID=?;' % (store, store) > 111 cursor.execute(sql, (categoryID,)) > 112 packageIDs = [itm[0] for itm in cursor] > 113 for pid in packageIDs: > global cursor = , cursor.execute = method Cursor.execute of >, sql = 'select > distinct p.ID from productsPackages p join productsCategoriesPackages c > where c.CategoryID=?;', categoryID = 1L > /usr/lib64/python2.4/site-packages/MySQLdb/cursors.py in > execute(self=, query='select distinct > p.ID from productsPackages p join productsCategoriesPackages c where > c.CategoryID=?;', args=(1L,)) > 146 query = query.encode(charset) > 147 if args is not None: > 148 query = query % db.literal(args) > 149 try: > 150 r = self._query(query) > query = 'select distinct p.ID from productsPackages p join > productsCategoriesPackages c where c.CategoryID=?;', db = 0x2b79db9dc470 to Connection>, db.literal = Connection.literal of <_mysql.connection open to 'localhost' at > 142be8b0>>, args = (1L,) > > TypeError: not all arguments converted during string formatting > args = ('not all arguments converted during string formatting',) > > > This helps prevent SQL-injection attacks (assuming you have full > control over the value of "store"...otherwise, as you've been > advised, if the remote user has control over the value in "store", > you're asking to be exploited). > > > They have control over it. I pass it in the url. Please advise. > > > You'd have to check the place-holder character for your particular > back-end: > > >>> import as db > >>> print db.paramstyle > > Printed "format". What's that mean? I use MySQLdb > TIA, > beno > Given that you actually started this thread by asking a good question that showed you had done some independent work, I'll bite. The problem is something that was discussed in one of your other numerous threads by John Machin and me. The issue is the parameterization of (i.e. sticking variable bits into) SQL queries. When you write curs.execute("some sql query with %s and %s in it", (data1, data2)) the second argument to execute is supposed to contain data values. This allows the SQL engine to do the preparatory work for a query once, and then use the same "prepared query" then next time it's executed. The preparation involves scanning the SQL query to make sure the syntax is correct, validating the table and column names, and developing a "query execution plan" that is a sequence of internal operations the database performs to get you the answer you want. (SQL, unlike Python, is a "declarative" language - rather than telling it what to do you describe the results you want to see and the engine works out how to provide it). Of course, if different tables are used for different queries then there is no hope that the same execution plan can be used for them. For this reason most database processors (and this certainly includes the one you are using) don't allow you to parameterize anything in SQL statement other than data values. So curs.execute("select field1 from tableA where id=%s", (3, )) is OK, but curs.execute("select field1 from tableA where %s=3", ("id", )) is definitely not. And curs.execute("select field1 from %stable where id=3", (storename, )) is right out. This goes back to the issue of your database design. You have chosen to represent each store with its own set of tables, where an experienced database designer would instead have used just one set of tables, each table having a "store" column that would allow you to use parameterized queries to select the relevant data. There are many other sound reasons for making such a design choice, which primarily boil down to operational and management efficiency and simplicity. But we are now in the realm of theory as far as you are concerned, since you have already stated several times that you aren't interested in correcting your design until after you have got the current mess into production. So good luck with that. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From gagsl-py2 at yahoo.com.ar Sat Jan 9 09:42:16 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Sat, 09 Jan 2010 11:42:16 -0300 Subject: How to get many places of pi from Machin's Equation? References: Message-ID: En Sat, 09 Jan 2010 08:31:49 -0300, Richard D. Moores escribi?: > Is there a way in Python 3.1 to calculate pi to greater accuracy using > Machin's Equation? Even to an arbitrary number of places? You may be interested in Demo/scripts/pi.py in the source distribution. It can generate pi with infinite precision, limited by available memory only. And this thread from last month: http://groups.google.com/group/comp.lang.python/t/e37bb8c59f2e5582/ -- Gabriel Genellina From gagsl-py2 at yahoo.com.ar Sat Jan 9 10:01:47 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Sat, 09 Jan 2010 12:01:47 -0300 Subject: Something More Elegant References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> Message-ID: En Sat, 09 Jan 2010 11:01:25 -0300, Victor Subervi escribi?: > On Sat, Jan 9, 2010 at 8:39 AM, Tim Chase > wrote: > >> It would also help if you didn't pass the categoryID as a >> string-formatted >> value, but as a proper parameter, something like >> >> sql = "... where c.categoryid=?" % (store, store) >> cursor.execute(sql, (category_id,)) >> > > I now have the following: > > sql = 'select distinct p.ID from %sPackages p join > %sCategoriesPackages c where c.CategoryID=?;' % (store, store) > cursor.execute(sql, (categoryID,)) > packageIDs = [itm[0] for itm in cursor] > > It threw this error: > > TypeError: not all arguments converted during string formatting > args = ('not all arguments converted during string formatting',) > >> You'd have to check the place-holder character for your particular >> back-end: >> >> >>> import as db >> >>> print db.paramstyle >> >> Printed "format". What's that mean? I use MySQLdb That means, MySQLdb uses %s as a placeholder for parameter substitution -- same as Python when doing string interpolation. Unfortunately this will confuse things. In your code above, the ? near the end should become %s -- but you don't want THAT %s to be interpreted by Python at that time, instead it must remain as a literal %s until the cursor.execute line. You have to escape the % by doubling it: %%s sql = 'select distinct p.ID from %sPackages p join %sCategoriesPackages c where c.CategoryID=%%s;' % (store, store) cursor.execute(sql, (categoryID,)) packageIDs = [itm[0] for itm in cursor] -- Gabriel Genellina From vmail at mycircuit.org Sat Jan 9 10:09:32 2010 From: vmail at mycircuit.org (Peter) Date: Sat, 09 Jan 2010 16:09:32 +0100 Subject: restructuredText editor ? In-Reply-To: <5e6l17-28k.ln1@mid.florian-diesch.de> References: <5e6l17-28k.ln1@mid.florian-diesch.de> Message-ID: <4B489C2C.5030409@mycircuit.org> On 01/09/2010 03:32 AM, Florian Diesch wrote: > Peter writes: > > >> What editor do people out there use to edit .rst files for >> sphinx-python documentation ? >> > Emacs with ReST mode and YASnippet > > > Florian > Great, works very well and thanks for mentionning YASnippets ( useful for many types of documents ) Peter From 3lvss0809 at gmail.com Sat Jan 9 10:12:18 2010 From: 3lvss0809 at gmail.com (3lvss0809 at gmail.com) Date: Sat, 9 Jan 2010 07:12:18 -0800 (PST) Subject: Microsoft Office Word and Python (Win XP) Message-ID: Hi. Im very new with python. I have got some answer on my issue to use interop or COM ''plugins'' to access MS Word through python but i don't even know what those two ''plugins'' are so I cannot use them. What I want to do is the following: I need the script that moves (only moves, not change or delete!) entire (100% of the text) text from one .doc file to another. But its not so easy as it sounds. The target .doc file is not the only one but can be many of them. All the target .doc files are always in the same folder (same path) but all of them don't have the same name. The .doc file FROM where I want to move entire text is only one, always in the same folder (same path) and always with the same file name. Names of the target are only similar but as I have said before, not the same. Here is the point of whole script: Target .doc files have the names: HD1.doc HD2.doc HD3.doc HD4.doc and so on What I would like to have is moved the entire (but really all of the text, must be 100% all) text into the .doc file with the highest ( ! ) number. The target .doc files will always start with ''HD'' and always be similar to above examples. It is possible that the doc file (target file) is only one, so only HD1.doc. Therefore ''1'' is the maximum number and the text is moved into this file. Sometimes the target file is empty but usually won't be. If it won't be then the text should be moved to the end of the text, into first new line (no empty lines inbetween). So for example in the target file which has the maximum number in its name is the following text: a b c In the file from which I want to move the text is: d This means I need in the target file this: a b c d Could someone tell me please how to do this? Thank you. From 3lvss0809 at gmail.com Sat Jan 9 10:12:18 2010 From: 3lvss0809 at gmail.com (3lvss0809 at gmail.com) Date: Sat, 9 Jan 2010 07:12:18 -0800 (PST) Subject: Microsoft Office Word and Python (Win XP) Message-ID: Hi. Im very new with python. I have got some answer on my issue to use interop or COM ''plugins'' to access MS Word through python but i don't even know what those two ''plugins'' are so I cannot use them. What I want to do is the following: I need the script that moves (only moves, not change or delete!) entire (100% of the text) text from one .doc file to another. But its not so easy as it sounds. The target .doc file is not the only one but can be many of them. All the target .doc files are always in the same folder (same path) but all of them don't have the same name. The .doc file FROM where I want to move entire text is only one, always in the same folder (same path) and always with the same file name. Names of the target are only similar but as I have said before, not the same. Here is the point of whole script: Target .doc files have the names: HD1.doc HD2.doc HD3.doc HD4.doc and so on What I would like to have is moved the entire (but really all of the text, must be 100% all) text into the .doc file with the highest ( ! ) number. The target .doc files will always start with ''HD'' and always be similar to above examples. It is possible that the doc file (target file) is only one, so only HD1.doc. Therefore ''1'' is the maximum number and the text is moved into this file. Sometimes the target file is empty but usually won't be. If it won't be then the text should be moved to the end of the text, into first new line (no empty lines inbetween). So for example in the target file which has the maximum number in its name is the following text: a b c In the file from which I want to move the text is: d This means I need in the target file this: a b c d Could someone tell me please how to do this? Thank you. From iurisilvio at gmail.com Sat Jan 9 10:14:39 2010 From: iurisilvio at gmail.com (Iuri) Date: Sat, 9 Jan 2010 13:14:39 -0200 Subject: Something More Elegant In-Reply-To: References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> Message-ID: <789aac5a1001090714q1453984cp8754d27fd5be6102@mail.gmail.com> And you should use cursor.fetchall() instead of cursor in list comprehension: packageIDs = [itm[0] for itm in cursor.fetchall()] On Sat, Jan 9, 2010 at 1:01 PM, Gabriel Genellina wrote: > En Sat, 09 Jan 2010 11:01:25 -0300, Victor Subervi < > victorsubervi at gmail.com> escribi?: > >> On Sat, Jan 9, 2010 at 8:39 AM, Tim Chase > >wrote: >> >> It would also help if you didn't pass the categoryID as a >>> string-formatted >>> value, but as a proper parameter, something like >>> >>> sql = "... where c.categoryid=?" % (store, store) >>> cursor.execute(sql, (category_id,)) >>> >>> >> I now have the following: >> >> sql = 'select distinct p.ID from %sPackages p join >> %sCategoriesPackages c where c.CategoryID=?;' % (store, store) >> cursor.execute(sql, (categoryID,)) >> packageIDs = [itm[0] for itm in cursor] >> >> It threw this error: >> >> TypeError: not all arguments converted during string formatting >> args = ('not all arguments converted during string formatting',) >> >> You'd have to check the place-holder character for your particular >>> back-end: >>> >>> >>> import as db >>> >>> print db.paramstyle >>> >>> Printed "format". What's that mean? I use MySQLdb >>> >> > That means, MySQLdb uses %s as a placeholder for parameter substitution -- > same as Python when doing string interpolation. Unfortunately this will > confuse things. In your code above, the ? near the end should become %s -- > but you don't want THAT %s to be interpreted by Python at that time, instead > it must remain as a literal %s until the cursor.execute line. You have to > escape the % by doubling it: %%s > > sql = 'select distinct p.ID from %sPackages p join > %sCategoriesPackages c where c.CategoryID=%%s;' % (store, store) > > cursor.execute(sql, (categoryID,)) > packageIDs = [itm[0] for itm in cursor] > > -- > Gabriel Genellina > > > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Sat Jan 9 10:15:48 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 9 Jan 2010 10:15:48 -0500 Subject: Something More Elegant In-Reply-To: References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> Message-ID: <4dc0cfea1001090715md97518tf618309eda1deabb@mail.gmail.com> On Sat, Jan 9, 2010 at 9:35 AM, Steve Holden wrote: > But we are now in the realm of theory as far as you are concerned, since > you have already stated several times that you aren't interested in > correcting your design until after you have got the current mess into > production. So good luck with that. > And if you were in my shoes, I'm sure you'd do the same thing. Well, it *is* working now :)) And I am interested in cleaning this up. I should probably start with the matter of databases, since that's something I won't be able to easily change once clients actually start entering data. Please share with me any further concepts or questions to get me thinking how to redesign the databases. TIA, beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From sjmachin at lexicon.net Sat Jan 9 10:23:31 2010 From: sjmachin at lexicon.net (John Machin) Date: Sat, 9 Jan 2010 07:23:31 -0800 (PST) Subject: Porblem with xlutils/xlrd/xlwt References: <8e9245ec-05cb-48ed-ae32-f9fb0c46509f@j5g2000yqm.googlegroups.com> <972fdcfc-73a6-4e87-9752-3bc10da0dcad@s3g2000yqs.googlegroups.com> <43a1e323-8ccd-48b7-a6a4-e540aca2b1cd@35g2000yqa.googlegroups.com> <5ac38519-9ad4-4a89-9b3f-ccd56fb18721@m3g2000yqf.googlegroups.com> Message-ID: On Jan 9, 9:56?pm, pp wrote: > On Jan 9, 3:52?am, Jon Clements wrote: > > > > > On Jan 9, 10:44?am, pp wrote: > > > > On Jan 9, 3:42?am, Jon Clements wrote: > > > > > On Jan 9, 10:24?am, pp wrote: > > > yeah all my versions are latest fromhttp://www.python-excel.org. > > > just checked!! How did you check? > > > what could be the problem? > > > Does rb = xlrd.open_workbook('somesheet.xls', on_demand=True) work by > > itself? > > Yes it does. The problem is with line: wb = ?copy(rb) > here I am getting the error: AttributeError: 'Book' object has no > attribute 'on_demand' Please replace the first 4 lines of your script by these 6 lines: import xlrd assert xlrd.__VERSION__ == "0.7.1" from xlwt import easyxf from xlutils.copy import copy rb = xlrd.open_workbook( 'source.xls',formatting_info=True, on_demand=False) and run it again. Please copy all the output and paste it into your response. From victorsubervi at gmail.com Sat Jan 9 10:28:31 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 9 Jan 2010 10:28:31 -0500 Subject: Something More Elegant In-Reply-To: <789aac5a1001090714q1453984cp8754d27fd5be6102@mail.gmail.com> References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> <789aac5a1001090714q1453984cp8754d27fd5be6102@mail.gmail.com> Message-ID: <4dc0cfea1001090728g7997b969pcf5a59070dc83591@mail.gmail.com> On Sat, Jan 9, 2010 at 10:14 AM, Iuri wrote: > And you should use cursor.fetchall() instead of cursor in list > comprehension: > > packageIDs = [itm[0] for itm in cursor.fetchall()] > Now, someone else on this list told me the other. Can you explain the difference? TIA, beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From dickinsm at gmail.com Sat Jan 9 10:31:18 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 9 Jan 2010 07:31:18 -0800 (PST) Subject: How to get many places of pi from Machin's Equation? References: Message-ID: <3a3b9062-e01c-4115-a0e9-3b1b271e6c12@v25g2000yqk.googlegroups.com> On Jan 9, 11:31?am, "Richard D. Moores" wrote: > Is there a way in Python 3.1 to calculate pi to greater accuracy using > Machin's Equation? Even to an arbitrary number of places? There's no arbitrary-precision version of atan included with Python. You could write your own (e.g., based on argument reduction + Taylor series) for use with the decimal module, or you could use one of the various 3rd party arbitrary-precision arithmetic packages that do provide atan. Mark From mccolgst at gmail.com Sat Jan 9 10:37:48 2010 From: mccolgst at gmail.com (McColgst) Date: Sat, 9 Jan 2010 07:37:48 -0800 (PST) Subject: Scripting (was Re: Python books, literature etc) References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: <741365c6-d955-40ec-bbe9-104c8c524534@k19g2000yqc.googlegroups.com> Just to kind of get back on topic: Before buying a book or making a terribly large investment, OP should consider the fact that Python 3 is out and gaining some popularity. From dickinsm at gmail.com Sat Jan 9 10:57:05 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 9 Jan 2010 07:57:05 -0800 (PST) Subject: How to get many places of pi from Machin's Equation? References: Message-ID: On Jan 9, 11:31?am, "Richard D. Moores" wrote: > Machin's Equation is > > 4 arctan (1/5) - arctan(1/239) = pi/4 > [...] > > Is there a way in Python 3.1 to calculate pi to greater accuracy using > Machin's Equation? Even to an arbitrary number of places? Here's some crude code (no error bounds, possibility of infinite loops, ...) that computes pi to 1000 places using Machin's formula and the decimal module. The last few digits will be bogus, and should be ignored. from decimal import Decimal, getcontext def atan(x): # reductions reductions = 0 while 100*abs(x) > 1: reductions += 1 x /= 1 + (1+x*x).sqrt() # Taylor series sum = 0 xpow = x x2 = x*x k = 1 while True: term = xpow/k oldsum = sum sum += term if sum == oldsum: break k += 2 xpow *= -x2 return sum * 2**reductions getcontext().prec = 1000 one = Decimal(1) print(16*atan(one/5) - 4*atan(one/239)) From victorsubervi at gmail.com Sat Jan 9 11:11:28 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 9 Jan 2010 11:11:28 -0500 Subject: Easy Q Message-ID: <4dc0cfea1001090811y198462a8s39976a0622ef6c59@mail.gmail.com> Hi; I have a string.join statement on a variable that comes from a cgi.FieldStorage().getlist. The variable may be a list or a single value. I need to treat it differently depending on which it is. How can I distinguish it? len(var) will obviously give me the length of the string if it's a string and the length of the list if it's a list. TIA, beno -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at REMOVE-THIS-cybersource.com.au Sat Jan 9 11:19:30 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 09 Jan 2010 16:19:30 GMT Subject: Link to module Stack References: <0358384b$0$1336$c3e8da3@news.astraweb.com> Message-ID: <03589968$0$1336$c3e8da3@news.astraweb.com> On Sat, 09 Jan 2010 05:56:36 -0500, Dave Angel wrote: >> "InnerInterpreterError" is the most inappropriate exception name I've >> ever seen. It has nothing to do with the interpreter, it's a stack >> error. >> >> > It has everything to do with the (Forth) interpreter. Exceptions can > readily be named according to their application -- it's not always about > Python. Anyway, Forth has an inner-interpreter and an > outer-interpreter, and the name will make sense to a Forth programmer. Pardon me, but I *am* a Forth programmer. Or was, it's been many years, and I'm rusty. I guess this is a difference of terminology: what you're calling an inner interpreter and an outer interpreter I know of as the Forth engine and the (text) interpreter. Gforth refers to them as such, so did Leo Brodie's Forth books, and the (ancient) Macintosh Forth compiler "Mach 2". But in any case... a stack is an general-purpose data structure, and the error message shouldn't be coupled so tightly to one use. That would be like this (made-up) example: >>> 1/0 Traceback (most recent call last): File "", line 1, in GraphicsApplicationError: too few pixels to calculate average Ridiculous, yes? Yes, Forth uses a stack (technically two, a parameter stack and a return stack, and some implementations include a third, floating point, stack). Virtually all languages use stacks in their implementation, and Python byte-code is also stack-based. >>> result = self.__heap[-1] >>> del self.__heap[-1] >>> >>> >> That is better written as result = self.__heap.pop(). >> >> >> > or even better, without the extra local var: > > def pop (self): > if len(self.__heap) == 0: > raise InnerInterpreterError, "stack underflow" > return self.__heap.pop(1) pop(1)? I don't think so. >>> L = list('abcdef') >>> L.pop(1) 'b' >>> L ['a', 'c', 'd', 'e', 'f'] You probably meant pop(-1), but that's unnecessary because pop defaults to popping from the end of the list. > P.S. - I'm puzzled why the OP even put this message here. There's no > question posted with it. Me too. -- Steven From mail.to.daniel.platz at googlemail.com Sat Jan 9 11:30:07 2010 From: mail.to.daniel.platz at googlemail.com (Daniel Platz) Date: Sat, 9 Jan 2010 08:30:07 -0800 (PST) Subject: Manipulating pointers in C using ctypes References: <08bda34e-9bee-44b1-b2de-80d647151786@a15g2000yqm.googlegroups.com> Message-ID: <78689d52-b729-4bc4-8c6b-34f415c6c869@u41g2000yqe.googlegroups.com> Thanks for valuable answers. Both solutions work and I understand my mistake. Best regards, Daniel From rdmoores at gmail.com Sat Jan 9 11:32:48 2010 From: rdmoores at gmail.com (Richard D. Moores) Date: Sat, 9 Jan 2010 08:32:48 -0800 Subject: How to get many places of pi from Machin's Equation? In-Reply-To: References: Message-ID: On Sat, Jan 9, 2010 at 07:57, Mark Dickinson wrote: > On Jan 9, 11:31?am, "Richard D. Moores" wrote: >> Machin's Equation is >> >> 4 arctan (1/5) - arctan(1/239) = pi/4 >> [...] >> >> Is there a way in Python 3.1 to calculate pi to greater accuracy using >> Machin's Equation? Even to an arbitrary number of places? > > Here's some crude code (no error bounds, ?possibility of infinite > loops, ...) that computes pi to 1000 places using Machin's formula and > the decimal module. ?The last few digits will be bogus, and should be > ignored. > > from decimal import Decimal, getcontext > > def atan(x): > ? ?# reductions > ? ?reductions = 0 > ? ?while 100*abs(x) > 1: > ? ? ? ?reductions += 1 > ? ? ? ?x /= 1 + (1+x*x).sqrt() > > ? ?# Taylor series > ? ?sum = 0 > ? ?xpow = x > ? ?x2 = x*x > ? ?k = 1 > ? ?while True: > ? ? ? ?term = xpow/k > ? ? ? ?oldsum = sum > ? ? ? ?sum += term > ? ? ? ?if sum == oldsum: > ? ? ? ? ? ?break > ? ? ? ?k += 2 > ? ? ? ?xpow *= -x2 > > ? ?return sum * 2**reductions > > getcontext().prec = 1000 > one = Decimal(1) > print(16*atan(one/5) - 4*atan(one/239)) > -- > http://mail.python.org/mailman/listinfo/python-list > Great! Done in Python 3 with arctan and the decimal module. And the first 997 digits were accurate. Just what I was after. I don't believe the Chudnovsky algorithm has been mentioned. It isn't what I wanted, but it is amazing. () Thanks, everyone! Dick Moores From dreadpiratejeff at gmail.com Sat Jan 9 11:34:29 2010 From: dreadpiratejeff at gmail.com (J) Date: Sat, 9 Jan 2010 11:34:29 -0500 Subject: table from csv file In-Reply-To: <8e1d52da-f6e8-4600-8afa-3cc9be7bee80@a21g2000yqc.googlegroups.com> References: <59653a09-c3f7-464f-a36f-475f0d759e46@e27g2000yqd.googlegroups.com> <77b18520-c2c4-4e21-9147-1164d81b9f25@l30g2000yqb.googlegroups.com> <8e1d52da-f6e8-4600-8afa-3cc9be7bee80@a21g2000yqc.googlegroups.com> Message-ID: <36dec4ff1001090834q756aa826u66486a8535484861@mail.gmail.com> On Sat, Jan 9, 2010 at 05:26, Jon Clements wrote: >> reader = open('C:/test.txt','rb') >> data = csv.DictReader(reader,restval='000',restkey='Misc') > > [snip] > > DictReader works, but what use to bug me was the fact you couldn't > then output the cols in the 'correct' order afterwards, so you had > to store the header row anyway to re-order the rows... > (although admittedly this doesn't affect the OP's question). > > However, I see that 2.6+ offers .fieldnames on DictReader objects. At a guess, and I stipulate that because I am just learning about all this, I'd say that's because DictReader returns a dictionary for each row retrieved... It DOES take the first row and make those the dict keys, but after that, it just returns a Dict for each rows and dictionaries seem to be randomly ordered. It may not be that great if you're doing something that needs order, but all he's doing is pulling arbitrary columns from a file and printing as a table, so that works great in this case... It was quite interesting messing around with that yesterday. I saw the question, and it looked interesting, so I went and learned and applied ;-) I'm rather smugly satisfied with myself ... Cheers, Jeff -- Samuel Goldwyn - "I'm willing to admit that I may not always be right, but I am never wrong." - http://www.brainyquote.com/quotes/authors/s/samuel_goldwyn.html From python at mrabarnett.plus.com Sat Jan 9 11:48:45 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 09 Jan 2010 16:48:45 +0000 Subject: Easy Q In-Reply-To: <4dc0cfea1001090811y198462a8s39976a0622ef6c59@mail.gmail.com> References: <4dc0cfea1001090811y198462a8s39976a0622ef6c59@mail.gmail.com> Message-ID: <4B48B36D.3000402@mrabarnett.plus.com> Victor Subervi wrote: > Hi; > I have a string.join statement on a variable that comes from a > cgi.FieldStorage().getlist. The variable may be a list or a single > value. I need to treat it differently depending on which it is. How can > I distinguish it? len(var) will obviously give me the length of the > string if it's a string and the length of the list if it's a list. > 1. string.join isn't a statement, it's a function, and why are you using it anyway? Strings have a .join method: >>> ", ".join(["first", "second", "third"]) 'first, second, third' 2. Are you sure cgi.FieldStorage().getlist sometimes returns a single value instead of a list? If that's the case then it's a very odd name for the method! From gherron at islandtraining.com Sat Jan 9 11:56:12 2010 From: gherron at islandtraining.com (Gary Herron) Date: Sat, 09 Jan 2010 08:56:12 -0800 Subject: Easy Q In-Reply-To: <4dc0cfea1001090811y198462a8s39976a0622ef6c59@mail.gmail.com> References: <4dc0cfea1001090811y198462a8s39976a0622ef6c59@mail.gmail.com> Message-ID: <4B48B52C.3020409@islandtraining.com> Victor Subervi wrote: > Hi; > I have a string.join statement on a variable that comes from a > cgi.FieldStorage().getlist. The variable may be a list or a single > value. I need to treat it differently depending on which it is. How > can I distinguish it? len(var) will obviously give me the length of > the string if it's a string and the length of the list if it's a list. > TIA, > beno Like this: if isinstance(var, list): ... join ... else: ... ??? ... Gary Herron From chambon.pascal at gmail.com Sat Jan 9 12:25:06 2010 From: chambon.pascal at gmail.com (Pakal) Date: Sat, 9 Jan 2010 09:25:06 -0800 (PST) Subject: Direct use of bytearray buffers with ctypes ? Message-ID: <9694d6f5-5189-4371-a933-f7c5da5b2d30@e27g2000yqd.googlegroups.com> Hello I'm having big trouble wrapping the win32 ReadFile() function with ctypes. I wanted to allow the user to give a bytearray to this function, so that the writable buffer of this bytearray is directly used to receive the data from the file ; thus, no temporary copy in a separate ctype buffer would be required. However, I've found no way to provide ReadFile with a pointer to the inner buffer of the bytearray object. I get misc. TypeErrors and ValueErrors all the time, by trying to use the from_buffer() method or other ctypes functions. Is that actually posisble to expose with ctypes the internals of a python object to the system, or is that unsupported at the moment ? From benjamin at python.org Sat Jan 9 12:29:33 2010 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 9 Jan 2010 11:29:33 -0600 Subject: [RELEASED] Python 2.7 alpha 2 Message-ID: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> On behalf of the Python development team, I'm gleeful to announce the second alpha release of Python 2.7. Python 2.7 is scheduled to be the last major version in the 2.x series. It includes many features that were first released in Python 3.1. The faster io module, the new nested with statement syntax, improved float repr, and the memoryview object have been backported from 3.1. Other features include an ordered dictionary implementation, unittests improvements, and support for ttk Tile in Tkinter. For a more extensive list of changes in 2.7, see http://doc.python.org/dev/whatsnew/2.7.html or Misc/NEWS in the Python distribution. To download Python 2.7 visit: http://www.python.org/download/releases/2.7/ Please note that this is a development release, intended as a preview of new features for the community, and is thus not suitable for production use. The 2.7 documentation can be found at: http://docs.python.org/2.7 Please consider trying Python 2.7 with your code and reporting any bugs you may notice to: http://bugs.python.org Have fun! -- Benjamin Peterson 2.7 Release Manager benjamin at python.org (on behalf of the entire python-dev team and 2.7's contributors) From nawijn at gmail.com Sat Jan 9 12:58:48 2010 From: nawijn at gmail.com (Marco Nawijn) Date: Sat, 9 Jan 2010 09:58:48 -0800 (PST) Subject: Microsoft Office Word and Python (Win XP) References: Message-ID: <0c53a831-98f8-4e17-8aff-75759d3789ed@m25g2000yqc.googlegroups.com> On Jan 9, 4:12?pm, "3lvss0... at gmail.com" <3lvss0... at gmail.com> wrote: > Hi. > Im very new with python. I have got some answer on my issue to use > interop or COM ''plugins'' to access MS Word through python but i > don't even know what those two ''plugins'' are so I cannot use them. > What I want to do is the following: > > I need the script that moves (only moves, not change or delete!) > entire (100% of the text) text from one .doc file to another. But its > not so easy as it sounds. The target .doc file is not the only one but > can be many of them. All the target .doc files are always in the same > folder (same path) but all of them don't have the same name. The .doc > file FROM where I want to move entire text is only one, always in the > same folder (same path) and always with the same file name. > Names of the target are only similar but as I have said before, not > the same. Here is the point of whole script: > Target .doc files have the names: > HD1.doc > HD2.doc > HD3.doc > HD4.doc > and so on > > What I would like to have is moved the entire (but really all of the > text, must be 100% all) text into the .doc file with the highest ( ! ) > number. The target .doc files will always start with ''HD'' and always > be similar to above examples. > It is possible that the doc file (target file) is only one, so only > HD1.doc. Therefore ''1'' is the maximum number and the text is moved > into this file. > Sometimes the target file is empty but usually won't be. If it won't > be then the text should be moved to the end of the text, into first > new line (no empty lines inbetween). > So for example in the target file which has the maximum number in its > name is the following text: > > a > b > c > > In the file from which I want to move the text is: > > d > > This means I need in the target file this: > > a > b > c > d > > Could someone tell me please how to do this? > > Thank you. Hi, I will try to head you in the right direction with the Python/MS.Word link. First of all, you need to install the win32 extension. See http://sourceforge.net/projects/pywin32/ Once you have this installed you can instantiate a MS.Word application like this (code untested): >>> from win32com.client import Dispatch >>> app = Dispatch("Word.Application") >>> app.Visible = True The code so-far is more or less equivalent to opening Word without opening a document (normally Word will start with an empty document). To open a document do something like the following. >>> doc = app.Documents.Open("c:\\example.doc") Further builtin Python modules that could be helpfull are: glob -> for searching files matching a pattern os, os.path -> for path related functionality like stripping directory names from a complete path Take a look at the online documentation for more information http://docs.python.org/modindex.html Good luck and let us know the result. Marco From tjreedy at udel.edu Sat Jan 9 13:00:07 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 09 Jan 2010 13:00:07 -0500 Subject: Microsoft Office Word and Python (Win XP) In-Reply-To: References: Message-ID: On 1/9/2010 10:12 AM, 3lvss0809 at gmail.com wrote: > I need the script that moves (only moves, not change or delete!) > entire (100% of the text) text from one .doc file to another. If you want to copy files without modification, use the OS copy command. You can use the subprocess module to do that from Python. From davea at ieee.org Sat Jan 9 13:02:22 2010 From: davea at ieee.org (Dave Angel) Date: Sat, 09 Jan 2010 13:02:22 -0500 Subject: Easy Q In-Reply-To: <4dc0cfea1001090811y198462a8s39976a0622ef6c59@mail.gmail.com> References: <4dc0cfea1001090811y198462a8s39976a0622ef6c59@mail.gmail.com> Message-ID: <4B48C4AE.2060100@ieee.org> Victor Subervi wrote: > Hi; > I have a string.join statement on a variable that comes from a > cgi.FieldStorage().getlist. The variable may be a list or a single value. I > need to treat it differently depending on which it is. How can I distinguish > it? len(var) will obviously give me the length of the string if it's a > string and the length of the list if it's a list. > TIA, > beno > > If you have to do an explicit type check of your variable, use an if statement, and the isinstance() built-in function. Note that you can check for multiple types, like list and tuple, in the same function call. DaveA From Dave Sat Jan 9 13:06:53 2010 From: Dave (Dave) Date: Sat, 09 Jan 2010 12:06:53 -0600 Subject: interactive terminal in Ubuntu Linux : libreadline5-dev works only in Python 2.6 not 3.1 Message-ID: On Jan 6 I inquired how to "fix" the 3.1.1 interactive terminal in Ubuntu Linux. Left arrow yields ^[[D , etc. casevh helped by suggesting "libreadline5-dev" be installed. Did so with Synaptic Package Manager. The behavior of the Python 3.3.1 terminal is unchanged but the 2.6.2 terminal is corrected. Is there any way to fix the 3.1.1 terminal command line ? Thanks, Dave WB3DWE pdlemper at earthlink.net From victorsubervi at gmail.com Sat Jan 9 13:18:26 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 9 Jan 2010 13:18:26 -0500 Subject: Easy Q In-Reply-To: <4B48B52C.3020409@islandtraining.com> References: <4dc0cfea1001090811y198462a8s39976a0622ef6c59@mail.gmail.com> <4B48B52C.3020409@islandtraining.com> Message-ID: <4dc0cfea1001091018t726f283g381f815723e20f94@mail.gmail.com> On Sat, Jan 9, 2010 at 11:56 AM, Gary Herron wrote: > Victor Subervi wrote: > >> Hi; >> I have a string.join statement on a variable that comes from a >> cgi.FieldStorage().getlist. The variable may be a list or a single value. I >> need to treat it differently depending on which it is. How can I distinguish >> it? len(var) will obviously give me the length of the string if it's a >> string and the length of the list if it's a list. >> TIA, >> beno >> > > Like this: > > if isinstance(var, list): > ... join ... > else: > ... ??? ... > Right.. Thanks! beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Sat Jan 9 13:22:10 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sat, 9 Jan 2010 13:22:10 -0500 Subject: Something More Elegant In-Reply-To: References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> <789aac5a1001090714q1453984cp8754d27fd5be6102@mail.gmail.com> <4dc0cfea1001090728g7997b969pcf5a59070dc83591@mail.gmail.com> Message-ID: <4dc0cfea1001091022n57f901c7gead57d0d9b31ebec@mail.gmail.com> On Sat, Jan 9, 2010 at 1:00 PM, Dennis Lee Bieber wrote: > On Sat, 9 Jan 2010 10:28:31 -0500, Victor Subervi > declaimed the following in > gmane.comp.python.general: > > > On Sat, Jan 9, 2010 at 10:14 AM, Iuri wrote: > > > > > And you should use cursor.fetchall() instead of cursor in list > > > comprehension: > > > > > > packageIDs = [itm[0] for itm in cursor.fetchall()] > > > > > > > Now, someone else on this list told me the other. Can you explain the > > difference? > > Since a (one-liner) list comprehension is going to result in > processing all the data, it may be faster to fetch all the data at the > start. Depending upon the implementation of the database API, iterating > over a cursor object to retrieve one record at a time may result in a > hit on the database engine (some engines may hold the data in the server > and only transmit it record by record unless explicitly asked for all of > it -- and if the database server is on a different machine that could > make for a lot of slow network traffic). > > Also, you need to take the database locking system into account -- > until you not only read all the data, but commit the transaction (even a > read-only transaction), the engine may lock any other user from > accessing those records (and some engines may lock the entire table, not > just records). > > Iterating over a cursor may be useful if: 1) you have a really > massive database and your query is not filtering the data to manageable > levels (better done by looping over the query itself using limit and > offset features to control how many records are returned in a batch); 2) > it is a single user/single access database where locking data won't have > an impact; 3) you need to use the data in each record to update or > otherwise perform some other query on the database (though unless this > is done within the same transaction you might still have locking > issues). > > Wow! Thanks. I'll digest this over the weekend. beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From fetchinson at googlemail.com Sat Jan 9 14:01:33 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Sat, 9 Jan 2010 20:01:33 +0100 Subject: lightweight encryption of text file In-Reply-To: References: Message-ID: >> I have a plain text file which I would like to protect in a very >> simple minded, yet for my purposes sufficient, way. I'd like to >> encrypt/convert it into a binary file in such a way that possession of >> a password allows anyone to convert it back into the original text >> file while not possessing the password one would only see the >> following with the standard linux utility 'file': >> >> [fetchinson at fetch ~]$ file encrypted.data >> encrypted.data: data > [...] > > This is probably not what you want, but it is very simple and doesn't > import any module:) I am not qualified to say how easy it is to discover > the message without the password. > > def str2int(txt): > return reduce(lambda n, c: n*255 + ord(c), txt, 0) > > def int2str(n): > chars = [] > while n: > n, o = divmod(n, 255) > chars.append(chr(o)) > return ''.join(reversed(chars)) > > def encrypt(txt, pwd): > return int2str(str2int(txt)*str2int(pwd)) > > def decrypt(txt, pwd): > return int2str(str2int(txt)/str2int(pwd)) > > def test(txt, pwd): > encrypted_txt = encrypt(txt, pwd) > decrypted_txt = decrypt(encrypted_txt, pwd) > print "text:%r" % txt > print "encrypted:%r" % encrypted_txt > print "decrypted:%r" % decrypted_txt > > >>>> test("This encryption scheme is definitely unbreakable.", "secret") > text:'This encryption scheme is definitely unbreakable.' > encrypted:'&2\xa5\xd4\x17i+E\x01k\xfa\x94\xf80\xa8\x8f\xea.w\x128\xf1\xd9\x0f9\xf2t\xc9\r`\x90%\xd6\xf3~\x1f\x00%u&\x8a\xe4\xe0\xa7\xb8\xb0ec)S>\xcb\xf2>\xec' > decrypted:'This encryption scheme is definitely unbreakable.' Thanks, this looks pretty simple too, I will go with either Steven's or with your solution. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From 3lvss0809 at gmail.com Sat Jan 9 14:18:12 2010 From: 3lvss0809 at gmail.com (3lvss0809 at gmail.com) Date: Sat, 9 Jan 2010 11:18:12 -0800 (PST) Subject: Microsoft Office Word and Python (Win XP) References: Message-ID: Marco Nawijn: I have had installed pywin32 already. The three lines that you mentoined don't do this, also what did you mean with "doc = app.Documents.Open("c:\\example.doc")". Which document should I open with this line? It shouldn't be opened anything. I was asking about the script as automated process. If you know how could i do this? Terry Reedy: I have never mentoined copying files but moving the whole text from, always the same (same name, same path), .doc file with. However copying (=moving) text to correct .doc file would be good yes. I know command prompt and its copy function but this way it wouldn't work because I would have to define the target file - the file INTO which I want to move the text. But I will never know the file name (target file). The only thing I know is: - the file is one of .doc files that start with "HD" - whole name of those .doc file is always HDX.doc where X is a number and I need to move the text into the file with maximum X (the most high number) - all the HD files will be always in the same path (same folder) but I would like to use the path inside the code (it might be obvious that I have to) because on PC and laptop, I have two different usernames and since HD files are located inside Documents And Settings, I have to use two copies of the script - one for PC, one for laptop. Dennis Lee Bieber: Im not familiar with python, also Im not programmer. Thats why Im not able to do so when people tell me "do this then use XYZ function which will give you ZYX from what you can do that and you will get result". Im still willing to learn but there are thousands of python tutorials and the one for exsactly this topic probably doesn't exsist. The .doc extension is required, so I cannot use .txt because I need the HD files in .doc. From bartc at freeuk.com Sat Jan 9 14:19:33 2010 From: bartc at freeuk.com (bartc) Date: Sat, 09 Jan 2010 19:19:33 GMT Subject: Scripting (was Re: Python books, literature etc) In-Reply-To: References: <9F8149B4-EED1-4F98-87C4-3868E5D71838@milochik.com> <36dec4ff1001060717h6cf62e38ka2433d5745665f1f@mail.gmail.com> <36dec4ff1001061101x1e4b6ddbu9369242073dbf442@mail.gmail.com> Message-ID: <9F42n.24112$Ym4.2976@text.news.virginmedia.com> "Peter" wrote in message news:mailman.661.1262978839.28905.python-list at python.org... > >> Sounds good. >> >> Regarding the book's title: is it just me, or are Python programmers >> in general put off when people call it "scripting"? >> >> I won't attempt a strict definition of the term "scripting language", >> but it seems like non-programmers use it to mean "less scary than what >> you might think of as programming", while programmers interpret it as >> "not useful as a general-purpose language". >> >> > It took me a while to take "scripting" seriously. I grew up with Pascal > and Eiffel and I found it difficult to appreciate dynamic typing and > scripting. The author Langtangen is explaining in detail why he considers > scripting useful, in particular he provides an automatic test suite to run > different language versions ( perl, python, c, c++) of the same program to > compare performance. The results are amazing, in that some of the examples > run faster than the C++ version. I think if you can get Python to run fast (compared to compiled languages), then that's scripting (ie. just using it to sequence lots of built-in functions and operations). If it runs a lot slower than those other languages, then you're probably doing some programming. And with programs where the runtime is not significant, it could be either... -- Bartc From cousinstanley at gmail.com Sat Jan 9 14:52:52 2010 From: cousinstanley at gmail.com (Cousin Stanley) Date: Sat, 9 Jan 2010 19:52:52 +0000 (UTC) Subject: PIL show() not working for 2nd pic References: Message-ID: > I am using PIL for image processing in ubuntu 9.04. When i give two > im.show() commands for two different images, the second image is not > displayed (eye of gnome is the display program). It says no such file > or directory. Any ideas? Suresh .... I also had problems with show() when using eye of gnome as the image viewer with your code but no problems using the display viewer from the imagemagick package .... im.show( command = 'eog' ) # problems im.show( command = 'display' ) # no problems # ---------------------------------------------------------- #!/usr/bin/python ''' NewsGroup .... comp.lang.python Subject ...... PIL show() not working for 2nd pic Date ......... 2010-01-07 Post_By ...... suresh.amritapuri Edit_By ...... Stanley C. Kitching ''' import math import Image list_source = [ 'image/beach.tif' , 'image/colors.tif' ] list_target = [ ] def neg( x ) : return 255 - 1 - x def logtr( x ) : y = math.log( 1 + x , 10 ) print y return y * 100 for this_image in list_source : im_source = Image.open( this_image ) im_neg = im_source.point( neg ) im_logtr = im_source.point( logtr ) list_target.append( im_source ) list_target.append( im_neg ) list_target.append( im_logtr ) for this_image in list_target : this_image.show( command = 'display' ) # ---------------------------------------------------------- -- Stanley C. Kitching Human Being Phoenix, Arizona From zuo at chopin.edu.pl Sat Jan 9 14:53:42 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sat, 09 Jan 2010 20:53:42 +0100 Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2 In-Reply-To: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> Message-ID: Hello, I have a question: are class decorator planned to be backported from 3.x? All the best, *j -- Jan Kaliszewski (zuo) From apt.shansen at gmail.com Sat Jan 9 15:00:27 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Sat, 9 Jan 2010 12:00:27 -0800 Subject: Something More Elegant In-Reply-To: <4dc0cfea1001090715md97518tf618309eda1deabb@mail.gmail.com> References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> <4dc0cfea1001090715md97518tf618309eda1deabb@mail.gmail.com> Message-ID: <7a9c25c21001091200r10c3c7b2jbc16040efa9f7b34@mail.gmail.com> On Sat, Jan 9, 2010 at 7:15 AM, Victor Subervi wrote: > On Sat, Jan 9, 2010 at 9:35 AM, Steve Holden wrote: > >> But we are now in the realm of theory as far as you are concerned, since >> you have already stated several times that you aren't interested in >> correcting your design until after you have got the current mess into >> production. So good luck with that. >> > > And if you were in my shoes, I'm sure you'd do the same thing. ... Really, no, he wouldn't :) We're not all just hobbyists here who only do work for random open source projects. A lot of us are professionals who actually do have clients, actually do have deadlines, actually do have an understanding for production requirements. Getting something into production as soon as possible is certainly an important goal in commercial work. But it is not the only goal. Proper database design is very important because if you don't do it, you'll actually end up usually wasting *more* time and effort then if you just bite the bullet and fix it now. Proper database design -- in particular in your case, not having multiple tables with various names that even need "%sThis" or "%sThat", and using parameterized queries to access those tables, is really important. It will save you time, it will save you effort, and it'll save you money-- because /not/ doing it is among other things, a major security risk. Getting code out fast and now is a noble goal in commercial projects, getting code out which is by design prone to attacks by hackers is negligence. Well, it *is* working now :)) And I am interested in cleaning this up. I > should probably start with the matter of databases, since that's something I > won't be able to easily change once clients actually start entering data. > Please share with me any further concepts or questions to get me thinking > how to redesign the databases. > Your first goal needs to be in the layout of your tables, yes. Instead of having multiple "Packages" tables that vary, have a single Packages table, with an additional column that determines what kind of package it is (this information used to be in the name of the table). This may seem like everything's jumbled together, but it works. Put an index on that column if you need to: don't worry about if that one table might have thousands or tens of thousands of records. Databases are -designed- to handle that, and handle it well and faster. Do the same for any other table which has various names-- CategoriesPackages seems to be another one. Segment the data by adding columns and indexes when needed, and not by breaking it out into different tables: if two tables have approximately the same columns, they belong in one table, with a column to simply differentiate between the two. These steps are taking your database towards the ultimate goal of normalization-- a laudable goal, but I won't go into that in detail. It takes a book. The next thing to do is to re-do your SQL queries. You should never do string interpolation, e.g: SQL="SELECT x FROM y WHERE z = %s" % (arg,) cur.execute(SQL) If you have done the first step, your tables always have a set names so you never need to interpolate to do the queries-- and then at this point, under no circumstances ever, ever-- consider this a law that you will get fined for violation-- use string interpolation to generate your queries. Instead, do: cur.execute("SELECT x FROM y WHERE z = %s", (arg,)) That looks really similar, but is lightyears away. Its very unfortunate that some database drivers use 'format' as the paramstyle because it confuses issues, but the two effects are very different. In one, Python is just munging together and creating a new string. In the other, the database driver is doing a few things. Its analyzing the query, its storing it (often caching it, which speeds up further executions of that query), etc, etc, and then finally its seeing that you are passing arguments into it, and it is -safely- binding those arguments into the expression; this prevents SQL Injection attacks. You can use interpolation and prevent injection if you -meticulously- check -every- string that comes from the user, and -never- trust it (even if that string was written out to a hidden and legitimate users have no way to alter, because illegitimate users will alter it anyways). Or you can use parameterized queries and just avoid it, while getting plenty of other benefits as well. At work, we had a third-party package that we re-sold as part of our offering, and glancing over its source, I noticed something. It did, in essence to check login: cur.execute("SELECT user_id FROM usertable WHERE username = '%s' AND password = '%s' % (username, password)) I blinked, and emailed them to point out the problem. I suggested they log in as: Username = dummyuser Password = '; DROP usertable You see, when using interpolation, the string that got sent to the database was: SELECT user_id FROM usertable WHERE username = 'dummyuser' AND password = ''; DROP usertable And thus, from the login screen of this app-- I destroyed their environment. Its sort of important that you not put code out into production which is susceptible to such things. Your clients will not at all appreciate it if and when some hacker discovers it and destroys their site, loosing them money. But even beyond that, there are many other benefits to just doing this right. You want to, believe me. Now, before you put anything into production. It might take a day or two longer, but its worth it. Finally, go finish getting rid of the bare excepts, I saw one recently :) Just rip them all out, every one. All at once. Then fix any errors that come along with specific excepts or pre-tests if appropriate. ;) --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From no.email at nospam.invalid Sat Jan 9 15:03:16 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 09 Jan 2010 12:03:16 -0800 Subject: lightweight encryption of text file References: Message-ID: <7xr5pzoxkr.fsf@ruckus.brouhaha.com> Daniel Fetchinson writes: > I have a plain text file which I would like to protect in a very > simple minded, yet for my purposes sufficient, way. For encrypting strings, use this module: http://nightsong.com/phr/crypto/p3.py Obviously this is limited to strings that fit in memory, which might be a problem with large files. Some day I might get around to adding a streaming interface to it. The "file" command will not recognize the ciphertext as encrypted data. It will just say "data". If you want to be more serious, use pgp or gpg with the -c option (password-based encryption). I think "file" does recognize the pgp file format as encrypted data (RFC 2440). From no.email at nospam.invalid Sat Jan 9 15:04:40 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 09 Jan 2010 12:04:40 -0800 Subject: lightweight encryption of text file References: Message-ID: <7xmy0noxif.fsf@ruckus.brouhaha.com> Daniel Fetchinson writes: >> http://www.nightsong.com/phr/crypto/p3.py > > Thanks a lot, currently I'm having trouble using this code on python > 2.6 but probably some small tweaking will fix it. Yikes, this is the first I've heard of such a problem. I will look into it. Thanks. (Also thanks to Robert for the recommendation). From no.email at nospam.invalid Sat Jan 9 15:18:33 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 09 Jan 2010 12:18:33 -0800 Subject: lightweight encryption of text file References: Message-ID: <7xiqbbowva.fsf@ruckus.brouhaha.com> Robert Kern writes: > Are you on a 64-bit platform? Unfortunately, array's integer typecodes > are platform-specific, but p3.py requires a 32-bit integer ... Oh yes, thanks, I never did get around to dealing with 64 bit platforms. I also notice that some of the unit test functions use print statements, which won't compile under Python 3.0. I will try to release a version in the next few days with fixes for both of these issues. From steve at holdenweb.com Sat Jan 9 15:31:18 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 09 Jan 2010 15:31:18 -0500 Subject: Link to module Stack In-Reply-To: <03589968$0$1336$c3e8da3@news.astraweb.com> References: <0358384b$0$1336$c3e8da3@news.astraweb.com> <03589968$0$1336$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > On Sat, 09 Jan 2010 05:56:36 -0500, Dave Angel wrote: > >>> "InnerInterpreterError" is the most inappropriate exception name I've >>> ever seen. It has nothing to do with the interpreter, it's a stack >>> error. >>> >>> >> It has everything to do with the (Forth) interpreter. Exceptions can >> readily be named according to their application -- it's not always about >> Python. Anyway, Forth has an inner-interpreter and an >> outer-interpreter, and the name will make sense to a Forth programmer. > > Pardon me, but I *am* a Forth programmer. Or was, it's been many years, > and I'm rusty. I guess this is a difference of terminology: what you're > calling an inner interpreter and an outer interpreter I know of as the > Forth engine and the (text) interpreter. Gforth refers to them as such, > so did Leo Brodie's Forth books, and the (ancient) Macintosh Forth > compiler "Mach 2". > > But in any case... a stack is an general-purpose data structure, and the > error message shouldn't be coupled so tightly to one use. That would be > like this (made-up) example: > >>>> 1/0 > Traceback (most recent call last): > File "", line 1, in > GraphicsApplicationError: too few pixels to calculate average > > > Ridiculous, yes? > > > Yes, Forth uses a stack (technically two, a parameter stack and a return > stack, and some implementations include a third, floating point, stack). > Virtually all languages use stacks in their implementation, and Python > byte-code is also stack-based. > > >>>> result = self.__heap[-1] >>>> del self.__heap[-1] >>>> >>>> >>> That is better written as result = self.__heap.pop(). >>> >>> >>> >> or even better, without the extra local var: >> >> def pop (self): >> if len(self.__heap) == 0: Since self.__heap is a list, the canonical Python for the above test would, of course, be the much simpler if not self.__heap regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From casevh at gmail.com Sat Jan 9 16:27:07 2010 From: casevh at gmail.com (casevh) Date: Sat, 9 Jan 2010 13:27:07 -0800 (PST) Subject: interactive terminal in Ubuntu Linux : libreadline5-dev works only in Python 2.6 not 3.1 References: Message-ID: <23c7de94-44ba-4567-b31f-8a5bb656e9e9@m3g2000yqf.googlegroups.com> On Jan 9, 10:06?am, Dave WB3DWE wrote: > On Jan 6 I inquired how to "fix" the 3.1.1 interactive terminal > in Ubuntu Linux. ? Left arrow yields ^[[D , etc. > > casevh helped by suggesting "libreadline5-dev" be installed. > Did so with Synaptic Package Manager. > The behavior of the Python 3.3.1 terminal is unchanged but > the 2.6.2 terminal is corrected. > > Is there any way to fix the 3.1.1 terminal command line ? > > Thanks, ? ? Dave WB3DWE ? ? ?pdlem... at earthlink.net Did you recompile Python 3.1.1 after installing libreadline5-dev? (From the Python 3.1.1 directory. Your options to configure may vary.) make distclean ./configure --prefix=/usr/local --with-computed-gotos --with-wide- unicode make make altinstall casevh From nawijn at gmail.com Sat Jan 9 16:30:48 2010 From: nawijn at gmail.com (Marco Nawijn) Date: Sat, 9 Jan 2010 13:30:48 -0800 (PST) Subject: Microsoft Office Word and Python (Win XP) References: Message-ID: On Jan 9, 8:18?pm, "3lvss0... at gmail.com" <3lvss0... at gmail.com> wrote: > Marco Nawijn: I have had installed pywin32 already. The three lines > that you mentoined don't do this I checked at my own computer and it works fine. > also what did you mean with "doc = > app.Documents.Open("c:\\example.doc")". Which document should I open > with this line? This was just meant as an example on how to open a Word document from within python. This would be the basis for copying/appending from your source document to your target document (e.g. HD10.doc). > It shouldn't be opened anything. I was asking about > the script as automated process. If you know how could i do this? Well it is a python script. So you can run it as an automated process. You should just set app.Visible=False so the Word user interface component is not shown. > > Terry Reedy: I have never mentoined copying files but moving the whole > text from, always the same (same name, same path), .doc file with. > However copying (=moving) text to correct .doc file would be good yes. > I know command prompt and its copy function but this way it wouldn't > work because I would have to define the target file - the file INTO > which I want to move the text. But I will never know the file name > (target file). The only thing I know is: > - the file is one of .doc files that start with "HD" > - whole name of those .doc file is always HDX.doc where X is a number > and I need to move the text into the file with maximum X (the most > high number) > - all the HD files will be always in the same path (same folder) but I > would like to use the path inside the code (it might be obvious that I > have to) because on PC and laptop, I have two different usernames and > since HD files are located inside Documents And Settings, I have to > use two copies of the script - one for PC, one for laptop. > > Dennis Lee Bieber: Im not familiar with python, also Im not > programmer. Thats why Im not able to do so when people tell me "do > this then use XYZ function which will give you ZYX from what you can > do that and you will get result". Im still willing to learn but there > are thousands of python tutorials and the one for exsactly this topic > probably doesn't exsist. The .doc extension is required, so I cannot > use .txt because I need the HD files in .doc. From jackdied at gmail.com Sat Jan 9 16:34:28 2010 From: jackdied at gmail.com (Jack Diederich) Date: Sat, 9 Jan 2010 16:34:28 -0500 Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2 In-Reply-To: References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> Message-ID: On Sat, Jan 9, 2010 at 2:53 PM, Jan Kaliszewski wrote: > Hello, > > I have a question: are class decorator planned to be backported from 3.x? > Eh? Class decorators have been in the 2.x series since 2.6. If you want to know more about class decorators check out this talk from PyCon 2009 http://pycon.blip.tv/file/1949345 by a very handsome and well beloved man, and come to pycon 2010 where there will be a couple more talks on the subject by slightly more homely but none the less competent speakers. -Jack From zuo at chopin.edu.pl Sat Jan 9 16:44:44 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sat, 09 Jan 2010 22:44:44 +0100 Subject: [Python-Dev] [RELEASED] Python 2.7 alpha 2 In-Reply-To: References: <1afaf6161001090929x228deda5id07cc762522ca53e@mail.gmail.com> Message-ID: 09-01-2010 o 22:34:28 Jack Diederich wrote: > On Sat, Jan 9, 2010 at 2:53 PM, Jan Kaliszewski >> I have a question: are class decorator planned to be backported from >> 3.x? >> > Eh? Class decorators have been in the 2.x series since 2.6. Oops, I overlooked the fact :) Thank you! *j -- Jan Kaliszewski (zuo) From niels.ellegaard at gmail.com Sat Jan 9 17:33:28 2010 From: niels.ellegaard at gmail.com (Niels L. Ellegaard) Date: Sat, 09 Jan 2010 23:33:28 +0100 Subject: Append to an Excel file References: <313cb2b5-4100-4d4d-bb38-dbfe3ca71e17@j5g2000yqm.googlegroups.com> <74a95a2f-90b7-48e6-9b69-c7250b979978@l30g2000yqb.googlegroups.com> Message-ID: <87tyuu7vt3.fsf@langager.dbnet.dk> pp writes: > On Jan 9, 1:47?am, Jason Scheirer wrote: >> On Jan 9, 12:30?am, pp wrote: >> >> > Hi All, >> >> > How do I add a line to an existing file. This should append to the >> > existing data in the excel file, which was saved previously. >> >> > Thanks, >> > PP >> >> http://pypi.python.org/pypi/xlwt > > Hi Jason and all, > > Thanks > > I have seen this.. my question is there a way to append to a excel > file which has been closed. Any specific modes which can be added to > the sheet so that it adds a line to the data which was return in some > earlier running of the program. I may be wrong, but I think that you have to do the following 1) Use xlrd to read the file. This creates an xlrd.Book 2) Use xlutils to transform the xlrd.Book into a xlwt.WorkBook 3) Edit the xlwt.WorkBook 4) Save the xlwt.WorkBook https://secure.simplistix.co.uk/svn/xlutils/trunk/xlutils/docs/copy.txt Niels From MLMDev at Comcast.net Sat Jan 9 17:43:34 2010 From: MLMDev at Comcast.net (Mitchell L Model) Date: Sat, 9 Jan 2010 17:43:34 -0500 Subject: Python-list Digest, Vol 76, Issue 97 In-Reply-To: References: Message-ID: <18FFC0A5-87C2-4E2F-B12D-BF269EF21477@Comcast.net> On Jan 8, 2010, at 7:35:39 PM EST, Terry Reedy wrote: > > On 1/8/2010 12:02 PM, Mitchell L Model wrote: > > >> On further reflection, I will add that >> what appears to be happening is that during import both the global >> and >> local dictionaries are set to a copy of the globals() from the >> importing >> scope and that copy becomes the value of the module's __dict__ once >> import has completed successfully. > > I have no idea why you think that. The module dict starts empty > except for __name__, __file__, and perhaps a couple of other > 'hidden' items. It is not a copy and has nothing to do with > importing scopes. Why I think -- or, rather, thought -- that was because of some defective experiments I ran. It was purely a delusion. Thank you for correcting it. > > > and that copy becomes the value of the module's __dict__ once > > import has completed successfully. > > That new dict becomes .... . > >> Because exec leaves locals() and globals() distinct, > > Not necessarily. > > In 3.x, at least, > exec(s) > executes s in the current scope. If this is top level, where locals > is globals, then same should be true within exec. Yes. To simplify some of my ramblings and incorporate the points you and others have made, and to once again acknowledge Python's elegance, an important observation which I bet even a lot of serious Python programs don't realize (or at least not consciously) is that: globals() is locals() in the following contexts: the interpreter top level the top level of a module (though as you point out, starts out as a very bare dictionary during import) a string being exec'd when the call to exec includes no dictionary argument(s) one dictionary argument the same dictionary as both the second and third arguments The identity does not hold for: a string being exec'd when a different dictionary is provided as the second and third arguments to exec inside anything that creates a scope: a function definition, class definition, etc. Did I get all that right? Are there any other contexts that should be included in these? > > d = {} > exec(s, d) > > In 3.x, at least, d will also be used as locals. Yes, talking about 3.x. > > exec(s, d, d) > > Again, globals and locals are not distinct. > > It would seem that in 3.x, the only way for exec to have distinct > globals and locals is to call exec(s) where they are distinct or to > pass distince globals and locals. Apparently so. To clarify "where they are distinct", that would mean from a context in which they were already distinct, which is not the case if exec is called from the top level, but is the case if called from within, say, a function, as my code does. > > Some of the issues of this thread are discussed in Language > Reference 4.1, Naming and Binding. I suppose it could be clearer > that it is, but the addition of nonlocal scope complicated things. I pretty much have that section memorized and reread it at least monthly. It's part of what I meant by starting my original comments by saying that I thought I understood all of this. Thank you (and others) for helping clarify exactly what's going on. As with so many things in Python, it is not always easy to keep one's preconceptions, delusions, and experiences with other languages out of the way of its simplicity, even if one is a very experienced and knowledgeable Python programmer. --- Mitchell -------------- next part -------------- An HTML attachment was scrubbed... URL: From pdlemper at earthlink.net Sat Jan 9 18:10:20 2010 From: pdlemper at earthlink.net (pdlemper at earthlink.net) Date: Sat, 09 Jan 2010 17:10:20 -0600 Subject: interactive terminal in Ubuntu Linux : libreadline5-dev works only in Python 2.6 not 3.1 References: <23c7de94-44ba-4567-b31f-8a5bb656e9e9@m3g2000yqf.googlegroups.com> Message-ID: On Sat, 9 Jan 2010 13:27:07 -0800 (PST), casevh wrote: > >Did you recompile Python 3.1.1 after installing libreadline5-dev? > >(From the Python 3.1.1 directory. Your options to configure may vary.) > >make distclean >./configure --prefix=/usr/local --with-computed-gotos --with-wide- >unicode >make >make altinstall > >casevh Thanks so much for your help . . . but I'm going backwards, probably due to my inadequate knowledge of Linux. Ran the above vebatim, except had to do sudo make install . Python recompiled and the system appears to be in same path/dir as before : /home/dave/python31/Python-3.1.1 Called up interactive prompt >>> with $ python3 , as before. Problems : 1. prompt keys remain fouled up as before. 2. will not import random module, either from prompt or from any of my prewritten modules. Get ImportError /usr/local/lib/Python3.1/lib-dynload/_collections.so : undefined symbol : PyUnicode UCS4_FromString Some other modules will import : math , sys, os 3. Some of my pre-existing modules will not run : have not checked them all, but apparently those with random. 4. Attempted to read my modules with gedit. Pulls them up but refuses to save : could not save file /usr/local/lib/python3.1/dlmodules/Ackermann.py You do not have permission necessary to save file. This is same path my modules were in before the recompile and the dir I made for them. Had no trouble saving them previously. If no fix available, I'll reinstall from the Python-3.1.1 that I extracted from the tarball , and go back to XP for a while : ( Dave WB3DWE pdlemper at earthlink.net From davea at ieee.org Sat Jan 9 19:17:08 2010 From: davea at ieee.org (Dave Angel) Date: Sat, 09 Jan 2010 19:17:08 -0500 Subject: Link to module Stack In-Reply-To: <03589968$0$1336$c3e8da3@news.astraweb.com> References: <0358384b$0$1336$c3e8da3@news.astraweb.com> <03589968$0$1336$c3e8da3@news.astraweb.com> Message-ID: <4B491C84.2000809@ieee.org> Steven D'Aprano wrote: > On Sat, 09 Jan 2010 05:56:36 -0500, Dave Angel wrote: > > >>> "InnerInterpreterError" is the most inappropriate exception name I've >>> ever seen. It has nothing to do with the interpreter, it's a stack >>> error. >>> >>> >>> >> It has everything to do with the (Forth) interpreter. Exceptions can >> readily be named according to their application -- it's not always about >> Python. Anyway, Forth has an inner-interpreter and an >> outer-interpreter, and the name will make sense to a Forth programmer. >> > > Pardon me, but I *am* a Forth programmer. Or was, it's been many years, > and I'm rusty. I guess this is a difference of terminology: what you're > calling an inner interpreter and an outer interpreter I know of as the > Forth engine and the (text) interpreter. Gforth refers to them as such, > so did Leo Brodie's Forth books, and the (ancient) Macintosh Forth > compiler "Mach 2". > > I'm pretty sure FIG-Forth called them an inner interpreter and outer interpreter, but I don't remember other sources. FIG-Forth was my first Forth system, gotten on an 8" diskette. The inner interpreter was LOADSW, JMP AX, I believe, as it was an indirected threaded interpreter implementation. > > >> or even better, without the extra local var: >> >> def pop (self): >> if len(self.__heap) == 0: >> raise InnerInterpreterError, "stack underflow" >> return self.__heap.pop(1) >> > > pop(1)? I don't think so. > > > That was a typo; I meant pop(). And of course others have improved on my remarks anyway. DaveA From 3lvss0809 at gmail.com Sat Jan 9 19:23:37 2010 From: 3lvss0809 at gmail.com (3lvss0809 at gmail.com) Date: Sat, 9 Jan 2010 16:23:37 -0800 (PST) Subject: Microsoft Office Word and Python (Win XP) References: Message-ID: <4dd1834d-b03e-4f75-a6f6-117d6f522ed9@a15g2000yqm.googlegroups.com> Marco did you also make the HD files to it worked for you? Because I cannot even imagine how only three lines would do everything - find correct folder (path) of the files, find maximum number, move the entire text,... In this 3 lines is not stated that we are talking about the files that start with "HD" so I wonder how it worked for you. Nothing was moved on my PC. I have done some research about the line "app = Dispatch("Word.Application")" and saw this: http://www.programmingforums.org/post105986.html The script described here is different comparing to what I want to do but I might be able to use some ideas - of course, since Im not a programmer, I need to study the functions before. On the link the user is trying to replace parts (predefinited or not - i don't know) of the .doc files - this is not moving/copying entire text but the topic is still about writing something into .doc through python. Unfortunatelly for me, his script doesn't need to search for correct .doc file. So I kept searching and came to this: http://www.daniweb.com/forums/thread129924.html This task is very close to what I want but still different. If we try to compare; "I am a text" (his task) = already exsisting (if any) text in my HDX file where X is the highest number (my task) "Hello" (his task) = whole text inside .doc with always the same name on the same location (my task) then thats it. I believe just those two differences exsist. From casevh at gmail.com Sat Jan 9 19:48:52 2010 From: casevh at gmail.com (casevh) Date: Sat, 9 Jan 2010 16:48:52 -0800 (PST) Subject: interactive terminal in Ubuntu Linux : libreadline5-dev works only in Python 2.6 not 3.1 References: <23c7de94-44ba-4567-b31f-8a5bb656e9e9@m3g2000yqf.googlegroups.com> Message-ID: <15a78d94-34d5-4bef-8344-6dc8843de4e5@k19g2000yqc.googlegroups.com> On Jan 9, 3:10?pm, pdlem... at earthlink.net wrote: > On Sat, 9 Jan 2010 13:27:07 -0800 (PST), casevh > wrote: > >Did you recompile Python 3.1.1 after installing libreadline5-dev? > > >(From the Python 3.1.1 directory. Your options to configure may vary.) > > >make distclean > >./configure --prefix=/usr/local --with-computed-gotos --with-wide- > >unicode > >make > >make altinstall > > >casevh > > Thanks so much for your help . . . but I'm going backwards, probably > due to my inadequate knowledge of Linux. > > Ran the above vebatim, except had to do ? ? ?sudo make install ? ?. > Python recompiled and the system appears to be in same path/dir as > before : ?/home/dave/python31/Python-3.1.1 > Called up interactive prompt ?>>> ?with ?$ python3 , as before. > It looks like you now have two copies of Python 3.1.1 installed. That's probably a side-effect of my instructions. I'll got through each instruction individually. It looks like your Python source code is in "/home/dave/python31/ Python-3.1.1". The commnad "make distclean" should remove the results of the prior configuration so to won't need to extract the source code again. The command "./configure" accepts several options that control how the source code is configured. To see all the available options, use the command "./configure --help". The first option - "--prefix=/usr/local" - identifies the location where Python 3.1.1 will be installed. The "/ usr/local" directory is a common location for user-compiled programs. If no location is specified, many applications assume "/usr/local". The option - "--with-computed-gotos" - is just a compiler options that produces a slightly faster interpreter. The final option - "--with- wide-unicode" - identifies the proper Unicode format to use. (I looks like this option may have been split via line wrapping on my earlier email. I think it is causing the UCS4 error.) The command "make" compiles Python 3.1.1. The command "sudo make altinstall" installs Python 3.1.1 under the "/ usr/local" directory. The option "altinstall" installs Python 3.1.1 and leaves its name as "python3.1". It should be located in "/usr/ local/bin/python3.1". Your own programs (Ackermann.py) should be kept somewhere in your home directory, for example "/home/dave/code". Typing "python3.1" will look in the current directory first, and then search the path where it should find "/usr/local/bin/python3.1". What will you need to do fix this? 1) Try the commands again. Make sure all the "./configure" options are on one line. Make sure to do "sudo make altinstall". (Don't use "sudo make install"; it will give your version of Python the name "python" and that can cause confusion on your system.) 2) Move your applications to another directory. 3) Try running "python3.1" while you are in that directory. If this doesn't work, report back on the error messages you receive. Don't give up; we can fix this. casevh > Problems : > ? ? 1. prompt keys remain fouled up as before. > ? ? 2. will not import ?random ?module, either from prompt or > ? ? ? ? ? ? from any of my prewritten modules. ?Get ImportError > ? ? ? ? ? ?/usr/local/lib/Python3.1/lib-dynload/_collections.so : > ? ? ? ? ? ? undefined symbol : PyUnicode UCS4_FromString > ? ? ? ? ? ? ? ? Some other modules will import : math , sys, os > ? ? 3. Some of my pre-existing modules will not run : have > ? ? ? ? ? not checked them all, but apparently those with random. > ? ? 4. Attempted to read my modules with gedit. ?Pulls them > ? ? ? ? ? up but refuses to save : ? ? ? ? ?could not save file > ? ? ? ? ? /usr/local/lib/python3.1/dlmodules/Ackermann.py > ? ? ? ? ? ? ? ?You do not have permission necessary to save file. > ? ? ? ? ? This is same path my modules were in before the > ? ? ? ? ? recompile and the dir I made for them. Had no trouble > ? ? ? ? ? saving them previously. > > If no fix available, I'll reinstall from the Python-3.1.1 that > I extracted from the tarball , and go back to XP for a while ?: ( > > Dave WB3DWE ? ? ? pdlem... at earthlink.net From peloko45 at gmail.com Sat Jan 9 20:01:54 2010 From: peloko45 at gmail.com (Joan Miller) Date: Sat, 9 Jan 2010 17:01:54 -0800 (PST) Subject: Prepend to logging message Message-ID: <5aa06ab5-61fd-46a8-9cac-7f3a2a5d382b@d20g2000yqh.googlegroups.com> How to prepend anything to a logging message? Is possible to do it from the dictionary object (ExtraLog) or is there is that override process() [1]? ------------------ class ExtraLog(object): def __getitem__(self, name): if name == 'foo': result = 'testing' return result def __iter__(self): keys = ['foo',] keys.extend(self.__dict__.keys()) return iter(keys) logger = logging.LoggerAdapter(logging.getLogger('foo'), ExtraLog()) ------------------ [1] http://docs.python.org/library/logging.html#logging.LoggerAdapter From monaghand.david at gmail.com Sat Jan 9 20:09:46 2010 From: monaghand.david at gmail.com (David Monaghan) Date: Sun, 10 Jan 2010 01:09:46 +0000 Subject: Microsoft Office Word and Python (Win XP) References: Message-ID: On Sat, 9 Jan 2010 11:18:12 -0800 (PST), "3lvss0809 at gmail.com" <3lvss0809 at gmail.com> wrote: >Dennis Lee Bieber: Im not familiar with python, also Im not >programmer. What you want to do isn't complicated, but it isn't simple either, unless you're familiar with VBA/VBS. I approach these problems by first getting the VBA code by recording a macro within Word. I then convert it to VB Script, which is a learning process in itself. Converting that script to Python com is another learning process and then putting the whole thing together with file finding and saving is another job. When you've done that, you won't feel able to say you're not a programmer - and you should feel familiar with Python, too. DaveM From ishwor.gurung at gmail.com Sat Jan 9 22:27:10 2010 From: ishwor.gurung at gmail.com (Ishwor Gurung) Date: Sun, 10 Jan 2010 14:27:10 +1100 Subject: Prepend to logging message In-Reply-To: <5aa06ab5-61fd-46a8-9cac-7f3a2a5d382b@d20g2000yqh.googlegroups.com> References: <5aa06ab5-61fd-46a8-9cac-7f3a2a5d382b@d20g2000yqh.googlegroups.com> Message-ID: <34534aed1001091927i8e9fe6dsd38cc46657cdfe2c@mail.gmail.com> Joan, 2010/1/10 Joan Miller : > How to prepend anything to a logging message? Is possible to do it > from the dictionary object (ExtraLog) or is there is that override > process() [1]? > > ------------------ > class ExtraLog(object): > > ? ?def __getitem__(self, name): > ? ? ? ?if name == 'foo': > ? ? ? ? ? ?result = 'testing' > ? ? ? ?return result > > ? ?def __iter__(self): > ? ? ? ?keys = ['foo',] > ? ? ? ?keys.extend(self.__dict__.keys()) > ? ? ? ?return iter(keys) > > logger = logging.LoggerAdapter(logging.getLogger('foo'), ExtraLog()) > ------------------ Yep. Just subclass LoggerAdapter and override process(..) Read this: http://docs.python.org/library/logging.html#adding-contextual-information-to-your-logging-output -- Regards Ishwor Gurung Key id:0xa98db35e Key fingerprint:FBEF 0D69 6DE1 C72B A5A8 35FE 5A9B F3BB 4E5E 17B5 From lie.1296 at gmail.com Sun Jan 10 00:24:01 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Sun, 10 Jan 2010 16:24:01 +1100 Subject: PIL how to display multiple images side by side In-Reply-To: <6d9fef57-7fdc-46ec-9b53-7aa824e785e9@s31g2000yqs.googlegroups.com> References: <6d9fef57-7fdc-46ec-9b53-7aa824e785e9@s31g2000yqs.googlegroups.com> Message-ID: <4b496473$1@dnews.tpgi.com.au> On 1/9/2010 8:43 AM, suresh.amritapuri wrote: > Hi, > > In PIL, how to display multiple images in say m rows and n colums when > I have m*n images. > > suresh Tkinter has PhotoImage widget and PIL has support for this widget: http://www.pythonware.com/library/pil/handbook/imagetk.htm From alfps at start.no Sun Jan 10 00:51:58 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 10 Jan 2010 06:51:58 +0100 Subject: PIL how to display multiple images side by side In-Reply-To: <4b496473$1@dnews.tpgi.com.au> References: <6d9fef57-7fdc-46ec-9b53-7aa824e785e9@s31g2000yqs.googlegroups.com> <4b496473$1@dnews.tpgi.com.au> Message-ID: * Lie Ryan: > On 1/9/2010 8:43 AM, suresh.amritapuri wrote: >> Hi, >> >> In PIL, how to display multiple images in say m rows and n colums when >> I have m*n images. >> >> suresh > > Tkinter has PhotoImage widget and PIL has support for this widget: > http://www.pythonware.com/library/pil/handbook/imagetk.htm Maybe I've misunderstood something (in that case glad to learn!), but I believe PhotoImage is not a widget, and that a PhotoImage has to be presented in e.g. a Label widget or some other widget that's able to display images. Cheers, - Alf From nobody at nowhere.com Sun Jan 10 02:51:02 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 10 Jan 2010 07:51:02 +0000 Subject: subprocess.Popen does not close pipe in an error case References: <07641018-9581-44af-bb20-8f9b96a44c4f@p8g2000yqb.googlegroups.com> Message-ID: On Wed, 06 Jan 2010 11:39:37 -0800, Steven K. Wong wrote: > Suppose now all the prog1.poll() calls/loop are replaced by a single > prog1.wait(). Without the explicit prog1.stdout.close(), prog1.wait() > will not return, so the calling process still hangs. Because calling > prog1.wait() means that the calling process will naturally never read > prog1.stdout, I would argue that prog1.wait() should close the pipe > before actually waiting for prog1 to exit. Makes sense? prog1.stdout might be being read by a different thread. From nobody at nowhere.com Sun Jan 10 02:54:02 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 10 Jan 2010 07:54:02 +0000 Subject: subprocess.Popen does not close pipe in an error case References: <07641018-9581-44af-bb20-8f9b96a44c4f@p8g2000yqb.googlegroups.com> <40a2adab-941a-4844-9279-10667bf2ba5e@v25g2000yqk.googlegroups.com> Message-ID: On Wed, 06 Jan 2010 19:05:40 -0800, Steven K. Wong wrote: > Well, the example code at > http://www.python.org/ ... /subprocess.html#replacing-shell-pipeline > has the same issue: > Perhaps the doc can be improved to remind folks to close p1.stdout if > the calling process doesn't need it, unless wait() is changed to close > it and p1.wait() is called. > > Am I making any sense here? The docs should include the p1.stdout.close(). It isn't needed in the typical case, where p2 runs until EOF on stdin, but (as you have noticed) it matters if p2 terminates prematurely. From nobody at nowhere.com Sun Jan 10 03:16:28 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 10 Jan 2010 08:16:28 +0000 Subject: Ask how to use HTMLParser References: <87vdfdtg3z.fsf@ymail.invalid> Message-ID: On Fri, 08 Jan 2010 11:44:48 +0800, Water Lin wrote: > I am a new guy to use Python, but I want to parse a html page now. I > tried to use HTMLParse. Here is my sample code: > ---------------------- > from HTMLParser import HTMLParser Note that HTMLParser only tokenises HTML; it doesn't actually *parse* it. You just get a stream of tag, text, entity, text, tag, ..., not a parse tree. In particular, if an element has its start and/or end tags omitted, you won't get any notification about the start and/or end of the element; you have to figure that out yourself from the fact that you're getting a tag which wouldn't be allowed outside or inside the element. E.g. if the document has omitted

tags, if you get a

tag when you are (or *thought* that you were) already within a paragraph, you can infer the omitted

tag. If you want an actual parser, look at BeautifulSoup. This also does a good job of handling invalid HTML (which seems to be far more common than genuine HTML). From nobody at nowhere.com Sun Jan 10 03:33:42 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 10 Jan 2010 08:33:42 +0000 Subject: lightweight encryption of text file References: Message-ID: On Fri, 08 Jan 2010 20:14:51 +0100, Daniel Fetchinson wrote: > I have a plain text file which I would like to protect in a very > simple minded, yet for my purposes sufficient, way. I'd like to > encrypt/convert it into a binary file in such a way that possession of > a password allows anyone to convert it back into the original text > file while not possessing the password one would only see the > following with the standard linux utility 'file': > What would be the simplest way to achieve this using preferably stock > python without 3rd party modules? If a not too complex 3rd party > module made it really simple that would be acceptable too. RC4 (aka ArcFour) is quite trivial to implement, and better than inventing your own cipher or using a Vignere: import itertools class arcfour: def __init__(self, key): self.s = range(256) self.schedule(map(ord, key)) self.pad = self.prng() def swap(self, i, j): self.s[i], self.s[j] = self.s[j], self.s[i] def schedule(self, key): j = 0 for i, c in zip(xrange(256), itertools.cycle(key)): j = (j + self.s[i] + c) % 256 self.swap(i, j) def prng(self): i = j = 0 while True: i = (i + 1) % 256 j = (j + self.s[i]) % 256 self.swap(i, j) yield self.s[(self.s[i] + self.s[j]) % 256] def crypt(self, string): chars = (chr(c ^ r) for c, r in zip(map(ord, string), self.pad)) return ''.join(chars) I suggest that you don't use the password itself as the key, unless you're sure that a low-entropy string won't be used. Instead, create an SHA hash (see the sha and hashlib modules) of the password and use that. From fetchinson at googlemail.com Sun Jan 10 03:59:31 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Sun, 10 Jan 2010 09:59:31 +0100 Subject: lightweight encryption of text file In-Reply-To: References: Message-ID: >> I have a plain text file which I would like to protect in a very >> simple minded, yet for my purposes sufficient, way. I'd like to >> encrypt/convert it into a binary file in such a way that possession of >> a password allows anyone to convert it back into the original text >> file while not possessing the password one would only see the >> following with the standard linux utility 'file': > >> What would be the simplest way to achieve this using preferably stock >> python without 3rd party modules? If a not too complex 3rd party >> module made it really simple that would be acceptable too. > > RC4 (aka ArcFour) is quite trivial to implement, and better than inventing > your own cipher or using a Vignere: > > import itertools > > class arcfour: > def __init__(self, key): > self.s = range(256) > self.schedule(map(ord, key)) > self.pad = self.prng() > > def swap(self, i, j): > self.s[i], self.s[j] = self.s[j], self.s[i] > > def schedule(self, key): > j = 0 > for i, c in zip(xrange(256), itertools.cycle(key)): > j = (j + self.s[i] + c) % 256 > self.swap(i, j) > > def prng(self): > i = j = 0 > while True: > i = (i + 1) % 256 > j = (j + self.s[i]) % 256 > self.swap(i, j) > yield self.s[(self.s[i] + self.s[j]) % 256] > > def crypt(self, string): > chars = (chr(c ^ r) for c, r in zip(map(ord, string), self.pad)) > return ''.join(chars) > > I suggest that you don't use the password itself as the key, unless you're > sure that a low-entropy string won't be used. Instead, create an SHA hash > (see the sha and hashlib modules) of the password and use that. Thanks, this looks very simple too, but where is the decryption code? Wikipedia seems to suggest that encryption and decryption are both the same but running crypt on the output of crypt doesn't give back the original string. So probably I'm misunderstanding something. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From peter at www.pjb.com.au Sun Jan 10 04:15:54 2010 From: peter at www.pjb.com.au (Peter Billam) Date: 10 Jan 2010 09:15:54 GMT Subject: Computer Music Toolkit (CMT) ? Message-ID: Greetings. Is there a way to get at the Computer Music Toolkit (CMT) http://www.ladspa.org/cmt/ functionality from Python (Python3 in my case) ? Googling is confusing because of www.cmt.com and cmt-graph and openscientist.lal.in2p3.fr/v9/cmt.html and pyAMISecure_and_cmt and so on... Regards, Peter -- Peter Billam www.pjb.com.au www.pjb.com.au/comp/contact.html From chambon.pascal at gmail.com Sun Jan 10 04:42:23 2010 From: chambon.pascal at gmail.com (Pakal) Date: Sun, 10 Jan 2010 01:42:23 -0800 (PST) Subject: Direct use of bytearray buffers with ctypes ? References: <9694d6f5-5189-4371-a933-f7c5da5b2d30@e27g2000yqd.googlegroups.com> Message-ID: <18fdc1e3-416b-445e-af44-80851c194e0f@s31g2000yqs.googlegroups.com> Oki, it seems I've found. To directly use a bytearray buffer from ctypes, you must first create a compatible ctypes type (I.E, a char array of same size), and only then instanciate this new type with newtype.from_buffer (bytearray_object). The little danger is : you must NOT change the size of bytearray as long as the ctypes array targets its inner buffer, else of cousre memory reallocations occurs, and your ctypes array points to invalid memory (awaiting a pretty segfault). But modifying chars or characters ranges from the buffer, as well via the bytearray object than via the ctypes array, is fine. I don't think problems can occur if both sides are accessed simultaneously, even though ctypes operations may release the GIL while they execute... the only race condition is the access of a memory segment, isn't it ? No crash shall occur with this... IDLE 2.6.4 >>> import ctypes >>> a = bytearray(3) >>> mytype = ctypes.c_char * 3 >>> h = mytype.from_buffer(a) >>> h <__main__.c_char_Array_3 object at 0x02B06350> >>> h[:] = "abc" >>> a bytearray(b'abc') >>> h[0] = "k" >>> a bytearray(b'kbc') From stefan_ml at behnel.de Sun Jan 10 04:47:48 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 10 Jan 2010 10:47:48 +0100 Subject: Computer Music Toolkit (CMT) ? In-Reply-To: References: Message-ID: <4b49a244$0$6726$9b4e6d93@newsspool2.arcor-online.net> Peter Billam, 10.01.2010 10:15: > Greetings. Is there a way to get at the Computer Music Toolkit (CMT) > http://www.ladspa.org/cmt/ > functionality from Python (Python3 in my case) ? > > Googling is confusing because of www.cmt.com and cmt-graph and > openscientist.lal.in2p3.fr/v9/cmt.html and pyAMISecure_and_cmt > and so on... Just a note on the last paragraph: searching for acronyms is usually simplified by adding the spelled-out name to the search keywords. It's highly unlikely that someone would write a Python tool for a library or application without putting the complete name of the thing it's made for on the project homepage. Stefan From peloko45 at gmail.com Sun Jan 10 04:47:52 2010 From: peloko45 at gmail.com (Joan Miller) Date: Sun, 10 Jan 2010 01:47:52 -0800 (PST) Subject: Prepend to logging message References: <5aa06ab5-61fd-46a8-9cac-7f3a2a5d382b@d20g2000yqh.googlegroups.com> Message-ID: <40d0e6b8-15be-4e3b-aca2-26e07d7fc8cf@v25g2000yqk.googlegroups.com> On 10 ene, 03:27, Ishwor Gurung wrote: > Joan, > > 2010/1/10 Joan Miller : > > > > > How to prepend anything to a logging message? Is possible to do it > > from the dictionary object (ExtraLog) or is there is that override > > process() [1]? > > > ------------------ > > class ExtraLog(object): > > > ? ?def __getitem__(self, name): > > ? ? ? ?if name == 'foo': > > ? ? ? ? ? ?result = 'testing' > > ? ? ? ?return result > > > ? ?def __iter__(self): > > ? ? ? ?keys = ['foo',] > > ? ? ? ?keys.extend(self.__dict__.keys()) > > ? ? ? ?return iter(keys) > > > logger = logging.LoggerAdapter(logging.getLogger('foo'), ExtraLog()) > > ------------------ > > Yep. Just subclass LoggerAdapter and override process(..) > Read this:http://docs.python.org/library/logging.html#adding-contextual-informa... > -- > Regards > Ishwor Gurung > Key id:0xa98db35e > Key fingerprint:FBEF 0D69 6DE1 C72B A5A8 ?35FE 5A9B F3BB 4E5E 17B5 Any example to override process() ? From parul.pandey85 at gmail.com Sun Jan 10 04:51:41 2010 From: parul.pandey85 at gmail.com (pp) Date: Sun, 10 Jan 2010 01:51:41 -0800 (PST) Subject: Porblem with xlutils/xlrd/xlwt References: <8e9245ec-05cb-48ed-ae32-f9fb0c46509f@j5g2000yqm.googlegroups.com> <972fdcfc-73a6-4e87-9752-3bc10da0dcad@s3g2000yqs.googlegroups.com> <43a1e323-8ccd-48b7-a6a4-e540aca2b1cd@35g2000yqa.googlegroups.com> <5ac38519-9ad4-4a89-9b3f-ccd56fb18721@m3g2000yqf.googlegroups.com> Message-ID: <610f640f-b03b-4aaa-93fe-a3fdf1381801@j4g2000yqe.googlegroups.com> On Jan 9, 8:23?am, John Machin wrote: > On Jan 9, 9:56?pm, pp wrote: > > > On Jan 9, 3:52?am, Jon Clements wrote: > > > > On Jan 9, 10:44?am, pp wrote: > > > > > On Jan 9, 3:42?am, Jon Clements wrote: > > > > > > On Jan 9, 10:24?am, pp wrote: > > > > yeah all my versions are latest fromhttp://www.python-excel.org. > > > > just checked!! > > How did you check? > > > > > what could be the problem? > > > > Does rb = xlrd.open_workbook('somesheet.xls', on_demand=True) work by > > > itself? > > > Yes it does. The problem is with line: wb = ?copy(rb) > > here I am getting the error: AttributeError: 'Book' object has no > > attribute 'on_demand' > > Please replace the first 4 lines of your script by these 6 lines: > > import xlrd > assert xlrd.__VERSION__ == "0.7.1" > from xlwt import easyxf > from xlutils.copy import copy > rb = xlrd.open_workbook( > ? ? 'source.xls',formatting_info=True, on_demand=False) > > and run it again. Please copy all the output and paste it into your > response. This time when I ran the code sent by you I got the following results:I am using ipython for running the code. AssertionError Traceback (most recent call last) /home/parul/CODES/copy_1.py in () 1 ----> 2 import xlrd 3 assert xlrd.__VERSION__ == "0.7.1" 4 from xlwt import easyxf 5 from xlutils.copy import copy 6 rb = xlrd.open_workbook('source.xls',formatting_info=True, on_demand=False) AssertionError: WARNING: Failure executing file: I used www.python-excel.org to get xlrd and xlwt .. so they are latest versions. From __peter__ at web.de Sun Jan 10 05:26:55 2010 From: __peter__ at web.de (Peter Otten) Date: Sun, 10 Jan 2010 11:26:55 +0100 Subject: Prepend to logging message References: <5aa06ab5-61fd-46a8-9cac-7f3a2a5d382b@d20g2000yqh.googlegroups.com> Message-ID: Joan Miller wrote: > How to prepend anything to a logging message? Is possible to do it > from the dictionary object (ExtraLog) or is there is that override > process() [1]? > > ------------------ > class ExtraLog(object): > > def __getitem__(self, name): > if name == 'foo': > result = 'testing' > return result > > def __iter__(self): > keys = ['foo',] > keys.extend(self.__dict__.keys()) > return iter(keys) format = "foo=%(foo)s " + logging.BASIC_FORMAT logging.basicConfig(format=format) logger = logging.LoggerAdapter(logging.getLogger('foo'), ExtraLog()) logger.error("yadda") Is that what you want? Peter From dikkie at nospam.org Sun Jan 10 05:59:03 2010 From: dikkie at nospam.org (Dikkie Dik) Date: Sun, 10 Jan 2010 11:59:03 +0100 Subject: (non-python) upgrading a MySQL database In-Reply-To: References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> <4dc0cfea1001090715md97518tf618309eda1deabb@mail.gmail.com> Message-ID: <4b49b2f7$0$8105$bf4948fe@news.tele2.nl> >> ... And I am interested in cleaning this up. I should probably >> start with the matter of databases, since that's something I won't be able >> to easily change once clients actually start entering data. Please share >> with me any further concepts or questions to get me thinking how to redesign >> the databases. I maintain my (MySQL) databases in an agile fashion. That means I start small, and let my databases evolve along with my applications. So the schema changes a lot during development. I take into account that the databases already contain data and that it must be preserved and migrated. I wrote a howto on my upgrading method and it can be found here: http://www.howtoforge.org/node/4833 Hope this is of any help. Good luck, Dikkie From sjmachin at lexicon.net Sun Jan 10 05:59:48 2010 From: sjmachin at lexicon.net (John Machin) Date: Sun, 10 Jan 2010 02:59:48 -0800 (PST) Subject: Porblem with xlutils/xlrd/xlwt References: <8e9245ec-05cb-48ed-ae32-f9fb0c46509f@j5g2000yqm.googlegroups.com> <972fdcfc-73a6-4e87-9752-3bc10da0dcad@s3g2000yqs.googlegroups.com> <43a1e323-8ccd-48b7-a6a4-e540aca2b1cd@35g2000yqa.googlegroups.com> <5ac38519-9ad4-4a89-9b3f-ccd56fb18721@m3g2000yqf.googlegroups.com> <610f640f-b03b-4aaa-93fe-a3fdf1381801@j4g2000yqe.googlegroups.com> Message-ID: <8bff475c-212a-4d15-85dd-a9e4fb103d35@26g2000yqo.googlegroups.com> On Jan 10, 8:51?pm, pp wrote: > On Jan 9, 8:23?am, John Machin wrote: > > > > > On Jan 9, 9:56?pm, pp wrote: > > > > On Jan 9, 3:52?am, Jon Clements wrote: > > > > > On Jan 9, 10:44?am, pp wrote: > > > > > > On Jan 9, 3:42?am, Jon Clements wrote: > > > > > > > On Jan 9, 10:24?am, pp wrote: > > > > > yeah all my versions are latest fromhttp://www.python-excel.org. > > > > > just checked!! > > > How did you check? You didn't answer this question. > > > > > > what could be the problem? > > > > > Does rb = xlrd.open_workbook('somesheet.xls', on_demand=True) work by > > > > itself? > > > > Yes it does. The problem is with line: wb = ?copy(rb) > > > here I am getting the error: AttributeError: 'Book' object has no > > > attribute 'on_demand' > > > Please replace the first 4 lines of your script by these 6 lines: > > > import xlrd > > assert xlrd.__VERSION__ == "0.7.1" > > from xlwt import easyxf > > from xlutils.copy import copy > > rb = xlrd.open_workbook( > > ? ? 'source.xls',formatting_info=True, on_demand=False) > > > and run it again. Please copy all the output and paste it into your > > response. > > This time when I ran the code sent by you I got the following > results:I am using ipython for running the code. > > AssertionError ? ? ? ? ? ? ? ? ? ? ? ? ? ?Traceback (most recent call > last) > > /home/parul/CODES/copy_1.py in () > ? ? ? 1 > ----> 2 import xlrd > ? ? ? 3 assert xlrd.__VERSION__ == "0.7.1" > ? ? ? 4 from xlwt import easyxf > ? ? ? 5 from xlutils.copy import copy > ? ? ? 6 rb = xlrd.open_workbook('source.xls',formatting_info=True, > on_demand=False) > > AssertionError: > WARNING: Failure executing file: > Your traceback appears to show an AssertionError from an import statement. We could do without an extra layer of noise in the channel; please consider giving ipython the flick (for debug purposes, at least) and use Python to run your script from the shell prompt. Change the second line to read: print xlrd.__VERSION__ > I used www.python-excel.org to get xlrd and xlwt .. so they are latest > versions. Let's concentrate on xlrd. I presume that means that you clicked on the xlrd Download link which took you to http://pypi.python.org/pypi/xlrd from which you can download the latest version of the package. That page has "xlrd 0.7.1" in a relatively large font at the top. You would have been presented with options to download one of these xlrd-0.7.1.tar.gz xlrd-0.7.1.win32.exe xlrd-0.7.1.zip (each uploaded on 2009-06-01). Which one did you download, and then what did you do with it? Or perhaps you ignored those and read further down to "Download link" which took you to an out-of-date page but you didn't notice the "0.6.1" in large bold type at the top nor the "Page last updated on 11 June 2007" at the bottom nor the "0.6.1" in the name of the file that you downloaded ... sorry about that; I've smacked the webmaster about the chops :-) Cheers, John From 3lvss0809 at gmail.com Sun Jan 10 06:00:29 2010 From: 3lvss0809 at gmail.com (3lvss0809 at gmail.com) Date: Sun, 10 Jan 2010 03:00:29 -0800 (PST) Subject: Microsoft Office Word and Python (Win XP) References: Message-ID: ? From peloko45 at gmail.com Sun Jan 10 06:23:37 2010 From: peloko45 at gmail.com (Joan Miller) Date: Sun, 10 Jan 2010 03:23:37 -0800 (PST) Subject: Prepend to logging message References: <5aa06ab5-61fd-46a8-9cac-7f3a2a5d382b@d20g2000yqh.googlegroups.com> Message-ID: On 10 ene, 10:26, Peter Otten <__pete... at web.de> wrote: > Joan Miller wrote: > > How to prepend anything to a logging message? Is possible to do it > > from the dictionary object (ExtraLog) or is there is that override > > process() [1]? > > > ------------------ > > class ExtraLog(object): > > > ? ? def __getitem__(self, name): > > ? ? ? ? if name == 'foo': > > ? ? ? ? ? ? result = 'testing' > > ? ? ? ? return result > > > ? ? def __iter__(self): > > ? ? ? ? keys = ['foo',] > > ? ? ? ? keys.extend(self.__dict__.keys()) > > ? ? ? ? return iter(keys) > > format = "foo=%(foo)s " + logging.BASIC_FORMAT > logging.basicConfig(format=format) > logger = logging.LoggerAdapter(logging.getLogger('foo'), ExtraLog()) > logger.error("yadda") > > Is that what you want? > > Peter I want that a message can be modified before of being logged. i.e. for "yadda" I would that were preppend 2 spaces. (And I want not manage that in the format to manage the indentation of all text) From matt at bubblegen.co.uk Sun Jan 10 06:30:24 2010 From: matt at bubblegen.co.uk (Matthew Lear) Date: Sun, 10 Jan 2010 11:30:24 +0000 Subject: getopt not raising exception Message-ID: <4B49BA50.8050207@bubblegen.co.uk> Hello, I'm having problems getting getopt to function correctly. Basically, no exception is being raised if no argument is passed to the code snippet below. I've read the Python documentation and tried example code from various sources which should cause an exception, only they don't. I've also tried executing the code on different machines too but to no avail. I'm sure I'm obviously doing something wrong but can't determine what. Any help would be much appreciated indeed. import sys, getopt try: opts, args = getopt.getopt(sys.argv, "h:", ["help"]) except getopt.GetoptError: print "error" sys.exit(2) If no args are passed when the script is run there is no exception raised. Why? Surely the "h:" means that this option must be passed? Thanks, -- Matt From mnordhoff at mattnordhoff.com Sun Jan 10 06:57:43 2010 From: mnordhoff at mattnordhoff.com (Matt Nordhoff) Date: Sun, 10 Jan 2010 11:57:43 +0000 Subject: getopt not raising exception In-Reply-To: <4B49BA50.8050207@bubblegen.co.uk> References: <4B49BA50.8050207@bubblegen.co.uk> Message-ID: <4B49C0B7.2000702@mattnordhoff.com> Matthew Lear wrote: > Hello, > > I'm having problems getting getopt to function correctly. Basically, no > exception is being raised if no argument is passed to the code snippet > below. I've read the Python documentation and tried example code from > various sources which should cause an exception, only they don't. I've > also tried executing the code on different machines too but to no avail. > I'm sure I'm obviously doing something wrong but can't determine what. > Any help would be much appreciated indeed. > > import sys, getopt > > try: > opts, args = getopt.getopt(sys.argv, "h:", ["help"]) > except getopt.GetoptError: > print "error" > sys.exit(2) > > If no args are passed when the script is run there is no exception > raised. Why? Surely the "h:" means that this option must be passed? > > Thanks, > -- Matt That's not what ":" means. It means that, *if* the option is passed, it must be followed by an additional argument, e.g.: foo.py -h something If you require that -h be passed, it's not much of an option! You should use a regular argument and check len(args) or somesuch. Or, if you must keep it an "option", do something equivalent for that. BTW: Checked out optparse? It's bigger and sexier! -- Matt Nordhoff From mail at hellmutweber.de Sun Jan 10 07:26:00 2010 From: mail at hellmutweber.de (Hellmut Weber) Date: Sun, 10 Jan 2010 13:26:00 +0100 Subject: integer and string compare, is that correct? Message-ID: <4B49C758.9050204@hellmutweber.de> Hi, being a causal python user (who likes the language quite a lot) it took me a while to realize the following: leo at sylvester py_count $ python Python 2.6.3 (r263:75183, Oct 26 2009, 12:34:23) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> max = '5' >>> n = 5 >>> n >= max False >>> n + max Traceback (most recent call last): File "", line 1, in TypeError: unsupported operand type(s) for +: 'int' and 'str' >>> Section 5.9 Comparison describes this. Can someone give me examples of use cases TIA Hellmut -- Dr. Hellmut Weber mail at hellmutweber.de Degenfeldstra?e 2 tel +49-89-3081172 D-80803 M?nchen-Schwabing mobil +49-172-8450321 please: No DOCs, no PPTs. why: tinyurl.com/cbgq From __peter__ at web.de Sun Jan 10 07:36:06 2010 From: __peter__ at web.de (Peter Otten) Date: Sun, 10 Jan 2010 13:36:06 +0100 Subject: Prepend to logging message References: <5aa06ab5-61fd-46a8-9cac-7f3a2a5d382b@d20g2000yqh.googlegroups.com> Message-ID: Joan Miller wrote: > On 10 ene, 10:26, Peter Otten <__pete... at web.de> wrote: >> Joan Miller wrote: >> > How to prepend anything to a logging message? Is possible to do it >> > from the dictionary object (ExtraLog) or is there is that override >> > process() [1]? >> >> > ------------------ >> > class ExtraLog(object): >> >> > def __getitem__(self, name): >> > if name == 'foo': >> > result = 'testing' >> > return result >> >> > def __iter__(self): >> > keys = ['foo',] >> > keys.extend(self.__dict__.keys()) >> > return iter(keys) >> >> format = "foo=%(foo)s " + logging.BASIC_FORMAT >> logging.basicConfig(format=format) >> logger = logging.LoggerAdapter(logging.getLogger('foo'), ExtraLog()) >> logger.error("yadda") >> >> Is that what you want? >> >> Peter > > I want that a message can be modified before of being logged. i.e. for > "yadda" I would that were preppend 2 spaces. (And I want not manage > that in the format to manage the indentation of all text) Following Ishwor's advice: import logging class MyLoggerAdapter(logging.LoggerAdapter): def process(self, msg, kwargs): return "message->" + msg.upper(), kwargs logging.basicConfig() logger = MyLoggerAdapter(logging.getLogger('foo'), {}) logger.error("yadda") Peter From clp2 at rebertia.com Sun Jan 10 07:46:03 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 10 Jan 2010 04:46:03 -0800 Subject: integer and string compare, is that correct? In-Reply-To: <4B49C758.9050204@hellmutweber.de> References: <4B49C758.9050204@hellmutweber.de> Message-ID: <50697b2c1001100446u41d9f20dt37c3dba252c9d460@mail.gmail.com> On Sun, Jan 10, 2010 at 4:26 AM, Hellmut Weber wrote: > Hi, > being a causal python user (who likes the language quite a lot) > it took me a while to realize the following: > > > leo at sylvester py_count $ python > Python 2.6.3 (r263:75183, Oct 26 2009, 12:34:23) > [GCC 4.4.1] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> max = '5' >>>> n = 5 >>>> n >= max > False >>>> n + max > Traceback (most recent call last): > ?File "", line 1, in > TypeError: unsupported operand type(s) for +: 'int' and 'str' >>>> > > > Section 5.9 Comparison describes this. > > Can someone give me examples of use cases The behavior of disparate types being comparable is deprecated and has been removed in Python 3.0+; don't rely upon it. (The ordering used is arbitrary but consistent) IIRC, the feature existed in prior versions to make lists containing mixed types sortable; this was found to be not all that useful and to hide what are quite arguably errors. Cheers, Chris -- http://blog.rebertia.com From clp2 at rebertia.com Sun Jan 10 07:48:19 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 10 Jan 2010 04:48:19 -0800 Subject: integer and string compare, is that correct? In-Reply-To: <50697b2c1001100446u41d9f20dt37c3dba252c9d460@mail.gmail.com> References: <4B49C758.9050204@hellmutweber.de> <50697b2c1001100446u41d9f20dt37c3dba252c9d460@mail.gmail.com> Message-ID: <50697b2c1001100448j57cfe8bewe1e4f8521c02e40d@mail.gmail.com> On Sun, Jan 10, 2010 at 4:46 AM, Chris Rebert wrote: > The behavior of disparate types being comparable is deprecated and has > been removed in Python 3.0+; don't rely upon it. Clarification: Equality testing between disparate types still works unaltered however. By "comparable", I meant >, <, >=, <= comparisons. Cheers, Chris From peloko45 at gmail.com Sun Jan 10 08:10:57 2010 From: peloko45 at gmail.com (Joan Miller) Date: Sun, 10 Jan 2010 05:10:57 -0800 (PST) Subject: Prepend to logging message References: <5aa06ab5-61fd-46a8-9cac-7f3a2a5d382b@d20g2000yqh.googlegroups.com> Message-ID: <06b9dbb3-dbbf-4929-a3a1-9a50893a8eb8@k19g2000yqc.googlegroups.com> On 10 ene, 12:36, Peter Otten <__pete... at web.de> wrote: > Joan Miller wrote: > > On 10 ene, 10:26, Peter Otten <__pete... at web.de> wrote: > >> Joan Miller wrote: > >> > How to prepend anything to a logging message? Is possible to do it > >> > from the dictionary object (ExtraLog) or is there is that override > >> > process() [1]? > > >> > ------------------ > >> > class ExtraLog(object): > > >> > def __getitem__(self, name): > >> > if name == 'foo': > >> > result = 'testing' > >> > return result > > >> > def __iter__(self): > >> > keys = ['foo',] > >> > keys.extend(self.__dict__.keys()) > >> > return iter(keys) > > >> format = "foo=%(foo)s " + logging.BASIC_FORMAT > >> logging.basicConfig(format=format) > >> logger = logging.LoggerAdapter(logging.getLogger('foo'), ExtraLog()) > >> logger.error("yadda") > > >> Is that what you want? > > >> Peter > > > I want that a message can be modified before of being logged. i.e. for > > "yadda" I would that were preppend 2 spaces. (And I want not manage > > that in the format to manage the indentation of all text) > > Following Ishwor's advice: > > import logging > > class MyLoggerAdapter(logging.LoggerAdapter): > ? ? def process(self, msg, kwargs): > ? ? ? ? return "message->" + msg.upper(), kwargs > > logging.basicConfig() > logger = MyLoggerAdapter(logging.getLogger('foo'), {}) > logger.error("yadda") > > Peter Thanks! I had to see the code to override it correctly [1] -------------------- class LoggerAdapter_(logging.LoggerAdapter): def __init__(self, logger, extra): self.logger = logger self.extra = extra def process(self, msg, kwargs): kwargs["extra"] = self.extra return "message->" + msg.upper(), kwargs -------------------- [1] http://bitbucket.org/mirror/python-trunk/src/tip/Lib/logging/__init__.py#cl-1264 From __peter__ at web.de Sun Jan 10 08:30:31 2010 From: __peter__ at web.de (Peter Otten) Date: Sun, 10 Jan 2010 14:30:31 +0100 Subject: integer and string compare, is that correct? References: Message-ID: Hellmut Weber wrote: > being a causal python user (who likes the language quite a lot) > it took me a while to realize the following: > > > leo at sylvester py_count $ python > Python 2.6.3 (r263:75183, Oct 26 2009, 12:34:23) > [GCC 4.4.1] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> max = '5' > >>> n = 5 > >>> n >= max > False > Section 5.9 Comparison describes this. > > Can someone give me examples of use cases The use cases for an order that works across types like int and str are weak to non-existent. Implementing it was considered a mistake and has been fixed in Python 3: Python 3.1.1+ (r311:74480, Nov 2 2009, 15:45:00) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 5 > "5" Traceback (most recent call last): File "", line 1, in TypeError: unorderable types: int() > str() Checking for equality is still possible: >>> 5 == "5" False Peter From drncode1 at live.com Sun Jan 10 08:35:36 2010 From: drncode1 at live.com (flow) Date: 11 Jan 2010 00:35:36 +1100 Subject: "Advanced" Python programming book? Message-ID: <4b49d7a8$1@dnews.tpgi.com.au> I've just finished reading a sort of beginner Python book, and I know quite a bit now but I'm looking for a book that can teach me advanced aspects of Python - code optimisation, threading, etc. Any recommendations? Cheers. From peloko45 at gmail.com Sun Jan 10 08:49:35 2010 From: peloko45 at gmail.com (Joan Miller) Date: Sun, 10 Jan 2010 05:49:35 -0800 (PST) Subject: Prepend to logging message References: <5aa06ab5-61fd-46a8-9cac-7f3a2a5d382b@d20g2000yqh.googlegroups.com> <06b9dbb3-dbbf-4929-a3a1-9a50893a8eb8@k19g2000yqc.googlegroups.com> Message-ID: <808f3af7-4652-4ad3-8c3d-d2c25465846f@r24g2000yqd.googlegroups.com> On 10 ene, 13:10, Joan Miller wrote: > On 10 ene, 12:36, Peter Otten <__pete... at web.de> wrote: > > > > > Joan Miller wrote: > > > On 10 ene, 10:26, Peter Otten <__pete... at web.de> wrote: > > >> Joan Miller wrote: > > >> > How to prepend anything to a logging message? Is possible to do it > > >> > from the dictionary object (ExtraLog) or is there is that override > > >> > process() [1]? > > > >> > ------------------ > > >> > class ExtraLog(object): > > > >> > def __getitem__(self, name): > > >> > if name == 'foo': > > >> > result = 'testing' > > >> > return result > > > >> > def __iter__(self): > > >> > keys = ['foo',] > > >> > keys.extend(self.__dict__.keys()) > > >> > return iter(keys) > > > >> format = "foo=%(foo)s " + logging.BASIC_FORMAT > > >> logging.basicConfig(format=format) > > >> logger = logging.LoggerAdapter(logging.getLogger('foo'), ExtraLog()) > > >> logger.error("yadda") > > > >> Is that what you want? > > > >> Peter > > > > I want that a message can be modified before of being logged. i.e. for > > > "yadda" I would that were preppend 2 spaces. (And I want not manage > > > that in the format to manage the indentation of all text) > > > Following Ishwor's advice: > > > import logging > > > class MyLoggerAdapter(logging.LoggerAdapter): > > ? ? def process(self, msg, kwargs): > > ? ? ? ? return "message->" + msg.upper(), kwargs > > > logging.basicConfig() > > logger = MyLoggerAdapter(logging.getLogger('foo'), {}) > > logger.error("yadda") > > > Peter > > Thanks! > > I had to see the code to override it correctly [1] > > -------------------- > class LoggerAdapter_(logging.LoggerAdapter): > > ? ? def __init__(self, logger, extra): > ? ? ? ? self.logger = logger > ? ? ? ? self.extra = extra > > ? ? def process(self, msg, kwargs): > ? ? ? ? kwargs["extra"] = self.extra > ? ? ? ? return "message->" + msg.upper(), kwargs > -------------------- > > [1]http://bitbucket.org/mirror/python-trunk/src/tip/Lib/logging/__init__... Sorry! It isn't necessary to copy __init__(). From ryniek90 at gmail.com Sun Jan 10 10:19:41 2010 From: ryniek90 at gmail.com (Ryniek90) Date: Sun, 10 Jan 2010 16:19:41 +0100 Subject: "Advanced" Python programming book? In-Reply-To: References: Message-ID: <4B49F00D.6000704@gmail.com> > I've just finished reading a sort of beginner Python book, and I know > quite a bit now but I'm looking for a book that can teach me advanced > aspects of Python - code optimisation, threading, etc. > > Any recommendations? > > Cheers. Check those link: http://www.amazon.com/Beginning-Python-Visualization-Transformation-Professionals/dp/1430218436/ref=sr_1_11?ie=UTF8&s=books&qid=1263136036&sr=1-11 http://www.amazon.com/Gray-Hat-Python-Programming-Engineers/dp/1593271921/ref=sr_1_12?ie=UTF8&s=books&qid=1263136036&sr=1-12 http://www.amazon.com/Python-Cookbook-Alex-Martelli/dp/0596007973/ref=sr_1_4?ie=UTF8&s=books&qid=1263136036&sr=1-4 http://www.amazon.com/High-Performance-Python-Anthony-Lewis/dp/0596522088/ref=sr_1_14?ie=UTF8&s=books&qid=1263136082&sr=1-14 http://www.amazon.com/Natural-Language-Processing-Python-Steve/dp/0596516495/ref=sr_1_16?ie=UTF8&s=books&qid=1263136082&sr=1-16 http://www.amazon.com/Python-Scripting-Computational-Science-Engineering/dp/3540739157/ref=sr_1_17?ie=UTF8&s=books&qid=1263136082&sr=1-17 http://www.amazon.com/Expert-Python-Programming-practices-distributing/dp/184719494X/ref=sr_1_20?ie=UTF8&s=books&qid=1263136082&sr=1-20 http://www.djangobook.com/ Similar topics you can find in Google. Check python technologies, like: PyCuda, SciPy, NumPy, PyGame, Django, Pylons, Zope, TurboGears, Twisted, PyGTK, PyQt and so on. Check this link: http://www.python.org/about/apps/ and this http://wiki.python.org/moin/ Cheers :) From steve at REMOVE-THIS-cybersource.com.au Sun Jan 10 10:30:12 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 10 Jan 2010 15:30:12 GMT Subject: lightweight encryption of text file References: Message-ID: <0359df54$0$1336$c3e8da3@news.astraweb.com> On Sun, 10 Jan 2010 09:59:31 +0100, Daniel Fetchinson wrote: > Thanks, this looks very simple too, but where is the decryption code? > Wikipedia seems to suggest that encryption and decryption are both the > same but running crypt on the output of crypt doesn't give back the > original string. So probably I'm misunderstanding something. Yes, the nature of a stream cipher :) What you're probably doing is what I did, before I had my Aha!!! moment: >>> arc = arcfour("password") >>> plaintext = "attack at dawn" >>> ciphertext = arc.crypt(plaintext) >>> print plaintext; print ciphertext.encode("hex").upper() attack at dawn 6371736C6E7C3F495C185629210B >>> x = arc.crypt(ciphertext) >>> assert x == plaintext Traceback (most recent call last): File "", line 1, in AssertionError >>> x '\x16\xf7\xf1\xcc\xda\xb5\xe0\xbf\x0b\x13 bF\x8f' So what's going on? Consider: Because Arcfour uses xor for the encryption step, decryption is exactly the same. So you only need one method to do both. But because Arcfour stores state, calling it twice in a row doesn't give the same result: >>> arc.crypt("hello").encode("hex").upper() 'CAA48DE953' >>> arc.crypt("hello").encode("hex").upper() '03405412CA' Arcfour is a stream cipher. When you call it twice on two different strings, that is logically equivalent to calling it once on a single long string made up of concatenating the two strings together. Each time you encrypt a single character, the internal state ("self.s") changes. To undo the change, you need the same state. The easiest way to do this is by creating a new instance: >>> encrypter = arcfour("password") >>> decrypter = arcfour("password") >>> plaintext = "attack at dawn" >>> ciphertext = encrypter.crypt(plaintext) >>> assert decrypter.crypt(ciphertext) == plaintext >>> So long as the two instances stay in lock-step (every time you use up a byte from the keystream in one, you do the same in the other) you can use one to decrypt the output of the other. It doesn't even really matter which one you use: >>> x = decrypter.crypt("Nobody expects the Spanish Inquisition!!!") >>> encrypter.crypt(x) 'Nobody expects the Spanish Inquisition!!!' In summary: use the arcfour class to create a stream. If you are just encrypting, or just decrypting, you can use one stream, or as many streams as you like, using different keys. But to do both, you need two streams, initiated with the same key, and kept in lockstep. The advantage of a stream cipher is that you can encrypt a text without needing all the text at once, and then decrypt it the same way: >>> output = [] >>> output.append(encrypt.crypt("abcdefghi")) >>> output.append(encrypt.crypt("jklmno")) >>> output.append(encrypt.crypt("p")) >>> output.append(encrypt.crypt("qrstuvwxyz")) >>> output = ''.join(output) >>> >>> plain = [] >>> plain.append(decrypt.crypt(output[0:20])) >>> plain.append(decrypt.crypt(output[20:24])) >>> plain.append(decrypt.crypt(output[24:])) >>> ''.join(plain) 'abcdefghijklmnopqrstuvwxyz' -- Steven From contact at pythonxy.com Sun Jan 10 10:50:20 2010 From: contact at pythonxy.com (Pierre Raybaut) Date: Sun, 10 Jan 2010 16:50:20 +0100 Subject: [ANN] Spyder v1.0.2 released Message-ID: <4B49F73C.9060402@pythonxy.com> Hi all, I'm pleased to announce here that Spyder version 1.0.2 has been released: http://packages.python.org/spyder Previously known as Pydee, Spyder (Scientific PYthon Development EnviRonment) is a free open-source Python development environment providing MATLAB-like features in a simple and light-weighted software, available for Windows XP/Vista/7, GNU/Linux and MacOS X: * advanced code editing features (code analysis, ...) * interactive console with MATLAB-like workpace (with GUI-based list, dictionary, tuple, text and array editors -- screenshots: http://packages.python.org/spyder/console.html#the-workspace) and integrated matplotlib figures * external console to open an interpreter or run a script in a separate process (with a global variable explorer providing the same features as the interactive console's workspace) * code analysis with pyflakes and pylint * search in files features * documentation viewer: automatically retrieves docstrings or source code of the function/class called in the interactive/external console * integrated file/directories explorer * MATLAB-like path management ...and more! Spyder is part of spyderlib, a Python module based on PyQt4 and QScintilla2 which provides powerful console-related PyQt4 widgets. Spyder v1.0.2 is a bugfix release: * External console: subprocess python calls were using the external console's sitecustomize.py (instead of system sitecustomize.py) * Added workaround for PyQt4 v4.6+ major bug with matplotlib * Added option to customize the way matplotlib figures are embedded (docked or floating window) * Matplotlib's "Option" dialog box is now supporting subplots * Array editor now supports complex arrays * Editor: replaced "Run selection or current line" option by "Run selection or current block" (without selection, this feature is similar to MATLAB's cell mode) * ...and a lot of minor bugfixes. - Pierre From nobody at nowhere.com Sun Jan 10 11:20:45 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 10 Jan 2010 16:20:45 +0000 Subject: lightweight encryption of text file References: Message-ID: On Sun, 10 Jan 2010 09:59:31 +0100, Daniel Fetchinson wrote: > Thanks, this looks very simple too, but where is the decryption code? > Wikipedia seems to suggest that encryption and decryption are both the > same but running crypt on the output of crypt doesn't give back the > original string. So probably I'm misunderstanding something. The encryption is stateful (it wouldn't be very good if it wasn't), so you need to create and initialise a new arcfour instance for decryption; you can't re-use the existing instance. From nobody at nowhere.com Sun Jan 10 11:24:53 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 10 Jan 2010 16:24:53 +0000 Subject: lightweight encryption of text file References: <0359df54$0$1336$c3e8da3@news.astraweb.com> Message-ID: On Sun, 10 Jan 2010 15:30:12 +0000, Steven D'Aprano wrote: >> Thanks, this looks very simple too, but where is the decryption code? >> Wikipedia seems to suggest that encryption and decryption are both the >> same but running crypt on the output of crypt doesn't give back the >> original string. So probably I'm misunderstanding something. > > Yes, the nature of a stream cipher :) It isn't limited to stream ciphers. You would have the same issue for a block cipher with chaining (i.e. *not* ECB mode). From nobody at nowhere.com Sun Jan 10 11:34:46 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 10 Jan 2010 16:34:46 +0000 Subject: integer and string compare, is that correct? References: Message-ID: Hellmut Weber wrote: >> being a causal python user (who likes the language quite a lot) >> it took me a while to realize the following: >> >>> max = '5' >> >>> n = 5 >> >>> n >= max >> False > >> Section 5.9 Comparison describes this. >> >> Can someone give me examples of use cases Peter Otten wrote: > The use cases for an order that works across types like int and str are weak > to non-existent. Implementing it was considered a mistake and has been fixed > in Python 3: >>>> 5 > "5" > Traceback (most recent call last): > File "", line 1, in > TypeError: unorderable types: int() > str() If you actually need to perform comparisons across types, you can rely upon the fact that tuple comparisons are non-strict and use e.g.: > a = 5 > b = '5' > (type(a).__name__, a) < (type(b).__name__, b) True > (type(a).__name__, a) > (type(b).__name__, b) False The second elements will only be compared if the first elements are equal (i.e. the values have the same type). From tomslists at netp.org Sun Jan 10 11:44:21 2010 From: tomslists at netp.org (Tom Pacheco) Date: Sun, 10 Jan 2010 11:44:21 -0500 Subject: "Advanced" Python programming book? In-Reply-To: <4B49F00D.6000704@gmail.com> References: <4B49F00D.6000704@gmail.com> Message-ID: <4B4A03E5.2000205@netp.org> Ryniek90 wrote: >> I've just finished reading a sort of beginner Python book, and I know >> quite a bit now but I'm looking for a book that can teach me advanced >> aspects of Python - code optimisation, threading, etc. >> >> Any recommendations? >> >> Cheers. >> > Check those link: > http://www.amazon.com/Beginning-Python-Visualization-Transformation-Professionals/dp/1430218436/ref=sr_1_11?ie=UTF8&s=books&qid=1263136036&sr=1-11 > > http://www.amazon.com/Gray-Hat-Python-Programming-Engineers/dp/1593271921/ref=sr_1_12?ie=UTF8&s=books&qid=1263136036&sr=1-12 > > http://www.amazon.com/Python-Cookbook-Alex-Martelli/dp/0596007973/ref=sr_1_4?ie=UTF8&s=books&qid=1263136036&sr=1-4 > > http://www.amazon.com/High-Performance-Python-Anthony-Lewis/dp/0596522088/ref=sr_1_14?ie=UTF8&s=books&qid=1263136082&sr=1-14 > > http://www.amazon.com/Natural-Language-Processing-Python-Steve/dp/0596516495/ref=sr_1_16?ie=UTF8&s=books&qid=1263136082&sr=1-16 > > http://www.amazon.com/Python-Scripting-Computational-Science-Engineering/dp/3540739157/ref=sr_1_17?ie=UTF8&s=books&qid=1263136082&sr=1-17 > > http://www.amazon.com/Expert-Python-Programming-practices-distributing/dp/184719494X/ref=sr_1_20?ie=UTF8&s=books&qid=1263136082&sr=1-20 > > http://www.djangobook.com/ > > > Similar topics you can find in Google. > Check python technologies, like: PyCuda, SciPy, NumPy, PyGame, Django, > Pylons, Zope, TurboGears, Twisted, PyGTK, PyQt and so on. > > Check this link: http://www.python.org/about/apps/ and this > http://wiki.python.org/moin/ > > Cheers :) > some good books.. i might add this to the list http://www.amazon.com/Python-Essential-Reference-David-Beazley/dp/0672329786/ref=pd_sim_b_2 From fetchinson at googlemail.com Sun Jan 10 11:48:02 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Sun, 10 Jan 2010 17:48:02 +0100 Subject: lightweight encryption of text file In-Reply-To: <0359df54$0$1336$c3e8da3@news.astraweb.com> References: <0359df54$0$1336$c3e8da3@news.astraweb.com> Message-ID: >> Thanks, this looks very simple too, but where is the decryption code? >> Wikipedia seems to suggest that encryption and decryption are both the >> same but running crypt on the output of crypt doesn't give back the >> original string. So probably I'm misunderstanding something. > > Yes, the nature of a stream cipher :) > > What you're probably doing is what I did, before I had my Aha!!! moment: > > >>>> arc = arcfour("password") >>>> plaintext = "attack at dawn" >>>> ciphertext = arc.crypt(plaintext) >>>> print plaintext; print ciphertext.encode("hex").upper() > attack at dawn > 6371736C6E7C3F495C185629210B >>>> x = arc.crypt(ciphertext) >>>> assert x == plaintext > Traceback (most recent call last): > File "", line 1, in > AssertionError >>>> x > '\x16\xf7\xf1\xcc\xda\xb5\xe0\xbf\x0b\x13 bF\x8f' > > > So what's going on? Consider: > > Because Arcfour uses xor for the encryption step, decryption is exactly > the same. So you only need one method to do both. > > But because Arcfour stores state, calling it twice in a row doesn't give > the same result: > >>>> arc.crypt("hello").encode("hex").upper() > 'CAA48DE953' >>>> arc.crypt("hello").encode("hex").upper() > '03405412CA' > > > Arcfour is a stream cipher. When you call it twice on two different > strings, that is logically equivalent to calling it once on a single long > string made up of concatenating the two strings together. Each time you > encrypt a single character, the internal state ("self.s") changes. To > undo the change, you need the same state. The easiest way to do this is > by creating a new instance: > > >>>> encrypter = arcfour("password") >>>> decrypter = arcfour("password") >>>> plaintext = "attack at dawn" >>>> ciphertext = encrypter.crypt(plaintext) >>>> assert decrypter.crypt(ciphertext) == plaintext >>>> > > > So long as the two instances stay in lock-step (every time you use up a > byte from the keystream in one, you do the same in the other) you can use > one to decrypt the output of the other. It doesn't even really matter > which one you use: > >>>> x = decrypter.crypt("Nobody expects the Spanish Inquisition!!!") >>>> encrypter.crypt(x) > 'Nobody expects the Spanish Inquisition!!!' > > > In summary: use the arcfour class to create a stream. If you are just > encrypting, or just decrypting, you can use one stream, or as many > streams as you like, using different keys. But to do both, you need two > streams, initiated with the same key, and kept in lockstep. > > The advantage of a stream cipher is that you can encrypt a text without > needing all the text at once, and then decrypt it the same way: > >>>> output = [] >>>> output.append(encrypt.crypt("abcdefghi")) >>>> output.append(encrypt.crypt("jklmno")) >>>> output.append(encrypt.crypt("p")) >>>> output.append(encrypt.crypt("qrstuvwxyz")) >>>> output = ''.join(output) >>>> >>>> plain = [] >>>> plain.append(decrypt.crypt(output[0:20])) >>>> plain.append(decrypt.crypt(output[20:24])) >>>> plain.append(decrypt.crypt(output[24:])) >>>> ''.join(plain) > 'abcdefghijklmnopqrstuvwxyz' Thanks Steven, this was very helpful! Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From no.email at nospam.invalid Sun Jan 10 11:54:51 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 10 Jan 2010 08:54:51 -0800 Subject: lightweight encryption of text file References: Message-ID: <7xfx6dhpd0.fsf@ruckus.brouhaha.com> Nobody writes: > RC4 (aka ArcFour) is quite trivial to implement, and better than inventing > your own cipher or using a Vignere: ... That's a cute implementation, but it has no authentication and doesn't include any randomness, which means if you use the same key for two inputs, there is a security failure (xor'ing the two ciphertexts reveals the xor of the plaintexts). It also looks rather slow. I don't make any guarantees about p3.py, but it has been reviewed by several experts and appears to be reasonably sound for the type of casual use being discussed here, and it is tuned for speed (given the implementation constraints). For more demanding purposes, you should use a more serious library like one of the OpenSSL wrappers. From victorsubervi at gmail.com Sun Jan 10 12:07:30 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sun, 10 Jan 2010 12:07:30 -0500 Subject: Academic Question Message-ID: <4dc0cfea1001100907t6bc032cftea2e9e258e7c1937@mail.gmail.com> Hi; The following code that works: #! /usr/bin/python import string import cgitb; cgitb.enable() import cgi import MySQLdb import sys,os from sets import Set import fpformat cwd = os.getcwd() sys.path.append(cwd) from login import login from particulars import ourOptions form = cgi.FieldStorage() store = form.getfirst('store') cat = form.getfirst('cat') id = form.getfirst('id') patientID = form.getfirst('patientID') try: browser = form.getfirst('browser', 'all') except: browser = headers() os.chdir('%s/..' % cwd) sys.path.append(os.getcwd()) from templateFrame import top, bottom os.chdir(cwd) Why doesn't it work if I move the bottom imports to the top? The form values get lost, even though I chdir to cwd. TIA, beno -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Sun Jan 10 12:17:50 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sun, 10 Jan 2010 12:17:50 -0500 Subject: Something More Elegant In-Reply-To: <7a9c25c21001091200r10c3c7b2jbc16040efa9f7b34@mail.gmail.com> References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> <4dc0cfea1001090715md97518tf618309eda1deabb@mail.gmail.com> <7a9c25c21001091200r10c3c7b2jbc16040efa9f7b34@mail.gmail.com> Message-ID: <4dc0cfea1001100917v1021e052y104642295303d7b6@mail.gmail.com> On Sat, Jan 9, 2010 at 3:00 PM, Stephen Hansen wrote: > On Sat, Jan 9, 2010 at 7:15 AM, Victor Subervi wrote: > >> On Sat, Jan 9, 2010 at 9:35 AM, Steve Holden wrote: >> >>> But we are now in the realm of theory as far as you are concerned, since >>> you have already stated several times that you aren't interested in >>> correcting your design until after you have got the current mess into >>> production. So good luck with that. >>> >> >> And if you were in my shoes, I'm sure you'd do the same thing. > > > ... Really, no, he wouldn't :) > Let's not argue an academic point. > We're not all just hobbyists here who only do work for random open source > projects. A lot of us are professionals who actually do have clients, > actually do have deadlines, actually do have an understanding for production > requirements. Getting something into production as soon as possible is > certainly an important goal in commercial work. But it is not the only goal. > Proper database design is very important because if you don't do it, you'll > actually end up usually wasting *more* time and effort then if you just bite > the bullet and fix it now. > Clearly. That's why when it was pointed out to me, I jumped on it. > > Proper database design -- in particular in your case, not having multiple > tables with various names that even need "%sThis" or "%sThat", and using > parameterized queries to access those tables, is really important. It will > save you time, it will save you effort, and it'll save you money-- because > /not/ doing it is among other things, a major security risk. Getting code > out fast and now is a noble goal in commercial projects, getting code out > which is by design prone to attacks by hackers is negligence. > You're preaching to the choir ;) Read the above. The only exception to this is putting all store stuff in the same table. The columns of each store table are different and automatically created through what I believe are called "mixins". > > Well, it *is* working now :)) And I am interested in cleaning this up. I >> should probably start with the matter of databases, since that's something I >> won't be able to easily change once clients actually start entering data. >> Please share with me any further concepts or questions to get me thinking >> how to redesign the databases. >> > > The next thing to do is to re-do your SQL queries. You should never do > string interpolation, e.g: > > SQL="SELECT x FROM y WHERE z = %s" % (arg,) > cur.execute(SQL) > > If you have done the first step, your tables always have a set names so you > never need to interpolate to do the queries-- and then at this point, under > no circumstances ever, ever-- consider this a law that you will get fined > for violation-- use string interpolation to generate your queries. > > Instead, do: > > cur.execute("SELECT x FROM y WHERE z = %s", (arg,)) > > That looks really similar, but is lightyears away. Its very unfortunate > that some database drivers use 'format' as the paramstyle because it > confuses issues, but the two effects are very different. In one, Python is > just munging together and creating a new string. In the other, the database > driver is doing a few things. Its analyzing the query, its storing it (often > caching it, which speeds up further executions of that query), etc, etc, and > then finally its seeing that you are passing arguments into it, and it is > -safely- binding those arguments into the expression; this prevents SQL > Injection attacks. You can use interpolation and prevent injection if you > -meticulously- check -every- string that comes from the user, and -never- > trust it (even if that string was written out to a hidden and > legitimate users have no way to alter, because illegitimate users will alter > it anyways). Or you can use parameterized queries and just avoid it, while > getting plenty of other benefits as well. > > > At work, we had a third-party package that we re-sold as part of our > offering, and glancing over its source, I noticed something. It did, in > essence to check login: > > cur.execute("SELECT user_id FROM usertable WHERE username = '%s' AND > password = '%s' % (username, password)) > > I blinked, and emailed them to point out the problem. I suggested they log > in as: > > Username = dummyuser > Password = '; DROP usertable > > You see, when using interpolation, the string that got sent to the database > was: > > SELECT user_id FROM usertable WHERE username = 'dummyuser' AND password > = ''; DROP usertable > > And thus, from the login screen of this app-- I destroyed their > environment. > > Its sort of important that you not put code out into production which is > susceptible to such things. Your clients will not at all appreciate it if > and when some hacker discovers it and destroys their site, loosing them > money. But even beyond that, there are many other benefits to just doing > this right. You want to, believe me. Now, before you put anything into > production. It might take a day or two longer, but its worth it. > > Finally, go finish getting rid of the bare excepts, I saw one recently :) > Just rip them all out, every one. All at once. Then fix any errors that come > along with specific excepts or pre-tests if appropriate. ;) > Frankly, none of this will take too long (except normalization, of course). And I'm on every one of them. Thank you very much! beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From wolftracks at invalid.com Sun Jan 10 12:28:15 2010 From: wolftracks at invalid.com (W. eWatson) Date: Sun, 10 Jan 2010 09:28:15 -0800 Subject: Fractional Hours from datetime? Message-ID: Maybe there's a more elegant way to do this. I want to express the result of datetime.datetime.now() in fractional hours. Here's one way. dt=datetime.datetime.now() xtup = dt.timetuple() h = xtup[3]+xtup[4]/60.0+xtup[5]/3600.00+xtup[6]/10**6 # now is in fractions of an hour From __peter__ at web.de Sun Jan 10 12:37:27 2010 From: __peter__ at web.de (Peter Otten) Date: Sun, 10 Jan 2010 18:37:27 +0100 Subject: integer and string compare, is that correct? References: Message-ID: Somebody wrote: > If you actually need to perform comparisons across types, you can rely > upon the fact that tuple comparisons are non-strict and use e.g.: > > > a = 5 > > b = '5' > > (type(a).__name__, a) < (type(b).__name__, b) > True > > (type(a).__name__, a) > (type(b).__name__, b) > False > > The second elements will only be compared if the first elements are equal > (i.e. the values have the same type). The same type *name*. To play it safe you'll have to compare the id, too: >>> items = [] >>> for i in range(2): ... class A: pass ... items.append(A()) ... >>> sorted(items, key=lambda a: (type(a).__name__, a)) Traceback (most recent call last): File "", line 1, in TypeError: unorderable types: A() < A() >>> sorted(items, key=lambda a: (type(a).__name__, id(type(a)), a)) [<__main__.A object at 0x14dfbd0>, <__main__.A object at 0x14dfc50>] Peter From aahz at pythoncraft.com Sun Jan 10 12:48:02 2010 From: aahz at pythoncraft.com (Aahz) Date: 10 Jan 2010 09:48:02 -0800 Subject: problem with cheetah References: <0fd84b05-cf12-485b-a14e-608e47679b32@s20g2000yqd.googlegroups.com> Message-ID: In article <0fd84b05-cf12-485b-a14e-608e47679b32 at s20g2000yqd.googlegroups.com>, mlowicki wrote: > >Hi!, i get such error when I try to install cheetah: Probably better to ask on the Cheetah list: https://lists.sourceforge.net/lists/listinfo/cheetahtemplate-discuss -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." --Red Adair From peloko45 at gmail.com Sun Jan 10 14:04:08 2010 From: peloko45 at gmail.com (Joan Miller) Date: Sun, 10 Jan 2010 11:04:08 -0800 (PST) Subject: Milliseconds in logging format Message-ID: How the logging '%(asctime)s' [1] specifier to gets the millisecond portion of the time if there is not a directive to get it from the time module [2] ? "The date format string follows the requirements of strftime()" [1] http://docs.python.org/library/logging.html#basic-example [2] http://docs.python.org/library/time.html#time.strftime From python at mrabarnett.plus.com Sun Jan 10 14:09:17 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 10 Jan 2010 19:09:17 +0000 Subject: Academic Question In-Reply-To: <4dc0cfea1001100907t6bc032cftea2e9e258e7c1937@mail.gmail.com> References: <4dc0cfea1001100907t6bc032cftea2e9e258e7c1937@mail.gmail.com> Message-ID: <4B4A25DD.4040703@mrabarnett.plus.com> Victor Subervi wrote: > Hi; > The following code that works: > > #! /usr/bin/python > > import string > import cgitb; cgitb.enable() > import cgi > import MySQLdb > import sys,os > from sets import Set > import fpformat > cwd = os.getcwd() > sys.path.append(cwd) > from login import login > from particulars import ourOptions > > form = cgi.FieldStorage() > store = form.getfirst('store') > cat = form.getfirst('cat') > id = form.getfirst('id') > patientID = form.getfirst('patientID') > > try: > browser = form.getfirst('browser', 'all') > except: > browser = headers() > A bare except, and if an exception _does_ occur, they'll be a NameError because 'headers' isn't defined. > os.chdir('%s/..' % cwd) > sys.path.append(os.getcwd()) > from templateFrame import top, bottom > os.chdir(cwd) > > Why doesn't it work if I move the bottom imports to the top? The form > values get lost, even though I chdir to cwd. > I try to avoid changing the directory. From awilliam at opengroupware.us Sun Jan 10 14:45:24 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Sun, 10 Jan 2010 14:45:24 -0500 Subject: "Bad file descriptor" in HTTPServer using Multiprocessing. Message-ID: <1263152724.13227.5.camel@linux-m3mt> I have a Python multiprocessing application where a master process starts server sub-processes and communicates with them via Pipes; that works very well. But one of the subprocesses, in turn, starts a collection of HTTPServer 'workers' (almost exactly as demonstrated in the docs). This works perfectly.... so long as the subprocess that starts the HTTPServer workers is the *first* process started by the master process. Otherwise the HTTPServer's serve_forever immediately abends with: "(9, 'Bad file descriptor')" I'm confused why the order of starting the processes matter, especially given that the HTTPServer's are start in a subprocess of a subprocess. The master doesn't do much of anything between starting each subprocess (which it does in a loop; identify all the subprocesses to start, and start them.). subprocess -------------------- ...... self._httpd = HTTPServer((HTTP_HOST, HTTP_PORT), HTTPRequestHandler) print 'HTTPServer created.' for i in range(10): p = multiprocessing.Process(target=serve_forever, args=(self._httpd, i, self)) self._workers.append(p) p.start() ..... def serve_forever(server, i, control): print 'coils.http starting HTTP worker #{0}'.format(i) try: server.serve_forever() except KeyboardInterrupt: pass except Exception, e: control.log.exception(e) print 'coils.http worker #{0} abended with exception.'.format(i) print e return -- OpenGroupware developer: awilliam at whitemice.org OpenGroupare & Cyrus IMAPd documenation @ From debatem1 at gmail.com Sun Jan 10 14:59:52 2010 From: debatem1 at gmail.com (geremy condra) Date: Sun, 10 Jan 2010 14:59:52 -0500 Subject: lightweight encryption of text file In-Reply-To: References: Message-ID: Not sure why in the world you would homebrew something like this- a small dependency isn't that bad, and aes can be pretty simple to use. Might as well go for the industrial strength approach. Geremy Condra From irmen-NOSPAM- at xs4all.nl Sun Jan 10 15:05:06 2010 From: irmen-NOSPAM- at xs4all.nl (Irmen de Jong) Date: Sun, 10 Jan 2010 21:05:06 +0100 Subject: Milliseconds in logging format In-Reply-To: References: Message-ID: <4b4a32a6$0$22940$e4fe514c@news.xs4all.nl> On 10-1-2010 20:04, Joan Miller wrote: > How the logging '%(asctime)s' [1] specifier to gets the millisecond > portion of the time if there is not a directive to get it from the > time module [2] ? > > > "The date format string follows the requirements of strftime()" > [1] http://docs.python.org/library/logging.html#basic-example > > [2] http://docs.python.org/library/time.html#time.strftime It appends them to the string that strftime() returned. If you set a custom datefmt, it will stop doing this and you'll have to use %(msecs) in your log format to get them back. See the source, Lib/logging/__init__.py around line 400. (Python 2.6.4) -irmen From bryanvick at gmail.com Sun Jan 10 15:21:40 2010 From: bryanvick at gmail.com (Bryan) Date: Sun, 10 Jan 2010 12:21:40 -0800 (PST) Subject: How to use list type generated by SWIG? Message-ID: I am writing a small script to manage my ipod. I am using the python bindings for libgpod. I have never used swig, or used python to program against a c/c++ library. I can get the library to find my ipod, and parse its DB format, but I'm not sure how to interact with the types that some of the functions return, specifically a Swig Glist (gtk's doubly-linked list) import gpod db = gpod.itdb_parse('/media/ipod', None) print db.tracks >>> Glist is not iterable or subscriptable. It contains a 'next' function which always seems to return None after printing "swig/python detected a memory leak of type 'GList *', no destructor found." to the console. How can I interact with this list? From no.email at nospam.invalid Sun Jan 10 15:26:05 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 10 Jan 2010 12:26:05 -0800 Subject: lightweight encryption of text file References: Message-ID: <7xiqb9yaea.fsf@ruckus.brouhaha.com> geremy condra writes: > Not sure why in the world you would homebrew something like this- a > small dependency isn't that bad, and aes can be pretty simple to use. > Might as well go for the industrial strength approach. In my experience, 1) small dependencies ARE that bad, since they mean you have to develop and test on every platform that you want your code to run on; 2) using a serious library requires quite a bit of knowledge and decision-making which not everyone is equipped to do. "AES" is not so simple to use unless you know what you're doing in terms of modes, nonces, etc. Having supported this kind of package in a commercial setting in the past, IMO, for the sort of (common) application in question, it's best to keep things as simple as possible and supply a single interface that provides encryption, authentication, and random initialization all in one call. The cost is a little bit of ciphertext bloat, but it prevents all kinds of security failures frequently overlooked by novices. I'd like it a lot if the Python stdlib could include a serious cryptography module. That was rejected for regulatory reasons several years ago, but maybe things are changing enough that the issue can be revisited sometime. From 3lvss0809 at gmail.com Sun Jan 10 15:27:58 2010 From: 3lvss0809 at gmail.com (3lvss0809 at gmail.com) Date: Sun, 10 Jan 2010 12:27:58 -0800 (PST) Subject: Microsoft Office Word and Python (Win XP) References: Message-ID: <86b15d62-0c8f-4341-980b-0ec99f15f1f4@21g2000yqj.googlegroups.com> no idea :-( From 3lvss0809 at gmail.com Sun Jan 10 15:30:33 2010 From: 3lvss0809 at gmail.com (3lvss0809 at gmail.com) Date: Sun, 10 Jan 2010 12:30:33 -0800 (PST) Subject: Microsoft Office Word and Python (Win XP) References: Message-ID: so does anyone know how I could do this? From matt at bubblegen.co.uk Sun Jan 10 16:39:49 2010 From: matt at bubblegen.co.uk (Matthew Lear) Date: Sun, 10 Jan 2010 21:39:49 +0000 Subject: getopt not raising exception In-Reply-To: <4B49C0B7.2000702@mattnordhoff.com> References: <4B49BA50.8050207@bubblegen.co.uk> <4B49C0B7.2000702@mattnordhoff.com> Message-ID: <4B4A4925.9000805@bubblegen.co.uk> Matt Nordhoff wrote: > BTW: Checked out optparse? It's bigger and sexier! Thanks for the tip. Works a treat. -- Matt From emmanuel.surleau at gmail.com Sun Jan 10 17:08:28 2010 From: emmanuel.surleau at gmail.com (Emmanuel Surleau) Date: Sun, 10 Jan 2010 23:08:28 +0100 Subject: getopt not raising exception In-Reply-To: <4B4A4925.9000805@bubblegen.co.uk> References: <4B49BA50.8050207@bubblegen.co.uk> <4B49C0B7.2000702@mattnordhoff.com> <4B4A4925.9000805@bubblegen.co.uk> Message-ID: <201001102308.28179.emmanuel.surleau@gmail.com> > Matt Nordhoff wrote: > > BTW: Checked out optparse? It's bigger and sexier! If you're doing things with positional arguments, you should consider using argparse (http://argparse.googlecode.com/svn/tags/r101/doc/index.html). It's like optparse, but (much) better. Cheers, Emm From stef.mientki at gmail.com Sun Jan 10 18:48:22 2010 From: stef.mientki at gmail.com (Stef Mientki) Date: Mon, 11 Jan 2010 00:48:22 +0100 Subject: class viewer ? Message-ID: <4B4A6746.9020405@gmail.com> hello, I'd like to have a class viewer, something different from pydoc, and I wonder if someone has made something similar. from the given class, it's ancestors and it's derived classes, I'ld like to get the following information in a tree like structure: - the file were the class is definied - the attributes, split in inherited / created / overriden - the methodes, split in inherited / created / overriden - the files were instances of the class are created - and probably I forget a few any suggestions ? thanks, Stef Mientki From zabin.farishta at gmail.com Sun Jan 10 19:10:01 2010 From: zabin.farishta at gmail.com (Zabin) Date: Sun, 10 Jan 2010 16:10:01 -0800 (PST) Subject: setTextAlignment function Qtablewidget PyQt Message-ID: Hey! I am trying to align contents of some table cells but find the code below working for some fields and not for others. I am stuck as to why this is happening. Help will be gretly appreciated! setTextAlignment(QtCore.Qt.AlignVCenter) Cheers Zabin From mail at hellmutweber.de Sun Jan 10 19:44:30 2010 From: mail at hellmutweber.de (Hellmut Weber) Date: Mon, 11 Jan 2010 01:44:30 +0100 Subject: integer and string compare, is that correct? In-Reply-To: References: Message-ID: <4B4A746E.4010109@hellmutweber.de> Hi, thanks to all who answered. I'm using Python 2.6.5 on my machine and consulted the corresponding documentation. I do appreciate the modified definition of python 3, that seems much more reasonable. Thanks for indicating. Greetings from Munich in Winter Hellmut Am 10.01.2010 17:34, schrieb Nobody: > Hellmut Weber wrote: > >>> being a causal python user (who likes the language quite a lot) >>> it took me a while to realize the following: > >>> >>> max = '5' >>> >>> n = 5 >>> >>> n>= max >>> False >> >>> Section 5.9 Comparison describes this. >>> >>> Can someone give me examples of use cases > > Peter Otten wrote: > >> The use cases for an order that works across types like int and str are weak >> to non-existent. Implementing it was considered a mistake and has been fixed >> in Python 3: > >>>>> 5> "5" >> Traceback (most recent call last): >> File "", line 1, in >> TypeError: unorderable types: int()> str() > > If you actually need to perform comparisons across types, you can rely > upon the fact that tuple comparisons are non-strict and use e.g.: > > > a = 5 > > b = '5' > > (type(a).__name__, a)< (type(b).__name__, b) > True > > (type(a).__name__, a)> (type(b).__name__, b) > False > > The second elements will only be compared if the first elements are equal > (i.e. the values have the same type). > -- Dr. Hellmut Weber mail at hellmutweber.de Degenfeldstra?e 2 tel +49-89-3081172 D-80803 M?nchen-Schwabing mobil +49-172-8450321 please: No DOCs, no PPTs. why: tinyurl.com/cbgq From MLMDev at Comcast.net Sun Jan 10 19:45:59 2010 From: MLMDev at Comcast.net (Mitchell L Model) Date: Sun, 10 Jan 2010 19:45:59 -0500 Subject: sys.stdout vs. sys.stderr Message-ID: <6C60CA7B-E01C-449F-B956-E9E58A0ADD34@Comcast.net> In Python 3.1 is there any difference in the buffering behavior of the initial sys.stdout and sys.stderr streams? They are both line_buffered and stdout doesn't seem to use a larger-grain buffering, so they seem to be identical with respect to buffering. Were they different at some earlier point in Python's evolution? From zabin.farishta at gmail.com Sun Jan 10 20:07:10 2010 From: zabin.farishta at gmail.com (Zabin) Date: Sun, 10 Jan 2010 17:07:10 -0800 (PST) Subject: setTextAlignment function Qtablewidget PyQt References: Message-ID: <10cb3536-abb7-4962-b1b9-0a4b07b26c5b@35g2000yqa.googlegroups.com> On Jan 11, 1:10?pm, Zabin wrote: > Hey! > > I am trying to align contents of some table cells but find the code > below working for some fields and not for others. I am stuck as to why > this is happening. Help will be gretly appreciated! > > setTextAlignment(QtCore.Qt.AlignVCenter) > > Cheers > Zabin Figured it out! It was just the '\n' character being read with the line from the text file that messed up the formatting. just needed to use the line.strip() function before passing it to the table as a table widget item. From tjreedy at udel.edu Sun Jan 10 20:43:15 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 10 Jan 2010 20:43:15 -0500 Subject: Computer Music Toolkit (CMT) ? In-Reply-To: References: Message-ID: On 1/10/2010 4:15 AM, Peter Billam wrote: > Greetings. Is there a way to get at the Computer Music Toolkit (CMT) > http://www.ladspa.org/cmt/ > functionality from Python (Python3 in my case) ? You can access compiled C shared libraries most easily via the ctypes module. Searching Python CMT "Computer Music Toolkit" (following Stefan's suggestion) returns a few hits that might be useful. Searching PyPI for CMT does not turn up any relevant packages, so if you do develop a ctypes wrapping, consider contributing it. Terry Jan Reedy From tjreedy at udel.edu Sun Jan 10 21:36:37 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 10 Jan 2010 21:36:37 -0500 Subject: Python Goldmine has been updated: http://preciseinfo.org/Convert/index_Convert_Python.html In-Reply-To: References: Message-ID: On 1/8/2010 11:50 AM, tanix wrote: > Python Goldmine collection contains the extensive collection of articles > going back several years. It includes thousands of code > examples and expert discussions on all major topics. > > The information is organized by relevant topics, covered > by the corresponding chapters. > > The information was filtered with sophisticated filters and vast > majority of artilces with little relevance have been filtered out. > > If you have any specific requests for some new chapters to be added > and it is of interest to others, please post your requests on this > thread. > > If anyone feels he has above average level of competence, or can > reccommend someone who posts on this group, you may request to be > included in the expert chapters. > > The Python Goldmine is at: > > http://preciseinfo.org/Convert/index_Convert_Python.html > > -- > Programmer's Goldmine collections: > > http://preciseinfo.org > > Tens of thousands of code examples and expert discussions on > C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, > organized by major topics of language, tools, methods, techniques. This site pops up spam windowns. One was blocked, one managed to bypass the popup blocker. Tnis is not friendly behaviour. Some categories have 100s of entries. Better, I think, to use a search engine such as Google with more specific search terms and a snippet of context for each result. tjr From steve at holdenweb.com Sun Jan 10 22:00:22 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 10 Jan 2010 22:00:22 -0500 Subject: Python Goldmine has been updated: http://preciseinfo.org/Convert/index_Convert_Python.html In-Reply-To: References: Message-ID: <4B4A9446.7020009@holdenweb.com> Terry Reedy wrote: > On 1/8/2010 11:50 AM, tanix wrote: >> Python Goldmine collection contains the extensive collection of articles >> going back several years. It includes thousands of code >> examples and expert discussions on all major topics. >> >> The information is organized by relevant topics, covered >> by the corresponding chapters. >> >> The information was filtered with sophisticated filters and vast >> majority of artilces with little relevance have been filtered out. >> >> If you have any specific requests for some new chapters to be added >> and it is of interest to others, please post your requests on this >> thread. >> >> If anyone feels he has above average level of competence, or can >> reccommend someone who posts on this group, you may request to be >> included in the expert chapters. >> >> The Python Goldmine is at: >> >> http://preciseinfo.org/Convert/index_Convert_Python.html >> >> -- >> Programmer's Goldmine collections: >> >> http://preciseinfo.org >> >> Tens of thousands of code examples and expert discussions on >> C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, >> organized by major topics of language, tools, methods, techniques. > > This site pops up spam windowns. One was blocked, one managed to bypass > the popup blocker. Tnis is not friendly behaviour. > > Some categories have 100s of entries. Better, I think, to use a search > engine such as Google with more specific search terms and a snippet of > context for each result. > Because I habitually run the NoScript extension to Firefox the popups didn't appear, but there didn't seem to be any original content on this site. Google continues to be your friend. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Sun Jan 10 22:34:51 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 10 Jan 2010 22:34:51 -0500 Subject: Python Goldmine has been updated: http://preciseinfo.org/Convert/index_Convert_Python.html In-Reply-To: <4B4A9446.7020009@holdenweb.com> References: <4B4A9446.7020009@holdenweb.com> Message-ID: Steve Holden wrote: [...] > Because I habitually run the NoScript extension to Firefox the popups > didn't appear, but there didn't seem to be any original content on this > site. Google continues to be your friend. > And dammit, why didn't I think to strip the links out instead of creating yet one more link to it? Sorry ... regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From nobody at nowhere.com Sun Jan 10 22:36:17 2010 From: nobody at nowhere.com (Nobody) Date: Mon, 11 Jan 2010 03:36:17 +0000 Subject: lightweight encryption of text file References: <7xfx6dhpd0.fsf@ruckus.brouhaha.com> Message-ID: On Sun, 10 Jan 2010 08:54:51 -0800, Paul Rubin wrote: > Nobody writes: >> RC4 (aka ArcFour) is quite trivial to implement, and better than inventing >> your own cipher or using a Vignere: ... > > That's a cute implementation, but it has no authentication and doesn't > include any randomness, which means if you use the same key for two > inputs, there is a security failure (xor'ing the two ciphertexts reveals > the xor of the plaintexts). Right. RC4 is a cipher, not a cryptosystem. But, yeah, the OP needs to be aware of the difference (and probably isn't, yet). So to take that a step further ... The key passed to arcfour.schedule() shouldn't be re-used. If you need to encrypt multiple files, use a different key for each. If you want to encrypt multiple files with the same "password", generate a unique key by hashing a combination of the password and a random salt (e.g. from /dev/random), and prepend the salt to the beginning of the stream. To decrypt, extract the salt from the stream to generate the key. If you need to verify the data, append a hash of the ciphertext (a hash of the plaintext would allow an attacker to confirm a guessed plaintext or to confirm that two files contain the same plaintext). Stream ciphers are vulnerable to replacement attacks: (p1 xor r) xor (p1 xor p2) == (p2 xor r) So if you can guess any part of the plaintext p1, you can replace it with alternative plaintext p2 without needing to decrypt/encrypt or knowing anything about the pad r. Also, if this is for something important, I'd be concerned about how to protect the key. That's hard enough to do in C, let alone in Python. > It also looks rather slow. Any kind of bulk binary data processing in pure Python is slow. The code was written mainly for simplicity, e.g. using generators means that you don't have to deal with buffer sizes. Replacing " % 256" with " & 255" might be worthwhile. > I don't make > any guarantees about p3.py, but it has been reviewed by several experts > and appears to be reasonably sound for the type of casual use being > discussed here, and it is tuned for speed (given the implementation > constraints). For more demanding purposes, you should use a more > serious library like one of the OpenSSL wrappers. The OP specifically wanted to avoid third-party libraries. From nobody at nowhere.com Sun Jan 10 22:44:26 2010 From: nobody at nowhere.com (Nobody) Date: Mon, 11 Jan 2010 03:44:26 +0000 Subject: lightweight encryption of text file References: <7xiqb9yaea.fsf@ruckus.brouhaha.com> Message-ID: On Sun, 10 Jan 2010 12:26:05 -0800, Paul Rubin wrote: > I'd like it a lot if the Python stdlib could include a serious > cryptography module. And I'd like a truckload of gold ;) Right now, even asking for HTTPS support is too much to ask. Heck, even asking for the fake HTTPS support to be identified as such is too much, apparently. From steve at holdenweb.com Sun Jan 10 22:52:24 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 10 Jan 2010 22:52:24 -0500 Subject: lightweight encryption of text file In-Reply-To: References: <7xiqb9yaea.fsf@ruckus.brouhaha.com> Message-ID: Nobody wrote: > On Sun, 10 Jan 2010 12:26:05 -0800, Paul Rubin wrote: > >> I'd like it a lot if the Python stdlib could include a serious >> cryptography module. > > And I'd like a truckload of gold ;) > > Right now, even asking for HTTPS support is too much to ask. Heck, > even asking for the fake HTTPS support to be identified as such is too > much, apparently. > No, Paul, nobody will complain if you *ask* ... A question I've been asking myself quite a lot recently is how the PSF could, were funding to be available, direct the development of Python in specific directions, and what those directions should be. Unfortunately there are probably as many answers as Python programmers in terms of the priorities to be adopted. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From austyn at gmail.com Sun Jan 10 23:04:18 2010 From: austyn at gmail.com (Austyn) Date: Sun, 10 Jan 2010 20:04:18 -0800 (PST) Subject: Fractional Hours from datetime? References: Message-ID: <3ca2bf74-744d-421e-a78e-38dbeac2cda1@m3g2000yqf.googlegroups.com> How about: import time arizona_utc_offset = -7.00 h = (time.time() / 3600 + arizona_utc_offset) % 24 dt.timetuple()[6] is the day of the week; struct tm_time doesn't include a sub-second field. On Jan 10, 10:28?am, "W. eWatson" wrote: > Maybe there's a more elegant way to do this. I want to express the > result of datetime.datetime.now() in fractional hours. > > Here's one way. > > dt=datetime.datetime.now() > xtup = dt.timetuple() > h = xtup[3]+xtup[4]/60.0+xtup[5]/3600.00+xtup[6]/10**6 > # ?now is in fractions of an hour From Dave Sun Jan 10 23:16:03 2010 From: Dave (Dave) Date: Sun, 10 Jan 2010 22:16:03 -0600 Subject: interactive terminal in Ubuntu Linux : libreadline5-dev works only in Python 2.6 not 3.1 References: <23c7de94-44ba-4567-b31f-8a5bb656e9e9@m3g2000yqf.googlegroups.com> <15a78d94-34d5-4bef-8344-6dc8843de4e5@k19g2000yqc.googlegroups.com> Message-ID: On Sat, 9 Jan 2010 16:48:52 -0800 (PST), casevh wrote: >On Jan 9, 3:10?pm, pdlem... at earthlink.net wrote: >> On Sat, 9 Jan 2010 13:27:07 -0800 (PST), casevh >> wrote: > >1) Try the commands again. Make sure all the "./configure" options are >on one line. Make sure to do "sudo make altinstall". (Don't use "sudo >make install"; it will give your version of Python the name "python" >and that can cause confusion on your system.) > >2) Move your applications to another directory. > >3) Try running "python3.1" while you are in that directory. > >If this doesn't work, report back on the error messages you receive. > Thanks casevh for your time & patience. The ./configure . . . was a "continuous" line , although it ran over to next line even on 132 char wide terminal because of names of system & dir. I was careful with the spaces and hyphens. In my reply the "make install" was a typo , I did run make altinstall. Moved all my code to pycode dir on my home directory. Removed all files from /usr/local/lib/python3.1/dlmodules and removed that dir. Twice ran the recompile : make distclean ./configure --prefix=/usr/local --with-computed-gotos --with-wide-unicode <^ one space> make sudo make altinstall After each reinstall had same problems : cannot import random or any code using it, although random.py is in Lib : Traceback File "", line 1, in File "random.py", line 46, in import collections as _collections File "collections.py", line 9 in from _collections import deque, default dict ImportError: /usr/local/lib/python3.1/lib-dynload/collections.so: undefined symbol: PyUnicodeUCS4_FromString After second reinstall today I also found that a module importing "time" would not run. Likewise could not import time at >>> . Same error, ending : undefined symbol: PyUnicode UCS4_FromString And my original problem still there : fouled up keys in interactive terminal. Seems minor now ; ) Should I try to remove everything and reopen the tarball ? Dave WB3DWE, central Texas From no.email at nospam.invalid Sun Jan 10 23:22:50 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 10 Jan 2010 20:22:50 -0800 Subject: lightweight encryption of text file References: <7xiqb9yaea.fsf@ruckus.brouhaha.com> Message-ID: <7xmy0li82t.fsf@ruckus.brouhaha.com> Steve Holden writes: >> Right now, even asking for HTTPS support is too much to ask. Heck, >> even asking for the fake HTTPS support to be identified as such is too >> much, apparently. >> > No, Paul, nobody will complain if you *ask* ... Er, that wasn't me... > A question I've been asking myself quite a lot recently is how the PSF > could, were funding to be available, direct the development of Python in > specific directions,... Crypto in the stdlib is not a matter of funding or (technical) priorities. It's a policy issue; the maintainers were (at least as of a few years ago) concerned about legal restrictions on crypto in some jurisdictions causing problems with distributing Python if it included crypto. I know that other systems (Java, Mozilla, etc) include crypto but they may have had to jump through some hoops for that purpose. From austyn at gmail.com Sun Jan 10 23:23:19 2010 From: austyn at gmail.com (Austyn) Date: Sun, 10 Jan 2010 20:23:19 -0800 (PST) Subject: Fractional Hours from datetime? References: <3ca2bf74-744d-421e-a78e-38dbeac2cda1@m3g2000yqf.googlegroups.com> Message-ID: <05ca0e15-b55e-4ede-81fe-c8f4459e3932@r5g2000yqb.googlegroups.com> Here's an improvement in case you want your code to work outside of Arizona: from time import time, timezone h = ((time() - timezone) / 3600) % 24 On Jan 10, 9:04?pm, Austyn wrote: > How about: > > import time > arizona_utc_offset = -7.00 > h = (time.time() / 3600 + arizona_utc_offset) % 24 > > dt.timetuple()[6] is the day of the week; struct tm_time doesn't > include a sub-second field. > > On Jan 10, 10:28?am, "W. eWatson" wrote: > > > > > Maybe there's a more elegant way to do this. I want to express the > > result of datetime.datetime.now() in fractional hours. > > > Here's one way. > > > dt=datetime.datetime.now() > > xtup = dt.timetuple() > > h = xtup[3]+xtup[4]/60.0+xtup[5]/3600.00+xtup[6]/10**6 > > # ?now is in fractions of an hour From no.email at nospam.invalid Sun Jan 10 23:27:55 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 10 Jan 2010 20:27:55 -0800 Subject: lightweight encryption of text file References: <7xfx6dhpd0.fsf@ruckus.brouhaha.com> Message-ID: <7xiqb9i7uc.fsf@ruckus.brouhaha.com> Nobody writes: > But, yeah, the OP needs to be aware of the difference (and probably isn't, > yet). So to take that a step further ... > The key passed to arcfour.schedule() shouldn't be re-used.... > If you need to verify the data, append a hash of the ciphertext ... > If you want to encrypt multiple files with the same "password", > generate a unique key by hashing a combination of the password and a > random salt (e.g. from /dev/random)... Yep, a whole lot of stuff that's easy to get wrong if it's left to the user (for example, you mean "a MAC of the ciphertext", not "a hash of the ciphertext"), and a lot of extra work even if the user gets it right. It's simplest for the library to provide a single interface that does all of it. >> It also looks rather slow. > > Any kind of bulk binary data processing in pure Python is slow. Well, slow is a relative term, but p3.py is about 5x faster than the fastest pure-Python rc4 implementation that I compared it to. Its heavy lifting is done by the SHA and array modules, that are written in C. From aahz at pythoncraft.com Sun Jan 10 23:53:42 2010 From: aahz at pythoncraft.com (Aahz) Date: 10 Jan 2010 20:53:42 -0800 Subject: py2exe "for loop" hangs in compiled program References: Message-ID: In article , p_tierchen <1JoS at sms.at> wrote: > >the application is an interface to a sqlite database and stores image >metadata (such as year, event, photographer, people on image etc.). i >use pyqt4 for the interface and developed this application on a linux >platform (python 2.5.4). friends of mine liked what i have done an want >it for their windows computers. so i setup a test system (win XP and >python 2.6.?). copying the code to this system and starting it: works >as intended. however they complain python whats this, this is sooo >complicated etc. so i took refuge in using py2exe, planing to give them >a zip compressed archive ... testing this option i found out that on >the surface all looks nice (GUI and functions etc) however: extending >to "larger" datasets (>~50 images) some of the for loops stop after >some time. https://lists.sourceforge.net/lists/listinfo/py2exe-users -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." --Red Adair From aahz at pythoncraft.com Sun Jan 10 23:54:38 2010 From: aahz at pythoncraft.com (Aahz) Date: 10 Jan 2010 20:54:38 -0800 Subject: How to validate the __init__ parameters References: <803c4db7-3813-448a-b428-0ec11c57f098@t42g2000yqd.googlegroups.com> Message-ID: In article , Jean-Michel Pichavant wrote: > >class A: > def __init__(self, foo = None, bar = None): > if len(foo) > 5: > raise ValueError('foo cannot exceed 5 characters') Bad Idea -- what happens when foo is None? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." --Red Adair From steve at holdenweb.com Sun Jan 10 23:57:41 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 10 Jan 2010 23:57:41 -0500 Subject: lightweight encryption of text file In-Reply-To: <7xmy0li82t.fsf@ruckus.brouhaha.com> References: <7xiqb9yaea.fsf@ruckus.brouhaha.com> <7xmy0li82t.fsf@ruckus.brouhaha.com> Message-ID: Paul Rubin wrote: > Steve Holden writes: >>> Right now, even asking for HTTPS support is too much to ask. Heck, >>> even asking for the fake HTTPS support to be identified as such is too >>> much, apparently. >>> >> No, Paul, nobody will complain if you *ask* ... > > Er, that wasn't me... > Oh sorry, no more it was. >> A question I've been asking myself quite a lot recently is how the PSF >> could, were funding to be available, direct the development of Python in >> specific directions,... > > Crypto in the stdlib is not a matter of funding or (technical) > priorities. It's a policy issue; the maintainers were (at least as of a > few years ago) concerned about legal restrictions on crypto in some > jurisdictions causing problems with distributing Python if it included > crypto. I know that other systems (Java, Mozilla, etc) include crypto > but they may have had to jump through some hoops for that purpose. > There are administrative hoops to jump through still in the US to export cryptographic code. Further, if the standard distribution were to include it then the requirements of the US government do still come into play, and it would be a pain to have to have people downloading the software certify (for example) that they weren't intending to re-export it to a "prohibited" country. We can get around some of these issues by retaining the hosting on mainland Europe rather than in the USA, but these issues do demand careful study which apparently nobody really has time for at present. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From pavlovevidence at gmail.com Mon Jan 11 00:21:14 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Sun, 10 Jan 2010 21:21:14 -0800 (PST) Subject: lightweight encryption of text file References: Message-ID: <7a2fbce3-d4cf-437b-9c0c-40f975189353@s31g2000yqs.googlegroups.com> On Jan 8, 11:14?am, Daniel Fetchinson wrote: > I have a plain text file which I would like to protect in a very > simple minded, yet for my purposes sufficient, way. I'd like to > encrypt/convert it into a binary file in such a way that possession of > a password allows anyone to convert it back into the original text > file while not possessing the password one would only see the > following with the standard linux utility 'file': > > [fetchinson at fetch ~]$ file encrypted.data > encrypted.data: data > > and the effort required to convert the file back to the original text > file without the password would be equivalent to guessing the > password. gpg -c simpletextfile.txt -o simpletextfile.gpg But I guess you can't depend on users to have gpg installed so you have to roll out some unvetted Python tool. Carl Banks From debatem1 at gmail.com Mon Jan 11 00:31:23 2010 From: debatem1 at gmail.com (geremy condra) Date: Mon, 11 Jan 2010 00:31:23 -0500 Subject: lightweight encryption of text file In-Reply-To: <7xiqb9yaea.fsf@ruckus.brouhaha.com> References: <7xiqb9yaea.fsf@ruckus.brouhaha.com> Message-ID: On Sun, Jan 10, 2010 at 3:26 PM, Paul Rubin wrote: > geremy condra writes: >> Not sure why in the world you would homebrew something like this- a >> small dependency isn't that bad, and aes can be pretty simple to use. >> Might as well go for the industrial strength approach. > > In my experience, 1) small dependencies ARE that bad, since they mean > you have to develop and test on every platform that you want your code > to run on; And having no dependencies frees you from the burden of testing where your software will be deployed? I don't think so. > 2) using a serious library requires quite a bit of knowledge > and decision-making which not everyone is equipped to do. Homebrewing is not a good solution to the problem of being ignorant of modern cryptography. >?"AES" is not so simple to use unless you know what you're doing in > terms of modes, nonces, etc. Seems pretty simple to me- use AES 192, don't use ECB mode, and use your library of choice's key strengthening utilities. Even blatantly ignoring that advice would still probably give you better results than homebrewing though, so I don't really see the issue here. > Having supported this kind of package in a commercial > setting in the past, IMO, for the sort of (common) application in > question, it's best to keep things as simple as possible and supply a > single interface that provides encryption, authentication, and random > initialization all in one call. ?The cost is a little bit of ciphertext > bloat, but it prevents all kinds of security failures frequently > overlooked by novices. > > I'd like it a lot if the Python stdlib could include a serious > cryptography module. ?That was rejected for regulatory reasons several > years ago, but maybe things are changing enough that the issue can be > revisited sometime. I agree. I inquired about it not too long ago on python-ideas; little serious discussion ensued. Geremy Condra From steve at holdenweb.com Mon Jan 11 00:54:04 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 11 Jan 2010 00:54:04 -0500 Subject: lightweight encryption of text file In-Reply-To: <7a2fbce3-d4cf-437b-9c0c-40f975189353@s31g2000yqs.googlegroups.com> References: <7a2fbce3-d4cf-437b-9c0c-40f975189353@s31g2000yqs.googlegroups.com> Message-ID: Carl Banks wrote: > On Jan 8, 11:14 am, Daniel Fetchinson > wrote: >> I have a plain text file which I would like to protect in a very >> simple minded, yet for my purposes sufficient, way. I'd like to >> encrypt/convert it into a binary file in such a way that possession of >> a password allows anyone to convert it back into the original text >> file while not possessing the password one would only see the >> following with the standard linux utility 'file': >> >> [fetchinson at fetch ~]$ file encrypted.data >> encrypted.data: data >> >> and the effort required to convert the file back to the original text >> file without the password would be equivalent to guessing the >> password. > > > gpg -c simpletextfile.txt -o simpletextfile.gpg > > But I guess you can't depend on users to have gpg installed so you > have to roll out some unvetted Python tool. > > The OP's statement of requirements would be pretty much satisfied by the "crypt" utility. He can even run it under Cygwin on Windows if necessary. Cryptographic sophistication (or even cryptographic security) was not requested (and would not be provided anyway by most of the suggested solutions). If any real protection is required then an ad-hoc program is definitely not the way to provide it. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From casevh at gmail.com Mon Jan 11 01:08:20 2010 From: casevh at gmail.com (casevh) Date: Sun, 10 Jan 2010 22:08:20 -0800 (PST) Subject: interactive terminal in Ubuntu Linux : libreadline5-dev works only in Python 2.6 not 3.1 References: <23c7de94-44ba-4567-b31f-8a5bb656e9e9@m3g2000yqf.googlegroups.com> <15a78d94-34d5-4bef-8344-6dc8843de4e5@k19g2000yqc.googlegroups.com> Message-ID: <43ee50ae-7a0e-4c93-b844-8bb6fb1ea07a@u41g2000yqe.googlegroups.com> On Jan 10, 8:16?pm, Dave WB3DWE wrote: > On Sat, 9 Jan 2010 16:48:52 -0800 (PST), casevh > wrote: > > >On Jan 9, 3:10?pm, pdlem... at earthlink.net wrote: > >> On Sat, 9 Jan 2010 13:27:07 -0800 (PST), casevh > >> wrote: > > >1) Try the commands again. Make sure all the "./configure" options are > >on one line. Make sure to do "sudo make altinstall". (Don't use "sudo > >make install"; it will give your version of Python the name "python" > >and that can cause confusion on your system.) > > >2) Move your applications to another directory. > > >3) Try running "python3.1" while you are in that directory. > > >If this doesn't work, report back on the error messages you receive. > > Thanks casevh for your time & patience. > The ./configure . . . ?was a "continuous" line , although it ran over to > next line even on 132 char wide terminal because of names of system & > dir. ?I was careful with the spaces and hyphens. > In my reply the "make install" was a typo , I did run ?make altinstall. > > Moved all my code to pycode dir on my home directory. ?Removed all > files from /usr/local/lib/python3.1/dlmodules and removed that dir. > > Twice ran the recompile : > ? ? make distclean > ? ? ./configure --prefix=/usr/local --with-computed-gotos > ? ? ? ? ? ? --with-wide-unicode > ? ? ? ? <^ one space> > ? ? make > ? ? sudo make altinstall > After each reinstall had same problems : ?cannot import ?random ?or > any code using it, although random.py is in Lib : > ? ? Traceback > ? ? ? ? File "", line 1, in > ? ? ? ? File "random.py", line 46, in > ? ? ? ? ? ? import collections as _collections > ? ? ? ? File "collections.py", line 9 in > ? ? ? ? ? ? from _collections import deque, default dict > ? ? ImportError: /usr/local/lib/python3.1/lib-dynload/collections.so: > ? ? ? ? undefined symbol: PyUnicodeUCS4_FromString > > After second reinstall today I also found that a module importing > ? ? "time" ?would not run. ?Likewise could not import ?time ?at ?>>> . > Same error, ending : undefined symbol: PyUnicode UCS4_FromString > > And my original problem still there : fouled up keys in interactive > terminal. Seems minor now ; ) ? Should I try to remove everything > and reopen the tarball ? > Dave WB3DWE, ?central Texas Are you sure you are using the new version of python3.1 (the one located in /usr/local/bin/)? What is the result of "which python3.1"? What happens if you run "/usr/local/bin/python3.1"? casevh From sebastian.langer at gmx.de Mon Jan 11 01:21:54 2010 From: sebastian.langer at gmx.de (Sebastian) Date: Sun, 10 Jan 2010 22:21:54 -0800 (PST) Subject: how to duplicate array entries Message-ID: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> Hi there, I have an array x=[1,2,3] Is there an operator which I can use to get the result [1,1,1,2,2,2,3,3,3] ? I tried x*3, which resulted in [1,2,3,1,2,3,1,2,3] I also tried [[b,b,b] for b in x] which led to [[1,2,3],[1,2,3], [1,2,3]], but this isn't what I want either. Cheers, Sebastian From sebastian.langer at gmx.de Mon Jan 11 01:24:07 2010 From: sebastian.langer at gmx.de (Sebastian) Date: Sun, 10 Jan 2010 22:24:07 -0800 (PST) Subject: how to duplicate array entries References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> Message-ID: <2f4fa997-b044-426c-8600-007e51c25c32@h9g2000yqa.googlegroups.com> On Jan 11, 4:21 pm, Sebastian wrote: > I also tried [[b,b,b] for b in x] which led to [[1,2,3],[1,2,3], > [1,2,3]] Sorry, I have to correct myself. The quoted line above resulted in [[1,1,1],[2,2,2],[3,3,3]] of course! Cheers, Sebastian From clp2 at rebertia.com Mon Jan 11 01:30:04 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 10 Jan 2010 22:30:04 -0800 Subject: how to duplicate array entries In-Reply-To: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> Message-ID: <50697b2c1001102230l4d547e5cxded09801daa0261a@mail.gmail.com> On Sun, Jan 10, 2010 at 10:21 PM, Sebastian wrote: > Hi there, > > I have an array ?x=[1,2,3] > > Is there an operator which I can use to get the result > [1,1,1,2,2,2,3,3,3] ? > > I tried x*3, which resulted in [1,2,3,1,2,3,1,2,3] > I also tried [[b,b,b] for b in x] which led to [[1,2,3],[1,2,3], > [1,2,3]], but this isn't what I want either. from itertools import chain, repeat n = 3 stretched = list(chain(*[repeat(item, n) for item in x])) Cheers, Chris -- http://blog.rebertia.com From paul.nospam at rudin.co.uk Mon Jan 11 02:07:07 2010 From: paul.nospam at rudin.co.uk (Paul Rudin) Date: Mon, 11 Jan 2010 07:07:07 +0000 Subject: how to duplicate array entries References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> Message-ID: <87ocl1ce78.fsf@rudin.co.uk> Sebastian writes: > Hi there, > > I have an array x=[1,2,3] In python such an object is called a "list". (In cpython it's implemented as an automatically resizable array.) > > Is there an operator which I can use to get the result > [1,1,1,2,2,2,3,3,3] ? There's no operator that will give you that directly - but there are plenty of one-liners that will yield that list. e.g: >>> list(itertools.chain(*([x]*3 for x in [1,2,3]))) [1, 1, 1, 2, 2, 2, 3, 3, 3] From danb_83 at yahoo.com Mon Jan 11 02:13:55 2010 From: danb_83 at yahoo.com (Dan Bishop) Date: Sun, 10 Jan 2010 23:13:55 -0800 (PST) Subject: integer and string compare, is that correct? References: Message-ID: <2f718029-1682-47f6-a2c3-6e40b51bd71a@v25g2000yqk.googlegroups.com> On Jan 10, 10:34?am, Nobody wrote: > Hellmut Weber wrote: > >> being a causal python user (who likes the language quite a lot) > >> it took me a while to realize the following: > >> ?>>> max = '5' > >> ?>>> n = 5 > >> ?>>> n >= max > >> False > > >> Section 5.9 Comparison describes this. > > >> Can someone give me examples of use cases > Peter Otten wrote: > > The use cases for an order that works across types like int and str are weak > > to non-existent. Implementing it was considered a mistake and has been fixed > > in Python 3: > >>>> 5 > "5" > > Traceback (most recent call last): > > ? File "", line 1, in > > TypeError: unorderable types: int() > str() > > If you actually need to perform comparisons across types, you can rely > upon the fact that tuple comparisons are non-strict and use e.g.: > > ? ? ? ? > a = 5 > ? ? ? ? > b = '5' > ? ? ? ? > (type(a).__name__, a) < (type(b).__name__, b) > ? ? ? ? True > ? ? ? ? > (type(a).__name__, a) > (type(b).__name__, b) > ? ? ? ? False > > The second elements will only be compared if the first elements are equal > (i.e. the values have the same type). But this method gives you 3.0 < 2 because 'float' < 'int'. Probably not what you want. From gherron at islandtraining.com Mon Jan 11 02:20:52 2010 From: gherron at islandtraining.com (Gary Herron) Date: Sun, 10 Jan 2010 23:20:52 -0800 Subject: how to duplicate array entries In-Reply-To: <87ocl1ce78.fsf@rudin.co.uk> References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <87ocl1ce78.fsf@rudin.co.uk> Message-ID: <4B4AD154.1020501@islandtraining.com> Paul Rudin wrote: > Sebastian writes: > > >> Hi there, >> >> I have an array x=[1,2,3] >> > > In python such an object is called a "list". > > (In cpython it's implemented as an automatically resizable array.) > > >> Is there an operator which I can use to get the result >> [1,1,1,2,2,2,3,3,3] ? >> > > There's no operator that will give you that directly - but there are > plenty of one-liners that will yield that list. > e.g: > > >>>> list(itertools.chain(*([x]*3 for x in [1,2,3]))) >>>> > [1, 1, 1, 2, 2, 2, 3, 3, 3] > List comprehension also works nicely for this problem, and may be clearer to some. >>> x = [1,2,3] >>> print [i for i in x for k in range(3)] [1, 1, 1, 2, 2, 2, 3, 3, 3] Gary Herron From steven at REMOVE.THIS.cybersource.com.au Mon Jan 11 02:34:04 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 11 Jan 2010 07:34:04 GMT Subject: how to duplicate array entries References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> Message-ID: On Sun, 10 Jan 2010 22:21:54 -0800, Sebastian wrote: > Hi there, > > I have an array x=[1,2,3] You have a list. Python has an array type, but you have to "import array" to use it. > Is there an operator which I can use to get the result > [1,1,1,2,2,2,3,3,3] ? Not an operator, but you can do it easily with a function. Here's the simple version: >>> def duplicate(items, count): ... L = [] ... for item in items: ... L.extend([item]*count) ... return L ... >>> duplicate([1,2,3], 3) [1, 1, 1, 2, 2, 2, 3, 3, 3] Here's a version which is short and simple enough to use in-line, but will be slow for large lists: >>> x = [1,2,3] >>> count = 3 >>> sum([[item]*count for item in x], []) [1, 1, 1, 2, 2, 2, 3, 3, 3] Finally, here's a nasty hack that you should never, ever, ever use for any reason except to win a bet: >>> [locals()['_[1]'].extend([item]*(count-1)) or item for item in x] [1, 1, 1, 2, 2, 2, 3, 3, 3] -- Steven From marco.salden at gmail.com Mon Jan 11 02:47:34 2010 From: marco.salden at gmail.com (Marco Salden) Date: Sun, 10 Jan 2010 23:47:34 -0800 (PST) Subject: "Advanced" Python programming book? References: <4b49d7a8$1@dnews.tpgi.com.au> Message-ID: <5db2ed03-2669-4b74-ab29-47692001ff34@b2g2000yqi.googlegroups.com> On Jan 10, 2:35?pm, flow wrote: > I've just finished reading a sort of beginner Python book, and I know > quite a bit now but I'm looking for a book that can teach me advanced > aspects of Python - code optimisation, threading, etc. > > Any recommendations? > > Cheers. I like to add this one, which is nice to read just after reading the "starters", covers all kinds of topics, and written by someone who knows Python (I think at least). http://oreilly.com/catalog/9780596009250/ HTH, Marco From ben+python at benfinney.id.au Mon Jan 11 02:48:03 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 11 Jan 2010 18:48:03 +1100 Subject: class viewer ? References: Message-ID: <87aawlt74c.fsf@benfinney.id.au> Stef Mientki writes: > from the given class, it's ancestors and it's derived classes, > I'ld like to get the following information in a tree like structure: > > - the file were the class is definied > - the attributes, split in inherited / created / overriden > - the methodes, split in inherited / created / overriden > - the files were instances of the class are created > - and probably I forget a few > > any suggestions ? The ?Emacs Code Browser? was seemingly designed for languages like C++ and Java, but it serves well enough for Python also. I find it annoying to use, since it seems to assume mouse navigation and has rather impenetrable (for me) documentation. But it does seem to do much of what you're asking for, and a big plus is that it's not specific to Python. -- \ ?The most common way people give up their power is by thinking | `\ they don't have any.? ?Alice Walker | _o__) | Ben Finney From marco.salden at gmail.com Mon Jan 11 02:50:21 2010 From: marco.salden at gmail.com (Marco Salden) Date: Sun, 10 Jan 2010 23:50:21 -0800 (PST) Subject: integer and string compare, is that correct? References: Message-ID: <7b3e40df-4b6a-41dc-95ff-f3b7ca14b6ac@b2g2000yqi.googlegroups.com> On Jan 10, 1:26?pm, Hellmut Weber wrote: > Hi, > being a causal python user (who likes the language quite a lot) > it took me a while to realize the following: > > leo at sylvester py_count $ python > Python 2.6.3 (r263:75183, Oct 26 2009, 12:34:23) > [GCC 4.4.1] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > ?>>> max = '5' > ?>>> n = 5 > ?>>> n >= max > False > ?>>> n + max > Traceback (most recent call last): > ? ?File "", line 1, in > TypeError: unsupported operand type(s) for +: 'int' and 'str' > ?>>> > > Section 5.9 Comparison describes this. > > Can someone give me examples of use cases > > TIA > > Hellmut > > -- > Dr. Hellmut Weber ? ? ? ? m... at hellmutweber.de > Degenfeldstra?e 2 ? ? ? ? tel ? +49-89-3081172 > D-80803 M?nchen-Schwabing mobil +49-172-8450321 > please: No DOCs, no PPTs. why: tinyurl.com/cbgq I would say you want to compare semantically an integer value with an integer value so why not: IDLE 1.1.3 >>> max = '5' >>> n = 5 >>> n==(int(max)) True >>> ? (in Python 2.4...) Regards, Marco From alfps at start.no Mon Jan 11 02:56:36 2010 From: alfps at start.no (Alf P. Steinbach) Date: Mon, 11 Jan 2010 08:56:36 +0100 Subject: how to duplicate array entries In-Reply-To: <87ocl1ce78.fsf@rudin.co.uk> References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <87ocl1ce78.fsf@rudin.co.uk> Message-ID: * Paul Rudin: > Sebastian writes: > >> I have an array x=[1,2,3] > > In python such an object is called a "list". > > (In cpython it's implemented as an automatically resizable array.) I don't think the OP's terminology needs correction. A Python "list" is an array functionality-wise. If one isn't observant of that fact then one ends up with O(n^2) time for the simplest things. Using the term "array" accentuates and clarifies this most important aspect. Using the misleading term "list", even if that's the type's name in Python, hides this most important aspect, and so is not, IMHO, a Good Idea except where it really matters that it's a 'list' array as opposed to, say, a 'tuple' array. >> Is there an operator which I can use to get the result >> [1,1,1,2,2,2,3,3,3] ? > > There's no operator that will give you that directly - but there are > plenty of one-liners that will yield that list. > e.g: > >>>> list(itertools.chain(*([x]*3 for x in [1,2,3]))) > [1, 1, 1, 2, 2, 2, 3, 3, 3] And I think it's worth noting that, for the general case, this notation is also hiding a gross inefficiency, first constructing sub-arrays and then copying them and joining them. It doesn't even buy clarity. So, just >>> def repeat_items_in( s, n ): ... a = [] ... for item in s: ... for i in range( n ): ... a.append( item ) ... return a ... >>> repeat_items_in( [1, 2, 3], 3 ) [1, 1, 1, 2, 2, 2, 3, 3, 3] >>> _ And if one absolutely feels like trading some efficiency and clarity for some more functional-programming expression like thing (I don't understand why people desire that!), just replace the 'append' line with a 'yield' and then write list( repeat_items_in( [1, 2, 3], 3 ) ) Re the thing I don't understand: it's the same in C++, people using hours on figuring out how to do something very simple in an ungrokkable indirect and "compiled" way using template metaprogramming stuff, when they could just write a simple 'for' loop and be done with in, say, 3 seconds, and much clearer too! Cheers, - Alf From steven at REMOVE.THIS.cybersource.com.au Mon Jan 11 03:26:05 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 11 Jan 2010 08:26:05 GMT Subject: how to duplicate array entries References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <87ocl1ce78.fsf@rudin.co.uk> Message-ID: On Mon, 11 Jan 2010 08:56:36 +0100, Alf P. Steinbach wrote: > * Paul Rudin: >> Sebastian writes: >> >>> I have an array x=[1,2,3] >> >> In python such an object is called a "list". >> >> (In cpython it's implemented as an automatically resizable array.) > > I don't think the OP's terminology needs correction. > > A Python "list" is an array functionality-wise. > > If one isn't observant of that fact then one ends up with O(n^2) time > for the simplest things. Well that's certainly not true. Some operations may be O(N**2), but others are not: list.append() is amortized O(N) and for individual appends, may be can be as fast as O(1). > Using the term "array" accentuates and clarifies this most important > aspect. But Python lists are designed to behave as lists. Just because CPython implements them using arrays doesn't make them arrays. Other Python implementations might use other implementations... If the creator of CLPython is out there, perhaps might like to comment on whether he uses Lisp linked-lists for the Python list type? > Using the misleading term "list", even if that's the type's name in > Python, hides this most important aspect, and so is not, IMHO, a Good > Idea except where it really matters that it's a 'list' array as opposed > to, say, a 'tuple' array. Or an "array" array. >>> from array import array >>> array >>> Is there an operator which I can use to get the result >>> [1,1,1,2,2,2,3,3,3] ? >> >> There's no operator that will give you that directly - but there are >> plenty of one-liners that will yield that list. e.g: >> >>>>> list(itertools.chain(*([x]*3 for x in [1,2,3]))) >> [1, 1, 1, 2, 2, 2, 3, 3, 3] > > And I think it's worth noting that, for the general case, this notation > is also hiding a gross inefficiency, first constructing sub-arrays and > then copying them and joining them. I wouldn't call that a gross inefficiency -- that's a gross exaggeration unless count is very large, and even then, possibly not that large. Only one such sub-array (sub-list) exists at any time. (Unless I am grossly misinformed.) > It doesn't even buy clarity. Not if you're unused to the functional, iterator-based idioms, no. But if you are, it does. > And if one absolutely feels like trading some efficiency and clarity for > some more functional-programming expression like thing (I don't > understand why people desire that!), I don't understand why you assume that functional forms are necessarily less efficient and clear than non-functional. Which is easier to read? >>> print sum([1,2,3]) 6 versus >>> total = 0 >>> for i in [1, 2, 3]: ... total += i ... >>> print total 6 [...] > Re the thing I don't understand: it's the same in C++, people using > hours on figuring out how to do something very simple in an ungrokkable > indirect and "compiled" way using template metaprogramming stuff, when > they could just write a simple 'for' loop and be done with in, say, 3 > seconds, and much clearer too! Amen to that brother! It's the obsession with one-liners and the desire for a single built-in command to do every imaginable task. -- Steven From sebastian.langer at gmx.de Mon Jan 11 03:39:10 2010 From: sebastian.langer at gmx.de (Sebastian) Date: Mon, 11 Jan 2010 00:39:10 -0800 (PST) Subject: how to duplicate array entries References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <87ocl1ce78.fsf@rudin.co.uk> Message-ID: Thank you for your answers! I actually implemented it using for loops before I posted here, but I was curious if there is a more elegant solution (judging from the post, Alf will probably say, that for loops are already elegant). Sebastian From frank at chagford.com Mon Jan 11 03:51:13 2010 From: frank at chagford.com (Frank Millman) Date: Mon, 11 Jan 2010 10:51:13 +0200 Subject: TypeError: __name__ must be set to a string object Message-ID: Hi all This problem is similar to one I posted recently regarding the multiprocessing module and unicode. However, although this one manifests itself while using the multiprocessing module, is caused by Python itself (2.6.2). At the top of my program I have 'from __future__ import unicode_literals'. The relevant lines from my program read - from multiprocessing.managers import BaseManager class MyManager(BaseManager): pass MyManager.register('my_function', my_function) Inside the multiprocessing module, the following lines are executed - @classmethod def register(cls, typeid, ...) [...] def temp(...): [...] temp.__name__ = typeid At this point, Python raises the exception 'TypeError: __name__ must be set to a string object'. I can fix it by changing my last line to - MyManager.register(str('my_function'), my_function) Is there any reason why __name__ cannot be a unicode object in Python 2.x? If so, there is probably little chance of this being changed, so it is probably not worth reporting. Any thoughts? Frank Millman From cloudgiant at gmail.com Mon Jan 11 03:56:46 2010 From: cloudgiant at gmail.com (Munir) Date: Mon, 11 Jan 2010 00:56:46 -0800 (PST) Subject: how to duplicate array entries References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> Message-ID: <14437829-e3ef-41ac-b533-bc7003c1434b@j19g2000yqk.googlegroups.com> > I have an array ?x=[1,2,3] > > Is there an operator which I can use to get the result > [1,1,1,2,2,2,3,3,3] ? > > I tried x*3, which resulted in [1,2,3,1,2,3,1,2,3] Have you tried: y = x*3 y.sort() Munir From __peter__ at web.de Mon Jan 11 03:57:30 2010 From: __peter__ at web.de (Peter Otten) Date: Mon, 11 Jan 2010 09:57:30 +0100 Subject: how to duplicate array entries References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <87ocl1ce78.fsf@rudin.co.uk> Message-ID: Alf P. Steinbach wrote: > Re the thing I don't understand: it's the same in C++, people using hours > on figuring out how to do something very simple in an ungrokkable indirect > and "compiled" way using template metaprogramming stuff, when they could > just write a simple 'for' loop and be done with in, say, 3 seconds, and > much clearer too! Most of that stuff doesn't end in code meant to do anything important. It's more like gymnastics that helps you keep your mind in shape. Or so I would hope. >>> items = [1, 2, 3] >>> result = 3*len(items)*[None] >>> result[::3] = result[1::3] = result[2::3] = items >>> result [1, 1, 1, 2, 2, 2, 3, 3, 3] >>> from itertools import * >>> list(chain.from_iterable(starmap(repeat, izip(items, repeat(3))))) [1, 1, 1, 2, 2, 2, 3, 3, 3] ;) Peter From alfps at start.no Mon Jan 11 04:03:04 2010 From: alfps at start.no (Alf P. Steinbach) Date: Mon, 11 Jan 2010 10:03:04 +0100 Subject: how to duplicate array entries In-Reply-To: References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <87ocl1ce78.fsf@rudin.co.uk> Message-ID: * Steven D'Aprano: > On Mon, 11 Jan 2010 08:56:36 +0100, Alf P. Steinbach wrote: > >> * Paul Rudin: >>> Sebastian writes: >>> >>>> I have an array x=[1,2,3] >>> In python such an object is called a "list". >>> >>> (In cpython it's implemented as an automatically resizable array.) >> I don't think the OP's terminology needs correction. >> >> A Python "list" is an array functionality-wise. >> >> If one isn't observant of that fact then one ends up with O(n^2) time >> for the simplest things. > > Well that's certainly not true. Some operations may be O(N**2), but > others are not: list.append() is amortized O(N) and for individual > appends, may be can be as fast as O(1). The second sentence may or may not be true. I don't know of any fundamental 'list' operations that have quadratic time. Is there? The first sentence is just baffling -- what on Earth is the "that" that you think is not true? OK, I can guess (correct me if I'm guessing wrong, please): you think I'm talking about elementary operations. I'm not. I'm talking about algorithmic complexity for loops doing e.g. insertions. >> Using the term "array" accentuates and clarifies this most important >> aspect. > > But Python lists are designed to behave as lists. No, I'm sorry, they're not. A Python 'list' has de facto constant time indexing, or "random access". A linked list -- what the informal "list" means in programming -- does not have constant time indexing. A linked list has constant time insertion. A Python 'list' has de facto linear time insertion (except when used as cursor gap array, of course, or e.g. implementing a linked list on top, such things). So in short, a Python 'list' has all the properties of arrays, and none of the properties of linked lists. > Just because CPython > implements them using arrays doesn't make them arrays. Other Python > implementations might use other implementations... No, I'm sorry, not without screwing up existing Python programs. Indexing is assumed as constant time in every CPython program. That is, in your own words, but here correct, that's "certainly not true". ;-) No (sensible) programming language allows a language implementation to change the running time of common loops from linear to quadratic. It would be decidedly un-pythonic. ;-) > If the creator of CLPython is out there, perhaps might like to comment on > whether he uses Lisp linked-lists for the Python list type? If he does then you're talking about a different language than the one that CPython implements: constant time indexing is very different from linear time. It doesn't matter if some bananas are called oranges. They don't turn into oranges no matter what they're called. >> Using the misleading term "list", even if that's the type's name in >> Python, hides this most important aspect, and so is not, IMHO, a Good >> Idea except where it really matters that it's a 'list' array as opposed >> to, say, a 'tuple' array. > > Or an "array" array. For example, yes. These different kinds of arrays have different restrictions: can't be used as dictionary key, can't be modified, has fixed item type. And when talking about such characteristics the type name can be relevant. >>>> from array import array >>>> array > > > > >>>> Is there an operator which I can use to get the result >>>> [1,1,1,2,2,2,3,3,3] ? >>> There's no operator that will give you that directly - but there are >>> plenty of one-liners that will yield that list. e.g: >>> >>>>>> list(itertools.chain(*([x]*3 for x in [1,2,3]))) >>> [1, 1, 1, 2, 2, 2, 3, 3, 3] >> And I think it's worth noting that, for the general case, this notation >> is also hiding a gross inefficiency, first constructing sub-arrays and >> then copying them and joining them. > > I wouldn't call that a gross inefficiency -- that's a gross exaggeration > unless count is very large, and even then, possibly not that large. Only > one such sub-array (sub-list) exists at any time. (Unless I am grossly > misinformed.) I'm sorry but to the best of my knowledge you're misinformed. Unless there's some pretty advanced lazy evaluation involved the * operator has to collect the subarrays into an array formal argument for the 'chain' routine. And at that point they all exist at the same time. >>> def knurre( *poff ): ... print( type( poff ) ) ... print( poff ) ... >>> a = [1, 2, 3] >>> knurre( *(3*[x] for x in a) ) ([1, 1, 1], [2, 2, 2], [3, 3, 3]) >>> _ >> It doesn't even buy clarity. > > Not if you're unused to the functional, iterator-based idioms, no. > > But if you are, it does. He he -- see above, with 99.x certainty you *misunderstood* the code. That's *not* clear code. That's, hereby (almost) proven :-), code that makes even experienced programmers misunderstand what's going on! >> And if one absolutely feels like trading some efficiency and clarity for >> some more functional-programming expression like thing (I don't >> understand why people desire that!), > > I don't understand why you assume that functional forms are necessarily > less efficient and clear than non-functional. Which is easier to read? > >>>> print sum([1,2,3]) > 6 > > versus > >>>> total = 0 >>>> for i in [1, 2, 3]: > ... total += i > ... >>>> print total > 6 No, here I very much agree with you: the first is bestest. :-) I like abstraction. But not for its own sake: there has to be some increase in clarity, some details that the abstraction removes from consideration, instead of introducing additional details for consideration! > [...] >> Re the thing I don't understand: it's the same in C++, people using >> hours on figuring out how to do something very simple in an ungrokkable >> indirect and "compiled" way using template metaprogramming stuff, when >> they could just write a simple 'for' loop and be done with in, say, 3 >> seconds, and much clearer too! > > Amen to that brother! > > It's the obsession with one-liners and the desire for a single built-in > command to do every imaginable task. He he... :-) Cheers, - Alf From drncode1 at live.com Mon Jan 11 04:06:38 2010 From: drncode1 at live.com (flow) Date: 11 Jan 2010 20:06:38 +1100 Subject: "Advanced" Python programming book? References: <4b49d7a8$1@dnews.tpgi.com.au> Message-ID: <4b4aea1e$1@dnews.tpgi.com.au> Cheers guys - I'll check the books out :) Thanks very much. From martin at v.loewis.de Mon Jan 11 04:09:36 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Mon, 11 Jan 2010 10:09:36 +0100 Subject: sys.stdout vs. sys.stderr In-Reply-To: References: Message-ID: > In Python 3.1 is there any difference in the buffering behavior of the > initial sys.stdout and sys.stderr streams? No. > Were they different at some earlier point in Python's evolution? That depends on the operating system. These used to be whatever the C library set up as stdout and stderr. Typically, they were buffered in the same way. Regards, Martin From clp2 at rebertia.com Mon Jan 11 04:21:40 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Mon, 11 Jan 2010 01:21:40 -0800 Subject: how to duplicate array entries In-Reply-To: References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <87ocl1ce78.fsf@rudin.co.uk> Message-ID: <50697b2c1001110121ua673e5ex446d5deaecad8043@mail.gmail.com> On Mon, Jan 11, 2010 at 1:03 AM, Alf P. Steinbach wrote: > * Steven D'Aprano: >> >> On Mon, 11 Jan 2010 08:56:36 +0100, Alf P. Steinbach wrote: >> >>> * Paul Rudin: >>>> >>>> Sebastian writes: >>>> >>>>> I have an array ?x=[1,2,3] >>>> >>>> In python such an object is called a "list". >>>> >>>> (In cpython it's implemented as an automatically resizable array.) >>> >>> I don't think the OP's terminology needs correction. >>> >>> A Python "list" is an array functionality-wise. >>> >>> If one isn't observant of that fact then one ends up with O(n^2) time >>> for the simplest things. >> >> Well that's certainly not true. Some operations may be O(N**2), but others >> are not: list.append() is amortized O(N) and for individual appends, may be >> can be as fast as O(1). > > The second sentence may or may not be true. I don't know of any fundamental > 'list' operations that have quadratic time. Is there? > > The first sentence is just baffling ?-- ?what on Earth is the "that" that > you think is not true? > > OK, I can guess (correct me if I'm guessing wrong, please): you think I'm > talking about elementary operations. I'm not. I'm talking about algorithmic > complexity for loops doing e.g. insertions. > > >>> Using the term "array" accentuates and clarifies this most important >>> aspect. >> >> But Python lists are designed to behave as lists. > > No, I'm sorry, they're not. > > A Python 'list' has de facto constant time indexing, or "random access". > > A linked list ?-- ?what the informal "list" means in programming Eh, it's a bit context-dependent. The abstract data type definition is a superset that includes both linked lists and dynamic arrays. FWIW, Java likewise uses "list" in its ADT sense. Cheers, Chris -- http://blog.rebertia.com From no.email at nospam.invalid Mon Jan 11 04:56:36 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 11 Jan 2010 01:56:36 -0800 Subject: lightweight encryption of text file References: <7xiqb9yaea.fsf@ruckus.brouhaha.com> Message-ID: <7xy6k53qy3.fsf@ruckus.brouhaha.com> geremy condra writes: > And having no dependencies frees you from the burden of testing > where your software will be deployed? I don't think so. If you just use the stdlib and are a bit careful about OS dependent features, your code can run pretty much everywhere. More to the point, if (say) you develop a pure Python app on Linux and a Windows user reports a problem, you can probably straighten it out without having to actually use a Windows development system. If you need C extensions you need Windows compilers. >> 2) using a serious library requires quite a bit of knowledge >> and decision-making which not everyone is equipped to do. > > Homebrewing is not a good solution to the problem of being > ignorant of modern cryptography. Not sure what you're getting at. If you're referring to p3.py as a homebrew algorithm designed by someone ignorant, I don't think that is accurate. I've been a fulltime crypto developer, and p3.py was reviewed by several experts on sci.crypt who all felt that its design is sound. I don't claim it's suitable for high-value data (stick with something standards-conformant for that) but it was designed as a replacement for the now-defunct rotor module, and is just about certainly better in every regard. Its only cryptographic assumption is that SHA1(key+X) is a pseudorandom function for fixed length X. That is not a certified characteristic of SHA1, but the HMAC standard (RFC 2104) is based on the same assumption (see Krawczyk's paper cited in the RFC). I'd go as far as to say that it is just about certainly better than RC4, which has well-known distinguishers of quite low complexity. >>?"AES" is not so simple to use unless you know what you're doing in >> terms of modes, nonces, etc. > > Seems pretty simple to me- use AES 192, don't use ECB mode, and > use your library of choice's key strengthening utilities. That does not address any questions of authentication, ciphertext malleability, IV generation, etc. p3.py handles all of that with no fuss imposed on the user. Really, p3.py was written because the same basic desire (simple, pure-Python encryption) kept coming up over and over in my own code and others', and it really seems to address the constraints about as well as anything I've been able to think of. From alfps at start.no Mon Jan 11 05:20:49 2010 From: alfps at start.no (Alf P. Steinbach) Date: Mon, 11 Jan 2010 11:20:49 +0100 Subject: how to duplicate array entries In-Reply-To: References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <87ocl1ce78.fsf@rudin.co.uk> Message-ID: * Chris Rebert: > On Mon, Jan 11, 2010 at 1:03 AM, Alf P. Steinbach wrote: >> * Steven D'Aprano: >>> On Mon, 11 Jan 2010 08:56:36 +0100, Alf P. Steinbach wrote: >>> >>>> * Paul Rudin: >>>>> Sebastian writes: >>>>> >>>>>> I have an array x=[1,2,3] >>>>> In python such an object is called a "list". >>>>> >>>>> (In cpython it's implemented as an automatically resizable array.) >>>> I don't think the OP's terminology needs correction. >>>> >>>> A Python "list" is an array functionality-wise. >>>> >>>> If one isn't observant of that fact then one ends up with O(n^2) time >>>> for the simplest things. >>> Well that's certainly not true. Some operations may be O(N**2), but others >>> are not: list.append() is amortized O(N) and for individual appends, may be >>> can be as fast as O(1). >> The second sentence may or may not be true. I don't know of any fundamental >> 'list' operations that have quadratic time. Is there? >> >> The first sentence is just baffling -- what on Earth is the "that" that >> you think is not true? >> >> OK, I can guess (correct me if I'm guessing wrong, please): you think I'm >> talking about elementary operations. I'm not. I'm talking about algorithmic >> complexity for loops doing e.g. insertions. >> >> >>>> Using the term "array" accentuates and clarifies this most important >>>> aspect. >>> But Python lists are designed to behave as lists. >> No, I'm sorry, they're not. >> >> A Python 'list' has de facto constant time indexing, or "random access". >> >> A linked list -- what the informal "list" means in programming > > Eh, it's a bit context-dependent. The abstract data type definition is > a superset that includes both linked lists and dynamic arrays. Assuming you're talking about some abstract type definition that's in some PEP somewhere (there's no abstract data type in the language specification, it's all informal) then that's a deficiency of the specification, since the type is designed around indexing operations. Perhaps the designers thought it would be "obvious", that no-one could mistake it for other than what it is? Anyway, that doesn't make it context-dependent: if true, it only makes it poorly specified. > FWIW, Java likewise uses "list" in its ADT sense. I'm sorry, I'm unfamiliar with that Java terminology (but then, reportedly, many Java programmers think that Java has "pass by reference", so nothing coming from that direction will surprise me very much!). The Java language specification has a section about arrays, none about lists AFAICS. Do you have a reference? Cheers & hth., - Alf From clp2 at rebertia.com Mon Jan 11 05:31:41 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Mon, 11 Jan 2010 02:31:41 -0800 Subject: how to duplicate array entries In-Reply-To: References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <87ocl1ce78.fsf@rudin.co.uk> Message-ID: <50697b2c1001110231l2ee64e3cub10ad46dbc5d6e2c@mail.gmail.com> On Mon, Jan 11, 2010 at 2:20 AM, Alf P. Steinbach wrote: > * Chris Rebert: >> On Mon, Jan 11, 2010 at 1:03 AM, Alf P. Steinbach wrote: >>> * Steven D'Aprano: >>>> On Mon, 11 Jan 2010 08:56:36 +0100, Alf P. Steinbach wrote: >>>>> * Paul Rudin: >>>>>> Sebastian writes: >>>>> Using the term "array" accentuates and clarifies this most important >>>>> aspect. >>>> >>>> But Python lists are designed to behave as lists. >>> >>> No, I'm sorry, they're not. >>> >>> A Python 'list' has de facto constant time indexing, or "random access". >>> >>> A linked list ?-- ?what the informal "list" means in programming >> >> Eh, it's a bit context-dependent. The abstract data type definition is >> a superset that includes both linked lists and dynamic arrays. > > Assuming you're talking about some abstract type definition that's in some > PEP somewhere No, I mean the computer science definition/term: http://en.wikipedia.org/wiki/List_(computer_science) >> FWIW, Java likewise uses "list" in its ADT sense. > > I'm sorry, I'm unfamiliar with that Java terminology (but then, reportedly, > many Java programmers think that Java has "pass by reference", so nothing > coming from that direction will surprise me very much!). The Java language > specification has a section about arrays, none about lists AFAICS. Do you > have a reference? http://java.sun.com/j2se/1.4.2/docs/api/java/util/List.html Cheers, Chris -- http://blog.rebertia.com From fetchinson at googlemail.com Mon Jan 11 05:46:07 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Mon, 11 Jan 2010 11:46:07 +0100 Subject: easy_install fails on python-magic Message-ID: I'm not sure whose fault this is, the author of python-magic or easy_install (I'm guessing easy_install) but in any case easy_install python-magic (as root) fails with Searching for python-magic Reading http://pypi.python.org/simple/python-magic/ Reading http://hupp.org/adam/hg/python-magic No local packages or download links found for python-magic Best match: None Traceback (most recent call last): File "/usr/bin/easy_install", line 8, in load_entry_point('setuptools==0.6c11', 'console_scripts', 'easy_install')() File "/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py", line 1712, in main File "/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py", line 1700, in with_ei_usage File "/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py", line 1716, in File "/usr/lib64/python2.6/distutils/core.py", line 152, in setup dist.run_commands() File "/usr/lib64/python2.6/distutils/dist.py", line 975, in run_commands self.run_command(cmd) File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command cmd_obj.run() File "/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py", line 211, in run File "/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py", line 434, in easy_install File "/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/package_index.py", line 475, in fetch_distribution AttributeError: 'NoneType' object has no attribute 'clone' As far as I can see the problem is that no version of python-magic has been supplied for python 2.6 which is the version I'm using, only 2.4 and 2.5. Nevertheless easy_install should I think fail with a more user friendly message, and definitely not with a long traceback. It should I think just inform the user that no version has been found, good bye. Or something like that. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From alfps at start.no Mon Jan 11 06:12:56 2010 From: alfps at start.no (Alf P. Steinbach) Date: Mon, 11 Jan 2010 12:12:56 +0100 Subject: how to duplicate array entries In-Reply-To: References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <87ocl1ce78.fsf@rudin.co.uk> Message-ID: * Chris Rebert: > On Mon, Jan 11, 2010 at 2:20 AM, Alf P. Steinbach wrote: >> * Chris Rebert: >>> On Mon, Jan 11, 2010 at 1:03 AM, Alf P. Steinbach wrote: >>>> * Steven D'Aprano: >>>>> On Mon, 11 Jan 2010 08:56:36 +0100, Alf P. Steinbach wrote: >>>>>> * Paul Rudin: >>>>>>> Sebastian writes: > >>>>>> Using the term "array" accentuates and clarifies this most important >>>>>> aspect. >>>>> But Python lists are designed to behave as lists. >>>> No, I'm sorry, they're not. >>>> >>>> A Python 'list' has de facto constant time indexing, or "random access". >>>> >>>> A linked list -- what the informal "list" means in programming >>> Eh, it's a bit context-dependent. The abstract data type definition is >>> a superset that includes both linked lists and dynamic arrays. >> Assuming you're talking about some abstract type definition that's in some >> PEP somewhere > > No, I mean the computer science definition/term: > http://en.wikipedia.org/wiki/List_(computer_science) Note that the default meaning is a list with the characteristics of a linked list. The abstract data type specified in that article is a bit more restricted than the usual meaning of list -- as the article notes, the ADT it presents is equivalent to an abstract stack, and it's essentially the Lisp view of lists, not only just linked list but a restricted view of linked lists. To understand it you have to keep in mind that such an ADT is a simplification, for the purpose of specifying logical functionality and nothing else. The algorithmic efficiency is simply not specified, but is implied. Unfortunately, as noted there, the article doesn't cite any references or sources... Here's one: http://www.itl.nist.gov/div897/sqg/dads/HTML/list.html >>> FWIW, Java likewise uses "list" in its ADT sense. >> I'm sorry, I'm unfamiliar with that Java terminology (but then, reportedly, >> many Java programmers think that Java has "pass by reference", so nothing >> coming from that direction will surprise me very much!). The Java language >> specification has a section about arrays, none about lists AFAICS. Do you >> have a reference? > > http://java.sun.com/j2se/1.4.2/docs/api/java/util/List.html Thanks. I didn't know that. It's a very dirty hodgepodge interface with *optional* methods that throw exceptions if not implemented and apparently no constraints on algorithmic complexity for various methods. As such, it's a very good example why 'list' should not be conflated with 'array'. It leads to such monstrosities where neither correctness nor any kind of efficiency is guaranteed :-) Cheers, & thanks, - Alf PS: Please do not mail me copies of your replies to the group. A mail copy means that I may have to answer your article twice, as in this case. From mal at egenix.com Mon Jan 11 07:12:19 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 11 Jan 2010 13:12:19 +0100 Subject: Fractional Hours from datetime? In-Reply-To: References: Message-ID: <4B4B15A3.7000305@egenix.com> W. eWatson wrote: > Maybe there's a more elegant way to do this. I want to express the > result of datetime.datetime.now() in fractional hours. > > Here's one way. > > dt=datetime.datetime.now() > xtup = dt.timetuple() > h = xtup[3]+xtup[4]/60.0+xtup[5]/3600.00+xtup[6]/10**6 > # now is in fractions of an hour Here's how you'd do that with mxDateTime: >>> from mx.DateTime import now >>> now().abstime / 3600.0 13.17341068830755 .abstime gives you the time in fractional seconds. http://www.egenix.com/products/python/mxBase/mxDateTime/ -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 11 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From john_re at fastmail.us Mon Jan 11 07:29:36 2010 From: john_re at fastmail.us (giovanni_re) Date: Mon, 11 Jan 2010 04:29:36 -0800 Subject: TONIGHT Join 5-6P Mon 11th - 1st Evening Meeting test IRC & VOIP online Python at BerkeleyTIP-Global - for forwarding Message-ID: <1263212976.2403.1354001243@webmail.messagingengine.com> You're invited to the first test of the Global Python bimonthly evening meetings at BerkeleyTIP-Global. :) Join in tonight, Monday Jan 11, 5-6P Pacific, 8-9P Eastern, = Tues Jan 12 1A-2A UTC. http://sites.google.com/site/berkeleytip/schedule On #berkeleytip on irc.freenode.net, & on voip - whatever is working - try btip server first. http://sites.google.com/site/berkeleytip/remote-attendance This will be an online only meeting - no in person meeting at UCB. Hot topics: Community Leadership Summit review of interesting sessions, Spring 2010 efforts for UCB & all UC's & all college activities, Upcoming KDE conference end of next week, for 1 week, in Los Angeles. What do _you_ want to discuss? == Some people have asked for an evening meeting, because: a) they can't make weekend meetings, b) they want more BTIP-Global. ;) So, this will be a test, everyone invited, to see if we can make this work. == BerkeleyTIP-Global is the Global All Free SW HW & Culture meeting online via VOIP. http://sites.google.com/site/berkeleytip/ Join the global mailing list, say "hi", & what you're interested in. :) http://groups.google.com/group/BerkTIPGlobal For Forwarding: You are invited to forward this announcement wherever it might be appreciated. From martin.hellwig at dcuktec.org Mon Jan 11 08:18:17 2010 From: martin.hellwig at dcuktec.org (Martin P. Hellwig) Date: Mon, 11 Jan 2010 13:18:17 +0000 Subject: Fractional Hours from datetime? In-Reply-To: References: Message-ID: W. eWatson wrote: > Maybe there's a more elegant way to do this. I want to express the > result of datetime.datetime.now() in fractional hours. > > Here's one way. > > dt=datetime.datetime.now() > xtup = dt.timetuple() > h = xtup[3]+xtup[4]/60.0+xtup[5]/3600.00+xtup[6]/10**6 > # now is in fractions of an hour Here is another (though personally I don't find this more elegant than yours, perhaps a bit more readable): >>> now = datetime.datetime.now() >>> fractional_hour = int(now.strftime('%H')) + int(now.strftime('%M')) / 60.0 -- MPH http://blog.dcuktec.com 'If consumed, best digested with added seasoning to own preference.' From martin.hellwig at dcuktec.org Mon Jan 11 08:25:22 2010 From: martin.hellwig at dcuktec.org (Martin P. Hellwig) Date: Mon, 11 Jan 2010 13:25:22 +0000 Subject: Fractional Hours from datetime? In-Reply-To: References: Message-ID: Martin P. Hellwig wrote: > W. eWatson wrote: >> Maybe there's a more elegant way to do this. I want to express the >> result of datetime.datetime.now() in fractional hours. >> >> Here's one way. >> >> dt=datetime.datetime.now() >> xtup = dt.timetuple() >> h = xtup[3]+xtup[4]/60.0+xtup[5]/3600.00+xtup[6]/10**6 >> # now is in fractions of an hour > > Here is another (though personally I don't find this more elegant than > yours, perhaps a bit more readable): > > >>> now = datetime.datetime.now() > >>> fractional_hour = int(now.strftime('%H')) + int(now.strftime('%M')) > / 60.0 > Actually my version is overcomplicated: >>> now = datetime.datetime.now() >>> fractional_hour = now.hour + now.minute / 60.0 -- MPH http://blog.dcuktec.com 'If consumed, best digested with added seasoning to own preference.' From jeanmichel at sequans.com Mon Jan 11 08:52:51 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 11 Jan 2010 14:52:51 +0100 Subject: How to validate the __init__ parameters In-Reply-To: References: <803c4db7-3813-448a-b428-0ec11c57f098@t42g2000yqd.googlegroups.com> Message-ID: <4B4B2D33.5020109@sequans.com> Aahz wrote: > In article , > Jean-Michel Pichavant wrote: > >> class A: >> def __init__(self, foo = None, bar = None): >> if len(foo) > 5: >> raise ValueError('foo cannot exceed 5 characters') >> > > Bad Idea -- what happens when foo is None? > You're right. That perfectly illustrates how the simplest solution is the often most valuable one: it is much more easy to find bugs, when there is any. JM From victorsubervi at gmail.com Mon Jan 11 08:55:21 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Mon, 11 Jan 2010 09:55:21 -0400 Subject: Academic Question In-Reply-To: <4B4A25DD.4040703@mrabarnett.plus.com> References: <4dc0cfea1001100907t6bc032cftea2e9e258e7c1937@mail.gmail.com> <4B4A25DD.4040703@mrabarnett.plus.com> Message-ID: <4dc0cfea1001110555q3730f9f2s1bb41522f7f16ff8@mail.gmail.com> On Sun, Jan 10, 2010 at 3:09 PM, MRAB wrote: > browser = form.getfirst('browser', 'all') >> except: >> browser = headers() >> >> try: > A bare except, and if an exception _does_ occur, they'll be a NameError > because 'headers' isn't defined. > > Oh, not the large halibut again! (I will be cleaning them up ;) > > > os.chdir('%s/..' % cwd) >> sys.path.append(os.getcwd()) >> from templateFrame import top, bottom >> os.chdir(cwd) >> >> Why doesn't it work if I move the bottom imports to the top? The form >> values get lost, even though I chdir to cwd. >> >> I try to avoid changing the directory. > It's either that or copying them all over to the other dir, which is even worse, since I don't want to maintain identical scripts. Thanks, beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From joaopcf at gmail.com Mon Jan 11 09:00:15 2010 From: joaopcf at gmail.com (=?ISO-8859-1?Q?Jo=E3o?=) Date: Mon, 11 Jan 2010 06:00:15 -0800 (PST) Subject: force URLencoding script Message-ID: <6bc42c49-e26c-4f7b-9538-dd2e319accfb@26g2000yqo.googlegroups.com> Hi. I'm trying to figure out how to force URLencoding in my Python 2.4.3 environment receiving data an input argument but I'm really at a loss here. What am I doing wrong? #!/usr/bin/env python import sys from urllib import urlencode, urlopen from urllib2 import Request import urlparse destination = sys.argv[1] msg = sys.argv[2] #Will I have problems with this one if the input is multiline? # the browser identifies itself using the User-Agent header # after creating the Request object, it's possible to pass in a dictionary of headers user_agent = 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)' # force the request to be identified as IE 5.5.... headers = { ?User-Agent? : user_agent } # force no proxy authentication = 'UID=22541&PW=gdyb21LQTcIANtvYMT7QVQ==&' # force Unicode display format message = u'M=%s&' % msg dest_number = 'N=%s' % destination data = authentication + message + dest_number url = 'http://10.112.28.221:38080/GwHTTPin/sendtext' print 'Encoded URL:', url #get full URL adding ? to it, followed by the encoded values #full_url = url + '?' url_values #should I force url_values = urllib.urlencode(data) instead? full_url = urllib2.Request(url, data, headers) response = urllib2.urlopen(full_url) #.urlopen works transparently with proxies which do not require authentication processed = urllib.open(full_url) From jeanmichel at sequans.com Mon Jan 11 09:00:40 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 11 Jan 2010 15:00:40 +0100 Subject: Easy Q In-Reply-To: <4dc0cfea1001091018t726f283g381f815723e20f94@mail.gmail.com> References: <4dc0cfea1001090811y198462a8s39976a0622ef6c59@mail.gmail.com> <4B48B52C.3020409@islandtraining.com> <4dc0cfea1001091018t726f283g381f815723e20f94@mail.gmail.com> Message-ID: <4B4B2F08.4010800@sequans.com> Victor Subervi wrote: > On Sat, Jan 9, 2010 at 11:56 AM, Gary Herron > > wrote: > > Victor Subervi wrote: > > Hi; > I have a string.join statement on a variable that comes from a > cgi.FieldStorage().getlist. The variable may be a list or a > single value. I need to treat it differently depending on > which it is. How can I distinguish it? len(var) will obviously > give me the length of the string if it's a string and the > length of the list if it's a list. > TIA, > beno > > > Like this: > > if isinstance(var, list): > ... join ... > else: > ... ??? ... > > > Right.. Thanks! > beno You should definitely check again MRAB's answer. Having getlist returning something else than a list, especially a non iterable single item *is* suspicious. Usually, properly designed functions return consistent types over calls. JM From victorsubervi at gmail.com Mon Jan 11 09:20:33 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Mon, 11 Jan 2010 10:20:33 -0400 Subject: Easy Q In-Reply-To: <4B4B2F08.4010800@sequans.com> References: <4dc0cfea1001090811y198462a8s39976a0622ef6c59@mail.gmail.com> <4B48B52C.3020409@islandtraining.com> <4dc0cfea1001091018t726f283g381f815723e20f94@mail.gmail.com> <4B4B2F08.4010800@sequans.com> Message-ID: <4dc0cfea1001110620we2dd8eey582a3fb6c7784c75@mail.gmail.com> On Mon, Jan 11, 2010 at 10:00 AM, Jean-Michel Pichavant < jeanmichel at sequans.com> wrote: > Victor Subervi wrote: > > On Sat, Jan 9, 2010 at 11:56 AM, Gary Herron > gherron at islandtraining.com>> wrote: >> >> Victor Subervi wrote: >> >> Hi; >> I have a string.join statement on a variable that comes from a >> cgi.FieldStorage().getlist. The variable may be a list or a >> single value. I need to treat it differently depending on >> which it is. How can I distinguish it? len(var) will obviously >> give me the length of the string if it's a string and the >> length of the list if it's a list. >> TIA, >> beno >> >> >> Like this: >> >> if isinstance(var, list): >> ... join ... >> else: >> ... ??? ... >> >> >> Right.. Thanks! >> beno >> > You should definitely check again MRAB's answer. > Having getlist returning something else than a list, especially a non > iterable single item *is* suspicious. > Usually, properly designed functions return consistent types over calls. > Obviously. Thanks, beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From michele.petrazzo at REMOVE_me_unipex.it Mon Jan 11 09:34:37 2010 From: michele.petrazzo at REMOVE_me_unipex.it (Michele Petrazzo) Date: Mon, 11 Jan 2010 15:34:37 +0100 Subject: Computer Music Toolkit (CMT) ? In-Reply-To: References: Message-ID: Terry Reedy wrote: > On 1/10/2010 4:15 AM, Peter Billam wrote: >> Greetings. Is there a way to get at the Computer Music Toolkit (CMT) >> http://www.ladspa.org/cmt/ >> functionality from Python (Python3 in my case) ? > > You can access compiled C shared libraries most easily via the ctypes > module. > > so if you do develop a ctypes wrapping, consider contributing it. Since I'm interested to do it for a my new project, if the OP are interested, please contact me directly. Michele From jhcepas at gmail.com Mon Jan 11 10:16:02 2010 From: jhcepas at gmail.com (Jaime Huerta Cepas) Date: Mon, 11 Jan 2010 16:16:02 +0100 Subject: does anyone know a way to package a "static" or "portable" version of a python module? Message-ID: does anyone know a way to package a "static" version of a python module? This is, the same that pyinstaller does with single scripts, but with a complete module directory. I'm specially interested in packaging all pyqt4, sip and qt4 dependencies within the module itself. thanks, Jaime. -------------- next part -------------- An HTML attachment was scrubbed... URL: From davea at ieee.org Mon Jan 11 10:26:40 2010 From: davea at ieee.org (Dave Angel) Date: Mon, 11 Jan 2010 10:26:40 -0500 Subject: Academic Question In-Reply-To: <4dc0cfea1001110555q3730f9f2s1bb41522f7f16ff8@mail.gmail.com> References: <4dc0cfea1001100907t6bc032cftea2e9e258e7c1937@mail.gmail.com> <4B4A25DD.4040703@mrabarnett.plus.com> <4dc0cfea1001110555q3730f9f2s1bb41522f7f16ff8@mail.gmail.com> Message-ID: <4B4B4330.9010508@ieee.org> Victor Subervi wrote: > On Sun, Jan 10, 2010 at 3:09 PM, MRAB wrote: > > >> browser = form.getfirst('browser', 'all') >> >>> except: >>> browser = headers() >>> >>> try: >>> >> A bare except, and if an exception _does_ occur, they'll be a NameError >> because 'headers' isn't defined. >> >> >> > > > Oh, not the large halibut again! (I will be cleaning them up ;) > > >> os.chdir('%s/..' % cwd) >> >>> sys.path.append(os.getcwd()) >>> from templateFrame import top, bottom >>> os.chdir(cwd) >>> >>> Why doesn't it work if I move the bottom imports to the top? The form >>> values get lost, even though I chdir to cwd. >>> >>> I try to avoid changing the directory. >>> > > It's either that or copying them all over to the other dir, which is even > worse, since I don't want to maintain identical scripts. > Thanks, > beno > > You miss the point. Rather than doing chdir to change the current directory, in order to use getcwd in your sys.path.append, calculate the appropriate directory yourself, and use it directly, without changing the current directory. Brute force would be something like (untested): sys.path.append(os.path.join(os.getcwd(), "..")) You could get trickier by stripping off the last node of the directory path, but it shouldn't be necessary. Incidentally, I'd tend to use os.path.dirname( __main__.file ) rather than os.getcwd(). That way it'd work even if current directory was changed by something else. In other words (untested): sys.path.append(os.path.join(os.path.dirname(__file__), "..")) this will add the parent directory of the current module to the os.path. HTH DaveA From narkewoody at gmail.com Mon Jan 11 10:27:03 2010 From: narkewoody at gmail.com (Steven Woody) Date: Mon, 11 Jan 2010 23:27:03 +0800 Subject: pyserial: Unexpected Local Echo Message-ID: Hi, I am using pyserial. But I always get the local echo after I write some characters onto serial port and I find no way to disable this behavior. When I say 'local echo', I mean the next read operation will get characters that was just write to the same port. I run my program on cygwin (pyserial was also built on the system from source code) and the serial port i am using is a USB adapter that simulates a port (COM4 on my XP) because my laptop don't have a real serial port. But I checked my COM4 settings, there is no any think like 'local echo'. Thanks in advance. -- Life is the only flaw in an otherwise perfect nonexistence -- Schopenhauer narke public key at http://subkeys.pgp.net:11371 (narkewoody at gmail.com) From victorsubervi at gmail.com Mon Jan 11 10:31:59 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Mon, 11 Jan 2010 11:31:59 -0400 Subject: Academic Question In-Reply-To: <4B4B4330.9010508@ieee.org> References: <4dc0cfea1001100907t6bc032cftea2e9e258e7c1937@mail.gmail.com> <4B4A25DD.4040703@mrabarnett.plus.com> <4dc0cfea1001110555q3730f9f2s1bb41522f7f16ff8@mail.gmail.com> <4B4B4330.9010508@ieee.org> Message-ID: <4dc0cfea1001110731r57cb1bb9s423f50e9c80c1743@mail.gmail.com> On Mon, Jan 11, 2010 at 11:26 AM, Dave Angel wrote: > Victor Subervi wrote: > >> On Sun, Jan 10, 2010 at 3:09 PM, MRAB wrote: >> >> >> >>> browser = form.getfirst('browser', 'all') >>> >>> >>>> except: >>>> browser = headers() >>>> >>>> try: >>>> >>>> >>> A bare except, and if an exception _does_ occur, they'll be a NameError >>> because 'headers' isn't defined. >>> >>> >>> >>> >> >> >> Oh, not the large halibut again! (I will be cleaning them up ;) >> >> >> >>> os.chdir('%s/..' % cwd) >>> >>> >>>> sys.path.append(os.getcwd()) >>>> from templateFrame import top, bottom >>>> os.chdir(cwd) >>>> >>>> Why doesn't it work if I move the bottom imports to the top? The form >>>> values get lost, even though I chdir to cwd. >>>> >>>> I try to avoid changing the directory. >>>> >>>> >>> >> It's either that or copying them all over to the other dir, which is even >> worse, since I don't want to maintain identical scripts. >> Thanks, >> beno >> >> >> > You miss the point. Rather than doing chdir to change the current > directory, in order to use getcwd in your sys.path.append, calculate the > appropriate directory yourself, and use it directly, without changing the > current directory. Brute force would be something like (untested): > > sys.path.append(os.path.join(os.getcwd(), "..")) > > You could get trickier by stripping off the last node of the directory > path, but it shouldn't be necessary. > > Incidentally, I'd tend to use os.path.dirname( __main__.file ) rather > than os.getcwd(). That way it'd work even if current directory was changed > by something else. In other words (untested): > > sys.path.append(os.path.join(os.path.dirname(__file__), "..")) > > this will add the parent directory of the current module to the os.path. > Well put. Thanks! beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From jlconlin at gmail.com Mon Jan 11 10:35:41 2010 From: jlconlin at gmail.com (Jeremy) Date: Mon, 11 Jan 2010 07:35:41 -0800 (PST) Subject: Different number of matches from re.findall and re.split Message-ID: <2f8b08af-65fd-4ff9-b90b-db3f689aca01@u7g2000yqm.googlegroups.com> Hello all, I am using re.split to separate some text into logical structures. The trouble is that re.split doesn't find everything while re.findall does; i.e.: > found = re.findall('^ 1', line, re.MULTILINE) > len(found) 6439 > tables = re.split('^ 1', line, re.MULTILINE) > len(tables) > 1 Can someone explain why these two commands are giving different results? I thought I should have the same number of matches (or maybe different by 1, but not 6000!) Thanks, Jeremy From steve at holdenweb.com Mon Jan 11 10:43:54 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 11 Jan 2010 10:43:54 -0500 Subject: pyserial: Unexpected Local Echo In-Reply-To: References: Message-ID: Steven Woody wrote: > Hi, > > I am using pyserial. But I always get the local echo after I write > some characters onto serial port and I find no way to disable this > behavior. When I say 'local echo', I mean the next read operation will > get characters that was just write to the same port. > > I run my program on cygwin (pyserial was also built on the system from > source code) and the serial port i am using is a USB adapter that > simulates a port (COM4 on my XP) because my laptop don't have a real > serial port. But I checked my COM4 settings, there is no any think > like 'local echo'. > > > Thanks in advance. > It sounds to me like the device you are connecting to implements echoing. Have you tried connecting a terminal emulator to it? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From iainking at gmail.com Mon Jan 11 10:44:45 2010 From: iainking at gmail.com (Iain King) Date: Mon, 11 Jan 2010 07:44:45 -0800 (PST) Subject: Different number of matches from re.findall and re.split References: <2f8b08af-65fd-4ff9-b90b-db3f689aca01@u7g2000yqm.googlegroups.com> Message-ID: <42ac849d-fecd-4489-8c9c-4ac4dc939898@m3g2000yqf.googlegroups.com> On Jan 11, 3:35?pm, Jeremy wrote: > Hello all, > > I am using re.split to separate some text into logical structures. > The trouble is that re.split doesn't find everything while re.findall > does; i.e.: > > > > > found = re.findall('^ 1', line, re.MULTILINE) > > len(found) > ? ?6439 > > tables = re.split('^ 1', line, re.MULTILINE) > > len(tables) > > 1 > > Can someone explain why these two commands are giving different > results? ?I thought I should have the same number of matches (or maybe > different by 1, but not 6000!) > > Thanks, > Jeremy re.split doesn't take re.MULTILINE as a flag: it doesn't take any flags. It does take a maxsplit parameter, which you are passing the value of re.MULTILINE (which happens to be 8 in my implementation). Since your pattern is looking for line starts, and your first line presumably has more splits than the maxsplits you are specifying, your re.split never finds more than 1. >>> a 'split(pattern, string, maxsplit=0)\n Split the source string by the occurren ces of the pattern,\n returning a list containing the resulting substrings.\n ' >>> re.split(" ", a, re.MULTILINE) ['split(pattern,', 'string,', 'maxsplit=0)\n', '', '', '', 'Split', 'the', 'sour ce string by the occurrences of the pattern,\n returning a list containing th e resulting substrings.\n'] >>> re.split(" ", a) ['split(pattern,', 'string,', 'maxsplit=0)\n', '', '', '', 'Split', 'the', 'sour ce', 'string', 'by', 'the', 'occurrences', 'of', 'the', 'pattern,\n', '', '', '' , 'returning', 'a', 'list', 'containing', 'the', 'resulting', 'substrings.\n'] Iain From jlconlin at gmail.com Mon Jan 11 10:49:28 2010 From: jlconlin at gmail.com (Jeremy) Date: Mon, 11 Jan 2010 07:49:28 -0800 (PST) Subject: Different number of matches from re.findall and re.split References: <2f8b08af-65fd-4ff9-b90b-db3f689aca01@u7g2000yqm.googlegroups.com> <42ac849d-fecd-4489-8c9c-4ac4dc939898@m3g2000yqf.googlegroups.com> Message-ID: <4a5ebe07-aac7-40e6-ad1d-12f3d5316703@j19g2000yqk.googlegroups.com> On Jan 11, 8:44?am, Iain King wrote: > On Jan 11, 3:35?pm, Jeremy wrote: > > > > > > > Hello all, > > > I am using re.split to separate some text into logical structures. > > The trouble is that re.split doesn't find everything while re.findall > > does; i.e.: > > > > found = re.findall('^ 1', line, re.MULTILINE) > > > len(found) > > ? ?6439 > > > tables = re.split('^ 1', line, re.MULTILINE) > > > len(tables) > > > 1 > > > Can someone explain why these two commands are giving different > > results? ?I thought I should have the same number of matches (or maybe > > different by 1, but not 6000!) > > > Thanks, > > Jeremy > > re.split doesn't take re.MULTILINE as a flag: it doesn't take any > flags. It does take a maxsplit parameter, which you are passing the > value of re.MULTILINE (which happens to be 8 in my implementation). > Since your pattern is looking for line starts, and your first line > presumably has more splits than the maxsplits you are specifying, your > re.split never finds more than 1. Yep. Thanks for pointing that out. I guess I just assumed that re.split was similar to re.search/match/findall in what it accepted as function parameters. I guess I'll have to use a \n instead of a ^ for split. Thanks, Jeremy From howe.steven at gmail.com Mon Jan 11 10:52:26 2010 From: howe.steven at gmail.com (Steven Howe) Date: Mon, 11 Jan 2010 07:52:26 -0800 Subject: how to duplicate array entries In-Reply-To: <14437829-e3ef-41ac-b533-bc7003c1434b@j19g2000yqk.googlegroups.com> References: <25568f3b-637b-4ae8-87c3-7c5bc004cca0@22g2000yqr.googlegroups.com> <14437829-e3ef-41ac-b533-bc7003c1434b@j19g2000yqk.googlegroups.com> Message-ID: <4B4B493A.8050406@gmail.com> try --------------------------------------- #!/usr/bin/env python from types import ListType, IntType def array_expander( ar=None, ex=None ): if type( ex ) != IntType: return [] if ex <= 0: return [] if type( ar ) != ListType: return [] # working code starts here # res = [] for t in ar: for x in range( ex ): res.append( t ) return res # function ends # res = array_expander( [1,11,3,5], 4 ) print res --------------------------------------- [1, 1, 1, 1, 11, 11, 11, 11, 3, 3, 3, 3, 5, 5, 5, 5] -- *Kiri-kin-tha's* First Law of Metaphysics is /"Nothing unreal exists."/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From kw at codebykevin.com Mon Jan 11 10:54:57 2010 From: kw at codebykevin.com (Kevin Walzer) Date: Mon, 11 Jan 2010 10:54:57 -0500 Subject: Procedural API inside class--scoping questions Message-ID: I'm trying to make use of a Python library, aemreceive, that provides a procedural API. (aemreceive is a library for Python on the Mac that allows the application to receive and respond to Apple Events.) My Python apps basically run in a single fooApp class, and everything runs inside the class. To launch the app (my apps are Tkinter-based), I use something like this: if __name__== '__main__': app = fooApp(None) app.mainloop() In keeping with aemreceive's procedural API, I've added a runCommand function inside the app class to provide some basic functionality in response to Apple Events it will set some objects/variables, then display the output in the app. In my code it looks like this: def runCommand(string): self.searchterm=string self.servertree.selection_set('Default') self.getInfo() When I test this command, I get an error from Python: "self" is not defined. I think I understand what is going here. All of the other functions in the fooApp class take "self" as a parameter, i.e. they are components of the class. runCommand is not, so therefore "self" is undefined. My question is, how can I get the values from the class (self.searchterm, et.al) inside the runCommand function? Thanks, Kevin -- Kevin Walzer Code by Kevin http://www.codebykevin.com From arnodel at googlemail.com Mon Jan 11 10:56:19 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 11 Jan 2010 07:56:19 -0800 (PST) Subject: Different number of matches from re.findall and re.split References: <2f8b08af-65fd-4ff9-b90b-db3f689aca01@u7g2000yqm.googlegroups.com> Message-ID: <085d5292-00af-4452-9f9c-da881f580b88@a21g2000yqc.googlegroups.com> On 11 Jan, 15:35, Jeremy wrote: > Hello all, > > I am using re.split to separate some text into logical structures. > The trouble is that re.split doesn't find everything while re.findall > does; i.e.: > > > > > found = re.findall('^ 1', line, re.MULTILINE) > > len(found) > ? ?6439 > > tables = re.split('^ 1', line, re.MULTILINE) > > len(tables) > > 1 > > Can someone explain why these two commands are giving different > results? ?I thought I should have the same number of matches (or maybe > different by 1, but not 6000!) > > Thanks, > Jeremy When in doubt, the documentation is a good place to start :) http://docs.python.org/library/re.html#re.split re.split(pattern, string[, maxsplit=0]) http://docs.python.org/library/re.html#re.findall re.findall(pattern, string[, flags]) Notice that re.split's optional third argument is not for passing flags. HTH -- Arnaud From arnodel at googlemail.com Mon Jan 11 11:01:16 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 11 Jan 2010 08:01:16 -0800 (PST) Subject: Procedural API inside class--scoping questions References: Message-ID: On 11 Jan, 15:54, Kevin Walzer wrote: > I'm trying to make use of a Python library, aemreceive, that provides a > procedural API. (aemreceive is a library for Python on the Mac that > allows the application to receive and respond to Apple Events.) > > My Python apps basically run in a single fooApp class, and everything > runs inside the class. To launch the app (my apps are Tkinter-based), I > use something like this: > > if __name__== '__main__': > ? ? ?app = fooApp(None) > ? ? ?app.mainloop() > > In keeping with aemreceive's procedural API, I've added a runCommand > function inside the app class to provide some basic functionality in > response to Apple Events it will set some objects/variables, then > display the output in the app. In my code it looks like this: > > ? ? ?def runCommand(string): > ? ? ? ? ?self.searchterm=string > ? ? ? ? ?self.servertree.selection_set('Default') > ? ? ? ? ?self.getInfo() > > When I test this command, I get an error from Python: "self" is not > defined. > Is runcommand is method of your class? In that case you should define it as: def runCommand(self, string): ... HTH -- Arnaud From __peter__ at web.de Mon Jan 11 11:03:42 2010 From: __peter__ at web.de (Peter Otten) Date: Mon, 11 Jan 2010 17:03:42 +0100 Subject: Different number of matches from re.findall and re.split References: <2f8b08af-65fd-4ff9-b90b-db3f689aca01@u7g2000yqm.googlegroups.com> <42ac849d-fecd-4489-8c9c-4ac4dc939898@m3g2000yqf.googlegroups.com> <4a5ebe07-aac7-40e6-ad1d-12f3d5316703@j19g2000yqk.googlegroups.com> Message-ID: Jeremy wrote: > On Jan 11, 8:44 am, Iain King wrote: >> On Jan 11, 3:35 pm, Jeremy wrote: >> >> >> >> >> >> > Hello all, >> >> > I am using re.split to separate some text into logical structures. >> > The trouble is that re.split doesn't find everything while re.findall >> > does; i.e.: >> >> > > found = re.findall('^ 1', line, re.MULTILINE) >> > > len(found) >> > 6439 >> > > tables = re.split('^ 1', line, re.MULTILINE) >> > > len(tables) >> > > 1 >> >> > Can someone explain why these two commands are giving different >> > results? I thought I should have the same number of matches (or maybe >> > different by 1, but not 6000!) >> >> > Thanks, >> > Jeremy >> >> re.split doesn't take re.MULTILINE as a flag: it doesn't take any >> flags. It does take a maxsplit parameter, which you are passing the >> value of re.MULTILINE (which happens to be 8 in my implementation). >> Since your pattern is looking for line starts, and your first line >> presumably has more splits than the maxsplits you are specifying, your >> re.split never finds more than 1. > > Yep. Thanks for pointing that out. I guess I just assumed that > re.split was similar to re.search/match/findall in what it accepted as > function parameters. I guess I'll have to use a \n instead of a ^ for > split. You can precompile the pattern and then invoke the split() method: >>> re.compile("^X", re.MULTILINE).split("""X alpha ... beta ... X gamma ... delta X ... X ... zeta ... """) ['', ' alpha\nbeta\n', ' gamma\ndelta X\n', '\nzeta\n'] Peter From python at mrabarnett.plus.com Mon Jan 11 11:04:07 2010 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 11 Jan 2010 16:04:07 +0000 Subject: Different number of matches from re.findall and re.split In-Reply-To: <4a5ebe07-aac7-40e6-ad1d-12f3d5316703@j19g2000yqk.googlegroups.com> References: <2f8b08af-65fd-4ff9-b90b-db3f689aca01@u7g2000yqm.googlegroups.com> <42ac849d-fecd-4489-8c9c-4ac4dc939898@m3g2000yqf.googlegroups.com> <4a5ebe07-aac7-40e6-ad1d-12f3d5316703@j19g2000yqk.googlegroups.com> Message-ID: <4B4B4BF7.6040903@mrabarnett.plus.com> Jeremy wrote: > On Jan 11, 8:44 am, Iain King wrote: >> On Jan 11, 3:35 pm, Jeremy wrote: >> >> >> >> >> >>> Hello all, >>> I am using re.split to separate some text into logical structures. >>> The trouble is that re.split doesn't find everything while re.findall >>> does; i.e.: >>>> found = re.findall('^ 1', line, re.MULTILINE) >>>> len(found) >>> 6439 >>>> tables = re.split('^ 1', line, re.MULTILINE) >>>> len(tables) >>>> 1 >>> Can someone explain why these two commands are giving different >>> results? I thought I should have the same number of matches (or maybe >>> different by 1, but not 6000!) >>> Thanks, >>> Jeremy >> re.split doesn't take re.MULTILINE as a flag: it doesn't take any >> flags. It does take a maxsplit parameter, which you are passing the >> value of re.MULTILINE (which happens to be 8 in my implementation). >> Since your pattern is looking for line starts, and your first line >> presumably has more splits than the maxsplits you are specifying, your >> re.split never finds more than 1. > > Yep. Thanks for pointing that out. I guess I just assumed that > re.split was similar to re.search/match/findall in what it accepted as > function parameters. I guess I'll have to use a \n instead of a ^ for > split. > You could use the .split method of a pattern object instead: tables = re.compile('^ 1', re.MULTILINE).split(line) From ethanwinograd at gmail.com Mon Jan 11 11:04:08 2010 From: ethanwinograd at gmail.com (ethan) Date: Mon, 11 Jan 2010 08:04:08 -0800 (PST) Subject: Python POS/cash register projects? Message-ID: <599fd6e0-5d0e-4ed4-b5df-8ae8e237e8bb@w12g2000vbj.googlegroups.com> Anybody have any experience with creating a basic POS register system in Python? Any existing projects out there you are aware of? This would be a GUI app, standalone with some basic export and print functions. I see this as a great opportunity to deepen my Python experience but dont want to reinvent the wheel completely.. doesn't look like there is a lot out there.. From duncan.booth at invalid.invalid Mon Jan 11 11:28:54 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 11 Jan 2010 16:28:54 GMT Subject: Different number of matches from re.findall and re.split References: <4a5ebe07-aac7-40e6-ad1d-12f3d5316703@j19g2000yqk.googlegroups.com> Message-ID: MRAB wrote: >> Yep. Thanks for pointing that out. I guess I just assumed that >> re.split was similar to re.search/match/findall in what it accepted as >> function parameters. I guess I'll have to use a \n instead of a ^ for >> split. >> > You could use the .split method of a pattern object instead: > > tables = re.compile('^ 1', re.MULTILINE).split(line) or you might include the flag in the regular expression literal: '(?m)^ 1' -- Duncan Booth http://kupuguy.blogspot.com From robert.kern at gmail.com Mon Jan 11 11:44:10 2010 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 11 Jan 2010 10:44:10 -0600 Subject: lightweight encryption of text file In-Reply-To: <4B4851C6.1050009@bluewin.ch> References: <4B4851C6.1050009@bluewin.ch> Message-ID: On 2010-01-09 03:52 AM, Anthra Norell wrote: > Daniel Fetchinson wrote: > > I have a plain text file which I would like to protect in a very > > simple minded, yet for my purposes sufficient, way. I'd like to > > encrypt/convert it into a binary file in such a way that possession of > > a password allows anyone to convert it back into the original text > > file while not possessing the password one would only see the > > following with the standard linux utility 'file': > > > > [fetchinson at fetch ~]$ file encrypted.data > > encrypted.data: data > > > > and the effort required to convert the file back to the original text > > file without the password would be equivalent to guessing the > > password. > > > > I'm fully aware of the security implications of this loose > > specification, but for my purposes this would be a good solution. > > > > What would be the simplest way to achieve this using preferably stock > > python without 3rd party modules? If a not too complex 3rd part > > module made it really simple that would be acceptable too. > > Daniel, > > Here's what looks like another thread veering off into package-ology, > leaving a stumped OP behind. > > "Don't use a random generator for encryption purposes!" warns the > manual, of which fact I was reminded in no uncertain terms on this forum > a few years ago when I proposed the following little routine in response > to a post very similar to yours. One critic challenged me to encode my > credit card data and post it. Which I did. Actually, you just "encrypted" your credit card number and challenged comp.lang.python to crack it. No one challenged you to do anything of the sort. Fortunately, the ever-watchful eye of Google was upon us that day: http://groups.google.com/group/comp.lang.python/browse_thread/thread/5fb9ffada975bae9?pli=1 > Upon which another critic > conjured up the horror vision of gigahertzes hacking my pathetic little > effort to pieces as I was reading his message. Of the well-meaning kind, > he urged me to put an immediate stop to this foolishness. I didn't. > > No unplanned expenditures ensued. That's because comp.lang.python is not full of thieves, not because your algorithm is worth a damn. p3.py imposes no more overhead than this, but it has some real security properties. To quote Paul Rubin from that previous thread: """ Since good encryption schemes that don't have significant performance penalties are widely available, why mess with a crap scheme EVER? Why use a solution that "might or might not be adequate" when you can use one that's definitely ok? """ -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From wolftracks at invalid.com Mon Jan 11 11:44:26 2010 From: wolftracks at invalid.com (W. eWatson) Date: Mon, 11 Jan 2010 08:44:26 -0800 Subject: Fractional Hours from datetime? In-Reply-To: <05ca0e15-b55e-4ede-81fe-c8f4459e3932@r5g2000yqb.googlegroups.com> References: <3ca2bf74-744d-421e-a78e-38dbeac2cda1@m3g2000yqf.googlegroups.com> <05ca0e15-b55e-4ede-81fe-c8f4459e3932@r5g2000yqb.googlegroups.com> Message-ID: Austyn wrote: > Here's an improvement in case you want your code to work outside of > Arizona: > > from time import time, timezone > h = ((time() - timezone) / 3600) % 24 > > On Jan 10, 9:04 pm, Austyn wrote: >> How about: >> >> import time >> arizona_utc_offset = -7.00 >> h = (time.time() / 3600 + arizona_utc_offset) % 24 >> >> dt.timetuple()[6] is the day of the week; struct tm_time doesn't >> include a sub-second field. >> >> On Jan 10, 10:28 am, "W. eWatson" wrote: >> >> >> >>> Maybe there's a more elegant way to do this. I want to express the >>> result of datetime.datetime.now() in fractional hours. >>> Here's one way. >>> dt=datetime.datetime.now() >>> xtup = dt.timetuple() >>> h = xtup[3]+xtup[4]/60.0+xtup[5]/3600.00+xtup[6]/10**6 >>> # now is in fractions of an hour > There seems to be some controversy about this and other matters of datetime. From bruno.42.desthuilliers at websiteburo.invalid Mon Jan 11 12:05:43 2010 From: bruno.42.desthuilliers at websiteburo.invalid (Bruno Desthuilliers) Date: Mon, 11 Jan 2010 18:05:43 +0100 Subject: Procedural API inside class--scoping questions In-Reply-To: References: Message-ID: <4b4b5a65$0$32713$426a74cc@news.free.fr> Kevin Walzer a ?crit : > I'm trying to make use of a Python library, aemreceive, that provides a > procedural API. (aemreceive is a library for Python on the Mac that > allows the application to receive and respond to Apple Events.) > > My Python apps basically run in a single fooApp class, and everything > runs inside the class. To launch the app (my apps are Tkinter-based), I > use something like this: > > if __name__== '__main__': > app = fooApp(None) > app.mainloop() > > In keeping with aemreceive's procedural API, I've added a runCommand > function inside the app class to provide some basic functionality in > response to Apple Events it will set some objects/variables, then > display the output in the app. In my code it looks like this: > > def runCommand(string): > self.searchterm=string > self.servertree.selection_set('Default') > self.getInfo() > > When I test this command, I get an error from Python: "self" is not > defined. Indeed. > I think I understand what is going here. All of the other functions in > the fooApp class take "self" as a parameter, i.e. they are components of > the class. It actually works the other way round : it's because they are "components" (we prefer to say "attributes") of the class that they take the current instance as first param. FWIW, the following syntaxes are functionnaly equivalent: obj = MyClass() obj.some_method() MyClass.some_method(obj) # if some_method is not inherited: MyClass.__dict__['some_method'](obj) Anyway: > runCommand is not, so therefore "self" is undefined. My > question is, how can I get the values from the class (self.searchterm, s/class/instance/ here. > et.al) inside the runCommand function? The usual way - adds the relevant param: def runCommand(self, string): self.searchterm=string self.servertree.selection_set('Default') self.getInfo() The problem is that you don't provide much information about how this function is actually called. From mrkafk at gmail.com Mon Jan 11 12:29:49 2010 From: mrkafk at gmail.com (mk) Date: Mon, 11 Jan 2010 18:29:49 +0100 Subject: Interesting (?) problem Message-ID: Hello everyone, I have two lists of IP addresses: hostips = [ 'a', 'b', 'c', 'd', 'e' ] thread_results = [ 'd', 'b', 'c' ] I need to sort thread_results in the same order as hostips. (Obviously, hostips can contain any valid ip addresses as strings, they are sorted alphabetically here just for sake of example.) Since explicit is better than implicit, I will clarify: thread_results is obviously result of threads communicating with IPs from hostips, and that can finish at various times, thus returning ips into thread_results in any order. Sorting would be trivial to do if thread_results were not a subset of hostips (obviously, for some IPs communication can fail which excludes them from the result). One approach I can see is constructing hostips_limited list that would contain only ips that are in thread_results but that would preserve order of hostips: hostips_limited = [] for h in hostips: if h in thread_results: hostips_limited.append(h) ..and then doing sorting thread_results. But maybe there is more elegant / faster approach? Incidentally, it *seems* that list comprehension preserves order: hostips_limited = [ h for h in hostips if h in thread_results ] Empirically speaking it seems to work (I tested it on real ips), but please correct me if that's wrong. Regards, mk From jeanmichel at sequans.com Mon Jan 11 12:48:35 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 11 Jan 2010 18:48:35 +0100 Subject: Interesting (?) problem In-Reply-To: References: Message-ID: <4B4B6473.5050309@sequans.com> mk wrote: > > Incidentally, it *seems* that list comprehension preserves order: > > hostips_limited = [ h for h in hostips if h in thread_results ] > > Empirically speaking it seems to work (I tested it on real ips), but > please correct me if that's wrong. > > > > Regards, > mk > Sounds good to me. List are *ordered* items ; it does not suprise me that list comprehension consistently keep the order. JM From mrkafk at gmail.com Mon Jan 11 12:57:24 2010 From: mrkafk at gmail.com (mk) Date: Mon, 11 Jan 2010 18:57:24 +0100 Subject: Interesting (?) problem In-Reply-To: References: Message-ID: mk wrote: > Hello everyone, > > I have two lists of IP addresses: > > hostips = [ 'a', 'b', 'c', 'd', 'e' ] > > thread_results = [ 'd', 'b', 'c' ] > > I need to sort thread_results in the same order as hostips. P.S. One clarification: those lists are actually more complicated (thread_result is a list of tuples (ip, thread)), which is why I need thread_results sorted in order of hostips (instead of just constructing [ h for h in hostips if h in thread_results ] and be done with it). From arnodel at googlemail.com Mon Jan 11 13:17:38 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 11 Jan 2010 18:17:38 +0000 Subject: Interesting (?) problem References: Message-ID: mk writes: > Hello everyone, > > I have two lists of IP addresses: > > hostips = [ 'a', 'b', 'c', 'd', 'e' ] > > thread_results = [ 'd', 'b', 'c' ] > > I need to sort thread_results in the same order as hostips. [...solution:] > hostips_limited = [] > for h in hostips: > if h in thread_results: > hostips_limited.append(h) > > ..and then doing sorting thread_results. What do you mean by that last sentence? [... or:] > Incidentally, it *seems* that list comprehension preserves order: > > hostips_limited = [ h for h in hostips if h in thread_results ] That's ok, but why not keep thread_results as a set instead of a list if the ordering in that container is not significant but you are testing membership a lot? -- Arnaud From Erman.CELEN at 3ds.com Mon Jan 11 13:27:07 2010 From: Erman.CELEN at 3ds.com (CELEN Erman) Date: Mon, 11 Jan 2010 13:27:07 -0500 Subject: C Module's '1.#INF' changes to 'inf' at Python In-Reply-To: References: Message-ID: > Numeric.log10() will check to see if the errno was set to ERANGE. It does not > check if a floating point exception flag was set, which is tricky to do across > platforms. The newer numpy can do it because we've finally managed to implement > all of that platform-specific code, but the earlier Numeric does not. > Presumably, the msvc8 C runtime's implementation of log10() sets errno and the > msvc9 runtime's version does not. It doesn't seem like C part is changed. I confirmed that the behavior of log10(0.0) in C's standard math.h library didn't change between compilers (msvc8 and msvc9 both sets the errno to 34(ERANGE)). Now I'm thinking that this setting errno problem is happening somewhere in Numeric's log10 wrappers. As I see, Numeric's "PyUFunc_GenericFunction" checks errno value to see if the called function has set it and if it is non-zero, it calls math_error which raises the exception. The problem is that now errno is not being set but I can't see why since I can't step into that redirected function call ("*(double *)op = ((DoubleUnaryFunc *)func)(*(double *)ip1)" where function value is "0x01c4ede0 log10"). I am wondering which functions are actually being called when I call log10(0.0). Could you (or anybody) point me where this errno is supposed to be set in Numeric or umath when I call log10(0.0) so that I can take a look at why this is not being the case. (I also noticed that this behavior is same under standard NumPy 1.4 with standard Python 2.6 on Windows. If you call numpy.log10(0.0) you will get an "-inf" and no exceptions will be raised. Which is not the case with Python's standard math.log10(0.0) which will raise a ValueError) Best Regards, Ali Erman CELEN Platform Specialists / Porting This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged. If you are not one of the named recipients or have received this email in error, (i) you should not read, disclose, or copy it, (ii) please notify sender of your receipt by reply email and delete this email and all attachments, (iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.For other languages, go to http://www.3ds.com/terms/email-disclaimer. From __peter__ at web.de Mon Jan 11 13:34:51 2010 From: __peter__ at web.de (Peter Otten) Date: Mon, 11 Jan 2010 19:34:51 +0100 Subject: Interesting (?) problem References: Message-ID: mk wrote: > mk wrote: >> Hello everyone, >> >> I have two lists of IP addresses: >> >> hostips = [ 'a', 'b', 'c', 'd', 'e' ] >> >> thread_results = [ 'd', 'b', 'c' ] >> >> I need to sort thread_results in the same order as hostips. > > P.S. One clarification: those lists are actually more complicated > (thread_result is a list of tuples (ip, thread)), which is why I need > thread_results sorted in order of hostips (instead of just constructing > [ h for h in hostips if h in thread_results ] and be done with it). Make it [(host, thread) for host, thread in thread_results if host in hostips] then. However, if you care to explain what you're intending to do with the resulting list I'm sure someone will come up with a more efficient approach based on sets or dicts. Peter From scott.freemire at gmail.com Mon Jan 11 13:35:04 2010 From: scott.freemire at gmail.com (Scott) Date: Mon, 11 Jan 2010 10:35:04 -0800 (PST) Subject: Fundamental Function Question (beginner) Message-ID: <8d7ef0f0-2a0f-4a43-aced-8f4755406249@e27g2000yqd.googlegroups.com> When creating a function is there any difference between putting everything under the "def" or not? Here I created a function called CscoPortNum to convert the network port number field in a Cisco syslog string from a an ascii name back into its numeric form if required. Does it matter at all that I created the translation dictionary first and then started the def? # def CscoPortNum(RulL) # Accept a single ACL Rule as a List split into individual words and # return Port number. Convert from Cisco syslog Port name if required portfpth = "\\progra~1\\syslogd\\ACL_Logs\\Port-Translations.txt" # Create dictionary of portnames to portnumbers portD = {} for prtnmS in open(portfpth): prtnmS = prtnmS.rstrip() spltprtL = prtnmS.split(" ") portD[spltprtL[2]] = [spltprtL[1]] def CscoPortNum(RulL): if "eq" in RulL: # Is the Port listed? if RulL[RulL.index("eq")+1][0].isdigit(): # Is it numeric? # if re.search("\d", RulL[RulL.index("eq")+1][0]): # Is it numeric? portnum = RulL[RulL.index("eq")+1] # If numeric, use as is. else: # If named, look up numeric translation portnum = portD[RulL[RulL.index("eq")+1]] portnum = str(portnum).strip("[]'") else: portnum = "noeq" return portnum From jeanmichel at sequans.com Mon Jan 11 13:36:05 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 11 Jan 2010 19:36:05 +0100 Subject: Interesting (?) problem In-Reply-To: References: Message-ID: <4B4B6F95.1020704@sequans.com> mk wrote: > mk wrote: >> Hello everyone, >> >> I have two lists of IP addresses: >> >> hostips = [ 'a', 'b', 'c', 'd', 'e' ] >> >> thread_results = [ 'd', 'b', 'c' ] >> >> I need to sort thread_results in the same order as hostips. > > P.S. One clarification: those lists are actually more complicated > (thread_result is a list of tuples (ip, thread)), which is why I need > thread_results sorted in order of hostips (instead of just > constructing [ h for h in hostips if h in thread_results ] and be done > with it). > > > > Could be easily done by changing the thread_results structure. You seems to want to lookup using ips: thread_results = {'ip1':['t1','t2','t3'], 'ip3':['t4','t8']} # tx are threads results = [] for ip in hostips: results += [(ip, thread) for thread in thread_results.get(ip, [])] >>> print results Out[10]: [('ip1', 't1'), ('ip1', 't2'), ('ip1', 't3'), ('ip3', 't4'), ('ip3', 't8')] In a general manner, if you end up with duplicated informations inside your structures (like ips in you thread_results structure) that means you'll need sooner or later to do additional loop work to factorize the data. JM From nobody at nowhere.com Mon Jan 11 13:39:25 2010 From: nobody at nowhere.com (Nobody) Date: Mon, 11 Jan 2010 18:39:25 +0000 Subject: integer and string compare, is that correct? References: <2f718029-1682-47f6-a2c3-6e40b51bd71a@v25g2000yqk.googlegroups.com> Message-ID: On Sun, 10 Jan 2010 23:13:55 -0800, Dan Bishop wrote: >> If you actually need to perform comparisons across types, you can rely >> upon the fact that tuple comparisons are non-strict and use e.g.: >> >> ? ? ? ? > a = 5 >> ? ? ? ? > b = '5' >> ? ? ? ? > (type(a).__name__, a) < (type(b).__name__, b) >> ? ? ? ? True >> ? ? ? ? > (type(a).__name__, a) > (type(b).__name__, b) >> ? ? ? ? False >> >> The second elements will only be compared if the first elements are equal >> (i.e. the values have the same type). > > But this method gives you 3.0 < 2 because 'float' < 'int'. Probably > not what you want. If you're comparing instances of entirely arbitrary types, what you probably want (and the only thing you're going to get) is an entirely arbitrary ordering. The main case where such a comparison makes sense is for algorithms which require a total ordering (e.g. tree-like structures), and those won't care if 3<2 so long as the axioms for a total ordering hold. From no.email at nospam.invalid Mon Jan 11 13:42:32 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 11 Jan 2010 10:42:32 -0800 Subject: Interesting (?) problem References: Message-ID: <7xaawkpjon.fsf@ruckus.brouhaha.com> mk writes: > I have two lists of IP addresses: > > hostips = [ 'a', 'b', 'c', 'd', 'e' ] > > thread_results = [ 'd', 'b', 'c' ] > > I need to sort thread_results in the same order as hostips. Assuming each address in hostips appears just once: from itertools import izip,count d = dict(izip(hostips, count())) sorted_results = sorted(thread_results, key=d.get) From nobody at nowhere.com Mon Jan 11 13:47:31 2010 From: nobody at nowhere.com (Nobody) Date: Mon, 11 Jan 2010 18:47:31 +0000 Subject: sys.stdout vs. sys.stderr References: Message-ID: On Mon, 11 Jan 2010 10:09:36 +0100, Martin v. Loewis wrote: >> In Python 3.1 is there any difference in the buffering behavior of the >> initial sys.stdout and sys.stderr streams? > > No. > >> Were they different at some earlier point in Python's evolution? > > That depends on the operating system. These used to be whatever the > C library set up as stdout and stderr. Typically, they were buffered > in the same way. On Unix, stdout will be line buffered if it is associated with a tty and fully buffered otherwise, while stderr is always unbuffered. On Windows, stdout and stderr are unbuffered if they refer to a character device, fully buffered otherwise (Windows doesn't have line buffering; setvbuf(_IOLBF) is equivalent to setvbuf(_IOFBF)). ANSI C says: As initially opened, the standard error stream is not fully buffered; the standard input and standard output streams are fully buffered if and only if the stream can be determined not to refer to an interactive device. From robert.kern at gmail.com Mon Jan 11 13:52:53 2010 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 11 Jan 2010 12:52:53 -0600 Subject: C Module's '1.#INF' changes to 'inf' at Python In-Reply-To: References: Message-ID: On 2010-01-11 12:27 PM, CELEN Erman wrote: >> Numeric.log10() will check to see if the errno was set to ERANGE. It does not >> check if a floating point exception flag was set, which is tricky to do across >> platforms. The newer numpy can do it because we've finally managed to implement >> all of that platform-specific code, but the earlier Numeric does not. >> Presumably, the msvc8 C runtime's implementation of log10() sets errno and the >> msvc9 runtime's version does not. > > It doesn't seem like C part is changed. I confirmed that the behavior of log10(0.0) in C's standard math.h library didn't change between compilers (msvc8 and msvc9 both sets the errno to 34(ERANGE)). Now I'm thinking that this setting errno problem is happening somewhere in Numeric's log10 wrappers. > > As I see, Numeric's "PyUFunc_GenericFunction" checks errno value to see if the called function has set it and if it is non-zero, it calls math_error which raises the exception. The problem is that now errno is not being set but I can't see why since I can't step into that redirected function call ("*(double *)op = ((DoubleUnaryFunc *)func)(*(double *)ip1)" where function value is "0x01c4ede0 log10"). This is the math library's log10() function. > I am wondering which functions are actually being called when I call log10(0.0). Could you (or anybody) point me where this errno is supposed to be set in Numeric or umath when I call log10(0.0) so that I can take a look at why this is not being the case. errno gets set to 0 before PyUFunc_GenericFunction calls the underlying log10() function. Other than that, Numeric does not set errno. > (I also noticed that this behavior is same under standard NumPy 1.4 with standard Python 2.6 on Windows. If you call numpy.log10(0.0) you will get an "-inf" and no exceptions will be raised. Which is not the case with Python's standard math.log10(0.0) which will raise a ValueError) Correct. This is numpy's intended behavior. See numpy.seterr() to enable exceptions if you want them. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From python at mrabarnett.plus.com Mon Jan 11 13:57:09 2010 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 11 Jan 2010 18:57:09 +0000 Subject: Fundamental Function Question (beginner) In-Reply-To: <8d7ef0f0-2a0f-4a43-aced-8f4755406249@e27g2000yqd.googlegroups.com> References: <8d7ef0f0-2a0f-4a43-aced-8f4755406249@e27g2000yqd.googlegroups.com> Message-ID: <4B4B7485.7050409@mrabarnett.plus.com> Scott wrote: > When creating a function is there any difference between putting > everything under the "def" or not? > > Here I created a function called CscoPortNum to convert the network > port number field in a Cisco syslog string from a an ascii name back > into its numeric form if required. Does it matter at all that I > created the translation dictionary first and then started the def? > > # def CscoPortNum(RulL) > # Accept a single ACL Rule as a List split into individual words and > # return Port number. Convert from Cisco syslog Port name if required > portfpth = "\\progra~1\\syslogd\\ACL_Logs\\Port-Translations.txt" > # Create dictionary of portnames to portnumbers > portD = {} > for prtnmS in open(portfpth): > prtnmS = prtnmS.rstrip() > spltprtL = prtnmS.split(" ") > portD[spltprtL[2]] = [spltprtL[1]] > def CscoPortNum(RulL): > if "eq" in RulL: # Is the Port listed? > if RulL[RulL.index("eq")+1][0].isdigit(): # Is it numeric? > # if re.search("\d", RulL[RulL.index("eq")+1][0]): # Is it > numeric? > portnum = RulL[RulL.index("eq")+1] # If numeric, use > as is. > else: > # If named, look up numeric translation > portnum = portD[RulL[RulL.index("eq")+1]] > portnum = str(portnum).strip("[]'") > else: portnum = "noeq" > return portnum There's nothing wrong with building dicts or other lookup tables outside a function in order to avoid re-creating them every time the function is called. From nobody at nowhere.com Mon Jan 11 13:59:01 2010 From: nobody at nowhere.com (Nobody) Date: Mon, 11 Jan 2010 18:59:01 +0000 Subject: pyserial: Unexpected Local Echo References: Message-ID: On Mon, 11 Jan 2010 23:27:03 +0800, Steven Woody wrote: > I am using pyserial. But I always get the local echo after I write > some characters onto serial port and I find no way to disable this > behavior. When I say 'local echo', I mean the next read operation will > get characters that was just write to the same port. That explains the "echo" part. What makes you think that it's local? From aioe.org at technicalbloke.com Mon Jan 11 14:08:49 2010 From: aioe.org at technicalbloke.com (r0g) Date: Mon, 11 Jan 2010 19:08:49 +0000 Subject: Fundamental Function Question (beginner) References: <8d7ef0f0-2a0f-4a43-aced-8f4755406249@e27g2000yqd.googlegroups.com> Message-ID: Scott wrote: > When creating a function is there any difference between putting > everything under the "def" or not? > > Here I created a function called CscoPortNum to convert the network > port number field in a Cisco syslog string from a an ascii name back > into its numeric form if required. Does it matter at all that I > created the translation dictionary first and then started the def? > > # def CscoPortNum(RulL) > # Accept a single ACL Rule as a List split into individual words and > # return Port number. Convert from Cisco syslog Port name if required > portfpth = "\\progra~1\\syslogd\\ACL_Logs\\Port-Translations.txt" > # Create dictionary of portnames to portnumbers > portD = {} > for prtnmS in open(portfpth): > prtnmS = prtnmS.rstrip() > spltprtL = prtnmS.split(" ") > portD[spltprtL[2]] = [spltprtL[1]] > def CscoPortNum(RulL): > if "eq" in RulL: # Is the Port listed? > if RulL[RulL.index("eq")+1][0].isdigit(): # Is it numeric? > # if re.search("\d", RulL[RulL.index("eq")+1][0]): # Is it > numeric? > portnum = RulL[RulL.index("eq")+1] # If numeric, use > as is. > else: > # If named, look up numeric translation > portnum = portD[RulL[RulL.index("eq")+1]] > portnum = str(portnum).strip("[]'") > else: portnum = "noeq" > return portnum In this snippet no, you're not calling the function in the preceding code so there's no problem. You can intersperse functions with the rest of your code however you like, they just wont be visible to the preceding code, but it's better to stick them all at the top of your script. Even better when you have more than a handful is to bundle functions into separate py files and then import that file e.g. ---contents of foobar.py----- def foo(): print "foo" def bar: print "bar" ---contents of your main script----- import foobar print foo(),bar() Roger. From nobody at nowhere.com Mon Jan 11 14:10:30 2010 From: nobody at nowhere.com (Nobody) Date: Mon, 11 Jan 2010 19:10:30 +0000 Subject: Interesting (?) problem References: Message-ID: On Mon, 11 Jan 2010 18:57:24 +0100, mk wrote: >> I have two lists of IP addresses: >> >> hostips = [ 'a', 'b', 'c', 'd', 'e' ] >> >> thread_results = [ 'd', 'b', 'c' ] >> >> I need to sort thread_results in the same order as hostips. > > P.S. One clarification: those lists are actually more complicated > (thread_result is a list of tuples (ip, thread)), which is why I need > thread_results sorted in order of hostips (instead of just constructing > [ h for h in hostips if h in thread_results ] and be done with it). 1. thread_results_dict = dict([(v[0], v) for v in thread_results]) [thread_results_dict[h] for h in hostips if h in thread_results_dict] 2. hostips_dict = dict([(ip, idx) for idx, ip in enumerate(hostips)]) sorted(thread_results, key = lambda r: hostips_dict[r[0]]) From steve at holdenweb.com Mon Jan 11 14:11:11 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 11 Jan 2010 14:11:11 -0500 Subject: Different number of matches from re.findall and re.split In-Reply-To: <4a5ebe07-aac7-40e6-ad1d-12f3d5316703@j19g2000yqk.googlegroups.com> References: <2f8b08af-65fd-4ff9-b90b-db3f689aca01@u7g2000yqm.googlegroups.com> <42ac849d-fecd-4489-8c9c-4ac4dc939898@m3g2000yqf.googlegroups.com> <4a5ebe07-aac7-40e6-ad1d-12f3d5316703@j19g2000yqk.googlegroups.com> Message-ID: Jeremy wrote: > On Jan 11, 8:44 am, Iain King wrote: >> On Jan 11, 3:35 pm, Jeremy wrote: >> >> >> >> >> >>> Hello all, >>> I am using re.split to separate some text into logical structures. >>> The trouble is that re.split doesn't find everything while re.findall >>> does; i.e.: >>>> found = re.findall('^ 1', line, re.MULTILINE) >>>> len(found) >>> 6439 >>>> tables = re.split('^ 1', line, re.MULTILINE) >>>> len(tables) >>>> 1 >>> Can someone explain why these two commands are giving different >>> results? I thought I should have the same number of matches (or maybe >>> different by 1, but not 6000!) >>> Thanks, >>> Jeremy >> re.split doesn't take re.MULTILINE as a flag: it doesn't take any >> flags. It does take a maxsplit parameter, which you are passing the >> value of re.MULTILINE (which happens to be 8 in my implementation). >> Since your pattern is looking for line starts, and your first line >> presumably has more splits than the maxsplits you are specifying, your >> re.split never finds more than 1. > > Yep. Thanks for pointing that out. I guess I just assumed that > re.split was similar to re.search/match/findall in what it accepted as > function parameters. I guess I'll have to use a \n instead of a ^ for > split. > > Thanks, > Jeremy Remember you can specify flags inside the pattern itself. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From arnodel at googlemail.com Mon Jan 11 14:12:10 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 11 Jan 2010 19:12:10 +0000 Subject: integer and string compare, is that correct? References: <2f718029-1682-47f6-a2c3-6e40b51bd71a@v25g2000yqk.googlegroups.com> Message-ID: Nobody writes: > On Sun, 10 Jan 2010 23:13:55 -0800, Dan Bishop wrote: > >>> If you actually need to perform comparisons across types, you can rely >>> upon the fact that tuple comparisons are non-strict and use e.g.: >>> >>> ? ? ? ? > a = 5 >>> ? ? ? ? > b = '5' >>> ? ? ? ? > (type(a).__name__, a) < (type(b).__name__, b) >>> ? ? ? ? True >>> ? ? ? ? > (type(a).__name__, a) > (type(b).__name__, b) >>> ? ? ? ? False >>> >>> The second elements will only be compared if the first elements are equal >>> (i.e. the values have the same type). >> >> But this method gives you 3.0 < 2 because 'float' < 'int'. Probably >> not what you want. > > If you're comparing instances of entirely arbitrary types, what > you probably want (and the only thing you're going to get) is an > entirely arbitrary ordering. > > The main case where such a comparison makes sense is for algorithms which > require a total ordering (e.g. tree-like structures), and those won't care > if 3<2 so long as the axioms for a total ordering hold. It won't work for several reasons. Offhand I can think of two: 1. lists, tuples: >>> [1] < ['a'] Traceback (most recent call last): File "", line 1, in TypeError: unorderable types: int() < str() 2. Partially ordered or unordered builtin types: >>> 1j < 1+1j Traceback (most recent call last): File "", line 1, in TypeError: no ordering relation is defined for complex numbers -- Arnaud From mfmdevine at gmail.com Mon Jan 11 14:13:01 2010 From: mfmdevine at gmail.com (amadain) Date: Mon, 11 Jan 2010 11:13:01 -0800 (PST) Subject: xml.sax parsing elements with the same name Message-ID: <51ad9cee-6b7b-42c8-8f9e-44c0f8b96bc7@j19g2000yqk.googlegroups.com> I have an event log with 100s of thousands of entries with logs of the form: I am using xml.sax to parse the event log. The trouble with the file above is when I parse for result value I get the last result value (Blocked from above). I want to get the result value triggered (the second in the event). my code is as follows: def startElement(self, name, attrs): if name == 'event': self.eventTime = attrs.get('eventTimestamp',"") self.eventUniqueId = attrs.get('uniqueId', "") if name == 'result': self.resultValue = attrs.get('value',"") return def endElement(self, name): if name=="event": result= eval(self.filter) if result: ... How do I get the result value I require when events have the same names like above? From aioe.org at technicalbloke.com Mon Jan 11 14:16:36 2010 From: aioe.org at technicalbloke.com (r0g) Date: Mon, 11 Jan 2010 19:16:36 +0000 Subject: Fundamental Function Question (beginner) References: <8d7ef0f0-2a0f-4a43-aced-8f4755406249@e27g2000yqd.googlegroups.com> Message-ID: r0g wrote: > Scott wrote: >> When creating a function is there any difference between putting >> everything under the "def" or not? >> >> Here I created a function called CscoPortNum to convert the network >> port number field in a Cisco syslog string from a an ascii name back >> into its numeric form if required. Does it matter at all that I >> created the translation dictionary first and then started the def? >> >> # def CscoPortNum(RulL) >> # Accept a single ACL Rule as a List split into individual words and >> # return Port number. Convert from Cisco syslog Port name if required >> portfpth = "\\progra~1\\syslogd\\ACL_Logs\\Port-Translations.txt" >> # Create dictionary of portnames to portnumbers >> portD = {} >> for prtnmS in open(portfpth): >> prtnmS = prtnmS.rstrip() >> spltprtL = prtnmS.split(" ") >> portD[spltprtL[2]] = [spltprtL[1]] >> def CscoPortNum(RulL): >> if "eq" in RulL: # Is the Port listed? >> if RulL[RulL.index("eq")+1][0].isdigit(): # Is it numeric? >> # if re.search("\d", RulL[RulL.index("eq")+1][0]): # Is it >> numeric? >> portnum = RulL[RulL.index("eq")+1] # If numeric, use >> as is. >> else: >> # If named, look up numeric translation >> portnum = portD[RulL[RulL.index("eq")+1]] >> portnum = str(portnum).strip("[]'") >> else: portnum = "noeq" >> return portnum > > > In this snippet no, you're not calling the function in the preceding > code so there's no problem. You can intersperse functions with the rest > of your code however you like, they just wont be visible to the > preceding code, but it's better to stick them all at the top of your > script. Even better when you have more than a handful is to bundle > functions into separate py files and then import that file e.g. > > ---contents of foobar.py----- > def foo(): > print "foo" > def bar: > print "bar" > > ---contents of your main script----- > import foobar > print foo(),bar() > > Roger. Whoops, that should have been... > ---contents of your main script----- > import foobar > print foobar.foo(),foobar.bar() Roger. > From jlconlin at gmail.com Mon Jan 11 14:19:03 2010 From: jlconlin at gmail.com (Jeremy) Date: Mon, 11 Jan 2010 11:19:03 -0800 (PST) Subject: Different number of matches from re.findall and re.split References: <4a5ebe07-aac7-40e6-ad1d-12f3d5316703@j19g2000yqk.googlegroups.com> Message-ID: On Jan 11, 9:28?am, Duncan Booth wrote: > MRAB wrote: > >> Yep. ?Thanks for pointing that out. ?I guess I just assumed that > >> re.split was similar to re.search/match/findall in what it accepted as > >> function parameters. ?I guess I'll have to use a \n instead of a ^ for > >> split. > > > You could use the .split method of a pattern object instead: > > > ? ? ?tables = re.compile('^ 1', re.MULTILINE).split(line) > > or you might include the flag in the regular expression literal: '(?m)^ 1' Another great solution. This is what I will do. Thanks, Jeremy From jlconlin at gmail.com Mon Jan 11 14:20:34 2010 From: jlconlin at gmail.com (Jeremy) Date: Mon, 11 Jan 2010 11:20:34 -0800 (PST) Subject: What is built-in method sub Message-ID: <477f47ea-7cca-4ba3-b225-e16562bdae5e@21g2000yqj.googlegroups.com> I just profiled one of my Python scripts and discovered that >99% of the time was spent in {built-in method sub} What is this function and is there a way to optimize it? Thanks, Jeremy From john at castleamber.com Mon Jan 11 14:26:30 2010 From: john at castleamber.com (John Bokma) Date: Mon, 11 Jan 2010 13:26:30 -0600 Subject: xml.sax parsing elements with the same name References: <51ad9cee-6b7b-42c8-8f9e-44c0f8b96bc7@j19g2000yqk.googlegroups.com> Message-ID: <87k4vo5tp5.fsf@castleamber.com> amadain writes: > I have an event log with 100s of thousands of entries with logs of the > form: > > uniqueId="1261124569.35725_PFS_1_1340035961"> > > > > > > > > > > > > > > > > I am using xml.sax to parse the event log. The trouble with the file > above is when I parse for result value I get the last result value > (Blocked from above). I want to get the result value triggered (the > second in the event). > > my code is as follows: > > def startElement(self, name, attrs): > if name == 'event': > self.eventTime = attrs.get('eventTimestamp',"") > self.eventUniqueId = attrs.get('uniqueId', "") > if name == 'result': > self.resultValue = attrs.get('value',"") > return > > def endElement(self, name): > if name=="event": > result= eval(self.filter) > if result: > ... > > How do I get the result value I require when events have the same > names like above? You have to keep track if you're inside a filters section, and keep track of the filter elements (first, second, etc.) assuming you want the result value of the first filter. -- John Bokma Read my blog: http://johnbokma.com/ Hire me (Perl/Python): http://castleamber.com/ From phlip2005 at gmail.com Mon Jan 11 14:50:14 2010 From: phlip2005 at gmail.com (Phlip) Date: Mon, 11 Jan 2010 11:50:14 -0800 Subject: Fundamental Function Question (beginner) In-Reply-To: References: <8d7ef0f0-2a0f-4a43-aced-8f4755406249@e27g2000yqd.googlegroups.com> Message-ID: MRAB wrote: > Scott wrote: >> for prtnmS in open(portfpth): >> prtnmS = prtnmS.rstrip() > There's nothing wrong with building dicts or other lookup tables outside > a function in order to avoid re-creating them every time the function is > called. However, please consider writing complete, pronounceable names for variables. This looks like Fortran! From tanix at mongo.net Mon Jan 11 14:50:35 2010 From: tanix at mongo.net (tanix) Date: Mon, 11 Jan 2010 19:50:35 GMT Subject: Python Goldmine has been updated: http://preciseinfo.org/Convert/index_Convert_Python.html References: Message-ID: In article , Terry Reedy wrote: >On 1/8/2010 11:50 AM, tanix wrote: >> Python Goldmine collection contains the extensive collection of articles >> going back several years. It includes thousands of code >> examples and expert discussions on all major topics. >> >> The information is organized by relevant topics, covered >> by the corresponding chapters. >> >> The information was filtered with sophisticated filters and vast >> majority of artilces with little relevance have been filtered out. >> >> If you have any specific requests for some new chapters to be added >> and it is of interest to others, please post your requests on this >> thread. >> >> If anyone feels he has above average level of competence, or can >> reccommend someone who posts on this group, you may request to be >> included in the expert chapters. >> >> The Python Goldmine is at: >> >> http://preciseinfo.org/Convert/index_Convert_Python.html >> >> -- >> Programmer's Goldmine collections: >> >> http://preciseinfo.org >> >> Tens of thousands of code examples and expert discussions on >> C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, >> organized by major topics of language, tools, methods, techniques. > >This site pops up spam windowns. One was blocked, one managed to bypass >the popup blocker. Tnis is not friendly behaviour. I am sorry. But this is a known issue. This is one of counter vendors doing these popups. They were contacted about this but they refuse to deal with it. These popups are totally inappropriate. These counters will be removed with the next major site update. >Some categories have 100s of entries. Better, I think, to use a search >engine such as Google with more specific search terms and a snippet of >context for each result. Well, I can not tell you what is better for you. You have to decide on your own. What I can tell you is this: when you do Google search, you are not going to get the most appropriate results. Because their filtering is orders of magnitude less precise. There is currenly work going on to do internal site search that will further increase precision. But even as it stands right now, you are getting the code examples on related issues with > 90% certainty of articles to be on topic. Beyond that, do as you wish. >tjr -- Programmer's Goldmine collections: http://preciseinfo.org Tens of thousands of code examples and expert discussions on C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, organized by major topics of language, tools, methods, techniques. From tanix at mongo.net Mon Jan 11 14:52:02 2010 From: tanix at mongo.net (tanix) Date: Mon, 11 Jan 2010 19:52:02 GMT Subject: Python Goldmine has been updated: http://preciseinfo.org/Convert/index_Convert_Python.html References: Message-ID: In article , Steve Holden wrote: >Terry Reedy wrote: >> On 1/8/2010 11:50 AM, tanix wrote: >>> Python Goldmine collection contains the extensive collection of articles >>> going back several years. It includes thousands of code >>> examples and expert discussions on all major topics. >>> >>> The information is organized by relevant topics, covered >>> by the corresponding chapters. >>> >>> The information was filtered with sophisticated filters and vast >>> majority of artilces with little relevance have been filtered out. >>> >>> If you have any specific requests for some new chapters to be added >>> and it is of interest to others, please post your requests on this >>> thread. >>> >>> If anyone feels he has above average level of competence, or can >>> reccommend someone who posts on this group, you may request to be >>> included in the expert chapters. >>> >>> The Python Goldmine is at: >>> >>> http://preciseinfo.org/Convert/index_Convert_Python.html >>> >>> -- >>> Programmer's Goldmine collections: >>> >>> http://preciseinfo.org >>> >>> Tens of thousands of code examples and expert discussions on >>> C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, >>> organized by major topics of language, tools, methods, techniques. >> >> This site pops up spam windowns. One was blocked, one managed to bypass >> the popup blocker. Tnis is not friendly behaviour. >> >> Some categories have 100s of entries. Better, I think, to use a search >> engine such as Google with more specific search terms and a snippet of >> context for each result. >> >Because I habitually run the NoScript extension to Firefox the popups >didn't appear, but there didn't seem to be any original content on this >site. The site contains the selected articles from this group, filtered with high precision filters. > Google continues to be your friend. >regards > Steve -- Programmer's Goldmine collections: http://preciseinfo.org Tens of thousands of code examples and expert discussions on C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, organized by major topics of language, tools, methods, techniques. From tanix at mongo.net Mon Jan 11 14:54:22 2010 From: tanix at mongo.net (tanix) Date: Mon, 11 Jan 2010 19:54:22 GMT Subject: Python Goldmine has been updated: http://preciseinfo.org/Convert/index_Convert_Python.html References: <4B4A9446.7020009@holdenweb.com> Message-ID: In article , Steve Holden wrote: >Steve Holden wrote: >[...] >> Because I habitually run the NoScript extension to Firefox the popups >> didn't appear, but there didn't seem to be any original content on this >> site. Google continues to be your friend. >> >And dammit, why didn't I think to strip the links out instead of >creating yet one more link to it? Sorry ... There isn't a single ad on any of these sites beyond those nasty popups displayed by 3rd party counter vendor. Sorry, but this is the way it is right now. The simpliest thing is just to close the popup window without even looking at it because it is pretty much guaranteed to be totally off the wall add no one would be interested in seeing anyway. >regards > Steve -- Programmer's Goldmine collections: http://preciseinfo.org Tens of thousands of code examples and expert discussions on C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, organized by major topics of language, tools, methods, techniques. From pavlovevidence at gmail.com Mon Jan 11 14:54:47 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Mon, 11 Jan 2010 11:54:47 -0800 (PST) Subject: What is built-in method sub References: <477f47ea-7cca-4ba3-b225-e16562bdae5e@21g2000yqj.googlegroups.com> Message-ID: <4bfd1268-8bca-4813-be16-709cbff7bf8c@g31g2000vbr.googlegroups.com> On Jan 11, 11:20?am, Jeremy wrote: > I just profiled one of my Python scripts and discovered that >99% of > the time was spent in > > {built-in method sub} > > What is this function and is there a way to optimize it? I'm guessing this is re.sub (or, more likely, a method sub of an internal object that is called by re.sub). If all your script does is to make a bunch of regexp substitutions, then spending 99% of the time in this function might be reasonable. Optimize your regexps to improve performance. (We can help you if you care to share any.) If my guess is wrong, you'll have to be more specific about what your sctipt does, and maybe share the profile printout or something. Carl Banks From mrabarnett at mrabarnett.plus.com Mon Jan 11 14:56:57 2010 From: mrabarnett at mrabarnett.plus.com (Matthew Barnett) Date: Mon, 11 Jan 2010 19:56:57 +0000 Subject: What is built-in method sub In-Reply-To: <477f47ea-7cca-4ba3-b225-e16562bdae5e@21g2000yqj.googlegroups.com> References: <477f47ea-7cca-4ba3-b225-e16562bdae5e@21g2000yqj.googlegroups.com> Message-ID: <4B4B8289.6000901@mrabarnett.plus.com> Jeremy wrote: > I just profiled one of my Python scripts and discovered that >99% of > the time was spent in > > {built-in method sub} > > What is this function and is there a way to optimize it? > > Thanks, > Jeremy From python at mrabarnett.plus.com Mon Jan 11 14:58:29 2010 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 11 Jan 2010 19:58:29 +0000 Subject: What is built-in method sub In-Reply-To: <477f47ea-7cca-4ba3-b225-e16562bdae5e@21g2000yqj.googlegroups.com> References: <477f47ea-7cca-4ba3-b225-e16562bdae5e@21g2000yqj.googlegroups.com> Message-ID: <4B4B82E5.30807@mrabarnett.plus.com> Jeremy wrote: > I just profiled one of my Python scripts and discovered that >99% of > the time was spent in > > {built-in method sub} > > What is this function and is there a way to optimize it? > I think it's the subtraction operator. The only way to optimise it is to reduce the number of subtractions that you do! From philip at semanchuk.com Mon Jan 11 15:00:44 2010 From: philip at semanchuk.com (Philip Semanchuk) Date: Mon, 11 Jan 2010 15:00:44 -0500 Subject: Fundamental Function Question (beginner) In-Reply-To: References: <8d7ef0f0-2a0f-4a43-aced-8f4755406249@e27g2000yqd.googlegroups.com> Message-ID: <3168D89D-0A24-4732-9A53-5E692A28412D@semanchuk.com> On Jan 11, 2010, at 2:50 PM, Phlip wrote: > MRAB wrote: > >> Scott wrote: > >>> for prtnmS in open(portfpth): >>> prtnmS = prtnmS.rstrip() > >> There's nothing wrong with building dicts or other lookup tables >> outside >> a function in order to avoid re-creating them every time the >> function is >> called. > > However, please consider writing complete, pronounceable names for > variables. This looks like Fortran! Wht doYu mn? I thnk hisCde is ez2rd! From jlconlin at gmail.com Mon Jan 11 15:02:06 2010 From: jlconlin at gmail.com (Jeremy) Date: Mon, 11 Jan 2010 12:02:06 -0800 (PST) Subject: What is built-in method sub References: <477f47ea-7cca-4ba3-b225-e16562bdae5e@21g2000yqj.googlegroups.com> <4bfd1268-8bca-4813-be16-709cbff7bf8c@g31g2000vbr.googlegroups.com> Message-ID: <4777eb9a-0aa4-4e5f-8d61-225852691856@26g2000yqo.googlegroups.com> On Jan 11, 12:54?pm, Carl Banks wrote: > On Jan 11, 11:20?am, Jeremy wrote: > > > I just profiled one of my Python scripts and discovered that >99% of > > the time was spent in > > > {built-in method sub} > > > What is this function and is there a way to optimize it? > > I'm guessing this is re.sub (or, more likely, a method sub of an > internal object that is called by re.sub). > > If all your script does is to make a bunch of regexp substitutions, > then spending 99% of the time in this function might be reasonable. > Optimize your regexps to improve performance. ?(We can help you if you > care to share any.) > > If my guess is wrong, you'll have to be more specific about what your > sctipt does, and maybe share the profile printout or something. > > Carl Banks Your guess is correct. I had forgotten that I was using that function. I am using the re.sub command to remove trailing whitespace from lines in a text file. The commands I use are copied below. If you have any suggestions on how they could be improved, I would love to know. Thanks, Jeremy lines = self._outfile.readlines() self._outfile.close() line = string.join(lines) if self.removeWS: # Remove trailing white space on each line trailingPattern = '(\S*)\ +?\n' line = re.sub(trailingPattern, '\\1\n', line) From invalid at invalid.invalid Mon Jan 11 15:02:37 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Mon, 11 Jan 2010 20:02:37 +0000 (UTC) Subject: pyserial: Unexpected Local Echo References: Message-ID: On 2010-01-11, Steven Woody wrote: > I am using pyserial. But I always get the local echo after I > write some characters onto serial port I really doubt you're getting a local echo. Is the data coming out the serial port? Do you get the echo if you disconnect the serial cable? > and I find no way to disable this behavior. When I say 'local > echo', I mean the next read operation will get characters that > was just write to the same port. The device to which you're connected is echoing them. There's also a chance that your rxd line is floating and there's enough crosstalk in the cable to "echo" the data, but I'll bet money it's not being done locally (in the serial driver or port). > I run my program on cygwin (pyserial was also built on the > system from source code) and the serial port i am using is a > USB adapter that simulates a port (COM4 on my XP) because my > laptop don't have a real serial port. But I checked my COM4 > settings, there is no any think like 'local echo'. -- Grant Edwards grante Yow! Will it improve my at CASH FLOW? visi.com From mfmdevine at gmail.com Mon Jan 11 15:08:13 2010 From: mfmdevine at gmail.com (amadain) Date: Mon, 11 Jan 2010 12:08:13 -0800 (PST) Subject: xml.sax parsing elements with the same name References: <51ad9cee-6b7b-42c8-8f9e-44c0f8b96bc7@j19g2000yqk.googlegroups.com> <87k4vo5tp5.fsf@castleamber.com> Message-ID: <74552e4f-f60f-482f-9830-98c104e938a7@m16g2000yqc.googlegroups.com> On Jan 11, 7:26?pm, John Bokma wrote: > amadain writes: > > I have an event log with 100s of thousands of entries with logs of the > > form: > > > > uniqueId="1261124569.35725_PFS_1_1340035961"> > > ? ? > > ? ? ? > > ? ? ? ? ? > > ? ? ? ? ? ? ? > > ? ? ? ? ? ? ? ? ? ? > > ? ? ? ? ? ? ? > > ? ? ? ? ? > > ? ? ? ? ? > > ? ? ? ? ? ? ? > > ? ? ? ? ? ? ? ? ? ? > > ? ? ? ? ? ? ? > > ? ? ? ? ? > > ? ? ? > > > > > I am using xml.sax to parse the event log. The trouble with the file > > above is when I parse for result value I get the last result value > > (Blocked from above). I want to get the result value triggered (the > > second in the event). > > > my code is as follows: > > > ? ? def startElement(self, name, attrs): > > ? ? ? ? if name == 'event': > > ? ? ? ? ? ? self.eventTime = attrs.get('eventTimestamp',"") > > ? ? ? ? ? ? self.eventUniqueId = attrs.get('uniqueId', "") > > ? ? ? ? if name == 'result': > > ? ? ? ? ? ? self.resultValue = attrs.get('value',"") > > ? ? ? ? return > > > ? ? def endElement(self, name): > > ? ? ? ? if name=="event": > > ? ? ? ? ? ? result= eval(self.filter) > > ? ? ? ? ? ? if result: > > ? ? ? ? ? ?... > > > How do I get the result value I require when events have the same > > names like above? > > You have to keep track if you're inside a filters section, and keep > track of the filter elements (first, second, etc.) assuming you want the > result value of the first filter. > > -- > John Bokma > > Read my blog:http://johnbokma.com/ > Hire me (Perl/Python):http://castleamber.com/ how do I keep track? The first result value is outside a filters section and the rest are. Do you mean something like: def startElement(self, name, attrs): if name == 'event': self.eventTime = attrs.get('eventTimestamp',"") self.eventUniqueId = attrs.get('uniqueId', "") if name == 'result': self.resultValue = attrs.get('value',"") if name == filters: if name == 'result': self.resultValueF = attrs.get('value',"") return A From anthra.norell at bluewin.ch Mon Jan 11 15:09:43 2010 From: anthra.norell at bluewin.ch (Anthra Norell) Date: Mon, 11 Jan 2010 21:09:43 +0100 Subject: lightweight encryption of text file In-Reply-To: References: <4B4851C6.1050009@bluewin.ch> Message-ID: <4B4B8587.9080601@bluewin.ch> Robert Kern wrote: > On 2010-01-09 03:52 AM, Anthra Norell wrote: >> Daniel Fetchinson wrote: >> > I have a plain text file which I would like to protect in a very >> > simple minded, yet for my purposes sufficient, way. I'd like to >> > encrypt/convert it into a binary file in such a way that >> possession of >> > a password allows anyone to convert it back into the original text >> > file while not possessing the password one would only see the >> > following with the standard linux utility 'file': >> > >> > [fetchinson at fetch ~]$ file encrypted.data >> > encrypted.data: data >> > >> > and the effort required to convert the file back to the original text >> > file without the password would be equivalent to guessing the >> > password. >> > >> > I'm fully aware of the security implications of this loose >> > specification, but for my purposes this would be a good solution. >> > >> > What would be the simplest way to achieve this using preferably stock >> > python without 3rd party modules? If a not too complex 3rd part >> > module made it really simple that would be acceptable too. >> >> Daniel, >> >> Here's what looks like another thread veering off into package-ology, >> leaving a stumped OP behind. >> >> "Don't use a random generator for encryption purposes!" warns the >> manual, of which fact I was reminded in no uncertain terms on this forum >> a few years ago when I proposed the following little routine in response >> to a post very similar to yours. One critic challenged me to encode my >> credit card data and post it. Which I did. > > Actually, you just "encrypted" your credit card number and challenged > comp.lang.python to crack it. No one challenged you to do anything of > the sort. Fortunately, the ever-watchful eye of Google was upon us > that day: > > http://groups.google.com/group/comp.lang.python/browse_thread/thread/5fb9ffada975bae9?pli=1 My dear Robert. Thank you for the clarification. You are right. The thread recorded by Google doesn't mention the credit card detail. I remember it distinctly, though. I also remember that it wasn't my idea. And I recall being urged by another, well-mannered, member of this group to call it off right away. He wrote--I pretty much quote: "...there must be a number of machines out there grinding away at your code right now!" >> Upon which another critic >> conjured up the horror vision of gigahertzes hacking my pathetic little >> effort to pieces as I was reading his message. Of the well-meaning kind, >> he urged me to put an immediate stop to this foolishness. I didn't. >> >> No unplanned expenditures ensued. > > That's because comp.lang.python is not full of thieves, not because > your algorithm is worth a damn. You're right about the thieves. You have a point about my algorithm, although you might express it in a fashion that lives up to its merits. My algorithm would not resist a brute-force attack that iterates through all possible keys and analyzes the outcome for non-randomness. I knew that then and so I posted a second-level encryption, that is, an encryption of an encryption. Thus the brute-force attack wouldn't find anything non-random. By not disclosing the detail I may have breached some formal rule of the craft. If I had disclosed it, I have my doubts that the processing power available at a sensible cost would have done the job. The keys I said I used were long integers. There are roughly 4.3 billion of them. Cycling through all of them would surely take a considerable number of hours on a PC. If each one of the 4.3 billion top-level decryptions required another bottom-level run many hours long, the chances of cracking the code before dying of old age are very, very, very remote. I may be wrong about that. If you know better, your knowledge would serve the community a lot better than your ill temper. > > p3.py imposes no more overhead than this, but it has some real > security properties. To quote Paul Rubin from that previous thread: > > """ > Since good encryption schemes that don't have significant performance > penalties are widely available, why mess with a crap scheme EVER? Why > use a solution that "might or might not be adequate" when you can use > one that's definitely ok? > """ > Excellent point! Why EVER make anything yourself when you can buy it? Another excellent point! Cheers Frederic From deets at nospam.web.de Mon Jan 11 15:15:25 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Mon, 11 Jan 2010 21:15:25 +0100 Subject: What is built-in method sub In-Reply-To: <4777eb9a-0aa4-4e5f-8d61-225852691856@26g2000yqo.googlegroups.com> References: <477f47ea-7cca-4ba3-b225-e16562bdae5e@21g2000yqj.googlegroups.com> <4bfd1268-8bca-4813-be16-709cbff7bf8c@g31g2000vbr.googlegroups.com> <4777eb9a-0aa4-4e5f-8d61-225852691856@26g2000yqo.googlegroups.com> Message-ID: <7r1f6tFcrfU1@mid.uni-berlin.de> Jeremy schrieb: > On Jan 11, 12:54 pm, Carl Banks wrote: >> On Jan 11, 11:20 am, Jeremy wrote: >> >>> I just profiled one of my Python scripts and discovered that >99% of >>> the time was spent in >>> {built-in method sub} >>> What is this function and is there a way to optimize it? >> I'm guessing this is re.sub (or, more likely, a method sub of an >> internal object that is called by re.sub). >> >> If all your script does is to make a bunch of regexp substitutions, >> then spending 99% of the time in this function might be reasonable. >> Optimize your regexps to improve performance. (We can help you if you >> care to share any.) >> >> If my guess is wrong, you'll have to be more specific about what your >> sctipt does, and maybe share the profile printout or something. >> >> Carl Banks > > Your guess is correct. I had forgotten that I was using that > function. > > I am using the re.sub command to remove trailing whitespace from lines > in a text file. The commands I use are copied below. If you have any > suggestions on how they could be improved, I would love to know. > > Thanks, > Jeremy > > lines = self._outfile.readlines() > self._outfile.close() > > line = string.join(lines) > > if self.removeWS: > # Remove trailing white space on each line > trailingPattern = '(\S*)\ +?\n' > line = re.sub(trailingPattern, '\\1\n', line) line = line.rstrip()? Diez From suresh.amritapuri at gmail.com Mon Jan 11 15:19:03 2010 From: suresh.amritapuri at gmail.com (suresh.amritapuri) Date: Mon, 11 Jan 2010 12:19:03 -0800 (PST) Subject: PIL how to display multiple images side by side References: <6d9fef57-7fdc-46ec-9b53-7aa824e785e9@s31g2000yqs.googlegroups.com> <4b496473$1@dnews.tpgi.com.au> Message-ID: <34dd6abe-d9e5-4a9f-b396-256c19541b69@b2g2000yqi.googlegroups.com> On Jan 9, 9:51?pm, "Alf P. Steinbach" wrote: > * Lie Ryan: > > > On 1/9/2010 8:43 AM, suresh.amritapuri wrote: > >> Hi, > > >> In PIL, how to display multiple images in say m rows and n colums when > >> I have m*n images. > > >> suresh > > > Tkinter has PhotoImage widget and PIL has support for this widget: > >http://www.pythonware.com/library/pil/handbook/imagetk.htm > > Maybe I've misunderstood something (in that case glad to learn!), but I believe > PhotoImage is not a widget, and that a PhotoImage has to be presented in e.g. a > Label widget or some other widget that's able to display images. > > Cheers, > > - Alf Hi, Let me paste the code I have adapted from somewhere. I dont get a clue on how to go further so that all the images would stay in their respective positions. thanks suresh def button_click_exit_mainloop (event): event.widget.quit() # this will cause mainloop to unblock. root = Tkinter.Tk() root.bind("    
> b) If there's already a PYTHONPATH, Edit it, adding a semi-colon and > the full path of folder Module to the end. > > 5) Put your module folders into the folder Module. > > 6) (Here's a really arcane bit.) Into each module folder, put a file > named __init__.py. It will be executed when you load the module. It > can be empty, but it has to be there or else the module folder will be > ignored. Cheers & hth., - Alf Notes: [1] Windows XP and I believe also Vista places a limit of about 8 KiB on the length of any environment variable. From wuwei23 at gmail.com Sun Jan 17 23:15:27 2010 From: wuwei23 at gmail.com (alex23) Date: Sun, 17 Jan 2010 20:15:27 -0800 (PST) Subject: The answer References: Message-ID: <8fc5ec25-03ba-4347-9521-cd3a73864474@a15g2000yqm.googlegroups.com> On Jan 18, 12:30?pm, Jive Dadson wrote: > These instructions are for MS Windows. > > 1) Create your modules folder. Let's say it's named "Modules." ?The > documentation calls it a "package." > > 2) In an explorer window or on the desktop, right click on My Computer, > and select Properties. > > 3) Select the Advanced tab, and click on Environment Variables near the > bottom. > > 4) Look for an environment variable named PYTHONPATH. > > ? ? a) If you do not find one, create one using the New button(s). I > don't know if it has to be in User Variables or System Variables. ?To > save time experimenting, I just put one in both. For the value, put the > full path of the folder Modules. > > ? ? b) If there's already a PYTHONPATH, ?Edit it, adding a semi-colon > and the full path of folder Module to the end. > > 5) Put your module folders into the folder Module. > > 6) (Here's a really arcane bit.) Into each module folder, put a file > named __init__.py. ?It will be executed when you load the module. ?It > can be empty, but it has to be there or else the module folder will be > ignored. Actually, if you're using Python 2.6+/3.x, you can effectively skip steps 1-5, as these versions now support user site-packages. Rather than create a Module folder and modify your PYTHONPATH, add (if it doesn't exist already) the following folder: %APPDATA%/Python/Python26/site-packages Modules can sit directly in the folder, or within packages. For more details: http://www.python.org/dev/peps/pep-0370/ From wuwei23 at gmail.com Sun Jan 17 23:40:41 2010 From: wuwei23 at gmail.com (alex23) Date: Sun, 17 Jan 2010 20:40:41 -0800 (PST) Subject: IDLE Namespace Toolbox? Windows. References: Message-ID: <8cacd724-4654-4ef8-9407-f501b55b8986@a15g2000yqm.googlegroups.com> On Jan 18, 8:22?am, "W. eWatson" wrote: > I've found it a bit aggravating that using dir and > help, for example, that the output just rolls on off the screen and I > have to play around with the shell scroll bars to find what I'm looking > for. A few simple changes to mydir should change that, but I would hope > or think maybe there are even more tools to generally help. If you haven't already, you really should check out IPython: http://ipython.scipy.org It's an enhanced interactive shell packed full of convenience features. For example, output is by default paged, so you'll get the behaviour you want from help() and dir() straight away. However, you'll probably end up using IPython's help instead: ? will not only display the docstring, it'll provide metadata about the object, such as its base class, the file it was defined in and even the namespace it exists in. %page will pretty print the object and run it through the pager. %timeit is an _exceptionally_ handy wrapper around the timeit module. %bg runs in a separate, background thread There's a directory stack, macros, code in history can be edited, profiling & debugging, functions can be called without parenthesis (nice if you use IPython as a shell replacement), and you can easily capture the results of a command line call to a variable. But yes, along with all that, it pages object printing :) From notontheweb at noisp.com Sun Jan 17 23:49:41 2010 From: notontheweb at noisp.com (Jive Dadson) Date: Sun, 17 Jan 2010 20:49:41 -0800 Subject: The answer In-Reply-To: References: <4OP4n.99891$IU1.53386@en-nntp-04.dc1.easynews.com> Message-ID: alex23 wrote: > > Actually, if you're using Python 2.6+/3.x, you can effectively skip > steps 1-5, as these versions now support user site-packages. > > Rather than create a Module folder and modify your PYTHONPATH, add (if > it doesn't exist already) the following folder: > %APPDATA%/Python/Python26/site-packages > > Modules can sit directly in the folder, or within packages. > > For more details: http://www.python.org/dev/peps/pep-0370/ That requires a directory whose name embeds the Python version number, which is the evil from which I flee, or rather sought to flee. Imagine if all your C++ code had to go into directories that were named for some specific C++ compiler. It's just WRONG. It's a maintenance nightmare to have a bunch of different source files that all have to be updated whenever you fix a bug or add a feature. From wuwei23 at gmail.com Mon Jan 18 00:00:16 2010 From: wuwei23 at gmail.com (alex23) Date: Sun, 17 Jan 2010 21:00:16 -0800 (PST) Subject: The answer References: <4OP4n.99891$IU1.53386@en-nntp-04.dc1.easynews.com> Message-ID: <70363743-37ed-4c0d-b0c5-a70e75ddd7f9@e16g2000yqc.googlegroups.com> Jive Dadson wrote: > That requires a directory whose name embeds the Python version number, > which is the evil from which I flee, or rather sought to flee. ?Imagine > if all your C++ code had to go into directories that were named for some > specific C++ compiler. ?It's just WRONG. ?It's a maintenance nightmare > to have a bunch of different source files that all have to be updated > whenever you fix a bug or add a feature. With the PEP 370 approach, you can just designate a folder the core one and symlink all other versions to it. Or better yet, use version control to push updates to each folder. With all of the versions sharing one folder, as you'd prefer, it would be a lot more difficult to actually achieve version-level granularity if and when you need it. It's not WRONG, it's just providing a level of control that you don't need at this point. From afriere at yahoo.co.uk Mon Jan 18 00:30:07 2010 From: afriere at yahoo.co.uk (Asun Friere) Date: Sun, 17 Jan 2010 21:30:07 -0800 (PST) Subject: enhancing 'list' References: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> Message-ID: <88c73449-222f-4646-8fc3-4307e4bffcb4@u41g2000yqe.googlegroups.com> On Jan 18, 9:37?am, samwyse wrote: > Consider this a wish list. ?I know I'm unlikely to get any of these in > time for for my birthday, but still I felt the need to toss it out and > see what happens. > > Lately, I've slinging around a lot of lists, and there are some simple > things I'd like to do that just aren't there. > If memory serves me correctly, it has been possible to subclass 'built- in' types since Py2.2 or thereabouts. From steven at REMOVE.THIS.cybersource.com.au Mon Jan 18 00:34:59 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 18 Jan 2010 05:34:59 GMT Subject: Inheriting methods but over-riding docstrings References: <0361ea2d$0$1336$c3e8da3@news.astraweb.com> Message-ID: On Mon, 18 Jan 2010 01:45:17 +0100, Alf P. Steinbach wrote: > Steven: on a personal note, earlier when I saw you (I think it was you) > using the "Norwegian Parrot" example I thought it referred to me because > that was the only sense I could make of it, it followed right after some > discussion we had. Thus my impression of you or or responses in this > group was colored by a false interpretation. But, checking, which is > often a good idea!, and *which I should have done then*, as far as I can > see the term was first used in this group in April 2001, http://groups.google.com/group/comp.lang.python/browse_thread/ thread/12a125ceddd401e2/c021547a1dc14a41>. > > It's still a mystery to me what it refers to, though... :-) It refers to the famous Monty Python "Dead Parrot Sketch", involving a Norwegian Blue parrot that definitely isn't dead but merely pining for the fjords. http://www.mtholyoke.edu/~ebarnes/python/dead-parrot.htm http://en.wikipedia.org/wiki/Dead_Parrot No doubt you'll be able to find it on Youtube. A little-known FAQ is that Python is named for Monty Python, not the snake, and that traditionally metasyntactic variables such as foo, bar etc. are frequently named after Monty Python sketches. E.g. I will frequently reference the Cheeseshop sketch, the Spanish Inquisition, Ethel the Aardvark, Spam (the lunch meat, not the email), and similar. E.g. instead of foo, bar, baz, we frequently use spam, ham, eggs. http://www.python.org/doc/faq/general/#why-is-it-called-python -- Steven From notontheweb at noisp.com Mon Jan 18 00:44:44 2010 From: notontheweb at noisp.com (Jive Dadson) Date: Sun, 17 Jan 2010 21:44:44 -0800 Subject: Arrrrgh! Another module broken In-Reply-To: References: Message-ID: Matt Newville wrote: > On Jan 17, 7:25 pm, Jive Dadson wrote: >> I just found another module that broke when I went to 2.6. > Gnuplot. Apparently one of its routines has a parameter >> named "with." That used to be okay, and now it's not. > > This was fixed in version 1.8 of Gnuplot.py > >> Once I get everything to work under 2.6, I am using it >> forever or until new releases no longer break working >> code, whichever comes first. > > Hey, good luck with that forever plan. > > --Matt I have source code that uses the "with" kwarg. What do I replace it with? From clp2 at rebertia.com Mon Jan 18 00:55:36 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 17 Jan 2010 21:55:36 -0800 Subject: Arrrrgh! Another module broken In-Reply-To: References: Message-ID: <50697b2c1001172155s2e9e251ep185143be280d7fb2@mail.gmail.com> On Sun, Jan 17, 2010 at 9:44 PM, Jive Dadson wrote: > Matt Newville wrote: >> >> On Jan 17, 7:25 pm, Jive Dadson wrote: >>> >>> I just found another module that broke when I went to 2.6. > Gnuplot. >>> ?Apparently one of its routines has a parameter >>> named "with." ?That used to be okay, and now it's not. >> >> This was fixed in version 1.8 of Gnuplot.py >> >>> Once I get everything to work under 2.6, I am using it >>> forever or until new releases no longer break working >>> code, whichever comes first. >> >> Hey, good luck with that forever plan. > > I have source code that uses the "with" kwarg. What do I replace it with? Rename the argument to something else that's not a Python keyword. When keyword conflicts occur, appending an underscore (i.e. with_) is a popular option, as is using a nonstandard spelling (e.g. wif) Cheers, Chris -- http://blog.rebertia.com From aioe.org at technicalbloke.com Mon Jan 18 00:57:18 2010 From: aioe.org at technicalbloke.com (r0g) Date: Mon, 18 Jan 2010 05:57:18 +0000 Subject: Arrrrgh! Another module broken References: Message-ID: Jive Dadson wrote: > Matt Newville wrote: >> On Jan 17, 7:25 pm, Jive Dadson wrote: >>> I just found another module that broke when I went to 2.6. > >>> Gnuplot. Apparently one of its routines has a parameter >>> named "with." That used to be okay, and now it's not. >> >> This was fixed in version 1.8 of Gnuplot.py >> >>> Once I get everything to work under 2.6, I am using it >>> forever or until new releases no longer break working >>> code, whichever comes first. >> >> Hey, good luck with that forever plan. >> >> --Matt > > I have source code that uses the "with" kwarg. What do I replace it with? Good old fashioned open() and close() Roger. From steve at holdenweb.com Mon Jan 18 01:09:41 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 18 Jan 2010 01:09:41 -0500 Subject: pyserial: Unexpected Local Echo In-Reply-To: References: <4b50c6bf$0$1676$742ec2ed@news.sonic.net> Message-ID: Steven Woody wrote: > 2010/1/16 John Nagle : >> Grant Edwards wrote: >>> On 2010-01-11, Steven Woody wrote: >>> >>>> I am using pyserial. But I always get the local echo after I >>>> write some characters onto serial port >>> I really doubt you're getting a local echo. Is the data coming >>> out the serial port? Do you get the echo if you disconnect the >>> serial cable? >>> >>>> and I find no way to disable this behavior. When I say 'local >>>> echo', I mean the next read operation will get characters that >>>> was just write to the same port. >>> The device to which you're connected is echoing them. There's >>> also a chance that your rxd line is floating and there's enough >>> crosstalk in the cable to "echo" the data, but I'll bet money >>> it's not being done locally (in the serial driver or port). >>> >>>> I run my program on cygwin (pyserial was also built on the >>>> system from source code) and the serial port i am using is a >>>> USB adapter that simulates a port (COM4 on my XP) because my >>>> laptop don't have a real serial port. But I checked my COM4 >>>> settings, there is no any think like 'local echo'. >> You're using what? Some version of Python built on Cygwin >> running on a Windows XP system? What if you just run a stock >> Python built for Windows on Windows XP? Or run Linux? That >> half-and-half environment may not work right. pyserial has >> special cases in it for Windows and Linux, and it's not >> clear what it will do on Cygwin. >> >> That said, if you're getting echo from output back to input, >> I'd look at the USB to serial device. I've used devices with >> the Silicon Laboratories CP2102 part, and they work fine. >> ("http://www.aetherltd.com/connectingusb.html") >> >> Do you have something plugged into the serial port? If >> so, what? >> >> John Nagle >> -- >> http://mail.python.org/mailman/listinfo/python-list >> > > Now I tried run the same test code using pure Windows python + > pyserial, the result is same. > > Actually, the USB device is a optic head, that read data from an > electric energy meter: > > PC USB Port -> Optic Head -> Meter > > I get the echo even when the meter itself is disconnected, that mean > the echo is not generated by firmware inside the meter. So, if the > echo is also not generated from my PC, that much be from the optic > head itself. Is it possible? > > Regards, > woody > Of course this echo is good, as it allows you to verify that your transmission has been received ... somewhere ... regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From lie.1296 at gmail.com Mon Jan 18 01:14:12 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Mon, 18 Jan 2010 17:14:12 +1100 Subject: The answer In-Reply-To: References: Message-ID: <4b53fc49$1@dnews.tpgi.com.au> On 01/18/10 13:30, Jive Dadson wrote: > Okay, with your help I've figured it out. Instructions are below, but > read the caveat by Ben Fenny in this thread. All this stuff is good for > one default version of Python only. The PYTHONPATH described below, for > example, cannot specify a version number. Yes, that's a pain in the > butt, but there's no way around it. If you switch versions, you may > have to delete all the .pyc files that will show up in the module > folders. Python ought to check them to see if they are valid, but I do > not know if it does so. Err... "The answer" to... what? From dieter at handshake.de Mon Jan 18 01:25:58 2010 From: dieter at handshake.de (Dieter Maurer) Date: 18 Jan 2010 07:25:58 +0100 Subject: setattr() oddness In-Reply-To: <4b517acc$1@dnews.tpgi.com.au> References: <3eba07d0-fc14-4f51-a777-1c47cc866387@j14g2000yqm.googlegroups.com> <409dce36-691c-4c9f-ae69-aba358377b6a@j4g2000yqe.googlegroups.com> <4b517acc$1@dnews.tpgi.com.au> Message-ID: Lie Ryan writes on Sat, 16 Jan 2010 19:37:29 +1100: > On 01/16/10 10:10, Sean DiZazzo wrote: > > Interesting. I can understand the "would take time" argument, but I > > don't see any legitimate use case for an attribute only accessible via > > getattr(). Well, at least not a pythonic use case. > > mostly for people (ab)using attributes instead of dictionary. Here is one use case: A query application. Queries are described by complex query objects. For efficiency reasons, query results should be cached. For this, it is not unnatural to use query objects as cache keys. Then, query objects must not get changed in an uncontrolled way. I use "__setattr__" to control access to the objects. Dieter From steven at REMOVE.THIS.cybersource.com.au Mon Jan 18 01:47:59 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 18 Jan 2010 06:47:59 GMT Subject: setattr() oddness References: <3eba07d0-fc14-4f51-a777-1c47cc866387@j14g2000yqm.googlegroups.com> <409dce36-691c-4c9f-ae69-aba358377b6a@j4g2000yqe.googlegroups.com> <4b517acc$1@dnews.tpgi.com.au> Message-ID: On Mon, 18 Jan 2010 07:25:58 +0100, Dieter Maurer wrote: > Lie Ryan writes on Sat, 16 Jan 2010 19:37:29 +1100: >> On 01/16/10 10:10, Sean DiZazzo wrote: >> > Interesting. I can understand the "would take time" argument, but I >> > don't see any legitimate use case for an attribute only accessible >> > via getattr(). Well, at least not a pythonic use case. >> >> mostly for people (ab)using attributes instead of dictionary. > > Here is one use case: > > A query application. Queries are described by complex query objects. > For efficiency reasons, query results should be cached. For this, it is > not unnatural to use query objects as cache keys. Then, query objects > must not get changed in an uncontrolled way. I use "__setattr__" to > control access to the objects. (1) Wouldn't it be more natural to store these query keys in a list or dictionary rather than as attributes on an object? e.g. instead of: cache.__setattr__('complex query object', value) use: cache['complex query object'] = value (2) How does __setattr__ let you control access to the object? If a user wants to modify the cache, and they know the complex query object, what's stopping them from using __setattr__ too? -- Steven From nagle at animats.com Mon Jan 18 01:56:47 2010 From: nagle at animats.com (John Nagle) Date: Sun, 17 Jan 2010 22:56:47 -0800 Subject: Problems with collision response In-Reply-To: References: Message-ID: <4b5402b9$0$1589$742ec2ed@news.sonic.net> Joabos wrote: > I'm doing a project, and I need to insert collision detection and response on > it. Here's the code. What am I doing wrong? Where to begin... 1. Most of your code will never be executed because it is inside multiline triple-quoted strings. 2. Your approach to collision response probably won't handle multiple simultaneous collisions. 3. You're expecting others to debug your code. John Nagle From notontheweb at noisp.com Mon Jan 18 02:08:02 2010 From: notontheweb at noisp.com (Jive Dadson) Date: Sun, 17 Jan 2010 23:08:02 -0800 Subject: More version woes Message-ID: This has to do with Komodo. I cannot use Python 2.4, because numpy is broken on my machine for that release for reasons unknown. I want to use 2.6 anyway. But when I use Python 2.6 and Komodo 3.5, it runs slow as death. I think it might have something to do with the warning I'm getting. Does anyone know what this is about, and what to do about it? Or can anyone tell me why Komodo 3.5 is slow horribly slow with Python 2.6? C:\Program Files\ActiveState Komodo 3.5\lib\support\dbgp\pythonlib\dbgp\client.py:813: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 self.message = message C:\Program Files\ActiveState Komodo 3.5\lib\support\dbgp\pythonlib\dbgp\client.py:815: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 return self._cmd_error % (self.name,self.tid,self.errid,self.message) The offending bit of source... class CommandError(Exception): """Breakpoint class a simple exception that knows how to serialize itself to xml. """ _cmd_error = '' def __init__(self, name, tid, errid, message): self.name = name self.tid = tid self.errid = errid self.message = message # THIS IS WHAT IT DOESN'T LIKE def __str__(self): return self._cmd_error % (self.name,self.tid,self.errid,self.message) From notontheweb at noisp.com Mon Jan 18 02:11:38 2010 From: notontheweb at noisp.com (Jive Dadson) Date: Sun, 17 Jan 2010 23:11:38 -0800 Subject: Arrrrgh! Another module broken In-Reply-To: References: Message-ID: Jive Dadson wrote: > Matt Newville wrote: >> On Jan 17, 7:25 pm, Jive Dadson wrote: >>> I just found another module that broke when I went to 2.6. > >>> Gnuplot. Apparently one of its routines has a parameter >>> named "with." That used to be okay, and now it's not. >> >> This was fixed in version 1.8 of Gnuplot.py >> >>> Once I get everything to work under 2.6, I am using it >>> forever or until new releases no longer break working >>> code, whichever comes first. >> >> Hey, good luck with that forever plan. >> >> --Matt > > I have source code that uses the "with" kwarg. What do I replace it with? I found the answer. The new kwarg in Gnuplot is "with_". From notontheweb at noisp.com Mon Jan 18 02:15:28 2010 From: notontheweb at noisp.com (Jive Dadson) Date: Sun, 17 Jan 2010 23:15:28 -0800 Subject: More version woes In-Reply-To: References: Message-ID: Sorry. That deprecation warning has nothing to do with the slowness. It does torque my jaw, however. Komodo costs money, and Python 2.6 broke it. @#^&!!! (Again.) So, the new question is, does anyone know how to make Komodo 3.5 run at speed with Python 2.6? Or perhaps better yet, can someone suggest a good IDE that doesn't cost much? From tjreedy at udel.edu Mon Jan 18 02:56:16 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 18 Jan 2010 02:56:16 -0500 Subject: enhancing 'list' In-Reply-To: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> References: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> Message-ID: On 1/17/2010 5:37 PM, samwyse wrote: > Consider this a wish list. I know I'm unlikely to get any of these in > time for for my birthday, but still I felt the need to toss it out and > see what happens. > > Lately, I've slinging around a lot of lists, and there are some simple > things I'd like to do that just aren't there. > > s.count(x[, cmp[, key]]) > - return number of i?s for which s[i] == x. 'cmp' specifies a custom > comparison function of two arguments, as in '.sort'. 'key' specifies > a custom key extraction function of one argument. > s.index(x[, i[, j[, cmp[, key]]]]) > - return smallest k such that s[k] == x and i<= k< j. 'cmp' and > 'key' are as above. > s.rindex(x[, i[, j[, cmp[, key]]]]) > - return largest k such that s[k] == x and i<= k< j. 'cmp' and > 'key' are as above. > > There are two overlapping proposals here. One is to add the .rindex > method, which strings already have. The other is to extend the > optional arguments of .sort to all other methods that test for item > equality. > > One last thing, the Python 2.6.2 spec says .count and .index only > apply to mutable sequence types. I see no reason why they > (and .rindex) couldn't also apply to immutable sequences (tuples, in > particular). In 3.x, tuple does have those methods, even though the doc is not clear (unless fixed by now). From steven at REMOVE.THIS.cybersource.com.au Mon Jan 18 03:02:36 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 18 Jan 2010 08:02:36 GMT Subject: More version woes References: Message-ID: On Sun, 17 Jan 2010 23:15:28 -0800, Jive Dadson wrote: > Sorry. That deprecation warning has nothing to do with the slowness. It > does torque my jaw, however. Komodo costs money, and Python 2.6 broke > it. @#^&!!! (Again.) > > So, the new question is, does anyone know how to make Komodo 3.5 run at > speed with Python 2.6? Or perhaps better yet, can someone suggest a > good IDE that doesn't cost much? Perhaps you should be asking the Komodo forums? Or since you paid for it, perhaps you should be asking customer support? I suspect that they'll probably tell you that since the latest version of Komodo is 5.2, and you're using 3.5, you should stop using a version that is over four years old and almost certainly not supported. -- Steven From notontheweb at noisp.com Mon Jan 18 03:08:35 2010 From: notontheweb at noisp.com (Jive Dadson) Date: Mon, 18 Jan 2010 00:08:35 -0800 Subject: More version woes In-Reply-To: References: Message-ID: <7GU4n.73060$oC1.63466@en-nntp-01.dc1.easynews.com> Steven D'Aprano wrote: > On Sun, 17 Jan 2010 23:15:28 -0800, Jive Dadson wrote: > >> Sorry. That deprecation warning has nothing to do with the slowness. It >> does torque my jaw, however. Komodo costs money, and Python 2.6 broke >> it. @#^&!!! (Again.) >> >> So, the new question is, does anyone know how to make Komodo 3.5 run at >> speed with Python 2.6? Or perhaps better yet, can someone suggest a >> good IDE that doesn't cost much? > > Perhaps you should be asking the Komodo forums? > > Or since you paid for it, perhaps you should be asking customer support? > > I suspect that they'll probably tell you that since the latest version of > Komodo is 5.2, and you're using 3.5, you should stop using a version that > is over four years old and almost certainly not supported. > > Yep. They want $295, and I cannot justify that for personal use. I'll be looking for a cheaper one. From steven at REMOVE.THIS.cybersource.com.au Mon Jan 18 03:59:23 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 18 Jan 2010 08:59:23 GMT Subject: More version woes References: <7GU4n.73060$oC1.63466@en-nntp-01.dc1.easynews.com> Message-ID: On Mon, 18 Jan 2010 00:08:35 -0800, Jive Dadson wrote: >> I suspect that they'll probably tell you that since the latest version >> of Komodo is 5.2, and you're using 3.5, you should stop using a version >> that is over four years old and almost certainly not supported. >> >> >> > Yep. They want $295, and I cannot justify that for personal use. I'll > be looking for a cheaper one. This is why I use a text editor and the interactive console :) -- Steven From __peter__ at web.de Mon Jan 18 04:06:42 2010 From: __peter__ at web.de (Peter Otten) Date: Mon, 18 Jan 2010 10:06:42 +0100 Subject: enhancing 'list' References: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> Message-ID: samwyse wrote: > Lately, I've slinging around a lot of lists, and there are some simple > things I'd like to do that just aren't there. > > s.count(x[, cmp[, key]]) > - return number of i?s for which s[i] == x. 'cmp' specifies a custom > comparison function of two arguments, as in '.sort'. 'key' specifies > a custom key extraction function of one argument. What's your use case exactly? If I were to enhance count/index/rindex I would go for the simpler >>> missing = object() >>> class List(list): ... def count(self, value=missing, predicate=missing): ... if value is missing: ... if predicate is missing: ... raise TypeError ... return sum(1 for item in self if predicate(item)) ... else: ... if predicate is not missing: ... raise TypeError ... return list.count(self, value) ... >>> items = List(range(10)) >>> items.count(7) 1 >>> items.count(predicate=lambda item: item%3) 6 which nicely covers all applications I can imagine. Peter From deets at nospam.web.de Mon Jan 18 04:09:27 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Mon, 18 Jan 2010 10:09:27 +0100 Subject: More version woes In-Reply-To: <7GU4n.73060$oC1.63466@en-nntp-01.dc1.easynews.com> References: <7GU4n.73060$oC1.63466@en-nntp-01.dc1.easynews.com> Message-ID: <7rimq8Fcp7U1@mid.uni-berlin.de> > Yep. They want $295, and I cannot justify that for personal use. I'll be > looking for a cheaper one. There are bazillion discussions in this NG about IDEs and editors. GIYF. Diez From bettini at dsi.unifi.it Mon Jan 18 04:13:04 2010 From: bettini at dsi.unifi.it (Lorenzo Bettini) Date: Mon, 18 Jan 2010 10:13:04 +0100 Subject: LAST CALL FOR PAPERS: TOOLS EUROPE 2010 Message-ID: <4b542623$2@news.x-privat.org> ========================================================================== LAST CALL FOR PAPERS (Deadline: January 22, 2010) TOOLS EUROPE 2010 48th International Conference Objects, Models, Components, Patterns Co-located with *** International Conference on Model Transformation (ICMT 2010) *** *** International Conference on Software Composition (SC 2010) *** *** International Conference on Tests and Proofs (TAP 2010) *** M?laga - Spain, 28 June - 02 July 2010 http://malaga2010.lcc.uma.es/ ========================================================================== TOOLS EUROPE is devoted to the combination of technologies that have emerged as a result of object technology becoming "mainstream". Like its predecessors, TOOLS EUROPE combines an emphasis on quality with a strong practical focus. Started in 1989, TOOLS conferences, held in Europe, the USA, Australia, China and Eastern Europe, have played a major role in the development of object technology; many of seminal concepts were first presented at TOOLS. After an interruption of four years, the conference was revived in 2007 to reflect the maturing of the field and the new challenges ahead and has become a yearly event. Contributions are solicited on all aspects of object technology and related fields, in particular model-based development, component-based development, and patterns (design, analysis and other applications); more generally, any contribution addressing topics in advanced software technology fall within the scope of TOOLS. Reflecting the practical emphasis of TOOLS, contributions showcasing applications along with a sound conceptual contribution are particularly welcome. Topics include: * Object technology, including programming techniques, languages, tools * Testing of object-oriented systems * Patterns, pattern languages, tool support for patterns * Distributed and concurrent object systems * Real-time object-oriented programming and design * Experience reports, including efforts at standardisation * Applications to safety- and security-related software * Component-based programming, modelling, tools * Aspects and aspect-oriented programming and modelling * Frameworks for component-based development * Trusted and reliable components * Model-driven development and Model-Driven Architecture * Domain specific languages and language design * Tools and frameworks for supporting model-driven development * Language implementation techniques, compilers, run-time systems * Practical applications of program verification and analysis * Open source solutions & Reproduction studies All contributions will be subject to a rigorous selection process by the international Program Committee, with a stress on originality, practicality and overall quality. The proceedings will be published in Springer LNBIP. For detailed submission information see the conference page. Important Dates: Papers submission deadline: January 22, 2010 Acceptance notification: March 24, 2010 Camera-ready final copy: April 5, 2010 Conference: June 28 -- July 02, 2010 Conference Chair: Bertrand Meyer, ETH Z?rich and Eiffel Software Program Chair: Jan Vitek, Purdue University Publicity Chair: Osmar Santos, University of York Program Committee: Uwe Assman, University of Dresden, Germany Elisa Baniassad, Chinese University of Hong Kong, Hong Kong Alexandre Bergel, University of Chile, Chile Lorenzo Bettini, University of Torino, Italy Judith Bishop, Microsoft Research, USA William Cook, University of Texas Austin, USA Sophia Drossopolou, Imperial College London, UK Catherine Dubois, ENSIIE, France St?phane Ducasse, INRIA Lille, France Manuel Fahndrich, Microsoft Research, USA Harald Gall, University of Zurich, Switzerland Benoit Garbinato, University of Lausanne, Switzerland Angelo Gargantini, University of Bergamo, Italy Jeff Gray, University of Alabama Birmingham, USA Kathryn Gray, University of Cambridge, UK Thomas Gschwind, IBM Research, Switzerland Matthias Hauswith, University of Lugano, Switzerland Nigel Horspool, University of Victoria, Canada Tomas Kalibera, Charles University, Czech Republic Gerti Kappel, Vienna University of Technology, Austria Doug Lea, State University of New York Oswego, USA Shane Markstrum, Brucknell University, USA Peter M?ller, ETH Zurich, Switzerland Oscar Nierstrasz, University of Bern, Switzerland James Noble, Victoria University of Wellington, New Zealand Nate Nystrom, University of Texas Arlington, USA Manuel Oriol, University of York, UK Jonathan Ostroff, York University, Canada Richard Paige, University of York, UK Shaz Qadeer, Microsoft Research, USA Awais Rashid, Lancaster University, UK Vivek Sarkar, Rice University, USA Doug Schmidt, Vanderbilt University, USA Manuel Serrano, INRIA Sophia Antipolis, France Peter Thiemann, University of Freiburg, Germany Dave Thomas, Bedarra Research Labs, Canada Laurence Tratt, Bournemouth University, UK Mandana Vaziri, IBM Research, USA Tian Zhao, University of Wisconsin-Milwaukee, USA From gabriele.modena at gmail.com Mon Jan 18 04:33:44 2010 From: gabriele.modena at gmail.com (Gabriele Modena) Date: Mon, 18 Jan 2010 10:33:44 +0100 Subject: ctypes: nested structures and pointers Message-ID: <1fe1d5d61001180133gb27a01dsb7434087909aff0@mail.gmail.com> Hi all, I am trying to learn ctypes and I am facing some problems In wrapping two nested structs. <--- begin C code ----> struct dev_callbacks; // Prototype the callback struct typedef struct { const struct dev_callbacks* pdc; char acName[DEVICE_NAME_LENGTH]; chip_type ct; dev_spec ds; bool bActive; bool bCrc; bool bPar; uint8_t ui8TxBits; } dev_info; struct dev_callbacks { const char* acDriver; dev_info* (*connect)(const uint32_t uiIndex); bool (*transceive)(const dev_spec ds, const byte_t* pbtTx, const uint32_t uiTxLen, byte_t* pbtRx, uint32_t* puiRxLen); void (*disconnect)(dev_info* pdi); }; <---- End C code ----> I have two problems with this code: 1. what is the correct (pythonic) way to capture the prototype definition of dev_callbacks and the relation between that structure and dev_info? 2. is it correct to wrap "connect", "transceive" and "disconnect" in separate structures and reference them within pyDEV_CALLBACKS? The (stub) code I wrote for now looks like this: class pyDEV_CALLBACKS(Structure): _fields_ = [("acDriver", c_char_p), ("connect", c_void_p), ("transceive", c_bool), ("disconnect", c_void_p) ] class pyDEV_INFO(Structure): _fields_ = [ ("pdc", POINTER(pyDEV_CALLBACKS)), ("ct", c_ubyte), ("ds", c_void_p), ("acName", c_char * 256), ("bActive", c_bool), ("bCrc", c_bool), ("bPar", c_bool), ("ui8TxBits", c_ubyte) ] Passing the data structures to wrapper functions seem to work (the data is initialized), but the result is not what expected (I presume the problems are related to memory alignment due to wrong declarations), -- Gabriele From anandvaidya.ml at gmail.com Mon Jan 18 04:58:42 2010 From: anandvaidya.ml at gmail.com (Anand Vaidya) Date: Mon, 18 Jan 2010 01:58:42 -0800 (PST) Subject: Generic Python Benchmark suite? Message-ID: Is there a generic python benchmark suite in active development? I am looking forward to comparing some code on various python implementations (primarily CPython 2.x, CPython 3.x, UnladenSwallow, Psyco). I am happy with something that gives me a relative number eg: ULS is 30% faster than CPy2.x etc I found pybench which is probably not maintained actively. What do you suggest? PS: I think a benchmark should cover file / network, database I/O, data structures (dict, list etc), object creation/manipulation, numbers, measure looping inefficiencies, effects of caching (memcache etc) at the minimum From utente at esempio.net Mon Jan 18 05:15:37 2010 From: utente at esempio.net (superpollo) Date: Mon, 18 Jan 2010 11:15:37 +0100 Subject: substitution Message-ID: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> hi. what is the most pythonic way to substitute substrings? eg: i want to apply: foo --> bar baz --> quux quuux --> foo so that: fooxxxbazyyyquuux --> barxxxquuxyyyfoo bye From utente at esempio.net Mon Jan 18 05:21:54 2010 From: utente at esempio.net (superpollo) Date: Mon, 18 Jan 2010 11:21:54 +0100 Subject: substitution In-Reply-To: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> Message-ID: <4b543642$0$1109$4fafbaef@reader4.news.tin.it> superpollo ha scritto: > hi. > > what is the most pythonic way to substitute substrings? > > eg: i want to apply: > > foo --> bar > baz --> quux > quuux --> foo > > so that: > > fooxxxbazyyyquuux --> barxxxquuxyyyfoo > > bye i explain better: say the subs are: quuux --> foo foo --> bar baz --> quux then i cannot apply the subs in sequence (say, .replace() in a loop), otherwise: fooxxxbazyyyquuux --> fooxxxbazyyyfoo --> barxxxbazyyybar --> barxxxquuxyyybar not as intended... From stefan_ml at behnel.de Mon Jan 18 05:30:16 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 18 Jan 2010 11:30:16 +0100 Subject: Generic Python Benchmark suite? In-Reply-To: References: Message-ID: <4b543838$0$7618$9b4e6d93@newsspool1.arcor-online.net> Anand Vaidya, 18.01.2010 10:58: > Is there a generic python benchmark suite in active development? > [...] > PS: I think a benchmark should cover file / network, database I/O, > data structures (dict, list etc), object creation/manipulation, > numbers, measure looping inefficiencies, effects of caching (memcache > etc) at the minimum That doesn't sound generic at all. Maybe you should prefer an application based benchmark instead. Stefan From __peter__ at web.de Mon Jan 18 05:45:37 2010 From: __peter__ at web.de (Peter Otten) Date: Mon, 18 Jan 2010 11:45:37 +0100 Subject: substitution References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> Message-ID: superpollo wrote: > superpollo ha scritto: >> hi. >> >> what is the most pythonic way to substitute substrings? >> >> eg: i want to apply: >> >> foo --> bar >> baz --> quux >> quuux --> foo >> >> so that: >> >> fooxxxbazyyyquuux --> barxxxquuxyyyfoo >> >> bye > > i explain better: > > say the subs are: > > quuux --> foo > foo --> bar > baz --> quux > > then i cannot apply the subs in sequence (say, .replace() in a loop), > otherwise: > > fooxxxbazyyyquuux --> fooxxxbazyyyfoo --> barxxxbazyyybar --> > barxxxquuxyyybar > > not as intended... If you want to avoid regular expressions: def replace_many(s, pairs): if len(pairs): a, b = pairs[0] rest = pairs[1:] return b.join(replace_many(t, rest) for t in s.split(a)) else: return s assert replace_many("abc", ["ab", "bc", "ca"]) == "bca" assert (replace_many("fooxxxbazyyyquuux", [("quuux", "foo"), ("foo", "bar"), ("baz", "quux")]) == "barxxxquuxyyyfoo") Not tested. Peter From gklein at xs4all.nl Mon Jan 18 05:52:36 2010 From: gklein at xs4all.nl (Gertjan Klein) Date: Mon, 18 Jan 2010 11:52:36 +0100 Subject: maintain 2 versions of python on my computer References: <88886fe9-31ee-4ecc-9109-b645ceb46260@c34g2000yqn.googlegroups.com> <7a1f55ee-4c12-4f56-989a-abcd1c441365@j4g2000yqe.googlegroups.com> Message-ID: <16f8l51ui0eiq7p7lldoha772oslo5tqs6@4ax.com> Duncan Booth wrote: > @(C:\Python26\Python -x %~f0 %* || pause) && goto:EOF > import sys > print sys.version > # raise RuntimeError # uncomment to trigger the 'pause' That is nice! This should probably be in the documentation, here: http://docs.python.org/using/cmdline.html#cmdoption-x ... instead of "This is intended for a DOS specific hack only.". Gertjan. From phlip2005 at gmail.com Mon Jan 18 06:03:26 2010 From: phlip2005 at gmail.com (Phlip) Date: Mon, 18 Jan 2010 03:03:26 -0800 (PST) Subject: Is python not good enough? References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> Message-ID: <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> On Jan 12, 7:09?am, ikuta liu wrote: > Go language try to merge low level, hight level and browser language. Go uses := for assignment. This means, to appease the self-righteous indignation of the math professor who would claim = should mean "equality"... ...you gotta type a shift and 2 characters for a very common operator. Pass! -- Phlip From davea at ieee.org Mon Jan 18 06:03:52 2010 From: davea at ieee.org (Dave Angel) Date: Mon, 18 Jan 2010 06:03:52 -0500 Subject: The answer In-Reply-To: References: <4OP4n.99891$IU1.53386@en-nntp-04.dc1.easynews.com> Message-ID: <4B544018.7090508@ieee.org> Jive Dadson wrote: >
alex23 > wrote: > > > > Actually, if you're using Python 2.6+/3.x, you can effectively skip > > steps 1-5, as these versions now support user site-packages. > > > > Rather than create a Module folder and modify your PYTHONPATH, add (if > > it doesn't exist already) the following folder: > > %APPDATA%/Python/Python26/site-packages > > > > Modules can sit directly in the folder, or within packages. > > > > For more details: http://www.python.org/dev/peps/pep-0370/ > > That requires a directory whose name embeds the Python version number, > which is the evil from which I flee, or rather sought to flee. > Imagine if all your C++ code had to go into directories that were > named for some specific C++ compiler. It's just WRONG. It's a > maintenance nightmare to have a bunch of different source files that > all have to be updated whenever you fix a bug or add a feature. > > As others have pointed out, you need a "deploy" script, which in your case would copy the files from source control to the appropriate production folder. That's analogous to the compile, link and deploy steps of C++. And if you want to be even more analogous, copy just the .pyc files, after building them. Certainly you have lots more files in your version control system which are not intended to be copied to the "Modules" folder, such as the test suite. DaveA From steve at REMOVE-THIS-cybersource.com.au Mon Jan 18 06:10:39 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 18 Jan 2010 11:10:39 GMT Subject: substitution References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> Message-ID: <00ddc6b1$0$15570$c3e8da3@news.astraweb.com> On Mon, 18 Jan 2010 11:15:37 +0100, superpollo wrote: > hi. > > what is the most pythonic way to substitute substrings? > > eg: i want to apply: > > foo --> bar > baz --> quux > quuux --> foo > > so that: > > fooxxxbazyyyquuux --> barxxxquuxyyyfoo For simple cases, just use replace: >>> s = 'fooxxxbazyyyquuux' >>> s = s.replace('foo', 'bar') >>> s = s.replace('baz', 'quux') >>> s = s.replace('quuux', 'foo') >>> s == 'barxxxquuxyyyfoo' True In complicated cases, such as if there are conflicts or overlaps between strings, you may need to use a regular expression, or even parse the string yourself. The problem is that "substitute multiple strings" is not well defined, because in general it depends on the order you perform them. You can define a function to do the replacements in one order, but for another use you might need a different order. E.g.: replacing "a" -> "X" and "aa" -> "Y", if you have the string "aaa" what result do you expect? You could get "XXX", "XY" or "YX". None of these are wrong, it depends on which you prefer. -- Steven From python.list at tim.thechases.com Mon Jan 18 06:19:48 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Mon, 18 Jan 2010 05:19:48 -0600 Subject: Is python not good enough? In-Reply-To: <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> Message-ID: <4B5443D4.4020408@tim.thechases.com> Phlip wrote: > On Jan 12, 7:09 am, ikuta liu wrote: >> Go language try to merge low level, hight level and browser language. > > Go uses := for assignment. > > This means, to appease the self-righteous indignation of the math > professor who would claim = should mean "equality"... > > ...you gotta type a shift and 2 characters for a very common operator. > > Pass! Pass?! no...Pascal! :-) -tkc From samwyse at gmail.com Mon Jan 18 06:22:59 2010 From: samwyse at gmail.com (samwyse) Date: Mon, 18 Jan 2010 03:22:59 -0800 (PST) Subject: enhancing 'list' References: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> Message-ID: <375ab6da-58c2-4975-bc79-22b2219a3216@r24g2000yqd.googlegroups.com> On Jan 18, 1:56?am, Terry Reedy wrote: > On 1/17/2010 5:37 PM, samwyse wrote: > > > > > > > Consider this a wish list. ?I know I'm unlikely to get any of these in > > time for for my birthday, but still I felt the need to toss it out and > > see what happens. > > > Lately, I've slinging around a lot of lists, and there are some simple > > things I'd like to do that just aren't there. > > > s.count(x[, cmp[, key]]) > > - return number of i?s for which s[i] == x. ?'cmp' specifies a custom > > comparison function of two arguments, as in '.sort'. ?'key' specifies > > a custom key extraction function of one argument. > > s.index(x[, i[, j[, cmp[, key]]]]) > > - return smallest k such that s[k] == x and i<= k< ?j. ?'cmp' and > > 'key' are as above. > > s.rindex(x[, i[, j[, cmp[, key]]]]) > > - return largest k such that s[k] == x and i<= k< ?j. ?'cmp' and > > 'key' are as above. > > > There are two overlapping proposals here. ?One is to add the .rindex > > method, which strings already have. ?The other is to extend the > > optional arguments of .sort to all other methods that test for item > > equality. > > > One last thing, the Python 2.6.2 spec says .count and .index only > > apply to mutable sequence types. ?I see no reason why they > > (and .rindex) couldn't also apply to immutable sequences (tuples, in > > particular). > > In 3.x, tuple does have those methods, even though the doc is not clear > (unless fixed by now). That's good to hear. Perhaps I should have tried them directyly, but my 3.1 docs still echo the 2.x docs, which only show them for immutable sequences. From samwyse at gmail.com Mon Jan 18 06:59:07 2010 From: samwyse at gmail.com (samwyse) Date: Mon, 18 Jan 2010 03:59:07 -0800 (PST) Subject: enhancing 'list' References: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> Message-ID: On Jan 18, 3:06?am, Peter Otten <__pete... at web.de> wrote: > samwyse wrote: > > Lately, I've slinging around a lot of lists, and there are some simple > > things I'd like to do that just aren't there. > > > s.count(x[, cmp[, key]]) > > - return number of i?s for which s[i] == x. ?'cmp' specifies a custom > > comparison function of two arguments, as in '.sort'. ?'key' specifies > > a custom key extraction function of one argument. > > What's your use case exactly? If I were to enhance count/index/rindex I > would go for the simpler > > >>> missing = object() ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? > >>> class List(list): ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? > > ... ? ? def count(self, value=missing, predicate=missing): ? ? ? ? ? ? > ... ? ? ? ? ? ? if value is missing: > ... ? ? ? ? ? ? ? ? ? ? if predicate is missing: > ... ? ? ? ? ? ? ? ? ? ? ? ? ? ? raise TypeError > ... ? ? ? ? ? ? ? ? ? ? return sum(1 for item in self if predicate(item)) > ... ? ? ? ? ? ? else: > ... ? ? ? ? ? ? ? ? ? ? if predicate is not missing: > ... ? ? ? ? ? ? ? ? ? ? ? ? ? ? raise TypeError > ... ? ? ? ? ? ? ? ? ? ? return list.count(self, value) > ...>>> items = List(range(10)) > >>> items.count(7) > 1 > >>> items.count(predicate=lambda item: item%3) > > 6 > > which nicely covers all applications I can imagine. > > Peter That is a good idea. However, I was looking more at the simplicity of building of ideas that are already present in .sort. And this implementation is pretty simple as well. >>> class List(list): import __builtin__ def count(self, value, cmp=__builtin__.cmp): return sum(1 for item in self if not cmp(item, value)) >>> items = List(range(10)) >>> items.count(7) 1 >>> items.count(3, lambda a, b: not a%b) # My way 6 >>> items.count(Ellipsis, lambda a, b: not a%3) # Your way 6 As a side note, wouldn't it be nice if '...' could be used in more places than just slices? IMHO, a useful idiom would be to use it to signify "irrelevant" or "don't care", as opposed to 'None' which (in my mind, at least) signifies "missing" or "unknown". From samwyse at gmail.com Mon Jan 18 06:59:53 2010 From: samwyse at gmail.com (samwyse) Date: Mon, 18 Jan 2010 03:59:53 -0800 (PST) Subject: enhancing 'list' References: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> <88c73449-222f-4646-8fc3-4307e4bffcb4@u41g2000yqe.googlegroups.com> Message-ID: On Jan 17, 11:30?pm, Asun Friere wrote: > On Jan 18, 9:37?am, samwyse wrote: > > > Consider this a wish list. ?I know I'm unlikely to get any of these in > > time for for my birthday, but still I felt the need to toss it out and > > see what happens. > > > Lately, I've slinging around a lot of lists, and there are some simple > > things I'd like to do that just aren't there. > > If memory serves me correctly, it has been possible to subclass 'built- > in' types since Py2.2 or thereabouts. True, but I've had bad experiences doing that. See, for example, http://groups.google.com/group/comp.lang.python/browse_thread/thread/10cfe2affc265ac where I tried to subclass 'int'. More importantly, subclassing means that people have to keep re-inventing the same methods. Having a single implementation would save time, not to mention the speed advantages of implementing them in the hosting language (C, Java, .NET, etc). From __peter__ at web.de Mon Jan 18 07:20:19 2010 From: __peter__ at web.de (Peter Otten) Date: Mon, 18 Jan 2010 13:20:19 +0100 Subject: enhancing 'list' References: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> Message-ID: samwyse wrote: > On Jan 18, 3:06 am, Peter Otten <__pete... at web.de> wrote: >> samwyse wrote: >> > Lately, I've slinging around a lot of lists, and there are some simple >> > things I'd like to do that just aren't there. >> >> > s.count(x[, cmp[, key]]) >> > - return number of i?s for which s[i] == x. 'cmp' specifies a custom >> > comparison function of two arguments, as in '.sort'. 'key' specifies >> > a custom key extraction function of one argument. >> >> What's your use case exactly? If I were to enhance count/index/rindex I >> would go for the simpler >> >> >>> missing = object() >> >>> class List(list): >> >> ... def count(self, value=missing, predicate=missing): >> ... if value is missing: >> ... if predicate is missing: >> ... raise TypeError >> ... return sum(1 for item in self if predicate(item)) >> ... else: >> ... if predicate is not missing: >> ... raise TypeError >> ... return list.count(self, value) >> ...>>> items = List(range(10)) >> >>> items.count(7) >> 1 >> >>> items.count(predicate=lambda item: item%3) >> >> 6 >> >> which nicely covers all applications I can imagine. >> >> Peter > > That is a good idea. However, I was looking more at the simplicity of > building of ideas that are already present in .sort. And this > implementation is pretty simple as well. Note that the cmp() builtin and the cmp parameter for list.sort() are gone in Python 3. >>>> class List(list): > import __builtin__ > def count(self, value, cmp=__builtin__.cmp): > return sum(1 for item in self if not cmp(item, value)) > > >>>> items = List(range(10)) >>>> items.count(7) > 1 >>>> items.count(3, lambda a, b: not a%b) # My way > 6 >>>> items.count(Ellipsis, lambda a, b: not a%3) # Your way > 6 > > As a side note, wouldn't it be nice if '...' could be used in more > places than just slices? IMHO, a useful idiom would be to use it to > signify "irrelevant" or "don't care", as opposed to 'None' which (in > my mind, at least) signifies "missing" or "unknown". That is a pretty subtle distinction... I prefer keyword arguments, but in Python 3 you can use the ellipsis literal freely: >>> ... == ... True >>> [..., 42, ...].count(...) 2 Peter From samwyse at gmail.com Mon Jan 18 07:28:51 2010 From: samwyse at gmail.com (samwyse) Date: Mon, 18 Jan 2010 04:28:51 -0800 (PST) Subject: The answer References: Message-ID: <460a00bf-e87d-443b-b631-eaaa42ed4b4c@r24g2000yqd.googlegroups.com> On Jan 17, 8:30?pm, Jive Dadson wrote: > Okay, with your help I've figured it out. ?Instructions are below, but > read the caveat by Ben Fenny in this thread. ?All this stuff is good for > one default version of Python only. ?The PYTHONPATH described below, for > example, cannot specify a version number. ?Yes, that's a pain in the > butt, but there's no way around it. ?If you switch versions, you may > have to delete all the .pyc files that will show up in the module > folders. ?Python ought to check them to see if they are valid, but I do > not know if it does so. > > These instructions are for MS Windows. > > 1) Create your modules folder. Let's say it's named "Modules." ?The > documentation calls it a "package." > > 2) In an explorer window or on the desktop, right click on My Computer, > and select Properties. > > 3) Select the Advanced tab, and click on Environment Variables near the > bottom. > > 4) Look for an environment variable named PYTHONPATH. > > ? ? a) If you do not find one, create one using the New button(s). I > don't know if it has to be in User Variables or System Variables. ?To > save time experimenting, I just put one in both. For the value, put the > full path of the folder Modules. > > ? ? b) If there's already a PYTHONPATH, ?Edit it, adding a semi-colon > and the full path of folder Module to the end. > > 5) Put your module folders into the folder Module. > > 6) (Here's a really arcane bit.) Into each module folder, put a file > named __init__.py. ?It will be executed when you load the module. ?It > can be empty, but it has to be there or else the module folder will be > ignored. In your original thread, you never quite said why you can't use site- packages and .pth files. Are you not allowed to modify your local installation? If you are writing something for distribution to others, then site-packages and .pth files are the best way to go, since they don't assume any particular operating system. If you can't (or won't) use them, then just create Module as a sub-directory of wherever your program lives, since that directory is always prepended to PYTHONPATH. If you need to use the same module from multiple directories, most modern operating systems support symbolic links; if you're using Windows, well, here's a nickel kid, get yourself a better computer (http://farm1.static.flickr.com/ 89/240711122_f9888e5a3b_o.jpg). I don't think that __init__.py is very arcane, since it is described in detail in the documentation. It's also a great place to use the standard site.addsitedir() function, which is another platform independent way to manipulate Python's search path. From iainking at gmail.com Mon Jan 18 07:41:41 2010 From: iainking at gmail.com (Iain King) Date: Mon, 18 Jan 2010 04:41:41 -0800 (PST) Subject: substitution References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> Message-ID: <924f9bbd-ae54-4984-b6c7-a8950b5d4000@c29g2000yqd.googlegroups.com> On Jan 18, 10:21?am, superpollo wrote: > superpollo ha scritto: > > > hi. > > > what is the most pythonic way to substitute substrings? > > > eg: i want to apply: > > > foo --> bar > > baz --> quux > > quuux --> foo > > > so that: > > > fooxxxbazyyyquuux --> barxxxquuxyyyfoo > > > bye > > i explain better: > > say the subs are: > > quuux --> foo > foo --> bar > baz --> quux > > then i cannot apply the subs in sequence (say, .replace() in a loop), > otherwise: > > fooxxxbazyyyquuux --> fooxxxbazyyyfoo --> barxxxbazyyybar --> > barxxxquuxyyybar > > not as intended... Not sure if it's the most pythonic, but I'd probably do it like this: def token_replace(string, subs): subs = dict(subs) tokens = {} for i, sub in enumerate(subs): tokens[sub] = i tokens[i] = sub current = [string] for sub in subs: new = [] for piece in current: if type(piece) == str: chunks = piece.split(sub) new.append(chunks[0]) for chunk in chunks[1:]: new.append(tokens[sub]) new.append(chunk) else: new.append(piece) current = new output = [] for piece in current: if type(piece) == str: output.append(piece) else: output.append(subs[tokens[piece]]) return ''.join(output) >>> token_replace("fooxxxbazyyyquuux", [("quuux", "foo"), ("foo", "bar"), ("baz", "quux")]) 'barxxxquuxyyyfoo' I'm sure someone could whittle that down to a handful of list comps... Iain From anthra.norell at bluewin.ch Mon Jan 18 07:43:49 2010 From: anthra.norell at bluewin.ch (Anthra Norell) Date: Mon, 18 Jan 2010 13:43:49 +0100 Subject: substitution In-Reply-To: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> Message-ID: <4B545785.8090003@bluewin.ch> superpollo wrote: > hi. > > what is the most pythonic way to substitute substrings? > > eg: i want to apply: > > foo --> bar > baz --> quux > quuux --> foo > > so that: > > fooxxxbazyyyquuux --> barxxxquuxyyyfoo > > bye Try the code below the dotted line. It does any number of substitutions and handles overlaps correctly (long over short) Your case: >>> substitutions = (('foo', 'bar'), ('baz', 'quux'), ('quuux', 'foo')) # Sequence of doublets >>> T = Translator (substitutions) # Compile substitutions -> translator >>> s = 'fooxxxbazyyyquuux' # Your source string >>> d = 'barxxxquuxyyyfoo' # Your destination string >>> print T (s) barxxxquuxyyyfoo >>> print T (s) == d True Frederic ------------------------------------------------------------- class Translator: r""" Will translate any number of targets, handling them correctly if some overlap. Making Translator T = Translator (definitions, [eat = 1]) 'definitions' is a sequence of pairs: ((target, substitute),(t2, s2), ...) 'eat' says whether untargeted sections pass (translator) or are skipped (extractor). Makes a translator by default (eat = False) T.eat is an instance attribute that can be changed at any time. Definitions example: (('a','A'),('b','B'),('ab','ab'),('abc','xyz') # ('ab','ab') see Tricks. ('\x0c', 'page break'), ('\r\n','\n'), (' ','\t')) Order doesn't matter. Running translation = T (source) Tricks Deletion: ('target', '') Exception: (('\n',''), ('\n\n','\n\n')) # Eat LF except paragraph breaks. Exception: (('\n', '\r\n'), ('\r\n',\r\n')) # Unix to DOS, would leave DOS unchanged Translation cascade: # Rejoin text lines per paragraph Unix or DOS, inserting inter-word space if missing Mark_LF = Translator ((('\n','+LF+'),('\r\n','+LF+'),('\r\n\r\n','\r\n\r\n'),('\n\n','\n\n'))) # Pick positively identifiable mark for Unix and DOS end of lines Single_Space_Mark = Translator (((' +LF+', ' '),('+LF+', ' '),('-+LF+', ''))) no_lf_text = Single_Space_Mark (Mark_LF (text)) Translation cascade: # Nesting calls reptiles = T_latin_english (T_german_latin (reptilien)) Limitations 1. The number of substitutions and the maximum size of input depends on the respective capabilities of the Python re module. 2. Regular expressions will not work as such. Author: Frederic Rentsch (anthra.norell at bluewin.ch). """ def __init__ (self, definitions, eat = 0): ''' definitions: a sequence of pairs of strings. ((target, substitute), (t, s), ...) eat: False (0) means translate: unaffected data passes unaltered. True (1) means extract: unaffected data doesn't pass (gets eaten). Extraction filters typically require substitutes to end with some separator, else they fuse together. (E.g. ' ', '\t' or '\n') 'eat' is an attribute that can be switched anytime. ''' self.eat = eat self.compile_sequence_of_pairs (definitions) def compile_sequence_of_pairs (self, definitions): ''' Argument 'definitions' is a sequence of pairs: (('target 1', 'substitute 1'), ('t2', 's2'), ...) Order doesn't matter. ''' import re self.definitions = definitions targets, substitutes = zip (*definitions) re_targets = [re.escape (item) for item in targets] re_targets.sort (reverse = True) self.targets_set = set (targets) self.table = dict (definitions) regex_string = '|'.join (re_targets) self.regex = re.compile (regex_string, re.DOTALL) def __call__ (self, s): hits = self.regex.findall (s) nohits = self.regex.split (s) valid_hits = set (hits) & self.targets_set # Ignore targets with illegal re modifiers. if valid_hits: substitutes = [self.table [item] for item in hits if item in valid_hits] + [] # Make lengths equal for zip to work right if self.eat: return ''.join (substitutes) else: zipped = zip (nohits, substitutes) return ''.join (list (reduce (lambda a, b: a + b, [zipped][0]))) + nohits [-1] else: if self.eat: return '' else: return s From samwyse at gmail.com Mon Jan 18 07:49:06 2010 From: samwyse at gmail.com (samwyse) Date: Mon, 18 Jan 2010 04:49:06 -0800 (PST) Subject: enhancing 'list' References: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> Message-ID: <86ae75ac-f174-4e90-b1a3-301004967d1b@34g2000yqp.googlegroups.com> On Jan 18, 6:20?am, Peter Otten <__pete... at web.de> wrote: > Note that the cmp() builtin and the cmp parameter for list.sort() are gone > in Python 3. I've got Python 3 installed, and am using it for most new development. In this case case, however, I'm writing for the Google App Engine, which is stuck at 2.5. :( (Curiously, no matter how I order my PATH, the wrong version seems to appear first more than half the time! I'm seriously considering renaming all my Python 3 code to use a .py3 file extension.) > samwyse wrote: > > As a side note, wouldn't it be nice if '...' could be used in more > > places than just slices? ?IMHO, a useful idiom would be to use it to > > signify "irrelevant" or "don't care", as opposed to 'None' which (in > > my mind, at least) signifies "missing" or "unknown". > > That is a pretty subtle distinction... Hey, I'm a pretty subtle person... > I prefer keyword arguments, but in Python 3 you can use the ellipsis literal > freely: > > >>> ... == ... > True > >>> [..., 42, ...].count(...) > > 2 > > Peter I must have either missed or forgotten about that. Thanks! From stefan_ml at behnel.de Mon Jan 18 07:50:48 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 18 Jan 2010 13:50:48 +0100 Subject: enhancing 'list' In-Reply-To: <86ae75ac-f174-4e90-b1a3-301004967d1b@34g2000yqp.googlegroups.com> References: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> <86ae75ac-f174-4e90-b1a3-301004967d1b@34g2000yqp.googlegroups.com> Message-ID: <4b545928$0$7620$9b4e6d93@newsspool1.arcor-online.net> samwyse, 18.01.2010 13:49: > Curiously, no matter how I > order my PATH, the wrong version seems to appear first more than half > the time! I'm seriously considering renaming all my Python 3 code to > use a .py3 file extension. You should be able to start the interpreter as "python3.1" to be sure. Stefan From kaklis at gmail.com Mon Jan 18 07:52:41 2010 From: kaklis at gmail.com (kaklis at gmail.com) Date: Mon, 18 Jan 2010 04:52:41 -0800 (PST) Subject: Parse a log file Message-ID: <92293df0-7d2e-453a-b5bc-d4a7eb905fcf@l19g2000yqb.googlegroups.com> Hello to all! I want to parse a log file with the following format for example: TIMESTAMPE Operation FileName Bytes 12/Jan/2010:16:04:59 +0200 EXISTS sample3.3gp 37151 12/Jan/2010:16:04:59 +0200 EXISTS sample3.3gp 37151 12/Jan/2010:16:04:59 +0200 EXISTS sample3.3gp 37151 12/Jan/2010:16:04:59 +0200 EXISTS sample3.3gp 37151 12/Jan/2010:16:04:59 +0200 EXISTS sample3.3gp 37151 12/Jan/2010:16:05:05 +0200 DELETE sample3.3gp 37151 How can i count the operations for a month(e.g total of 40 Operations, 30 exists, 10 delete?) Any tips? Thanks in advance Antonis From cournape at gmail.com Mon Jan 18 07:56:31 2010 From: cournape at gmail.com (David Cournapeau) Date: Mon, 18 Jan 2010 21:56:31 +0900 Subject: Is python not good enough? In-Reply-To: <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> Message-ID: <5b8d13221001180456w3fab510bl10b69474476cbf62@mail.gmail.com> On Mon, Jan 18, 2010 at 8:03 PM, Phlip wrote: > This means, to appease the self-righteous indignation of the math > professor who would claim = should mean "equality"... Much more likely, this is part of the stated goal of making go very easy to analyse (to build tools and so that go is very fast to compile), as stated in its FAQ. cheers, David From cjwilliams43 at gmail.com Mon Jan 18 08:01:28 2010 From: cjwilliams43 at gmail.com (Colin W.) Date: Mon, 18 Jan 2010 08:01:28 -0500 Subject: r"string" vs R"string In-Reply-To: <00dd21f5$0$15570$c3e8da3@news.astraweb.com> References: <00dd21f5$0$15570$c3e8da3@news.astraweb.com> Message-ID: On 17-Jan-10 18:27 PM, Steven D'Aprano wrote: > On Sun, 17 Jan 2010 11:13:48 -0500, Roy Smith wrote: > >> In article, >> "Colin W." wrote: >> >>> On 17-Jan-10 02:16 AM, Terry Reedy wrote: >>>> On 1/17/2010 1:55 AM, Brendan Miller wrote: >>>>> Is there any difference whatsoever between a raw string beginning >>>>> with the captical R or one with the lower case r e.g. r"string" vs >>>>> R"string"? >>>> >>>> No. Nor is there and difference between the strings created with raw >>>> literals and cooked literals. >>>> >>>> >>> "cooked" literal ?? >> >> I've never seen it referred to this way in the Python literature, but >> "cooked" is a well-known term meaning, "not raw". The usage goes back >> decades. > > I think the use of "cooked" meaning "not raw" goes back a little bit more > than decades. The verb "to cook" dates from the late 14th century, and > the adjective form "cooked" would follow soon after. The use of "cooked" > to mean manipulated (as in "cooking the books") comes from the 1630s. > Extending it to strings (as in raw versus cooked strings) is an obvious > extension. > Yes, I should have cottoned on, but perhaps "... between the strings, raw and other." would have conveyed the idea. Colin W. From samwyse at gmail.com Mon Jan 18 08:05:01 2010 From: samwyse at gmail.com (samwyse) Date: Mon, 18 Jan 2010 05:05:01 -0800 (PST) Subject: Parse a log file References: <92293df0-7d2e-453a-b5bc-d4a7eb905fcf@l19g2000yqb.googlegroups.com> Message-ID: <419bf009-da09-4b89-8fe0-96f854d9f9b4@j14g2000yqm.googlegroups.com> On Jan 18, 6:52?am, "kak... at gmail.com" wrote: > Hello to all! > I want to parse a log file with the following format for > example: > ? ? ? ? ? ? ? TIMESTAMPE ? ? ? ? ? ?Operation ? ? FileName > Bytes > 12/Jan/2010:16:04:59 +0200 ? EXISTS ? ? ? sample3.3gp ? 37151 > 12/Jan/2010:16:04:59 +0200 ?EXISTS ? ? ? ?sample3.3gp ? 37151 > 12/Jan/2010:16:04:59 +0200 ?EXISTS ? ? ? ?sample3.3gp ? 37151 > 12/Jan/2010:16:04:59 +0200 ?EXISTS ? ? ? ?sample3.3gp ? 37151 > 12/Jan/2010:16:04:59 +0200 ?EXISTS ? ? ? ?sample3.3gp ? 37151 > 12/Jan/2010:16:05:05 +0200 ?DELETE ? ? ?sample3.3gp ? 37151 > > How can i count the operations for a month(e.g total of 40 Operations, > 30 exists, 10 delete?) > Any tips? > > Thanks in advance > Antonis time.strptime(string[, format]) Parse a string representing a time according to a format. The return value is a struct_time as returned by gmtime() or localtime(). The format parameter uses the same directives as those used by strftime (); it defaults to "%a %b %d %H:%M:%S %Y" which matches the formatting returned by ctime(). If string cannot be parsed according to format, or if it has excess data after parsing, ValueError is raised. The default values used to fill in any missing data when more accurate values cannot be inferred are (1900, 1, 1, 0, 0, 0, 0, 1, -1). >>> import time >>> ts='12/Jan/2010:16:04:59 +0200' >>> time.strptime(ts[:-6], '%d/%b/%Y:%H:%M:%S') time.struct_time(tm_year=2010, tm_mon=1, tm_mday=12, tm_hour=16, tm_min=4, tm_sec=59, tm_wday=1, tm_yday=12, tm_isdst=-1) I leave the conversion of the last six characters (the time zone offset) as an exercise for the student. :) From iainking at gmail.com Mon Jan 18 08:22:37 2010 From: iainking at gmail.com (Iain King) Date: Mon, 18 Jan 2010 05:22:37 -0800 (PST) Subject: substitution References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> <924f9bbd-ae54-4984-b6c7-a8950b5d4000@c29g2000yqd.googlegroups.com> Message-ID: <0fa11055-66c4-47ef-9e9c-01cad10d998e@a6g2000yqm.googlegroups.com> On Jan 18, 12:41?pm, Iain King wrote: > On Jan 18, 10:21?am, superpollo wrote: > > > > > superpollo ha scritto: > > > > hi. > > > > what is the most pythonic way to substitute substrings? > > > > eg: i want to apply: > > > > foo --> bar > > > baz --> quux > > > quuux --> foo > > > > so that: > > > > fooxxxbazyyyquuux --> barxxxquuxyyyfoo > > > > bye > > > i explain better: > > > say the subs are: > > > quuux --> foo > > foo --> bar > > baz --> quux > > > then i cannot apply the subs in sequence (say, .replace() in a loop), > > otherwise: > > > fooxxxbazyyyquuux --> fooxxxbazyyyfoo --> barxxxbazyyybar --> > > barxxxquuxyyybar > > > not as intended... > > Not sure if it's the most pythonic, but I'd probably do it like this: > > def token_replace(string, subs): > ? ? ? ? subs = dict(subs) > ? ? ? ? tokens = {} > ? ? ? ? for i, sub in enumerate(subs): > ? ? ? ? ? ? ? ? tokens[sub] = i > ? ? ? ? ? ? ? ? tokens[i] = sub > ? ? ? ? current = [string] > ? ? ? ? for sub in subs: > ? ? ? ? ? ? ? ? new = [] > ? ? ? ? ? ? ? ? for piece in current: > ? ? ? ? ? ? ? ? ? ? ? ? if type(piece) == str: > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? chunks = piece.split(sub) > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? new.append(chunks[0]) > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? for chunk in chunks[1:]: > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? new.append(tokens[sub]) > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? new.append(chunk) > ? ? ? ? ? ? ? ? ? ? ? ? else: > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? new.append(piece) > ? ? ? ? ? ? ? ? current = new > ? ? ? ? output = [] > ? ? ? ? for piece in current: > ? ? ? ? ? ? ? ? if type(piece) == str: > ? ? ? ? ? ? ? ? ? ? ? ? output.append(piece) > ? ? ? ? ? ? ? ? else: > ? ? ? ? ? ? ? ? ? ? ? ? output.append(subs[tokens[piece]]) > ? ? ? ? return ''.join(output) > > >>> token_replace("fooxxxbazyyyquuux", [("quuux", "foo"), ("foo", "bar"), ("baz", "quux")]) > > 'barxxxquuxyyyfoo' > > I'm sure someone could whittle that down to a handful of list comps... > Iain Slightly better (lets you have overlapping search strings, used in the order they are fed in): def token_replace(string, subs): tokens = {} if type(subs) == dict: for i, sub in enumerate(subs): tokens[sub] = i tokens[i] = subs[sub] else: s = [] for i, (k,v) in enumerate(subs): tokens[k] = i tokens[i] = v s.append(k) subs = s current = [string] for sub in subs: new = [] for piece in current: if type(piece) == str: chunks = piece.split(sub) new.append(chunks[0]) for chunk in chunks[1:]: new.append(tokens[sub]) new.append(chunk) else: new.append(piece) current = new output = [] for piece in current: if type(piece) == str: output.append(piece) else: output.append(tokens[piece]) return ''.join(output) From __peter__ at web.de Mon Jan 18 08:31:50 2010 From: __peter__ at web.de (Peter Otten) Date: Mon, 18 Jan 2010 14:31:50 +0100 Subject: substitution References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> <924f9bbd-ae54-4984-b6c7-a8950b5d4000@c29g2000yqd.googlegroups.com> Message-ID: Iain King wrote: > Not sure if it's the most pythonic, but I'd probably do it like this: > > def token_replace(string, subs): > subs = dict(subs) > tokens = {} > for i, sub in enumerate(subs): > tokens[sub] = i > tokens[i] = sub > current = [string] > for sub in subs: > new = [] > for piece in current: > if type(piece) == str: > chunks = piece.split(sub) > new.append(chunks[0]) > for chunk in chunks[1:]: > new.append(tokens[sub]) > new.append(chunk) > else: > new.append(piece) > current = new > output = [] > for piece in current: > if type(piece) == str: > output.append(piece) > else: > output.append(subs[tokens[piece]]) > return ''.join(output) > > >>> token_replace("fooxxxbazyyyquuux", [("quuux", "foo"), ("foo", "bar"), ("baz", "quux")]) > 'barxxxquuxyyyfoo' > > I'm sure someone could whittle that down to a handful of list comps... I tried, but failed: def join(chunks, separator): chunks = iter(chunks) yield next(chunks) for chunk in chunks: yield separator yield chunk def token_replace(string, subs): tokens = {} current = [string] for i, (find, replace) in enumerate(subs): tokens[i] = replace new = [] for piece in current: if piece in tokens: new.append(piece) else: new.extend(join(piece.split(find), i)) current = new return ''.join(tokens.get(piece, piece) for piece in current) You could replace the inner loop with sum(..., []), but that would be really ugly. Peter From utente at esempio.net Mon Jan 18 08:43:46 2010 From: utente at esempio.net (superpollo) Date: Mon, 18 Jan 2010 14:43:46 +0100 Subject: substitution In-Reply-To: References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> <924f9bbd-ae54-4984-b6c7-a8950b5d4000@c29g2000yqd.googlegroups.com> Message-ID: <4b546593$0$1114$4fafbaef@reader4.news.tin.it> it looked simpler when i posted, but i realize that the problem is non trivial. thanks to everybody. i guess that the algorithm would be easier if it was known in advance that the string to substitute must have some specific property, say: 1) they all must start with "XYZ" 2) they all have the same length N (e.g. 5) like this: qweXYZ12asdXYZ1345XYZ ---> qweIWAS12asdIWAS1345XYZ bye From anh.hai.trinh at gmail.com Mon Jan 18 08:59:59 2010 From: anh.hai.trinh at gmail.com (Anh Hai Trinh) Date: Mon, 18 Jan 2010 05:59:59 -0800 (PST) Subject: Is python not good enough? References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> Message-ID: <37f8195f-3f02-4a13-b677-6b5d5e1589e8@22g2000yqr.googlegroups.com> On Jan 18, 6:03?pm, Phlip wrote: > On Jan 12, 7:09?am, ikuta liu wrote: > > > Go language try to merge low level, hight level and browser language. > > Go uses := for assignment. Except that it doesn't. := is a declaration. s := "foo" is short for var s string = "foo" Cheers, ----aht From jeanmichel at sequans.com Mon Jan 18 09:10:37 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 18 Jan 2010 15:10:37 +0100 Subject: Changing var names In-Reply-To: <4dc0cfea1001151203w2d1c9520t692d2367a1882ee3@mail.gmail.com> References: <4dc0cfea1001150927i48b6c386w53f4487cd894d078@mail.gmail.com> <1263582938.3142.12.camel@linux-m3mt> <4dc0cfea1001151203w2d1c9520t692d2367a1882ee3@mail.gmail.com> Message-ID: <4B546BDD.1030602@sequans.com> Victor Subervi wrote: > On Fri, Jan 15, 2010 at 3:15 PM, Adam Tauno Williams > > wrote: > > On Fri, 2010-01-15 at 13:27 -0400, Victor Subervi wrote: > > Hi; > > Well it took me *less than a day* to fix the following problems: > > -- bare excepts (accidentally left a couple I think) > > -- sql injection attacks > > -- recreating tables to make them more reasonable > > Now, I believe someone once mentioned that there is some s/w > tool out > > there for changing the names of my variables more easily than going > > through every script one-by-one. If so, please share with me again > > what it is. > > This process is called 'refactoring' [a good term to Google], and > every > decent IDE provides some support [if it doesn't, it isn't a "decent" > IDE] > > > Thanks. I'll work with Eric, I guess, once I get my Mac. > beno Here are 2 well known IDEs, that help with refactoring (and many other things). http://netbeans.org/ http://www.eclipse.org/ Both supports python. I personnaly will stick to vim :o) JM From gagsl-py2 at yahoo.com.ar Mon Jan 18 09:11:46 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 18 Jan 2010 11:11:46 -0300 Subject: Inheriting methods but over-riding docstrings References: <0361ea2d$0$1336$c3e8da3@news.astraweb.com> Message-ID: En Sun, 17 Jan 2010 23:23:45 -0300, Steve Holden escribi?: > Gabriel Genellina wrote: >> Methods don't have docstrings; functions do. So one has to "clone" the >> function to set a new docstring. >> > On behalf of all methods I would like to say "We demand the right to > docstrings". > >>>> print abs.__doc__ > abs(number) -> number > > Return the absolute value of the argument. >>>> > > Perhaps I am misunderstanding "don't have docstrings". This code: Sorry, I wasn't clear at all. What I wanted to say is that docstrings are actually stored as *function* attributes; methods inherit the docstring from the function they are built on (as a read-only attribute). py> class Base(object): ... def method(self): ... "my docstring" ... py> Base.method.__doc__ 'my docstring' py> Base.method.__doc__ = "another docstring" Traceback (most recent call last): File "", line 1, in AttributeError: attribute '__doc__' of 'instancemethod' objects is not writable py> Base.method.im_func.__doc__ 'my docstring' py> Base.method.im_func.__doc__ = "another docstring" py> Base.method.__doc__ 'another docstring' So, if you want Base.foo and Derived.foo to share the same behavior but have separate docstrings (what Steven asked for), you have to create separate functions for both. That's what my already posted code does. > So, are the demands of the methods reasonable, or do they already have > what they are so loudly demanding? Well, they could demand the right to have a docstring of their own. Since 3.x eliminated a whole category of methods (unbound methods are gone), their ability to press for demands was greatly reduced. I don't think they could get such right granted in the near future... -- Gabriel Genellina From adi at digitaltrowel.com Mon Jan 18 09:17:56 2010 From: adi at digitaltrowel.com (Adi Eyal) Date: Mon, 18 Jan 2010 16:17:56 +0200 Subject: substitution Message-ID: > From:?superpollo > To: > Date:?Mon, 18 Jan 2010 11:15:37 +0100 > Subject:?substitution > hi. > > what is the most pythonic way to substitute substrings? > > eg: i want to apply: > > foo --> bar > baz --> quux > quuux --> foo > > so that: > > fooxxxbazyyyquuux --> barxxxquuxyyyfoo > > bye Using regular expressions the answer is short (and sweet) mapping = { "foo" : "bar", "baz" : "quux", "quuux" : "foo" } pattern = "(%s)" % "|".join(mapping.keys()) repl = lambda x : mapping.get(x.group(1), x.group(1)) s = "fooxxxbazyyyquuux" re.subn(pattern, repl, s) From iainking at gmail.com Mon Jan 18 09:23:44 2010 From: iainking at gmail.com (Iain King) Date: Mon, 18 Jan 2010 06:23:44 -0800 (PST) Subject: substitution References: Message-ID: <93831c02-6e5a-48f7-a698-3b3f1ef6e5b6@v25g2000yqk.googlegroups.com> On Jan 18, 2:17?pm, Adi Eyal wrote: > > From:?superpollo > > To: > > Date:?Mon, 18 Jan 2010 11:15:37 +0100 > > Subject:?substitution > > hi. > > > what is the most pythonic way to substitute substrings? > > > eg: i want to apply: > > > foo --> bar > > baz --> quux > > quuux --> foo > > > so that: > > > fooxxxbazyyyquuux --> barxxxquuxyyyfoo > > > bye > > Using regular expressions the answer is short (and sweet) > > mapping = { > ? ? ? ? "foo" : "bar", > ? ? ? ? "baz" : "quux", > ? ? ? ? "quuux" : "foo" > > } > > pattern = "(%s)" % "|".join(mapping.keys()) > repl = lambda x : mapping.get(x.group(1), x.group(1)) > s = "fooxxxbazyyyquuux" > re.subn(pattern, repl, s) Winner! :) Iain From ted at sjksdjk.it Mon Jan 18 09:32:36 2010 From: ted at sjksdjk.it (ted) Date: Mon, 18 Jan 2010 15:32:36 +0100 Subject: python gui ide under linux..like visual studio ;) ? Message-ID: <4b5470ce$0$1103$4fafbaef@reader4.news.tin.it> Hi at all... Can someone please give me some advice, about a good IDE with control GUI under Linux ? Actually i know QT Creator by Nokia which i can use with Python (but i don't know how). And, a good library for access to database (mysql, sql server, oracle) ? Thank you very much ! bye From oktakaweb at gmail.com Mon Jan 18 09:40:47 2010 From: oktakaweb at gmail.com (Oktaka Com) Date: Mon, 18 Jan 2010 06:40:47 -0800 (PST) Subject: multithreading, performance, again... References: Message-ID: <4826604d-85f3-48b9-863e-42401d29bf6a@l19g2000yqb.googlegroups.com> On 30 Aral?k 2009, 17:44, mk wrote: > Hello everyone, > > I have figured out (sort of) how to do profiling of multithreaded > programs with cProfile, it goes something like this: > > #!/usr/local/bin/python > > import cProfile > import threading > > class TestProf(threading.Thread): > ? ? ?def __init__(self, ip): > > ? ? ? ? ?threading.Thread.__init__(self) > ? ? ? ? ?self.ip = ip > > ? ? ?def run(self): > ? ? ? ? ?prof = cProfile.Profile() > ? ? ? ? ?retval = prof.runcall(self.runmethod) > ? ? ? ? ?prof.dump_stats('tprof' + self.ip) > > ? ? ?def runmethod(self): > ? ? ? ? ?pass > > tp = TestProf('10.0.10.10') > > tp.start() > tp.join() > > The problem is, now that I've done profiling in the actual program > (profiled version here:http://python.domeny.com/cssh_profiled.py) with > 9 threads and added up stats (using pstats.Stats.add()), the times I get > are trivial: > > ?>>> p.strip_dirs().sort_stats('cumulative').print_stats(10) > Wed Dec 30 16:23:59 2009 ? ?csshprof9.156.44.113 > Wed Dec 30 16:23:59 2009 ? ?csshprof9.156.46.243 > Wed Dec 30 16:23:59 2009 ? ?csshprof9.156.46.89 > Wed Dec 30 16:24:00 2009 ? ?csshprof9.156.47.125 > Wed Dec 30 16:24:00 2009 ? ?csshprof9.156.47.17 > Wed Dec 30 16:24:00 2009 ? ?csshprof9.156.47.29 > Wed Dec 30 16:24:01 2009 ? ?csshprof9.167.41.241 > Wed Dec 30 16:24:02 2009 ? ?csshprof9.168.119.15 > Wed Dec 30 16:24:02 2009 ? ?csshprof9.168.119.218 > > ? ? ? ? ? 39123 function calls (38988 primitive calls) in 6.004 CPU seconds > > ? ? Ordered by: cumulative time > ? ? List reduced from 224 to 10 due to restriction <10> > > ? ? ncalls ?tottime ?percall ?cumtime ?percall filename:lineno(function) > ? ? ? ? ?9 ? ?0.000 ? ?0.000 ? ?6.004 ? ?0.667 cssh.py:696(runmethod) > ? ? ? ?100 ? ?0.004 ? ?0.000 ? ?5.467 ? ?0.055 threading.py:389(wait) > ? ? ? ? 82 ? ?0.025 ? ?0.000 ? ?5.460 ? ?0.067 threading.py:228(wait) > ? ? ? ?400 ? ?5.400 ? ?0.013 ? ?5.400 ? ?0.013 {time.sleep} > ? ? ? ? ?9 ? ?0.000 ? ?0.000 ? ?5.263 ? ?0.585 cssh.py:452(ssh_connect) > ? ? ? ? ?9 ? ?0.003 ? ?0.000 ? ?5.262 ? ?0.585 client.py:226(connect) > ? ? ? ? ?9 ? ?0.001 ? ?0.000 ? ?2.804 ? ?0.312 > transport.py:394(start_client) > ? ? ? ? ?9 ? ?0.005 ? ?0.001 ? ?2.254 ? ?0.250 client.py:391(_auth) > ? ? ? ? 18 ? ?0.001 ? ?0.000 ? ?2.115 ? ?0.117 > transport.py:1169(auth_publickey) > ? ? ? ? 18 ? ?0.001 ? ?0.000 ? ?2.030 ? ?0.113 > auth_handler.py:156(wait_for_response) > > > > It's not burning CPU time in the main thread (profiling with cProfile > indicated smth similar to the above), it's not burning it in the > individual worker threads - so where the heck it is burning this CPU > time? bc 'top' shows heavy CPU load during most of the time of the > program run. > > help... > > regards, > mk See http://code.google.com/p/yappi/ if you want to profile multithreaded python app. From mwilson at the-wire.com Mon Jan 18 09:44:50 2010 From: mwilson at the-wire.com (Mel) Date: Mon, 18 Jan 2010 09:44:50 -0500 Subject: substitution References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> Message-ID: superpollo wrote: > hi. > > what is the most pythonic way to substitute substrings? > > eg: i want to apply: > > foo --> bar > baz --> quux > quuux --> foo > > so that: > > fooxxxbazyyyquuux --> barxxxquuxyyyfoo This is simple -- maybe a bit brutal -- and if the strings get long it will be slow: def replaced (input, replacements): output = [] while input: for target in replacements: if input.startswith (target): output.append (replacements [target]) input = input [len (target):] break else: output.append (input[0]) input = input [1:] return ''.join (output) original = 'fooxxxbazyyyquux' replacements = {'quux':'foo', 'foo':'bar', 'baz':'quux'} print original, '-->', replaced (original, replacements) Mel. From bartc at freeuk.com Mon Jan 18 10:03:57 2010 From: bartc at freeuk.com (bartc) Date: Mon, 18 Jan 2010 15:03:57 GMT Subject: basic Class in Python In-Reply-To: References: <86a6a8c8-49fe-4a3d-81a0-5a44a179c7d6@o28g2000yqh.googlegroups.com> Message-ID: "Wolfgang Rohdewald" wrote in message news:mailman.1056.1263771299.28905.python-list at python.org... > On Monday 18 January 2010, BarryJOgorman wrote: >> TypeError: object._new_() takes no parameters > >> def _init_(self, name, job=None, pay=0): > > __init__ needs two underscores left and right Any particular reason why two, and not one (or three)? In some fonts it's difficult to tell how many as they run together. -- Bartc From jeanmichel at sequans.com Mon Jan 18 10:07:56 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 18 Jan 2010 16:07:56 +0100 Subject: Proper display of XMLRPCserver exceptions Message-ID: <4B54794C.6020207@sequans.com> To all using xmlrpclib, I had trouble getting a proper display of my exceptions occuring on the server. Basically, xmlrpclib only display the exception itself without any traceback, on the client side. Something like :2"> Ok then there's a key error, but how the hell am I supposed to know where it occured ? So I wrote this: from SimpleXMLRPCServer import SimpleXMLRPCServer server = SimpleXMLRPCServer(('localhost', 8000), logRequests=False, allow_none=True) old_dispatch = server._dispatch def _new_dispatch(method, params): try: return old_dispatch(method,params) except Exception, exc: import traceback traceback.print_exc() raise server._dispatch = _new_dispatch Pretty dirty, isn't it, accessing the private attributes (too lazy to subclass) ! I guess there's an obvious way to do this with xmlrpclib that I just missed. Anyone knows ? JM From nobody at nowhere.com Mon Jan 18 10:09:31 2010 From: nobody at nowhere.com (Nobody) Date: Mon, 18 Jan 2010 15:09:31 +0000 Subject: substitution References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> Message-ID: On Mon, 18 Jan 2010 11:21:54 +0100, superpollo wrote: >> what is the most pythonic way to substitute substrings? > say the subs are: > > quuux --> foo > foo --> bar > baz --> quux > > then i cannot apply the subs in sequence (say, .replace() in a loop), > otherwise: > > fooxxxbazyyyquuux --> fooxxxbazyyyfoo --> barxxxbazyyybar --> > barxxxquuxyyybar > > not as intended... Are there any characters which are guaranteed never to occur in the string? If so: s = s.replace('quuux', '@1').replace('foo', '@2').replace('baz', '@3') s = s.replace('@1', 'foo').replace('@2', 'bar').replace('@3', 'quux') E.g.: def replace(subs, s): for i, (src, dst) in enumerate(subs): s = s.replace(src, '@%06d' % i) for i, (src, dst) in enumerate(subs): s = s.replace('@%06d' % i, dst) return s Not the most efficient solution (and problematic if there aren't any "unused" characters), but somewhat shorter than the other solutions. From R.Brodie at rl.ac.uk Mon Jan 18 10:21:36 2010 From: R.Brodie at rl.ac.uk (Richard Brodie) Date: Mon, 18 Jan 2010 15:21:36 -0000 Subject: basic Class in Python References: <86a6a8c8-49fe-4a3d-81a0-5a44a179c7d6@o28g2000yqh.googlegroups.com> Message-ID: "bartc" wrote in message news:xL_4n.28001$Ym4.5658 at text.news.virginmedia.com... > Any particular reason why two, and not one (or three)? In some fonts it's difficult to > tell how many as they run together. It follows the C convention for reserved identifers. From phlip2005 at gmail.com Mon Jan 18 10:37:36 2010 From: phlip2005 at gmail.com (Phlip) Date: Mon, 18 Jan 2010 07:37:36 -0800 (PST) Subject: Is python not good enough? References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> <37f8195f-3f02-4a13-b677-6b5d5e1589e8@22g2000yqr.googlegroups.com> Message-ID: <1e692369-cee9-4e63-8dce-33d70bf96368@p24g2000yqm.googlegroups.com> On Jan 18, 5:59?am, Anh Hai Trinh wrote: > > Go uses := for assignment. > > Except that it doesn't. := is a declaration. Ah, and that's why Go is easy for cheap parsers to rip. Tx all! I was formerly too mortified to proceed - now I'm back in the Go camp. They fixed the hideous redundancy of Java without the ill-defined scope issues of Python & Ruby, and without the tacky little 'var' of JavaScript! From dotancohen at gmail.com Mon Jan 18 10:42:38 2010 From: dotancohen at gmail.com (Dotan Cohen) Date: Mon, 18 Jan 2010 17:42:38 +0200 Subject: Generic Python Benchmark suite? In-Reply-To: References: Message-ID: <880dece01001180742h62f2febi37e44f282b583d50@mail.gmail.com> > What do you suggest? > $ man time -- Dotan Cohen http://what-is-what.com http://gibberish.co.il From utente at esempio.net Mon Jan 18 10:43:25 2010 From: utente at esempio.net (superpollo) Date: Mon, 18 Jan 2010 16:43:25 +0100 Subject: not and is not problem Message-ID: <4b54819e$0$1106$4fafbaef@reader4.news.tin.it> hi: #!/usr/bin/env python data = "seq=123" name , value = data.split("=") print name print value if not name == "seq": print "DOES NOT PRINT OF COURSE..." if name is not "seq": print "WTF! WHY DOES IT PRINT?" help please. bye From showell30 at yahoo.com Mon Jan 18 10:46:00 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 18 Jan 2010 07:46:00 -0800 (PST) Subject: ANN: shpaml 0.94b Message-ID: <8b8967a0-5fe3-471f-84fc-9f5d31a3c990@j19g2000yqk.googlegroups.com> Hi everybody, I would like to announce the latest version of shpaml, which is an indentation-based markup language similar to haml, but different. Shpaml allows you to author HTML-like content with an indentation- based syntax that eliminates the need to write and read close tags and angle brackets. It also has haml-like syntax for specifying divs, classes, and ids, which also makes your documents easier to read, particularly when you are also using CSS and jQuery-like tools. Finally, it is very transparent with respect to HTML semantics, as well as letting most HTML syntax pass through as needed. It is probably best understood by simply looking at a few examples: http://shpaml.webfactional.com/examples There is also a tutorial, and you can try out your own examples there, so need to download the software if you just want to poke around a bit: http://shpaml.webfactional.com/tutorial/1 If you do decide to download it, it is a pretty small module at 225 lines or so: http://shpaml.webfactional.com/source_code The project is about two months old now, and it is being actively maintained. We have a mailing list, users submitting patches, all that good stuff. I am labelling the current version as "beta," but it has been well tested on over a thousand lines of markup. Cheers, Steve From arnodel at googlemail.com Mon Jan 18 10:55:10 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 18 Jan 2010 07:55:10 -0800 (PST) Subject: not and is not problem References: <4b54819e$0$1106$4fafbaef@reader4.news.tin.it> Message-ID: <3e8d242a-7965-46c8-ab51-f61124aacc1a@j19g2000yqk.googlegroups.com> On 18 Jan, 15:43, superpollo wrote: > hi: > > #!/usr/bin/env python > data = "seq=123" > name , value = data.split("=") > print name > print value > if not name == "seq": > ? ? ?print "DOES NOT PRINT OF COURSE..." > if name is not "seq": > ? ? ?print "WTF! WHY DOES IT PRINT?" > > help please. > > bye is and == are different operators. * A == B means A.__eq__(B) * A is B means that A and B are the same object, living at the same physical location in the computer's memory. HTH -- Arnaud From apt.shansen at gmail.com Mon Jan 18 10:56:47 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Mon, 18 Jan 2010 07:56:47 -0800 Subject: not and is not problem In-Reply-To: <4b54819e$0$1106$4fafbaef@reader4.news.tin.it> References: <4b54819e$0$1106$4fafbaef@reader4.news.tin.it> Message-ID: <7a9c25c21001180756i13061fd5he51d9b0e6f3f38a5@mail.gmail.com> On Mon, Jan 18, 2010 at 7:43 AM, superpollo wrote: > #!/usr/bin/env python > data = "seq=123" > name , value = data.split("=") > print name > print value > if not name == "seq": > print "DOES NOT PRINT OF COURSE..." > if name is not "seq": > print "WTF! WHY DOES IT PRINT?" > Because name really is not "seq"; "seq" is an entirely new and different string. The "is" operator tests for object identity, == tests for equality. "name is name" will always return True, everything else will always return False. For example, you can do: other = name if name is other: print "OK" And it'll work, because those are the exact same object. You only use 'is' and 'is not' when you really want to ask, 'Is this the precisely same object?' Generally, that's only when you're testing if something is or is not None. Sometimes is or is not True/False, but usually 'if x' is preferred. --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthra.norell at bluewin.ch Mon Jan 18 10:59:57 2010 From: anthra.norell at bluewin.ch (Anthra Norell) Date: Mon, 18 Jan 2010 16:59:57 +0100 Subject: substitution In-Reply-To: <4B545785.8090003@bluewin.ch> References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4B545785.8090003@bluewin.ch> Message-ID: <4B54857D.407@bluewin.ch> Anthra Norell wrote: > superpollo wrote: >> hi. >> >> what is the most pythonic way to substitute substrings? >> >> eg: i want to apply: >> >> foo --> bar >> baz --> quux >> quuux --> foo >> >> so that: >> >> fooxxxbazyyyquuux --> barxxxquuxyyyfoo >> >> bye > So it goes. The more it matters, the sillier the misatakes. The method __init__ () was incomplete and __call__ () was missing, Sorry abount that. Here the whole thing again: class Translator: r""" Will translate any number of targets, handling them correctly if some overlap. Making Translator T = Translator (definitions, [eat = 1]) 'definitions' is a sequence of pairs: ((target, substitute),(t2, s2), ...) 'eat' says whether untargeted sections pass (translator) or are skipped (extractor). Makes a translator by default (eat = False) T.eat is an instance attribute that can be changed at any time. Definitions example: (('a','A'),('b','B'),('ab','ab'),('abc','xyz') # ('ab','ab') see Tricks. ('\x0c', 'page break'), ('\r\n','\n'), (' ','\t')) Order doesn't matter. Running translation = T (source) Tricks Deletion: ('target', '') Exception: (('\n',''), ('\n\n','\n\n')) # Eat LF except paragraph breaks. Exception: (('\n', '\r\n'), ('\r\n',\r\n')) # Unix to DOS, would leave DOS unchanged Translation cascade: # Rejoin text lines per paragraph Unix or DOS, inserting inter-word space if missing Mark_LF = Translator ((('\n','+LF+'),('\r\n','+LF+'),('\r\n\r\n','\r\n\r\n'),('\n\n','\n\n'))) # Pick positively identifiable mark for Unix and DOS end of lines Single_Space_Mark = Translator (((' +LF+', ' '),('+LF+', ' '),('-+LF+', ''))) no_lf_text = Single_Space_Mark (Mark_LF (text)) Translation cascade: # Nesting calls reptiles = T_latin_english (T_german_latin (reptilien)) Limitations 1. The number of substitutions and the maximum size of input depends on the respective capabilities of the Python re module. 2. Regular expressions will not work as such. Author: Frederic Rentsch (anthra.norell at bluewin.ch). """ def __init__ (self, definitions, eat = 0): ''' definitions: a sequence of pairs of strings. ((target, substitute), (t, s), ...) eat: False (0) means translate: unaffected data passes unaltered. True (1) means extract: unaffected data doesn't pass (gets eaten). Extraction filters typically require substitutes to end with some separator, else they fuse together. (E.g. ' ', '\t' or '\n') 'eat' is an attribute that can be switched anytime. ''' self.eat = eat self.compile_sequence_of_pairs (definitions) def compile_sequence_of_pairs (self, definitions): ''' Argument 'definitions' is a sequence of pairs: (('target 1', 'substitute 1'), ('t2', 's2'), ...) Order doesn't matter. ''' import re self.definitions = definitions targets, substitutes = zip (*definitions) re_targets = [re.escape (item) for item in targets] re_targets.sort (reverse = True) self.targets_set = set (targets) self.table = dict (definitions) regex_string = '|'.join (re_targets) self.regex = re.compile (regex_string, re.DOTALL) def __call__ (self, s): hits = self.regex.findall (s) nohits = self.regex.split (s) valid_hits = set (hits) & self.targets_set # Ignore targets with illegal re modifiers. if valid_hits: substitutes = [self.table [item] for item in hits if item in valid_hits] + [] # Make lengths equal for zip to work right if self.eat: return ''.join (substitutes) else: zipped = zip (nohits, substitutes) return ''.join (list (reduce (lambda a, b: a + b, [zipped][0]))) + nohits [-1] else: if self.eat: return '' else: return s From darcy at druid.net Mon Jan 18 11:03:31 2010 From: darcy at druid.net (D'Arcy J.M. Cain) Date: Mon, 18 Jan 2010 11:03:31 -0500 Subject: not and is not problem In-Reply-To: <4b54819e$0$1106$4fafbaef@reader4.news.tin.it> References: <4b54819e$0$1106$4fafbaef@reader4.news.tin.it> Message-ID: <20100118110331.49199fe8.darcy@druid.net> On Mon, 18 Jan 2010 16:43:25 +0100 superpollo wrote: > hi: > if name is not "seq": > print "WTF! WHY DOES IT PRINT?" Perhaps they aren't the same object. Some values are guaranteed to be the same (e.g. True and False) but not all. Try "!=" instead of "is not" in that experssion. -- D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner. From wkfung.eric at gmail.com Mon Jan 18 11:07:41 2010 From: wkfung.eric at gmail.com (Kit) Date: Mon, 18 Jan 2010 08:07:41 -0800 (PST) Subject: What is a list compression in Python? Message-ID: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> Hello Everyone, I am not sure if I have posted this question in a correct board. Can anyone please teach me: What is a list compression in Python? Would you mind give me some list compression examples? Thanks & really appreciate that. Kit From dreadpiratejeff at gmail.com Mon Jan 18 11:11:35 2010 From: dreadpiratejeff at gmail.com (J) Date: Mon, 18 Jan 2010 11:11:35 -0500 Subject: not and is not problem In-Reply-To: <4b54819e$0$1106$4fafbaef@reader4.news.tin.it> References: <4b54819e$0$1106$4fafbaef@reader4.news.tin.it> Message-ID: <36dec4ff1001180811h6a36c6ccm8f11626dd9ba3c55@mail.gmail.com> On Mon, Jan 18, 2010 at 10:43, superpollo wrote: > hi: > > #!/usr/bin/env python > data = "seq=123" > name , value = data.split("=") > print name > print value > if not name == "seq": > ? ?print "DOES NOT PRINT OF COURSE..." > if name is not "seq": > ? ?print "WTF! WHY DOES IT PRINT?" is will return True if two variables point to the same object, == if the objects referred to by the variables are equal. >> if not name == "seq": says if the object that the variable name points to is NOT the same as the string "seq" then do the following. Since the object that "name" points to contains the string "seq" and the string "seq" is identical in value to the "seq" in your comparison, the result is TRUE (b = a) and your if statment only proceeds if the comparison result is FALSE. >> if name is not "seq" is and is not relate to pointing to an object, not the object's contents. Example: >> name = "foo" >> name1 = name >> print name foo >> print name1 foo >> name is name1 True >> name1 is name True >> name = "bar" >> print name bar >> print name1 foo >> name is name1 False >> name is not name1 True Or even better: >> name = "foo" >> name1 = name >> id(name) 11875840 >> id(name1) 11875840 >> name = "bar" >> id(name) 11875744 >> id(name1) 11875840 >> id("foo") 11875840 >> id("bar") 11875744 shows the location for each object in relation to the name pointing to that object... -- Joan Crawford - "I, Joan Crawford, I believe in the dollar. Everything I earn, I spend." - http://www.brainyquote.com/quotes/authors/j/joan_crawford.html From utente at esempio.net Mon Jan 18 11:15:01 2010 From: utente at esempio.net (superpollo) Date: Mon, 18 Jan 2010 17:15:01 +0100 Subject: What is a list compression in Python? In-Reply-To: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> Message-ID: <4b548906$0$1113$4fafbaef@reader4.news.tin.it> Kit ha scritto: > Hello Everyone, I am not sure if I have posted this question in a > correct board. Can anyone please teach me: > > What is a list compression in Python? > > Would you mind give me some list compression examples? > > > Thanks & really appreciate that. > Kit i think that's compreHENsion... http://www.python.org/doc/2.5.4/tut/node7.html#SECTION007140000000000000000 From duncan.booth at invalid.invalid Mon Jan 18 11:20:46 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 18 Jan 2010 16:20:46 GMT Subject: substitution References: Message-ID: Adi Eyal wrote: > Using regular expressions the answer is short (and sweet) > > mapping = { > "foo" : "bar", > "baz" : "quux", > "quuux" : "foo" > } > > pattern = "(%s)" % "|".join(mapping.keys()) > repl = lambda x : mapping.get(x.group(1), x.group(1)) > s = "fooxxxbazyyyquuux" > re.subn(pattern, repl, s) I'd use a def as being IMHO clearer than the lambda but YMMV. More importantly I'd also escape the keys in case they contain any characters special to regular expressions: >>> mapping = { "foo" : "bar", "baz" : "quux", "quuux" : "foo" } >>> import re >>> pattern = "(%s)" % "|".join(re.escape(k) for k in mapping) >>> def repl(x): return mapping[x.group(1)] >>> s = "fooxxxbazyyyquuux" >>> re.subn(pattern, repl, s) ('barxxxquuxyyyfoo', 3) >>> re.sub(pattern, repl, s) 'barxxxquuxyyyfoo' >>> -- Duncan Booth http://kupuguy.blogspot.com From steve at REMOVE-THIS-cybersource.com.au Mon Jan 18 11:24:25 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 18 Jan 2010 16:24:25 GMT Subject: not and is not problem References: <4b54819e$0$1106$4fafbaef@reader4.news.tin.it> Message-ID: <00de103b$0$15570$c3e8da3@news.astraweb.com> On Mon, 18 Jan 2010 16:43:25 +0100, superpollo wrote: > hi: > > #!/usr/bin/env python > data = "seq=123" > name , value = data.split("=") > print name > print value > if not name == "seq": > print "DOES NOT PRINT OF COURSE..." > if name is not "seq": > print "WTF! WHY DOES IT PRINT?" `is` is not an alternative spelling for `==`. `is` tests for object identity, not equality. Unless you care about object identity, always use equals. >>> 'seq 1' is 'seq 1' # the SAME object is used twice True >>> s = "seq 1" # TWO objects are used >>> t = "seq 1" >>> s is t False Beware: Python caches strings that look like identifiers. This will include "seq". So, purely as an implementation-specific detail, Python will sometimes re-use the same string object, and sometimes not. Do not use `is` when you are testing for equality. By the way, instead of not name == "seq" you should write name != "seq" -- Steven From simon at brunningonline.net Mon Jan 18 11:24:54 2010 From: simon at brunningonline.net (Simon Brunning) Date: Mon, 18 Jan 2010 16:24:54 +0000 Subject: What is a list compression in Python? In-Reply-To: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> Message-ID: <8c7f10c61001180824i5cdd66d0xb8a4dc93422bb925@mail.gmail.com> 2010/1/18 Kit : > Hello Everyone, I am not sure if I have posted this question in a > correct board. Can anyone please teach me: > > What is a list compression in Python? Perhaps you mean a list comprehension? If so, see . -- Cheers, Simon B. From apt.shansen at gmail.com Mon Jan 18 11:25:24 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Mon, 18 Jan 2010 08:25:24 -0800 Subject: What is a list compression in Python? In-Reply-To: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> Message-ID: <7a9c25c21001180825ka187892s324f0887e3bee8b@mail.gmail.com> On Mon, Jan 18, 2010 at 8:07 AM, Kit wrote: > Hello Everyone, I am not sure if I have posted this question in a > correct board. Can anyone please teach me: > > What is a list compression in Python? > > Would you mind give me some list compression examples? > Do you mean list *comprehension*? If so, its a special syntax for list construct which can be used for shorter, clearer code (provided one doesn't abuse it, at which point it becomes quite obtuse). Its never required: nothing you do with a list comprehension you couldn't do with a standard loop. For example: evens = [] for n in range(100): if n %2 == 0: evens.append(n) verses: my_list = [n for n in range(100) if n % 2 == 0] The basic syntax is: [ for in ] The 'if' part at the end is optional. The syntax is converted into a for loop that builds a list, and then returns it to you. So, basically it becomes: temp = [] for in : temp.append() Except the 'temp' variable doesn't really have a name and is returned from the comprehension, where you can give it a name. Sometimes you need an 'if' clause, sometimes you don't. Sometimes your 'expression to be appended' is very simple, other times you mutate it. For example, an easy way to convert a list of numbers into a list of strings is: [str(x) for x in range(10)] HTH, --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthra.norell at bluewin.ch Mon Jan 18 11:26:45 2010 From: anthra.norell at bluewin.ch (Anthra Norell) Date: Mon, 18 Jan 2010 17:26:45 +0100 Subject: substitution In-Reply-To: <4B545785.8090003@bluewin.ch> References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4B545785.8090003@bluewin.ch> Message-ID: <4B548BC5.4010902@bluewin.ch> superpollo wrote: >> hi. >> >> what is the most pythonic way to substitute substrings? >> >> eg: i want to apply: >> >> foo --> bar >> baz --> quux >> quuux --> foo >> >> so that: >> >> fooxxxbazyyyquuux --> barxxxquuxyyyfoo >> >> bye > > Third attempt. Clearly something doesn't work right. My code gets clipped on the way up. I have to send it as an attachment. Here's again what it does: >>> substitutions = (('foo', 'bar'), ('baz', 'quux'), ('quuux', 'foo')) # Sequence of doublets >>> T = Translator (substitutions) # Compile substitutions -> translator >>> s = 'fooxxxbazyyyquuux' # Your source string >>> d = 'barxxxquuxyyyfoo' # Your destination string >>> print T (s) barxxxquuxyyyfoo >>> print T (s) == d True Code attached Regards Frederic -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: translator.py URL: From steve at REMOVE-THIS-cybersource.com.au Mon Jan 18 11:26:48 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 18 Jan 2010 16:26:48 GMT Subject: substitution References: <93831c02-6e5a-48f7-a698-3b3f1ef6e5b6@v25g2000yqk.googlegroups.com> Message-ID: <00de10c9$0$15570$c3e8da3@news.astraweb.com> On Mon, 18 Jan 2010 06:23:44 -0800, Iain King wrote: > On Jan 18, 2:17?pm, Adi Eyal wrote: [...] >> Using regular expressions the answer is short (and sweet) >> >> mapping = { >> ? ? ? ? "foo" : "bar", >> ? ? ? ? "baz" : "quux", >> ? ? ? ? "quuux" : "foo" >> >> } >> >> pattern = "(%s)" % "|".join(mapping.keys()) >> repl = lambda x : mapping.get(x.group(1), x.group(1)) >> s = "fooxxxbazyyyquuux" >> re.subn(pattern, repl, s) > > Winner! :) What are the rules for being declared "Winner"? For the simple case given, calling s.replace three times is much faster: more than twice as fast. But a bigger problem is that the above "winner" may not work correctly if there are conflicts between the target strings (e.g. 'a'->'X', 'aa'->'Y'). The problem is that the result you get depends on the order of the searches, BUT as given, that order is non-deterministic. dict.keys() returns in an arbitrary order, which means the caller can't specify the order except by accident. For example: >>> repl = lambda x : m[x.group(1)] >>> m = {'aa': 'Y', 'a': 'X'} >>> pattern = "(%s)" % "|".join(m.keys()) >>> subn(pattern, repl, 'aaa') # expecting 'YX' ('XXX', 3) The result that you get using this method will be consistent but arbitrary and unpredictable. For those who care, here's my timing code: from timeit import Timer setup = """ mapping = {"foo" : "bar", "baz" : "quux", "quuux" : "foo"} pattern = "(%s)" % "|".join(mapping.keys()) repl = lambda x : mapping.get(x.group(1), x.group(1)) repl = lambda x : mapping[x.group(1)] s = "fooxxxbazyyyquuux" from re import subn """ t1 = Timer("subn(pattern, repl, s)", setup) t2 = Timer( "s.replace('foo', 'bar').replace('baz', 'quux').replace('quuux', 'foo')", "s = 'fooxxxbazyyyquuux'") And the results on my PC: >>> min(t1.repeat(number=100000)) 1.1273870468139648 >>> min(t2.repeat(number=100000)) 0.49491715431213379 -- Steven From steve at REMOVE-THIS-cybersource.com.au Mon Jan 18 11:30:22 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 18 Jan 2010 16:30:22 GMT Subject: What is a list compression in Python? References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> Message-ID: <00de11a0$0$15570$c3e8da3@news.astraweb.com> On Mon, 18 Jan 2010 08:07:41 -0800, Kit wrote: > Hello Everyone, I am not sure if I have posted this question in a > correct board. Can anyone please teach me: > > What is a list compression in Python? Google "python list comprehension". If Google is broken for you, try Yahoo, or any other search engine. > Would you mind give me some list compression examples? Instead of this: L = [] for x in range(10): L.append(x**2) you can write: L = [x**2 for x in range(10)] Instead of this example: L = [] for x in range(10): if x % 2 == 0: L.append(x**2) you can write: L = [x**2 for x in range(10) if x % 2 == 0] -- Steven From holger at merlinux.eu Mon Jan 18 11:36:52 2010 From: holger at merlinux.eu (holger krekel) Date: Mon, 18 Jan 2010 17:36:52 +0100 Subject: py.test-1.2.0: junitxml, standalone test scripts, pluginization Message-ID: <20100118163652.GB31033@trillke.net> Hi all, i just released some bits related to automated testing with Python: py-1.2.0: py.test core which grew junitxml, standalone-script generation pytest-xdist-1.0: separately installable dist-testing & looponfailing plugin pytest-figleaf-1.0: separately installable figleaf-coverage testing plugin See below or at this URL for the announcement: http://pylib.org/announce/release-1.2.0.html If you didn't experience much speed-up or previously had problems with distributed testing i recommend you try to install "pytest-xdist" now and see if it works better. For me it speeds up some tests runs by 500% on a 4 CPU machine due to its better internal model and several fixes. (It's five times because several tests depend on IO and don't block CPU meanwhile). Another tip: if you use "pip" (best with a virtualenv) you can do e.g.: pip install pytest-xdist pip uninstall pytest-xdist to conveniently activate/deactivate plugins for py.test. easy_install works ok as well but has no uninstall, yet remains the only option for installing with Python3 at the moment, though. You need to use the fine 'distribute' project's easy_install for the latter. cheers & have fun, holger py.test/pylib 1.2.0: junitxml, standalone test scripts, pluginization -------------------------------------------------------------------------------- py.test is an advanced automated testing tool working with Python2, Python3 and Jython versions on all major operating systems. It has a simple plugin architecture and can run many existing common Python test suites without modification. It offers some unique features not found in other testing tools. See http://pytest.org for more info. py.test 1.2.0 brings many bug fixes and interesting new abilities: * --junitxml=path will create an XML file for use with CI processing * --genscript=path creates a standalone py.test-equivalent test-script * --ignore=path prevents collection of anything below that path * --confcutdir=path only lookup conftest.py test configs below that path * a 'pytest_report_header' hook to add info to the terminal report header * a 'pytestconfig' function argument gives direct access to option values * 'pytest_generate_tests' can now be put into a class as well * on CPython py.test additionally installs as "py.test-VERSION", on Jython as py.test-jython and on PyPy as py.test-pypy-XYZ Apart from many bug fixes 1.2.0 also has better pluginization: Distributed testing and looponfailing testing now live in the separately installable 'pytest-xdist' plugin. The same is true for 'pytest-figleaf' for doing coverage reporting. Those two plugins can serve well now as blue prints for doing your own. thanks to all who helped and gave feedback, have fun, holger krekel, January 2010 Changes between 1.2.0 and 1.1.1 ===================================== - moved dist/looponfailing from py.test core into a new separately released pytest-xdist plugin. - new junitxml plugin: --junitxml=path will generate a junit style xml file which is processable e.g. by the Hudson CI system. - new option: --genscript=path will generate a standalone py.test script which will not need any libraries installed. thanks to Ralf Schmitt. - new option: --ignore will prevent specified path from collection. Can be specified multiple times. - new option: --confcutdir=dir will make py.test only consider conftest files that are relative to the specified dir. - new funcarg: "pytestconfig" is the pytest config object for access to command line args and can now be easily used in a test. - install 'py.test' and `py.which` with a ``-$VERSION`` suffix to disambiguate between Python3, python2.X, Jython and PyPy installed versions. - new "pytestconfig" funcarg allows access to test config object - new "pytest_report_header" hook can return additional lines to be displayed at the header of a test run. - (experimental) allow "py.test path::name1::name2::..." for pointing to a test within a test collection directly. This might eventually evolve as a full substitute to "-k" specifications. - streamlined plugin loading: order is now as documented in customize.html: setuptools, ENV, commandline, conftest. also setuptools entry point names are turned to canonical namees ("pytest_*") - automatically skip tests that need 'capfd' but have no os.dup - allow pytest_generate_tests to be defined in classes as well - deprecate usage of 'disabled' attribute in favour of pytestmark - deprecate definition of Directory, Module, Class and Function nodes in conftest.py files. Use pytest collect hooks instead. - collection/item node specific runtest/collect hooks are only called exactly on matching conftest.py files, i.e. ones which are exactly below the filesystem path of an item - change: the first pytest_collect_directory hook to return something will now prevent further hooks to be called. - change: figleaf plugin now requires --figleaf to run. Also change its long command line options to be a bit shorter (see py.test -h). - change: pytest doctest plugin is now enabled by default and has a new option --doctest-glob to set a pattern for file matches. - change: remove internal py._* helper vars, only keep py._pydir - robustify capturing to survive if custom pytest_runtest_setup code failed and prevented the capturing setup code from running. - make py.test.* helpers provided by default plugins visible early - works transparently both for pydoc and for interactive sessions which will regularly see e.g. py.test.mark and py.test.importorskip. - simplify internal plugin manager machinery - simplify internal collection tree by introducing a RootCollector node - fix assert reinterpreation that sees a call containing "keyword=..." - fix issue66: invoke pytest_sessionstart and pytest_sessionfinish hooks on slaves during dist-testing, report module/session teardown hooks correctly. - fix issue65: properly handle dist-testing if no execnet/py lib installed remotely. - skip some install-tests if no execnet is available - fix docs, fix internal bin/ script generation -- ----- End forwarded message ----- -- From invalid at invalid.invalid Mon Jan 18 11:37:22 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Mon, 18 Jan 2010 16:37:22 +0000 (UTC) Subject: Arrrrgh! Another module broken References: Message-ID: On 2010-01-18, Jive Dadson wrote: > I just found another module that broke when I went to 2.6. Gnuplot. > Apparently one of its routines has a parameter named "with." That used > to be okay, and now it's not. I remember seeing depreicated warnings about that _years_ ago, and I would have sworn it's been fixed for at least a couple years. -- Grant From steve at REMOVE-THIS-cybersource.com.au Mon Jan 18 11:37:29 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 18 Jan 2010 16:37:29 GMT Subject: Is python not good enough? References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> Message-ID: <00de134b$0$15570$c3e8da3@news.astraweb.com> On Mon, 18 Jan 2010 03:03:26 -0800, Phlip wrote: > On Jan 12, 7:09?am, ikuta liu wrote: > >> Go language try to merge low level, hight level and browser language. > > Go uses := for assignment. > > This means, to appease the self-righteous indignation of the math > professor who would claim = should mean "equality"... > > ...you gotta type a shift and 2 characters for a very common operator. I doubt it has anything to do with "the math professor". Any maths professor will tell you that, in mathematics, = is used for both assignment and equality, since in maths they are the same thing. And besides, equality testing is no less common than assignment. To appease the "self-righteous indignation of the C coders", we have to type == instead of = for a very common operator. No matter what convention you use, you're going to upset some group of people. Seriously, I programmed in Pascal for many years, and typing := for assignment is not a burden. -- Steven From steve at REMOVE-THIS-cybersource.com.au Mon Jan 18 11:39:40 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 18 Jan 2010 16:39:40 GMT Subject: Is python not good enough? References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> <37f8195f-3f02-4a13-b677-6b5d5e1589e8@22g2000yqr.googlegroups.com> <1e692369-cee9-4e63-8dce-33d70bf96368@p24g2000yqm.googlegroups.com> Message-ID: <00de13ce$0$15570$c3e8da3@news.astraweb.com> On Mon, 18 Jan 2010 07:37:36 -0800, Phlip wrote: > They fixed the hideous redundancy of Java without the ill-defined scope > issues of Python Which ill-defined scope issues are you referring to? -- Steven From jonathan.slenders at gmail.com Mon Jan 18 11:44:41 2010 From: jonathan.slenders at gmail.com (Jonathan S) Date: Mon, 18 Jan 2010 08:44:41 -0800 (PST) Subject: Python decorator syntax limitations Message-ID: Hi all, The following is what I want to do, but this results in a syntax error: @news_page('template.html').lookup(News, 'news_id', 'news') def view(request, group, news): pass What does work is the equivalent old way of doing decorating: def view(request, group, news): pass view = news_page('template.html').lookup(News, 'news_id', 'news') (view) Any suggestions? I have my reasons for doing this, (news_page is a class, and __call__ is used to wrap the template.) I'm sure this is a limitation in the syntax, but would parenthesis somewhere help? Thanks! Jonathan From iainking at gmail.com Mon Jan 18 11:46:45 2010 From: iainking at gmail.com (Iain King) Date: Mon, 18 Jan 2010 08:46:45 -0800 (PST) Subject: substitution References: <93831c02-6e5a-48f7-a698-3b3f1ef6e5b6@v25g2000yqk.googlegroups.com> <00de10c9$0$15570$c3e8da3@news.astraweb.com> Message-ID: <2fbd0038-7e19-4d22-862a-9e60b6c04aa0@22g2000yqr.googlegroups.com> On Jan 18, 4:26?pm, Steven D'Aprano wrote: > On Mon, 18 Jan 2010 06:23:44 -0800, Iain King wrote: > > On Jan 18, 2:17?pm, Adi Eyal wrote: > [...] > >> Using regular expressions the answer is short (and sweet) > > >> mapping = { > >> ? ? ? ? "foo" : "bar", > >> ? ? ? ? "baz" : "quux", > >> ? ? ? ? "quuux" : "foo" > > >> } > > >> pattern = "(%s)" % "|".join(mapping.keys()) > >> repl = lambda x : mapping.get(x.group(1), x.group(1)) > >> s = "fooxxxbazyyyquuux" > >> re.subn(pattern, repl, s) > > > Winner! :) > > What are the rules for being declared "Winner"? For the simple case > given, calling s.replace three times is much faster: more than twice as > fast. > > But a bigger problem is that the above "winner" may not work correctly if > there are conflicts between the target strings (e.g. 'a'->'X', > 'aa'->'Y'). The problem is that the result you get depends on the order > of the searches, BUT as given, that order is non-deterministic. > dict.keys() returns in an arbitrary order, which means the caller can't > specify the order except by accident. For example: > > >>> repl = lambda x : m[x.group(1)] > >>> m = {'aa': 'Y', 'a': 'X'} > >>> pattern = "(%s)" % "|".join(m.keys()) > >>> subn(pattern, repl, 'aaa') ?# expecting 'YX' > > ('XXX', 3) > > The result that you get using this method will be consistent but > arbitrary and unpredictable. > > For those who care, here's my timing code: > > from timeit import Timer > > setup = """ > mapping = {"foo" : "bar", "baz" : "quux", "quuux" : "foo"} > pattern = "(%s)" % "|".join(mapping.keys()) > repl = lambda x : mapping.get(x.group(1), x.group(1)) > repl = lambda x : mapping[x.group(1)] > s = "fooxxxbazyyyquuux" > from re import subn > """ > > t1 = Timer("subn(pattern, repl, s)", setup) > t2 = Timer( > "s.replace('foo', 'bar').replace('baz', 'quux').replace('quuux', 'foo')", > "s = 'fooxxxbazyyyquuux'") > > And the results on my PC: > > >>> min(t1.repeat(number=100000)) > 1.1273870468139648 > >>> min(t2.repeat(number=100000)) > > 0.49491715431213379 > > -- > Steven Adi elicited that response from me because his solution was vastly more succinct than everything else that had appeared up til that point while still meeting the OP's requirements. The OP never cared about overlap between 2 'find' strings, just between the 'find' and 'replace' strings (though I did take it into account in my second post for the sake of completeness). His code could have been a little cleaner, I'd have trimmed it to: mapping = {"foo": "bar", "baz": "quux", "quuux": "foo"} pattern = "(%s)" % "|".join(mapping) repl = lambda x : mapping[x.group(1)] s = "fooxxxbazyyyquuux" re.subn(pattern, repl, s) but apart from that was very pythonic: explicit, succinct, and all the heavy work is done by the module (i.e. in compiled c code in the majority case of CPython). It can be 'upgraded' to cover the find- find overlap if you really want (I *believe* regexps will match the leftmost option in a group first): subs = [("foo", "bar"), ("baz", "quux"), ("quuux", "foo")] pattern = "(%s)" % "|".join((x[0] for x in subs)) mapping = dict(subs) repl = lambda x : mapping[x.group(1)] s = "fooxxxbazyyyquuux" re.subn(pattern, repl, s) Anyway, there's no prize for winning, but by all means: if you think someone else's code and not a variation on this should win for most pythonic, then make your nomination :) Iain From showell30 at yahoo.com Mon Jan 18 12:00:44 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 18 Jan 2010 09:00:44 -0800 (PST) Subject: Python decorator syntax limitations References: Message-ID: On Jan 18, 8:44?am, Jonathan S wrote: > Hi all, > The following is what I want to do, but this results in a syntax > error: > > @news_page('template.html').lookup(News, 'news_id', 'news') > def view(request, group, news): > ? ? pass > > What does work is the equivalent old way of doing decorating: > > def view(request, group, news): > ? ? pass > view = news_page('template.html').lookup(News, 'news_id', 'news') > (view) > > Any suggestions? I have my reasons for doing this, (news_page is a > class, and __call__ is used to wrap the template.) > I'm sure this is a limitation in the syntax, but would parenthesis > somewhere help? You might want to consider doing something like this: def lookup_view(page_class, template, class_, id_, label): def decorator(func): return page_class(template).lookup(class_, id_, label) (func) return decorator @lookup_view(news_page, 'template.html', News, 'news_id', 'news') def view(...): pass From jeanmichel at sequans.com Mon Jan 18 12:05:37 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 18 Jan 2010 18:05:37 +0100 Subject: not and is not problem In-Reply-To: <00de103b$0$15570$c3e8da3@news.astraweb.com> References: <4b54819e$0$1106$4fafbaef@reader4.news.tin.it> <00de103b$0$15570$c3e8da3@news.astraweb.com> Message-ID: <4B5494E1.4080205@sequans.com> Steven D'Aprano wrote: > On Mon, 18 Jan 2010 16:43:25 +0100, superpollo wrote: > > >> hi: >> >> #!/usr/bin/env python >> data = "seq=123" >> name , value = data.split("=") >> print name >> print value >> if not name == "seq": >> print "DOES NOT PRINT OF COURSE..." >> if name is not "seq": >> print "WTF! WHY DOES IT PRINT?" >> > [snip] > Beware: Python caches strings that look like identifiers. This will > include "seq". So, purely as an implementation-specific detail, Python > will sometimes re-use the same string object, and sometimes not. > This is the most important point to understand. Some people use equality where in fact they should test for identity, because of this 'cache' effect Steven described that makes you feel that the 'is' operator is not reliable. >>>'seq' is 'seq' True >>>name = 'seq' >>>name is 'seq' True >>>name, _ = 'seq _'.split() >>>name is 'seq' False In your case, what you want is equality, because you don't care if name and 'seq' are the same object in memory, there's a good chance they are not in fact (see last example). A simple example to illustrate the difference: >>>class Foo: >>> def __eq__(self, any): >>> """Foo equals everything." >>> return True >>>f = Foo() >>>f == None True >>>f is None False f equals None, but f is not None. JM From lie.1296 at gmail.com Mon Jan 18 12:25:11 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Tue, 19 Jan 2010 04:25:11 +1100 Subject: Python decorator syntax limitations In-Reply-To: References: Message-ID: <4b54998b$1@dnews.tpgi.com.au> On 01/19/10 03:44, Jonathan S wrote: > Any suggestions? I have my reasons for doing this, (news_page is a > class, and __call__ is used to wrap the template.) > I'm sure this is a limitation in the syntax, but would parenthesis > somewhere help? The restriction[1] is put in there since Guido has a "gut feeling" that allowing arbitrary expression in decorator syntax severely harms readability. http://mail.python.org/pipermail/python-dev/2004-August/046711.html A workaround is to put an assignment above it: nplookup = news_page('template.html').lookup @nplookup(News, 'news_id', 'news') def view(request, group, news): pass If you are sure you can put up a convincing argument for lifting this restriction, and you are willing to put some time arguing, you are welcome to start a thread in the python-dev mailing list. Be sure to read about previous discussions, as repeating arguments wouldn't change anything. [1] there is a subtle difference between "restriction" and "limitation" From gabriele.modena at gmail.com Mon Jan 18 12:25:20 2010 From: gabriele.modena at gmail.com (Gabriele Modena) Date: Mon, 18 Jan 2010 18:25:20 +0100 Subject: ctypes: nested structures and pointers In-Reply-To: <1fe1d5d61001180133gb27a01dsb7434087909aff0@mail.gmail.com> References: <1fe1d5d61001180133gb27a01dsb7434087909aff0@mail.gmail.com> Message-ID: <1fe1d5d61001180925gcad560ep1bdc82866c19bd29@mail.gmail.com> On Mon, Jan 18, 2010 at 10:33 AM, Gabriele Modena wrote: > ?1. what is the correct (pythonic) way to capture the prototype > definition of dev_callbacks and the relation between that structure > and dev_info? > > ?2. is it correct to wrap "connect", "transceive" and "disconnect" in > separate structures and reference them within ?pyDEV_CALLBACKS? Solved. The answer was in the doc: http://docs.python.org/library/ctypes.html#structured-data-types (_fields_ & _pack_). From solipsis at pitrou.net Mon Jan 18 12:42:53 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 18 Jan 2010 17:42:53 +0000 (UTC) Subject: Generic Python Benchmark suite? References: <4b543838$0$7618$9b4e6d93@newsspool1.arcor-online.net> Message-ID: Le Mon, 18 Jan 2010 11:30:16 +0100, Stefan Behnel a ?crit?: > Anand Vaidya, 18.01.2010 10:58: >> Is there a generic python benchmark suite in active development? [...] >> PS: I think a benchmark should cover file / network, database I/O, >> data structures (dict, list etc), object creation/manipulation, >> numbers, measure looping inefficiencies, effects of caching (memcache >> etc) at the minimum > > That doesn't sound generic at all. Maybe you should prefer an > application based benchmark instead. Actually, it sounds much more generic that most of the benchmarks we usually rely on :-) From __peter__ at web.de Mon Jan 18 12:46:08 2010 From: __peter__ at web.de (Peter Otten) Date: Mon, 18 Jan 2010 18:46:08 +0100 Subject: Python decorator syntax limitations References: Message-ID: Jonathan S wrote: > Hi all, > The following is what I want to do, but this results in a syntax > error: > > > @news_page('template.html').lookup(News, 'news_id', 'news') > def view(request, group, news): > pass > > > What does work is the equivalent old way of doing decorating: > > > def view(request, group, news): > pass > view = news_page('template.html').lookup(News, 'news_id', 'news') > (view) > > > > Any suggestions? I have my reasons for doing this, (news_page is a > class, and __call__ is used to wrap the template.) > I'm sure this is a limitation in the syntax, but would parenthesis > somewhere help? @apply(lambda:news_page('template.html').lookup(News, 'news_id', 'news')) def view(request, group, news): pass But I'd go with a dedicated helper function. Peter From solipsis at pitrou.net Mon Jan 18 12:47:40 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 18 Jan 2010 17:47:40 +0000 (UTC) Subject: Generic Python Benchmark suite? References: Message-ID: Le Mon, 18 Jan 2010 01:58:42 -0800, Anand Vaidya a ?crit?: > Is there a generic python benchmark suite in active development? I am > looking forward to comparing some code on various python implementations > (primarily CPython 2.x, CPython 3.x, UnladenSwallow, Psyco). > > I am happy with something that gives me a relative number eg: ULS is 30% > faster than CPy2.x etc > > I found pybench which is probably not maintained actively. pybench is not abandoned. However, it only gets an addition now and then. More importantly, it's a set of low-level microbenchmarks designed to stress the execution cost of certain primitives. It won't give you the answer to any high-level questions. In the SVN sandbox (*) you'll find a couple of other benchmarks: - stringbench, stressing string operations - iobench, stressing common file I/O operations - ccbench, trying to measure interpreter efficiency in the face of multi- threaded workloads (*) http://svn.python.org/view/sandbox/trunk/ Regards Antoine. From jeanmichel at sequans.com Mon Jan 18 12:54:06 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 18 Jan 2010 18:54:06 +0100 Subject: using super In-Reply-To: References: <0361ea2d$0$1336$c3e8da3@news.astraweb.com> Message-ID: <4B54A03E.6010702@sequans.com> >>> class SubClass(Base): >>> colour = "Red" >>> def parrot(self): >>> """docstring for Subclass""" >>> return super(Subclass, self).parrot() I'm not a big fan of super, but I'm still wondering if >>> return super(self.__class__, self).parrot() would have made it. What if Subclass has more than one base class ? JM From solipsis at pitrou.net Mon Jan 18 12:55:41 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 18 Jan 2010 17:55:41 +0000 (UTC) Subject: python gui ide under linux..like visual studio ;) ? References: <4b5470ce$0$1103$4fafbaef@reader4.news.tin.it> Message-ID: Le Mon, 18 Jan 2010 15:32:36 +0100, ted a ?crit?: > > And, a good library for access to database (mysql, sql server, oracle) ? If you want something high-level: http://www.sqlalchemy.org/ You won't regret it :) From duncan.booth at invalid.invalid Mon Jan 18 13:11:14 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 18 Jan 2010 18:11:14 GMT Subject: using super References: Message-ID: Jean-Michel Pichavant wrote: > >>>> class SubClass(Base): >>>> colour = "Red" >>>> def parrot(self): >>>> """docstring for Subclass""" >>>> return super(Subclass, self).parrot() > I'm not a big fan of super, but I'm still wondering if > > >>> return super(self.__class__, self).parrot() > > would have made it. No, it wouldn't. > What if Subclass has more than one base class ? super() will work in that case provided you specify the current class. It never works correctly if you specify the class of self. Consider another level of subclasses and it may be clearer: >>> class Base(object): def parrot(self): print "Base.parrot" >>> class SubClass(Base): def parrot(self): print "SubClass.parrot" return super(SubClass, self).parrot() >>> class SubSubClass(SubClass): def parrot(self): print "SubSubClass.parrot" return super(SubSubClass, self).parrot() >>> SubSubClass().parrot() SubSubClass.parrot SubClass.parrot Base.parrot >>> class SubClass(Base): def parrot(self): print "SubClass.parrot" return super(self.__class__, self).parrot() >>> class SubSubClass(SubClass): def parrot(self): print "SubSubClass.parrot" return super(self.__class__, self).parrot() >>> SubSubClass().parrot() SubClass.parrot SubClass.parrot SubClass.parrot SubClass.parrot SubClass.parrot SubClass.parrot SubClass.parrot SubClass.parrot SubClass.parrot ... (you're in an infinite loop now) ... From python at mrabarnett.plus.com Mon Jan 18 14:00:09 2010 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 18 Jan 2010 19:00:09 +0000 Subject: Is python not good enough? In-Reply-To: <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> <7fc69a90-0ac1-451f-a403-95d4f8de2a49@o28g2000yqh.googlegroups.com> Message-ID: <4B54AFB9.6000905@mrabarnett.plus.com> Phlip wrote: > On Jan 12, 7:09 am, ikuta liu wrote: > >> Go language try to merge low level, hight level and browser language. > > Go uses := for assignment. > > This means, to appease the self-righteous indignation of the math > professor who would claim = should mean "equality"... > > ...you gotta type a shift and 2 characters for a very common operator. > > Pass! > If I were going to list what I didn't like about Go, that wouldn't be one of them! From python at mrabarnett.plus.com Mon Jan 18 14:04:08 2010 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 18 Jan 2010 19:04:08 +0000 Subject: basic Class in Python In-Reply-To: References: <86a6a8c8-49fe-4a3d-81a0-5a44a179c7d6@o28g2000yqh.googlegroups.com> Message-ID: <4B54B0A8.9040706@mrabarnett.plus.com> bartc wrote: > > "Wolfgang Rohdewald" wrote in message > news:mailman.1056.1263771299.28905.python-list at python.org... >> On Monday 18 January 2010, BarryJOgorman wrote: >>> TypeError: object._new_() takes no parameters >> >>> def _init_(self, name, job=None, pay=0): >> >> __init__ needs two underscores left and right > > Any particular reason why two, and not one (or three)? In some fonts > it's difficult to tell how many as they run together. > I believe it was borrowed from C, which has __FILE__ and __LINE__ (CPython is written in C). With hindsight, of course, single underscores would've been sufficient... From aahz at pythoncraft.com Mon Jan 18 14:15:48 2010 From: aahz at pythoncraft.com (Aahz) Date: 18 Jan 2010 11:15:48 -0800 Subject: Python decorator syntax limitations References: <4b54998b$1@dnews.tpgi.com.au> Message-ID: In article <4b54998b$1 at dnews.tpgi.com.au>, Lie Ryan wrote: > >If you are sure you can put up a convincing argument for lifting this >restriction, and you are willing to put some time arguing, you are >welcome to start a thread in the python-dev mailing list. Be sure to >read about previous discussions, as repeating arguments wouldn't change >anything. While I haven't been reading python-dev in recent months, I'm pretty sure the moratorium makes the only appropriate place for this python-ideas, and even there people are not likely to invest much time: http://www.python.org/dev/peps/pep-3003/ -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." --Red Adair From adi at digitaltrowel.com Mon Jan 18 14:24:24 2010 From: adi at digitaltrowel.com (Adi Eyal) Date: Mon, 18 Jan 2010 21:24:24 +0200 Subject: substitution Message-ID: > From:?Steven D'Aprano > To:?python-list at python.org > Date:?18 Jan 2010 16:26:48 GMT > Subject:?Re: substitution > On Mon, 18 Jan 2010 06:23:44 -0800, Iain King wrote: > >> On Jan 18, 2:17?pm, Adi Eyal wrote: > [...] >>> Using regular expressions the answer is short (and sweet) >>> >>> mapping = { >>> ? ? ? ? "foo" : "bar", >>> ? ? ? ? "baz" : "quux", >>> ? ? ? ? "quuux" : "foo" >>> >>> } >>> >>> pattern = "(%s)" % "|".join(mapping.keys()) >>> repl = lambda x : mapping.get(x.group(1), x.group(1)) >>> s = "fooxxxbazyyyquuux" >>> re.subn(pattern, repl, s) >> >> Winner! :) > > What are the rules for being declared "Winner"? For the simple case > given, calling s.replace three times is much faster: more than twice as > fast. :) - The solution above is short and easy to read and digest. It is also easy to maintain - you simply need to add additional entries to the mapping dict if you need to add additional strings. Calling replace 3 times in a row doesn't work correctly e.g. foo => bar bar => baz foo.replace("foo", "bar").replace("bar", "baz") > > But a bigger problem is that the above "winner" may not work correctly if > there are conflicts between the target strings (e.g. 'a'->'X', > 'aa'->'Y'). The problem is that the result you get depends on the order > of the searches, BUT as given, that order is non-deterministic. > dict.keys() returns in an arbitrary order, which means the caller can't > specify the order except by accident. For example: > >>>> repl = lambda x : m[x.group(1)] >>>> m = {'aa': 'Y', 'a': 'X'} >>>> pattern = "(%s)" % "|".join(m.keys()) >>>> subn(pattern, repl, 'aaa') ?# expecting 'YX' > ('XXX', 3) > > The result that you get using this method will be consistent but > arbitrary and unpredictable. > That's a simple fix import re mapping = { "foo" : "bar", "baz" : "quux", "quuux" : "foo" } keys = [(len(key), key) for key in mapping.keys()] keys.sort(reverse=True) keys = [key for (_, key) in keys] pattern = "(%s)" % "|".join(keys) repl = lambda x : mapping[x.group(1)] s = "fooxxxbazyyyquuux" re.subn(pattern, repl, s) With regards to timing - I suspect that as the input string grows, the ratio of the regexp solution to the string solution will grow. If the map grows, the regexp solution seems faster (see below). Python's regular expression state machine is very inefficient with lots of disjunctions since it involves a lot of backtracking - this can be hand optimised for a much greater speed improvement. N = 1000 alphabet = "abcdefghijklmnopqrstuvwxyz" make_token = lambda : "".join(sample(alphabet, 5)) mapping = dict([(make_token(), make_token()) for i in range(N)]) keys = [(len(key), key) for key in mapping.keys()] keys.sort(reverse=True) keys = [key for (_, key) in keys] pattern = "(%s)" % "|".join(keys) replace_strs = ["replace('%s', '%s')" % (key, mapping[key]) for key in keys] command = "s." + ".".join(replace_strs) setup = """ import re regexp = re.compile("%s") repl = lambda x : mapping[x.group(1)] s = "fooxxxbazyyyquuux" """ % pattern t1 = Timer("regexp.subn(repl, s)", setup) t2 = Timer(command, "s = 'fooxxxbazyyyquuux'") res1 = sum(t1.repeat(number=1000)) res2 = sum(t2.repeat(number=1000)) print res1 / res2 on my machine this comes to >>> 0.316323109737 From arnodel at googlemail.com Mon Jan 18 14:31:39 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 18 Jan 2010 19:31:39 +0000 Subject: Python decorator syntax limitations References: <4b54998b$1@dnews.tpgi.com.au> Message-ID: aahz at pythoncraft.com (Aahz) writes: > In article <4b54998b$1 at dnews.tpgi.com.au>, > Lie Ryan wrote: >> >>If you are sure you can put up a convincing argument for lifting this >>restriction, and you are willing to put some time arguing, you are >>welcome to start a thread in the python-dev mailing list. Be sure to >>read about previous discussions, as repeating arguments wouldn't change >>anything. > > While I haven't been reading python-dev in recent months, I'm pretty sure > the moratorium makes the only appropriate place for this python-ideas, > and even there people are not likely to invest much time: > > http://www.python.org/dev/peps/pep-3003/ In actual fact there is a recent discussion of this on python-ideas: http://groups.google.co.uk/group/python-ideas/browse_thread/thread/1eebf486969c39a1/ -- Arnaud From aahz at pythoncraft.com Mon Jan 18 14:45:59 2010 From: aahz at pythoncraft.com (Aahz) Date: 18 Jan 2010 11:45:59 -0800 Subject: lightweight encryption of text file References: <0357cddd$0$1336$c3e8da3@news.astraweb.com> Message-ID: In article , Daniel Fetchinson wrote: > >Well, that's sort of true about learning a complex API :) But it's >also true that I'm not storing anything really valuable in the file >but still wouldn't want to leave it lying around in plain text. In >case I lose the laptop with the file I seriously doubt anybody who >finds it will go through each and every file and try to find what's in >it, even though they look like data files and there is no hint what so >ever that any one of them contains encrypted info. If they see a text >file, well, that can give them ideas, so let's encrypt a little bit. One reason I like OSX is that it's dirt-simple to encrypt my home dir. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." --Red Adair From jeanmichel at sequans.com Mon Jan 18 14:50:45 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 18 Jan 2010 20:50:45 +0100 Subject: using super In-Reply-To: References: Message-ID: <4B54BB95.1090407@sequans.com> Duncan Booth wrote: > Jean-Michel Pichavant wrote: > > >>>>> class SubClass(Base): >>>>> colour = "Red" >>>>> def parrot(self): >>>>> """docstring for Subclass""" >>>>> return super(Subclass, self).parrot() >>>>> >> I'm not a big fan of super, but I'm still wondering if >> >> >>>>> return super(self.__class__, self).parrot() >>>>> >> would have made it. >> > > No, it wouldn't. > > >> What if Subclass has more than one base class ? >> > > super() will work in that case provided you specify the current class. It > never works correctly if you specify the class of self. > > Consider another level of subclasses and it may be clearer: > > >>>> class Base(object): >>>> > def parrot(self): > print "Base.parrot" > > > >>>> class SubClass(Base): >>>> > def parrot(self): > print "SubClass.parrot" > return super(SubClass, self).parrot() > > > >>>> class SubSubClass(SubClass): >>>> > def parrot(self): > print "SubSubClass.parrot" > return super(SubSubClass, self).parrot() > > > >>>> SubSubClass().parrot() >>>> > SubSubClass.parrot > SubClass.parrot > Base.parrot > > >>>> class SubClass(Base): >>>> > def parrot(self): > print "SubClass.parrot" > return super(self.__class__, self).parrot() > > > >>>> class SubSubClass(SubClass): >>>> > def parrot(self): > print "SubSubClass.parrot" > return super(self.__class__, self).parrot() > > >>>> SubSubClass().parrot() >>>> > SubClass.parrot > SubClass.parrot > SubClass.parrot > SubClass.parrot > SubClass.parrot > SubClass.parrot > SubClass.parrot > SubClass.parrot > SubClass.parrot > ... (you're in an infinite loop now) ... > I see. Then is there a reason why >>> return super(Subclass, self).parrot() would be prefered over the "classic" >>> return Base.parrot(self) ? Or is it just a matter of preference ? JM From vicente.soler at gmail.com Mon Jan 18 15:02:51 2010 From: vicente.soler at gmail.com (vsoler) Date: Mon, 18 Jan 2010 12:02:51 -0800 (PST) Subject: Unable to install numpy Message-ID: <3a772f49-aee6-4450-b60e-8f05daec136a@21g2000yqj.googlegroups.com> Hi all, I just download Numpy, and tried to install it using "numpy-1.4.0- win32-superpack-python2.6.exe" I get an error: "Python version 2.6 required, which was not found in the Registry" However, I am using Python 2.6 every day. I'm running Windows 7. What can I do? From robert.kern at gmail.com Mon Jan 18 15:08:41 2010 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 18 Jan 2010 14:08:41 -0600 Subject: Unable to install numpy In-Reply-To: <3a772f49-aee6-4450-b60e-8f05daec136a@21g2000yqj.googlegroups.com> References: <3a772f49-aee6-4450-b60e-8f05daec136a@21g2000yqj.googlegroups.com> Message-ID: On 2010-01-18 14:02 PM, vsoler wrote: > Hi all, > > I just download Numpy, and tried to install it using "numpy-1.4.0- > win32-superpack-python2.6.exe" > > I get an error: "Python version 2.6 required, which was not found in > the Registry" > > However, I am using Python 2.6 every day. I'm running Windows 7. > > What can I do? Please ask numpy installation questions on the numpy mailing list. http://www.scipy.org/Mailing_Lists -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From arnodel at googlemail.com Mon Jan 18 15:19:40 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 18 Jan 2010 20:19:40 +0000 Subject: using super References: Message-ID: Jean-Michel Pichavant writes: [...] > Then is there a reason why >>>> return super(Subclass, self).parrot() > would be prefered over the "classic" >>>> return Base.parrot(self) > ? > > Or is it just a matter of preference ? Using super() calls the next method in the class's Method Resolution Order (mro - to see it on class A, use A.mro()). This guarantees that ancestor methods won't be called twice when the inheritance graph is not a tree. Here is a (minimal?) example. >>> class A(object): ... def foo(self): print 'A' ... >>> class B(A): ... def foo(self): ... super(B, self).foo() ... print 'B' ... >>> class C(A): ... def foo(self): ... super(C, self).foo() ... print 'C' ... >>> class D(B, C): ... def foo(self): ... super(D, self).foo() ... print 'D' ... >>> d = D() >>> d.foo() A C B D >>> D.mro() [, , , , ] The reason why super() may be necessary is that if an object is an instance of say class C, its method resolution order above class C is not known at compile time. HTH -- Arnaud From joaopcf at gmail.com Mon Jan 18 15:50:01 2010 From: joaopcf at gmail.com (=?ISO-8859-1?Q?Jo=E3o?=) Date: Mon, 18 Jan 2010 12:50:01 -0800 (PST) Subject: force URLencoding script References: <6bc42c49-e26c-4f7b-9538-dd2e319accfb@26g2000yqo.googlegroups.com> <9bc83518-b641-47a9-84ef-c68c9f825248@26g2000yqo.googlegroups.com> <4c68f05e-1fde-4e39-bb8a-803aa32de201@c34g2000yqn.googlegroups.com> Message-ID: <3877ca5f-2412-425b-bc98-2a36f6480346@m26g2000yqb.googlegroups.com> On Jan 15, 4:46?pm, r0g wrote: > Jo?o wrote: > > On Jan 15, 2:38 pm, r0g wrote: > >> Jo?o wrote: > >>> On Jan 14, 5:58 pm, r0g wrote: > >>>> Jo?o wrote: > >>>>> On Jan 12, 10:07 pm, r0g wrote: > >>>>>> Jo?o wrote: > >>>>> for the following data, > >>>>> authentication = "UID=somestring&" > >>>>> message = 'PROBLEM severity High: OperatorX Plat1(locationY) global > >>>>> Succ. : 94.470000%' > >>>>> dest_number = 'XXXXXXXXXXX' > >>>>> url_values = urlencode({'M':message}) > >>>>> enc_data = authentication + url_values + dest_number > >>>>> I'm getting null for > >>>>> full_url = Request(url, enc_data, headers) > >>>>> and thus, > >>>>> response = urlopen(full_url).read() > >>>>> returns, > >>>>> TypeError: > >>>>> ) > >>>> Are you sure it's returning a null and not just some other unexpected > >>>> type? > >>>> I think your problem may be that you are passing a urllib2 class to > >>>> urllib(1)'s urlopen. Try using urllib2's urlopen instead e.g. > >>>> import urllib2 > >>>> request_object = urllib2.Request('http://www.example.com') > >>>> response = urllib2.urlopen(request_object) > >>>> the_page = response.read() > >>>> Roger. > >>> Thanks Roger. > >>> I think it's a null because i did a print(full_url) right after the > >>> Request > >>> I tried > >>> request_object = urllib2.Request('http://www.example.com') > >>> print(request_object) > >>> but when printing I get: > >> Hi Jo?o, > > >> That's exactly what you want, an object that is an instance of the > >> Request class. That object doesn't do anything by itself, you still need > >> to a) Connect to the server and request that URL and b) Read the data > >> from the server. > > >> a) To connect to the web server and initialize the request you need to > >> call urllib2.urlopen() with the Request object you just created and > >> assign the result to a name e.g. > > >>>> response = urllib2.urlopen(request_object) > >> That will give you an object (response) that you can call the .read() > >> method of to get the web page data. > > >>>> the_page = response.read() > >> If that doesn't make sense or seem to work for you then please try > >> reading the following website from top to bottom before taking any > >> further steps... > > >>http://www.voidspace.org.uk/python/articles/urllib2.shtml > > >>> I've read about Python 2.4 not playing well with proxies even with no > >>> proxy activated. > >>> Any sugestion? > >> I doubt any language can play well with proxies if there are none so I > >> doubt it's a factor ;) > > >> Good luck, > > >> Roger. > > > lol. > > I've expressed myself poorly, > > I meant I read about some issues when getting the Request + urlopen > > working when there's a proxy involved (like in my case) > > even when activating a no_proxy configuration, something like, > > > proxy_support = urllib.ProxyHandler({}) > > opener = urllib.build_opener(proxy_support) > > urllib.install_opener(opener) > > > But I don't know how to use it :( > > That is how you use it IIRC, this installs the proxy handler into urllib > and subsequent objects you subclass from urllib will use the custom handler. > > From what I can tell, you should be using urllib2 though, not urllib. > > Lets take a step back. You had the following line... > > request_object = urllib2.Request('http://www.example.com') > > ...You printed it and it showed that you had created a Request object > right. Now what happens when you type... > > response = urllib2.urlopen(request_object) > print response > > ? > > Roger. Thanks for the patience Roger. Your explanation opened my eyes. I finally got it to work, and it turned out I didn't have to include any custom proxy handler to avoid our proxy. It ended on such a small and simple block of code after getting the pieces together.. #!/usr/bin/env python import sys from urllib2 import Request, urlopen from urllib import urlencode authentication = "UID=124675&PW=gdberishyb8LcIANtvT89QVQ==&" url = 'http://10.112.28.221:38080/GwHTTPin/sendtext' encoded_data = urlencode({'M':sys.argv[2], 'N':sys.argv[1]}) # Was having problem with this one, shouldn't have tried to pass the authentication as an urlencode parameter because it was breaking the password string! sent_data = authentication + encoded_data full_url = Request(url,sent_data) response = urlopen(full_url).read() From joaopcf at gmail.com Mon Jan 18 15:50:22 2010 From: joaopcf at gmail.com (=?ISO-8859-1?Q?Jo=E3o?=) Date: Mon, 18 Jan 2010 12:50:22 -0800 (PST) Subject: force URLencoding script References: <6bc42c49-e26c-4f7b-9538-dd2e319accfb@26g2000yqo.googlegroups.com> <9bc83518-b641-47a9-84ef-c68c9f825248@26g2000yqo.googlegroups.com> <4c68f05e-1fde-4e39-bb8a-803aa32de201@c34g2000yqn.googlegroups.com> Message-ID: <9ff54fda-9b55-4f9c-b1bc-48dd2ef66017@c29g2000yqd.googlegroups.com> On Jan 15, 4:46?pm, r0g wrote: > Jo?o wrote: > > On Jan 15, 2:38 pm, r0g wrote: > >> Jo?o wrote: > >>> On Jan 14, 5:58 pm, r0g wrote: > >>>> Jo?o wrote: > >>>>> On Jan 12, 10:07 pm, r0g wrote: > >>>>>> Jo?o wrote: > >>>>> for the following data, > >>>>> authentication = "UID=somestring&" > >>>>> message = 'PROBLEM severity High: OperatorX Plat1(locationY) global > >>>>> Succ. : 94.470000%' > >>>>> dest_number = 'XXXXXXXXXXX' > >>>>> url_values = urlencode({'M':message}) > >>>>> enc_data = authentication + url_values + dest_number > >>>>> I'm getting null for > >>>>> full_url = Request(url, enc_data, headers) > >>>>> and thus, > >>>>> response = urlopen(full_url).read() > >>>>> returns, > >>>>> TypeError: > >>>>> ) > >>>> Are you sure it's returning a null and not just some other unexpected > >>>> type? > >>>> I think your problem may be that you are passing a urllib2 class to > >>>> urllib(1)'s urlopen. Try using urllib2's urlopen instead e.g. > >>>> import urllib2 > >>>> request_object = urllib2.Request('http://www.example.com') > >>>> response = urllib2.urlopen(request_object) > >>>> the_page = response.read() > >>>> Roger. > >>> Thanks Roger. > >>> I think it's a null because i did a print(full_url) right after the > >>> Request > >>> I tried > >>> request_object = urllib2.Request('http://www.example.com') > >>> print(request_object) > >>> but when printing I get: > >> Hi Jo?o, > > >> That's exactly what you want, an object that is an instance of the > >> Request class. That object doesn't do anything by itself, you still need > >> to a) Connect to the server and request that URL and b) Read the data > >> from the server. > > >> a) To connect to the web server and initialize the request you need to > >> call urllib2.urlopen() with the Request object you just created and > >> assign the result to a name e.g. > > >>>> response = urllib2.urlopen(request_object) > >> That will give you an object (response) that you can call the .read() > >> method of to get the web page data. > > >>>> the_page = response.read() > >> If that doesn't make sense or seem to work for you then please try > >> reading the following website from top to bottom before taking any > >> further steps... > > >>http://www.voidspace.org.uk/python/articles/urllib2.shtml > > >>> I've read about Python 2.4 not playing well with proxies even with no > >>> proxy activated. > >>> Any sugestion? > >> I doubt any language can play well with proxies if there are none so I > >> doubt it's a factor ;) > > >> Good luck, > > >> Roger. > > > lol. > > I've expressed myself poorly, > > I meant I read about some issues when getting the Request + urlopen > > working when there's a proxy involved (like in my case) > > even when activating a no_proxy configuration, something like, > > > proxy_support = urllib.ProxyHandler({}) > > opener = urllib.build_opener(proxy_support) > > urllib.install_opener(opener) > > > But I don't know how to use it :( > > That is how you use it IIRC, this installs the proxy handler into urllib > and subsequent objects you subclass from urllib will use the custom handler. > > From what I can tell, you should be using urllib2 though, not urllib. > > Lets take a step back. You had the following line... > > request_object = urllib2.Request('http://www.example.com') > > ...You printed it and it showed that you had created a Request object > right. Now what happens when you type... > > response = urllib2.urlopen(request_object) > print response > > ? > > Roger. Thanks for the patience Roger. Your explanation opened my eyes. I finally got it to work, and it turned out I didn't have to include any custom proxy handler to avoid our proxy. It ended on such a small and simple block of code after getting the pieces together.. #!/usr/bin/env python import sys from urllib2 import Request, urlopen from urllib import urlencode authentication = "UID=124675&PW=gdberishyb8LcIANtvT89QVQ==&" url = 'http://10.112.28.221:38080/GwHTTPin/sendtext' encoded_data = urlencode({'M':sys.argv[2], 'N':sys.argv[1]}) # Was having problem with this one, shouldn't have tried to pass the authentication as an urlencode parameter because it was breaking the password string! sent_data = authentication + encoded_data full_url = Request(url,sent_data) response = urlopen(full_url).read() From jonathan.slenders at gmail.com Mon Jan 18 15:57:03 2010 From: jonathan.slenders at gmail.com (Jonathan S) Date: Mon, 18 Jan 2010 12:57:03 -0800 (PST) Subject: Python decorator syntax limitations References: <4b54998b$1@dnews.tpgi.com.au> Message-ID: <28e5220e-82f8-44d0-bede-a82fd3fbfd95@s31g2000yqs.googlegroups.com> Thanks a lot, all of you! This was really helpful. (or at least give me the inspiration I needed to finish it.) I'm sure this is a use case where most other options are less readable than the chain of methods in the decorator. In this use case, I had a lot of Django views to which access permissions had to be attached. Chaining the permissions was in my opinion the easiest way to go but appeared not to work because Guido had a gut feeling about that. Using multiple decorators in the usual way (like every following wraps the previous) does not work, because these decorators need to access the same authentication class. (little hard to explain, but take from me that I had a case where it didn't work.) Placing all possible options in the constructor method of the decorator like Steve Howell proposed would work but is ugly. This is what I made. I'll upgrade my news_page decorator to work in the following way. (Applying the news_page decorator to 'view', will turn it into a class, but make it callable buy wrapping the actual view in __call__. So we can still apply the other methods to the view. @news_page('template.html') def view(request, group, news): ... pass view.lookup(News, 'news_id', 'news') view.require_administrator() That is still rather ugly, because the 'lookup' option is appended behind the view. I kept playing with the code and came to the following: @require_administrator @do_lookup(News, 'news_id', 'news) @news_page('template.html') def view(request, group, news): ... pass Where 'do_lookup' and 'require_administrator' passes the method, but sets some parameters. So, 'do_lookup' can access class members of news_page. It would look like: def do_lookup(*lookup_options): def set_options(news_page_view): news_page_view.lookup(*lookup_options) return news_page_view return set_options def require_administrator(news_page_view): news_page_view.require_administrator() return news_page_view Maybe, I'll join the discussion later on, when I have time to read all the conversations and write down good arguments. But, honestly, I'm satisfied with the last result. Have a nice day! From kyosohma at gmail.com Mon Jan 18 15:59:20 2010 From: kyosohma at gmail.com (Mike Driscoll) Date: Mon, 18 Jan 2010 12:59:20 -0800 (PST) Subject: python gui ide under linux..like visual studio ;) ? References: <4b5470ce$0$1103$4fafbaef@reader4.news.tin.it> Message-ID: <3f3650e2-1c9d-43e1-b759-6c3d939de7bb@h9g2000yqa.googlegroups.com> On Jan 18, 8:32?am, ted wrote: > Hi at all... > Can someone please give me some advice, about a good IDE with control > GUI under Linux ? > > Actually i know QT Creator by Nokia which i can use with Python (but i > don't know how). > > And, a good library for access to database (mysql, sql server, oracle) ? > > Thank you very much ! > > bye Check out Dabo for the database stuff: http://dabodev.com/ There is nothing like Visual Studio. The closest I've found are things like the QT Creator and wxGlade / Boa Constructor / wxFormBuilder. I know I've heard of another one that was commercial, but I can't recall the name at the moment. ------------------- Mike Driscoll Blog: http://blog.pythonlibrary.org PyCon 2010 Atlanta Feb 19-21 http://us.pycon.org/ From timur at freescale.com Mon Jan 18 16:00:26 2010 From: timur at freescale.com (Timur Tabi) Date: Mon, 18 Jan 2010 15:00:26 -0600 Subject: I really need webbrowser.open('file://') to open a web browser In-Reply-To: <126148f0-fdcf-442d-b537-fd43b7bfb6b5@f12g2000yqn.googlegroups.com> References: <126148f0-fdcf-442d-b537-fd43b7bfb6b5@f12g2000yqn.googlegroups.com> Message-ID: On Sat, Jan 16, 2010 at 3:43 PM, Paul Boddie wrote: > Generally, the desktop-specific tools should know that a browser is > the appropriate application for an HTML file, and testing with both > xdg-open, gnome-open and "kfmclient openURL" seems to open browsers on > HTML files (using file:///...) for me (using KDE, Kubuntu 8.04). Of > course, this depends on the settings in use on your desktop, but it > should be noted that using "kfmclient exec" could have the effect you > describe. I'm using Gnome, and I have HTML files associated with Firefox. However, my default web browser is Seamonkey, and when I do webbrowser.open('http://...'), it opens that URL in Seamonkey, not Firefox. So if there is some Gnome association between an .html file and a text editor, I don't know where it is defined. > Not that I'm aware of. Sadly, standardisation of applications and > services - having a command which can open a particular class of > application (such as "e-mail reader", "Web browser") - seems to be > absent from the free desktop arena, although I do recall there being a > preferred applications dialogue in KDE, at least. I would be sympathetic to this problem if the API were called desktop.open(...). But it's called webbrowser.open(), so it has to be certain that a web browser is being at all times. IMHO, any other behavior is a bug. -- Timur Tabi Linux kernel developer at Freescale From mailbox394 at gmail.com Mon Jan 18 16:29:59 2010 From: mailbox394 at gmail.com (Gregory) Date: Mon, 18 Jan 2010 13:29:59 -0800 (PST) Subject: form mailer for info + files Message-ID: <3d2e2746-8e80-46fa-9e70-958955142fdc@36g2000yqu.googlegroups.com> There is code all over on how to create a form mailer. I need an example that will show how to upload two files and send them along with form field info. Is it required to overwrite the python cgi.fieldstorage to do this? Working with Python 2.4.3. From wayne.dads.bell at gmail.com Mon Jan 18 16:39:14 2010 From: wayne.dads.bell at gmail.com (dads) Date: Mon, 18 Jan 2010 13:39:14 -0800 (PST) Subject: searching and storing large quantities of xml! References: <4b52bdb8$0$6733$9b4e6d93@newsspool2.arcor-online.net> Message-ID: Thanks all, took your advice and have been playing all weekend which has been great fun. ElementTree is awesome. I created a script that organises the xml as they're in year blocks and I didn't realise the required xml is mixed up with other xml. Plus the volumes are much greater than I realised, I checked as back at work and it was something like 600,000 files in a year, just over a gig for each year. I'm going to add zipping up of the files and getting the required info and putting it in a db this week hopefully. It's been completely overhauled, originally I used modified date now it gets the date from the parsed xml, safer that way. The code is below but word of caution, it's hobbyist code so it'll probably make your eyes bleed =), thanks again: There was one thing that I forgot about - when ElementTree fails to parse due to an element not being closed why doesn't it close the file like object. As later on I would raise 'WindowsError: [Error 32] ...file being used by other process' when using shutil.move(). I got round this by using a 'try except' block. from __future__ import print_function import xml.etree.cElementTree as ET import calendar import zipfile import os.path import shutil import zlib import os class Xmlorg(object): def __init__(self): self.cwd = os.getcwd() self.year = os.path.basename(self.cwd) def _mkMonthAndDaysDirs(self): ''' creates dirs for every month and day of a of specidifed year. Works for leap years as well. (specified)year/(year)month/day ...2010/201001/01 ...2010/201001/02 ...2010/201001/03 ''' def addZero(n): if len(str(n)) < 2: return '0' + str(n) else: return str(n) dim = [ calendar.monthrange(year,month)[1] for year in \ [int(self.year)] for month in range(1,13) ] count = 1 for n in dim: month = addZero(count) count += 1 ym = os.path.join(self.cwd, self.year + month) os.mkdir(ym) for x in range(1,n+1): x = addZero(x) os.mkdir(os.path.join(ym, x)) def ParseAndOrg(self): '''requires dir and zip struct: .../(year)/(year).zip - example .../2008/2008.zip ''' def movef(fp1,fp2): '''moves files with exception handling''' try: shutil.move(fp1,fp2) except IOError, e: print(e) except WindowsError, e: print(e) self._mkMonthAndDaysDirs() os.mkdir(os.path.join(self.cwd, 'otherFileType')) # dir struct .../(year)/(year).zip - ex. .../2008/2008.zip zf = zipfile.ZipFile(os.path.join(self.cwd, self.year + '.zip')) zf.extractall() ld = os.listdir(self.cwd) for i in ld: if os.path.isfile(i) and i.endswith('.xml'): try: tree = ET.parse(i) except: print('%s np' % i) #not parsed root = tree.getroot() if root.findtext('Summary/FileType') == 'Order': date = root.findtext('OrderHeader/OrderDate')[:10] #dd/mm/yyyy dc = date.split('/') fp1 = os.path.join(self.cwd, i) fp2 = os.path.join(self.cwd, dc[2] + dc[1], dc[0]) movef(fp1,fp2) else: fp1 = os.path.join(self.cwd, i) fp2 = os.path.join(self.cwd, 'otherFileType') movef(fp1,fp2) if __name__ == '__main__': os.chdir('c:/sv_zip_test/2010/') #remove xo = Xmlorg() xo.ParseAndOrg() From tjreedy at udel.edu Mon Jan 18 16:42:19 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 18 Jan 2010 16:42:19 -0500 Subject: Generic Python Benchmark suite? In-Reply-To: References: Message-ID: On 1/18/2010 4:58 AM, Anand Vaidya wrote: > Is there a generic python benchmark suite in active development? I am > looking forward to comparing some code on various python > implementations (primarily CPython 2.x, CPython 3.x, UnladenSwallow, > Psyco). You might find this interesting if you have not seen it before: http://code.google.com/p/unladen-swallow/wiki/Benchmarks From python.list at tim.thechases.com Mon Jan 18 16:56:40 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Mon, 18 Jan 2010 15:56:40 -0600 Subject: Parse a log file In-Reply-To: <92293df0-7d2e-453a-b5bc-d4a7eb905fcf@l19g2000yqb.googlegroups.com> References: <92293df0-7d2e-453a-b5bc-d4a7eb905fcf@l19g2000yqb.googlegroups.com> Message-ID: <4B54D918.8040103@tim.thechases.com> kaklis at gmail.com wrote: > I want to parse a log file with the following format for > example: > TIMESTAMPE Operation FileName > Bytes > 12/Jan/2010:16:04:59 +0200 EXISTS sample3.3gp 37151 > 12/Jan/2010:16:04:59 +0200 EXISTS sample3.3gp 37151 > 12/Jan/2010:16:04:59 +0200 EXISTS sample3.3gp 37151 > 12/Jan/2010:16:04:59 +0200 EXISTS sample3.3gp 37151 > 12/Jan/2010:16:04:59 +0200 EXISTS sample3.3gp 37151 > 12/Jan/2010:16:05:05 +0200 DELETE sample3.3gp 37151 > > How can i count the operations for a month(e.g total of 40 Operations, > 30 exists, 10 delete?) It can be done pretty easily with a regexp to parse the relevant bits: import re r = re.compile(r'\d+/([^/]+)/(\d+)\S+\s+\S+\s+(\w+)') stats = {} for line in file('log.txt'): m = r.match(line) if m: stats[m.groups()] = stats.get(m.groups(), 0) + 1 print stats This prints out {('Jan', '2010', 'EXISTS'): 5, ('Jan', '2010', 'DELETE'): 1} With the resulting data structure, you can manipulate it to do coarser-grained aggregates such as the total operations, or remap month-name abbreviations into integers so they could be sorted for output. -tkc From showell30 at yahoo.com Mon Jan 18 17:00:17 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 18 Jan 2010 14:00:17 -0800 (PST) Subject: inner methods and recursion Message-ID: Hi, I have a style/design question relating to recursion and inner methods. I wrote some code recently that implements a recursive algorithm that takes several parameters from the original caller. Once the algorithm starts running and recursing, those parameters remain the same, so I define an inner method called "recurse" that essentially curries those parameters. In particular, the recurse method can get passed to a callback method that the caller supplies, so that the caller can wrap the recursive step with their own logic. The only thing I don't like about the technique that I'm using is that it seems needlessly repetitive to define mostly the same parameter list in two different places. The repetition seems mostly harmless, but it seems like some kind of a smell that I'm overlooking a simpler way to write the code. I just can't put my finger on what it is. def indent_lines(lines, branch_method, leaf_method, pass_syntax, flush_left_syntax, flush_left_empty_line, indentation_method, get_block, ): def recurse(lines): return indent_lines( lines, branch_method, leaf_method, pass_syntax, flush_left_syntax, flush_left_empty_line, indentation_method, get_block, ) output = [] while lines: if lines[0].strip() == '': lines.pop(0) output.append('') else: prefix, i = get_block(lines, indentation_method) if i == 1: line = lines.pop(0)[len(prefix):] if line == pass_syntax: pass elif line.startswith(flush_left_syntax): output.append(line[len(flush_left_syntax):]) elif line.startswith(flush_left_empty_line): output.append('') else: output.append(prefix + leaf_method(line)) else: block = lines[:i] lines = lines[i:] output += branch_method(prefix, block, recurse) return output Does anybody have any inspiration here? I vaguely recall a discussion some time back about having some kind of @recursive decorator, but I can't seem to find the thread. From aahz at pythoncraft.com Mon Jan 18 17:11:37 2010 From: aahz at pythoncraft.com (Aahz) Date: 18 Jan 2010 14:11:37 -0800 Subject: monitor reading file with thread References: <4b47712a$0$1115$4fafbaef@reader4.news.tin.it> Message-ID: In article <4b47712a$0$1115$4fafbaef at reader4.news.tin.it>, wiso wrote: > >class Reader(): > def __init__(self,filename): > self.filename = filename > self.lineno = 0 > def __iter__(self): > f = open(self.filename) > for line in f: > self.lineno += 1 > time.sleep(0.01) # slow down > yield line > f.close() There's no reason for the sleep. >class Monitor(threading.Thread): > def __init__(self,reader,stop_queue,interval=2): > threading.Thread.__init__(self) > self.interval = interval > self.reader = reader > self.stop_queue = stop_queue > def run(self): > while True: > try: > if self.stop_queue.get(timeout=self.interval) == "stop": > break > except Queue.Empty: > pass > print "MONITOR: ", reader.lineno I'd make this simpler, you only need a queue when you're transferring data between threads: class Monitor(threading.Thread): def __init__(self, reader, interval=2): threading.Thread.__init__(self) self.interval = interval self.reader = reader self.stop = False def run(self): while not self.stop: print "MONITOR:", reader.lineno time.sleep(self.interval) ...and to stop the monitor, just set m.stop=True. Note that you have a bug that needs fixing, I left it for you. ;-) -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." --Red Adair From jabronson at gmail.com Mon Jan 18 17:12:20 2010 From: jabronson at gmail.com (Joshua Bronson) Date: Mon, 18 Jan 2010 14:12:20 -0800 (PST) Subject: heapq._siftdown / decrease-key support? References: <0ba8b786-2360-4a74-84c4-97e0564fbf3c@q16g2000vbc.googlegroups.com> Message-ID: <82e3cf4e-f58c-47d7-ad21-5d5a8dd2bafa@m25g2000yqc.googlegroups.com> On Sun, Jan 17, 2010 at 11:30 PM, Raymond Hettinger wrote: > > > Raymond, do you think this technique is worth documenting in the heapq > > module? It'd be too bad if any future users incorrectly think that it > > won't meet their needs the way I did. > > Yes. ?Please assign a tracker issue to me to expand the heapq documention > to discuss the mark-as-invalid trick for priority queues. > > > Raymond I've created http://bugs.python.org/issue7734. I don't enough have privileges to assign it to you, but I added you to the nosy list. Thanks! Josh From tjreedy at udel.edu Mon Jan 18 17:12:28 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 18 Jan 2010 17:12:28 -0500 Subject: r"string" vs R"string In-Reply-To: References: <00dd21f5$0$15570$c3e8da3@news.astraweb.com> Message-ID: On 1/18/2010 8:01 AM, Colin W. wrote: > On 17-Jan-10 18:27 PM, Steven D'Aprano wrote: >> On Sun, 17 Jan 2010 11:13:48 -0500, Roy Smith wrote: >> >>> In article, >>> "Colin W." wrote: >>> >>>> On 17-Jan-10 02:16 AM, Terry Reedy wrote: >>>>> On 1/17/2010 1:55 AM, Brendan Miller wrote: >>>>>> Is there any difference whatsoever between a raw string beginning >>>>>> with the captical R or one with the lower case r e.g. r"string" vs >>>>>> R"string"? >>>>> >>>>> No. Nor is there and difference between the strings created with raw >>>>> literals and cooked literals. >>>>> >>>>> >>>> "cooked" literal ?? >>> >>> I've never seen it referred to this way in the Python literature, but >>> "cooked" is a well-known term meaning, "not raw". The usage goes back >>> decades. >> >> I think the use of "cooked" meaning "not raw" goes back a little bit more >> than decades. The verb "to cook" dates from the late 14th century, and >> the adjective form "cooked" would follow soon after. The use of "cooked" >> to mean manipulated (as in "cooking the books") comes from the 1630s. >> Extending it to strings (as in raw versus cooked strings) is an obvious >> extension. >> > Yes, I should have cottoned on, but perhaps "... between the strings, > raw and other." would have conveyed the idea. I intentionally did not say that because Python only has one type of string object (in 3.x), not raw and normal. Python *code* has string literals optionally prefixed with 'r' (or 'R', which I did not know before) to suppress the normal processing that reduces escape sequences to one char. The term 'cooked' versus 'raw' was used in Unixland at least through the 1980s. For instance, the input stream from a terminal could be cooked or left raw. I presume that is whence came Python's use of 'raw', with much the same meaning. Terry Jan Reedy From gnarlodious at gmail.com Mon Jan 18 17:43:54 2010 From: gnarlodious at gmail.com (Gnarlodious) Date: Mon, 18 Jan 2010 14:43:54 -0800 (PST) Subject: Py 3: Terminal script can't find relative path Message-ID: <58e9b1e4-049a-4a7f-bea9-8d7e7e9f191a@e27g2000yqd.googlegroups.com> I am running a script in a browser that finds the file in subfolder Data: Content=Plist('Data/Content.plist') However, running the same script in Terminal errors: IOError: [Errno 2] No such file or directory: 'Data/Content.plist' Is Py 3 unable to find relative paths? I tried all kinds of tricks to no avail. What is wrong? -- Gnarlie From kaklis at gmail.com Mon Jan 18 17:46:26 2010 From: kaklis at gmail.com (kaklis at gmail.com) Date: Mon, 18 Jan 2010 14:46:26 -0800 (PST) Subject: Parse a log file References: <92293df0-7d2e-453a-b5bc-d4a7eb905fcf@l19g2000yqb.googlegroups.com> Message-ID: <9ca93ef0-396f-4f60-9fce-d1354c0d4c32@j19g2000yqk.googlegroups.com> On Jan 18, 11:56?pm, Tim Chase wrote: > kak... at gmail.com wrote: > > I want to parse a log file with the following format for > > example: > > ? ? ? ? ? ? ? TIMESTAMPE ? ? ? ? ? ?Operation ? ? FileName > > Bytes > > 12/Jan/2010:16:04:59 +0200 ? EXISTS ? ? ? sample3.3gp ? 37151 > > 12/Jan/2010:16:04:59 +0200 ?EXISTS ? ? ? ?sample3.3gp ? 37151 > > 12/Jan/2010:16:04:59 +0200 ?EXISTS ? ? ? ?sample3.3gp ? 37151 > > 12/Jan/2010:16:04:59 +0200 ?EXISTS ? ? ? ?sample3.3gp ? 37151 > > 12/Jan/2010:16:04:59 +0200 ?EXISTS ? ? ? ?sample3.3gp ? 37151 > > 12/Jan/2010:16:05:05 +0200 ?DELETE ? ? ?sample3.3gp ? 37151 > > > How can i count the operations for a month(e.g total of 40 Operations, > > 30 exists, 10 delete?) > > It can be done pretty easily with a regexp to parse the relevant > bits: > > ? ?import re > ? ?r = re.compile(r'\d+/([^/]+)/(\d+)\S+\s+\S+\s+(\w+)') > ? ?stats = {} > ? ?for line in file('log.txt'): > ? ? ?m = r.match(line) > ? ? ?if m: > ? ? ? ?stats[m.groups()] = stats.get(m.groups(), 0) + 1 > ? ?print stats > > This prints out > > ? ?{('Jan', '2010', 'EXISTS'): 5, ('Jan', '2010', 'DELETE'): 1} > > With the resulting data structure, you can manipulate it to do > coarser-grained aggregates such as the total operations, or remap > month-name abbreviations into integers so they could be sorted > for output. > > -tkc Thank you both so much Antonis From g.bogle at auckland.no.spam.ac.nz Mon Jan 18 18:15:37 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Tue, 19 Jan 2010 12:15:37 +1300 Subject: PyQwt installation Message-ID: Should there be a problem installing PyQwt5.2.0 with PyQt4.4.4 and Python2.6 on Windows? My student is saying she can't get the Windows installer to work, and she points out that the download site says "Binary installation of PyQwt-5.2.0 on Windows for Python-2.6.x, PyQt-4.5.4" From john at castleamber.com Mon Jan 18 18:21:48 2010 From: john at castleamber.com (John Bokma) Date: Mon, 18 Jan 2010 17:21:48 -0600 Subject: Py 3: Terminal script can't find relative path References: <58e9b1e4-049a-4a7f-bea9-8d7e7e9f191a@e27g2000yqd.googlegroups.com> Message-ID: <87d417578z.fsf@castleamber.com> Gnarlodious writes: > I am running a script in a browser that finds the file in subfolder > Data: > > Content=Plist('Data/Content.plist') > > However, running the same script in Terminal errors: > > IOError: [Errno 2] No such file or directory: 'Data/Content.plist' What does: ls -l Data/Content.plist in the terminal give? -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From python at mrabarnett.plus.com Mon Jan 18 18:34:22 2010 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 18 Jan 2010 23:34:22 +0000 Subject: searching and storing large quantities of xml! In-Reply-To: References: <4b52bdb8$0$6733$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <4B54EFFE.7070208@mrabarnett.plus.com> dads wrote: [snip] > import os.path > import shutil > import zlib > import os > There's no need to import both os.path and os. Import just os; you can still refer to os.path in the rest of the code. [snip] > > def _mkMonthAndDaysDirs(self): > > ''' creates dirs for every month and day of a of specidifed > year. > Works for leap years as well. > > (specified)year/(year)month/day > > > ...2010/201001/01 > ...2010/201001/02 > ...2010/201001/03 ''' > There's nothing except a docstring in this method! > > def addZero(n): > > if len(str(n)) < 2: > return '0' + str(n) > else: > return str(n) > A shorter and quicker way is: return "%02d" % n which means "pad with leading zeros to be at least 2 characters". [snip] > > # dir struct .../(year)/(year).zip - ex. .../2008/2008.zip > zf = zipfile.ZipFile(os.path.join(self.cwd, self.year + > '.zip')) > zf.extractall() You might want to close the zipfile after use. [snip] > > if __name__ == '__main__': > os.chdir('c:/sv_zip_test/2010/') #remove I recommend that you work with the full paths instead of changing the current directory and then using relative paths. > xo = Xmlorg() > xo.ParseAndOrg() From wkfung.eric at gmail.com Mon Jan 18 19:24:11 2010 From: wkfung.eric at gmail.com (Kit) Date: Mon, 18 Jan 2010 16:24:11 -0800 (PST) Subject: What is a list compression in Python? References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> <00de11a0$0$15570$c3e8da3@news.astraweb.com> Message-ID: Thank you so much guys. Just out of curiosity: can I do something like this to "square all even numbers in the range 1-10"? print [x^2 for x in range (1,11) if x % 2 == 0] Or is there a better way of doing it? Thanks for the help, and I am really appreciate your help. Kit. On 1?19?, ??12?30?, Steven D'Aprano wrote: > On Mon, 18 Jan 2010 08:07:41 -0800, Kit wrote: > > Hello Everyone, I am not sure if I have posted this question in a > > correct board. Can anyone please teach me: > > > What is a list compression in Python? > > Google "python list comprehension". > > If Google is broken for you, try Yahoo, or any other search engine. > > > Would you mind give me some list compression examples? > > Instead of this: > > L = [] > for x in range(10): > ? ? L.append(x**2) > > you can write: > > L = [x**2 for x in range(10)] > > Instead of this example: > > L = [] > for x in range(10): > ? ? if x % 2 == 0: > ? ? ? ? L.append(x**2) > > you can write: > > L = [x**2 for x in range(10) if x % 2 == 0] > > -- > Steven From wkfung.eric at gmail.com Mon Jan 18 19:26:31 2010 From: wkfung.eric at gmail.com (Kit) Date: Mon, 18 Jan 2010 16:26:31 -0800 (PST) Subject: What is a list compression in Python? References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> <00de11a0$0$15570$c3e8da3@news.astraweb.com> Message-ID: <21602b5a-0d88-46b3-9cb9-bcfcd5c6707c@b10g2000yqa.googlegroups.com> Oops... > print [x^2 for x in range (1,11) if x % 2 == 0] print [x^2 for x in range (1,10) if x % 2 == 0] On 1?19?, ??8?24?, Kit wrote: > Thank you so much guys. > > Just out of curiosity: can I do something like this to "square all > even numbers in the range 1-10"? > print [x^2 for x in range (1,11) if x % 2 == 0] > > Or is there a better way of doing it? Thanks for the help, and I am > really appreciate your help. > > Kit. > > On 1?19?, ??12?30?, Steven D'Aprano > > > cybersource.com.au> wrote: > > On Mon, 18 Jan 2010 08:07:41 -0800, Kit wrote: > > > Hello Everyone, I am not sure if I have posted this question in a > > > correct board. Can anyone please teach me: > > > > What is a list compression in Python? > > > Google "python list comprehension". > > > If Google is broken for you, try Yahoo, or any other search engine. > > > > Would you mind give me some list compression examples? > > > Instead of this: > > > L = [] > > for x in range(10): > > ? ? L.append(x**2) > > > you can write: > > > L = [x**2 for x in range(10)] > > > Instead of this example: > > > L = [] > > for x in range(10): > > ? ? if x % 2 == 0: > > ? ? ? ? L.append(x**2) > > > you can write: > > > L = [x**2 for x in range(10) if x % 2 == 0] > > > -- > > Steven From python at mrabarnett.plus.com Mon Jan 18 19:35:04 2010 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 19 Jan 2010 00:35:04 +0000 Subject: What is a list compression in Python? In-Reply-To: References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> <00de11a0$0$15570$c3e8da3@news.astraweb.com> Message-ID: <4B54FE38.1010704@mrabarnett.plus.com> Kit wrote: > Thank you so much guys. > > Just out of curiosity: can I do something like this to "square all > even numbers in the range 1-10"? > print [x^2 for x in range (1,11) if x % 2 == 0] > > Or is there a better way of doing it? Thanks for the help, and I am > really appreciate your help. > That would be: print [x ** 2 for x in range(1, 11) if x % 2 == 0] Note that Python uses "**" for raising to a power and "^" (like in C) for bitwise exclusive-or. From gherron at islandtraining.com Mon Jan 18 19:36:45 2010 From: gherron at islandtraining.com (Gary Herron) Date: Mon, 18 Jan 2010 16:36:45 -0800 Subject: What is a list compression in Python? In-Reply-To: References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> <00de11a0$0$15570$c3e8da3@news.astraweb.com> Message-ID: <4B54FE9D.8070601@islandtraining.com> Kit wrote: > Thank you so much guys. > > Just out of curiosity: can I do something like this to "square all > even numbers in the range 1-10"? > print [x^2 for x in range (1,11) if x % 2 == 0] > > Or is there a better way of doing it? Thanks for the help, and I am > really appreciate your help. > That's a fine way to do it, although: xrange is slightly more efficient than range for large sets of values. (This isn't such a large set for it to matter.) You can get range and xrange to count by two's, eliminating the need for the "if" portion in this case. The exponent operator is ** not ^, and x*x is more efficient than x**2. So. print [x**2 for x in xrange(1,11,2)] Gary Herron > Kit. > > On 1?19?, ??12?30?, Steven D'Aprano cybersource.com.au> wrote: > >> On Mon, 18 Jan 2010 08:07:41 -0800, Kit wrote: >> >>> Hello Everyone, I am not sure if I have posted this question in a >>> correct board. Can anyone please teach me: >>> >>> What is a list compression in Python? >>> >> Google "python list comprehension". >> >> If Google is broken for you, try Yahoo, or any other search engine. >> >> >>> Would you mind give me some list compression examples? >>> >> Instead of this: >> >> L = [] >> for x in range(10): >> L.append(x**2) >> >> you can write: >> >> L = [x**2 for x in range(10)] >> >> Instead of this example: >> >> L = [] >> for x in range(10): >> if x % 2 == 0: >> L.append(x**2) >> >> you can write: >> >> L = [x**2 for x in range(10) if x % 2 == 0] >> >> -- >> Steven >> > > From wkfung.eric at gmail.com Mon Jan 18 19:41:49 2010 From: wkfung.eric at gmail.com (Kit) Date: Mon, 18 Jan 2010 16:41:49 -0800 (PST) Subject: What is a list compression in Python? References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> <00de11a0$0$15570$c3e8da3@news.astraweb.com> Message-ID: Cool! Thank you very much. You mean this instead right? print [x*x for x in xrange(1,11,2)] Kit On 1?19?, ??8?36?, Gary Herron wrote: > Kit wrote: > > Thank you so much guys. > > > Just out of curiosity: can I do something like this to "square all > > even numbers in the range 1-10"? > > print [x^2 for x in range (1,11) if x % 2 == 0] > > > Or is there a better way of doing it? Thanks for the help, and I am > > really appreciate your help. > > That's a fine way to do it, although: > > ? xrange is slightly more efficient than range for large > ? sets of values. ?(This isn't such a large set for it to matter.) > > ? You can get range and xrange to count by two's, eliminating > ? the need for the "if" portion in this case. > > ? The exponent operator is ** not ^, and ?x*x is more efficient than x**2. > > So. > > print [x**2 for x in xrange(1,11,2)] > > Gary Herron > > > > > Kit. > > > On 1?19?, ??12?30?, Steven D'Aprano > cybersource.com.au> wrote: > > >> On Mon, 18 Jan 2010 08:07:41 -0800, Kit wrote: > > >>> Hello Everyone, I am not sure if I have posted this question in a > >>> correct board. Can anyone please teach me: > > >>> What is a list compression in Python? > > >> Google "python list comprehension". > > >> If Google is broken for you, try Yahoo, or any other search engine. > > >>> Would you mind give me some list compression examples? > > >> Instead of this: > > >> L = [] > >> for x in range(10): > >> ? ? L.append(x**2) > > >> you can write: > > >> L = [x**2 for x in range(10)] > > >> Instead of this example: > > >> L = [] > >> for x in range(10): > >> ? ? if x % 2 == 0: > >> ? ? ? ? L.append(x**2) > > >> you can write: > > >> L = [x**2 for x in range(10) if x % 2 == 0] > > >> -- > >> Steven From python at mrabarnett.plus.com Mon Jan 18 19:50:42 2010 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 19 Jan 2010 00:50:42 +0000 Subject: What is a list compression in Python? In-Reply-To: <21602b5a-0d88-46b3-9cb9-bcfcd5c6707c@b10g2000yqa.googlegroups.com> References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> <00de11a0$0$15570$c3e8da3@news.astraweb.com> <21602b5a-0d88-46b3-9cb9-bcfcd5c6707c@b10g2000yqa.googlegroups.com> Message-ID: <4B5501E2.5070103@mrabarnett.plus.com> Kit wrote: > Oops... > >> print [x^2 for x in range (1,11) if x % 2 == 0] > print [x^2 for x in range (1,10) if x % 2 == 0] > The start is inclusive and the end is exclusive. This is a general rule in Python. In fact, there's only one exception that I know of: randint(a, b) in the 'random' module. From marlowequart at hotmail.com Mon Jan 18 19:53:51 2010 From: marlowequart at hotmail.com (marlowe) Date: Mon, 18 Jan 2010 16:53:51 -0800 (PST) Subject: syntax Message-ID: <3ab2b14d-3248-4570-b332-b64ac4b37758@j14g2000yqm.googlegroups.com> I wrote this program, but i have a feeling like there might be a more practical way of writing it. Can someone give me an idea of how to simplify this? Here is an example of the csv file i am using. This program calculates the exponential moving average of the 20 day range. USOtable.csv (full table found at http://ichart.finance.yahoo.com/table.csv?s=USO) Date,Open,High,Low,Close,Volume,Adj Close 2010-01-15,38.97,39.02,38.28,38.40,12615300,38.40 2010-01-14,39.30,39.44,38.88,39.06,8575900,39.06 2010-01-13,39.40,39.71,38.63,39.21,15502700,39.21 2010-01-12,40.07,40.36,39.53,39.63,11960100,39.63 2010-01-11,41.09,41.19,40.46,40.54,8902200,40.54 2010-01-08,40.63,41.17,40.45,40.93,9393500,40.93 2010-01-07,40.87,41.08,40.68,40.72,10012000,40.72 2010-01-06,40.32,41.19,39.89,40.97,19789800,40.97 2010-01-05,40.25,40.45,39.93,40.41,10450200,40.41 test.py import csv reader = open('/prog/USOtable.csv','rb') data = [row for row in csv.reader(reader)] Nvals= [1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1] H=float(data[40][2])+float(data[41][2])+float(data[42][2])+float(data [43][2])\ +float(data[44][2])+float(data[45][2])+float(data[46][2])+float(data [47][2])\ +float(data[48][2])+float(data[49][2])+float(data[50][2])+float(data [51][2])\ +float(data[52][2])+float(data[53][2])+float(data[54][2])+float(data [55][2])\ +float(data[56][2])+float(data[57][2])+float(data[58][2])+float(data [59][2])\ +float(data[60][2]) L=float(data[40][3])+float(data[41][3])+float(data[42][3])+float(data [43][3])\ +float(data[44][3])+float(data[45][3])+float(data[46][3])+float(data [47][3])\ +float(data[48][3])+float(data[49][3])+float(data[50][3])+float(data [51][3])\ +float(data[52][3])+float(data[53][3])+float(data[54][3])+float(data [55][3])\ +float(data[56][3])+float(data[57][3])+float(data[58][3])+float(data [59][3])\ +float(data[60][3]) Nvals[39]=(H-L)/2 i=1 while i <=38: high=float(data[39-i][2]) low=float(data[39-i][3]) TR=high-low Nvals[39-i]=(19*Nvals[40-i]+TR)/20 i+=1 for value in Nvals: print value From gagsl-py2 at yahoo.com.ar Mon Jan 18 19:57:15 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 18 Jan 2010 21:57:15 -0300 Subject: using super References: <4B54BB95.1090407@sequans.com> Message-ID: En Mon, 18 Jan 2010 16:50:45 -0300, Jean-Michel Pichavant escribi?: > Duncan Booth wrote: >> Jean-Michel Pichavant wrote: >> >> >>>>>> class SubClass(Base): >>>>>> colour = "Red" >>>>>> def parrot(self): >>>>>> """docstring for Subclass""" >>>>>> return super(Subclass, self).parrot() >>>>>> >>> I'm not a big fan of super, but I'm still wondering if >>> return super(self.__class__, self).parrot() >>> would have made it. >> >> No, it wouldn't. [...] >> > I see. > Then is there a reason why > return super(Subclass, self).parrot() > would be prefered over the "classic" > return Base.parrot(self) > ? > Or is it just a matter of preference ? For a longer explanation, see: James Knight: Python's Super Considered Harmful http://fuhm.net/super-harmful/ Michele Simionato: Things to Know About Python Super http://www.artima.com/weblogs/viewpost.jsp?thread=236275 http://www.artima.com/weblogs/viewpost.jsp?thread=236278 http://www.artima.com/weblogs/viewpost.jsp?thread=237121 -- Gabriel Genellina From clp2 at rebertia.com Mon Jan 18 20:23:34 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Mon, 18 Jan 2010 17:23:34 -0800 Subject: syntax In-Reply-To: <3ab2b14d-3248-4570-b332-b64ac4b37758@j14g2000yqm.googlegroups.com> References: <3ab2b14d-3248-4570-b332-b64ac4b37758@j14g2000yqm.googlegroups.com> Message-ID: <50697b2c1001181723t2fe41ca5xeb19ea048db00706@mail.gmail.com> On Mon, Jan 18, 2010 at 4:53 PM, marlowe wrote: > I wrote this program, but i have a feeling like there might be a more > practical way of writing it. Can someone give me an idea of how to > simplify this? Here is an example of the csv file i am using. This > program calculates the exponential moving average of the 20 day range. > H=float(data[40][2])+float(data[41][2])+float(data[42][2])+float(data > [43][2])\ > +float(data[44][2])+float(data[45][2])+float(data[46][2])+float(data > [47][2])\ > +float(data[48][2])+float(data[49][2])+float(data[50][2])+float(data > [51][2])\ > +float(data[52][2])+float(data[53][2])+float(data[54][2])+float(data > [55][2])\ > +float(data[56][2])+float(data[57][2])+float(data[58][2])+float(data > [59][2])\ > +float(data[60][2]) H = sum(float(data[i][2]) for i in xrange(40,61)) > L=float(data[40][3])+float(data[41][3])+float(data[42][3])+float(data > [43][3])\ > +float(data[44][3])+float(data[45][3])+float(data[46][3])+float(data > [47][3])\ > +float(data[48][3])+float(data[49][3])+float(data[50][3])+float(data > [51][3])\ > +float(data[52][3])+float(data[53][3])+float(data[54][3])+float(data > [55][3])\ > +float(data[56][3])+float(data[57][3])+float(data[58][3])+float(data > [59][3])\ > +float(data[60][3]) H = sum(float(data[i][3]) for i in xrange(40,61)) Might I suggest you learn more about comprehensions and looping?: http://docs.python.org/tutorial/datastructures.html#list-comprehensions http://www.python.org/dev/peps/pep-0289/ http://docs.python.org/library/functions.html#range http://docs.python.org/library/functions.html#sum Cheers, Chris -- http://blog.rebertia.com From clp2 at rebertia.com Mon Jan 18 20:24:59 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Mon, 18 Jan 2010 17:24:59 -0800 Subject: syntax In-Reply-To: <50697b2c1001181723t2fe41ca5xeb19ea048db00706@mail.gmail.com> References: <3ab2b14d-3248-4570-b332-b64ac4b37758@j14g2000yqm.googlegroups.com> <50697b2c1001181723t2fe41ca5xeb19ea048db00706@mail.gmail.com> Message-ID: <50697b2c1001181724ve0e2e52o51e27cde0061e22e@mail.gmail.com> On Mon, Jan 18, 2010 at 5:23 PM, Chris Rebert wrote: > On Mon, Jan 18, 2010 at 4:53 PM, marlowe wrote: >> L=float(data[40][3])+float(data[41][3])+float(data[42][3])+float(data >> [43][3])\ >> +float(data[44][3])+float(data[45][3])+float(data[46][3])+float(data >> [47][3])\ >> +float(data[48][3])+float(data[49][3])+float(data[50][3])+float(data >> [51][3])\ >> +float(data[52][3])+float(data[53][3])+float(data[54][3])+float(data >> [55][3])\ >> +float(data[56][3])+float(data[57][3])+float(data[58][3])+float(data >> [59][3])\ >> +float(data[60][3]) > > H = sum(float(data[i][3]) for i in xrange(40,61)) Erm, L for that second one obviously, not H: L = sum(float(data[i][3]) for i in xrange(40,61)) - Chris From mrabarnett at mrabarnett.plus.com Mon Jan 18 20:31:47 2010 From: mrabarnett at mrabarnett.plus.com (Matthew Barnett) Date: Tue, 19 Jan 2010 01:31:47 +0000 Subject: syntax In-Reply-To: <3ab2b14d-3248-4570-b332-b64ac4b37758@j14g2000yqm.googlegroups.com> References: <3ab2b14d-3248-4570-b332-b64ac4b37758@j14g2000yqm.googlegroups.com> Message-ID: <4B550B83.8090609@mrabarnett.plus.com> marlowe wrote: > I wrote this program, but i have a feeling like there might be a more > practical way of writing it. Can someone give me an idea of how to > simplify this? Here is an example of the csv file i am using. This > program calculates the exponential moving average of the 20 day range. > > USOtable.csv (full table found at http://ichart.finance.yahoo.com/table.csv?s=USO) > > Date,Open,High,Low,Close,Volume,Adj Close > 2010-01-15,38.97,39.02,38.28,38.40,12615300,38.40 > 2010-01-14,39.30,39.44,38.88,39.06,8575900,39.06 > 2010-01-13,39.40,39.71,38.63,39.21,15502700,39.21 > 2010-01-12,40.07,40.36,39.53,39.63,11960100,39.63 > 2010-01-11,41.09,41.19,40.46,40.54,8902200,40.54 > 2010-01-08,40.63,41.17,40.45,40.93,9393500,40.93 > 2010-01-07,40.87,41.08,40.68,40.72,10012000,40.72 > 2010-01-06,40.32,41.19,39.89,40.97,19789800,40.97 > 2010-01-05,40.25,40.45,39.93,40.41,10450200,40.41 > > > test.py > > import csv > > > reader = open('/prog/USOtable.csv','rb') > data = [row for row in csv.reader(reader)] > data = list(csv.reader(reader)) > Nvals= > [1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1] > Nvals = [1] * 40 > > H=float(data[40][2])+float(data[41][2])+float(data[42][2])+float(data > [43][2])\ > +float(data[44][2])+float(data[45][2])+float(data[46][2])+float(data > [47][2])\ > +float(data[48][2])+float(data[49][2])+float(data[50][2])+float(data > [51][2])\ > +float(data[52][2])+float(data[53][2])+float(data[54][2])+float(data > [55][2])\ > +float(data[56][2])+float(data[57][2])+float(data[58][2])+float(data > [59][2])\ > +float(data[60][2]) > H = sum(float(d[2]) for d in data[40 : 61]) > L=float(data[40][3])+float(data[41][3])+float(data[42][3])+float(data > [43][3])\ > +float(data[44][3])+float(data[45][3])+float(data[46][3])+float(data > [47][3])\ > +float(data[48][3])+float(data[49][3])+float(data[50][3])+float(data > [51][3])\ > +float(data[52][3])+float(data[53][3])+float(data[54][3])+float(data > [55][3])\ > +float(data[56][3])+float(data[57][3])+float(data[58][3])+float(data > [59][3])\ > +float(data[60][3]) > L = sum(float(d[3]) for d in data[40 : 61]) > > Nvals[39]=(H-L)/2 > > > i=1 > while i <=38: > high=float(data[39-i][2]) > low=float(data[39-i][3]) > TR=high-low > Nvals[39-i]=(19*Nvals[40-i]+TR)/20 > i+=1 > Instead of: i=1 while i <=38: ... i+=1 you could use: for i in range(1, 39): ... Note that the start value is inclusive and the end value is exclusive. > for value in Nvals: > print value From python at mrabarnett.plus.com Mon Jan 18 20:32:06 2010 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 19 Jan 2010 01:32:06 +0000 Subject: syntax In-Reply-To: <3ab2b14d-3248-4570-b332-b64ac4b37758@j14g2000yqm.googlegroups.com> References: <3ab2b14d-3248-4570-b332-b64ac4b37758@j14g2000yqm.googlegroups.com> Message-ID: <4B550B96.80303@mrabarnett.plus.com> marlowe wrote: > I wrote this program, but i have a feeling like there might be a more > practical way of writing it. Can someone give me an idea of how to > simplify this? Here is an example of the csv file i am using. This > program calculates the exponential moving average of the 20 day range. > > USOtable.csv (full table found at http://ichart.finance.yahoo.com/table.csv?s=USO) > > Date,Open,High,Low,Close,Volume,Adj Close > 2010-01-15,38.97,39.02,38.28,38.40,12615300,38.40 > 2010-01-14,39.30,39.44,38.88,39.06,8575900,39.06 > 2010-01-13,39.40,39.71,38.63,39.21,15502700,39.21 > 2010-01-12,40.07,40.36,39.53,39.63,11960100,39.63 > 2010-01-11,41.09,41.19,40.46,40.54,8902200,40.54 > 2010-01-08,40.63,41.17,40.45,40.93,9393500,40.93 > 2010-01-07,40.87,41.08,40.68,40.72,10012000,40.72 > 2010-01-06,40.32,41.19,39.89,40.97,19789800,40.97 > 2010-01-05,40.25,40.45,39.93,40.41,10450200,40.41 > > > test.py > > import csv > > > reader = open('/prog/USOtable.csv','rb') > data = [row for row in csv.reader(reader)] > data = list(csv.reader(reader)) > Nvals= > [1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1] > Nvals = [1] * 40 > > H=float(data[40][2])+float(data[41][2])+float(data[42][2])+float(data > [43][2])\ > +float(data[44][2])+float(data[45][2])+float(data[46][2])+float(data > [47][2])\ > +float(data[48][2])+float(data[49][2])+float(data[50][2])+float(data > [51][2])\ > +float(data[52][2])+float(data[53][2])+float(data[54][2])+float(data > [55][2])\ > +float(data[56][2])+float(data[57][2])+float(data[58][2])+float(data > [59][2])\ > +float(data[60][2]) > H = sum(float(d[2]) for d in data[40 : 61]) > L=float(data[40][3])+float(data[41][3])+float(data[42][3])+float(data > [43][3])\ > +float(data[44][3])+float(data[45][3])+float(data[46][3])+float(data > [47][3])\ > +float(data[48][3])+float(data[49][3])+float(data[50][3])+float(data > [51][3])\ > +float(data[52][3])+float(data[53][3])+float(data[54][3])+float(data > [55][3])\ > +float(data[56][3])+float(data[57][3])+float(data[58][3])+float(data > [59][3])\ > +float(data[60][3]) > L = sum(float(d[3]) for d in data[40 : 61]) > > Nvals[39]=(H-L)/2 > > > i=1 > while i <=38: > high=float(data[39-i][2]) > low=float(data[39-i][3]) > TR=high-low > Nvals[39-i]=(19*Nvals[40-i]+TR)/20 > i+=1 > Instead of: i=1 while i <=38: ... i+=1 you could use: for i in range(1, 39): ... Note that the start value is inclusive and the end value is exclusive. > for value in Nvals: > print value From marlowequart at hotmail.com Mon Jan 18 20:51:08 2010 From: marlowequart at hotmail.com (marlowe) Date: Mon, 18 Jan 2010 17:51:08 -0800 (PST) Subject: syntax References: <3ab2b14d-3248-4570-b332-b64ac4b37758@j14g2000yqm.googlegroups.com> Message-ID: oh, and i need to make those a new column that is added to the csv file. Thanks From gagsl-py2 at yahoo.com.ar Mon Jan 18 21:07:11 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 18 Jan 2010 23:07:11 -0300 Subject: inner methods and recursion References: Message-ID: En Mon, 18 Jan 2010 19:00:17 -0300, Steve Howell escribi?: > Hi, I have a style/design question relating to recursion and inner > methods. > > I wrote some code recently that implements a recursive algorithm that > takes several parameters from the original caller. Once the algorithm > starts running and recursing, those parameters remain the same, so I > define an inner method called "recurse" that essentially curries those > parameters. In particular, the recurse method can get passed to a > callback method that the caller supplies, so that the caller can wrap > the recursive step with their own logic. Python already have lexical scoping, you can take advantage of it. On any non-prehistoric version of Python you may write: def indent_lines(lines, branch_method, leaf_method, pass_syntax, flush_left_syntax, flush_left_empty_line, indentation_method, get_block, ): def _indent_lines(lines): output = [] while lines: # ... the real work ... # recursive call: output += branch_method(prefix, block, _indent_lines) return output return _indent_lines(lines) The real work happens inside _indent_lines, and it has access to the outer indent_lines scope, where all remaining parameters are defined. > The only thing I don't like about the technique that I'm using is that > it seems needlessly repetitive to define mostly the same parameter > list in two different places. The repetition seems mostly harmless, > but it seems like some kind of a smell that I'm overlooking a simpler > way to write the code. I just can't put my finger on what it is. Is the above technique what you were looking for? > Does anybody have any inspiration here? I vaguely recall a discussion > some time back about having some kind of @recursive decorator, but I > can't seem to find the thread. The thread I remember was about making "true" recursive calls (normal recursive calls at global or method scope are not truly recursive: they actually perform a name lookup, which may or may not yield the original function). In the above case, that doesn't happen, the recursive call resolves the _indent_lines name directly using a cell "pointing" into its container local scope. -- Gabriel Genellina From ndbecker2 at gmail.com Mon Jan 18 21:18:26 2010 From: ndbecker2 at gmail.com (Neal Becker) Date: Mon, 18 Jan 2010 21:18:26 -0500 Subject: multiprocessing question Message-ID: I'm using multiprocessing as a crude batch queuing system, like this: import my_test_program as prog (where my_test_program has a function called 'run') def run_test (args): prog.run (args[1:]) cases = [] for t in test_conditions: args = [prog.__name__]+[more args...] cases.append (args) (leaving out details, but 'cases' will be the list of test cases to run) results = pool.map (run_test, cases) Problem is, it doesn't seem to keep all my cpus busy, even though there are more test cases than cpus. Ideas? From g.bogle at auckland.no.spam.ac.nz Mon Jan 18 21:44:53 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Tue, 19 Jan 2010 15:44:53 +1300 Subject: syntax References: <3ab2b14d-3248-4570-b332-b64ac4b37758@j14g2000yqm.googlegroups.com> Message-ID: Looks like homework. From g.bogle at auckland.no.spam.ac.nz Mon Jan 18 21:48:59 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Tue, 19 Jan 2010 15:48:59 +1300 Subject: What is a list compression in Python? References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> <00de11a0$0$15570$c3e8da3@news.astraweb.com> Message-ID: Kit wrote: > Thank you so much guys. > > Just out of curiosity: can I do something like this to "square all > even numbers in the range 1-10"? > print [x^2 for x in range (1,11) if x % 2 == 0] Why not try it? From python at mrabarnett.plus.com Mon Jan 18 21:49:36 2010 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 19 Jan 2010 02:49:36 +0000 Subject: multiprocessing question In-Reply-To: References: Message-ID: <4B551DC0.2030407@mrabarnett.plus.com> Neal Becker wrote: > I'm using multiprocessing as a crude batch queuing system, like this: > > import my_test_program as prog > (where my_test_program has a function called 'run') > > def run_test (args): > prog.run (args[1:]) > > cases = [] > for t in test_conditions: > args = [prog.__name__]+[more args...] > > cases.append (args) > > (leaving out details, but 'cases' will be the list of test cases to run) > > results = pool.map (run_test, cases) > > Problem is, it doesn't seem to keep all my cpus busy, even though there are > more test cases than cpus. Ideas? > If they do a lot of I/O then that could be the bottleneck. From showell30 at yahoo.com Mon Jan 18 21:55:58 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 18 Jan 2010 18:55:58 -0800 (PST) Subject: inner methods and recursion References: Message-ID: <4d3231c3-a5c7-41b8-b2b3-ffad9201487e@l19g2000yqb.googlegroups.com> On Jan 18, 6:07?pm, "Gabriel Genellina" wrote: > En Mon, 18 Jan 2010 19:00:17 -0300, Steve Howell ? > escribi?: > > > Hi, I have a style/design question relating to recursion and inner > > methods. > > > I wrote some code recently that implements a recursive algorithm that > > takes several parameters from the original caller. ?Once the algorithm > > starts running and recursing, those parameters remain the same, so I > > define an inner method called "recurse" that essentially curries those > > parameters. ?In particular, the recurse method can get passed to a > > callback method that the caller supplies, so that the caller can wrap > > the recursive step with their own logic. > > Python already have lexical scoping, you can take advantage of it. On any ? > non-prehistoric version of Python you may write: > > def indent_lines(lines, > ? ? ? ? ? ? ?branch_method, > ? ? ? ? ? ? ?leaf_method, > ? ? ? ? ? ? ?pass_syntax, > ? ? ? ? ? ? ?flush_left_syntax, > ? ? ? ? ? ? ?flush_left_empty_line, > ? ? ? ? ? ? ?indentation_method, > ? ? ? ? ? ? ?get_block, > ? ? ? ? ? ? ?): > > ? ? ?def _indent_lines(lines): > ? ? ? ?output = [] > ? ? ? ?while lines: > ? ? ? ? ?# ... the real work ... > ? ? ? ? ?# recursive call: > ? ? ? ? ?output += branch_method(prefix, block, _indent_lines) > ? ? ? ?return output > > ? ? ?return _indent_lines(lines) > > The real work happens inside _indent_lines, and it has access to the outer ? > indent_lines scope, where all remaining parameters are defined. > > > The only thing I don't like about the technique that I'm using is that > > it seems needlessly repetitive to define mostly the same parameter > > list in two different places. ?The repetition seems mostly harmless, > > but it seems like some kind of a smell that I'm overlooking a simpler > > way to write the code. ?I just can't put my finger on what it is. > > Is the above technique what you were looking for? > That was exactly what I was looking for. I just tried it out, and it works perfectly. Thanks! From sccolbert at gmail.com Mon Jan 18 22:31:49 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Mon, 18 Jan 2010 22:31:49 -0500 Subject: PyQwt installation In-Reply-To: References: Message-ID: <7f014ea61001181931t62da8633ucb19afdef2e5769c@mail.gmail.com> On Mon, Jan 18, 2010 at 6:15 PM, Gib Bogle wrote: > Should there be a problem installing PyQwt5.2.0 with PyQt4.4.4 and > Python2.6 on Windows? My student is saying she can't get the Windows > installer to work, and she points out that the download site says "Binary > installation of PyQwt-5.2.0 on Windows for Python-2.6.x, PyQt-4.5.4" > -- > http://mail.python.org/mailman/listinfo/python-list > It's a sad day when the professor can't google. from http://pyqwt.sourceforge.net/doc5/installation.html#windows-binary-installer Windows Binary Installer Make sure that you have installed: python-2.6.2.msi numpy-1.3.0-win32-superpack-python2.6.exe PyQt-Py2.6-gpl-4.5.4-1.exe before installing PyQwt5.2.0-Python2.6-PyQt4.5.4-NumPy1.3.0-1.exe. So I would say, yes, they are probably not compatible. The PyQt4.5 brought many changes from 4.4. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven at REMOVE.THIS.cybersource.com.au Mon Jan 18 22:49:37 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 19 Jan 2010 03:49:37 GMT Subject: What is a list compression in Python? References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> <00de11a0$0$15570$c3e8da3@news.astraweb.com> Message-ID: On Mon, 18 Jan 2010 16:24:11 -0800, Kit wrote: > Thank you so much guys. > > Just out of curiosity: can I do something like this to "square all even > numbers in the range 1-10"? > print [x^2 for x in range (1,11) if x % 2 == 0] ^ is the XOR operator in Python. You want: [x**2 for x in range (1,11) if x % 2 == 0] or even better: [x**2 for x in range (2, 11, 2)] -- Steven From badouglas at gmail.com Mon Jan 18 23:04:57 2010 From: badouglas at gmail.com (tom) Date: Mon, 18 Jan 2010 20:04:57 -0800 (PST) Subject: python replace/sub/wildcard/regex issue Message-ID: <9d80d7f8-d78f-4b0b-9d8b-afae52633c8a@b10g2000yqa.googlegroups.com> hi... trying to figure out how to solve what should be an easy python/regex/ wildcard/replace issue. i've tried a number of different approaches.. so i must be missing something... my initial sample text are: Soo ChoiLONGEDITBOX">Apryl Berney Soo ChoiLONGEDITBOX">Joel Franks Joel FranksGEDITBOX">Alexander Yamato and i'm trying to get Soo Choi foo Apryl Berney Soo Choi foo Joel Franks Joel Franks foo Alexander Yamato the issue i'm facing.. is how to start at "' and substitute inclusive of the stuff inside the regex... i've tried derivations of name=re.sub("]*\">"," foo ",name) but i'm missing something... thoughts... thanks tom From badouglas at gmail.com Mon Jan 18 23:04:57 2010 From: badouglas at gmail.com (tom) Date: Mon, 18 Jan 2010 20:04:57 -0800 (PST) Subject: python replace/sub/wildcard/regex issue Message-ID: <9d80d7f8-d78f-4b0b-9d8b-afae52633c8a@b10g2000yqa.googlegroups.com> hi... trying to figure out how to solve what should be an easy python/regex/ wildcard/replace issue. i've tried a number of different approaches.. so i must be missing something... my initial sample text are: Soo ChoiLONGEDITBOX">Apryl Berney Soo ChoiLONGEDITBOX">Joel Franks Joel FranksGEDITBOX">Alexander Yamato and i'm trying to get Soo Choi foo Apryl Berney Soo Choi foo Joel Franks Joel Franks foo Alexander Yamato the issue i'm facing.. is how to start at "' and substitute inclusive of the stuff inside the regex... i've tried derivations of name=re.sub("]*\">"," foo ",name) but i'm missing something... thoughts... thanks tom From clp2 at rebertia.com Mon Jan 18 23:31:55 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Mon, 18 Jan 2010 20:31:55 -0800 Subject: python replace/sub/wildcard/regex issue In-Reply-To: <9d80d7f8-d78f-4b0b-9d8b-afae52633c8a@b10g2000yqa.googlegroups.com> References: <9d80d7f8-d78f-4b0b-9d8b-afae52633c8a@b10g2000yqa.googlegroups.com> Message-ID: <50697b2c1001182031v893cfecyfa7dc540cabb3a@mail.gmail.com> On Mon, Jan 18, 2010 at 8:04 PM, tom wrote: > hi... > > trying to figure out how to solve what should be an easy python/regex/ > wildcard/replace issue. > > i've tried a number of different approaches.. so i must be missing > something... > > my initial sample text are: > > Soo ChoiLONGEDITBOX">Apryl Berney > Soo ChoiLONGEDITBOX">Joel Franks > Joel FranksGEDITBOX">Alexander Yamato > > and i'm trying to get > > Soo Choi foo Apryl Berney > Soo Choi foo Joel Franks > Joel Franks foo Alexander Yamato > > the issue i'm facing.. is how to start at "' and > substitute inclusive of the stuff inside the regex... > > i've tried derivations of > > name=re.sub("]*\">"," foo ",name) > > but i'm missing something... > > thoughts... thanks "Some people, when confronted with a problem, think 'I know, I'll use regular expressions.' Now they have two problems." Assuming your sample text is representative of all your test: new_text = "\n".join(line[:line.index('<')] + line[line.rindex('>')+1:] for line in your_text.split('\n')) Cheers, Chris -- http://blog.rebertia.com From wuwei23 at gmail.com Mon Jan 18 23:33:53 2010 From: wuwei23 at gmail.com (alex23) Date: Mon, 18 Jan 2010 20:33:53 -0800 (PST) Subject: python replace/sub/wildcard/regex issue References: <9d80d7f8-d78f-4b0b-9d8b-afae52633c8a@b10g2000yqa.googlegroups.com> Message-ID: On Jan 19, 2:04?pm, tom wrote: > trying to figure out how to solve what should be an easy python/regex/ > wildcard/replace issue. > but i'm missing something... Well, some would say you've missed the most obvious solution of _not_ using regexps :) I'd probably do it via string methods wrapped up in a helper function: >>> def extract(text): ... first, rest = text.split('<', 1) ... ignore, last = rest.rsplit('>', 1) ... return '%s foo %s' % (first, last) ... >>> extract('Soo ChoiLONGEDITBOX">Apryl Berney') 'Soo Choi foo Apryl Berney' >>> extract('Soo ChoiLONGEDITBOX">Joel Franks') 'Soo Choi foo Joel Franks' >>> extract('Joel FranksGEDITBOX">Alexander Yamato') 'Joel Franks foo Alexander Yamato' From bedouglas at earthlink.net Mon Jan 18 23:38:18 2010 From: bedouglas at earthlink.net (tom) Date: Mon, 18 Jan 2010 20:38:18 -0800 Subject: python replace/sub/wildcard/regex issue In-Reply-To: <50697b2c1001182031v893cfecyfa7dc540cabb3a@mail.gmail.com> Message-ID: <19b501ca98c1$39ae6140$0301a8c0@tmesa.com> hi chris... should the reply you sent me be implemented all on one line? like: test='Soo ChoiLONGEDITBOX">Apryl Berney' new_text = "\n".join(line[:line.index('<')] + line[line.rindex('>')+1:] for line in test.split('\n')) and what would line be? thanks -----Original Message----- From: python-list-bounces+bedouglas=earthlink.net at python.org [mailto:python-list-bounces+bedouglas=earthlink.net at python.org]On Behalf Of Chris Rebert Sent: Monday, January 18, 2010 8:32 PM To: tom Cc: python-list at python.org Subject: Re: python replace/sub/wildcard/regex issue On Mon, Jan 18, 2010 at 8:04 PM, tom wrote: > hi... > > trying to figure out how to solve what should be an easy python/regex/ > wildcard/replace issue. > > i've tried a number of different approaches.. so i must be missing > something... > > my initial sample text are: > > Soo ChoiLONGEDITBOX">Apryl Berney > Soo ChoiLONGEDITBOX">Joel Franks > Joel FranksGEDITBOX">Alexander Yamato > > and i'm trying to get > > Soo Choi foo Apryl Berney > Soo Choi foo Joel Franks > Joel Franks foo Alexander Yamato > > the issue i'm facing.. is how to start at "' and > substitute inclusive of the stuff inside the regex... > > i've tried derivations of > > name=re.sub("]*\">"," foo ",name) > > but i'm missing something... > > thoughts... thanks "Some people, when confronted with a problem, think 'I know, I'll use regular expressions.' Now they have two problems." Assuming your sample text is representative of all your test: new_text = "\n".join(line[:line.index('<')] + line[line.rindex('>')+1:] for line in your_text.split('\n')) Cheers, Chris -- http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list From clp2 at rebertia.com Mon Jan 18 23:40:46 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Mon, 18 Jan 2010 20:40:46 -0800 Subject: python replace/sub/wildcard/regex issue In-Reply-To: <50697b2c1001182031v893cfecyfa7dc540cabb3a@mail.gmail.com> References: <9d80d7f8-d78f-4b0b-9d8b-afae52633c8a@b10g2000yqa.googlegroups.com> <50697b2c1001182031v893cfecyfa7dc540cabb3a@mail.gmail.com> Message-ID: <50697b2c1001182040v1a070bc0uf610d8e2246a74a2@mail.gmail.com> On Mon, Jan 18, 2010 at 8:31 PM, Chris Rebert wrote: > On Mon, Jan 18, 2010 at 8:04 PM, tom wrote: >> hi... >> >> trying to figure out how to solve what should be an easy python/regex/ >> wildcard/replace issue. >> >> i've tried a number of different approaches.. so i must be missing >> something... >> >> my initial sample text are: >> >> Soo ChoiLONGEDITBOX">Apryl Berney >> Soo ChoiLONGEDITBOX">Joel Franks >> Joel FranksGEDITBOX">Alexander Yamato >> >> and i'm trying to get >> >> Soo Choi foo Apryl Berney >> Soo Choi foo Joel Franks >> Joel Franks foo Alexander Yamato >> >> the issue i'm facing.. is how to start at "' and >> substitute inclusive of the stuff inside the regex... >> >> i've tried derivations of >> >> name=re.sub("]*\">"," foo ",name) >> >> but i'm missing something... >> >> thoughts... thanks > > "Some people, when confronted with a problem, think 'I know, I'll use > regular expressions.' Now they have two problems." > > Assuming your sample text is representative of all your test: > > new_text = "\n".join(line[:line.index('<')] + line[line.rindex('>')+1:] for line in your_text.split('\n')) Erm, remembering to intersperse the "foo" (should be all 1-line, bloody Gmail): new_text = "\n".join(line[:line.index('<')] + " foo " + line[line.rindex('>')+1:] for line in your_text.split('\n')) Or just use alex23's method, which seems all-round superior. :-) Cheers, Chris From anandvaidya.ml at gmail.com Tue Jan 19 00:05:26 2010 From: anandvaidya.ml at gmail.com (Anand Vaidya) Date: Mon, 18 Jan 2010 21:05:26 -0800 (PST) Subject: Generic Python Benchmark suite? References: Message-ID: <08431d7e-7ce9-40d0-8a9c-1c2b885ce7b4@p8g2000yqb.googlegroups.com> On Jan 19, 5:42?am, Terry Reedy wrote: > On 1/18/2010 4:58 AM, Anand Vaidya wrote: > > > Is there a generic python benchmark suite in active development? I am > > looking forward to comparing some code on various python > > implementations (primarily CPython 2.x, CPython 3.x, UnladenSwallow, > > Psyco). > > You might find this interesting if you have not seen it before:http://code.google.com/p/unladen-swallow/wiki/Benchmarks @Antoine, Terry, Thanks for the suggestions. I will investigate those. I just ran the pybench, doesn't run on 3.x, 2to3 fails. I was hoping to locate something like Phoronix Test Suite (that is a suite of microbenchmarks for Linux OS) Regards Anand From wissme at hotmail.com Tue Jan 19 02:09:03 2010 From: wissme at hotmail.com (Jean Guillaume Pyraksos) Date: Tue, 19 Jan 2010 08:09:03 +0100 Subject: Python IDE for MacOS-X Message-ID: What's the best one to use with beginners ? Something with integrated syntax editor, browser of doc... Thanks, JG From moekaveli at gmail.com Tue Jan 19 02:18:30 2010 From: moekaveli at gmail.com (Dr. Benjamin David Clarke) Date: Mon, 18 Jan 2010 23:18:30 -0800 (PST) Subject: Updating an OptionMenu every time the text file it reads from is updated (Tkinter) Message-ID: <45d53456-0673-4829-aa73-9c98feba7b64@s31g2000yqs.googlegroups.com> I currently have a program that reads in values for an OptionMenu from a text file. I also have an option to add a line to that text file which corresponds to a new value for that OptionMenu. How can I make that OptionMenu update its values based on that text file without restarting the program? In other words, every time I add a value to the text file, I want the OptionMenu to immediately update to take note of this change. I'll provide code if needed. From mal at egenix.com Tue Jan 19 03:52:22 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 19 Jan 2010 09:52:22 +0100 Subject: Generic Python Benchmark suite? In-Reply-To: References: Message-ID: <4B5572C6.9040009@egenix.com> Anand Vaidya wrote: > Is there a generic python benchmark suite in active development? I am > looking forward to comparing some code on various python > implementations (primarily CPython 2.x, CPython 3.x, UnladenSwallow, > Psyco). > > I am happy with something that gives me a relative number eg: ULS is > 30% faster than CPy2.x etc > > I found pybench which is probably not maintained actively. Oh, it is. In fact, I'm preparing a new version for Python 2.7. > What do you suggest? > > PS: I think a benchmark should cover file / network, database I/O, > data structures (dict, list etc), object creation/manipulation, > numbers, measure looping inefficiencies, effects of caching (memcache > etc) at the minimum pybench addresses many of the low-level aspects you're asking for. It doesn't have an I/O tests, since these usually don't have much to do with Python's performance, but rather that of the underlying OS and hardware. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 19 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From stefan_ml at behnel.de Tue Jan 19 04:41:20 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 19 Jan 2010 10:41:20 +0100 Subject: searching and storing large quantities of xml! In-Reply-To: References: <4b52bdb8$0$6733$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <4b557e40$0$6586$9b4e6d93@newsspool3.arcor-online.net> dads, 18.01.2010 22:39: > There was one thing that I forgot about - when ElementTree fails to > parse due to an element not being closed why doesn't it close the file > like object. Because it didn't open it? Stefan From rajat.dudeja at gmail.com Tue Jan 19 04:51:02 2010 From: rajat.dudeja at gmail.com (Rajat) Date: Tue, 19 Jan 2010 01:51:02 -0800 (PST) Subject: thread return code Message-ID: <08974f1d-4334-4165-8272-7c178f589bbc@m16g2000yqc.googlegroups.com> Hi, I'm using threading module in Python 2.6.4. I'm using thread's join() method. On the new thread I'm running a function which returns a code at the end. Is there a way I access that code in the parent thread after thread finishes? Simply, does join() could get me that code? Regards, Rajat From ted at sjksdjk.it Tue Jan 19 04:58:49 2010 From: ted at sjksdjk.it (ted) Date: Tue, 19 Jan 2010 10:58:49 +0100 Subject: python gui ide under linux..like visual studio ;) ? In-Reply-To: <3f3650e2-1c9d-43e1-b759-6c3d939de7bb@h9g2000yqa.googlegroups.com> References: <4b5470ce$0$1103$4fafbaef@reader4.news.tin.it> <3f3650e2-1c9d-43e1-b759-6c3d939de7bb@h9g2000yqa.googlegroups.com> Message-ID: <4b558226$0$823$4fafbaef@reader5.news.tin.it> Il 18/01/2010 21:59, Mike Driscoll ha scritto: > On Jan 18, 8:32 am, ted wrote: >> Hi at all... >> Can someone please give me some advice, about a good IDE with control >> GUI under Linux ? >> >> Actually i know QT Creator by Nokia which i can use with Python (but i >> don't know how). >> >> And, a good library for access to database (mysql, sql server, oracle) ? >> >> Thank you very much ! >> >> bye > > Check out Dabo for the database stuff: http://dabodev.com/ > > There is nothing like Visual Studio. The closest I've found are things > like the QT Creator and wxGlade / Boa Constructor / wxFormBuilder. I > know I've heard of another one that was commercial, but I can't recall > the name at the moment. > > ------------------- > Mike Driscoll > > Blog: http://blog.pythonlibrary.org > > PyCon 2010 Atlanta Feb 19-21 http://us.pycon.org/ Thank you very much. Bye! From alfps at start.no Tue Jan 19 05:25:55 2010 From: alfps at start.no (Alf P. Steinbach) Date: Tue, 19 Jan 2010 11:25:55 +0100 Subject: thread return code In-Reply-To: <08974f1d-4334-4165-8272-7c178f589bbc@m16g2000yqc.googlegroups.com> References: <08974f1d-4334-4165-8272-7c178f589bbc@m16g2000yqc.googlegroups.com> Message-ID: * Rajat: > Hi, > > I'm using threading module in Python 2.6.4. I'm using thread's join() > method. > > On the new thread I'm running a function which returns a code at the > end. Is there a way I access that code in the parent thread after > thread finishes? Simply, does join() could get me that code? join() always returns None. But you can store the code in the thread object, and then access it after the join(). Cheers & hth., - Alf From chris at simplistix.co.uk Tue Jan 19 05:56:34 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 19 Jan 2010 10:56:34 +0000 Subject: what test runner should I use? Message-ID: <4B558FE2.40701@simplistix.co.uk> Hi All, I'm wondering what test runner I should use. Here's my list of requirements: - cross platform (I develop for and on Windows, Linux and Mac) - should not prevent tests from running with other test runners (so no plugins/layers/etc that only work with one specific test runner) - should work with zc.buildout (preferably without a specialist recipe!) So far I've tried the following with the resultant problems: zope.testing - requires a special recipe to be useful - now emits deprecation warnings from itself: https://mail.zope.org/pipermail/zope-dev/2009-December/038965.html - coverage support is baroque to put it politely twisted's trial - only has old-style script definition in setup.py, so doesn't work with buildout without hackery - drops _twisted_trial folders all over the place and doesn't clear them up nose - can't see to get it to run only my packages tests, rather than including the tests of packages my package depends on - seems to be focused towards files rather than modules (which makes it not play nicely with buildout) - seems to be difficult to provide options to at configuration time that can then be overridden on the command line I did also look at py.test's homepage but found it pretty scary. What other options do people recommend? Failing that, any ideas how to fix the problems above? cheers, Chris From alfps at start.no Tue Jan 19 05:57:09 2010 From: alfps at start.no (Alf P. Steinbach) Date: Tue, 19 Jan 2010 11:57:09 +0100 Subject: Updating an OptionMenu every time the text file it reads from is updated (Tkinter) In-Reply-To: <45d53456-0673-4829-aa73-9c98feba7b64@s31g2000yqs.googlegroups.com> References: <45d53456-0673-4829-aa73-9c98feba7b64@s31g2000yqs.googlegroups.com> Message-ID: * Dr. Benjamin David Clarke: > I currently have a program that reads in values for an OptionMenu from > a text file. I also have an option to add a line to that text file > which corresponds to a new value for that OptionMenu. How can I make > that OptionMenu update its values based on that text file without > restarting the program? In other words, every time I add a value to > the text file, I want the OptionMenu to immediately update to take > note of this change. I'll provide code if needed. It's a bit unclear to me whether the update of the text file is from within the process as where the menu is, or not. If the problem is connecting file updates to menu changes: If it's the same process, perhaps you can wrap all access of the text file so that whenever an operation to change the text file is performed, it calls back on interested parties (like event handling)? If it's not the same process you need some "file changed" notification. Windows has this functionality. I don't know if it's there in *nix. Anyway, since I'm still essentially a Python newbie I don't know of any Python modules for such functionality, if that's what you need. But perhaps someone else does... Otherwise, if the problem is actually updating the menu, and it is a tkinter OptionMenu: It appears that an OptionMenu has a logical attribute 'menu' that is a tkinter Menu, representing the options; if your OptionMenu is an object 'o' then you can write 'o["menu"]' or 'o.cget( "menu" )' to get at that logical attribute. And tkinter Menu has various methods that you can use to inspect or update the menu; see . Disclaimer: I haven't tried updating, but I see no reason why it shouldn't work. :-) Cheers & hth., - Alf From michels at mps.mpg.de Tue Jan 19 06:06:12 2010 From: michels at mps.mpg.de (Helmut Michels) Date: Tue, 19 Jan 2010 12:06:12 +0100 Subject: [ANN] Data Plotting Library DISLIN 10.0 Message-ID: Dear Pytnon users, I am pleased to announce version 10.0 of the data plotting software DISLIN. DISLIN is a high-level and easy to use plotting library for displaying data as curves, bar graphs, pie charts, 3D-colour plots, surfaces, contours and maps. Several output formats are supported such as X11, VGA, PostScript, PDF, CGM, WMF, HPGL, TIFF, GIF, PNG, BMP and SVG. The software is available for the most C, Fortran 77 and Fortran 90/95 compilers. Plotting extensions for the interpreting languages Perl, Python and Java are also supported. DISLIN distributions and manuals in PDF, PostScript and HTML format are available from the DISLIN home page http://www.dislin.de and via FTP from the server ftp://ftp.gwdg.de/pub/grafik/dislin All DISLIN distributions are free for non-commercial use. Licenses for commercial use are available from the site http://www.dislin.de. ------------------- Helmut Michels Max Planck Institute for Solar System Research Phone: +49 5556 979-334 Max-Planck-Str. 2 Fax : +49 5556 979-240 D-37191 Katlenburg-Lindau Mail : michels at mps.mpg.de From fossist at gmail.com Tue Jan 19 06:16:30 2010 From: fossist at gmail.com (fossist) Date: Tue, 19 Jan 2010 03:16:30 -0800 (PST) Subject: Is HTML report of tests run using PyUnit (unittest) possible? Message-ID: <584f31d6-4c4d-4661-836f-c8552d892d38@j19g2000yqk.googlegroups.com> I am using PyUnit (unittest module) for loading test cases from our modules and executing them. Is it possible to export the test results as HTML report? Currently the results appear as text on standard output while the tests execute. But is there something out of the box available in PyUnit to make this possible? From stefan_ml at behnel.de Tue Jan 19 06:42:02 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 19 Jan 2010 12:42:02 +0100 Subject: twenty years ago Guido created Python In-Reply-To: <555a57f6-6593-47d5-9a3a-1fe01a91b51d@z7g2000vbl.googlegroups.com> References: <871vi628ft.fsf@benfinney.id.au> <871vi6zhfp.fsf@benfinney.id.au> <87oclaxksq.fsf@benfinney.id.au> <877hrxy3zz.fsf@benfinney.id.au> <02d9bd25-5ae0-4de3-910d-dc36bec0fc11@m3g2000yqf.googlegroups.com> <555a57f6-6593-47d5-9a3a-1fe01a91b51d@z7g2000vbl.googlegroups.com> Message-ID: <4b559a8a$0$6582$9b4e6d93@newsspool3.arcor-online.net> n00m, 06.01.2010 01:25: > My mom a little girl wsas pursued by Germans alsatians dogs in a > forest > Germans just made a fun of it. > They screamead Zurich and laughed They laughed at her scare and you > now teach me. Now her hands tremble froom malntrination > She ate ate at WWII grass. And you teach me? Sorry, this thread wasn't long enough yet to put Godwin's law in action. Stefan From michele.simionato at gmail.com Tue Jan 19 06:44:09 2010 From: michele.simionato at gmail.com (Michele Simionato) Date: Tue, 19 Jan 2010 03:44:09 -0800 (PST) Subject: Inheriting methods but over-riding docstrings References: <0361ea2d$0$1336$c3e8da3@news.astraweb.com> Message-ID: <28aa31f4-7331-4ac2-918b-0c59f5a834de@e37g2000yqn.googlegroups.com> On Jan 16, 6:55?pm, Steven D'Aprano wrote: > I have a series of subclasses that inherit methods from a base class, but > I'd like them to have their own individual docstrings. The following is not tested more than you see and will not work for builtin methods, but it should work in the common cases: from types import FunctionType, CodeType def newfunc(func, docstring): c = func.func_code nc = CodeType(c.co_argcount, c.co_nlocals, c.co_stacksize, c.co_flags, c.co_code, c.co_consts, c.co_names, c.co_varnames, c.co_filename, func.__name__, c.co_firstlineno, c.co_lnotab, c.co_freevars, c.co_cellvars) nf = FunctionType(nc, func.func_globals, func.__name__) nf.__doc__ = docstring return nf def setdocstring(method, docstring): cls = method.im_class basefunc = getattr(super(cls, cls), method.__name__).im_func setattr(cls, method.__name__, newfunc(basefunc, docstring)) # example of use class B(object): def m(self): "base" return 'ok' class C(B): pass setdocstring(C.m, 'C.m docstring') print B.m.__doc__ # the base docstring print C.m.__doc__ # the new docstring From __peter__ at web.de Tue Jan 19 07:00:10 2010 From: __peter__ at web.de (Peter Otten) Date: Tue, 19 Jan 2010 13:00:10 +0100 Subject: Updating an OptionMenu every time the text file it reads from is updated (Tkinter) References: <45d53456-0673-4829-aa73-9c98feba7b64@s31g2000yqs.googlegroups.com> Message-ID: Dr. Benjamin David Clarke wrote: > I currently have a program that reads in values for an OptionMenu from > a text file. I also have an option to add a line to that text file > which corresponds to a new value for that OptionMenu. How can I make > that OptionMenu update its values based on that text file without > restarting the program? In other words, every time I add a value to > the text file, I want the OptionMenu to immediately update to take > note of this change. I'll provide code if needed. Inferred from looking into the Tkinter source code: # python 2.6 import Tkinter as tk root = tk.Tk() var = tk.StringVar() var.set("One") optionmenu = tk.OptionMenu(root, var, "One", "Two", "Three") optionmenu.grid(row=0, column=1) def add_option(): value = entry_add.get() menu = optionmenu["menu"] variable = var command = None # what you passed as command argument to optionmenu menu.add_command(label=value, command=tk._setit(variable, value, command)) label_show = tk.Label(root, text="current value") label_show.grid(row=1, column=0) entry_show = tk.Entry(root, textvariable=var) entry_show.grid(row=1, column=1) label_add = tk.Label(root, text="new option") label_add.grid(row=2, column=0) entry_add = tk.Entry(root) entry_add.grid(row=2, column=1) button_add = tk.Button(root, text="add option", command=add_option) button_add.grid(row=2, column=2) root.mainloop() Peter From solipsis at pitrou.net Tue Jan 19 07:04:13 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 19 Jan 2010 12:04:13 +0000 (UTC) Subject: Generic Python Benchmark suite? References: <08431d7e-7ce9-40d0-8a9c-1c2b885ce7b4@p8g2000yqb.googlegroups.com> Message-ID: Le Mon, 18 Jan 2010 21:05:26 -0800, Anand Vaidya a ?crit?: > @Antoine, Terry, > > Thanks for the suggestions. > > I will investigate those. I just ran the pybench, doesn't run on 3.x, > 2to3 fails. You just have to use the pybench version that is bundled with 3.x (in the Tools directory). From utente at esempio.net Tue Jan 19 07:12:14 2010 From: utente at esempio.net (superpollo) Date: Tue, 19 Jan 2010 13:12:14 +0100 Subject: [ANN] Data Plotting Library DISLIN 10.0 In-Reply-To: References: Message-ID: <4b55a19f$0$1106$4fafbaef@reader4.news.tin.it> Helmut Michels ha scritto: > Dear Pytnon users, > > I am pleased to announce version 10.0 of the data plotting software > DISLIN. why dont you make it free software (i mean. GPL'ed) bye From david.mcwright at usbfmi.com Tue Jan 19 07:23:00 2010 From: david.mcwright at usbfmi.com (dippim) Date: Tue, 19 Jan 2010 04:23:00 -0800 (PST) Subject: python replace/sub/wildcard/regex issue References: <9d80d7f8-d78f-4b0b-9d8b-afae52633c8a@b10g2000yqa.googlegroups.com> Message-ID: <6a7693f0-88d8-48b9-ac3a-896f5d02643b@l30g2000yqb.googlegroups.com> On Jan 18, 11:04?pm, tom wrote: > hi... > > trying to figure out how to solve what should be an easy python/regex/ > wildcard/replace issue. > > i've tried a number of different approaches.. so i must be missing > something... > > my initial sample text are: > > Soo ChoiLONGEDITBOX">Apryl Berney > Soo ChoiLONGEDITBOX">Joel Franks > Joel FranksGEDITBOX">Alexander Yamato > > and i'm trying to get > > Soo Choi foo Apryl Berney > Soo Choi foo Joel Franks > Joel Franks foo Alexander Yamato > > the issue i'm facing.. is how to start at "' and > substitute inclusive of the stuff inside the regex... > > i've tried derivations of > > name=re.sub("]*\">"," foo ",name) > > but i'm missing something... > > thoughts... thanks > > tom The problem here is that ]* consumes anything that's not > and then stops when it hits something that is >. So, [^>]* consumes "pan" in each case, then tries to match \">, but fails since there isn't a ", so the match ends. It never makes it to the second >. I agree with Chris Rebert, regexes are dangerous because the number of possible cases where you can match isn't always clear (see the above explanation :). Also, if the number of comparisons you have to do isn't high, they can be inefficient. However, for your limited set of examples the following should work: aList = ['Soo ChoiLONGEDITBOX">Apryl Berney', 'Soo ChoiLONGEDITBOX">Joel Franks', 'Joel FranksGEDITBOX">Alexander Yamato'] matcher = re.compile(r"<[\w\W]*>") newList = [] for x in aList: newList.append(matcher.sub(" foo ", x)) print newList David From jeanmichel at sequans.com Tue Jan 19 07:33:54 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Tue, 19 Jan 2010 13:33:54 +0100 Subject: using super In-Reply-To: References: <4B54BB95.1090407@sequans.com> Message-ID: <4B55A6B2.107@sequans.com> Gabriel Genellina wrote: >> I see. >> Then is there a reason why >> return super(Subclass, self).parrot() >> would be prefered over the "classic" >> return Base.parrot(self) >> ? >> Or is it just a matter of preference ? > > For a longer explanation, see: > > James Knight: Python's Super Considered Harmful > http://fuhm.net/super-harmful/ > > Michele Simionato: Things to Know About Python Super > http://www.artima.com/weblogs/viewpost.jsp?thread=236275 > http://www.artima.com/weblogs/viewpost.jsp?thread=236278 > http://www.artima.com/weblogs/viewpost.jsp?thread=237121 > Thanks to all who replied to this thread. I didn't remember why I didn't want to dive into super in the first place, now I remember :o) I'm sure about one thing about super: it has a misleading name. JM From gnarlodious at gmail.com Tue Jan 19 08:03:52 2010 From: gnarlodious at gmail.com (Gnarlodious) Date: Tue, 19 Jan 2010 05:03:52 -0800 (PST) Subject: Py 3: Terminal script can't find relative path References: <58e9b1e4-049a-4a7f-bea9-8d7e7e9f191a@e27g2000yqd.googlegroups.com> <87d417578z.fsf@castleamber.com> Message-ID: <9541dcf0-8060-4851-bdbd-0aed6de13bf9@v25g2000yqk.googlegroups.com> On Jan 18, 4:21?pm, John Bokma wrote: > Gnarlodious writes: > > I am running a script in a browser that finds the file in subfolder > > Data: > > > Content=Plist('Data/Content.plist') > > > However, running the same script in Terminal errors: > > > IOError: [Errno 2] No such file or directory: 'Data/Content.plist' > > What does: > > ls -l Data/Content.plist > > in the terminal give? I can replace with absolute paths and it works as expected. Could this be a Python 3 bug? Where as a CGI script it finds the relative path but not in Terminal? -- Gnarlie From rpjday at crashcourse.ca Tue Jan 19 10:29:08 2010 From: rpjday at crashcourse.ca (Robert P. J. Day) Date: Tue, 19 Jan 2010 10:29:08 -0500 (EST) Subject: can i examine the svn rev used to build a python 3 executable? Message-ID: i'm currently checking out python3 from the svn repo, configuring, building and installing under /usr/local/bin on my fedora 12 system, all good. i'm curious, though -- is there a query i can make of that executable that would tell me what svn rev it was built from? i'm guessing not, but i thought i'd ask. rday -- ======================================================================== Robert P. J. Day Waterloo, Ontario, CANADA Linux Consulting, Training and Kernel Pedantry. Web page: http://crashcourse.ca Twitter: http://twitter.com/rpjday ======================================================================== From gerald.britton at gmail.com Tue Jan 19 10:30:21 2010 From: gerald.britton at gmail.com (Gerald Britton) Date: Tue, 19 Jan 2010 10:30:21 -0500 Subject: Performance of lists vs. list comprehensions Message-ID: <5d1a32001001190730u4fa2f26cn62e79c2c73a08b@mail.gmail.com> Yesterday I stumbled across some old code in a project I was working on. It does something like this: mystring = '\n'.join( [ line for line in lines if ] ) where "lines" is a simple list of strings. I realized that the code had been written before you could put a list comprehension as an argument to join(). I figured that I could drop the '[' and ']' and leave the rest as a list comprehension since that works with current Python (and the project's base is 2.5). So I rewrote the original statement like this: mystring = '\n'.join( line for line in lines if ) It works as expected. Then I got curious as to how it performs. I was surprised to learn that the rewritten expression runs more than twice as _slow_. e.g.: >>> l ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9'] >>> Timer("' '.join([x for x in l])", 'l = map(str,range(10))').timeit() 2.9967339038848877 >>> Timer("' '.join(x for x in l)", 'l = map(str,range(10))').timeit() 7.2045478820800781 Notice that I dropped the condition testing that was in my original code. I just wanted to see the effect of two different expressions. I thought that maybe there was some lower bound on the number of the items in the list or list comprehension beyond which the comprehension would prove more efficient. There doesn't appear to be one. I scaled the length of the input list up to 1 million items and got more or less the same relative performance. Now I'm really curious and I'd like to know: 1. Can anyone else confirm this observation? 2. Why should the "pure" list comprehension be slower than the same comprehension enclosed in '[...]' ? -- Gerald Britton From rpjday at crashcourse.ca Tue Jan 19 10:32:31 2010 From: rpjday at crashcourse.ca (Robert P. J. Day) Date: Tue, 19 Jan 2010 10:32:31 -0500 (EST) Subject: can i examine the svn rev used to build a python 3 executable? In-Reply-To: References: Message-ID: On Tue, 19 Jan 2010, Robert P. J. Day wrote: > i'm currently checking out python3 from the svn repo, configuring, > building and installing under /usr/local/bin on my fedora 12 system, > all good. > > i'm curious, though -- is there a query i can make of that > executable that would tell me what svn rev it was built from? i'm > guessing not, but i thought i'd ask. never mind. just discovered that while "python3 -V" won't do it, executing it gives me: $ python3 Python 3.2a0 (py3k:77609, Jan 19 2010, 04:10:16) ... and it's that 77609 rev number i was after. rday -- ======================================================================== Robert P. J. Day Waterloo, Ontario, CANADA Linux Consulting, Training and Kernel Pedantry. Web page: http://crashcourse.ca Twitter: http://twitter.com/rpjday ======================================================================== From ryniek90 at gmail.com Tue Jan 19 10:40:02 2010 From: ryniek90 at gmail.com (ryniek90) Date: Tue, 19 Jan 2010 16:40:02 +0100 Subject: python gui ide under linux..like visual studio ;) ? In-Reply-To: References: Message-ID: <4B55D252.8020800@gmail.com> > Il 18/01/2010 21:59, Mike Driscoll ha scritto: > >> > On Jan 18, 8:32 am, ted wrote: >> >>> >> Hi at all... >>> >> Can someone please give me some advice, about a good IDE with control >>> >> GUI under Linux ? >>> >> >>> >> Actually i know QT Creator by Nokia which i can use with Python (but i >>> >> don't know how). >>> >> >>> >> And, a good library for access to database (mysql, sql server, oracle) ? >>> >> >>> >> Thank you very much ! >>> >> >>> >> bye >>> >> > >> > Check out Dabo for the database stuff:http://dabodev.com/ >> > >> > There is nothing like Visual Studio. The closest I've found are things >> > like the QT Creator and wxGlade / Boa Constructor / wxFormBuilder. I >> > know I've heard of another one that was commercial, but I can't recall >> > the name at the moment. >> > >> > ------------------- >> > Mike Driscoll >> > >> > Blog:http://blog.pythonlibrary.org >> > >> > PyCon 2010 Atlanta Feb 19-21http://us.pycon.org/ >> > Thank you very much. > > Bye! > > > Hi, check this Python Wiki pages: http://wiki.python.org/moin/IntegratedDevelopmentEnvironments Cheers. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gnarlodious at gmail.com Tue Jan 19 10:50:54 2010 From: gnarlodious at gmail.com (Gnarlodious) Date: Tue, 19 Jan 2010 07:50:54 -0800 (PST) Subject: Py 3: How to switch application to Unicode strings? Message-ID: <92288825-8b01-47fb-8592-f65a7d7aeb8a@k35g2000yqb.googlegroups.com> I am using Python 3, getting an error from SQLite: sqlite3.ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. So... how do I switch to Unicode? I thought I was doing it when I put # coding:utf-8 at the start of my script. -- Gnarlie From alfps at start.no Tue Jan 19 10:57:39 2010 From: alfps at start.no (Alf P. Steinbach) Date: Tue, 19 Jan 2010 16:57:39 +0100 Subject: Performance of lists vs. list comprehensions In-Reply-To: References: Message-ID: * Gerald Britton: > Yesterday I stumbled across some old code in a project I was working > on. It does something like this: > > mystring = '\n'.join( [ line for line in lines if depending on line> ] ) > > where "lines" is a simple list of strings. I realized that the code > had been written before you could put a list comprehension as an > argument to join(). I figured that I could drop the '[' and ']' and > leave the rest as a list comprehension since that works with current > Python (and the project's base is 2.5). So I rewrote the original > statement like this: > > mystring = '\n'.join( line for line in lines if depending on line> ) > > It works as expected. Then I got curious as to how it performs. I > was surprised to learn that the rewritten expression runs more than > twice as _slow_. e.g.: > >>>> l > ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9'] > >>>> Timer("' '.join([x for x in l])", 'l = map(str,range(10))').timeit() > 2.9967339038848877 > >>>> Timer("' '.join(x for x in l)", 'l = map(str,range(10))').timeit() > 7.2045478820800781 > > Notice that I dropped the condition testing that was in my original > code. I just wanted to see the effect of two different expressions. > > I thought that maybe there was some lower bound on the number of the > items in the list or list comprehension beyond which the comprehension > would prove more efficient. There doesn't appear to be one. I scaled > the length of the input list up to 1 million items and got more or > less the same relative performance. > > Now I'm really curious and I'd like to know: > > 1. Can anyone else confirm this observation? >>> Timer("' '.join([x for x in l])", 'l = map(str,range(10))').timeit() 5.8625191190500345 >>> Timer("' '.join(x for x in l)", 'l = map(str,range(10))').timeit() 12.093135300715574 >>> _ > 2. Why should the "pure" list comprehension be slower than the same > comprehension enclosed in '[...]' ? Regarding (2) the unparenthesized expression in join is *not* a list comprehension but a generator expression. And as such it involves join calling next() on the generator object repeatedly, with each next() call involving a light-weight context shift. In addition the docs mumble something about "lazy" evaluation, and that may also contribute to the overhead. I think that in contrast, the interpreter can evaluate a list comprehension, [x for x in blah], directly without any context shifting, just by transforming it to equivalent code and putting the target expressions innermost there. And so the main factor causing a slowdown for a list comprehension would, I think, be paging and such if the list it produced was Really Huge, while for the generator there's no memory issue but rather much calling & context shifting. Cheers & hth., - Alf From gerald.britton at gmail.com Tue Jan 19 11:10:43 2010 From: gerald.britton at gmail.com (Gerald Britton) Date: Tue, 19 Jan 2010 11:10:43 -0500 Subject: Performance of lists vs. list comprehensions In-Reply-To: References: Message-ID: <5d1a32001001190810w7d0c78b2r8723185e2d8c23f8@mail.gmail.com> Thanks! Good explanation. On Tue, Jan 19, 2010 at 10:57 AM, Alf P. Steinbach wrote: > * Gerald Britton: >> >> Yesterday I stumbled across some old code in a project I was working >> on. ?It does something like this: >> >> mystring = '\n'.join( [ line for line in lines if > depending on line> ] ) >> >> where "lines" is a simple list of strings. ?I realized that the code >> had been written before you could put a list comprehension as an >> argument to join(). ?I figured that I could drop the '[' and ']' and >> leave the rest as a list comprehension since that works with current >> Python (and the project's base is 2.5). ?So I rewrote the original >> statement like this: >> >> mystring = '\n'.join( line for line in lines if > depending on line> ) >> >> It works as expected. ?Then I got curious as to how it performs. ?I >> was surprised to learn that the rewritten expression runs more than >> twice as _slow_. ?e.g.: >> >>>>> l >> >> ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9'] >> >>>>> Timer("' '.join([x for x in l])", 'l = map(str,range(10))').timeit() >> >> 2.9967339038848877 >> >>>>> Timer("' '.join(x for x in l)", 'l = map(str,range(10))').timeit() >> >> 7.2045478820800781 >> >> Notice that I dropped the condition testing that was in my original >> code. ?I just wanted to see the effect of two different expressions. >> >> I thought that maybe there was some lower bound on the number of the >> items in the list or list comprehension beyond which the comprehension >> would prove more efficient. ?There doesn't appear to be one. ?I scaled >> the length of the input list up to 1 million items and got more or >> less the same relative performance. >> >> Now I'm really curious and I'd like to know: >> >> 1. Can anyone else confirm this observation? > >>>> Timer("' '.join([x for x in l])", 'l = map(str,range(10))').timeit() > 5.8625191190500345 >>>> Timer("' '.join(x for x in l)", 'l = map(str,range(10))').timeit() > 12.093135300715574 >>>> _ > > >> 2. Why should the "pure" list comprehension be slower than the same >> comprehension enclosed in '[...]' ? > > Regarding (2) the unparenthesized expression in join is *not* a list > comprehension but a generator expression. > > And as such it involves join calling next() on the generator object > repeatedly, with each next() call involving a light-weight context shift. > > In addition the docs mumble something about "lazy" evaluation, and that may > also contribute to the overhead. > > I think that in contrast, the interpreter can evaluate a list comprehension, > [x for x in blah], directly without any context shifting, just by > transforming it to equivalent code and putting the target expressions > innermost there. > > And so the main factor causing a slowdown for a list comprehension would, I > think, be paging and such if the list it produced was Really Huge, while for > the generator there's no memory issue but rather much calling & context > shifting. > > > Cheers & hth., > > - Alf > -- > http://mail.python.org/mailman/listinfo/python-list > -- Gerald Britton From doxalogos at gmail.com Tue Jan 19 11:15:22 2010 From: doxalogos at gmail.com (DoxaLogos) Date: Tue, 19 Jan 2010 08:15:22 -0800 (PST) Subject: multiprocessing problems Message-ID: <74594510-9366-4227-8e1e-d221a9220271@k35g2000yqb.googlegroups.com> Hi, I decided to play around with the multiprocessing module, and I'm having some strange side effects that I can't explain. It makes me wonder if I'm just overlooking something obvious or not. Basically, I have a script parses through a lot of files doing search and replace on key strings inside the file. I decided the split the work up on multiple processes on each processor core (4 total). I've tried many various ways doing this form using pool to calling out separate processes, but the result has been the same: computer crashes from endless process spawn. Here's the guts of my latest incarnation. def ProcessBatch(files): p = [] for file in files: p.append(Process(target=ProcessFile,args=file)) for x in p: x.start() for x in p: x.join() p = [] return Now, the function calling ProcessBatch looks like this: def ReplaceIt(files): """ All this does is walks through all the files passed to it and verifies the file is a legitimate file to be processed (project file). @param files: files to be processed """ processFiles = [] for replacefile in files: if(CheckSkipFile(replacefile)): processFiles.append(replacefile) if(len(processFiles) == 4): ProcessBatch(processFiles) processFiles = [] #check for left over files once main loop is done and process them if(len(processFiles) > 0): ProcessBatch(processFiles) return Specs: Windows 7 64-bit Python v2.6.2 Intel i5 Thanks From apt.shansen at gmail.com Tue Jan 19 11:18:00 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Tue, 19 Jan 2010 08:18:00 -0800 Subject: Performance of lists vs. list comprehensions In-Reply-To: <5d1a32001001190730u4fa2f26cn62e79c2c73a08b@mail.gmail.com> References: <5d1a32001001190730u4fa2f26cn62e79c2c73a08b@mail.gmail.com> Message-ID: <7a9c25c21001190818g7bd0ca6n336e6b1d52af0b2c@mail.gmail.com> On Tue, Jan 19, 2010 at 7:30 AM, Gerald Britton wrote: [snip] > mystring = '\n'.join( line for line in lines if depending on line> ) > Note, this is not a list comprehension, but a generator comprehension. A list comprehension is used to build, in one sweep, a list and return it. A generator comprehension is used to build an generator that can be iterated over to produce a sequence of items. I think you're seeing not performance of the expression, but the function call overhead which generators include. A generator requires one to call its next method to get each item: a list comprehension is just syntactical sugar for a for loop. As to which is faster, I think it depends. Your test-case is using *really* small ranges-- just ten items. In this case, just doing a simple loop to build a list and then pass it through join is probably faster. If you're using a much larger list though, the characteristics of the problem may change, where the lazy evaluation of a generator expression may be more desirable. A list comprehension includes a waste of memory, too: you have to build up a complete list before you return it, and if you have a lot of lines? That can be a problem. As you can see, the performance characteristics between the two narrow considerably if you compare a larger sample: >>> Timer("' '.join([str(x) for x in l])", 'l = xrange(100)').timeit() 50.092024087905884 >>> Timer("' '.join(str(x) for x in l)", 'l = xrange(100)').timeit() 54.591049909591675 --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From pruebauno at latinmail.com Tue Jan 19 11:18:17 2010 From: pruebauno at latinmail.com (nn) Date: Tue, 19 Jan 2010 08:18:17 -0800 (PST) Subject: Py 3: Terminal script can't find relative path References: <58e9b1e4-049a-4a7f-bea9-8d7e7e9f191a@e27g2000yqd.googlegroups.com> <87d417578z.fsf@castleamber.com> <9541dcf0-8060-4851-bdbd-0aed6de13bf9@v25g2000yqk.googlegroups.com> Message-ID: <317ebb29-8c20-4fe5-82c4-b2fe05efef38@h9g2000yqa.googlegroups.com> On Jan 19, 8:03?am, Gnarlodious wrote: > On Jan 18, 4:21?pm, John Bokma wrote: > > > Gnarlodious writes: > > > I am running a script in a browser that finds the file in subfolder > > > Data: > > > > Content=Plist('Data/Content.plist') > > > > However, running the same script in Terminal errors: > > > > IOError: [Errno 2] No such file or directory: 'Data/Content.plist' > > > What does: > > > ls -l Data/Content.plist > > > in the terminal give? > > I can replace with absolute paths and it works as expected. Could this > be a Python 3 bug? Where as a CGI script it finds the relative path > but not in Terminal? > > -- Gnarlie Stop and think for a second what you are saying: It works with absolute paths, it works as CGI script with relative paths, it doesn't work in the terminal. What is different? Do you know for sure what folder you are starting at when using the relative path? Most likely the terminal starts in a different place than the CGI script. From awilliam at opengroupware.us Tue Jan 19 11:20:17 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Tue, 19 Jan 2010 11:20:17 -0500 Subject: Closing a Curses Window??? Message-ID: <1263918017.3165.12.camel@linux-m3mt> I'm uses the curses module of Python to create a TUI. I can create windows, and subwindows, but I can't find anything on deleting or disposing of a a subwindow. How do I get rid of a subwindow? From oswald.harry at gmail.com Tue Jan 19 11:22:15 2010 From: oswald.harry at gmail.com (harryos) Date: Tue, 19 Jan 2010 08:22:15 -0800 (PST) Subject: use of super Message-ID: <8fbe9650-abee-4f0a-93f2-040fc8697e82@u41g2000yqe.googlegroups.com> hi I was going thru the weblog appln in practical django book by bennet .I came across this class Entry(Model): def save(self): dosomething() super(Entry,self).save() I couldn't make out why Entry and self are passed as arguments to super ().Can someone please explain? thanks harry From gerald.britton at gmail.com Tue Jan 19 11:26:43 2010 From: gerald.britton at gmail.com (Gerald Britton) Date: Tue, 19 Jan 2010 11:26:43 -0500 Subject: Performance of lists vs. list comprehensions In-Reply-To: <7a9c25c21001190818g7bd0ca6n336e6b1d52af0b2c@mail.gmail.com> References: <5d1a32001001190730u4fa2f26cn62e79c2c73a08b@mail.gmail.com> <7a9c25c21001190818g7bd0ca6n336e6b1d52af0b2c@mail.gmail.com> Message-ID: <5d1a32001001190826o2d5478bfm2b29a6f0f1818ab9@mail.gmail.com> Interestingly, I scaled it up to a million list items with more or less the same results. It's helpful to see that your results are different. That leads me to suspect that mine are somehow related to my own environment. Still I think the key is the overhead in calling next() for each item in the generator expression. That in itself probably accounts for the differences since function calls are somewhat expensive IIRC. On Tue, Jan 19, 2010 at 11:18 AM, Stephen Hansen wrote: > On Tue, Jan 19, 2010 at 7:30 AM, Gerald Britton > wrote: > [snip] >> >> mystring = '\n'.join( line for line in lines if > depending on line> ) > > Note, this is not a list comprehension, but a generator comprehension. > A list comprehension is used to build, in one sweep, a list and return it. > A generator comprehension is used to build an generator that can be iterated > over to produce a sequence of items. > I think you're seeing not performance of the expression, but the function > call overhead which generators include. A generator requires one to call its > next method to get each item: a list comprehension is just syntactical sugar > for a for loop. > As to which is faster, I think it depends. Your test-case is using *really* > small ranges-- just ten items. In this case, just doing a simple loop to > build a list and then pass it through join is probably faster. If you're > using a much larger list though, the characteristics of the problem may > change, where the lazy evaluation of a generator expression may be more > desirable. > A list comprehension includes a waste of memory, too: you have to build up a > complete list before you return it, and if you have a lot of lines? That can > be a problem. > As you can see, the performance characteristics between the two narrow > considerably if you compare a larger sample: > ?>>> Timer("' '.join([str(x) for x in l])", 'l = xrange(100)').timeit() > 50.092024087905884 >>>> Timer("' '.join(str(x) for x in l)", 'l = xrange(100)').timeit() > 54.591049909591675 > --S -- Gerald Britton From awilliam at opengroupware.us Tue Jan 19 11:26:49 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Tue, 19 Jan 2010 11:26:49 -0500 Subject: multiprocessing problems In-Reply-To: <74594510-9366-4227-8e1e-d221a9220271@k35g2000yqb.googlegroups.com> References: <74594510-9366-4227-8e1e-d221a9220271@k35g2000yqb.googlegroups.com> Message-ID: <1263918409.3165.15.camel@linux-m3mt> > I decided to play around with the multiprocessing module, and I'm > having some strange side effects that I can't explain. It makes me > wonder if I'm just overlooking something obvious or not. Basically, I > have a script parses through a lot of files doing search and replace > on key strings inside the file. I decided the split the work up on > multiple processes on each processor core (4 total). I've tried many > various ways doing this form using pool to calling out separate > processes, but the result has been the same: computer crashes from > endless process spawn. Are you hitting a ulimit error? The number of processes you can create is probably limited. TIP: close os.stdin on your subprocesses. > Here's the guts of my latest incarnation. > def ProcessBatch(files): > p = [] > for file in files: > p.append(Process(target=ProcessFile,args=file)) > for x in p: > x.start() > for x in p: > x.join() > p = [] > return > Now, the function calling ProcessBatch looks like this: > def ReplaceIt(files): > processFiles = [] > for replacefile in files: > if(CheckSkipFile(replacefile)): > processFiles.append(replacefile) > if(len(processFiles) == 4): > ProcessBatch(processFiles) > processFiles = [] > #check for left over files once main loop is done and process them > if(len(processFiles) > 0): > ProcessBatch(processFiles) According to this you will create files is sets of four, but an unknown number of sets of four. From doxalogos at gmail.com Tue Jan 19 11:33:25 2010 From: doxalogos at gmail.com (DoxaLogos) Date: Tue, 19 Jan 2010 08:33:25 -0800 (PST) Subject: multiprocessing problems References: <74594510-9366-4227-8e1e-d221a9220271@k35g2000yqb.googlegroups.com> Message-ID: <63c5df56-7b86-4ee7-ba4f-b14984f94097@e16g2000yqc.googlegroups.com> On Jan 19, 10:26?am, Adam Tauno Williams wrote: > > I decided to play around with the multiprocessing module, and I'm > > having some strange side effects that I can't explain. ?It makes me > > wonder if I'm just overlooking something obvious or not. ?Basically, I > > have a script parses through a lot of files doing search and replace > > on key strings inside the file. ?I decided the split the work up on > > multiple processes on each processor core (4 total). ?I've tried many > > various ways doing this form using pool to calling out separate > > processes, but the result has been the same: computer crashes from > > endless process spawn. > > Are you hitting a ulimit error? ?The number of processes you can create > is probably limited. > > TIP: close os.stdin on your subprocesses. > > > > > Here's the guts of my latest incarnation. > > def ProcessBatch(files): > > ? ? p = [] > > ? ? for file in files: > > ? ? ? ? p.append(Process(target=ProcessFile,args=file)) > > ? ? for x in p: > > ? ? ? ? x.start() > > ? ? for x in p: > > ? ? ? ? x.join() > > ? ? p = [] > > ? ? return > > Now, the function calling ProcessBatch looks like this: > > def ReplaceIt(files): > > ? ? processFiles = [] > > ? ? for replacefile in files: > > ? ? ? ? if(CheckSkipFile(replacefile)): > > ? ? ? ? ? ? processFiles.append(replacefile) > > ? ? ? ? ? ? if(len(processFiles) == 4): > > ? ? ? ? ? ? ? ? ProcessBatch(processFiles) > > ? ? ? ? ? ? ? ? processFiles = [] > > ? ? #check for left over files once main loop is done and process them > > ? ? if(len(processFiles) > 0): > > ? ? ? ? ProcessBatch(processFiles) > > According to this you will create files is sets of four, but an unknown > number of sets of four. What would be the proper way to only do a set of 4, stop, then do another set of 4? I'm trying to only 4 files at time before doing another set of 4. From gregchagnon at gmail.com Tue Jan 19 11:33:38 2010 From: gregchagnon at gmail.com (SoxFan44) Date: Tue, 19 Jan 2010 08:33:38 -0800 (PST) Subject: Create list/dict from string Message-ID: <2956b67c-c4b7-4757-bda8-48e8f799d4df@e16g2000yqc.googlegroups.com> I was wondering if there was a way to create a list (which in this case would contain several dicts) based on a string passed in by the user. Security is not an issue. Basically I want to be able to have the user pass in using optparse: --actions=[{"action_name": "action_1", "val": "asdf", "val2": "asdf"}, {"action_name": "action_2", "val": "asdf", "val2": "asdf"}, {"action_name": "action_1", "val": "asdf", "val2": "asdf"}] And have this create a list/dict. I'm aware of pickle, but it won't work as far as I can tell. Thanks. From pruebauno at latinmail.com Tue Jan 19 11:44:12 2010 From: pruebauno at latinmail.com (nn) Date: Tue, 19 Jan 2010 08:44:12 -0800 (PST) Subject: Arrrrgh! Another module broken References: Message-ID: <381e6798-f263-4db4-9aa3-0006578721eb@r5g2000yqb.googlegroups.com> On Jan 18, 11:37?am, Grant Edwards wrote: > On 2010-01-18, Jive Dadson wrote: > > > I just found another module that broke when I went to 2.6. ?Gnuplot. > > Apparently one of its routines has a parameter named "with." ?That used > > to be okay, and now it's not. > > I remember seeing depreicated warnings about that _years_ ago, > and I would have sworn it's been fixed for at least a couple > years. > > -- > Grant FWIW, "with" deprecation warnings exist since September 19, 2006 when Python 2.5 was released. From rbrt.somerville at gmail.com Tue Jan 19 11:48:23 2010 From: rbrt.somerville at gmail.com (robert somerville) Date: Tue, 19 Jan 2010 08:48:23 -0800 Subject: how to sort two dimensional array Message-ID: <2fb4a5011001190848mf8af3fcib7753783b223dc97@mail.gmail.com> Hi; i am having trouble trying to sort the rows of a 2 dimensional array by the values in the first column .. does anybody know how or have an example of how to do this ??? while leaving the remain columns remain relative to the leading column from numpy import * a=array( [ [4, 4, 3], [4, 5, 2], [3, 1, 1] ] ) i would like to generate the output (or get the output ...) b = [ [3,1,1], [4,4,3], [4,5,2] ] thanks; bob -------------- next part -------------- An HTML attachment was scrubbed... URL: From rbrt.somerville at gmail.com Tue Jan 19 11:50:21 2010 From: rbrt.somerville at gmail.com (robert somerville) Date: Tue, 19 Jan 2010 08:50:21 -0800 Subject: how to sort two dimensional array ?? Message-ID: <2fb4a5011001190850u58296a82yabb73af8b1bd21ab@mail.gmail.com> Hi; i am having trouble trying to sort the rows of a 2 dimensional array by the values in the first column .. does anybody know how or have an example of how to do this ??? while leaving the remain columns remain relative to the leading column from numpy import * a=array( [ [4, 4, 3], [4, 5, 2], [3, 1, 1] ] ) i would like to generate the output (or get the output ...) b = [ [3,1,1], [4,4,3], [4,5,2] ] thanks; bob -------------- next part -------------- An HTML attachment was scrubbed... URL: From simon at brunningonline.net Tue Jan 19 11:52:56 2010 From: simon at brunningonline.net (Simon Brunning) Date: Tue, 19 Jan 2010 16:52:56 +0000 Subject: use of super In-Reply-To: <8fbe9650-abee-4f0a-93f2-040fc8697e82@u41g2000yqe.googlegroups.com> References: <8fbe9650-abee-4f0a-93f2-040fc8697e82@u41g2000yqe.googlegroups.com> Message-ID: <8c7f10c61001190852m58eb51e3ubeec14c9b9dcdd7b@mail.gmail.com> 2010/1/19 harryos : > I was going thru the weblog appln in practical django book by > bennet .I came across this > > class Entry(Model): > ? ? ? ?def save(self): > ? ? ? ? ? ? ? ?dosomething() > ? ? ? ? ? ? ? ?super(Entry,self).save() > > I couldn't make out why Entry and self are passed as arguments to super > ().Can someone please explain? Does make anything clearer? -- Cheers, Simon B. From __peter__ at web.de Tue Jan 19 11:57:46 2010 From: __peter__ at web.de (Peter Otten) Date: Tue, 19 Jan 2010 17:57:46 +0100 Subject: Create list/dict from string References: <2956b67c-c4b7-4757-bda8-48e8f799d4df@e16g2000yqc.googlegroups.com> Message-ID: SoxFan44 wrote: > I was wondering if there was a way to create a list (which in this > case would contain several dicts) based on a string passed in by the > user. Security is not an issue. Basically I want to be able to have > the user pass in using optparse: > --actions=[{"action_name": "action_1", "val": "asdf", "val2": > "asdf"}, > {"action_name": "action_2", "val": "asdf", "val2": > "asdf"}, > {"action_name": "action_1", "val": "asdf", "val2": > "asdf"}] > > And have this create a list/dict. I'm aware of pickle, but it won't > work as far as I can tell. Both eval() and json.loads() will do. eval() is dangerous as it allows the user to run arbitrary python code. Peter From phlip2005 at gmail.com Tue Jan 19 11:58:21 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 19 Jan 2010 08:58:21 -0800 (PST) Subject: Is HTML report of tests run using PyUnit (unittest) possible? References: <584f31d6-4c4d-4661-836f-c8552d892d38@j19g2000yqk.googlegroups.com> Message-ID: <8d5070ee-1ec2-4f55-b56e-d841fb023cb9@j19g2000yqk.googlegroups.com> On Jan 19, 3:16?am, fossist wrote: > I am using PyUnit (unittest module) for loading test cases from our > modules and executing them. Is it possible to export the test results > as HTML report? Currently the results appear as text on standard > output while the tests execute. But is there something out of the box > available in PyUnit to make this possible? django-test-extensions can do this, but I'm unaware how well it works without Django. (And I _could_ complain that it comes with one or two irritations _with_ Django;) I would download it and read its source to see how the --xml option works. (Then you'd use a XSL filter to rip the XML into HTML...) -- Phlip http://c2.com/cgi/wiki?ZeekLand From rsomerville at sjgeophysics.com Tue Jan 19 12:00:26 2010 From: rsomerville at sjgeophysics.com (Robert Somerville) Date: Tue, 19 Jan 2010 09:00:26 -0800 Subject: how to sort two dimensional array ?? Message-ID: <4B55E52A.9000900@sjgeophysics.com> Hi; i am having trouble trying to sort the rows of a 2 dimensional array by the values in the first column .. does anybody know how or have an example of how to do this ??? while leaving the remain columns remain relative to the leading column from numpy import * a=array( [ [4, 4, 3], [4, 5, 2], [3, 1, 1] ] ) i would like to generate the output (or get the output ...) b = [ [3,1,1], [4,4,3], [4,5,2] ] thanks; bob From briandenzer at gmail.com Tue Jan 19 12:14:29 2010 From: briandenzer at gmail.com (Brian D) Date: Tue, 19 Jan 2010 09:14:29 -0800 (PST) Subject: Iterate over group names in a regex match? Message-ID: Here's a simple named group matching pattern: >>> s = "1,2,3" >>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") >>> m = re.match(p, s) >>> m <_sre.SRE_Match object at 0x011BE610> >>> print m.groups() ('1', '2', '3') Is it possible to call the group names, so that I can iterate over them? The result I'm looking for would be: ('one', 'two', 'three') From phlip2005 at gmail.com Tue Jan 19 12:19:55 2010 From: phlip2005 at gmail.com (Phlip) Date: Tue, 19 Jan 2010 09:19:55 -0800 (PST) Subject: Python IDE for MacOS-X References: Message-ID: <6b836c4b-0297-44d1-bcad-f3423925a19c@34g2000yqp.googlegroups.com> On Jan 18, 11:09?pm, Jean Guillaume Pyraksos wrote: > What's the best one to use with beginners ? > Something with integrated syntax editor, browser of doc... > Thanks, Before this message goes stale, there's TextMate (which I have too much experience with to consider redeemable in any way)... ...and there's Komodo Edit. The problems I have with that are... - the code browsing breaks when the wind blows, and you must Find in Files (my library, Django, is symlinked below my project, so FiF can see it) - the editor refuses to let me run a script - such as a test script - each time I hit F5. For whatever reason - poor keyboard remapping, or lack of a plug-in for Django - I must use Ctrl+R to run a batch file, just to test. Testing should be ONE (1) keystroke. - FiF sees my .git folder, and everything in it. WTF?? - after searching or anything, I must click the editor with the mouse to resume editing. should move the focus out of the frame to the editor, and of course any activity that takes you out of the editor should dump you back into it by default Other than that, it has all the standard edito My problem with Mac in general is the keystrokes are always so broken they force me to switch to a mouse. Nobody seems to usability test these things and see how long you can type & manipulate windows without From slais-www at ucl.ac.uk Tue Jan 19 12:23:32 2010 From: slais-www at ucl.ac.uk (djc) Date: Tue, 19 Jan 2010 17:23:32 +0000 Subject: Iterate over group names in a regex match? In-Reply-To: References: Message-ID: Brian D wrote: > Here's a simple named group matching pattern: > >>>> s = "1,2,3" >>>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") >>>> m = re.match(p, s) >>>> m > <_sre.SRE_Match object at 0x011BE610> >>>> print m.groups() > ('1', '2', '3') > > Is it possible to call the group names, so that I can iterate over > them? > > The result I'm looking for would be: > > ('one', 'two', 'three') print(m.groupdict()) {'one': '1', 'three': '3', 'two': '2'} >>> print(m.groupdict().keys()) ['one', 'three', 'two'] -- David Clark, MSc, PhD. Dept of Information Studies Systems & Web Development Manager University College London UCL Centre for Publishing Gower Str London WCIE 6BT From oswald.harry at gmail.com Tue Jan 19 12:26:38 2010 From: oswald.harry at gmail.com (harryos) Date: Tue, 19 Jan 2010 09:26:38 -0800 (PST) Subject: use of super References: <8fbe9650-abee-4f0a-93f2-040fc8697e82@u41g2000yqe.googlegroups.com> Message-ID: <584b87a7-bc72-4a1c-8846-79c37fd1baf3@m25g2000yqc.googlegroups.com> thanks Simon..I should have checked it From gerald.britton at gmail.com Tue Jan 19 12:27:13 2010 From: gerald.britton at gmail.com (Gerald Britton) Date: Tue, 19 Jan 2010 12:27:13 -0500 Subject: Create list/dict from string In-Reply-To: <2956b67c-c4b7-4757-bda8-48e8f799d4df@e16g2000yqc.googlegroups.com> References: <2956b67c-c4b7-4757-bda8-48e8f799d4df@e16g2000yqc.googlegroups.com> Message-ID: <5d1a32001001190927n1fd0916cs78bb31d54010bf34@mail.gmail.com> Can't you just use the dict() built-in function like this: dict({"action_name": "action_1", "val": "asdf"}) Of course if the list is not properly formed, this will fail. But I guess you have thought of that already. On Tue, Jan 19, 2010 at 11:33 AM, SoxFan44 wrote: > I was wondering if there was a way to create a list (which in this > case would contain several dicts) based on a string passed in by the > user. ?Security is not an issue. ?Basically I want to be able to have > the user pass in using optparse: > --actions=[{"action_name": "action_1", "val": "asdf", "val2": > "asdf"}, > ? ? ? ? ? ? ? {"action_name": "action_2", "val": "asdf", "val2": > "asdf"}, > ? ? ? ? ? ? ? {"action_name": "action_1", "val": "asdf", "val2": > "asdf"}] > > And have this create a list/dict. ?I'm aware of pickle, but it won't > work as far as I can tell. > > Thanks. > -- > http://mail.python.org/mailman/listinfo/python-list > -- Gerald Britton From simon at brunningonline.net Tue Jan 19 12:27:33 2010 From: simon at brunningonline.net (Simon Brunning) Date: Tue, 19 Jan 2010 17:27:33 +0000 Subject: Create list/dict from string In-Reply-To: References: <2956b67c-c4b7-4757-bda8-48e8f799d4df@e16g2000yqc.googlegroups.com> Message-ID: <8c7f10c61001190927r2b973817t41e134c7b8a2ca09@mail.gmail.com> 2010/1/19 Peter Otten <__peter__ at web.de>: > Both eval() and json.loads() will do. eval() is dangerous as it allows the > user to run arbitrary python code. Something like might be worth a look too. -- Cheers, Simon B. From __peter__ at web.de Tue Jan 19 12:28:11 2010 From: __peter__ at web.de (Peter Otten) Date: Tue, 19 Jan 2010 18:28:11 +0100 Subject: Iterate over group names in a regex match? References: Message-ID: Brian D wrote: > Here's a simple named group matching pattern: > >>>> s = "1,2,3" >>>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") >>>> m = re.match(p, s) >>>> m > <_sre.SRE_Match object at 0x011BE610> >>>> print m.groups() > ('1', '2', '3') > > Is it possible to call the group names, so that I can iterate over > them? > > The result I'm looking for would be: > > ('one', 'two', 'three') >>> s = "1,2,3" >>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") >>> m = re.match(p, s) >>> dir(m) ['__copy__', '__deepcopy__', 'end', 'expand', 'group', 'groupdict', 'groups', 'span', 'start'] >>> m.groupdict().keys() ['one', 'three', 'two'] >>> sorted(m.groupdict(), key=m.span) ['one', 'two', 'three'] Peter From alfps at start.no Tue Jan 19 12:28:20 2010 From: alfps at start.no (Alf P. Steinbach) Date: Tue, 19 Jan 2010 18:28:20 +0100 Subject: Iterate over group names in a regex match? In-Reply-To: References: Message-ID: * Brian D: > Here's a simple named group matching pattern: > >>>> s = "1,2,3" >>>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") >>>> m = re.match(p, s) >>>> m > <_sre.SRE_Match object at 0x011BE610> >>>> print m.groups() > ('1', '2', '3') > > Is it possible to call the group names, so that I can iterate over > them? > > The result I'm looking for would be: > > ('one', 'two', 'three') I never used that beast (I'm in a sense pretty new to Python, although starting some months back I only investigate what's needed for my writings), but checking things in the interpreter: >>> import re >>> re >>> s = "1,2,3" >>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") >>> m = re.match(p, s) >>> m <_sre.SRE_Match object at 0x01319F70> >>> m.groups() ('1', '2', '3') >>> type( m.groups() ) >>> dir( m ) ['__copy__', '__deepcopy__', 'end', 'expand', 'group', 'groupdict', 'groups', 'span', 'start'] >>> m.groupdict >>> m.groupdict() {'one': '1', 'three': '3', 'two': '2'} >>> print( tuple( m.groupdict().keys() ) ) ('one', 'three', 'two') >>> _ Cheers & hth., - Alf From pych3m4 at gmail.com Tue Jan 19 12:36:27 2010 From: pych3m4 at gmail.com (Chema Cortes) Date: Tue, 19 Jan 2010 18:36:27 +0100 Subject: [ANN] Python-es mailing list changes home Message-ID: <2c9fb0dd1001190936q3116f0a3k221f55485daf685f@mail.gmail.com> === Python-es mailing list changes home === Due to technical problems with the site that usually ran the Python-es mailing list (Python list for the Spanish speaking community), we are setting up a new one under the python.org umbrella. Hence, the new list will become (the old one was ). Please feel free to subscribe to the new list in: http://mail.python.org/mailman/listinfo/python-es Thanks! === La lista de distribuci?n Python-es cambia de lugar === Debido a problemas t?cnicos con el sitio que normalmente albergaba la lista de Python-es (Lista de Python para la comunidad hispano-hablante), estamos configurando una nueva en el sitio python.org. As? que la nueva lista ser? (en sustituci?n de la antigua ). Por favor, si lo deseas, date de alta en la nueva lista en: http://mail.python.org/mailman/listinfo/python-es ?Gracias! Chema Cortes, Oswaldo Hern?ndez y Francesc Alted From robert.kern at gmail.com Tue Jan 19 12:39:09 2010 From: robert.kern at gmail.com (Robert Kern) Date: Tue, 19 Jan 2010 11:39:09 -0600 Subject: how to sort two dimensional array ?? In-Reply-To: <4B55E52A.9000900@sjgeophysics.com> References: <4B55E52A.9000900@sjgeophysics.com> Message-ID: On 2010-01-19 11:00 AM, Robert Somerville wrote: > Hi; > i am having trouble trying to sort the rows of a 2 dimensional array by > the values in the first column .. does anybody know how or have an > example of how to do this ??? while leaving the remain columns remain > relative to the leading column You will want to ask numpy questions on the numpy mailing list: http://www.scipy.org/Mailing_Lists > from numpy import * > > a=array( [ [4, 4, 3], [4, 5, 2], [3, 1, 1] ] ) > > i would like to generate the output (or get the output ...) > > b = [ [3,1,1], [4,4,3], [4,5,2] ] In [4]: import numpy as np In [5]: a = np.array( [ [4, 4, 3], [4, 5, 2], [3, 1, 1] ] ) In [6]: i = a[:,0].argsort() In [7]: b = a[i] In [8]: b Out[8]: array([[3, 1, 1], [4, 4, 3], [4, 5, 2]]) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From apt.shansen at gmail.com Tue Jan 19 12:39:53 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Tue, 19 Jan 2010 09:39:53 -0800 Subject: how to sort two dimensional array ?? In-Reply-To: <4B55E52A.9000900@sjgeophysics.com> References: <4B55E52A.9000900@sjgeophysics.com> Message-ID: <7a9c25c21001190939x187d685egae92ee1367272c76@mail.gmail.com> On Tue, Jan 19, 2010 at 9:00 AM, Robert Somerville < rsomerville at sjgeophysics.com> wrote: > Hi; > Hi, why did you post this three times? > i am having trouble trying to sort the rows of a 2 dimensional array by the > values in the first column .. does anybody know how or have an example of > how to do this ??? while leaving the remain columns remain relative to the > leading column > > from numpy import * > > a=array( [ [4, 4, 3], [4, 5, 2], [3, 1, 1] ] ) > > i would like to generate the output (or get the output ...) > > b = [ [3,1,1], [4,4,3], [4,5,2] ] > I don't use numpy, so this may or may not work. But for a regular python list-of-lists (2 dimensional array), you simply do: a.sort(key=lambda x: x[0]) --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From briandenzer at gmail.com Tue Jan 19 12:51:54 2010 From: briandenzer at gmail.com (Brian D) Date: Tue, 19 Jan 2010 09:51:54 -0800 (PST) Subject: Iterate over group names in a regex match? References: Message-ID: On Jan 19, 11:28?am, Peter Otten <__pete... at web.de> wrote: > Brian D wrote: > > Here's a simple named group matching pattern: > > >>>> s = "1,2,3" > >>>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") > >>>> m = re.match(p, s) > >>>> m > > <_sre.SRE_Match object at 0x011BE610> > >>>> print m.groups() > > ('1', '2', '3') > > > Is it possible to call the group names, so that I can iterate over > > them? > > > The result I'm looking for would be: > > > ('one', 'two', 'three') > >>> s = "1,2,3" > >>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") > >>> m = re.match(p, s) > >>> dir(m) > > ['__copy__', '__deepcopy__', 'end', 'expand', 'group', 'groupdict', > 'groups', 'span', 'start']>>> m.groupdict().keys() > > ['one', 'three', 'two']>>> sorted(m.groupdict(), key=m.span) > > ['one', 'two', 'three'] > > Peter groupdict() does it. I've never seen it used before. Very cool! Thank you all for taking time to answer the question. From tim.arnold at sas.com Tue Jan 19 12:53:19 2010 From: tim.arnold at sas.com (Tim Arnold) Date: Tue, 19 Jan 2010 12:53:19 -0500 Subject: Python IDE for MacOS-X References: Message-ID: "Jean Guillaume Pyraksos" wrote in message news:wissme-9248E1.08090319012010 at news.free.fr... > What's the best one to use with beginners ? > Something with integrated syntax editor, browser of doc... > Thanks, > > JG eclipse + pydev works well for me. --Tim Arnold From briandenzer at gmail.com Tue Jan 19 13:01:44 2010 From: briandenzer at gmail.com (Brian D) Date: Tue, 19 Jan 2010 10:01:44 -0800 (PST) Subject: Iterate over group names in a regex match? References: Message-ID: <5067f1f2-863f-4187-8b81-bdebe7a92df3@p24g2000yqm.googlegroups.com> On Jan 19, 11:51?am, Brian D wrote: > On Jan 19, 11:28?am, Peter Otten <__pete... at web.de> wrote: > > > > > Brian D wrote: > > > Here's a simple named group matching pattern: > > > >>>> s = "1,2,3" > > >>>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") > > >>>> m = re.match(p, s) > > >>>> m > > > <_sre.SRE_Match object at 0x011BE610> > > >>>> print m.groups() > > > ('1', '2', '3') > > > > Is it possible to call the group names, so that I can iterate over > > > them? > > > > The result I'm looking for would be: > > > > ('one', 'two', 'three') > > >>> s = "1,2,3" > > >>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") > > >>> m = re.match(p, s) > > >>> dir(m) > > > ['__copy__', '__deepcopy__', 'end', 'expand', 'group', 'groupdict', > > 'groups', 'span', 'start']>>> m.groupdict().keys() > > > ['one', 'three', 'two']>>> sorted(m.groupdict(), key=m.span) > > > ['one', 'two', 'three'] > > > Peter > > groupdict() does it. I've never seen it used before. Very cool! > > Thank you all for taking time to answer the question. FYI, here's an example of the working result ... >>> for k, v in m.groupdict().iteritems(): k, v ('one', '1') ('three', '3') ('two', '2') The use for this is that I'm pulling data from a flat text file using regex, and storing values in a dictionary that will be used to update a database. From python at mrabarnett.plus.com Tue Jan 19 13:20:42 2010 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 19 Jan 2010 18:20:42 +0000 Subject: Iterate over group names in a regex match? In-Reply-To: References: Message-ID: <4B55F7FA.4080003@mrabarnett.plus.com> Brian D wrote: > Here's a simple named group matching pattern: > >>>> s = "1,2,3" >>>> p = re.compile(r"(?P\d),(?P\d),(?P\d)") >>>> m = re.match(p, s) >>>> m > <_sre.SRE_Match object at 0x011BE610> >>>> print m.groups() > ('1', '2', '3') > > Is it possible to call the group names, so that I can iterate over > them? > > The result I'm looking for would be: > > ('one', 'two', 'three') > The closest you can get is with groupdict(): >>> print m.groupdict() {'one': '1', 'three': '3', 'two': '2'} From lordkrandel at gmail.com Tue Jan 19 13:42:02 2010 From: lordkrandel at gmail.com (Wyrmskull) Date: Tue, 19 Jan 2010 10:42:02 -0800 (PST) Subject: substitution In-Reply-To: References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> Message-ID: <4b55fcfa.0d0db80a.1657.fffff15d@mx.google.com> Peter Otten wrote: def replace_many(s, pairs): if len(pairs): a, b = pairs[0] rest = pairs[1:] return b.join(replace_many(t, rest) for t in s.split(a)) else: return s ------------- Proves wrong, this way x -> y -> z. You call replace_many again on the central part of the split Specifics indicate that x -> y in the end. Your flowing pythonicity (if len(x):) gave me lots of inspiration. I bet this will win the prize ;) ------------- def mySubst(reps,string): if not(len(reps)): return string a,b,c = string.partition(reps[0][0]) if b: return mySubst(reps,a) + reps[0][1] + mySubst (reps,c) else: return mySubst(reps[1:],string) print mySubst( ( ('foo','bar'), ('bar','qux'), ('qux','foo') ), 'foobarquxfoo') ------- Wyrmskull From __peter__ at web.de Tue Jan 19 13:49:40 2010 From: __peter__ at web.de (Peter Otten) Date: Tue, 19 Jan 2010 19:49:40 +0100 Subject: substitution References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> Message-ID: Wyrmskull wrote: > Peter Otten wrote: >> def replace_many(s, pairs): >> if len(pairs): >> a, b = pairs[0] >> rest = pairs[1:] >> return b.join(replace_many(t, rest) for t in s.split(a)) >> else: >> return s > Proves wrong, this way x -> y -> z. > You call replace_many again on the central part of the split > Specifics indicate that x -> y in the end. Sorry, I don't understand what you want to say with the above. > Try with this: > > def mySubst(reps,string): > if not(len(reps)): > return string > current = reps[0][0] > a,b,c = string.partition(current) > if b: > return mySubst(reps,a) + reps[0][1] + mySubst (reps,c) > else: > return mySubst(reps[1:],string) > > print mySubst( ( ('foo','bar'), ('bar','qux'), ('qux','foo') ), > 'foobarquxfoo') > > ------- > Wyrmskull I don't see at first glance where the results of replace_many() and mySubst() differ. Perhaps you could give an example? Peter PS: Please keep the conversation on-list From lordkrandel at gmail.com Tue Jan 19 13:56:41 2010 From: lordkrandel at gmail.com (Wyrmskull) Date: Tue, 19 Jan 2010 10:56:41 -0800 (PST) Subject: substitution In-Reply-To: <4b55fcfa.0d0db80a.1657.fffff15d@mx.google.com> References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> <4b55fcfa.0d0db80a.1657.fffff15d@mx.google.com> Message-ID: <4b560069.100db80a.249f.ffffc009@mx.google.com> Cleaned. You can remove the 'else's if you want, because 'return' breaks the instruction flow. Should also work with other sequence types. ---------------- def mySubst(reps,seq): if reps: a,b,c = string.partition(reps[0][0]) if b: return mySubst(reps,a) + reps[0][1] + mySubst (reps,c) else: return mySubst(reps[1:], seq) else: return seq print mySubst( ( ('foo','bar'), ('bar','qux'), ('qux','foo') ), 'foobarquxfoo') print mySubst( ( ('foo','bar'), ('bar','qux'), ('qux','foo') ), 'foobarquxxxfoo') ------- Wyrmskull From __peter__ at web.de Tue Jan 19 13:58:24 2010 From: __peter__ at web.de (Peter Otten) Date: Tue, 19 Jan 2010 19:58:24 +0100 Subject: substitution References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> Message-ID: Wyrmskull wrote: > Your flowing pythonicity (if len(x):) gave me lots of inspiration. Actually, instead of if len(pairs): ... that should have been just if pairs: ... When I started writing the function I initially wanted to special-case len(pairs) == 1. The len() call is a superfluous leftover. Peter From lordkrandel at gmail.com Tue Jan 19 14:06:38 2010 From: lordkrandel at gmail.com (Wyrmskull) Date: Tue, 19 Jan 2010 11:06:38 -0800 (PST) Subject: substitution In-Reply-To: References: <4b5434c9$0$1114$4fafbaef@reader4.news.tin.it> <4b543642$0$1109$4fafbaef@reader4.news.tin.it> Message-ID: <4b5602be.16125e0a.1a2d.0c2d@mx.google.com> Nvm, my bad, I misunderstood the split instruction. No difference :) ------- Wyrmskull P.S Sorry about the P.M., I misclicked on a GUI From gd_usenet at spamfence.net Tue Jan 19 14:10:56 2010 From: gd_usenet at spamfence.net (=?UTF-8?Q?G=C3=BCnther?= Dietrich) Date: Tue, 19 Jan 2010 20:10:56 +0100 Subject: Python IDE for MacOS-X References: Message-ID: <0nch27-m2t.ln1@spamfence.net> Jean Guillaume Pyraksos wrote: >What's the best one to use with beginners ? >Something with integrated syntax editor, browser of doc... >Thanks, I started with 'EasyEclipse for Python', but soon changed to Eclipse with PyDev, MercurialEclipse and AnyEditTools plugins installed manually. I can recommend the combination Eclipse/PyDev. Best regards, G?nther From sccolbert at gmail.com Tue Jan 19 14:27:28 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Tue, 19 Jan 2010 14:27:28 -0500 Subject: Python IDE for MacOS-X In-Reply-To: References: Message-ID: <7f014ea61001191127h6af98916l1b4ed7f6d9bb703c@mail.gmail.com> On Tue, Jan 19, 2010 at 2:09 AM, Jean Guillaume Pyraksos wrote: > What's the best one to use with beginners ? > Something with integrated syntax editor, browser of doc... > Thanks, > > JG > -- > http://mail.python.org/mailman/listinfo/python-list > I whole-heartedly recommend WingIDE. It's commercial and the only piece of commercial Linux software I use, but it is worth every penny. And support emails are answered within hours, if not minutes...they are great guys over there. **I am not affiliated with Wingware in any way. Just a happy customer.** -------------- next part -------------- An HTML attachment was scrubbed... URL: From gnarlodious at gmail.com Tue Jan 19 14:29:00 2010 From: gnarlodious at gmail.com (Gnarlodious) Date: Tue, 19 Jan 2010 11:29:00 -0800 (PST) Subject: Py 3: Terminal script can't find relative path References: <58e9b1e4-049a-4a7f-bea9-8d7e7e9f191a@e27g2000yqd.googlegroups.com> <87d417578z.fsf@castleamber.com> <9541dcf0-8060-4851-bdbd-0aed6de13bf9@v25g2000yqk.googlegroups.com> <317ebb29-8c20-4fe5-82c4-b2fe05efef38@h9g2000yqa.googlegroups.com> Message-ID: <98c70d6f-5872-436b-b160-01b39d17d270@a32g2000yqm.googlegroups.com> OK I guess that is normal, I fixed it with this: path=os.path.dirname(__file__)+"/Data/" -- Gnarlie From wolfram.hinderer at googlemail.com Tue Jan 19 14:29:15 2010 From: wolfram.hinderer at googlemail.com (Wolfram Hinderer) Date: Tue, 19 Jan 2010 11:29:15 -0800 (PST) Subject: Performance of lists vs. list comprehensions References: Message-ID: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> On 19 Jan., 16:30, Gerald Britton wrote: > >>> Timer("' '.join([x for x in l])", 'l = map(str,range(10))').timeit() > > 2.9967339038848877 > > >>> Timer("' '.join(x for x in l)", 'l = map(str,range(10))').timeit() > > 7.2045478820800781 [...] > 2. Why should the "pure" list comprehension be slower than the same > comprehension enclosed in '[...]' ? Others have already commented on "generator expression" vs. "list comprehension". I'll try to shed some light on the cause of the slowness. For me it's >>> Timer("' '.join([x for x in l])", 'l = map(str,range(10))').timeit() 0.813948839866498 >>> Timer("' '.join(x for x in l)", 'l = map(str,range(10))').timeit() 2.226825476422391 But wait! I'm on Python 3.1 and the setup statement has to be changed to make this test meaningful. >>> Timer("' '.join([x for x in l])", 'l = list(map(str,range(10)))').timeit() 2.5788493369966545 >>> Timer("' '.join(x for x in l)", 'l = list(map(str,range(10)))').timeit() 3.7431774848480472 Much smaller factor now. But wait! If we want to test list comprehension against generator comprehension, we should try a function that just consumes the iterable. >>> setup = """l = list(map(str,range(10))) ... def f(a): ... for i in a: pass ... """ >>> Timer("f([x for x in l])", setup).timeit() 3.288511528699928 >>> Timer("f(x for x in l)", setup).timeit() 2.410873798206012 Oops! Iteration over generator expressions is not inherently more expension than iteration over list comprehensions. But certainly building a list from a generator expression is more expensive than a list comprehension? >>> Timer("[x for x in l]", 'l = list(map(str,range(10)))').timeit() 2.088602950933364 >>> Timer("list(x for x in l)", 'l = list(map(str,range(10)))').timeit() 3.691566805277944 Yes, list building from a generator expression *is* expensive. And join has to do it, because it has to iterate twice over the iterable passed in: once for calculating the memory needed for the joined string, and once more to actually do the join (this is implementation dependent, of course). If the iterable is a list already, the list building is not needed. From apt.shansen at gmail.com Tue Jan 19 14:56:13 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Tue, 19 Jan 2010 11:56:13 -0800 Subject: Py 3: How to switch application to Unicode strings? In-Reply-To: <92288825-8b01-47fb-8592-f65a7d7aeb8a@k35g2000yqb.googlegroups.com> References: <92288825-8b01-47fb-8592-f65a7d7aeb8a@k35g2000yqb.googlegroups.com> Message-ID: <7a9c25c21001191156j46a7fdadt58b728477b85e651@mail.gmail.com> On Tue, Jan 19, 2010 at 7:50 AM, Gnarlodious wrote: > I am using Python 3, getting an error from SQLite: > > sqlite3.ProgrammingError: You must not use 8-bit bytestrings unless > you use a text_factory that can interpret 8-bit bytestrings (like > text_factory = str). It is highly recommended that you instead just > switch your application to Unicode strings. > > So... how do I switch to Unicode? I thought I was doing it when I put # coding:utf-8 > > at the start of my script. > All that does is mean that the script itself is encoded as utf8. In Py3, anything you do "this" or use str(), you are using unicode strings. The problem appears to be that you are passing bytestrings to sqlite; things created with b"this" or bytes(), or read from a file as bytes and not decoded before passing it to the database. To really help further, you should provide the line that threw that warning and show where any variables in it come from. As for that error message, I believe it means text_factory = bytes. --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From gerald.britton at gmail.com Tue Jan 19 15:06:13 2010 From: gerald.britton at gmail.com (Gerald Britton) Date: Tue, 19 Jan 2010 15:06:13 -0500 Subject: Performance of lists vs. list comprehensions In-Reply-To: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> Message-ID: <5d1a32001001191206x3e2cc28ey6bc5c983fb739d67@mail.gmail.com> [snip] > > Yes, list building from a generator expression *is* expensive. And > join has to do it, because it has to iterate twice over the iterable > passed in: once for calculating the memory needed for the joined > string, and once more to actually do the join (this is implementation > dependent, of course). If the iterable is a list already, the list > building is not needed. if join has to iterate twice over the iterable, how does this work? $ python3.1 Python 3.1.1+ (r311:74480, Nov 2 2009, 14:49:22) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> l = map(str, (x for x in range(10) if int(x)%2)) >>> '.'.join(l) '1.3.5.7.9' >>> If join had to iterate twice over l, it would be consumed on the first pass. If it works as you say then join would have to copy the iterable on the first pass, effectively turning it into a list. Though I didn't read through it, I would suppose that join could use a dynamic-table approach to hold the result, starting with some guesstimate then expanding the result buffer if and when needed. > -- > http://mail.python.org/mailman/listinfo/python-list > -- Gerald Britton From steve at holdenweb.com Tue Jan 19 15:52:15 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 19 Jan 2010 15:52:15 -0500 Subject: enhancing 'list' In-Reply-To: <375ab6da-58c2-4975-bc79-22b2219a3216@r24g2000yqd.googlegroups.com> References: <84109e69-62ce-431f-b00e-f6bc0e7ac56c@q4g2000yqm.googlegroups.com> <375ab6da-58c2-4975-bc79-22b2219a3216@r24g2000yqd.googlegroups.com> Message-ID: samwyse wrote: > On Jan 18, 1:56 am, Terry Reedy wrote: >> On 1/17/2010 5:37 PM, samwyse wrote: >> >> >> >> >> >>> Consider this a wish list. I know I'm unlikely to get any of these in >>> time for for my birthday, but still I felt the need to toss it out and >>> see what happens. >>> Lately, I've slinging around a lot of lists, and there are some simple >>> things I'd like to do that just aren't there. >>> s.count(x[, cmp[, key]]) >>> - return number of i?s for which s[i] == x. 'cmp' specifies a custom >>> comparison function of two arguments, as in '.sort'. 'key' specifies >>> a custom key extraction function of one argument. >>> s.index(x[, i[, j[, cmp[, key]]]]) >>> - return smallest k such that s[k] == x and i<= k< j. 'cmp' and >>> 'key' are as above. >>> s.rindex(x[, i[, j[, cmp[, key]]]]) >>> - return largest k such that s[k] == x and i<= k< j. 'cmp' and >>> 'key' are as above. >>> There are two overlapping proposals here. One is to add the .rindex >>> method, which strings already have. The other is to extend the >>> optional arguments of .sort to all other methods that test for item >>> equality. >>> One last thing, the Python 2.6.2 spec says .count and .index only >>> apply to mutable sequence types. I see no reason why they >>> (and .rindex) couldn't also apply to immutable sequences (tuples, in >>> particular). >> In 3.x, tuple does have those methods, even though the doc is not clear >> (unless fixed by now). > > That's good to hear. Perhaps I should have tried them directyly, but > my 3.1 docs still echo the 2.x docs, which only show them for > immutable sequences. The tuple IS an immutable sequence. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From arnodel at googlemail.com Tue Jan 19 16:01:31 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Tue, 19 Jan 2010 21:01:31 +0000 Subject: Performance of lists vs. list comprehensions References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> Message-ID: Gerald Britton writes: > [snip] > >> >> Yes, list building from a generator expression *is* expensive. And >> join has to do it, because it has to iterate twice over the iterable >> passed in: once for calculating the memory needed for the joined >> string, and once more to actually do the join (this is implementation >> dependent, of course). If the iterable is a list already, the list >> building is not needed. > > if join has to iterate twice over the iterable, how does this work? > > $ python3.1 > Python 3.1.1+ (r311:74480, Nov 2 2009, 14:49:22) > [GCC 4.4.1] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> l = map(str, (x for x in range(10) if int(x)%2)) >>>> '.'.join(l) > '1.3.5.7.9' >>>> > > If join had to iterate twice over l, it would be consumed on the first > pass. If it works as you say then join would have to copy the > iterable on the first pass, effectively turning it into a list. > Though I didn't read through it, I would suppose that join could use a > dynamic-table approach to hold the result, starting with some > guesstimate then expanding the result buffer if and when needed. Looking at the source (py3k): PyObject * PyUnicode_Join(PyObject *separator, PyObject *seq) { [skip declarations] fseq = PySequence_Fast(seq, ""); if (fseq == NULL) { return NULL; } [code that works out the length of the joined string then allocates memory, then fills it] } Where PySequence_Fast(seq, "") returns seq if seq is already a tuple or a list and otherwise returns a new tuple built from the elements of seq. So no, it doesn't guess the size of the joined string and yes, it iterates twice over the "sequence" (I would have thought it should be called an iterable) by copying it into a tuple. -- Arnaud From wolfram.hinderer at googlemail.com Tue Jan 19 16:15:14 2010 From: wolfram.hinderer at googlemail.com (Wolfram Hinderer) Date: Tue, 19 Jan 2010 13:15:14 -0800 (PST) Subject: Performance of lists vs. list comprehensions References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> Message-ID: <57a7f03b-7d43-484d-8952-53bf5fc2774f@c29g2000yqd.googlegroups.com> On 19 Jan., 21:06, Gerald Britton wrote: > [snip] > > > > > Yes, list building from a generator expression *is* expensive. And > > join has to do it, because it has to iterate twice over the iterable > > passed in: once for calculating the memory needed for the joined > > string, and once more to actually do the join (this is implementation > > dependent, of course). If the iterable is a list already, the list > > building is not needed. > > if join has to iterate twice over the iterable, how does this work? > > $ python3.1 > Python 3.1.1+ (r311:74480, Nov ?2 2009, 14:49:22) > [GCC 4.4.1] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > > >>> l = map(str, (x for x in range(10) if int(x)%2)) > >>> '.'.join(l) > '1.3.5.7.9' > > If join had to iterate twice over l, it would be consumed on the first > pass. Yes. (Coincidentally, l is consumed in the first execution of the Timer ()-statement, which is why I had to add the call to list. Not to mention the xrange example of Stephen. But all this is not linked to what join does internally.) > If it works as you say then join would have to copy the > iterable on the first pass, effectively turning it into a list. Yes, that's what I'm saying above in the first two lines of what you quoted. I should have added somehting like "AFAIK CPython does it that way". > Though I didn't read through it, I would suppose that join could use a > dynamic-table approach to hold the result, starting with some > guesstimate then expanding the result buffer if and when needed. Probably. But that's not what happens. Try "".join("" for x in range(10**10) and watch join eating memory. From aahz at pythoncraft.com Tue Jan 19 16:15:35 2010 From: aahz at pythoncraft.com (Aahz) Date: 19 Jan 2010 13:15:35 -0800 Subject: integer and string compare, is that correct? References: Message-ID: In article , Peter Otten <__peter__ at web.de> wrote: > >The use cases for an order that works across types like int and str are >weak to non-existent. Implementing it was considered a mistake and has >been fixed in Python 3: That is not precisely correct from my POV. The primary use case for order that works across types is sorting lists of heterogeneous data. Many people relied on that feature; however, experience showed that it caused more bugs than it fixed. But that doesn't obviate the use of the feature. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." --Red Adair From gerald.britton at gmail.com Tue Jan 19 16:20:42 2010 From: gerald.britton at gmail.com (Gerald Britton) Date: Tue, 19 Jan 2010 16:20:42 -0500 Subject: Performance of lists vs. list comprehensions In-Reply-To: References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> Message-ID: <5d1a32001001191320m549a124bw7dd392fd036409a4@mail.gmail.com> That's surprising. I wouldn't implement it that way at all. I'd use a dynamically-expanding buffer as I suggested. That way you get a single pass and don't have to calculate anything before you begin. In the best case, you'd use half the memory (the result just fits in the buffer after its last expansion and no saved tuple). In the worst case, the memory use is about the same (you just expanded the buffer using a 2x expansion rule then you hit the last item). Still I suppose the author thought of that approach and rejected it for reasons I can't yet see. On Tue, Jan 19, 2010 at 4:01 PM, Arnaud Delobelle wrote: > Gerald Britton writes: > >> [snip] >> >>> >>> Yes, list building from a generator expression *is* expensive. And >>> join has to do it, because it has to iterate twice over the iterable >>> passed in: once for calculating the memory needed for the joined >>> string, and once more to actually do the join (this is implementation >>> dependent, of course). If the iterable is a list already, the list >>> building is not needed. >> >> if join has to iterate twice over the iterable, how does this work? >> >> $ python3.1 >> Python 3.1.1+ (r311:74480, Nov ?2 2009, 14:49:22) >> [GCC 4.4.1] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >>>>> l = map(str, (x for x in range(10) if int(x)%2)) >>>>> '.'.join(l) >> '1.3.5.7.9' >>>>> >> >> If join had to iterate twice over l, it would be consumed on the first >> pass. ?If it works as you say then join would have to copy the >> iterable on the first pass, effectively turning it into a list. >> Though I didn't read through it, I would suppose that join could use a >> dynamic-table approach to hold the result, starting with some >> guesstimate then expanding the result buffer if and when needed. > > Looking at the source (py3k): > > PyObject * > PyUnicode_Join(PyObject *separator, PyObject *seq) > { > ? ?[skip declarations] > > ? ?fseq = PySequence_Fast(seq, ""); > ? ?if (fseq == NULL) { > ? ? ? ?return NULL; > ? ?} > > ? ?[code that works out the length of the joined string then allocates > ? ? memory, then fills it] > } > > Where PySequence_Fast(seq, "") returns seq if seq is already a tuple or > a list and otherwise returns a new tuple built from the elements of seq. > > So no, it doesn't guess the size of the joined string and yes, it > iterates twice over the "sequence" (I would have thought it should be > called an iterable) by copying it into a tuple. > > -- > Arnaud > -- > http://mail.python.org/mailman/listinfo/python-list > -- Gerald Britton From jgardner at jonathangardner.net Tue Jan 19 16:22:02 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Tue, 19 Jan 2010 13:22:02 -0800 (PST) Subject: Recommended "new" way for config files References: <4B46075D.8080606@mycircuit.org> <4B460CA0.20800@sequans.com> <4B4625BC.6090201@mycircuit.org> <87iqbcus0p.fsf@benfinney.id.au> Message-ID: On Jan 8, 2:54?pm, Ben Finney wrote: > Chris Rebert writes: > > JSON is one option:http://docs.python.org/library/json.html > > YAML is another contender. > Compared to JSON, it is yet to gain as much mind-share, but even more > human-friendly and no less expressive. > > Here are some discussions of YAML that can help you evaluate it: > > ? ? > ? ? > ? ? > > YAML is far too complex to be useful. I played with it a while and found the syntax even more confusing than XML, which is quite a feat. From moekaveli at gmail.com Tue Jan 19 16:23:35 2010 From: moekaveli at gmail.com (Dr. Benjamin David Clarke) Date: Tue, 19 Jan 2010 13:23:35 -0800 (PST) Subject: Updating an OptionMenu every time the text file it reads from is updated (Tkinter) References: <45d53456-0673-4829-aa73-9c98feba7b64@s31g2000yqs.googlegroups.com> Message-ID: <6aae9aa4-1a5a-45ce-8c4c-bcacca977ba7@a13g2000vbf.googlegroups.com> On Jan 19, 7:00?am, Peter Otten <__pete... at web.de> wrote: > Dr. Benjamin David Clarke wrote: > > > I currently have a program that reads in values for an OptionMenu from > > a text file. I also have an option to add a line to that text file > > which corresponds to a new value for that OptionMenu. How can I make > > that OptionMenu update its values based on that text file without > > restarting the program? In other words, every time I add a value to > > the text file, I want the OptionMenu to immediately update to take > > note of this change. I'll provide code if needed. > > Inferred from looking into the Tkinter source code: > > # python 2.6 > import Tkinter as tk > > root = tk.Tk() > > var ?= tk.StringVar() > var.set("One") > > optionmenu = tk.OptionMenu(root, var, "One", "Two", "Three") > optionmenu.grid(row=0, column=1) > > def add_option(): > ? ? value = entry_add.get() > ? ? menu = optionmenu["menu"] > ? ? variable = var > ? ? command = None # what you passed as command argument to optionmenu > ? ? menu.add_command(label=value, > ? ? ? ? ? ? ? ? ? ? ?command=tk._setit(variable, value, command)) > > label_show = tk.Label(root, text="current value") > label_show.grid(row=1, column=0) > entry_show = tk.Entry(root, textvariable=var) > entry_show.grid(row=1, column=1) > > label_add = tk.Label(root, text="new option") > label_add.grid(row=2, column=0) > entry_add = tk.Entry(root) > entry_add.grid(row=2, column=1) > > button_add = tk.Button(root, text="add option", > ? ? ? ? ? ? ? ? ? ?command=add_option) > button_add.grid(row=2, column=2) > > root.mainloop() > > Peter The problem turned out to be fairly simple. I just had to get a little creative with nesting my functions and add or remove the option from the OptionMenu right after adding/removing to the text file. Thanks for pointing me in the right direction. From python at rcn.com Tue Jan 19 16:54:02 2010 From: python at rcn.com (Raymond Hettinger) Date: Tue, 19 Jan 2010 13:54:02 -0800 (PST) Subject: Performance of lists vs. list comprehensions References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> Message-ID: [Wolfram Hinderer] > Yes, list building from a generator expression *is* expensive. And > join has to do it, because it has to iterate twice over the iterable > passed in: once for calculating the memory needed for the joined > string, and once more to actually do the join (this is implementation > dependent, of course). If the iterable is a list already, the list > building is not needed. Good analysis. That is exactly what is going on under the hood. Raymond From arnodel at googlemail.com Tue Jan 19 16:55:49 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Tue, 19 Jan 2010 21:55:49 +0000 Subject: Performance of lists vs. list comprehensions References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> Message-ID: Gerald Britton writes: > That's surprising. I wouldn't implement it that way at all. I'd use a > dynamically-expanding buffer as I suggested. That way you get a > single pass and don't have to calculate anything before you begin. In > the best case, you'd use half the memory (the result just fits in the > buffer after its last expansion and no saved tuple). In the worst > case, the memory use is about the same (you just expanded the buffer > using a 2x expansion rule then you hit the last item). > > Still I suppose the author thought of that approach and rejected it > for reasons I can't yet see. I don't know the reasons, but I'm guessing they could be historic. Before Python had iterators, str.join would mostly have been only given lists and tuples as arguments, in which case the current approach seems to be the most appropriate. Later, when things like generator functions and generator expressions were introduced, perhaps str.join wasn't optimized to accomodate them. -- Arnaud From martin at v.loewis.de Tue Jan 19 17:05:36 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Tue, 19 Jan 2010 23:05:36 +0100 Subject: can i examine the svn rev used to build a python 3 executable? In-Reply-To: References: Message-ID: <4B562CB0.9040204@v.loewis.de> > never mind. just discovered that while "python3 -V" won't do it, > executing it gives me: > > $ python3 > Python 3.2a0 (py3k:77609, Jan 19 2010, 04:10:16) > ... > > and it's that 77609 rev number i was after. If you want that in a command line fashion, do python -c 'import sys;print(sys.subversion[2])' Regards, Martin From peter.milliken at gmail.com Tue Jan 19 17:07:46 2010 From: peter.milliken at gmail.com (Peter) Date: Tue, 19 Jan 2010 14:07:46 -0800 (PST) Subject: thread return code References: <08974f1d-4334-4165-8272-7c178f589bbc@m16g2000yqc.googlegroups.com> Message-ID: On Jan 19, 9:25?pm, "Alf P. Steinbach" wrote: > * Rajat: > > > Hi, > > > I'm using threading module in Python 2.6.4. I'm using thread's join() > > method. > > > On the new thread I'm running a function which returns a code at the > > end. Is there a way I access that code in the parent thread after > > thread finishes? Simply, does join() could get me that code? > > join() always returns None. > > But you can store the code in the thread object, and then access it after the > join(). > > Cheers & hth., > > - Alf The typical way to communicate with a thread is via a queue or pipe. You can do what Alf suggests or you could create a queue (or pipe), pass it to the thread as an argument and have the thread put the "return value" into the queue as the last action prior to exit. After the join() just read the results from the queue. Using a queue or pipe is just a suggestion, the multiprocessing module offers numerous ways to communicate between tasks, have a read and pick whatever mechanism seems appropriate for your situation. Peter From fabiofz at gmail.com Tue Jan 19 18:47:51 2010 From: fabiofz at gmail.com (Fabio Zadrozny) Date: Tue, 19 Jan 2010 21:47:51 -0200 Subject: Pydev 1.5.4 Released Message-ID: Hi All, Pydev 1.5.4 has been released Details on Pydev: http://pydev.org Details on its development: http://pydev.blogspot.com Release Highlights: ------------------------------- * Actions: o Go to matching bracket (Ctrl + Shift + P) o Copy the qualified name of the current context to the clipboard. o Ctrl + Shift + T keybinding is resolved to show globals in any context (note: a conflict may occur if JDT is present -- it can be fixed at the keys preferences if wanted). o Ctrl + 2 shows a dialog with the list of available options. o Wrap paragraph is available in the source menu. o Globals browser will start with the current word if no selection is available (if possible). * Templates: o Scripting engine can be used to add template variables to Pydev. o New template variables for next, previous class or method, current module, etc. o New templates for super and super_raw. o print is now aware of Python 3.x or 2.x * Code analysis and code completion: o Fixed problem when getting builtins with multiple Python interpreters configured. o If there's a hasattr(obj, 'attr), 'attr' will be considered in the code completion and code analysis. o Fixed issue where analysis was only done once when set to only analyze open editor. o Proper namespace leakage semantic in list comprehension. o Better calltips in IronPython. o Support for code-completion in Mac OS (interpreter was crashing if _CF was not imported in the main thread). * Grammar: o Fixed issues with 'with' being used as name or keyword in 2.5. o Fixed error when using nested list comprehension. o Proper 'as' and 'with' handling in 2.4 and 2.5. o 'with' statement accepts multiple items in python 3.0. * Improved hover: o Showing the actual contents of method or class when hovering. o Link to the definition of the token being hovered (if class or method). * Others: o Completions for [{( are no longer duplicated when on block mode. o String substitution can now be configured in the interpreter. o Fixed synchronization issue that could make Pydev halt. o Fixed problem when editing with collapsed code. o Import wasn't found for auto-import location if it import started with 'import' (worked with 'from') o Fixed interactive console problem with help() function in Python 3.1 o NullPointerException fix in compare editor. What is Pydev? --------------------------- Pydev is a plugin that enables users to use Eclipse for Python, Jython and IronPython development -- making Eclipse a first class Python IDE -- It comes with many goodies such as code completion, syntax highlighting, syntax analysis, refactor, debug and many others. Cheers, -- Fabio Zadrozny ------------------------------------------------------ Software Developer Aptana http://aptana.com/python Pydev - Python Development Environment for Eclipse http://pydev.org http://pydev.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at REMOVE-THIS-cybersource.com.au Tue Jan 19 20:52:41 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 20 Jan 2010 01:52:41 GMT Subject: Performance of lists vs. list comprehensions References: <5d1a32001001190730u4fa2f26cn62e79c2c73a08b@mail.gmail.com> <7a9c25c21001190818g7bd0ca6n336e6b1d52af0b2c@mail.gmail.com> Message-ID: <00dfe6ee$0$15570$c3e8da3@news.astraweb.com> On Tue, 19 Jan 2010 11:26:43 -0500, Gerald Britton wrote: > Interestingly, I scaled it up to a million list items with more or less > the same results. A million items is not a lot of data. Depending on the size of each object, that might be as little as 4 MB of data: >>> L = ['' for _ in xrange(10**6)] >>> sys.getsizeof(L) 4348732 Try generating a billion items, or even a hundred million, and see how you go. This is a good lesson in the dangers of premature optimization. I can't think how many times I've written code using a generator expression passed to join, thinking that would surely be faster than using a list comprehension ("save building a temporary list first"). -- Steven From steve at REMOVE-THIS-cybersource.com.au Tue Jan 19 21:52:16 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 20 Jan 2010 02:52:16 GMT Subject: Performance of lists vs. list comprehensions References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> Message-ID: <00dff4e6$0$15570$c3e8da3@news.astraweb.com> On Tue, 19 Jan 2010 16:20:42 -0500, Gerald Britton wrote: > That's surprising. I wouldn't implement it that way at all. I'd use a > dynamically-expanding buffer as I suggested. That way you get a single > pass and don't have to calculate anything before you begin. In the best > case, you'd use half the memory (the result just fits in the buffer > after its last expansion and no saved tuple). In the worst case, the > memory use is about the same (you just expanded the buffer using a 2x > expansion rule then you hit the last item). In the worst case, you waste 50% of the memory allocated. And because strings are immutable (unlike lists and dicts, which also use this approach), you can never use that memory until the string is garbage collected. In the current approach, join produces a temporary sequence, but it doesn't last very long. With your suggested approach, you could end up with a large number of long-lasting strings up to twice the size necessary. Since join is particularly useful for building large strings, this could be a significant memory pessimation. The obvious fix is for join to shrink the buffer once it has finished building the string, but the success of that may be operating system dependent. I don't know -- it sounds like a recipe for memory fragmentation to me. And even if it can be done, reclaiming the memory will take time, potentially more time than the current approach. Still, it's probably worth trying, and seeing if it speeds join up. -- Steven From metolone+gmane at gmail.com Tue Jan 19 22:34:12 2010 From: metolone+gmane at gmail.com (Mark Tolonen) Date: Tue, 19 Jan 2010 19:34:12 -0800 Subject: Py 3: How to switch application to Unicode strings? References: <92288825-8b01-47fb-8592-f65a7d7aeb8a@k35g2000yqb.googlegroups.com> <7a9c25c21001191156j46a7fdadt58b728477b85e651@mail.gmail.com> Message-ID: "Stephen Hansen" wrote in message news:7a9c25c21001191156j46a7fdadt58b728477b85e651 at mail.gmail.com... > On Tue, Jan 19, 2010 at 7:50 AM, Gnarlodious > wrote: > >> I am using Python 3, getting an error from SQLite: >> >> sqlite3.ProgrammingError: You must not use 8-bit bytestrings unless >> you use a text_factory that can interpret 8-bit bytestrings (like >> text_factory = str). It is highly recommended that you instead just >> switch your application to Unicode strings. >> >> So... how do I switch to Unicode? I thought I was doing it when I put > > # coding:utf-8 >> >> at the start of my script. >> > > All that does is mean that the script itself is encoded as utf8. > Actually it means that the user has declared that the source file is encoded in utf-8. A common source of errors is that the source file is *not* encoded in utf-8. Make sure to save the source file in the encoding declared. -Mark From gnarlodious at gmail.com Tue Jan 19 23:16:40 2010 From: gnarlodious at gmail.com (Gnarlodious) Date: Tue, 19 Jan 2010 20:16:40 -0800 (PST) Subject: Py 3: How to switch application to Unicode strings? References: <92288825-8b01-47fb-8592-f65a7d7aeb8a@k35g2000yqb.googlegroups.com> <7a9c25c21001191156j46a7fdadt58b728477b85e651@mail.gmail.com> Message-ID: <646ab38b-0710-4d31-b9e1-8a6ee7bfa42f@21g2000yqj.googlegroups.com> Well, Python 3 is supposed to be all Unicode by default. I shouldn't even need to say # coding:UTF-8 And, the file is saved as Unicode. There are many mentions of this error found by Google, but none seen to clearly say what the problem is or how to fix it. FYI, the problem line says: cursor.execute('insert into Data values (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)', frameTuple) and one of the strings in the tuple contains a character like '?'. I have a version of the SQLite editor that works as expected in a browser, I don't know why. -- Gnarlie From alfps at start.no Tue Jan 19 23:25:22 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 20 Jan 2010 05:25:22 +0100 Subject: Performance of lists vs. list comprehensions In-Reply-To: <00dff4e6$0$15570$c3e8da3@news.astraweb.com> References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> <00dff4e6$0$15570$c3e8da3@news.astraweb.com> Message-ID: * Steven D'Aprano: > On Tue, 19 Jan 2010 16:20:42 -0500, Gerald Britton wrote: > >> That's surprising. I wouldn't implement it that way at all. I'd use a >> dynamically-expanding buffer as I suggested. That way you get a single >> pass and don't have to calculate anything before you begin. In the best >> case, you'd use half the memory (the result just fits in the buffer >> after its last expansion and no saved tuple). In the worst case, the >> memory use is about the same (you just expanded the buffer using a 2x >> expansion rule then you hit the last item). > > In the worst case, you waste 50% of the memory allocated. Yes. That is a good argument for not doing the expanding buffer thing. But such buffers may be generally present anyway, resulting from optimization of "+". Using CPython 2.6.4 in Windows XP: >>> def elapsed_time_for( f, n_calls ): ... return timeit.timeit( f, number = n_calls ) ... >>> def appender( n ): ... def makestr( n = n ): ... s = "" ... for i in xrange( n ): ... s = s + "A" ... return s ... return makestr ... >>> appender( 1000 )() == 1000*"A" True >>> >>> for i in xrange( 10 ): ... print( elapsed_time_for( appender( 10000*(i+1) ), 100 ) ) ... 0.782596670811 1.37728454314 2.10189898437 2.76442173517 3.34536707878 4.08251830889 4.79620119317 5.42201844089 6.12892811796 6.84236460221 >>> _ Here the linear increase of times indicate that the "+" is being optimized using an expanding buffer for the string. If only the minimal space was allocated each time then one would expect O(n^2) behavior instead of the apparently O(n) above. Example of that O(n^2) behavior given below. >>> def exact_appender( n ): ... def makestr( n = n ): ... s = "" ... for i in xrange( n ): ... new_s = s + "A" ... s = new_s ... return s ... return makestr ... >>> exact_appender( 1000 )() == 1000*"A" True >>> for i in xrange( 10 ): ... print( elapsed_time_for( exact_appender( 10000*(i+1) ), 100 ) ) ... 3.28094241027 9.30584501661 19.5319170453 33.6563767183 52.3327800042 66.5475022663 84.8809736992 Traceback (most recent call last): File "", line 2, in File "", line 2, in elapsed_time_for File "C:\Program Files\cpython\python26\lib\timeit.py", line 227, in timeit return Timer(stmt, setup, timer).timeit(number) File "C:\Program Files\cpython\python26\lib\timeit.py", line 193, in timeit timing = self.inner(it, self.timer) File "C:\Program Files\cpython\python26\lib\timeit.py", line 99, in inner _func() File "", line 5, in makestr KeyboardInterrupt >>> _ So, given that apparently the simple '+' in the first example is optimized using an expanding buffer, which then hangs around, it's not clear to me that the space optimization in 'join' really helps. It may be (but isn't necessarily) like shoveling snow in a snowstorm. Then the effort/cost could be for naught. > And because > strings are immutable (unlike lists and dicts, which also use this > approach), you can never use that memory until the string is garbage > collected. I think that the simple '+', with the apparent optimization shown in the first example above, can use that space. I know for a fact that when you control a string implementation then it can do that (since I've implemented that). But I don't know for a fact that it's practical to do so in Python. In order to use the space the implementation must know that there's only one reference to the string. And I don't know whether that information is readily available in a CPython implementation (say), although I suspect that it is. Cheers, - Alf From apt.shansen at gmail.com Tue Jan 19 23:45:11 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Tue, 19 Jan 2010 20:45:11 -0800 Subject: Py 3: How to switch application to Unicode strings? In-Reply-To: <646ab38b-0710-4d31-b9e1-8a6ee7bfa42f@21g2000yqj.googlegroups.com> References: <92288825-8b01-47fb-8592-f65a7d7aeb8a@k35g2000yqb.googlegroups.com> <7a9c25c21001191156j46a7fdadt58b728477b85e651@mail.gmail.com> <646ab38b-0710-4d31-b9e1-8a6ee7bfa42f@21g2000yqj.googlegroups.com> Message-ID: <7a9c25c21001192045t460629c0pb928c5abd24186f8@mail.gmail.com> On Tue, Jan 19, 2010 at 8:16 PM, Gnarlodious wrote: > Well, Python 3 is supposed to be all Unicode by default. I shouldn't > even need to say > # coding:UTF-8 > > And, the file is saved as Unicode. > > There are many mentions of this error found by Google, but none seen > to clearly say what the problem is or how to fix it. > > FYI, the problem line says: > > cursor.execute('insert into Data values > (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)', frameTuple) > > and one of the strings in the tuple contains a character like '?'. > I have a version of the SQLite editor that works as expected in a > browser, I don't know why. > > But is it a -unicode- string, or a -byte- string? Print it with repr(). By that error, it seems like its a bytestring. So you read it or got it from a source which provided it to you not as unicode. In that case, find out what the encoding is-- and decode it. after = before.decode("utf8") Python 3 is not 'all unicode'; or 'by default'. Python 3 has a firm line in the sand. Everything is either explicitly a byte string (bytes) or explicitly a unicode string (str). --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at REMOVE-THIS-cybersource.com.au Wed Jan 20 01:12:18 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 20 Jan 2010 06:12:18 GMT Subject: Performance of lists vs. list comprehensions References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> <00dff4e6$0$15570$c3e8da3@news.astraweb.com> Message-ID: <00e023c8$0$15570$c3e8da3@news.astraweb.com> On Wed, 20 Jan 2010 05:25:22 +0100, Alf P. Steinbach wrote: > * Steven D'Aprano: >> On Tue, 19 Jan 2010 16:20:42 -0500, Gerald Britton wrote: >> >>> That's surprising. I wouldn't implement it that way at all. I'd use a >>> dynamically-expanding buffer as I suggested. That way you get a >>> single pass and don't have to calculate anything before you begin. In >>> the best case, you'd use half the memory (the result just fits in the >>> buffer after its last expansion and no saved tuple). In the worst >>> case, the memory use is about the same (you just expanded the buffer >>> using a 2x expansion rule then you hit the last item). >> >> In the worst case, you waste 50% of the memory allocated. > > Yes. That is a good argument for not doing the expanding buffer thing. > But such buffers may be generally present anyway, resulting from > optimization of "+". As near as I can determine, the CPython optimization you are referring to doesn't use the "double the buffer when needed" technique. It operates on a completely different strategy. As near as I can tell (as a non-C speaker), it re-sizes the string in place to the size actually needed, thus reducing the amount of copying needed. The optimization patch is here: http://bugs.python.org/issue980695 and some history (including Guido's opposition to the patch) here: http://mail.python.org/pipermail/python-dev/2004-August/046686.html Nevertheless, the patch is now part of CPython since 2.4, but must be considered an implementation-specific optimization. It doesn't apply to Jython, and likely other implementations. > Using CPython 2.6.4 in Windows XP: [snip time measurements] > Here the linear increase of times indicate that the "+" is being > optimized using an expanding buffer for the string. Be careful of jumping to the conclusion from timings that a certain algorithm is used. All you can really tell is that, whatever the algorithm is, it has such-and-such big-oh behaviour. > If only the minimal > space was allocated each time then one would expect O(n^2) behavior > instead of the apparently O(n) above. I don't follow that reasoning. > Example of that O(n^2) behavior given below. The example shown demonstrates that the + optimization only applies in certain specific cases. In fact, it only applies to appending: s += t s = s + t but not prepending or concatenation with multiple strings: s = t + s s += t + u However, keep in mind that the CPython keyhole optimizer will take something like this: s += "x" + "y" and turn it into this: s += "xy" which the concatenation optimization does apply to. Optimizations make it hard to reason about the behaviour of algorithms! -- Steven From r.grimm at science-computing.de Wed Jan 20 01:39:11 2010 From: r.grimm at science-computing.de (Rainer Grimm) Date: Tue, 19 Jan 2010 22:39:11 -0800 (PST) Subject: What is a list compression in Python? References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> Message-ID: Hallo, you can also look at list comprehension as syntactic sugar for the functions map and filter. The two functions from the functional world can be expressed in a comprehensive way with list comprehension. >>> [x**2 for x in range(10) ] == map ( lambda x: x*x, range(10)) True >>> [ x for x in range(10) if x%2 == 0 ] == filter ( lambda x: x%2 == 0 , range(10)) True Greetings From dieter at handshake.de Wed Jan 20 01:41:13 2010 From: dieter at handshake.de (Dieter Maurer) Date: 20 Jan 2010 07:41:13 +0100 Subject: setattr() oddness In-Reply-To: References: <3eba07d0-fc14-4f51-a777-1c47cc866387@j14g2000yqm.googlegroups.com> <409dce36-691c-4c9f-ae69-aba358377b6a@j4g2000yqe.googlegroups.com> <4b517acc$1@dnews.tpgi.com.au> Message-ID: Steven D'Aprano writes on 18 Jan 2010 06:47:59 GMT: > On Mon, 18 Jan 2010 07:25:58 +0100, Dieter Maurer wrote: > > > Lie Ryan writes on Sat, 16 Jan 2010 19:37:29 +1100: > >> On 01/16/10 10:10, Sean DiZazzo wrote: > >> > Interesting. I can understand the "would take time" argument, but I > >> > don't see any legitimate use case for an attribute only accessible > >> > via getattr(). Well, at least not a pythonic use case. > >> > >> mostly for people (ab)using attributes instead of dictionary. > > > > Here is one use case: > > > > A query application. Queries are described by complex query objects. > > For efficiency reasons, query results should be cached. For this, it is > > not unnatural to use query objects as cache keys. Then, query objects > > must not get changed in an uncontrolled way. I use "__setattr__" to > > control access to the objects. > > > (1) Wouldn't it be more natural to store these query keys in a list or > dictionary rather than as attributes on an object? > > e.g. instead of: > > cache.__setattr__('complex query object', value) > > use: > > cache['complex query object'] = value Few will use "cache.__setattr__(...)" but "cache.attr = ..." which is nicer than "cache['attr'] = ...". Moreover, it is not the cache but the query of which I want to protect modification. My cache indeed uses "cache[query_object] = ...". But I want to prevent "query_object" from being changed after a potential caching. > (2) How does __setattr__ let you control access to the object? If a user > wants to modify the cache, and they know the complex query object, what's > stopping them from using __setattr__ too? In my specific case, "__setattr__" prevents all modifications via attribute assignment. The class uses "__dict__" access to set attributes when it knows it is still safe. Of course, this is no real protection against attackers (which could use "__dict__" as well). It only protects against accidental change of query objects. Meanwhile, I remembered a more important use case for "__setattr__": providing for transparent persistancy. The "ZODB" (Zope Object DataBase) customizes "__setattr__" in order to intercept object modifications and register automatically that the change needs to be persisted at the next transaction commit. Dieter From arnodel at googlemail.com Wed Jan 20 02:17:27 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Wed, 20 Jan 2010 07:17:27 +0000 Subject: Py 3: How to switch application to Unicode strings? References: <92288825-8b01-47fb-8592-f65a7d7aeb8a@k35g2000yqb.googlegroups.com> <7a9c25c21001191156j46a7fdadt58b728477b85e651@mail.gmail.com> <646ab38b-0710-4d31-b9e1-8a6ee7bfa42f@21g2000yqj.googlegroups.com> Message-ID: Gnarlodious writes: > Well, Python 3 is supposed to be all Unicode by default. I shouldn't > even need to say > # coding:UTF-8 > > And, the file is saved as Unicode. > When a filed is saved, shouldn't it be in a specific encoding? I don't see how you can save your file 'as unicode'. You should save your file with UTF-8 encoding. HTH -- Arnaud From metolone+gmane at gmail.com Wed Jan 20 02:22:06 2010 From: metolone+gmane at gmail.com (Mark Tolonen) Date: Tue, 19 Jan 2010 23:22:06 -0800 Subject: Py 3: How to switch application to Unicode strings? References: <92288825-8b01-47fb-8592-f65a7d7aeb8a@k35g2000yqb.googlegroups.com><7a9c25c21001191156j46a7fdadt58b728477b85e651@mail.gmail.com> <646ab38b-0710-4d31-b9e1-8a6ee7bfa42f@21g2000yqj.googlegroups.com> Message-ID: "Gnarlodious" wrote in message news:646ab38b-0710-4d31-b9e1-8a6ee7bfa42f at 21g2000yqj.googlegroups.com... > Well, Python 3 is supposed to be all Unicode by default. I shouldn't > even need to say > # coding:UTF-8 Yes, in Python 3, an absence of a 'coding' line assumes UTF-8. > And, the file is saved as Unicode. There is no such thing as "saved as Unicode". Unicode is not an encoding. For example, '?' is the Unicode codepoint 241. This can be stored in a file in a number of ways. UTF-8 is the two bytes 0xc3 0xB1. UTF-16LE is 0xF1 0x00. UTF-16BE is 0x00 0xF1. latin-1 is the single byte 0xF1. If your editor saves your file in the encoding "latin-1", and you don't use a coding line to declare it, Python 3 will throw an error if it finds a non-UTF-8 byte sequence in the file. > There are many mentions of this error found by Google, but none seen > to clearly say what the problem is or how to fix it. > FYI, the problem line says: > cursor.execute('insert into Data values > (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)', frameTuple) Is frameTuple a byte string or Unicode string. print(repr(frameTuple)). > and one of the strings in the tuple contains a character like '?'. > I have a version of the SQLite editor that works as expected in a > browser, I don't know why. Post the simplest, complete source code that exhibits the problem. -Mark From apt.shansen at gmail.com Wed Jan 20 02:32:31 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Tue, 19 Jan 2010 23:32:31 -0800 Subject: What is a list compression in Python? In-Reply-To: References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> Message-ID: <7a9c25c21001192332m17e8509blc7e3c67bf9ade364@mail.gmail.com> On Tue, Jan 19, 2010 at 10:39 PM, Rainer Grimm wrote: > Hallo, > you can also look at list comprehension as syntactic sugar for the > functions map and filter. The two functions from the functional world > can be expressed in a comprehensive way with list comprehension. > >>> [x**2 for x in range(10) ] == map ( lambda x: x*x, range(10)) > True > >>> [ x for x in range(10) if x%2 == 0 ] == filter ( lambda x: x%2 == 0 , > range(10)) > True > I really don't think you can call comprehensions as mere syntactic sugar, as there are measurable performance differences between the two. For something to be syntactic sugar, it must be essentially equivalent. A list comprehension is, IIUC, real syntactic sugar around a for loop. Meaning it, in a real way, simply creates a for-loop with special syntax. A list comprehension can be /equivalent in effect/ to the functions map and filter, but it is not syntactic sugar. It is generating entirely different code-paths with entirely different performance characteristics to achieve the same goals. But that's not sugar. Syntactic sugar is a sweet, cute, or cute way to express a more complex thought in a simpler or more concise way. But the -effect- and -result- of both thoughts are the same: if you do it manually, or with the sugar, they're essentially equivalent. That's why its sugar... its just something to make the experience sweeter, it doesn't -really- add any value beyond making the experience sweeter. Consider your tests: >>> Timer("[x**2 for x in range(10) ]").timeit() 3.4986340999603271 >>> Timer("map ( lambda x: x*x, range(10))").timeit() 4.5014309883117676 >>> Timer("[ x for x in range(10) if x%2 == 0 ]").timeit() 3.3268649578094482 >>> Timer("filter ( lambda x: x%2 == 0 , range(10))").timeit() 5.3649170398712158 The list comprehension version performs distinctly better in each case. The end result of the two are the same, but one is not sugar for the other. A list comprehension is in essence a compact way of writing a for loop that builds a list. A map (or filter) operation is a functional approach to build a list: the difference is that the latter requires you to call a function a lot, and function calls in Python are not cheap. --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From apt.shansen at gmail.com Wed Jan 20 02:34:18 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Tue, 19 Jan 2010 23:34:18 -0800 Subject: What is a list compression in Python? In-Reply-To: <7a9c25c21001192332m17e8509blc7e3c67bf9ade364@mail.gmail.com> References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> <7a9c25c21001192332m17e8509blc7e3c67bf9ade364@mail.gmail.com> Message-ID: <7a9c25c21001192334r5ed45f8fo891e37fcbe2caf2e@mail.gmail.com> On Tue, Jan 19, 2010 at 11:32 PM, Stephen Hansen wrote: > > I really don't think you can call comprehensions as mere syntactic sugar, > Err, I misspoke. I don't really think you can call comprehensions mere syntactic sugar /for map and filter/. It IS mere syntactic sugar for a "for loop". But not for the functional equivalent. --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at REMOVE-THIS-cybersource.com.au Wed Jan 20 02:36:22 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 20 Jan 2010 07:36:22 GMT Subject: Best way to convert sequence of bytes to long integer Message-ID: <00e0377c$0$15570$c3e8da3@news.astraweb.com> I have a byte string (Python 2.x string), e.g.: s = "g%$f yg\n1\05" assert len(s) == 10 I wish to convert it to a long integer, treating it as base-256. Currently I'm using: def makelong(s): n = 0 for c in s: n *= 256 n += ord(c) return n which gives: >>> makelong(s) 487088900085839492165893L Is this the best way, or have I missed some standard library function? Thanks in advance, -- Steven From stefan_ml at behnel.de Wed Jan 20 02:38:12 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 20 Jan 2010 08:38:12 +0100 Subject: Performance of lists vs. list comprehensions In-Reply-To: <00e023c8$0$15570$c3e8da3@news.astraweb.com> References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> <00dff4e6$0$15570$c3e8da3@news.astraweb.com> <00e023c8$0$15570$c3e8da3@news.astraweb.com> Message-ID: <4b56b2e5$0$7628$9b4e6d93@newsspool1.arcor-online.net> Steven D'Aprano, 20.01.2010 07:12: > On Wed, 20 Jan 2010 05:25:22 +0100, Alf P. Steinbach wrote: >> That is a good argument for not doing the expanding buffer thing. >> But such buffers may be generally present anyway, resulting from >> optimization of "+". > > As near as I can determine, the CPython optimization you are referring to > doesn't use the "double the buffer when needed" technique. It operates on > a completely different strategy. As near as I can tell (as a non-C > speaker), it re-sizes the string in place to the size actually needed, > thus reducing the amount of copying needed. > >> Using CPython 2.6.4 in Windows XP: > [snip time measurements] >> Here the linear increase of times indicate that the "+" is being >> optimized using an expanding buffer for the string. > > Be careful of jumping to the conclusion from timings that a certain > algorithm is used. All you can really tell is that, whatever the > algorithm is, it has such-and-such big-oh behaviour. Which is particularly tricky here since the algorithms depends more on the OS than on the code in CPython. The better timings come from the fact that the OS does *not* need to copy the buffer on each iteration, but does smarter things when asked to enlarge the buffer. If you ran the benchmark on an OS that *did* copy the buffer each time, the runtime would really be quadratic. BTW, I think it would actually be worth trying to apply the same approach to str.join() if the argument is not a sequence (obviously followed by a benchmark on different platforms). Stefan From stefan_ml at behnel.de Wed Jan 20 02:56:48 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 20 Jan 2010 08:56:48 +0100 Subject: Best way to convert sequence of bytes to long integer In-Reply-To: <00e0377c$0$15570$c3e8da3@news.astraweb.com> References: <00e0377c$0$15570$c3e8da3@news.astraweb.com> Message-ID: <4b56b740$0$7615$9b4e6d93@newsspool1.arcor-online.net> Steven D'Aprano, 20.01.2010 08:36: > I have a byte string (Python 2.x string), e.g.: > > s = "g%$f yg\n1\05" > assert len(s) == 10 > > I wish to convert it to a long integer, treating it as base-256. > Currently I'm using: > > def makelong(s): > n = 0 > for c in s: > n *= 256 > n += ord(c) > return n > > > which gives: > >>>> makelong(s) > 487088900085839492165893L > > > Is this the best way, or have I missed some standard library function? Have you checked if the struct module offers anything here? Stefan From alfps at start.no Wed Jan 20 03:21:30 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 20 Jan 2010 09:21:30 +0100 Subject: Performance of lists vs. list comprehensions In-Reply-To: <00e023c8$0$15570$c3e8da3@news.astraweb.com> References: <64c68195-f31f-4f55-8a82-2b3e593748d5@p8g2000yqb.googlegroups.com> <00dff4e6$0$15570$c3e8da3@news.astraweb.com> <00e023c8$0$15570$c3e8da3@news.astraweb.com> Message-ID: * Steven D'Aprano: > On Wed, 20 Jan 2010 05:25:22 +0100, Alf P. Steinbach wrote: > >> * Steven D'Aprano: >>> On Tue, 19 Jan 2010 16:20:42 -0500, Gerald Britton wrote: >>> >>>> That's surprising. I wouldn't implement it that way at all. I'd use a >>>> dynamically-expanding buffer as I suggested. That way you get a >>>> single pass and don't have to calculate anything before you begin. In >>>> the best case, you'd use half the memory (the result just fits in the >>>> buffer after its last expansion and no saved tuple). In the worst >>>> case, the memory use is about the same (you just expanded the buffer >>>> using a 2x expansion rule then you hit the last item). >>> In the worst case, you waste 50% of the memory allocated. >> Yes. That is a good argument for not doing the expanding buffer thing. >> But such buffers may be generally present anyway, resulting from >> optimization of "+". > > > As near as I can determine, the CPython optimization you are referring to > doesn't use the "double the buffer when needed" technique. It operates on > a completely different strategy. As near as I can tell (as a non-C > speaker), it re-sizes the string in place to the size actually needed, > thus reducing the amount of copying needed. > > The optimization patch is here: > > http://bugs.python.org/issue980695 > > and some history (including Guido's opposition to the patch) here: > > http://mail.python.org/pipermail/python-dev/2004-August/046686.html > > Nevertheless, the patch is now part of CPython since 2.4, but must be > considered an implementation-specific optimization. It doesn't apply to > Jython, and likely other implementations. Provided that the CPython code is that code then you're right, it only calls PyObject_REALLOC, depending on that operation having (amortized) constant time. And PyObject_REALLOC can in principle achieve that by relying on paging, e.g. using the OS' virtual memory allocator, instead of doubling and copying. However, I'm baffled by the code I find via Google Code search; there PyObject_REALLOC simply calls malloc and copies, which if that were the code used in CPython 2.4 and greater, and if the '+' code is the code that you refer to above, would produce O(n^2) time for the first '+' example. >> Using CPython 2.6.4 in Windows XP: > [snip time measurements] >> Here the linear increase of times indicate that the "+" is being >> optimized using an expanding buffer for the string. > > Be careful of jumping to the conclusion from timings that a certain > algorithm is used. All you can really tell is that, whatever the > algorithm is, it has such-and-such big-oh behaviour. Well, as for intended meaning you're right about that, it needs not be a doubling buffer. >> If only the minimal >> space was allocated each time then one would expect O(n^2) behavior >> instead of the apparently O(n) above. > > I don't follow that reasoning. The reasoning assumes independent allocations rather than reallocation (extension of existing allocation). Then quadratic time for the example follows from sum(range(n)) = (n^2-n)/2 of the sizes of the data copied. But with PyObject_REALLOC as essentially constant time also 'join' could take advantage of this. :-) Cheers, - Alf From __peter__ at web.de Wed Jan 20 03:43:30 2010 From: __peter__ at web.de (Peter Otten) Date: Wed, 20 Jan 2010 09:43:30 +0100 Subject: Best way to convert sequence of bytes to long integer References: <00e0377c$0$15570$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > I have a byte string (Python 2.x string), e.g.: > > s = "g%$f yg\n1\05" > assert len(s) == 10 > > I wish to convert it to a long integer, treating it as base-256. > Currently I'm using: > > def makelong(s): > n = 0 > for c in s: > n *= 256 > n += ord(c) > return n > > > which gives: > >>>> makelong(s) > 487088900085839492165893L > > > Is this the best way, or have I missed some standard library function? The pickle module uses >>> import binascii >>> s = "g%$f yg\n1\05" >>> long(binascii.hexlify(s), 16) 487088900085839492165893L Peter From rpjday at crashcourse.ca Wed Jan 20 03:58:45 2010 From: rpjday at crashcourse.ca (Robert P. J. Day) Date: Wed, 20 Jan 2010 03:58:45 -0500 (EST) Subject: how to check if a module is importable? Message-ID: finally getting back to clawing my way thru the python 3 book so probably a number of newbie questions coming up. first one -- can i check if a module is importable (equivalently, exists on sys.path, i assume) without trying to import it first? i can see that i can use try/except and just run "import" -- is that the accepted way to test? rday -- ======================================================================== Robert P. J. Day Waterloo, Ontario, CANADA Linux Consulting, Training and Kernel Pedantry. Web page: http://crashcourse.ca Twitter: http://twitter.com/rpjday ======================================================================== From no.email at nospam.invalid Wed Jan 20 04:10:41 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 20 Jan 2010 01:10:41 -0800 Subject: Best way to convert sequence of bytes to long integer References: <00e0377c$0$15570$c3e8da3@news.astraweb.com> Message-ID: <7xk4vdf8fi.fsf@ruckus.brouhaha.com> Steven D'Aprano writes: > s = "g%$f yg\n1\05" > Is this the best way, or have I missed some standard library function? It is really a shame that there is not a standard library function for this. I usually do it using hexadecimal conversion: d = int(s.encode('hex'), 16) From dickinsm at gmail.com Wed Jan 20 04:10:47 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Wed, 20 Jan 2010 01:10:47 -0800 (PST) Subject: Best way to convert sequence of bytes to long integer References: <00e0377c$0$15570$c3e8da3@news.astraweb.com> Message-ID: <5858a2bd-116b-4dd8-bff3-0039b345bff7@34g2000yqp.googlegroups.com> On Jan 20, 7:36?am, Steven D'Aprano wrote: > I have a byte string (Python 2.x string), e.g.: > > s = "g%$f yg\n1\05" > assert len(s) == 10 > > I wish to convert it to a long integer, treating it as base-256. Not that it helps you right now, but provided someone finds the time, there should be an int/long method for this in Python 2.7. It's already implemented for Python 3.2, so it just needs backporting. Python 3.2a0 (py3k:77612, Jan 20 2010, 09:04:15) [GCC 4.0.1 (Apple Inc. build 5490)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> int.from_bytes(b"g%$f yg\n1\05", 'big') 487088900085839492165893 Until then, Peter Otten's solution is about as close as you can get, I think. -- Mark From mal at egenix.com Wed Jan 20 05:02:29 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 20 Jan 2010 11:02:29 +0100 Subject: setattr() oddness In-Reply-To: References: <3eba07d0-fc14-4f51-a777-1c47cc866387@j14g2000yqm.googlegroups.com> <409dce36-691c-4c9f-ae69-aba358377b6a@j4g2000yqe.googlegroups.com> <4b517acc$1@dnews.tpgi.com.au> Message-ID: <4B56D4B5.7000408@egenix.com> Dieter Maurer wrote: > Steven D'Aprano writes on 18 Jan 2010 06:47:59 GMT: >> On Mon, 18 Jan 2010 07:25:58 +0100, Dieter Maurer wrote: >> >>> Lie Ryan writes on Sat, 16 Jan 2010 19:37:29 +1100: >>>> On 01/16/10 10:10, Sean DiZazzo wrote: >>>>> Interesting. I can understand the "would take time" argument, but I >>>>> don't see any legitimate use case for an attribute only accessible >>>>> via getattr(). Well, at least not a pythonic use case. >>>> >>>> mostly for people (ab)using attributes instead of dictionary. >>> >>> Here is one use case: >>> >>> A query application. Queries are described by complex query objects. >>> For efficiency reasons, query results should be cached. For this, it is >>> not unnatural to use query objects as cache keys. Then, query objects >>> must not get changed in an uncontrolled way. I use "__setattr__" to >>> control access to the objects. >> >> >> (1) Wouldn't it be more natural to store these query keys in a list or >> dictionary rather than as attributes on an object? >> >> e.g. instead of: >> >> cache.__setattr__('complex query object', value) >> >> use: >> >> cache['complex query object'] = value > > Few will use "cache.__setattr__(...)" but "cache.attr = ..." which > is nicer than "cache['attr'] = ...". > > Moreover, it is not the cache but the query of which I want to protect > modification. My cache indeed uses "cache[query_object] = ...". > But I want to prevent "query_object" from being changed after a potential > caching. You might want to look at mxProxy which provides object-level access control: http://www.egenix.com/products/python/mxBase/mxProxy/ See the "Access Protocol" in the documentation: http://www.egenix.com/products/python/mxBase/mxProxy/doc/#_Toc162774446 >> (2) How does __setattr__ let you control access to the object? If a user >> wants to modify the cache, and they know the complex query object, what's >> stopping them from using __setattr__ too? > > In my specific case, "__setattr__" prevents all modifications via attribute > assignment. The class uses "__dict__" access to set attributes when > it knows it is still safe. > > Of course, this is no real protection against attackers (which could > use "__dict__" as well). It only protects against accidental change > of query objects. > > > Meanwhile, I remembered a more important use case for "__setattr__": > providing for transparent persistancy. The "ZODB" (Zope Object DataBase) > customizes "__setattr__" in order to intercept object modifications > and register automatically that the change needs to be persisted at > the next transaction commit. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 20 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From jeanmichel at sequans.com Wed Jan 20 05:06:08 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Wed, 20 Jan 2010 11:06:08 +0100 Subject: how to check if a module is importable? In-Reply-To: References: Message-ID: <4B56D590.1070400@sequans.com> Robert P. J. Day wrote: > finally getting back to clawing my way thru the python 3 book so > probably a number of newbie questions coming up. first one -- can i > check if a module is importable (equivalently, exists on sys.path, i > assume) without trying to import it first? > > i can see that i can use try/except and just run "import" -- is that > the accepted way to test? > > rday > -- > AFAIK, that is why ImportError is for. try: import module except ImportError: # handle me pass is commonly used. JM From bornstub at gmail.com Wed Jan 20 05:52:25 2010 From: bornstub at gmail.com (Victor Lin) Date: Wed, 20 Jan 2010 02:52:25 -0800 (PST) Subject: Memory usage problem of twisted server Message-ID: Hi, I encountered an increasing memory usage problem of my twisted server. I have posted a question on stackoverflow: http://stackoverflow.com/questions/2100192/how-to-find-the-source-of-increasing-memory-usage-of-a-twisted-server I have read the article "Improving Python's Memory Allocator" ( http://evanjones.ca/memoryallocator/ ) and Python Memory Management ( http://evanjones.ca/python-memory.html ). And I now know little about how Python manages memory. I am wondering, is that the increasing memory usage problem of my audio broadcasting caused by the how python manage memory? In my server, there are lots of audio data chunks, they are not all in fixed size. For example, the size of audio stream might looks like this: ... chunk size 822 chunk size 878 chunk size 1690 chunk size 1659 chunk size 1643 chunk size 1952 chunk size 373 chunk size 763 chunk size 1535 chunk size 1665 ... All of those chunks are str object. The size of chunks depend on mp3 encoder. You can see most the size of those chunks is bigger than 256 bytes, so that Python uses malloc to allocate spaces for those chunks. In my mind, I saw those chunks occupy different place in heap, all of them are not in fixed size, so that even when the chunk is freed, it is difficult for malloc to find the right place for the new chunk. As the result, there are so many chunks placed in large range of heap, it is difficult for OS to swap pages out. Then the RSS is always high. Is that my guessing correct? How can I monitor the memory allocation of Python? Is there any available tools for this? Or is that modifying the Python source code and recompiling the only way to monitor allocation of memory? Thanks. Victor Lin. From davea at ieee.org Wed Jan 20 06:05:50 2010 From: davea at ieee.org (Dave Angel) Date: Wed, 20 Jan 2010 06:05:50 -0500 Subject: Performance of lists vs. list comprehensions In-Reply-To: <00dfe6ee$0$15570$c3e8da3@news.astraweb.com> References: <5d1a32001001190730u4fa2f26cn62e79c2c73a08b@mail.gmail.com> <7a9c25c21001190818g7bd0ca6n336e6b1d52af0b2c@mail.gmail.com> <00dfe6ee$0$15570$c3e8da3@news.astraweb.com> Message-ID: <4B56E38E.5020907@ieee.org> Steven D'Aprano wrote: > A million items is not a lot of data. Depending on the size of each > object, that might be as little as 4 MB of data: > > >>>> L = ['' for _ in xrange(10**6)] >>>> sys.getsizeof(L) >>>> > 4348732 > > Note that this sys.getsizeof() is only saying the size of the list, not the size of the objects referenced in the list. So that number doesn't depend on "the size of the object." DaveA From rpjday at crashcourse.ca Wed Jan 20 06:59:19 2010 From: rpjday at crashcourse.ca (Robert P. J. Day) Date: Wed, 20 Jan 2010 06:59:19 -0500 (EST) Subject: examining an initial, pristine python3 shell session Message-ID: still working my way through "dive into python 3" and i've already been asked to give a newbie tutorial on it -- blind leading the blind, as it were. that should be hilarious. i'll be using python 3 and it occurred to me that it would be educational (at least for me :-) to display what an initial p3 shell session looks like before doing any imports whatsoever. as in, i run "python3" on my fedora box and, at the ">>>" prompt, i want to show what's already there for the new user. from what little i know so far, i'd start with: >>> __name__ '__main__' >>> to display the name of the current scope(?). backing up a bit, i could run either of: >>> dir() ['__builtins__', '__doc__', '__name__', '__package__'] >>> globals() {'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None} >>> then i might go a bit further to examine some of *those* objects. i admit it might seem a bit dry, but i think it would be handy to have a handle on what a clean shell session looks like before starting to import things, then seeing how that importing changes the session before getting down to actual programming. what other useful commands might i run immediately after starting a session whose output would be informative? i can certainly poke at some of those objects to see them in more detail. i'm just curious what others might recommend. thanks. rday -- ======================================================================== Robert P. J. Day Waterloo, Ontario, CANADA Linux Consulting, Training and Kernel Pedantry. Web page: http://crashcourse.ca Twitter: http://twitter.com/rpjday ======================================================================== From alfps at start.no Wed Jan 20 07:43:23 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 20 Jan 2010 13:43:23 +0100 Subject: examining an initial, pristine python3 shell session In-Reply-To: References: Message-ID: * Robert P. J. Day: > still working my way through "dive into python 3" and i've already > been asked to give a newbie tutorial on it -- blind leading the blind, > as it were. that should be hilarious. > > i'll be using python 3 and it occurred to me that it would be > educational (at least for me :-) to display what an initial p3 shell > session looks like before doing any imports whatsoever. as in, i run > "python3" on my fedora box and, at the ">>>" prompt, i want to show > what's already there for the new user. > > from what little i know so far, i'd start with: > >>>> __name__ > '__main__' > > to display the name of the current scope(?). backing up a bit, i > could run either of: > >>>> dir() > ['__builtins__', '__doc__', '__name__', '__package__'] >>>> globals() > {'__builtins__': , '__name__': > '__main__', '__doc__': None, '__package__': None} > > then i might go a bit further to examine some of *those* objects. i > admit it might seem a bit dry, but i think it would be handy to have a > handle on what a clean shell session looks like before starting to > import things, then seeing how that importing changes the session > before getting down to actual programming. > > what other useful commands might i run immediately after starting a > session whose output would be informative? i can certainly poke at > some of those objects to see them in more detail. i'm just curious > what others might recommend. thanks. That depends on what you mean by "newbie". If it's someone who knows a little bit of programming but is new to Python, then 'help' would definitely be about the first thing I'd show her. But if it's someone who doesn't even know anything about programming, then I'd recommend (blatant plug) -- its first two chapters are constructed around complete, concrete examples. However, you would have to adapt just the *sense* of the first chapter, which is only about tool usage, to *nix, since it's written for Windows. I'd not dive into 'help' for the someone who doesn't know anything because it gets technical pretty fast, and because she will get back to that on her own when it's time. Whatever you do, and whatever the background of the newbie, do introduce turtle graphics right away. The ch 2 of the above reference contains some t.g. examples that you might use (initial silly figures, graphs of functions, recursive figures). It doesn't go into the turtle module objects. But if objects are what you want to show right away, then I think the turtle module is great also for that, because those objects are simple and can be easily explored. Cheers & hth., - Alf From dickinsm at gmail.com Wed Jan 20 09:18:55 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Wed, 20 Jan 2010 06:18:55 -0800 (PST) Subject: Best way to convert sequence of bytes to long integer References: <00e0377c$0$15570$c3e8da3@news.astraweb.com> Message-ID: <9b870cc2-0966-4cb2-be93-573317c06eb0@a6g2000yqm.googlegroups.com> On Jan 20, 7:36?am, Steven D'Aprano wrote: > I have a byte string (Python 2.x string), e.g.: > > s = "g%$f yg\n1\05" > assert len(s) == 10 > > I wish to convert it to a long integer, treating it as base-256. By the way, are you willing to divulge what you're using this functionality for? When trying to hack out the API for int.to_bytes and int.from_bytes on the mailing list and bug tracker, some of the discussion about use-cases seemed a little too much like guesswork; I'd be curious to find out about real-life use-cases. Mark From lie.1296 at gmail.com Wed Jan 20 09:37:58 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Thu, 21 Jan 2010 01:37:58 +1100 Subject: how to check if a module is importable? In-Reply-To: References: Message-ID: <4b57155b@dnews.tpgi.com.au> On 01/20/10 19:58, Robert P. J. Day wrote: > > finally getting back to clawing my way thru the python 3 book so > probably a number of newbie questions coming up. first one -- can i > check if a module is importable (equivalently, exists on sys.path, i > assume) without trying to import it first? > > i can see that i can use try/except and just run "import" -- is that > the accepted way to test? check imp.find_module() From bblais at bryant.edu Wed Jan 20 09:40:32 2010 From: bblais at bryant.edu (Brian Blais) Date: Wed, 20 Jan 2010 09:40:32 -0500 Subject: turtle, ipython, and pylab don't mix Message-ID: Hello, I am noticing a hard crash of the ipython interpreter, with the pylab option, with the turtle module. I am on Win XP, Enthought Edition 6.0.0 which uses Python 2.6. I can't reproduce it without the pylab option, nor on the Mac (with an earlier Enthought Edition). The crash happens only when I try to move the turtle window, after it has drawn. The following commands are enough to give the resulting error. In [1]: from turtle import * In [2]: pendown() In [3]: forward(10) In [4]: Fatal Python error: PyEval_RestoreThread: NULL tstate This application has requested the Runtime to terminate it in an unusual way. Please contact the application's support team for more information. Any ideas? Of course, I'll use it without the pylab option, but that means that there are two icons I need instead of one. Minor inconvenience for me, but a bit more of a pain for my students. thanks, Brian Blais -- Brian Blais bblais at bryant.edu http://web.bryant.edu/~bblais http://bblais.blogspot.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From alfps at start.no Wed Jan 20 09:40:59 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 20 Jan 2010 15:40:59 +0100 Subject: examining an initial, pristine python3 shell session In-Reply-To: References: Message-ID: * Alf P. Steinbach: > * Robert P. J. Day: >> still working my way through "dive into python 3" and i've already >> been asked to give a newbie tutorial on it -- blind leading the blind, >> as it were. that should be hilarious. >> >> i'll be using python 3 and it occurred to me that it would be >> educational (at least for me :-) to display what an initial p3 shell >> session looks like before doing any imports whatsoever. as in, i run >> "python3" on my fedora box and, at the ">>>" prompt, i want to show >> what's already there for the new user. >> >> from what little i know so far, i'd start with: >> >>>>> __name__ >> '__main__' >> >> to display the name of the current scope(?). backing up a bit, i >> could run either of: >> >>>>> dir() >> ['__builtins__', '__doc__', '__name__', '__package__'] >>>>> globals() >> {'__builtins__': , '__name__': >> '__main__', '__doc__': None, '__package__': None} >> >> then i might go a bit further to examine some of *those* objects. i >> admit it might seem a bit dry, but i think it would be handy to have a >> handle on what a clean shell session looks like before starting to >> import things, then seeing how that importing changes the session >> before getting down to actual programming. >> >> what other useful commands might i run immediately after starting a >> session whose output would be informative? i can certainly poke at >> some of those objects to see them in more detail. i'm just curious >> what others might recommend. thanks. > > That depends on what you mean by "newbie". > > If it's someone who knows a little bit of programming but is new to > Python, then 'help' would definitely be about the first thing I'd show her. He he... Try "help antigravity". :-) > But if it's someone who doesn't even know anything about programming, > then I'd recommend (blatant plug) http://tinyurl.com/programmingbookP3> -- its first two chapters are > constructed around complete, concrete examples. However, you would have > to adapt just the *sense* of the first chapter, which is only about tool > usage, to *nix, since it's written for Windows. I'd not dive into 'help' > for the someone who doesn't know anything because it gets technical > pretty fast, and because she will get back to that on her own when it's > time. > > Whatever you do, and whatever the background of the newbie, do introduce > turtle graphics right away. > > The ch 2 of the above reference contains some t.g. examples that you > might use (initial silly figures, graphs of functions, recursive > figures). It doesn't go into the turtle module objects. But if objects > are what you want to show right away, then I think the turtle module is > great also for that, because those objects are simple and can be easily > explored. > > > Cheers & hth., > > - Alf From rpjday at crashcourse.ca Wed Jan 20 09:51:07 2010 From: rpjday at crashcourse.ca (Robert P. J. Day) Date: Wed, 20 Jan 2010 09:51:07 -0500 (EST) Subject: how to check if a module is importable? In-Reply-To: <4b57155b@dnews.tpgi.com.au> References: <4b57155b@dnews.tpgi.com.au> Message-ID: On Thu, 21 Jan 2010, Lie Ryan wrote: > On 01/20/10 19:58, Robert P. J. Day wrote: > > > > finally getting back to clawing my way thru the python 3 book so > > probably a number of newbie questions coming up. first one -- can i > > check if a module is importable (equivalently, exists on sys.path, i > > assume) without trying to import it first? > > > > i can see that i can use try/except and just run "import" -- is that > > the accepted way to test? > > check imp.find_module() thanks, that's what i was looking for. i'm sure i'll run across that shortly as i keep working my way thru this python 3 book. rday -- ======================================================================== Robert P. J. Day Waterloo, Ontario, CANADA Linux Consulting, Training and Kernel Pedantry. Web page: http://crashcourse.ca Twitter: http://twitter.com/rpjday ======================================================================== From rpjday at crashcourse.ca Wed Jan 20 10:02:42 2010 From: rpjday at crashcourse.ca (Robert P. J. Day) Date: Wed, 20 Jan 2010 10:02:42 -0500 (EST) Subject: examining an initial, pristine python3 shell session In-Reply-To: References: Message-ID: On Wed, 20 Jan 2010, Alf P. Steinbach wrote: > * Robert P. J. Day: ... snip ... > > what other useful commands might i run immediately after > > starting a session whose output would be informative? i can > > certainly poke at some of those objects to see them in more > > detail. i'm just curious what others might recommend. thanks. > > That depends on what you mean by "newbie". > > If it's someone who knows a little bit of programming but is new to > Python, then 'help' would definitely be about the first thing I'd > show her. > > But if it's someone who doesn't even know anything about > programming, then I'd recommend (blatant plug) http://tinyurl.com/programmingbookP3> -- its first two chapters are > constructed around complete, concrete examples. However, you would > have to adapt just the *sense* of the first chapter, which is only > about tool usage, to *nix, since it's written for Windows. I'd not > dive into 'help' for the someone who doesn't know anything because > it gets technical pretty fast, and because she will get back to that > on her own when it's time. > > Whatever you do, and whatever the background of the newbie, do > introduce turtle graphics right away. ah, thank you, i appreciate that reference. i'm expecting the small audience to be relatively tech-savvy with OO dev experience, and i'm betting that some of them will be wondering right off the very first thing *i* was wondering -- when i start that python3 shell and get dumped into it, what am i looking at? which is why i wanted to collect a few commands to give the attendees at least a vague idea of what was already there. i've collected the following to start with: >>> dir() ['__builtins__', '__doc__', '__name__', '__package__'] >>> globals() ... stuff ... then move on to examine *those* things: >>> type(dir) >>> type(globals) >>> type(__builtins__) zoom in a bit further and pick on specific examples: >>> dir(__builtins__) ... lots of output ... and so on. as i said, i know it looks dry but i figure i can take 5 minutes or so just to lay out the terrain and what a shell session looks like before you do *anything* with it. and i'm betting most of my audience will appreciate getting that high-level view before launching into some programming. they'll just want to know the initial session setup before they start importing stuff into it. rday -- ======================================================================== Robert P. J. Day Waterloo, Ontario, CANADA Linux Consulting, Training and Kernel Pedantry. Web page: http://crashcourse.ca Twitter: http://twitter.com/rpjday ======================================================================== From lie.1296 at gmail.com Wed Jan 20 10:05:44 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Thu, 21 Jan 2010 02:05:44 +1100 Subject: examining an initial, pristine python3 shell session In-Reply-To: References: Message-ID: <4b571bde$1@dnews.tpgi.com.au> On 01/20/10 22:59, Robert P. J. Day wrote: > then i might go a bit further to examine some of *those* objects. i > admit it might seem a bit dry, but i think it would be handy to have a > handle on what a clean shell session looks like before starting to > import things, then seeing how that importing changes the session > before getting down to actual programming. I would recommend to teach these introspection capabilities *after* some basic programming. If you tell them these dir(), globals(), __name__, etc before even getting into basic python, your precious newbies would sit there thinking "what the hell is he talking about?" From victorsubervi at gmail.com Wed Jan 20 10:19:01 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 20 Jan 2010 11:19:01 -0400 Subject: Interesting Problem Message-ID: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> Hi; I think I finally have an interesting problem for y'all. I need to import a script from a lower dir, forcing me to change dirs: cwd = os.getcwd() os.chdir('%s/..' % cwd) sys.path.append(os.getcwd()) from templateFrame import top, bottom os.chdir(cwd) Because I've found I must do my form evaluations *before* changing dir as above, I am forced to call these values as globals: form = cgi.FieldStorage() store = form.getfirst('store') cat = form.getfirst('cat') id = form.getfirst('id') pkg = form.getfirst('pkg') patientID = form.getfirst('patientID') Now, apparently because of python's problem with globals, when I call "id" as follows: cursor.execute('select ProductID from productsPackages where PackageID=%s' % id) I get the following error: /var/www/html/angrynates.com/cart/Store_frame2.py 135 cursor.close() 136 bottom() 137 138 Store_frame2() 139 Store_frame2 = /var/www/html/angrynates.com/cart/Store_frame2.py in Store_frame2() 119 printAProduct() 120 else: 121 cursor.execute('select ProductID from productsPackages where PackageID=%s' % id) 122 for id in [itm[0] for itm in cursor]: 123 printAProduct(id) global cursor = , cursor.execute = >, global id = '1' UnboundLocalError: local variable 'id' referenced before assignment args = ("local variable 'id' referenced before assignment",) Full code below. Please advise. TIA, beno #! /usr/bin/python import string import cgitb; cgitb.enable() import cgi import MySQLdb import sys,os from sets import Set import fpformat cwd = os.getcwd() sys.path.append(cwd) from login import login from particulars import truncate form = cgi.FieldStorage() store = form.getfirst('store') cat = form.getfirst('cat') id = form.getfirst('id') pkg = form.getfirst('pkg') patientID = form.getfirst('patientID') try: browser = form.getfirst('browser', 'all') except: browser = headers() os.chdir('%s/..' % cwd) sys.path.append(os.getcwd()) from templateFrame import top, bottom os.chdir(cwd) ourFile = string.split(__file__, "/") p = ourFile[len(ourFile) - 1] p = p[: - 9] site = ourFile[4][:-10] if site != '': site = site[:-1] user, passwd, db, host = login() db = MySQLdb.connect(host, user, passwd, db) cursor= db.cursor() cursor.execute('describe %s;' % store) descr = cursor.fetchall() cols = [] for elt in descr: cols.append(elt[0]) def printAProduct(id=id): lastID = '' cursor.execute('select * from %s where ID="%s"' % (store, id)) vals = cursor.fetchone() prodDict = dict(zip(cols, vals)) for col, item in prodDict.iteritems(): if col == 'ID': print '' % item try: # Stores such as products (but not prescriptions) will pass through from particulars import ourOptions if col in ourOptions(): print '' % (col, item) except: pass if col[:3] != 'pic': notSet = 1 if isinstance(item, (str, int, long, float, long, complex, unicode, list, buffer, xrange, tuple)): print '%s: %s
\n' % (col, item) else: try: html = "%s:
" print html except: pass # if notSet == 1: # if len(col) > 49: # colX = col[:50] + '...' # else: # colX = col # print '%s: %s
\n' % (colX, item) elif col == 'pic1': try: if (item != None): # if (item != None) and (len(item > 0)): print '
\n' % (store, col[3:], id, store, col[3:], id, store, col[3:], id) except TypeError: raise except: raise # i += 1 # try: # content = item[x].tostring() # pic = "tmp" + str(i) + ".jpg" # try: # os.remove(pic) # except: # pass # img = open(pic, "w") # img.write(content) # print '

' % pic # img.close() # except: # pass def Store_frame2(): top(browser, p) i = 0 print '\n' print '
' print "" % store print "" if pkg == 'no': printAProduct() else: cursor.execute('select ProductID from productsPackages where PackageID=%s' % id) for id in [itm[0] for itm in cursor]: printAProduct(id) if store == 'prescriptions': print "" print "" elif truncate()[store] != '': pass else: print "Quantity:
" print "" print '
' print '\n' print '\n' cursor.close() bottom() Store_frame2() -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Wed Jan 20 10:24:51 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 20 Jan 2010 11:24:51 -0400 Subject: Something More Elegant In-Reply-To: References: <4dc0cfea1001090523y77dafa0et86fd63d0aa859d91@mail.gmail.com> <4B4886FF.2030503@tim.thechases.com> <4dc0cfea1001090601o6da46244wb556f1be9d1b9592@mail.gmail.com> <4dc0cfea1001090715md97518tf618309eda1deabb@mail.gmail.com> <4dc0cfea1001171006x743c233cy557e3930db3feba9@mail.gmail.com> Message-ID: <4dc0cfea1001200724i7415593dv7cdb3d2a1ac764c5@mail.gmail.com> On Sun, Jan 17, 2010 at 5:36 PM, Dennis Lee Bieber wrote: > I don't think that would be efficient, considering that "the above" > entails what would be something like three or four courses all by > themselves (Database Analysis, Database Design, SQL [these three are > independent of any particular implementation language, though Design and > SQL may be impacted by the database engine one is using -- for example, > foreign key integrity & cascading deletes are not enforced by MySQL > MyISAM tables, but can be enforced using InnoDB tables; SQLite doesn't > have user access levels, but all client/server engines do, etc.), and > then a course on Python with an emphasis on Web applications [since I > recall you are trying to generate HTML at some stage]) the only example > I'd be able to give would require me writing a complete application -- > the last time I supplied a large snippet of pseudo-code it took over a > month of daily questions and replies to fix your attempts to modify it; > most problems coming down to not understanding the algorithm, how to use > parameterized queries, etc. > LOL! Yeah... <:-} > > The shortest hints I can demonstrate -- using a text console for > input, AND in pseudo-code (I've not ensured this will run) would be: > > DON'T: > tname = raw_input("Enter the name of the table to be searched: ") > fname = raw_input("Enter the name of the field in %s to be searched: " > % tname) > fvalue = raw_input("Enter the value of %s defining the desired data: " > % fname) > > SQL = """select * from %s > where %s like "%%%s%%"""" % (tname, fname, fvalue) > crs.execute(SQL) > > > DO > tables = [ "Table1", > "Table2", > "Table3" ] > fields = { "Table1" : [ "t1field1", > "t1field2", > "t1field3" ], > "Table2" : [ "t2field1", > "t2field2", > "t2field3" ], > "Table3" : [ "t3field1", > "t3field2", > "t3field3" ] > } > > for i, t in enumerate(tables): > print "%s\t%s" % (i, t) > tnum = int(raw_input( > "Enter the number of the table to be searched: ")) > tname = tables[tnum] > > for i, f in enumerate(fields[tname]): > print "%s\t%s" % (i, f) > fnum = int(raw_input( > "Enter the name of the field in %s to be searched: " > % tname) > fname = fields[tname][fnum] > > fvalue = raw_input("Enter the value of %s defining the desired data: " > % fname) > > SQL = """select * from %s > where %s like %%s""" % (tname, fname) > crs.execute(SQL, ("%"+fvalue+"%",)) > > In a real application, one would use the ability of the database > engine itself to retrieve the list of table names, and the fields for > each table -- that way the code would not need to be changed if the > database has additions or deletions of tables/fields. > > Also note that I did NOT include exception handling -- all those > raw_input() in the "DO" version should have some exception handling (the > first two might not be given numeric data so the int() fails, or the > integer supplied may not be in the range of indices for the list > look-ups; all three might get an EOF entry by the user trying to abort > the look-up. > > Fancier versions would build multi-table joins, or allow for AND (or > OR) clauses in the WHERE, using a list of table.field/value pairs. > Thanks. I think I followed that :/ beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Wed Jan 20 10:26:27 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 20 Jan 2010 11:26:27 -0400 Subject: Changing var names In-Reply-To: <4B546BDD.1030602@sequans.com> References: <4dc0cfea1001150927i48b6c386w53f4487cd894d078@mail.gmail.com> <1263582938.3142.12.camel@linux-m3mt> <4dc0cfea1001151203w2d1c9520t692d2367a1882ee3@mail.gmail.com> <4B546BDD.1030602@sequans.com> Message-ID: <4dc0cfea1001200726g57a828e7vfca360357ec75cc6@mail.gmail.com> On Mon, Jan 18, 2010 at 10:10 AM, Jean-Michel Pichavant < jeanmichel at sequans.com> wrote: > Victor Subervi wrote: > >> On Fri, Jan 15, 2010 at 3:15 PM, Adam Tauno Williams < >> awilliam at opengroupware.us > wrote: >> >> On Fri, 2010-01-15 at 13:27 -0400, Victor Subervi wrote: >> > Hi; >> > Well it took me *less than a day* to fix the following problems: >> > -- bare excepts (accidentally left a couple I think) >> > -- sql injection attacks >> > -- recreating tables to make them more reasonable >> > Now, I believe someone once mentioned that there is some s/w >> tool out >> > there for changing the names of my variables more easily than going >> > through every script one-by-one. If so, please share with me again >> > what it is. >> >> This process is called 'refactoring' [a good term to Google], and >> every >> decent IDE provides some support [if it doesn't, it isn't a "decent" >> IDE] >> >> >> Thanks. I'll work with Eric, I guess, once I get my Mac. >> beno >> > Here are 2 well known IDEs, that help with refactoring (and many other > things). > http://netbeans.org/ > http://www.eclipse.org/ > > Both supports python. > Thanks. > > I personnaly will stick to vim :o) > I'll keep that in mind. I haven't used any WYSIWIGS to date. Thanks, beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From keith at nekotaku.com Wed Jan 20 10:32:36 2010 From: keith at nekotaku.com (KB) Date: Wed, 20 Jan 2010 07:32:36 -0800 (PST) Subject: Using jython to call python procedures/methods Message-ID: Hi there, I have an application that only publishes a Java API. I can use jython to access java classes, but jython currently (to the best of my knowledge) does not support numpy/scipy. Ideally I would like to have jython call a "native" python routine where I have the numpy/scipy procedures already written. Does anyone have any experience with this? Is it possible? I had toyed with the idea of having jython/java write the data to a file/database and then manually kick off the python process, but ideally I would like this as automated as possible. Thanks in advance. From robert.kern at gmail.com Wed Jan 20 11:06:26 2010 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 20 Jan 2010 10:06:26 -0600 Subject: turtle, ipython, and pylab don't mix In-Reply-To: References: Message-ID: On 2010-01-20 08:40 AM, Brian Blais wrote: > Hello, > > I am noticing a hard crash of the ipython interpreter, with the pylab > option, with the turtle module. I am on Win XP, Enthought Edition 6.0.0 > which uses Python 2.6. I can't reproduce it without the pylab option, > nor on the Mac (with an earlier Enthought Edition). The crash happens > only when I try to move the turtle window, after it has drawn. The > following commands are enough to give the resulting error. > > In [1]: from turtle import * > > In [2]: pendown() > > In [3]: forward(10) > > In [4]: Fatal Python error: PyEval_RestoreThread: NULL tstate > > This application has requested the Runtime to terminate it in an unusual > way. > Please contact the application's support team for more information. > > > Any ideas? Of course, I'll use it without the pylab option, but that > means that there are two icons I need instead of one. Minor > inconvenience for me, but a bit more of a pain for my students. The IPython list is probably a better place for this question. Is suspect that the problem is the mixing of GUIs. turtle uses Tk, but I suspect that your configured matplotlib backend is WxAgg. "ipython -pylab" will bring up a wx application as the main thread for execution of the code you type. When you try to bring in another GUI event loop, something goes awry. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From kwmsmith at gmail.com Wed Jan 20 11:08:41 2010 From: kwmsmith at gmail.com (Kurt Smith) Date: Wed, 20 Jan 2010 10:08:41 -0600 Subject: Using jython to call python procedures/methods In-Reply-To: References: Message-ID: On Wed, Jan 20, 2010 at 9:32 AM, KB wrote: > Hi there, > > I have an application that only publishes a Java API. I can use jython > to access java classes, but jython currently (to the best of my > knowledge) does not support numpy/scipy. > > Ideally I would like to have jython call a "native" python routine > where I have the numpy/scipy procedures already written. > > Does anyone have any experience with this? Is it possible? I have no experience with these technologies, and others can point you to more detailed info, but you could try using XML-RPC (see the docs for the xmlrpclib module in the standard library) or SOAP (Google it). These would be better than rolling your own. Perhaps there's a more Pythonic solution though? Kurt > > I had toyed with the idea of having jython/java write the data to a > file/database and then manually kick off the python process, but > ideally I would like this as automated as possible. > > Thanks in advance. > -- > http://mail.python.org/mailman/listinfo/python-list > From doxalogos at gmail.com Wed Jan 20 11:10:34 2010 From: doxalogos at gmail.com (DoxaLogos) Date: Wed, 20 Jan 2010 08:10:34 -0800 (PST) Subject: multiprocessing problems References: <74594510-9366-4227-8e1e-d221a9220271@k35g2000yqb.googlegroups.com> <63c5df56-7b86-4ee7-ba4f-b14984f94097@e16g2000yqc.googlegroups.com> Message-ID: <63ef9fe9-6e9b-4766-9a0c-4aebc28d4148@p24g2000yqm.googlegroups.com> On Jan 19, 10:33?am, DoxaLogos wrote: > On Jan 19, 10:26?am, Adam Tauno Williams > wrote: > > > > > > I decided to play around with the multiprocessing module, and I'm > > > having some strange side effects that I can't explain. ?It makes me > > > wonder if I'm just overlooking something obvious or not. ?Basically, I > > > have a script parses through a lot of files doing search and replace > > > on key strings inside the file. ?I decided the split the work up on > > > multiple processes on each processor core (4 total). ?I've tried many > > > various ways doing this form using pool to calling out separate > > > processes, but the result has been the same: computer crashes from > > > endless process spawn. > > > Are you hitting a ulimit error? ?The number of processes you can create > > is probably limited. > > > TIP: close os.stdin on your subprocesses. > > > > Here's the guts of my latest incarnation. > > > def ProcessBatch(files): > > > ? ? p = [] > > > ? ? for file in files: > > > ? ? ? ? p.append(Process(target=ProcessFile,args=file)) > > > ? ? for x in p: > > > ? ? ? ? x.start() > > > ? ? for x in p: > > > ? ? ? ? x.join() > > > ? ? p = [] > > > ? ? return > > > Now, the function calling ProcessBatch looks like this: > > > def ReplaceIt(files): > > > ? ? processFiles = [] > > > ? ? for replacefile in files: > > > ? ? ? ? if(CheckSkipFile(replacefile)): > > > ? ? ? ? ? ? processFiles.append(replacefile) > > > ? ? ? ? ? ? if(len(processFiles) == 4): > > > ? ? ? ? ? ? ? ? ProcessBatch(processFiles) > > > ? ? ? ? ? ? ? ? processFiles = [] > > > ? ? #check for left over files once main loop is done and process them > > > ? ? if(len(processFiles) > 0): > > > ? ? ? ? ProcessBatch(processFiles) > > > According to this you will create files is sets of four, but an unknown > > number of sets of four. > > What would be the proper way to only do a set of 4, stop, then do > another set of 4? ?I'm trying to only 4 files at time before doing > another set of 4. I found out my problems. One thing I did was followed the test queue example in the documentation, but the biggest problem turned out to be a pool instantiated globally in my script was causing most of the endless process spawn, even with the "if __name__ == "__main__":" block. From keith at nekotaku.com Wed Jan 20 11:24:38 2010 From: keith at nekotaku.com (KB) Date: Wed, 20 Jan 2010 08:24:38 -0800 (PST) Subject: Using jython to call python procedures/methods References: Message-ID: <1ab8341e-9d72-43d7-acb3-1a0559d3381a@c34g2000yqn.googlegroups.com> Hmmm, XML is an interesting angle... I'll noodle it... From apt.shansen at gmail.com Wed Jan 20 11:30:07 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Wed, 20 Jan 2010 08:30:07 -0800 Subject: Interesting Problem In-Reply-To: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> References: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> Message-ID: <7a9c25c21001200830k5d40d1dcv6813933b96d13cbb@mail.gmail.com> On Wed, Jan 20, 2010 at 7:19 AM, Victor Subervi wrote: > Hi; > I think I finally have an interesting problem for y'all. I need to import a > script from a lower dir, forcing me to change dirs: > Don't do that. If you must, then the correct way to do it is to adjust your sys.path and not change directory. "sys.path.append('..')" or whatever. But I really would just re-organize your code so you don't need such things. Have a single top-level directory that's on your path, put a blank __init__.py in all your directories, and use absolute imports everywhere. from myapp.templateFrame import top, bottom from myapp.some_directory.some_file import this, that etc. > Now, apparently because of python's problem with globals, when I call "id" > as follows: > > Python does not have problems with globals. You are repeatedly re-using the same variable names, causing confusion and errors; when you say 'id' in that line of code, Python doesn't think you are talking about the "id" that is global. It thinks you are talking about the "id" that is local-- that you define one line beneath. Local has precedence over global. Don't shadow global variables (and 'id' is a bad name, as it shadows a builtin variable) --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From gherron at islandtraining.com Wed Jan 20 11:37:55 2010 From: gherron at islandtraining.com (Gary Herron) Date: Wed, 20 Jan 2010 08:37:55 -0800 Subject: Interesting Problem In-Reply-To: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> References: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> Message-ID: <4B573163.4010607@islandtraining.com> Victor Subervi wrote: > Hi; > I think I finally have an interesting problem for y'all. I need to > import a script from a lower dir, forcing me to change dirs: Wait a moment... Your assumption that you need to change directories is invalid. (And that means everything you do below this point unnecessary.) Python provides many ways to easily import from arbitrary directories: * Before run time, modify the environment variable PYTHONPATH with the desired directory. Then import as normal. * At run time, import sys and append the desired path to sys.path, and again import as normal. * Check out the "imp" module which allows you to import from files specified via a path. Straighten that out, then we can examine the following (suspicious) clams: * that you must evaluate *before* some import * that you must use globals * python's problem with globals Gary Herron > > cwd = os.getcwd() > os.chdir('%s/..' % cwd) > sys.path.append(os.getcwd()) > from templateFrame import top, bottom > os.chdir(cwd) > > Because I've found I must do my form evaluations *before* changing dir > as above, I am forced to call these values as globals: > > form = cgi.FieldStorage() > store = form.getfirst('store') > cat = form.getfirst('cat') > id = form.getfirst('id') > pkg = form.getfirst('pkg') > patientID = form.getfirst('patientID') > > Now, apparently because of python's problem with globals, when I call > "id" as follows: > > cursor.execute('select ProductID from productsPackages where > PackageID=%s' % id) > > I get the following error: > > /var/www/html/angrynates.com/cart/Store_frame2.py > > 135 cursor.close() > 136 bottom() > 137 > 138 Store_frame2() > 139 > Store_frame2 = > /var/www/html/angrynates.com/cart/Store_frame2.py > in Store_frame2() > 119 printAProduct() > 120 else: > 121 cursor.execute('select ProductID from productsPackages where > PackageID=%s' % id) > 122 for id in [itm[0] for itm in cursor]: > 123 printAProduct(id) > global cursor = , cursor.execute = > >, > global id = '1' > > UnboundLocalError: local variable 'id' referenced before assignment > args = ("local variable 'id' referenced before assignment",) > > Full code below. Please advise. > TIA, > beno > > #! /usr/bin/python > > import string > import cgitb; cgitb.enable() > import cgi > import MySQLdb > import sys,os > from sets import Set > import fpformat > cwd = os.getcwd() > sys.path.append(cwd) > from login import login > from particulars import truncate > > form = cgi.FieldStorage() > store = form.getfirst('store') > cat = form.getfirst('cat') > id = form.getfirst('id') > pkg = form.getfirst('pkg') > patientID = form.getfirst('patientID') > > try: > browser = form.getfirst('browser', 'all') > except: > browser = headers() > > os.chdir('%s/..' % cwd) > sys.path.append(os.getcwd()) > from templateFrame import top, bottom > os.chdir(cwd) > > ourFile = string.split(__file__, "/") > p = ourFile[len(ourFile) - 1] > p = p[: - 9] > site = ourFile[4][:-10] > if site != '': > site = site[:-1] > > user, passwd, db, host = login() > db = MySQLdb.connect(host, user, passwd, db) > cursor= db.cursor() > > cursor.execute('describe %s;' % store) > descr = cursor.fetchall() > cols = [] > for elt in descr: > cols.append(elt[0]) > > def printAProduct(id=id): > lastID = '' > cursor.execute('select * from %s where ID="%s"' % (store, id)) > vals = cursor.fetchone() > prodDict = dict(zip(cols, vals)) > for col, item in prodDict.iteritems(): > if col == 'ID': > print '' % item > try: # Stores such as products (but not prescriptions) will pass > through > from particulars import ourOptions > if col in ourOptions(): > print '' % (col, item) > except: > pass > if col[:3] != 'pic': > notSet = 1 > if isinstance(item, (str, int, long, float, long, complex, > unicode, list, buffer, xrange, tuple)): > print '%s: %s
\n' % (col, item) > else: > try: > html = "%s:
" > print html > except: > pass > # if notSet == 1: > # if len(col) > 49: > # colX = col[:50] + '...' > # else: > # colX = col > # print '%s: %s
\n' % (colX, item) > elif col == 'pic1': > try: > if (item != None): > # if (item != None) and (len(item > 0)): > print ' class="highslide" href="getpic.py?store=%s&pic=%s&id=%s" > onclick="return hs.expand(this)"> src="getpic.py?store=%s&pic=%s&id=%s" width="100" height="80" alt="" > align="left" border="0" title="Click to enlarge" style="border: > 0px">
\n' % (store, col[3:], id, store, col[3:], > id, store, col[3:], id) > except TypeError: > raise > except: > raise > # i += 1 > # try: > # content = item[x].tostring() > # pic = "tmp" + str(i) + ".jpg" > # try: > # os.remove(pic) > # except: > # pass > # img = open(pic, "w") > # img.write(content) > # print '

' % pic > # img.close() > # except: > # pass > > def Store_frame2(): > top(browser, p) > i = 0 > print '\n' > print '
' > print "" % store > print "" > if pkg == 'no': > printAProduct() > else: > cursor.execute('select ProductID from productsPackages where > PackageID=%s' % id) > for id in [itm[0] for itm in cursor]: > printAProduct(id) > if store == 'prescriptions': > print "" > print "" > elif truncate()[store] != '': > pass > else: > print "Quantity: width='3' maxlength='3' />
" > print "" > print '
' > print '\n' > print '\n' > cursor.close() > bottom() > > Store_frame2() > > > -- > The Logos has come to bear > http://logos.13gems.com/ From python at mrabarnett.plus.com Wed Jan 20 11:38:07 2010 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 20 Jan 2010 16:38:07 +0000 Subject: Interesting Problem In-Reply-To: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> References: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> Message-ID: <4B57316F.4030009@mrabarnett.plus.com> Victor Subervi wrote: > Hi; > I think I finally have an interesting problem for y'all. I need to > import a script from a lower dir, forcing me to change dirs: > > cwd = os.getcwd() > os.chdir('%s/..' % cwd) > sys.path.append(os.getcwd()) > from templateFrame import top, bottom > os.chdir(cwd) > There's no need to actually change current directory. Just do this: sys.path.append(os.path.dirname(os.getcwd())) from templateFrame import top, bottom > Because I've found I must do my form evaluations *before* changing dir > as above, I am forced to call these values as globals: > > form = cgi.FieldStorage() > store = form.getfirst('store') > cat = form.getfirst('cat') > id = form.getfirst('id') > pkg = form.getfirst('pkg') > patientID = form.getfirst('patientID') > > Now, apparently because of python's problem with globals, when I call > "id" as follows: > > cursor.execute('select ProductID from productsPackages where > PackageID=%s' % id) > > I get the following error: > > /var/www/html/angrynates.com/cart/Store_frame2.py > > 135 cursor.close() > 136 bottom() > 137 > 138 Store_frame2() > 139 > Store_frame2 = > /var/www/html/angrynates.com/cart/Store_frame2.py > in Store_frame2() > 119 printAProduct() > 120 else: > 121 cursor.execute('select ProductID from productsPackages where > PackageID=%s' % id) > 122 for id in [itm[0] for itm in cursor]: > 123 printAProduct(id) > global cursor = , cursor.execute = method Cursor.execute of >, global id = '1' > > UnboundLocalError: local variable 'id' referenced before assignment > args = ("local variable 'id' referenced before assignment",) > > Full code below. Please advise. > Line 121 refers to 'id', but line 122 assigns to 'id' (the 'for' loop). In a function, if you assign to a variable then that variable defaults to being local. You're trying to use the local variable before assigning a value to it. Having a local variable called 'id' means that you can't refer to the global variable of the same name. You're also assigning to 'i' on line 113, but not referring to it elsewhere in the function. From darkneter at gmail.com Wed Jan 20 11:43:38 2010 From: darkneter at gmail.com (NighterNet) Date: Wed, 20 Jan 2010 08:43:38 -0800 (PST) Subject: Object Integer mapping Message-ID: <60e22629-c7a9-4d8f-930f-a3ed777b5f42@p24g2000yqm.googlegroups.com> Need help on python version 3.1.x. I can't seem to know what going on a bit. The code that I check that the hash is different every time. Is there a way to make the hash the same? I using as to check the class is the same variables but if it different variable in the class that it add to the array and return the index. # Generic Object->Integer mapping # the object must be usable as a dictionary key class ObjMap: def __init__(self): self.dict = {} self.next = 0 def get(self, obj): if obj in self.dict: return self.dict[obj] else: id = self.next self.next = self.next + 1 self.dict[obj] = id return id def items(self): getval = operator.itemgetter(0) getkey = operator.itemgetter(1) return map(getval, sorted(self.dict.items(), key=getkey)) class Point: def __init__(self): self.x = 0 self.y = 0 self.z = 0 points = ObjMap() Testme = Point() Testme.x = 0 print(points.get(Testme)) Testme2 = Point() Testme2.y = 1 print(points.get(Testme2)) Testme3 = Point() Testme3.y = 1 print(points.get(Testme3)) Ttestme4 = Point() Ttestme4.y = 1 print( points.get(Ttestme4)) It keep adding new array from this but doesn't match hash. Need help on how to fix this class code. From nytrokiss at gmail.com Wed Jan 20 11:49:19 2010 From: nytrokiss at gmail.com (James Matthews) Date: Wed, 20 Jan 2010 08:49:19 -0800 Subject: Interesting Problem In-Reply-To: <4B57316F.4030009@mrabarnett.plus.com> References: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> <4B57316F.4030009@mrabarnett.plus.com> Message-ID: <8a6b8e351001200849v6baf2597y3adb98d0d2d43636@mail.gmail.com> Also as a side point... It's best to anonymize the code as best you can so people cannot attach your code to your site (IP theft and security risk) On Wed, Jan 20, 2010 at 8:38 AM, MRAB wrote: > Victor Subervi wrote: > >> Hi; >> I think I finally have an interesting problem for y'all. I need to import >> a script from a lower dir, forcing me to change dirs: >> >> cwd = os.getcwd() >> os.chdir('%s/..' % cwd) >> sys.path.append(os.getcwd()) >> from templateFrame import top, bottom >> os.chdir(cwd) >> >> There's no need to actually change current directory. Just do this: > > sys.path.append(os.path.dirname(os.getcwd())) > > from templateFrame import top, bottom > > Because I've found I must do my form evaluations *before* changing dir as >> above, I am forced to call these values as globals: >> >> form = cgi.FieldStorage() >> store = form.getfirst('store') >> cat = form.getfirst('cat') >> id = form.getfirst('id') >> pkg = form.getfirst('pkg') >> patientID = form.getfirst('patientID') >> >> Now, apparently because of python's problem with globals, when I call "id" >> as follows: >> >> cursor.execute('select ProductID from productsPackages where PackageID=%s' >> % id) >> >> I get the following error: >> >> /var/www/html/angrynates.com/cart/Store_frame2.py < >> http://angrynates.com/cart/Store_frame2.py> >> >> 135 cursor.close() >> 136 bottom() >> 137 >> 138 Store_frame2() >> 139 >> Store_frame2 = >> /var/www/html/angrynates.com/cart/Store_frame2.py < >> http://angrynates.com/cart/Store_frame2.py> in Store_frame2() >> >> 119 printAProduct() >> 120 else: >> 121 cursor.execute('select ProductID from productsPackages where >> PackageID=%s' % id) >> 122 for id in [itm[0] for itm in cursor]: >> 123 printAProduct(id) >> global cursor = , cursor.execute = > method Cursor.execute of >, global id = '1' >> >> UnboundLocalError: local variable 'id' referenced before assignment >> args = ("local variable 'id' referenced before assignment",) >> >> Full code below. Please advise. >> >> Line 121 refers to 'id', but line 122 assigns to 'id' (the 'for' loop). > > In a function, if you assign to a variable then that variable defaults > to being local. You're trying to use the local variable before assigning > a value to it. Having a local variable called 'id' means that you can't > refer to the global variable of the same name. > > You're also assigning to 'i' on line 113, but not referring to it > elsewhere in the function. > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://www.astorandblack.com -- -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorsubervi at gmail.com Wed Jan 20 12:06:05 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Wed, 20 Jan 2010 13:06:05 -0400 Subject: Interesting Problem In-Reply-To: <7a9c25c21001200830k5d40d1dcv6813933b96d13cbb@mail.gmail.com> References: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> <7a9c25c21001200830k5d40d1dcv6813933b96d13cbb@mail.gmail.com> Message-ID: <4dc0cfea1001200906h22843e0amf5b98e2ea7ad18bb@mail.gmail.com> On Wed, Jan 20, 2010 at 12:30 PM, Stephen Hansen wrote: > On Wed, Jan 20, 2010 at 7:19 AM, Victor Subervi wrote: > >> Hi; >> I think I finally have an interesting problem for y'all. I need to import >> a script from a lower dir, forcing me to change dirs: >> > > Don't do that. If you must, then the correct way to do it is to adjust your > sys.path and not change directory. "sys.path.append('..')" or whatever. > > But I really would just re-organize your code so you don't need such > things. Have a single top-level directory that's on your path, put a blank > __init__.py in all your directories, and use absolute imports everywhere. > > from myapp.templateFrame import top, bottom > from myapp.some_directory.some_file import this, that > > etc. > I didn't know I could do that. Thanks! > > > >> Now, apparently because of python's problem with globals, when I call "id" >> as follows: >> >> > Python does not have problems with globals. You are repeatedly re-using the > same variable names, causing confusion and errors; when you say 'id' in that > line of code, Python doesn't think you are talking about the "id" that is > global. It thinks you are talking about the "id" that is local-- that you > define one line beneath. > I didn't think that was a problem because it cascaded. Thanks, that fixed it! beno -------------- next part -------------- An HTML attachment was scrubbed... URL: From chardster at gmail.com Wed Jan 20 12:18:27 2010 From: chardster at gmail.com (Richard Thomas) Date: Wed, 20 Jan 2010 09:18:27 -0800 (PST) Subject: Object Integer mapping References: <60e22629-c7a9-4d8f-930f-a3ed777b5f42@p24g2000yqm.googlegroups.com> Message-ID: On Jan 20, 4:43?pm, NighterNet wrote: > Need help on python version 3.1.x. I can't seem to know what going on > a bit. The code that I check that the hash is different every time. Is > there a way to make the hash the same? I using as to check the class > is the same variables but if it different variable in the class that > it add to the array and return the index. > > # Generic Object->Integer mapping > # the object must be usable as a dictionary key > class ObjMap: > ? ? ? ? def __init__(self): > ? ? ? ? ? ? ? ? self.dict = {} > ? ? ? ? ? ? ? ? self.next = 0 > ? ? ? ? def get(self, obj): > ? ? ? ? ? ? ? ? if obj in self.dict: > ? ? ? ? ? ? ? ? ? ? ? ? return self.dict[obj] > ? ? ? ? ? ? ? ? else: > ? ? ? ? ? ? ? ? ? ? ? ? id = self.next > ? ? ? ? ? ? ? ? ? ? ? ? self.next = self.next + 1 > ? ? ? ? ? ? ? ? ? ? ? ? self.dict[obj] = id > ? ? ? ? ? ? ? ? ? ? ? ? return id > > ? ? ? ? def items(self): > ? ? ? ? ? ? ? ? getval = operator.itemgetter(0) > ? ? ? ? ? ? ? ? getkey = operator.itemgetter(1) > ? ? ? ? ? ? ? ? return map(getval, sorted(self.dict.items(), key=getkey)) > > class Point: > ? ? ? ? def __init__(self): > ? ? ? ? ? ? ? ? self.x = 0 > ? ? ? ? ? ? ? ? self.y = 0 > ? ? ? ? ? ? ? ? self.z = 0 > > points = ObjMap() > > Testme = Point() > Testme.x = 0 > print(points.get(Testme)) > Testme2 = Point() > Testme2.y = 1 > print(points.get(Testme2)) > Testme3 = Point() > Testme3.y = 1 > print(points.get(Testme3)) > Ttestme4 = Point() > Ttestme4.y = 1 > print( points.get(Ttestme4)) > > It keep adding new array from this but doesn't match hash. Need help > on how to fix this class code. You need to define how the Point class hashes. You can do this by writing a __hash__ method like so: class Point(object): ... def __hash__(self): return hash(self.x) ^ hash(self.y) ^ hash(self.z) However you will also need to define how the Point class relates as equal using the __eq__ or __cmp__ methods. class Point(object): ... def __eq__(self, other): return self.x == other.x and self.y == other.y and self.z == other.z This addition makes sure that if two points a equivalent then they count as the same key in the dictionary. Richard. From robert.kern at gmail.com Wed Jan 20 12:43:20 2010 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 20 Jan 2010 11:43:20 -0600 Subject: Object Integer mapping In-Reply-To: References: <60e22629-c7a9-4d8f-930f-a3ed777b5f42@p24g2000yqm.googlegroups.com> Message-ID: On 2010-01-20 11:18 AM, Richard Thomas wrote: > On Jan 20, 4:43 pm, NighterNet wrote: >> Need help on python version 3.1.x. I can't seem to know what going on >> a bit. The code that I check that the hash is different every time. Is >> there a way to make the hash the same? I using as to check the class >> is the same variables but if it different variable in the class that >> it add to the array and return the index. >> >> # Generic Object->Integer mapping >> # the object must be usable as a dictionary key >> class ObjMap: >> def __init__(self): >> self.dict = {} >> self.next = 0 >> def get(self, obj): >> if obj in self.dict: >> return self.dict[obj] >> else: >> id = self.next >> self.next = self.next + 1 >> self.dict[obj] = id >> return id >> >> def items(self): >> getval = operator.itemgetter(0) >> getkey = operator.itemgetter(1) >> return map(getval, sorted(self.dict.items(), key=getkey)) >> >> class Point: >> def __init__(self): >> self.x = 0 >> self.y = 0 >> self.z = 0 >> >> points = ObjMap() >> >> Testme = Point() >> Testme.x = 0 >> print(points.get(Testme)) >> Testme2 = Point() >> Testme2.y = 1 >> print(points.get(Testme2)) >> Testme3 = Point() >> Testme3.y = 1 >> print(points.get(Testme3)) >> Ttestme4 = Point() >> Ttestme4.y = 1 >> print( points.get(Ttestme4)) >> >> It keep adding new array from this but doesn't match hash. Need help >> on how to fix this class code. > > You need to define how the Point class hashes. You can do this by > writing a __hash__ method like so: > > class Point(object): > ... > def __hash__(self): > return hash(self.x) ^ hash(self.y) ^ hash(self.z) > > However you will also need to define how the Point class relates as > equal using the __eq__ or __cmp__ methods. > > class Point(object): > ... > def __eq__(self, other): > return self.x == other.x and self.y == other.y and self.z == > other.z > > This addition makes sure that if two points a equivalent then they > count as the same key in the dictionary. I recommend a simpler approach that duplicates less code and makes it easier to maintain the __hash__/__eq__ invariants: class Point(object): ... def _key(self): # The type(self).__name__ bit is optional, but usually handy. return (type(self).__name__, self.x, self.y, self.z) def __hash__(self): return hash(self._key()) def __eq__(self, other): if not hasattr(other, '_key'): return False return self._key() == other._key() It is also worth noting that once one allows hashing by value, one should treat the objects as immutable, so one should not change the attributes as the OP does in his example. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From python at mrabarnett.plus.com Wed Jan 20 13:27:29 2010 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 20 Jan 2010 18:27:29 +0000 Subject: Interesting Problem In-Reply-To: <8a6b8e351001200849v6baf2597y3adb98d0d2d43636@mail.gmail.com> References: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> <4B57316F.4030009@mrabarnett.plus.com> <8a6b8e351001200849v6baf2597y3adb98d0d2d43636@mail.gmail.com> Message-ID: <4B574B11.80308@mrabarnett.plus.com> James Matthews wrote: > Also as a side point... It's best to anonymize the code as best you can > so people cannot attach your code to your site (IP theft and security risk) > [snip] Security risk, yes; IP theft, not so much. ;-) From alfps at start.no Wed Jan 20 13:27:34 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 20 Jan 2010 19:27:34 +0100 Subject: Best way to convert sequence of bytes to long integer In-Reply-To: <9b870cc2-0966-4cb2-be93-573317c06eb0@a6g2000yqm.googlegroups.com> References: <00e0377c$0$15570$c3e8da3@news.astraweb.com> <9b870cc2-0966-4cb2-be93-573317c06eb0@a6g2000yqm.googlegroups.com> Message-ID: * Mark Dickinson: > On Jan 20, 7:36 am, Steven D'Aprano cybersource.com.au> wrote: >> I have a byte string (Python 2.x string), e.g.: >> >> s = "g%$f yg\n1\05" >> assert len(s) == 10 >> >> I wish to convert it to a long integer, treating it as base-256. > > By the way, are you willing to divulge what you're using this > functionality for? When trying to hack out the API for int.to_bytes > and int.from_bytes on the mailing list and bug tracker, some of the > discussion about use-cases seemed a little too much like guesswork; > I'd be curious to find out about real-life use-cases. One possibility is that Steven wants to apply bitlevel and/or arithmetic operations for e.g. some custom cryptographic thing. The string that he uses as example is not completely unlike a message digest. Another possibility is custom serialization/deserialization. But it would be nice to know what it's actually about... Cheers, - Alf From ng1.ben at gmail.com Wed Jan 20 13:38:18 2010 From: ng1.ben at gmail.com (ben) Date: Wed, 20 Jan 2010 10:38:18 -0800 (PST) Subject: * operator in python tutorial Message-ID: <7b2ef0aa-8b93-4cb5-b944-beea3e2c98b1@p16g2000vbe.googlegroups.com> Hello, I am following through the python tutorial which gets to a line that uses the * operator with zip(). I searched and searched but could find no information on the operator or how to use it in general. The example from the tut is as follows: >>> x = [1, 2, 3] >>> y = [4, 5, 6] >>> zipped = zip(x, y) >>> zipped [(1, 4), (2, 5), (3, 6)] >>> x2, y2 = zip(*zipped) >>> x == list(x2) and y == list(y2) True How would i apply the * operator in general? Thanks. From ng1.ben at gmail.com Wed Jan 20 13:48:06 2010 From: ng1.ben at gmail.com (ben) Date: Wed, 20 Jan 2010 10:48:06 -0800 (PST) Subject: * operator in python tutorial References: <7b2ef0aa-8b93-4cb5-b944-beea3e2c98b1@p16g2000vbe.googlegroups.com> Message-ID: I missed it because I skipped to the explanation of zip. I hit the back arrow and went on and saw the link to "Unpacking Argument Lists" which explained what I needed to know. Thanks. From mal at egenix.com Wed Jan 20 14:11:55 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 20 Jan 2010 20:11:55 +0100 Subject: Interesting Problem In-Reply-To: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> References: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> Message-ID: <4B57557B.9070408@egenix.com> Victor Subervi wrote: > Hi; > I think I finally have an interesting problem for y'all. I need to import a > script from a lower dir, forcing me to change dirs: > > cwd = os.getcwd() > os.chdir('%s/..' % cwd) > sys.path.append(os.getcwd()) > from templateFrame import top, bottom > os.chdir(cwd) I suggest you read up on the built-in __import__() function: http://docs.python.org/library/functions.html#__import__ There's no need to change sys.path just to import a module. If you only need to execute some Python from a file, use execfile(): http://docs.python.org/library/functions.html#execfile -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 20 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From luismgz at gmail.com Wed Jan 20 14:13:07 2010 From: luismgz at gmail.com (=?ISO-8859-1?Q?Luis_M=2E_Gonz=E1lez?=) Date: Wed, 20 Jan 2010 11:13:07 -0800 (PST) Subject: What is a list compression in Python? References: <2ded0fd1-2bec-4ba1-a42c-b4262831140b@34g2000yqp.googlegroups.com> Message-ID: <55bd3026-c900-4a6a-b420-2d0d69e4b314@l30g2000yqb.googlegroups.com> On Jan 18, 1:07?pm, Kit wrote: > Hello Everyone, I am not sure if I have posted this question in a > correct board. Can anyone please teach me: > > What is a list compression in Python? > > Would you mind give me some list compression examples? > > Thanks & really appreciate that. > Kit It's also worth noting that from Python 3.0 on, you have also dict and set comprehensions. From mal at egenix.com Wed Jan 20 14:22:40 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 20 Jan 2010 20:22:40 +0100 Subject: Using jython to call python procedures/methods In-Reply-To: References: Message-ID: <4B575800.8000202@egenix.com> KB wrote: > Hi there, > > I have an application that only publishes a Java API. I can use jython > to access java classes, but jython currently (to the best of my > knowledge) does not support numpy/scipy. > > Ideally I would like to have jython call a "native" python routine > where I have the numpy/scipy procedures already written. > > Does anyone have any experience with this? Is it possible? > > I had toyed with the idea of having jython/java write the data to a > file/database and then manually kick off the python process, but > ideally I would like this as automated as possible. Have a look at this talk for some inspiration: http://www.slideshare.net/onyame/mixing-python-and-java Here's an example of using Pyro to connect Jython and CPython: http://www.razorvine.net/python/PyroAndJython As always with such solutions, there are a lot of weird problems to be expected :-/ -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 20 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From george.trojan at noaa.gov Wed Jan 20 16:03:10 2010 From: george.trojan at noaa.gov (George Trojan) Date: Wed, 20 Jan 2010 21:03:10 +0000 Subject: html code generation Message-ID: I need an advice on table generation. The table is essentially a fifo, containing about 200 rows. The rows are inserted every few minutes or so. The simplest solution is to store row data per line and write directly html code: line = "value1value2>... " each run of the program would read the previous table into a list of lines, insert the first row and drop the last one, taking care of table header and trailer. Is there a more classy solution? George From steven at REMOVE.THIS.cybersource.com.au Wed Jan 20 16:23:24 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 20 Jan 2010 21:23:24 GMT Subject: Best way to convert sequence of bytes to long integer References: <00e0377c$0$15570$c3e8da3@news.astraweb.com> <9b870cc2-0966-4cb2-be93-573317c06eb0@a6g2000yqm.googlegroups.com> Message-ID: On Wed, 20 Jan 2010 19:27:34 +0100, Alf P. Steinbach wrote: > * Mark Dickinson: >> On Jan 20, 7:36 am, Steven D'Aprano > cybersource.com.au> wrote: >>> I have a byte string (Python 2.x string), e.g.: >>> >>> s = "g%$f yg\n1\05" >>> assert len(s) == 10 >>> >>> I wish to convert it to a long integer, treating it as base-256. >> >> By the way, are you willing to divulge what you're using this >> functionality for? When trying to hack out the API for int.to_bytes >> and int.from_bytes on the mailing list and bug tracker, some of the >> discussion about use-cases seemed a little too much like guesswork; I'd >> be curious to find out about real-life use-cases. > > One possibility is that Steven wants to apply bitlevel and/or arithmetic > operations for e.g. some custom cryptographic thing. The string that he > uses as example is not completely unlike a message digest. Good guess! I'm writing a module that handles, I won't call it encryption, obfuscation using classical cryptographic algorithms. One of the functions needs a deterministic but unpredictable integer generated from a human-generated password or passphrase, so I'm using: hashlib.md5(key).digest() to get a string of bytes, then converting it to an int. int.from_bytes would be perfect for me, but in the meantime I'm using Paul Rubin's trick of int(s.encode("hex"), 16). Thanks to all who responded. -- Steven From sccolbert at gmail.com Wed Jan 20 16:52:07 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Wed, 20 Jan 2010 16:52:07 -0500 Subject: html code generation In-Reply-To: References: Message-ID: <7f014ea61001201352k639123b4m405c9e272bf628d6@mail.gmail.com> use a deque with a 'junk' as each element http://docs.python.org/library/collections.html On Wed, Jan 20, 2010 at 4:03 PM, George Trojan wrote: > I need an advice on table generation. The table is essentially a fifo, > containing about 200 rows. The rows are inserted every few minutes or so. > The simplest solution is to store row data per line and write directly html > code: > line = "value1value2>... " > each run of the program would read the previous table into a list of lines, > insert the first row and drop the last one, taking care of table header and > trailer. > Is there a more classy solution? > > George > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sccolbert at gmail.com Wed Jan 20 16:55:25 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Wed, 20 Jan 2010 16:55:25 -0500 Subject: html code generation In-Reply-To: <7f014ea61001201352k639123b4m405c9e272bf628d6@mail.gmail.com> References: <7f014ea61001201352k639123b4m405c9e272bf628d6@mail.gmail.com> Message-ID: <7f014ea61001201355i3d4071d6md171c1b7d234c859@mail.gmail.com> On Wed, Jan 20, 2010 at 4:52 PM, Chris Colbert wrote: > use a deque with a 'junk' as each element > > http://docs.python.org/library/collections.html > > On Wed, Jan 20, 2010 at 4:03 PM, George Trojan wrote: > >> I need an advice on table generation. The table is essentially a fifo, >> containing about 200 rows. The rows are inserted every few minutes or so. >> The simplest solution is to store row data per line and write directly html >> code: >> line = "value1value2>... " >> each run of the program would read the previous table into a list of >> lines, insert the first row and drop the last one, taking care of table >> header and trailer. >> Is there a more classy solution? >> >> George >> -- >> http://mail.python.org/mailman/listinfo/python-list > > In [1]: from collections import deque In [2]: a = deque(['foo', 'bar']) In [3]: ''.join(a) Out[3]: 'foobar' In [4]: a.popleft() Out[4]: 'foo' In [5]: a.append('baz') In [6]: ''.join(a) Out[6]: 'barbaz' -------------- next part -------------- An HTML attachment was scrubbed... URL: From sccolbert at gmail.com Wed Jan 20 16:59:38 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Wed, 20 Jan 2010 16:59:38 -0500 Subject: html code generation In-Reply-To: <7f014ea61001201355i3d4071d6md171c1b7d234c859@mail.gmail.com> References: <7f014ea61001201352k639123b4m405c9e272bf628d6@mail.gmail.com> <7f014ea61001201355i3d4071d6md171c1b7d234c859@mail.gmail.com> Message-ID: <7f014ea61001201359x1444106frd38f58bf9cf7dfde@mail.gmail.com> ah ok, i misread your post. Store each 'junk' as an item in the deque. the idea is the same though. On Wed, Jan 20, 2010 at 4:55 PM, Chris Colbert wrote: > > > On Wed, Jan 20, 2010 at 4:52 PM, Chris Colbert wrote: > >> use a deque with a 'junk' as each element >> >> http://docs.python.org/library/collections.html >> >> On Wed, Jan 20, 2010 at 4:03 PM, George Trojan wrote: >> >>> I need an advice on table generation. The table is essentially a fifo, >>> containing about 200 rows. The rows are inserted every few minutes or so. >>> The simplest solution is to store row data per line and write directly html >>> code: >>> line = "value1value2>... " >>> each run of the program would read the previous table into a list of >>> lines, insert the first row and drop the last one, taking care of table >>> header and trailer. >>> Is there a more classy solution? >>> >>> George >>> -- >>> http://mail.python.org/mailman/listinfo/python-list >> >> > In [1]: from collections import deque > > In [2]: a = deque(['foo', 'bar']) > > In [3]: ''.join(a) > Out[3]: 'foobar' > > In [4]: a.popleft() > Out[4]: 'foo' > > In [5]: a.append('baz') > > In [6]: ''.join(a) > Out[6]: 'barbaz' > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From darkneter at gmail.com Wed Jan 20 17:01:14 2010 From: darkneter at gmail.com (NighterNet) Date: Wed, 20 Jan 2010 14:01:14 -0800 (PST) Subject: Object Integer mapping References: <60e22629-c7a9-4d8f-930f-a3ed777b5f42@p24g2000yqm.googlegroups.com> Message-ID: <0209aa58-d355-4a6a-9061-216ea7aa8c59@e16g2000yqc.googlegroups.com> On Jan 20, 9:43?am, Robert Kern wrote: > On 2010-01-20 11:18 AM, Richard Thomas wrote: > > > > > On Jan 20, 4:43 pm, NighterNet ?wrote: > >> Need help on python version 3.1.x. I can't seem to know what going on > >> a bit. The code that I check that the hash is different every time. Is > >> there a way to make the hash the same? I using as to check the class > >> is the same variables but if it different variable in the class that > >> it add to the array and return the index. > > >> # Generic Object->Integer mapping > >> # the object must be usable as a dictionary key > >> class ObjMap: > >> ? ? ? ? ?def __init__(self): > >> ? ? ? ? ? ? ? ? ?self.dict = {} > >> ? ? ? ? ? ? ? ? ?self.next = 0 > >> ? ? ? ? ?def get(self, obj): > >> ? ? ? ? ? ? ? ? ?if obj in self.dict: > >> ? ? ? ? ? ? ? ? ? ? ? ? ?return self.dict[obj] > >> ? ? ? ? ? ? ? ? ?else: > >> ? ? ? ? ? ? ? ? ? ? ? ? ?id = self.next > >> ? ? ? ? ? ? ? ? ? ? ? ? ?self.next = self.next + 1 > >> ? ? ? ? ? ? ? ? ? ? ? ? ?self.dict[obj] = id > >> ? ? ? ? ? ? ? ? ? ? ? ? ?return id > > >> ? ? ? ? ?def items(self): > >> ? ? ? ? ? ? ? ? ?getval = operator.itemgetter(0) > >> ? ? ? ? ? ? ? ? ?getkey = operator.itemgetter(1) > >> ? ? ? ? ? ? ? ? ?return map(getval, sorted(self.dict.items(), key=getkey)) > > >> class Point: > >> ? ? ? ? ?def __init__(self): > >> ? ? ? ? ? ? ? ? ?self.x = 0 > >> ? ? ? ? ? ? ? ? ?self.y = 0 > >> ? ? ? ? ? ? ? ? ?self.z = 0 > > >> points = ObjMap() > > >> Testme = Point() > >> Testme.x = 0 > >> print(points.get(Testme)) > >> Testme2 = Point() > >> Testme2.y = 1 > >> print(points.get(Testme2)) > >> Testme3 = Point() > >> Testme3.y = 1 > >> print(points.get(Testme3)) > >> Ttestme4 = Point() > >> Ttestme4.y = 1 > >> print( points.get(Ttestme4)) > > >> It keep adding new array from this but doesn't match hash. Need help > >> on how to fix this class code. > > > You need to define how the Point class hashes. You can do this by > > writing a __hash__ method like so: > > > class Point(object): > > ? ? ?... > > ? ? ?def __hash__(self): > > ? ? ? ? ?return hash(self.x) ^ hash(self.y) ^ hash(self.z) > > > However you will also need to define how the Point class relates as > > equal using the __eq__ or __cmp__ methods. > > > class Point(object): > > ? ? ?... > > ? ? ?def __eq__(self, other): > > ? ? ? ? ?return self.x == other.x and self.y == other.y and self.z == > > other.z > > > This addition makes sure that if two points a equivalent then they > > count as the same key in the dictionary. > > I recommend a simpler approach that duplicates less code and makes it easier to > maintain the __hash__/__eq__ invariants: > > class Point(object): > ? ? ?... > ? ? ?def _key(self): > ? ? ? ? ?# The type(self).__name__ bit is optional, but usually handy. > ? ? ? ? ?return (type(self).__name__, self.x, self.y, self.z) > > ? ? ?def __hash__(self): > ? ? ? ? ?return hash(self._key()) > > ? ? ?def __eq__(self, other): > ? ? ? ? ?if not hasattr(other, '_key'): > ? ? ? ? ? ? ?return False > ? ? ? ? ?return self._key() == other._key() > > It is also worth noting that once one allows hashing by value, one should treat > the objects as immutable, so one should not change the attributes as the OP does > in his example. > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless enigma > ? that is made terrible by our own mad attempt to interpret it as though it had > ? an underlying truth." > ? ?-- Umberto Eco Thanks you, it help fix my hash check against my classes. From darcy at druid.net Wed Jan 20 17:03:42 2010 From: darcy at druid.net (D'Arcy J.M. Cain) Date: Wed, 20 Jan 2010 17:03:42 -0500 Subject: html code generation In-Reply-To: References: Message-ID: <20100120170342.ba73aeb1.darcy@druid.net> On Wed, 20 Jan 2010 21:03:10 +0000 George Trojan wrote: > I need an advice on table generation. The table is essentially a fifo, > containing about 200 rows. The rows are inserted every few minutes or > so. The simplest solution is to store row data per line and write > directly html code: > line = "value1value2>... " > each run of the program would read the previous table into a list of > lines, insert the first row and drop the last one, taking care of table > header and trailer. > Is there a more classy solution? Almost positively. It's hard to say for sure though without knowing more. However, I have a few pointers for you. First, think about a proper database. You can store the raw data into something like PostgreSQL and then you have your historical record. Then you can extract and format the latest 200 records at run time. You can change that value any time you want and even look at historical information such as the output exactly 24 hours ago. Whether you generate the page on demand or pre-calculate at intervals will depend on the ratio of updates to displays. I would start with generating on demand to start with and profile usage. Look into server side HTML. It can be a bit ugly generating your entire web page in code. Write the static part as a regular file and include your small Python script to generate the table data. The cool thing with that is that you can give your webaster the task of making pretty pages and simply deal with the variable generation. HTH. -- D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner. From iainspeed at gmail.com Wed Jan 20 17:45:44 2010 From: iainspeed at gmail.com (Iain Barnett) Date: Wed, 20 Jan 2010 22:45:44 +0000 Subject: installing psycopg2-2.0.13 with python3.1 Message-ID: <03495BC1-8736-42D5-9B59-5B0148F643AA@gmail.com> Hi, Would anyone know if it's possible to install psycopg2-2.0.13 with python3.1.1 (or similar)? When I try I get the following $ sudo python setup.py install Password: File "setup.py", line 233 except Warning, w: ^ SyntaxError: invalid syntax I can install it with python2.6 with no problems, but obviously I'd prefer to use the latest version. My system is OSX10.6, and I'm new to Python. Any help is much appreciated. Iain From gagsl-py2 at yahoo.com.ar Wed Jan 20 18:09:17 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Wed, 20 Jan 2010 20:09:17 -0300 Subject: Inheriting methods but over-riding docstrings References: <0361ea2d$0$1336$c3e8da3@news.astraweb.com> <28aa31f4-7331-4ac2-918b-0c59f5a834de@e37g2000yqn.googlegroups.com> Message-ID: En Tue, 19 Jan 2010 08:44:09 -0300, Michele Simionato escribi?: > On Jan 16, 6:55 pm, Steven D'Aprano cybersource.com.au> wrote: >> I have a series of subclasses that inherit methods from a base class, >> but >> I'd like them to have their own individual docstrings. > > from types import FunctionType, CodeType > > def newfunc(func, docstring): > c = func.func_code > nc = CodeType(c.co_argcount, c.co_nlocals, c.co_stacksize, > c.co_flags, c.co_code, c.co_consts, c.co_names, > c.co_varnames, c.co_filename, func.__name__, > c.co_firstlineno, c.co_lnotab, c.co_freevars, > c.co_cellvars) > nf = FunctionType(nc, func.func_globals, func.__name__) > nf.__doc__ = docstring > return nf > > def setdocstring(method, docstring): > cls = method.im_class > basefunc = getattr(super(cls, cls), method.__name__).im_func > setattr(cls, method.__name__, newfunc(basefunc, docstring)) > > class B(object): > def m(self): > "base" > return 'ok' > > class C(B): > pass > > setdocstring(C.m, 'C.m docstring') This is basically the same technique as in but there is a difference: you clone the function object *and* the code object it is based on. As I understand it, code objects are immutable and there is no need to clone them, but I may be wrong. Why did you feel the need to clone the code object too? -- Gabriel Genellina From joncle at googlemail.com Wed Jan 20 18:15:01 2010 From: joncle at googlemail.com (Jon Clements) Date: Wed, 20 Jan 2010 15:15:01 -0800 (PST) Subject: html code generation References: Message-ID: <5a451761-8bd8-446d-bb5d-35017661a589@r24g2000yqd.googlegroups.com> On Jan 20, 10:03?pm, "D'Arcy J.M. Cain" wrote: > On Wed, 20 Jan 2010 21:03:10 +0000 > > George Trojan wrote: > > I need an advice on table generation. The table is essentially a fifo, > > containing about 200 rows. The rows are inserted every few minutes or > > so. The simplest solution is to store row data per line and write > > directly html code: > > line = "value1value2>... " > > each run of the program would read the previous table into a list of > > lines, insert the first row and drop the last one, taking care of table > > header and trailer. > > Is there a more classy solution? > > Almost positively. ?It's hard to say for sure though without knowing > more. ?However, I have a few pointers for you. > > First, think about a proper database. ?You can store the raw data into > something like PostgreSQL and then you have your historical record. > Then you can extract and format the latest 200 records at run time. ?You > can change that value any time you want and even look at historical > information such as the output exactly 24 hours ago. > > Whether you generate the page on demand or pre-calculate at intervals > will depend on the ratio of updates to displays. ?I would start with > generating on demand to start with and profile usage. > > Look into server side HTML. ?It can be a bit ugly generating your > entire web page in code. ?Write the static part as a regular file and > include your small Python script to generate the table data. ?The cool > thing with that is that you can give your webaster the task of making > pretty pages and simply deal with the variable generation. > > HTH. > > -- > D'Arcy J.M. Cain ? ? ? ? | ?Democracy is three wolveshttp://www.druid.net/darcy/? ? ? ? ? ? ? ?| ?and a sheep voting on > +1 416 425 1212 ? ? (DoD#0082) ? ?(eNTP) ? | ?what's for dinner. I'd second this and go for a DB solution; possibly even SQLite to start with. I would tend to go with a web framework (maybe OTT) or a templating engine, so if you wish, someone can do the CSS/HTML and you just provide something that says (gimme the last 200 hundred rows). Very similar to what D'Arcy J.M. Cain has said but should hopefully formalise the possible problem and be applicable across the project. Further more you might want to look into AJAX, but ummm, that's a completely new debate :) Cheers, Jon. From nils at ccsg.de Wed Jan 20 18:30:21 2010 From: nils at ccsg.de (Nils Ruettershoff) Date: Thu, 21 Jan 2010 00:30:21 +0100 Subject: multiprocessing problems In-Reply-To: <63ef9fe9-6e9b-4766-9a0c-4aebc28d4148@p24g2000yqm.googlegroups.com> References: <74594510-9366-4227-8e1e-d221a9220271@k35g2000yqb.googlegroups.com> <63c5df56-7b86-4ee7-ba4f-b14984f94097@e16g2000yqc.googlegroups.com> <63ef9fe9-6e9b-4766-9a0c-4aebc28d4148@p24g2000yqm.googlegroups.com> Message-ID: <4B57920D.1030601@ccsg.de> Hi Doxa, DoxaLogos wrote: [...] > I found out my problems. One thing I did was followed the test queue > example in the documentation, but the biggest problem turned out to be > a pool instantiated globally in my script was causing most of the > endless process spawn, even with the "if __name__ == "__main__":" > block. > Problems who solves them self, are the best problems ;) One tip: currently your algorithm has some overhead. 'Cause you are starting 4 time an additional python interpreter, compute the files and, closing all new spawned interpreter and starting again 4 interpreter, which are processing the files. For such kind of jobs I prefer to start processes once and feeding them with data via a queue. This reduces some overhead and increase runtime performance. This could look like this: (due some pseudo functions not directly executeable -> untested) import multiprocessing import Queue class Worker(multiprocessing.Process): def __init__(self, feeder_q, queue_filled): multiprocessing.Process.__init__(self) self.feeder_q = feeder_q self.queue_filled = queue_filled def run(self): serve = True # start infinite loop while serve: try: # scan queue for work, will block process up to 5 seconds. If until then no item is in queue a Queue.Empty will be raised text = self.feeder_q.get(True, timeout=5) if text: do_stuff(text) # very important! tell the queue that the fetched work has been finished # otherwise the feeder_q.join() would block infinite self.input_queue.task_done() except Queue.Empty: # as soon as queue is empty and all work has been enqueued # process can terminate itself if self.queue_filled.is_set() and feeder_q.empty(): serve = False return if __name__ == '__main__': number_of_processes = 4 queue_filled = multiprocessing.Event() feeder_q = multiprocessing.JoinableQueue() process_list =[] # get file name which need to be processed all_files = get_all_files() # start processes for i in xrange(0,number_of_processes): process = Worker(feeder_q, queue_filled) process.start() process_list.append(thread) # start feeding for file in all_files: feeder_q.put(file) # inform processes that all work has been ordered queue_filled.set() # wait until queue is empty feeder_q.join() # wait until all processed have finished their jobs for process in process_list: process.join() Cheers, Nils From nils at ccsg.de Wed Jan 20 18:45:32 2010 From: nils at ccsg.de (Nils Ruettershoff) Date: Thu, 21 Jan 2010 00:45:32 +0100 Subject: multiprocessing problems In-Reply-To: <1263918409.3165.15.camel@linux-m3mt> References: <74594510-9366-4227-8e1e-d221a9220271@k35g2000yqb.googlegroups.com> <1263918409.3165.15.camel@linux-m3mt> Message-ID: <4B57959C.4020201@ccsg.de> Hi, Adam Tauno Williams wrote: [...] >> Here's the guts of my latest incarnation. >> def ProcessBatch(files): >> p = [] >> for file in files: >> p.append(Process(target=ProcessFile,args=file)) >> for x in p: >> x.start() >> for x in p: >> x.join() >> p = [] >> return >> Now, the function calling ProcessBatch looks like this: >> def ReplaceIt(files): >> processFiles = [] >> for replacefile in files: >> if(CheckSkipFile(replacefile)): >> processFiles.append(replacefile) >> if(len(processFiles) == 4): >> ProcessBatch(processFiles) >> processFiles = [] >> #check for left over files once main loop is done and process them >> if(len(processFiles) > 0): >> ProcessBatch(processFiles) >> > > According to this you will create files is sets of four, but an unknown > number of sets of four. > > This is not correct, 'cause of the x.join(). This will block the parent process until all processes have been terminated. So as soon as the current set of processes have finished their job, a new set will be spawned. Cheers, Nils -------------- next part -------------- An HTML attachment was scrubbed... URL: From philip at semanchuk.com Wed Jan 20 18:58:37 2010 From: philip at semanchuk.com (Philip Semanchuk) Date: Wed, 20 Jan 2010 18:58:37 -0500 Subject: installing psycopg2-2.0.13 with python3.1 In-Reply-To: <03495BC1-8736-42D5-9B59-5B0148F643AA@gmail.com> References: <03495BC1-8736-42D5-9B59-5B0148F643AA@gmail.com> Message-ID: On Jan 20, 2010, at 5:45 PM, Iain Barnett wrote: > Hi, > > Would anyone know if it's possible to install psycopg2-2.0.13 with > python3.1.1 (or similar)? When I try I get the following > > $ sudo python setup.py install > Password: > File "setup.py", line 233 > except Warning, w: > ^ > SyntaxError: invalid syntax > > > I can install it with python2.6 with no problems, but obviously I'd > prefer to use the latest version. My system is OSX10.6, and I'm new > to Python. Hi Iain, I've been out of the psycopg loop for a while now, but my guess is that the answer to your question is no. There are some Python 3- related checkins in the psycopg2 repository, but it looks like they're not ready for public use yet (unless you're adventurous). Here's the source repository, which contains a commit commented "First round of changes for Python 3": https://dndg.it/cgi-bin/gitweb.cgi?p=public/psycopg2.git If you scroll to the bottom of that page you'll see a "head" created especially for Python 2 which is where 2.0.13 came from. If you're willing to do some hacking on your own, Martin v. L?wis ported psycopg2 to Python 3 and provided a diff: http://mail.python.org/pipermail/python-porting/2008-December/000018.html Last but not least, I should note that psycopg has its own mailing list, and I bet they know more about it there: http://lists.initd.org/mailman/listinfo/psycopg Hope this helps. Psycopg & Postgres both worked wonderfully well for me. Cheers Philip From gagsl-py2 at yahoo.com.ar Wed Jan 20 19:11:49 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Wed, 20 Jan 2010 21:11:49 -0300 Subject: installing psycopg2-2.0.13 with python3.1 References: <03495BC1-8736-42D5-9B59-5B0148F643AA@gmail.com> Message-ID: En Wed, 20 Jan 2010 19:45:44 -0300, Iain Barnett escribi?: > Would anyone know if it's possible to install psycopg2-2.0.13 with > python3.1.1 (or similar)?I can install it with python2.6 with no > problems, but obviously I'd prefer to use the latest version. My system > is OSX10.6, and I'm new to Python. If you're new to Python, perhaps it's better to stay with the 2.6 version. Python 3.x introduced some incompatible changes in the language; not all third-party packages are available for Python 3.x yet. Regarding psycopg2, although some work has been done [1], there is no "official" release compatible with Python 3 yet. If you insist on using Python 3.1, there is another interface to PostgreSQL called pg8000 that claims to be Python 3.x compatible (I've not actually tested it). [1] http://mail.python.org/pipermail/python-porting/2008-December/000004.html [2] http://pybrary.net/pg8000/ -- Gabriel Genellina From g.bogle at auckland.no.spam.ac.nz Wed Jan 20 19:22:19 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Thu, 21 Jan 2010 13:22:19 +1300 Subject: py2exe and pydocs. Downloads? References: Message-ID: Gabriel Genellina wrote: > c:\temp>python -m pydoc sys > Help on built-in module sys: > [...same info...] When I do this I get: No module named tempfile From icanbob at gmail.com Wed Jan 20 20:00:10 2010 From: icanbob at gmail.com (bobicanprogram) Date: Wed, 20 Jan 2010 17:00:10 -0800 (PST) Subject: Using jython to call python procedures/methods References: Message-ID: On Jan 20, 10:32 am, KB wrote: > Hi there, > > I have an application that only publishes a Java API. I can use jython > to access java classes, but jython currently (to the best of my > knowledge) does not support numpy/scipy. > > Ideally I would like to have jython call a "native" python routine > where I have the numpy/scipy procedures already written. > > Does anyone have any experience with this? Is it possible? > > I had toyed with the idea of having jython/java write the data to a > file/database and then manually kick off the python process, but > ideally I would like this as automated as possible. > > Thanks in advance. You might want to take a look at the SIMPL toolkit (http:// www.icanprogram.com/simpl). It sports both JAVA and Python hooks so you should be able to pump the data from your JAVA module to a Python/ numpy/scipy module. bob From zuo at chopin.edu.pl Wed Jan 20 20:02:02 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Thu, 21 Jan 2010 02:02:02 +0100 Subject: Sorted dictionary Message-ID: Hello, Inspired by some my needs as well as some discussions in the net, I've implemented a sorted dictionary (i.e. a dict that returns keys, values and items always sorted by keys): http://code.activestate.com/recipes/576998/ Maybe it'll appear to be useful for somebody... And I'm curious about your opinions. Regards, *j -- Jan Kaliszewski (zuo) From someone at somewhere.com Wed Jan 20 20:30:25 2010 From: someone at somewhere.com (Gringo) Date: Wed, 20 Jan 2010 19:30:25 -0600 Subject: * operator in python tutorial In-Reply-To: <7b2ef0aa-8b93-4cb5-b944-beea3e2c98b1@p16g2000vbe.googlegroups.com> References: <7b2ef0aa-8b93-4cb5-b944-beea3e2c98b1@p16g2000vbe.googlegroups.com> Message-ID: <0036df65$0$25085$c3e8da3@news.astraweb.com> On 1/20/2010 12:38, ben wrote: > Hello, > > I am following through the python tutorial which gets to a line that > uses the * operator with zip(). I searched and searched but could find > no information on the operator or how to use it in general. The > example from the tut is as follows: >>>> x = [1, 2, 3] >>>> y = [4, 5, 6] >>>> zipped = zip(x, y) >>>> zipped > [(1, 4), (2, 5), (3, 6)] >>>> x2, y2 = zip(*zipped) >>>> x == list(x2) and y == list(y2) > True > > How would i apply the * operator in general? > Thanks. The * operator, when used in this context, unpacks the sequence and it's as if you passed each item to the function as a different parameter. For example, if you have a list x with 4 items, these two statements would be the same: f(x[0],x[1],x[2],x[3]) f(*x) Hope this helps. From steven at REMOVE.THIS.cybersource.com.au Wed Jan 20 20:50:58 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 21 Jan 2010 01:50:58 GMT Subject: Sorted dictionary References: Message-ID: On Thu, 21 Jan 2010 02:02:02 +0100, Jan Kaliszewski wrote: > Hello, > > Inspired by some my needs as well as some discussions in the net, I've > implemented a sorted dictionary (i.e. a dict that returns keys, values > and items always sorted by keys): > > http://code.activestate.com/recipes/576998/ What's the advantage of that over sorting the keys as needed? E.g. for key in sorted(dict): print key works. -- Steven From gnarlodious at gmail.com Wed Jan 20 20:56:16 2010 From: gnarlodious at gmail.com (Gnarlodious) Date: Wed, 20 Jan 2010 17:56:16 -0800 (PST) Subject: Py 3: How to switch application to Unicode strings? References: <92288825-8b01-47fb-8592-f65a7d7aeb8a@k35g2000yqb.googlegroups.com><7a9c25c21001191156j46a7fdadt58b728477b85e651@mail.gmail.com> <646ab38b-0710-4d31-b9e1-8a6ee7bfa42f@21g2000yqj.googlegroups.com> Message-ID: Thanks for the help, but I am going to skip this problem because I don't need Unicode characters in a script anyway. -- Gnarlie From gnarlodious at gmail.com Wed Jan 20 22:02:49 2010 From: gnarlodious at gmail.com (Gnarlodious) Date: Wed, 20 Jan 2010 19:02:49 -0800 (PST) Subject: Create object name from string value? Message-ID: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> I want to declare several objects from names in a list: objects=['object1', 'object2', 'object3', 'object4'] for objectName in objects: objectName=classname() That fails, and so do these: exec(objectName)=classname() eval(objectName)=classname() So how to make an object whose name is the value in the variable? -- Gnarlie From ben+python at benfinney.id.au Wed Jan 20 22:31:11 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 21 Jan 2010 14:31:11 +1100 Subject: Create object name from string value? References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> Message-ID: <87ska0dthc.fsf@benfinney.id.au> Gnarlodious writes: > I want to declare several objects from names in a list: > > objects=['object1', 'object2', 'object3', 'object4'] > for objectName in objects: > objectName=classname() Since (I presume) you are a newcomer to Python, it's best to learn the common style . I will assume the above is written as:: object_names = ['object1', 'object2', 'object3', 'object4'] for object_name in object_names: object_name = Class() > That fails Well, it succeeds (runs successfully). But I get what you mean: it fails to do what you expected. > So how to make an object whose name is the value in the variable? This is almost certainly better done with a dictionary. Like so:: object_names = ['object1', 'object2', 'object3', 'object4'] objects = {} for object_name in object_names: objects[object_name] = Class() There are shortcuts that can be used to create the dictionary without the separate list of names. But for now the above should make sense immediately and help you get to the meat of the problem. -- \ ?We now have access to so much information that we can find | `\ support for any prejudice or opinion.? ?David Suzuki, 2008-06-27 | _o__) | Ben Finney From gnarlodious at gmail.com Wed Jan 20 23:36:33 2010 From: gnarlodious at gmail.com (Gnarlodious) Date: Wed, 20 Jan 2010 20:36:33 -0800 (PST) Subject: Create object name from string value? References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> <87ska0dthc.fsf@benfinney.id.au> Message-ID: <0fa4a0fa-b813-4ba4-b032-8a4d6fbe0132@m26g2000yqb.googlegroups.com> On Jan 20, 8:31?pm, Ben Finney wrote: > Since (I presume) you are a newcomer to Python, it's best to learn the > common style . Thanks for that. > This is almost certainly better done with a dictionary. Like so:: > > ? ? object_names = ['object1', 'object2', 'object3', 'object4'] > ? ? objects = {} > ? ? for object_name in object_names: > ? ? ? ? objects[object_name] = Class() And thanks for that too. I see it returns a list of objects, Python is very cool. -- Gnarlie From qq263020776 at gmail.com Wed Jan 20 23:56:07 2010 From: qq263020776 at gmail.com (yousay) Date: Wed, 20 Jan 2010 20:56:07 -0800 (PST) Subject: why the super class can access the subclass's attribute Message-ID: I have sees aprogram like this ,i confusing why super class can access the subclass's attribute ,this is the program,thanks in advance: class MyThread(threading.Thread): def join(self): super(MyThread,self).join() return self.result class Worker(MyThread): import random import pdb pdb.set_trace() def run(self): total = 0 for i in range(random.randrange(10,100)): total +=i self.result = total From steven at REMOVE.THIS.cybersource.com.au Thu Jan 21 00:35:54 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 21 Jan 2010 05:35:54 GMT Subject: Create object name from string value? References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> Message-ID: On Wed, 20 Jan 2010 19:02:49 -0800, Gnarlodious wrote: > I want to declare several objects from names in a list: > > objects=['object1', 'object2', 'object3', 'object4'] for objectName in > objects: > objectName=classname() That's the wrong way to handle the problem. Named objects are only useful if you know the name of the object when writing the code. Otherwise, how do you know what name to use in the code? > That fails, and so do these: > > exec(objectName)=classname() > eval(objectName)=classname() The right way to solve this problem is with a dictionary: for name in ["object1", "object2", "object3"]: d = {name: classname()} print d[name] but for the record, the way to use exec is like this: exec("object1 = classname()") But beware, exec is not only much slower than the alternatives, but it risks putting a serious security flaw in your software. As a general rule, 9 times out of 10 if you think you want exec, you don't. -- Steven From wuwei23 at gmail.com Thu Jan 21 01:08:29 2010 From: wuwei23 at gmail.com (alex23) Date: Wed, 20 Jan 2010 22:08:29 -0800 (PST) Subject: Interesting Problem References: <4dc0cfea1001200719p6044cba5k4035b456a12b61cd@mail.gmail.com> Message-ID: <2d6f733e-e00c-4b87-9ba1-a3d9d8478461@k35g2000yqb.googlegroups.com> Victor Subervi wrote: > I think I finally have an interesting problem for y'all. I need to import a > script from a lower dir, forcing me to change dirs: I'm surprised that no one has yet mentioned packages. Suppose you have the following folder layout and you stick an empty __init__.py in each subfolder, like so: app/ tools/ __init__.py open.py close.py extensions/ __init__.py inner.py outer.py Scripts in the 'app' folder can then import from the subfolders using dotted notation: import tools from tools import open, close import tools.extensions from tools.extensions import inner, outer However, with empty __init__ files, you can't use attribute lookup on nested modules, so if you started with an 'import tools' any attempts to reference the contents of the module in the following ways will fail: tools.open tools.close tools.ext.inner tools.ext.outer To provide this functionality, you need to import the items you want available in the __init__ of the package they belong to. So by adding the following: tools/__init__.py: import open, close, ext tools/ext/__init__.py: import inner, outer You can then do a single 'import tools' and use standard attribute lookup to access the contents of your subfolders. From michele.simionato at gmail.com Thu Jan 21 01:17:40 2010 From: michele.simionato at gmail.com (Michele Simionato) Date: Wed, 20 Jan 2010 22:17:40 -0800 (PST) Subject: Inheriting methods but over-riding docstrings References: <0361ea2d$0$1336$c3e8da3@news.astraweb.com> <28aa31f4-7331-4ac2-918b-0c59f5a834de@e37g2000yqn.googlegroups.com> Message-ID: <5ecdae50-b417-44bb-ae5b-c1d9aea865a2@y23g2000yqm.googlegroups.com> On Jan 21, 12:09?am, "Gabriel Genellina" wrote: > This is basically the same technique as in ? > but there is ? > a difference: you clone the function object *and* the code object it is ? > based on. As I understand it, code objects are immutable and there is no ? > need to clone them, but I may be wrong. Why did you feel the need to clone ? > the code object too? No need. I just had the newfunc utility in my library (I think copied from a recipe in the Python cookbook by Alex Martelli) so I used it. In this case it is overkill, though. Also, I had no read your post when I posted my solution, otherwise I would not have sent it ;) Anyway, the setdocstring utility to extract the parent method was not explicit in your post and may be of some use to somebody. M. Simionato From gagsl-py2 at yahoo.com.ar Thu Jan 21 01:40:49 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Thu, 21 Jan 2010 03:40:49 -0300 Subject: py2exe and pydocs. Downloads? References: Message-ID: En Wed, 20 Jan 2010 21:22:19 -0300, Gib Bogle escribi?: > Gabriel Genellina wrote: > >> c:\temp>python -m pydoc sys >> Help on built-in module sys: >> [...same info...] > > When I do this I get: > > No module named tempfile You found a bug. Looks like it depends on the environment, or what packages are installed, or something like that, because it worked on my other PC but not here. Please report it at http://bugs.python.org so it doesn't get forgotten. -- Gabriel Genellina From michele.simionato at gmail.com Thu Jan 21 01:51:04 2010 From: michele.simionato at gmail.com (Michele Simionato) Date: Wed, 20 Jan 2010 22:51:04 -0800 (PST) Subject: counting lines of code Message-ID: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> I need a small utility to count the lines of Python code in a directory, traversing subdirectories and ignoring comments and docstrings. I am sure there is already something doing that, what do you suggest? TIA, Michele Simionato From steven at REMOVE.THIS.cybersource.com.au Thu Jan 21 01:54:35 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 21 Jan 2010 06:54:35 GMT Subject: Sorted dictionary References: Message-ID: On Wed, 20 Jan 2010 22:21:22 -0800, Dennis Lee Bieber wrote: > On Thu, 21 Jan 2010 02:02:02 +0100, "Jan Kaliszewski" > declaimed the following in > gmane.comp.python.general: > >> Hello, >> >> Inspired by some my needs as well as some discussions in the net, I've >> implemented a sorted dictionary (i.e. a dict that returns keys, values >> and items always sorted by keys): >> > How does this differ from all the other "ordered dictionary" > schemes out there (and maybe even included in 2.7? I'm still on 2.5) Ordered dicts remember the insertion order of keys. Sorted dicts return the keys in sorted order. -- Steven From skippy.hammond at gmail.com Thu Jan 21 02:06:15 2010 From: skippy.hammond at gmail.com (Mark Hammond) Date: Thu, 21 Jan 2010 18:06:15 +1100 Subject: counting lines of code In-Reply-To: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> Message-ID: <4B57FCE7.5010704@gmail.com> On 21/01/2010 5:51 PM, Michele Simionato wrote: > I need a small utility to count the lines of Python code in a > directory, traversing subdirectories and ignoring comments and > docstrings. I am sure there is already something doing that, what do > you suggest? I suggest typing your subject line into google and hitting the "I feel lucky" button :) HTH, Mark From ben+python at benfinney.id.au Thu Jan 21 02:12:06 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 21 Jan 2010 18:12:06 +1100 Subject: counting lines of code References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> Message-ID: <87bpgodj95.fsf@benfinney.id.au> Michele Simionato writes: > I need a small utility to count the lines of Python code in a > directory, traversing subdirectories and ignoring comments and > docstrings. I am sure there is already something doing that, what do > you suggest? Any of the static code checkers (?pylint?, ?pyflakes?, etc.) would already be doing this. You would at least be able to crib from them, maybe import the functionality; ideally they can directly report what you want to see. -- \ ?My business is to teach my aspirations to conform themselves | `\ to fact, not to try and make facts harmonise with my | _o__) aspirations.? ?Thomas Henry Huxley, 1860-09-23 | Ben Finney From michele.simionato at gmail.com Thu Jan 21 02:16:47 2010 From: michele.simionato at gmail.com (Michele Simionato) Date: Wed, 20 Jan 2010 23:16:47 -0800 (PST) Subject: counting lines of code References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> Message-ID: I did not known about cloc, it does more that I need, but it looks cool (it is perl and not Python, by who cares? ;) Thanks, Michele From michele.simionato at gmail.com Thu Jan 21 02:20:13 2010 From: michele.simionato at gmail.com (Michele Simionato) Date: Wed, 20 Jan 2010 23:20:13 -0800 (PST) Subject: counting lines of code References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> <87bpgodj95.fsf@benfinney.id.au> Message-ID: <1f166703-fdfc-4f87-b336-ef80e492635b@a32g2000yqm.googlegroups.com> On Jan 21, 8:12?am, Ben Finney wrote: > Michele Simionato writes: > > I need a small utility to count the lines of Python code in a > > directory, traversing subdirectories and ignoring comments and > > docstrings. I am sure there is already something doing that, what do > > you suggest? > > Any of the static code checkers (?pylint?, ?pyflakes?, etc.) would > already be doing this. pylint does too many things, I want something fast that just counts the lines and can be run on thousands of files at once. cloc seems fine, I have just tried on 2,000 files and it gives me a report in just a few seconds. From clp2 at rebertia.com Thu Jan 21 02:38:56 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Wed, 20 Jan 2010 23:38:56 -0800 Subject: why the super class can access the subclass's attribute In-Reply-To: References: Message-ID: <50697b2c1001202338q3d944121i7a2705557aac271e@mail.gmail.com> On Wed, Jan 20, 2010 at 8:56 PM, yousay wrote: > I have sees aprogram like this ,i confusing why super class can access > the subclass's attribute To make this easy to understand, I'm going to ***drastically*** oversimplify. With that disclaimer out of the way... When you access an instance attribute, as in self.result, Python actually returns self.__dict__["result"]; that is, it does a dictionary lookup using the attribute name as a string as the key; instance attribute assignment works analogously. Every object has a special dictionary associated with it that is used to store its instance attributes; this dictionary can itself be accessed through the __dict__ attribute. (And if you're about to ask how the __dict__ attribute gets looked up, well, it's magic?!) This "instance attribute access is just dictionary manipulation" principle is universal, even in subclasses, superclasses, and unrelated objects. Thus, in both Worker and its superclass MyThread, accessing the `result` instance attribute manipulates the very same dictionary (self.__dict__) and thus works exactly the same in both cases. So, essentially, `result` is accessible to MyThread because there's nothing to prevent it from being accessible there. Other related principles that may aid in your understanding (these are not oversimplified): - Attribute lookups all happen dynamically at runtime - Python has no language-enforced notion of `protected` or `private` attributes; everything's public - Python is dynamically typed and thus does no compile-time typechecking Cheers, Chris -- Simplifications include overlooking __slots__, metaclasses, __getattribute__, and instance.class_attribute, among others. http://blog.rebertia.com > ,this is the program,thanks in advance: > class MyThread(threading.Thread): > ? ?def join(self): > ? ? ? ?super(MyThread,self).join() > ? ? ? ?return self.result > > class Worker(MyThread): > ? ?import random > ? ?import pdb > ? ?pdb.set_trace() > ? ?def run(self): > ? ? ? ?total = 0 > ? ? ? ?for i in range(random.randrange(10,100)): > ? ? ? ? ? ?total +=i > ? ? ? ?self.result = total From Martin.Drautzburg at web.de Thu Jan 21 02:43:12 2010 From: Martin.Drautzburg at web.de (Martin Drautzburg) Date: Thu, 21 Jan 2010 08:43:12 +0100 Subject: Symbols as parameters? Message-ID: <1495297.2JfmrJjJTh@beaureve.gmx.net> Hello all, When passing parameters to a function, you sometimes need a paramter which can only assume certain values, e.g. def move (direction): ... If direction can only be "up", "down", "left" or "right", you can solve this by passing strings, but this is not quite to the point: - you could pass invalid strings easily - you need to quote thigs, which is a nuisance - the parameter IS REALLY NOT A STRING, but a direction Alternatively you could export such symbols, so when you "import *" you have them available in the caller's namespace. But that forces you to "import *" which pollutes your namespace. What I am really looking for is a way - to be able to call move(up) - having the "up" symbol only in the context of the function call So it should look something like this ... magic, magic ... move(up) ... unmagic, unmagic ... print up This should complain that "up" is not defined during the "print" call, but not when move() is called. And of course there should be as little magic as possible. Any way to achieve this? From clp2 at rebertia.com Thu Jan 21 02:49:22 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Wed, 20 Jan 2010 23:49:22 -0800 Subject: Sorted dictionary In-Reply-To: References: Message-ID: <50697b2c1001202349m7b346df5q4686dfa51b29bc13@mail.gmail.com> On Wed, Jan 20, 2010 at 5:50 PM, Steven D'Aprano wrote: > On Thu, 21 Jan 2010 02:02:02 +0100, Jan Kaliszewski wrote: > >> Hello, >> >> Inspired by some my needs as well as some discussions in the net, I've >> implemented a sorted dictionary (i.e. a dict that returns keys, values >> and items always sorted by keys): >> >> http://code.activestate.com/recipes/576998/ > > > What's the advantage of that over sorting the keys as needed? > > > E.g. > > for key in sorted(dict): > ? ?print key > > > works. Well, it does spread out the cost of sorting the key list over each of the N distinct key assignments, versus sorting the entire list all at once, on-demand at the end of the process. And you avoid having to traverse the M buckets in the dictionary to locate the N keys in the first place. So it has different performance characteristics, which could theoretically matter depending on the use case; e.g. it could avoid a GUI application hanging for several seconds while it sorts a large list of keys. Cheers, Chris -- http://blog.rebertia.com From andreengels at gmail.com Thu Jan 21 03:18:32 2010 From: andreengels at gmail.com (Andre Engels) Date: Thu, 21 Jan 2010 09:18:32 +0100 Subject: Symbols as parameters? In-Reply-To: <1495297.2JfmrJjJTh@beaureve.gmx.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: <6faf39c91001210018v4a88b77eg3058f30ea41257a4@mail.gmail.com> On Thu, Jan 21, 2010 at 8:43 AM, Martin Drautzburg wrote: > Hello all, > > When passing parameters to a function, you sometimes need a paramter > which can only assume certain values, e.g. > > ? ? ? ?def move (direction): > ? ? ? ? ? ? ? ?... > If direction can only be "up", "down", "left" or "right", you can solve > this by passing strings, but this is not quite to the point: > > ? ? ? ?- you could pass invalid strings easily > ? ? ? ?- you need to quote thigs, which is a nuisance > ? ? ? ?- the parameter IS REALLY NOT A STRING, but a direction > > Alternatively you could export such symbols, so when you "import *" you > have them available in the caller's namespace. But that forces you > to "import *" which pollutes your namespace. > > What I am really looking for is a way > > ? ? ? ?- to be able to call move(up) > ? ? ? ?- having the "up" symbol only in the context of the function call > > So it should look something like this > > ... magic, magic ... > move(up) > ... unmagic, unmagic ... > print up > > This should complain that "up" is not defined during the "print" call, > but not when move() is called. And of course there should be as little > magic as possible. > > Any way to achieve this? It probably can be done, but in my opinion even it can, it would be ugly. I would just define "up" as some global constant - I don't see why it would be a problem that it's defined when you don't need it. In fact, I think it would be beneficial - you can then have direction-valued variables, so that you can (for example) implement "go the same direction I went last time". If that kind of thing happens more often, it might be useful to have a Direction class, to be able to easily program in things like "the opposite of up is down" and "right from right is back". -- Andr? Engels, andreengels at gmail.com From python at rcn.com Thu Jan 21 03:27:52 2010 From: python at rcn.com (Raymond Hettinger) Date: Thu, 21 Jan 2010 00:27:52 -0800 (PST) Subject: Sorted dictionary References: Message-ID: <1c74df0b-b22e-4aea-934a-c6d25e818f41@e37g2000yqn.googlegroups.com> On Jan 20, 5:02?pm, "Jan Kaliszewski" wrote: > Hello, > > Inspired by some my needs as well as some discussions in the net, I've ? > implemented a sorted dictionary (i.e. a dict that returns keys, values and ? > items always sorted by keys): > > http://code.activestate.com/recipes/576998/ > > Maybe it'll appear to be useful for somebody... And I'm curious about your ? > opinions. Using an underlying list to track sorted items means that insertion and deletion take O(n) time. That could be reduced to O(log n) time by using a blist or skiplist as the underlying structure for maintaining a sort. The other alternative is to just append items to the list and sort the list on demand. Since the Timsort takes advantage of existing order, the process will approach O(n) if sorting after every few updates. This is close the O(n) time it takes to iterate the list in the first place: s = SortedDict(items) list(s) # O(n log n) to sort and O(n) to iterate s[newkey] = newval list(s) # close to O(n) my two cents, Raymond From alfps at start.no Thu Jan 21 03:30:40 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 09:30:40 +0100 Subject: Symbols as parameters? In-Reply-To: <1495297.2JfmrJjJTh@beaureve.gmx.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: * Martin Drautzburg: > Hello all, > > When passing parameters to a function, you sometimes need a paramter > which can only assume certain values, e.g. > > def move (direction): > ... > If direction can only be "up", "down", "left" or "right", you can solve > this by passing strings, but this is not quite to the point: > > - you could pass invalid strings easily > - you need to quote thigs, which is a nuisance > - the parameter IS REALLY NOT A STRING, but a direction > > Alternatively you could export such symbols, so when you "import *" you > have them available in the caller's namespace. But that forces you > to "import *" which pollutes your namespace. > > What I am really looking for is a way > > - to be able to call move(up) > - having the "up" symbol only in the context of the function call > > So it should look something like this > > ... magic, magic ... > move(up) > ... unmagic, unmagic ... > print up > > This should complain that "up" is not defined during the "print" call, > but not when move() is called. And of course there should be as little > magic as possible. > > Any way to achieve this? >>> def move( direction ): ... print( "move " + str( direction ) ) ... >>> move( "up" ) move up >>> >>> class using_directions: ... up = 42 ... move( up ) ... move 42 >>> up Traceback (most recent call last): File "", line 1, in NameError: name 'up' is not defined >>> _ Of course it's an abuse of the language. :-) So I wouldn't recommend it, but it's perhaps worth having seen it. Cheers & hth., - Alf PS: I hope this was not a homework question. From nagle at animats.com Thu Jan 21 03:30:57 2010 From: nagle at animats.com (John Nagle) Date: Thu, 21 Jan 2010 00:30:57 -0800 Subject: html code generation In-Reply-To: References: Message-ID: <4b580d15$0$1621$742ec2ed@news.sonic.net> George Trojan wrote: > I need an advice on table generation. The table is essentially a fifo, > containing about 200 rows. The rows are inserted every few minutes or > so. The simplest solution is to store row data per line and write > directly html code: > line = "value1value2>... " > each run of the program would read the previous table into a list of > lines, insert the first row and drop the last one, taking care of table > header and trailer. > Is there a more classy solution? > > George The "HTMLTemplate" module is good for simple tasks like that. There are much more elaborate frameworks available, but if all you need to do is build a table periodically, it's a simple solution. This page http://www.sitetruth.com/reports/phishes.html is built with HTMLtemplate. It changes only every three hours, and computing it requires a big SQL join, so it's an automatically updated static page. John Nagle From javier.collado at gmail.com Thu Jan 21 03:38:24 2010 From: javier.collado at gmail.com (Javier Collado) Date: Thu, 21 Jan 2010 09:38:24 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: Hello, I'd say that isn't totally incorrect to use strings instead of symbols. Please note that in other programming languages symbols, atoms and the like are in fact immutable strings, which is what python provides by default. Best regards, Javier 2010/1/21 Alf P. Steinbach : > * Martin Drautzburg: >> >> Hello all, >> >> When passing parameters to a function, you sometimes need a paramter >> which can only assume certain values, e.g. >> >> ? ? ? ?def move (direction): >> ? ? ? ? ? ? ? ?... >> If direction can only be "up", "down", "left" or "right", you can solve >> this by passing strings, but this is not quite to the point: >> >> ? ? ? ?- you could pass invalid strings easily >> ? ? ? ?- you need to quote thigs, which is a nuisance >> ? ? ? ?- the parameter IS REALLY NOT A STRING, but a direction >> >> Alternatively you could export such symbols, so when you "import *" you >> have them available in the caller's namespace. But that forces you >> to "import *" which pollutes your namespace. >> >> What I am really looking for is a way >> >> ? ? ? ?- to be able to call move(up) >> ? ? ? ?- having the "up" symbol only in the context of the function call >> >> So it should look something like this >> >> ... magic, magic ... >> move(up) >> ... unmagic, unmagic ... >> print up >> >> This should complain that "up" is not defined during the "print" call, >> but not when move() is called. And of course there should be as little >> magic as possible. >> >> Any way to achieve this? > >>>> def move( direction ): > ... ? print( "move " + str( direction ) ) > ... >>>> move( "up" ) > move up >>>> >>>> class using_directions: > ... ? ? up = 42 > ... ? ? move( up ) > ... > move 42 >>>> up > Traceback (most recent call last): > ?File "", line 1, in > NameError: name 'up' is not defined >>>> _ > > > Of course it's an abuse of the language. :-) > > So I wouldn't recommend it, but it's perhaps worth having seen it. > > > Cheers & hth., > > - Alf > > PS: I hope this was not a homework question. > -- > http://mail.python.org/mailman/listinfo/python-list > From ben+python at benfinney.id.au Thu Jan 21 04:10:34 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 21 Jan 2010 20:10:34 +1100 Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: <877hrbesc5.fsf@benfinney.id.au> Martin Drautzburg writes: > When passing parameters to a function, you sometimes need a paramter > which can only assume certain values You might like to try the ?enum? library for this . -- \ ?You could augment an earwig to the point where it understood | `\ nuclear physics, but it would still be a very stupid thing to | _o__) do!? ?The Doctor, _The Two Doctors_ | Ben Finney From bearophileHUGS at lycos.com Thu Jan 21 04:15:38 2010 From: bearophileHUGS at lycos.com (Bearophile) Date: Thu, 21 Jan 2010 01:15:38 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: Martin Drautzburg, having symbols spread in the namespace is bad. And too much magic is even worse. You seem to need something like an enum that must be used with its qualified name, as: move(direction.up) That works well unless the symbols name are keywords. Creating a small class like this (that warns if you give it a string that contains a keyword) looks not too much hard: direction = Enum("up", "down", "left", "right") Or: direction = Enum("up, down, left, right") Or: direction = Enum("up down left right") Bye, bearophile From bearophileHUGS at lycos.com Thu Jan 21 04:24:25 2010 From: bearophileHUGS at lycos.com (Bearophile) Date: Thu, 21 Jan 2010 01:24:25 -0800 (PST) Subject: Sorted dictionary References: <1c74df0b-b22e-4aea-934a-c6d25e818f41@e37g2000yqn.googlegroups.com> Message-ID: Raymond Hettinger: > Using an underlying list to track sorted items > means that insertion and deletion take O(n) time. > That could be reduced to O(log n) time by using > a blist or skiplist as the underlying structure > for maintaining a sort. In the collections module it can be useful to have ordered dicts and sets based on search trees. I'm thinking about B+ trees to use CPU cache locality better. It can be fun :-) Bye, bearophile From iainking at gmail.com Thu Jan 21 04:30:19 2010 From: iainking at gmail.com (Iain King) Date: Thu, 21 Jan 2010 01:30:19 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: <5cfc086f-e812-498c-a387-c7266aa810e6@p24g2000yqm.googlegroups.com> On Jan 21, 7:43?am, Martin Drautzburg wrote: > Hello all, > > When passing parameters to a function, you sometimes need a paramter > which can only assume certain values, e.g. > > ? ? ? ? def move (direction): > ? ? ? ? ? ? ? ? ... > If direction can only be "up", "down", "left" or "right", you can solve > this by passing strings, but this is not quite to the point: > > ? ? ? ? - you could pass invalid strings easily > ? ? ? ? - you need to quote thigs, which is a nuisance > ? ? ? ? - the parameter IS REALLY NOT A STRING, but a direction > > Alternatively you could export such symbols, so when you "import *" you > have them available in the caller's namespace. But that forces you > to "import *" which pollutes your namespace. > > What I am really looking for is a way > > ? ? ? ? - to be able to call move(up) > ? ? ? ? - having the "up" symbol only in the context of the function call > > So it should look something like this > > ... magic, magic ... > move(up) > ... unmagic, unmagic ... > print up > > This should complain that "up" is not defined during the "print" call, > but not when move() is called. And of course there should be as little > magic as possible. > > Any way to achieve this? class Direction(object): pass def is_direction(d): return type(d)==Direction up = Direction() down = Direction() left = Direction() right = Direction() Now you can do your move(up), print up, etc. You can also check a valid direction was passed in by calling is_direction. 'Course, you can extend this so it does something a little more useful: class Direction(object): def __init__(self, vx=0, vy=0): self.vx = vx self.vy = vy up = Direction(0, -1) down = Direction(0, 1) left = Direction(-1, 0) right = Direction(1, 0) def move(direction): spaceship.x += direction.vx spaceship.y += direction.vy Iain From rpjday at crashcourse.ca Thu Jan 21 05:04:40 2010 From: rpjday at crashcourse.ca (Robert P. J. Day) Date: Thu, 21 Jan 2010 05:04:40 -0500 (EST) Subject: deleting, then re-importing a class method Message-ID: (once again, never ashamed to ask the dumb questions.) still playing with python3, and testing whether i can delete/unimport a specific method, then re-import it: >>> import sys >>> print(sys.__doc__) ... blah blah blah ... >>> del(sys.__doc__) >>> print(sys.__doc__) module(name[, doc]) Create a module object. The name must be a string; the optional doc argument can have any type. >>> ok, now that i've deleted just that member of "sys", can i re-import it? i know this doesn't seem to work: >>> import sys or is there an operator other than "import" that more represents a full refresh of a class? rday p.s. no, i don't have a valid application of the above, i'm just trying to break things. -- ======================================================================== Robert P. J. Day Waterloo, Ontario, CANADA Linux Consulting, Training and Kernel Pedantry. Web page: http://crashcourse.ca Twitter: http://twitter.com/rpjday ======================================================================== From alfps at start.no Thu Jan 21 05:06:39 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 11:06:39 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: Huh? I guess you meant to reply to the OP, not me. Cheers, - Alf * Javier Collado: > Hello, > > I'd say that isn't totally incorrect to use strings instead of > symbols. Please note that in other programming languages symbols, > atoms and the like are in fact immutable strings, which is what python > provides by default. > > Best regards, > Javier > > 2010/1/21 Alf P. Steinbach : >> * Martin Drautzburg: >>> Hello all, >>> >>> When passing parameters to a function, you sometimes need a paramter >>> which can only assume certain values, e.g. >>> >>> def move (direction): >>> ... >>> If direction can only be "up", "down", "left" or "right", you can solve >>> this by passing strings, but this is not quite to the point: >>> >>> - you could pass invalid strings easily >>> - you need to quote thigs, which is a nuisance >>> - the parameter IS REALLY NOT A STRING, but a direction >>> >>> Alternatively you could export such symbols, so when you "import *" you >>> have them available in the caller's namespace. But that forces you >>> to "import *" which pollutes your namespace. >>> >>> What I am really looking for is a way >>> >>> - to be able to call move(up) >>> - having the "up" symbol only in the context of the function call >>> >>> So it should look something like this >>> >>> ... magic, magic ... >>> move(up) >>> ... unmagic, unmagic ... >>> print up >>> >>> This should complain that "up" is not defined during the "print" call, >>> but not when move() is called. And of course there should be as little >>> magic as possible. >>> >>> Any way to achieve this? >>>>> def move( direction ): >> ... print( "move " + str( direction ) ) >> ... >>>>> move( "up" ) >> move up >>>>> class using_directions: >> ... up = 42 >> ... move( up ) >> ... >> move 42 >>>>> up >> Traceback (most recent call last): >> File "", line 1, in >> NameError: name 'up' is not defined >>>>> _ >> >> Of course it's an abuse of the language. :-) >> >> So I wouldn't recommend it, but it's perhaps worth having seen it. >> >> >> Cheers & hth., >> >> - Alf >> >> PS: I hope this was not a homework question. >> -- >> http://mail.python.org/mailman/listinfo/python-list >> From jeanmichel at sequans.com Thu Jan 21 05:20:09 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Thu, 21 Jan 2010 11:20:09 +0100 Subject: why the super class can access the subclass's attribute In-Reply-To: References: Message-ID: <4B582A59.9040604@sequans.com> yousay wrote: > I have sees aprogram like this ,i confusing why super class can access > the subclass's attribute > ,this is the program,thanks in advance: > class MyThread(threading.Thread): > def join(self): > super(MyThread,self).join() > return self.result > > class Worker(MyThread): > import random > import pdb > pdb.set_trace() > def run(self): > total = 0 > for i in range(random.randrange(10,100)): > total +=i > self.result = total > > > I don't thing I got your problem. But in case you are talking about 'result', you could probably say that "the subclass Worker can access to the super class MyThread 'result' attribute". Also keep in mind that there is no private attributes in python, and that any object can access any other object attribute, no matter its class. Of course, you dont want to do that. JM From tompelka at gmail.com Thu Jan 21 05:25:08 2010 From: tompelka at gmail.com (Tomas Pelka) Date: Thu, 21 Jan 2010 11:25:08 +0100 Subject: subprocess troubles Message-ID: <4B582B84.9040009@gmail.com> Hey all, have a problem with following piece of code: -------------------------------------------------- import subprocess paattern = "python" cmd = "/usr/bin/locate" arg1 = " -i" arg2 = " -d /var/www/books/mlocate.db" arg3 = str(" " + pattern) p1 = subprocess.Popen([cmd, arg1, arg2, arg3], shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE) (stdoutdata, stderrdata) = p1.communicate() print p1.returncode print "%s -- %s" % (stdoutdata, stderrdata) -------------------------------------------------- But return code is always 1 and command do not return any result/error (stdoutdata, stderrdata are None). If I run this command (/usr/bin/locate -i -d /var/www/books/mlocate.db python) from standard shell everything goes fine. Could you please give me an advice what I'm doing wrong? Thanks Cheers -- Tom From pavlovevidence at gmail.com Thu Jan 21 05:25:36 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Thu, 21 Jan 2010 02:25:36 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> On Jan 20, 11:43?pm, Martin Drautzburg wrote: > Hello all, > > When passing parameters to a function, you sometimes need a paramter > which can only assume certain values, e.g. > > ? ? ? ? def move (direction): > ? ? ? ? ? ? ? ? ... > If direction can only be "up", "down", "left" or "right", you can solve > this by passing strings, but this is not quite to the point: > > ? ? ? ? - you could pass invalid strings easily > ? ? ? ? - you need to quote thigs, which is a nuisance > ? ? ? ? - the parameter IS REALLY NOT A STRING, but a direction Nothing you can pass to the function is really a direction. A symbol is no more a direction than a string is. > Alternatively you could export such symbols, so when you "import *" you > have them available in the caller's namespace. But that forces you > to "import *" which pollutes your namespace. No it doesn't. You can still write module.UP in the function call, or you can write from module import UP. > What I am really looking for is a way > > ? ? ? ? - to be able to call move(up) > ? ? ? ? - having the "up" symbol only in the context of the function call Short answer is, you can't do it. Long answer is, you don't really want to have a symbol that's only in context during the function call, because you might just want to write some code that operates on directions (e.g., return random.choice ([UP,DOWN]), but it's a moot point since you can't do it. > So it should look something like this > > ... magic, magic ... > move(up) > ... unmagic, unmagic ... > print up > > This should complain that "up" is not defined during the "print" call, > but not when move() is called. And of course there should be as little > magic as possible. > > Any way to achieve this? Apart from writing a custom DSL (domain specific language), no. It is not, in case your were wondering, a feature that would be easily added to Python (the moratorium on new syntax notwithstanding). Python wouldn't be able to do this at compile time (since Python doesn't know what symbols refer to till run time). I convinced myself that it would be techincally possible to do it at run time, but at a penalty of higher function call overhead (which is already high enough as it is). And with the performance hit, extra scoping complexity, the inability to be used outside the function context, and no real benefit except to save typing, I don't see it ever being approved. Carl Banks From alfps at start.no Thu Jan 21 05:38:19 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 11:38:19 +0100 Subject: Symbols as parameters? In-Reply-To: <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> Message-ID: * Carl Banks: > On Jan 20, 11:43 pm, Martin Drautzburg [snip] > >> What I am really looking for is a way >> >> - to be able to call move(up) >> - having the "up" symbol only in the context of the function call > > Short answer is, you can't do it. > On the contrary, it's not difficult to do. I provided an example in my answer to the OP (first reply in the thread). However, it's IMHO an abuse of the language, not something that one should do. Cheers & hth., - Alf From javier.collado at gmail.com Thu Jan 21 05:39:11 2010 From: javier.collado at gmail.com (Javier Collado) Date: Thu, 21 Jan 2010 11:39:11 +0100 Subject: subprocess troubles In-Reply-To: <4B582B84.9040009@gmail.com> References: <4B582B84.9040009@gmail.com> Message-ID: Hello, If you set shell=False, then I think that arg2 should be separated into two different parts. Also, arg3 could be set just to pattern (no need to add extra spaces or using str function). Best regards, Javier 2010/1/21 Tomas Pelka : > Hey all, > > have a problem with following piece of code: > > -------------------------------------------------- > import subprocess > > paattern = "python" > cmd = "/usr/bin/locate" > arg1 = " -i" > arg2 = " -d /var/www/books/mlocate.db" > arg3 = str(" " + pattern) > > p1 = subprocess.Popen([cmd, arg1, arg2, arg3], shell=False, > stdout=subprocess.PIPE, stderr=subprocess.PIPE) > (stdoutdata, stderrdata) = p1.communicate() > > print p1.returncode > print "%s -- %s" % (stdoutdata, stderrdata) > -------------------------------------------------- > > But return code is always 1 and command do not return any result/error > (stdoutdata, stderrdata are None). If I run this command (/usr/bin/locate -i > -d /var/www/books/mlocate.db python) from standard shell everything goes > fine. > > Could you please give me an advice what I'm doing wrong? > > Thanks > Cheers > > -- > Tom > > -- > http://mail.python.org/mailman/listinfo/python-list > From jeanmichel at sequans.com Thu Jan 21 05:43:38 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Thu, 21 Jan 2010 11:43:38 +0100 Subject: deleting, then re-importing a class method In-Reply-To: References: Message-ID: <4B582FDA.2080601@sequans.com> Robert P. J. Day wrote: > (once again, never ashamed to ask the dumb questions.) > > still playing with python3, and testing whether i can > delete/unimport a specific method, then re-import it: > > >>>> import sys >>>> print(sys.__doc__) >>>> > ... blah blah blah ... > >>>> del(sys.__doc__) >>>> print(sys.__doc__) >>>> > module(name[, doc]) > > Create a module object. > The name must be a string; the optional doc argument can have any > type. > > > ok, now that i've deleted just that member of "sys", can i re-import > it? i know this doesn't seem to work: > > >>>> import sys >>>> > > or is there an operator other than "import" that more represents a > full refresh of a class? > > rday > > p.s. no, i don't have a valid application of the above, i'm just > trying to break things. > > -- > > > ======================================================================== > Robert P. J. Day Waterloo, Ontario, CANADA > > Linux Consulting, Training and Kernel Pedantry. > > Web page: http://crashcourse.ca > Twitter: http://twitter.com/rpjday > ======================================================================== > reload(sys) JM From __peter__ at web.de Thu Jan 21 05:51:01 2010 From: __peter__ at web.de (Peter Otten) Date: Thu, 21 Jan 2010 11:51:01 +0100 Subject: deleting, then re-importing a class method References: Message-ID: Robert P. J. Day wrote: > > (once again, never ashamed to ask the dumb questions.) > > still playing with python3, and testing whether i can > delete/unimport a specific method, then re-import it: > >>>> import sys >>>> print(sys.__doc__) > ... blah blah blah ... >>>> del(sys.__doc__) >>>> print(sys.__doc__) > module(name[, doc]) > > Create a module object. > The name must be a string; the optional doc argument can have any > type. >>>> > > ok, now that i've deleted just that member of "sys", can i re-import > it? i know this doesn't seem to work: Actually you haven't: >>> import sys >>> del sys.__doc__ >>> hasattr(sys, "__doc__") True > >>>> import sys > > or is there an operator other than "import" that more represents a > full refresh of a class? imp.reload() > rday > > p.s. no, i don't have a valid application of the above, i'm just > trying to break things. That is indeed likely to happen: Python 3.1.1+ (r311:74480, Nov 2 2009, 15:45:00) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> walk = os.walk >>> del os.walk >>> hasattr(os, "walk") False >>> import imp >>> imp.reload(os) >>> os.walk >>> os.walk is walk False So you have to two distinct walk() functions now. This becomes especially nasty when isinstance(obj, module.Class) tests begin to fail because the module and the class was reloaded, but obj is an instance of module.Class before the reload. Peter From jeanmichel at sequans.com Thu Jan 21 05:57:18 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Thu, 21 Jan 2010 11:57:18 +0100 Subject: Symbols as parameters? In-Reply-To: <1495297.2JfmrJjJTh@beaureve.gmx.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: <4B58330E.5040900@sequans.com> Martin Drautzburg wrote: > Hello all, > > When passing parameters to a function, you sometimes need a paramter > which can only assume certain values, e.g. > > def move (direction): > ... > If direction can only be "up", "down", "left" or "right", you can solve > this by passing strings, but this is not quite to the point: > > - you could pass invalid strings easily > - you need to quote thigs, which is a nuisance > - the parameter IS REALLY NOT A STRING, but a direction > > Alternatively you could export such symbols, so when you "import *" you > have them available in the caller's namespace. But that forces you > to "import *" which pollutes your namespace. > > What I am really looking for is a way > > - to be able to call move(up) > - having the "up" symbol only in the context of the function call > > So it should look something like this > > ... magic, magic ... > move(up) > ... unmagic, unmagic ... > print up > > This should complain that "up" is not defined during the "print" call, > but not when move() is called. And of course there should be as little > magic as possible. > > Any way to achieve this? > > in move.py def move(direction): print "moving %s" % direction move.UP = 'up' move.DOWN = 'down' Then >>> from move import move >>> move(move.UP) moving up JM From jarausch at igpm.rwth-aachen.de Thu Jan 21 06:02:53 2010 From: jarausch at igpm.rwth-aachen.de (Helmut Jarausch) Date: Thu, 21 Jan 2010 12:02:53 +0100 Subject: Best way to convert sequence of bytes to long integer In-Reply-To: References: <00e0377c$0$15570$c3e8da3@news.astraweb.com> <9b870cc2-0966-4cb2-be93-573317c06eb0@a6g2000yqm.googlegroups.com> Message-ID: <7rqqisFn94U1@mid.dfncis.de> On 01/20/10 22:23, Steven D'Aprano wrote: > I'm writing a module that handles, I won't call it encryption, > obfuscation using classical cryptographic algorithms. One of the > functions needs a deterministic but unpredictable integer generated from > a human-generated password or passphrase, so I'm using: > > hashlib.md5(key).digest() > > to get a string of bytes, then converting it to an int. > > int.from_bytes would be perfect for me, but in the meantime I'm using > Paul Rubin's trick of int(s.encode("hex"), 16). > Sorry, but this doesn't work for me (in Python 3.2a0) since hashlib.md5(key).digest() returns a byte string which has no .encode method. Just my 5 cents, Helmut. -- Helmut Jarausch Lehrstuhl fuer Numerische Mathematik RWTH - Aachen University D 52056 Aachen, Germany From anuhacks at gmail.com Thu Jan 21 06:18:07 2010 From: anuhacks at gmail.com (anusha k) Date: Thu, 21 Jan 2010 16:48:07 +0530 Subject: covert number into string Message-ID: <61f9a5fb1001210318r46991edake19f8008711d52e9@mail.gmail.com> Hi, Can anyone tell me how to convert number to words For example: If number =9999 then it should give me *Nine thousand nine hundred ninetynine* Is there any build-in function or something that can do this for me Thank you in advance Anusha Kadambala -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan_ml at behnel.de Thu Jan 21 06:41:04 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 21 Jan 2010 12:41:04 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: <4b583d50$0$6551$9b4e6d93@newsspool4.arcor-online.net> Alf P. Steinbach, 21.01.2010 09:30: > * Martin Drautzburg: >> - to be able to call move(up) >> - having the "up" symbol only in the context of the function call >> >> So it should look something like this >> >> ... magic, magic ... >> move(up) >> ... unmagic, unmagic ... >> print up Looks like a terribly bad design to me. >> This should complain that "up" is not defined during the "print" call, >> but not when move() is called. And of course there should be as little >> magic as possible. >> >> Any way to achieve this? > > >>> def move( direction ): > ... print( "move " + str( direction ) ) > ... > >>> move( "up" ) > move up > >>> > >>> class using_directions: > ... up = 42 > ... move( up ) > ... > move 42 > >>> up > Traceback (most recent call last): > File "", line 1, in > NameError: name 'up' is not defined > >>> _ > > Of course it's an abuse of the language. :-) And, of course, it's totally useless as the move() function doesn't know about the 'up' thing in the first place. Stefan From stefan_ml at behnel.de Thu Jan 21 06:43:08 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 21 Jan 2010 12:43:08 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> Message-ID: <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> Alf P. Steinbach, 21.01.2010 11:38: > * Carl Banks: >> On Jan 20, 11:43 pm, Martin Drautzburg > [snip] >> >>> What I am really looking for is a way >>> >>> - to be able to call move(up) >>> - having the "up" symbol only in the context of the function >>> call >> >> Short answer is, you can't do it. >> > > On the contrary, it's not difficult to do. > > I provided an example in my answer to the OP (first reply in the thread). Erm, no, you didn't. You showed a) how to pass a string constant into a function and b) how to pass a value from a bound variable. None of that is helpful to the OP's problem. Stefan From juhasecke at googlemail.com Thu Jan 21 06:48:28 2010 From: juhasecke at googlemail.com (Jan Ulrich Hasecke) Date: Thu, 21 Jan 2010 12:48:28 +0100 Subject: covert number into string In-Reply-To: <61f9a5fb1001210318r46991edake19f8008711d52e9@mail.gmail.com> References: <61f9a5fb1001210318r46991edake19f8008711d52e9@mail.gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 21.01.10 12:18, anusha k wrote: > Hi, > > Can anyone tell me how to convert number to words > For example: > If number =9999 then it should give me *Nine thousand nine hundred > ninetynine* > Is there any build-in function or something that can do this for me > Thank you in advance > Anusha Kadambala > Hey thats like an exercise in the wonderful German book "Python for Kids". Should be pretty easy to implement it by yourself. Maybe you can understand the code without the german comments. But keep in mind that this is an excercise for kids in chapter 5. I am sure there are better implementations. # Python f?r Kids -- 3. Auflage, Kapitel 5 # Autor: Gregor Lingl # Datum: 01. 03. 2004 # Loesung von Kapitel 5, Aufgabe 3: def zahlwort(zahl): einer = zahl % 10 # Einerstelle: Rest bei der Division von zahl durch 10 if einer == 1: e = "ein" elif einer == 2: e = "zwei" elif einer == 3: e = "drei" elif einer == 4: e = "vier" elif einer == 5: e = "f?nf" elif einer == 6: e = "sechs" elif einer == 7: e = "sieben" elif einer == 8: e = "acht" elif einer == 9: e = "neun" zehner = zahl // 10 # Zehnerstelle: Ergebnis der Ganzzahldivision # von zahl durch 10 if zehner == 0: z = "" elif zehner == 1: z = "zehn" elif zehner == 2: z = "zwanzig" elif zehner == 3: z = "dreissig" elif zehner == 4: z = "vierzig" elif zehner == 5: z = "f?nfzig" elif zehner == 6: z = "sechzig" elif zehner == 7: z = "siebzig" elif zehner == 8: z = "achtzig" elif zehner == 9: z ="neunzig" if zahl == 11: print "elf" elif zahl == 12: print "zw?lf" elif zahl == 17: print "siebzehn" elif einer == 0: print z elif zehner == 1: print e+z else: print e+"und"+z zahl = raw_input("Gib eine zweistellige ganze Zahl ein: ") zahl = int(zahl) # macht aus dem String zahl eine ganze Zahl zahlwort(zahl) print Cheers! juh -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.10 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAktYPwwACgkQPUzUEFbILMRLWwCgjugY7Wxzec7AH8Esz1hNIf8d WgoAoKgFo59NoBVk0jGZsqe5slOc7P9W =cP1x -----END PGP SIGNATURE----- From alfps at start.no Thu Jan 21 06:57:41 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 12:57:41 +0100 Subject: Symbols as parameters? In-Reply-To: <4b583d50$0$6551$9b4e6d93@newsspool4.arcor-online.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <4b583d50$0$6551$9b4e6d93@newsspool4.arcor-online.net> Message-ID: * Stefan Behnel: > Alf P. Steinbach, 21.01.2010 09:30: >> * Martin Drautzburg: >>> - to be able to call move(up) >>> - having the "up" symbol only in the context of the function call >>> >>> So it should look something like this >>> >>> ... magic, magic ... >>> move(up) >>> ... unmagic, unmagic ... >>> print up > > Looks like a terribly bad design to me. That it is, in Python. >>> This should complain that "up" is not defined during the "print" call, >>> but not when move() is called. And of course there should be as little >>> magic as possible. >>> >>> Any way to achieve this? >>>>> def move( direction ): >> ... print( "move " + str( direction ) ) >> ... >>>>> move( "up" ) >> move up >>>>> class using_directions: >> ... up = 42 >> ... move( up ) >> ... >> move 42 >>>>> up >> Traceback (most recent call last): >> File "", line 1, in >> NameError: name 'up' is not defined >>>>> _ >> Of course it's an abuse of the language. :-) > > And, of course, it's totally useless as the move() function doesn't know > about the 'up' thing in the first place. Hm, it seems as if you thought that the example was *literally* what the OP should do. It was not: it just demonstrated the possibility. Consider replacing the assignment with whatever, or alternatively, consider that perhaps 42 is indeed the value of "up" that the move() function knows about. Cheers & hth., - Alf From alfps at start.no Thu Jan 21 06:58:51 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 12:58:51 +0100 Subject: Symbols as parameters? In-Reply-To: <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> Message-ID: * Stefan Behnel: > Alf P. Steinbach, 21.01.2010 11:38: >> * Carl Banks: >>> On Jan 20, 11:43 pm, Martin Drautzburg >> [snip] >>>> What I am really looking for is a way >>>> >>>> - to be able to call move(up) >>>> - having the "up" symbol only in the context of the function >>>> call >>> Short answer is, you can't do it. >>> >> On the contrary, it's not difficult to do. >> >> I provided an example in my answer to the OP (first reply in the thread). > > Erm, no, you didn't. You showed a) how to pass a string constant into a > function and b) how to pass a value from a bound variable. None of that is > helpful to the OP's problem. Perhaps look again at that example. It demonstrates exactly what the OP asks for. Cheers & hth., - Alf From ndbecker2 at gmail.com Thu Jan 21 07:13:48 2010 From: ndbecker2 at gmail.com (Neal Becker) Date: Thu, 21 Jan 2010 07:13:48 -0500 Subject: multiprocessing as batch system Message-ID: I'm using multiprocessing as a poor man's batch system. It is working OK, except that it doesn't seem to do load balancing quite right. I have a 8-cpu machine. If I start, using multiprocessing pool, calling map with more than 8 elements, it will start 8 processes. It seems, though, that when some of them finish, it does not start others right away. The load average may be a lot less than 8 but still additional processes are not active. Is there a way to use multiprocessing to achieve the effect I want? (I really want a simple batch system, but without complex setup) From joel.goldstick at columbuswebmakers.com Thu Jan 21 07:49:33 2010 From: joel.goldstick at columbuswebmakers.com (Joel Goldstick) Date: Thu, 21 Jan 2010 07:49:33 -0500 Subject: covert number into string In-Reply-To: <61f9a5fb1001210318r46991edake19f8008711d52e9@mail.gmail.com> References: <61f9a5fb1001210318r46991edake19f8008711d52e9@mail.gmail.com> Message-ID: <4B584D5D.5090209@columbuswebmakers.com> anusha k wrote: > Hi, > > Can anyone tell me how to convert number to words > For example: > If number =9999 then it should give me *Nine thousand nine hundred > ninetynine* > Is there any build-in function or something that can do this for me > Thank you in advance > Anusha Kadambala > > This might help: http://snippets.dzone.com/posts/show/704 From tompelka at gmail.com Thu Jan 21 08:27:19 2010 From: tompelka at gmail.com (Tomas Pelka) Date: Thu, 21 Jan 2010 14:27:19 +0100 Subject: subprocess troubles In-Reply-To: References: <4B582B84.9040009@gmail.com> Message-ID: <4B585637.1040300@gmail.com> On 01/21/2010 11:39 AM, Javier Collado wrote: > Hello, > > If you set shell=False, then I think that arg2 should be separated > into two different parts. > > Also, arg3 could be set just to pattern (no need to add extra spaces > or using str function). > > Best regards, > Javier > > 2010/1/21 Tomas Pelka: > >> Hey all, >> >> have a problem with following piece of code: >> >> -------------------------------------------------- >> import subprocess >> >> paattern = "python" >> cmd = "/usr/bin/locate" >> arg1 = " -i" >> arg2 = " -d /var/www/books/mlocate.db" >> arg3 = str(" " + pattern) >> >> p1 = subprocess.Popen([cmd, arg1, arg2, arg3], shell=False, >> stdout=subprocess.PIPE, stderr=subprocess.PIPE) >> (stdoutdata, stderrdata) = p1.communicate() >> >> print p1.returncode >> print "%s -- %s" % (stdoutdata, stderrdata) >> -------------------------------------------------- >> >> But return code is always 1 and command do not return any result/error >> (stdoutdata, stderrdata are None). If I run this command (/usr/bin/locate -i >> -d /var/www/books/mlocate.db python) from standard shell everything goes >> fine. >> >> Could you please give me an advice what I'm doing wrong? >> >> Thanks >> Cheers >> >> -- >> Tom >> >> -- >> http://mail.python.org/mailman/listinfo/python-list >> >> Thanks Javier for advice, but sill same result. I'm running this code as cgi script from apache. Weird is that when i run it from shell as apache user, like ---------------- # su -s /bin/bash -c "/usr/bin/locate -i -d /var/www/books/mlocate.db python; echo $?" apache 0 --------------- I always get "0", but as cgi it returns "1". When I run this script by other user (tom), I'll obtain nonzero output what is OK. Additional info: # su -s /bin/bash -c "ls -l /var/www/books/mlocate.db" apache -rw-rw-r-- 1 tom books 1465653 Jan 20 13:33 /var/www/books/mlocate.db so db is readable by apache Whore source attached. -- Tom -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: search.py URL: From tim at reportlab.com Thu Jan 21 08:40:27 2010 From: tim at reportlab.com (Tim) Date: Thu, 21 Jan 2010 05:40:27 -0800 (PST) Subject: ReportLab PDF Toolkit v2.4 released Message-ID: We're pleased to announce the latest version of the ReportLab open source PDF toolkit, now available for download here: https://www.reportlab.com/software/opensource/rl-toolkit/download/ The ReportLab Toolkit is a library for programatically creating documents in PDF format. It's free, open-source software written in Python, and released under a BSD type license. Thanks, -The ReportLab Team

ReportLab Toolkit v2.4 - The Open Source Library for creating PDF Documents. (21-Jan-2010)

From wbsoft at xs4all.nl Thu Jan 21 09:18:54 2010 From: wbsoft at xs4all.nl (Wilbert Berendsen) Date: Thu, 21 Jan 2010 15:18:54 +0100 Subject: substitution In-Reply-To: References: Message-ID: <201001211518.55011.wbsoft@xs4all.nl> Op maandag 18 januari 2010 schreef Adi: > keys = [(len(key), key) for key in mapping.keys()] > keys.sort(reverse=True) > keys = [key for (_, key) in keys] > > pattern = "(%s)" % "|".join(keys) > repl = lambda x : mapping[x.group(1)] > s = "fooxxxbazyyyquuux" > > re.subn(pattern, repl, s) I managed to make it even shorted, using the key argument for sorted, not putting the whole regexp inside parentheses and pre-compiling the regular expression: import re mapping = { "foo" : "bar", "baz" : "quux", "quuux" : "foo" } # sort the keys, longest first, so 'aa' gets matched before 'a', because # in Python regexps the first match (going from left to right) in a # |-separated group is taken keys = sorted(mapping.keys(), key=len) rx = re.compile("|".join(keys)) repl = lambda x: mapping[x.group()] s = "fooxxxbazyyyquuux" rx.sub(repl, s) One thing remaining: if the replacement keys could contain non-alphanumeric characters, they should be escaped using re.escape: rx = re.compile("|".join(re.escape(key) for key in keys)) Met vriendelijke groet, Wilbert Berendsen -- http://www.wilbertberendsen.nl/ "You must be the change you wish to see in the world." -- Mahatma Gandhi From gnarlodious at gmail.com Thu Jan 21 09:23:17 2010 From: gnarlodious at gmail.com (Gnarlodious) Date: Thu, 21 Jan 2010 06:23:17 -0800 (PST) Subject: Create object name from string value? References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> Message-ID: <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> On Jan 20, 10:35?pm, Steven D'Aprano wrote: > That's the wrong way to handle the problem. Named objects are only useful > if you know the name of the object when writing the code. Otherwise, how > do you know what name to use in the code? Thank you for the help. I am gathering the names of all *.plist files in a folder, creating objects named the filename, and accessing the data like this: Data.Server.Config.BaseURL > http://Spectrumology.com/ Adding a .plist file would automatically create a plist dictionary object inside the Data module. > The right way to solve this problem is with a dictionary: > > for name in ["object1", "object2", "object3"]: > ? ? d = {name: classname()} > ? ? print d[name] This works! However I end up saying: d['Server'].Config.BaseURL to get the data, when I should be saying: Server.Config.BaseURL > but for the record, the way to use exec is like this: > > exec("object1 = classname()") I failed to make that work. So back to the original question. How to make an instance named according to a string inside a variable? I guess it should be in the top-level namespace, not inside a list or dictionary. -- Gnarlie http://Gnarlodious.com/Gnarlodious From phil at riverbankcomputing.com Thu Jan 21 09:26:55 2010 From: phil at riverbankcomputing.com (Phil Thompson) Date: Thu, 21 Jan 2010 14:26:55 +0000 Subject: SIP v4.10 Released (Python Bindings Generator) Message-ID: SIP v4.10 has been released and can be downloaded from http://www.riverbankcomputing.com/software/sip/. SIP is a tool for generating Python modules that wrap C or C++ libraries. It is similar to SWIG. It is used to generate PyQt and PyKDE. The SIP license is similar to the Python License and is also licensed under the GPL v2 and v3. SIP runs on Windows, UNIX, Linux and MacOS/X. SIP requires Python v2.3 or later and fully supports Python v3. This release adds support for keyword arguments and docstrings. Docstrings may be either explictly specified or automatically generated. Automatically generated docstrings describe the Python signatures of all available overloads for a callable. A significantly improved error reporting mechanism uses those docstrings in exceptions raised when arguments with incorrect types are passed. Other features of SIP include: - extension modules are implemented as a single binary .pyd or .so file (no Python stubs) - support for Python new-style classes - the ability to specify the super-type and meta-type used to wrap instances - generated modules are quick to import, even for large libraries - thread support - the ability to re-implement C++ abstract and virtual methods in Python - the ability to define Python classes that derive from abstract C++ classes - the ability to spread a class hierarchy across multiple Python modules - the ability to wrap a C++ class in different ways and allow an application to select a particular implementation at run-time - support for C++ namespaces - support for C++ exceptions - support for C++ operators - an extensible build system written in Python that supports over 50 platform/compiler combinations - the generation of API files for IDEs that support autocompletion and call tips. From python at mrabarnett.plus.com Thu Jan 21 09:42:02 2010 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 21 Jan 2010 14:42:02 +0000 Subject: substitution In-Reply-To: <201001211518.55011.wbsoft@xs4all.nl> References: <201001211518.55011.wbsoft@xs4all.nl> Message-ID: <4B5867BA.4070804@mrabarnett.plus.com> Wilbert Berendsen wrote: > Op maandag 18 januari 2010 schreef Adi: >> keys = [(len(key), key) for key in mapping.keys()] >> keys.sort(reverse=True) >> keys = [key for (_, key) in keys] >> >> pattern = "(%s)" % "|".join(keys) >> repl = lambda x : mapping[x.group(1)] >> s = "fooxxxbazyyyquuux" >> >> re.subn(pattern, repl, s) > > I managed to make it even shorted, using the key argument for sorted, not > putting the whole regexp inside parentheses and pre-compiling the regular > expression: > > import re > > mapping = { > "foo" : "bar", > "baz" : "quux", > "quuux" : "foo" > } > > # sort the keys, longest first, so 'aa' gets matched before 'a', because > # in Python regexps the first match (going from left to right) in a > # |-separated group is taken > keys = sorted(mapping.keys(), key=len) > For longest first you need: keys = sorted(mapping.keys(), key=len, reverse=True) > rx = re.compile("|".join(keys)) > repl = lambda x: mapping[x.group()] > s = "fooxxxbazyyyquuux" > rx.sub(repl, s) > > One thing remaining: if the replacement keys could contain non-alphanumeric > characters, they should be escaped using re.escape: > Strictly speaking, not all non-alphanumeric characters, but only the special ones. > rx = re.compile("|".join(re.escape(key) for key in keys)) > From daniel at stutzbachenterprises.com Thu Jan 21 09:42:26 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Thu, 21 Jan 2010 08:42:26 -0600 Subject: Sorted dictionary In-Reply-To: <1c74df0b-b22e-4aea-934a-c6d25e818f41@e37g2000yqn.googlegroups.com> References: <1c74df0b-b22e-4aea-934a-c6d25e818f41@e37g2000yqn.googlegroups.com> Message-ID: On Thu, Jan 21, 2010 at 2:27 AM, Raymond Hettinger wrote: > Using an underlying list to track sorted items > means that insertion and deletion take O(n) time. > That could be reduced to O(log n) time by using > a blist or skiplist as the underlying structure > for maintaining a sort. > Indeed. In fact, blist 1.1 (to be released within a month or so) will include sorteddict, sortedset, sortedlist, weaksortedset, and weaksortedlist types. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From bouncyinc at gmail.com Thu Jan 21 09:42:38 2010 From: bouncyinc at gmail.com (John Haggerty) Date: Thu, 21 Jan 2010 07:42:38 -0700 Subject: creating a python program to control the cursor and click a location on the screen at specified delay interval? Message-ID: Hi....So just testing the feasibility of doing this but I was interested in creating a primitive way of scripting gui applications w/o using a remote network client by having the application up and then almost just using the pointer position to figure out where it is and then click it and any subsequent dialogs. I would have to assume that someone has done something like this. In principle this could be done I don't know if this is possible to implement in python cross platform but the system I am using is just your standard Ubuntu Linux 9.10 system. Would be interesting to see something already in existance however to avoid duplication of work. -------------- next part -------------- An HTML attachment was scrubbed... URL: From alfps at start.no Thu Jan 21 09:43:05 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 15:43:05 +0100 Subject: Create object name from string value? In-Reply-To: <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> Message-ID: * Gnarlodious: > On Jan 20, 10:35 pm, Steven D'Aprano wrote: > >> That's the wrong way to handle the problem. Named objects are only useful >> if you know the name of the object when writing the code. Otherwise, how >> do you know what name to use in the code? > > Thank you for the help. I am gathering the names of all *.plist files > in a folder, creating objects named the filename, and accessing the > data like this: > > Data.Server.Config.BaseURL >> http://Spectrumology.com/ > > Adding a .plist file would automatically create a plist dictionary > object inside the Data module. > >> The right way to solve this problem is with a dictionary: >> >> for name in ["object1", "object2", "object3"]: >> d = {name: classname()} >> print d[name] > > This works! However I end up saying: > > d['Server'].Config.BaseURL > > to get the data, when I should be saying: > > Server.Config.BaseURL > >> but for the record, the way to use exec is like this: >> >> exec("object1 = classname()") > > I failed to make that work. So back to the original question. How to > make an instance named according to a string inside a variable? I > guess it should be in the top-level namespace, not inside a list or > dictionary. I don't understand how you intend to use a variable whose name comes from dynamic data. So I agree with Steven that it's the wrong way to handle the problem -- whatever the problem is! But, if it can help: >>> import __main__ >>> setattr( __main__, "blah", 123 ) >>> blah 123 >>> _ Cheers, - Alf From phil at riverbankcomputing.com Thu Jan 21 09:46:57 2010 From: phil at riverbankcomputing.com (Phil Thompson) Date: Thu, 21 Jan 2010 14:46:57 +0000 Subject: PyQt v4.7 Released Message-ID: <99dcfcfa3cc9487368d1684816771153@localhost> PyQt v4.7 has been released and is available from http://www.riverbankcomputing.com/software/pyqt/. PyQt is a comprehensive set of bindings for the Qt application and UI framework from Nokia. It supports the same platforms as Qt (Windows, Linux and MacOS/X). PyQt supports Python v3 and Python v2.3 and later. The highlights of this release include: - full support for Qt v4.6.1 including the new animation and state machine frameworks, gesture and multi-touch support, and advanced graphics effects (blurring, colourising, drop shadows) - all callables have docstrings that describe the Python signatures of all available overloads - keyword arguments are supported for all optional arguments. Windows installers are provided for the GPL version of PyQt which contains everything needed for PyQt development (including Qt, Qt Designer, QScintilla, and MySQL, PostgreSQL, SQLite and ODBC drivers) except Python itself. PyQt v4 is implemented as a set of 19 extension modules containing over 400 classes and over 6,000 functions and methods. QtCore The non-GUI infrastructure including event loops, threads, i18n, Unicode, signals and slots, user and application settings, mapped files and shared memory. QtDesigner A set of classes that allow the Qt Designer GUI design tool to be extended with PyQt. QtGui A rich collection of GUI widgets. QtHelp A set of classes for creating and viewing searchable documentation and being able to integrate online help with PyQt applications. It includes the C++ port of the Lucene text search engine. QtNetwork A set of classes to support TCP and UDP socket programming and higher level protocols (eg. HTTP, SSL). QtOpenGL A set of classes that allows PyOpenGL to render onto Qt widgets. QtScript A set of classes that implements a JavaScript interpreter. Python objects may be exposed in the interpreter as JavaScript objects. QtScriptTools A debugger for the JavaScript interpreter. QtSql A set of classes that implement SQL data models and interfaces to industry standard databases. The Windows installers include support for SQLite, MySQL, PostgreSQL and ODBC. QtSvg A set of classes to render SVG files onto Qt widgets. QtTest A set of classes to automate unit testing of PyQt applications and GUIs. QtWebKit This implements a web browser engine based on the WebKit engine used by Apple's Safari browser. It allows the methods and properties of Python objects to be published and appear as JavaScript objects to scripts embedded in HTML pages. QtXML A set of classes that implement DOM and SAX parsers. QtXMLPatterns A set of classes that implement XQuery and XPath support for XML and custom data models. QtAssistant A set of classes that enables the Qt Assistant online help browser to be integrated with an application. QAxContainer A set of classes for Windows that allows the integration of ActiveX controls and COM objects. phonon A cross-platform multimedia framework that enables the use of audio and video content in PyQt applications. DirectX is used as the Windows backend, QuickTime as the MacOS/X backend, and GStreamer as the Linux backend. QtMultimedia A set of classes that provide low-level multimedia functions. Application developers would normally use the phonon module. DBus PyQt includes dbus.mainloop.qt that allows the Qt event loop to be used with the standard DBus Python bindings. PyQt includes the pyuic4 utility which generates Python code to implement user interfaces created with Qt Designer in the same way that the uic utility generates C++ code. It is also able to load Designer XML files dynamically. PyQt is available under the GPL and a commercial license. Unlike Qt, PyQt is not available under the LGPL. The commercial PyQt license allows GPL applications to be relicensed at any time. From iainking at gmail.com Thu Jan 21 09:58:14 2010 From: iainking at gmail.com (Iain King) Date: Thu, 21 Jan 2010 06:58:14 -0800 (PST) Subject: substitution References: Message-ID: On Jan 21, 2:18?pm, Wilbert Berendsen wrote: > Op maandag 18 januari 2010 schreef Adi: > > > keys = [(len(key), key) for key in mapping.keys()] > > keys.sort(reverse=True) > > keys = [key for (_, key) in keys] > > > pattern = "(%s)" % "|".join(keys) > > repl = lambda x : mapping[x.group(1)] > > s = "fooxxxbazyyyquuux" > > > re.subn(pattern, repl, s) > > I managed to make it even shorted, using the key argument for sorted, not > putting the whole regexp inside parentheses and pre-compiling the regular > expression: > > import re > > mapping = { > ? ? ? ? "foo" : "bar", > ? ? ? ? "baz" : "quux", > ? ? ? ? "quuux" : "foo" > > } > > # sort the keys, longest first, so 'aa' gets matched before 'a', because > # in Python regexps the first match (going from left to right) in a > # |-separated group is taken > keys = sorted(mapping.keys(), key=len) > > rx = re.compile("|".join(keys)) > repl = lambda x: mapping[x.group()] > s = "fooxxxbazyyyquuux" > rx.sub(repl, s) > > One thing remaining: if the replacement keys could contain non-alphanumeric > characters, they should be escaped using re.escape: > > rx = re.compile("|".join(re.escape(key) for key in keys)) > > Met vriendelijke groet, > Wilbert Berendsen > > --http://www.wilbertberendsen.nl/ > "You must be the change you wish to see in the world." > ? ? ? ? -- Mahatma Gandhi Sorting it isn't the right solution: easier to hold the subs as tuple pairs and by doing so let the user specify order. Think of the following subs: "fooxx" -> "baz" "oxxx" -> "bar" does the user want "bazxbazyyyquuux" or "fobarbazyyyquuux"? Iain From wbsoft at xs4all.nl Thu Jan 21 10:08:21 2010 From: wbsoft at xs4all.nl (Wilbert Berendsen) Date: Thu, 21 Jan 2010 16:08:21 +0100 Subject: substitution In-Reply-To: <4B5867BA.4070804@mrabarnett.plus.com> References: <201001211518.55011.wbsoft@xs4all.nl> <4B5867BA.4070804@mrabarnett.plus.com> Message-ID: <201001211608.21475.wbsoft@xs4all.nl> Op donderdag 21 januari 2010 schreef MRAB: > For longest first you need: > > keys = sorted(mapping.keys(), key=len, reverse=True) Oh yes, I cut/pasted the wrong line :-) Just for clarity: import re mapping = { "foo" : "bar", "baz" : "quux", "quuux" : "foo" } # sort the keys, longest first, so 'aa' gets matched before 'a', because # in Python regexps the first match (going from left to right) in a # |-separated group is taken keys = sorted(mapping.keys(), key=len, reverse=True) rx = re.compile("|".join(keys)) repl = lambda x: mapping[x.group()] s = "fooxxxbazyyyquuux" rx.sub(repl, s) >> One thing remaining: if the replacement keys could contain non-alphanumeric >> characters, they should be escaped using re.escape: >> rx = re.compile("|".join(re.escape(key) for key in keys)) >> >Strictly speaking, not all non-alphanumeric characters, but only the >special ones. True, although the re.escape function simply escapes all non-alphanumeric characters :) And here is a factory function that returns a translator given a mapping. The translator can be called to perform replacements in a string: import re def translator(mapping): keys = sorted(mapping.keys(), key=len, reverse=True) rx = re.compile("|".join(keys)) repl = lambda m: mapping[m.group()] return lambda s: rx.sub(repl, s) #Usage: >>> t = translator(mapping) >>> t('fooxxxbazyyyquuux') 'barxxxquuxyyyfoo' w best regards, Wilbert Berendsen -- http://www.wilbertberendsen.nl/ From python.list at tim.thechases.com Thu Jan 21 10:32:52 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 21 Jan 2010 09:32:52 -0600 Subject: Create object name from string value? In-Reply-To: <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> Message-ID: <4B5873A4.6090707@tim.thechases.com> Gnarlodious wrote: >> for name in ["object1", "object2", "object3"]: >> d = {name: classname()} >> print d[name] > > This works! However I end up saying: > > d['Server'].Config.BaseURL > > to get the data, when I should be saying: > > Server.Config.BaseURL It sounds like you want a mapping of strings to class objects (not instances), so you can do something like mapping = { "object1": object1, "object2": some_module.object2, "object3": other_module.namespace.object3, } MyClass = mapping["object1"] server = MyClass(initialization_params) # the above two lines can be written as # server = mapping["object1"](init_params) print server.Config.BaseURL -tkc From jlconlin at gmail.com Thu Jan 21 10:41:20 2010 From: jlconlin at gmail.com (Jeremy) Date: Thu, 21 Jan 2010 07:41:20 -0800 (PST) Subject: distutils not finding all of my pure python modules Message-ID: <543c08a0-30c8-49fc-882f-212e099bbd96@h9g2000yqa.googlegroups.com> I have a small set of Python packages/modules that I am putting together. I'm having trouble in that when I run python setup.py sdist I don't get all of my pure python modules. The setup.py script I use is: # ===================================== from distutils.core import setup purePythonModules = ['regex', 'gnuFile'] setup(name='PythonForSafeguards', version='0.9.1', description = 'Python code for MCNP and Safeguards analysis.', author = 'Jake the Snake', author_email = 'something at blah.com', packages = ['MCNP', 'Safeguards'], url='http://lanl.gov', py_modules = purePythonModules, ) # ========================================= Everythin seems to work fine except the gnuFile.py script does not get put into the distribution. I know the file exists in the same directory as regex.py and has the same permissions. Does anyone know what is going on here? I'm using Python 2.6.4. Thanks, Jeremy From davea at ieee.org Thu Jan 21 11:01:18 2010 From: davea at ieee.org (Dave Angel) Date: Thu, 21 Jan 2010 11:01:18 -0500 Subject: Symbols as parameters? In-Reply-To: <1495297.2JfmrJjJTh@beaureve.gmx.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: <4B587A4E.9050508@ieee.org> Martin Drautzburg wrote: > Hello all, > > When passing parameters to a function, you sometimes need a paramter > which can only assume certain values, e.g. > > def move (direction): > ... > If direction can only be "up", "down", "left" or "right", you can solve > this by passing strings, but this is not quite to the point: > > - you could pass invalid strings easily > - you need to quote thigs, which is a nuisance > - the parameter IS REALLY NOT A STRING, but a direction > > Alternatively you could export such symbols, so when you "import *" you > have them available in the caller's namespace. But that forces you > to "import *" which pollutes your namespace. > > What I am really looking for is a way > > - to be able to call move(up) > - having the "up" symbol only in the context of the function call > > So it should look something like this > > ... magic, magic ... > move(up) > ... unmagic, unmagic ... > print up > > This should complain that "up" is not defined during the "print" call, > but not when move() is called. And of course there should be as little > magic as possible. > > Any way to achieve this? > > > My favorite answer provided by others was adding the attributes to the function right after the function definition, then using move(move.up) But another approach is to define the function as: def move(up=False, down=False, right=False, left=False): if up: ... if down: .... and call it as: move(up=True) DaveA From awilliam at opengroupware.us Thu Jan 21 11:12:22 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Thu, 21 Jan 2010 11:12:22 -0500 Subject: Create object name from string value? In-Reply-To: <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> Message-ID: <1264090342.8831.7.camel@linux-m3mt> > > but for the record, the way to use exec is like this: > > exec("object1 = classname()") > I failed to make that work. So back to the original question. How to > make an instance named according to a string inside a variable? I > guess it should be in the top-level namespace, not inside a list or > dictionary. I think what you want is more like: classclass = eval(classname) x = classclass(...args...) -- OpenGroupware developer: awilliam at whitemice.org OpenGroupare & Cyrus IMAPd documenation @ From awilliam at opengroupware.us Thu Jan 21 11:16:31 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Thu, 21 Jan 2010 11:16:31 -0500 Subject: multiprocessing as batch system In-Reply-To: References: Message-ID: <1264090591.8831.10.camel@linux-m3mt> On Thu, 2010-01-21 at 07:13 -0500, Neal Becker wrote: > I'm using multiprocessing as a poor man's batch system. It is working OK, > except that it doesn't seem to do load balancing quite right. > I have a 8-cpu machine. If I start, using multiprocessing pool, calling map > with more than 8 elements, it will start 8 processes. It seems, though, > that when some of them finish, it does not start others right away. The > load average may be a lot less than 8 but still additional processes are not > active. If your processes are I/O bound your system will never appear to be saturated. Multiple pending processes and a low load average usually indicated waiting on some other resource. > Is there a way to use multiprocessing to achieve the effect I want? (I > really want a simple batch system, but without complex setup) Well, honestly, that isn't possible (unless you just choose to ignore all the things that can go wrong: crashed worked, hung workers, etc...) -- OpenGroupware developer: awilliam at whitemice.org OpenGroupare & Cyrus IMAPd documenation @ From davea at ieee.org Thu Jan 21 11:21:26 2010 From: davea at ieee.org (Dave Angel) Date: Thu, 21 Jan 2010 11:21:26 -0500 Subject: Create object name from string value? In-Reply-To: <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> Message-ID: <4B587F06.2070904@ieee.org> Gnarlodious wrote: > On Jan 20, 10:35 pm, Steven D'Aprano wrote: > > >> That's the wrong way to handle the problem. Named objects are only useful >> if you know the name of the object when writing the code. Otherwise, how >> do you know what name to use in the code? >> > > Thank you for the help. I am gathering the names of all *.plist files > in a folder, creating objects named the filename, and accessing the > data like this: > > Data.Server.Config.BaseURL > >> http://Spectrumology.com/ >> > > Adding a .plist file would automatically create a plist dictionary > object inside the Data module. > > >> The right way to solve this problem is with a dictionary: >> >> for name in ["object1", "object2", "object3"]: >> d =name: classname()} >> print d[name] >> > > This works! However I end up saying: > > d['Server'].Config.BaseURL > > to get the data, when I should be saying: > > Server.Config.BaseURL > > >> but for the record, the way to use exec is like this: >> >> exec("object1 =lassname()") >> > > I failed to make that work. So back to the original question. How to > make an instance named according to a string inside a variable? I > guess it should be in the top-level namespace, not inside a list or > dictionary. > > -- Gnarlie > http://Gnarlodious.com/Gnarlodious > > I know you figure you have a specific list of files, and that they'll never conflict with the other global variables you've defined. But adding globals dynamically is "magic", and can lead to very obscure bugs. Suppose sometime in the future somebody adds another plist file with the same name as one of your functions? Put it inside a dummy class, as follows: >>> class Plist: pass ... >>> plists = Plist() >>> setattr(plists, "Server", 42) >>> plists.Server 42 >>> That avoids the ["Server"] syntax, but still doesn't risk polluting your global namespace with arbitrary names. And in the example above, you'd be using plists.Server.Config.BaseURL And if you must use the global namespace, you can simply do: >>> globals()["Server"] = 42 >>> Server 42 DaveA From apt.shansen at gmail.com Thu Jan 21 11:54:43 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Thu, 21 Jan 2010 08:54:43 -0800 Subject: Create object name from string value? In-Reply-To: <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> Message-ID: <7a9c25c21001210854o542fd9c6k9409fced12f42242@mail.gmail.com> On Thu, Jan 21, 2010 at 6:23 AM, Gnarlodious wrote: > This works! However I end up saying: > > d['Server'].Config.BaseURL > > to get the data, when I should be saying: > > Server.Config.BaseURL That's the thing: you should not actually be saying that. d['Server'].Config.BaseURL is precisely what you should be saying. You need to adjust your expectations to the language you are using and how it operates. Whatever aesthetic sense in you that is objecting to using a dictionary is something you just need to get past. If you don't like the dictionary getitem syntax, you can always do: >>> class AttrDict(dict): ... def __getattr__(self, key): ... return self[key] ... >>> a = AttrDict() >>> a["test"] = 1 >>> a.test 1 Is there any real reason you can't write Servers["Server"].Config.BaseURL or Servers.Server.Config.BaseURL, or is it just an itch that's bugging you? If its the latter, then it's better to just not scratch it and it'll go away when you get more used to the language :) Its not easy to do what you're asking for. People ask for it every once in awhile. But it's just not the Pythonic way to approach the problem, and while there are things you can do to accomplish it, they're hacks and are not guaranteed to work consistently. Its not even a oversight in Python, something the devs never considered: its intentionally designed to be this way. How come a dictionary (or class as above) isn't good enough? --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From showell30 at yahoo.com Thu Jan 21 11:54:44 2010 From: showell30 at yahoo.com (Steve Howell) Date: Thu, 21 Jan 2010 08:54:44 -0800 (PST) Subject: simple pub/sub Message-ID: <0af0eff2-50e2-44aa-81f1-b3d12cb265a7@a15g2000yqm.googlegroups.com> Hi, I'm looking for ideas on building a simple architecture that allows a bunch of independent Python processes to exchange data using files and perform calculations. One Python program would be collecting data from boat instruments on a serial port, then writing that info out to a file, and also transmitting data to instruments as needed (again through the serial port). It would be the most complex program in terms of having to poll file descriptors, or something similar, but I want to limit its job to pure communication handling. Then, I would have a bunch of programs doing various calculations on the input stream. It's totally acceptable if they just sleep a bit, wake up, see if there is any new data, and if so, do some calculations, so I will probably just use this: http://www.dabeaz.com/generators/follow.py Some of the "calculator" programs might be looking for trends in the data, and when a certain threshold is reached, they will need to notify the comm program of the new message. To avoid collisions between all the different calculator programs, I am thinking the simplest thing is just have them each write to their own output file. So then the comm program has multiple data sources. It wants to track to the serial port, but it also wants to monitor the output files from the various calculator processes. I want to do this in a mostly nonblocking way, so I would probably poll on the file descriptors. The only problems are that select "...cannot be used on regular files to determine whether a file has grown since it was last read," and it only works on Unix for files. (I'm using Unix myself, but one of the other programmers insists on Windows, and is resisting Python as well.) So a variation would be that the calculator programs talk to the comm program via sockets, which seems slightly on the heavy side, but it is certainly feasible. I know that something like twisted could probably solve my problem, but I'm also wondering if there is some solution that is a little more lightweight and batteries-included. At the heart of the matter, I just want the programs to be able to do the following tasks. 1) All programs will occasionally need to write a line of text to some data stream without having to lock it. (I don't care if the write blocks.) 2) All programs will need to be able to do a non-blocking read of a line of text from a data stream. (And there may be multiple consumers.) 3) The comm program will need to be able to poll the serial port and input data streams to see which ones are ready. Any thoughts? From zachar at awst.at Thu Jan 21 11:56:55 2010 From: zachar at awst.at (zachar at awst.at) Date: Thu, 21 Jan 2010 17:56:55 +0100 (CET) Subject: =?utf-8?Q?Python_sys=2Eprefix?= Message-ID: Hi All, I need some help about my "problem": I compiled and installed python with ./configure --prefix=/opt/pyhon2.6.4... But I have to move it to a different directory (the reason is very difficult but I must do this)... I started the interpreter from shell and I was surprised because It still worked and It changed the sys.prefix so it pointed to the correct place so the sys.path was correct.... The questions are: HOW? Is it a feature which is not documented (at least I couldn't find anything about it)? >From the official python documentation (http://docs.python.org/library/sys.html?highlight=sys#sys.prefix): "sys.prefix? A string giving the site-specific directory prefix where the platform independent Python files are installed; by default, this is the string '/usr/local'. This can be set at build time with the --prefix argument to the configure script. The main collection of Python library modules is installed in the directory prefix + '/lib/pythonversion' while the platform independent header files (all except pyconfig.h) are stored in prefix + '/include/pythonversion', where version is equal to version[:3]." In my case, this is not correct: it looks, after I call the interpreter, something change the sys.prefix variable and point it to the right place and not to that directory which I used with the configuration script... This is very useful (I am happy because it solves one of my problems) but I am afraid of it will be do it in a different way in the future (as it is not documented). Thanks, Regards, Balazs From awilliam at opengroupware.us Thu Jan 21 12:03:26 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Thu, 21 Jan 2010 12:03:26 -0500 Subject: simple pub/sub In-Reply-To: <0af0eff2-50e2-44aa-81f1-b3d12cb265a7@a15g2000yqm.googlegroups.com> References: <0af0eff2-50e2-44aa-81f1-b3d12cb265a7@a15g2000yqm.googlegroups.com> Message-ID: <1264093406.8831.25.camel@linux-m3mt> On Thu, 2010-01-21 at 08:54 -0800, Steve Howell wrote: > Hi, I'm looking for ideas on building a simple architecture that > allows a bunch of independent Python processes to exchange data using > files and perform calculations. > One Python program would be collecting data from boat instruments on a > serial port, then writing that info out to a file, and also > transmitting data to instruments as needed (again through the serial > port). It would be the most complex program in terms of having to > poll file descriptors, or something similar, but I want to limit its > job to pure communication handling. This should be pretty easy using multiprocessing. In OpenGroupware Coils we have a master process that spins up children (workers, that each provide a distinct service) it opens a Pipe to each child to send messages to the child and a Queue from which it reads [all children (clients) write to the Queue and listen on their Pipe]. Then the master just listens on its pipe and forwards messages from children to other children. All children are Service objects . Then implementing a new service [type of worker] is as easy as (our brutally simple pubsub service) Hope that helps. -- OpenGroupware developer: awilliam at whitemice.org OpenGroupare & Cyrus IMAPd documenation @ From showell30 at yahoo.com Thu Jan 21 12:28:52 2010 From: showell30 at yahoo.com (Steve Howell) Date: Thu, 21 Jan 2010 09:28:52 -0800 (PST) Subject: simple pub/sub References: <0af0eff2-50e2-44aa-81f1-b3d12cb265a7@a15g2000yqm.googlegroups.com> Message-ID: <3edfbb90-d32e-40e5-ba33-6720faaacbc9@k19g2000yqc.googlegroups.com> On Jan 21, 9:03?am, Adam Tauno Williams wrote: > On Thu, 2010-01-21 at 08:54 -0800, Steve Howell wrote: > > Hi, I'm looking for ideas on building a simple architecture that > > allows a bunch of independent Python processes to exchange data using > > files and perform calculations. > > One Python program would be collecting data from boat instruments on a > > serial port, then writing that info out to a file, and also > > transmitting data to instruments as needed (again through the serial > > port). ?It would be the most complex program in terms of having to > > poll file descriptors, or something similar, but I want to limit its > > job to pure communication handling. > > This should be pretty easy using multiprocessing. ?In OpenGroupware > Coils we have a master process > that spins up children (workers, that each provide a distinct service) it opens a Pipe to each child to send messages to the child and a Queue from which it reads [all children (clients) write to the Queue and listen on their Pipe]. ?Then the master just listens on its pipe and forwards messages from children to other children. > > All children are Service objects > . ?Then implementing a new service [type of worker] is as easy as (our brutally simple pubsub service) > > Hope that helps. > It does indeed! I'm gonna try this approach for now. From aahz at pythoncraft.com Thu Jan 21 12:37:58 2010 From: aahz at pythoncraft.com (Aahz) Date: 21 Jan 2010 09:37:58 -0800 Subject: simple pub/sub References: <0af0eff2-50e2-44aa-81f1-b3d12cb265a7@a15g2000yqm.googlegroups.com> Message-ID: In article <0af0eff2-50e2-44aa-81f1-b3d12cb265a7 at a15g2000yqm.googlegroups.com>, Steve Howell wrote: > >Hi, I'm looking for ideas on building a simple architecture that >allows a bunch of independent Python processes to exchange data using >files and perform calculations. SQLite? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From pavlovevidence at gmail.com Thu Jan 21 12:50:55 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Thu, 21 Jan 2010 09:50:55 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> Message-ID: <06bb8c18-371f-4d61-a45a-ae70d5e20f44@o28g2000yqh.googlegroups.com> On Jan 21, 2:38?am, "Alf P. Steinbach" wrote: > * Carl Banks: > > > On Jan 20, 11:43 pm, Martin Drautzburg > [snip] > > >> What I am really looking for is a way > > >> ? ? ? ? - to be able to call move(up) > >> ? ? ? ? - having the "up" symbol only in the context of the function call > > > Short answer is, you can't do it. > > On the contrary, it's not difficult to do. > > I provided an example in my answer to the OP (first reply in the thread). Your example doesn't remotely do what the OP was asking for. In fact your example is so preposterous I question your sanity. > However, it's IMHO an abuse of the language, not something that one should do. Usually people abuse the language to achieve something (ostensibly) useful. Your example is so useless I don't think I would even call it abuse. As best as I can tell, what it is is you attempting to make yourself look like some kind of badass by answering some absurdly literal interpretation of the OP's question. Except you haven't even done that: class using_directions: up = 42 move( up ) print up # <- clearly not confined to context of function call Carl Banks From garrickp at gmail.com Thu Jan 21 12:52:32 2010 From: garrickp at gmail.com (Falcolas) Date: Thu, 21 Jan 2010 09:52:32 -0800 (PST) Subject: Closures in metaclasses Message-ID: <2c126442-f0b8-4b38-82c7-7fb226b15b60@u41g2000yqe.googlegroups.com> I'm running into an issue with closures in metaclasses - that is, if I create a function with a closure in a metaclass, the closure appears to be lost when I access the final class. I end up getting the text 'param' instead of the actual tags I am expecting: ALL_TAGS = ['a', 'abbr', 'acronym', 'address', 'applet', 'b', 'bdo', 'big'] #snip def _tag_meta(name, bases, dict_): for tag in ALL_TAGS: def generic_tag(*args, **kwargs): return Tag._generate_tag(tag, *args, **kwargs) #generic_tag = eval("lambda *args, **kwargs: Tag._generate_tag ('%s', *args, **kwargs)" % tag) dict_[tag] = staticmethod(generic_tag) return type(name, bases, dict_) class Tag(object): __metaclass__ = _tag_meta @staticmethod def _generate_tag(tag_name, *args, **kwargs): # Does the expected, following is just for the example's sake return tag_name Here's the output from my actual class: $ python Python 2.6.4 (r264:75706, Dec 7 2009, 18:45:15) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from htmlgen import Tag >>> Tag.html("foo") '\nfoo\n' >>> Using the eval shown above works as expected, but I'd rather go with the closures, if someone can help me figure out what's going on. From zuo at chopin.edu.pl Thu Jan 21 13:08:45 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Thu, 21 Jan 2010 19:08:45 +0100 Subject: Sorted dictionary In-Reply-To: References: Message-ID: 21-01-2010, 07:21:22 Dennis Lee Bieber wrote: > How does this differ from all the other "ordered dictionary" schemes > out there (and maybe even included in 2.7? I'm still on 2.5) It's completely different, please read first paragraph of page I linked (http://code.activestate.com/recipes/576998/). Regards, *j -- Jan Kaliszewski (zuo) From iainspeed at gmail.com Thu Jan 21 13:13:38 2010 From: iainspeed at gmail.com (Iain Barnett) Date: Thu, 21 Jan 2010 18:13:38 +0000 Subject: installing psycopg2-2.0.13 with python3.1 In-Reply-To: References: <03495BC1-8736-42D5-9B59-5B0148F643AA@gmail.com> Message-ID: <31AC458E-97F9-48D3-9401-2D865232AE53@gmail.com> On 21 Jan 2010, at 00:11, Gabriel Genellina wrote: > En Wed, 20 Jan 2010 19:45:44 -0300, Iain Barnett escribi?: > >> Would anyone know if it's possible to install psycopg2-2.0.13 with python3.1.1 (or similar)?I can install it with python2.6 with no problems, but obviously I'd prefer to use the latest version. My system is OSX10.6, and I'm new to Python. > > If you're new to Python, perhaps it's better to stay with the 2.6 version. > > Python 3.x introduced some incompatible changes in the language; not all third-party packages are available for Python 3.x yet. Regarding psycopg2, although some work has been done [1], there is no "official" release compatible with Python 3 yet. > > If you insist on using Python 3.1, there is another interface to PostgreSQL called pg8000 that claims to be Python 3.x compatible (I've not actually tested it). > > [1] http://mail.python.org/pipermail/python-porting/2008-December/000004.html > [2] http://pybrary.net/pg8000/ > > -- > Gabriel Genellina > > -- > http://mail.python.org/mailman/listinfo/python-list On 20 Jan 2010, at 23:58, Philip Semanchuk wrote: > > On Jan 20, 2010, at 5:45 PM, Iain Barnett wrote: > >> Hi, >> >> Would anyone know if it's possible to install psycopg2-2.0.13 with python3.1.1 (or similar)? When I try I get the following >> >> $ sudo python setup.py install >> Password: >> File "setup.py", line 233 >> except Warning, w: >> ^ >> SyntaxError: invalid syntax >> >> >> I can install it with python2.6 with no problems, but obviously I'd prefer to use the latest version. My system is OSX10.6, and I'm new to Python. > > Hi Iain, > I've been out of the psycopg loop for a while now, but my guess is that the answer to your question is no. There are some Python 3-related checkins in the psycopg2 repository, but it looks like they're not ready for public use yet (unless you're adventurous). > > Here's the source repository, which contains a commit commented "First round of changes for Python 3": > https://dndg.it/cgi-bin/gitweb.cgi?p=public/psycopg2.git > > If you scroll to the bottom of that page you'll see a "head" created especially for Python 2 which is where 2.0.13 came from. > > If you're willing to do some hacking on your own, Martin v. L?wis ported psycopg2 to Python 3 and provided a diff: > http://mail.python.org/pipermail/python-porting/2008-December/000018.html > > Last but not least, I should note that psycopg has its own mailing list, and I bet they know more about it there: > http://lists.initd.org/mailman/listinfo/psycopg > > Hope this helps. Psycopg & Postgres both worked wonderfully well for me. > > Cheers > Philip > -- > http://mail.python.org/mailman/listinfo/python-list Thanks to both of you, it really is much appreciated. Regards Iain From arnodel at googlemail.com Thu Jan 21 13:24:32 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Thu, 21 Jan 2010 18:24:32 +0000 Subject: Closures in metaclasses References: <2c126442-f0b8-4b38-82c7-7fb226b15b60@u41g2000yqe.googlegroups.com> Message-ID: Falcolas writes: > I'm running into an issue with closures in metaclasses - that is, if I > create a function with a closure in a metaclass, the closure appears > to be lost when I access the final class. I end up getting the text > 'param' instead of the actual tags I am expecting: > > ALL_TAGS = ['a', 'abbr', 'acronym', 'address', 'applet', 'b', 'bdo', > 'big'] #snip > > def _tag_meta(name, bases, dict_): > for tag in ALL_TAGS: > def generic_tag(*args, **kwargs): > return Tag._generate_tag(tag, *args, **kwargs) > #generic_tag = eval("lambda *args, **kwargs: Tag._generate_tag > ('%s', *args, **kwargs)" % tag) > dict_[tag] = staticmethod(generic_tag) > return type(name, bases, dict_) This is almost a FAQ and has nothing to do with metaclasses. The simplest solution usually involves adding a default argument 'tag=tag' to the function you define (here, generic_tag), but you can't do this here because you have a **kwargs argument. Instead, you can use a closure and do this for example: def factory(tag): def generic_tag(*args, **kwargs): return Tag._generate_tag(tag, *args, **kwargs) return generic_tag def _tag_meta(name, bases, dict_): for tag in ALL_TAGS: dict_[tag] = staticmethod(factory(tag)) return type(name, bases, dict_) Then your Tag example class will work as you expect: class Tag(object): __metaclass__ = _tag_meta @staticmethod def _generate_tag(tag_name, *args, **kwargs): # Does the expected, following is just for the example's sake return tag_name I am sure that there are plenty of discussions of this issue in the archives of c.l.python. I just can't think of a good keyword to google it ATM. However, I am usure about why you are using a metaclass. HTH -- Arnaud From garrickp at gmail.com Thu Jan 21 13:37:47 2010 From: garrickp at gmail.com (Falcolas) Date: Thu, 21 Jan 2010 10:37:47 -0800 (PST) Subject: Closures in metaclasses References: <2c126442-f0b8-4b38-82c7-7fb226b15b60@u41g2000yqe.googlegroups.com> Message-ID: <7ceaaaa1-45c2-4875-a091-971c14e2d41e@f12g2000yqn.googlegroups.com> On Jan 21, 11:24?am, Arnaud Delobelle wrote: > Falcolas writes: > > I'm running into an issue with closures in metaclasses - that is, if I > > create a function with a closure in a metaclass, the closure appears > > to be lost when I access the final class. I end up getting the text > > 'param' instead of the actual tags I am expecting: > > > ALL_TAGS = ?['a', 'abbr', 'acronym', 'address', 'applet', 'b', 'bdo', > > 'big'] #snip > > > def _tag_meta(name, bases, dict_): > > ? ? for tag in ALL_TAGS: > > ? ? ? ? def generic_tag(*args, **kwargs): > > ? ? ? ? ? ? return Tag._generate_tag(tag, *args, **kwargs) > > ? ? ? ? #generic_tag = eval("lambda *args, **kwargs: Tag._generate_tag > > ('%s', *args, **kwargs)" % tag) > > ? ? ? ? dict_[tag] = staticmethod(generic_tag) > > ? ? return type(name, bases, dict_) > > This is almost a FAQ and has nothing to do with metaclasses. ?The > simplest solution usually involves adding a default argument 'tag=tag' > to the function you define (here, generic_tag), but you can't do this > here because you have a **kwargs argument. ?Instead, you can use a > closure and do this for example: > > def factory(tag): > ? ? def generic_tag(*args, **kwargs): > ? ? ? ? return Tag._generate_tag(tag, *args, **kwargs) > ? ? return generic_tag > > def _tag_meta(name, bases, dict_): > ? ? for tag in ALL_TAGS: > ? ? ? ? dict_[tag] = staticmethod(factory(tag)) > ? ? return type(name, bases, dict_) I see - I was thinking it would preserve the closure from the for statement, but I can see now why that would be wrong. > > However, I am usure about why you are using a metaclass. > > HTH > > -- > Arnaud It was the easiest way I found to add a lot of static methods to the Tag class without writing each one out. __getattr__ was not working for this application. This is for a very simple application, and I didn't want to add a lot of complexity to it's use. I'm always open for other options OTOH. From deets at nospam.web.de Thu Jan 21 13:43:41 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Thu, 21 Jan 2010 19:43:41 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> Message-ID: <7rrliuFl3gU1@mid.uni-berlin.de> Am 21.01.10 12:58, schrieb Alf P. Steinbach: > * Stefan Behnel: >> Alf P. Steinbach, 21.01.2010 11:38: >>> * Carl Banks: >>>> On Jan 20, 11:43 pm, Martin Drautzburg >>> [snip] >>>>> What I am really looking for is a way >>>>> >>>>> - to be able to call move(up) >>>>> - having the "up" symbol only in the context of the function >>>>> call >>>> Short answer is, you can't do it. >>>> >>> On the contrary, it's not difficult to do. >>> >>> I provided an example in my answer to the OP (first reply in the >>> thread). >> >> Erm, no, you didn't. You showed a) how to pass a string constant into a >> function and b) how to pass a value from a bound variable. None of >> that is >> helpful to the OP's problem. > > Perhaps look again at that example. It demonstrates exactly what the OP > asks for. Perhaps look again at the requirements: """ - the parameter IS REALLY NOT A STRING, but a direction """ Diez From zuo at chopin.edu.pl Thu Jan 21 13:43:45 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Thu, 21 Jan 2010 19:43:45 +0100 Subject: Sorted dictionary In-Reply-To: <50697b2c1001202349m7b346df5q4686dfa51b29bc13@mail.gmail.com> References: <50697b2c1001202349m7b346df5q4686dfa51b29bc13@mail.gmail.com> Message-ID: Dnia 21-01-2010 o 08:49:22 Chris Rebert napisa?(a): > On Wed, Jan 20, 2010 at 5:50 PM, Steven D'Aprano > wrote: >> On Thu, 21 Jan 2010 02:02:02 +0100, Jan Kaliszewski wrote: >>> http://code.activestate.com/recipes/576998/ >> What's the advantage of that over sorting the keys as needed? >> >> >> E.g. >> >> for key in sorted(dict): >> ? ?print key >> >> >> works. > > Well, it does spread out the cost of sorting the key list over each of > the N distinct key assignments, versus sorting the entire list all at > once, on-demand at the end of the process. And you avoid having to > traverse the M buckets in the dictionary to locate the N keys in the > first place. So it has different performance characteristics, which > could theoretically matter depending on the use case; e.g. it could > avoid a GUI application hanging for several seconds while it sorts a > large list of keys. Generally, I see 3 possible approaches: 1. What Steven wrote about: simply sort all keys when you need to get them sorted (it needs iteration over whole a dict). In many cases it's good enough (so is the best, because of simplicity) -- e.g. when a dict is not very long (very common case), or when that sorting it is a rare operation. Sort always when a key is added/deleted, maintaining an auxiliary list of sorted keys (sufficient especially when you modify a dict rarely and get from it often). 2. Sort all keys on-demand -- but only if a dict is marked as "modified". Mark a dict as "modified" when you add/delete any key; and mark a dict as "unmodified" when sorting. (Of course, it can be automatized, see e.g.: http://pypi.python.org/pypi/sorteddict). Nice, if your use-pattern is: add/del a lot, *then* only get/iter a lot... 3. Continually (at each set/del of a key) maintain auxiliary sorted list of keys -- using binary search (as I did), heap queue (see: http://pypi.python.org/pypi/HeapDict) or such an algorithm. I think this approach is the best when you have quite long dict, and you add/del fairly often and get/iter very often. Regards, *j -- Jan Kaliszewski (zuo) From zuo at chopin.edu.pl Thu Jan 21 13:45:56 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Thu, 21 Jan 2010 19:45:56 +0100 Subject: Sorted dictionary In-Reply-To: <1c74df0b-b22e-4aea-934a-c6d25e818f41@e37g2000yqn.googlegroups.com> References: <1c74df0b-b22e-4aea-934a-c6d25e818f41@e37g2000yqn.googlegroups.com> Message-ID: Dnia 21-01-2010 o 09:27:52 Raymond Hettinger napisa?(a): > On Jan 20, 5:02?pm, "Jan Kaliszewski" wrote: >> http://code.activestate.com/recipes/576998/ > Using an underlying list to track sorted items > means that insertion and deletion take O(n) time. > That could be reduced to O(log n) time by using > a blist or skiplist as the underlying structure > for maintaining a sort. Please note that I used funcions from bisect, that use binary search. Doesn't it take O(log n) time? *j -- Jan Kaliszewski (zuo) From alfps at start.no Thu Jan 21 13:46:00 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 19:46:00 +0100 Subject: Symbols as parameters? In-Reply-To: <06bb8c18-371f-4d61-a45a-ae70d5e20f44@o28g2000yqh.googlegroups.com> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <06bb8c18-371f-4d61-a45a-ae70d5e20f44@o28g2000yqh.googlegroups.com> Message-ID: * Carl Banks: > On Jan 21, 2:38 am, "Alf P. Steinbach" wrote: >> * Carl Banks: >> >>> On Jan 20, 11:43 pm, Martin Drautzburg >> [snip] >> >>>> What I am really looking for is a way >>>> - to be able to call move(up) >>>> - having the "up" symbol only in the context of the function call >>> Short answer is, you can't do it. >> On the contrary, it's not difficult to do. >> >> I provided an example in my answer to the OP (first reply in the thread). > > Your example doesn't remotely do what the OP was asking for. In fact > your example is so preposterous I question your sanity. Your first sentence is incorrect, your second sentence is a silly attempt at getting personal. >> However, it's IMHO an abuse of the language, not something that one should do. > > Usually people abuse the language to achieve something (ostensibly) > useful. Your example is so useless I don't think I would even call it > abuse. You are correct that it's useless. The OP asked for a construct to do a useless thing. The presented construct does exactly what the OP asked for: useless. > As best as I can tell, what it is is you attempting to make yourself > look like some kind of badass by answering some absurdly literal > interpretation of the OP's question. Hm, there are ways to do things, even the ad hominem thing. You just present yourself as one using strong words when you're proven wrong. Myself I'm not foreign to strong words :-), but I wouldn't dream of applying them to a person. The above is very, uh, primitive. Besides, it's quite silly to get angry when you're proved to be wrong. :-) > Except you haven't even done that: > > class using_directions: > up = 42 > move( up ) > print up # <- clearly not confined to context of function call You know, I didn't think of that ingenious thing, that it would be possible to *modify* the example so that it no longer fit the OP's literal description. Thx! Cheers & hth., - Alf From alfps at start.no Thu Jan 21 13:48:02 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 19:48:02 +0100 Subject: Symbols as parameters? In-Reply-To: <7rrliuFl3gU1@mid.uni-berlin.de> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> <7rrliuFl3gU1@mid.uni-berlin.de> Message-ID: * Diez B. Roggisch: > Am 21.01.10 12:58, schrieb Alf P. Steinbach: >> * Stefan Behnel: >>> Alf P. Steinbach, 21.01.2010 11:38: >>>> * Carl Banks: >>>>> On Jan 20, 11:43 pm, Martin Drautzburg >>>> [snip] >>>>>> What I am really looking for is a way >>>>>> >>>>>> - to be able to call move(up) >>>>>> - having the "up" symbol only in the context of the function >>>>>> call >>>>> Short answer is, you can't do it. >>>>> >>>> On the contrary, it's not difficult to do. >>>> >>>> I provided an example in my answer to the OP (first reply in the >>>> thread). >>> >>> Erm, no, you didn't. You showed a) how to pass a string constant into a >>> function and b) how to pass a value from a bound variable. None of >>> that is >>> helpful to the OP's problem. >> >> Perhaps look again at that example. It demonstrates exactly what the OP >> asks for. > > Perhaps look again at the requirements: > > """ > - the parameter IS REALLY NOT A STRING, but a direction > """ So? Cheers & hth., - Alf From deets at nospam.web.de Thu Jan 21 13:53:01 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Thu, 21 Jan 2010 19:53:01 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> <7rrliuFl3gU1@mid.uni-berlin.de> Message-ID: <7rrm4dFofqU1@mid.uni-berlin.de> Am 21.01.10 19:48, schrieb Alf P. Steinbach: > * Diez B. Roggisch: >> Am 21.01.10 12:58, schrieb Alf P. Steinbach: >>> * Stefan Behnel: >>>> Alf P. Steinbach, 21.01.2010 11:38: >>>>> * Carl Banks: >>>>>> On Jan 20, 11:43 pm, Martin Drautzburg >>>>> [snip] >>>>>>> What I am really looking for is a way >>>>>>> >>>>>>> - to be able to call move(up) >>>>>>> - having the "up" symbol only in the context of the function >>>>>>> call >>>>>> Short answer is, you can't do it. >>>>>> >>>>> On the contrary, it's not difficult to do. >>>>> >>>>> I provided an example in my answer to the OP (first reply in the >>>>> thread). >>>> >>>> Erm, no, you didn't. You showed a) how to pass a string constant into a >>>> function and b) how to pass a value from a bound variable. None of >>>> that is >>>> helpful to the OP's problem. >>> >>> Perhaps look again at that example. It demonstrates exactly what the OP >>> asks for. >> >> Perhaps look again at the requirements: >> >> """ >> - the parameter IS REALLY NOT A STRING, but a direction >> """ > > So? Oh please. You claim you provided exactly what the OP asked for. But in the body of move, all you can do is to compare the direction parameter tco "up", not to UP. So no, you provided *something*, but not what the OP asked for. Diez From alfps at start.no Thu Jan 21 14:01:04 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 20:01:04 +0100 Subject: Symbols as parameters? In-Reply-To: <7rrm4dFofqU1@mid.uni-berlin.de> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> <7rrliuFl3gU1@mid.uni-berlin.de> <7rrm4dFofqU1@mid.uni-berlin.de> Message-ID: * Diez B. Roggisch: > Am 21.01.10 19:48, schrieb Alf P. Steinbach: >> * Diez B. Roggisch: >>> Am 21.01.10 12:58, schrieb Alf P. Steinbach: >>>> * Stefan Behnel: >>>>> Alf P. Steinbach, 21.01.2010 11:38: >>>>>> * Carl Banks: >>>>>>> On Jan 20, 11:43 pm, Martin Drautzburg >>>>>> [snip] >>>>>>>> What I am really looking for is a way >>>>>>>> >>>>>>>> - to be able to call move(up) >>>>>>>> - having the "up" symbol only in the context of the function >>>>>>>> call >>>>>>> Short answer is, you can't do it. >>>>>>> >>>>>> On the contrary, it's not difficult to do. >>>>>> >>>>>> I provided an example in my answer to the OP (first reply in the >>>>>> thread). >>>>> >>>>> Erm, no, you didn't. You showed a) how to pass a string constant >>>>> into a >>>>> function and b) how to pass a value from a bound variable. None of >>>>> that is >>>>> helpful to the OP's problem. >>>> >>>> Perhaps look again at that example. It demonstrates exactly what the OP >>>> asks for. >>> >>> Perhaps look again at the requirements: >>> >>> """ >>> - the parameter IS REALLY NOT A STRING, but a direction >>> """ >> >> So? > > Oh please. You claim you provided exactly what the OP asked for. But in > the body of move, all you can do is to compare the direction parameter > tco "up", not to UP. So no, you provided *something*, but not what the > OP asked for. Pardon me for not taking you seriously or not getting the joke, if that's what it's meant as. Cheers, - Alf From deets at nospam.web.de Thu Jan 21 14:06:35 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Thu, 21 Jan 2010 20:06:35 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> <7rrliuFl3gU1@mid.uni-berlin.de> <7rrm4dFofqU1@mid.uni-berlin.de> Message-ID: <7rrmttFtgjU1@mid.uni-berlin.de> Am 21.01.10 20:01, schrieb Alf P. Steinbach: > * Diez B. Roggisch: >> Am 21.01.10 19:48, schrieb Alf P. Steinbach: >>> * Diez B. Roggisch: >>>> Am 21.01.10 12:58, schrieb Alf P. Steinbach: >>>>> * Stefan Behnel: >>>>>> Alf P. Steinbach, 21.01.2010 11:38: >>>>>>> * Carl Banks: >>>>>>>> On Jan 20, 11:43 pm, Martin Drautzburg >>>>>>> [snip] >>>>>>>>> What I am really looking for is a way >>>>>>>>> >>>>>>>>> - to be able to call move(up) >>>>>>>>> - having the "up" symbol only in the context of the function >>>>>>>>> call >>>>>>>> Short answer is, you can't do it. >>>>>>>> >>>>>>> On the contrary, it's not difficult to do. >>>>>>> >>>>>>> I provided an example in my answer to the OP (first reply in the >>>>>>> thread). >>>>>> >>>>>> Erm, no, you didn't. You showed a) how to pass a string constant >>>>>> into a >>>>>> function and b) how to pass a value from a bound variable. None of >>>>>> that is >>>>>> helpful to the OP's problem. >>>>> >>>>> Perhaps look again at that example. It demonstrates exactly what >>>>> the OP >>>>> asks for. >>>> >>>> Perhaps look again at the requirements: >>>> >>>> """ >>>> - the parameter IS REALLY NOT A STRING, but a direction >>>> """ >>> >>> So? >> >> Oh please. You claim you provided exactly what the OP asked for. But >> in the body of move, all you can do is to compare the direction >> parameter tco "up", not to UP. So no, you provided *something*, but >> not what the OP asked for. > > Pardon me for not taking you seriously or not getting the joke, if > that's what it's meant as. No joking, unless you started it somewhere earlier. The OP didn't want to compare to a string, yet your "example" forces him to exactly do that. If not, would you care to show us the fulfillment of "the parameter IS REALLY NOT A STRING, but a direction" in your example? Thanks. You see, it seems that you agreeing to be wrong is something your are so adamantly refusing to acknowledge that I'm inclined to believe that in the event of this really happen one day, a great rift in the very fabric of space and time would appear, and the world would end. If this conversation was in my native language and not in yours, I'd might give it a shot, just to see what really happens. But as it isn't, and I guess everybody else except you knows that your wrong - I wont. HTH, Diez From arnodel at googlemail.com Thu Jan 21 14:10:07 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Thu, 21 Jan 2010 11:10:07 -0800 (PST) Subject: Closures in metaclasses References: <2c126442-f0b8-4b38-82c7-7fb226b15b60@u41g2000yqe.googlegroups.com> <7ceaaaa1-45c2-4875-a091-971c14e2d41e@f12g2000yqn.googlegroups.com> Message-ID: <48807f29-3157-43b8-9b01-37ccf5f00616@a15g2000yqm.googlegroups.com> On Jan 21, 6:37?pm, Falcolas wrote: > On Jan 21, 11:24?am, Arnaud Delobelle wrote: > > > > > > > Falcolas writes: > > > I'm running into an issue with closures in metaclasses - that is, if I > > > create a function with a closure in a metaclass, the closure appears > > > to be lost when I access the final class. I end up getting the text > > > 'param' instead of the actual tags I am expecting: > > > > ALL_TAGS = ?['a', 'abbr', 'acronym', 'address', 'applet', 'b', 'bdo', > > > 'big'] #snip > > > > def _tag_meta(name, bases, dict_): > > > ? ? for tag in ALL_TAGS: > > > ? ? ? ? def generic_tag(*args, **kwargs): > > > ? ? ? ? ? ? return Tag._generate_tag(tag, *args, **kwargs) > > > ? ? ? ? #generic_tag = eval("lambda *args, **kwargs: Tag._generate_tag > > > ('%s', *args, **kwargs)" % tag) > > > ? ? ? ? dict_[tag] = staticmethod(generic_tag) > > > ? ? return type(name, bases, dict_) > > > This is almost a FAQ and has nothing to do with metaclasses. ?The > > simplest solution usually involves adding a default argument 'tag=tag' > > to the function you define (here, generic_tag), but you can't do this > > here because you have a **kwargs argument. ?Instead, you can use a > > closure and do this for example: > > > def factory(tag): > > ? ? def generic_tag(*args, **kwargs): > > ? ? ? ? return Tag._generate_tag(tag, *args, **kwargs) > > ? ? return generic_tag > > > def _tag_meta(name, bases, dict_): > > ? ? for tag in ALL_TAGS: > > ? ? ? ? dict_[tag] = staticmethod(factory(tag)) > > ? ? return type(name, bases, dict_) > > I see - I was thinking it would preserve the closure from the for > statement, but I can see now why that would be wrong. > > > > > However, I am usure about why you are using a metaclass. > > > HTH > > > -- > > Arnaud > > It was the easiest way I found to add a lot of static methods to the > Tag class without writing each one out. __getattr__ was not working > for this application. This is for a very simple application, and I > didn't want to add a lot of complexity to it's use. I'm always open > for other options OTOH. This should work (untested): class Tag(object): @staticmethod def _generate_tag(tag_name, *args, **kwargs): # Does the expected, following is just for the example's sake return tag for tag in ALL_TAGS: setattr(Tag, tag, staticmethod(factory(tag))) Or you could override __getattr__ -- Arnaud From alfps at start.no Thu Jan 21 14:24:12 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 20:24:12 +0100 Subject: Symbols as parameters? In-Reply-To: <7rrmttFtgjU1@mid.uni-berlin.de> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> <7rrliuFl3gU1@mid.uni-berlin.de> <7rrm4dFofqU1@mid.uni-berlin.de> <7rrmttFtgjU1@mid.uni-berlin.de> Message-ID: * Diez B. Roggisch: > Am 21.01.10 20:01, schrieb Alf P. Steinbach: >> * Diez B. Roggisch: >>> Am 21.01.10 19:48, schrieb Alf P. Steinbach: >>>> * Diez B. Roggisch: >>>>> Am 21.01.10 12:58, schrieb Alf P. Steinbach: >>>>>> * Stefan Behnel: >>>>>>> Alf P. Steinbach, 21.01.2010 11:38: >>>>>>>> * Carl Banks: >>>>>>>>> On Jan 20, 11:43 pm, Martin Drautzburg >>>>>>>> [snip] >>>>>>>>>> What I am really looking for is a way >>>>>>>>>> >>>>>>>>>> - to be able to call move(up) >>>>>>>>>> - having the "up" symbol only in the context of the function >>>>>>>>>> call >>>>>>>>> Short answer is, you can't do it. >>>>>>>>> >>>>>>>> On the contrary, it's not difficult to do. >>>>>>>> >>>>>>>> I provided an example in my answer to the OP (first reply in the >>>>>>>> thread). >>>>>>> >>>>>>> Erm, no, you didn't. You showed a) how to pass a string constant >>>>>>> into a >>>>>>> function and b) how to pass a value from a bound variable. None of >>>>>>> that is >>>>>>> helpful to the OP's problem. >>>>>> >>>>>> Perhaps look again at that example. It demonstrates exactly what >>>>>> the OP >>>>>> asks for. >>>>> >>>>> Perhaps look again at the requirements: >>>>> >>>>> """ >>>>> - the parameter IS REALLY NOT A STRING, but a direction >>>>> """ >>>> >>>> So? >>> >>> Oh please. You claim you provided exactly what the OP asked for. But >>> in the body of move, all you can do is to compare the direction >>> parameter tco "up", not to UP. So no, you provided *something*, but >>> not what the OP asked for. >> >> Pardon me for not taking you seriously or not getting the joke, if >> that's what it's meant as. > > No joking, unless you started it somewhere earlier. The OP didn't want > to compare to a string, yet your "example" forces him to exactly do > that. If not, would you care to show us the fulfillment of "the > parameter IS REALLY NOT A STRING, but a direction" in your example? Thanks. Just use a "direction" instead of a number (if you didn't like the number). I'm sorry that I couldn't believe that you didn't understand that. But OK, you didn't. The OP gave no definition of his "direction" though, so you'll have to use your good sense about possibilities for what it could be. From the context and later comments (especially about "import") my guess us that he's talking about numbers denoting directions, but you'd have to ask him. > You see, it seems that you agreeing to be wrong is something your are so > adamantly refusing to acknowledge that I'm inclined to believe that in > the event of this really happen one day, a great rift in the very fabric > of space and time would appear, and the world would end. If this > conversation was in my native language and not in yours, I'd might give > it a shot, just to see what really happens. But as it isn't, and I guess > everybody else except you knows that your wrong - I wont. The above is a silly ad hominem attack, containing 1 lie, and trying to confuse the issue: you're posting this in a different part of the thread as a response to Carls Banks being proved wrong by my example. Incidentally, he also resorts to ad hominem. Sadly it seems to be a common response pattern in this group. Do you understand how bad that makes you look? Cheers & hth., - Alf From ndbecker2 at gmail.com Thu Jan 21 14:27:14 2010 From: ndbecker2 at gmail.com (Neal Becker) Date: Thu, 21 Jan 2010 14:27:14 -0500 Subject: multiprocessing as batch system References: <1264090591.8831.10.camel@linux-m3mt> Message-ID: Adam Tauno Williams wrote: > On Thu, 2010-01-21 at 07:13 -0500, Neal Becker wrote: >> I'm using multiprocessing as a poor man's batch system. It is working >> OK, except that it doesn't seem to do load balancing quite right. >> I have a 8-cpu machine. If I start, using multiprocessing pool, calling >> map >> with more than 8 elements, it will start 8 processes. It seems, though, >> that when some of them finish, it does not start others right away. The >> load average may be a lot less than 8 but still additional processes are >> not active. > > If your processes are I/O bound your system will never appear to be > saturated. Multiple pending processes and a low load average usually > indicated waiting on some other resource. No, 100% compute bound. My impression was that some processes completed, but new ones were not immediately started. From daniel at stutzbachenterprises.com Thu Jan 21 14:35:55 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Thu, 21 Jan 2010 13:35:55 -0600 Subject: Sorted dictionary In-Reply-To: References: <1c74df0b-b22e-4aea-934a-c6d25e818f41@e37g2000yqn.googlegroups.com> Message-ID: On Thu, Jan 21, 2010 at 12:45 PM, Jan Kaliszewski wrote: > Please note that I used funcions from bisect, that use binary search. > > Doesn't it take O(log n) time? > It takes O(log n) time to find the point to insert, but O(n) time to perform the actual insertion. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnodel at googlemail.com Thu Jan 21 14:41:53 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Thu, 21 Jan 2010 19:41:53 +0000 Subject: Sorted dictionary References: <1c74df0b-b22e-4aea-934a-c6d25e818f41@e37g2000yqn.googlegroups.com> Message-ID: "Jan Kaliszewski" writes: > Dnia 21-01-2010 o 09:27:52 Raymond Hettinger napisa?(a): > >> On Jan 20, 5:02?pm, "Jan Kaliszewski" wrote: > >>> http://code.activestate.com/recipes/576998/ > >> Using an underlying list to track sorted items >> means that insertion and deletion take O(n) time. >> That could be reduced to O(log n) time by using >> a blist or skiplist as the underlying structure >> for maintaining a sort. > > Please note that I used funcions from bisect, that use binary search. > > Doesn't it take O(log n) time? Insertion and deletion are still O(n) as all items to the right of the inserted/deleted one have to be shifted by one place. -- Arnaud From pavlovevidence at gmail.com Thu Jan 21 14:43:12 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Thu, 21 Jan 2010 11:43:12 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <06bb8c18-371f-4d61-a45a-ae70d5e20f44@o28g2000yqh.googlegroups.com> Message-ID: On Jan 21, 10:46?am, "Alf P. Steinbach" wrote: > * Carl Banks: > > > > > > > On Jan 21, 2:38 am, "Alf P. Steinbach" wrote: > >> * Carl Banks: > > >>> On Jan 20, 11:43 pm, Martin Drautzburg > >> [snip] > > >>>> What I am really looking for is a way > >>>> ? ? ? ? - to be able to call move(up) > >>>> ? ? ? ? - having the "up" symbol only in the context of the function call > >>> Short answer is, you can't do it. > >> On the contrary, it's not difficult to do. > > >> I provided an example in my answer to the OP (first reply in the thread). > > > Your example doesn't remotely do what the OP was asking for. ?In fact > > your example is so preposterous I question your sanity. > > Your first sentence is incorrect, your second sentence is a silly attempt at > getting personal. > > >> However, it's IMHO an abuse of the language, not something that one should do. > > > Usually people abuse the language to achieve something (ostensibly) > > useful. ?Your example is so useless I don't think I would even call it > > abuse. > > You are correct that it's useless. The OP asked for a construct to do a useless > thing. The presented construct does exactly what the OP asked for: useless. > > > As best as I can tell, what it is is you attempting to make yourself > > look like some kind of badass by answering some absurdly literal > > interpretation of the OP's question. > > Hm, there are ways to do things, even the ad hominem thing. You just present > yourself as one using strong words when you're proven wrong. Myself I'm not > foreign to strong words :-), but I wouldn't dream of applying them to a person. > > The above is very, uh, primitive. > > Besides, it's quite silly to get angry when you're proved to be wrong. :-) > > > Except you haven't even done that: > > > class using_directions: > > ? ? up = 42 > > ? ? move( up ) > > ? ? print up # <- clearly not confined to context of function call > > You know, I didn't think of that ingenious thing, that it would be possible to > *modify* the example so that it no longer fit the OP's literal description. Thx! I'm not sure if you're trolling, insane, or just stupid; regardless, I'm done with you. Carl Banks From alfps at start.no Thu Jan 21 14:53:19 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 20:53:19 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <06bb8c18-371f-4d61-a45a-ae70d5e20f44@o28g2000yqh.googlegroups.com> Message-ID: * Carl Banks: > On Jan 21, 10:46 am, "Alf P. Steinbach" wrote: >> * Carl Banks: >>> On Jan 21, 2:38 am, "Alf P. Steinbach" wrote: >>>> * Carl Banks: >>>>> On Jan 20, 11:43 pm, Martin Drautzburg >>>> [snip] >>>>>> What I am really looking for is a way >>>>>> - to be able to call move(up) >>>>>> - having the "up" symbol only in the context of the function call >>>>> Short answer is, you can't do it. >>>> On the contrary, it's not difficult to do. >>>> I provided an example in my answer to the OP (first reply in the thread). >>> Your example doesn't remotely do what the OP was asking for. In fact >>> your example is so preposterous I question your sanity. >> Your first sentence is incorrect, your second sentence is a silly attempt at >> getting personal. >> >>>> However, it's IMHO an abuse of the language, not something that one should do. >>> Usually people abuse the language to achieve something (ostensibly) >>> useful. Your example is so useless I don't think I would even call it >>> abuse. >> You are correct that it's useless. The OP asked for a construct to do a useless >> thing. The presented construct does exactly what the OP asked for: useless. >> >>> As best as I can tell, what it is is you attempting to make yourself >>> look like some kind of badass by answering some absurdly literal >>> interpretation of the OP's question. >> Hm, there are ways to do things, even the ad hominem thing. You just present >> yourself as one using strong words when you're proven wrong. Myself I'm not >> foreign to strong words :-), but I wouldn't dream of applying them to a person. >> >> The above is very, uh, primitive. >> >> Besides, it's quite silly to get angry when you're proved to be wrong. :-) >> >>> Except you haven't even done that: >>> class using_directions: >>> up = 42 >>> move( up ) >>> print up # <- clearly not confined to context of function call >> You know, I didn't think of that ingenious thing, that it would be possible to >> *modify* the example so that it no longer fit the OP's literal description. Thx! > > I'm not sure if you're trolling, insane, or just stupid; regardless, > I'm done with you. Piling more ad hominem attacks on top of your previous is pretty silly. You were technically wrong, that's no big issue, everybody is wrong now and then. But reacting to a polite notification of your error, by going all out with various personal accusations, doesn't give anybody a positive impression of you. Cheers & hth., - Alf From phlip2005 at gmail.com Thu Jan 21 15:24:31 2010 From: phlip2005 at gmail.com (Phlip) Date: Thu, 21 Jan 2010 12:24:31 -0800 (PST) Subject: counting lines of code References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> <87bpgodj95.fsf@benfinney.id.au> <1f166703-fdfc-4f87-b336-ef80e492635b@a32g2000yqm.googlegroups.com> Message-ID: <7e09df6a-cda1-480e-a971-8f8a70ac4118@b9g2000yqd.googlegroups.com> On Jan 20, 11:20?pm, Michele Simionato wrote: > pylint does too many things, I want something fast that just counts > the lines and can be run on thousands of files at once. > cloc seems fine, I have just tried on 2,000 files and it gives me a > report in just a few seconds. In my experience with Python codebases that big... ...how many of those lines are duplicated, and might merge together into a better design? The LOC would go down, too. -- Phlip From aahz at pythoncraft.com Thu Jan 21 15:36:41 2010 From: aahz at pythoncraft.com (Aahz) Date: 21 Jan 2010 12:36:41 -0800 Subject: Dynamic HTML controls References: Message-ID: In article , Alan Harris-Reid wrote: > >Does anyone know where I can find any decent dynamically-constructed >HTML control classes (dropdown list, table, input field, checkbox, etc.) >written in Python. For example, for a HTML table I would like something >like... You might look at Quixote: http://quixote.ca/ -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From garrickp at gmail.com Thu Jan 21 15:38:09 2010 From: garrickp at gmail.com (Falcolas) Date: Thu, 21 Jan 2010 12:38:09 -0800 (PST) Subject: Closures in metaclasses References: <2c126442-f0b8-4b38-82c7-7fb226b15b60@u41g2000yqe.googlegroups.com> <7ceaaaa1-45c2-4875-a091-971c14e2d41e@f12g2000yqn.googlegroups.com> <48807f29-3157-43b8-9b01-37ccf5f00616@a15g2000yqm.googlegroups.com> Message-ID: On Jan 21, 12:10?pm, Arnaud Delobelle wrote: > On Jan 21, 6:37?pm, Falcolas wrote: > > > On Jan 21, 11:24?am, Arnaud Delobelle wrote: > > > It was the easiest way I found to add a lot of static methods to the > > Tag class without writing each one out. __getattr__ was not working > > for this application. This is for a very simple application, and I > > didn't want to add a lot of complexity to it's use. I'm always open > > for other options OTOH. > > This should work (untested): > > class Tag(object): > ? ? @staticmethod > ? ? def _generate_tag(tag_name, *args, **kwargs): > ? ? ? ? # Does the expected, following is just for the example's sake > ? ? ? ? return tag > > for tag in ALL_TAGS: > ? ? setattr(Tag, tag, staticmethod(factory(tag))) > > Or you could override __getattr__ > > -- > Arnaud I tried overriding __getattr__ and got an error at runtime (the instance did not have xyz key, etc), and the Tag dict is not modifiable (granted, I tried Tag.__dict__[tag] = spam, not setattr()). From __peter__ at web.de Thu Jan 21 15:55:09 2010 From: __peter__ at web.de (Peter Otten) Date: Thu, 21 Jan 2010 21:55:09 +0100 Subject: Closures in metaclasses References: <2c126442-f0b8-4b38-82c7-7fb226b15b60@u41g2000yqe.googlegroups.com> <7ceaaaa1-45c2-4875-a091-971c14e2d41e@f12g2000yqn.googlegroups.com> <48807f29-3157-43b8-9b01-37ccf5f00616@a15g2000yqm.googlegroups.com> Message-ID: Falcolas wrote: > I tried overriding __getattr__ and got an error at runtime (the You can either move __getattr__() into the metaclass or instantiate the class. I prefer the latter. Both approaches in one example: >>> class Tag: ... class __metaclass__(type): ... def __getattr__(self, name): return "<%s> via metaclass" % name ... def __getattr__(self, name): return "<%s> via class" % name ... >>> Tag.yadda ' via metaclass' >>> tag = Tag() >>> tag.yadda ' via class' From lacrima.maxim at gmail.com Thu Jan 21 15:58:58 2010 From: lacrima.maxim at gmail.com (Lacrima) Date: Thu, 21 Jan 2010 12:58:58 -0800 (PST) Subject: what test runner should I use? References: Message-ID: <3bebc5ef-f867-4566-a5b4-b1cc34fc7c81@b10g2000yqa.googlegroups.com> On Jan 19, 12:56?pm, Chris Withers wrote: > Hi All, > > I'm wondering what test runner I should use. Here's my list of requirements: > > - cross platform (I develop for and on Windows, Linux and Mac) > > - should not prevent tests from running with other test runners > ? ?(so no plugins/layers/etc that only work with one specific test > ? ? runner) > > - should work with zc.buildout (preferably without a specialist recipe!) > > So far I've tried the following with the resultant problems: > > zope.testing > > ? - requires a special recipe to be useful > ? - now emits deprecation warnings from itself: > ? ?https://mail.zope.org/pipermail/zope-dev/2009-December/038965.html > ? - coverage support is baroque to put it politely > > twisted's trial > > ? - only has old-style script definition in setup.py, so doesn't work > ? ? with buildout without hackery > > ? - drops _twisted_trial folders all over the place and doesn't clear > ? ? them up > > nose > > ? - can't see to get it to run only my packages tests, rather than > ? ? including the tests of packages my package depends on > > ? - seems to be focused towards files rather than modules > ? ? (which makes it not play nicely with buildout) > > ? - seems to be difficult to provide options to at configuration time > ? ? that can then be overridden on the command line > > I did also look at py.test's homepage but found it pretty scary. > > What other options do people recommend? > Failing that, any ideas how to fix the problems above? > > cheers, > > Chris Nose should work pretty well with buildout. You need this in your buildout.cfg [buildout] parts = test [test] recipe = pbp.recipe.noserunner eggs = yourpackage1 yourpackage2 This will generate test script (bin/test), which will search and run tests only in packages, specified in eggs option. From garrickp at gmail.com Thu Jan 21 16:01:33 2010 From: garrickp at gmail.com (Falcolas) Date: Thu, 21 Jan 2010 13:01:33 -0800 (PST) Subject: Closures in metaclasses References: <2c126442-f0b8-4b38-82c7-7fb226b15b60@u41g2000yqe.googlegroups.com> <7ceaaaa1-45c2-4875-a091-971c14e2d41e@f12g2000yqn.googlegroups.com> <48807f29-3157-43b8-9b01-37ccf5f00616@a15g2000yqm.googlegroups.com> Message-ID: <054fc7de-6706-4a41-9136-17ce0b437bf4@b10g2000yqa.googlegroups.com> On Jan 21, 1:55?pm, Peter Otten <__pete... at web.de> wrote: > Falcolas wrote: > > I tried overriding __getattr__ and got an error at runtime (the > > You can either move __getattr__() into the metaclass or instantiate the > class. I prefer the latter. > > Both approaches in one example: > > >>> class Tag: > > ... ? ? class __metaclass__(type): > ... ? ? ? ? ? ? def __getattr__(self, name): return "<%s> via metaclass" % > name > ... ? ? def __getattr__(self, name): return "<%s> via class" % name > ...>>> Tag.yadda > > ' via metaclass'>>> tag = Tag() > >>> tag.yadda > > ' via class' Very nice, thanks! From arnodel at googlemail.com Thu Jan 21 16:06:30 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Thu, 21 Jan 2010 21:06:30 +0000 Subject: Closures in metaclasses References: <2c126442-f0b8-4b38-82c7-7fb226b15b60@u41g2000yqe.googlegroups.com> <7ceaaaa1-45c2-4875-a091-971c14e2d41e@f12g2000yqn.googlegroups.com> <48807f29-3157-43b8-9b01-37ccf5f00616@a15g2000yqm.googlegroups.com> Message-ID: Falcolas writes: > On Jan 21, 12:10?pm, Arnaud Delobelle wrote: [...] >> Or you could override __getattr__ >> >> -- >> Arnaud > > I tried overriding __getattr__ and got an error at runtime (the > instance did not have xyz key, etc), and the Tag dict is not > modifiable (granted, I tried Tag.__dict__[tag] = spam, not setattr()). Yes that's because __getattr__ works on instances, not on the class itself. To achieve this, you'd have to override the *metaclass*'s __getattr__ method. But are you sure that you want lots of staticmethods? Why not instanciate your class Tag instead and have normal methods to generate tags? This would have the added benefit that you could set some options in the __init__ method of Tag to customize its behaviour. If you have a class which only contains static methods, you'd be better off with a module which contains good old functions. -- Arnaud From showell30 at yahoo.com Thu Jan 21 16:08:02 2010 From: showell30 at yahoo.com (Steve Howell) Date: Thu, 21 Jan 2010 13:08:02 -0800 (PST) Subject: simple pub/sub References: <0af0eff2-50e2-44aa-81f1-b3d12cb265a7@a15g2000yqm.googlegroups.com> Message-ID: On Jan 21, 9:37?am, a... at pythoncraft.com (Aahz) wrote: > In article <0af0eff2-50e2-44aa-81f1-b3d12cb26... at a15g2000yqm.googlegroups.com>, > Steve Howell ? wrote: > > > > >Hi, I'm looking for ideas on building a simple architecture that > >allows a bunch of independent Python processes to exchange data using > >files and perform calculations. > > SQLite? The data is just a stream of instrument readings, so putting it into a relational database doesn't really buy me much. So far "multiprocessing" seems to be the way to go. From jenn.duerr at gmail.com Thu Jan 21 16:17:59 2010 From: jenn.duerr at gmail.com (noydb) Date: Thu, 21 Jan 2010 13:17:59 -0800 (PST) Subject: Rounding up to the next 100 Message-ID: <2fff27ae-179b-4c60-95e3-c3eb6dfe94cc@n31g2000vbt.googlegroups.com> If one has a floating number as a string, is there a spiffy way to round that string-number UP to the nearest 100? XstrNmbr = 3579.127893 -- would want to round that to 3600. Thanks for any help! From aahz at pythoncraft.com Thu Jan 21 16:20:41 2010 From: aahz at pythoncraft.com (Aahz) Date: 21 Jan 2010 13:20:41 -0800 Subject: counting lines of code References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> <87bpgodj95.fsf@benfinney.id.au> <1f166703-fdfc-4f87-b336-ef80e492635b@a32g2000yqm.googlegroups.com> <7e09df6a-cda1-480e-a971-8f8a70ac4118@b9g2000yqd.googlegroups.com> Message-ID: In article <7e09df6a-cda1-480e-a971-8f8a70ac4118 at b9g2000yqd.googlegroups.com>, Phlip wrote: >On Jan 20, 11:20=A0pm, Michele Simionato >wrote: >> >> pylint does too many things, I want something fast that just counts >> the lines and can be run on thousands of files at once. >> cloc seems fine, I have just tried on 2,000 files and it gives me a >> report in just a few seconds. > >In my experience with Python codebases that big... > >...how many of those lines are duplicated, and might merge together >into a better design? Um... do you have any clue who you followed up to? If you don't, Google is your friend. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From g.bogle at auckland.no.spam.ac.nz Thu Jan 21 16:24:03 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Fri, 22 Jan 2010 10:24:03 +1300 Subject: py2exe and pydocs. Downloads? References: Message-ID: Gabriel Genellina wrote: > You found a bug. Looks like it depends on the environment, or what > packages are installed, or something like that, because it worked on my > other PC but not here. > Please report it at http://bugs.python.org so it doesn't get forgotten. > Done From Michael.Coll-Barth at VerizonWireless.com Thu Jan 21 16:30:13 2010 From: Michael.Coll-Barth at VerizonWireless.com (Michael.Coll-Barth at VerizonWireless.com) Date: Thu, 21 Jan 2010 16:30:13 -0500 Subject: Rounding up to the next 100 In-Reply-To: References: Message-ID: > From: noydb > If one has a floating number as a string, is there a spiffy way to > round that string-number UP to the nearest 100? > > XstrNmbr = 3579.127893 -- would want to round that to 3600. What's wrong with round? round( XstrNmbr, -2 ) seems to do the trick. Or do you want to get rid of the decimal point as well? The information contained in this message and any attachment may be proprietary, confidential, and privileged or subject to the work product doctrine and thus protected from disclosure. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify me immediately by replying to this message and deleting it and all copies and backups thereof. Thank you. From peter.milliken at gmail.com Thu Jan 21 16:30:29 2010 From: peter.milliken at gmail.com (Peter) Date: Thu, 21 Jan 2010 13:30:29 -0800 (PST) Subject: Python and Tkinter Programming by John Grayson References: <0738732e-d8df-469d-be4a-3699f4fd5f64@j14g2000yqm.googlegroups.com> <801d251d-ae12-4ce6-95a1-58f594cd2f81@p8g2000yqb.googlegroups.com> <4B4F96D6.2070905@codebykevin.com> Message-ID: <326dc723-3909-4293-9bf6-48a367b9251c@e16g2000yqc.googlegroups.com> On Jan 15, 9:12?am, Kevin Walzer wrote: > On 1/14/10 3:39 PM, Peter wrote: > > > > > > > On Jan 15, 6:24 am, Mark Roseman ?wrote: > >> ? Peter ?wrote: > >>> Besides, the book is mainly about using Python with Tkinter - and > >>> Tkinter hasn't changed that much since 2000, so I believe it is just > >>> as relevant today as it was back then. > > >> I'd say that Tkinter has substantially changed - with the introduction > >> of the 'ttk' themed widgets. ?I cover these in my tutorial athttp://www.tkdocs.com > > >> Mark > > > But is the ttk themed widgets a "change" to Tkinter or an "addition/ > > improvement"? i.e. does the themed widgets invalidate any of the > > Tkinter stuff in Grayson's book? That certainly isn't my impression of > > the themed widgets, I was of the impression that the themed widgets > > just added some convenient widgets (such as Scale, Spinbox etc) which > > a user could find in other GUI frameworks, for example, Pmw > > supplemented the basic Tkinter widget set and offered some (essential > > IMO) widgets that were missing in Tkinter - such as the Notebook > > widget. > > It's both a change and an improvement. The themed widgets can be used in > conjunction with the traditional widgets, but in many cases it's > possible (and desirable) to update your entire application to use the > themed widgets. > > > Lets face it, if somebody wants to get up to speed on Python and GUI > > development then the book is still very, very relevant and necessary > > (IMO). The documentation for the themed widgets still leaves a lot to > > be desired from the perspective of somebody who wants to start using > > Python to create GUI applications. As Lord Eldritch reminded me in his > > post, the book even has a section on Pmw - which is what I use mainly > > for my GUI applications - because Tkinter was missing some vital > > widgets that are now available in the ttk themed set. > > > Personally I will start to incorporate some of the ttk themed widgets > > into my applications - but Pmw will remain the 'basis' for my GUI's as > > the entire framework (IMO) supports a class oriented approach that > > allows easy creation of extensible and reconfigurable (at run time) > > GUI interfaces. > > PMW is certainly a helpful addition to the Tkinter developer's toolbox, > but it also has limitations. It suffers from some of the same > limitations as Tk itself, i.e. it is rather dated in appearance, and > even it lacks some modern UI features such as a good multicolumn > listbox, treeview, etc. In addition to the ttk themed widgets, other, > more configurable pure Tk-based megawidget packages exist, such as > BWidgets and Tablelist, and there are Python wrappers for these at the > Tkinter wiki (http://tkinter.unpythonic.net/wiki/). > > > > > Ultimately Grayson does a good job of providing information and > > reference to toolkit(s) that allow a beginner to quickly get up to > > speed on producing a GUI using Python. It is purely up to the user > > afterwards as to whether they stick with Tkinter/Pmw (and now the ttk > > themed set) or venture into wxPython or Jython (as two examples of GUI > > 'systems' that provide 'better' facilities to a Python programmer). > > Another book I've found very helpful for learning Tkinter is Programming > Python by Mark Lutz--a lot of coverage there of GUI development. > > --Kevin > > -- > Kevin Walzer > Code by Kevinhttp://www.codebykevin.com Another possible consideration when choosing a GUI to learn Python - will you want to print from within your GUI application? Admittedly, I haven't looked to see whether this situation has changed or not for some years now, but a GUI based on Tkinter (whether it is ttk or Pmw or whatever) has no support for "printing". Other GUI frameworks, such as wxPython, Jython, PyQT etc have API's that allow printing of information from within your GUI application. Now if you go the Tk route, I am sure you can use the Win32 bindings to do printing - but I personally could never work them out :-) Maybe there is some reasonable documentation or example code for doing that these days - but there wasn't when I first approached the problem (which was back in the python 2.1 days - ancient history!). Printing using Windows API was a complete mystery to me. My first GUI application required information to be printed out - I wrote the whole thing using Pmw and Tkinter - only to find that I had no way of getting stuff to a printer (I was learning Python and Tkinter at the time :-)). I briefly explored "porting" my application to wxPython or Jython because both frameworks provided printer API functions - at that time, PyQT was commercial license only for a PC, so that wasn't an option. I found wxPython stunningly hard to understand from the documentation (I have since purchased wxPython in Action by Rappin and Dunn but have never found the time to read it). I had some problems porting the code to Jython (my code relied heavily on the pickle module and Jython had some severe bugs in that area at the time), so in the end I came up with a fairly micky-mouse system that placed information that required printing into .txt files and the GUI application started Microsoft Word using mail-merge templates and I got the information to the printer that way - not ideal! But it worked :-) So consider that in your decision process - maybe somebody can (kindly :-)) hop in here and say "that's easy, just look at xyz or read this book abc" to find out how to print from within an application using Tkinter. Regards Peter From phlip2005 at gmail.com Thu Jan 21 16:31:37 2010 From: phlip2005 at gmail.com (Phlip) Date: Thu, 21 Jan 2010 13:31:37 -0800 Subject: counting lines of code In-Reply-To: References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> <87bpgodj95.fsf@benfinney.id.au> <1f166703-fdfc-4f87-b336-ef80e492635b@a32g2000yqm.googlegroups.com> <7e09df6a-cda1-480e-a971-8f8a70ac4118@b9g2000yqd.googlegroups.com> Message-ID: <%I36n.5962$4p5.2277@newsfe22.iad> Aahz wrote: > In article <7e09df6a-cda1-480e-a971-8f8a70ac4118 at b9g2000yqd.googlegroups.com>, > Phlip wrote: >> On Jan 20, 11:20=A0pm, Michele Simionato >> wrote: >>> pylint does too many things, I want something fast that just counts >>> the lines and can be run on thousands of files at once. >>> cloc seems fine, I have just tried on 2,000 files and it gives me a >>> report in just a few seconds. >> In my experience with Python codebases that big... >> >> ...how many of those lines are duplicated, and might merge together >> into a better design? > > Um... do you have any clue who you followed up to? If you don't, Google > is your friend. Oh, sorry, did I have the wrong opinion? From arnodel at googlemail.com Thu Jan 21 16:32:12 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Thu, 21 Jan 2010 21:32:12 +0000 Subject: Rounding up to the next 100 References: <2fff27ae-179b-4c60-95e3-c3eb6dfe94cc@n31g2000vbt.googlegroups.com> Message-ID: noydb writes: > If one has a floating number as a string, is there a spiffy way to > round that string-number UP to the nearest 100? > > XstrNmbr = 3579.127893 -- would want to round that to 3600. > > Thanks for any help! >>> XstrNmbr = 3579.127893 >>> round(float(XstrNmbr), -2) 3600.0 >>> HTH -- Arnaud From grahn+nntp at snipabacken.se Thu Jan 21 16:33:18 2010 From: grahn+nntp at snipabacken.se (Jorgen Grahn) Date: 21 Jan 2010 21:33:18 GMT Subject: py2exe and pydocs. Downloads? References: Message-ID: On Thu, 2010-01-21, Gib Bogle wrote: > Gabriel Genellina wrote: > >> You found a bug. Looks like it depends on the environment, or what >> packages are installed, or something like that, because it worked on my >> other PC but not here. >> Please report it at http://bugs.python.org so it doesn't get forgotten. >> > > Done And for reference, it's , "pydoc error". /Jorgen -- // Jorgen Grahn O o . From arnodel at googlemail.com Thu Jan 21 16:36:06 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Thu, 21 Jan 2010 21:36:06 +0000 Subject: Rounding up to the next 100 References: <2fff27ae-179b-4c60-95e3-c3eb6dfe94cc@n31g2000vbt.googlegroups.com> Message-ID: Arnaud Delobelle writes: > >>> XstrNmbr = 3579.127893 I meant >>> XstrNmbr = "3579.127893" > >>> round(float(XstrNmbr), -2) > 3600.0 -- Arnaud From diesch at spamfence.net Thu Jan 21 16:51:33 2010 From: diesch at spamfence.net (Florian Diesch) Date: Thu, 21 Jan 2010 22:51:33 +0100 Subject: Rounding up to the next 100 References: <2fff27ae-179b-4c60-95e3-c3eb6dfe94cc@n31g2000vbt.googlegroups.com> Message-ID: <5sum27-m3d.ln1@mid.florian-diesch.de> noydb writes: > If one has a floating number as a string, is there a spiffy way to > round that string-number UP to the nearest 100? > > XstrNmbr = 3579.127893 -- would want to round that to 3600. math.ceil(3579.127893/100)*100 Florian -- GUIs programmieren mit Python und Glade: From Martin.Drautzburg at web.de Thu Jan 21 16:51:34 2010 From: Martin.Drautzburg at web.de (Martin Drautzburg) Date: Thu, 21 Jan 2010 22:51:34 +0100 Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: <5552238.LgDixP2R6m@beaureve.gmx.net> Thanks for all the answers. Let me summarize (1) I fail to see the relevance of ?>>> def move( direction ): ... ? print( "move " + str( direction ) ) ... ?>>> move( "up" ) move up not only in the context of my question. And I don't see an abuse of the language either. Maybe this could pass as a Zen Puzzle. (2) Using enum's was suggested. That is good to know, but again it is just a way to define constants in the caller's namespace. (3) Then somone suggested to tie the constants to the function itself, as in def move(direction): ? ? print "moving %s" % direction move.UP = 'up' move.DOWN = 'down' This is quite nice. Then again the "move." is just some object which allows attributes, and which only happens to have the same name as the function. Well in this case it IS the function, alright, but I could just as well have used a Class as in class m: pass m.UP = 'up' (4) Finally someone mentioned DSLs. I guess thats absolutely correct. This is what I am struggeling to achieve. I did a little googling ("how to write DSLs in python"), but haven't found anything appealing yet. Any pointers would be appreciated. (5) Here is something I came up with myself: def symbols(aDict): aDict["foo"] = "bar" def someFunction(aFoo): print aFoo symbols(locals()) someFunction (foo) #Eh voila: foo is magically defined prints: bar The call to symbols(locals()) is the "magic, magic" I supected would be required in my original posting. If someFunction was a member of a class, the symbols would be properly tied to that class (albeit not the individual function), but still good enough. I suppose I could wrap it in a decorator, which would also do the "unmagic". In any case, getting the context right seems to be the biggest problem. If I don't want to pollute my namespace, those symbols need to be defined in some context but undefined in others. AFAIK there are not really "blocks" in python and lexical scoping is present primarily in functions. From jenn.duerr at gmail.com Thu Jan 21 16:52:36 2010 From: jenn.duerr at gmail.com (noydb) Date: Thu, 21 Jan 2010 13:52:36 -0800 (PST) Subject: Rounding up to the next 100 References: Message-ID: <74d51f20-b693-4ab2-809f-6f9df6de08c0@r19g2000yqb.googlegroups.com> On Jan 21, 4:30?pm, Michael.Coll-Ba... at VerizonWireless.com wrote: > > From: noydb > > If one has a floating number as a string, is there a spiffy way to > > round that string-number UP to the nearest 100? > > > XstrNmbr = 3579.127893 -- would want to round that to 3600. > > What's wrong with round? ?round( XstrNmbr, -2 ) seems to do the trick. > Or do you want to get rid of the decimal point as well? > > The information contained in this message and any attachment may be > proprietary, confidential, and privileged or subject to the work > product doctrine and thus protected from disclosure. ?If the reader > of this message is not the intended recipient, or an employee or > agent responsible for delivering this message to the intended > recipient, you are hereby notified that any dissemination, > distribution or copying of this communication is strictly prohibited. > If you have received this communication in error, please notify me > immediately by replying to this message and deleting it and all > copies and backups thereof. ?Thank you. Thanks Arnaud! Michael - Nothing is wrong with round -- when I tried it initially, I was confused on the base -- seeing it from this example helped clear it up. From alfps at start.no Thu Jan 21 16:53:27 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 22:53:27 +0100 Subject: Rounding up to the next 100 In-Reply-To: References: Message-ID: * Michael.Coll-Barth at VerizonWireless.com: > > >> From: noydb > >> If one has a floating number as a string, is there a spiffy way to >> round that string-number UP to the nearest 100? >> >> XstrNmbr = 3579.127893 -- would want to round that to 3600. > > > What's wrong with round? round( XstrNmbr, -2 ) seems to do the trick. > Or do you want to get rid of the decimal point as well? Perhaps completely irrelevant, but just in passing, round() changed semantics from 2.x to 3.x, in 3.x always returning int when called with just 1 argument: >>> import sys >>> sys.version '2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]' >>> print round.__doc__ round(number[, ndigits]) -> floating point number Round a number to a given precision in decimal digits (default 0 digits). This always returns a floating point number. Precision may be negative. >>> _ >>> import sys >>> sys.version '3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)]' >>> print( round.__doc__ ) round(number[, ndigits]) -> number Round a number to a given precision in decimal digits (default 0 digits). This returns an int when called with one argument, otherwise the same type as the number. ndigits may be negative. >>> _ Might be useful to know regarding "get rid of the decimal point": in 3.x round(x) does that, in 2.x it doesn't. Cheers, - Alf From jenn.duerr at gmail.com Thu Jan 21 16:58:53 2010 From: jenn.duerr at gmail.com (noydb) Date: Thu, 21 Jan 2010 13:58:53 -0800 (PST) Subject: Rounding up to the next 100 References: <74d51f20-b693-4ab2-809f-6f9df6de08c0@r19g2000yqb.googlegroups.com> Message-ID: Sorry, although what I really need is the string-number rounded UP every time. So if the number is 3890.32, it needs to go to 3900; if the number is 3811.345, it needs to go to 3900 also. So, Florian's answer works. From Michael.Coll-Barth at VerizonWireless.com Thu Jan 21 17:03:50 2010 From: Michael.Coll-Barth at VerizonWireless.com (Michael.Coll-Barth at VerizonWireless.com) Date: Thu, 21 Jan 2010 17:03:50 -0500 Subject: Rounding up to the next 100 In-Reply-To: References: <2fff27ae-179b-4c60-95e3-c3eb6dfe94cc@n31g2000vbt.googlegroups.com> Message-ID: > From: Arnaud Delobelle > > > > >>> XstrNmbr = 3579.127893 > > I meant > >>> XstrNmbr = "3579.127893" > > > >>> round(float(XstrNmbr), -2) > > 3600.0 Ah, then you will need to cast it first. >>> XstrNmbr = '3579.127893' >>> round(float(XstrNmbr) ,-2) 3600.0 The information contained in this message and any attachment may be proprietary, confidential, and privileged or subject to the work product doctrine and thus protected from disclosure. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify me immediately by replying to this message and deleting it and all copies and backups thereof. Thank you. From g.bogle at auckland.no.spam.ac.nz Thu Jan 21 17:10:46 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Fri, 22 Jan 2010 11:10:46 +1300 Subject: py2exe and pydocs. Downloads? References: Message-ID: Gabriel Genellina wrote: > En Wed, 20 Jan 2010 21:22:19 -0300, Gib Bogle > escribi?: > >> Gabriel Genellina wrote: >> >>> c:\temp>python -m pydoc sys >>> Help on built-in module sys: >>> [...same info...] >> >> When I do this I get: >> >> No module named tempfile > > You found a bug. Looks like it depends on the environment, or what > packages are installed, or something like that, because it worked on my > other PC but not here. > Please report it at http://bugs.python.org so it doesn't get forgotten. > The bug checkers are not able to reproduce the error. What is your other PC running? From alfps at start.no Thu Jan 21 17:20:08 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 23:20:08 +0100 Subject: Rounding up to the next 100 In-Reply-To: References: <74d51f20-b693-4ab2-809f-6f9df6de08c0@r19g2000yqb.googlegroups.com> Message-ID: * noydb: > Sorry, although what I really need is the string-number rounded UP > every time. So if the number is 3890.32, it needs to go to 3900; if > the number is 3811.345, it needs to go to 3900 also. > > So, Florian's answer works. You might also consider -100*(-3579.127893//100) :-) Which avoids the math.ceil but assumes the number is positive (or zero). Cheers & hth., - Alf PS: Note that this trick doesn't work with most other common languages that I'm familiar with, since the round towards zero instead of down to minus infinity, but Python has more clean semantics in this regard. From aharrisreid at googlemail.com Thu Jan 21 17:34:00 2010 From: aharrisreid at googlemail.com (Alan Harris-Reid) Date: Thu, 21 Jan 2010 22:34:00 +0000 Subject: Dynamic HTML controls In-Reply-To: References: Message-ID: <4B58D658.3080609@googlemail.com> Aahz wrote: > In article , > Alan Harris-Reid wrote: > >> Does anyone know where I can find any decent dynamically-constructed >> HTML control classes (dropdown list, table, input field, checkbox, etc.) >> written in Python. For example, for a HTML table I would like something >> like... >> > > You might look at Quixote: > http://quixote.ca/ > Thanks for that Aahz - I'll check it out. Regards, Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From alfps at start.no Thu Jan 21 17:49:59 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 21 Jan 2010 23:49:59 +0100 Subject: Symbols as parameters? In-Reply-To: <5552238.LgDixP2R6m@beaureve.gmx.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> Message-ID: * Martin Drautzburg: > Thanks for all the answers. Let me summarize > > (1) I fail to see the relevance of > >>> def move( direction ): > ... print( "move " + str( direction ) ) > ... > >>> move( "up" ) > move up > > not only in the context of my question. And I don't see an abuse of the > language either. Maybe this could pass as a Zen Puzzle. Oh. You focused on the wrong details, then. Sorry for not providing a textual explanation, but I thought anyone not understanding it would just ask (that's normal in other Usenet groups that I'm familiar with). So, explanation... Here's your problem description, from the start of the thread: What I am really looking for is a way - to be able to call move(up) - having the "up" symbol only in the context of the function call So it should look something like this ... magic, magic ... move(up) ... unmagic, unmagic ... print up This should complain that "up" is not defined during the "print" call, but not when move() is called. And of course there should be as little magic as possible. The code corresponding to your first "magic, magic ..." is class using_directions: # Whatever statement (import?) defines symbolic directions goes here Then comes your -- note the added indentation move( up ) Your "... unmagic, unmagic" is simply to go back to the previous indent level. So your example's following print up at this point complaints that "up" is not defined, as you require (no general namespace pollution, although there is a slight but controllable pollution, namely the class name). Not that I recommend this technique. ;-) But it does exactly what you asked, quoted above. The best technique in my view is what you came up with yourself in the article I'm responding to (but snipped by me), namely a class with the constants and the function. It doesn't do what you required originally. But it's much better! :-) Cheers & hth., - Alf From Martin.Drautzburg at web.de Thu Jan 21 17:57:37 2010 From: Martin.Drautzburg at web.de (Martin Drautzburg) Date: Thu, 21 Jan 2010 23:57:37 +0100 Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> Message-ID: <7174001.cQaEHfC5Mm@beaureve.gmx.net> Here is a complete expample using a decorator, still a bit noisy def move(aDirection): print "moving " + aDirection #Here comes the decorator def scope(aDict): def save(locals): """Set symbols in locals and remember their original state""" setSymbols={} unsetSymbols=[] for i in ("up", "down", "left", "right"): if locals.has_key(i): setSymbols[i] = locals[i] else: unsetSymbols.append(i) # define the new symbols locals[i] = i return setSymbols, unsetSymbols def restore (locals, set, unset): """restore locals from set and unset""" for i in set.keys(): locals[i] = set[i] for i in unset: del(locals[i]) def callFunc(f): """Main decorator""" set, unset = save(aDict) f() restore(aDict, set, unset) return callFunc # -------------------------------------- # using it # -------------------------------------- # a variable defined in the outer scope up="outerScopeUp" # magic, magic (still too noisy for my taste) @scope (locals()) def _(): move(up) move(down) move(left) move(right) #verify the the outer scope variable hasn't changed print "in the outer scope up is still:", up print print "this should fail:" down # -------------------------------------- # Output # -------------------------------------- moving up moving down moving left moving right in the outer scope up is still: outerScopeUp this should fail: Traceback (most recent call last): File "", line 50, in NameError: name 'down' is not defined From deets at nospam.web.de Thu Jan 21 18:16:31 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Fri, 22 Jan 2010 00:16:31 +0100 Subject: Symbols as parameters? In-Reply-To: <5552238.LgDixP2R6m@beaureve.gmx.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> Message-ID: <7rs5ihFjqjU1@mid.uni-berlin.de> Am 21.01.10 22:51, schrieb Martin Drautzburg: > Thanks for all the answers. Let me summarize > > (1) I fail to see the relevance of > >>> def move( direction ): > ... print( "move " + str( direction ) ) > ... > >>> move( "up" ) > move up > > not only in the context of my question. And I don't see an abuse of the > language either. Maybe this could pass as a Zen Puzzle. > > (2) Using enum's was suggested. That is good to know, but again it is > just a way to define constants in the caller's namespace. > > (3) Then somone suggested to tie the constants to the function itself, > as in > def move(direction): > print "moving %s" % direction > > move.UP = 'up' > move.DOWN = 'down' > > This is quite nice. Then again the "move." is just some object which > allows attributes, and which only happens to have the same name as the > function. Well in this case it IS the function, alright, but I could > just as well have used a Class as in > > class m: pass > m.UP = 'up' > > > (4) Finally someone mentioned DSLs. I guess thats absolutely correct. > This is what I am struggeling to achieve. I did a little googling ("how > to write DSLs in python"), but haven't found anything appealing yet. > Any pointers would be appreciated. > > (5) Here is something I came up with myself: > > def symbols(aDict): > aDict["foo"] = "bar" > > def someFunction(aFoo): > print aFoo > > symbols(locals()) > someFunction (foo) #Eh voila: foo is magically defined http://docs.python.org/library/functions.html#locals If it works, it's more an accident than anything else, and can go away without prior notice. Diez From pavlovevidence at gmail.com Thu Jan 21 18:19:58 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Thu, 21 Jan 2010 15:19:58 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> Message-ID: <89e6f34c-1a7e-4c2e-88cb-ee0397678948@k17g2000yqh.googlegroups.com> On Jan 21, 1:51?pm, Martin Drautzburg wrote: > Thanks for all the answers. Let me summarize [snip] > (2) Using enum's was suggested. That is good to know, but again it is > just a way to define constants in the caller's namespace. It'll at least corral the symbols you want. [snip] > (4) Finally someone mentioned DSLs. I guess thats absolutely correct. > This is what I am struggeling to achieve. I see. Well, Python is a poor choice for defining an internal DSL (i.e., DSL using the general language's syntax), because it's (deliberately) rigid in both grammar and semantics. Soon as you want to do something a little different you're out of luck. Python is somewhat better suited for external DSLs. > I did a little googling ("how > to write DSLs in python"), but haven't found anything appealing yet. > Any pointers would be appreciated. Paul McGuire should be by to recommend PyParsing shortly. It's not really hard to write a simple DSL if you know how. It is pretty hard to learn how, though. Tools like PyParsing help a lot. > (5) Here is something I came up with myself: > > def symbols(aDict): > ? ? aDict["foo"] = "bar" > > def someFunction(aFoo): > ? ? print aFoo > > symbols(locals()) > someFunction (foo) #Eh voila: foo is magically defined > > prints: bar > > The call to symbols(locals()) is the "magic, magic" I supected would be > required in my original posting. If someFunction was a member of a > class, the symbols would be properly tied to that class (albeit not the > individual function), but still good enough. I suppose I could wrap it > in a decorator, which would also do the "unmagic". > > In any case, getting the context right seems to be the biggest problem. > If I don't want to pollute my namespace, those symbols need to be > defined in some context but undefined in others. AFAIK there are not > really "blocks" in python and lexical scoping is present primarily in > functions. That's pretty much the issue. BTW, I'm sorry that the thread got a little flamey there. Carl Banks From robert.kern at gmail.com Thu Jan 21 18:52:24 2010 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 21 Jan 2010 17:52:24 -0600 Subject: counting lines of code In-Reply-To: <%I36n.5962$4p5.2277@newsfe22.iad> References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> <87bpgodj95.fsf@benfinney.id.au> <1f166703-fdfc-4f87-b336-ef80e492635b@a32g2000yqm.googlegroups.com> <7e09df6a-cda1-480e-a971-8f8a70ac4118@b9g2000yqd.googlegroups.com> <%I36n.5962$4p5.2277@newsfe22.iad> Message-ID: On 2010-01-21 15:31 , Phlip wrote: > Aahz wrote: >> In article >> <7e09df6a-cda1-480e-a971-8f8a70ac4118 at b9g2000yqd.googlegroups.com>, >> Phlip wrote: >>> On Jan 20, 11:20=A0pm, Michele Simionato >>> wrote: >>>> pylint does too many things, I want something fast that just counts >>>> the lines and can be run on thousands of files at once. >>>> cloc seems fine, I have just tried on 2,000 files and it gives me a >>>> report in just a few seconds. >>> In my experience with Python codebases that big... >>> >>> ...how many of those lines are duplicated, and might merge together >>> into a better design? >> >> Um... do you have any clue who you followed up to? If you don't, Google >> is your friend. > > Oh, sorry, did I have the wrong opinion? You had a condescending attitude. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ben+python at benfinney.id.au Thu Jan 21 20:06:13 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 22 Jan 2010 12:06:13 +1100 Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <89e6f34c-1a7e-4c2e-88cb-ee0397678948@k17g2000yqh.googlegroups.com> Message-ID: <87d413c5iy.fsf@benfinney.id.au> Carl Banks writes: > On Jan 21, 1:51?pm, Martin Drautzburg > wrote: > > (2) Using enum's was suggested. That is good to know, but again it > > is just a way to define constants in the caller's namespace. > > It'll at least corral the symbols you want. It also satisfies the strongly-expressed demand you had for these constants to *not* be strings. It keeps the semantic meaning of these constants very distinct, and greatly reduces the possibility of an object accidentally having the same value as one of these constants. -- \ ?Roll dice!? ?Why?? ?Shut up! I don't need your fucking | `\ *input*, I need you to roll dice!? ?Luke Crane, demonstrating | _o__) his refined approach to play testing, 2009 | Ben Finney From ethan at stoneleaf.us Thu Jan 21 20:18:23 2010 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 21 Jan 2010 17:18:23 -0800 Subject: Python and Tkinter Programming by John Grayson In-Reply-To: <326dc723-3909-4293-9bf6-48a367b9251c@e16g2000yqc.googlegroups.com> References: <0738732e-d8df-469d-be4a-3699f4fd5f64@j14g2000yqm.googlegroups.com> <801d251d-ae12-4ce6-95a1-58f594cd2f81@p8g2000yqb.googlegroups.com> <4B4F96D6.2070905@codebykevin.com> <326dc723-3909-4293-9bf6-48a367b9251c@e16g2000yqc.googlegroups.com> Message-ID: <4B58FCDF.4000402@stoneleaf.us> Peter wrote: > On Jan 15, 9:12 am, Kevin Walzer wrote: >>> On Jan 15, 6:24 am, Mark Roseman wrote: >>>> Peter wrote: >>>>> Besides, the book is mainly about using Python with Tkinter - and >>>>> Tkinter hasn't changed that much since 2000, so I believe it is just >>>>> as relevant today as it was back then. >>>> I'd say that Tkinter has substantially changed - with the introduction >>>> of the 'ttk' themed widgets. I cover these in my tutorial athttp://www.tkdocs.com >> Another book I've found very helpful for learning Tkinter is Programming >> Python by Mark Lutz--a lot of coverage there of GUI development. >> > > Another possible consideration when choosing a GUI to learn Python - > will you want to print from within your GUI application? Excellent point. Many thanks to all who responded, especially for the reminder of the gui sections in Programming Python (forgot I had that book!). I'll start by going over that again, and we'll see how confident I feel afterwards. ;) ~Ethan~ From tekion at gmail.com Thu Jan 21 20:48:36 2010 From: tekion at gmail.com (tekion) Date: Thu, 21 Jan 2010 17:48:36 -0800 (PST) Subject: deriving MySQLdb class Message-ID: All, I am trying to write a class which inherits from MySQLdb class. Below is code snippet: import MySQLdb import sys class msql_connect(MySQLdb): def __init__(self): self.host = "hostname" self.user = "user" self.password = "passoword" self.database = "database name" I am running into below error: class msql_connect(MySQLdb): TypeError: Error when calling the metaclass bases module.__init__() takes at most 2 arguments (3 given) Does any one have an idea why? Thanks. From gburdell1 at gmail.com Thu Jan 21 20:56:00 2010 From: gburdell1 at gmail.com (gburdell1 at gmail.com) Date: Thu, 21 Jan 2010 17:56:00 -0800 (PST) Subject: Default return values for out-of-bounds list item Message-ID: Is there a built-in method in python that lets you specify a "default" value that will be returned whenever you try to access a list item that is out of bounds? Basically, it would be a function like this: def item(x,index,default): try: return x[index] except IndexError: return default So if a=[0,1,2,3], then item(a,0,44)=0, item(a,1,44)=1, and item(a, 1000,44)=44, item(a,-1000,44)=44 What I want to know is whether there is a built-in method or notation for this. What if, for example, we could do something like a [1000,44] ? From steve at REMOVE-THIS-cybersource.com.au Thu Jan 21 21:20:44 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 02:20:44 GMT Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> Message-ID: <0368f809$0$1357$c3e8da3@news.astraweb.com> On Thu, 21 Jan 2010 22:51:34 +0100, Martin Drautzburg wrote: > Thanks for all the answers. Let me summarize > > (1) I fail to see the relevance of > ?>>> def move( direction ): > ... ? print( "move " + str( direction ) ) ... > ?>>> move( "up" ) > move up I'm glad it's not just me then. > not only in the context of my question. And I don't see an abuse of the > language either. Maybe this could pass as a Zen Puzzle. > > (2) Using enum's was suggested. That is good to know, but again it is > just a way to define constants in the caller's namespace. I think this really is the correct solution for your problem. In Python, the standard place to have such public constants is at the module level, not the function or class. I think you're worrying unnecessarily about "namespace pollution" -- the module namespace is *exactly* the right place for them. If two functions both need UP, DOWN, etc symbols, then either: (1) they can just use the same symbols; or (2) if they can't, then they don't belong in the same module. An example from the standard library: the re module defines constants I, L, M, etc. representing flags that are passed to the re.compile. They are implemented as integers so they can easily be combined with &, but another implementation might use symbols. You will notice that they're not limited to the re.compile function itself. The caller may very well want to do something like this: # Get some flags for compile: flags = re.I & re.M # ... # much later on # x = re.compile(s, flags) You would force them to do this: # Get some flags for compile: flags = re.compile.I & re.compile.M # ... # much later on # x = re.compile(s, flags) which is, in my opinion, a needless level of indirection and possibly in violation of Demeter's Law. > (3) Then somone suggested to tie the constants to the function itself, > as in > def move(direction): > ? ? print "moving %s" % direction > > move.UP = 'up' > move.DOWN = 'down' > > This is quite nice. I would call it a horrible, horrible, horrible code smell. A stench in fact. In my opinion, such attributes tied to the function should be treated as internal to the function, and not the public interface. I wouldn't go quite so far as to say they should be treated as private, but having the caller use them should be rare and unusual. > Then again the "move." is just some object which > allows attributes, and which only happens to have the same name as the > function. Well in this case it IS the function, alright, but I could > just as well have used a Class as in > > class m: pass > m.UP = 'up' Either way, when you go to *use* the direction, you're still passing a string. There's no difference between: move(m.UP) and just move("up") Furthermore, the extra layer of indirection with the m.* doesn't give you anything useful. Think about using this: # choose a direction at random direction = random.choice([m.UP, m.DOWN, m.LEFT, m.RIGHT]) move(direction) What benefit is the extra layer of indirection? It just adds noise to the code. Surely this is better? UP, DOWN, LEFT, RIGHT = "up down left right".strip() direction = random.choice([UP, DOWN, LEFT, RIGHT]) move(direction) That's much clearer. In my opinion, if you want to prohibit users from passing a string (or integer) equal to your constants, so that move('up') does not work (in other words, they are forced to use the constants you provide) then Ben Finney's enum solution is probably the correct way to do it. But if you don't care, then the simplest solution is to define the constants you care about in the module, using either strings or ints, and then let the caller choose between using your named constants or not: move(UP) move('up') > (4) Finally someone mentioned DSLs. I guess thats absolutely correct. > This is what I am struggeling to achieve. I did a little googling ("how > to write DSLs in python"), but haven't found anything appealing yet. Any > pointers would be appreciated. That truly is using a bulldozer to crack a peanut. > (5) Here is something I came up with myself: > > def symbols(aDict): > aDict["foo"] = "bar" > > def someFunction(aFoo): > print aFoo > > symbols(locals()) > someFunction (foo) #Eh voila: foo is magically defined > prints: bar > > The call to symbols(locals()) is the "magic, magic" I supected would be > required in my original posting. If someFunction was a member of a > class, the symbols would be properly tied to that class (albeit not the > individual function), but still good enough. I disagree about it being "proper" to tie such public symbols to the class. But in any case, what you're trying to do is not supported by Python. If it works, that's a happy accident. "The contents of this dictionary should not be modified; changes may not affect the values of local and free variables used by the interpreter." http://docs.python.org/library/functions.html#locals > I suppose I could wrap it > in a decorator, which would also do the "unmagic". > > In any case, getting the context right seems to be the biggest problem. > If I don't want to pollute my namespace, It's not pollution. The module namespace is the right place for such public constants. -- Steven From steve at REMOVE-THIS-cybersource.com.au Thu Jan 21 21:21:58 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 02:21:58 GMT Subject: Create object name from string value? References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> Message-ID: <0368f853$0$1357$c3e8da3@news.astraweb.com> On Thu, 21 Jan 2010 06:23:17 -0800, Gnarlodious wrote: >> The right way to solve this problem is with a dictionary: >> >> for name in ["object1", "object2", "object3"]: >> ? ? d = {name: classname()} >> ? ? print d[name] > > This works! However I end up saying: > > d['Server'].Config.BaseURL > > to get the data, when I should be saying: > > Server.Config.BaseURL There are three moments in time that the programmer needs to care about: when you write the code, when you compile the code, and when you run the code. In this case, the two relevant ones are write-time and run-time. If you expect to be able to write Server.Config.BaseURL then that means that you MUST know the name "Server" at write-time. (Otherwise, how do you know to write "Server", instead of "MyServer" or "ServerSix"?) That means that the *name* Server isn't specified at runtime, but at write-time. Since you already know the name, then the right solution is to do something like: Server = classname() and then refer to Server, just like you would do: x = 45 print x+1 or similar. But if you are getting the name "Server" from a config file at runtime, say something like this: name=Server class=classname flag=True then you can't write: Server.Config.BaseURL in your source code, because you don't know if it will be called "Server", or "MyServer" or "Server42", or "WebServe", or "Fred". In general, you don't even know how many servers there will be: there could be one, or none, or fifty. So you need a level of indirection, hence the dictionary. -- Steven From steve at REMOVE-THIS-cybersource.com.au Thu Jan 21 21:28:23 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 02:28:23 GMT Subject: Default return values for out-of-bounds list item References: Message-ID: <0368f9d5$0$1357$c3e8da3@news.astraweb.com> On Thu, 21 Jan 2010 17:56:00 -0800, gburdell1 at gmail.com wrote: > Is there a built-in method in python that lets you specify a "default" > value that will be returned whenever you try to access a list item that > is out of bounds? No. > Basically, it would be a function like this: > > def item(x,index,default): > try: > return x[index] > except IndexError: > return default That's probably the best way to do it. > So if a=[0,1,2,3], then item(a,0,44)=0, item(a,1,44)=1, and item(a, > 1000,44)=44, item(a,-1000,44)=44 > > What I want to know is whether there is a built-in method or notation > for this. What if, for example, we could do something like a [1000,44] ? You can use slicing instead: >>> a=[0,1,2,3] >>> a[2:3] [2] >>> a[100:101] [] and then detect the empty list and use default: def item(x, index, default): a = x[index:index+1] return a[0] if a else default -- Steven From python at mrabarnett.plus.com Thu Jan 21 21:41:05 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 22 Jan 2010 02:41:05 +0000 Subject: Default return values for out-of-bounds list item In-Reply-To: References: Message-ID: <4B591041.8080905@mrabarnett.plus.com> gburdell1 at gmail.com wrote: > Is there a built-in method in python that lets you specify a "default" > value that will be returned whenever you try to access a list item > that is out of bounds? Basically, it would be a function like this: > > def item(x,index,default): > try: > return x[index] > except IndexError: > return default > > So if a=[0,1,2,3], then item(a,0,44)=0, item(a,1,44)=1, and item(a, > 1000,44)=44, item(a,-1000,44)=44 > > What I want to know is whether there is a built-in method or notation > for this. > There's no built-in method or notation for that. > What if, for example, we could do something like a [1000,44] ? That's actually using a tuple (1000, 44) as the index. Such tuples can be used as keys in a dict and are used in numpy for indexing multi-dimensional arrays, so it's definitely a bad idea. If such a method were added to the 'list' class then the best option (ie most consistent with other classes) would be get(index, default=None). From python at mrabarnett.plus.com Thu Jan 21 21:42:55 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 22 Jan 2010 02:42:55 +0000 Subject: Symbols as parameters? In-Reply-To: <0368f809$0$1357$c3e8da3@news.astraweb.com> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <0368f809$0$1357$c3e8da3@news.astraweb.com> Message-ID: <4B5910AF.3070104@mrabarnett.plus.com> Steven D'Aprano wrote: [snip] > An example from the standard library: the re module defines constants I, > L, M, etc. representing flags that are passed to the re.compile. They are > implemented as integers so they can easily be combined with &, but > another implementation might use symbols. You will notice that they're > not limited to the re.compile function itself. > > The caller may very well want to do something like this: > > # Get some flags for compile: > flags = re.I & re.M [snip] That should be: flags = re.I | re.M of course. :-) From steve at REMOVE-THIS-cybersource.com.au Thu Jan 21 22:08:57 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 03:08:57 GMT Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <0368f809$0$1357$c3e8da3@news.astraweb.com> Message-ID: <03690356$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 02:42:55 +0000, MRAB wrote: > Steven D'Aprano wrote: > [snip] >> An example from the standard library: the re module defines constants >> I, L, M, etc. representing flags that are passed to the re.compile. >> They are implemented as integers so they can easily be combined with &, >> but another implementation might use symbols. You will notice that >> they're not limited to the re.compile function itself. >> >> The caller may very well want to do something like this: >> >> # Get some flags for compile: >> flags = re.I & re.M > [snip] > > That should be: > > flags = re.I | re.M > > of course. :-) Dammit! No wonder my regexes never work as I expect!!! Thanks for the correction. -- Steven From python.list at tim.thechases.com Thu Jan 21 22:38:33 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 21 Jan 2010 21:38:33 -0600 Subject: Default return values for out-of-bounds list item In-Reply-To: <4B591041.8080905@mrabarnett.plus.com> References: <4B591041.8080905@mrabarnett.plus.com> Message-ID: <4B591DB9.7090100@tim.thechases.com> MRAB wrote: > gburdell1 at gmail.com wrote: >> Is there a built-in method in python that lets you specify a "default" >> value that will be returned whenever you try to access a list item >> that is out of bounds? Basically, it would be a function like this: >> >> def item(x,index,default): >> try: >> return x[index] >> except IndexError: >> return default >> >> So if a=[0,1,2,3], then item(a,0,44)=0, item(a,1,44)=1, and item(a, >> 1000,44)=44, item(a,-1000,44)=44 >> >> What I want to know is whether there is a built-in method or notation >> for this. > > > There's no built-in method or notation for that. > > > What if, for example, we could do something like a [1000,44] ? > > That's actually using a tuple (1000, 44) as the index. Such tuples can > be used as keys in a dict and are used in numpy for indexing > multi-dimensional arrays, so it's definitely a bad idea. > > If such a method were added to the 'list' class then the best option (ie > most consistent with other classes) would be get(index, default=None). But there's nothing stopping you from creating your own subclass of "list" that allows for defaults: class DefaultList(list): def __init__(self, default, *args, **kwargs): list.__init__(self, *args, **kwargs) self.default = default def __getitem__(self, index): try: return list.__getitem__(self, index) except IndexError: return self.default ml = DefaultList(42, [1,2,3,4]) for i in range(-5,5): print i, ml[i] (yeah, there's likely a "proper" way of using super() to do the above instead of "list.____" but the above worked for a quick 3-minute example composed in "ed"). The output of the above is -5 42 -4 1 -3 2 -2 3 -1 4 0 1 1 2 2 3 3 4 4 42 One might want to add other methods for __add__/__radd__ so that a DefaultList is returned instead of a "list", but this is python...the sky's the limit. -tkc From half.italian at gmail.com Thu Jan 21 22:42:35 2010 From: half.italian at gmail.com (Sean DiZazzo) Date: Thu, 21 Jan 2010 19:42:35 -0800 (PST) Subject: deriving MySQLdb class References: Message-ID: <9b9755ad-c3ca-42eb-b95c-0e41c5052cd7@p24g2000yqm.googlegroups.com> On Jan 21, 5:48?pm, tekion wrote: > All, > I am trying to write a class which inherits from MySQLdb class. ?Below > is code snippet: > import MySQLdb > import sys > > class ?msql_connect(MySQLdb): > ? ? def __init__(self): > ? ? ? ? self.host ? ? = ?"hostname" > ? ? ? ? self.user ? ? = "user" > ? ? ? ? self.password ?= "passoword" > ? ? ? ? self.database = "database name" > > I am running into below error: > ?class ?msql_connect(MySQLdb): > TypeError: Error when calling the metaclass bases > ? ? module.__init__() takes at most 2 arguments (3 given) > > Does any one have an idea why? ?Thanks. MySQLdb is the name of the module, not the class you want to subclass. But MySQLdb.connect() is not the class either...it's a factory function that returns instances of the class you actually want to subclass...connections.Connection(). The below works for me. from MySQLdb import connections import sys class mysql_connect(connections.Connection): def __init__(self): self.host = "host" self.user = "user" self.password = "password" self.database = "database" connections.Connection.__init__(self, host=self.host, user=self.user, passwd=self.password, db=self.database) p = mysql_connect() ~Sean From tekion at gmail.com Thu Jan 21 23:00:48 2010 From: tekion at gmail.com (tekion) Date: Thu, 21 Jan 2010 20:00:48 -0800 (PST) Subject: deriving MySQLdb class References: <9b9755ad-c3ca-42eb-b95c-0e41c5052cd7@p24g2000yqm.googlegroups.com> Message-ID: Sean, Thanks. This is useful. For future reference, how do I know what class is in MySQLdb module? From half.italian at gmail.com Thu Jan 21 23:13:16 2010 From: half.italian at gmail.com (Sean DiZazzo) Date: Thu, 21 Jan 2010 20:13:16 -0800 (PST) Subject: deriving MySQLdb class References: <9b9755ad-c3ca-42eb-b95c-0e41c5052cd7@p24g2000yqm.googlegroups.com> Message-ID: <10c80803-4934-4857-80e9-ba63648d9be4@14g2000yqp.googlegroups.com> On Jan 21, 8:00?pm, tekion wrote: > Sean, > Thanks. ?This is useful. ?For future reference, how do I know what > class is in MySQLdb module? You have to explore. ;) I found the MySQLdb module, and looked inside the __init__.py. Then looked for "connect" and followed the trail. From tekion at gmail.com Thu Jan 21 23:17:52 2010 From: tekion at gmail.com (tekion) Date: Thu, 21 Jan 2010 20:17:52 -0800 (PST) Subject: deriving MySQLdb class References: <9b9755ad-c3ca-42eb-b95c-0e41c5052cd7@p24g2000yqm.googlegroups.com> <10c80803-4934-4857-80e9-ba63648d9be4@14g2000yqp.googlegroups.com> Message-ID: <9f413105-5572-4bc7-93f4-ac7f252e3174@y23g2000yqm.googlegroups.com> Sean, I did a little investigation, there are other classes besides Connection. So, could I only set up a derived class from Connection and still be able to use the connection to query database and retrieve data? From half.italian at gmail.com Thu Jan 21 23:48:15 2010 From: half.italian at gmail.com (Sean DiZazzo) Date: Thu, 21 Jan 2010 20:48:15 -0800 (PST) Subject: deriving MySQLdb class References: <9b9755ad-c3ca-42eb-b95c-0e41c5052cd7@p24g2000yqm.googlegroups.com> <10c80803-4934-4857-80e9-ba63648d9be4@14g2000yqp.googlegroups.com> <9f413105-5572-4bc7-93f4-ac7f252e3174@y23g2000yqm.googlegroups.com> Message-ID: On Jan 21, 8:17?pm, tekion wrote: > Sean, > I did a little investigation, there are other classes besides > Connection. So, could I only set up a derived class from Connection > and still be able to use the connection to query database and retrieve > data? Im not sure I understand you completely... In theory, you could use the C API directly and subclass _mysql.connection to get at the database. But I think the point of MySQLdb is that they've done all the hard work. Why not use it? I think the other stuff in the module is in support of the Connection () class. ie. You cant get a cursor unless you already have a connection. From michele.simionato at gmail.com Fri Jan 22 00:00:30 2010 From: michele.simionato at gmail.com (Michele Simionato) Date: Thu, 21 Jan 2010 21:00:30 -0800 (PST) Subject: counting lines of code References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> <87bpgodj95.fsf@benfinney.id.au> <1f166703-fdfc-4f87-b336-ef80e492635b@a32g2000yqm.googlegroups.com> <7e09df6a-cda1-480e-a971-8f8a70ac4118@b9g2000yqd.googlegroups.com> Message-ID: <980775cc-9101-4673-af03-fc17914d63f8@c4g2000yqa.googlegroups.com> On Jan 21, 9:24?pm, Phlip wrote: > On Jan 20, 11:20?pm, Michele Simionato > wrote: > > > pylint does too many things, I want something fast that just counts > > the lines and can be run on thousands of files at once. > > cloc seems fine, I have just tried on 2,000 files and it gives me a > > report in just a few seconds. > > In my experience with Python codebases that big... > > ...how many of those lines are duplicated, and might merge together > into a better design? > > The LOC would go down, too. Actually 2,000 files is a very small portion of our code base, the one I am working on now. I have spent the last couple of months on a big refactoring project (which is still only at the beginning) and I wanted to count the difference between the lines of code before the refactoring and after the refactoring. I guess the new code is less than half than the old one. There was no cut and paste in the old code but a lot of subtle duplication, i.e. a code that could be unified in common libraries, but only after a lot of grunt work. The core parts were written 10 years ago, with a wrong architecture starting from the beginning, and then things started growing and growing on that monster. Just for fun I have run cloc on our trunk: Language files blank comment code scale 3rd gen. equiv -------------------------------------------------------------------------------- C++ 1528 67150 48251 304365 x 1.51 = 459591.15 XML 560 2769 2517 223223 x 1.90 = 424123.70 ASP 731 40136 4630 216713 x 1.29 = 279559.77 Python 2027 38825 47261 179532 x 4.20 = 754034.40 C/C++ Header 2150 51352 72619 141356 x 1.00 = 141356.00 Javascript 153 26196 9819 115311 x 1.48 = 170660.28 C 332 14147 12871 97918 x 0.77 = 75396.86 SQL 426 16432 4214 93598 x 2.29 = 214339.42 CSS 110 1493 1013 23087 x 1.00 = 23087.00 C# 83 3301 1990 19827 x 1.36 = 26964.72 Visual Basic 35 4363 5927 14633 x 2.76 = 40387.08 make 259 1617 650 8339 x 2.50 = 20847.50 Bourne Shell 52 598 1282 6557 x 3.81 = 24982.17 m4 28 611 627 5612 x 1.00 = 5612.00 IDL 23 560 0 3895 x 3.80 = 14801.00 HTML 33 354 76 3834 x 1.90 = 7284.60 MSBuild scripts 3 2 7 3419 x 1.90 = 6496.10 Lisp 33 562 648 2695 x 1.25 = 3368.75 Ruby 13 272 97 1141 x 4.20 = 4792.20 DOS Batch 77 790 410 1034 x 0.63 = 651.42 Java 4 148 181 972 x 1.36 = 1321.92 Perl 6 104 131 922 x 4.00 = 3688.00 XSD 6 0 0 506 x 1.90 = 961.40 awk 5 65 17 366 x 3.81 = 1394.46 DTD 4 117 50 351 x 1.90 = 666.90 ASP.Net 36 153 561 280 x 1.29 = 361.20 Bourne Again Shell 12 63 8 245 x 3.81 = 933.45 XSLT 1 15 14 196 x 1.90 = 372.40 NAnt scripts 3 27 0 119 x 1.90 = 226.10 Teamcenter def 10 16 0 93 x 1.00 = 93.00 -------------------------------------------------------------------------------- SUM: 8743 272238 215871 1470139 x 1.84 = 2708354.95 From casevh at gmail.com Fri Jan 22 00:02:44 2010 From: casevh at gmail.com (casevh) Date: Thu, 21 Jan 2010 21:02:44 -0800 (PST) Subject: Rounding up to the next 100 References: <74d51f20-b693-4ab2-809f-6f9df6de08c0@r19g2000yqb.googlegroups.com> Message-ID: On Jan 21, 1:58?pm, noydb wrote: > Sorry, although what I really need is the string-number rounded UP > every time. ?So if the number is 3890.32, it needs to go to 3900; if > the number is 3811.345, it needs to go to 3900 also. > > So, Florian's answer works. Another option is using math.ceil and math.floor. >>> import math >>> 100*math.ceil(1234.5678/100) 1300 >>> 100*math.floor(1234.5678/100) 1200 >>> 100*math.ceil(-1234.5678/100) -1200 >>> 100*math.floor(-1234.5678/100) -1300 casevh From gagsl-py2 at yahoo.com.ar Fri Jan 22 00:13:52 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 22 Jan 2010 02:13:52 -0300 Subject: Inheriting methods but over-riding docstrings References: <0361ea2d$0$1336$c3e8da3@news.astraweb.com> <28aa31f4-7331-4ac2-918b-0c59f5a834de@e37g2000yqn.googlegroups.com> <5ecdae50-b417-44bb-ae5b-c1d9aea865a2@y23g2000yqm.googlegroups.com> Message-ID: En Thu, 21 Jan 2010 03:17:40 -0300, Michele Simionato escribi?: > On Jan 21, 12:09 am, "Gabriel Genellina" > wrote: >> This is basically the same technique as in >> but there >> is a difference: you clone the function object *and* the code object it >> is based on. As I understand it, code objects are immutable and there >> is no need to clone them, but I may be wrong. Why did you feel the need >> to clone the code object too? > > No need. I just had the newfunc utility in my library (I think copied > from a recipe in the Python cookbook by Alex Martelli) so I used it. > In this case it is overkill, though. Also, I had no read your post > when I posted my solution, otherwise I would not have sent it ;) > Anyway, the setdocstring utility to extract the parent method was not > explicit in your post and may be of some use to somebody. Ah, ok! I was afraid I missed something -- but reusing code is a perfectly valid reason! -- Gabriel Genellina From dieter at handshake.de Fri Jan 22 01:33:49 2010 From: dieter at handshake.de (Dieter Maurer) Date: 22 Jan 2010 07:33:49 +0100 Subject: Memory usage problem of twisted server In-Reply-To: References: Message-ID: Victor Lin writes on Wed, 20 Jan 2010 02:52:25 -0800 (PST): > Hi, > > I encountered an increasing memory usage problem of my twisted server. > I have posted a question on stackoverflow: > http://stackoverflow.com/questions/2100192/how-to-find-the-source-of-increasing-memory-usage-of-a-twisted-server > > I have read the article "Improving Python's Memory Allocator" ( > http://evanjones.ca/memoryallocator/ ) and Python Memory Management > ( http://evanjones.ca/python-memory.html ). And I now know little > about how Python manages memory. I am wondering, is that the > increasing memory usage problem of my audio broadcasting caused by the > how python manage memory? Your careful reading has already told you that Python delegates memory allocation for larger blocks (>= 256 bytes) to the underlying C runtime library ("malloc" and friends). The C runtime library does not use memory compaction, i.e. it does not relocate used memory blocks in order to free space in few large chunks. Therefore, it is sensible to memory fragmentation: the free space gets scattered around in a large number of rather small blocks. The fragmentation rate is especially high when the memory request sizes have a high variance. > .... > Is that my guessing correct? How can I monitor the memory allocation > of Python? Look at "http://guppy-pe.sourceforge.net/" -- Dieter From anthra.norell at bluewin.ch Fri Jan 22 02:29:10 2010 From: anthra.norell at bluewin.ch (Anthra Norell) Date: Fri, 22 Jan 2010 08:29:10 +0100 Subject: substitution In-Reply-To: References: Message-ID: <4B5953C6.30803@bluewin.ch> Iain King wrote: > On Jan 21, 2:18 pm, Wilbert Berendsen wrote: > >> Op maandag 18 januari 2010 schreef Adi: >> >> >>> keys = [(len(key), key) for key in mapping.keys()] >>> keys.sort(reverse=True) >>> keys = [key for (_, key) in keys] >>> >>> pattern = "(%s)" % "|".join(keys) >>> repl = lambda x : mapping[x.group(1)] >>> s = "fooxxxbazyyyquuux" >>> >>> re.subn(pattern, repl, s) >>> >> I managed to make it even shorted, using the key argument for sorted, not >> putting the whole regexp inside parentheses and pre-compiling the regular >> expression: >> >> import re >> >> mapping = { >> "foo" : "bar", >> "baz" : "quux", >> "quuux" : "foo" >> >> } >> >> # sort the keys, longest first, so 'aa' gets matched before 'a', because >> # in Python regexps the first match (going from left to right) in a >> # |-separated group is taken >> keys = sorted(mapping.keys(), key=len) >> >> rx = re.compile("|".join(keys)) >> repl = lambda x: mapping[x.group()] >> s = "fooxxxbazyyyquuux" >> rx.sub(repl, s) >> >> One thing remaining: if the replacement keys could contain non-alphanumeric >> characters, they should be escaped using re.escape: >> >> rx = re.compile("|".join(re.escape(key) for key in keys)) >> >> Met vriendelijke groet, >> Wilbert Berendsen >> >> --http://www.wilbertberendsen.nl/ >> "You must be the change you wish to see in the world." >> -- Mahatma Gandhi >> > > Sorting it isn't the right solution: easier to hold the subs as tuple > pairs and by doing so let the user specify order. Think of the > following subs: > > "fooxx" -> "baz" > "oxxx" -> "bar" > > does the user want "bazxbazyyyquuux" or "fobarbazyyyquuux"? > > Iain > There is no way you can automate a user's choice. If he wants the second choice (oxxx->bar) he would have to add a third pattern: fooxxx -> fobar. In general, the rules 'upstream over downstream' and 'long over short' make sense in practically all cases. With all but simple substitution runs whose functionality is obvious, the result needs to be checked for unintended hits. To use an example from my SE manual which runs a (whimsical) text through a set of substitutions concentrating overlapping targets: >>> substitutions = [['be', 'BE'], ['being', 'BEING'], ['been', 'BEEN'], ['bee', 'BEE'], ['belong', 'BELONG'], ['long', 'LONG'], ['longer', 'LONGER']] >>> T = Translator (substitutions) # Code further up in this thread handling precedence by the two rules mentioned >>> text = "There was a bee named Mabel belonging to hive nine longing to be a beetle and thinking that being a bee was okay, but she had been a bee long enough and wouldn't be one much longer." >>> print T (text) There was a BEE named MaBEl BELONGing to hive nine LONGing to BE a BEEtle and thinking that BEING a BEE was okay, but she had BEEN a BEE LONG enough and wouldn't BE one much LONGER. All word-length substitutions resolve correctly. There are four unintended translations, though: MaBEl, BELONGing, LONGing and BEEtle. Adding the substitution Mabel->Mabel would prevent the first miss. The others could be taken care of similarly by replacing the target with itself. With large substitution sets and extensive data, this amounts to an iterative process of running, checking and fixing, many times over. That just isn't practical and may have to be abandoned when the substitutions catalog grows out of reasonable bounds. Dependable are runs where the targets are predictably singular, such as long id numbers that cannot possibly match anything but id numbers. Frederic From stefan_ml at behnel.de Fri Jan 22 02:40:23 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 22 Jan 2010 08:40:23 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> <7rrliuFl3gU1@mid.uni-berlin.de> <7rrm4dFofqU1@mid.uni-berlin.de> <7rrmttFtgjU1@mid.uni-berlin.de> Message-ID: <4b595667$0$7618$9b4e6d93@newsspool1.arcor-online.net> Alf P. Steinbach, 21.01.2010 20:24: > Do you understand how bad that makes you look? I think the right thing to say at this point is "don't feed the troll". Stefan From Martin.Drautzburg at web.de Fri Jan 22 02:43:39 2010 From: Martin.Drautzburg at web.de (Martin Drautzburg) Date: Fri, 22 Jan 2010 08:43:39 +0100 Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <89e6f34c-1a7e-4c2e-88cb-ee0397678948@k17g2000yqh.googlegroups.com> Message-ID: <1717313.IJevIQAxM0@beaureve.gmx.net> Carl Banks wrote: > I see. Well, Python is a poor choice for defining an internal DSL > (i.e., DSL using the general language's syntax), because it's > (deliberately) rigid in both grammar and semantics. I had this impression too. > Paul McGuire should be by to recommend PyParsing shortly. I looked it up and it seems to be about parsing strings. This is not what I am looking for as it would create a separate world outside of python. But I haven't looked deeply yet. From Martin.Drautzburg at web.de Fri Jan 22 03:12:46 2010 From: Martin.Drautzburg at web.de (Martin Drautzburg) Date: Fri, 22 Jan 2010 09:12:46 +0100 Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <0368f809$0$1357$c3e8da3@news.astraweb.com> Message-ID: <2002929.SUtrqfrzdN@beaureve.gmx.net> Steven D'Aprano wrote: > I think this really is the correct solution for your problem. In > Python, the standard place to have such public constants is at the > module level, not the function or class. I think you're worrying > unnecessarily about "namespace pollution" -- the module namespace is > *exactly* the right place for them. If two functions both need UP, > DOWN, etc symbols, then either: Defining those symbols at the module level is absolutely fine with me. The namespace pollution is indeed my biggest worry. You see, I want to be able to type in lots of lines with little effort. Preferably I would want to type move up I could still live with move(up) But then I need to import "from Movements import directions" or something like that. If another module defines "up" in some different way, I am in trouble. To circumvent this I would have to "import Movements", but then I's have to write move(directions.up) This is so noisy, I'd rather write move ("up") I don't like the quotes. I don't mind that "up" is a string (as someone suspected), what I dislike is that "up" was created ad-hoc by the caller. Could it be move("UP") as well? You could not tell without looking at the code of move(). Defining the symbols at module level solves THIS problem, but it leaves the above other problems. Still I like this best so far, because it is the standard way of doing this. > You would force them to do this: > > # Get some flags for compile: > flags = re.compile.I & re.compile.M > # ... > # much later on > # x = re.compile(s, flags) > > which is, in my opinion, a needless level of indirection and possibly > in violation of Demeter's Law. So scoping should be at module level? That makes some sense. > Either way, when you go to *use* the direction, you're still passing a > string. There's no difference between: > > move(m.UP) > > and just > > move("up") The difference is that move(m.UPx) would automatically raise an attribute error wheras move("upx") requires extra code in move() to raise an exception. move("up") just looks sematically wrong to me, in contrast len("up") is correct, because it really is an operation on Strings. When the caller writes move(up) should should not (need to) know what "up" really is behind the scenes. >> (4) Finally someone mentioned DSLs. I guess thats absolutely correct. >> This is what I am struggeling to achieve. I did a little googling >> ("how to write DSLs in python"), but haven't found anything appealing >> yet. Any pointers would be appreciated. > > That truly is using a bulldozer to crack a peanut. Well I guess I am really trying to implement a DSL (its about music). I just wasn't aware of that until someone mentioned DLSs here. The "symbols" problem is one problem I could not come up with anything delightful (the other one is getting rid of parenthesis). >> (5) Here is something I came up with myself: >> >> def symbols(aDict): >> aDict["foo"] = "bar" >> >> def someFunction(aFoo): >> print aFoo >> >> symbols(locals()) >> someFunction (foo) #Eh voila: foo is magically defined >> prints: bar >> >> The call to symbols(locals()) is the "magic, magic" I supected would >> be required in my original posting. If someFunction was a member of a >> class, the symbols would be properly tied to that class (albeit not >> the individual function), but still good enough. > > > I disagree about it being "proper" to tie such public symbols to the > class. But in any case, what you're trying to do is not supported by > Python. If it works, that's a happy accident. > > "The contents of this dictionary should not be modified; changes may > not affect the values of local and free variables used by the > interpreter." Two posters stronly discouraged that solution. And even with a decorator it does not look all that beautiful. I'll abandon this one. From __peter__ at web.de Fri Jan 22 03:21:48 2010 From: __peter__ at web.de (Peter Otten) Date: Fri, 22 Jan 2010 09:21:48 +0100 Subject: Default return values for out-of-bounds list item References: <0368f9d5$0$1357$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > On Thu, 21 Jan 2010 17:56:00 -0800, gburdell1 at gmail.com wrote: > >> Is there a built-in method in python that lets you specify a "default" >> value that will be returned whenever you try to access a list item that >> is out of bounds? > > No. > > >> Basically, it would be a function like this: >> >> def item(x,index,default): >> try: >> return x[index] >> except IndexError: >> return default > > That's probably the best way to do it. > > >> So if a=[0,1,2,3], then item(a,0,44)=0, item(a,1,44)=1, and item(a, >> 1000,44)=44, item(a,-1000,44)=44 >> >> What I want to know is whether there is a built-in method or notation >> for this. What if, for example, we could do something like a [1000,44] ? > > You can use slicing instead: > >>>> a=[0,1,2,3] >>>> a[2:3] > [2] >>>> a[100:101] > [] > > > and then detect the empty list and use default: > > def item(x, index, default): > a = x[index:index+1] > return a[0] if a else default You need to special-case -1: >>> for i in range(-4, 4): ... print "x[%d] -> %r" % (i, item("abc", i, "default")) ... x[-4] -> 'default' x[-3] -> 'a' x[-2] -> 'b' x[-1] -> 'default' x[0] -> 'a' x[1] -> 'b' x[2] -> 'c' x[3] -> 'default' Peter From alfps at start.no Fri Jan 22 03:27:26 2010 From: alfps at start.no (Alf P. Steinbach) Date: Fri, 22 Jan 2010 09:27:26 +0100 Subject: Symbols as parameters? In-Reply-To: <4b595667$0$7618$9b4e6d93@newsspool1.arcor-online.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <482a8a93-bfba-4e7f-bec6-7803e4b73863@k35g2000yqb.googlegroups.com> <4b583dcc$0$6551$9b4e6d93@newsspool4.arcor-online.net> <7rrliuFl3gU1@mid.uni-berlin.de> <7rrm4dFofqU1@mid.uni-berlin.de> <7rrmttFtgjU1@mid.uni-berlin.de> <4b595667$0$7618$9b4e6d93@newsspool1.arcor-online.net> Message-ID: * Stefan Behnel: > Alf P. Steinbach, 21.01.2010 20:24: >> Do you understand how bad that makes you look? > > I think the right thing to say at this point is "don't feed the troll". I find it amazing that you continue this kind of ad hominem attack. You leave it open who you regard as trolling, but when you do ad hominem you're essentially saying "I'm technically wrong but I'll win in the personal attack domain". Do you understand how bad that makes you look? Cheers & hth., - Alf From alfps at start.no Fri Jan 22 03:29:18 2010 From: alfps at start.no (Alf P. Steinbach) Date: Fri, 22 Jan 2010 09:29:18 +0100 Subject: Symbols as parameters? In-Reply-To: <7174001.cQaEHfC5Mm@beaureve.gmx.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> Message-ID: * Martin Drautzburg: > Here is a complete expample using a decorator, still a bit noisy > > def move(aDirection): > print "moving " + aDirection > > #Here comes the decorator > def scope(aDict): > def save(locals): > """Set symbols in locals and remember their original state""" > setSymbols={} > unsetSymbols=[] > for i in ("up", "down", "left", "right"): > if locals.has_key(i): > setSymbols[i] = locals[i] > else: > unsetSymbols.append(i) > # define the new symbols > locals[i] = i > > return setSymbols, unsetSymbols > > def restore (locals, set, unset): > """restore locals from set and unset""" > for i in set.keys(): > locals[i] = set[i] > for i in unset: > del(locals[i]) > > def callFunc(f): > """Main decorator""" > set, unset = save(aDict) > f() > restore(aDict, set, unset) > return callFunc > > > # -------------------------------------- > # using it > # -------------------------------------- > # a variable defined in the outer scope > up="outerScopeUp" > > # magic, magic (still too noisy for my taste) > @scope (locals()) > def _(): > move(up) > move(down) > move(left) > move(right) > > #verify the the outer scope variable hasn't changed > print "in the outer scope up is still:", up > print > print "this should fail:" > down > > # -------------------------------------- > # Output > # -------------------------------------- > > moving up > moving down > moving left > moving right > in the outer scope up is still: outerScopeUp > > this should fail: > Traceback (most recent call last): > File "", line 50, in > NameError: name 'down' is not defined Uhm, interesting technique, technically. But have you tested this within a function or class, which is what the use of "locals" implies? The reason that I ask is that in the documentation of locals() it says this: Note The contents of this dictionary should not be modified; changes may not affect the values of local variables used by the interpreter. (There's no such note for 'globals'). I have to admit that I was afraid to post this question since my experience in [comp.lang.python] is that when some technical error is pointed out by me, then most often the person starts a personal attack and credibility attack, injecting all kinds of noise -- actually that happened yet again as I was writing this response to you! But, I figure one shouldn't give up one humanity just because of one group where that happens regularly. I'm sort of counting on you to prove that there are, counting myself and one other, and perhaps now you, at least three persons here who are happy for technical corrections from me. Or, perhaps there's some aspect of locals(), e.g. in the context of decorators, that I don't know about and can now learn. :-) Cheers & hth., - Alf From zachar at awst.at Fri Jan 22 03:39:50 2010 From: zachar at awst.at (Balazs Zachar) Date: Fri, 22 Jan 2010 09:39:50 +0100 Subject: Python sys.prefix In-Reply-To: References: Message-ID: <4B596456.90503@awst.at> It looks like the python install environment is easily movable between two machine with the same OS (I mean, with the correct libraries)... Can I use this in production? Please consider the following test, after I use ./configure --prefix=/opt/python2.6.4 && make && make install: [root at CentOS-5-4-test1 ~]# /opt/python2.6.4/bin/python Python 2.6.4 (r264:75706, Jan 20 2010, 20:26:15) [GCC 4.1.2 20080704 (Red Hat 4.1.2-46)] on linux2 Type "help", "copyright", "credits" or "license" for more information. ...import sys ...sys.prefix '/opt/python2.6.4' ...sys.path ['', '/opt/python2.6.4/lib/python26.zip', '/opt/python2.6.4/lib/python2.6', '/opt/python2.6.4/lib/python2.6/plat-linux2', '/opt/python2.6.4/lib/python2.6/lib-tk', '/opt/python2.6.4/lib/python2.6/lib-old', '/opt/python2.6.4/lib/python2.6/lib-dynload', '/opt/python2.6.4/lib/python2.6/site-packages'] [root at CentOS-5-4-test1 ~]# mv /opt/python2.6.4/ /opt/python2.6.4_newplace [root at CentOS-5-4-test1 ~]# /opt/python2.6.4_newplace/bin/python Python 2.6.4 (r264:75706, Jan 20 2010, 20:26:15) [GCC 4.1.2 20080704 (Red Hat 4.1.2-46)] on linux2 Type "help", "copyright", "credits" or "license" for more information. ...import sys ...sys.prefix '/opt/python2.6.4_newplace' ...sys.path ['', '/opt/python2.6.4_newplace/lib/python26.zip', '/opt/python2.6.4_newplace/lib/python2.6', '/opt/python2.6.4_newplace/lib/python2.6/plat-linux2', '/opt/python2.6.4_newplace/lib/python2.6/lib-tk', '/opt/python2.6.4_newplace/lib/python2.6/lib-old', '/opt/python2.6.4_newplace/lib/python2.6/lib-dynload', '/opt/python2.6.4_newplace/lib/python2.6/site-packages'] On 01/21/2010 05:56 PM, zachar at awst.at wrote: > Hi All, > > I need some help about my "problem": > I compiled and installed python with ./configure --prefix=/opt/pyhon2.6.4... But I have to move it to a different directory (the reason is very difficult but I must do this)... I started the interpreter from shell and I was surprised because It still worked and It changed the sys.prefix so it pointed to the correct place so the sys.path was correct.... The questions are: HOW? Is it a feature which is not documented (at least I couldn't find anything about it)? > > From the official python documentation (http://docs.python.org/library/sys.html?highlight=sys#sys.prefix): > "sys.prefix? > > A string giving the site-specific directory prefix where the platform independent Python files are installed; by default, this is the string '/usr/local'. This can be set at build time with the --prefix argument to the configure script. The main collection of Python library modules is installed in the directory prefix + '/lib/pythonversion' while the platform independent header files (all except pyconfig.h) are stored in prefix + '/include/pythonversion', where version is equal to version[:3]." > > In my case, this is not correct: it looks, after I call the interpreter, something change the sys.prefix variable and point it to the right place and not to that directory which I used with the configuration script... This is very useful (I am happy because it solves one of my problems) but I am afraid of it will be do it in a different way in the future (as it is not documented). > > Thanks, > Regards, > Balazs > From duncan.booth at invalid.invalid Fri Jan 22 03:39:52 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 22 Jan 2010 08:39:52 GMT Subject: substitution References: <201001211518.55011.wbsoft@xs4all.nl> <4B5867BA.4070804@mrabarnett.plus.com> Message-ID: Wilbert Berendsen wrote: > # sort the keys, longest first, so 'aa' gets matched before 'a', because > # in Python regexps the first match (going from left to right) in a > # |-separated group is taken > keys = sorted(mapping.keys(), key=len, reverse=True) > This would do just as well (although see Iain King's response for the correct answer): keys = sorted(mapping, reverse=True) You don't need to specify key=len because the default sorting for two strings where one is a prefix of the other will always sort them that way anyway, and you don't need to call mapping.keys() because that's what sorted will sort anyway. -- Duncan Booth http://kupuguy.blogspot.com From georgeolivergo at gmail.com Fri Jan 22 04:16:17 2010 From: georgeolivergo at gmail.com (George Oliver) Date: Fri, 22 Jan 2010 01:16:17 -0800 (PST) Subject: looking for Python live code reloading IDEs Message-ID: <40aa7d9f-aa35-4bfd-b0d9-46a10585731b@b9g2000yqd.googlegroups.com> hi, I'm wondering if there are any Python programming environments that enable live code reloading, for example something like the Scheme- based impromptu (but also meant for any kind of Python program, not just audio/visual generation). Currently I do this directly in my editor (for game development), but I'm curious if there are other options. From pavlovevidence at gmail.com Fri Jan 22 04:18:33 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Fri, 22 Jan 2010 01:18:33 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <89e6f34c-1a7e-4c2e-88cb-ee0397678948@k17g2000yqh.googlegroups.com> <1717313.IJevIQAxM0@beaureve.gmx.net> Message-ID: <67ffb554-8c91-4bf8-9d40-4039ad91eb75@a6g2000yqm.googlegroups.com> On Jan 21, 11:43?pm, Martin Drautzburg wrote: > > Paul McGuire should be by to recommend PyParsing shortly. > > I looked it up and it seems to be about parsing strings. This is not > what I am looking for as it would create a separate world outside of > python. But I haven't looked deeply yet. Well, it's not necessarily a whole world outside Python, since an external DSL can be tied to the underlying to some degree. For example, a simple external DSL might input a string like this: "move up" parse it (such as with PyParsing), and output a string like this: "move(direction.up)" which can then be execed. It wouldn't be a whole new language, just an altered syntax. However, you are right in that, if you don't want to go creating your own syntax, PyParsing will be of no help. You'll have to shoehorn your DSL in a language not well suited for the task as best you can. Carl Banks From rdv at roalddevries.nl Fri Jan 22 05:56:02 2010 From: rdv at roalddevries.nl (Roald de Vries) Date: Fri, 22 Jan 2010 11:56:02 +0100 Subject: Symbols as parameters? In-Reply-To: <1495297.2JfmrJjJTh@beaureve.gmx.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: <4CD97E94-B40E-481D-B367-5FCE2820F48D@roalddevries.nl> Hi Martin, On Jan 21, 2010, at 8:43 AM, Martin Drautzburg wrote: > Hello all, > > When passing parameters to a function, you sometimes need a paramter > which can only assume certain values, e.g. > > def move (direction): > ... > If direction can only be "up", "down", "left" or "right", you can > solve > this by passing strings, but this is not quite to the point: > > - you could pass invalid strings easily > - you need to quote thigs, which is a nuisance > - the parameter IS REALLY NOT A STRING, but a direction > > Alternatively you could export such symbols, so when you "import *" > you > have them available in the caller's namespace. But that forces you > to "import *" which pollutes your namespace. > > What I am really looking for is a way > > - to be able to call move(up) > - having the "up" symbol only in the context of the function > call > > So it should look something like this > > ... magic, magic ... > move(up) > ... unmagic, unmagic ... > print up > > This should complain that "up" is not defined during the "print" call, > but not when move() is called. And of course there should be as little > magic as possible. > > Any way to achieve this? You could do something like this: class Move(object): def __call__(self, direction): print(direction) return 0 def up(self): return self('up') move = Move() Now move.up() means move('up'), and you can obviously do similar things for other directions. From jeanmichel at sequans.com Fri Jan 22 05:56:16 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Fri, 22 Jan 2010 11:56:16 +0100 Subject: Symbols as parameters? In-Reply-To: <0368f809$0$1357$c3e8da3@news.astraweb.com> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <0368f809$0$1357$c3e8da3@news.astraweb.com> Message-ID: <4B598450.1090208@sequans.com> Steven D'Aprano wrote: > (3) Then somone suggested to tie the constants to the function itself, >> as in >> def move(direction): >> print "moving %s" % direction >> >> move.UP = 'up' >> move.DOWN = 'down' >> >> This is quite nice. >> > > I would call it a horrible, horrible, horrible code smell. A stench in > fact. In my opinion, such attributes tied to the function should be > treated as internal to the function, and not the public interface. > > I wouldn't go quite so far as to say they should be treated as private, > but having the caller use them should be rare and unusual. > > I don't think so, my solution is perfect :o) Let me clarify, I wouldn't write this piece of code, but the OP seems to really worry about the 'noise' around its direction definitions. That is why I removed the constant class definition of DIRECTION, and add directly those constants in the function itself, allowing to use them without additional import statements. It is not usual but it is not *that* smelling. def move(direction): """Move to the given direction. @param direction: one of the function constant move.UP or move.DOWN """ print "moving %s" % direction move.UP = 'up' move.DOWN = 'down' As soon as it is properly documented, as a public interface should be, it becomes an acceptable design, IMO. Not the design I would choose in normal circonstances though. Jean-Michel From zuo at chopin.edu.pl Fri Jan 22 05:56:26 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Fri, 22 Jan 2010 11:56:26 +0100 Subject: Symbols as parameters? In-Reply-To: <5552238.LgDixP2R6m@beaureve.gmx.net> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> Message-ID: 21-01-2010, 22:51:34 Martin Drautzburg wrote: > Thanks for all the answers. Let me summarize > > (1) [...] > (2) Using enum's was suggested. That is good to know, but again it is > just a way to define constants in the caller's namespace. [...] > (3) Then somone suggested to tie the constants to the function itself, > [...] > (4) Finally someone mentioned DSLs. [...] > (5) Here is something I came up with myself: > > def symbols(aDict): > aDict["foo"] = "bar" > > def someFunction(aFoo): > print aFoo > > symbols(locals()) > someFunction (foo) #Eh voila: foo is magically defined [...] And what about such a sollution (6): from functools import wraps def restrict(*arg_options): "Decorator that restricts positional arg values to a limited set." options = set(arg_options) def actual_decorator(func): @wraps(func) def wrapper(*args, **kwargs): if options.issuperset(args): func(*args, **kwargs) else: raise ValueError("possible positional arguments" " for %s() limited to: %s" % (func.__name__, ", ".join(map(repr, arg_options)))) return wrapper return actual_decorator @restrict('up', 'down', 'right', 'left') def move(direction): print(direction) # ... move('up') # OK move('down') # OK move('left') # OK move('right') # OK move('rihgtt') # raises ValueError Although it uses strings, it solves (in Pythonic, explicit way) the main problem: "you could pass invalid strings easily". Typing a two characters more ('') isn't a big effort. Please also note that you can apply not only str-based symbols but any hashable objects (obviously it could be event implemented in less efficient way to accept *any* objects, though I doubt it's worth to do...). Regards, *j -- Jan Kaliszewski (zuo) From ben+python at benfinney.id.au Fri Jan 22 06:07:45 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 22 Jan 2010 22:07:45 +1100 Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <0368f809$0$1357$c3e8da3@news.astraweb.com> Message-ID: <87eili9z3y.fsf@benfinney.id.au> Jean-Michel Pichavant writes: > Steven D'Aprano wrote: > > I would call it a horrible, horrible, horrible code smell. A stench > > in fact. [?] > As soon as it is properly documented, as a public interface should be, > it becomes an acceptable design, IMO. [?] So your position seems to be that any design fault ceases to be a fault if it is documented. If that's not your position, I would be interested to know what relevance the above statement has to Stephen's objection. -- \ ?I have one rule to live by: Don't make it worse.? ?Hazel | `\ Woodcock | _o__) | Ben Finney From zuo at chopin.edu.pl Fri Jan 22 06:09:08 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Fri, 22 Jan 2010 12:09:08 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> Message-ID: s/sollution/solution s/event implemented/even implemented Sorry *j From robin at reportlab.com Fri Jan 22 06:11:35 2010 From: robin at reportlab.com (Robin Becker) Date: Fri, 22 Jan 2010 11:11:35 +0000 Subject: decimal threading cost? Message-ID: <4B5987E7.8070401@chamonix.reportlab.co.uk> Does using the decimal module incur a penalty because it imports threading or do I have to actually start a thread? -- Robin Becker From dickinsm at gmail.com Fri Jan 22 06:50:58 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 22 Jan 2010 03:50:58 -0800 (PST) Subject: decimal threading cost? References: Message-ID: <4abc165a-eb6c-450f-ac8b-68e076115c22@n7g2000yqb.googlegroups.com> On Jan 22, 11:11?am, Robin Becker wrote: > Does using the decimal module incur a penalty because it imports threading or do > I have to actually start a thread? There is at least one threading-related performance penalty that doesn't involve the user actually starting a thread: any arithmetic operation without an explicitly specified context (e.g., as invoked by one of the standard arithmetic operators) has to make a call to threading.local() to look up the current (thread-local) context. I don't know offhand whether this penalty is significant for the Python version of decimal, though it definitely *is* significant for decimal-in-C rewrite that's in the works: it would be interesting to do some timings against a thread-unaware version of decimal.py. -- Mark From davea at ieee.org Fri Jan 22 06:56:01 2010 From: davea at ieee.org (Dave Angel) Date: Fri, 22 Jan 2010 06:56:01 -0500 Subject: Symbols as parameters? In-Reply-To: <4CD97E94-B40E-481D-B367-5FCE2820F48D@roalddevries.nl> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <4CD97E94-B40E-481D-B367-5FCE2820F48D@roalddevries.nl> Message-ID: <4B599251.6090404@ieee.org> Roald de Vries wrote: >
Hi Martin, > > On Jan 21, 2010, at 8:43 AM, Martin Drautzburg wrote: >> Hello all, >> >> When passing parameters to a function, you sometimes need a paramter >> which can only assume certain values, e.g. >> >> def move (direction): >> ... >> If direction can only be "up", "down", "left" or "right", you can solve >> this by passing strings, but this is not quite to the point: >> >> - you could pass invalid strings easily >> - you need to quote thigs, which is a nuisance >> - the parameter IS REALLY NOT A STRING, but a direction >> >> Alternatively you could export such symbols, so when you "import *" you >> have them available in the caller's namespace. But that forces you >> to "import *" which pollutes your namespace. >> >> What I am really looking for is a way >> >> - to be able to call move(up) >> - having the "up" symbol only in the context of the function call >> >> So it should look something like this >> >> ... magic, magic ... >> move(up) >> ... unmagic, unmagic ... >> print up >> >> This should complain that "up" is not defined during the "print" call, >> but not when move() is called. And of course there should be as little >> magic as possible. >> >> Any way to achieve this? > > You could do something like this: > > class Move(object): > def __call__(self, direction): > print(direction) > return 0 > > def up(self): > return self('up') > > move = Move() > > Now move.up() means move('up'), and you can obviously do similar > things for other directions. > Once pointed down that road, how about: class Move(object): def __call__(self, direction): print(direction) return 0 @property def up(self): return self('up') move = Move() Now you can just say move.up with no parentheses. When I've wanted a DSL (Domain Specific Language) in the past, I've used Forth. It has so little syntax of its own, it's not hard to design your own simple syntax for a particular problem. And one of the things you can do is to define keywords that intercept the compile process of the following token (or line, or whatever). I am not claiming it's quick to become that proficient in Forth, however. The learning curve has a couple of steep sections, and this sort of thing is one of them. DaveA From drautzburg at googlemail.com Fri Jan 22 07:06:38 2010 From: drautzburg at googlemail.com (Martin Drautzburg) Date: Fri, 22 Jan 2010 04:06:38 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> Message-ID: <80071018-8653-43ad-8bc8-66ddaeb971ef@k35g2000yqb.googlegroups.com> On 22 Jan., 11:56, Roald de Vries wrote: > Hi Martin, > > On Jan 21, 2010, at 8:43 AM, Martin Drautzburg wrote: > > > > > > > Hello all, > > > When passing parameters to a function, you sometimes need a paramter > > which can only assume certain values, e.g. > > > ? ? ? ?def move (direction): > > ? ? ? ? ? ? ? ?... > > If direction can only be "up", "down", "left" or "right", you can ? > > solve > > this by passing strings, but this is not quite to the point: > > > ? ? ? ?- you could pass invalid strings easily > > ? ? ? ?- you need to quote thigs, which is a nuisance > > ? ? ? ?- the parameter IS REALLY NOT A STRING, but a direction > > > Alternatively you could export such symbols, so when you "import *" ? > > you > > have them available in the caller's namespace. But that forces you > > to "import *" which pollutes your namespace. > > > What I am really looking for is a way > > > ? ? ? ?- to be able to call move(up) > > ? ? ? ?- having the "up" symbol only in the context of the function ? > > call > > > So it should look something like this > > > ... magic, magic ... > > move(up) > > ... unmagic, unmagic ... > > print up > > > This should complain that "up" is not defined during the "print" call, > > but not when move() is called. And of course there should be as little > > magic as possible. > > > Any way to achieve this? > > You could do something like this: > > class Move(object): > ? ? ?def __call__(self, direction): > ? ? ? ? ?print(direction) > ? ? ? ? ?return 0 > > ? ? ?def up(self): > ? ? ? ? ?return self('up') > > move = Move() > > Now move.up() means move('up'), and you can obviously do similar ? > things for other directions.- Zitierten Text ausblenden - > > - Zitierten Text anzeigen - I had thought about that too. It gets a bit tricky when there is more than one parameter and it completely fails whan a parameter can REALLY be a number or an arbitrary string. Think: move(direction, distance) From nospam at nospam.com Fri Jan 22 07:17:44 2010 From: nospam at nospam.com (Gilles Ganault) Date: Fri, 22 Jan 2010 13:17:44 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? Message-ID: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Hello I use a dictionary to keep a list of users connected to a web site. To avoid users from creating login names that start with digits in order to be listed at the top, I'd like to sort the list differently every minute so that it'll start with the next letter, eg. display the list from A...Zdigits the first time, then B...ZAdigits, etc. That way, users have no incentive to create login names that start with either a digit or letter A. I see that dictionaries can be sorted using the... sort() method, but is it possible to have Python start sorting from a different letter? Thank you. From rdv at roalddevries.nl Fri Jan 22 07:29:22 2010 From: rdv at roalddevries.nl (Roald de Vries) Date: Fri, 22 Jan 2010 13:29:22 +0100 Subject: Symbols as parameters? In-Reply-To: <80071018-8653-43ad-8bc8-66ddaeb971ef@k35g2000yqb.googlegroups.com> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <80071018-8653-43ad-8bc8-66ddaeb971ef@k35g2000yqb.googlegroups.com> Message-ID: <14798C82-47F6-42B0-97AA-C603B81C5BF7@roalddevries.nl> On Jan 22, 2010, at 1:06 PM, Martin Drautzburg wrote: > On 22 Jan., 11:56, Roald de Vries wrote: >> Hi Martin, >> >> On Jan 21, 2010, at 8:43 AM, Martin Drautzburg wrote: >> >> >> >> >> >>> Hello all, >> >>> When passing parameters to a function, you sometimes need a paramter >>> which can only assume certain values, e.g. >> >>> def move (direction): >>> ... >>> If direction can only be "up", "down", "left" or "right", you can >>> solve >>> this by passing strings, but this is not quite to the point: >> >>> - you could pass invalid strings easily >>> - you need to quote thigs, which is a nuisance >>> - the parameter IS REALLY NOT A STRING, but a direction >> >>> Alternatively you could export such symbols, so when you "import *" >>> you >>> have them available in the caller's namespace. But that forces you >>> to "import *" which pollutes your namespace. >> >>> What I am really looking for is a way >> >>> - to be able to call move(up) >>> - having the "up" symbol only in the context of the function >>> call >> >>> So it should look something like this >> >>> ... magic, magic ... >>> move(up) >>> ... unmagic, unmagic ... >>> print up >> >>> This should complain that "up" is not defined during the "print" >>> call, >>> but not when move() is called. And of course there should be as >>> little >>> magic as possible. >> >>> Any way to achieve this? >> >> You could do something like this: >> >> class Move(object): >> def __call__(self, direction): >> print(direction) >> return 0 >> >> def up(self): >> return self('up') >> >> move = Move() >> >> Now move.up() means move('up'), and you can obviously do similar >> things for other directions.- Zitierten Text ausblenden - >> >> - Zitierten Text anzeigen - > > I had thought about that too. It gets a bit tricky when there is more > than one parameter and it completely fails whan a parameter can REALLY > be a number or an arbitrary string. Think: move(direction, distance) For extra parameters: class Move(object): def __call__(self, direction, param1, param2, *params, **dict): print(direction) return 0 def up(self, *params, **dict): return self('up', *params, **dict) move = Move() On Jan 22, 2010, at 1:06 PM, Martin Drautzburg wrote: > On 22 Jan., 11:56, Roald de Vries wrote: >> Hi Martin, >> >> On Jan 21, 2010, at 8:43 AM, Martin Drautzburg wrote: >> >> >> >> >> >>> Hello all, >> >>> When passing parameters to a function, you sometimes need a paramter >>> which can only assume certain values, e.g. >> >>> def move (direction): >>> ... >>> If direction can only be "up", "down", "left" or "right", you can >>> solve >>> this by passing strings, but this is not quite to the point: >> >>> - you could pass invalid strings easily >>> - you need to quote thigs, which is a nuisance >>> - the parameter IS REALLY NOT A STRING, but a direction >> >>> Alternatively you could export such symbols, so when you "import *" >>> you >>> have them available in the caller's namespace. But that forces you >>> to "import *" which pollutes your namespace. >> >>> What I am really looking for is a way >> >>> - to be able to call move(up) >>> - having the "up" symbol only in the context of the function >>> call >> >>> So it should look something like this >> >>> ... magic, magic ... >>> move(up) >>> ... unmagic, unmagic ... >>> print up >> >>> This should complain that "up" is not defined during the "print" >>> call, >>> but not when move() is called. And of course there should be as >>> little >>> magic as possible. >> >>> Any way to achieve this? >> >> You could do something like this: >> >> class Move(object): >> def __call__(self, direction): >> print(direction) >> return 0 >> >> def up(self): >> return self('up') >> >> move = Move() >> >> Now move.up() means move('up'), and you can obviously do similar >> things for other directions.- Zitierten Text ausblenden - >> >> - Zitierten Text anzeigen - > > I had thought about that too. It gets a bit tricky when there is more > than one parameter For extra parameters: class Move(object): def __call__(self, direction, param1, param2, *params, **dict): print(direction) return 0 def up(self, *params, **dict): return self('up', *params, **dict) move = Move() > and it completely fails whan a parameter can REALLY > be a number or an arbitrary string. Think: move(direction, distance) For a number, move.up(10) wouldn't be too bad. For an arbitrary string, you always have __getattr__ and __getattribute__. But if you like this style of coding, you should probably switch to Ruby. From dickinsm at gmail.com Fri Jan 22 07:34:06 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 22 Jan 2010 04:34:06 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> Message-ID: <065e450f-2190-42f4-b9c6-4b95260092be@f12g2000yqn.googlegroups.com> On Jan 21, 10:57?pm, Martin Drautzburg wrote: > Here is a complete expample using a decorator, still a bit noisy > > def move(aDirection): > ? ? print "moving " + aDirection > > #Here comes the decorator > def scope(aDict): > ? ? def save(locals): > [...] Have you considered making 'scope' a context manager? Your modifying-locals hacks could be put into the __enter__ and __exit__ methods, and you'd use the context manager with something like: with scope(): # ... # use up, down, left, right here # up, down, left, right no longer defined after the with block exits. -- Mark From robin at reportlab.com Fri Jan 22 07:35:02 2010 From: robin at reportlab.com (Robin Becker) Date: Fri, 22 Jan 2010 12:35:02 +0000 Subject: decimal threading cost? In-Reply-To: <4abc165a-eb6c-450f-ac8b-68e076115c22@n7g2000yqb.googlegroups.com> References: <4abc165a-eb6c-450f-ac8b-68e076115c22@n7g2000yqb.googlegroups.com> Message-ID: <4B599B76.2000600@chamonix.reportlab.co.uk> On 22/01/2010 11:50, Mark Dickinson wrote: > On Jan 22, 11:11 am, Robin Becker wrote: >> Does using the decimal module incur a penalty because it imports threading or do >> I have to actually start a thread? > > There is at least one threading-related performance penalty that > doesn't involve the user actually starting a thread: any arithmetic > operation without an explicitly specified context (e.g., as invoked by > one of the standard arithmetic operators) has to make a call to > threading.local() to look up the current (thread-local) context. > > I don't know offhand whether this penalty is significant for the > Python version of decimal, though it definitely *is* significant for > decimal-in-C rewrite that's in the works: it would be interesting to > do some timings against a thread-unaware version of decimal.py. > > -- > Mark I guess I was more worried that the import might start the per thread instruction count check even though I'm in a single thread process. -- Robin Becker From steve at REMOVE-THIS-cybersource.com.au Fri Jan 22 07:35:40 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 12:35:40 GMT Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <0368f809$0$1357$c3e8da3@news.astraweb.com> <2002929.SUtrqfrzdN@beaureve.gmx.net> Message-ID: <03698827$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 09:12:46 +0100, Martin Drautzburg wrote: > Defining those symbols at the module level is absolutely fine with me. > The namespace pollution is indeed my biggest worry. You see, I want to > be able to type in lots of lines with little effort. Preferably I would > want to type > > move up > > I could still live with > > move(up) > > But then I need to import "from Movements import directions" or > something like that. That's one line. It's hardly any cost, and it makes the code understandable: the reader can see exactly where directions comes from. This is a Good Thing. > If another module defines "up" in some different > way, I am in trouble. Only if you do this: from movements import up, down, left, right from wossnames import up, down, left, right The solution to this problem is simple: don't do it. Honestly, you're over-thinking this problem. Such names clashes can't happen by accident. They can happen through carelessness, but that's no different from this: x = 42 # ... much later on ... x = 23 # ... and later ... assert x == 42 The only time such name clashes can happen by accident is if you do from movements import * which is precisely why that form of import is not recommended. > To circumvent this I would have to "import > Movements", but then I's have to write > > move(directions.up) You'd have to say "import directions" to write "move(directions.up)". Another solution: import directions up = directions.up move(up) > This is so noisy, I'd rather write > > move ("up") > > I don't like the quotes. I don't mind that "up" is a string (as someone > suspected), what I dislike is that "up" was created ad-hoc by the > caller. Could it be move("UP") as well? You could not tell without > looking at the code of move(). Or the documentation. This is an API design decision the designer has to make. Should the move function use strings or integers or something else to specify the direction? If strings, should they be case-sensitive or insensitive? There's no right or wrong answer, all of these things have arguments in favour and against. [...] >> Either way, when you go to *use* the direction, you're still passing a >> string. There's no difference between: >> >> move(m.UP) >> >> and just >> >> move("up") > > The difference is that move(m.UPx) would automatically raise an > attribute error and move(UPx) would raise NameError, and move(U P) would raise SyntaxError. What's your point? > wheras move("upx") requires extra code in move() to > raise an exception. What extra code? Surely move already validates its input? Surely it looks something like this? def move(direction): if direction == 'up': foo elif direction == 'down': bar elif direction == 'left': baz elif direction == 'right': foobar else: raise ValueError("invalid direction") (or any variation that does the same sort of thing). The point is, you have to validate that direction is a valid direction anyway -- you can't trust that the user will only pass valid directions. There's nothing stopping the caller from saying move([42, 23]) or move(None), so you have to validate the argument inside the function anyway. > move("up") just looks sematically wrong to me, in > contrast len("up") is correct, because it really is an operation on > Strings. When the caller writes move(up) should should not (need to) > know what "up" really is behind the scenes. Again, I point you to Ben Finney's enum module, which I think is exactly what you want. -- Steven From jeanmichel at sequans.com Fri Jan 22 07:53:58 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Fri, 22 Jan 2010 13:53:58 +0100 Subject: Symbols as parameters? In-Reply-To: <87eili9z3y.fsf@benfinney.id.au> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <0368f809$0$1357$c3e8da3@news.astraweb.com> <87eili9z3y.fsf@benfinney.id.au> Message-ID: <4B599FE6.9010004@sequans.com> Ben Finney wrote: > Jean-Michel Pichavant writes: > > >> Steven D'Aprano wrote: >> >>> I would call it a horrible, horrible, horrible code smell. A stench >>> in fact. >>> > [?] > > >> As soon as it is properly documented, as a public interface should be, >> it becomes an acceptable design, IMO. >> > [?] > > So your position seems to be that any design fault ceases to be a fault > if it is documented. If that's not your position, I would be interested > to know what relevance the above statement has to Stephen's objection. > > Well, this disign is: 1/ working 2/ simple 3/ unusual I don't now if unusual means necessarily faulty or horrible. Still I don't want to defend this design at all cost, 'cause I'm not using it anyway, and I happily acknowledge that there are better ones (commonly used). I just wanted to help the OP with writing some simple and short code to use constants instead of strings in a context of one function. Note how I used *acceptable* in my previous post to qualify this design which is a rather cautious statement. Cheers, Jean-Michel From steve at REMOVE-THIS-cybersource.com.au Fri Jan 22 07:55:40 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 12:55:40 GMT Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> Message-ID: <03698cd8$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 09:29:18 +0100, Alf P. Steinbach wrote: > But have you tested this within a function or class, which is what the > use of "locals" implies? > > The reason that I ask is that in the documentation of locals() it says > this: > > Note > The contents of this dictionary should not be modified; changes may > not affect the values of local variables used by the interpreter. > > (There's no such note for 'globals'). > > I have to admit that I was afraid to post this question since my > experience in [comp.lang.python] is that when some technical error is > pointed out by me, then most often the person starts a personal attack > and credibility attack, injecting all kinds of noise -- actually that > happened yet again as I was writing this response to you! But, I figure > one shouldn't give up one humanity just because of one group where that > happens regularly. I'm sort of counting on you to prove that there are, > counting myself and one other, and perhaps now you, at least three > persons here who are happy for technical corrections from me. I've previously said, and I'll say it again, that you do bring much of value to this community, tech-wise. Shame that it comes along with such a thin skin. It's getting so that I often find myself afraid to disagree with anything you say lest you accuse me of lying again. > Or, perhaps there's some aspect of locals(), e.g. in the context of > decorators, that I don't know about and can now learn. :-) No, you got it spot on. Not to discourage you, but you're at least the third person who pointed this out in this thread. One implementation-specific trick is that modifying locals does actually work inside a class definition (at least in Python 2.5): >>> class Foo(object): ... x = 1 ... print locals() ... locals()['x'] = 2 ... {'x': 1, '__module__': '__main__'} >>> Foo.x 2 But it doesn't work in functions. That is because the local variables in CPython functions aren't stored in a dict, for efficiency reasons, so locals() makes a copy of those variables rather than returning the actual dict used as a namespace. This suggests we can cause locals() to malfunction in a class too, by using slots, since slotted attributes aren't stored in a dictionary. That's what I would predict, but alas I'm wrong: >>> class Foo(object): ... __slots__ = 'x' ... x = 1 ... print locals() ... locals()['x'] = 2 ... {'x': 1, '__module__': '__main__', '__slots__': 'x'} >>> Foo.x 2 So I don't understand why this works. Anyone know? Bottom line is, modifying locals() is not supported as a language feature. If it works, it's an accident. (Since Python can't guarantee that modifications to locals() will take, I wonder whether it would be better to ensure that they *never* take, rather than sometimes. It would only require locals() to return a copy of the dict, a shallow copy would probably do.) -- Steven From jeanmichel at sequans.com Fri Jan 22 08:06:35 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Fri, 22 Jan 2010 14:06:35 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? In-Reply-To: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: <4B59A2DB.4080809@sequans.com> Gilles Ganault wrote: > Hello > > I use a dictionary to keep a list of users connected to a web site. > > To avoid users from creating login names that start with digits in > order to be listed at the top, I'd like to sort the list differently > every minute so that it'll start with the next letter, eg. display the > list from A...Zdigits the first time, then B...ZAdigits, etc. > > That way, users have no incentive to create login names that start > with either a digit or letter A. > > I see that dictionaries can be sorted using the... sort() method, but > is it possible to have Python start sorting from a different letter? > > Thank you. > Here is one possible solution l = ['1a', 'a', 'b','c','av','ac'] # you mentioned a dictionary in your post, if so, l = myDict.keys() l.sort() # sort your list once and for all for start in '1abcd': result = [name for name in l if name[0] >= start] + [name for name in l if name[0] < start] print result ['1a', 'a', 'b', 'c', 'av', 'ac'] ['a', 'b', 'c', 'av', 'ac', '1a'] ['b', 'c', '1a', 'a', 'av', 'ac'] ['c', '1a', 'a', 'b', 'av', 'ac'] ['1a', 'a', 'b', 'c', 'av', 'ac'] Jean-Michel From nospam at nospam.com Fri Jan 22 08:07:04 2010 From: nospam at nospam.com (Gilles Ganault) Date: Fri, 22 Jan 2010 14:07:04 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: On Fri, 22 Jan 2010 13:17:44 +0100, Gilles Ganault wrote: >I see that dictionaries can be sorted using the... sort() method, but >is it possible to have Python start sorting from a different letter? Looks like the solution is to read the list of keys into a list, sort the list, and then use this to read the dictionary: http://code.activestate.com/recipes/52306/ But I haven't found whether a list can be sorted by starting from a given letter instead of "a". From jeanmichel at sequans.com Fri Jan 22 08:09:43 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Fri, 22 Jan 2010 14:09:43 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? In-Reply-To: <4B59A2DB.4080809@sequans.com> References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <4B59A2DB.4080809@sequans.com> Message-ID: <4B59A397.4010102@sequans.com> Jean-Michel Pichavant wrote: > Gilles Ganault wrote: >> Hello >> >> I use a dictionary to keep a list of users connected to a web site. >> >> To avoid users from creating login names that start with digits in >> order to be listed at the top, I'd like to sort the list differently >> every minute so that it'll start with the next letter, eg. display the >> list from A...Zdigits the first time, then B...ZAdigits, etc. >> >> That way, users have no incentive to create login names that start >> with either a digit or letter A. >> >> I see that dictionaries can be sorted using the... sort() method, but >> is it possible to have Python start sorting from a different letter? >> >> Thank you. >> > Here is one possible solution > > l = ['1a', 'a', 'b','c','av','ac'] # you mentioned a dictionary in > your post, if so, l = myDict.keys() > l.sort() # sort your list once and for all > for start in '1abcd': > result = [name for name in l if name[0] >= start] + [name for name > in l if name[0] < start] > print result > > > ['1a', 'a', 'b', 'c', 'av', 'ac'] > ['a', 'b', 'c', 'av', 'ac', '1a'] > ['b', 'c', '1a', 'a', 'av', 'ac'] > ['c', '1a', 'a', 'b', 'av', 'ac'] > ['1a', 'a', 'b', 'c', 'av', 'ac'] > > > Jean-Michel Sorry, the code I provided produce this output: ['1a', 'a', 'ac', 'av', 'b', 'c'] ['a', 'ac', 'av', 'b', 'c', '1a'] ['b', 'c', '1a', 'a', 'ac', 'av'] ['c', '1a', 'a', 'ac', 'av', 'b'] ['1a', 'a', 'ac', 'av', 'b', 'c'] which is actually what you are searching for. I just messed up with my ipython shell history :o) JM From alfps at start.no Fri Jan 22 08:16:50 2010 From: alfps at start.no (Alf P. Steinbach) Date: Fri, 22 Jan 2010 14:16:50 +0100 Subject: Symbols as parameters? In-Reply-To: <03698cd8$0$1357$c3e8da3@news.astraweb.com> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> <03698cd8$0$1357$c3e8da3@news.astraweb.com> Message-ID: * Steven D'Aprano -> Alf P. Steinbach: > > > No, you got it spot on. Not to discourage you, but you're at least the > third person who pointed this out in this thread. I get the impression that there's some message traffic that I don't see, perhaps on the mailing list, since (a) I haven't seen that about 'locals' pointed out by anyone else in this thread, and I think I've read all messages in the thread, and (b) sometimes threads pop up that start with a reply. For example, the recent thread "Covert number into string" started with a /reply/ in my newreader, using EternalSeptember's NNTP host. It also starts with a reply in Google's archives, . Cheers, - Alf PS: No, I'm not metaphorically thin-skinned. :-) As you can see in this thread I'm a pretty calm person, not getting upset by e.g. accusations of idiocy and insanity. The reason for my comments about being afraid to post is that even though rare cases of such language and ad-hominem just reflects on the one doing it, when it becomes a group/mob phenomenon then it creates an impression, like e.g. the one implicit in your "thin skin", that somehow I'm the one reacting to things by characterizing people -- which the archives show is not so. From steve at REMOVE-THIS-cybersource.com.au Fri Jan 22 08:24:05 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 13:24:05 GMT Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: <03699381$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 13:17:44 +0100, Gilles Ganault wrote: > Hello > > I use a dictionary to keep a list of users connected to a web site. > > To avoid users from creating login names that start with digits in order > to be listed at the top, I'd like to sort the list differently every > minute so that it'll start with the next letter, eg. display the list > from A...Zdigits the first time, then B...ZAdigits, etc. > > That way, users have no incentive to create login names that start with > either a digit or letter A. If you want to prohibit users from starting their login names with a digit, prohibit them from creating a login name with a digit. > I see that dictionaries can be sorted using the... sort() method, but is > it possible to have Python start sorting from a different letter? You can write a customer sort routine by using the key parameter to sort, which will probably be messy. What I'd do is keep 27 lists of user names, according to the first letter (26 letters from A to Z, plus one extra): users = [ # keep separate lists for each starting letter ['aaron', 'avril', 'adam'], ['betty', 'bob'], ['craig', 'cathy'], ['danni', 'da5id', 'donna'], # ... and so on ['zoe', 'zach'], ['4dam', '1-4m-t00-c001'] ] # add a new user users[3].append('daniel') Sort each one individually, and display them in whatever order you like: def display(start=0): start = start % 27 all = users[start:] + users[:start] for l in all: l.sort() print l And in use: >>> display(0) ['aaron', 'adam', 'avril'] ['betty', 'bob'] ['cathy', 'craig'] ['da5id', 'daniel', 'danni', 'donna'] ['zach', 'zoe'] ['1-4m-t00-c001', '4dam'] >>> >>> display(3) ['da5id', 'daniel', 'danni', 'donna'] ['zach', 'zoe'] ['1-4m-t00-c001', '4dam'] ['aaron', 'adam', 'avril'] ['betty', 'bob'] ['cathy', 'craig'] -- Steven From steve at REMOVE-THIS-cybersource.com.au Fri Jan 22 08:32:24 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 13:32:24 GMT Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <03699381$0$1357$c3e8da3@news.astraweb.com> Message-ID: <03699574$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 13:24:05 +0000, Steven D'Aprano wrote: > On Fri, 22 Jan 2010 13:17:44 +0100, Gilles Ganault wrote: [...] >> I see that dictionaries can be sorted using the... sort() method, but >> is it possible to have Python start sorting from a different letter? > > You can write a customer sort routine by using the key parameter to > sort, which will probably be messy. What I'd do is keep 27 lists of user > names, according to the first letter (26 letters from A to Z, plus one > extra): Replying to myself... the first sign of insanity *wink* While I'd still stick to that basic strategy, I'd wrap the functionality in a class so that the whole thing was transparent to the caller. So instead of the example I gave: > # add a new user > users[3].append('daniel') I'd write the class so the caller just needed to do: users.add('daniel') and the class would calculate which inner list to append it to. I just thought I'd make that clear in case you thought I expected you to manually keep track of which inner list each name should go into. -- Steven From wolfgang at rohdewald.de Fri Jan 22 08:35:07 2010 From: wolfgang at rohdewald.de (Wolfgang Rohdewald) Date: Fri, 22 Jan 2010 14:35:07 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <03698cd8$0$1357$c3e8da3@news.astraweb.com> Message-ID: <201001221435.20607.wolfgang@rohdewald.de> On Friday 22 January 2010, Alf P. Steinbach wrote: > I get the impression that there's some message traffic that I don't > see > For example, the recent thread "Covert number into string" started > with a reply in my newreader, using EternalSeptember's NNTP host. > > It also starts with a reply in Google's archives, http://groups.google.com/group/comp.lang.python/browse_thread/threa > d/b8097d4de4a9c9b0/cb3a2e6ccd7736ef>. did you check your spam folder? I got the original. If you want to, I can mail you the headers privately -- Wolfgang From neilc at norwich.edu Fri Jan 22 08:35:26 2010 From: neilc at norwich.edu (Neil Cerutti) Date: 22 Jan 2010 13:35:26 GMT Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: <7rtnsuFu4tU1@mid.individual.net> On 2010-01-22, Gilles Ganault wrote: > Hello > > I use a dictionary to keep a list of users connected to a web > site. > > To avoid users from creating login names that start with digits > in order to be listed at the top, I'd like to sort the list > differently every minute so that it'll start with the next > letter, eg. display the list from A...Zdigits the first time, > then B...ZAdigits, etc. Resorting is more work than is needed. Just choose a different starting index each time you display the names, and set up your lister to wrap-around to your arbitrary starting index. -- Neil Cerutti From nospam at nospam.com Fri Jan 22 08:58:58 2010 From: nospam at nospam.com (Gilles Ganault) Date: Fri, 22 Jan 2010 14:58:58 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <7rtnsuFu4tU1@mid.individual.net> Message-ID: <4lbjl5p7s67orgc91isk3ccvgp9s5m78l8@4ax.com> On 22 Jan 2010 13:35:26 GMT, Neil Cerutti wrote: >Resorting is more work than is needed. Just choose a different >starting index each time you display the names, and set up your >lister to wrap-around to your arbitrary starting index. Thanks. In this case, it means that in each loop iteration, I must search the list to find which item starts with the letter I'd like to begin sorting, eg. "B". Does Python include a search method or do I have to use a for loop to locate this starting item? From malkarouri at gmail.com Fri Jan 22 09:13:30 2010 From: malkarouri at gmail.com (Muhammad Alkarouri) Date: Fri, 22 Jan 2010 06:13:30 -0800 (PST) Subject: Consume an iterable Message-ID: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> In the python help for itertools, the following function is provided: def consume(iterator, n): "Advance the iterator n-steps ahead. If n is none, consume entirely." collections.deque(islice(iterator, n), maxlen=0) What is the advantage of using a collections.deque against, say, the following code? def consume(iterator, n): for _ in islice(iterator, n): pass Regards, Muhammad Alkarouri From alfps at start.no Fri Jan 22 09:36:32 2010 From: alfps at start.no (Alf P. Steinbach) Date: Fri, 22 Jan 2010 15:36:32 +0100 Subject: Symbols as parameters? In-Reply-To: <03698cd8$0$1357$c3e8da3@news.astraweb.com> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> <03698cd8$0$1357$c3e8da3@news.astraweb.com> Message-ID: * Steven D'Aprano: > > One implementation-specific trick is that modifying locals does actually > work inside a class definition (at least in Python 2.5): > >>>> class Foo(object): > ... x = 1 > ... print locals() > ... locals()['x'] = 2 > ... > {'x': 1, '__module__': '__main__'} >>>> Foo.x > 2 > > But it doesn't work in functions. That is because the local variables in > CPython functions aren't stored in a dict, for efficiency reasons, so > locals() makes a copy of those variables rather than returning the actual > dict used as a namespace. > > This suggests we can cause locals() to malfunction in a class too, by > using slots, since slotted attributes aren't stored in a dictionary. > That's what I would predict, but alas I'm wrong: > >>>> class Foo(object): > ... __slots__ = 'x' > ... x = 1 > ... print locals() > ... locals()['x'] = 2 > ... > {'x': 1, '__module__': '__main__', '__slots__': 'x'} >>>> Foo.x > 2 > > So I don't understand why this works. Anyone know? I don't *know*, but I have a speculation. It goes like this: 1. Perhaps first the statements in the class body are evaluated using a dictionary for locals, with as yet no existing class object. 2. Perhaps that dictionary plus some other info is then passed to __new__ of the class' metaclass, e.g. by default 'type' as metaclass, which perhaps produces a class object, filling in its slots and/or __dict__ from the supplied dictionary. :-) I'd have to read documentation and PEPs to say anything more for sure. > Bottom line is, modifying locals() is not supported as a language > feature. If it works, it's an accident. By the way, when I used 'class' to create a local scope I didn't have this behavior in mind, even if it might seem to be a strange coincidence. I just learned the above about metaclasses, if it is a correct impression, by checking whether the OP's declarative-like-usage code could be rescued in some way. Unfortunately my idea, fixing up the class' __dict__ in the metaclass, didn't work because the metaclass __new__ turned out to be called after, not before. > (Since Python can't guarantee that modifications to locals() will take, I > wonder whether it would be better to ensure that they *never* take, > rather than sometimes. It would only require locals() to return a copy of > the dict, a shallow copy would probably do.) I agree. Cheers, - Alf From joncle at googlemail.com Fri Jan 22 09:37:19 2010 From: joncle at googlemail.com (Jon Clements) Date: Fri, 22 Jan 2010 06:37:19 -0800 (PST) Subject: Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <7rtnsuFu4tU1@mid.individual.net> <4lbjl5p7s67orgc91isk3ccvgp9s5m78l8@4ax.com> Message-ID: On Jan 22, 1:58?pm, Gilles Ganault wrote: > On 22 Jan 2010 13:35:26 GMT, Neil Cerutti wrote: > > >Resorting is more work than is needed. Just choose a different > >starting index each time you display the names, and set up your > >lister to wrap-around to your arbitrary starting index. > > Thanks. In this case, it means that in each loop iteration, I must > search the list to find which item starts with the letter I'd like to > begin sorting, eg. "B". Does Python include a search method or do I > have to use a for loop to locate this starting item? How about a deque of lists... untested from collections import deque; from itertools import groupby deq = deque(list(items) for key, items in groupby(sorted(usernames), lambda L: L[0].upper())) Then everytime you use deq, rotate it? hth Jon. From nospam at nospam.com Fri Jan 22 09:38:29 2010 From: nospam at nospam.com (Gilles Ganault) Date: Fri, 22 Jan 2010 15:38:29 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <4B59A2DB.4080809@sequans.com> Message-ID: On Fri, 22 Jan 2010 14:09:43 +0100, Jean-Michel Pichavant wrote: >Sorry, the code I provided produce this output: > >['1a', 'a', 'ac', 'av', 'b', 'c'] >['a', 'ac', 'av', 'b', 'c', '1a'] >['b', 'c', '1a', 'a', 'ac', 'av'] >['c', '1a', 'a', 'ac', 'av', 'b'] >['1a', 'a', 'ac', 'av', 'b', 'c'] > >which is actually what you are searching for. I just messed up with my >ipython shell history :o) Thanks for the help. I'm a Python newbie, and have a difficult time understanding what the [] + [] line does :-/ I'll simplify things by using a list instead of a dictionary: ============ connected = [] connected.append("0test") connected.append("aa") connected.append("bb") connected.append("cc") for start in '1abcd': result = [name for name in connected if name[0] >= start] + [name for name in connected if name[0] < start] print result ============ C:\>test.py ['aa', 'bb', 'cc', '0test'] ['aa', 'bb', 'cc', '0test'] ['bb', 'cc', '0test', 'aa'] ['cc', '0test', 'aa', 'bb'] ['0test', 'aa', 'bb', 'cc'] ============ Pretty close to what I need to do but.. 1. Why is the first iteration done twice? 2. How can I have just one line, save the character that I used as starting point, increment it, and save it into a file so it can be read the next time this program runs? For instance, let's say we used "b" to start looking for items, I'll save "c" in a file for the next time. Thank you. From alfps at start.no Fri Jan 22 09:41:05 2010 From: alfps at start.no (Alf P. Steinbach) Date: Fri, 22 Jan 2010 15:41:05 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <03698cd8$0$1357$c3e8da3@news.astraweb.com> Message-ID: * Wolfgang Rohdewald: > On Friday 22 January 2010, Alf P. Steinbach wrote: >> I get the impression that there's some message traffic that I don't >> see > >> For example, the recent thread "Covert number into string" started >> with a reply in my newreader, using EternalSeptember's NNTP host. >> >> It also starts with a reply in Google's archives, > http://groups.google.com/group/comp.lang.python/browse_thread/threa >> d/b8097d4de4a9c9b0/cb3a2e6ccd7736ef>. > > did you check your spam folder? No, I don't have one for Usenet traffic, using Thunderbird as client. > I got the original. If you want to, I can mail you the headers > privately :-) Thanks!, but it's not necessary. It was just an example that somehow *some* articles are evidently missing in the view that I have (and Google has). Perhaps there is some spam filter somewhere. And what such filters do can be rather arbitrary and completely ungrokkable. Cheers, - Alf From zuo at chopin.edu.pl Fri Jan 22 09:44:28 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Fri, 22 Jan 2010 15:44:28 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? In-Reply-To: <4lbjl5p7s67orgc91isk3ccvgp9s5m78l8@4ax.com> References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <7rtnsuFu4tU1@mid.individual.net> <4lbjl5p7s67orgc91isk3ccvgp9s5m78l8@4ax.com> Message-ID: 22-01-2010 Steven D'Aprano wrote: > On Fri, 22 Jan 2010 13:17:44 +0100, Gilles Ganault wrote: >> To avoid users from creating login names that start with digits in order >> to be listed at the top, I'd like to sort the list differently every >> minute so that it'll start with the next letter, eg. display the list >> from A...Zdigits the first time, then B...ZAdigits, etc. >> >> That way, users have no incentive to create login names that start with >> either a digit or letter A. > > If you want to prohibit users from starting their login names with a > digit, prohibit them from creating a login name with a digit. I understand that a real problem is to make all names -- regardless of characters they are made of -- having "the equal rights" in terms of visibility. >> I see that dictionaries can be sorted using the... sort() method List, not dictionaries (they have nothing to do with the matter). >> but is it possible to have Python start sorting from a different >> letter? > > You can write a customer sort routine by using the key parameter to sort, > which will probably be messy. It could be done e.g. in such a way: # (for Python 2.x) from collections import deque import string CHARS = string.ascii_lowercase + string.digits def rotated_key(rotated_chars=deque(CHARS)): rotated_chars.rotate(1) tr_table = string.maketrans(CHARS, ''.join(rotated_chars)) def key(item): return item.translate(tr_table) return key # generate some names import random users = [''.join(random.choice(CHARS) for i in xrange(3)) for j in xrange(50)] for i in xrange(50): users.sort(key=rotated_key()) print ','.join(users) print But it still doesn't guarantee real "equal rights in visibility" (holders of names starting with rarely used characters are in better situation). > What I'd do is keep 27 lists of user names, > according to the first letter (26 letters from A to Z, plus one extra): > users = [ # keep separate lists for each starting letter > ['aaron', 'avril', 'adam'], > ['betty', 'bob'], > ['craig', 'cathy'], > ['danni', 'da5id', 'donna'], > # ... and so on > ['zoe', 'zach'], > ['4dam', '1-4m-t00-c001'] > ] But here the second letter becomes important for "visibility" and users still are not equal. And all this mess in unnecessary, because there is one perfect and simple solution -- see the Neil Cerutti's post. 22-01-2010, 14:58:58 Gilles Ganault wrote: > On 22 Jan 2010 13:35:26 GMT, Neil Cerutti wrote: >> Resorting is more work than is needed. Just choose a different >> starting index each time you display the names, and set up your >> lister to wrap-around to your arbitrary starting index. > > Thanks. In this case, it means that in each loop iteration, I must > search the list to find which item starts with the letter I'd like to > begin sorting, eg. "B". Does Python include a search method or do I > have to use a for loop to locate this starting item? There is e.g. `bisect' module -- you can search as well as insert with its functions. But IMHO you shouldn't search for the next starting *letter*, but for the next *name* in the list (basing on name that was used recently). If the list were immutable, no searching would be needed (indexes would be sufficient), but in real life users can be added and deleted in the meantime (so index of a particular name changes). Regards, *j -- Jan Kaliszewski (zuo) From davea at ieee.org Fri Jan 22 09:49:32 2010 From: davea at ieee.org (Dave Angel) Date: Fri, 22 Jan 2010 09:49:32 -0500 Subject: [2.5.1.1/dictionary] Change sorting order? In-Reply-To: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: <4B59BAFC.9040401@ieee.org> Gilles Ganault wrote: > Hello > > I use a dictionary to keep a list of users connected to a web site. > > To avoid users from creating login names that start with digits in > order to be listed at the top, I'd like to sort the list differently > every minute so that it'll start with the next letter, eg. display the > list from A...Zdigits the first time, then B...ZAdigits, etc. > > That way, users have no incentive to create login names that start > with either a digit or letter A. > > I see that dictionaries can be sorted using the... sort() method, but > is it possible to have Python start sorting from a different letter? > > Thank you. > > Seems to me the other solutions I've seen so far are more complex than needed. I figure you either want an unordered list, in which case you could use random.shuffle(), or you want a list that's sorted, but starts somewhere in the middle, at an arbitrary place, goes to the end, and wraps back to the beginning. So use random.randint() to choose an index within the list, and concatenate two slices of the list, based on that index in reverse order. And don't bother generating it each minute, but simply generate it each time you need it. I doubt if the cost of generating it is much different than the cost of checking the time. I guess there's a third possibility, that you don't want some of the G names at the beginning, and some at the end. In that case, I'd generate the random index as above, then increment it till the first character of the item changes. Use that index as your split point. If you like any of these, I could elaborate with some code. But each approach is pretty straightforward. DaveA From nospam at nospam.com Fri Jan 22 10:01:28 2010 From: nospam at nospam.com (Gilles Ganault) Date: Fri, 22 Jan 2010 16:01:28 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: <04fjl5psivncp4187f2edvaougn8j0edai@4ax.com> On Fri, 22 Jan 2010 09:49:32 -0500, Dave Angel wrote: >Seems to me the other solutions I've seen so far are more complex than >needed. I figure you either want an unordered list, in which case you >could use random.shuffle(), or you want a list that's sorted, but starts >somewhere in the middle, at an arbitrary place, goes to the end, and >wraps back to the beginning. So use random.randint() to choose an index >within the list, and concatenate two slices of the list, based on that >index in reverse order. Yes, this is exactly what I need: Start listing items from a given index (actually, using a character since the list contains names) all the way to the end of the list; If the character wasn't the very first, go back to the first time and display the list until we get to the current character. Python is so feature-rich, I'm sure there's a much simpler way to do this than this crappy code of mine: ============= connected = [] connected.append("0dummy") connected.append("aa") connected.append("bb") connected.append("cc") index = 0 for item in connected: #For testing purposes; #Final code will read/increment character from file if item[index] == "b": break else: index = index + 1 #Print items between current character and end of list tempindex = index while(tempindex < len(connected)): print connected[tempindex] tempindex = tempindex + 1 #if current letter not first character, #display beginning of list up to current character if index != 0: tempindex = 0 while tempindex < index: print connected[tempindex] tempindex = tempindex + 1 ============= Thank you for any help From arnodel at googlemail.com Fri Jan 22 10:07:19 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Fri, 22 Jan 2010 15:07:19 +0000 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> Message-ID: Muhammad Alkarouri writes: > In the python help for itertools, the following function is provided: > > def consume(iterator, n): > "Advance the iterator n-steps ahead. If n is none, consume > entirely." > collections.deque(islice(iterator, n), maxlen=0) > > What is the advantage of using a collections.deque against, say, the > following code? > > def consume(iterator, n): > for _ in islice(iterator, n): pass > deque is written in C, and so is islice, so it's less work for the bytecode interpreter? -- Arnaud From arnodel at googlemail.com Fri Jan 22 10:19:44 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Fri, 22 Jan 2010 15:19:44 +0000 Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <04fjl5psivncp4187f2edvaougn8j0edai@4ax.com> Message-ID: Gilles Ganault writes: > On Fri, 22 Jan 2010 09:49:32 -0500, Dave Angel wrote: >>Seems to me the other solutions I've seen so far are more complex than >>needed. I figure you either want an unordered list, in which case you >>could use random.shuffle(), or you want a list that's sorted, but starts >>somewhere in the middle, at an arbitrary place, goes to the end, and >>wraps back to the beginning. So use random.randint() to choose an index >>within the list, and concatenate two slices of the list, based on that >>index in reverse order. > > Yes, this is exactly what I need: Start listing items from a given > index (actually, using a character since the list contains names) all > the way to the end of the list; If the character wasn't the very > first, go back to the first time and display the list until we get to > the current character. > > Python is so feature-rich, I'm sure there's a much simpler way to do > this than this crappy code of mine: > > ============= > connected = [] > connected.append("0dummy") > connected.append("aa") > connected.append("bb") > connected.append("cc") > > index = 0 > for item in connected: > #For testing purposes; > #Final code will read/increment character from file > if item[index] == "b": > break > else: > index = index + 1 > > #Print items between current character and end of list > tempindex = index > while(tempindex < len(connected)): > print connected[tempindex] > tempindex = tempindex + 1 > > #if current letter not first character, > #display beginning of list up to current character > if index != 0: > tempindex = 0 > while tempindex < index: > print connected[tempindex] > tempindex = tempindex + 1 > ============= > > Thank you for any help Here's a straightforward way to do it, taking advantage of negative indices (no doubt there are many others): >>> connected = '0dummy aa bb cc'.split() >>> connected ['0dummy', 'aa', 'bb', 'cc'] >>> startindex = random.randrange(len(connected)) >>> startindex 2 >>> for i in xrange(startindex - len(connected), startindex): ... print connected[i] ... bb cc 0dummy aa >>> connected[-1] is the last element of connected connected[-2] is the one before last etc... I'll let you figure out why the loop works as it does. HTH -- Arnaud From arnodel at googlemail.com Fri Jan 22 10:24:33 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Fri, 22 Jan 2010 15:24:33 +0000 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> Message-ID: Muhammad Alkarouri writes: > In the python help for itertools, the following function is provided: > > def consume(iterator, n): > "Advance the iterator n-steps ahead. If n is none, consume > entirely." > collections.deque(islice(iterator, n), maxlen=0) > > What is the advantage of using a collections.deque against, say, the > following code? > > def consume(iterator, n): > for _ in islice(iterator, n): pass > I remember suggesting this function (under the name of 'exhaust'!) a while ago - although I don't think there was an optional parameter for the number of elements consumed. I hadn't realised it had been deemed useful by others! -- Arnaud From duncan.booth at invalid.invalid Fri Jan 22 10:24:58 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 22 Jan 2010 15:24:58 GMT Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: Jean-Michel Pichavant wrote: > Here is one possible solution > > l = ['1a', 'a', 'b','c','av','ac'] # you mentioned a dictionary in your > post, if so, l = myDict.keys() > l.sort() # sort your list once and for all > for start in '1abcd': > result = [name for name in l if name[0] >= start] + [name for name > in l if name[0] < start] > print result Here's another: >>> import bisect >>> def rotated_sort(data, startch): data = sorted(data) pos = bisect.bisect_left(data, startch) return data[pos:] + data[:pos] >>> for ch in " 1abcd": print ch, rotated_sort(['1a', 'a', 'b','c','av','ac'], ch) ['1a', 'a', 'ac', 'av', 'b', 'c'] 1 ['1a', 'a', 'ac', 'av', 'b', 'c'] a ['a', 'ac', 'av', 'b', 'c', '1a'] b ['b', 'c', '1a', 'a', 'ac', 'av'] c ['c', '1a', 'a', 'ac', 'av', 'b'] d ['1a', 'a', 'ac', 'av', 'b', 'c'] >>> -- Duncan Booth http://kupuguy.blogspot.com From zuo at chopin.edu.pl Fri Jan 22 10:25:33 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Fri, 22 Jan 2010 16:25:33 +0100 Subject: Consume an iterable In-Reply-To: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> Message-ID: > In the python help for itertools, the following function is provided: > > def consume(iterator, n): > "Advance the iterator n-steps ahead. If n is none, consume > entirely." > collections.deque(islice(iterator, n), maxlen=0) > > What is the advantage of using a collections.deque against, say, the > following code? > > def consume(iterator, n): > for _ in islice(iterator, n): pass Probably the former is faster. But I haven't check it. If you are curious, use timeit module... Regards, *j -- Jan Kaliszewski (zuo) From zuo at chopin.edu.pl Fri Jan 22 10:34:11 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Fri, 22 Jan 2010 16:34:11 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? In-Reply-To: References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <7rtnsuFu4tU1@mid.individual.net> <4lbjl5p7s67orgc91isk3ccvgp9s5m78l8@4ax.com> Message-ID: PS. 22-01-2010 o 15:44:28 Jan Kaliszewski wrote: > 22-01-2010, 14:58:58 Gilles Ganault wrote: > >> On 22 Jan 2010 13:35:26 GMT, Neil Cerutti wrote: >>> Resorting is more work than is needed. Just choose a different >>> starting index each time you display the names, and set up your >>> lister to wrap-around to your arbitrary starting index. >> >> Thanks. In this case, it means that in each loop iteration, I must >> search the list to find which item starts with the letter I'd like to >> begin sorting, eg. "B". Does Python include a search method or do I >> have to use a for loop to locate this starting item? > > There is e.g. `bisect' module -- you can search as well as insert with > its functions. But IMHO you shouldn't search for the next starting > *letter*, but for the next *name* in the list (basing on name that was > used recently). Or simply choose the starting index randomly -- using random.randrange(len(userlist)) or random.randint(0, len(userlist)-1) -- and use bisect.insort() for inserting to the list keeping it sorted (if you need to list users sorted+rotated). Regards, *j From steve at REMOVE-THIS-cybersource.com.au Fri Jan 22 10:48:41 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 15:48:41 GMT Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <7rtnsuFu4tU1@mid.individual.net> Message-ID: <0369b563$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 13:35:26 +0000, Neil Cerutti wrote: > On 2010-01-22, Gilles Ganault wrote: >> Hello >> >> I use a dictionary to keep a list of users connected to a web site. >> >> To avoid users from creating login names that start with digits in >> order to be listed at the top, I'd like to sort the list differently >> every minute so that it'll start with the next letter, eg. display the >> list from A...Zdigits the first time, then B...ZAdigits, etc. > > Resorting is more work than is needed. Just choose a different starting > index each time you display the names, and set up your lister to > wrap-around to your arbitrary starting index. The original poster's requirement is to start at a new *letter* each time, not whatever name happens to be at index N. Unless you can predict what index to use for (say) names starting with "B", then your scheme doesn't work. In order to find that index, you have to do a linear search of the list after every sort, turning sorting into O(N**2) instead of O(N*log N). -- Steven From neilc at norwich.edu Fri Jan 22 10:57:07 2010 From: neilc at norwich.edu (Neil Cerutti) Date: 22 Jan 2010 15:57:07 GMT Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <7rtnsuFu4tU1@mid.individual.net> <0369b563$0$1357$c3e8da3@news.astraweb.com> Message-ID: <7ru06jFo6hU1@mid.individual.net> On 2010-01-22, Steven D'Aprano wrote: > On Fri, 22 Jan 2010 13:35:26 +0000, Neil Cerutti wrote: >> On 2010-01-22, Gilles Ganault wrote: >>> Hello >>> >>> I use a dictionary to keep a list of users connected to a web site. >>> >>> To avoid users from creating login names that start with digits in >>> order to be listed at the top, I'd like to sort the list differently >>> every minute so that it'll start with the next letter, eg. display the >>> list from A...Zdigits the first time, then B...ZAdigits, etc. >> >> Resorting is more work than is needed. Just choose a different starting >> index each time you display the names, and set up your lister to >> wrap-around to your arbitrary starting index. > > The original poster's requirement is to start at a new *letter* > each time, not whatever name happens to be at index N. > > Unless you can predict what index to use for (say) names > starting with "B", then your scheme doesn't work. In order to > find that index, you have to do a linear search of the list > after every sort, turning sorting into O(N**2) instead of > O(N*log N). O(N*Log N) + O(N) == O(N**2)? Besides, the idea was to avoid sorting altogether. -- Neil Cerutti From albert at spenarnc.xs4all.nl Fri Jan 22 11:00:54 2010 From: albert at spenarnc.xs4all.nl (Albert van der Horst) Date: 22 Jan 2010 16:00:54 GMT Subject: Writing a string.ishex function References: <5f48d4ce-8985-4188-828d-0fdf57600659@g25g2000yqd.googlegroups.com> <20100114112254.eed71352.darcy@druid.net> Message-ID: In article , MRAB wrote: >D'Arcy J.M. Cain wrote: >> On Thu, 14 Jan 2010 07:52:58 -0800 (PST) >> chandra wrote: >>> Folks, >>> >>> I am new to Python and could not find a function along the lines of >> >> Welcome. >> >>> string.ishex in Python. There is however, a string.hexdigits constant >>> in the string module. I thought I would enhance the existing modlue >>> but am unsure how I should go about it. Specifically, I have attempted >>> this much: >> >> You should always test code before posting and post the exact code that >> you tested. >> >>> ---cut--- >>> #! /usr/bin/python >>> # -*- coding: utf-8 -*- >>> >>> import string >>> >>> def ishex(string): >> >> Bad idea to name your variable after a module. This function fails >> because of that. >> >>> ishex = False >>> for i in strdef ishex(sing: >>> if i in string.hexdigits: >>> ishex = True >>> else: >>> ishex = False >>> break >>> return ishex >> >> After renaming the variable this works but you can simplify it. >> >> >>> ---cut--- >> >> Just return False once you find a non-hex digit. >> >> def ishex(s): >> for c in s: >> if not c in string.hexdigits: return False >> >> return True >> >> And here are your unit tests. Every line should print "True". >> >> print ishex('123') is True >> print ishex('abc') is True >> print ishex('xyz') is False >> print ishex('0123456789abcdefABCDEF') is True >> print ishex('0123456789abcdefABCDEFG') is False >> >Don't use 'is', use '=='. > >BTW, ishex('') should return False. You are very wrong. Not with the above statement, but the very act of issuing a statement like that is wrong. The OP didn't specify ishex(). In absence of a specification, border cases are not defined. If the specification was: any character of string s must be a hex character then ishex('') should return True. If the specification was: the string must represent a hex number then ishex('') should probably return False. I can imagine a specification where it is appropriate to throw an exception for an empty string. This is also the safest thing to do, if there is the slightest hesitation. So the correct behaviour is: "Please mister customer, what exactly did you have in mind?" Groetjes Albert -- -- Albert van der Horst, UTRECHT,THE NETHERLANDS Economic growth -- being exponential -- ultimately falters. albert at spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst From jeanmichel at sequans.com Fri Jan 22 11:02:45 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Fri, 22 Jan 2010 17:02:45 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? In-Reply-To: References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <4B59A2DB.4080809@sequans.com> Message-ID: <4B59CC25.7010302@sequans.com> Gilles Ganault wrote: > On Fri, 22 Jan 2010 14:09:43 +0100, Jean-Michel Pichavant > wrote: > >> Sorry, the code I provided produce this output: >> >> ['1a', 'a', 'ac', 'av', 'b', 'c'] >> ['a', 'ac', 'av', 'b', 'c', '1a'] >> ['b', 'c', '1a', 'a', 'ac', 'av'] >> ['c', '1a', 'a', 'ac', 'av', 'b'] >> ['1a', 'a', 'ac', 'av', 'b', 'c'] >> >> which is actually what you are searching for. I just messed up with my >> ipython shell history :o) >> > > Thanks for the help. I'm a Python newbie, and have a difficult time > understanding what the [] + [] line does :-/ > > I'll simplify things by using a list instead of a dictionary: > > ============ > connected = [] > connected.append("0test") > connected.append("aa") > connected.append("bb") > connected.append("cc") > > for start in '1abcd': > result = [name for name in connected if name[0] >= start] + [name > for name in connected if name[0] < start] > print result > ============ > C:\>test.py > ['aa', 'bb', 'cc', '0test'] > ['aa', 'bb', 'cc', '0test'] > ['bb', 'cc', '0test', 'aa'] > ['cc', '0test', 'aa', 'bb'] > ['0test', 'aa', 'bb', 'cc'] > ============ > > Pretty close to what I need to do but.. > > 1. Why is the first iteration done twice? > > 2. How can I have just one line, save the character that I used as > starting point, increment it, and save it into a file so it can be > read the next time this program runs? For instance, let's say we used > "b" to start looking for items, I'll save "c" in a file for the next > time. > > Thank you. > 1/ [] + [] is using 2 python lists comprehension, Google it for details. It is quite difficult to read until you become familiar with it. Once you get it, you can write magical stuff :o) Basically, list comrehension allows to map functions to list elements and / or filter those elements. So what I'm using is the filter feature of list comprehension: [list of names for which the first char is greater that 'start'] + [list of names for which the first char is less than 'start'] 2/ Are you sure you want to do that ? looks like you are using a hammer to smash a fly. As someone has suggested before, maybe you want to pickup some starting index random. connected = ['aa', 'bb', 'cc', '0test'] import random def getNewOrder(myList): index = random.randint(0,len(myList)-1) print index return myList[index:] + myList[:index] # using slicing instead of list comprehension (suggested by DaveA) print getNewOrder(connected) JM From aahz at pythoncraft.com Fri Jan 22 11:03:32 2010 From: aahz at pythoncraft.com (Aahz) Date: 22 Jan 2010 08:03:32 -0800 Subject: maintain 2 versions of python on my computer References: <88886fe9-31ee-4ecc-9109-b645ceb46260@c34g2000yqn.googlegroups.com> <7a1f55ee-4c12-4f56-989a-abcd1c441365@j4g2000yqe.googlegroups.com> Message-ID: In article , Duncan Booth wrote: > >That seems overkill. This does pretty much the same thing: > > @(C:\Python26\Python -x %~f0 %* || pause) && goto:EOF > import sys > print sys.version > # raise RuntimeError # uncomment to trigger the 'pause' What version of Windows is necessary to use this? Anything with cmd.exe? (Sure doesn't look anything like DOS batch files...) -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From alfps at start.no Fri Jan 22 11:15:39 2010 From: alfps at start.no (Alf P. Steinbach) Date: Fri, 22 Jan 2010 17:15:39 +0100 Subject: maintain 2 versions of python on my computer In-Reply-To: References: <88886fe9-31ee-4ecc-9109-b645ceb46260@c34g2000yqn.googlegroups.com> <7a1f55ee-4c12-4f56-989a-abcd1c441365@j4g2000yqe.googlegroups.com> Message-ID: * Aahz: > In article , > Duncan Booth wrote: >> That seems overkill. This does pretty much the same thing: >> >> @(C:\Python26\Python -x %~f0 %* || pause) && goto:EOF >> import sys >> print sys.version >> # raise RuntimeError # uncomment to trigger the 'pause' > > What version of Windows is necessary to use this? Anything with > cmd.exe? (Sure doesn't look anything like DOS batch files...) Well, all of that has been there since, I don't know, long ago... -x A Python option that skips the first line of the script. @ suppresses echo of the line. () group commands (creates a compound command) || shortcut-evaluated OR && shortcut-evaluated AND % see below... C:\test> for /? | find "%~" %~I - expands %I removing any surrounding quotes (") %~fI - expands %I to a fully qualified path name %~dI - expands %I to a drive letter only %~pI - expands %I to a path only %~nI - expands %I to a file name only %~xI - expands %I to a file extension only %~sI - expanded path contains short names only %~aI - expands %I to file attributes of file %~tI - expands %I to date/time of file %~zI - expands %I to size of file %~$PATH:I - searches the directories listed in the PATH %~dpI - expands %I to a drive letter and path only %~nxI - expands %I to a file name and extension only %~fsI - expands %I to a full path name with short names only %~dp$PATH:I - searches the directories listed in the PATH %~ftzaI - expands %I to a DIR like output line values. The %~ syntax is terminated by a valid FOR variable name. C:\test> _ Cheers & hth. (even if a bit off-topic!), - Alf From steve at REMOVE-THIS-cybersource.com.au Fri Jan 22 11:17:12 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 16:17:12 GMT Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: <0369bc12$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 09:49:32 -0500, Dave Angel wrote: > Seems to me the other solutions I've seen so far are more complex than > needed. I figure you either want an unordered list, in which case you > could use random.shuffle(), or you want a list that's sorted, but starts > somewhere in the middle, at an arbitrary place, goes to the end, and > wraps back to the beginning. So use random.randint() to choose an index > within the list, and concatenate two slices of the list, based on that > index in reverse order. > > And don't bother generating it each minute, but simply generate it each > time you need it. According to the OP's stated requirements, he needs it every minute. Personally, from a usability perspective, I think a refresh of a (potentially huge) list of users every minute would be horrible. I think the whole design needs to be rethought, but that's not my decision. > I doubt if the cost of generating it is much > different than the cost of checking the time. Checking the time is quite fast -- it's a simple system call. Slicing and copying a list containing potentially tens or hundreds of thousands of names is anything but cheap! >>> from timeit import Timer >>> t1 = Timer('x = time.time()', 'import time') >>> t2 = Timer('y = x[:1000] + x[1000:]', 'x = [None]*100000') >>> >>> t1.timeit(1000) 0.0025529861450195312 >>> t2.timeit(1000) 3.0900199413299561 Slicing is more than 1000 times slower than checking the time. If you think I'm being unfair, that the OP's application will never have 100,000 names, try it with 1000 names: >>> t3 = Timer('y = x[:1000] + x[1000:]', 'x = [None]*1000') >>> t3.timeit(1000) 0.041487932205200195 That's 20 times slower than checking the time. That's the advantage of my suggestion: the only slicing done is the tiny master list, which means copying 27 pointers. Easy and fast, and it doesn't matter whether you have one user or one million, that operation will take exactly the same amount of time. Your suggestion to keep all the users in one giant list, and slice it repeatedly, scales terribly. > I guess there's a third possibility, that you don't want some of the G > names at the beginning, and some at the end. In that case, I'd generate > the random index as above, then increment it till the first character of > the item changes. Use that index as your split point. Another solution that doesn't scale terrible well. (Although not as bad as the previous one.) And what's with the random index thing? Why are you throwing away perfectly good information every time you want to shift letters? Your solution is something like this: You have a book opened at the start of Chapter 11, and you now want to jump to the start of Chapter 12. (1) Open the book at a random page. (2) Is it the start of Chapter 12? If yes, you're done. If no, turn to the next page. If you reach the end of the book, start again at the beginning. (3) Go to step 2. There are search techniques that start with a random index, but they don't advance one position at a time! (E.g. binary search, or hunt-and- bisect search, or similar.) But why search each time when you can keep an index to the start of each chapter and jump straight there in constant time? -- Steven From jeanmichel at sequans.com Fri Jan 22 11:21:02 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Fri, 22 Jan 2010 17:21:02 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? In-Reply-To: <4B59CC25.7010302@sequans.com> References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <4B59A2DB.4080809@sequans.com> <4B59CC25.7010302@sequans.com> Message-ID: <4B59D06E.3070406@sequans.com> Jean-Michel Pichavant wrote: > Gilles Ganault wrote: >> On Fri, 22 Jan 2010 14:09:43 +0100, Jean-Michel Pichavant >> wrote: >> >>> Sorry, the code I provided produce this output: >>> >>> ['1a', 'a', 'ac', 'av', 'b', 'c'] >>> ['a', 'ac', 'av', 'b', 'c', '1a'] >>> ['b', 'c', '1a', 'a', 'ac', 'av'] >>> ['c', '1a', 'a', 'ac', 'av', 'b'] >>> ['1a', 'a', 'ac', 'av', 'b', 'c'] >>> >>> which is actually what you are searching for. I just messed up with >>> my ipython shell history :o) >>> >> >> Thanks for the help. I'm a Python newbie, and have a difficult time >> understanding what the [] + [] line does :-/ >> >> I'll simplify things by using a list instead of a dictionary: >> >> ============ >> connected = [] >> connected.append("0test") >> connected.append("aa") >> connected.append("bb") >> connected.append("cc") >> >> for start in '1abcd': >> result = [name for name in connected if name[0] >= start] + [name >> for name in connected if name[0] < start] >> print result >> ============ >> C:\>test.py >> ['aa', 'bb', 'cc', '0test'] >> ['aa', 'bb', 'cc', '0test'] >> ['bb', 'cc', '0test', 'aa'] >> ['cc', '0test', 'aa', 'bb'] >> ['0test', 'aa', 'bb', 'cc'] >> ============ >> >> Pretty close to what I need to do but.. >> >> 1. Why is the first iteration done twice? >> >> 2. How can I have just one line, save the character that I used as >> starting point, increment it, and save it into a file so it can be >> read the next time this program runs? For instance, let's say we used >> "b" to start looking for items, I'll save "c" in a file for the next >> time. >> >> Thank you. >> > > 1/ [] + [] is using 2 python lists comprehension, Google it for > details. It is quite difficult to read until you become familiar with > it. Once you get it, you can write magical stuff :o) > Basically, list comrehension allows to map functions to list elements > and / or filter those elements. > > So what I'm using is the filter feature of list comprehension: > [list of names for which the first char is greater that 'start'] + > [list of names for which the first char is less than 'start'] > > 2/ Are you sure you want to do that ? looks like you are using a > hammer to smash a fly. As someone has suggested before, maybe you want > to pickup some starting index random. > > connected = ['aa', 'bb', 'cc', '0test'] > > import random > > def getNewOrder(myList): > index = random.randint(0,len(myList)-1) > print index > return myList[index:] + myList[:index] # using slicing instead of > list comprehension (suggested by DaveA) > > print getNewOrder(connected) > > JM > > Ok I realized that picking up a random index prevent from grouping names starting with the same letter (to ease visual lookup). Then go for the random char, and use char comparison (my first example). JM From steve at REMOVE-THIS-cybersource.com.au Fri Jan 22 11:27:21 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 22 Jan 2010 16:27:21 GMT Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <7rtnsuFu4tU1@mid.individual.net> <0369b563$0$1357$c3e8da3@news.astraweb.com> <7ru06jFo6hU1@mid.individual.net> Message-ID: <0369be73$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 15:57:07 +0000, Neil Cerutti wrote: > On 2010-01-22, Steven D'Aprano > wrote: >> Unless you can predict what index to use for (say) names starting with >> "B", then your scheme doesn't work. In order to find that index, you >> have to do a linear search of the list after every sort, turning >> sorting into O(N**2) instead of O(N*log N). > > O(N*Log N) + O(N) == O(N**2)? Oops! :( Of course, the sort is in fast C, and the linear search is in relatively slow Python, so it is quite conceivable that for realistic amounts of data, the time could be dominated by the searching. Or the OP might just change his requirements and allow starting the display in the middle of the letter. > Besides, the idea was to avoid sorting altogether. I don't see why you think that's necessary. Python's sort is very fast when the list is nearly sorted, so if you re-sort after any addition of a username, it might even be faster than trying to keep the list sorted all the time. That's the sort of thing that we'd need to do some timing tests to see which was faster. -- Steven From nobody at nowhere.com Fri Jan 22 11:30:54 2010 From: nobody at nowhere.com (Nobody) Date: Fri, 22 Jan 2010 16:30:54 +0000 Subject: subprocess troubles References: Message-ID: On Thu, 21 Jan 2010 11:25:08 +0100, Tomas Pelka wrote: > have a problem with following piece of code: > > -------------------------------------------------- > import subprocess > > paattern = "python" > cmd = "/usr/bin/locate" > arg1 = " -i" > arg2 = " -d /var/www/books/mlocate.db" > arg3 = str(" " + pattern) > p1 = subprocess.Popen([cmd, arg1, arg2, arg3], shell=False, > stdout=subprocess.PIPE, stderr=subprocess.PIPE) You probably want: arg1 = "-i" # no leading space arg2 = "-d" # no leading space arg3 = "/var/www/books/mlocate.db" arg4 = pattern p1 = subprocess.Popen([cmd, arg1, arg2, arg3, arg4], ... > If I run this command > (/usr/bin/locate -i -d /var/www/books/mlocate.db python) from standard > shell everything goes fine. > > Could you please give me an advice what I'm doing wrong? You're putting spaces where they aren't wanted. From neilc at norwich.edu Fri Jan 22 11:52:05 2010 From: neilc at norwich.edu (Neil Cerutti) Date: 22 Jan 2010 16:52:05 GMT Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <7rtnsuFu4tU1@mid.individual.net> <0369b563$0$1357$c3e8da3@news.astraweb.com> <7ru06jFo6hU1@mid.individual.net> <0369be73$0$1357$c3e8da3@news.astraweb.com> Message-ID: <7ru3dlFb8kU1@mid.individual.net> On 2010-01-22, Steven D'Aprano wrote: >> O(N*Log N) + O(N) == O(N**2)? > > Oops! :( > > Of course, the sort is in fast C, and the linear search is in > relatively slow Python, so it is quite conceivable that for > realistic amounts of data, the time could be dominated by the > searching. > > Or the OP might just change his requirements and allow starting > the display in the middle of the letter. That's what I would advocate. The scheme of starting at a random (or cycling through) first letters still grants an arbitrary advantage to some user names. A random starting position is thus more fair *and* more efficient. ;) -- Neil Cerutti From davea at ieee.org Fri Jan 22 11:54:58 2010 From: davea at ieee.org (Dave Angel) Date: Fri, 22 Jan 2010 11:54:58 -0500 Subject: [2.5.1.1/dictionary] Change sorting order? In-Reply-To: <04fjl5psivncp4187f2edvaougn8j0edai@4ax.com> References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <04fjl5psivncp4187f2edvaougn8j0edai@4ax.com> Message-ID: <4B59D862.3000207@ieee.org> Gilles Ganault wrote: > On Fri, 22 Jan 2010 09:49:32 -0500, Dave Angel wrote: > >> Seems to me the other solutions I've seen so far are more complex than >> needed. I figure you either want an unordered list, in which case you >> could use random.shuffle(), or you want a list that's sorted, but starts >> somewhere in the middle, at an arbitrary place, goes to the end, and >> wraps back to the beginning. So use random.randint() to choose an index >> within the list, and concatenate two slices of the list, based on that >> index in reverse order. >> > > Yes, this is exactly what I need: Start listing items from a given > index (actually, using a character since the list contains names) all > the way to the end of the list; If the character wasn't the very > first, go back to the first time and display the list until we get to > the current character. > > Python is so feature-rich, I'm sure there's a much simpler way to do > this than this crappy code of mine: > > ============= > connected = [] > connected.append("0dummy") > connected.append("aa") > connected.append("bb") > connected.append("cc") > > index = 0 > for item in connected: > #For testing purposes; > #Final code will read/increment character from file > if item[index] == "b": > break > else: > index = index + 1 > > #Print items between current character and end of list > tempindex = index > while(tempindex < len(connected)): > print connected[tempindex] > tempindex = tempindex + 1 > > #if current letter not first character, > #display beginning of list up to current character > if index != 0: > tempindex = 0 > while tempindex < index: > print connected[tempindex] > tempindex = tempindex + 1 > ============= > > Thank you for any help > > Your code would fail if you picked a letter for which there were no entries. That's easily fixed, just use >= instead of == in your if test. Jean-Michel Pichavant post implements my second option. So I'll borrow his code here. But apparently you want my third option: random starting place, but with all the names starting with the same character together. To get that, but with the distribution proportional to the number of names that start with the given character, you need to get the index in two stages. First use the random.randint(). Then do a search for the first name that starts with that same character. For that, I borrow from Duncan Booth, and use bisect.bisect(). import random, bisect connected = sorted( ['jj', '0test', '3test', 'aa', 'aj', 'bb', 'bz', 'dc', 'df'] ) def getNewOrder(myList): index = random.randint(0,len(myList)-1) startch = myList[index][0] pos = bisect.bisect_left(myList, startch) print index, myList[index], pos, myList[pos] return myList[pos:] + myList[:pos] # using slicing instead of list comprehension (suggested by DaveA) print connected for i in range(6): print getNewOrder(connected) I have the extra print inside the function to show (for example) that if 'aj' is picked, it actually will start with'aa' Note that with your constraints, there's no way it can be fair. But experiment a bit, and see if it meets your needs. DaveA From duncan.booth at invalid.invalid Fri Jan 22 12:15:28 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 22 Jan 2010 17:15:28 GMT Subject: maintain 2 versions of python on my computer References: <88886fe9-31ee-4ecc-9109-b645ceb46260@c34g2000yqn.googlegroups.com> <7a1f55ee-4c12-4f56-989a-abcd1c441365@j4g2000yqe.googlegroups.com> Message-ID: aahz at pythoncraft.com (Aahz) wrote: > In article , > Duncan Booth wrote: >> >>That seems overkill. This does pretty much the same thing: >> >> @(C:\Python26\Python -x %~f0 %* || pause) && goto:EOF >> import sys >> print sys.version >> # raise RuntimeError # uncomment to trigger the 'pause' > > What version of Windows is necessary to use this? Anything with > cmd.exe? (Sure doesn't look anything like DOS batch files...) Windows NT, Windows 2000, Windows XP, Windows Vista, Windows 7, Server 2003, Server 2008 will all handle the && and || stuff. I didn't think that NT recognised the goto:EOF as special but a quick search with Google seems to indicate that it did. If so this script probably worked since 1993.[*] Windows 7 still supports this cmd.exe syntax, but it also has Microsoft Powershell which seems to be Microsoft's latest answer to shell scripts and allows direct access to COM objects and .Net classes. [*] or at least would have worked if Guido used his time machine to take a copy of Python 2.6 back to 1993. :^) From nospam at nospam.com Fri Jan 22 12:43:00 2010 From: nospam at nospam.com (Gilles Ganault) Date: Fri, 22 Jan 2010 18:43:00 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: On 22 Jan 2010 15:24:58 GMT, Duncan Booth wrote: >Here's another: Thanks for the sample. It work great, except that it also runs when the header character doesn't match any item in the list: ======= import bisect connected = [] connected.append("_test") connected.append("-test") connected.append("0test") connected.append("1test") connected.append("Aa") connected.append("Bb") connected.append("Cc") def rotated_sort(data, startch): data = sorted(data) pos = bisect.bisect_left(data, startch) return data[pos:] + data[:pos] for ch in "_-0123456789ABCDEFGHIJKLMNOPQRSTUVWYZ": print ch, rotated_sort(connected, ch) ======= C:\>test.py _ ['_test', '-test', '0test', '1test', 'Aa', 'Bb', 'Cc'] - ['-test', '0test', '1test', 'Aa', 'Bb', 'Cc', '_test'] 0 ['0test', '1test', 'Aa', 'Bb', 'Cc', '_test', '-test'] 1 ['1test', 'Aa', 'Bb', 'Cc', '_test', '-test', '0test'] 2 ['Aa', 'Bb', 'Cc', '_test', '-test', '0test', '1test'] 3 ['Aa', 'Bb', 'Cc', '_test', '-test', '0test', '1test'] 4 ['Aa', 'Bb', 'Cc', '_test', '-test', '0test', '1test'] ======= Should I add an "if" block in the "for ch in"? From nospam at nospam.com Fri Jan 22 12:44:08 2010 From: nospam at nospam.com (Gilles Ganault) Date: Fri, 22 Jan 2010 18:44:08 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <4B59A2DB.4080809@sequans.com> <4B59CC25.7010302@sequans.com> Message-ID: On Fri, 22 Jan 2010 17:21:02 +0100, Jean-Michel Pichavant wrote: >Ok I realized that picking up a random index prevent from grouping names >starting with the same letter (to ease visual lookup). >Then go for the random char, and use char comparison (my first example). Yup, I think it's a good enough solution. Thanks much for the help. From ng1.ben at gmail.com Fri Jan 22 12:57:52 2010 From: ng1.ben at gmail.com (ben) Date: Fri, 22 Jan 2010 09:57:52 -0800 (PST) Subject: * operator in python tutorial References: <7b2ef0aa-8b93-4cb5-b944-beea3e2c98b1@p16g2000vbe.googlegroups.com> <0036df65$0$25085$c3e8da3@news.astraweb.com> Message-ID: On Jan 20, 8:30?pm, Gringo wrote: > On 1/20/2010 12:38, ben wrote: > > > > > Hello, > > > I am following through the python tutorial which gets to a line that > > uses the * operator with zip(). I searched and searched but could find > > no information on the operator or how to use it in general. The > > example from the tut is as follows: > >>>> x = [1, 2, 3] > >>>> y = [4, 5, 6] > >>>> zipped = zip(x, y) > >>>> zipped > > [(1, 4), (2, 5), (3, 6)] > >>>> x2, y2 = zip(*zipped) > >>>> x == list(x2) and y == list(y2) > > True > > > How would i apply the * operator in general? > > Thanks. > > The * operator, when used in this context, unpacks the sequence and it's > as if you passed each item to the function as a different parameter. > For example, if you have a list x with 4 items, these two statements > would be the same: > f(x[0],x[1],x[2],x[3]) > f(*x) > > Hope this helps. It does. Thanks. From davea at ieee.org Fri Jan 22 13:40:26 2010 From: davea at ieee.org (Dave Angel) Date: Fri, 22 Jan 2010 13:40:26 -0500 Subject: [2.5.1.1/dictionary] Change sorting order? In-Reply-To: <0369bc12$0$1357$c3e8da3@news.astraweb.com> References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> <0369bc12$0$1357$c3e8da3@news.astraweb.com> Message-ID: <4B59F11A.7030401@ieee.org> Steven D'Aprano wrote: > On Fri, 22 Jan 2010 09:49:32 -0500, Dave Angel wrote: > > >> Seems to me the other solutions I've seen so far are more complex than >> needed. I figure you either want an unordered list, in which case you >> could use random.shuffle(), or you want a list that's sorted, but starts >> somewhere in the middle, at an arbitrary place, goes to the end, and >> wraps back to the beginning. So use random.randint() to choose an index >> within the list, and concatenate two slices of the list, based on that >> index in reverse order. >> >> And don't bother generating it each minute, but simply generate it each >> time you need it. >> > > According to the OP's stated requirements, he needs it every minute. > > I read it as he wants viewers of the page that are more than a minute apart to see a different order. > Personally, from a usability perspective, I think a refresh of a > (potentially huge) list of users every minute would be horrible. I think > the whole design needs to be rethought, but that's not my decision. > > > Certainly if there are more than 100 users or so, the idea of presenting a web page with a single list of users is unwieldy. So I assumed this problem did not have to scale. Once the list goes on multiple pages, entirely different algorithms are appropriate, along with the display mechanisms to support them. >> I doubt if the cost of generating it is much >> different than the cost of checking the time. >> > > Checking the time is quite fast -- it's a simple system call. Slicing and > copying a list containing potentially tens or hundreds of thousands of > names is anything but cheap! > > >>>> from timeit import Timer >>>> t1 = Timer('x = time.time()', 'import time') >>>> t2 = Timer('y = x[:1000] + x[1000:]', 'x = [None]*100000') >>>> >>>> t1.timeit(1000) >>>> > 0.0025529861450195312 > >>>> t2.timeit(1000) >>>> > 3.0900199413299561 > > Slicing is more than 1000 times slower than checking the time. If you > think I'm being unfair, that the OP's application will never have 100,000 > names, try it with 1000 names: > > >>>> t3 = Timer('y = x[:1000] + x[1000:]', 'x = [None]*1000') >>>> t3.timeit(1000) >>>> > 0.041487932205200195 > > That's 20 times slower than checking the time. > > That's the advantage of my suggestion: the only slicing done is the tiny > master list, which means copying 27 pointers. Easy and fast, and it > doesn't matter whether you have one user or one million, that operation > will take exactly the same amount of time. Your suggestion to keep all > the users in one giant list, and slice it repeatedly, scales terribly. > > > >> I guess there's a third possibility, that you don't want some of the G >> names at the beginning, and some at the end. In that case, I'd generate >> the random index as above, then increment it till the first character of >> the item changes. Use that index as your split point. >> > > Another solution that doesn't scale terrible well. (Although not as bad > as the previous one.) > > And what's with the random index thing? Why are you throwing away > perfectly good information every time you want to shift letters? Your > solution is something like this: > > You have a book opened at the start of Chapter 11, and you now want to > jump to the start of Chapter 12. > > (1) Open the book at a random page. > (2) Is it the start of Chapter 12? > Wrong. We're checking whether it's the start of whatever the next chapter is, so if we randomly jumped into chapter 5, we'd be going a page at a time till chapter 6. After I thought about it, I realized we should be going backwards, not forward, but the idea is the same. We're only walking through a half chapter, typically. > If yes, you're done. > If no, turn to the next page. If you reach the end of > the book, start again at the beginning. > (3) Go to step 2. > > There are search techniques that start with a random index, but they > don't advance one position at a time! (E.g. binary search, or hunt-and- > bisect search, or similar.) But why search each time when you can keep an > index to the start of each chapter and jump straight there in constant > time? > > First, the time that will be spent turning this list into a web page will far exceed any of these numbers. And if we assume he's using CGI, then the time spent reloading and unmarshaling the list will exceed by even more. Next, when it comes to an actual implementation, I used bisect(), which is O(logn). And finally, it was more important to get the concepts together, as the spec wasn't well-thought out, than it was to get some optimal solution. That's why I used slice instead of doing the index-len() trick. In real code, he'll be looping over the list, calling some logic to add

and

for example. By running that loop from index-len() to index, it can operate on the original list. In my opinion, the only "fair" answer is to choose the starting point randomly, or sequentially, but without regard to starting letter. And random has the distinct advantage of not needing to store state between CGI invocations. The rest are premature optimizations, in my opinion. But once a final spec is decided, if the letters wanted to be equally weighted, then a dictionary of sorted lists might very well be the best storage mechanism. And once each starting letter gets its own web page, it also makes sense. DaveA From phlip2005 at gmail.com Fri Jan 22 13:51:22 2010 From: phlip2005 at gmail.com (Phlip) Date: Fri, 22 Jan 2010 10:51:22 -0800 (PST) Subject: counting lines of code References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> <87bpgodj95.fsf@benfinney.id.au> <1f166703-fdfc-4f87-b336-ef80e492635b@a32g2000yqm.googlegroups.com> <7e09df6a-cda1-480e-a971-8f8a70ac4118@b9g2000yqd.googlegroups.com> <980775cc-9101-4673-af03-fc17914d63f8@c4g2000yqa.googlegroups.com> Message-ID: <0d54eff2-59f0-4d95-aafb-478c9041c1b9@p8g2000yqb.googlegroups.com> On Jan 21, 9:00?pm, Michele Simionato wrote: > Just for fun I have run cloc on our trunk: > > SUM: ? ? ? ? ? ? ? ?8743 ? ?272238 ? ?215871 ? 1470139 x ? 1.84 = > 2708354.95 Nice! My favorite version of a cloc system can distinguish test from production code. That's why I always use executable cloc to measure the ratio of test to production code (where 1.2:1 is almost comfortable an 2:1 is sacred). Just so long as nobody confuses "more lines of code!" with progress... -- Phlip From python at rcn.com Fri Jan 22 14:11:21 2010 From: python at rcn.com (Raymond Hettinger) Date: Fri, 22 Jan 2010 11:11:21 -0800 (PST) Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> Message-ID: <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> On Jan 22, 6:13?am, Muhammad Alkarouri wrote: > In the python help for itertools, the following function is provided: > > def consume(iterator, n): > ? ? "Advance the iterator n-steps ahead. If n is none, consume > entirely." > ? ? collections.deque(islice(iterator, n), maxlen=0) > > What is the advantage of using a collections.deque against, say, the > following code? > > def consume(iterator, n): > ? ? for _ in islice(iterator, n): pass The deque version is faster. Raymond From showell30 at yahoo.com Fri Jan 22 14:14:32 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 11:14:32 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue Message-ID: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> The v2.6.4 version of the tutorial says this: ''' It is also possible to use a list as a queue, where the first element added is the first element retrieved (?first-in, first-out?); however, lists are not efficient for this purpose. While appends and pops from the end of list are fast, doing inserts or pops from the beginning of a list is slow (because all of the other elements have to be shifted by one). ''' Is that really true in CPython? It seems like you could advance the pointer instead of shifting all the elements. It would create some nuances with respect to reclaiming the memory, but it seems like an easy way to make lists perform better under a pretty reasonable use case. Does anybody know off the top of their head if the "have-to-be-shifted- by-one" warning is actually valid? http://docs.python.org/tutorial/datastructures.html From Martin.Drautzburg at web.de Fri Jan 22 14:16:31 2010 From: Martin.Drautzburg at web.de (Martin Drautzburg) Date: Fri, 22 Jan 2010 20:16:31 +0100 Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> <065e450f-2190-42f4-b9c6-4b95260092be@f12g2000yqn.googlegroups.com> Message-ID: <28103200.7M7XYOBnsK@beaureve.gmx.net> Mark Dickinson wrote: > On Jan 21, 10:57?pm, Martin Drautzburg > wrote: >> Here is a complete expample using a decorator, still a bit noisy >> >> def move(aDirection): >> print "moving " + aDirection >> >> #Here comes the decorator >> def scope(aDict): >> def save(locals): >> [...] > > Have you considered making 'scope' a context manager? Your > modifying-locals hacks could be put into the __enter__ and __exit__ > methods, and you'd use the context manager with something like: > > with scope(): > # ... > # use up, down, left, right here > > # up, down, left, right no longer defined after the with block exits. Wow! no I wasn't aware of that. I found some refererences to the magical "with" statement, but I had the impressions that it was a future thing and not available in python 2.5. Other than that it looks exactly like what I was looking for. Thanks. From Martin.Drautzburg at web.de Fri Jan 22 14:29:13 2010 From: Martin.Drautzburg at web.de (Martin Drautzburg) Date: Fri, 22 Jan 2010 20:29:13 +0100 Subject: A.x vs. A["x"] Message-ID: <1718504.hrdDYYapAE@beaureve.gmx.net> This has probably been asekd a million times, but if someone could give a short answer anyways I's be most grateful. What is it that allows one to write A.x? If I have a variable A, then what to I have to assign to it to A.x becomes valid? Or even further: what do I have to do so I can write A.x=1 without having done anything magical for x (but just for A)? I know you can do this with classes, but not with plain objects, but why is that so? From susan_kijiji at yahoo.ca Fri Jan 22 14:35:35 2010 From: susan_kijiji at yahoo.ca (susan_kijiji at yahoo.ca) Date: Fri, 22 Jan 2010 11:35:35 -0800 (PST) Subject: py2exe deal with python command line inside a program Message-ID: Hi, I need to create a python subprogress, like this: myProcess = subprocess.Popen([sys.executable, 'C:\myscript.py'], env=env, stdin=subprocess.PIPE, stdout=subprocess.PIPE) sys.executable was printed out as ''C:\\Python25\\python.exe'', how can I make this work in executable package through py2exe? I have to fix the following problems: -Source code shouldn't exposed in an executable program -Since python environment is not required when running an executable program, how to deal with the situation that "C:\\Python25\ \python.exe" is required as part of command? Thanks in advance! From Martin.Drautzburg at web.de Fri Jan 22 14:39:31 2010 From: Martin.Drautzburg at web.de (Martin Drautzburg) Date: Fri, 22 Jan 2010 20:39:31 +0100 Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> <065e450f-2190-42f4-b9c6-4b95260092be@f12g2000yqn.googlegroups.com> <28103200.7M7XYOBnsK@beaureve.gmx.net> Message-ID: <9734672.qlcV6ekshW@beaureve.gmx.net> Martin Drautzburg wrote: >> with scope(): >> # ... >> # use up, down, left, right here >> >> # up, down, left, right no longer defined after the with block exits. Just looked it up again. It's a cool thing. Too bad my locals() hack would still be required. The result would be less noisy (and actually really beautiful) than the decorator implementation though. Thanks again for pointing this out to me. From showell30 at yahoo.com Fri Jan 22 14:59:29 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 11:59:29 -0800 (PST) Subject: A.x vs. A["x"] References: <1718504.hrdDYYapAE@beaureve.gmx.net> Message-ID: On Jan 22, 11:29?am, Martin Drautzburg wrote: > This has probably been asekd a million times, but if someone could give > a short answer anyways I's be most grateful. Not sure there is exactly a short answer, and I am only qualified to maybe clarify some of the things you can and cannot do, not explain the reasons they are so. > What is it that allows one to write A.x? If I have a variable A, then > what to I have to assign to it to A.x becomes valid? Although not super concise, you'll find some good reading here: http://docs.python.org/tutorial/classes.html#a-word-about-names-and-objects It maybe does not address your question exactly, but it might give you insight into the overall philosophy. > Or even further: what do I have to do so I can write A.x=1 without > having done anything magical for x (but just for A)? I know you can do > this with classes, but not with plain objects, but why is that so? Here are examples where adding attributes on the fly does not work: Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41) [GCC 4.3.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> s = '' >>> s.x = 1 Traceback (most recent call last): File "", line 1, in AttributeError: 'str' object has no attribute 'x' >>> d = {} >>> d.x = 1 Traceback (most recent call last): File "", line 1, in AttributeError: 'dict' object has no attribute 'x' >>> n = 0 >>> n.x = 1 Traceback (most recent call last): File "", line 1, in AttributeError: 'int' object has no attribute 'x' Here are examples when you can assign new attributes: >>> class C: pass ... >>> C.x = 1 >>> C().x = 1 >>> def f(): pass ... >>> f.x = 1 >>> lam = lambda n: n >>> lam.x = 1 Hope that helps or at least gets the discussion started. From clp2 at rebertia.com Fri Jan 22 15:14:25 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Fri, 22 Jan 2010 12:14:25 -0800 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: <50697b2c1001221214x2c0798bas415296ed4f61ab50@mail.gmail.com> On Fri, Jan 22, 2010 at 11:14 AM, Steve Howell wrote: > The v2.6.4 version of the tutorial says this: > > ''' > It is also possible to use a list as a queue, where the first element > added is the first element retrieved (?first-in, first-out?); however, > lists are not efficient for this purpose. While appends and pops from > the end of list are fast, doing inserts or pops from the beginning of > a list is slow (because all of the other elements have to be shifted > by one). > ''' > > Is that really true in CPython? ?It seems like you could advance the > pointer instead of shifting all the elements. ?It would create some > nuances with respect to reclaiming the memory, but it seems like an > easy way to make lists perform better under a pretty reasonable use > case. > > Does anybody know off the top of their head if the "have-to-be-shifted- > by-one" warning is actually valid? Judging by the "Sorted dictionary" thread responses: Yes. Cheers, Chris -- http://blog.rebertia.com From showell30 at yahoo.com Fri Jan 22 15:22:21 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 12:22:21 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: On Jan 22, 12:14?pm, Chris Rebert wrote: > On Fri, Jan 22, 2010 at 11:14 AM, Steve Howell wrote: > > The v2.6.4 version of the tutorial says this: > > > ''' > > It is also possible to use a list as a queue, where the first element > > added is the first element retrieved (?first-in, first-out?); however, > > lists are not efficient for this purpose. While appends and pops from > > the end of list are fast, doing inserts or pops from the beginning of > > a list is slow (because all of the other elements have to be shifted > > by one). > > ''' > > > Is that really true in CPython? ?It seems like you could advance the > > pointer instead of shifting all the elements. ?It would create some > > nuances with respect to reclaiming the memory, but it seems like an > > easy way to make lists perform better under a pretty reasonable use > > case. > > > Does anybody know off the top of their head if the "have-to-be-shifted- > > by-one" warning is actually valid? > > Judging by the "Sorted dictionary" thread responses: Yes. > I think you are referring to this comment: ''' Insertion and deletion are still O(n) as all items to the right of the inserted/deleted one have to be shifted by one place. ''' http://groups.google.com/group/comp.lang.python/browse_thread/thread/d3699724d94d5b5a I can certainly see why most reasonable implementations of a list would have insertion/deletion in the middle of the list be O(N), but I don't think that limitation has to apply for the special cases of the beginning and end of the list. From jjposner at optimum.net Fri Jan 22 15:29:17 2010 From: jjposner at optimum.net (John Posner) Date: Fri, 22 Jan 2010 15:29:17 -0500 Subject: [2.5.1.1/dictionary] Change sorting order? In-Reply-To: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: <4b5a0af2$0$4995$607ed4bc@cv.net> On 1/22/2010 7:17 AM, Gilles Ganault wrote: > Hello > > I use a dictionary to keep a list of users connected to a web site. > > To avoid users from creating login names that start with digits in > order to be listed at the top, I'd like to sort the list differently > every minute so that it'll start with the next letter, eg. display the > list from A...Zdigits the first time, then B...ZAdigits, etc. > I don't believe anyone has suggested customizing the way that the dictionary's keys are sorted. How about using a string-like object ("RotaKey"), with a changeable sort order: #-------------------------- import string, time class RotaKey(object): """ string with rotating sort order """ LETTERS = string.ascii_uppercase def __init__(self, instring): self.string = instring @staticmethod def RotateSortOrder(): """ rotate the letter sequence for sorting RotaKey objects """ RotaKey.LETTERS = RotaKey.LETTERS[1:] + RotaKey.LETTERS[0] def __repr__(self): return "<%s>" % self.string def __cmp__(self, other): self_first, self_rest = self.string[0], self.string[1:] other_first, other_rest = other.string[0], other.string[1:] result = cmp(self.LETTERS.index(self_first.upper()), self.LETTERS.index(other_first.upper())) if result != 0: # first letters differ return result else: # first letters same, compare rest of strings return cmp(self_rest, other_rest) if __name__ == "__main__": # create dictionary: keys are RotaKey's, values in range 1001 - ... names = map(RotaKey, "Ant Zebra Bob Cat2 Ant2 Eagle Bob3 Bob2 Cat Dog".split()) datadict = dict(zip(names, range(1001, 1001+len(names)))) # TEST: print, rotate, print, rotate, ... for i in range(7): if i > 0: RotaKey.RotateSortOrder() print "=" * 15, i for k in sorted(datadict): print k, datadict[k] #-------------------------- NOTE: I tried implementing RotaKey as a subclass of "str", but found that the redefinition of __cmp__() was ignored. You can invoke RotaKey.RotateSortOrder() every minute (or whatever) to change the sort order. Program output: =============== 0 1001 1005 1003 1008 1007 1009 1004 1010 1006 1002 =============== 1 1003 1008 1007 1009 1004 1010 1006 1002 1001 1005 =============== 2 1009 1004 1010 1006 1002 ... etc. -John From lists at cheimes.de Fri Jan 22 15:40:08 2010 From: lists at cheimes.de (Christian Heimes) Date: Fri, 22 Jan 2010 21:40:08 +0100 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: <4B5A0D28.5080007@cheimes.de> Steve Howell wrote: > Is that really true in CPython? It seems like you could advance the > pointer instead of shifting all the elements. It would create some > nuances with respect to reclaiming the memory, but it seems like an > easy way to make lists perform better under a pretty reasonable use > case. > > Does anybody know off the top of their head if the "have-to-be-shifted- > by-one" warning is actually valid? Why do you think the documentation has such obvious errors? CPython's list type uses an array of pointers to store its members. The type is optimized for the most common list operations in Python: iteration and appending. Python code rarely changes the head or middle of a list. The dequeue type is an optimized data structure for popping and inserting data at both ends. When you list.pop() or list.insert() the pointers in the internal array must be shifted. appending is much faster because the internal array is overallocated meaning it contains free slots at the tail of the data structure. A linked list of pointers requires more memory and iteration is slower since since it can't utilize the CPU's cache as good as an array. Christian From showell30 at yahoo.com Fri Jan 22 15:57:46 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 12:57:46 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> On Jan 22, 12:40?pm, Christian Heimes wrote: > Steve Howell wrote: > > Is that really true in CPython? ?It seems like you could advance the > > pointer instead of shifting all the elements. ?It would create some > > nuances with respect to reclaiming the memory, but it seems like an > > easy way to make lists perform better under a pretty reasonable use > > case. > > > Does anybody know off the top of their head if the "have-to-be-shifted- > > by-one" warning is actually valid? > > Why do you think the documentation has such obvious errors? I wasn't making any assumptions, hence the question mark. The Python docs are very good, but even the best of projects make advances that aren't reflected in the docs. > CPython's list type uses an array of pointers to store its members. The > type is optimized for the most common list operations in Python: > iteration and appending. Python code rarely changes the head or middle > of a list. The dequeue type is an optimized data structure for popping > and inserting data at both ends. > I disagree that Python code rarely pops elements off the top of a list. There are perfectly valid use cases for wanting a list over a dequeue without having to pay O(N) for pop(0). Maybe we are just quibbling over the meaning of "rarely." > When you list.pop() or list.insert() the pointers in the internal array > must be shifted. appending is much faster because the internal array is > overallocated meaning it contains free slots at the tail of the data > structure. A linked list of pointers requires more memory and iteration > is slower since since it can't utilize the CPU's cache as good as an array. > I am not proposing a linked list of pointers. I am wondering about something like this: p = &p[1]; (and then reclaim p[0] as free memory, I already said I understood that was the tricky bit) The pointer arithmetic for accessing each element would still work in O (1), right? From arnodel at googlemail.com Fri Jan 22 16:08:48 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Fri, 22 Jan 2010 21:08:48 +0000 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: Steve Howell writes: > On Jan 22, 12:14?pm, Chris Rebert wrote: >> On Fri, Jan 22, 2010 at 11:14 AM, Steve Howell wrote: >> > The v2.6.4 version of the tutorial says this: >> >> > ''' >> > It is also possible to use a list as a queue, where the first element >> > added is the first element retrieved (?first-in, first-out?); however, >> > lists are not efficient for this purpose. While appends and pops from >> > the end of list are fast, doing inserts or pops from the beginning of >> > a list is slow (because all of the other elements have to be shifted >> > by one). >> > ''' >> >> > Is that really true in CPython? ?It seems like you could advance the >> > pointer instead of shifting all the elements. ?It would create some >> > nuances with respect to reclaiming the memory, but it seems like an >> > easy way to make lists perform better under a pretty reasonable use >> > case. >> >> > Does anybody know off the top of their head if the "have-to-be-shifted- >> > by-one" warning is actually valid? >> >> Judging by the "Sorted dictionary" thread responses: Yes. >> > > I think you are referring to this comment: > > ''' > Insertion and deletion are still O(n) as all items to the right of the > inserted/deleted one have to be shifted by one place. > ''' > > > I can certainly see why most reasonable implementations of a list > would have insertion/deletion in the middle of the list be O(N), but I > don't think that limitation has to apply for the special cases of the > beginning and end of the list. I made the comment you quoted. In CPython, it is O(n) to delete/insert an element at the start of the list - I know it because I looked at the implementation a while ago. This is why collections.deque exists I guess. I don't know how they are implemented but insertion/deletion at either end are O(1) and so is random access. So they are the data structure that you want. If you want evidence for lists, rather than my word, try: >>> import timeit >>> timeit.Timer('while t:del t[0]', 't=[0]*100000').timeit(1) 1.8452401161193848 >>> timeit.Timer('while t:del t[-1]', 't=[0]*100000').timeit(1) 0.017747163772583008 >>> timeit.Timer( 'while t:del t[0]', 'from collections import deque; t=deque([0]*100000)' ).timeit(1) 0.022077083587646484 >>> timeit.Timer( 'while t:del t[-1]', 'from collections import deque; t=deque([0]*100000)' ).timeit(1) 0.027813911437988281 -- Arnaud From kyosohma at gmail.com Fri Jan 22 16:19:43 2010 From: kyosohma at gmail.com (Mike Driscoll) Date: Fri, 22 Jan 2010 13:19:43 -0800 (PST) Subject: looking for Python live code reloading IDEs References: <40aa7d9f-aa35-4bfd-b0d9-46a10585731b@b9g2000yqd.googlegroups.com> Message-ID: <1161e21d-489c-40f6-a239-ca5c585abb6b@y23g2000yqm.googlegroups.com> On Jan 22, 3:16?am, George Oliver wrote: > hi, I'm wondering if there are any Python programming environments > that enable live code reloading, for example something like the Scheme- > based impromptu (but also meant for any kind of Python program, not > just audio/visual generation). > > Currently I do this directly in my editor (for game development), but > I'm curious if there are other options. Maybe you could get an idea from TurboGears or Django. They seem to reload their environments when a file is saved. ------------------- Mike Driscoll Blog: http://blog.pythonlibrary.org PyCon 2010 Atlanta Feb 19-21 http://us.pycon.org/ From kyosohma at gmail.com Fri Jan 22 16:24:20 2010 From: kyosohma at gmail.com (Mike Driscoll) Date: Fri, 22 Jan 2010 13:24:20 -0800 (PST) Subject: simple pub/sub References: <0af0eff2-50e2-44aa-81f1-b3d12cb265a7@a15g2000yqm.googlegroups.com> Message-ID: <1b44693a-e91e-4ac8-93ed-4934e126fdee@g1g2000yqi.googlegroups.com> On Jan 21, 10:54?am, Steve Howell wrote: > Hi, I'm looking for ideas on building a simple architecture that > allows a bunch of independent Python processes to exchange data using > files and perform calculations. > > One Python program would be collecting data from boat instruments on a > serial port, then writing that info out to a file, and also > transmitting data to instruments as needed (again through the serial > port). ?It would be the most complex program in terms of having to > poll file descriptors, or something similar, but I want to limit its > job to pure communication handling. > > Then, I would have a bunch of programs doing various calculations on > the input stream. ?It's totally acceptable if they just sleep a bit, > wake up, see if there is any new data, and if so, do some > calculations, so I will probably just use this: > > http://www.dabeaz.com/generators/follow.py > > Some of the "calculator" programs might be looking for trends in the > data, and when a certain threshold is reached, they will need to > notify the comm program of the new message. ?To avoid collisions > between all the different calculator programs, I am thinking the > simplest thing is just have them each write to their own output file. > > So then the comm program has multiple data sources. ?It wants to track > to the serial port, but it also wants to monitor the output files from > the various calculator processes. ?I want to do this in a mostly > nonblocking way, so I would probably poll on the file descriptors. > > The only problems are that select "...cannot be used on regular files > to determine whether a file has grown since it was last read," and it > only works on Unix for files. ?(I'm using Unix myself, but one of the > other programmers insists on Windows, and is resisting Python as > well.) > > So a variation would be that the calculator programs talk to the comm > program via sockets, which seems slightly on the heavy side, but it is > certainly feasible. > > I know that something like twisted could probably solve my problem, > but I'm also wondering if there is some solution that is a little more > lightweight and batteries-included. > > At the heart of the matter, I just want the programs to be able to do > the following tasks. > > ? 1) All programs will occasionally need to write a line of text to > some data stream without having to lock it. ?(I don't care if the > write blocks.) > ? 2) All programs will need to be able to do a non-blocking read of a > line of text from a data stream. ?(And there may be multiple > consumers.) > ? 3) The comm program will need to be able to poll the serial port and > input data streams to see which ones are ready. > > Any thoughts? Did you look at the pubsub module at all? See http://pubsub.sourceforge.net/ for more information. I use it a lot in my wxPython programs. ------------------- Mike Driscoll Blog: http://blog.pythonlibrary.org PyCon 2010 Atlanta Feb 19-21 http://us.pycon.org/ From showell30 at yahoo.com Fri Jan 22 16:26:03 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 13:26:03 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: <82dcbdd1-bf17-4f3c-8158-df054e69c79b@k35g2000yqb.googlegroups.com> On Jan 22, 1:08?pm, Arnaud Delobelle wrote: > Steve Howell writes: > > On Jan 22, 12:14?pm, Chris Rebert wrote: > >> On Fri, Jan 22, 2010 at 11:14 AM, Steve Howell wrote: > >> > The v2.6.4 version of the tutorial says this: > > >> > ''' > >> > It is also possible to use a list as a queue, where the first element > >> > added is the first element retrieved (?first-in, first-out?); however, > >> > lists are not efficient for this purpose. While appends and pops from > >> > the end of list are fast, doing inserts or pops from the beginning of > >> > a list is slow (because all of the other elements have to be shifted > >> > by one). > >> > ''' > > >> > Is that really true in CPython? ?It seems like you could advance the > >> > pointer instead of shifting all the elements. ?It would create some > >> > nuances with respect to reclaiming the memory, but it seems like an > >> > easy way to make lists perform better under a pretty reasonable use > >> > case. > > >> > Does anybody know off the top of their head if the "have-to-be-shifted- > >> > by-one" warning is actually valid? > > >> Judging by the "Sorted dictionary" thread responses: Yes. > > > I think you are referring to this comment: > > > ''' > > Insertion and deletion are still O(n) as all items to the right of the > > inserted/deleted one have to be shifted by one place. > > ''' > > > I can certainly see why most reasonable implementations of a list > > would have insertion/deletion in the middle of the list be O(N), but I > > don't think that limitation has to apply for the special cases of the > > beginning and end of the list. > > I made the comment you quoted. ?In CPython, it is O(n) to delete/insert > an element at the start of the list - I know it because I looked at the > implementation a while ago. ?This is why collections.deque exists I > guess. ?I don't know how they are implemented but insertion/deletion at > either end are O(1) and so is random access. ?So they are the data > structure that you want. > > If you want evidence for lists, rather than my word, try: > > >>> import timeit > >>> timeit.Timer('while t:del t[0]', 't=[0]*100000').timeit(1) > 1.8452401161193848 > >>> timeit.Timer('while t:del t[-1]', 't=[0]*100000').timeit(1) > > 0.017747163772583008 > > >>> timeit.Timer( > > ? ? 'while t:del t[0]', > ? ? 'from collections import deque; t=deque([0]*100000)' > ).timeit(1) > 0.022077083587646484>>> timeit.Timer( > > ? ? 'while t:del t[-1]', > ? ? 'from collections import deque; t=deque([0]*100000)' > ).timeit(1) > 0.027813911437988281 > Ok, thanks, good to know. I assume it's the colorly named list_ass_slice that would have to special case ilow == 0 and you'd need a memory manager that would let you realloc from ilow:ihigh to ilow+n:high. Am I reading that much of the code correctly? From tjreedy at udel.edu Fri Jan 22 16:27:14 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 22 Jan 2010 16:27:14 -0500 Subject: * operator in python tutorial In-Reply-To: References: <7b2ef0aa-8b93-4cb5-b944-beea3e2c98b1@p16g2000vbe.googlegroups.com> <0036df65$0$25085$c3e8da3@news.astraweb.com> Message-ID: On 1/22/2010 12:57 PM, ben wrote: > On Jan 20, 8:30 pm, Gringo wrote: >>> I am following through the python tutorial which gets to a line that >>> uses the * operator with zip(). All such questions are at least briefly answered in my complete python3 symbol glossary (all syntax usages) at http://code.google.com/p/xploro/downloads/list Terry Jan Reedy From lists at cheimes.de Fri Jan 22 16:29:23 2010 From: lists at cheimes.de (Christian Heimes) Date: Fri, 22 Jan 2010 22:29:23 +0100 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> Message-ID: <4B5A18B3.20200@cheimes.de> Steve Howell wrote: > I disagree that Python code rarely pops elements off the top of a > list. There are perfectly valid use cases for wanting a list over a > dequeue without having to pay O(N) for pop(0). Maybe we are just > quibbling over the meaning of "rarely." I was speaking from my own point of view. I've written several tenths of thousands of lines of Python code in the last seven years, mostly related to data manipulation, web applications and operating system interaction but also GUI stuff and scientific code. I can't recall any performance critical or prominent code that modifies the head of a list a lot. Of course there a use cases where you may want to use list.pop(). Unless you need a FILO structure you can always replace a LILO with a FIFO -- instead of list.insert(0, value) and list.pop(0) use list.append(value) and list.pop(). It's not possible to optimize a data structure for all use cases. > I am not proposing a linked list of pointers. I am wondering about > something like this: > > p = &p[1]; > (and then reclaim p[0] as free memory, I already said I understood > that was the tricky bit) > > The pointer arithmetic for accessing each element would still work in O > (1), right? You mean it's an impossible trick unless you come up with your own memory management system. Realloc(), the standard function to change the size of chunk of allocated memory in C, doesn't support your desired operation. Christian From steve at holdenweb.com Fri Jan 22 16:30:38 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 22 Jan 2010 16:30:38 -0500 Subject: counting lines of code In-Reply-To: References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> <87bpgodj95.fsf@benfinney.id.au> <1f166703-fdfc-4f87-b336-ef80e492635b@a32g2000yqm.googlegroups.com> <7e09df6a-cda1-480e-a971-8f8a70ac4118@b9g2000yqd.googlegroups.com> <%I36n.5962$4p5.2277@newsfe22.iad> Message-ID: Robert Kern wrote: > On 2010-01-21 15:31 , Phlip wrote: >> Aahz wrote: >>> In article >>> <7e09df6a-cda1-480e-a971-8f8a70ac4118 at b9g2000yqd.googlegroups.com>, >>> Phlip wrote: >>>> On Jan 20, 11:20=A0pm, Michele Simionato >>>> wrote: >>>>> pylint does too many things, I want something fast that just counts >>>>> the lines and can be run on thousands of files at once. >>>>> cloc seems fine, I have just tried on 2,000 files and it gives me a >>>>> report in just a few seconds. >>>> In my experience with Python codebases that big... >>>> >>>> ...how many of those lines are duplicated, and might merge together >>>> into a better design? >>> >>> Um... do you have any clue who you followed up to? If you don't, Google >>> is your friend. >> >> Oh, sorry, did I have the wrong opinion? > > You had a condescending attitude. > Towards someone who is fairly obviously not a Python neophyte. Please don't think we are telling you you can't have any opinion you like. Just don't expect to get away with it when you are wrong ;-) Welcome to c.l.py. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From icanbob at gmail.com Fri Jan 22 16:30:54 2010 From: icanbob at gmail.com (bobicanprogram) Date: Fri, 22 Jan 2010 13:30:54 -0800 (PST) Subject: simple pub/sub References: <0af0eff2-50e2-44aa-81f1-b3d12cb265a7@a15g2000yqm.googlegroups.com> Message-ID: <5dcf12b7-73b1-4ec4-ac9c-b984862bdc43@b10g2000vbh.googlegroups.com> On Jan 21, 11:54 am, Steve Howell wrote: > Hi, I'm looking for ideas on building a simple architecture that > allows a bunch of independent Python processes to exchange data using > files and perform calculations. > > One Python program would be collecting data from boat instruments on a > serial port, then writing that info out to a file, and also > transmitting data to instruments as needed (again through the serial > port). It would be the most complex program in terms of having to > poll file descriptors, or something similar, but I want to limit its > job to pure communication handling. > > Then, I would have a bunch of programs doing various calculations on > the input stream. It's totally acceptable if they just sleep a bit, > wake up, see if there is any new data, and if so, do some > calculations, so I will probably just use this: > > http://www.dabeaz.com/generators/follow.py > > Some of the "calculator" programs might be looking for trends in the > data, and when a certain threshold is reached, they will need to > notify the comm program of the new message. To avoid collisions > between all the different calculator programs, I am thinking the > simplest thing is just have them each write to their own output file. > > So then the comm program has multiple data sources. It wants to track > to the serial port, but it also wants to monitor the output files from > the various calculator processes. I want to do this in a mostly > nonblocking way, so I would probably poll on the file descriptors. > > The only problems are that select "...cannot be used on regular files > to determine whether a file has grown since it was last read," and it > only works on Unix for files. (I'm using Unix myself, but one of the > other programmers insists on Windows, and is resisting Python as > well.) > > So a variation would be that the calculator programs talk to the comm > program via sockets, which seems slightly on the heavy side, but it is > certainly feasible. > > I know that something like twisted could probably solve my problem, > but I'm also wondering if there is some solution that is a little more > lightweight and batteries-included. > > At the heart of the matter, I just want the programs to be able to do > the following tasks. > > 1) All programs will occasionally need to write a line of text to > some data stream without having to lock it. (I don't care if the > write blocks.) > 2) All programs will need to be able to do a non-blocking read of a > line of text from a data stream. (And there may be multiple > consumers.) > 3) The comm program will need to be able to poll the serial port and > input data streams to see which ones are ready. > > Any thoughts? You should definitely check out the SIMPL toolkit (http:// www.icanprogram.com/simpl). It has some pretty mature Python hooks. SIMPL has been used to construct data acquisition applications not unlike the one you describe. bob From tjreedy at udel.edu Fri Jan 22 16:32:47 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 22 Jan 2010 16:32:47 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: On 1/22/2010 2:14 PM, Steve Howell wrote: > The v2.6.4 version of the tutorial says this: > Is that really true in CPython? It seems like you could advance the > pointer instead of shifting all the elements. It would create some > nuances with respect to reclaiming the memory, but it seems like an > easy way to make lists perform better under a pretty reasonable use > case. Something like this was one proposed (ie, leave space at both ends of a list) but was rejected as over-complicating the list implementation for *relatively* rare use cases. I believe deque was written subsequently to address such other use cases. From lists at cheimes.de Fri Jan 22 16:35:37 2010 From: lists at cheimes.de (Christian Heimes) Date: Fri, 22 Jan 2010 22:35:37 +0100 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: <4B5A1A29.1050506@cheimes.de> Arnaud Delobelle wrote: > I made the comment you quoted. In CPython, it is O(n) to delete/insert > an element at the start of the list - I know it because I looked at the > implementation a while ago. This is why collections.deque exists I > guess. I don't know how they are implemented but insertion/deletion at > either end are O(1) and so is random access. So they are the data > structure that you want. Your assumption is correct. The collections.dequeue type uses a double linked list of blocks. Each blocks contains a fixed amount of pointers to Python objects. The implementation makes the implementation less memory hungry than a standard double linked list with just one element for each block. Christian From wbsoft at xs4all.nl Fri Jan 22 16:40:21 2010 From: wbsoft at xs4all.nl (Wilbert Berendsen) Date: Fri, 22 Jan 2010 22:40:21 +0100 Subject: counting lines of code In-Reply-To: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> Message-ID: <201001222240.21081.wbsoft@xs4all.nl> Op donderdag 21 januari 2010 schreef Michele: > I need a small utility to count the lines of Python code in a > directory, traversing subdirectories and ignoring comments and > docstrings. sloccount can do this. http://www.dwheeler.com/sloccount/ Met vriendelijke groet, Wilbert Berendsen -- http://www.wilbertberendsen.nl/ "You must be the change you wish to see in the world." -- Mahatma Gandhi From cs at zip.com.au Fri Jan 22 16:41:52 2010 From: cs at zip.com.au (Cameron Simpson) Date: Sat, 23 Jan 2010 08:41:52 +1100 Subject: looking for Python live code reloading IDEs In-Reply-To: <1161e21d-489c-40f6-a239-ca5c585abb6b@y23g2000yqm.googlegroups.com> References: <1161e21d-489c-40f6-a239-ca5c585abb6b@y23g2000yqm.googlegroups.com> Message-ID: <20100122214151.GA29661@cskk.homeip.net> On 22Jan2010 13:19, Mike Driscoll wrote: | On Jan 22, 3:16?am, George Oliver wrote: | > hi, I'm wondering if there are any Python programming environments | > that enable live code reloading, for example something like the Scheme- | > based impromptu (but also meant for any kind of Python program, not | > just audio/visual generation). | > | > Currently I do this directly in my editor (for game development), but | > I'm curious if there are other options. | | Maybe you could get an idea from TurboGears or Django. They seem to | reload their environments when a file is saved. CherrypPy too. -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ From tjreedy at udel.edu Fri Jan 22 16:46:03 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 22 Jan 2010 16:46:03 -0500 Subject: A.x vs. A["x"] In-Reply-To: <1718504.hrdDYYapAE@beaureve.gmx.net> References: <1718504.hrdDYYapAE@beaureve.gmx.net> Message-ID: On 1/22/2010 2:29 PM, Martin Drautzburg wrote: > This has probably been asekd a million times, but if someone could give > a short answer anyways I's be most grateful. > > What is it that allows one to write A.x? If I have a variable A, You do not really have a 'variable'. You have a name A bound to an object that is an instance of class C = type(A). > what to I have to assign to it to A.x becomes valid? If C has the appropriate special methods for get/set/del attribute, then you can get/set/del attributes for instances of C (such as A). If not, you cannot. In general, builtin classes, including object, allow get but not set/del on both the class and instances thereof. Subclasses of object allow all three. > Or even further: what do I have to do so I can write A.x=1 without > having done anything magical for x (but just for A)? Make A be a user-defined class or an instance thereof. > I know you can do this with classes, but not with plain objects, but why is that so? You exact meaning here is not clear, but I suspect it is somewhat incorrect, at least for built-in classes. Terry Jan Reedy From cjns1989 at gmail.com Fri Jan 22 16:47:55 2010 From: cjns1989 at gmail.com (Chris Jones) Date: Fri, 22 Jan 2010 16:47:55 -0500 Subject: Using dictionary key as a regular expression class Message-ID: <20100122214755.GH28576@turki.gavron.org> I was writing a script that counts occurrences of characters in source code files: #!/usr/bin/python import codecs tcounters = {} f = codecs.open('/home/gavron/git/screen/src/screen.c', 'r', "utf-8") for uline in f: lline = [] for char in uline[:-1]: lline += [char] counters = {} for i in set(lline): counters[i] = lline.count(i) for c in counters.keys(): if c in tcounters: tcounters[c] += counters[c] else: tcounters.update({c: counters[c]}) counters = {} for c in tcounters.keys(): print c, '\t', tcounters[c] I was looking for a way to stat by finger actions on a US keyboard for a traditional typist, and I was thinking I could use another dictionary with keys that would be made up of the characters correponding to each finger, such as '!1QqAaZz' for the left pinky, etc., hoping I would be able to iterate the keys and match the currently processed character with the key and increment it. Is this something that makes sense, or should I look elsewhere? Suggestions to improve the above snippet are also welcome. Thanks, CJ From aahz at pythoncraft.com Fri Jan 22 16:52:34 2010 From: aahz at pythoncraft.com (Aahz) Date: 22 Jan 2010 13:52:34 -0800 Subject: Bare Excepts References: <4dc0cfea0912300209x15e5bc96nd6e6224d89df0638@mail.gmail.com> <034fd208$0$1277$c3e8da3@news.astraweb.com> <80c56956-f28e-47a3-a723-3a5e3fd294b0@j19g2000yqk.googlegroups.com> Message-ID: In article <80c56956-f28e-47a3-a723-3a5e3fd294b0 at j19g2000yqk.googlegroups.com>, sjdevnull at yahoo.com wrote: >On Jan 2, 9:35=A0pm, Dave Angel wrote: >> >> In Windows, there is a way to do it. It's just not exposed to the >> Python built-in function open(). You use the CreateFile() function, >> with /dwCreationDisposition/ of CREATE_NEW. >> >> It's atomic, and fails politely if the file already exists. >> >> No idea if Unix has a similar functionality. > >It does. In Unix, you'd pass O_CREAT|O_EXCL to the open(2) system >call (O_CREAT means create a new file, O_EXCL means exclusive mode: >fail if the file exists already). Thanks! -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From gnarlodious at gmail.com Fri Jan 22 17:00:33 2010 From: gnarlodious at gmail.com (Gnarlodious) Date: Fri, 22 Jan 2010 14:00:33 -0800 (PST) Subject: Create object name from string value? References: <178a1b27-09e5-4584-964b-28430a010fd0@m25g2000yqc.googlegroups.com> <379b5939-f837-4964-9d80-4bebe541f6bc@36g2000yqu.googlegroups.com> Message-ID: <603b8956-bed4-4006-a466-c2e2096f37f7@c29g2000yqd.googlegroups.com> On Jan 21, 9:21?am, Dave Angel wrote: > Put it inside a dummy class, as follows: This has been very educational, thank you for all the suggestions. Here is the resulting code: class Property: pass # example of a dummy class Plist = Property() # create a Plist object from the dummy class # accumulate dictionary attributes based on every plist file dataFolder=os.path.dirname(__file__)+'/Data/' plistFiles=glob.glob(dataFolder+'*.plist') for file in range(len(plistFiles)): name=plistFiles[file].replace(dataFolder,'').replace('.plist','') setattr(Plist, name, Dict(plistFiles[file]).Config) # load the Plist object with the dictionary This is a vast improvement over my previous system. Now I can call any plist item from anywhere on my website: Data.Plist.Site.Trace where "Site.plist" is the name of the Plist file. This is such an elegant system I'm surprised it is not implemented as part of Python. -- Gnarlie http://Gnarlodious.com/Gnarlodious From arnodel at googlemail.com Fri Jan 22 17:07:13 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Fri, 22 Jan 2010 22:07:13 +0000 Subject: Using dictionary key as a regular expression class References: Message-ID: Chris Jones writes: > I was writing a script that counts occurrences of characters in source > code files: > > #!/usr/bin/python > import codecs > tcounters = {} > f = codecs.open('/home/gavron/git/screen/src/screen.c', 'r', "utf-8") > for uline in f: > lline = [] > for char in uline[:-1]: > lline += [char] > counters = {} > for i in set(lline): > counters[i] = lline.count(i) > for c in counters.keys(): > if c in tcounters: > tcounters[c] += counters[c] > else: > tcounters.update({c: counters[c]}) > counters = {} > for c in tcounters.keys(): > print c, '\t', tcounters[c] > > I was looking for a way to stat by finger actions on a US keyboard for a > traditional typist, and I was thinking I could use another dictionary > with keys that would be made up of the characters correponding to each > finger, such as '!1QqAaZz' for the left pinky, etc., hoping I would be > able to iterate the keys and match the currently processed character > with the key and increment it. > > Is this something that makes sense, or should I look elsewhere? > > Suggestions to improve the above snippet are also welcome. > > Thanks, > > CJ Why not just start with (untested): import codecs from collections import defaultdict tcounters = defaultdict(int) f = codecs.open('/home/gavron/git/screen/src/screen.c', 'r', "utf-8") for c in f.read(): tcounters[c] += 1 for c, n in tcounters.iteritems(): print "%r\t%i" % (c, n) -- Arnaud From showell30 at yahoo.com Fri Jan 22 17:38:18 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 14:38:18 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> Message-ID: <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> On Jan 22, 1:29?pm, Christian Heimes wrote: > Steve Howell wrote: > > I disagree that Python code rarely pops elements off the top of a > > list. ?There are perfectly valid use cases for wanting a list over a > > dequeue without having to pay O(N) for pop(0). ?Maybe we are just > > quibbling over the meaning of "rarely." > > I was speaking from my own point of view. I've written several tenths of > thousands of lines of Python code in the last seven years, mostly > related to data manipulation, web applications and operating system > interaction but also GUI stuff and scientific code. I can't recall any > performance critical or prominent code that modifies the head of a list > a lot. > That maybe would be an argument for just striking the paragraph from the tutorial. If it's rare that people pop the head off the list in code that is performance critical or prominent, why bother to even mention it in the tutorial? > Of course there a use cases where you may want to use list.pop(). Unless > you need a FILO structure you can always replace a LILO with a FIFO -- > instead of list.insert(0, value) and list.pop(0) use list.append(value) > and list.pop(). It's not possible to optimize a data structure for all > use cases. > > > I am not proposing a linked list of pointers. ?I am wondering about > > something like this: > > > p = &p[1]; > > (and then reclaim p[0] as free memory, I already said I understood > > that was the tricky bit) > > > The pointer arithmetic for accessing each element would still work in O > > (1), right? > > You mean it's an impossible trick unless you come up with your own > memory management system. Realloc(), the standard function to change the > size of chunk of allocated memory in C, doesn't support your desired > operation. > Impossible is a strong word. You could be lazy about giving the memory back, and just wait till the whole list is garbage collected. I don't think there's anything in Python's contract that says memory has to be freed the exact moment you stop using it, especially since we're talking about doing an O(N) memmove just to free up one pointer's worth of memory. I know the Python programmer could simply iterate through the list rather than popping off unused elements, but that just means that you not only tie up the memory for the pointers just as long, but you also prevent the objects themselves from being garbage collected. In my case I'm not really concerned about giving the memory back right away, it's more about keeping my code simple. Once I'm done with an element, I do just want to pop it off and keep all the simplicity of the lists, but I am just concerned enough speed that for a 1000 element list, I don't want to be doing 1000 memmoves for an average of 500 pointers, which effectively moves about a million bytes around for no reason. I suppose the solution is to either give up the sugar of lists, or try to wrap something like deque or list-with-offset into a sequence. From arobson73 at yahoo.co.uk Fri Jan 22 17:47:35 2010 From: arobson73 at yahoo.co.uk (G73) Date: Fri, 22 Jan 2010 14:47:35 -0800 (PST) Subject: Patch for IDLE/OS X to work with Tk-Cocoa Message-ID: <15238f19-e008-4e01-b284-721d3deba8ee@c29g2000yqd.googlegroups.com> im trying to update a patch. here is link to various patches http://bugs.python.org/issue6075 how do i update the patch, say for EditorWindow.patch. i have located my python installation the EditorWindow.py, and i can see some differences (which lines from the patch actually go into the EditorWindow.py ?. Also assuming i know which lines to edit, do i just edit the EditorWindow.py and that is it ? will i then be abler to run IDLE and this then pick up the changes i have made in EditorWindow/py ? im new to python and mac, but trying to learn it. From showell30 at yahoo.com Fri Jan 22 17:52:41 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 14:52:41 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: <0cac37c0-f858-4377-9653-4740cb655cdb@e11g2000yqe.googlegroups.com> On Jan 22, 1:32?pm, Terry Reedy wrote: > On 1/22/2010 2:14 PM, Steve Howell wrote: > > > The v2.6.4 version of the tutorial says this: > > Is that really true in CPython? ?It seems like you could advance the > > pointer instead of shifting all the elements. ?It would create some > > nuances with respect to reclaiming the memory, but it seems like an > > easy way to make lists perform better under a pretty reasonable use > > case. > > Something like this was one proposed (ie, leave space at both ends of a > list) but was rejected as over-complicating the list implementation for > *relatively* rare use cases. I believe deque was written subsequently to > address such other use cases. Bummer. I guess I get to do my own over-complicating of code, being in that unfortunate minority. From davea at ieee.org Fri Jan 22 17:54:15 2010 From: davea at ieee.org (Dave Angel) Date: Fri, 22 Jan 2010 17:54:15 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> Message-ID: <4B5A2C97.7060706@ieee.org> Steve Howell wrote: > On Jan 22, 12:40 pm, Christian Heimes wrote: > >> Steve Howell wrote: >> >>> Is that really true in CPython? It seems like you could advance the >>> pointer instead of shifting all the elements. It would create some >>> nuances with respect to reclaiming the memory, but it seems like an >>> easy way to make lists perform better under a pretty reasonable use >>> case. >>> >>> Does anybody know off the top of their head if the "have-to-be-shifted- >>> by-one" warning is actually valid? >>> >> Why do you think the documentation has such obvious errors? >> > > I wasn't making any assumptions, hence the question mark. The Python > docs are very good, but even the best of projects make advances that > aren't reflected in the docs. > > >> CPython's list type uses an array of pointers to store its members. The >> type is optimized for the most common list operations in Python: >> iteration and appending. Python code rarely changes the head or middle >> of a list. The dequeue type is an optimized data structure for popping >> and inserting data at both ends. >> >> > > I disagree that Python code rarely pops elements off the top of a > list. There are perfectly valid use cases for wanting a list over a > dequeue without having to pay O(N) for pop(0). Maybe we are just > quibbling over the meaning of "rarely." > > >> When you list.pop() or list.insert() the pointers in the internal array >> must be shifted. appending is much faster because the internal array is >> overallocated meaning it contains free slots at the tail of the data >> structure. A linked list of pointers requires more memory and iteration >> is slower since since it can't utilize the CPU's cache as good as an array. >> >> > > I am not proposing a linked list of pointers. I am wondering about > something like this: > > p =p[1]; > (and then reclaim p[0] as free memory, I already said I understood > that was the tricky bit) > > The pointer arithmetic for accessing each element would still work in O > (1), right? > > I think it was Dijkstr (sp?) who said you can accomplish anything with just one more level of indirection. Clearly each attribute (variable) that has a binding to a given list must point to a fixed piece of memory, that cannot safely be moved, because there's no efficient way to find all the attributes. That fixed piece is the list object, and I expect it's 16 or 20 bytes, on a 32bit implementation. It must in turn point to the actual malloc'ed block that contains pointers to all the elements of the list. So that block will be 4*n where n is the number of reserved cells, at least as large as len(). This is the block where copying takes place when you insert or delete from the beginning. The list object must contain a pointer to the beginning of this block, or it wouldn't be able to free() it later. So you'd be suggesting a second pointer that actually points to the current 0th pointer. And a pop would simply increment this second pointer. Such an approach might be worthwhile if you expect lots of pops and pushes, with a minimal net change. But of course each time you did a pop, you'd have to decide whether it was time to normalize/compact the block. As you say, reclaiming the 0th element of this block to the memory pool would be tricky. Doubly so, because 1) the C memory allocator has no such notion as resizing the beginning of a block. and 2) it would have nothing useful to do with the 4 bytes freed. The minimum allocated block in Python is probably 16 bytes of actual address space. I'd guess that's 4 bytes for malloc's overhead, and 8 bytes for the minimum object header, and 4 bytes for data. To check me, try: >>> a = 5.3 >>> b = 49.6 >>> id(a), id(b) (11074136, 11074152) Anyway, this could be done, where once the two pointers get some distance apart, you do a realloc, and copy everything. But of course you'd want to build some hysteresis into it, to avoid thrashing. There wouldn't be much of a performance hit, but it would increase every list size by 4 bytes minimum. So I doubt it would be a list replacement. This'd be an interesting project.to do as an addon module. DaveA From davea at ieee.org Fri Jan 22 17:57:36 2010 From: davea at ieee.org (Dave Angel) Date: Fri, 22 Jan 2010 17:57:36 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: <4B5A2D60.8080101@ieee.org> Arnaud Delobelle wrote: > Steve Howell writes: > > >> On Jan 22, 12:14 pm, Chris Rebert wrote: >> >> > I made the comment you quoted. In CPython, it is O(n) to delete/insert > an element at the start of the list - I know it because I looked at the > implementation a while ago. This is why collections.deque exists I > guess. I don't know how they are implemented but insertion/deletion at > either end are O(1) and so is random access. So they are the data > structure that you want. > > Not according to the 2.6 docs. Indexed access is O(1) at both ends but slows to O(n) in the middle. For fast random access, use lists instead. That sounds to me like a doubly-linked list implementation. From lists at cheimes.de Fri Jan 22 18:17:17 2010 From: lists at cheimes.de (Christian Heimes) Date: Sat, 23 Jan 2010 00:17:17 +0100 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> Message-ID: <4B5A31FD.5090300@cheimes.de> Steve Howell wrote: > That maybe would be an argument for just striking the paragraph from > the tutorial. If it's rare that people pop the head off the list in > code that is performance critical or prominent, why bother to even > mention it in the tutorial? How else are you going to teach new Python developers that they should favor append() and pop() in place of insert() and pop(0)? :) > Impossible is a strong word. You could be lazy about giving the > memory back, and just wait till the whole list is garbage collected. > I don't think there's anything in Python's contract that says memory > has to be freed the exact moment you stop using it, especially since > we're talking about doing an O(N) memmove just to free up one > pointer's worth of memory. Your proposal would increase the size of every list object of sizeof(ptr) or ssize_t (32 or 64bits) in order to store the information where the first element is. It would also unnecessarily complicate the code and possible slow down a lot of list operations. > I know the Python programmer could simply iterate through the list > rather than popping off unused elements, but that just means that you > not only tie up the memory for the pointers just as long, but you also > prevent the objects themselves from being garbage collected. > > In my case I'm not really concerned about giving the memory back right > away, it's more about keeping my code simple. Once I'm done with an > element, I do just want to pop it off and keep all the simplicity of > the lists, but I am just concerned enough speed that for a 1000 > element list, I don't want to be doing 1000 memmoves for an average of > 500 pointers, which effectively moves about a million bytes around for > no reason. The simplicity of Python is gained with some performance drawbacks. You have to learn to use Python algorithms. You can't simply re implement a fast C algorithm and expect it to be fast in Python, too. Christian From showell30 at yahoo.com Fri Jan 22 18:17:21 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 15:17:21 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> Message-ID: On Jan 22, 2:54?pm, Dave Angel wrote: > Steve Howell wrote: > > On Jan 22, 12:40 pm, Christian Heimes wrote: > > >> Steve Howell wrote: > > >>> Is that really true in CPython? ?It seems like you could advance the > >>> pointer instead of shifting all the elements. ?It would create some > >>> nuances with respect to reclaiming the memory, but it seems like an > >>> easy way to make lists perform better under a pretty reasonable use > >>> case. > > >>> Does anybody know off the top of their head if the "have-to-be-shifted- > >>> by-one" warning is actually valid? > > >> Why do you think the documentation has such obvious errors? > > > I wasn't making any assumptions, hence the question mark. ?The Python > > docs are very good, but even the best of projects make advances that > > aren't reflected in the docs. > > >> CPython's list type uses an array of pointers to store its members. The > >> type is optimized for the most common list operations in Python: > >> iteration and appending. Python code rarely changes the head or middle > >> of a list. The dequeue type is an optimized data structure for popping > >> and inserting data at both ends. > > > I disagree that Python code rarely pops elements off the top of a > > list. ?There are perfectly valid use cases for wanting a list over a > > dequeue without having to pay O(N) for pop(0). ?Maybe we are just > > quibbling over the meaning of "rarely." > > >> When you list.pop() or list.insert() the pointers in the internal array > >> must be shifted. appending is much faster because the internal array is > >> overallocated meaning it contains free slots at the tail of the data > >> structure. A linked list of pointers requires more memory and iteration > >> is slower since since it can't utilize the CPU's cache as good as an array. > > > I am not proposing a linked list of pointers. ?I am wondering about > > something like this: > > > p =p[1]; > > (and then reclaim p[0] as free memory, I already said I understood > > that was the tricky bit) > > > The pointer arithmetic for accessing each element would still work in O > > (1), right? > > I think it was Dijkstr (sp?) who said you can accomplish anything with > just one more level of indirection. ?Clearly each attribute (variable) > that has a binding to a given list must point to a fixed piece of > memory, that cannot safely be moved, because there's no efficient way to > find all the attributes. ? That fixed piece is the list object, and I > expect it's 16 or 20 bytes, on a 32bit implementation. ?It must in turn > point to the actual malloc'ed block that contains pointers to all the > elements of the list. ?So that block will be 4*n where n is the number > of reserved cells, at least as large as len(). ?This is the block where > copying takes place when you insert or delete from the beginning. > The indirection is already in Python, as it (at least appears to me) that everything is deferenced off of an ob_item pointer: http://svn.python.org/view/python/trunk/Objects/listobject.c?view=markup > The list object must contain a pointer to the beginning of this block, > or it wouldn't be able to free() it later. ?So you'd be suggesting a > second pointer that actually points to the current 0th pointer. ?And a > pop would simply increment this second pointer. > Yes, ob_item would point to the 0th element pointer and pop would simply increment it. The additional bookkeeping would be the original pointer. > Such an approach might be worthwhile if you expect lots of pops and > pushes, with a minimal net change. ?But of course each time you did a > pop, you'd have to decide whether it was time to normalize/compact ?the > block. > Yes, and that of course is the tricky bit. > As you say, reclaiming the 0th element of this block to the memory pool > would be tricky. ? I should clarify that a bit. Reclaiming the 0th element *cheaply* is tricky, unless you want to rewrite the memory manager. But I also think you can, of course, defer reclaiming the element. > Doubly so, because ?1) the C memory allocator has no > such notion as resizing the beginning of a block. and 2) it would have > nothing useful to do with the 4 bytes freed. ?The minimum allocated > block in Python is probably 16 bytes of actual address space. ?I'd guess > that's 4 bytes for malloc's overhead, and 8 bytes for the minimum object > header, and 4 bytes for data. ?To check me, try: > > ?>>> a = 5.3 > ?>>> b = 49.6 > ?>>> id(a), id(b) > (11074136, 11074152) > > Anyway, this could be done, where once the two pointers get some > distance apart, you do a realloc, and copy everything. ?But of course > you'd want to build some hysteresis into it, to avoid thrashing. > , but There wouldn't be any additional thrashing beyond what happens now. You'd simply avoid the first N-1 memmoves of up to kN bytes at the cost of not reclaiming those k(N-1) bytes right away. I suppose it's a more "bursty" penalty you'd be paying, but the peak of the "bursty" curve is no wider than the "constant" curve, it's just N times narrower! > There wouldn't be much of a performance hit, but it would increase every > list size by 4 bytes minimum. ?So I doubt it would be a list replacement. > I don't think that's the reason people would oppose this, although you are true about the penalty. Memory's cheap, you'd need about a quarter million lists just to fill up a meg. CPU cycles, on the other hand, are expensive, as users' demand for responsive programs seems to do a better job of keeping up with Moore's Law. I'd also argue that the memory you keep around to avoid unnecessary memmoves() will almost always be dwarfed by the memory used by the list elements themselves. From mrm at unknown.nospam Fri Jan 22 18:23:20 2010 From: mrm at unknown.nospam (Mr.M) Date: Fri, 22 Jan 2010 23:23:20 GMT Subject: PyArg_ParseTupleAndKeywords Message-ID: Hi, i can't understand what i'm doing wrong. I have a c/api that implements a new class. In (initproc) function i have somethink like this: [code] (some declarations omitted here) static char* keywordlist[] = {"service", "event_type", "free_text", "group_uid", "remote_name", "remote_abook_uid", "start_time", "end_time", "storage_time", "flags", "bytes_sent", "bytes_received", NULL}; if (!PyArg_ParseTupleAndKeywords(args, keywords, "|ssssssiiiiii", keywordlist, &service, &event_type, &free_text, &group_uid, &remote_name, &remote_abook_uid, &start_time, &end_time, &storage_time, &flags, &bytes_sent, &bytes_received)) return -1; [/code] Ok, what i want is something like this: [code] import mymodule a = mymodule.myclass() # ok, this works a = mymodule.myclass(flags = 1) # bad, this won't work > TypeError: expected string or Unicode object, tuple found [/code] I found that if i write this everything works fine: [code] import mymodule a = mymodule.myclass(service = "blabla", event_type = "event", free_text = "free", group_uid = "group", remote_name = "remote name", remote_abook_uid = "abook", start_time = 1, end_time = 61, storage_time = 61, flags = 2) [/code] in other words, the code only works if *EVERY* argument is specified in exactly the same position it appears in keyword-null-terminated array. Could you please help me? Thank you, Luca. From showell30 at yahoo.com Fri Jan 22 18:27:57 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 15:27:57 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> Message-ID: <98105b09-f522-432e-bac6-caa5010e9699@p24g2000yqm.googlegroups.com> On Jan 22, 3:17?pm, Christian Heimes wrote: > Steve Howell wrote: > > That maybe would be an argument for just striking the paragraph from > > the tutorial. ?If it's rare that people pop the head off the list in > > code that is performance critical or prominent, why bother to even > > mention it in the tutorial? > > How else are you going to teach new Python developers that they should > favor append() and pop() in place of insert() and pop(0)? :) > > > Impossible is a strong word. ?You could be lazy about giving the > > memory back, and just wait till the whole list is garbage collected. > > I don't think there's anything in Python's contract that says memory > > has to be freed the exact moment you stop using it, especially since > > we're talking about doing an O(N) memmove just to free up one > > pointer's worth of memory. > > Your proposal would increase the size of every list object of > sizeof(ptr) or ssize_t (32 or 64bits) in order to store the information > where the first element is. It would also unnecessarily complicate the > code and possible slow down a lot of list operations. > 64 bits per list is negligible. Adding a check for (ilow == 0) is even more negligible. You are not "unnecessarily" complicating code for something as widely used as Python lists if it achieves the desired benefit at minimal cost. You are complicating the code, but not "unnecessarily." > > I know the Python programmer could simply iterate through the list > > rather than popping off unused elements, but that just means that you > > not only tie up the memory for the pointers just as long, but you also > > prevent the objects themselves from being garbage collected. > > > In my case I'm not really concerned about giving the memory back right > > away, it's more about keeping my code simple. ?Once I'm done with an > > element, I do just want to pop it off and keep all the simplicity of > > the lists, but I am just concerned enough speed that for a 1000 > > element list, I don't want to be doing 1000 memmoves for an average of > > 500 pointers, which effectively moves about a million bytes around for > > no reason. > > The simplicity of Python is gained with some performance drawbacks. You > have to learn to use Python algorithms. You can't simply re implement a > fast C algorithm and expect it to be fast in Python, too. > I actually do expect Python to solve performance problems for me that are more easily solved in core than in Python itself. So I guess that's where we differ. From cjns1989 at gmail.com Fri Jan 22 18:38:44 2010 From: cjns1989 at gmail.com (Chris Jones) Date: Fri, 22 Jan 2010 18:38:44 -0500 Subject: Using dictionary key as a regular expression class In-Reply-To: References: Message-ID: <20100122233844.GJ28576@turki.gavron.org> On Fri, Jan 22, 2010 at 05:07:13PM EST, Arnaud Delobelle wrote: [..] > import codecs > from collections import defaultdict > > tcounters = defaultdict(int) > f = codecs.open('/home/gavron/git/screen/src/screen.c', 'r', "utf-8") > > for c in f.read(): > tcounters[c] += 1 > > for c, n in tcounters.iteritems(): > print "%r\t%i" % (c, n) Ah, yes.. much better - I grew suspicious of my 'effort' when I realized I could have written a shorter version in C. :-) CJ From aahz at pythoncraft.com Fri Jan 22 18:51:14 2010 From: aahz at pythoncraft.com (Aahz) Date: 22 Jan 2010 15:51:14 -0800 Subject: Efficient Running Median References: <497af344-31b5-4d1a-9b1a-c3d82feb31d3@j5g2000yqm.googlegroups.com> Message-ID: In article <497af344-31b5-4d1a-9b1a-c3d82feb31d3 at j5g2000yqm.googlegroups.com>, Raymond Hettinger wrote: > >The performance of an IndexableSkiplist is similar to a B+tree but the >implementation in pure python is much simpler. Nice! Can you summarize why IndexableSkipList is simpler? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From daniel at stutzbachenterprises.com Fri Jan 22 19:05:36 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Fri, 22 Jan 2010 18:05:36 -0600 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <98105b09-f522-432e-bac6-caa5010e9699@p24g2000yqm.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <98105b09-f522-432e-bac6-caa5010e9699@p24g2000yqm.googlegroups.com> Message-ID: On Fri, Jan 22, 2010 at 5:27 PM, Steve Howell wrote: > I actually do expect Python to solve performance problems for me that > are more easily solved in core than in Python itself. So I guess > that's where we differ. > You might be interested in the extension type I wrote (the "blist") that looks, acts, and quacks like a list, but takes worst-case O(log n) time for inserting and removing elements anywhere in the list. It's available for download here: http://pypi.python.org/pypi/blist/ And there's a detailed performance comparison with the built-in list here: http://stutzbachenterprises.com/performance-blist -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From showell30 at yahoo.com Fri Jan 22 19:09:03 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 16:09:03 -0800 (PST) Subject: medians for degree measurements Message-ID: I just saw the thread for medians, and it reminded me of a problem that I need to solve. We are writing some Python software for sailing, and we need to detect when we've departed from the median heading on the leg. Calculating arithmetic medians is straightforward, but compass bearings add a twist. The numerical median of 1, 2, 3, 4, 5, 6, 359 is 4. But for navigational purposes you would actually order the numbers 359, 1, 2, 3, 4, 5, 6, so the desired median heading of the boat is actually 3. Of course, you could express 359 better as -1 degrees to north, then the sequence would be -1, 1, 2, 3, 4, 5, and 6. And you'd be good. But that trick does not generalize if you go south instead, as you have similar issues with -179, 174, 175, 176, 177, 178, and 179. Has anybody solved this in Python, either for compass bearings or a different domain? I can think of kind of a brute force solution where you keep rotating the sequence until the endpoints are closest together mod 360, but I wonder if there is something more elegant. From robert.kern at gmail.com Fri Jan 22 19:45:46 2010 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 22 Jan 2010 18:45:46 -0600 Subject: medians for degree measurements In-Reply-To: References: Message-ID: On 2010-01-22 18:09 PM, Steve Howell wrote: > I just saw the thread for medians, and it reminded me of a problem > that I need to solve. We are writing some Python software for > sailing, and we need to detect when we've departed from the median > heading on the leg. Calculating arithmetic medians is > straightforward, but compass bearings add a twist. > > The numerical median of 1, 2, 3, 4, 5, 6, 359 is 4. But for > navigational purposes you would actually order the numbers 359, 1, 2, > 3, 4, 5, 6, so the desired median heading of the boat is actually 3. > > Of course, you could express 359 better as -1 degrees to north, then > the sequence would be -1, 1, 2, 3, 4, 5, and 6. And you'd be good. > > But that trick does not generalize if you go south instead, as you > have similar issues with -179, 174, 175, 176, 177, 178, and 179. > > Has anybody solved this in Python, either for compass bearings or a > different domain? I can think of kind of a brute force solution where > you keep rotating the sequence until the endpoints are closest > together mod 360, but I wonder if there is something more elegant. Brute force doesn't suck too much when using numpy to do the heavy lifting. In [158]: import numpy as np # Worst case. A von Mises distribution "centered" around the "South pole" # at pi/-pi. In [159]: theta = np.random.vonmises(np.pi, 1.0, size=1000) # Complex division makes circular distances easier to compute. In [160]: z = np.exp(1j * theta) # The newaxis bit plus numpy's broadcasting yields a 1000x1000 array of # the quotient of each pair of points in the dataset. In [161]: zdiv = z / z[:,np.newaxis] # Convert back to angular distances. Take the absolute value. Sum along one # dimension. Find the index of the element with the smallest mean absolute # circular deviation when used as the putative median. In [162]: i = abs(np.arctan2(zdiv.imag, zdiv.real)).sum(axis=1).argmin() # As expected, the result is close to the mode of the distribution. In [163]: theta[i] Out[163]: 3.0886753423606521 -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From python at mrabarnett.plus.com Fri Jan 22 20:12:11 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 23 Jan 2010 01:12:11 +0000 Subject: medians for degree measurements In-Reply-To: References: Message-ID: <4B5A4CEB.1020901@mrabarnett.plus.com> Steve Howell wrote: > I just saw the thread for medians, and it reminded me of a problem > that I need to solve. We are writing some Python software for > sailing, and we need to detect when we've departed from the median > heading on the leg. Calculating arithmetic medians is > straightforward, but compass bearings add a twist. > > The numerical median of 1, 2, 3, 4, 5, 6, 359 is 4. But for > navigational purposes you would actually order the numbers 359, 1, 2, > 3, 4, 5, 6, so the desired median heading of the boat is actually 3. > > Of course, you could express 359 better as -1 degrees to north, then > the sequence would be -1, 1, 2, 3, 4, 5, and 6. And you'd be good. > > But that trick does not generalize if you go south instead, as you > have similar issues with -179, 174, 175, 176, 177, 178, and 179. > > Has anybody solved this in Python, either for compass bearings or a > different domain? I can think of kind of a brute force solution where > you keep rotating the sequence until the endpoints are closest > together mod 360, but I wonder if there is something more elegant. > When you read the headings clockwise the values normally increase and you pick the middle one. However, when you cross north the values decrease. You can spot whether the set of headings crosses north by checking whether the difference between the minimum and maximum is greater than 180. If it is then make the westerly headings negative, sort the values, and pick the middle one, adding 180 if it's negative. def compass_median(points): points = sorted(points) if points[-1] - points[0] > 180: points = [(p - 360 if p > 180 else p) for p in points] points.sort() median = points[len(points) // 2] return median + 360 if median < 0 else median From tjreedy at udel.edu Fri Jan 22 20:46:35 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 22 Jan 2010 20:46:35 -0500 Subject: Using dictionary key as a regular expression class In-Reply-To: <20100122214755.GH28576@turki.gavron.org> References: <20100122214755.GH28576@turki.gavron.org> Message-ID: On 1/22/2010 4:47 PM, Chris Jones wrote: > I was writing a script that counts occurrences of characters in source code files: > > #!/usr/bin/python > import codecs > tcounters = {} > f = codecs.open('/home/gavron/git/screen/src/screen.c', 'r', "utf-8") > for uline in f: > lline = [] > for char in uline[:-1]: > lline += [char] Same but slower than lline.append(char), however, this loop just uselessless copies uline[:1] > counters = {} > for i in set(lline): > counters[i] = lline.count(i) slow way to do this > for c in counters.keys(): > if c in tcounters: > tcounters[c] += counters[c] > else: > tcounters.update({c: counters[c]}) I do not see the reason for intermediate dict > counters = {} duplicate line > for c in tcounters.keys(): > print c, '\t', tcounters[c] To only count ascii chars, as should be the case for C code, achars = [0]*63 for c in open('xxx', 'c'): try: achars[ord(c)-32] += 1 except IndexError: pass for i,n in enumerate(achars) print chr(i), n or sum subsets as desired. Terry Jan Reedy From kw at codebykevin.com Fri Jan 22 20:52:13 2010 From: kw at codebykevin.com (Kevin Walzer) Date: Fri, 22 Jan 2010 20:52:13 -0500 Subject: Patch for IDLE/OS X to work with Tk-Cocoa In-Reply-To: <15238f19-e008-4e01-b284-721d3deba8ee@c29g2000yqd.googlegroups.com> References: <15238f19-e008-4e01-b284-721d3deba8ee@c29g2000yqd.googlegroups.com> Message-ID: <3dc5d$4b5a564d$4275d90a$26905@FUSE.NET> On 1/22/10 5:47 PM, G73 wrote: > im trying to update a patch. here is link to various patches > http://bugs.python.org/issue6075 > > how do i update the patch, say for EditorWindow.patch. > i have located my python installation the EditorWindow.py, and i can > see some differences (which lines from the patch actually go into the > EditorWindow.py ?. > Also assuming i know which lines to edit, do i just edit the > EditorWindow.py and that is it ? will i then be abler to run IDLE and > this then pick up the changes i have made in EditorWindow/py ? > > im new to python and mac, but trying to learn it. To apply a patch, run "patch < patchfile," with patchfile being the patch. The command-line tool will prompt you for the name of the file to patch. I'm the author of those patches--hope you find them useful! -- Kevin Walzer Code by Kevin http://www.codebykevin.com From pavlovevidence at gmail.com Fri Jan 22 21:08:26 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Fri, 22 Jan 2010 18:08:26 -0800 (PST) Subject: PyArg_ParseTupleAndKeywords References: Message-ID: <91afd9e3-93a1-488d-8e83-299bfc222e24@22g2000yqr.googlegroups.com> On Jan 22, 3:23?pm, "Mr.M" wrote: > Hi, > > i can't understand what i'm doing wrong. I have a c/api that implements > a new class. > In (initproc) function i have somethink like this: > > [code] > > (some declarations omitted here) You probably shouldn't have, that could be where the error is.... I'd include the whole function up to the call that raises the exception. > static char* keywordlist[] = {"service", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"event_type", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"free_text", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"group_uid", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"remote_name", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"remote_abook_uid", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"start_time", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"end_time", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"storage_time", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"flags", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"bytes_sent", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"bytes_received", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? NULL}; > > if (!PyArg_ParseTupleAndKeywords(args, keywords, "|ssssssiiiiii", > keywordlist, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &service, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &event_type, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &free_text, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &group_uid, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &remote_name, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &remote_abook_uid, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &start_time, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &end_time, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &storage_time, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &flags, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &bytes_sent, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? &bytes_received)) return -1; This should be return NULL in most cases, but not all (such as if it's an object's tp_init--and it does look like that's the case here, so probably not the issue). > [/code] > > Ok, what i want is something like this: > > [code] > import mymodule > a = mymodule.myclass() # ok, this works > a = mymodule.myclass(flags = 1) # bad, this won't work > ?> TypeError: expected string or Unicode object, tuple found > [/code] Are you sure that PyArg_ParseTupleAndKeywords is what's raising the error? Are you subclassing this type in Python, and passing one of the string parameters a tuple by mistake? For instance, did you do something like this: class myclass(_mycmodule._myctype): def __init__(self,*args,**kwargs): log_something_here() super(myclass,self).__init__(args,**kwargs) Note the missing * on args. > I found that if i write this everything works fine: > > [code] > import mymodule > a = mymodule.myclass(service = "blabla", event_type = "event", free_text > ? = "free", group_uid = "group", remote_name = "remote name", > remote_abook_uid = "abook", start_time = 1, end_time = 61, storage_time > = 61, flags = 2) > [/code] > > in other words, the code only works if *EVERY* argument is specified in > exactly the same position it appears in keyword-null-terminated array. That it would have to be in a certain order is strange. Are you sure it's just not merely that they all have to be present? > Could you please help me? Tricky, but I think it'd help if you provided more information. My gut feeling is that there exception is being raised somewhere other than you think it is (i.e., not by PyArg_ParseTuple), so I'd recommend adding some debugging statements to track down the exact point wher the error occurs. Carl Banks From steve at REMOVE-THIS-cybersource.com.au Fri Jan 22 21:20:14 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 23 Jan 2010 02:20:14 GMT Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> Message-ID: <036a4966$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 14:38:18 -0800, Steve Howell wrote: > I know the Python programmer could simply iterate through the list > rather than popping off unused elements, but that just means that you > not only tie up the memory for the pointers just as long, but you also > prevent the objects themselves from being garbage collected. > > In my case I'm not really concerned about giving the memory back right > away, it's more about keeping my code simple. Once I'm done with an > element, I do just want to pop it off and keep all the simplicity of the > lists, but I am just concerned enough speed that for a 1000 element > list, I don't want to be doing 1000 memmoves for an average of 500 > pointers, which effectively moves about a million bytes around for no > reason. > > I suppose the solution is to either give up the sugar of lists, or try > to wrap something like deque or list-with-offset into a sequence. I don't understand what the actual problem you're trying to solve is. Despite your disclaimer about not being concerned about reclaiming the memory, it sounds like you're trying to micro-optimize memory usage. That is, you give me the impression that you prefer this: while alist: x = alist.pop(0) process(x) over this: for x in alist: process(x) # allow alist to be garbage collected when it goes out of scope That strikes me as a pointlessly trivial optimization, even if deleting at the start of the list was efficient. But whatever your reason, if you want to insert and delete efficiently from both ends of the sequence, use a deque. If you are only doing a small number of insertions/deletions at the beginning, and so don't care about inefficiency, use a list. If you only want to insert/delete from one end, use a list. Instead of: alist.insert(0, x) alist.pop(0) use: alist.append(x) alist.pop() That's fast and efficient. In some cases it doesn't matter which order the list is, but if it does matter, the worst case is that you need to call alist.reverse() occasionally, which is quite fast. Or iterate over the list in reverse order, which is even faster. So what am I missing? -- Steven From steve at holdenweb.com Fri Jan 22 21:31:17 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 22 Jan 2010 21:31:17 -0500 Subject: [ANN] hgview 1.2.0 In-Reply-To: <4b59df90$0$6582$9b4e6d93@newsspool3.arcor-online.net> References: <4b59df90$0$6582$9b4e6d93@newsspool3.arcor-online.net> Message-ID: <4B5A5F75.2020502@holdenweb.com> Stefan Behnel wrote: > For those who are a bit less fluent in French: > [...] ... and, while having access to e-mail, have not yet come across translate.google.com? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Fri Jan 22 21:31:17 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 22 Jan 2010 21:31:17 -0500 Subject: [ANN] hgview 1.2.0 In-Reply-To: <4b59df90$0$6582$9b4e6d93@newsspool3.arcor-online.net> References: <4b59df90$0$6582$9b4e6d93@newsspool3.arcor-online.net> Message-ID: <4B5A5F75.2020502@holdenweb.com> Stefan Behnel wrote: > For those who are a bit less fluent in French: > [...] ... and, while having access to e-mail, have not yet come across translate.google.com? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Fri Jan 22 21:41:18 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 22 Jan 2010 21:41:18 -0500 Subject: Mastering Python 3 I/O - Special Preview - Feb 5, 2010 (Chicago) In-Reply-To: <4EB3255C-2B1E-44E5-BE72-D61E44E580B4@dabeaz.com> References: <4EB3255C-2B1E-44E5-BE72-D61E44E580B4@dabeaz.com> Message-ID: <4B5A61CE.2030608@holdenweb.com> Dave: New York classes went well this week, and there appears to be some demand for Chicago training. How can we satisfy this demand to our common profit? regards Steve David Beazley wrote: > Mastering Python 3 I/O > ** PyCON'2010 Tutorial Preview in Chicago ** > > with David Beazley > February 5, 2010, 12pm - 5pm > http://www.dabeaz.com/chicago/index.html > > Can't make it to PyCON, but want to attend a cutting-edge tutorial on > the latest Python features? Join David Beazley, author of the Python > Essential Reference, in Chicago for a preview of his new tutorial > "Mastering Python 3 I/O." The goal of this tutorial is to take a top > to bottom tour of the Python 3 I/O system and to focus on essential > features that you must know if you are ever going to port existing > applications to Python 3 or use it for real applications. This > tutorial promises to go far beyond what you find in the documentation > and books (Dave's included). You'll learn about tricky gotchas, see > interesting practical examples, and get a better grasp of how Python 3 > is put together. > > This tutorial preview includes a free copy of the "Python Essential > Reference, 4th Ed.", lunch at one of Chicago's finest new restaurants, > artisinal pastries and more--all for the same price as a tutorial at > PyCON. However, it's strictly limited to 8 attendees. > > More information is available at: > > http://www.dabeaz.com/chicago/index.html > > Cheers, > Dave > -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From cjns1989 at gmail.com Fri Jan 22 21:58:48 2010 From: cjns1989 at gmail.com (Chris Jones) Date: Fri, 22 Jan 2010 21:58:48 -0500 Subject: Using dictionary key as a regular expression class In-Reply-To: References: <20100122214755.GH28576@turki.gavron.org> Message-ID: <20100123025848.GC2958@turki.gavron.org> On Fri, Jan 22, 2010 at 08:46:35PM EST, Terry Reedy wrote: > On 1/22/2010 4:47 PM, Chris Jones wrote: >> I was writing a script that counts occurrences of characters in source code files: >> >> #!/usr/bin/python >> import codecs >> tcounters = {} >> f = codecs.open('/home/gavron/git/screen/src/screen.c', 'r', "utf-8") >> for uline in f: >> lline = [] >> for char in uline[:-1]: >> lline += [char] > > Same but slower than lline.append(char), however, this loop just > uselessless copies uline[:1] I'll change that. Do you mean I should just read the file one character at a time? That was my original intention but I didn't find the way to do it. >> counters = {} >> for i in set(lline): >> counters[i] = lline.count(i) > > slow way to do this > >> for c in counters.keys(): >> if c in tcounters: >> tcounters[c] += counters[c] >> else: >> tcounters.update({c: counters[c]}) > > I do not see the reason for intermediate dict Couldn't find a way to increment the counters in the 'grand total' dictionary. I always ended up with the counter values for the last input line :-( Moot point if I can do a for loop reading one character at a time till end of file. >> counters = {} > > duplicate line And totally useless since I never reference it after that. Something I move else where and forgot to delete. Sorry about that. >> for c in tcounters.keys(): >> print c, '\t', tcounters[c] Literals, comments, ?'s..? > To only count ascii chars, as should be the case for C code, > > achars = [0]*63 > for c in open('xxx', 'c'): > try: > achars[ord(c)-32] += 1 > except IndexError: > pass > > for i,n in enumerate(achars) > print chr(i), n > > or sum subsets as desired. Thanks much for the snippet, let me play with it and see if I can come up with a Unicode/utf-8 version.. since while I'm at it I might as well write something a bit more general than C code. Since utf-8 is backward-compatible with 7bit ASCII, this shouldn't be a problem. > Terry Jan Reedy Thank you for your comments! CJ From roy at panix.com Fri Jan 22 22:04:10 2010 From: roy at panix.com (Roy Smith) Date: Fri, 22 Jan 2010 22:04:10 -0500 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: In article , Christian Heimes wrote: > Steve Howell wrote: > > Is that really true in CPython? It seems like you could advance the > > pointer instead of shifting all the elements. It would create some > > nuances with respect to reclaiming the memory, but it seems like an > > easy way to make lists perform better under a pretty reasonable use > > case. > > > > Does anybody know off the top of their head if the "have-to-be-shifted- > > by-one" warning is actually valid? > > Why do you think the documentation has such obvious errors? > > CPython's list type uses an array of pointers to store its members. The > type is optimized for the most common list operations in Python: > iteration and appending. Python code rarely changes the head or middle > of a list. The dequeue type is an optimized data structure for popping > and inserting data at both ends. > > When you list.pop() or list.insert() the pointers in the internal array > must be shifted. Well, at least in theory you could make pop(0) run in O(1). All you need to do is have each list store an offset. Initially it's 0, and pop(0) would just increment the offset. Then, all references to alist[i] would turn into array[i+offset]. Of course, that's a lot of complexity to optimize a relatively rare use case. You're probably better off just using a dequeue :-) From roy at panix.com Fri Jan 22 22:09:06 2010 From: roy at panix.com (Roy Smith) Date: Fri, 22 Jan 2010 22:09:06 -0500 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> Message-ID: In article <3ac173bd-4124-434d-b726-0b9baaeec752 at 36g2000yqu.googlegroups.com>, Steve Howell wrote: > In my case I'm not really concerned about giving the memory back right > away, it's more about keeping my code simple. Once I'm done with an > element, I do just want to pop it off and keep all the simplicity of > the lists, but I am just concerned enough speed that for a 1000 > element list, I don't want to be doing 1000 memmoves for an average of > 500 pointers, which effectively moves about a million bytes around for > no reason. If you really want to pop all the elements from a long list, reverse the list and pop them off the end. Popping every element off the beginning of the list is O(n^2), as you pointed out. Reversing the list is O(n), and each pop after that is O(1), so the overall complexity is O(n). From sccolbert at gmail.com Fri Jan 22 22:47:20 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Fri, 22 Jan 2010 22:47:20 -0500 Subject: Mastering Python 3 I/O - Special Preview - Feb 5, 2010 (Chicago) In-Reply-To: <4B5A61CE.2030608@holdenweb.com> References: <4EB3255C-2B1E-44E5-BE72-D61E44E580B4@dabeaz.com> <4B5A61CE.2030608@holdenweb.com> Message-ID: <7f014ea61001221947o7e17f0ebv9784c427346f6f08@mail.gmail.com> oops :) On Fri, Jan 22, 2010 at 9:41 PM, Steve Holden wrote: > Dave: > > New York classes went well this week, and there appears to be some > demand for Chicago training. How can we satisfy this demand to our > common profit? > > regards > Steve > > David Beazley wrote: > > Mastering Python 3 I/O > > ** PyCON'2010 Tutorial Preview in Chicago ** > > > > with David Beazley > > February 5, 2010, 12pm - 5pm > > http://www.dabeaz.com/chicago/index.html > > > > Can't make it to PyCON, but want to attend a cutting-edge tutorial on > > the latest Python features? Join David Beazley, author of the Python > > Essential Reference, in Chicago for a preview of his new tutorial > > "Mastering Python 3 I/O." The goal of this tutorial is to take a top > > to bottom tour of the Python 3 I/O system and to focus on essential > > features that you must know if you are ever going to port existing > > applications to Python 3 or use it for real applications. This > > tutorial promises to go far beyond what you find in the documentation > > and books (Dave's included). You'll learn about tricky gotchas, see > > interesting practical examples, and get a better grasp of how Python 3 > > is put together. > > > > This tutorial preview includes a free copy of the "Python Essential > > Reference, 4th Ed.", lunch at one of Chicago's finest new restaurants, > > artisinal pastries and more--all for the same price as a tutorial at > > PyCON. However, it's strictly limited to 8 attendees. > > > > More information is available at: > > > > http://www.dabeaz.com/chicago/index.html > > > > Cheers, > > Dave > > > > > -- > Steve Holden +1 571 484 6266 +1 800 494 3119 > PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ > Holden Web LLC http://www.holdenweb.com/ > UPCOMING EVENTS: http://holdenweb.eventbrite.com/ > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alekseymv at gmail.com Sat Jan 23 00:23:37 2010 From: alekseymv at gmail.com (Aleksey) Date: Fri, 22 Jan 2010 21:23:37 -0800 (PST) Subject: Problems with the date of modification of files on the flash drive in windows Message-ID: Hi All, I write crossplatform program and have next problem under windows (under linux is all OK) : I'm trying to get the UTC modification date of files on the flash drive under windows. In the flash card system is FAT. In the winter time (2010 01 21) Date of modification: >>> op.getmtime('g:\\alex\\bag\\pybag.log') 1263998652.0 In the summer time (2010 07 21) Date of the modification on the hour (3600 sec) different from winter time: >>> op.getmtime('g:\\alex\\bag\\pybag.log') 1263995052.0 Also time zone is "GMT" but real is "OMST" and localtime is wrong. In windows "tm_isdst" is always "0". For files on the HDD with NTFS modification time is right. How I can get right GMT time for files in flash with FAT under windows? Winter for Windows ---------------------- >>> time.altzone -3600 >>> time.daylight 0 >>> time.localtime() time.struct_time(tm_year=2010, tm_mon=1, tm_mday=21, tm_hour=3, tm_min=32, tm_sec=42, tm_wday=3, tm_yday=21, tm_isdst=0) >>> time.gmtime() time.struct_time(tm_year=2010, tm_mon=1, tm_mday=21, tm_hour=3, tm_min=32, tm_sec=48, tm_wday=3, tm_yday=21, tm_isdst=0) >>> time.tzname ('GMT', '') >>> time.timezone 0 Summer for Windows ----------------------------- >>> time.altzone -3600 >>> time.daylight 0 >>> time.localtime() time.struct_time(tm_year=2010, tm_mon=7, tm_mday=21, tm_hour=2, tm_min=13, tm_sec=18, tm_wday=2, tm_yday=202, tm_isdst=0) >>> time.gmtime() time.struct_time(tm_year=2010, tm_mon=7, tm_mday=21, tm_hour=2, tm_min=13, tm_sec=26, tm_wday=2, tm_yday=202, tm_isdst=0) >>> time.tzname ('GMT', '') >>> time.timezone 0 >From Linux for summer: --------------------------------- >>> time.altzone -25200 >>> time.daylight 1 >>> time.localtime() time.struct_time(tm_year=2010, tm_mon=7, tm_mday=23, tm_hour=12, tm_min=7, tm_sec=24, tm_wday=4, tm_yday=204, tm_isdst=1) >>> time.gmtime() time.struct_time(tm_year=2010, tm_mon=7, tm_mday=23, tm_hour=5, tm_min=7, tm_sec=36, tm_wday=4, tm_yday=204, tm_isdst=0) >>> time.timezone -21600 >>> time.tzname ('OMST', 'OMSST') >From Linux for winter: ------------------------------------ >>> time.altzone -25200 >>> time.daylight 1 >>> time.localtime() time.struct_time(tm_year=2010, tm_mon=1, tm_mday=23, tm_hour=11, tm_min=8, tm_sec=26, tm_wday=5, tm_yday=23, tm_isdst=0) >>> time.gmtime() time.struct_time(tm_year=2010, tm_mon=1, tm_mday=23, tm_hour=5, tm_min=8, tm_sec=39, tm_wday=5, tm_yday=23, tm_isdst=0) >>> time.timezone -21600 >>> time.tzname ('OMST', 'OMSST') From showell30 at yahoo.com Sat Jan 23 00:42:43 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 21:42:43 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> Message-ID: On Jan 22, 6:20?pm, Steven D'Aprano wrote: > On Fri, 22 Jan 2010 14:38:18 -0800, Steve Howell wrote: > > I know the Python programmer could simply iterate through the list > > rather than popping off unused elements, but that just means that you > > not only tie up the memory for the pointers just as long, but you also > > prevent the objects themselves from being garbage collected. > > > In my case I'm not really concerned about giving the memory back right > > away, it's more about keeping my code simple. ?Once I'm done with an > > element, I do just want to pop it off and keep all the simplicity of the > > lists, but I am just concerned enough speed that for a 1000 element > > list, I don't want to be doing 1000 memmoves for an average of 500 > > pointers, which effectively moves about a million bytes around for no > > reason. > > > I suppose the solution is to either give up the sugar of lists, or try > > to wrap something like deque or list-with-offset into a sequence. > > I don't understand what the actual problem you're trying to solve is. > Despite your disclaimer about not being concerned about reclaiming the > memory, it sounds like you're trying to micro-optimize memory usage. My discussion of memory probably distracted you from the fact that I'm not proposing a micro-optimization of memory; I am proposing a mega- optimization of CPU. This innocent program here literally moves about a million bytes of memory around for no good reason: lst = [] for i in range(2000): lst.append(i) while lst: print lst.pop(0) Why? Because list.pop(0) is implemented in O(N) instead of O(1). Why wouldn't you get a competent C programmer simply make list_ass_slice smart enough to make list.pop(0) O(1)? The brilliant computer scientist, Christian Heimes, provides the answers, and I am paraphrasing here, of course: 1) You can save 64 bits for every list by not storing an extra pointer! 2) You can simplify the CPython implementation! 3) You can avoid the oh-so-expensive check "if ilow == 1" for all operations that don't need this optimization! Sounds like two micro-optimizations to me (and a copout to boot). > That > is, you give me the impression that you prefer this: > > while alist: > ? ? x = alist.pop(0) > ? ? process(x) > > over this: > > for x in alist: > ? ? process(x) > # allow alist to be garbage collected when it goes out of scope > No, to be more precise, I prefer this implementation of a recursive parser (using lists) to one that would have to use deque's because of the lameness of Python's list implementation: https://bitbucket.org/showell/shpaml_website/src/tip/shpaml.py From showell30 at yahoo.com Sat Jan 23 00:58:28 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 21:58:28 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> Message-ID: <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> On Jan 22, 7:09?pm, Roy Smith wrote: > In article > <3ac173bd-4124-434d-b726-0b9baaeec... at 36g2000yqu.googlegroups.com>, > ?Steve Howell wrote: > > > In my case I'm not really concerned about giving the memory back right > > away, it's more about keeping my code simple. ?Once I'm done with an > > element, I do just want to pop it off and keep all the simplicity of > > the lists, but I am just concerned enough speed that for a 1000 > > element list, I don't want to be doing 1000 memmoves for an average of > > 500 pointers, which effectively moves about a million bytes around for > > no reason. > > If you really want to pop all the elements from a long list, reverse the > list and pop them off the end. ?Popping every element off the beginning of > the list is O(n^2), as you pointed out. ?Reversing the list is O(n), and > each pop after that is O(1), so the overall complexity is O(n). I really want to use list *normally* with all its perfectly good semantics and reasonable implementation, except for its blind spot with respect to popping the first element off the list. The whole reason I use CPython vs. C in the first place is that CPython programmers can generally program basic data structures better than I can. But list.pop(0) is the exception. And, with the possible exception of dicts, lists are the most fundamental data structures that Python has. I know Python's number one concern will never be speed, but if Python makes an O(1) operation into an unnecessarily O(N) operation for no good reasons other than "it's too complicated, " or it "adds another pointer to the structure," or "it adds another conditional check to list_ass_slice for operations that aren't popping off the top," I think it's reasonable to challenge the design philosophy. From showell30 at yahoo.com Sat Jan 23 01:09:54 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 22:09:54 -0800 (PST) Subject: medians for degree measurements References: Message-ID: <044268c0-c308-406e-8d5b-a38ef5b5e381@p24g2000yqm.googlegroups.com> On Jan 22, 5:12?pm, MRAB wrote: > Steve Howell wrote: > > I just saw the thread for medians, and it reminded me of a problem > > that I need to solve. ?We are writing some Python software for > > sailing, and we need to detect when we've departed from the median > > heading on the leg. ?Calculating arithmetic medians is > > straightforward, but compass bearings add a twist. > > > The numerical median of 1, 2, 3, 4, 5, 6, 359 is 4. ?But for > > navigational purposes you would actually order the numbers 359, 1, 2, > > 3, 4, 5, 6, so the desired median heading of the boat is actually 3. > > > Of course, you could express 359 better as -1 degrees to north, then > > the sequence would be -1, 1, 2, 3, 4, 5, and 6. ?And you'd be good. > > > But that trick does not generalize if you go south instead, as you > > have similar issues with -179, 174, 175, 176, 177, 178, and 179. > > > Has anybody solved this in Python, either for compass bearings or a > > different domain? ?I can think of kind of a brute force solution where > > you keep rotating the sequence until the endpoints are closest > > together mod 360, but I wonder if there is something more elegant. > > When you read the headings clockwise the values normally increase and > you pick the middle one. However, when you cross north the values > decrease. You can spot whether the set of headings crosses north by > checking whether the difference between the minimum and maximum is > greater than 180. If it is then make the westerly headings negative, > sort the values, and pick the middle one, adding 180 if it's negative. > > def compass_median(points): > ? ? ?points = sorted(points) > ? ? ?if points[-1] - points[0] > 180: > ? ? ? ? ?points = [(p - 360 if p > 180 else p) for p in points] > ? ? ? ? ?points.sort() > ? ? ?median = points[len(points) // 2] > ? ? ?return median + 360 if median < 0 else median I like this implementation, and it would probably work 99.9999% of the time for my particular use case. The only (very contrived) edge case that I can think of is when you have 10 bearings to SSW, 10 bearings to SSE, and the two outliers are unfortunately in the NE and NW quadrants. It seems like the algorithm above would pick one of the outliers. Maybe the refinement to the algorithm above would be to find the mean first, by summing sines and cosines of the bearings, taking the quotient, and applying the arctangent. Then use the resulting angle as the equivalent of "due north" and adjust angles to be within (-180, 180) respect to the mean, pretty much as you do in the code above, with minor modifications. I realize the problem as I stated it as sort of ill-defined. The way you stated the solution made me realize more deeply that you basically have a list that needs to be rotated either clockwise or counterclockwise in certain situations. Instead of using the 180- degree check to coarsely (but usually effectively) rotate your frame of reference, you could instead use the mean to eliminate even more edge cases. From nobody at nowhere.com Sat Jan 23 01:29:01 2010 From: nobody at nowhere.com (Nobody) Date: Sat, 23 Jan 2010 06:29:01 +0000 Subject: medians for degree measurements References: Message-ID: On Fri, 22 Jan 2010 16:09:03 -0800, Steve Howell wrote: > I just saw the thread for medians, and it reminded me of a problem > that I need to solve. We are writing some Python software for > sailing, and we need to detect when we've departed from the median > heading on the leg. Calculating arithmetic medians is > straightforward, but compass bearings add a twist. > > The numerical median of 1, 2, 3, 4, 5, 6, 359 is 4. But for > navigational purposes you would actually order the numbers 359, 1, 2, > 3, 4, 5, 6, so the desired median heading of the boat is actually 3. > > Of course, you could express 359 better as -1 degrees to north, then > the sequence would be -1, 1, 2, 3, 4, 5, and 6. And you'd be good. > > But that trick does not generalize if you go south instead, as you > have similar issues with -179, 174, 175, 176, 177, 178, and 179. > > Has anybody solved this in Python, either for compass bearings or a > different domain? I can think of kind of a brute force solution where > you keep rotating the sequence until the endpoints are closest > together mod 360, but I wonder if there is something more elegant. First, there isn't always a solution; what would you consider to be the median of [0, 90, 180, 270]? In the case where the bearings are clustered, one approach is to convert each bearing from polar to cartesian coordinates, compute the centroid, then convert back to polar coordinates, i.e.: from math import degrees, radians, sin, cos, atan2 def mean(bearings): x = sum(sin(radians(a)) for a in bearings) y = sum(cos(radians(a)) for a in bearings) return degrees(atan2(x, y)) Then, subtract the mean from each bearing, coerce all angles into the range -180..+180, calculate the median, add the mean, coerce back to 0..360. def median(bearings): m = mean(bearings) bearings = [(a - m + 180) % 360 - 180 for a in bearings] bearings.sort() median = bearings[len(bearings) / 2] median += m median %= 360 return median From showell30 at yahoo.com Sat Jan 23 02:03:05 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 23:03:05 -0800 (PST) Subject: medians for degree measurements References: Message-ID: <4ba80d57-71cb-442a-8146-2e44dfdc11c9@v25g2000yqk.googlegroups.com> On Jan 22, 10:29?pm, Nobody wrote: > On Fri, 22 Jan 2010 16:09:03 -0800, Steve Howell wrote: > > I just saw the thread for medians, and it reminded me of a problem > > that I need to solve. ?We are writing some Python software for > > sailing, and we need to detect when we've departed from the median > > heading on the leg. ?Calculating arithmetic medians is > > straightforward, but compass bearings add a twist. > > > The numerical median of 1, 2, 3, 4, 5, 6, 359 is 4. ?But for > > navigational purposes you would actually order the numbers 359, 1, 2, > > 3, 4, 5, 6, so the desired median heading of the boat is actually 3. > > > Of course, you could express 359 better as -1 degrees to north, then > > the sequence would be -1, 1, 2, 3, 4, 5, and 6. ?And you'd be good. > > > But that trick does not generalize if you go south instead, as you > > have similar issues with -179, 174, 175, 176, 177, 178, and 179. > > > Has anybody solved this in Python, either for compass bearings or a > > different domain? ?I can think of kind of a brute force solution where > > you keep rotating the sequence until the endpoints are closest > > together mod 360, but I wonder if there is something more elegant. > > First, there isn't always a solution; what would you consider to be the > median of [0, 90, 180, 270]? > You probably posted before seeing my recent post. I agree that the problem is ill-defined for certain cases. > In the case where the bearings are clustered, one approach is to > convert each bearing from polar to cartesian coordinates, compute the > centroid, then convert back to polar coordinates, i.e.: > > ? ? ? ? from math import degrees, radians, sin, cos, atan2 > > ? ? ? ? def mean(bearings): > ? ? ? ? ? ? ? ? x = sum(sin(radians(a)) for a in bearings) > ? ? ? ? ? ? ? ? y = sum(cos(radians(a)) for a in bearings) > ? ? ? ? ? ? ? ? return degrees(atan2(x, y)) > > Then, subtract the mean from each bearing, coerce all angles into the > range -180..+180, calculate the median, add the mean, coerce back to > 0..360. > > ? ? ? ? def median(bearings): > ? ? ? ? ? ? ? ? m = mean(bearings) > ? ? ? ? ? ? ? ? bearings = [(a - m + 180) % 360 - 180 for a in bearings] > ? ? ? ? ? ? ? ? bearings.sort() > ? ? ? ? ? ? ? ? median = bearings[len(bearings) / 2] > ? ? ? ? ? ? ? ? median += m > ? ? ? ? ? ? ? ? median %= 360 > ? ? ? ? ? ? ? ? return median Yep, that's exactly the solution I'm leaning toward now. From aahz at pythoncraft.com Sat Jan 23 02:10:23 2010 From: aahz at pythoncraft.com (Aahz) Date: 22 Jan 2010 23:10:23 -0800 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: In article <83082e19-9130-45a8-91f2-8601c1fdaac3 at 22g2000yqr.googlegroups.com>, Steve Howell wrote: > >I really want to use list *normally* with all its perfectly good >semantics and reasonable implementation, except for its blind spot >with respect to popping the first element off the list. The whole >reason I use CPython vs. C in the first place is that CPython >programmers can generally program basic data structures better than I >can. But list.pop(0) is the exception. And, with the possible >exception of dicts, lists are the most fundamental data structures >that Python has. > >I know Python's number one concern will never be speed, but if Python >makes an O(1) operation into an unnecessarily O(N) operation for no >good reasons other than "it's too complicated, " or it "adds another >pointer to the structure," or "it adds another conditional check to >list_ass_slice for operations that aren't popping off the top," I >think it's reasonable to challenge the design philosophy. "Rough consensus and running code." You have a good point, but nobody will ever give your idea serious attention until there's a patch and benchmarks. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From nobody at nowhere.com Sat Jan 23 02:19:40 2010 From: nobody at nowhere.com (Nobody) Date: Sat, 23 Jan 2010 07:19:40 +0000 Subject: Problems with the date of modification of files on the flash drive in windows References: Message-ID: On Fri, 22 Jan 2010 21:23:37 -0800, Aleksey wrote: > I write crossplatform program and have next problem under windows > (under linux is all OK) : > > I'm trying to get the UTC modification date of files on the flash > drive under windows. In the flash card system is FAT. Timestamps stored on a FAT filesystem use an unspecified timezone. When a Windows PC writes a timestamp to a FAT filesystem, it stores the current local time. For devices such as cameras, the timestamp will be whatever's in the device's clock if it has one (even if it does, there's no guarantee that it's accurate), or some arbitrary value otherwise. When reading a timestamp, it's just used as-is, i.e. it's a local time with no timezone specified. There is no way to convert this value to UTC or to a specific timezone. If you're using FAT, timestamps are pretty much meaningless unless the device never travels between timezones (in which case, they'll probably be roughly correct or wrong by roughly an hour either way due to DST). When it comes to timezones, anything developed prior to NT basically pretends that they don't exist. From showell30 at yahoo.com Sat Jan 23 02:43:03 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 23:43:03 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: <8e4d3fe2-c4bd-4a73-9c50-7a336dab25be@o28g2000yqh.googlegroups.com> On Jan 22, 11:10?pm, a... at pythoncraft.com (Aahz) wrote: > > >I know Python's number one concern will never be speed, but if Python > >makes an O(1) operation into an unnecessarily O(N) operation for no > >good reasons other than "it's too complicated, " or it "adds another > >pointer to the structure," or "it adds another conditional check to > >list_ass_slice for operations that aren't popping off the top," I > >think it's reasonable to challenge the design philosophy. > > "Rough consensus and running code." > > You have a good point, but nobody will ever give your idea serious > attention until there's a patch and benchmarks. Here is a benchmark of O(N*N) vs. O(N) for two C programs. One does memmove; the other simply advances the pointer. showell at showell-laptop:~$ time ./slow real 0m1.446s user 0m1.444s sys 0m0.004s showell at showell-laptop:~$ time ./fast real 0m0.003s user 0m0.004s sys 0m0.000s showell at showell-laptop:~$ diff slow.c fast.c 23,24c23 < lst.size -= 1; < memmove(lst.p, lst.p + 1, lst.size); --- > lst.p += 1; showell at showell-laptop:~$ cat slow.c #include #include #include struct ob_item { int whatever; }; struct list { struct ob_item *p; int size; }; struct list make_list(int n) { struct list lst; lst.p = malloc(n); lst.size = n; return lst; } void list_pop_left(struct list lst) { lst.size -= 1; memmove(lst.p, lst.p + 1, lst.size); } int main() { struct list lst; int i; int n = 40000; int t; lst = make_list(n); for (i = 0; i < n; ++i) { list_pop_left(lst); } } From alfps at start.no Sat Jan 23 02:43:28 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sat, 23 Jan 2010 08:43:28 +0100 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: * Steve Howell: > On Jan 22, 7:09 pm, Roy Smith wrote: >> In article >> <3ac173bd-4124-434d-b726-0b9baaeec... at 36g2000yqu.googlegroups.com>, >> Steve Howell wrote: >> >>> In my case I'm not really concerned about giving the memory back right >>> away, it's more about keeping my code simple. Once I'm done with an >>> element, I do just want to pop it off and keep all the simplicity of >>> the lists, but I am just concerned enough speed that for a 1000 >>> element list, I don't want to be doing 1000 memmoves for an average of >>> 500 pointers, which effectively moves about a million bytes around for >>> no reason. >> If you really want to pop all the elements from a long list, reverse the >> list and pop them off the end. Popping every element off the beginning of >> the list is O(n^2), as you pointed out. Reversing the list is O(n), and >> each pop after that is O(1), so the overall complexity is O(n). > > I really want to use list *normally* with all its perfectly good > semantics and reasonable implementation, except for its blind spot > with respect to popping the first element off the list. The whole > reason I use CPython vs. C in the first place is that CPython > programmers can generally program basic data structures better than I > can. But list.pop(0) is the exception. And, with the possible > exception of dicts, lists are the most fundamental data structures > that Python has. Having optimized list.pop() for first element, then you would have a blind spot with respect to insertion at the front... Which could then be optimized for the cases where there is some buffer space at the front, left over from a pop. I think it would just be harder to understand and harder to explain. And I think that due to that, as usual people would build their own elaborate "explanations", with erroneous conclusions, and would then use it in inefficient ways (like, popping from the middle or inserting at the front). I think the "harder to use correctly" is the strongest argument against the optimization, but there is also a non-obvious *memory overhead*... Having popped just one element at the front, in order for the implementation to not /accumulate/ unused memory, that is, in order to avoid an ongoing /memory leak/, extending the buffer to accomodate e.g. an append() can no longer be done as a (on relevant systems) constant time reallocation (letting the OS do its virtual memory paging magic). Instead, a new buffer would have to be allocated and all data copied over. One could still have amortized constant time for appends by using a doubling buffer (which is the strategy used in C++ 'std::vector'), but at the cost of wasting some memory, a percentage... The implementation as a pure array is easy to understand and use correctly, and doesn't waste memory. But it would IMHO have been better if it wasn't called "list", which brings in the wrong associations for someone used to other languages. The name does matter. E.g. I don't think this discussion about a pop optimization would have been there except for the name, which makes that optimization sound reasonable. Perhaps some standard alternative name could be devised. Like, "array" would have been nice, except that that's already grabbed by the array module. > I know Python's number one concern will never be speed, but if Python > makes an O(1) operation into an unnecessarily O(N) operation for no > good reasons other than "it's too complicated, " or it "adds another > pointer to the structure," or "it adds another conditional check to > list_ass_slice for operations that aren't popping off the top," I > think it's reasonable to challenge the design philosophy. See above. In addition to being more difficult /to use/ correctly, that is, being much easier to misunderstand, it incurs a memory overhead -- not the one directly from the pop optimization, but by the requirements of buffer extension. Essentially, as discussed above, it would then have to use a doubling buffer. Cheers & hth., - Alf From tjreedy at udel.edu Sat Jan 23 02:45:41 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 23 Jan 2010 02:45:41 -0500 Subject: Using dictionary key as a regular expression class In-Reply-To: <20100123025848.GC2958@turki.gavron.org> References: <20100122214755.GH28576@turki.gavron.org> <20100123025848.GC2958@turki.gavron.org> Message-ID: On 1/22/2010 9:58 PM, Chris Jones wrote: > On Fri, Jan 22, 2010 at 08:46:35PM EST, Terry Reedy wrote: > Do you mean I should just read the file one character at a time? Whoops, my misdirection (you can .read(1), but this is s l o w. I meant to suggest processing it a char at a time. 1. If not too big, for c in open(x, 'rb').read() # left .read() off # 'b' will get bytes, though ord(c) same for ascii chars for byte or unicode 2. If too big for that, for line in open(): for c in line: # or left off this part >> To only count ascii chars, as should be the case for C code, >> >> achars = [0]*63 >> for c in open('xxx', 'c'): >> try: >> achars[ord(c)-32] += 1 >> except IndexError: >> pass >> >> for i,n in enumerate(achars) >> print chr(i), n >> >> or sum subsets as desired. > > Thanks much for the snippet, let me play with it and see if I can come > up with a Unicode/utf-8 version.. since while I'm at it I might as well > write something a bit more general than C code. > > Since utf-8 is backward-compatible with 7bit ASCII, this shouldn't be > a problem. For any extended ascii, use larger array without decoding (until print, if need be). For unicode, add encoding to open and 'c in line' will return unicode chars. Then use *one* dict or defaultdict. I think something like from collections import defaultdict d = defaultdict(int) ... d[c] += 1 # if c is new, d[c] defaults to int() == 0 Terry Jan Reedy From showell30 at yahoo.com Sat Jan 23 02:58:29 2010 From: showell30 at yahoo.com (Steve Howell) Date: Fri, 22 Jan 2010 23:58:29 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: <2ce05a04-ebc9-4f26-ba32-ff787925a564@c29g2000yqd.googlegroups.com> On Jan 22, 11:10?pm, a... at pythoncraft.com (Aahz) wrote: > In article <83082e19-9130-45a8-91f2-8601c1fda... at 22g2000yqr.googlegroups.com>, > Steve Howell ? wrote: > > > > > > >I really want to use list *normally* with all its perfectly good > >semantics and reasonable implementation, except for its blind spot > >with respect to popping the first element off the list. ?The whole > >reason I use CPython vs. C in the first place is that CPython > >programmers can generally program basic data structures better than I > >can. ?But list.pop(0) is the exception. ?And, with the possible > >exception of dicts, lists are the most fundamental data structures > >that Python has. > > >I know Python's number one concern will never be speed, but if Python > >makes an O(1) operation into an unnecessarily O(N) operation for no > >good reasons other than "it's too complicated, " or it "adds another > >pointer to the structure," or "it adds another conditional check to > >list_ass_slice for operations that aren't popping off the top," I > >think it's reasonable to challenge the design philosophy. > > "Rough consensus and running code." > > You have a good point, but nobody will ever give your idea serious > attention until there's a patch and benchmarks. Another benchmark is that deques are slower than lists for accessing elements. showell at showell-laptop:~$ python foo.py 0.0215361118317 <- list 0.0429010391235 <- deque import time from collections import deque n = 40000 lst = [] for i in range(n): lst.append(i) t = time.time() for i in range(n): lst[i] print time.time() - t lst = deque(lst) t = time.time() for i in range(n): lst[i] print time.time() - t So substituting deque for list suffers not just in convenience, but also in performance. From tjreedy at udel.edu Sat Jan 23 03:13:49 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 23 Jan 2010 03:13:49 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: On 1/23/2010 12:58 AM, Steve Howell wrote: > I really want to use list *normally* with all its perfectly good > semantics and reasonable implementation, except for its blind spot > with respect to popping the first element off the list. It was not designed for that. .pop() was added to lists about 10 years ago because I asked for it (with no parameter, pop off end only) and wrote what would now be a PEP -- and because Tim Peters later supported the idea. Adding the optional parameter was something of an afterthought (never publicly discussed as far as I know) for occasional use for 'short' lists where O(n) is tolerable. You have half persuaded me that that the parameter addition was a mistake. Perhaps is is too attractice a nuisance for some people ;=). > The whole > reason I use CPython vs. C in the first place is that CPython > programmers can generally program basic data structures better than I > can. They have given us three options other that .pop(0). 1. listiterator 2. queue.Queue 3. collections.deque\ Why are you so stubborn about not using a data structure intended for your use case instead of one that was not? There is also 4. a two-list design for queues: iterator through one (or pop() from a reversed version thereof) while appending to another; switch when the first is empty. I plan to write it up with tests some day this year. > I know Python's number one concern will never be speed, but if Python > makes an O(1) operation into an unnecessarily O(N) operation for no > good reasons other than "it's too complicated, " or it "adds another > pointer to the structure," or "it adds another conditional check to > list_ass_slice for operations that aren't popping off the top," I > think it's reasonable to challenge the design philosophy. Challenge yes, mock no. Part of writing good basic data structures is not adding needless complication from featuritis and not penalizing 99.99% of access to satify a .01% need better satisfied another way. Terry Jan Reedy From tjreedy at udel.edu Sat Jan 23 03:20:24 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 23 Jan 2010 03:20:24 -0500 Subject: medians for degree measurements In-Reply-To: <044268c0-c308-406e-8d5b-a38ef5b5e381@p24g2000yqm.googlegroups.com> References: <044268c0-c308-406e-8d5b-a38ef5b5e381@p24g2000yqm.googlegroups.com> Message-ID: On 1/23/2010 1:09 AM, Steve Howell wrote: > On Jan 22, 5:12 pm, MRAB wrote: > I like this implementation, and it would probably work 99.9999% of the > time for my particular use case. The only (very contrived) edge case > that I can think of is when you have 10 bearings to SSW, 10 bearings > to SSE, and the two outliers are unfortunately in the NE and NW > quadrants. It seems like the algorithm above would pick one of the > outliers. > > Maybe the refinement to the algorithm above would be to find the mean > first, by summing sines and cosines of the bearings, taking the > quotient, and applying the arctangent. Then use the resulting angle > as the equivalent of "due north" and adjust angles to be within (-180, > 180) respect to the mean, pretty much as you do in the code above, > with minor modifications. I was going to suggest this. Let us know if it seems to work. > I realize the problem as I stated it as sort of ill-defined. Yes, I think this one reason stat books typically ignore directional data. I think it is an unfortunate omission. Terry Jan Reedy From showell30 at yahoo.com Sat Jan 23 03:23:28 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sat, 23 Jan 2010 00:23:28 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: On Jan 23, 12:13?am, Terry Reedy wrote: > > Challenge yes, mock no. > > Part of writing good basic data structures is not adding needless > complication from featuritis and not penalizing 99.99% of access to > satify a .01% need better satisfied another way. > I would like to challenge your assertion that advancing ob_item instead of doing memmove during list_ass_slice would impact the performance of list accesses in any way. It would only slow down operations that add/insert items into the list by, and then only by a single conditional statement, and those add/insert operations are already O(N) to begin with. From alfps at start.no Sat Jan 23 03:32:26 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sat, 23 Jan 2010 09:32:26 +0100 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: * Steve Howell: > On Jan 23, 12:13 am, Terry Reedy wrote: >> Challenge yes, mock no. >> >> Part of writing good basic data structures is not adding needless >> complication from featuritis and not penalizing 99.99% of access to >> satify a .01% need better satisfied another way. >> > > I would like to challenge your assertion that advancing ob_item > instead of doing memmove during list_ass_slice would impact the > performance of list accesses in any way. It would only slow down > operations that add/insert items into the list by, and then only by a > single conditional statement, and those add/insert operations are > already O(N) to begin with. I'm sorry, no, the last part is incorrect. Appending to a 'list' can currently be constant time, if OS reallocation is constant time (as the string '+' optimization relies on). With the pop optimization it can no longer be constant time without risking an accumulation of unused memory, a memory leak, although it can be amortized constant time, at the cost of wasting some percentage of memory. Cheers & hth., - Alf From showell30 at yahoo.com Sat Jan 23 03:54:19 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sat, 23 Jan 2010 00:54:19 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: <2ffb4ab8-ae8f-4b69-b449-56d0232a404b@y23g2000yqm.googlegroups.com> On Jan 23, 12:32?am, "Alf P. Steinbach" wrote: > * Steve Howell: > > > On Jan 23, 12:13 am, Terry Reedy wrote: > >> Challenge yes, mock no. > > >> Part of writing good basic data structures is not adding needless > >> complication from featuritis and not penalizing 99.99% of access to > >> satify a .01% need better satisfied another way. > > > I would like to challenge your assertion that advancing ob_item > > instead of doing memmove during list_ass_slice would impact the > > performance of list accesses in any way. ?It would only slow down > > operations that add/insert items into the list by, and then only by a > > single conditional statement, and those add/insert operations are > > already O(N) to begin with. > > I'm sorry, no, the last part is incorrect. > > Appending to a 'list' can currently be constant time, if OS reallocation is > constant time (as the string '+' optimization relies on). > That's true, but it's also irrelevant, as the pop optimization would happen in a branch of code that never gets executed during list appending: if (d < 0) { /* Delete -d items */ /* ADD CODE HERE TO AVOID memmove when ilow == 0 (i.e. ihigh + d == 0) */ memmove(&item[ihigh+d], &item[ihigh], (Py_SIZE(a) - ihigh)*sizeof(PyObject *)); list_resize(a, Py_SIZE(a) + d); item = a->ob_item; } > With the pop optimization it can no longer be constant time without risking an > accumulation of unused memory, a memory leak, although it can be amortized > constant time, at the cost of wasting some percentage of memory. list_resize already overallocates memory to allow room for growth. Whenever you did an append to the list that would force a realloc, you could first check to see if there is unused stuff at the front and do the memmove then and reclaim the unfreed memory. So instead of doing a paying for memmove on every pop, you only pay for it when the list goes to size 0, 4, 8, 16, 25, 35, 46, 58, 72, 88, etc. > From groups_ads12 at yahoo.com Sat Jan 23 04:09:40 2010 From: groups_ads12 at yahoo.com (groups_ads12) Date: Sat, 23 Jan 2010 09:09:40 -0000 Subject: www.phptutorial.me Message-ID: www.phptutorial.me php php help free php free php scripts htaccess php learn php learning php open php php and php applications php array php arrays php book php books php cgi php checkbox php class php classes php code php comments php course php curl php database php date php developer php development php doc php download php example php extension php fgets php filter php find php fopen php foreach php form php forms php function php functions php game php gd php get php header php html php http php if php include php index php insert php install php installation php isset php job php jobs php language php live php login php magic php mail php max php next php now php outsourcing php pagination php parse php pear php perl php post php print php program php programmer php programming php query php redirect php replace php rest php return php script php scripts php search php select php session php sessions php set cookie php software php source code php sql php string php strtotime php support php syntax php table php template php test php training php tutorial php update php upload php url php user php variable php video php vs php website php windows php xml smarty php -------------- next part -------------- An HTML attachment was scrubbed... URL: From cjns1989 at gmail.com Sat Jan 23 04:22:39 2010 From: cjns1989 at gmail.com (Chris Jones) Date: Sat, 23 Jan 2010 04:22:39 -0500 Subject: Using dictionary key as a regular expression class In-Reply-To: References: <20100122214755.GH28576@turki.gavron.org> <20100123025848.GC2958@turki.gavron.org> Message-ID: <20100123092239.GC2963@turki.gavron.org> On Sat, Jan 23, 2010 at 02:45:41AM EST, Terry Reedy wrote: > On 1/22/2010 9:58 PM, Chris Jones wrote: >> Do you mean I should just read the file one character at a time? > > Whoops, my misdirection (you can .read(1), but this is s l o w. > I meant to suggest processing it a char at a time. Right.. that's how I understood it - i.e. asking python for the next character, and not worrying about how much is retrieved from the disk in one pass. > 1. If not too big, > > for c in open(x, 'rb').read() # left .read() off > # 'b' will get bytes, though ord(c) same for ascii chars for byte or > unicode > > 2. If too big for that, > > for line in open(): > for c in line: # or left off this part Well the script is not going to process anything larger that a few KiloBytes, but all the same that's something I want to understand better. Isn't there any way I can tell python to retrieve a fairly large chunk of the disk file, like 4-8K, maybe.. and increment a pointer behind the scenes while I iterate so that I have access to characters one at a time. I mean, that should be pretty fast, since disk access would be minimal, and no data would actually be copied.. I would have thought that 1. above would cause python to do something like that behind the scenes. [..] >> Thanks much for the snippet, let me play with it and see if I can >> come up with a Unicode/utf-8 version.. since while I'm at it I might >> as well write something a bit more general than C code. >> >> Since utf-8 is backward-compatible with 7bit ASCII, this shouldn't be >> a problem. > For any extended ascii, You mean 8-bit encodings, like latin1 right? > use larger array without decoding (until print, if need be). For > unicode, add encoding to open and 'c in line' will return unicode > chars. Then use *one* dict or defaultdict. I think something like > from collections import defaultdict > d = defaultdict(int) > ... > d[c] += 1 # if c is new, d[c] defaults to int() == 0 I don't know python, so I basically googled for the building blocks of my little script.. and I remember seeing defaultdict(int) mentioned some place or other, but somehow I didn't understand what it did. Cool feature. Even if it's a bit wasteful, with unicode/utf-8, it looks like working with the code points, either as dictionary keys or as index values into an array might make the logic simpler - i.e. for each char, obtain its code point 'cp' and add one to dict[cp] or array[cp] - and then loop and print all non-zero values when the end-of-file condition is reached. Food for thought in any case. CJ From tjreedy at udel.edu Sat Jan 23 04:24:40 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 23 Jan 2010 04:24:40 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: On 1/23/2010 3:23 AM, Steve Howell wrote: > On Jan 23, 12:13 am, Terry Reedy wrote: >> >> Challenge yes, mock no. >> >> Part of writing good basic data structures is not adding needless >> complication from featuritis and not penalizing 99.99% of access to >> satify a .01% need better satisfied another way. >> > > I would like to challenge your assertion that advancing ob_item > instead of doing memmove during list_ass_slice would impact the > performance of list accesses in any way. It would only slow down > operations that add/insert items into the list by, and then only by a > single conditional statement, and those add/insert operations are > already O(N) to begin with. If you try writing a full patch, as I believe someone did, or at least a prototype thereof, when the idea was discussed, you might have a better idea of what the tradeoffs are and why it was rejected. For instance, when you append to a full list, it is resized. I believe it is now doubled, but the policy has varied over the years. If you turn list from essentially a stack to a deque, you complicate the resizing policy and have to consider the addition of a shift policy. Do you split the over-allocated fore and aft or increase the overallocation from double to, say, triple? If the former, then for normal usage that never uses the fore part, the over-allocation has been effectively reduced from 2x to 1.5x (which is about what it once was, I believe). This means more frequent resizings and copyings, which means slower operation for most use cases. Adding extra usually wasted space is also a nuisance. Terry Jan Reedy From showell30 at yahoo.com Sat Jan 23 04:27:12 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sat, 23 Jan 2010 01:27:12 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: On Jan 23, 12:13?am, Terry Reedy wrote: > On 1/23/2010 12:58 AM, Steve Howell wrote: > > > I really want to use list *normally* with all its perfectly good > > semantics and reasonable implementation, except for its blind spot > > with respect to popping the first element off the list. > > It was not designed for that. .pop() was added to lists about 10 years > ago because I asked for it (with no parameter, pop off end only) and > wrote what would now be a PEP -- and because Tim Peters later supported > the idea. Adding the optional parameter was something of an afterthought > (never publicly discussed as far as I know) for occasional use for > 'short' lists where O(n) is tolerable. You have half persuaded me that > that the parameter addition was a mistake. Perhaps is is too attractice > a nuisance for some people ;=). > pop(0) is a useful idiom in parsers. You can see examples in ElementTree and lib2to3. Even without pop(0), people would still write code like this, found in pstats.py: arg = args[0] args = args[1:] It is sometimes overkill (and even inappropriate) to use a queue when really you just want a list. Iterators are great, but they also have slightly different semantics than the list itself. There is nothing wrong with a language specification that allows users to do insert, delete, and pop on a list. Once you freeze the language specification, then you can turn your attention to improving the implementation. From mrm at unknown.nospam Sat Jan 23 04:34:15 2010 From: mrm at unknown.nospam (Mr.M) Date: Sat, 23 Jan 2010 09:34:15 GMT Subject: PyArg_ParseTupleAndKeywords In-Reply-To: <91afd9e3-93a1-488d-8e83-299bfc222e24@22g2000yqr.googlegroups.com> References: <91afd9e3-93a1-488d-8e83-299bfc222e24@22g2000yqr.googlegroups.com> Message-ID: Carl Banks ha scritto: >> (some declarations omitted here) > > You probably shouldn't have, that could be where the error is.... I'd > include the whole function up to the call that raises the exception. Thank you so much Carl for your help, i'll provide more info so that you can try to fix my errors! Thank you again! First of all i'll try to explain what i've discovered over a night i spent awake trying to find where the bug is. My init behaves differently in this situations: [code] # this works dummy = mymodule.myclass() # this works dummy = mymodule.myclass("string for service") # this also works dummy = mymodule.myclass("string for service", "string for event_type") # and it works if i provide arguments in the right sequence, of course # this won't work dummy = mymodule.myclass("string for service", "string for event_type", free_text = "string for free_text") # but this works dummy = mymodule.myclass(service = "string for service") # the worst thing: this doesn't work but it did and i can't # understand what is changed dummy = mymodule.myclass(free_text = "text for free_text") ok, every time the code fails i get this exception: TypeError: expected string or Unicode object, tuple found so, i think for some reason PyArg_ParseTupleAndKeywords receives "args" as a touple instead of something else. This is everything i managed to discover. This is the code: [code] /* DEBUG INFO */ printf("event INIT\n"); if (args) { int i; printf("args = \"%s\"\n", PyString_AsString(args)); printf("type = %s\n", PyString_AsString(PyObject_Repr(PyObject_Type(args)))); printf("tuple size = %d\n", PyTuple_Size(args)); for (i = 0; i < PyTuple_Size(args); i++) { printf("%d) %s\n", i, PyString_AsString(PyTuple_GetItem(args, i))); } } else { printf("args = NULL\n"); } printf("dict:\n"); if (keywords) { printf(" = %s\n", PyString_AsString(PyObject_Repr(keywords))); printf("type = %s\n", PyString_AsString(PyObject_Repr(PyObject_Type(keywords)))); } else { printf("keywords = NULL\n"); } char* service = NULL; char* event_type = NULL; char* free_text = NULL; char* group_uid = NULL; char* remote_name = NULL; char* remote_abook_uid = NULL; time_t start_time = -1L; time_t end_time = -1L; time_t storage_time = -1L; int flags = 0; int bytes_sent = -1; int bytes_received = -1; PyObject* temp; static char* keywordlist[] = {"service", "event_type", "free_text", "group_uid", "remote_name", "remote_abook_uid", "start_time", "end_time", "storage_time", "flags", "bytes_sent", "bytes_received", NULL}; if (!PyArg_ParseTupleAndKeywords(args, keywords, "|ssssssiiiiii", keywordlist, &service, &event_type, &free_text, &group_uid, &remote_name, &remote_abook_uid, &start_time, &end_time, &storage_time, &flags, &bytes_sent, &bytes_received)) { return -1; } printf("PyArg_ParseTupleAndKeywords worked fine\n"); [/code] (sorry if my code is a little messy and my english rather bad!) > Are you sure that PyArg_ParseTupleAndKeywords is what's raising the > error? Yes, i think so. I have a lot of printf in my code! > Are you subclassing this type in Python, and passing one of the string > parameters a tuple by mistake? For instance, did you do something > like this: > > class myclass(_mycmodule._myctype): > def __init__(self,*args,**kwargs): > log_something_here() > super(myclass,self).__init__(args,**kwargs) > > Note the missing * on args. no, i'm not subclassing. I have to admit that i had some testcase and a few weeks ago they worked like a charm... i can't understand what i changed, of course. > > >> I found that if i write this everything works fine: >> >> [code] >> import mymodule >> a = mymodule.myclass(service = "blabla", event_type = "event", free_text >> = "free", group_uid = "group", remote_name = "remote name", >> remote_abook_uid = "abook", start_time = 1, end_time = 61, storage_time >> = 61, flags = 2) >> [/code] >> >> in other words, the code only works if *EVERY* argument is specified in >> exactly the same position it appears in keyword-null-terminated array. > > That it would have to be in a certain order is strange. Are you sure > it's just not merely that they all have to be present? > No, i tryied to remove the "|" and Python complains that 12 argument must be present. So they are, in a sense, optional, but (and this is strange), they must be present in the right order even i provide them with keyword arguments. > Tricky, but I think it'd help if you provided more information. My > gut feeling is that there exception is being raised somewhere other > than you think it is (i.e., not by PyArg_ParseTuple), so I'd recommend > adding some debugging statements to track down the exact point wher > the error occurs. Ok, thank you! The code i posted is exactly the same i run (and it fails). I've no problem sending you the whole module or adding any other debug info you think could be helpfull to find the source of the bug. Ciao, Luca. From alfps at start.no Sat Jan 23 04:38:55 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sat, 23 Jan 2010 10:38:55 +0100 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <2ffb4ab8-ae8f-4b69-b449-56d0232a404b@y23g2000yqm.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> <2ffb4ab8-ae8f-4b69-b449-56d0232a404b@y23g2000yqm.googlegroups.com> Message-ID: * Steve Howell: > On Jan 23, 12:32 am, "Alf P. Steinbach" wrote: >> * Steve Howell: >> >>> On Jan 23, 12:13 am, Terry Reedy wrote: >>>> Challenge yes, mock no. >>>> Part of writing good basic data structures is not adding needless >>>> complication from featuritis and not penalizing 99.99% of access to >>>> satify a .01% need better satisfied another way. >>> I would like to challenge your assertion that advancing ob_item >>> instead of doing memmove during list_ass_slice would impact the >>> performance of list accesses in any way. It would only slow down >>> operations that add/insert items into the list by, and then only by a >>> single conditional statement, and those add/insert operations are >>> already O(N) to begin with. >> I'm sorry, no, the last part is incorrect. >> >> Appending to a 'list' can currently be constant time, if OS reallocation is >> constant time (as the string '+' optimization relies on). >> > > That's true, but it's also irrelevant, as the pop optimization would > happen in a branch of code that never gets executed during list > appending: > > if (d < 0) { /* Delete -d items */ > /* ADD CODE HERE TO AVOID memmove > when ilow == 0 (i.e. ihigh + d == 0) > */ > memmove(&item[ihigh+d], &item[ihigh], > (Py_SIZE(a) - ihigh)*sizeof(PyObject *)); > list_resize(a, Py_SIZE(a) + d); > item = a->ob_item; > } > > >> With the pop optimization it can no longer be constant time without risking an >> accumulation of unused memory, a memory leak, although it can be amortized >> constant time, at the cost of wasting some percentage of memory. > > list_resize already overallocates memory to allow room for growth. > Whenever you did an append to the list that would force a realloc, you > could first check to see if there is unused stuff at the front and do > the memmove then and reclaim the unfreed memory. So instead of doing > a paying for memmove on every pop [at front], you only pay for it when > the list goes to size 0, 4, 8, 16, 25, 35, 46, 58, 72, 88, etc. Oh. If 'list' already uses a doubling buffer then the only overhead from the optimization would, AFAICS, be a single add in every indexing. Which might be acceptable (at least it sounds very reasonable in the context of Python). Re terminology: I write "doubling buffer" to mean increase of buffer size by a factor. It's often 2, but might be e.g. 1.5, whatever. The point of using a constant factor is to avoid quadratic time for loops doing appending, i.e. the constant factor size increase yields amortized constant time per append. The sizes that you quote above, on the other hand, look like rather arbitrary buffer size increases where the size to increase by is limited to a certain small range. With copying or moving of everything at each buffer size increase that would not yield amortized constant time. It yield linear time, and quadratic time for a loop doing appends. But, anyway, if 'list' already uses a doubling buffer then the only overhead from the pop optimization would, AFAICS, be a single add in every indexing. On the third & gripping hand, however, a proof-of-concept actual implementation (patch) would be needed to ensure that one doesn't overlook any showstopper or serious problem, and to provide timings. And the special case would have to be documented as a special case. Hm, it would be nice if the Python docs offered complexity (time) guarantees in general... Cheers, - Alf From bearophileHUGS at lycos.com Sat Jan 23 04:55:07 2010 From: bearophileHUGS at lycos.com (Bearophile) Date: Sat, 23 Jan 2010 01:55:07 -0800 (PST) Subject: Efficient Running Median References: <497af344-31b5-4d1a-9b1a-c3d82feb31d3@j5g2000yqm.googlegroups.com> Message-ID: <6ba43727-4b13-45cd-aa0d-3831d2281721@f12g2000yqn.googlegroups.com> Very nice. I have added a comment at the bottom, using a circular queue instead of a deque may be faster. Bye, bearophile From showell30 at yahoo.com Sat Jan 23 05:37:56 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sat, 23 Jan 2010 02:37:56 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: On Jan 23, 1:24?am, Terry Reedy wrote: > On 1/23/2010 3:23 AM, Steve Howell wrote: > > > On Jan 23, 12:13 am, Terry Reedy ?wrote: > > >> Challenge yes, mock no. > > >> Part of writing good basic data structures is not adding needless > >> complication from featuritis and not penalizing 99.99% of access to > >> satify a .01% need better satisfied another way. > > > I would like to challenge your assertion that advancing ob_item > > instead of doing memmove during list_ass_slice would impact the > > performance of list accesses in any way. ?It would only slow down > > operations that add/insert items into the list by, and then only by a > > single conditional statement, and those add/insert operations are > > already O(N) to begin with. > > If you try writing a full patch, as I believe someone did, or at least a > prototype thereof, when the idea was discussed, you might have a better > idea of what the tradeoffs are and why it was rejected. > > For instance, when you append to a full list, it is resized. I believe > it is now doubled, but the policy has varied over the years. If you turn > list from essentially a stack to a deque, you complicate the resizing > policy and have to consider the addition of a shift policy. Do you split > the over-allocated fore and aft or increase the overallocation from > double to, say, triple? If the former, then for normal usage that never > uses the fore part, the over-allocation has been effectively reduced > from 2x to 1.5x (which is about what it once was, I believe). This means > more frequent resizings and copyings, which means slower operation for > most use cases. Adding extra usually wasted space is also a nuisance. > It looks like most of the complication would be in list_resize. I'm gonna oversimplify a bit, but tell me if this is the gist of it. I would have ob_item continue to always refer to first element of the list, and then I'd have to introduce another variable to refer to the start of our allocated memory, ob_start_memory, whenever you do a realloc/free/malloc. I'd have a notion of fore_wastage, which would either be a variable I maintain or something that I just calculate as needed from the difference of ob_item and ob_start_memory. In deciding whether I want to give memory back to the memory manager, I just need to adjust my calculations to account for fore and aft wastage to see if it's time to do a shrink, and before shrinking, I do the memmove. On growth, I would just always do a memmove right away if there is fore_wastage, and then do the normal procedure for aft wastage. For the most common scenario of append, append, append, the only penalty is having to skip over fore_wastage logic by checking for fore_wastage == 0 or ob_item == ob_start_memory. For the scenario of several appends followed by several pops, I get the big win of only doing log 2 N memmoves instead of N as I shrink the list down to zero. If I start alternating between pops and appends, it's basically a wash...instead of doing the memmove on the pop, I do it on the next append. If I were to pop the top element and then prepend a new element, to be truly efficient, I'd want to use reserved right away, but for simplicity, I would probably not complicate list_ass_slice and just do the normal resize() dance, which would give me memmove in one direction followed immediately by a memmove in the other direction when I come back to list_ass_slice. (But it would all still be a wash, since I would have to do the same number of memmoves in the current implementation.) A lot of the essential complexity here seems to come from the fact that realloc() isn't a very robust abstraction. It seems to be expensive to tell it you want to shrink, and it also does not have an interface to tell it to give you a little growing room. On the other hand, the code within list_resize() actually provides a nice framework for amortizing memmoves exponentially. From michele.simionato at gmail.com Sat Jan 23 06:00:19 2010 From: michele.simionato at gmail.com (Michele Simionato) Date: Sat, 23 Jan 2010 03:00:19 -0800 (PST) Subject: counting lines of code References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> <87bpgodj95.fsf@benfinney.id.au> <1f166703-fdfc-4f87-b336-ef80e492635b@a32g2000yqm.googlegroups.com> <7e09df6a-cda1-480e-a971-8f8a70ac4118@b9g2000yqd.googlegroups.com> <980775cc-9101-4673-af03-fc17914d63f8@c4g2000yqa.googlegroups.com> <0d54eff2-59f0-4d95-aafb-478c9041c1b9@p8g2000yqb.googlegroups.com> Message-ID: <187b42f9-af0a-40aa-8c2d-65b4c60e274b@c4g2000yqa.googlegroups.com> On Jan 22, 7:51?pm, Phlip wrote: > On Jan 21, 9:00?pm, Michele Simionato > wrote: > > > Just for fun I have run cloc on our trunk: > > > SUM: ? ? ? ? ? ? ? ?8743 ? ?272238 ? ?215871 ? 1470139 x ? 1.84 = > > 2708354.95 > > Nice! > > My favorite version of a cloc system can distinguish test from > production code. That's why I always use executable cloc to measure > the ratio of test to production code (where 1.2:1 is almost > comfortable an 2:1 is sacred). Most of this code base is old, before we started using automatic tests, so tests are not a significant fraction of the code. And in any case I consider tests as code, since you have to maintain them, refactor them, etc. From michele.simionato at gmail.com Sat Jan 23 06:05:54 2010 From: michele.simionato at gmail.com (Michele Simionato) Date: Sat, 23 Jan 2010 03:05:54 -0800 (PST) Subject: counting lines of code References: <2530e939-b575-485f-b1a3-bb127a3baf39@h2g2000yqj.googlegroups.com> <87bpgodj95.fsf@benfinney.id.au> <1f166703-fdfc-4f87-b336-ef80e492635b@a32g2000yqm.googlegroups.com> <7e09df6a-cda1-480e-a971-8f8a70ac4118@b9g2000yqd.googlegroups.com> <%I36n.5962$4p5.2277@newsfe22.iad> Message-ID: <9f15ce5c-8a97-4551-8b83-e1076861b952@y23g2000yqm.googlegroups.com> On Jan 22, 10:30?pm, Steve Holden wrote: > >> Oh, sorry, did I have the wrong opinion? > > > You had a condescending attitude. > > Towards someone who is fairly obviously not a Python neophyte. > > Please don't think we are telling you you can't have any opinion you > like. Just don't expect to get away with it when you are wrong ;-) Come on, we are all friends here! The guy just said > In my experience with Python codebases that big... > > ...how many of those lines are duplicated, and might merge together > into a better design? > > The LOC would go down, too. and it was even partially right. It is certainly right for the part I was working on. If it was good code from the beginning we would not have started this refactoring project, right? Peace, Michele From peloko45 at gmail.com Sat Jan 23 06:20:48 2010 From: peloko45 at gmail.com (Joan Miller) Date: Sat, 23 Jan 2010 03:20:48 -0800 (PST) Subject: ISC License Message-ID: <00eb248d-c9c9-430f-bc83-41ac865c5111@e11g2000yqe.googlegroups.com> There is a license approved by the OSI, the ISC License [1], which should be included in the PyPi classifiers [2]. [1] https://www.isc.org/software/license http://www.opensource.org/licenses/isc-license.txt [2] http://pypi.python.org/pypi?%3Aaction=list_classifiers From malkarouri at gmail.com Sat Jan 23 06:23:17 2010 From: malkarouri at gmail.com (Muhammad Alkarouri) Date: Sat, 23 Jan 2010 03:23:17 -0800 (PST) Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> Message-ID: <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> Thanks everyone, but not on my machine (Python 2.6.1, OS X 10.6) it's not: In [1]: from itertools import count, islice In [2]: from collections import deque In [3]: i1=count() In [4]: def consume1(iterator, n): ...: deque(islice(iterator, n), maxlen=0) ...: ...: In [5]: i2=count() In [6]: def consume2(iterator, n): ...: for _ in islice(iterator, n): pass ...: ...: In [7]: timeit consume1(i1, 10) 1000000 loops, best of 3: 1.63 us per loop In [8]: timeit consume2(i2, 10) 1000000 loops, best of 3: 846 ns per loop Can somebody please test whether it is only my machine or is this reproducible? (Thanks Jan for making me actually carry the test) From Martin.Drautzburg at web.de Sat Jan 23 06:24:44 2010 From: Martin.Drautzburg at web.de (Martin Drautzburg) Date: Sat, 23 Jan 2010 12:24:44 +0100 Subject: A.x vs. A["x"] References: <1718504.hrdDYYapAE@beaureve.gmx.net> Message-ID: <84293188.TWlGDM3umM@beaureve.gmx.net> Terry Reedy wrote: > On 1/22/2010 2:29 PM, Martin Drautzburg wrote: >> This has probably been asekd a million times, but if someone could >> give a short answer anyways I's be most grateful. >> >> What is it that allows one to write A.x? If I have a variable A, >> I know you can do this with classes, but not with plain objects, but >> why is that so? > > You exact meaning here is not clear, but I suspect it is somewhat > incorrect, at least for built-in classes. You're right. I used to think you could do this to classes: G = Strum G.x=1 But not to objects (instances): g = Strum() g.y = 2 But in fact both work. Thanks for clarifying From steve at holdenweb.com Sat Jan 23 06:35:04 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 23 Jan 2010 06:35:04 -0500 Subject: Mastering Python 3 I/O - Special Preview - Feb 5, 2010 (Chicago) In-Reply-To: <7f014ea61001221947o7e17f0ebv9784c427346f6f08@mail.gmail.com> References: <4EB3255C-2B1E-44E5-BE72-D61E44E580B4@dabeaz.com> <4B5A61CE.2030608@holdenweb.com> <7f014ea61001221947o7e17f0ebv9784c427346f6f08@mail.gmail.com> Message-ID: Chris Colbert wrote: > oops :) > Yes, but only a little oops. I think it's probably fairly well-known that David Beazley and I both supply training, and you'd hardly expect profit to be off the list of objectives for US trainers. But my failure to hit "reply to sender only" certainly did show that it's been a long week. Sorry for the noise! regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Sat Jan 23 06:37:38 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 23 Jan 2010 06:37:38 -0500 Subject: Using dictionary key as a regular expression class In-Reply-To: <20100122233844.GJ28576@turki.gavron.org> References: <20100122233844.GJ28576@turki.gavron.org> Message-ID: Chris Jones wrote: > On Fri, Jan 22, 2010 at 05:07:13PM EST, Arnaud Delobelle wrote: > > [..] > >> import codecs >> from collections import defaultdict >> >> tcounters = defaultdict(int) >> f = codecs.open('/home/gavron/git/screen/src/screen.c', 'r', "utf-8") >> >> for c in f.read(): >> tcounters[c] += 1 >> >> for c, n in tcounters.iteritems(): >> print "%r\t%i" % (c, n) > > Ah, yes.. much better - I grew suspicious of my 'effort' when I realized > I could have written a shorter version in C. :-) > Congratulations. That perception shows a sound appreciation of Python's design. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at REMOVE-THIS-cybersource.com.au Sat Jan 23 06:52:43 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 23 Jan 2010 11:52:43 GMT Subject: medians for degree measurements References: <044268c0-c308-406e-8d5b-a38ef5b5e381@p24g2000yqm.googlegroups.com> Message-ID: <036acf91$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 22:09:54 -0800, Steve Howell wrote: > On Jan 22, 5:12?pm, MRAB wrote: >> Steve Howell wrote: >> > I just saw the thread for medians, and it reminded me of a problem >> > that I need to solve. ?We are writing some Python software for >> > sailing, and we need to detect when we've departed from the median >> > heading on the leg. ?Calculating arithmetic medians is >> > straightforward, but compass bearings add a twist. [...] > I like this implementation, and it would probably work 99.9999% of the > time for my particular use case. The only (very contrived) edge case > that I can think of is when you have 10 bearings to SSW, 10 bearings to > SSE, and the two outliers are unfortunately in the NE and NW quadrants. > It seems like the algorithm above would pick one of the outliers. The trouble is that median of angular measurements is not a meaningful concept. The median depends on the values being ordered, but angles can't be sensibly ordered. Which is larger, 1 degree north or 359 degrees? Is the midpoint between them 0 degree or 180 degree? The median of the number line 0 <= x <= 360 is 180, but what's the median of the circle 0 deg <= theta <= 360 deg? With no end points, it's meaningless to talk about the middle. For this reason, I expect that taking the median of bearing measurements isn't well defined. You can probably get away with it so long as the bearings are "close", where "close" itself is ill-defined. In any case, this isn't a programming problem, it's a mathematics problem. I think you're better off taking it to a maths or statistics forum, and see what they say. -- Steven From steve at REMOVE-THIS-cybersource.com.au Sat Jan 23 07:12:29 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 23 Jan 2010 12:12:29 GMT Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> Message-ID: <036ad433$0$1357$c3e8da3@news.astraweb.com> On Fri, 22 Jan 2010 21:42:43 -0800, Steve Howell wrote: > This innocent program here literally moves about a million bytes of > memory around for no good reason: > > lst = [] > for i in range(2000): > lst.append(i) > while lst: > print lst.pop(0) > > Why? Because list.pop(0) is implemented in O(N) instead of O(1). > > Why wouldn't you get a competent C programmer simply make list_ass_slice > smart enough to make list.pop(0) O(1)? Because there are always trade-offs, and the competent C programmers who coded the implementation for lists choose different tradeoffs to the ones you would prefer. Seems to me that the simple solution to your problem is for you to implement your own data structure that makes whatever trade-offs you like. If it is good enough and popular enough, it might even replace the existing list implementation. >> That is, you give me the impression that you prefer this: >> >> while alist: >> ? ? x = alist.pop(0) >> ? ? process(x) >> >> over this: >> >> for x in alist: >> ? ? process(x) >> # allow alist to be garbage collected when it goes out of scope >> >> > No, to be more precise, I prefer this implementation of a recursive > parser (using lists) to one that would have to use deque's because of > the lameness of Python's list implementation: > > https://bitbucket.org/showell/shpaml_website/src/tip/shpaml.py That's a lot of code. Am I supposed to study the whole module, or can you give me a hint as to what you're referring to? The lack of docstrings and comments doesn't fill me with enthusiasm for reading it. Nevertheless, on the basis of a quick scan, I suppose that you're probably talking about the nested function called recurse: def recurse(prefix_lines): while prefix_lines: prefix, line = prefix_lines[0] if line == '': prefix_lines.pop(0) append('') else: block_size = get_block(prefix_lines) if block_size == 1: prefix_lines.pop(0) if line == pass_syntax: pass elif line.startswith(flush_left_syntax): append(line[len(flush_left_syntax):]) elif line.startswith(flush_left_empty_line): append('') else: append(prefix + leaf_method(line)) else: block = prefix_lines[:block_size] prefix_lines = prefix_lines[block_size:] branch_method(output, block, recurse) return Since you're not even looking at the results of the pop, why don't you just call del prefix_lines[0]? It probably won't perform any better, but it is more idiomatic. An alternative would be to do exactly what you want lists to do: track the start of the list. Untested: def recurse(prefix_lines): start = 0 end = len(prefix_lines) while start < end: prefix, line = prefix_lines[start] if line == '': start += 1 append('') else: block_size = get_block(prefix_lines) if block_size == 1: start += 1 if line == pass_syntax: pass elif line.startswith(flush_left_syntax): append(line[len(flush_left_syntax):]) elif line.startswith(flush_left_empty_line): append('') else: append(prefix + leaf_method(line)) else: block = prefix_lines[:block_size] start = block_size branch_method(output, block, recurse) return No more O(N) deletions. Problem solved. -- Steven From arnodel at googlemail.com Sat Jan 23 07:24:32 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sat, 23 Jan 2010 12:24:32 +0000 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> Message-ID: Dave Angel writes: > Arnaud Delobelle wrote: >> Steve Howell writes: >> >> >>> On Jan 22, 12:14 pm, Chris Rebert wrote: >>> >> I made the comment you quoted. In CPython, it is O(n) to delete/insert >> an element at the start of the list - I know it because I looked at the >> implementation a while ago. This is why collections.deque exists I >> guess. I don't know how they are implemented but insertion/deletion at >> either end are O(1) and so is random access. So they are the data >> structure that you want. >> >> > Not according to the 2.6 docs. > > Indexed access is O(1) at both ends but slows to O(n) in the > middle. For fast random access, use lists instead. Yes you are correct. This will teach me (again!) to check my facts. > > That sounds to me like a doubly-linked list implementation. I've just looked and it is a doubly-linked list of 'blocks' of size BLOCKLEN, which is 62 on the source I have (I guess it's 62 because then the whole block structure is 64 exactly, 62 + 1 for each link). So a small list will have near constant random access, in a way. -- Arnaud From __peter__ at web.de Sat Jan 23 07:45:43 2010 From: __peter__ at web.de (Peter Otten) Date: Sat, 23 Jan 2010 13:45:43 +0100 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> Message-ID: Muhammad Alkarouri wrote: > Thanks everyone, but not on my machine (Python 2.6.1, OS X 10.6) it's > not: > > > In [1]: from itertools import count, islice > > In [2]: from collections import deque > > In [3]: i1=count() > > In [4]: def consume1(iterator, n): > ...: deque(islice(iterator, n), maxlen=0) > ...: > ...: > > In [5]: i2=count() > > In [6]: def consume2(iterator, n): > ...: for _ in islice(iterator, n): pass > ...: > ...: > > In [7]: timeit consume1(i1, 10) > 1000000 loops, best of 3: 1.63 us per loop > > In [8]: timeit consume2(i2, 10) > 1000000 loops, best of 3: 846 ns per loop > > > Can somebody please test whether it is only my machine or is this > reproducible? I can reproduce it. The deque-based approach has a bigger constant overhead but better per-item performance. Its asymptotical behaviour is therefore better. $ python consume_timeit.py consume_deque 10: 1.77500414848 100: 3.73333001137 1000: 24.7235469818 consume_forloop 10: 1.22008490562 100: 5.86271500587 1000: 52.2449371815 consume_islice 10: 0.897439956665 100: 1.51542806625 1000: 7.70061397552 $ cat consume_timeit.py from collections import deque from itertools import islice, repeat def consume_deque(n, items): deque(islice(items, n), maxlen=0) def consume_forloop(n, items): for _ in islice(items, n): pass def consume_islice(n, items): next(islice(items, n-1, None), None) def check(fs): for consume in fs: items = iter(range(10)) consume(3, items) rest = list(items) assert rest == range(3, 10), consume.__name__ if __name__ == "__main__": fs = consume_deque, consume_forloop, consume_islice check(fs) items = repeat(None) from timeit import Timer for consume in fs: print consume.__name__ for n in (10, 100, 1000): print "%6d:" % n, print Timer("consume(%s, items)" % n, "from __main__ import consume, items").timeit() print $ With next(islice(...), None) I seem to have found a variant that beats both competitors. Peter From duncan.booth at invalid.invalid Sat Jan 23 07:58:44 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 23 Jan 2010 12:58:44 GMT Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> Message-ID: Peter Otten <__peter__ at web.de> wrote: > With next(islice(...), None) I seem to have found a variant that beats > both competitors. > It has different behaviour for n==0 but I'm sure that's easily fixed. From __peter__ at web.de Sat Jan 23 08:13:37 2010 From: __peter__ at web.de (Peter Otten) Date: Sat, 23 Jan 2010 14:13:37 +0100 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> Message-ID: Duncan Booth wrote: > Peter Otten <__peter__ at web.de> wrote: > >> With next(islice(...), None) I seem to have found a variant that beats >> both competitors. >> > It has different behaviour for n==0 but I'm sure that's easily fixed. "Different behaviour" being a euphemism for broken ;) def consume_islice(n, items): if n == 0: return next(islice(items, n-1, None), None) Peter From malkarouri at gmail.com Sat Jan 23 08:14:34 2010 From: malkarouri at gmail.com (Muhammad Alkarouri) Date: Sat, 23 Jan 2010 05:14:34 -0800 (PST) Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> Message-ID: <06153e39-7aa8-416b-93bb-32001e0cb81b@e37g2000yqn.googlegroups.com> On 23 Jan, 12:45, Peter Otten <__pete... at web.de> wrote: > Muhammad Alkarouri wrote: > > Thanks everyone, but not on my machine (Python 2.6.1, OS X 10.6) it's > > not: > > > In [1]: from itertools import count, islice > > > In [2]: from collections import deque > > > In [3]: i1=count() > > > In [4]: def consume1(iterator, n): > > ? ?...: ? ? deque(islice(iterator, n), maxlen=0) > > ? ?...: > > ? ?...: > > > In [5]: i2=count() > > > In [6]: def consume2(iterator, n): > > ? ?...: ? ? for _ in islice(iterator, n): pass > > ? ?...: > > ? ?...: > > > In [7]: timeit consume1(i1, 10) > > 1000000 loops, best of 3: 1.63 us per loop > > > In [8]: timeit consume2(i2, 10) > > 1000000 loops, best of 3: 846 ns per loop > > > Can somebody please test whether it is only my machine or is this > > reproducible? > > I can reproduce it. The deque-based approach has a bigger constant overhead > but better per-item performance. Its asymptotical behaviour is therefore > better. > > $ python consume_timeit.py > consume_deque > ? ? 10: 1.77500414848 > ? ?100: 3.73333001137 > ? 1000: 24.7235469818 > > consume_forloop > ? ? 10: 1.22008490562 > ? ?100: 5.86271500587 > ? 1000: 52.2449371815 > > consume_islice > ? ? 10: 0.897439956665 > ? ?100: 1.51542806625 > ? 1000: 7.70061397552 > > $ cat consume_timeit.py > from collections import deque > from itertools import islice, repeat > > def consume_deque(n, items): > ? ? deque(islice(items, n), maxlen=0) > > def consume_forloop(n, items): > ? ? for _ in islice(items, n): > ? ? ? ? pass > > def consume_islice(n, items): > ? ? next(islice(items, n-1, None), None) > > def check(fs): > ? ? for consume in fs: > ? ? ? ? items = iter(range(10)) > ? ? ? ? consume(3, items) > ? ? ? ? rest = list(items) > ? ? ? ? assert rest == range(3, 10), consume.__name__ > > if __name__ == "__main__": > ? ? fs = consume_deque, consume_forloop, consume_islice > ? ? check(fs) > > ? ? items = repeat(None) > > ? ? from timeit import Timer > ? ? for consume in fs: > ? ? ? ? print consume.__name__ > ? ? ? ? for n in (10, 100, 1000): > ? ? ? ? ? ? print "%6d:" % n, > ? ? ? ? ? ? print Timer("consume(%s, items)" % n, > ? ? ? ? ? ? ? ? ? ? ? ? "from __main__ import consume, items").timeit() > ? ? ? ? print > $ > > With next(islice(...), None) I seem to have found a variant that beats both ? > competitors. > > Peter Thanks Peter, I got more or less the same result on my machine (Python 2.6.1, x86_64, OS X 10.6): ~/tmp> python consume_timeit.py consume_deque 10: 1.3138859272 100: 3.54495286942 1000: 24.9603481293 consume_forloop 10: 0.658113002777 100: 2.85697007179 1000: 24.6610429287 consume_islice 10: 0.637741088867 100: 1.09042882919 1000: 5.44473600388 The next function performs much better. It is also much more direct for the purposes of consume and much more understandable (at least for me) as it doesn't require a specialized data structure which is subsequently not used as such. I am thus inclined to report it as a python documentation enhancement (bug) request. Any comments? Cheers, Muhammad From __peter__ at web.de Sat Jan 23 08:15:25 2010 From: __peter__ at web.de (Peter Otten) Date: Sat, 23 Jan 2010 14:15:25 +0100 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> Message-ID: Duncan Booth wrote: > Peter Otten <__peter__ at web.de> wrote: > >> With next(islice(...), None) I seem to have found a variant that beats >> both competitors. >> > It has different behaviour for n==0 but I'm sure that's easily fixed. "Different behaviour" being a euphemism for broken ;) def consume_islice(n, items): if n == 0: return next(islice(items, n-1, None), None) Peter From __peter__ at web.de Sat Jan 23 08:32:37 2010 From: __peter__ at web.de (Peter Otten) Date: Sat, 23 Jan 2010 14:32:37 +0100 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <06153e39-7aa8-416b-93bb-32001e0cb81b@e37g2000yqn.googlegroups.com> Message-ID: Muhammad Alkarouri wrote: > The next function performs much better. It is also much more direct > for the purposes of consume and much more understandable (at least for > me) as it doesn't require a specialized data structure which is > subsequently not used as such. > I am thus inclined to report it as a python documentation enhancement > (bug) request. Any comments? I would support that as a the deque(..., maxlen=0) trick is a bit too clever for my taste. Peter PS: Remember to include the bug fix for n=0 if you proceed. From __peter__ at web.de Sat Jan 23 08:46:16 2010 From: __peter__ at web.de (Peter Otten) Date: Sat, 23 Jan 2010 14:46:16 +0100 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> Message-ID: Peter Otten wrote: > Duncan Booth wrote: > >> Peter Otten <__peter__ at web.de> wrote: >> >>> With next(islice(...), None) I seem to have found a variant that beats >>> both competitors. >>> >> It has different behaviour for n==0 but I'm sure that's easily fixed. > > "Different behaviour" being a euphemism for broken ;) > > def consume_islice(n, items): > if n == 0: > return > next(islice(items, n-1, None), None) Even better: def consume_islice(n, items): next(islice(items, n, n), None) Peter From lists at cheimes.de Sat Jan 23 08:46:37 2010 From: lists at cheimes.de (Christian Heimes) Date: Sat, 23 Jan 2010 14:46:37 +0100 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <2ce05a04-ebc9-4f26-ba32-ff787925a564@c29g2000yqd.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> <2ce05a04-ebc9-4f26-ba32-ff787925a564@c29g2000yqd.googlegroups.com> Message-ID: <4B5AFDBD.3090200@cheimes.de> Steve Howell wrote: > Another benchmark is that deques are slower than lists for accessing > elements. deques are optimized for accessing, inserting and removing data from both ends. For anything else it's slower than the list type. The fact was explained in this very thread yesterday. Christian From malkarouri at gmail.com Sat Jan 23 08:55:12 2010 From: malkarouri at gmail.com (Muhammad Alkarouri) Date: Sat, 23 Jan 2010 05:55:12 -0800 (PST) Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <06153e39-7aa8-416b-93bb-32001e0cb81b@e37g2000yqn.googlegroups.com> Message-ID: <84254f72-2bf0-4157-990a-9880115dac0d@q4g2000yqm.googlegroups.com> On 23 Jan, 13:32, Peter Otten <__pete... at web.de> wrote: > Muhammad Alkarouri wrote: > > The next function performs much better. It is also much more direct > > for the purposes of consume and much more understandable (at least for > > me) as it doesn't require a specialized data structure which is > > subsequently not used as such. > > I am thus inclined to report it as a python documentation enhancement > > (bug) request. Any comments? > > I would support that as a the deque(..., maxlen=0) trick is a bit too clever > for my taste. > > Peter > > PS: Remember to include the bug fix for n=0 if you proceed. Done. http://bugs.python.org/issue7764 Thanks for all the effort. Muhammad From malkarouri at gmail.com Sat Jan 23 09:06:32 2010 From: malkarouri at gmail.com (Muhammad Alkarouri) Date: Sat, 23 Jan 2010 06:06:32 -0800 (PST) Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> Message-ID: <98b43693-3ba2-4d0d-b009-739d841a1c85@36g2000yqu.googlegroups.com> On 23 Jan, 13:46, Peter Otten <__pete... at web.de> wrote: > Peter Otten wrote: > > Duncan Booth wrote: > > >> Peter Otten <__pete... at web.de> wrote: > > >>> With next(islice(...), None) I seem to have found a variant that beats > >>> both ?competitors. > > >> It has different behaviour for n==0 but I'm sure that's easily fixed. > > > "Different behaviour" being a euphemism for broken ;) > > > def consume_islice(n, items): > > ? ? if n == 0: > > ? ? ? ? return > > ? ? next(islice(items, n-1, None), None) > > Even better: > > def consume_islice(n, items): > ? ? next(islice(items, n, n), None) > > Peter I submitted the bug report before considering this alternative, which is better. I may add this later in the day, but I have to wait a little as it seems you are going to optimize/improve the function almost out of existence:) If you are happy with this one, and can add the comment on the issue (http://bugs.python.org/issue7764) yourself, please do so. What I can say is, I am definitely very happy that I asked the question. You live and learn:) Cheers, Muhammad From __peter__ at web.de Sat Jan 23 09:19:56 2010 From: __peter__ at web.de (Peter Otten) Date: Sat, 23 Jan 2010 15:19:56 +0100 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <98b43693-3ba2-4d0d-b009-739d841a1c85@36g2000yqu.googlegroups.com> Message-ID: Muhammad Alkarouri wrote: > On 23 Jan, 13:46, Peter Otten <__pete... at web.de> wrote: >> def consume_islice(n, items): >> next(islice(items, n, n), None) > I submitted the bug report before considering this alternative, which > is better. I may add this later in the day, but I have to wait a > little as it seems you are going to optimize/improve the function > almost out of existence:) One problem: the above function doesn't consume the entire iterator like the original example does for n=None. Passing sys.maxint instead is not pretty. Peter From roy at panix.com Sat Jan 23 09:40:56 2010 From: roy at panix.com (Roy Smith) Date: Sat, 23 Jan 2010 09:40:56 -0500 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> Message-ID: In article , Steve Howell wrote: > This innocent program here literally moves about a million bytes of > memory around for no good reason: > > lst = [] > for i in range(2000): > lst.append(i) > while lst: > print lst.pop(0) > > Why? Because list.pop(0) is implemented in O(N) instead of O(1). I think you're being a little pedantic here. Yes, it is true that pop(0) is O(n), and that if you put an O(n) operation in a loop, you get O(n^2) run time. The problem is that while it is well-known that putting something that's O(n) in a loop gets you O(n^2), it's not well known that pop(0) for a Python list is O(n). This is where you and I apparently start to differ in what we think about this. You are arguing that this is a bug in the implementation of list. While I suppose there's some validity to that argument, I disagree. What I would argue (and have done so several times over the years, with little success) is that this is a bug in the documentation! I'm looking at http://tinyurl.com/cdbwog. It shows all the operations of a list. What it does not show is their cost. For pop(), it has a note: "The pop() method is only supported by the list and array types. The optional argument i defaults to -1, so that by default the last item is removed and returned". There's nothing there that gives any hint that pop(0) is any more expensive than pop(-1). That is "secret knowledge", which you only get by following the newsgroup discussions or looking at the implementation. You shouldn't have to do either. There's lots of possible ways list could be implemented. Without knowing the details, I'm left to guess about important stuff like the cost of operations. Every one of these operations should list the cost. Even if it's something as vague as, "While not guaranteed by the language spec, in the current implemenation of CPython ...". From rdv at roalddevries.nl Sat Jan 23 09:44:07 2010 From: rdv at roalddevries.nl (Roald de Vries) Date: Sat, 23 Jan 2010 15:44:07 +0100 Subject: enumerated while loop Message-ID: <227C7474-8CD1-4BC8-BDC3-ADBFAE01D052@roalddevries.nl> Dear all, I sometimes want to use an infinite while loop with access to the loop index, like this: def naturals(): i = 0 while True: yield i y += 1 for i in naturals(): print(i) I assume a function like 'naturals' already exists, or a similar construction for the same purpose. But what is it called? Kind regards, Roald From deets at nospam.web.de Sat Jan 23 09:49:23 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Sat, 23 Jan 2010 15:49:23 +0100 Subject: enumerated while loop In-Reply-To: References: Message-ID: <7s0gjjFehnU1@mid.uni-berlin.de> Am 23.01.10 15:44, schrieb Roald de Vries: > Dear all, > > I sometimes want to use an infinite while loop with access to the loop > index, like this: > > def naturals(): > i = 0 > while True: > yield i > y += 1 > > for i in naturals(): > print(i) > > I assume a function like 'naturals' already exists, or a similar > construction for the same purpose. But what is it called? for i, item in enumerate(iterable): .... Diez From invalid at invalid.invalid Sat Jan 23 09:50:29 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Sat, 23 Jan 2010 14:50:29 +0000 (UTC) Subject: enumerated while loop References: Message-ID: On 2010-01-23, Roald de Vries wrote: > Dear all, > > I sometimes want to use an infinite while loop with access to the loop > index, like this: > > def naturals(): > i = 0 > while True: > yield i > y += 1 > > for i in naturals(): > print(i) > > I assume a function like 'naturals' already exists, or a similar > construction for the same purpose. But what is it called? xrange(), except that you need to provde an upper limit. -- Grant From arnodel at googlemail.com Sat Jan 23 09:57:00 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sat, 23 Jan 2010 14:57:00 +0000 Subject: medians for degree measurements References: Message-ID: Steve Howell writes: > I just saw the thread for medians, and it reminded me of a problem > that I need to solve. We are writing some Python software for > sailing, and we need to detect when we've departed from the median > heading on the leg. Calculating arithmetic medians is > straightforward, but compass bearings add a twist. > > The numerical median of 1, 2, 3, 4, 5, 6, 359 is 4. But for > navigational purposes you would actually order the numbers 359, 1, 2, > 3, 4, 5, 6, so the desired median heading of the boat is actually 3. > > Of course, you could express 359 better as -1 degrees to north, then > the sequence would be -1, 1, 2, 3, 4, 5, and 6. And you'd be good. > > But that trick does not generalize if you go south instead, as you > have similar issues with -179, 174, 175, 176, 177, 178, and 179. > > Has anybody solved this in Python, either for compass bearings or a > different domain? I can think of kind of a brute force solution where > you keep rotating the sequence until the endpoints are closest > together mod 360, but I wonder if there is something more elegant. Here's a simple (too simple?) way to do it: 1. sort the bearings in ascending order 2. find the largest gap between two consecutive bearings 3. cut the circle at this point and now find the median the normal way In Python: >>> def median_bearing(bearings): ... bearings = sorted(bearings) ... n = len(bearings) ... i = max(xrange(n), key=lambda i: (bearings[(i+1)%n] - bearings[i])%360) ... return bearings[(i + (n+1)//2)%n] ... >>> median_bearing([1,2,3,4,5,6,359]) 3 >>> median_bearing([-179, 174, 175, 176, 177, 178, 179]) 177 I guess there may be cases where the results that it gives are still not very good, as in general there isn't a good notion of median for cyclic data. E.g. what would be the median of [0, 180] - it could equally be 090 or 270. Or the median of [0, 120, 240] has the same problem. But I suppose your data couldn't be like this as then it would be ill-advised to try to apply a notion of median to it. But it will work well I believe with quite localized data set with a few, even wildly inaccurate, outliers. E.g. >>> median_bearing([123, 124, 125, 125, 126, 10, 240]) 125 HTH -- Arnaud From roy at panix.com Sat Jan 23 09:57:04 2010 From: roy at panix.com (Roy Smith) Date: Sat, 23 Jan 2010 09:57:04 -0500 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: In article , "Alf P. Steinbach" wrote: > But it would IMHO have been better if it wasn't called "list", which brings > in the wrong associations for someone used to other languages. +1. When I first started using Python (back in the 1.4 days), I assumed a list was a singly-linked list. Which, of course, leads to the assumption that pop(0) is O(1), and lots of other wrong thinking(*). Oddly enough, I was going to write in the above paragraph, "like a C++ STL list", until I happened to glance at the STL docs and refreshed my memory that an STL list is doubly-linked. Which just goes to show that making assumptions based on names is a bad idea. So, we're right back to my statement earlier in this thread that the docs are deficient in that they describe behavior with no hint about cost. Given that, it should be no surprise that users make incorrect assumptions about cost. (*) I suspect somebody is going to point out that list.pop was added in some version later than 1.4, but that's a detail. From dickinsm at gmail.com Sat Jan 23 09:58:09 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 23 Jan 2010 06:58:09 -0800 (PST) Subject: enumerated while loop References: Message-ID: On Jan 23, 2:44?pm, Roald de Vries wrote: > I assume a function like 'naturals' already exists, or a similar ? > construction for the same purpose. But what is it called? itertools.count() -- Mark From zuo at chopin.edu.pl Sat Jan 23 10:04:52 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sat, 23 Jan 2010 16:04:52 +0100 Subject: enumerated while loop In-Reply-To: <7s0gjjFehnU1@mid.uni-berlin.de> References: <7s0gjjFehnU1@mid.uni-berlin.de> Message-ID: 23-01-2010 o 15:49:23 Diez B. Roggisch wrote: > Am 23.01.10 15:44, schrieb Roald de Vries: >> Dear all, >> >> I sometimes want to use an infinite while loop with access to the loop >> index, like this: >> >> def naturals(): >> i = 0 >> while True: >> yield i >> y += 1 >> >> for i in naturals(): >> print(i) >> >> I assume a function like 'naturals' already exists, or a similar >> construction for the same purpose. But what is it called? itertools.count() -- see http://docs.python.org/library/itertools.html#itertools.count > for i, item in enumerate(iterable): > .... That's completely beside the point. Regards, *j From rdv at roalddevries.nl Sat Jan 23 10:06:16 2010 From: rdv at roalddevries.nl (Roald de Vries) Date: Sat, 23 Jan 2010 16:06:16 +0100 Subject: enumerated while loop In-Reply-To: References: Message-ID: <38B9A1E5-3775-463F-86DB-3535C7D041F9@roalddevries.nl> On Jan 23, 2010, at 3:50 PM, Grant Edwards wrote: > On 2010-01-23, Roald de Vries wrote: >> Dear all, >> >> I sometimes want to use an infinite while loop with access to the >> loop >> index, like this: >> >> def naturals(): >> i = 0 >> while True: >> yield i >> y += 1 >> >> for i in naturals(): >> print(i) >> >> I assume a function like 'naturals' already exists, or a similar >> construction for the same purpose. But what is it called? > > xrange(), except that you need to provde an upper limit. I'm sorry, Grant, but you exactly miss my point ;-). The essence of this is that it's endless. From rdv at roalddevries.nl Sat Jan 23 10:07:51 2010 From: rdv at roalddevries.nl (Roald de Vries) Date: Sat, 23 Jan 2010 16:07:51 +0100 Subject: enumerated while loop In-Reply-To: <7s0gjjFehnU1@mid.uni-berlin.de> References: <7s0gjjFehnU1@mid.uni-berlin.de> Message-ID: <750D5C07-1230-4950-81A2-45A7430CDF7F@roalddevries.nl> On Jan 23, 2010, at 3:49 PM, Diez B. Roggisch wrote: > Am 23.01.10 15:44, schrieb Roald de Vries: >> Dear all, >> >> I sometimes want to use an infinite while loop with access to the >> loop >> index, like this: >> >> def naturals(): >> i = 0 >> while True: >> yield i >> y += 1 >> >> for i in naturals(): >> print(i) >> >> I assume a function like 'naturals' already exists, or a similar >> construction for the same purpose. But what is it called? > > for i, item in enumerate(iterable): > .... This only moves the problem to the variable 'iterable'. And moreover is more complex than needed; it give '(i, item)', whereas I only need the index 'i'. From ceciliaseidel at gmx.de Sat Jan 23 10:13:20 2010 From: ceciliaseidel at gmx.de (ceciliaseidel at gmx.de) Date: Sat, 23 Jan 2010 16:13:20 +0100 Subject: iterating lists Message-ID: <4B5B1210.2060300@gmx.de> As you were talking about list.pop()... Is anyone able to reproduce the following and explain why this happens by chance? (Using 3.1.1) l1 = ["ready", "steady", "go"] l2 = ["one", "two", "tree"] l3 = ["lift off"] for w in l1: print(l1.pop()) #prints only "go steady" - why not "ready"?? for w in range(len(l2)): print(l2.pop()) #prints "three two one" as expected for w in l3: print(l3.pop()) #prints "lift off" - inconsistent to first case... At least for 2.2.3 I found the first way to iterate the list as default, I guess it is deprecated now but still what happens seems weird to me... From arnodel at googlemail.com Sat Jan 23 10:19:06 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sat, 23 Jan 2010 15:19:06 +0000 Subject: iterating lists References: Message-ID: ceciliaseidel at gmx.de writes: > As you were talking about list.pop()... > > Is anyone able to reproduce the following and explain why this happens > by chance? (Using 3.1.1) > > l1 = ["ready", "steady", "go"] > l2 = ["one", "two", "tree"] > l3 = ["lift off"] > > for w in l1: > print(l1.pop()) #prints only "go steady" - why not "ready"?? > I suggest you simulate the loop above using pen and paper, writing the value of w and the value of l1 at each iteration. The behaviour you are observing should then be clearly explained. And you should also realise that it's a really bad idea to mutate a list that you are iterating on! HTH -- Arnaud From rdv at roalddevries.nl Sat Jan 23 10:30:30 2010 From: rdv at roalddevries.nl (Roald de Vries) Date: Sat, 23 Jan 2010 16:30:30 +0100 Subject: enumerated while loop In-Reply-To: References: Message-ID: On Jan 23, 2010, at 3:58 PM, Mark Dickinson wrote: > On Jan 23, 2:44 pm, Roald de Vries wrote: >> I assume a function like 'naturals' already exists, or a similar >> construction for the same purpose. But what is it called? > > itertools.count() On Jan 23, 2010, at 4:04 PM, Jan Kaliszewski wrote: > itertools.count() -- see http://docs.python.org/library/itertools.html#itertools.count > >> for i, item in enumerate(iterable): >> .... > > That's completely beside the point. Thanks guys From alfps at start.no Sat Jan 23 10:45:30 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sat, 23 Jan 2010 16:45:30 +0100 Subject: iterating lists In-Reply-To: References: Message-ID: * ceciliaseidel at gmx.de: > As you were talking about list.pop()... > > Is anyone able to reproduce the following and explain why this happens > by chance? (Using 3.1.1) > > l1 = ["ready", "steady", "go"] > l2 = ["one", "two", "tree"] > l3 = ["lift off"] > > for w in l1: > print(l1.pop()) #prints only "go steady" - why not "ready"?? > > for w in range(len(l2)): > print(l2.pop()) #prints "three two one" as expected > for w in l3: > print(l3.pop()) #prints "lift off" - inconsistent to first case... > > > At least for 2.2.3 I found the first way to iterate the list as default, > I guess it is deprecated now but still what happens seems weird to me... Arnaud Delobelle has already answered your question, but you might alternatively try this angle: l1 = ["ready", "steady", "go"] l2 = ["one", "two", "tree"] l3 = ["lift off"] for w in l1: print( w, l1 ) print(l1.pop()) #prints only "go steady" - why not "ready"?? for w in range(len(l2)): print(l2.pop()) #prints "three two one" as expected for w in l3: print( w, l3 ) print(l3.pop()) #prints "lift off" - inconsistent to first case... If the list has at least one item you always get into the first iteration of the loop. I.e. there's no inconsistency, unless you count the lack of an exception as an inconsistency. I don't know whether the behavior is clearly defined or not; there is a possibility that it might be well-defined. Cheers & hth., - Alf From thinke365 at gmail.com Sat Jan 23 10:49:46 2010 From: thinke365 at gmail.com (thinke365) Date: Sat, 23 Jan 2010 07:49:46 -0800 (PST) Subject: how can i know if a python object have a attribute such as 'attr1'? Message-ID: <27286937.post@talk.nabble.com> for example, i may define a python class: class A: def sayHello(): print 'hello' a = A() a.attr1 = 'hello' a.attr2 = 'bb' b = A() a.attr2 = 'aa' how can i know whether an object have an attribute named attr1? -- View this message in context: http://old.nabble.com/how-can-i-know-if-a-python-object-have-a-attribute-such-as-%27attr1%27--tp27286937p27286937.html Sent from the Python - python-list mailing list archive at Nabble.com. From steve at REMOVE-THIS-cybersource.com.au Sat Jan 23 10:54:26 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 23 Jan 2010 15:54:26 GMT Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> Message-ID: <036b0836$0$1357$c3e8da3@news.astraweb.com> On Sat, 23 Jan 2010 09:57:04 -0500, Roy Smith wrote: > In article , > "Alf P. Steinbach" wrote: > >> But it would IMHO have been better if it wasn't called "list", which >> brings in the wrong associations for someone used to other languages. > > +1. > > When I first started using Python (back in the 1.4 days), I assumed a > list was a singly-linked list. Why would you do that? I can think of at least eight different implementations of the abstract list data structure: constant-size array variable-size array variable-size array with amortised O(1) appends singly-linked list singly-linked list with CDR coding doubly-linked list skip list indexable skip list One can reasonably disregard constant-sized arrays as a possibility, given that Python lists aren't fixed size (pity the poor Pascal and Fortran coders who are stuck with static arrays!), but the rest are all reasonable possibilities. Why assume one specific implementation in the absence of documentation promising certain performance characteristics? > Oddly enough, I was going to write in the above paragraph, "like a C++ > STL list", until I happened to glance at the STL docs and refreshed my > memory that an STL list is doubly-linked. Which just goes to show that > making assumptions based on names is a bad idea. Exactly :) > So, we're right back to my statement earlier in this thread that the > docs are deficient in that they describe behavior with no hint about > cost. Given that, it should be no surprise that users make incorrect > assumptions about cost. There are quite a few problems with having the documentation specify cost: (1) Who is going to do it? Any volunteers? (2) Big-oh notation can be misleading, especially for naive users, or those whose intuition for what's fast has been shaped by other languages. Big-oh doesn't tell you whether something is fast or slow, only how it scales -- and sometimes not even then. (3) Having documented a particular performance, that discourages implementation changes. Any would-be patch or new implementation not only has to consider that the functional behaviour doesn't change, but that the performance doesn't either. In practice the Python developers are unlikely to make an implementation change which leads to radically worse performance, particularly for critical types like list and dict. But in other cases, they might choose to change big-oh behaviour, and not wish to be tied down by documentation of the cost of operations. (4) How much detail is necessary? What about degenerate cases? E.g. dict lookup in CPython is typically O(1) amortised, but if all the keys hash to the same value, it falls to O(N). (5) Should the language guarantee such degenerate behaviour? Who decides which costs are guaranteed and which are not? (6) Such performance guarantees should be implementation specific, not language specific. CPython is only one implementation of the language out of many. -- Steven From arnodel at googlemail.com Sat Jan 23 10:56:29 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sat, 23 Jan 2010 15:56:29 +0000 Subject: how can i know if a python object have a attribute such as 'attr1'? References: Message-ID: thinke365 writes: > for example, i may define a python class: > class A: > def sayHello(): > print 'hello' > > a = A() > a.attr1 = 'hello' > a.attr2 = 'bb' > > b = A() > a.attr2 = 'aa' > > how can i know whether an object have an attribute named attr1? hasattr(a, 'attr1') -- Arnaud From steve at holdenweb.com Sat Jan 23 11:05:05 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 23 Jan 2010 11:05:05 -0500 Subject: A.x vs. A["x"] In-Reply-To: <84293188.TWlGDM3umM@beaureve.gmx.net> References: <1718504.hrdDYYapAE@beaureve.gmx.net> <84293188.TWlGDM3umM@beaureve.gmx.net> Message-ID: Martin Drautzburg wrote: > Terry Reedy wrote: > >> On 1/22/2010 2:29 PM, Martin Drautzburg wrote: >>> This has probably been asekd a million times, but if someone could >>> give a short answer anyways I's be most grateful. >>> >>> What is it that allows one to write A.x? If I have a variable A, > > >>> I know you can do this with classes, but not with plain objects, but >>> why is that so? >> You exact meaning here is not clear, but I suspect it is somewhat >> incorrect, at least for built-in classes. > > You're right. I used to think you could do this to classes: > > G = Strum > G.x=1 > > But not to objects (instances): > > g = Strum() > g.y = 2 > > But in fact both work. Thanks for clarifying Both work, in fact, because a class is (like almost any other Python object), an instance of some class. In Python the classes that are defined in the interpreter are usually called types, but since "new-style objects" were introduced into Python in 2.2 you can define your own types (and subclass the system types). The name for the particular types whose instances are classes is "metaclass". But once you realize that the "regular classes" you already know about are just instances of their metaclass (the metaclass , in many cases) it might seem less surprising. There is a necessary but strange and surprising relationship between types and objects that can be summarized as isinstance(object, type) == isinstance(type,object) == True Because the built-in types are implemented as a part of the interpreter they are somewhat more restricted. Consequently not only can you not add attributes to the classes, you can't add them to the instances either: >>> int.x = 2 Traceback (most recent call last): File "", line 1, in TypeError: can't set attributes of built-in/extension type 'int' >>> i = int() >>> i.y = "hello" Traceback (most recent call last): File "", line 1, in AttributeError: 'int' object has no attribute 'y' >>> regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From ceciliaseidel at gmx.de Sat Jan 23 11:29:06 2010 From: ceciliaseidel at gmx.de (ceciliaseidel at gmx.de) Date: Sat, 23 Jan 2010 17:29:06 +0100 Subject: iterating lists In-Reply-To: References: Message-ID: <4B5B23D2.5070002@gmx.de> Arnaud Delobelle schrieb: > ceciliaseidel at gmx.de writes: > >> As you were talking about list.pop()... >> >> Is anyone able to reproduce the following and explain why this happens >> by chance? (Using 3.1.1) >> >> l1 = ["ready", "steady", "go"] >> l2 = ["one", "two", "tree"] >> l3 = ["lift off"] >> >> for w in l1: Ouch... thanks Arnaud... The stable way would've been for w in l1[:]: #use copy of l1 for iteration print(l1.pop()) #decomposite list (just in case any other newbie is reading this...) >> print(l1.pop()) #prints only "go steady" - why not "ready"?? >> > > I suggest you simulate the loop above using pen and paper, writing the > value of w and the value of l1 at each iteration. The behaviour you are > observing should then be clearly explained. And you should also realise > that it's a really bad idea to mutate a list that you are iterating on! > > HTH > From duncan.booth at invalid.invalid Sat Jan 23 11:29:23 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 23 Jan 2010 16:29:23 GMT Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> Message-ID: Roy Smith wrote: > I'm looking at http://tinyurl.com/cdbwog. It shows all the operations > of a list. What it does not show is their cost. For pop(), it has a > note: > > "The pop() method is only supported by the list and array types. The > optional argument i defaults to -1, so that by default the last item > is removed and returned". The page you should probably be looking at is http://wiki.python.org/moin/TimeComplexity From susan_kijiji at yahoo.ca Sat Jan 23 11:33:42 2010 From: susan_kijiji at yahoo.ca (im_smialing) Date: Sat, 23 Jan 2010 08:33:42 -0800 (PST) Subject: py2exe deal with python command line inside a program References: Message-ID: On Jan 22, 2:35?pm, susan_kij... at yahoo.ca wrote: > Hi, > > I need to create a python subprogress, like this: > myProcess = subprocess.Popen([sys.executable, 'C:\myscript.py'], > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?env=env, stdin=subprocess.PIPE, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?stdout=subprocess.PIPE) > > sys.executable was printed out as ''C:\\Python25\\python.exe'', how > can I make this work in executable package through py2exe? > > I have to fix the following problems: > -Source code shouldn't exposed in an executable program > -Since python ?environment is not required when running an executable > program, how to deal with the situation that "C:\\Python25\ > \python.exe" is required as part of command? > > Thanks in advance! Anyone can help? The problem took me two days and still no solutions From alfps at start.no Sat Jan 23 11:37:22 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sat, 23 Jan 2010 17:37:22 +0100 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <036b0836$0$1357$c3e8da3@news.astraweb.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> <036b0836$0$1357$c3e8da3@news.astraweb.com> Message-ID: * Steven D'Aprano: > On Sat, 23 Jan 2010 09:57:04 -0500, Roy Smith wrote: > >> In article , >> "Alf P. Steinbach" wrote: >> >>> But it would IMHO have been better if it wasn't called "list", which >>> brings in the wrong associations for someone used to other languages. >> +1. >> >> When I first started using Python (back in the 1.4 days), I assumed a >> list was a singly-linked list. > > Why would you do that? I can think of at least eight different > implementations of the abstract list data structure: > > constant-size array > variable-size array > variable-size array with amortised O(1) appends > singly-linked list > singly-linked list with CDR coding > doubly-linked list > skip list > indexable skip list > > One can reasonably disregard constant-sized arrays as a possibility, > given that Python lists aren't fixed size (pity the poor Pascal and > Fortran coders who are stuck with static arrays!), but the rest are all > reasonable possibilities. A linked list implementation would yield O(n) indexing. A great many loops in e.g. Python libraries code now having linear time would then get quadratic time, O(n^2). Those libraries would then be effectively unusable without extensive rewriting: one version for ordinary Python and one for 'list-as-list' Pythons... Thus, the linked list implementations are IMO *not* reasonable. And the reason is precisely the implied complexity guarantees, especially on indexing -- which could reasonably be O(log n), but not worse than that. > Why assume one specific implementation in the > absence of documentation promising certain performance characteristics? > > >> Oddly enough, I was going to write in the above paragraph, "like a C++ >> STL list", until I happened to glance at the STL docs and refreshed my >> memory that an STL list is doubly-linked. Which just goes to show that >> making assumptions based on names is a bad idea. > > Exactly :) > > > >> So, we're right back to my statement earlier in this thread that the >> docs are deficient in that they describe behavior with no hint about >> cost. Given that, it should be no surprise that users make incorrect >> assumptions about cost. > > There are quite a few problems with having the documentation specify cost: > > (1) Who is going to do it? Any volunteers? This problem must have been addressed at each time the documentation for some version of Python was written or updated. > (2) Big-oh notation can be misleading, especially for naive users, or > those whose intuition for what's fast has been shaped by other languages. > Big-oh doesn't tell you whether something is fast or slow, only how it > scales -- and sometimes not even then. It's how things scale that are of interest. :-) Big-oh tells you an upper asymptotic limit. That's sufficient for e.g. the C++ standard -- which, by the way, constitutes a concrete example of the practicality of specifying complexity. > (3) Having documented a particular performance, that discourages > implementation changes. Any would-be patch or new implementation not only > has to consider that the functional behaviour doesn't change, but that > the performance doesn't either. > > In practice the Python developers are unlikely to make an implementation > change which leads to radically worse performance, particularly for > critical types like list and dict. But in other cases, they might choose > to change big-oh behaviour, and not wish to be tied down by documentation > of the cost of operations. Say that there was an O(log n) documented worst complexity for 'list' indexing. Above you have described it as "reasonable" to break that, having O(n) complexity... But in light of my comments on that, and especially thinking a bit about maintainance of two or more! versions of various libraries, don't you agree that it would be Just Bad(TM)? > (4) How much detail is necessary? What about degenerate cases? E.g. dict > lookup in CPython is typically O(1) amortised, but if all the keys hash > to the same value, it falls to O(N). From N1745, the Technical Report 1 on C++ library extensions (will be part of the C++0x standard), table 21 specifying general requirements of unordered associative containers: expression: b.find(k) return type: iterator; assertion: Returns an iterator pointing to an element with key equivalent to k, or b.end() if no such element exists. complexity: Average case O(1), worst case O(b.size()). > (5) Should the language guarantee such degenerate behaviour? Who decides > which costs are guaranteed and which are not? I think the C++ standard (the latest draft of C++0x is freely available as PDF from the commitee pages) provides good guidance in this regard. :-) > (6) Such performance guarantees should be implementation specific, not > language specific. CPython is only one implementation of the language out > of many. Disagree Very Strongly. An implementation may offer stricter guarantees. But what matters regarding e.g. avoiding having to maintain two or three or umpteen versions of a library, is the set of language level complexity guarantees. Cheers, - Alf From showell30 at yahoo.com Sat Jan 23 11:41:15 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sat, 23 Jan 2010 08:41:15 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> <2ce05a04-ebc9-4f26-ba32-ff787925a564@c29g2000yqd.googlegroups.com> Message-ID: <43a0f0a9-da8b-47d5-824c-31fd5201a290@l30g2000yqb.googlegroups.com> On Jan 23, 5:46?am, Christian Heimes wrote: > Steve Howell wrote: > > Another benchmark is that deques are slower than lists for accessing > > elements. > > deques are optimized for accessing, inserting and removing data from > both ends. For anything else it's slower than the list type. The fact > was explained in this very thread yesterday. > And the benchmark confirmed it. The slowness is fairly negligible, though. From showell30 at yahoo.com Sat Jan 23 11:46:18 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sat, 23 Jan 2010 08:46:18 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> Message-ID: <38693efe-5de0-4b07-afab-bc1a0df8c490@b2g2000yqi.googlegroups.com> On Jan 23, 6:40?am, Roy Smith wrote: > In article > , > ?Steve Howell wrote: > > > This innocent program here literally moves about a million bytes of > > memory around for no good reason: > > > ? ? lst = [] > > ? ? for i in range(2000): > > ? ? ? ? lst.append(i) > > ? ? while lst: > > ? ? ? ? print lst.pop(0) > > > Why? ?Because list.pop(0) is implemented in O(N) instead of O(1). > > I think you're being a little pedantic here. ?Yes, it is true that pop(0) > is O(n), and that if you put an O(n) operation in a loop, you get O(n^2) > run time. > > The problem is that while it is well-known that putting something that's > O(n) in a loop gets you O(n^2), it's not well known that pop(0) for a > Python list is O(n). ?This is where you and I apparently start to differ in > what we think about this. > The source for Python is open. It pretty clearly shows that you move N bytes when you pop from the top of the list. Less clear is how linear the performance of memmove is. My benchmarks on the C program show that, at least on my computer, the results do not seem to contradict the "roughly linear" assumption. > You are arguing that this is a bug in the implementation of list. ?While I > suppose there's some validity to that argument, I disagree. ?What I would > argue (and have done so several times over the years, with little success) > is that this is a bug in the documentation! > > I'm looking athttp://tinyurl.com/cdbwog. ?It shows all the operations of a > list. ?What it does not show is their cost. ?For pop(), it has a note: > > "The pop() method is only supported by the list and array types. The > optional argument i defaults to -1, so that by default the last item is > removed and returned". > > There's nothing there that gives any hint that pop(0) is any more expensive > than pop(-1). ?That is "secret knowledge", which you only get by following > the newsgroup discussions or looking at the implementation. ?You shouldn't > have to do either. ?There's lots of possible ways list could be > implemented. ?Without knowing the details, I'm left to guess about > important stuff like the cost of operations. > > Every one of these operations should list the cost. ?Even if it's something > as vague as, "While not guaranteed by the language spec, in the current > implemenation of CPython ...". I agree with that. From showell30 at yahoo.com Sat Jan 23 12:03:46 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sat, 23 Jan 2010 09:03:46 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> Message-ID: <002ffd1d-15dc-4ea3-9340-6c64c09db32a@j14g2000yqm.googlegroups.com> On Jan 23, 4:12?am, Steven D'Aprano wrote: > > > An alternative would be to do exactly what you want lists to do: track > the start of the list. Untested: > > ? ? def recurse(prefix_lines): > ? ? ? ? start = 0 > ? ? ? ? end = len(prefix_lines) > ? ? ? ? while start < end: > ? ? ? ? ? ? prefix, line = prefix_lines[start] > ? ? ? ? ? ? if line == '': > ? ? ? ? ? ? ? ? start += 1 > ? ? ? ? ? ? ? ? append('') > ? ? ? ? ? ? else: > ? ? ? ? ? ? ? ? block_size = get_block(prefix_lines) > ? ? ? ? ? ? ? ? if block_size == 1: > ? ? ? ? ? ? ? ? ? ? start += 1 > ? ? ? ? ? ? ? ? ? ? if line == pass_syntax: > ? ? ? ? ? ? ? ? ? ? ? ? pass > ? ? ? ? ? ? ? ? ? ? elif line.startswith(flush_left_syntax): > ? ? ? ? ? ? ? ? ? ? ? ? append(line[len(flush_left_syntax):]) > ? ? ? ? ? ? ? ? ? ? elif line.startswith(flush_left_empty_line): > ? ? ? ? ? ? ? ? ? ? ? ? append('') > ? ? ? ? ? ? ? ? ? ? else: > ? ? ? ? ? ? ? ? ? ? ? ? append(prefix + leaf_method(line)) > ? ? ? ? ? ? ? ? else: > ? ? ? ? ? ? ? ? ? ? block = prefix_lines[:block_size] > ? ? ? ? ? ? ? ? ? ? start = block_size > ? ? ? ? ? ? ? ? ? ? branch_method(output, block, recurse) > ? ? ? ? return > > No more O(N) deletions. Problem solved. > A minor modification of your solution does work, but it also slightly + complicates the implementation. Keeping track of the start variable requires bookkeeping not just in recurse(), but also in methods that it calls. This is a pretty small program, so it's acceptable to pass around an offset variable to anybody else who might want to be consuming the list. ############ Generic indentation stuff follows -def get_indented_block(prefix_lines): - prefix, line = prefix_lines[0] +def get_indented_block(prefix_lines, start): + prefix, line = prefix_lines[start] len_prefix = len(prefix) i = 1 - while i < len(prefix_lines): - new_prefix, line = prefix_lines[i] + while i + start < len(prefix_lines): + new_prefix, line = prefix_lines[start+i] if line and len(new_prefix) <= len_prefix: break i += 1 - while i-1 > 0 and prefix_lines[i-1][1] == '': + while i-1 > 0 and prefix_lines[start+i-1][1] == '': i -= 1 return i @@ -190,15 +190,16 @@ ): append = output.append def recurse(prefix_lines): - while prefix_lines: - prefix, line = prefix_lines[0] + start = 0 + while start < len(prefix_lines): + prefix, line = prefix_lines[start] if line == '': - prefix_lines.pop(0) + start += 1 append('') else: - block_size = get_block(prefix_lines) + block_size = get_block(prefix_lines, start) if block_size == 1: - prefix_lines.pop(0) + start += 1 if line == pass_syntax: pass elif line.startswith(flush_left_syntax): @@ -208,8 +209,8 @@ else: append(prefix + leaf_method(line)) else: - block = prefix_lines[:block_size] - prefix_lines = prefix_lines[block_size:] + block = prefix_lines[start:start+block_size] + start += block_size branch_method(output, block, recurse) return prefix_lines = map(indentation_method, lines) From showell30 at yahoo.com Sat Jan 23 12:17:31 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sat, 23 Jan 2010 09:17:31 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> Message-ID: <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> On Jan 23, 4:12?am, Steven D'Aprano wrote: > On Fri, 22 Jan 2010 21:42:43 -0800, Steve Howell wrote: > > This innocent program here literally moves about a million bytes of > > memory around for no good reason: > > > ? ? lst = [] > > ? ? for i in range(2000): > > ? ? ? ? lst.append(i) > > ? ? while lst: > > ? ? ? ? print lst.pop(0) > > > Why? ?Because list.pop(0) is implemented in O(N) instead of O(1). > > > Why wouldn't you get a competent C programmer simply make list_ass_slice > > smart enough to make list.pop(0) O(1)? > > Because there are always trade-offs, and the competent C programmers who > coded the implementation for lists choose different tradeoffs to the ones > you would prefer. > > Seems to me that the simple solution to your problem is for you to > implement your own data structure that makes whatever trade-offs you > like. If it is good enough and popular enough, it might even replace the > existing list implementation. > The data structure that would make the tradeoffs I want would be implemented within CPython itself. I give a sketch of the changes elsewhere in this thread. Terry Reedy said: ''' If you try writing a full patch, as I believe someone did, or at least a prototype thereof, when the idea was discussed, you might have a better idea of what the tradeoffs are and why it was rejected. ''' I have to run, but tomorrow I will try to dig through python-dev archives and find the patch. If anybody has hints on where to look for it (anybody remember the author, for example?), it would be much appreciated. If the patch looks simple, I will try to pitch the idea that its time has come. Now that the specification of the language itself is frozen, I think there might be more room for improving implementations. Also, I might be able to make the argument that tradeoffs of memory vs. CPU vs. code complexity have different forces in the 2010s. Thanks for your reply. From detlev at die-offenbachs.de Sat Jan 23 12:28:32 2010 From: detlev at die-offenbachs.de (Detlev Offenbach) Date: Sat, 23 Jan 2010 18:28:32 +0100 Subject: pyflakes and Python3 Message-ID: Hi, does anybody know, if pyflakes is being ported to Python3? Or is there any other tool like it around that works with Python3? Regards, Detlev -- Detlev Offenbach detlev at die-offenbachs.de From rschroev_nospam_ml at fastmail.fm Sat Jan 23 12:29:33 2010 From: rschroev_nospam_ml at fastmail.fm (Roel Schroeven) Date: Sat, 23 Jan 2010 18:29:33 +0100 Subject: iterating lists In-Reply-To: References: Message-ID: <1mG6n.74060$4N3.53053@newsfe06.ams2> Op 2010-01-23 17:29, ceciliaseidel at gmx.de schreef: > > > Arnaud Delobelle schrieb: >> ceciliaseidel at gmx.de writes: >> >>> As you were talking about list.pop()... >>> >>> Is anyone able to reproduce the following and explain why this happens >>> by chance? (Using 3.1.1) >>> >>> l1 = ["ready", "steady", "go"] >>> l2 = ["one", "two", "tree"] >>> l3 = ["lift off"] >>> >>> for w in l1: > > Ouch... thanks Arnaud... The stable way would've been > > for w in l1[:]: #use copy of l1 for iteration > print(l1.pop()) #decomposite list I would prefer: while l1: print(l1.pop()) -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven From stefan_ml at behnel.de Sat Jan 23 12:34:04 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 23 Jan 2010 18:34:04 +0100 Subject: iterating lists In-Reply-To: References: Message-ID: <4b5b330c$0$6716$9b4e6d93@newsspool2.arcor-online.net> ceciliaseidel at gmx.de, 23.01.2010 17:29: > Arnaud Delobelle wrote: >> ceciliaseidel at gmx.de writes: >> >>> As you were talking about list.pop()... >>> >>> Is anyone able to reproduce the following and explain why this happens >>> by chance? (Using 3.1.1) >>> >>> l1 = ["ready", "steady", "go"] >>> l2 = ["one", "two", "tree"] >>> l3 = ["lift off"] >>> >>> for w in l1: > > Ouch... thanks Arnaud... The stable way would've been > > for w in l1[:]: #use copy of l1 for iteration > print(l1.pop()) #decomposite list If the intention is to exhaust the list during the iteration, I'd go for this: while l1: print(l1.pop()) Stefan From thinke365 at gmail.com Sat Jan 23 12:37:58 2010 From: thinke365 at gmail.com (thinke365) Date: Sat, 23 Jan 2010 09:37:58 -0800 (PST) Subject: how to generate random numbers that satisfy certain distribution Message-ID: <27288180.post@talk.nabble.com> such as uniform distribution, Normal distribution or poisson distribution. is there any package that can be used to generate such random numbers. -- View this message in context: http://old.nabble.com/how-to-generate-random-numbers-that-satisfy-certain-distribution-tp27288180p27288180.html Sent from the Python - python-list mailing list archive at Nabble.com. From showell30 at yahoo.com Sat Jan 23 12:38:29 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sat, 23 Jan 2010 09:38:29 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> <036b0836$0$1357$c3e8da3@news.astraweb.com> Message-ID: <44ccd731-4a98-4eec-a270-08f66b192d93@l19g2000yqb.googlegroups.com> On Jan 23, 7:54?am, Steven D'Aprano wrote: > On Sat, 23 Jan 2010 09:57:04 -0500, Roy Smith wrote: > > In article , > > ?"Alf P. Steinbach" wrote: > > >> But it would IMHO have been better if it wasn't called "list", which > >> brings in the wrong associations for someone used to other languages. > > > +1. > > > When I first started using Python (back in the 1.4 days), I assumed a > > list was a singly-linked list. > > Why would you do that? I can think of at least eight different > implementations of the abstract list data structure: > > constant-size array > variable-size array > variable-size array with amortised O(1) appends > singly-linked list > singly-linked list with CDR coding > doubly-linked list > skip list > indexable skip list > > One can reasonably disregard constant-sized arrays as a possibility, > given that Python lists aren't fixed size (pity the poor Pascal and > Fortran coders who are stuck with static arrays!), but the rest are all > reasonable possibilities. Why assume one specific implementation in the > absence of documentation promising certain performance characteristics? > > > Oddly enough, I was going to write in the above paragraph, "like a C++ > > STL list", until I happened to glance at the STL docs and refreshed my > > memory that an STL list is doubly-linked. ?Which just goes to show that > > making assumptions based on names is a bad idea. > > Exactly :) > > > So, we're right back to my statement earlier in this thread that the > > docs are deficient in that they describe behavior with no hint about > > cost. Given that, it should be no surprise that users make incorrect > > assumptions about cost. > > There are quite a few problems with having the documentation specify cost: > > (1) Who is going to do it? Any volunteers? > > (2) Big-oh notation can be misleading, especially for naive users, or > those whose intuition for what's fast has been shaped by other languages. > Big-oh doesn't tell you whether something is fast or slow, only how it > scales -- and sometimes not even then. > > (3) Having documented a particular performance, that discourages > implementation changes. Any would-be patch or new implementation not only > has to consider that the functional behaviour doesn't change, but that > the performance doesn't either. > > In practice the Python developers are unlikely to make an implementation > change which leads to radically worse performance, particularly for > critical types like list and dict. But in other cases, they might choose > to change big-oh behaviour, and not wish to be tied down by documentation > of the cost of operations. > > (4) How much detail is necessary? What about degenerate cases? E.g. dict > lookup in CPython is typically O(1) amortised, but if all the keys hash > to the same value, it falls to O(N). > > (5) Should the language guarantee such degenerate behaviour? Who decides > which costs are guaranteed and which are not? > > (6) Such performance guarantees should be implementation specific, not > language specific. CPython is only one implementation of the language out > of many. > Bringing this thread full circle, does it make sense to strike this passage from the tutorial?: ''' It is also possible to use a list as a queue, where the first element added is the first element retrieved (?first-in, first-out?); however, lists are not efficient for this purpose. While appends and pops from the end of list are fast, doing inserts or pops from the beginning of a list is slow (because all of the other elements have to be shifted by one). ''' I think points #3 and #6 possibly apply. Regarding points #2 and #4, the tutorial is at least not overly technical or specific; it just explains the requirement to shift other elements one by one in simple layman's terms. From steve at REMOVE-THIS-cybersource.com.au Sat Jan 23 12:44:22 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 23 Jan 2010 17:44:22 GMT Subject: iterating lists References: <1mG6n.74060$4N3.53053@newsfe06.ams2> Message-ID: <036b21fb$0$1357$c3e8da3@news.astraweb.com> On Sat, 23 Jan 2010 18:29:33 +0100, Roel Schroeven wrote: >> for w in l1[:]: #use copy of l1 for iteration >> print(l1.pop()) #decomposite list > > I would prefer: > > while l1: > print(l1.pop()) I would prefer: for x in reversed(l1): print(x) l1[:] = [] And garbage dispose of the entire list in one go, instead of an item at a time. -- Steven From python at mrabarnett.plus.com Sat Jan 23 12:47:49 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 23 Jan 2010 17:47:49 +0000 Subject: PyArg_ParseTupleAndKeywords In-Reply-To: References: <91afd9e3-93a1-488d-8e83-299bfc222e24@22g2000yqr.googlegroups.com> Message-ID: <4B5B3645.3090009@mrabarnett.plus.com> Mr.M wrote: > Carl Banks ha scritto: >>> (some declarations omitted here) >> >> You probably shouldn't have, that could be where the error is.... I'd >> include the whole function up to the call that raises the exception. > > Thank you so much Carl for your help, i'll provide more info so that you > can try to fix my errors! > Thank you again! > > First of all i'll try to explain what i've discovered over a night i > spent awake trying to find where the bug is. > > My init behaves differently in this situations: > > [code] > # this works > dummy = mymodule.myclass() > > # this works > dummy = mymodule.myclass("string for service") > > # this also works > dummy = mymodule.myclass("string for service", "string for event_type") > # and it works if i provide arguments in the right sequence, of course > > # this won't work > dummy = mymodule.myclass("string for service", "string for event_type", > free_text = "string for free_text") > > # but this works > dummy = mymodule.myclass(service = "string for service") > > # the worst thing: this doesn't work but it did and i can't > # understand what is changed > > dummy = mymodule.myclass(free_text = "text for free_text") > > ok, every time the code fails i get this exception: > > TypeError: expected string or Unicode object, tuple found > > so, i think for some reason PyArg_ParseTupleAndKeywords receives "args" > as a touple instead of something else. > > This is everything i managed to discover. > > This is the code: > > [code] > /* DEBUG INFO */ > printf("event INIT\n"); > if (args) > { > int i; > printf("args = \"%s\"\n", PyString_AsString(args)); > printf("type = %s\n", > PyString_AsString(PyObject_Repr(PyObject_Type(args)))); > printf("tuple size = %d\n", PyTuple_Size(args)); > for (i = 0; i < PyTuple_Size(args); i++) > { > printf("%d) %s\n", i, > PyString_AsString(PyTuple_GetItem(args, i))); > } > } > else > { > printf("args = NULL\n"); > } > printf("dict:\n"); > if (keywords) > { > printf(" = %s\n", > PyString_AsString(PyObject_Repr(keywords))); > printf("type = %s\n", > PyString_AsString(PyObject_Repr(PyObject_Type(keywords)))); > } > else > { > printf("keywords = NULL\n"); > } > > char* service = NULL; > char* event_type = NULL; > char* free_text = NULL; > char* group_uid = NULL; > char* remote_name = NULL; > char* remote_abook_uid = NULL; > > time_t start_time = -1L; > time_t end_time = -1L; > time_t storage_time = -1L; > > int flags = 0; > > int bytes_sent = -1; > int bytes_received = -1; > > PyObject* temp; > > static char* keywordlist[] = {"service", > "event_type", > "free_text", > "group_uid", > "remote_name", > "remote_abook_uid", > "start_time", > "end_time", > "storage_time", > "flags", > "bytes_sent", > "bytes_received", > NULL}; > > if (!PyArg_ParseTupleAndKeywords(args, keywords, "|ssssssiiiiii", > keywordlist, > &service, > &event_type, > &free_text, > &group_uid, > &remote_name, > &remote_abook_uid, > &start_time, > &end_time, > &storage_time, > &flags, > &bytes_sent, > &bytes_received)) > { > return -1; > } > > printf("PyArg_ParseTupleAndKeywords worked fine\n"); > > [/code] > > (sorry if my code is a little messy and my english rather bad!) > >> Are you sure that PyArg_ParseTupleAndKeywords is what's raising the >> error? > > Yes, i think so. I have a lot of printf in my code! > >> Are you subclassing this type in Python, and passing one of the string >> parameters a tuple by mistake? For instance, did you do something >> like this: >> >> class myclass(_mycmodule._myctype): >> def __init__(self,*args,**kwargs): >> log_something_here() >> super(myclass,self).__init__(args,**kwargs) >> >> Note the missing * on args. > > no, i'm not subclassing. > I have to admit that i had some testcase and a few weeks ago they worked > like a charm... i can't understand what i changed, of course. > >> >> >>> I found that if i write this everything works fine: >>> >>> [code] >>> import mymodule >>> a = mymodule.myclass(service = "blabla", event_type = "event", free_text >>> = "free", group_uid = "group", remote_name = "remote name", >>> remote_abook_uid = "abook", start_time = 1, end_time = 61, storage_time >>> = 61, flags = 2) >>> [/code] >>> >>> in other words, the code only works if *EVERY* argument is specified in >>> exactly the same position it appears in keyword-null-terminated array. >> >> That it would have to be in a certain order is strange. Are you sure >> it's just not merely that they all have to be present? >> > > No, i tryied to remove the "|" and Python complains that 12 argument > must be present. So they are, in a sense, optional, but (and this is > strange), they must be present in the right order even i provide them > with keyword arguments. > >> Tricky, but I think it'd help if you provided more information. My >> gut feeling is that there exception is being raised somewhere other >> than you think it is (i.e., not by PyArg_ParseTuple), so I'd recommend >> adding some debugging statements to track down the exact point wher >> the error occurs. > > Ok, thank you! > The code i posted is exactly the same i run (and it fails). > I've no problem sending you the whole module or adding any other debug > info you think could be helpfull to find the source of the bug. > Did you specify that the method accepts keywords arguments with METH_KEYWORDS? The function would take parameters for the instance (self), the positional arguments (args) and the keyword arguments (kwargs). http://docs.python.org/c-api/structures.html If you don't use METH_KEYWORDS then it'll think that all the arguments are positional, which is what seems to be happening: From mrm at unknown.nospam Sat Jan 23 13:21:25 2010 From: mrm at unknown.nospam (Mr.M) Date: Sat, 23 Jan 2010 18:21:25 GMT Subject: PyArg_ParseTupleAndKeywords In-Reply-To: References: <91afd9e3-93a1-488d-8e83-299bfc222e24@22g2000yqr.googlegroups.com> Message-ID: MRAB ha scritto: > Did you specify that the method accepts keywords arguments with > METH_KEYWORDS? The function would take parameters for the instance > (self), the positional arguments (args) and the keyword arguments > (kwargs). > > http://docs.python.org/c-api/structures.html > > If you don't use METH_KEYWORDS then it'll think that all the arguments > are positional, which is what seems to be happening: > Thank you MRAB for your reply. No, i didn't specify METH_KEYWORDS flag, but i think init method (the one that you put in tp_init slot, it is "initproc" type) doesn't have to appear in the PyMethodDef structure. Maybe i'm wrong? Luca. From info at wilbertberendsen.nl Sat Jan 23 13:44:00 2010 From: info at wilbertberendsen.nl (Wilbert Berendsen) Date: Sat, 23 Jan 2010 19:44:00 +0100 Subject: counting character occurrences In-Reply-To: References: Message-ID: <201001231944.01002.info@wilbertberendsen.nl> Op vrijdag 22 januari 2010 schreef Arnaud: > Why not just start with (untested): > > import codecs > from collections import defaultdict > > tcounters = defaultdict(int) > f = codecs.open('/home/gavron/git/screen/src/screen.c', 'r', "utf-8") > > for c in f.read(): > tcounters[c] += 1 > > for c, n in tcounters.iteritems(): > print "%r\t%i" % (c, n) Or using Counter from Python3.1 collections: import codecs from collections import Counter filename = '/home/gavron/git/screen/src/screen.c' with codecs.open(filename, 'r', 'utf-8') as f: counted = Counter(f.read()) for c, n in counted: print(c, n, sep='\t') with best regards, Wilbert Berendsen -- http://www.wilbertberendsen.nl/ From aahz at pythoncraft.com Sat Jan 23 13:45:59 2010 From: aahz at pythoncraft.com (Aahz) Date: 23 Jan 2010 10:45:59 -0800 Subject: ISC License References: <00eb248d-c9c9-430f-bc83-41ac865c5111@e11g2000yqe.googlegroups.com> Message-ID: In article <00eb248d-c9c9-430f-bc83-41ac865c5111 at e11g2000yqe.googlegroups.com>, Joan Miller wrote: > >There is a license approved by the OSI, the ISC License [1], which >should be included in the PyPi classifiers [2]. > >[1] https://www.isc.org/software/license >http://www.opensource.org/licenses/isc-license.txt >[2] http://pypi.python.org/pypi?%3Aaction=list_classifiers http://pypi.python.org/pypi has a link named "Bug Reports" -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From aahz at pythoncraft.com Sat Jan 23 13:50:42 2010 From: aahz at pythoncraft.com (Aahz) Date: 23 Jan 2010 10:50:42 -0800 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> <8e4d3fe2-c4bd-4a73-9c50-7a336dab25be@o28g2000yqh.googlegroups.com> Message-ID: In article <8e4d3fe2-c4bd-4a73-9c50-7a336dab25be at o28g2000yqh.googlegroups.com>, Steve Howell wrote: >On Jan 22, 11:10=A0pm, a... at pythoncraft.com (Aahz) wrote: >> >>>I know Python's number one concern will never be speed, but if Python >>>makes an O(1) operation into an unnecessarily O(N) operation for no >>>good reasons other than "it's too complicated, " or it "adds another >>>pointer to the structure," or "it adds another conditional check to >>>list_ass_slice for operations that aren't popping off the top," I >>>think it's reasonable to challenge the design philosophy. >> >> "Rough consensus and running code." >> >> You have a good point, but nobody will ever give your idea serious >> attention until there's a patch and benchmarks. > >Here is a benchmark of O(N*N) vs. O(N) for two C programs. One does >memmove; the other simply advances the pointer. You should provide pybench numbers and probably also use the benchmarks produced by the Unladen Swallow project. The next step is to file a patch on bugs.python.org and request review. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From ra.ravi.rav at gmail.com Sat Jan 23 13:51:38 2010 From: ra.ravi.rav at gmail.com (Ravi) Date: Sat, 23 Jan 2010 10:51:38 -0800 (PST) Subject: how to generate random numbers that satisfy certain distribution References: Message-ID: On Jan 23, 10:37?pm, thinke365 wrote: > such as uniform distribution, Normal distribution or poisson distribution. > is there any package that can be used to generate such random numbers. > > -- > View this message in context:http://old.nabble.com/how-to-generate-random-numbers-that-satisfy-cer... > Sent from the Python - python-list mailing list archive at Nabble.com. Did you try random package? From roy at panix.com Sat Jan 23 14:02:53 2010 From: roy at panix.com (Roy Smith) Date: Sat, 23 Jan 2010 14:02:53 -0500 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> Message-ID: In article , Duncan Booth wrote: > Roy Smith wrote: > > > I'm looking at http://tinyurl.com/cdbwog. It shows all the operations > > of a list. What it does not show is their cost. For pop(), it has a > > note: > > > > "The pop() method is only supported by the list and array types. The > > optional argument i defaults to -1, so that by default the last item > > is removed and returned". > > The page you should probably be looking at is > http://wiki.python.org/moin/TimeComplexity I was not aware of this page; thanks for pointing it out. From thinke365 at gmail.com Sat Jan 23 14:10:52 2010 From: thinke365 at gmail.com (thinke365) Date: Sat, 23 Jan 2010 11:10:52 -0800 (PST) Subject: how to generate random numbers that satisfy certain distribution In-Reply-To: References: <27288180.post@talk.nabble.com> Message-ID: <27288996.post@talk.nabble.com> Bugzilla from ra.ravi.rav at gmail.com wrote: > > On Jan 23, 10:37?pm, thinke365 wrote: >> such as uniform distribution, Normal distribution or poisson >> distribution. >> is there any package that can be used to generate such random numbers. >> >> -- >> View this message in >> context:http://old.nabble.com/how-to-generate-random-numbers-that-satisfy-cer... >> Sent from the Python - python-list mailing list archive at Nabble.com. > > Did you try random package? > -- > http://mail.python.org/mailman/listinfo/python-list > > of course i have tried random package, but can this package generate random sequence that satisfy possion distribution , normal distribution and uniform distribution -- View this message in context: http://old.nabble.com/how-to-generate-random-numbers-that-satisfy-certain-distribution-tp27288180p27288996.html Sent from the Python - python-list mailing list archive at Nabble.com. From python at mrabarnett.plus.com Sat Jan 23 14:14:54 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 23 Jan 2010 19:14:54 +0000 Subject: PyArg_ParseTupleAndKeywords In-Reply-To: References: <91afd9e3-93a1-488d-8e83-299bfc222e24@22g2000yqr.googlegroups.com> Message-ID: <4B5B4AAE.3050805@mrabarnett.plus.com> Mr.M wrote: > MRAB ha scritto: > >> Did you specify that the method accepts keywords arguments with >> METH_KEYWORDS? The function would take parameters for the instance >> (self), the positional arguments (args) and the keyword arguments >> (kwargs). >> >> http://docs.python.org/c-api/structures.html >> >> If you don't use METH_KEYWORDS then it'll think that all the arguments >> are positional, which is what seems to be happening: >> > > Thank you MRAB for your reply. > No, i didn't specify METH_KEYWORDS flag, but i think init method (the > one that you put in tp_init slot, it is "initproc" type) doesn't have to > appear in the PyMethodDef structure. > > Maybe i'm wrong? > I think you're right. From peteRE at MpeteOzilla.Vco.ukE Sat Jan 23 14:17:47 2010 From: peteRE at MpeteOzilla.Vco.ukE (Peter Chant) Date: Sat, 23 Jan 2010 19:17:47 +0000 Subject: how to generate random numbers that satisfy certain distribution References: Message-ID: thinke365 wrote: > > such as uniform distribution, Normal distribution or poisson distribution. > is there any package that can be used to generate such random numbers. > I remeber being told that adding up 12 random numbers in the range 0-1 (which is what most computer random number genertors at the time chucked out) and subtracted 6 gives a pretty good normal distribution. I think I did try it once and it failed, but I must have done something odd. -- http://www.petezilla.co.uk From nulla.epistola at web.de Sat Jan 23 14:36:18 2010 From: nulla.epistola at web.de (Sibylle Koczian) Date: Sat, 23 Jan 2010 20:36:18 +0100 Subject: installing psycopg2-2.0.13 with python3.1 In-Reply-To: References: <03495BC1-8736-42D5-9B59-5B0148F643AA@gmail.com> Message-ID: Iain Barnett schrieb: > On 21 Jan 2010, at 00:11, Gabriel Genellina wrote: > >> If you insist on using Python 3.1, there is another interface to PostgreSQL called pg8000 that claims to be Python 3.x compatible (I've not actually tested it). >> >> [1] http://mail.python.org/pipermail/python-porting/2008-December/000004.html >> [2] http://pybrary.net/pg8000/ >> Or there is py-postgresql: http://python.projects.postgresql.org/ I've not yet used it very much, but it seems to work quite well, is reasonably documented and the mailing list is small and helpful. HTH Sibylle From arnodel at googlemail.com Sat Jan 23 14:57:07 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sat, 23 Jan 2010 19:57:07 +0000 Subject: how to generate random numbers that satisfy certain distribution References: Message-ID: thinke365 writes: > such as uniform distribution, Normal distribution or poisson distribution. > is there any package that can be used to generate such random numbers. It's all in the standard library, the module is called -surprisingly- 'random'. - use random.uniform for the uniform distributions - use random normalvariate for normal distributions There isn't a Poisson distribution function, but there is a expovariate function. I think you can get poisson like this, but take it with a grain of salt because my probability skills are very rusty! import random import itertools def poisson(l): e = random.expovariate acc = 0.0 for n in itertools.count(): acc += e(l) if acc >= 1.0: return n -- Arnaud From no.email at nospam.invalid Sat Jan 23 15:26:03 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 23 Jan 2010 12:26:03 -0800 Subject: how to generate random numbers that satisfy certain distribution References: <27288180.post@talk.nabble.com> Message-ID: <7xy6jo1sbo.fsf@ruckus.brouhaha.com> thinke365 writes: > of course i have tried random package, but can this package generate random > sequence that satisfy possion distribution , normal distribution and uniform > distribution Did you look at the documentation? From no.email at nospam.invalid Sat Jan 23 15:29:22 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 23 Jan 2010 12:29:22 -0800 Subject: how to generate random numbers that satisfy certain distribution References: Message-ID: <7xtyuc1s65.fsf@ruckus.brouhaha.com> Peter Chant writes: > I remeber being told that adding up 12 random numbers in the range 0-1 > (which is what most computer random number genertors at the time chucked > out) and subtracted 6 gives a pretty good normal distribution. I think I > did try it once and it failed, but I must have done something odd. That gives you a binomial distribution on 12 trials, which approximates a normal distribution when the number of trials is large. 12 isn't too bad. But there's a simpler way, the Box-Muller transform, that gives you a pair drawn from a precisely normal distribution from two uniform random samples: http://en.wikipedia.org/wiki/Box-Muller_transform From thinke365 at gmail.com Sat Jan 23 15:57:15 2010 From: thinke365 at gmail.com (thinke365) Date: Sat, 23 Jan 2010 12:57:15 -0800 (PST) Subject: this customize sort did not work ,what's wrong? Message-ID: <27289860.post@talk.nabble.com> l = list() l1 = list((1, 2, 3, 4)) l2 = list((1,2)) l3 = list((1, 2, 3, 4, 5)) l.append(l1) l.append(l2) l.append(l3) print l def sort_by_list(E1, E2): print len(E1), len(E2) return len(list(E1)) > len(list(E2)) l.sort(cmp=sort_by_list) print l output: [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] 2 4 5 2 [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] the order of the elements in the list did not change! -- View this message in context: http://old.nabble.com/this-customize-sort-did-not-work-%2Cwhat%27s-wrong--tp27289860p27289860.html Sent from the Python - python-list mailing list archive at Nabble.com. From thinke365 at gmail.com Sat Jan 23 16:06:22 2010 From: thinke365 at gmail.com (thinke365) Date: Sat, 23 Jan 2010 13:06:22 -0800 (PST) Subject: this customize sort did not work ,what's wrong? In-Reply-To: <27289860.post@talk.nabble.com> References: <27289860.post@talk.nabble.com> Message-ID: <27289922.post@talk.nabble.com> i mean the output i want is: [ [1, 2], [1, 2, 3, 4], [1, 2, 3, 4, 5]], that is sort according to the length of the list element thinke365 wrote: > > l = list() > l1 = list((1, 2, 3, 4)) > l2 = list((1,2)) > l3 = list((1, 2, 3, 4, 5)) > l.append(l1) > l.append(l2) > l.append(l3) > print l > > def sort_by_list(E1, E2): > print len(E1), len(E2) > return len(list(E1)) > len(list(E2)) > > l.sort(cmp=sort_by_list) > print l > > output: > [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] > 2 4 > 5 2 > [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] > > the order of the elements in the list did not change! > -- View this message in context: http://old.nabble.com/this-customize-sort-did-not-work-%2Cwhat%27s-wrong--tp27289860p27289922.html Sent from the Python - python-list mailing list archive at Nabble.com. From thinke365 at gmail.com Sat Jan 23 16:21:06 2010 From: thinke365 at gmail.com (thinke365) Date: Sat, 23 Jan 2010 13:21:06 -0800 (PST) Subject: this customize sort did not work ,what's wrong? In-Reply-To: <27289922.post@talk.nabble.com> References: <27289860.post@talk.nabble.com> <27289922.post@talk.nabble.com> Message-ID: <27290014.post@talk.nabble.com> jesus, now i fixed it, using odd lambda sort. l.sort(lambda x,y: cmp(len(x), len(y))) print l BUT I AM STILL CONFUSED WHY COSTOMIZED SORT FAILED TO SORT AS IT IS PROGRAMMER! thinke365 wrote: > > i mean the output i want is: > [ [1, 2], [1, 2, 3, 4], [1, 2, 3, 4, 5]], that is sort according to the > length of the list element > > thinke365 wrote: >> >> l = list() >> l1 = list((1, 2, 3, 4)) >> l2 = list((1,2)) >> l3 = list((1, 2, 3, 4, 5)) >> l.append(l1) >> l.append(l2) >> l.append(l3) >> print l >> >> def sort_by_list(E1, E2): >> print len(E1), len(E2) >> return len(list(E1)) > len(list(E2)) >> >> l.sort(cmp=sort_by_list) >> print l >> >> output: >> [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] >> 2 4 >> 5 2 >> [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] >> >> the order of the elements in the list did not change! >> > > -- View this message in context: http://old.nabble.com/this-customize-sort-did-not-work-%2Cwhat%27s-wrong--tp27289860p27290014.html Sent from the Python - python-list mailing list archive at Nabble.com. From alfps at start.no Sat Jan 23 16:37:40 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sat, 23 Jan 2010 22:37:40 +0100 Subject: this customize sort did not work ,what's wrong? In-Reply-To: References: <27289860.post@talk.nabble.com> <27289922.post@talk.nabble.com> Message-ID: * thinke365: > jesus, now i fixed it, using odd lambda sort. > l.sort(lambda x,y: cmp(len(x), len(y))) > print l This uses 'cmp'. Your earlier code, quoted below, used '>'. > BUT I AM STILL CONFUSED WHY COSTOMIZED SORT FAILED TO SORT AS IT IS > PROGRAMMER! Please don't shout. > thinke365 wrote: >> i mean the output i want is: >> [ [1, 2], [1, 2, 3, 4], [1, 2, 3, 4, 5]], that is sort according to the >> length of the list element >> >> thinke365 wrote: >>> l = list() >>> l1 = list((1, 2, 3, 4)) >>> l2 = list((1,2)) >>> l3 = list((1, 2, 3, 4, 5)) >>> l.append(l1) >>> l.append(l2) >>> l.append(l3) >>> print l >>> >>> def sort_by_list(E1, E2): >>> print len(E1), len(E2) >>> return len(list(E1)) > len(list(E2)) >>> >>> l.sort(cmp=sort_by_list) >>> print l >>> >>> output: >>> [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] >>> 2 4 >>> 5 2 >>> [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] >>> >>> the order of the elements in the list did not change! >>> Cheers & hth., - Alf From duncan.booth at invalid.invalid Sat Jan 23 16:40:29 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 23 Jan 2010 21:40:29 GMT Subject: this customize sort did not work ,what's wrong? References: <27289860.post@talk.nabble.com> <27289922.post@talk.nabble.com> Message-ID: thinke365 wrote: > jesus, now i fixed it, using odd lambda sort. > l.sort(lambda x,y: cmp(len(x), len(y))) > print l > > BUT I AM STILL CONFUSED WHY COSTOMIZED SORT FAILED TO SORT AS IT IS > PROGRAMMER! Try reading the documentation: >>> help(list.sort) Help on method_descriptor: sort(...) L.sort(cmp=None, key=None, reverse=False) -- stable sort *IN PLACE*; cmp(x, y) -> -1, 0, 1 Your comparison function returned True or False, not -1, 0, +1 From steve at REMOVE-THIS-cybersource.com.au Sat Jan 23 17:05:07 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 23 Jan 2010 22:05:07 GMT Subject: how to generate random numbers that satisfy certain distribution References: <27288180.post@talk.nabble.com> Message-ID: <036b5f17$0$1357$c3e8da3@news.astraweb.com> On Sat, 23 Jan 2010 11:10:52 -0800, thinke365 wrote: > of course i have tried random package, but can this package generate > random sequence that satisfy possion distribution , normal distribution > and uniform distribution Please don't talk garbage. If you had really tried the random module, you would know the answer. What do you think random.uniform does? In an interactive Python session: >>> import random >>> help(random) and read. When you've done that, if you still have any specific questions, please ask. -- Steven From steve at REMOVE-THIS-cybersource.com.au Sat Jan 23 17:06:25 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 23 Jan 2010 22:06:25 GMT Subject: how to generate random numbers that satisfy certain distribution References: <7xtyuc1s65.fsf@ruckus.brouhaha.com> Message-ID: <036b5f64$0$1357$c3e8da3@news.astraweb.com> On Sat, 23 Jan 2010 12:29:22 -0800, Paul Rubin wrote: > Peter Chant writes: >> I remeber being told that adding up 12 random numbers in the range 0-1 >> (which is what most computer random number genertors at the time >> chucked out) and subtracted 6 gives a pretty good normal distribution. >> I think I did try it once and it failed, but I must have done something >> odd. > > That gives you a binomial distribution on 12 trials, which approximates > a normal distribution when the number of trials is large. 12 isn't too > bad. But there's a simpler way, the Box-Muller transform, that gives > you a pair drawn from a precisely normal distribution from two uniform > random samples: > > http://en.wikipedia.org/wiki/Box-Muller_transform The Box-Muller transform is reasonably simple, but you can't be serious that it is simpler than adding twelve random numbers and subtracting six! def almost_normal(): return sum([random.random() for _ in xrange(12)], -6) Not that I'm recommending that anyone use this binomial approximation for anything but the quickest and dirtiest uses, particularly since the random module already has two excellent normal distributions (gauss and normalvariate). -- Steven From no.email at nospam.invalid Sat Jan 23 17:10:10 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 23 Jan 2010 14:10:10 -0800 Subject: how to generate random numbers that satisfy certain distribution References: <7xtyuc1s65.fsf@ruckus.brouhaha.com> <036b5f64$0$1357$c3e8da3@news.astraweb.com> Message-ID: <7xr5pgxykd.fsf@ruckus.brouhaha.com> Steven D'Aprano writes: > The Box-Muller transform is reasonably simple, but you can't be serious > that it is simpler than adding twelve random numbers and subtracting six! If you want a normal distribution, using the Box-Muller transform is simpler because it spares you the complication of figuring out whether the 12-trial binomial approximation is close enough to produce reliable results for your specific application, which you obviously have to do if you are using the approximation for anything serious. It also involves writing less code than that list comprehension, since it is already implemented in the random module so you can just call it directly. From mrm at unknown.nospam Sat Jan 23 17:48:11 2010 From: mrm at unknown.nospam (Mr.M) Date: Sat, 23 Jan 2010 22:48:11 GMT Subject: PyArg_ParseTupleAndKeywords In-Reply-To: References: <91afd9e3-93a1-488d-8e83-299bfc222e24@22g2000yqr.googlegroups.com> Message-ID: MRAB ha scritto: > I think you're right. > I have rewritten my code, a piece at a time, and (and this is very annoying) now it works fine. I really can't understand what went wrong with my old code. Luca. From no.email at please.post Sat Jan 23 17:55:38 2010 From: no.email at please.post (kj) Date: Sat, 23 Jan 2010 22:55:38 +0000 (UTC) Subject: ISO module for binomial coefficients, etc. Message-ID: Before I go off to re-invent a thoroughly invented wheel, I thought I'd ask around for some existing module for computing binomial coefficient, hypergeometric coefficients, and other factorial-based combinatorial indices. I'm looking for something that can handle fairly large factorials (on the order of 10000!), using floating-point approximations as needed, and is smart about optimizations, memoizations, etc. TIA! ~K From tjreedy at udel.edu Sat Jan 23 18:04:01 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 23 Jan 2010 18:04:01 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> Message-ID: On 1/23/2010 12:17 PM, Steve Howell wrote: > Terry Reedy said: > > ''' > If you try writing a full patch, as I believe someone did, or at least > a > prototype thereof, when the idea was discussed, you might have a > better > idea of what the tradeoffs are and why it was rejected. > ''' > > I have to run, but tomorrow I will try to dig through python-dev > archives and find the patch. If anybody has hints on where to look > for it (anybody remember the author, for example?), it would be much > appreciated. The approach you outlined in your other response to me is, I believe, what was considered, investigated, and then rejected (by Guido, with agreement). The discussion may have been on the now-closed and (misspelled) pyk3 (?), or maybe on python-ideas, but my memory is more likely the former. I am sure that Raymond H. was involved also. > If the patch looks simple, I will try to pitch the idea that its time > has come. Now that the specification of the language itself is > frozen, I think there might be more room for improving > implementations. Also, I might be able to make the argument that > tradeoffs of memory vs. CPU vs. code complexity have different forces > in the 2010s. I am not opposed to a possible change, just hasty, ill-informed criticism. If there is not a PEP on this issue, it would be good to have one that recorded the proposal and the pros and cons, regardless of the outcome, so there would be something to refer people to. If that had been already done, it would have shortened this thread considerably. Terry Jan Reedy From tjreedy at udel.edu Sat Jan 23 18:08:37 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 23 Jan 2010 18:08:37 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> Message-ID: On 1/23/2010 2:02 PM, Roy Smith wrote: > In article, > Duncan Booth wrote: > >> Roy Smith wrote: >> >>> I'm looking at http://tinyurl.com/cdbwog. It shows all the operations >>> of a list. What it does not show is their cost. For pop(), it has a >>> note: >>> >>> "The pop() method is only supported by the list and array types. The >>> optional argument i defaults to -1, so that by default the last item >>> is removed and returned". >> >> The page you should probably be looking at is >> http://wiki.python.org/moin/TimeComplexity > > I was not aware of this page; thanks for pointing it out. Perhaps you could suggest on the tracker a place or places in the doc where this relatively new wiki page could be referred to. Perhaps in the introductory paragraphs of the Built-in Type section of the lib ref. Where would you have like to have found it? The page was added in response to threads like this one, but it obviously is more obscure than it should be. Terry Jan Reedy From python at mrabarnett.plus.com Sat Jan 23 18:10:00 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 23 Jan 2010 23:10:00 +0000 Subject: this customize sort did not work ,what's wrong? In-Reply-To: <27290014.post@talk.nabble.com> References: <27289860.post@talk.nabble.com> <27289922.post@talk.nabble.com> <27290014.post@talk.nabble.com> Message-ID: <4B5B81C8.9040106@mrabarnett.plus.com> thinke365 wrote: > jesus, now i fixed it, using odd lambda sort. > l.sort(lambda x,y: cmp(len(x), len(y))) > print l > > BUT I AM STILL CONFUSED WHY COSTOMIZED SORT FAILED TO SORT AS IT IS > PROGRAMMER! > > > thinke365 wrote: >> i mean the output i want is: >> [ [1, 2], [1, 2, 3, 4], [1, 2, 3, 4, 5]], that is sort according to the >> length of the list element >> >> thinke365 wrote: >>> l = list() >>> l1 = list((1, 2, 3, 4)) >>> l2 = list((1,2)) >>> l3 = list((1, 2, 3, 4, 5)) >>> l.append(l1) >>> l.append(l2) >>> l.append(l3) >>> print l >>> >>> def sort_by_list(E1, E2): >>> print len(E1), len(E2) >>> return len(list(E1)) > len(list(E2)) >>> >>> l.sort(cmp=sort_by_list) >>> print l >>> >>> output: >>> [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] >>> 2 4 >>> 5 2 >>> [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] >>> >>> the order of the elements in the list did not change! >>> When comparing [1, 2] with [1, 2, 3, 4] you want it to return -1 to say that it's shorter, so they should therefore be swapped, but you're returning 0 (False), which says it's the same length, so they're not swapped. This is also works: l.sort(lambda x,y: len(x) - len(y)) because the sort really only looks at the sign. From tjreedy at udel.edu Sat Jan 23 18:34:53 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 23 Jan 2010 18:34:53 -0500 Subject: enumerated while loop In-Reply-To: <227C7474-8CD1-4BC8-BDC3-ADBFAE01D052@roalddevries.nl> References: <227C7474-8CD1-4BC8-BDC3-ADBFAE01D052@roalddevries.nl> Message-ID: On 1/23/2010 9:44 AM, Roald de Vries wrote: > Dear all, > > I sometimes want to use an infinite while loop with access to the loop > index, like this: > > def naturals(): > i = 0 > while True: > yield i > y += 1 > > for i in naturals(): > print(i) > > I assume a function like 'naturals' already exists, or a similar > construction for the same purpose. But what is it called? itertools.count From tjreedy at udel.edu Sat Jan 23 18:38:29 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 23 Jan 2010 18:38:29 -0500 Subject: how can i know if a python object have a attribute such as 'attr1'? In-Reply-To: References: Message-ID: On 1/23/2010 10:56 AM, Arnaud Delobelle wrote: > thinke365 writes: > >> for example, i may define a python class: >> class A: >> def sayHello(): >> print 'hello' >> >> a = A() >> a.attr1 = 'hello' >> a.attr2 = 'bb' >> >> b = A() >> a.attr2 = 'aa' >> >> how can i know whether an object have an attribute named attr1? > > hasattr(a, 'attr1') or try: a.attr1 except AttributeError: pass Terry Jan Reedy From tjreedy at udel.edu Sat Jan 23 18:44:57 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 23 Jan 2010 18:44:57 -0500 Subject: this customize sort did not work ,what's wrong? In-Reply-To: <27289860.post@talk.nabble.com> References: <27289860.post@talk.nabble.com> Message-ID: On 1/23/2010 3:57 PM, thinke365 wrote: > > l = list() > l1 = list((1, 2, 3, 4)) > l2 = list((1,2)) > l3 = list((1, 2, 3, 4, 5)) > l.append(l1) > l.append(l2) > l.append(l3) > print l > > def sort_by_list(E1, E2): > print len(E1), len(E2) > return len(list(E1))> len(list(E2)) > > l.sort(cmp=sort_by_list) > print l > > output: > [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] > 2 4 > 5 2 > [[1, 2, 3, 4], [1, 2], [1, 2, 3, 4, 5]] > > the order of the elements in the list did not change! Here is what is right! >>> l = [ (1,2,3), (1,2), (1,2,3,4,5), () ] >>> l.sort(key=len) >>> l [(), (1, 2), (1, 2, 3), (1, 2, 3, 4, 5)] The cmp param is gone in 3.x in favor of key. Use the latter. Terry Jan Reedy From roy at panix.com Sat Jan 23 22:02:22 2010 From: roy at panix.com (Roy Smith) Date: Sat, 23 Jan 2010 22:02:22 -0500 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> Message-ID: In article , Terry Reedy wrote: > >> The page you should probably be looking at is > >> http://wiki.python.org/moin/TimeComplexity > > > > I was not aware of this page; thanks for pointing it out. > > Perhaps you could suggest on the tracker a place or places in the doc > where this relatively new wiki page could be referred to. Perhaps in the > introductory paragraphs of the Built-in Type section of the lib ref. > Where would you have like to have found it? I think the most logical place would have been right at the table of operations (http://tinyurl.com/cdbwog). From apt.shansen at gmail.com Sat Jan 23 22:14:47 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Sat, 23 Jan 2010 19:14:47 -0800 Subject: this customize sort did not work ,what's wrong? In-Reply-To: <27289860.post@talk.nabble.com> References: <27289860.post@talk.nabble.com> Message-ID: <7a9c25c21001231914h21ff1277ra6ca485ca4405754@mail.gmail.com> On Sat, Jan 23, 2010 at 12:57 PM, thinke365 wrote: > def sort_by_list(E1, E2): > print len(E1), len(E2) > return len(list(E1)) > len(list(E2)) > > l.sort(cmp=sort_by_list) > The cmp function is defined as returning one of three values, -1, 0 and 1. You are returning true or false, so things aren't getting sorted because you're not giving cmp what it is looking for. You could rewrite that as return cmp(len(list(E1)), len(list(E2))) if you want. However, cmp is going away in Py3. You can just do, instead: l.sort(key=len) And you'll get precisely what you're looking for. --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at REMOVE-THIS-cybersource.com.au Sat Jan 23 22:30:46 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 24 Jan 2010 03:30:46 GMT Subject: how to generate random numbers that satisfy certain distribution References: <7xtyuc1s65.fsf@ruckus.brouhaha.com> <036b5f64$0$1357$c3e8da3@news.astraweb.com> <7xr5pgxykd.fsf@ruckus.brouhaha.com> Message-ID: <036bab68$0$1357$c3e8da3@news.astraweb.com> On Sat, 23 Jan 2010 14:10:10 -0800, Paul Rubin wrote: > Steven D'Aprano writes: >> The Box-Muller transform is reasonably simple, but you can't be serious >> that it is simpler than adding twelve random numbers and subtracting >> six! > > If you want a normal distribution, using the Box-Muller transform is > simpler because it spares you the complication of figuring out whether > the 12-trial binomial approximation is close enough to produce reliable > results for your specific application, which you obviously have to do if > you are using the approximation for anything serious. But using Box-Miller gives you the complication of figuring out whether you care about being thread safety, which you have to do if you're doing anything serious. (See the comments in the random module for the gauss method). > It also involves > writing less code than that list comprehension, since it is already > implemented in the random module so you can just call it directly. By that logic, the Linux kernel is simpler than a function to add one to the argument, because the Linux kernel has already been implemented but you have to write your own add_one function. (Except in Forth, which usually comes with a word to add one to the number at the top of the stack.) We can agree that, given that the random module already has two normal distributions, there's no real point in using the binomial approximation. Everything else is quibbling. By the way, does anyone know why there is no Poisson random numbers in the module? The implementation is quite simple (but not as simple as the Linux kernel *wink*): def poisson(lambda_=1): L = math.exp(-lambda_) k = 0 p = 1 while 1: k += 1 p *= random.random() if p <= L: break return k-1 although this can be improved upon for large values of lambda_. -- Steven From python at rcn.com Sat Jan 23 23:00:37 2010 From: python at rcn.com (Raymond Hettinger) Date: Sat, 23 Jan 2010 20:00:37 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> Message-ID: <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> [Steve Howell] > Why wouldn't you get a competent C programmer simply make > list_ass_slice smart enough to make list.pop(0) O(1)? When this suggestion was discussed on python-dev years ago, it was rejected. One reason is that it was somewhat common for C code to access the list data structure directly (bypassing API accessor functions). Changing the list to have a starting offset would break existing C extensions. Another reason is that Guido is non-tolerant of space or time trade-offs for lists and tuples because they pervade the language and are heavily used internally. Any additional space or time requirement however small would impact the language performance as a whole. FWIW, that is also the reason that lists are not weak-referenceable (it would cost one extra pointer field per instance and that wasn't deemed to be worth it). > The brilliant computer scientist, Christian Heimes, provides the > answers, and I am paraphrasing here, of course: IMHO, Christian IS a brilliant computer scientist, so I'll ignore the rude intention and take the sentence literally. > ? 1) You can save 64 bits for every list by not storing an extra > pointer! > ? 2) You can simplify the CPython implementation! > ? 3) You can avoid the oh-so-expensive check "if ilow == 1" for all > operations that don't need this optimization! > > Sounds like two micro-optimizations to me (and a copout to boot). Micro or not, these reasons were part of Guido's rationale. Tim Peters and I also participated in the conversation and did not disagree. So, collections.deque() was born and the problem stopped being an issue. Also, Daniel Stuzbach has published a blist implementation that offers high performance insertions and deletions and fast handling of sparse lists. Raymond From suresh.amritapuri at gmail.com Sun Jan 24 00:57:44 2010 From: suresh.amritapuri at gmail.com (suresh.amritapuri) Date: Sat, 23 Jan 2010 21:57:44 -0800 (PST) Subject: image processing - inverse filtering Message-ID: <5bfefbb6-89a8-49f6-9f02-7d36dfbc0f16@c29g2000yqd.googlegroups.com> Hi, If I am using scipy.ndimage.gaussian_filter() for filtering an image, how to do the inverse filtering? In general how to do this using scipy.ndimage? Thanks suresh From reply-to at works.fine.invalid Sun Jan 24 03:09:24 2010 From: reply-to at works.fine.invalid (NickC) Date: 24 Jan 2010 08:09:24 GMT Subject: How to Embed PHP in HTML print Message-ID: <036becb4$0$1280$c3e8da3@news.astraweb.com> I am running a Python application under apache web server, executing as a cgi script. Most of the output is print statements that write HTML. I'd like to embed some PHP code within the HTML. The PHP is a gallery plugin script that talks to the core photo gallery application, written in php. The idea is that my python app can display a random image from a photo album, using the photo gallery application's plugin script. If I do: 'print ""', it doesn't work. I imagine because the web server never sees it as php code. Is there a way to do this? Some possible ideas: Is there a way I can get python to call functions within a PHP app? Perhaps write the plugin in a separate script in PHP, and include that page within my output so that apache recognises the page inclusion? How to get apache to pay attention to the output so it "wakes up" and does some server-side includes? Many thanks for any help. -- NickC From idrevetnom at free.fr Sun Jan 24 03:09:44 2010 From: idrevetnom at free.fr (idrevetnom) Date: Sun, 24 Jan 2010 09:09:44 +0100 Subject: ISO module for binomial coefficients, etc. References: Message-ID: <4b5bffc6$0$21815$426a74cc@news.free.fr> Maybe this could be of interest : http://tnt.math.metro-u.ac.jp/nzmath/manual/modules/combinatorial.html hope this helps Id From gagsl-py2 at yahoo.com.ar Sun Jan 24 04:54:51 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Sun, 24 Jan 2010 06:54:51 -0300 Subject: distutils not finding all of my pure python modules References: <543c08a0-30c8-49fc-882f-212e099bbd96@h9g2000yqa.googlegroups.com> Message-ID: En Thu, 21 Jan 2010 12:41:20 -0300, Jeremy escribi?: > from distutils.core import setup > > purePythonModules = ['regex', 'gnuFile'] > > setup(name='PythonForSafeguards', > version='0.9.1', > description = 'Python code for MCNP and Safeguards analysis.', > author = 'Jake the Snake', > author_email = 'something at blah.com', > packages = ['MCNP', 'Safeguards'], > url='http://lanl.gov', > py_modules = purePythonModules, > ) > > # ========================================= > > Everythin seems to work fine except the gnuFile.py script does not get > put into the distribution. I know the file exists in the same > directory as regex.py and has the same permissions. regex.py and gnuFile.py must be in the same directory as setup.py, ok? -- Gabriel Genellina From showell30 at yahoo.com Sun Jan 24 05:33:36 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sun, 24 Jan 2010 02:33:36 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> Message-ID: On Jan 23, 8:00?pm, Raymond Hettinger wrote: > [Steve Howell] > > > Why wouldn't you get a competent C programmer simply make > > list_ass_slice smart enough to make list.pop(0) O(1)? > > When this suggestion was discussed on python-dev years ago, > it was rejected. ?One reason is that it was somewhat > common for C code to access the list data structure directly > (bypassing API accessor functions). ?Changing the list to have > a starting offset would break existing C extensions. > > Another reason is that Guido is non-tolerant of space or time > trade-offs for lists and tuples because they pervade the language > and are heavily used internally. ?Any additional space or time > requirement however small would impact the language performance > as a whole. ?FWIW, that is also the reason that lists are not > weak-referenceable (it would cost one extra pointer field per > instance and that wasn't deemed to be worth it). > > > The brilliant computer scientist, Christian Heimes, provides the > > answers, and I am paraphrasing here, of course: > > IMHO, Christian IS a brilliant computer scientist, so I'll ignore > the rude intention and take the sentence literally. > You are also a brilliant computer scientist, despite the fact that you are defending a list implemenation that can't pop the first element off the list in O(1) time. From duncan.booth at invalid.invalid Sun Jan 24 05:37:51 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 24 Jan 2010 10:37:51 GMT Subject: How to Embed PHP in HTML print References: <036becb4$0$1280$c3e8da3@news.astraweb.com> Message-ID: NickC wrote: > Some possible ideas: > Is there a way I can get python to call functions within a PHP app? > Perhaps write the plugin in a separate script in PHP, and include that > page within my output so that apache recognises the page inclusion? How > to get apache to pay attention to the output so it "wakes up" and does > some server-side includes? > One option would be to ditch either the Python or the PHP and do everything in one language. Probably your simplest option is to get the web server to generate the PHP output separately. Since it's the same web server you can just use some javascript in the final web page to call in the PHP generated content with ajax. Alternatively, use urllib in Python to retrieve a page from the Apache server and insert that into its own output: that way you won't requrie javascript in the client, but that might be messy if you have authentication or sessions going on. From steve at REMOVE-THIS-cybersource.com.au Sun Jan 24 06:20:09 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 24 Jan 2010 11:20:09 GMT Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> Message-ID: <036c1967$0$1357$c3e8da3@news.astraweb.com> On Sun, 24 Jan 2010 02:33:36 -0800, Steve Howell wrote: > You are also a brilliant computer scientist, despite the fact that you > are defending a list implemenation that can't pop the first element off > the list in O(1) time. You say that like it's a bad thing. It's very simple: the trade-offs that the Python development team have deliberately chosen aren't the same trade-offs that you prefer. That doesn't make your trade-offs right and Python's wrong. They're just different, and if Python lists had your preferred implementation, I guarantee that somebody would be complaining about it right now. If you're serious about wanting O(1) pops from the start of the list, write your own list implementation and use it. You might even like to make it public, so others can use it as well. But please stop with the snide remarks and poorly disguised insults and back-handed compliments, it's getting tedious. Or just change your algorithm slightly -- it's not hard to turn an algorithm that pops from the start of a list to one that pops from the end of the list. -- Steven From tartley at tartley.com Sun Jan 24 06:28:00 2010 From: tartley at tartley.com (Jonathan Hartley) Date: Sun, 24 Jan 2010 03:28:00 -0800 (PST) Subject: py2exe deal with python command line inside a program References: Message-ID: <16c80c55-f06d-4682-9e7d-594471561f46@b10g2000yqa.googlegroups.com> On Jan 22, 7:35?pm, susan_kij... at yahoo.ca wrote: > Hi, > > I need to create a python subprogress, like this: > myProcess = subprocess.Popen([sys.executable, 'C:\myscript.py'], > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?env=env, stdin=subprocess.PIPE, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?stdout=subprocess.PIPE) > > sys.executable was printed out as ''C:\\Python25\\python.exe'', how > can I make this work in executable package through py2exe? > > I have to fix the following problems: > -Source code shouldn't exposed in an executable program > -Since python ?environment is not required when running an executable > program, how to deal with the situation that "C:\\Python25\ > \python.exe" is required as part of command? > > Thanks in advance! Hi. What does it do when you try to execute it with py2exe? Does it fail to run? What is the error? From clp2 at rebertia.com Sun Jan 24 06:35:41 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 24 Jan 2010 03:35:41 -0800 Subject: py2exe deal with python command line inside a program In-Reply-To: <16c80c55-f06d-4682-9e7d-594471561f46@b10g2000yqa.googlegroups.com> References: <16c80c55-f06d-4682-9e7d-594471561f46@b10g2000yqa.googlegroups.com> Message-ID: <50697b2c1001240335xd0b6487p5caab0ce91f7a3ff@mail.gmail.com> On Sun, Jan 24, 2010 at 3:28 AM, Jonathan Hartley wrote: > On Jan 22, 7:35?pm, susan_kij... at yahoo.ca wrote: >> Hi, >> >> I need to create a python subprogress, like this: >> myProcess = subprocess.Popen([sys.executable, 'C:\myscript.py'], >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?env=env, stdin=subprocess.PIPE, >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?stdout=subprocess.PIPE) >> >> sys.executable was printed out as ''C:\\Python25\\python.exe'', how >> can I make this work in executable package through py2exe? >> >> I have to fix the following problems: >> -Source code shouldn't exposed in an executable program >> -Since python ?environment is not required when running an executable >> program, how to deal with the situation that "C:\\Python25\ >> \python.exe" is required as part of command? >> >> Thanks in advance! > > Hi. What does it do when you try to execute it with py2exe? Does it > fail to run? What is the error? The subprocess call would fail utterly since sys.executable is apparently inaccurate for py2exe-generated executables. Cheers, Chris -- http://blog.rebertia.com From peloko45 at gmail.com Sun Jan 24 06:52:47 2010 From: peloko45 at gmail.com (Joan Miller) Date: Sun, 24 Jan 2010 03:52:47 -0800 (PST) Subject: ISC License References: <00eb248d-c9c9-430f-bc83-41ac865c5111@e11g2000yqe.googlegroups.com> Message-ID: <4c2d6744-b101-4504-bcfb-d2ec08ab68a0@m26g2000yqb.googlegroups.com> On 23 ene, 18:45, a... at pythoncraft.com (Aahz) wrote: > In article <00eb248d-c9c9-430f-bc83-41ac865c5... at e11g2000yqe.googlegroups.com>, > Joan Miller ? wrote: > > > > >There is a license approved by the OSI, the ISC License [1], which > >should be included in the PyPi classifiers [2]. > > >[1]https://www.isc.org/software/license > >http://www.opensource.org/licenses/isc-license.txt > >[2]http://pypi.python.org/pypi?%3Aaction=list_classifiers > > http://pypi.python.org/pypihas a link named "Bug Reports" > -- > Aahz (a... at pythoncraft.com) ? ? ? ? ? <*> ? ? ? ?http://www.pythoncraft.com/ > > import antigravity Added! http://sourceforge.net/tracker/?func=detail&aid=2938526&group_id=66150&atid=513503 From stefan_ml at behnel.de Sun Jan 24 07:02:48 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 24 Jan 2010 13:02:48 +0100 Subject: iterating lists In-Reply-To: <036b21fb$0$1357$c3e8da3@news.astraweb.com> References: <1mG6n.74060$4N3.53053@newsfe06.ams2> <036b21fb$0$1357$c3e8da3@news.astraweb.com> Message-ID: <4b5c36e8$0$6579$9b4e6d93@newsspool3.arcor-online.net> Steven D'Aprano, 23.01.2010 18:44: > On Sat, 23 Jan 2010 18:29:33 +0100, Roel Schroeven wrote: > >>> for w in l1[:]: #use copy of l1 for iteration >>> print(l1.pop()) #decomposite list >> I would prefer: >> >> while l1: >> print(l1.pop()) > > > I would prefer: > > for x in reversed(l1): > print(x) > l1[:] = [] > > > And garbage dispose of the entire list in one go, instead of an item at a > time. IIRC, that's what CPython does anyway, so no need to make this more complex than necessary. Stefan From davea at ieee.org Sun Jan 24 07:07:55 2010 From: davea at ieee.org (Dave Angel) Date: Sun, 24 Jan 2010 07:07:55 -0500 Subject: ISO module for binomial coefficients, etc. In-Reply-To: References: Message-ID: <4B5C381B.8070909@ieee.org> kj wrote: > Before I go off to re-invent a thoroughly invented wheel, I thought > I'd ask around for some existing module for computing binomial > coefficient, hypergeometric coefficients, and other factorial-based > combinatorial indices. I'm looking for something that can handle > fairly large factorials (on the order of 10000!), using floating-point > approximations as needed, and is smart about optimizations, > memoizations, etc. > > TIA! > > ~K > > You do realize that a standard. python floating point number cannot possibly approximate a number like 10000! Better use longs. I'd check out the gamma function, which matches factorial for integer arguments (plus or minus 1). DaveA From alfps at start.no Sun Jan 24 07:55:50 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 24 Jan 2010 13:55:50 +0100 Subject: ISO module for binomial coefficients, etc. In-Reply-To: References: Message-ID: * Dave Angel: > kj wrote: >> Before I go off to re-invent a thoroughly invented wheel, I thought >> I'd ask around for some existing module for computing binomial >> coefficient, hypergeometric coefficients, and other factorial-based >> combinatorial indices. I'm looking for something that can handle >> fairly large factorials (on the order of 10000!), using floating-point >> approximations as needed, and is smart about optimizations, >> memoizations, etc. >> >> TIA! >> >> ~K >> >> > You do realize that a standard. python floating point number cannot > possibly approximate a number like 10000! I think what kj is looking for, provided she/he is knowledgable about the subject, is code that does something like >>> from math import * >>> log_fac = 0 >>> for i in range( 1, 10000+1 ): ... log_fac += log( i, 10 ) ... >>> print( "10000! = {}e{}".format( 10**(log_fac % 1), int( log_fac ) ) ) 10000! = 2.84625968062e35659 >>> _ which turned out to be accurate to 10 digits. > Better use longs. That would involve incredible overhead. E.g., how many bytes for the number above? Those bytes translate into arithmetic overhead. > I'd check out the gamma function, which matches factorial for integer > arguments (plus or minus 1). Or, e.g., logarithms... ;-) Cheers & hth., - Alf From jim.waddle at boeing.com Sun Jan 24 08:54:20 2010 From: jim.waddle at boeing.com (Waddle, Jim) Date: Sun, 24 Jan 2010 05:54:20 -0800 Subject: ctypes for AIX Message-ID: <9C240725D2ABD949A80B46DD93AB107042E8D44101@XCH-NW-08V.nw.nos.boeing.com> I need to use ctypes with python running on AIX. It appears that python is being developed mostly for windows. Is there a policy concerning getting functions like ctypes working on AIX. Jim Waddle KIT-D 425-785-5194 From reply-to at works.fine.invalid Sun Jan 24 08:59:46 2010 From: reply-to at works.fine.invalid (NickC) Date: 24 Jan 2010 13:59:46 GMT Subject: How to Embed PHP in HTML print References: <036becb4$0$1280$c3e8da3@news.astraweb.com> Message-ID: <036c3ed3$0$1277$c3e8da3@news.astraweb.com> On Sun, 24 Jan 2010 10:37:51 +0000, Duncan Booth wrote: > content with ajax. Alternatively, use urllib in Python to retrieve a > page from the Apache server and insert that into its own output: that Thanks for hint on urllib. I shake my head in amazement with python sometimes. I'll write it here: print urllib.urlopen('http://myserver.com/phpscript.php').read() That's it. *One* line. The output from the php script is: random image and the one-liner seamlessly prints that to insert it into the html output. And I thought it would be hard; I should have known better. -- NickC From clp2 at rebertia.com Sun Jan 24 09:30:57 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 24 Jan 2010 06:30:57 -0800 Subject: ctypes for AIX In-Reply-To: <9C240725D2ABD949A80B46DD93AB107042E8D44101@XCH-NW-08V.nw.nos.boeing.com> References: <9C240725D2ABD949A80B46DD93AB107042E8D44101@XCH-NW-08V.nw.nos.boeing.com> Message-ID: <50697b2c1001240630r33845a1bl7f94b40a4f0c4436@mail.gmail.com> On Sun, Jan 24, 2010 at 5:54 AM, Waddle, Jim wrote: > I need to use ctypes with python running on AIX. According to the ctypes readme, ctypes is based on libffi, which according to its website, supports AIX for PowerPC64. So, perhaps you could state what the actual error or problem you're encountering is? It is theoretically possible the ctypes-bundled libffi is either outdated or had the AIX-specific bits removed; I don't know, I'm not a CPython dev. > It appears that python is being developed mostly for windows. No, not really; your statement is especially ironic considering one of Python's primary areas of use is for web applications as part of a LAMP stack. > Is there a policy concerning getting functions like ctypes working on AIX. No idea. Someone will probably chime in though. Cheers, Chris -- http://blog.rebertia.com From sg552 at hotmail.co.uk Sun Jan 24 10:08:15 2010 From: sg552 at hotmail.co.uk (Rotwang) Date: Sun, 24 Jan 2010 15:08:15 +0000 Subject: Default path for files Message-ID: Hi all, can anybody tell me whether there's a way to change the default location for files to be opened by open()? I'd like to be able to create files somewhere other than my Python folder without having to write the full path in the filename every time. Sorry if this is a stupid question, I don't know much about programming. From krister.svanlund at gmail.com Sun Jan 24 10:14:25 2010 From: krister.svanlund at gmail.com (Krister Svanlund) Date: Sun, 24 Jan 2010 16:14:25 +0100 Subject: Default path for files In-Reply-To: References: Message-ID: <2cf430a61001240714x21ddac6av46145a10eec434ab@mail.gmail.com> On Sun, Jan 24, 2010 at 4:08 PM, Rotwang wrote: > Hi all, can anybody tell me whether there's a way to change the default > location for files to be opened by open()? I'd like to be able to create > files somewhere other than my Python folder without having to write the full > path in the filename every time. Sorry if this is a stupid question, I don't > know much about programming. Check out http://docs.python.org/library/os.html and the function chdir it is what you are looking for. From rpjday at crashcourse.ca Sun Jan 24 10:28:26 2010 From: rpjday at crashcourse.ca (Robert P. J. Day) Date: Sun, 24 Jan 2010 10:28:26 -0500 (EST) Subject: how to list the attributes of a class, not an object? Message-ID: once again, probably a trivial question but i googled and didn't get an obvious solution. how to list the attributes of a *class*? eg., i was playing with dicts and noticed that the type returned by the keys() method was "dict_keys". so i'm now curious as to the attributes of the dict_keys class. but i don't know how to look at that without first *creating* such an instance, then asking for "dir(dk)". surely there's a simpler way just using the class name, no? rday p.s. any recommendations for the most concise reference sheet for python 3 that exists? being able to print off the entire language spec on two or four pages and tacking it up in front of me would be just ducky. thanks. -- ======================================================================== Robert P. J. Day Waterloo, Ontario, CANADA Linux Consulting, Training and Kernel Pedantry. Web page: http://crashcourse.ca Twitter: http://twitter.com/rpjday ======================================================================== From bblais at bryant.edu Sun Jan 24 10:29:20 2010 From: bblais at bryant.edu (Brian Blais) Date: Sun, 24 Jan 2010 10:29:20 -0500 Subject: some turtle questions Message-ID: <8F7BCECA-CD55-425D-A173-94D7F30DFB53@bryant.edu> Hello, I am trying to think of things to do with the turtle module with my students, and I have some ideas where I am not sure whether the turtle module can do it. 1) is there a way to determine the current screen pixel color? I am thinking about having the turtle go forward until it reaches an object, say a red circle. I can probably do this by making circle objects (drawn with turtles themselves) which know their own position, and check against this info. But I thought it might be useful also for the turtle to know. 2) is there a way to put a limit on the extent the turtle can travel? it seems I can keep moving off of the screen. Is there a way to make it so that a forward(50) command, at the edge, either raises an exception (at the wall) or simply doesn't move the turtle because of the limit? thanks! bb -- Brian Blais bblais at bryant.edu http://web.bryant.edu/~bblais http://bblais.blogspot.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From zuo at chopin.edu.pl Sun Jan 24 10:34:47 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sun, 24 Jan 2010 16:34:47 +0100 Subject: Consume an iterable In-Reply-To: References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <98b43693-3ba2-4d0d-b009-739d841a1c85@36g2000yqu.googlegroups.com> Message-ID: Dnia 23-01-2010 o 15:19:56 Peter Otten <__peter__ at web.de> napisa?(a): >>> def consume_islice(n, items): >>> next(islice(items, n, n), None) > > One problem: the above function doesn't consume the entire iterator like > the original example does for n=None. Passing sys.maxint instead is not > pretty. Not very pretty, but noticeably (though not dramatically) faster for n=None. Consider a modified version of the script from http://bugs.python.org/issue7764: import collections, sys from itertools import islice, repeat def consume0(iterator, n): # the old one collections.deque(islice(iterator, n), maxlen=0) def consume1(iterator, n): # similar to the primary proposal if n is None: collections.deque(iterator, maxlen=0) elif n != 0: next(islice(iterator, n-1, None), None) def consume2(iterator, n): # the approved proposal (see #7764) if n is None: collections.deque(iterator, maxlen=0) else: next(islice(iterator, n, n), None) def consume3(iterator, n): # with sys.maxint if n is None: n = sys.maxint # (maybe should be sys.maxsize instead?) next(islice(iterator, n, n), None) def test(fs): for consume in fs: iterator = iter(range(10)) consume(iterator, 3) rest = list(iterator) assert rest == range(3, 10), consume.__name__ iterator = iter(range(10)) consume(iterator, 0) rest = list(iterator) assert rest == range(10), consume.__name__ iterator = iter(range(10)) consume(iterator, None) rest = list(iterator) assert rest == [], consume.__name__ if __name__ == "__main__": from timeit import Timer fs = (consume0, consume1, consume2, consume3) test(fs) iterator = repeat(None, 1000) for consume in fs: print consume.__name__ for n in (10, 100, 1000, None): print "%6s:" % n, print Timer("consume(iterator, %s)" % n, "import collections, sys\n" "from __main__ import consume, iterator").timeit() print Results [Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41) [GCC 4.3.3] on linux2 pentium4 2.4 GHz]: consume0 10: 2.94313001633 100: 2.91833305359 1000: 2.93242096901 None: 2.90090417862 consume1 10: 1.80793309212 100: 1.7936270237 1000: 1.83439803123 None: 2.37652015686 consume2 10: 1.58784389496 100: 1.5890610218 1000: 1.58557391167 None: 2.37005710602 consume3 10: 1.6071870327 100: 1.61109304428 1000: 1.60717701912 None: 1.81885385513 Regards, *j -- Jan Kaliszewski (zuo) From sg552 at hotmail.co.uk Sun Jan 24 10:49:00 2010 From: sg552 at hotmail.co.uk (Rotwang) Date: Sun, 24 Jan 2010 15:49:00 +0000 Subject: Default path for files In-Reply-To: References: Message-ID: Krister Svanlund wrote: > On Sun, Jan 24, 2010 at 4:08 PM, Rotwang wrote: >> Hi all, can anybody tell me whether there's a way to change the default >> location for files to be opened by open()? I'd like to be able to create >> files somewhere other than my Python folder without having to write the full >> path in the filename every time. Sorry if this is a stupid question, I don't >> know much about programming. > > Check out http://docs.python.org/library/os.html and the function > chdir it is what you are looking for. Thank you. So would adding import os os.chdir() to site.py (or any other module which is automatically imported during initialisation) change the default location to every time I used Python? From zuo at chopin.edu.pl Sun Jan 24 10:56:42 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sun, 24 Jan 2010 16:56:42 +0100 Subject: how to list the attributes of a class, not an object? In-Reply-To: References: Message-ID: 24-01-2010, 16:28:26 Robert P. J. Day wrote > once again, probably a trivial question but i googled and didn't > get an obvious solution. how to list the attributes of a *class*? dir(type(an_obj)) or more reliable: list(vars(type(an_obj))) (dir() uses __dir__ which can be implemented in any way, and default implementation, accordinto to the docs, "attempts to produce the most relevant, rather than complete, information"). > eg., i was playing with dicts and noticed that the type returned by > the keys() method was "dict_keys". so i'm now curious as to the > attributes of the dict_keys class. but i don't know how to look at > that without first *creating* such an instance, then asking for > "dir(dk)". Why you bother about creating an instance? Just do it: list(vars(type({}.keys()))) or dir(type({}.keys())) if dir() satisfies you. Regards, *j -- Jan Kaliszewski (zuo) From alfps at start.no Sun Jan 24 10:56:46 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 24 Jan 2010 16:56:46 +0100 Subject: how to list the attributes of a class, not an object? In-Reply-To: References: Message-ID: * Robert P. J. Day: > once again, probably a trivial question but i googled and didn't > get an obvious solution. how to list the attributes of a *class*? > > eg., i was playing with dicts and noticed that the type returned by > the keys() method was "dict_keys". so i'm now curious as to the > attributes of the dict_keys class. but i don't know how to look at > that without first *creating* such an instance, then asking for > "dir(dk)". Like, dir( list ) where 'list' is the built-in type. There's a pretty-printer for that somewhere, but I can't recall. And as I also recommended in your thread "examining an initial, pristine python3 shell session", help( list ) or more generally help( "list" ) > surely there's a simpler way just using the class name, no? Yes. :-) > rday > > p.s. any recommendations for the most concise reference sheet for > python 3 that exists? being able to print off the entire language > spec on two or four pages and tacking it up in front of me would be > just ducky. thanks. Sorry, don't know. Cheers & hth., - Alf From lists at cheimes.de Sun Jan 24 11:01:27 2010 From: lists at cheimes.de (Christian Heimes) Date: Sun, 24 Jan 2010 17:01:27 +0100 Subject: Default path for files In-Reply-To: References: Message-ID: Rotwang wrote: > import os > os.chdir() > > to site.py (or any other module which is automatically imported during > initialisation) change the default location to every time I used > Python? First of all you shouldn't alter the site module ever! The optional sitecustomize module exists to make global changes. A library must never change the current working directory. It's up to the application to choose the right working directory. If you mess with the working directory in a library or global module like site you *will* break applications. Python has multiple ways to modify the list of importable locations, either globally, for the current user or the current application. Use them wisely! Christian From rpjday at crashcourse.ca Sun Jan 24 11:02:48 2010 From: rpjday at crashcourse.ca (Robert P. J. Day) Date: Sun, 24 Jan 2010 11:02:48 -0500 (EST) Subject: how to list the attributes of a class, not an object? In-Reply-To: References: Message-ID: On Sun, 24 Jan 2010, Alf P. Steinbach wrote: > * Robert P. J. Day: > > once again, probably a trivial question but i googled and didn't > > get an obvious solution. how to list the attributes of a *class*? > > > > eg., i was playing with dicts and noticed that the type returned by > > the keys() method was "dict_keys". so i'm now curious as to the > > attributes of the dict_keys class. but i don't know how to look at > > that without first *creating* such an instance, then asking for > > "dir(dk)". > > Like, > > dir( list ) > > where 'list' is the built-in type. > > There's a pretty-printer for that somewhere, but I can't recall. except that doesn't work for >>> dir(dict_keys) so what's the difference there? rday -- ======================================================================== Robert P. J. Day Waterloo, Ontario, CANADA Linux Consulting, Training and Kernel Pedantry. Web page: http://crashcourse.ca Twitter: http://twitter.com/rpjday ======================================================================== From __peter__ at web.de Sun Jan 24 11:05:58 2010 From: __peter__ at web.de (Peter Otten) Date: Sun, 24 Jan 2010 17:05:58 +0100 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <98b43693-3ba2-4d0d-b009-739d841a1c85@36g2000yqu.googlegroups.com> Message-ID: Jan Kaliszewski wrote: > Dnia 23-01-2010 o 15:19:56 Peter Otten <__peter__ at web.de> napisa?(a): > >>>> def consume_islice(n, items): >>>> next(islice(items, n, n), None) >> >> One problem: the above function doesn't consume the entire iterator like >> the original example does for n=None. Passing sys.maxint instead is not >> pretty. > > Not very pretty, but noticeably (though not dramatically) faster for > n=None. Consider a modified version of the script from > http://bugs.python.org/issue7764: > > import collections, sys > from itertools import islice, repeat > > def consume0(iterator, n): # the old one > collections.deque(islice(iterator, n), maxlen=0) > > def consume1(iterator, n): # similar to the primary proposal > if n is None: > collections.deque(iterator, maxlen=0) > elif n != 0: > next(islice(iterator, n-1, None), None) > > def consume2(iterator, n): # the approved proposal (see #7764) > if n is None: > collections.deque(iterator, maxlen=0) > else: > next(islice(iterator, n, n), None) > > def consume3(iterator, n): # with sys.maxint > if n is None: > n = sys.maxint # (maybe should be sys.maxsize instead?) > next(islice(iterator, n, n), None) > > def test(fs): > for consume in fs: > iterator = iter(range(10)) > consume(iterator, 3) > rest = list(iterator) > assert rest == range(3, 10), consume.__name__ > > iterator = iter(range(10)) > consume(iterator, 0) > rest = list(iterator) > assert rest == range(10), consume.__name__ > > iterator = iter(range(10)) > consume(iterator, None) > rest = list(iterator) > assert rest == [], consume.__name__ > > if __name__ == "__main__": > from timeit import Timer > > fs = (consume0, consume1, > consume2, consume3) > test(fs) > > iterator = repeat(None, 1000) > for consume in fs: > print consume.__name__ > for n in (10, 100, 1000, None): > print "%6s:" % n, > print Timer("consume(iterator, %s)" % n, > "import collections, sys\n" > "from __main__ import consume, > iterator").timeit() > print > > > Results [Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41) [GCC 4.3.3] > on linux2 pentium4 2.4 GHz]: > > consume0 > 10: 2.94313001633 > 100: 2.91833305359 > 1000: 2.93242096901 > None: 2.90090417862 > > consume1 > 10: 1.80793309212 > 100: 1.7936270237 > 1000: 1.83439803123 > None: 2.37652015686 > > consume2 > 10: 1.58784389496 > 100: 1.5890610218 > 1000: 1.58557391167 > None: 2.37005710602 > > consume3 > 10: 1.6071870327 > 100: 1.61109304428 > 1000: 1.60717701912 > None: 1.81885385513 > > > Regards, > *j > Don't the results look suspicious to you? Try measuring with iterator = iter([]) I'm sure you'll get the same result. An "easy" fix which introduces some constant overhead but keeps the results comparable: for consume in fs: print consume.__name__ for n in (10, 100, 1000, None): print "%6s:" % n, print Timer("consume(repeat(None, 1000), %s)" % n, "import collections, sys\n" "from __main__ import consume, repeat").timeit() print Just for fun, here's a variant of consume3 for the paranoid: _sentinel = object() def consume4(iterator, n): if n is None: n = sys.maxint while next(islice(iterator, n, n), _sentinel) is not _sentinel: pass Peter From zuo at chopin.edu.pl Sun Jan 24 11:17:07 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sun, 24 Jan 2010 17:17:07 +0100 Subject: PS. In-Reply-To: References: Message-ID: 24-01-2010, 16:56:42 Jan Kaliszewski wrote: > 24-01-2010, 16:28:26 Robert P. J. Day wrote > >> once again, probably a trivial question but i googled and didn't >> get an obvious solution. how to list the attributes of a *class*? > > dir(type(an_obj)) > > or more reliable: > > list(vars(type(an_obj))) > > (dir() uses __dir__ which can be implemented in any way, and default > implementation, accordinto to the docs, "attempts to produce the most > relevant, rather than complete, information"). I missed one important thing: * dir(a_type) mostly applies to attributes of a_type *and* of its base types/classes [1]. * vars() applies only to attributes of this particular type (AFAIN vars(sth) and sth.__dict__ are practically the same). Example: >>> class D(dict): pass ... >>> dir(D) ['__class__', '__contains__', '__delattr__', '__delitem__', '__dict__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'clear', 'copy', 'fromkeys', 'get', 'items', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values'] >>> list(vars(D)) ['__dict__', '__module__', '__weakref__', '__doc__'] Regards, *j [1] In Python 3.x *type* and *class* is practically the same. (though built-in ones are denoted as *types* rather than *classes* -- using this naming convention a *class* is simply a user-defined *type*). -- Jan Kaliszewski (zuo) From sg552 at hotmail.co.uk Sun Jan 24 11:33:25 2010 From: sg552 at hotmail.co.uk (Rotwang) Date: Sun, 24 Jan 2010 16:33:25 +0000 Subject: Default path for files In-Reply-To: References: Message-ID: Christian Heimes wrote: > Rotwang wrote: >> import os >> os.chdir() >> >> to site.py (or any other module which is automatically imported during >> initialisation) change the default location to every time I used >> Python? > > First of all you shouldn't alter the site module ever! The optional > sitecustomize module exists to make global changes. > > A library must never change the current working directory. It's up to > the application to choose the right working directory. If you mess with > the working directory in a library or global module like site you *will* > break applications. Python has multiple ways to modify the list of > importable locations, either globally, for the current user or the > current application. Use them wisely! > > Christian > OK, thanks. From alfps at start.no Sun Jan 24 11:37:41 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 24 Jan 2010 17:37:41 +0100 Subject: how to list the attributes of a class, not an object? In-Reply-To: References: Message-ID: * Robert P. J. Day: > On Sun, 24 Jan 2010, Alf P. Steinbach wrote: > >> * Robert P. J. Day: >>> once again, probably a trivial question but i googled and didn't >>> get an obvious solution. how to list the attributes of a *class*? >>> >>> eg., i was playing with dicts and noticed that the type returned by >>> the keys() method was "dict_keys". so i'm now curious as to the >>> attributes of the dict_keys class. but i don't know how to look at >>> that without first *creating* such an instance, then asking for >>> "dir(dk)". >> Like, >> >> dir( list ) >> >> where 'list' is the built-in type. >> >> There's a pretty-printer for that somewhere, but I can't recall. > > except that doesn't work for > > >>> dir(dict_keys) > > so what's the difference there? 'list' is a built-in type that by default is available. 'dict_keys' is a type that you're not meant to use directly, so it's not made available by default. The 'type' function yields the type of its argument, so you *can* do e.g. DictKeys = type( {}.keys() ) dir( DictKeys ) list( vars( DictKeys ) ) help( DictKeys ) It doesn't help much though because the only method of interrest is __iter__, which produces an iterator that you can use e.g. in a for loop or to construct a list, whatever. The relevant place to find out more about keys() is in the documentation. Cheers & hth., - Alf From zuo at chopin.edu.pl Sun Jan 24 11:49:32 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sun, 24 Jan 2010 17:49:32 +0100 Subject: Consume an iterable In-Reply-To: References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <98b43693-3ba2-4d0d-b009-739d841a1c85@36g2000yqu.googlegroups.com> Message-ID: > Don't the results look suspicious to you? Try measuring with > > iterator = iter([]) You are obviously right, my brain doesn't work well today :-( But the corret results even more distinctly support my thesis -- that for n=None using n=sys.maxint (consume3) is noticeably faster than 0-maxlen deque (consume1 and consume2): consume0 10: 4.06217813492 100: 8.45529103279 1000: 53.237226963 None: 54.0063519478 consume1 10: 2.59927105904 100: 3.47109603882 1000: 12.7196500301 None: 26.0995740891 consume2 10: 2.36225390434 100: 3.22979712486 1000: 12.5794699192 None: 28.5096430779 consume3 10: 2.39173388481 100: 3.43043398857 1000: 14.3361399174 None: 14.8560190201 Regards, *j -- Jan Kaliszewski (zuo) From zuo at chopin.edu.pl Sun Jan 24 11:58:54 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sun, 24 Jan 2010 17:58:54 +0100 Subject: how to list the attributes of a class, not an object? In-Reply-To: References: Message-ID: 24-01-2010, 17:37:41 Alf P. Steinbach wrote: > DictKeys = type( {}.keys() ) > > dir( DictKeys ) > list( vars( DictKeys ) ) > help( DictKeys ) > > It doesn't help much though because the only method of interrest is > __iter__ Not only. Please, consider: >>> dictkeys = type({}.keys()) >>> set(dir(dictkeys)).difference(dir(object)) {'__ror__', '__rsub__', '__and__', '__rand__', '__contains__', '__len__', '__iter__', '__or__', '__rxor__', '__xor__', '__sub__'} And e.g. comparision-related mehods (__gt__, __le__ and so on) also are more interensing in dict keys views than in plain object() instances... Regards, *j From python at bdurham.com Sun Jan 24 12:05:36 2010 From: python at bdurham.com (python at bdurham.com) Date: Sun, 24 Jan 2010 12:05:36 -0500 Subject: Splitting text at whitespace but keeping the whitespace in the returned list Message-ID: <1264352736.30280.1356298585@webmail.messagingengine.com> I need to parse some ASCII text into 'word' sized chunks of text AND collect the whitespace that seperates the split items. By 'word' I mean any string of characters seperated by whitespace (newlines, carriage returns, tabs, spaces, soft-spaces, etc). This means that my split text can contain punctuation and numbers - just not whitespace. The split( None ) method works fine for returning the word sized chunks of text, but destroys the whitespace separators that I need. Is there a variation of split() that returns delimiters as well as tokens? Thank you, Malcolm -------------- next part -------------- An HTML attachment was scrubbed... URL: From gagsl-py2 at yahoo.com.ar Sun Jan 24 12:09:03 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Sun, 24 Jan 2010 14:09:03 -0300 Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> <03698cd8$0$1357$c3e8da3@news.astraweb.com> Message-ID: En Fri, 22 Jan 2010 10:16:50 -0300, Alf P. Steinbach escribi?: > I get the impression that there's some message traffic that I don't see, > perhaps on the mailing list, since (a) I haven't seen that about > 'locals' pointed out by anyone else in this thread, and I think I've > read all messages in the thread, and (b) sometimes threads pop up that > start with a reply. > > For example, the recent thread "Covert number into string" started with > a /reply/ in my newreader, using EternalSeptember's NNTP host. > > It also starts with a reply in Google's archives, http://groups.google.com/group/comp.lang.python/browse_thread/thread/b8097d4de4a9c9b0/cb3a2e6ccd7736ef>. gmane (news.gmane.org) is a newsserver that acts as a bridge news <-> mailing lists. The post that started the thread you mention is available there, as well as those previous posts in this thread about the danger of using locals() -- Gabriel Genellina From python at mrabarnett.plus.com Sun Jan 24 12:24:40 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 24 Jan 2010 17:24:40 +0000 Subject: Splitting text at whitespace but keeping the whitespace in the returned list In-Reply-To: <1264352736.30280.1356298585@webmail.messagingengine.com> References: <1264352736.30280.1356298585@webmail.messagingengine.com> Message-ID: <4B5C8258.6090508@mrabarnett.plus.com> python at bdurham.com wrote: > I need to parse some ASCII text into 'word' sized chunks of text AND > collect the whitespace that seperates the split items. By 'word' I mean > any string of characters seperated by whitespace (newlines, carriage > returns, tabs, spaces, soft-spaces, etc). This means that my split text > can contain punctuation and numbers - just not whitespace. > > The split( None ) method works fine for returning the word sized chunks > of text, but destroys the whitespace separators that I need. > > Is there a variation of split() that returns delimiters as well as tokens? > I'd use the re module: >>> import re >>> re.split(r'(\s+)', "Hello world!") ['Hello', ' ', 'world!'] From __peter__ at web.de Sun Jan 24 12:24:49 2010 From: __peter__ at web.de (Peter Otten) Date: Sun, 24 Jan 2010 18:24:49 +0100 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <98b43693-3ba2-4d0d-b009-739d841a1c85@36g2000yqu.googlegroups.com> Message-ID: Jan Kaliszewski wrote: > But the corret results even more distinctly support my thesis -- that for > n=None using n=sys.maxint (consume3) is noticeably faster than 0-maxlen > deque (consume1 and consume2): That advantage may not survive the next release: http://svn.python.org/view/python/trunk/Modules/_collectionsmodule.c?r1=68145&r2=70296&pathrev=70296 Peter From alfps at start.no Sun Jan 24 12:25:47 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 24 Jan 2010 18:25:47 +0100 Subject: Symbols as parameters? In-Reply-To: References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> <03698cd8$0$1357$c3e8da3@news.astraweb.com> Message-ID: * Gabriel Genellina: > En Fri, 22 Jan 2010 10:16:50 -0300, Alf P. Steinbach > escribi?: > >> I get the impression that there's some message traffic that I don't >> see, perhaps on the mailing list, since (a) I haven't seen that about >> 'locals' pointed out by anyone else in this thread, and I think I've >> read all messages in the thread, and (b) sometimes threads pop up that >> start with a reply. >> >> For example, the recent thread "Covert number into string" started >> with a /reply/ in my newreader, using EternalSeptember's NNTP host. >> >> It also starts with a reply in Google's archives, > http://groups.google.com/group/comp.lang.python/browse_thread/thread/b8097d4de4a9c9b0/cb3a2e6ccd7736ef>. >> > > gmane (news.gmane.org) is a newsserver that acts as a bridge news <-> > mailing lists. The post that started the thread you mention is available > there, as well as those previous posts in this thread about the danger > of using locals() Thanks. I have gmane in Thunderbird (after all, IIRC gmane is Norwegian! :-) ), but only with one obscure list. Cheers, - Alf From lognaturel at gmail.com Sun Jan 24 12:35:42 2010 From: lognaturel at gmail.com (Helene Martin) Date: Sun, 24 Jan 2010 09:35:42 -0800 Subject: [Edu-sig] some turtle questions In-Reply-To: <8F7BCECA-CD55-425D-A173-94D7F30DFB53@bryant.edu> References: <8F7BCECA-CD55-425D-A173-94D7F30DFB53@bryant.edu> Message-ID: I'm almost sure that there's no way for a turtle to know anything about the background. That's an unfortunate limitation! As for putting a limit on a turtle's travel, you need to write an appropriate conditional. For example, if you want your turtle to stay within a 200x200 square centered around the origin and stop if it gets out, do something roughly like: while(math.abs(t.xcor()) < 100 and math.abs(t.ycor()) < 100): move turtle Of course, you could instead use if statements and simulate bouncing (if my turtle's x coordinate is beyond my bounding box, subtract from its x coordinate). Best, H?l?ne. Computer Science Teacher Garfield High School http://garfieldcs.com On Sun, Jan 24, 2010 at 7:29 AM, Brian Blais wrote: > Hello, > I am trying to think of things to do with the turtle module with my > students, and I have some ideas where I am not sure whether the turtle > module can do it. > 1) is there a way to determine the current screen pixel color? ?I am > thinking about having the turtle go forward until it reaches an object, say > a red circle. ?I can probably do this by making circle objects (drawn with > turtles themselves) which know their own position, and check against this > info. ?But I thought it might be useful also for the turtle to know. > 2) is there a way to put a limit on the extent the turtle can travel? ?it > seems I can keep moving off of the screen. ?Is there a way to make it so > that a forward(50) command, at the edge, either raises an exception (at the > wall) or simply doesn't move the turtle because of the limit? > > thanks! > > bb > > -- > Brian Blais > bblais at bryant.edu > http://web.bryant.edu/~bblais > http://bblais.blogspot.com/ > > > > _______________________________________________ > Edu-sig mailing list > Edu-sig at python.org > http://mail.python.org/mailman/listinfo/edu-sig > > From python at bdurham.com Sun Jan 24 12:36:33 2010 From: python at bdurham.com (python at bdurham.com) Date: Sun, 24 Jan 2010 12:36:33 -0500 Subject: Splitting text at whitespace but keeping the whitespace in the returned list In-Reply-To: <4B5C8258.6090508@mrabarnett.plus.com> References: <1264352736.30280.1356298585@webmail.messagingengine.com> <4B5C8258.6090508@mrabarnett.plus.com> Message-ID: <1264354593.1381.1356302439@webmail.messagingengine.com> MRAB, "MRAB" wrote: > >>> import re > >>> re.split(r'(\s+)', "Hello world!") > ['Hello', ' ', 'world!'] That was exactly (EXACTLY!) the solution I was looking for. Thank you! Malcolm From zuo at chopin.edu.pl Sun Jan 24 13:01:59 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sun, 24 Jan 2010 19:01:59 +0100 Subject: Consume an iterable In-Reply-To: References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <98b43693-3ba2-4d0d-b009-739d841a1c85@36g2000yqu.googlegroups.com> Message-ID: 24-01-2010, 18:24:49 Peter Otten <__peter__ at web.de> wrote: >> n=None using n=sys.maxint (consume3) is noticeably faster than 0-maxlen >> deque (consume1 and consume2): > > That advantage may not survive the next release: > > http://svn.python.org/view/python/trunk/Modules/_collectionsmodule.c?r1=68145&r2=70296&pathrev=70296 Nice. :) *j -- Jan Kaliszewski (zuo) From gd_usenet at spamfence.net Sun Jan 24 13:04:48 2010 From: gd_usenet at spamfence.net (=?UTF-8?Q?G=C3=BCnther?= Dietrich) Date: Sun, 24 Jan 2010 19:04:48 +0100 Subject: Default path for files References: Message-ID: <0neu27-csn.ln1@spamfence.net> Rotwang wrote: >> Check out http://docs.python.org/library/os.html and the function >> chdir it is what you are looking for. > >Thank you. So would adding > >import os >os.chdir() > >to site.py (or any other module which is automatically imported during >initialisation) change the default location to every time I used >Python? Don't change the library modules. It would catch you anytime when you expect it least. See for the environment variable PYTHONSTARTUP and the associated startup file. Best regards, G?nther From george.sakkis at gmail.com Sun Jan 24 13:11:07 2010 From: george.sakkis at gmail.com (George Sakkis) Date: Sun, 24 Jan 2010 10:11:07 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> <065e450f-2190-42f4-b9c6-4b95260092be@f12g2000yqn.googlegroups.com> <28103200.7M7XYOBnsK@beaureve.gmx.net> <9734672.qlcV6ekshW@beaureve.gmx.net> Message-ID: <6ddbbbfe-402d-4b87-8b61-a5042071dd08@e37g2000yqn.googlegroups.com> On Jan 22, 8:39?pm, Martin Drautzburg wrote: > Martin Drautzburg wrote: > >> with scope(): > >> ? ? # ... > >> ? ? # use up, down, left, right here > > >> # up, down, left, right no longer defined after the with block exits. > > Just looked it up again. It's a cool thing. Too bad my locals() hack > would still be required. The result would be less noisy (and actually > really beautiful) than the decorator implementation though. Thanks > again for pointing this out to me. Both in your example and by using a context manager, you can get away with not passing locals() explicitly by introspecting the stack frame. Here's a context manager that does the trick: from __future__ import with_statement from contextlib import contextmanager import sys @contextmanager def enums(*consts): # 2 levels up the stack to bypass the contextmanager frame f_locals = sys._getframe(2).f_locals new_names = set() reset_locals, updated_locals = {}, {} for const in consts: updated_locals[const] = const if const in f_locals: reset_locals[const] = f_locals[const] else: new_names.add(const) f_locals.update(updated_locals) try: yield finally: for name in new_names: del f_locals[name] f_locals.update(reset_locals) if __name__ == '__main__': def move(aDirection): print "moving " + aDirection up = "outerScopeUp" with enums("up", "down", "left", "right"): move(up) move(down) move(left) move(right) print "in the outer scope up is still:", up print "this should fail:" down Of course, as all other attempts to mess with locals() shown in this thread, this only "works" when locals() is globals(). If you try it within a function, it fails: def test(): up = "outerScopeUp" with enums("up", "down", "left", "right"): move(up) move(down) move(left) move(right) print "in the outer scope up is still:", up print "this should fail:" down ## XXX: doesn't work within a function test() So it's utility is pretty limited; a custom DSL is probably better suited to your problem. George From aahz at pythoncraft.com Sun Jan 24 13:14:29 2010 From: aahz at pythoncraft.com (Aahz) Date: 24 Jan 2010 10:14:29 -0800 Subject: simple cgi program References: <4b531bf9$0$1140$4fafbaef@reader1.news.tin.it> Message-ID: In article <4b531bf9$0$1140$4fafbaef at reader1.news.tin.it>, superpollo wrote: > >i would like to submit the following code for review. it is a simple >common gateway interface program, which uses the least possible >libraries for the sake of mechanism undertanding. You should probably factor this into two or three functions. I would not rename urllib here -- it's too common to want to use "url" for an actual URL. You should not use readline() -- just use read(). Which then obviates the need for the test (there's no real need for using CONTENT_LENGTH for this purpose IIRC). The value for "me" is wrong unless your cgi is installed as a top-level script. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From susan_kijiji at yahoo.ca Sun Jan 24 13:14:37 2010 From: susan_kijiji at yahoo.ca (im_smialing) Date: Sun, 24 Jan 2010 10:14:37 -0800 (PST) Subject: py2exe deal with python command line inside a program References: <16c80c55-f06d-4682-9e7d-594471561f46@b10g2000yqa.googlegroups.com> Message-ID: <8de9116a-0b09-4919-aec2-a9c41e37e177@y12g2000yqh.googlegroups.com> Hi Jonathan, Here is the traceback I got, 'test.py' is where "main" starts, and I replaced 'sys.executable' with string 'python': args: ['python', 'C:\\myscript.py'] Traceback (most recent call last): File "test.py", line 22, in File "subprocess.pyc", line 594, in __init__ File "subprocess.pyc", line 816, in _execute_child WindowsError: [Error 2] The system cannot find the file specified On Jan 24, 6:28?am, Jonathan Hartley wrote: > On Jan 22, 7:35?pm, susan_kij... at yahoo.ca wrote: > > > > > > > Hi, > > > I need to create a python subprogress, like this: > > myProcess = subprocess.Popen([sys.executable, 'C:\myscript.py'], > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?env=env, stdin=subprocess.PIPE, > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?stdout=subprocess.PIPE) > > > sys.executable was printed out as ''C:\\Python25\\python.exe'', how > > can I make this work in executable package through py2exe? > > > I have to fix the following problems: > > -Source code shouldn't exposed in an executable program > > -Since python ?environment is not required when running an executable > > program, how to deal with the situation that "C:\\Python25\ > > \python.exe" is required as part of command? > > > Thanks in advance! > > Hi. What does it do when you try to execute it with py2exe? Does it > fail to run? What is the error?- Hide quoted text - > > - Show quoted text - From susan_kijiji at yahoo.ca Sun Jan 24 13:25:34 2010 From: susan_kijiji at yahoo.ca (im_smialing) Date: Sun, 24 Jan 2010 10:25:34 -0800 (PST) Subject: py2exe deal with python command line inside a program References: <16c80c55-f06d-4682-9e7d-594471561f46@b10g2000yqa.googlegroups.com> Message-ID: <08003ed8-04f5-4f33-a49f-a143da5d3882@h2g2000yqj.googlegroups.com> On Jan 24, 6:35?am, Chris Rebert wrote: > On Sun, Jan 24, 2010 at 3:28 AM, Jonathan Hartley wrote: > > On Jan 22, 7:35?pm, susan_kij... at yahoo.ca wrote: > >> Hi, > > >> I need to create a python subprogress, like this: > >> myProcess = subprocess.Popen([sys.executable, 'C:\myscript.py'], > >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?env=env, stdin=subprocess.PIPE, > >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?stdout=subprocess.PIPE) > > >> sys.executable was printed out as ''C:\\Python25\\python.exe'', how > >> can I make this work in executable package through py2exe? > > >> I have to fix the following problems: > >> -Source code shouldn't exposed in an executable program > >> -Since python ?environment is not required when running an executable > >> program, how to deal with the situation that "C:\\Python25\ > >> \python.exe" is required as part of command? > > >> Thanks in advance! > > > Hi. What does it do when you try to execute it with py2exe? Does it > > fail to run? What is the error? > Thanks for pointing that, this time I try to use 'python' as the arg, I got an error: WindowsError: [Error 2] The system cannot find the file specified Because the subprocess is looking for a source code location, and which was hard coded, any suggestion to work out the issue? > The subprocess call would fail utterly since sys.executable is > apparently inaccurate for py2exe-generated executables. > > Cheers, > Chris > --http://blog.rebertia.com- Hide quoted text - > > - Show quoted text - From clp2 at rebertia.com Sun Jan 24 13:50:50 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 24 Jan 2010 10:50:50 -0800 Subject: py2exe deal with python command line inside a program In-Reply-To: <08003ed8-04f5-4f33-a49f-a143da5d3882@h2g2000yqj.googlegroups.com> References: <16c80c55-f06d-4682-9e7d-594471561f46@b10g2000yqa.googlegroups.com> <08003ed8-04f5-4f33-a49f-a143da5d3882@h2g2000yqj.googlegroups.com> Message-ID: <50697b2c1001241050g108c9055u8d62ff36b039801c@mail.gmail.com> On Sun, Jan 24, 2010 at 10:25 AM, im_smialing wrote: > On Jan 24, 6:35?am, Chris Rebert wrote: >> On Sun, Jan 24, 2010 at 3:28 AM, Jonathan Hartley wrote: >> > On Jan 22, 7:35?pm, susan_kij... at yahoo.ca wrote: >> >> Hi, >> >> >> I need to create a python subprogress, like this: >> >> myProcess = subprocess.Popen([sys.executable, 'C:\myscript.py'], >> >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?env=env, stdin=subprocess.PIPE, >> >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?stdout=subprocess.PIPE) >> >> >> sys.executable was printed out as ''C:\\Python25\\python.exe'', how >> >> can I make this work in executable package through py2exe? >> >> >> I have to fix the following problems: >> >> -Source code shouldn't exposed in an executable program >> >> -Since python ?environment is not required when running an executable >> >> program, how to deal with the situation that "C:\\Python25\ >> >> \python.exe" is required as part of command? >> >> >> Thanks in advance! >> >> > Hi. What does it do when you try to execute it with py2exe? Does it >> > fail to run? What is the error? >> > Thanks for pointing that, this time I try to use 'python' as the arg, > I got an error: > WindowsError: [Error 2] The system cannot find the file specified > > Because the subprocess is looking for a source code location, and > which was hard coded, any suggestion to work out the issue? Famous last words, but I think it's impossible. You'd have to somehow specify the Python interpreter inside the py2exe-generated executable as the program for subprocess.Popen to run, but I sincerely doubt that can be done. I would suggest something involving os.fork(), but you're clearly on Windows, which doesn't support fork(), so that option's out. The closest thing that leaves is execfile(): http://docs.python.org/library/functions.html#execfile Or you could relax your constraints: You could require Python to be installed on the system (I think there are ways to have your program's installer run the Python installer without any user interaction), or you could give up trying to keep the source code secret (it's illegal for your users to redistribute or modify it anyway, assuming you use the right EULA, and py2exe doesn't keep your sourcecode secret anyway - http://stackoverflow.com/questions/261638/how-do-i-protect-python-code) Cheers, Chris -- http://blog.rebertia.com >> The subprocess call would fail utterly since sys.executable is >> apparently inaccurate for py2exe-generated executables. From showell30 at yahoo.com Sun Jan 24 13:59:32 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sun, 24 Jan 2010 10:59:32 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <036c1967$0$1357$c3e8da3@news.astraweb.com> Message-ID: On Jan 24, 3:20?am, Steven D'Aprano wrote: > On Sun, 24 Jan 2010 02:33:36 -0800, Steve Howell wrote: > > You are also a brilliant computer scientist, despite the fact that you > > are defending a list implemenation that can't pop the first element off > > the list in O(1) time. > > You say that like it's a bad thing. It is. > It's very simple: the trade-offs that the Python development team have > deliberately chosen aren't the same trade-offs that you prefer. That > doesn't make your trade-offs right and Python's wrong. They're just > different, and if Python lists had your preferred implementation, I > guarantee that somebody would be complaining about it right now. > > If you're serious about wanting O(1) pops from the start of the list, > write your own list implementation and use it. You might even like to > make it public, so others can use it as well. But please stop with the > snide remarks and poorly disguised insults and back-handed compliments, > it's getting tedious. I will stop being snide, but I will be blunt, and if anybody interprets my criticism as an insult, so be it. The current algorithm is broken. It's a 20th century implementation of lists built on top of a 20th century memory manager. It's at least ten years behind the times. > Or just change your algorithm slightly -- it's not hard to turn an > algorithm that pops from the start of a list to one that pops from the > end of the list. > The fact that you are proposing to reverse a list to work around its performance deficiencies just confirms to me that the algorithm is broken. I will concede the fact that most of CPython's tradeoffs are driven by the limitations of the underlying memory manager. If realloc () allowed you to easily release memory from the front of a previously allocated block, we'd be talking about maybe a 10-line patch here, and it wouldn't impact even list_resize() in a meaningful way. Even with realloc()'s brokenness, you could improve pop(0) in a way that does not impact list access at all, and the patch would not change the time complexity of any operation; it would just add negligible extract bookkeeping within list_resize() and a few other places. The objection that the extra pointer would increase the size of list objects is totally 20th century thinking. It would be totally negligible for any real world program. From showell30 at yahoo.com Sun Jan 24 14:26:32 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sun, 24 Jan 2010 11:26:32 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> Message-ID: On Jan 23, 3:04?pm, Terry Reedy wrote: > On 1/23/2010 12:17 PM, Steve Howell wrote: > > > Terry Reedy said: > > > ''' > > If you try writing a full patch, as I believe someone did, or at least > > a > > prototype thereof, when the idea was discussed, you might have a > > better > > idea of what the tradeoffs are and why it was rejected. > > ''' > > > I have to run, but tomorrow I will try to dig through python-dev > > archives and find the patch. ?If anybody has hints on where to look > > for it (anybody remember the author, for example?), it would be much > > appreciated. > > The approach you outlined in your other response to me is, I believe, > what was considered, investigated, and then rejected (by Guido, with > agreement). The discussion may have been on the now-closed and > (misspelled) pyk3 (?), or maybe on python-ideas, but my memory is more > likely the former. I am sure that Raymond H. was involved also. > > > If the patch looks simple, I will try to pitch the idea that its time > > has come. ?Now that the specification of the language itself is > > frozen, I think there might be more room for improving > > implementations. ?Also, I might be able to make the argument that > > tradeoffs of memory vs. CPU vs. code complexity have different forces > > in the 2010s. > > I am not opposed to a possible change, just hasty, ill-informed > criticism. If there is not a PEP on this issue, it would be good to have > one that recorded the proposal and the pros and cons, regardless of the > outcome, so there would be something to refer people to. If that had > been already done, it would have shortened this thread considerably. > I think it's a good idea to write a PEP on this issue, and I will attempt a first draft. I think I should submit the first draft to python-ideas, correct? I expect the PEP to be at least initially, if not permanently, rejected, but it would not be an exercise in futility, as I agree it's good to record pros and cons of the proposal in one place. The PEP probably would not include a proposed patch until there was a little bit of consensus behind it, but it would not take me a lot of time to present the basic argument. Here is my sketch of what the PEP would look like. Proposal: Improve list's implementation so that deleting elements from the front of the list does not require an O(N) memmove operation. Rationale: Some Python programs that process lists have multiple methods that consume the first element of the list and pop it off. The pattern comes up with parsers in particular, but there are other examples. It is possible now, of course, to use a data structure in Python that has O(1) for deleting off the top of the list, but none of the alternatives fully replicate the benefits of list itself. Specification: Improving CPython's performance does not affect the language itself, so there are no bikeshed arguments to be had with respect to syntax, etc. Any patch would, of course, affect the performance of nearly every Python program in existence, so any patch would have to, at a bare minimum: 1) Not increase the time or memory complexity of any other list operation. 2) Not affect list access at all. 3) Minimally affect list operations that mutate the list. 4) Be reasonably simple within CPython itself. 5) Not be grossly wasteful of memory. Backwards Compatibility: See above. An implementation of this PEP would not change the definition of the language in any way, but it would have to minimally impact the performance of lists for the normal use cases. Implementation: There are two ways to make deleting the first item of the list run more efficiently. The most ambitious proposal is to fix the memory manager itself to allow the release of memory from the start of the chunk. The advantage of this proposal is that it would simplify the changes to list itself, and possibly have collateral benefits for other CPython internal data structures. The disadvantage of the proposal is that there is a strong tradition in CPython to use native memory management, particularly with respect to the fact that it runs on many platforms. The less ambitious proposal is to change the memory management scheme within list itself. There is already precedent in list_resize() to optimistically allocate memory, so it is not a great break from tradition to optimistically defer the release of memory. But it would complicate things. References: I would refer to this thread on comp.lang.python for discussion, and I would also try to dig up older threads on python-dev or elsewhere. From babedoudi at yahoo.fr Sun Jan 24 14:27:44 2010 From: babedoudi at yahoo.fr (=?ISO-8859-1?Q?R=E9mi?=) Date: Sun, 24 Jan 2010 20:27:44 +0100 Subject: Terminal application with non-standard print Message-ID: Hello everyone, I would like to do a Python application that prints data to stdout, but not the common way. I do not want the lines to be printed after each other, but the old lines to be replaced with the new ones, like wget does it for example (when downloading a file you can see the percentage increasing on a same line). I looked into the curses module, but this seems adapted only to do a whole application, and the terminal history is not visible anymore when the application starts. Any ideas? Thanks, Remi From aahz at pythoncraft.com Sun Jan 24 14:28:53 2010 From: aahz at pythoncraft.com (Aahz) Date: 24 Jan 2010 11:28:53 -0800 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> Message-ID: In article , Steve Howell wrote: > >Even with realloc()'s brokenness, you could improve pop(0) in a way >that does not impact list access at all, and the patch would not change >the time complexity of any operation; it would just add negligible >extract bookkeeping within list_resize() and a few other places. Again, your responsibility is to provide a patch and a spectrum of benchmarking tests to prove it. Then you would still have to deal with the objection that extensions use the list internals -- that might be an okay sell given the effort otherwise required to port extensions to Python 3, but that's not the way to bet. Have you actually read the discussions you were pointed at? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From invalid at invalid.invalid Sun Jan 24 14:33:16 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Sun, 24 Jan 2010 19:33:16 +0000 (UTC) Subject: Terminal application with non-standard print References: Message-ID: On 2010-01-24, R?mi wrote: > I would like to do a Python application that prints data to stdout, but > not the common way. I do not want the lines to be printed after each > other, but the old lines to be replaced with the new ones, like wget > does it for example (when downloading a file you can see the percentage > increasing on a same line). sys.stdout.write("Here's the first line") time.sleep(1) sys.stdout.write("\rAnd this line replaces it.") -- Grant From babedoudi at yahoo.fr Sun Jan 24 14:39:53 2010 From: babedoudi at yahoo.fr (=?ISO-8859-1?Q?R=E9mi?=) Date: Sun, 24 Jan 2010 20:39:53 +0100 Subject: Terminal application with non-standard print In-Reply-To: References: Message-ID: Thank you for your answer, but that does not work : the second line is printed after the first one. -- R?mi Grant Edwards wrote: > On 2010-01-24, R?mi wrote: > >> I would like to do a Python application that prints data to stdout, but >> not the common way. I do not want the lines to be printed after each >> other, but the old lines to be replaced with the new ones, like wget >> does it for example (when downloading a file you can see the percentage >> increasing on a same line). > > sys.stdout.write("Here's the first line") > time.sleep(1) > sys.stdout.write("\rAnd this line replaces it.") > From martin at v.loewis.de Sun Jan 24 14:50:15 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 24 Jan 2010 20:50:15 +0100 Subject: [ANN] Python 2.5.5 Release Candidate 2. Message-ID: <4B5CA477.1060807@v.loewis.de> On behalf of the Python development team and the Python community, I'm happy to announce the release candidate 2 of Python 2.5.5. This is a source-only release that only includes security fixes. The last full bug-fix release of Python 2.5 was Python 2.5.4. Users are encouraged to upgrade to the latest release of Python 2.6 (which is 2.6.4 at this point). This releases fixes issues with the logging and tarfile modules, and with thread-local variables. Since the release candidate 1, additional bugs have been fixed in the expat module. See the detailed release notes at the website (also available as Misc/NEWS in the source distribution) for details of bugs fixed. For more information on Python 2.5.5, including download links for various platforms, release notes, and known issues, please see: http://www.python.org/2.5.5 Highlights of the previous major Python releases are available from the Python 2.5 page, at http://www.python.org/2.5/highlights.html Enjoy this release, Martin Martin v. Loewis martin at v.loewis.de Python Release Manager (on behalf of the entire python-dev team) From showell30 at yahoo.com Sun Jan 24 14:53:15 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sun, 24 Jan 2010 11:53:15 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> Message-ID: <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> On Jan 24, 11:28?am, a... at pythoncraft.com (Aahz) wrote: > In article , > Steve Howell ? wrote: > > > > >Even with realloc()'s brokenness, you could improve pop(0) in a way > >that does not impact list access at all, and the patch would not change > >the time complexity of any operation; it would just add negligible > >extract bookkeeping within list_resize() and a few other places. > > Again, your responsibility is to provide a patch and a spectrum of > benchmarking tests to prove it. ?Then you would still have to deal with > the objection that extensions use the list internals -- that might be an > okay sell given the effort otherwise required to port extensions to > Python 3, but that's not the way to bet. Ok. > Have you actually read the discussions you were pointed at? I don't think anybody provided an actual link, but please correct me if I overlooked it. From invalid at invalid.invalid Sun Jan 24 14:53:32 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Sun, 24 Jan 2010 19:53:32 +0000 (UTC) Subject: Terminal application with non-standard print References: Message-ID: On 2010-01-24, R?mi wrote: > Thank you for your answer, but that does not work: Works fine for me. > the second line is printed after the first one. Not when I run it. There's not much more I can say given the level of detail you've provided. -- Grant From python at rcn.com Sun Jan 24 14:57:01 2010 From: python at rcn.com (Raymond Hettinger) Date: Sun, 24 Jan 2010 11:57:01 -0800 (PST) Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <98b43693-3ba2-4d0d-b009-739d841a1c85@36g2000yqu.googlegroups.com> Message-ID: <6f7ef176-f9ea-4975-86d0-009fd01b8308@h34g2000yqm.googlegroups.com> > ? ? ?def consume2(iterator, n): ?# the approved proposal (see #7764) > ? ? ? ? ?if n is None: > ? ? ? ? ? ? ?collections.deque(iterator, maxlen=0) > ? ? ? ? ?else: > ? ? ? ? ? ? ?next(islice(iterator, n, n), None) FWIW, the deque() approach becomes even faster in Py2.7 and Py3.1 which has a high-speed path for the case where maxlen is zero. Here's a snippet from Modules/_collectionsmodule.c: /* Run an iterator to exhaustion. Shortcut for the extend/extendleft methods when maxlen == 0. */ static PyObject* consume_iterator(PyObject *it) { PyObject *item; while ((item = PyIter_Next(it)) != NULL) { Py_DECREF(item); } Py_DECREF(it); if (PyErr_Occurred()) return NULL; Py_RETURN_NONE; } This code consumes an iterator to exhaustion. It is short, sweet, and hard to beat. Raymond From tjreedy at udel.edu Sun Jan 24 15:13:16 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 24 Jan 2010 15:13:16 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> Message-ID: On 1/24/2010 2:26 PM, Steve Howell wrote: > I think it's a good idea to write a PEP on this issue, and I will > attempt a first draft. I think I should submit the first draft to > python-ideas, correct? That is not a *requirement* for drafts in general, but it is a good idea for a community or community-person generated proposal, such as this one. > I expect the PEP to be at least initially, if not permanently, > rejected, Guido sometimes rejects 'no-chance' proposals without waiting to be asked, but he usually waits until the PEP author feels the issue is ripe and asks for a pronouncement. tjr From alfps at start.no Sun Jan 24 15:14:32 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 24 Jan 2010 21:14:32 +0100 Subject: Symbols as parameters? In-Reply-To: <6ddbbbfe-402d-4b87-8b61-a5042071dd08@e37g2000yqn.googlegroups.com> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> <065e450f-2190-42f4-b9c6-4b95260092be@f12g2000yqn.googlegroups.com> <28103200.7M7XYOBnsK@beaureve.gmx.net> <9734672.qlcV6ekshW@beaureve.gmx.net> <6ddbbbfe-402d-4b87-8b61-a5042071dd08@e37g2000yqn.googlegroups.com> Message-ID: Just top-posting for clarity. :-) up = "UP" left = "LEFT" down = "DOWN" right = "RIGHT" # This code is not guaranteed to work by the language specification. # But it is one way to do the solution I presented earlier in the thread. import sys def import_from( module_name ): local_variables = sys._getframe( 1 ).f_locals m = __import__( module_name, globals(), local_variables, "*" ) for name in local_variables: if not name.startswith( "_" ): local_variables[name] = getattr( m, name ) def move( direction ): print( "Moving " + str( direction ) ) def test(): up = "outerScopeUp" class using_directions: (up, left, down, right) = 4*[None]; import_from( "directions" ) move( up ) move( down ) move( left ) move( right ) print( "in the outer scope up is still: " + up ) print( "this should fail:" ) down test() Moving UP Moving DOWN Moving LEFT Moving RIGHT in the outer scope up is still: outerScopeUp this should fail: Traceback (most recent call last): File "locals.py", line 29, in test() File "locals.py", line 27, in test down NameError: global name 'down' is not defined Cheers & hth., - Alf * George Sakkis: > On Jan 22, 8:39 pm, Martin Drautzburg > wrote: > >> Martin Drautzburg wrote: >>>> with scope(): >>>> # ... >>>> # use up, down, left, right here >>>> # up, down, left, right no longer defined after the with block exits. >> Just looked it up again. It's a cool thing. Too bad my locals() hack >> would still be required. The result would be less noisy (and actually >> really beautiful) than the decorator implementation though. Thanks >> again for pointing this out to me. > > Both in your example and by using a context manager, you can get away > with not passing locals() explicitly by introspecting the stack frame. > Here's a context manager that does the trick: > > from __future__ import with_statement > from contextlib import contextmanager > import sys > > @contextmanager > def enums(*consts): > # 2 levels up the stack to bypass the contextmanager frame > f_locals = sys._getframe(2).f_locals > new_names = set() > reset_locals, updated_locals = {}, {} > for const in consts: > updated_locals[const] = const > if const in f_locals: > reset_locals[const] = f_locals[const] > else: > new_names.add(const) > f_locals.update(updated_locals) > try: > yield > finally: > for name in new_names: > del f_locals[name] > f_locals.update(reset_locals) > > > if __name__ == '__main__': > def move(aDirection): > print "moving " + aDirection > > up = "outerScopeUp" > with enums("up", "down", "left", "right"): > move(up) > move(down) > move(left) > move(right) > print "in the outer scope up is still:", up > print "this should fail:" > down > > > Of course, as all other attempts to mess with locals() shown in this > thread, this only "works" when locals() is globals(). If you try it > within a function, it fails: > > def test(): > up = "outerScopeUp" > with enums("up", "down", "left", "right"): > move(up) > move(down) > move(left) > move(right) > print "in the outer scope up is still:", up > print "this should fail:" > down > > ## XXX: doesn't work within a function > test() > > So it's utility is pretty limited; a custom DSL is probably better > suited to your problem. From tekion at gmail.com Sun Jan 24 15:26:30 2010 From: tekion at gmail.com (tekion) Date: Sun, 24 Jan 2010 12:26:30 -0800 (PST) Subject: easy_install error ... Message-ID: <5e9c94df-531f-4725-bcd5-87c603d8df1c@o26g2000vbd.googlegroups.com> All, I am running into issue with easy install error, see error below: python setup.py easy_install -m 'docutils==0.4' running easy_install error: Not a URL, existing file, or requirement spec: "'docutils==0.4'" any idea as to why? thanks. From babedoudi at yahoo.fr Sun Jan 24 15:35:09 2010 From: babedoudi at yahoo.fr (=?ISO-8859-1?Q?R=E9mi?=) Date: Sun, 24 Jan 2010 12:35:09 -0800 (PST) Subject: Terminal application with non-standard print References: Message-ID: <8901726e-d32d-48bc-8a6b-39a05b65a46d@g29g2000yqe.googlegroups.com> My apologies, I did not run the lines properly. Thanks, that works great now. If I understand well, \r erases the last line. How about erasing the previous lines? For example when writing sys.stdout.write("1\n2\n") sys.stdout.write("\r3") the "1" is still visible. -- R?mi On 24 jan, 20:53, Grant Edwards wrote: > On 2010-01-24, R?mi wrote: > > > Thank you for your answer, but that does not work: > > Works fine for me. > > > the second line is printed after the first one. > > Not when I run it. > > There's not much more I can say given the level of detail > you've provided. > > -- > Grant From no.email at nospam.invalid Sun Jan 24 15:44:25 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 24 Jan 2010 12:44:25 -0800 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> Message-ID: <7xaaw3b5cm.fsf@ruckus.brouhaha.com> Steve Howell writes: > Proposal: Improve list's implementation so that deleting elements from > the front of the list does not require an O(N) memmove operation. ... > It is possible now, of course, to use a data structure in > Python that has O(1) for deleting off the top of the list, but none of > the alternatives fully replicate the benefits of list itself. I think you are mostly referring to deque. Why don't you instead say what you think is wrong with using deque, and how deque can be improved? > See above. An implementation of this PEP would not change the > definition of the language in any way, but it would have to minimally > impact the performance of lists for the normal use cases. But you're talking about adding one or two words to EVERY list, and many normal use cases allocate a LOT of lists. Those use cases are likely more common than use cases that pop from the front of the list but for some reason can't use deque. > The most ambitious proposal is to fix the memory manager itself to > allow the release of memory from the start of the chunk. That's inappropriate given the memory fragmentation it would cause. Really, you're describing a problem that arises in a few programs but up til now, as far as I know, everyone has found deque to be an adequate solution. Your approach of snarling against list is not persuading anyone that list needs to be changed, because most everyone is satisfied with the existing solution. You might change approaches and discuss deque, what's wrong with it, and whether it can be fixed. Getting a change approved for deque is probably much easier than getting one approved for list, just because nowhere near as many things depend on deque's performance. And when discussing performance in this context, additive constants do matter. From alfps at start.no Sun Jan 24 15:44:36 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 24 Jan 2010 21:44:36 +0100 Subject: [Edu-sig] some turtle questions In-Reply-To: References: <8F7BCECA-CD55-425D-A173-94D7F30DFB53@bryant.edu> Message-ID: * Helene Martin: > I'm almost sure that there's no way for a turtle to know anything > about the background. That's an unfortunate limitation! The "background" for the turtle is just a Tkinter canvas. So yes, it's technically possible to inspect things there, since there is method to obtain the canvas. But unfortunately the canvas is not a bitmapped picture: it just maintains a list of objects to draw (vector graphics, metafile graphics). And so there's no practical way to obtain the "current screen pixel color" as Brian asks for: there are no pixels in that canvas... Blatant plug: I included some perhaps interesting turtle examples in ch 2 of my writings at . It starts with some silly figures, then graph drawing, then some recursive figures (idealized tree, C-curve, dragoncurve). Helene: since I'm not on the original list, could you perhaps forward to that list or to the original poster? Thanks, - Alf > As for putting a limit on a turtle's travel, you need to write an > appropriate conditional. For example, if you want your turtle to stay > within a 200x200 square centered around the origin and stop if it gets > out, do something roughly like: > > while(math.abs(t.xcor()) < 100 and math.abs(t.ycor()) < 100): > move turtle > > Of course, you could instead use if statements and simulate bouncing > (if my turtle's x coordinate is beyond my bounding box, subtract from > its x coordinate). > > Best, > > H?l?ne. > Computer Science Teacher > Garfield High School > http://garfieldcs.com > > On Sun, Jan 24, 2010 at 7:29 AM, Brian Blais wrote: >> Hello, >> I am trying to think of things to do with the turtle module with my >> students, and I have some ideas where I am not sure whether the turtle >> module can do it. >> 1) is there a way to determine the current screen pixel color? I am >> thinking about having the turtle go forward until it reaches an object, say >> a red circle. I can probably do this by making circle objects (drawn with >> turtles themselves) which know their own position, and check against this >> info. But I thought it might be useful also for the turtle to know. >> 2) is there a way to put a limit on the extent the turtle can travel? it >> seems I can keep moving off of the screen. Is there a way to make it so >> that a forward(50) command, at the edge, either raises an exception (at the >> wall) or simply doesn't move the turtle because of the limit? >> >> thanks! From no.email at nospam.invalid Sun Jan 24 15:45:19 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 24 Jan 2010 12:45:19 -0800 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <98b43693-3ba2-4d0d-b009-739d841a1c85@36g2000yqu.googlegroups.com> <6f7ef176-f9ea-4975-86d0-009fd01b8308@h34g2000yqm.googlegroups.com> Message-ID: <7x636rb5b4.fsf@ruckus.brouhaha.com> Raymond Hettinger writes: > This code consumes an iterator to exhaustion. > It is short, sweet, and hard to beat. I've always used sum(1 for x in iterator) or some such. From davea at ieee.org Sun Jan 24 16:11:03 2010 From: davea at ieee.org (Dave Angel) Date: Sun, 24 Jan 2010 16:11:03 -0500 Subject: ISO module for binomial coefficients, etc. In-Reply-To: References: Message-ID: <4B5CB767.6010100@ieee.org> Alf P. Steinbach wrote: >
* Dave > Angel: >> kj wrote: >>> Before I go off to re-invent a thoroughly invented wheel, I thought >>> I'd ask around for some existing module for computing binomial >>> coefficient, hypergeometric coefficients, and other factorial-based >>> combinatorial indices. I'm looking for something that can handle >>> fairly large factorials (on the order of 10000!), using floating-point >>> approximations as needed, and is smart about optimizations, >>> memoizations, etc. >>> >>> TIA! >>> >>> ~K >>> >>> >> You do realize that a standard. python floating point number cannot >> possibly approximate a number like 10000! > > I think what kj is looking for, provided she/he is knowledgable about > the subject, is code that does something like > > >>> from math import * > >>> log_fac = 0 > >>> for i in range( 1, 10000+1 ): > ... log_fac += log( i, 10 ) > ... > >>> print( "10000! = {}e{}".format( 10**(log_fac % 1), int( log_fac > ) ) ) > 10000! = 2.84625968062e35659 > >>> _ > > which turned out to be accurate to 10 digits. > > >> Better use longs. > > That would involve incredible overhead. E.g., how many bytes for the > number above? Those bytes translate into arithmetic overhead. > About 14k. > >> I'd check out the gamma function, which matches factorial for integer >> arguments (plus or minus 1). > > Or, e.g., logarithms... ;-) > > > Cheers & hth., > > - Alf > I didn't think of simply summing the logs. I did have some optimizations in mind for the multiply of the longs. If you do lots of partial products, you can do a good bit of the work with smaller numbers, and only get to longs when those partial products get big enough. You could also use scaling when the numbers do start getting bigger. But I still think there must be code for the gamma function that would be quicker. But I haven't chased that lead. DaveA From susan_kijiji at yahoo.ca Sun Jan 24 16:17:44 2010 From: susan_kijiji at yahoo.ca (im_smiling) Date: Sun, 24 Jan 2010 13:17:44 -0800 (PST) Subject: py2exe deal with python command line inside a program References: <16c80c55-f06d-4682-9e7d-594471561f46@b10g2000yqa.googlegroups.com> <08003ed8-04f5-4f33-a49f-a143da5d3882@h2g2000yqj.googlegroups.com> Message-ID: <489fc373-e245-4519-a374-b836f480c553@k35g2000yqb.googlegroups.com> Thanks for taking time to help me. If I use either way, requiring python being installed, or allow source codes exposure, it seems in some degree, there's not necessarily to make the executable package any more, which is very frustrating.... Suppose I take the first way, python environment must be on a machine, how can I have py2exe find a specific file path, for instance as below, 2 up levels from running director, how to find myscript.py in py2exe? Since in customers' machines, C:\dev\mysricpt.py doesn't exist.... python environment: running directory - C:\dev\level1\level2\test.py, suprocess directory-C:\dev\mysricpt.py py2exe C:\dev\level1\level2\dist On Jan 24, 1:50?pm, Chris Rebert wrote: > On Sun, Jan 24, 2010 at 10:25 AM, im_smialing wrote: > > On Jan 24, 6:35?am, Chris Rebert wrote: > >> On Sun, Jan 24, 2010 at 3:28 AM, Jonathan Hartley wrote: > >> > On Jan 22, 7:35?pm, susan_kij... at yahoo.ca wrote: > >> >> Hi, > > >> >> I need to create a python subprogress, like this: > >> >> myProcess = subprocess.Popen([sys.executable, 'C:\myscript.py'], > >> >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?env=env, stdin=subprocess.PIPE, > >> >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?stdout=subprocess.PIPE) > > >> >> sys.executable was printed out as ''C:\\Python25\\python.exe'', how > >> >> can I make this work in executable package through py2exe? > > >> >> I have to fix the following problems: > >> >> -Source code shouldn't exposed in an executable program > >> >> -Since python ?environment is not required when running an executable > >> >> program, how to deal with the situation that "C:\\Python25\ > >> >> \python.exe" is required as part of command? > > >> >> Thanks in advance! > > >> > Hi. What does it do when you try to execute it with py2exe? Does it > >> > fail to run? What is the error? > > > Thanks for pointing that, this time I try to use 'python' as the arg, > > I got an error: > > WindowsError: [Error 2] The system cannot find the file specified > > > Because the subprocess is looking for a source code location, and > > which was hard coded, any suggestion to work out the issue? > > Famous last words, but I think it's impossible. You'd have to somehow > specify the Python interpreter inside the py2exe-generated executable > as the program for subprocess.Popen to run, but I sincerely doubt that > can be done. > > I would suggest something involving os.fork(), but you're clearly on > Windows, which doesn't support fork(), so that option's out. > > The closest thing that leaves is execfile():http://docs.python.org/library/functions.html#execfile > > Or you could relax your constraints: > You could require Python to be installed on the system (I think there > are ways to have your program's installer run the Python installer > without any user interaction), or you could give up trying to keep the > source code secret (it's illegal for your users to redistribute or > modify it anyway, assuming you use the right EULA, and py2exe doesn't > keep your sourcecode secret anyway -http://stackoverflow.com/questions/261638/how-do-i-protect-python-code) > > Cheers, > Chris > --http://blog.rebertia.com > > >> The subprocess call would fail utterly since sys.executable is > >> apparently inaccurate for py2exe-generated executables. > > From news123 at free.fr Sun Jan 24 16:19:38 2010 From: news123 at free.fr (News123) Date: Sun, 24 Jan 2010 22:19:38 +0100 Subject: start .pyo files with doubleclick on windows Message-ID: <4b5cb96a$0$23933$426a74cc@news.free.fr> Hi, I'd like to start .pyo files under windows with a double click. (I know I can just write a .bat wrapper, but somehow it would be more fun to start with a direct double click) Currently this works if the file does not import any other .pyo file. The problem is, that a dobleclick performs a "python.exe myfile.pyo" In order to import precompiled .pyo files however it seems, that "python.exe -OO myfile.pyo" would be required. Is there any trick to do this under windows? The only trick, that I can think of is to write a small wrapper executable, that would then call python -OO. Wouldn't it be useful if python searched also for .pyo files when trying to import and being invoked with "python.exe mycode.pyo" thanks in advance for any ideas suggestion From alfps at start.no Sun Jan 24 16:25:53 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 24 Jan 2010 22:25:53 +0100 Subject: start .pyo files with doubleclick on windows In-Reply-To: <4b5cb96a$0$23933$426a74cc@news.free.fr> References: <4b5cb96a$0$23933$426a74cc@news.free.fr> Message-ID: * News123: > Hi, > > > I'd like to start .pyo files under windows with a double click. > (I know I can just write a .bat wrapper, but somehow it would be more > fun to start with a direct double click) > > > Currently this works if the file does not import any other .pyo file. > > > The problem is, that a dobleclick performs a > "python.exe myfile.pyo" > > In order to import precompiled .pyo files however it seems, that > "python.exe -OO myfile.pyo" > > would be required. > > > Is there any trick to do this under windows? > The only trick, that I can think of is to write a small wrapper > executable, that would then call python -OO. > > > Wouldn't it be useful if python searched also for .pyo files when trying > to import and being invoked with > > "python.exe mycode.pyo" > > > thanks in advance for any ideas suggestion C:\> assoc .pyo .pyo=Python.CompiledFile C:\> ftype python.compiledfile python.compiledfile="C:\Program Files\cpython\python31\python.exe" "%1" %* C:\> _ Use ftype to change the association. Cheers & hth., - Alf From nospam at nospam.com Sun Jan 24 16:33:39 2010 From: nospam at nospam.com (Gilles Ganault) Date: Sun, 24 Jan 2010 22:33:39 +0100 Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: On Fri, 22 Jan 2010 13:17:44 +0100, Gilles Ganault wrote: >To avoid users from creating login names that start with digits in >order to be listed at the top, I'd like to sort the list differently >every minute so that it'll start with the next letter, eg. display the >list from A...Zdigits the first time, then B...ZAdigits, etc. Thanks everyone for the great feedback. I ended up using - a list instead of a dictionary to hold the list of names - use the random library to pick a character from which to start listing names (simpler than saving the next letter into a file, and starting from this the next time the loop is gone through) For those interested, here's some working code: ========== import random connected = ["_test","-test","0test","1test","Aa","Aab","Bb","Bbbbb","Cc","Ccaaaaaa"] connected.sort() #Fill list with non-duplicate first letter of all items characters=[] for name in connected: char = name[0] #if this character not in list, add to array if not char in characters: characters.append(char) #Pick a random character from which to start iterating index = random.randint(0,len(characters)-1) startch = characters[index] print "Will start from %s" % startch #Go through list starting with items that start with 'starch', and rotate from beginning result = [name for name in connected if name[0] >= startch] + [name for name in connected if name[0] < startch] print result ========== Thanks again. From casevh at gmail.com Sun Jan 24 16:49:36 2010 From: casevh at gmail.com (casevh) Date: Sun, 24 Jan 2010 13:49:36 -0800 (PST) Subject: ISO module for binomial coefficients, etc. References: Message-ID: On Jan 23, 2:55?pm, kj wrote: > Before I go off to re-invent a thoroughly invented wheel, I thought > I'd ask around for some existing module for computing binomial > coefficient, hypergeometric coefficients, and other factorial-based > combinatorial indices. ?I'm looking for something that can handle > fairly large factorials (on the order of 10000!), using floating-point > approximations as needed, and is smart about optimizations, > memoizations, etc. > > TIA! > > ~K If you need exact values, gmpy (http://code.google.com/p/gmpy/) has basic, but fast, support for calculating binomial coefficients and factorials. If you are floating point approximations, check out mpmath (http://code.google.com/p/mpmath/). casevh From daniel at stutzbachenterprises.com Sun Jan 24 16:51:00 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Sun, 24 Jan 2010 15:51:00 -0600 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> Message-ID: On Sun, Jan 24, 2010 at 1:53 PM, Steve Howell wrote: > I don't think anybody provided an actual link, but please correct me > if I overlooked it. I have to wonder if my messages are all ending up in your spam folder for some reason. :-) PEP 3128 (which solves your problem, but not using the implementation you suggest) http://www.python.org/dev/peps/pep-3128/ Implementation as an extension module: http://pypi.python.org/pypi/blist/ Related discussion: http://mail.python.org/pipermail/python-3000/2007-April/006757.html http://mail.python.org/pipermail/python-3000/2007-May/007491.html Detailed performance comparison: http://stutzbachenterprises.com/performance-blist I maintain a private fork of Python 3 with the blist replacing the regular list, as a way of rigorously testing the blist implementation. Although I originally proposed a PEP, I am content to have the blist exist as a third-party module. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC From news123 at free.fr Sun Jan 24 17:36:53 2010 From: news123 at free.fr (News123) Date: Sun, 24 Jan 2010 23:36:53 +0100 Subject: start .pyo files with doubleclick on windows In-Reply-To: References: <4b5cb96a$0$23933$426a74cc@news.free.fr> Message-ID: <4b5ccb85$0$22008$426a74cc@news.free.fr> Hi Alf, Alf P. Steinbach wrote: > * News123: >> Hi, >> >> >> I'd like to start .pyo files under windows with a double click. > > C:\> assoc .pyo > .pyo=Python.CompiledFile > > C:\> ftype python.compiledfile > python.compiledfile="C:\Program Files\cpython\python31\python.exe" "%1" %* > > C:\> _ > > > Use ftype to change the association. Thanks a lot, I leared something new about Windows What I did now is this: assoc .pyo=Python.CompiledOptimizedFile ftype Python.CompiledOptimizedFile="C:\Python26\python.exe" -OO "%1" %* bye N > > > Cheers & hth., > > - Alf From ladasky at my-deja.com Sun Jan 24 17:53:14 2010 From: ladasky at my-deja.com (John Ladasky) Date: Sun, 24 Jan 2010 14:53:14 -0800 (PST) Subject: Broken Python 2.6 installation on Ubuntu Linux 8.04 Message-ID: Hello everyone, I've posted this same question over on ubuntuforums.org, so I'm trying to get help in all of the logical places. I'm running Ubuntu Linux 8.04 (Hardy) on a fairly new x86 box, with two hard disks in a software RAID 1 configuration. Hardy comes with Python 2.5 as a standard package, but not 2.6. I would really like to have the "combinations" function that is included with itertools in Python 2.6. I tried writing a combinations function of my own, but it's SLOW and uses a HUGE amount of memory. So, in my case the Linux Synaptic Package Manager cannot be used to install Python 2.6. I therefore attempted a manual installation. I downloaded the Linux tarball for Python 2.6 from python.org. I followed the installation instructions, and they appeared to execute fine. But when I started IDLE, I still had Python 2.5. No good. I do most of my editing in SCIte. Apparently SCIte knows that I have Python 2.6, and is trying to use it. Alas, my programs depend on extra Python packages such as biopython, numpy and matplotlib. My Python 2.6 distro does not have these yet. None of my programs will run from SCIte! Looking down into the details of the install, I've discovered that Hardy placed the Python 2.4 and 2.5 executables in /usr/bin. My Python 2.6 installation ended up in /usr/local/bin. This may be contributing to my problems. Given the mess I've made by trying to just install plain-old Python, I don't know whether I should attempt to back out, or to press on. Can I convince IDLE to connect to Python 2.6? How do I manually install site packages? Alternately, I COULD upgrade my Ubuntu Linux to 9.4 (Jaunty) or 9.10 (Karmic). Python 2.6 comes standard with both of these. But this is why I mentioned that my storage is RAID1. Apparently, upgrading with RAID present is a serious headache. The Linux wizards are supposed to be fixing these problems in the next release, due in April. I could wait, I suppose. In the mean time, I may have to uninstall Python 2.6 and get my 2.5 running again. I have not found any instructions for how to do that. Help! From arnodel at googlemail.com Sun Jan 24 17:55:53 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sun, 24 Jan 2010 22:55:53 +0000 Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: Gilles Ganault writes: > On Fri, 22 Jan 2010 13:17:44 +0100, Gilles Ganault > wrote: >>To avoid users from creating login names that start with digits in >>order to be listed at the top, I'd like to sort the list differently >>every minute so that it'll start with the next letter, eg. display the >>list from A...Zdigits the first time, then B...ZAdigits, etc. > > Thanks everyone for the great feedback. I ended up using > - a list instead of a dictionary to hold the list of names > - use the random library to pick a character from which to start > listing names (simpler than saving the next letter into a file, and > starting from this the next time the loop is gone through) > > For those interested, here's some working code: > > ========== > import random > > connected = > ["_test","-test","0test","1test","Aa","Aab","Bb","Bbbbb","Cc","Ccaaaaaa"] > connected.sort() > > #Fill list with non-duplicate first letter of all items > characters=[] > for name in connected: > char = name[0] > #if this character not in list, add to array > if not char in characters: > characters.append(char) characters = list(set(name[0] for name in connected)) > #Pick a random character from which to start iterating > index = random.randint(0,len(characters)-1) > startch = characters[index] startch = random.choice(characters) > print "Will start from %s" % startch > > #Go through list starting with items that start with 'starch', and > rotate from beginning > result = [name for name in connected if name[0] >= startch] + [name > for name in connected if name[0] < startch] It would make sense to keep track of the indices where the first word starting with a certain letter starts in the earlier part of the code and to use it now to 'rotate' the list. > print result > ========== > > Thanks again. -- Arnaud From benjamin.kaplan at case.edu Sun Jan 24 18:05:16 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Sun, 24 Jan 2010 18:05:16 -0500 Subject: Broken Python 2.6 installation on Ubuntu Linux 8.04 In-Reply-To: References: Message-ID: On Sun, Jan 24, 2010 at 5:53 PM, John Ladasky wrote: > Hello everyone, > > I've posted this same question over on ubuntuforums.org, so I'm trying > to get help in all of the logical places. > > I'm running Ubuntu Linux 8.04 (Hardy) on a fairly new x86 box, with > two hard disks in a software RAID 1 configuration. > > Hardy comes with Python 2.5 as a standard package, but not 2.6. ?I > would really like to have the "combinations" function that is included > with itertools in Python 2.6. ?I tried writing a combinations function > of my own, but it's SLOW and uses a HUGE amount of memory. > > So, in my case the Linux Synaptic Package Manager cannot be used to > install Python 2.6. ?I therefore attempted a manual installation. ?I > downloaded the Linux tarball for Python 2.6 from python.org. ?I > followed the installation instructions, and they appeared to execute > fine. ?But when I started IDLE, I still had Python 2.5. ?No good. > How did you start IDLE? If you started it by going to the Applications menu, it still points to the Python 2.5 idle. If you open up a terminal and run idle, it should run Python 2.6. If it doesn't, make a .bashrc file in your home directory and add the line "$PATH=/usr/local/bin:$PATH". > I do most of my editing in SCIte. ?Apparently SCIte knows that I have > Python 2.6, and is trying to use it. Alas, my programs depend on extra > Python packages such as biopython, numpy and matplotlib. My Python 2.6 > distro does not have these yet. None of my programs will run from > SCIte! Extensions written in C must be recompiled for every version of Python. Since you're using a version of Python not available through the package manager, your packages are also not available through that. You'll have to download the sources for those and compile them by hand to. This is why most people stick with the precompiled binaries. > > Looking down into the details of the install, I've discovered that > Hardy placed the Python 2.4 and 2.5 executables in /usr/bin. ?My > Python 2.6 installation ended up in /usr/local/bin. ?This may be > contributing to my problems. > It shouldn't be. /usr/local/bin should already be on your path in front of /usr/bin. If it isn't put it there (that's what the .bashrc file I listed before does) > Given the mess I've made by trying to just install plain-old Python, I > don't know whether I should attempt to back out, or to press on. Can I > convince IDLE to connect to Python 2.6? ?How do I manually install > site packages? > Download the package sources and compile them yourself. Python packages have a very easy way to do that- just cd into the source folder and run "python setup.py install" > Alternately, I COULD upgrade my Ubuntu Linux to 9.4 (Jaunty) or 9.10 > (Karmic). ?Python 2.6 comes standard with both of these. ?But this is > why I mentioned that my storage is RAID1. ?Apparently, upgrading with > RAID present is a serious headache. ?The Linux wizards are supposed to > be fixing these problems in the next release, due in April. ?I could > wait, I suppose. > > In the mean time, I may have to uninstall Python 2.6 and get my 2.5 > running again. ?I have not found any instructions for how to do that. > Your Python2.5 installation is just fine. You can get to it by running python2.5 at the command line (as opposed to "python" which should run python2.6) > Help! > -- > http://mail.python.org/mailman/listinfo/python-list > From steve at holdenweb.com Sun Jan 24 18:07:46 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 24 Jan 2010 18:07:46 -0500 Subject: How to Embed PHP in HTML print In-Reply-To: <036c3ed3$0$1277$c3e8da3@news.astraweb.com> References: <036becb4$0$1280$c3e8da3@news.astraweb.com> <036c3ed3$0$1277$c3e8da3@news.astraweb.com> Message-ID: <4B5CD2C2.4080309@holdenweb.com> NickC wrote: > On Sun, 24 Jan 2010 10:37:51 +0000, Duncan Booth wrote: > >> content with ajax. Alternatively, use urllib in Python to retrieve a >> page from the Apache server and insert that into its own output: that > > Thanks for hint on urllib. I shake my head in amazement with python > sometimes. I'll write it here: > > print urllib.urlopen('http://myserver.com/phpscript.php').read() > > That's it. *One* line. > > The output from the php script is: > > alt="random image" > IMG_0759" /> > > and the one-liner seamlessly prints that to insert it into the html output. > > And I thought it would be hard; I should have known better. > > Always great to see Python doing impressive things for new users. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Sun Jan 24 18:07:46 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 24 Jan 2010 18:07:46 -0500 Subject: How to Embed PHP in HTML print In-Reply-To: <036c3ed3$0$1277$c3e8da3@news.astraweb.com> References: <036becb4$0$1280$c3e8da3@news.astraweb.com> <036c3ed3$0$1277$c3e8da3@news.astraweb.com> Message-ID: <4B5CD2C2.4080309@holdenweb.com> NickC wrote: > On Sun, 24 Jan 2010 10:37:51 +0000, Duncan Booth wrote: > >> content with ajax. Alternatively, use urllib in Python to retrieve a >> page from the Apache server and insert that into its own output: that > > Thanks for hint on urllib. I shake my head in amazement with python > sometimes. I'll write it here: > > print urllib.urlopen('http://myserver.com/phpscript.php').read() > > That's it. *One* line. > > The output from the php script is: > > alt="random image" > IMG_0759" /> > > and the one-liner seamlessly prints that to insert it into the html output. > > And I thought it would be hard; I should have known better. > > Always great to see Python doing impressive things for new users. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From __peter__ at web.de Sun Jan 24 18:21:36 2010 From: __peter__ at web.de (Peter Otten) Date: Mon, 25 Jan 2010 00:21:36 +0100 Subject: Consume an iterable References: <537312ad-ee75-4594-8455-1d999f411dc3@l19g2000yqb.googlegroups.com> <77036312-3cfb-4ec0-bf9f-3946e6ed7180@q4g2000yqm.googlegroups.com> <40cfc296-4287-4a8f-850e-4f4234c55731@p8g2000yqb.googlegroups.com> <98b43693-3ba2-4d0d-b009-739d841a1c85@36g2000yqu.googlegroups.com> <6f7ef176-f9ea-4975-86d0-009fd01b8308@h34g2000yqm.googlegroups.com> Message-ID: Raymond Hettinger wrote: > FWIW, the deque() approach becomes even faster in Py2.7 and Py3.1 > which has a high-speed path for the case where maxlen is zero. > Here's a snippet from Modules/_collectionsmodule.c: > > /* Run an iterator to exhaustion. Shortcut for > the extend/extendleft methods when maxlen == 0. */ > static PyObject* > consume_iterator(PyObject *it) > { > PyObject *item; > > while ((item = PyIter_Next(it)) != NULL) { > Py_DECREF(item); > } > Py_DECREF(it); > if (PyErr_Occurred()) > return NULL; > Py_RETURN_NONE; > } > > > This code consumes an iterator to exhaustion. > It is short, sweet, and hard to beat. islice() is still a tad faster. A possible optimization: static PyObject* consume_iterator(PyObject *it) { PyObject *item; PyObject *(*iternext)(PyObject *); iternext = *Py_TYPE(it)->tp_iternext; while ((item = iternext(it)) != NULL) { Py_DECREF(item); } Py_DECREF(it); if (PyErr_Occurred()) { if(PyErr_ExceptionMatches(PyExc_StopIteration)) PyErr_Clear(); else return NULL; } Py_RETURN_NONE; } Before: $ ./python -m timeit -s"from itertools import repeat, islice; from collections import deque; from sys import maxint" "next(islice(repeat(None, 1000), maxint, maxint), None)" 100000 loops, best of 3: 6.49 usec per loop $ ./python -m timeit -s"from itertools import repeat, islice; from collections import deque; from sys import maxint" "deque(repeat(None, 1000), maxlen=0)" 100000 loops, best of 3: 9.93 usec per loop After: $ ./python -m timeit -s"from itertools import repeat, islice; from collections import deque; from sys import maxint" "deque(repeat(None, 1000), maxlen=0)" 100000 loops, best of 3: 6.31 usec per loop Peter PS: Two more, for Paul Rubin: $ ./python -m timeit -s"from itertools import repeat" "sum(0 for _ in repeat(None, 1000))" 10000 loops, best of 3: 125 usec per loop $ ./python -m timeit -s"from itertools import repeat" "sum(0 for _ in repeat(None, 1000) if 0)" 10000 loops, best of 3: 68.3 usec per loop From andymac at bullseye.apana.org.au Sun Jan 24 18:43:50 2010 From: andymac at bullseye.apana.org.au (Andrew MacIntyre) Date: Mon, 25 Jan 2010 10:43:50 +1100 Subject: ctypes for AIX In-Reply-To: <9C240725D2ABD949A80B46DD93AB107042E8D44101@XCH-NW-08V.nw.nos.boeing.com> References: <9C240725D2ABD949A80B46DD93AB107042E8D44101@XCH-NW-08V.nw.nos.boeing.com> Message-ID: <4B5CDB36.4020904@bullseye.andymac.org> Waddle, Jim wrote: > Is there a policy concerning getting functions like ctypes working on AIX. If you can get it working, post a patch on the bug tracker. -- ------------------------------------------------------------------------- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac at bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac at pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From lists at cheimes.de Sun Jan 24 18:52:38 2010 From: lists at cheimes.de (Christian Heimes) Date: Mon, 25 Jan 2010 00:52:38 +0100 Subject: Broken Python 2.6 installation on Ubuntu Linux 8.04 In-Reply-To: References: Message-ID: <4B5CDD46.30303@cheimes.de> Benjamin Kaplan wrote: > Extensions written in C must be recompiled for every version of > Python. Since you're using a version of Python not available through > the package manager, your packages are also not available through > that. You'll have to download the sources for those and compile them > by hand to. This is why most people stick with the precompiled > binaries. As far as I remember you can use the Debian build system to create binaries for your Python version. You have to add Python 2.6 to /usr/share/python/debian_defaults and recompile the desired packages with "apt-get -b python-yourmodule". This should generate one to several deb files. Install them with "dpkg -i filename.deb". It's all documented in /usr/share/doc/python/python-policy.txt.gz, read the file with zless. By the way you mustn't install your own Python with "make install", use "make altinstall"! Your /usr/local/bin/python binary masks the original python command in /usr/bin. You should remove all /usr/local/bin/py* binaries that do not end with 2.6. Otherwise you may and will break existing programs on your system. Christian From mwilson at the-wire.com Sun Jan 24 19:03:08 2010 From: mwilson at the-wire.com (Mel) Date: Sun, 24 Jan 2010 19:03:08 -0500 Subject: ISO module for binomial coefficients, etc. References: Message-ID: Dave Angel wrote: > But I still think there must be code for the gamma function that would > be quicker. But I haven't chased that lead. Log of the gamma function is also an algorithm in its own right (e.g. published in _Numerical Recipes_.) Mel. From steven at REMOVE.THIS.cybersource.com.au Sun Jan 24 19:05:13 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 25 Jan 2010 00:05:13 GMT Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> <065e450f-2190-42f4-b9c6-4b95260092be@f12g2000yqn.googlegroups.com> <28103200.7M7XYOBnsK@beaureve.gmx.net> <9734672.qlcV6ekshW@beaureve.gmx.net> <6ddbbbfe-402d-4b87-8b61-a5042071dd08@e37g2000yqn.googlegroups.com> Message-ID: On Sun, 24 Jan 2010 10:11:07 -0800, George Sakkis wrote: > Both in your example and by using a context manager, you can get away > with not passing locals() explicitly by introspecting the stack frame. You say that as if it were less of an ugly hack than the locals() trick. But sys._getframe is a private implementation detail, so you're no better off. > So it's utility is pretty limited; a custom DSL is probably better > suited to your problem. Personally, I think the OP is worrying too much about namespace pollution. If you're concerned about the enums of one function clashing with the enums of another, that's a strong hint that they don't belong in the same module. -- Steven From ladasky at my-deja.com Sun Jan 24 19:13:16 2010 From: ladasky at my-deja.com (John Ladasky) Date: Sun, 24 Jan 2010 16:13:16 -0800 (PST) Subject: Broken Python 2.6 installation on Ubuntu Linux 8.04 References: Message-ID: <2c4702dd-1c66-46f4-ad40-44798a54abdc@21g2000yqj.googlegroups.com> Thanks, Benjamin, I am getting a handle on this. I've written my own Python modules before, and have installed them using distutils. So I know that procedure. I just downloaded the Numpy 1.4.0 tarball, and I succeeded in installing it. A program I wrote which depends on numpy ran successfully from SCIte. I'll move on to matplotlib and biopython next. I also succeeded in running both the Python 2.6 and the Python 2.5 interpreters from the terminal prompt as you suggested. Not sure whether I'll need to play with .bashrc yet. From george.sakkis at gmail.com Sun Jan 24 19:46:08 2010 From: george.sakkis at gmail.com (George Sakkis) Date: Sun, 24 Jan 2010 16:46:08 -0800 (PST) Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <5552238.LgDixP2R6m@beaureve.gmx.net> <7174001.cQaEHfC5Mm@beaureve.gmx.net> <065e450f-2190-42f4-b9c6-4b95260092be@f12g2000yqn.googlegroups.com> <28103200.7M7XYOBnsK@beaureve.gmx.net> <9734672.qlcV6ekshW@beaureve.gmx.net> <6ddbbbfe-402d-4b87-8b61-a5042071dd08@e37g2000yqn.googlegroups.com> Message-ID: <9aba7f45-6ed8-4397-ae63-71c1bfc23568@v25g2000yqk.googlegroups.com> On Jan 25, 1:05?am, Steven D'Aprano wrote: > On Sun, 24 Jan 2010 10:11:07 -0800, George Sakkis wrote: > > Both in your example and by using a context manager, you can get away > > with not passing locals() explicitly by introspecting the stack frame. > > You say that as if it were less of an ugly hack than the locals() trick. > But sys._getframe is a private implementation detail, so you're no better > off. The client *is* better off in that he doesn't have to pass locals() to some magic decorator to achieve his goal; how this is implemented is a different issue. As for being an ugly hack, you may use inspect.stack () if the underscore in sys._getframe() makes you feel dirty. George From robert.kern at gmail.com Sun Jan 24 20:26:08 2010 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 24 Jan 2010 19:26:08 -0600 Subject: medians for degree measurements In-Reply-To: <036acf91$0$1357$c3e8da3@news.astraweb.com> References: <044268c0-c308-406e-8d5b-a38ef5b5e381@p24g2000yqm.googlegroups.com> <036acf91$0$1357$c3e8da3@news.astraweb.com> Message-ID: On 2010-01-23 05:52 , Steven D'Aprano wrote: > On Fri, 22 Jan 2010 22:09:54 -0800, Steve Howell wrote: > >> On Jan 22, 5:12 pm, MRAB wrote: >>> Steve Howell wrote: >>>> I just saw the thread for medians, and it reminded me of a problem >>>> that I need to solve. We are writing some Python software for >>>> sailing, and we need to detect when we've departed from the median >>>> heading on the leg. Calculating arithmetic medians is >>>> straightforward, but compass bearings add a twist. > [...] >> I like this implementation, and it would probably work 99.9999% of the >> time for my particular use case. The only (very contrived) edge case >> that I can think of is when you have 10 bearings to SSW, 10 bearings to >> SSE, and the two outliers are unfortunately in the NE and NW quadrants. >> It seems like the algorithm above would pick one of the outliers. > > The trouble is that median of angular measurements is not a meaningful > concept. The median depends on the values being ordered, but angles can't > be sensibly ordered. Which is larger, 1 degree north or 359 degrees? Is > the midpoint between them 0 degree or 180 degree? Then don't define the median that way. Instead, define the median as the point that minimizes the sum of the absolute deviations of the data from that point (the L1 norm of the deviations, for those familiar with that terminology). For 1-D data on the real number line, that corresponds to sorting the data and taking the middle element (or the artithmetic mean of the middle two in the case of even-numbered data). My definition applies to other spaces, too, that don't have a total order attached to them including the space of angles. The "circular median" is a real, well-defined statistic that is used for exactly what the OP intends to use it for. > The median of the number line 0<= x<= 360 is 180, but what's the median > of the circle 0 deg<= theta<= 360 deg? With no end points, it's > meaningless to talk about the middle. > > For this reason, I expect that taking the median of bearing measurements > isn't well defined. You can probably get away with it so long as the > bearings are "close", where "close" itself is ill-defined. > > In any case, this isn't a programming problem, it's a mathematics > problem. I think you're better off taking it to a maths or statistics > forum, and see what they say. Same answer with my statistics hat on or off. :-) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From robert.kern at gmail.com Sun Jan 24 20:33:42 2010 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 24 Jan 2010 19:33:42 -0600 Subject: how to generate random numbers that satisfy certain distribution In-Reply-To: <036bab68$0$1357$c3e8da3@news.astraweb.com> References: <7xtyuc1s65.fsf@ruckus.brouhaha.com> <036b5f64$0$1357$c3e8da3@news.astraweb.com> <7xr5pgxykd.fsf@ruckus.brouhaha.com> <036bab68$0$1357$c3e8da3@news.astraweb.com> Message-ID: On 2010-01-23 21:30 , Steven D'Aprano wrote: > On Sat, 23 Jan 2010 14:10:10 -0800, Paul Rubin wrote: > >> Steven D'Aprano writes: >>> The Box-Muller transform is reasonably simple, but you can't be serious >>> that it is simpler than adding twelve random numbers and subtracting >>> six! >> >> If you want a normal distribution, using the Box-Muller transform is >> simpler because it spares you the complication of figuring out whether >> the 12-trial binomial approximation is close enough to produce reliable >> results for your specific application, which you obviously have to do if >> you are using the approximation for anything serious. > > But using Box-Miller gives you the complication of figuring out whether > you care about being thread safety, which you have to do if you're doing > anything serious. (See the comments in the random module for the gauss > method). If you care about thread-safety, each thread should get its own Random instance anyways. > By the way, does anyone know why there is no Poisson random numbers in > the module? The implementation is quite simple (but not as simple as the > Linux kernel *wink*): > > > def poisson(lambda_=1): > L = math.exp(-lambda_) > k = 0 > p = 1 > while 1: > k += 1 > p *= random.random() > if p<= L: > break > return k-1 > > > although this can be improved upon for large values of lambda_. Where large is about 10. That's where my implementation in numpy.random switches over to a more complicated, but more efficient implementation. It does require a log-gamma special function implementation, though. #define LS2PI 0.91893853320467267 #define TWELFTH 0.083333333333333333333333 long rk_poisson_ptrs(rk_state *state, double lam) { long k; double U, V, slam, loglam, a, b, invalpha, vr, us; slam = sqrt(lam); loglam = log(lam); b = 0.931 + 2.53*slam; a = -0.059 + 0.02483*b; invalpha = 1.1239 + 1.1328/(b-3.4); vr = 0.9277 - 3.6224/(b-2); while (1) { U = rk_double(state) - 0.5; V = rk_double(state); us = 0.5 - fabs(U); k = (long)floor((2*a/us + b)*U + lam + 0.43); if ((us >= 0.07) && (V <= vr)) { return k; } if ((k < 0) || ((us < 0.013) && (V > us))) { continue; } if ((log(V) + log(invalpha) - log(a/(us*us)+b)) <= (-lam + k*loglam - loggam(k+1))) { return k; } } } -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From tekion at gmail.com Sun Jan 24 22:12:02 2010 From: tekion at gmail.com (tekion) Date: Sun, 24 Jan 2010 19:12:02 -0800 (PST) Subject: easy_install error ... References: <5e9c94df-531f-4725-bcd5-87c603d8df1c@o26g2000vbd.googlegroups.com> Message-ID: <2d3c4b85-c0fe-4f65-94ea-d246acdd0345@d30g2000vbl.googlegroups.com> I am also having another issue with easy_install, python setup.py easy_install --find-links thirdparty 'docutils==0.4' running easy_install error: Not a URL, existing file, or requirement spec: "'docutils==0.4'" I think both problem are related. I have setuptool version .6C11 install and I am running python 2.5 on window OS. Is any one else having similar issue with setuptool version .6C11? Thanks. From ursusmaximus at gmail.com Sun Jan 24 22:18:12 2010 From: ursusmaximus at gmail.com (Ron) Date: Sun, 24 Jan 2010 19:18:12 -0800 (PST) Subject: Sikuli: the coolest Python project I have yet seen... Message-ID: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> Sikuli is the coolest Python project I have ever seen in my ten year hobbyist career. An MIT oepn source project, Sikuli uses Python to automate GUI tasks (in any GUI or GUI baed app that runs the JVM) by simply drag and dropping GUI elements into Python scripts as function arguments. Download at http://sikuli.csail.mit.edu/ I also did this podcast about Sikuli http://media.libsyn.com/media/awaretek/Python411_20100124_Sikuli.mp3 From showell30 at yahoo.com Sun Jan 24 23:12:11 2010 From: showell30 at yahoo.com (Steve Howell) Date: Sun, 24 Jan 2010 20:12:11 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> Message-ID: <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> On Jan 24, 12:44?pm, Paul Rubin wrote: > Steve Howell writes: > > Proposal: Improve list's implementation so that deleting elements from > > the front of the list does not require an O(N) memmove operation. ... > > It is possible now, of course, to use a data structure in > > Python that has O(1) for deleting off the top of the list, but none of > > the alternatives fully replicate the benefits of list itself. > > I think you are mostly referring to deque. ?Why don't you instead > say what you think is wrong with using deque, and how deque can > be improved? > There is nothing wrong with deque, at least as far as I know, if the data strucure actually applies to your use case. It does not apply to my use case. > > See above. ?An implementation of this PEP would not change the > > definition of the language in any way, but it would have to minimally > > impact the performance of lists for the normal use cases. > > But you're talking about adding one or two words to EVERY list, and many > normal use cases allocate a LOT of lists. ?Those use cases are likely > more common than use cases that pop from the front of the list but for > some reason can't use deque. For EVERY list, you are not only allocating memory for the list itself, but you are also allocating memory for the objects within the list. So the extra one or two words are NEGLIGIBLE. > > The most ambitious proposal is to fix the memory manager itself to > > allow the release of memory from the start of the chunk. > > That's inappropriate given the memory fragmentation it would cause. > Bullshit. Memory managers consolidate free memory chunks all the time. That's their job. > Really, you're describing a problem that arises in a few programs but up > til now, as far as I know, everyone has found deque to be an adequate > solution. ? Wrong. Many programs delete the first element of the list. > Your approach of snarling against list is not persuading > anyone that list needs to be changed, because most everyone is satisfied > with the existing solution. ? Please provide evidence of that. I am pretty sure that everybody who chooses alternatives to Python would disagree. > You might change approaches and discuss > deque, what's wrong with it, and whether it can be fixed. ?Getting a > change approved for deque is probably much easier than getting one > approved for list, just because nowhere near as many things depend on > deque's performance. ? Again...I am not looking to improve deque, which is a perfectly valid data structure for a limited set of problems. > And when discussing performance in this contextc > additive constants do matter. Wrong again. Operations that mutate lists are already expensive, and a few checks to see if unreleased memory can be reclaimed are totally NEGLIGIBLE. From steven at REMOVE.THIS.cybersource.com.au Mon Jan 25 01:07:48 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 25 Jan 2010 06:07:48 GMT Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> Message-ID: On Sun, 24 Jan 2010 20:12:11 -0800, Steve Howell wrote: >> > The most ambitious proposal is to fix the memory manager itself to >> > allow the release of memory from the start of the chunk. >> >> That's inappropriate given the memory fragmentation it would cause. >> >> > Bullshit. Memory managers consolidate free memory chunks all the time. > That's their job. So let me get this straight... You've complained that Python's list.pop(0) is lame because it moves memory around. And your solution to that is to have the memory manager move the memory around instead? Perhaps I'm missing something, but I don't see the advantage here. At best, you consolidate all those moves you wanted to avoid and do them all at once instead of a few at a time. At worst, you get a situation where the application periodically, and unpredictably, grinds to a halt while the memory manager tries to defrag all those lists. >> Really, you're describing a problem that arises in a few programs but >> up til now, as far as I know, everyone has found deque to be an >> adequate solution. > > Wrong. Many programs delete the first element of the list. I'm sure they do. Many programs do all sorts of things, of varying sensibleness. But I'm pretty sure I've never written a program that deleted the first element of a list. Even if I have, it's a vanishingly small use-case for me. YMMV. >> Your approach of snarling against list is not persuading anyone that >> list needs to be changed, because most everyone is satisfied with the >> existing solution. > > Please provide evidence of that. I am pretty sure that everybody who > chooses alternatives to Python would disagree. Do you honestly believe that "everybody" who prefers another language over Python does so because they dislike the performance of list.pop(0)? >> You might change approaches and discuss deque, what's wrong with it, >> and whether it can be fixed. ?Getting a change approved for deque is >> probably much easier than getting one approved for list, just because >> nowhere near as many things depend on deque's performance. > > Again...I am not looking to improve deque, which is a perfectly valid > data structure for a limited set of problems. > >> And when discussing performance in this contextc additive constants do >> matter. > > Wrong again. Operations that mutate lists are already expensive, and a > few checks to see if unreleased memory can be reclaimed are totally > NEGLIGIBLE. Popping from the end of the list isn't expensive. Reversing lists is relatively cheap. In-place modifications are very cheap. -- Steven From arnodel at googlemail.com Mon Jan 25 01:57:42 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 25 Jan 2010 06:57:42 +0000 Subject: [2.5.1.1/dictionary] Change sorting order? References: <1j5jl599as1ova18lkge87vamob0msh3mt@4ax.com> Message-ID: Arnaud Delobelle writes: > Gilles Ganault writes: > >> On Fri, 22 Jan 2010 13:17:44 +0100, Gilles Ganault >> wrote: >>>To avoid users from creating login names that start with digits in >>>order to be listed at the top, I'd like to sort the list differently >>>every minute so that it'll start with the next letter, eg. display the >>>list from A...Zdigits the first time, then B...ZAdigits, etc. >> >> Thanks everyone for the great feedback. I ended up using >> - a list instead of a dictionary to hold the list of names >> - use the random library to pick a character from which to start >> listing names (simpler than saving the next letter into a file, and >> starting from this the next time the loop is gone through) >> >> For those interested, here's some working code: >> >> ========== >> import random >> >> connected = >> ["_test","-test","0test","1test","Aa","Aab","Bb","Bbbbb","Cc","Ccaaaaaa"] >> connected.sort() >> >> #Fill list with non-duplicate first letter of all items >> characters=[] >> for name in connected: >> char = name[0] >> #if this character not in list, add to array >> if not char in characters: >> characters.append(char) > > characters = list(set(name[0] for name in connected)) > >> #Pick a random character from which to start iterating >> index = random.randint(0,len(characters)-1) >> startch = characters[index] > > startch = random.choice(characters) > >> print "Will start from %s" % startch >> >> #Go through list starting with items that start with 'starch', and >> rotate from beginning >> result = [name for name in connected if name[0] >= startch] + [name >> for name in connected if name[0] < startch] I forgot this line: result = sorted(connected, key=lambda name: name[0] < startch) This takes advantage of the fact that timsort is stable. So you could do it like this, assuming connected is sorted: characters = list(set(name[0] for name in connected)) startch = random.choice(characters) result = sorted(connected, key=lambda name: name[0] < startch) If connected is not sorted, you could replace the last line with: result = sorted(connected, key=lambda name: (name[0] < startch, name)) -- Arnaud From no.email at nospam.invalid Mon Jan 25 02:24:30 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 24 Jan 2010 23:24:30 -0800 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> Message-ID: <7xpr4ylk9d.fsf@ruckus.brouhaha.com> Steve Howell writes: > There is nothing wrong with deque, at least as far as I know, if the > data strucure actually applies to your use case. It does not apply to > my use case. You haven't explained why deque doesn't apply to your use case. Until a convincing explanation emerges, the sentiment you're creating seems to be "what's wrong with that guy and why doesn't he just use deque?". So, why aren't you using deque? If deque somehow isn't adequate for your use case, maybe it can be improved. >> Your approach of snarling against list is not persuading >> anyone that list needs to be changed, because most everyone is satisfied >> with the existing solution. ? > > Please provide evidence of that. I am pretty sure that everybody who > chooses alternatives to Python would disagree. I've heard of many reasons to choose alternatives to Python, and have chosen alternatives to Python in various situations myself. The list.pop(0) issue has never been one of those reasons for me or anyone else I know of to choose an alternative until you came along. Anyway, you're welcome to switch to another language; nobody's heart will be broken if you do that. I'd be interested to know which languages handle list.pop(0) the way you're proposing for Python. >> And when discussing performance in this context, additive constants >> do matter. > Wrong again. Operations that mutate lists are already expensive I'm talking about memory consumption, which is part of Python's concept of performance. You're proposing adding a word or two to every list, with insufficient justification presented so far. Any such justification would have to include a clear and detailed explanation of why using deque is insufficient, so that would be a good place to start. On another note: the idea you're suggesting, while not yet 100% convincing, is not crazy, which is why people are willing to discuss it with you reasonably. But your confrontational style is making discussion unpleasant. Can you dial it back a little? Your current approach is perhaps leading you towards people's ignore lists. From tim.yellow at gmail.com Mon Jan 25 02:29:25 2010 From: tim.yellow at gmail.com (tim) Date: Sun, 24 Jan 2010 23:29:25 -0800 (PST) Subject: Sikuli: the coolest Python project I have yet seen... References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> Message-ID: <3abeeada-28fb-4e04-8a8b-0b814918d7e0@33g2000yqo.googlegroups.com> On Jan 25, 11:18?am, Ron wrote: > Sikuli is the coolest Python project I have ever seen in my ten year > hobbyist career. An MIT oepn source project, Sikuli uses Python to > automate GUI tasks (in any GUI or GUI baed app that runs the JVM) by > simply drag and dropping GUI elements into Python scripts as function > arguments. Download athttp://sikuli.csail.mit.edu/I also did this > podcast about Sikulihttp://media.libsyn.com/media/awaretek/Python411_20100124_Sikuli.mp3 Wow , It look likes like better than autoIt !! Very impressed. From siddhartha.veedaluru at gmail.com Mon Jan 25 03:35:48 2010 From: siddhartha.veedaluru at gmail.com (siddu) Date: Mon, 25 Jan 2010 00:35:48 -0800 (PST) Subject: TypeError not caught by except statement Message-ID: <73caff4c-243e-4ee6-aa6a-6b14f46a6706@a6g2000yqm.googlegroups.com> Hi, except not able to caught the TypeError exception occured in the below code log.info("refer",ret) in the try block throws a TypeError which is not caught . Also sometimes process is getting hanged. ------------------------------------------------------------------------------------------------------------------------------------------------ import logging log = logging.getLogger() fileName = strftime("%d-%b-%Y-", gmtime()) + str(int(time.time())) + "- Log.log" log = logging.getLogger() log.setLevel(logging.NOTSET) fh = logging.FileHandler(logFile) logFileLevel = logging.DEBUG fh.setLevel(logFileLevel) format_string = '%(process)d %(thread)d %(asctime)-15s %(levelname)-5s at %(filename)-15s in %(funcName)-10s at line %(lineno)-3d "%(message) s"' fh.setFormatter(logging.Formatter(format_string)) log.addHandler(fh) try: log.info("start") log.info("refer",ret) log.info("end") except TypeError: log.exception("Exception raised") ---------------------------------------------------------------------------------------------------------------------------------------------- OUTPUT message: Traceback (most recent call last): File "C:\Python26\lib\logging\__init__.py", line 768, in emit msg = self.format(record) File "C:\Python26\lib\logging\__init__.py", line 648, in format return fmt.format(record) File "C:\Python26\lib\logging\__init__.py", line 436, in format record.message = record.getMessage() File "C:\Python26\lib\logging\__init__.py", line 306, in getMessage msg = msg % self.args TypeError: not all arguments converted during string formatting From alfps at start.no Mon Jan 25 03:55:45 2010 From: alfps at start.no (Alf P. Steinbach) Date: Mon, 25 Jan 2010 09:55:45 +0100 Subject: TypeError not caught by except statement In-Reply-To: <73caff4c-243e-4ee6-aa6a-6b14f46a6706@a6g2000yqm.googlegroups.com> References: <73caff4c-243e-4ee6-aa6a-6b14f46a6706@a6g2000yqm.googlegroups.com> Message-ID: * siddu: > Hi, > > except not able to caught the TypeError exception occured in the below > code > > log.info("refer",ret) in the try block > > throws a TypeError which is not caught . > Also sometimes process is getting hanged. > > ------------------------------------------------------------------------------------------------------------------------------------------------ > import logging > log = logging.getLogger() > fileName = strftime("%d-%b-%Y-", gmtime()) + str(int(time.time())) + "- > Log.log" > log = logging.getLogger() > log.setLevel(logging.NOTSET) > fh = logging.FileHandler(logFile) Where does 'logFile' come from. > logFileLevel = logging.DEBUG > fh.setLevel(logFileLevel) > format_string = '%(process)d %(thread)d %(asctime)-15s %(levelname)-5s > at %(filename)-15s in %(funcName)-10s at line %(lineno)-3d "%(message) > s"' > fh.setFormatter(logging.Formatter(format_string)) > log.addHandler(fh) > > try: > log.info("start") > log.info("refer",ret) > log.info("end") > except TypeError: > log.exception("Exception raised") Try to reduce the example to a small, complete program that exhibits the problem. > > ---------------------------------------------------------------------------------------------------------------------------------------------- > OUTPUT message: > > Traceback (most recent call last): > File "C:\Python26\lib\logging\__init__.py", line 768, in emit > msg = self.format(record) > File "C:\Python26\lib\logging\__init__.py", line 648, in format > return fmt.format(record) > File "C:\Python26\lib\logging\__init__.py", line 436, in format > record.message = record.getMessage() > File "C:\Python26\lib\logging\__init__.py", line 306, in getMessage > msg = msg % self.args > TypeError: not all arguments converted during string formatting Is this a complete listing? CHheers & hth., - Alf From frank at chagford.com Mon Jan 25 03:59:01 2010 From: frank at chagford.com (Frank Millman) Date: Mon, 25 Jan 2010 10:59:01 +0200 Subject: Is defaultdict thread safe? Message-ID: Hi all Is defaultdict thread safe? Assume I have - from collections import defaultdict my_dict = defaultdict(list) If two threads call "my_dict['abc'].append(...)" simultaneously, is it guaranteed that my_dict['abc'] will end up containing two elements? Thanks Frank Millman From pavlovevidence at gmail.com Mon Jan 25 04:19:57 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Mon, 25 Jan 2010 01:19:57 -0800 (PST) Subject: Sikuli: the coolest Python project I have yet seen... References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> Message-ID: On Jan 24, 7:18?pm, Ron wrote: > Sikuli is the coolest Python project I have ever seen in my ten year > hobbyist career. An MIT oepn source project, Sikuli uses Python to > automate GUI tasks (in any GUI or GUI baed app that runs the JVM) by > simply drag and dropping GUI elements into Python scripts as function > arguments. Download athttp://sikuli.csail.mit.edu/I also did this > podcast about Sikulihttp://media.libsyn.com/media/awaretek/Python411_20100124_Sikuli.mp3 Nice, thanks for the link. Very happy to see people using Python for cool stuff like this, also to see the alternate implementations in use. And it couldn't come at a better time for me as I am trying to figure out how to automate some GUI-only program I am forced to use at work. Carl Banks From python at rcn.com Mon Jan 25 04:26:52 2010 From: python at rcn.com (Raymond Hettinger) Date: Mon, 25 Jan 2010 01:26:52 -0800 (PST) Subject: Is defaultdict thread safe? References: Message-ID: <502c108e-7bed-46b6-a24b-e25b7b92773f@e25g2000yqh.googlegroups.com> On Jan 25, 12:59?am, "Frank Millman" wrote: > Hi all > > Is defaultdict thread safe? Sometimes. It depends on whether an operation has callbacks to pure Python. > Assume I have - > > ? ? from collections import defaultdict > ? ? my_dict = defaultdict(list) > > If two threads call "my_dict['abc'].append(...)" simultaneously, is it > guaranteed that my_dict['abc'] will end up containing two elements? Yes. But, if the constructor is a user defined class, the pure python code runs for the instantiation and all bets are off. class A: def __init__(self): . . . my_dict = defaultdict(A) # not thread-safe. Raymond From olof.bjarnason at gmail.com Mon Jan 25 04:40:31 2010 From: olof.bjarnason at gmail.com (Olof Bjarnason) Date: Mon, 25 Jan 2010 10:40:31 +0100 Subject: Sikuli: the coolest Python project I have yet seen... In-Reply-To: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> Message-ID: 2010/1/25 Ron : > Sikuli is the coolest Python project I have ever seen in my ten year > hobbyist career. An MIT oepn source project, Sikuli uses Python to > automate GUI tasks (in any GUI or GUI baed app that runs the JVM) by > simply drag and dropping GUI elements into Python scripts as function > arguments. Download at http://sikuli.csail.mit.edu/ I also did this > podcast about Sikuli http://media.libsyn.com/media/awaretek/Python411_20100124_Sikuli.mp3 > -- > http://mail.python.org/mailman/listinfo/python-list > It looks really nice, but the screenhost-taking did not work on my computer (Win7). Innovative yet simple idea this mix-visual-and-code. -- http://olofb.wordpress.com From frank at chagford.com Mon Jan 25 04:45:02 2010 From: frank at chagford.com (Frank Millman) Date: Mon, 25 Jan 2010 11:45:02 +0200 Subject: Is defaultdict thread safe? References: <502c108e-7bed-46b6-a24b-e25b7b92773f@e25g2000yqh.googlegroups.com> Message-ID: On Jan 25, 11:26 am, Raymond Hettinger wrote: > On Jan 25, 12:59 am, "Frank Millman" wrote: > > > Hi all > > > Is defaultdict thread safe? > > Sometimes. It depends on whether an operation has callbacks to pure > Python. > > > Assume I have - > > > from collections import defaultdict > > my_dict = defaultdict(list) > > > If two threads call "my_dict['abc'].append(...)" simultaneously, is it > > guaranteed that my_dict['abc'] will end up containing two elements? > > Yes. > > But, if the constructor is a user defined class, the pure python code > runs for the instantiation and all bets are off. > > class A: > def __init__(self): > . . . > my_dict = defaultdict(A) # not thread-safe. > > Raymond Many thanks Frank From technic.tec at gmail.com Mon Jan 25 04:56:56 2010 From: technic.tec at gmail.com (tec) Date: Mon, 25 Jan 2010 17:56:56 +0800 Subject: TypeError not caught by except statement In-Reply-To: <73caff4c-243e-4ee6-aa6a-6b14f46a6706@a6g2000yqm.googlegroups.com> References: <73caff4c-243e-4ee6-aa6a-6b14f46a6706@a6g2000yqm.googlegroups.com> Message-ID: On 2010-1-25 16:35, siddu wrote: > > Hi, > > except not able to caught the TypeError exception occured in the below > code > > log.info("refer",ret) in the try block > > throws a TypeError which is not caught . > Also sometimes process is getting hanged. > > ------------------------------------------------------------------------------------------------------------------------------------------------ > import logging > log = logging.getLogger() > fileName = strftime("%d-%b-%Y-", gmtime()) + str(int(time.time())) + "- > Log.log" > log = logging.getLogger() > log.setLevel(logging.NOTSET) > fh = logging.FileHandler(logFile) > logFileLevel = logging.DEBUG > fh.setLevel(logFileLevel) > format_string = '%(process)d %(thread)d %(asctime)-15s %(levelname)-5s > at %(filename)-15s in %(funcName)-10s at line %(lineno)-3d "%(message) > s"' > fh.setFormatter(logging.Formatter(format_string)) > log.addHandler(fh) > > try: > log.info("start") > log.info("refer",ret) ~~~~~~~~~~~~~~~~~~~~~~~ Seems this line causes the exception, and it is handled inside log.info(), which prints those traceback info. Usually log.info(msg, args) raises the same exception as print(msg%args). > log.info("end") > except TypeError: > log.exception("Exception raised") > > ---------------------------------------------------------------------------------------------------------------------------------------------- > OUTPUT message: > > Traceback (most recent call last): > File "C:\Python26\lib\logging\__init__.py", line 768, in emit > msg = self.format(record) > File "C:\Python26\lib\logging\__init__.py", line 648, in format > return fmt.format(record) > File "C:\Python26\lib\logging\__init__.py", line 436, in format > record.message = record.getMessage() > File "C:\Python26\lib\logging\__init__.py", line 306, in getMessage > msg = msg % self.args > TypeError: not all arguments converted during string formatting From pavlovevidence at gmail.com Mon Jan 25 05:03:49 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Mon, 25 Jan 2010 02:03:49 -0800 (PST) Subject: TypeError not caught by except statement References: <73caff4c-243e-4ee6-aa6a-6b14f46a6706@a6g2000yqm.googlegroups.com> Message-ID: <8100e8d0-9c30-4f74-9af5-e65c187fe415@a5g2000yqi.googlegroups.com> On Jan 25, 12:35?am, siddu wrote: > Hi, > > except not able to caught the TypeError exception occured in the below > code > > ? ? ? ? log.info("refer",ret) in the try block > > throws a TypeError which is not caught . > Also sometimes process is getting hanged. > > ------------------------------------------------------------------------------------------------------------------------------------------------ > import logging > log = logging.getLogger() > fileName = strftime("%d-%b-%Y-", gmtime()) + str(int(time.time())) + "- > Log.log" > log = logging.getLogger() > log.setLevel(logging.NOTSET) > fh = logging.FileHandler(logFile) > logFileLevel = logging.DEBUG > fh.setLevel(logFileLevel) > format_string = '%(process)d %(thread)d %(asctime)-15s %(levelname)-5s > at %(filename)-15s in %(funcName)-10s at line %(lineno)-3d "%(message) > s"' > fh.setFormatter(logging.Formatter(format_string)) > log.addHandler(fh) > > try: > ? ? log.info("start") > ? ? log.info("refer",ret) > ? ? log.info("end") > except TypeError: > ? ? log.exception("Exception raised") > > ---------------------------------------------------------------------------------------------------------------------------------------------- > OUTPUT message: > > Traceback (most recent call last): > ? File "C:\Python26\lib\logging\__init__.py", line 768, in emit > ? ? msg = self.format(record) > ? File "C:\Python26\lib\logging\__init__.py", line 648, in format > ? ? return fmt.format(record) > ? File "C:\Python26\lib\logging\__init__.py", line 436, in format > ? ? record.message = record.getMessage() > ? File "C:\Python26\lib\logging\__init__.py", line 306, in getMessage > ? ? msg = msg % self.args > TypeError: not all arguments converted during string formatting The logging module is swallowing the exception (and printing a traceback), I guess because logging is considered something that shouldn't bring your program down on error. You can apparently define a logging handler that overrides "handleError" to propogate the exception if you want. Can't tell you why it's hanging, but the logging error you're getting is probably because your string formatter is trying to perform the following operation: "refer" % (ret,) Carl Banks From carlisdjfriends at yahoo.com Mon Jan 25 05:18:29 2010 From: carlisdjfriends at yahoo.com (carlisdjfriends) Date: Mon, 25 Jan 2010 10:18:29 -0000 Subject: [Private Photo Share] Cali Girl- Has sent you private photos. Message-ID: I do not want the entire group seeing these photos.Because some may recognize me. Here's the link: http://www.ourlivespace.com/hotgirl/photos.htm Enjoy babe :) From stefan_ml at behnel.de Mon Jan 25 05:55:19 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 25 Jan 2010 11:55:19 +0100 Subject: Is defaultdict thread safe? In-Reply-To: References: Message-ID: <4b5d7897$0$6581$9b4e6d93@newsspool3.arcor-online.net> Frank Millman, 25.01.2010 09:59: > Is defaultdict thread safe? > > Assume I have - > > from collections import defaultdict > my_dict = defaultdict(list) > > If two threads call "my_dict['abc'].append(...)" simultaneously, is it > guaranteed that my_dict['abc'] will end up containing two elements? Thread-safety is implementation specific. Other runtime environments than CPython (e.g. Jython, IronPython, PyPy) may or may not provide any guarantees on thread safety here. Stefan From richard.lamboj at bilcom.at Mon Jan 25 05:59:13 2010 From: richard.lamboj at bilcom.at (Richard Lamboj) Date: Mon, 25 Jan 2010 11:59:13 +0100 Subject: reading from pipe Message-ID: <201001251159.13237.richard.lamboj@bilcom.at> Hello, is there any solution to catch if a pipe has closed? Maybe the signal modul? For Simulation: #!/usr/bin/env python # -*- coding: utf-8 -*- import sys while True: line = sys.stdin.readline() sys.stdout.write(line) sys.stdout.flush() time cat /tmp/proxy.test | test.py Solution: if line == "": break Kind Regards, Richi From lutz.horn at fastmail.fm Mon Jan 25 06:06:18 2010 From: lutz.horn at fastmail.fm (Lutz Horn) Date: Mon, 25 Jan 2010 12:06:18 +0100 Subject: reading from pipe In-Reply-To: References: Message-ID: Hi, Richard Lamboj schrieb: > is there any solution to catch if a pipe has closed? Maybe the signal modul? Since sys.stdin is a file object, you can use sys.stdin.closed to check if it has been closed. Lutz From ladasky at my-deja.com Mon Jan 25 06:18:10 2010 From: ladasky at my-deja.com (ladasky at my-deja.com) Date: Mon, 25 Jan 2010 03:18:10 -0800 (PST) Subject: WxPython upgrade trouble on Ubuntu 8.04 Message-ID: Hello everyone, You know the old saying, "in for a penny, in for a pound." Several hours ago I posted this... http://groups.google.com/group/comp.lang.python/msg/0a86e792c674adc8 ...in which I described my desire to acquire Python 2.6 without upgrading my Ubuntu Linux installation from 8.04. Since Python 2.6 is not part of the Ubuntu 8.04 repository, I installed Python 2.6 manually. Things are a little messy right now, since invoking IDLE from the desktop still defaults to Python 2.5. But I'm getting there. At least I can access Python 2.6 from the command prompt, and SCIte also finds 2.6. Of course, I also have to reinstall all of the Python modules that I use on 2.6. I have succeeded with numpy and biopython, both installed manually. WxPython is giving me trouble, though. I visited wxpython.org and noted that wx version 2.8.10.1 fixes some important bugs that can occur when using Python 2.6. The instructions here... http://wiki.wxpython.org/InstallingOnUbuntuOrDebian ...allowed me to add the wxWidgets apt repository to the list of web sites that my Synaptic Package Manager checks for updates. It found wx 2.8.10.1 and auto-installed it. I got error messages from Python 2.5 until I also installed python-wxtools, wx2.8-i18n, libwxgtk2.8- dev, and libgtk2.0-dev as recommended on the web page mentioned above. Now "import wx" worked again, with no error messages, from Python 2.5. Installing wx on Python 2.6 is my ultimate goal, and here's where I'm stuck. I actually tried "import wx" from Python 2.6 first, and I got this result: >>> import wx Traceback (most recent call last): File "", line 1, in File "wx/__init__.py", line 45, in from wx._core import * File "wx/_core.py", line 4, in import _core_ ImportError: No module named _core_ I went back to Synaptic, said that I wanted to reinstall wx 2.8.10.1 -- but not really, as I then selected "download package files only." I received the tarball. The top-level folder has the wx binary stuff, which I believe is already installed. I selected the wxPython directory at the second level. Tried "python setup.py build", and after several minutes and dozens of warning messages, I finally got this fatal error: In file included from /usr/include/wx-2.8/wx/glcanvas.h:54, from contrib/glcanvas/gtk/glcanvas_wrap.cpp:2661: /usr/include/wx-2.8/wx/gtk/glcanvas.h: At global scope: /usr/include/wx-2.8/wx/gtk/glcanvas.h:47: error: ?GLXContext? does not name a type /usr/include/wx-2.8/wx/gtk/glcanvas.h:124: error: ISO C++ forbids declaration of ?GLXFBConfig? with no type /usr/include/wx-2.8/wx/gtk/glcanvas.h:124: error: expected ?;? before ?*? token error: command 'gcc' failed with exit status 1 Worse -- NOW if I start Python 2.5, I get the same error message I got from 2.6. Now I'm stuck again. Any advice? Many thanks! From ladasky at my-deja.com Mon Jan 25 06:21:42 2010 From: ladasky at my-deja.com (John Ladasky) Date: Mon, 25 Jan 2010 03:21:42 -0800 (PST) Subject: Broken Python 2.6 installation on Ubuntu Linux 8.04 References: Message-ID: <79c8e373-e48d-42c3-ad97-08bee0f192d6@b2g2000yqi.googlegroups.com> On Jan 24, 3:52?pm, Christian Heimes wrote: > By the way you mustn't install your own Python with "make install", use > "make altinstall"! Your /usr/local/bin/python binary masks the original > python command in /usr/bin. You should remove all /usr/local/bin/py* > binaries that do not end with 2.6. Otherwise you may and will break > existing programs on your system. > > Christian Hello Christian, In my earlier response to Benjamin, I thought I was going to solve this problem quickly. Maybe not! I know for a fact that my Linux printer management program, HPLIP toolbox, uses wxPython. And now HPLIP won't start! However, my usr/local/bin ONLY contains references to Python 2.6. So I think this is a problem with me installing wx... see this other post... http://groups.google.com/group/comp.soft-sys.wxwindows/msg/f33e245eb0956067 Sigh. All this, just so I could use some itertools functions. From solipsis at pitrou.net Mon Jan 25 06:29:09 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 25 Jan 2010 11:29:09 +0000 (UTC) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> Message-ID: Le Sun, 24 Jan 2010 11:28:53 -0800, Aahz a ?crit?: > > Again, your responsibility is to provide a patch and a spectrum of > benchmarking tests to prove it. Then you would still have to deal with > the objection that extensions use the list internals -- that might be an > okay sell given the effort otherwise required to port extensions to > Python 3, but that's not the way to bet. IMO, code accessing the list internals should be considered broken. The macros (PyList_GET_ITEM, etc.) are there for a reason. We can't just freeze every internal characteristic of the interpreter just because someone might be messing around with it in unrecommended ways. From modpoc at gmail.com Mon Jan 25 07:27:13 2010 From: modpoc at gmail.com (mbpoc mbpoc) Date: Mon, 25 Jan 2010 17:57:13 +0530 Subject: modbus Message-ID: <38e2a7731001250427g4e6139a2sdd210242e1248a23@mail.gmail.com> Hi all, I started using pymodbus. I am trying to gain pointers as to how the communcation between devices be achieved through modbus. Suggestions on Simulators/ master-slave codes would be of gr8 help Regards -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeanmichel at sequans.com Mon Jan 25 07:27:37 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 25 Jan 2010 13:27:37 +0100 Subject: Sikuli: the coolest Python project I have yet seen... In-Reply-To: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> Message-ID: <4B5D8E39.1040600@sequans.com> Ron wrote: > Sikuli is the coolest Python project I have ever seen in my ten year > hobbyist career. An MIT oepn source project, Sikuli uses Python to > automate GUI tasks (in any GUI or GUI baed app that runs the JVM) by > simply drag and dropping GUI elements into Python scripts as function > arguments. Download at http://sikuli.csail.mit.edu/ I also did this > podcast about Sikuli http://media.libsyn.com/media/awaretek/Python411_20100124_Sikuli.mp3 > It looks like your web site is down. JM From ian at excess.org Mon Jan 25 07:53:05 2010 From: ian at excess.org (Ian Ward) Date: Mon, 25 Jan 2010 07:53:05 -0500 Subject: ANN: Urwid 0.9.9.1 - Console UI Library Message-ID: <4B5D9431.9060109@excess.org> Announcing Urwid 0.9.9.1 ------------------------ Urwid home page: http://excess.org/urwid/ Screen shots: http://excess.org/urwid/examples.html Tarball: http://excess.org/urwid/urwid-0.9.9.1.tar.gz About this release: =================== This maintenance release fixes a number of bugs including a backwards incompatibility introduced in the last release and a poor ListBox snapping behaviour. New in this release: ==================== * Fix for ListBox snapping to selectable widgets taller than the ListBox itself * raw_display switching to alternate buffer now works properly with Terminal.app * Fix for BoxAdapter backwards incompatibility introduced in 0.9.9 * Fix for a doctest failure under powerpc * Fix for systems with gpm_mev installed but not running gpm About Urwid =========== Urwid is a console UI library for Python. It features fluid interface resizing, UTF-8 support, multiple text layouts, simple attribute markup, powerful scrolling list boxes and flexible interface design. Urwid is released under the GNU LGPL. From robin at reportlab.com Mon Jan 25 08:34:32 2010 From: robin at reportlab.com (Robin Becker) Date: Mon, 25 Jan 2010 13:34:32 +0000 Subject: Sikuli: the coolest Python project I have yet seen... In-Reply-To: <4B5D8E39.1040600@sequans.com> References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> <4B5D8E39.1040600@sequans.com> Message-ID: <4B5D9DE8.7040303@chamonix.reportlab.co.uk> On 25/01/2010 12:27, Jean-Michel Pichavant wrote: > Ron wrote: >> Sikuli is the coolest Python project I have ever seen in my ten year >> hobbyist career. An MIT oepn source project, Sikuli uses Python to >> automate GUI tasks (in any GUI or GUI baed app that runs the JVM) by >> simply drag and dropping GUI elements into Python scripts as function >> arguments. Download at http://sikuli.csail.mit.edu/ I also did this >> podcast about Sikuli >> http://media.libsyn.com/media/awaretek/Python411_20100124_Sikuli.mp3 > It looks like your web site is down. > > JM there's a u-tube here http://www.youtube.com/watch?v=FxDOlhysFcM -- Robin Becker From vs at it.uu.se Mon Jan 25 08:39:21 2010 From: vs at it.uu.se (Virgil Stokes) Date: Mon, 25 Jan 2010 14:39:21 +0100 Subject: Sikuli: the coolest Python project I have yet seen... In-Reply-To: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> Message-ID: <4B5D9F09.7050607@it.uu.se> On 25-Jan-2010 04:18, Ron wrote: > Sikuli is the coolest Python project I have ever seen in my ten year > hobbyist career. An MIT oepn source project, Sikuli uses Python to > automate GUI tasks (in any GUI or GUI baed app that runs the JVM) by > simply drag and dropping GUI elements into Python scripts as function > arguments. Download at http://sikuli.csail.mit.edu/ I also did this > This link is broken! --V From simon at brunningonline.net Mon Jan 25 08:57:00 2010 From: simon at brunningonline.net (Simon Brunning) Date: Mon, 25 Jan 2010 13:57:00 +0000 Subject: Is python not good enough? In-Reply-To: References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> Message-ID: <8c7f10c61001250557u6b37f284x202574f20becd326@mail.gmail.com> 2010/1/25 Albert van der Horst : > If Go was to compete with anything, they would have give it a name > that was Googleable. ;-) If they want it Googleable, it will be. ;-) -- Cheers, Simon B. From groups_ads12 at yahoo.com Mon Jan 25 09:10:05 2010 From: groups_ads12 at yahoo.com (groups_ads12) Date: Mon, 25 Jan 2010 14:10:05 -0000 Subject: www.visualstudio2010.learn.net.in Message-ID: www.visualstudio2010.learn.net.in visual studio 2010 visual studio 2010 download visual studio 2010 release date microsoft visual studio 2010 visual studio 2010 release visual studio 2010 screenshots visual studio 2010 features visual studio 2010 wiki visual studio 2010 blog visual studio 2010 rapidshare visual studio 2010 review visual studio 2010 screenshot visual studio 2010 video microsoft visual studio 2010 download visual studio 2010 iso visual studio 2010 new features visual studio 2010 preview what's new in visual studio 2010 visual studio 2010 mfc visual studio 2010 msdn visual studio 2010 requirements visual studio 2010 wikipedia visual studio 2010 ctp2 visual studio 2010 news visual studio 2010 password visual studio 2010 professional visual studio 2010 sdk visual studio 2010 what's new visual studio 2010 downloads visual studio 2010 expired visual studio 2010 launch visual studio 2010 trial visual studio 2010 videos whats new in visual studio 2010 new features in visual studio 2010 new in visual studio 2010 visual studio 2010 c# visual studio 2010 demo visual studio 2010 extensibility visual studio 2010 ide visual studio 2010 intellisense visual studio 2010 launch date visual studio 2010 multi monitor visual studio 2010 price visual studio 2010 screen shots visual studio 2010 slow visual studio 2010 tutorial features of visual studio 2010 release date for visual studio 2010 visual studio 2010 activation visual studio 2010 forum visual studio 2010 parallel buy visual studio 2010 date visual studio 2010 how to install visual studio 2010 install visual studio 2010 installing visual studio 2010 release date of visual studio 2010 visual studio 2010 .net visual studio 2010 64 visual studio 2010 architecture visual studio 2010 asp net visual studio 2010 asp net mvc visual studio 2010 changes visual studio 2010 expiration visual studio 2010 expire visual studio 2010 extension visual studio 2010 feedback visual studio 2010 free visual studio 2010 installation visual studio 2010 jquery visual studio 2010 mef visual studio 2010 performance visual studio 2010 released visual studio 2010 screen visual studio 2010 ship date visual studio 2010 timeline visual studio 2010 training visual studio 2010 upgrade visual studio 2010 virtual visual studio 2010 web visual studio 2010 windows xp welcome to visual studio 2010 when visual studio 2010 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeanmichel at sequans.com Mon Jan 25 09:12:11 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Mon, 25 Jan 2010 15:12:11 +0100 Subject: ANN: Urwid 0.9.9.1 - Console UI Library In-Reply-To: <4B5D9431.9060109@excess.org> References: <4B5D9431.9060109@excess.org> Message-ID: <4B5DA6BB.8020106@sequans.com> Ian Ward wrote: > Announcing Urwid 0.9.9.1 > ------------------------ > > Urwid home page: > http://excess.org/urwid/ > > Screen shots: > http://excess.org/urwid/examples.html > > Tarball: > http://excess.org/urwid/urwid-0.9.9.1.tar.gz > > > About this release: > =================== > > This maintenance release fixes a number of bugs including a backwards > incompatibility introduced in the last release and a poor ListBox > snapping behaviour. > > > New in this release: > ==================== > > * Fix for ListBox snapping to selectable widgets taller than the > ListBox itself > > * raw_display switching to alternate buffer now works properly with > Terminal.app > > * Fix for BoxAdapter backwards incompatibility introduced in 0.9.9 > > * Fix for a doctest failure under powerpc > > * Fix for systems with gpm_mev installed but not running gpm > > > About Urwid > =========== > > Urwid is a console UI library for Python. It features fluid interface > resizing, UTF-8 support, multiple text layouts, simple attribute markup, > powerful scrolling list boxes and flexible interface design. > > Urwid is released under the GNU LGPL. > > > I tried to use urwid to provide some friendly interface to some of our command line tools. It's pretty much effective, but I had difficulties redirecting stdout/stdin to one of the widget. In fact I don't know how do it properly, so it's currently broken. Most of ours tools are written so they log and take input from stdout/stdin, is there a widget you can use to act as stdin/stdout ? Google didn't find any receip for that, it would have been great. If it exists, any code sample would be much appreciated. Jean-Michel From javier.collado at gmail.com Mon Jan 25 09:14:34 2010 From: javier.collado at gmail.com (Javier Collado) Date: Mon, 25 Jan 2010 15:14:34 +0100 Subject: Sikuli: the coolest Python project I have yet seen... In-Reply-To: <4B5D9F09.7050607@it.uu.se> References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> <4B5D9F09.7050607@it.uu.se> Message-ID: Hello, I think the site is under maintenance. I tried a couple of hours ago and it worked fine. As an alternative, I found that this link also worked: http://www.sikuli.org/ Unfortunately, it seems it's not working right now. Best regards, Javier 2010/1/25 Virgil Stokes : > On 25-Jan-2010 04:18, Ron wrote: >> >> Sikuli is the coolest Python project I have ever seen in my ten year >> hobbyist career. An MIT oepn source project, Sikuli uses Python to >> automate GUI tasks (in any GUI or GUI baed app that runs the JVM) by >> simply drag and dropping GUI elements into Python scripts as function >> arguments. Download at http://sikuli.csail.mit.edu/ I also did this >> > > This link is broken! > > --V > > -- > http://mail.python.org/mailman/listinfo/python-list > From kyosohma at gmail.com Mon Jan 25 09:16:26 2010 From: kyosohma at gmail.com (Mike Driscoll) Date: Mon, 25 Jan 2010 06:16:26 -0800 (PST) Subject: WxPython upgrade trouble on Ubuntu 8.04 References: Message-ID: On Jan 25, 5:18?am, "lada... at my-deja.com" wrote: > Hello everyone, > > You know the old saying, "in for a penny, in for a pound." > > Several hours ago I posted this... > > http://groups.google.com/group/comp.lang.python/msg/0a86e792c674adc8 > > ...in which I described my desire to acquire Python 2.6 without > upgrading my Ubuntu Linux installation from 8.04. > > Since Python 2.6 is not part of the Ubuntu 8.04 repository, I > installed Python 2.6 manually. ?Things are a little messy right now, > since invoking IDLE from the desktop still defaults to Python 2.5. > But I'm getting there. ?At least I can access Python 2.6 from the > command prompt, and SCIte also finds 2.6. > > Of course, I also have to reinstall all of the Python modules that I > use on 2.6. ?I have succeeded with numpy and biopython, both installed > manually. ?WxPython is giving me trouble, though. > > I visited wxpython.org and noted that wx version 2.8.10.1 fixes some > important bugs that can occur when using Python 2.6. ?The instructions > here... > > http://wiki.wxpython.org/InstallingOnUbuntuOrDebian > > ...allowed me to add the wxWidgets apt repository to the list of web > sites that my Synaptic Package Manager checks for updates. ?It found > wx 2.8.10.1 and auto-installed it. ?I got error messages from Python > 2.5 until I also installed python-wxtools, wx2.8-i18n, libwxgtk2.8- > dev, and libgtk2.0-dev as recommended on the web page mentioned > above. ?Now "import wx" worked again, with no error messages, from > Python 2.5. > > Installing wx on Python 2.6 is my ultimate goal, and here's where I'm > stuck. ?I actually tried "import wx" from Python 2.6 first, and I got > this result: > > >>> import wx > > Traceback (most recent call last): > ? File "", line 1, in > ? File "wx/__init__.py", line 45, in > ? ? from wx._core import * > ? File "wx/_core.py", line 4, in > ? ? import _core_ > ImportError: No module named _core_ > > I went back to Synaptic, said that I wanted to reinstall wx 2.8.10.1 > -- but not really, as I then selected "download package files only." > I received the tarball. ?The top-level folder has the wx binary stuff, > which I believe is already installed. ?I selected the wxPython > directory at the second level. ?Tried "python setup.py build", and > after several minutes and dozens of warning messages, I finally got > this fatal error: > > In file included from /usr/include/wx-2.8/wx/glcanvas.h:54, > ? ? ? ? ? ? ? ? ?from contrib/glcanvas/gtk/glcanvas_wrap.cpp:2661: > /usr/include/wx-2.8/wx/gtk/glcanvas.h: At global scope: > /usr/include/wx-2.8/wx/gtk/glcanvas.h:47: error: ?GLXContext? does not > name a type > /usr/include/wx-2.8/wx/gtk/glcanvas.h:124: error: ISO C++ forbids > declaration of ?GLXFBConfig? with no type > /usr/include/wx-2.8/wx/gtk/glcanvas.h:124: error: expected ?;? before > ?*? token > error: command 'gcc' failed with exit status 1 > > Worse -- NOW if I start Python 2.5, I get the same error message I got > from 2.6. > > Now I'm stuck again. ?Any advice? ?Many thanks! Try re-posting to the wxPython mailing list. Someone there will know what the next step is. ------------------- Mike Driscoll Blog: http://blog.pythonlibrary.org PyCon 2010 Atlanta Feb 19-21 http://us.pycon.org/ From albert at spenarnc.xs4all.nl Mon Jan 25 09:26:19 2010 From: albert at spenarnc.xs4all.nl (Albert van der Horst) Date: 25 Jan 2010 14:26:19 GMT Subject: Is python not good enough? References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> Message-ID: In article , Aahz wrote: >In article <1b42700d-139a-4653-8669-d4ee2fc488b6 at r5g2000yqb.googlegroups.com>, >ikuta liu wrote: >> >>Is python not good enough? for google, enhance python performance is >>the good way better then choose build Go language? > >It is not at all clear that -- despite some comments to the contrary -- >the Go developers are intending to compete with Python. Go seems much >more intended to compete with C++/Java. If they're successful, we may >eventually see GoPython. ;-) As far as I can tell, Go was not intended to compete with anything. It was their own itch they scratched. Then they opened it to the world, which I applaud. If Go was to compete with anything, they would have give it a name that was Googleable. ;-) >-- >Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Groetjes Albert -- -- Albert van der Horst, UTRECHT,THE NETHERLANDS Economic growth -- being exponential -- ultimately falters. albert at spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst From albert at spenarnc.xs4all.nl Mon Jan 25 09:50:35 2010 From: albert at spenarnc.xs4all.nl (Albert van der Horst) Date: 25 Jan 2010 14:50:35 GMT Subject: A simple-to-use sound file writer References: Message-ID: In article , Mel wrote: >Alf P. Steinbach wrote: >> * Steve Holden: > >>> It's not clear to me that you can approximate any waveform with a >>> suitable combination of square waves, >> >> Oh. It's simple to prove. At least conceptually! :-) >> >> Consider first that you need an infinite number of sine waves to create a >> perfect square wave. >> >> The opposite also holds: infinite number of square waves to create a >> perfect sine wave (in a way sines and squares are opposites, the most >> incompatible). > >No, it doesn't. The infinite set of sine waves that make a square wave >leave out the sine waves of frequency 2f, 4f, 6f, 8f, ... (2*n*f) ... . >Once you've left them out, you can never get them back. So sawtooth waves, >for example, can't generally be built out of sets of square waves. Bullshit. My Boehm (B\"ohm) electronic organ does exactly that. They even have a chip for it. In the 70's it was a great hype, a sawtooth organ. Well not exactly a hype, the sound, especially the low registers, is dramatically better. If you're interested in frequencies above audible (organ builders aren't), you need an infinity of squares to build a perfect sawtooth. But then you need an inifinity of sines to build a perfect square wave. > Mel. Groetjes Albert -- -- Albert van der Horst, UTRECHT,THE NETHERLANDS Economic growth -- being exponential -- ultimately falters. albert at spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst From ursusmaximus at gmail.com Mon Jan 25 10:38:32 2010 From: ursusmaximus at gmail.com (Ron) Date: Mon, 25 Jan 2010 07:38:32 -0800 (PST) Subject: Sikuli: the coolest Python project I have yet seen... References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> Message-ID: Thew link at MIT does appear to be down right now, but I presume it will come back up. Well, those of you who find it underwhelming are in good company. See the blog post at Lambda the Ultimate http://lambda-the-ultimate.org/node/3783 I was impressed though by the application to notify you when your bus gets close to the pickup point, using Google maps, and by the app to automatically chart a course to Houston from LA on I-10, again using Google maps. And perhaps most of all, the app to notify you when your sleeping baby wakes up, from a picture on a digital camera. Hey, most of life is non-deterministic. I am in the analog engineering world and simple, deterministic black and white situations are all fine and useful, but I can see this very easy to use and simple technology being useful also ;-)) All of the above apps are but a few lines of code. Ron From tavares at fe.up.pt Mon Jan 25 10:59:33 2010 From: tavares at fe.up.pt (tavares at fe.up.pt) Date: Mon, 25 Jan 2010 07:59:33 -0800 (PST) Subject: CompIMAGE 2010: Call for Short Communications Message-ID: Dear Colleague, In the framework of the International Symposium CompIMAGE'10 ?Computational Modeling of Objects Presented in Images: Fundamentals, Methods, and Applications,? May 5-7, 2010, Buffalo-Niagara, NY, USA (www.fe.up.pt/CompIMAGE), prospective authors are invited to submit short communications to be published in the Symposium proceedings and presented at the Symposium. Deadline for submission: February 23, 2010. This deadline is firm and will not be extended. Requirements: The manuscripts must be prepared according to the LNCS requirements and submitted electronically in PDF format, together with all source files and figures. The length should be between 2 and 6 pages. The papers will be reviewed by the Editors. The authors of accepted papers will be notified by February 26, 2010. The accepted papers will be published by Research Publishing Services (Singapore) in an edited book with an ISBN. A paper will be included in the proceedings only if at least one of its author registers by February 28, 2010 and sends a symposium fee for short communication, which includes access to all symposium materials, banquet, and tour of Niagara Falls (please, visit the symposium webpage at: www.fe.up.pt/CompIMAGE). The participants are supposed to present their work orally or at a poster session. Yours sincerely, Jo?o Manuel R. S. Tavares (University of Porto, symposium co-chair) From ladasky at my-deja.com Mon Jan 25 11:10:54 2010 From: ladasky at my-deja.com (John Ladasky) Date: Mon, 25 Jan 2010 08:10:54 -0800 (PST) Subject: WxPython upgrade trouble on Ubuntu 8.04 References: Message-ID: Hi Mike, Thanks, I forgot that wxPython-users is distinct from comp.soft- sys.wxwindows. I'll give it a try. From wegwerp at gmail.com Mon Jan 25 11:16:28 2010 From: wegwerp at gmail.com (Bas) Date: Mon, 25 Jan 2010 08:16:28 -0800 (PST) Subject: medians for degree measurements References: Message-ID: <5b9beaa6-f707-46f6-b439-c9adb1fd7794@r19g2000yqb.googlegroups.com> On Jan 23, 1:09?am, Steve Howell wrote: [snip problem with angle data wrapping around at 360 degrees] Hi, This problem is trivial to solve if you can assume that you that your data points are measured consecutively and that your boat does not turn by more than 180 degrees between two samples, which seems a reasonable use case. If you cannot make this assumption, the answer seems pretty arbitrary to me anyhow. The standard trick in this situation is to 'unwrap' the data (fix > 180 deg jumps by adding or subtracting 360 to subsequent points), do your thing and then 'rewrap' to your desired interval ([0-355] or [-180,179] degrees). In [1]: from numpy import * In [2]: def median_degree(degrees): ...: return mod(rad2deg(median(unwrap(deg2rad(degrees)))),360) ...: In [3]: print(median_degree([1, 2, 3, 4, 5, 6, 359])) 3.0 In [4]: print(median_degree([-179, 174, 175, 176, 177, 178, 179])) 177.0 If the deg2rad and rad2deg bothers you, you should write your own unwrap function that handles data in degrees. Hope this helps, Bas P.S. Slightly off-topic rant against both numpy and matlab implementation of unwrap: They always assume data is in radians. There is some option to specify the maximum jump size in radians, but to me it would be more useful to specify the interval of a complete cycle, so that you can do unwrapped_radians = unwrap(radians) unwrapped_degrees = unwrap(degrees, 360) unwrapped_32bit_counter = unwrap(overflowing_counter, 2**32) From robert.kern at gmail.com Mon Jan 25 11:34:48 2010 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 25 Jan 2010 10:34:48 -0600 Subject: medians for degree measurements In-Reply-To: <5b9beaa6-f707-46f6-b439-c9adb1fd7794@r19g2000yqb.googlegroups.com> References: <5b9beaa6-f707-46f6-b439-c9adb1fd7794@r19g2000yqb.googlegroups.com> Message-ID: On 2010-01-25 10:16 AM, Bas wrote: > P.S. > Slightly off-topic rant against both numpy and matlab implementation > of unwrap: They always assume data is in radians. There is some option > to specify the maximum jump size in radians, but to me it would be > more useful to specify the interval of a complete cycle, so that you > can do > > unwrapped_radians = unwrap(radians) > unwrapped_degrees = unwrap(degrees, 360) > unwrapped_32bit_counter = unwrap(overflowing_counter, 2**32) Rants accompanied with patches are more effective. :-) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From wegwerp at gmail.com Mon Jan 25 12:06:34 2010 From: wegwerp at gmail.com (Bas) Date: Mon, 25 Jan 2010 09:06:34 -0800 (PST) Subject: medians for degree measurements References: <5b9beaa6-f707-46f6-b439-c9adb1fd7794@r19g2000yqb.googlegroups.com> Message-ID: <5f4a7bb8-5e5e-408c-98a1-0d2077590ce5@v25g2000yqk.googlegroups.com> > On 2010-01-25 10:16 AM, Bas wrote: > > > P.S. > > Slightly off-topic rant against both numpy and matlab implementation > > of unwrap: They always assume data is in radians. There is some option > > to specify the maximum jump size in radians, but to me it would be > > more useful to specify the interval of a complete cycle, so that you > > can do > > > unwrapped_radians = unwrap(radians) > > unwrapped_degrees = unwrap(degrees, 360) > > unwrapped_32bit_counter = unwrap(overflowing_counter, 2**32) On Jan 25, 5:34 pm, Robert Kern wrote:> > Rants accompanied with patches are more effective. :-) As you wish (untested): def unwrap(p, cycle=2*pi, axis=-1): """docstring to be updated""" p = asarray(p) half_cycle = cycle / 2 nd = len(p.shape) dd = diff(p, axis=axis) slice1 = [slice(None, None)]*nd # full slices slice1[axis] = slice(1, None) ddmod = mod(dd+half_cycle, cycle)-half_cycle _nx.putmask(ddmod, (ddmod==-half_cycle) & (dd > 0), half_cycle) ph_correct = ddmod - dd; _nx.putmask(ph_correct, abs(dd) References: <1b42700d-139a-4653-8669-d4ee2fc488b6@r5g2000yqb.googlegroups.com> <8c7f10c61001250557u6b37f284x202574f20becd326@mail.gmail.com> Message-ID: <4eb0089f1001250922l2138556cl91c6091a0f97d402@mail.gmail.com> On Mon, Jan 25, 2010 at 8:57 AM, Simon Brunning wrote: > 2010/1/25 Albert van der Horst : >> If Go was to compete with anything, they would have give it a name >> that was Googleable. ;-) > If they want it Googleable, it will be. ;-) http://www.google.com/search?q=go+language From showell30 at yahoo.com Mon Jan 25 12:31:15 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 09:31:15 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> <7xpr4ylk9d.fsf@ruckus.brouhaha.com> Message-ID: On Jan 24, 11:24?pm, Paul Rubin wrote: > Steve Howell writes: > > There is nothing wrong with deque, at least as far as I know, if the > > data strucure actually applies to your use case. ?It does not apply to > > my use case. > > You haven't explained why deque doesn't apply to your use case. ?Until a > convincing explanation emerges, the sentiment you're creating seems to > be "what's wrong with that guy and why doesn't he just use deque?". ?So, > why aren't you using deque? ?If deque somehow isn't adequate for your > use case, maybe it can be improved. > These are the reasons I am not using deque: 1) I want to use native lists, so that downstream methods can use them as lists. 2) Lists are faster for accessing elements. 3) I want to be able to insert elements into the middle of the list. 4) I have no need for rotating elements. I also have reasons for not using the other workarounds, such as reversing the list. > >> And when discussing performance in this context, additive constants > >> do matter. > > Wrong again. ?Operations that mutate lists are already expensive > > I'm talking about memory consumption, which is part of Python's concept > of performance. ?You're proposing adding a word or two to every list, > with insufficient justification presented so far. ?Any such > justification would have to include a clear and detailed explanation of > why using deque is insufficient, so that would be a good place to start. > Adding a word or two to a list is an O(1) addition to a data structure that takes O(N) memory to begin with. That extra pointer should really be taken not just in context of the list itself taking O(N) memory, but also the fact that all the elements in the list are also consuming memory (until they get popped off). So adding the pointer has neglible cost. Another way of looking at it is that you would need to have 250 or so lists in memory at the same time before the extra pointer was even costing you kilobytes of memory. My consumer laptop has 3027908k of memory. From robert.kern at gmail.com Mon Jan 25 12:39:04 2010 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 25 Jan 2010 11:39:04 -0600 Subject: medians for degree measurements In-Reply-To: <5f4a7bb8-5e5e-408c-98a1-0d2077590ce5@v25g2000yqk.googlegroups.com> References: <5b9beaa6-f707-46f6-b439-c9adb1fd7794@r19g2000yqb.googlegroups.com> <5f4a7bb8-5e5e-408c-98a1-0d2077590ce5@v25g2000yqk.googlegroups.com> Message-ID: On 2010-01-25 11:06 AM, Bas wrote: >> On 2010-01-25 10:16 AM, Bas wrote: >> >>> P.S. >>> Slightly off-topic rant against both numpy and matlab implementation >>> of unwrap: They always assume data is in radians. There is some option >>> to specify the maximum jump size in radians, but to me it would be >>> more useful to specify the interval of a complete cycle, so that you >>> can do >> >>> unwrapped_radians = unwrap(radians) >>> unwrapped_degrees = unwrap(degrees, 360) >>> unwrapped_32bit_counter = unwrap(overflowing_counter, 2**32) > > On Jan 25, 5:34 pm, Robert Kern wrote:> >> Rants accompanied with patches are more effective. :-) > > As you wish (untested): > > def unwrap(p, cycle=2*pi, axis=-1): > """docstring to be updated""" > p = asarray(p) > half_cycle = cycle / 2 > nd = len(p.shape) > dd = diff(p, axis=axis) > slice1 = [slice(None, None)]*nd # full slices > slice1[axis] = slice(1, None) > ddmod = mod(dd+half_cycle, cycle)-half_cycle > _nx.putmask(ddmod, (ddmod==-half_cycle)& (dd> 0), half_cycle) > ph_correct = ddmod - dd; > _nx.putmask(ph_correct, abs(dd) up = array(p, copy=True, dtype='d') > up[slice1] = p[slice1] + ph_correct.cumsum(axis) > return up > > I never saw a use case for the discontinuity argument, so in my > preferred version it would be removed. Of course this breaks old code > (by who uses this option anyhow??) and breaks compatibility between > matlab and numpy. Sometimes legitimate features have phase discontinuities greater than pi. If you want your feature to be accepted, please submit a patch that does not break backwards compatibility and which updates the docstring and tests appropriately. I look forward to seeing the complete patch! Thank you. http://projects.scipy.org/numpy -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From tim.arnold at sas.com Mon Jan 25 12:47:25 2010 From: tim.arnold at sas.com (Tim Arnold) Date: Mon, 25 Jan 2010 12:47:25 -0500 Subject: Splitting text at whitespace but keeping the whitespace in thereturned list References: <1264352736.30280.1356298585@webmail.messagingengine.com> Message-ID: "MRAB" wrote in message news:mailman.1362.1264353878.28905.python-list at python.org... > python at bdurham.com wrote: >> I need to parse some ASCII text into 'word' sized chunks of text AND >> collect the whitespace that seperates the split items. By 'word' I mean >> any string of characters seperated by whitespace (newlines, carriage >> returns, tabs, spaces, soft-spaces, etc). This means that my split text >> can contain punctuation and numbers - just not whitespace. >> The split( None ) method works fine for returning the word sized chunks >> of text, but destroys the whitespace separators that I need. >> Is there a variation of split() that returns delimiters as well as >> tokens? >> > I'd use the re module: > > >>> import re > >>> re.split(r'(\s+)', "Hello world!") > ['Hello', ' ', 'world!'] also, partition works though it returns a tuple instead of a list. >>> s = 'hello world' >>> s.partition(' ') ('hello', ' ', 'world') >>> --Tim Arnold From xahlee at gmail.com Mon Jan 25 12:55:12 2010 From: xahlee at gmail.com (Xah Lee) Date: Mon, 25 Jan 2010 09:55:12 -0800 (PST) Subject: Xah's Edu Corner: Unix Pipe As Functional Language Message-ID: <4c5dad6d-d8e9-46af-8ed8-5bc6dce057df@l11g2000yqb.googlegroups.com> Dear comrades, Hot from the press: ? Unix Pipe As Functional Language http://xahlee.org/comp/unix_pipes_and_functional_lang.html plain text version follows: -------------------------------------------------- Unix Pipe As Functional Language Xah Lee, 2010-01-25 Found the following juicy interview snippet today: Q: Is there a connection between the idea of composing programs together from the command line throught pipes and the idea of writing little languages, each for a specific domain? Alfred Aho: I think there's a connection. Certainly in the early days of Unix, pipes facilitated function composition on the command line. You could take an input, perform some transformation on it, and then pipe the output into another program. ... Q: When you say ?function composition?, that brings to mind the mathematical approach of function composition. Alfred Aho: That's exactly what I mean. Q: Was that mathematical formalism in mind at the invention of the pipe, or was that a metaphor added later when someone realized it worked the same way? Alfred Aho: I think it was right there from the start. Doug McIlroy, at least in my book, deserves the credit for pipes. He thought like a mathematician and I think he had this connection right from the start. I think of the Unix command line as a prototypical functional language. It is from a interview with Alfred Aho, one of the creators of AWK. The source is from this book: Masterminds of Programming: Conversations with the Creators of Major Programming Languages (2009), by Federico Biancuzzi et al. (amazon) Since about 1998, when i got into the unix programing industry, i see the pipe as a post fix notation, and sequencing pipes as a form of functional programing, but finding it overall extremely badly designed. I've wrote a few essays explaining the functional programing connection and exposing the lousy syntax. (mostly in years around 2000) However, i've never seen another person expressing the idea that unix pipes is a form of postfix notation and functional programing. It is a great satisfaction to see one of the main unix author state so. -------------------------------------------------- Unix Pipe As Functional Programing The following email content (slighted edited) is posted to Mac OS X mailing list, 2002-05. Source From: xah / xahlee.org Subject: Re: mail handling/conversion between OSes/apps Date: May 12, 2002 8:41:58 PM PDT Cc: macosx-talk / omnigroup.com Yes, unix have this beautiful philosophy. The philosophy is functional programing. For example, define: power(x) := x*x so ?power(3)? returns ?9?. Here ?power? is a function that takes 2 arguments. First parameter specifies the number to be raised to power, the second the number of times to multiply itself. functions can be nested, f(g(h(x))) or composed compose(f,g,h)(x) Here the ?compose? itself is a function, which take other functions as arguments, and the output of compose is a new function that is equivalent to nesting f g h. Nesting does not necessarily involved nested syntax. Here's a post fix notation in Mathematica for example: x // h // g // h or prefix notation: f @ g @ h @ x or in lisp (f (g (h x))) The principle is that everything is either a function definition or function application, and function's behavior is strictly determined by its argument. Apple around 1997 or so have this OpenDoc technology, which is similar idea applied more broadly across OS. That is, instead of one monolithic browser or big image editors or other software, but have lots of small tools or components that each does one specific thing and all can call each other or embedded in a application framework as services or the like. For example, in a email apps, you can use BBEdit to write you email, use Microsoft's spell checker, use XYZ brand of recorder to record a message, without having to open many applications or use the Finder the way we would do today. This multiplies flexibility. (OpenDoc was killed when Steve Jobs become the iCEO around 1998 and did some serious house-keeping, against the ghastly anger of Mac developers and fanatics, I'm sure many of you remember this piece of history.) The unix pipe syntax ?|?, is a postfix notation for nesting. e.g. ps auwwx | awk '{print $2}' | sort -n | xargs echo in conventional syntax it might look like this: xargs( echo, sort(n, awk('print $2', ps(auwwx))) ) So when you use ?pipe? to string many commands in unix, you are doing supreme functional programing. That's why it is so flexible and useful, because each component or function does one thing, and you can combine them in myriad of ways. However, this beautiful functional programing idea, when it is implemented by the unix heads, becomes a fucking mess. Nothing works and nothing works right. I don't feel like writing a comprehensive exposition on this at the moment. Here's a quick summary: * Fantastically stupid syntax. * Inconsistencies everywhere. Everywhere. * Fucking stupid global variables reliance called environment variables, which fucks up the whole functional programing paradigm. * Implicit stuff everywhere. * Totally incompetent commands and their parameters. (promiscuously non-orthogonal, and missing things, and fucked up in just more ways than one can possibly imagine. there are one million way to do one thing, and none are correct, and much simple needs CANNOT be done! (that's why there are gazillion shells each smart-ass improving upon the other, and that's why Perl is born too! But asinine Larry Wall don't know shit but smugly created another complexity that don't do much.)) Maybe some other day when i'm pissed, i'll write a better exposition on this issue. I've been wanting to write a final-say essay on this for long. Don't feel like it now. -------------------------------------------------- Unix Syntatical and Semantical Stupidity Exposition The following is posted to a Mac OS X mailing list. Source From: xah at xahlee.org Subject: unix inanity: shell env syntax Date: June 7, 2002 12:00:29 AM PDT To: macosx-talk at omnigroup.com Unix Syntatical and Semantical Stupidity Exposition. (this is one of the many technical expositions of unix stupidity) (this is currently unpolished, but the meat is there. Input welcome.) arguments are given with a dash prefix. e.g. ls -a -l Order (usually) does not matter. So, ls -a -l is the same as ls -l -a but arguments can be combined, e.g. ls -al means the same thing as ls -a -l However, some option consists of more than one character. e.g. perl -version perl -help therefore, the meaning of a option string "-ab" is ad hoc dependent on the program. It can be "-a -b" or just a option named "ab". Then, sometimes there are two versions of the same optional argument. e.g. perl -help perl -h perl -version perl -v this equivalence is ad hoc. Different program will disagree on common options. For example, to get the version, here are common varieties: -v -V -version sometimes v/V stands for "verbose mode", i.e. to output more detail. Sometimes, if a option is given more than once, then it specifies a degree of that option. For example, some command accept the -v for "verbose", meaning that it will output more detail. Sometimes there are few levels of detail. The number of times a option is given determines the level of detail. e.g. on Solaris 8, /usr/ucb/ps -w /usr/ucb/ps -w -w Thus, meaning of repeated option may have special meaning depending on the program. Oftentimes some options automatically turn on or surpress a bunch of others. e.g. Solaris 8, /usr/bin/ls -f When a named optional parameter is of a boolean type, that is a toggle of yes/no, true/false, exist/nonexist, then it is often times that instead of taking a boolean value, their sole existence or non- existence defines their value. Toggle options are sometimes represented by one option name for yes, while another option name for no, and when both are present, the behavior is program dependent. Toggle options are represented by different option names. For named options, their syntax is slack but behavior is usually dependent on the program. i.e. not all of the following works for every program command -o="myvalue" command -omyvalue comand -o myvalue Often one option may have many synonyms... A example of a better design... (Mathematica, Scheme, Dylan, Python, Ruby... there's quite a lot elegance and practicality yet distinct designs and purposes and styles ...) (recall that unix have a bad design to begin with; it's a donkey shit pile from the beginning and continuation. Again, unix is not simply technically incompetent. If that, then that's easy to improve, and i don't have a problem with, since there are things in one way or another considered horrendous by today's standard like COBOL or FORTRAN or DOS etc. But, unix is a brain-washing idiot-making machine, churning out piles and piles of religiously idiotic and pigheaded keyboard punchers. For EVERY aspects of good engineering methodology improvement or language design progress opportunity, unixers will unanimously turn it down. Inevitably someone will ask me what's my point. My point in my series of unix-showcasing articles have always been clear for those who studies it: Unix is a crime that caused society inordinate harm, and i want unix geeks to wake up and realize it. Microsoft PowerShell Note: Microsoft's new shell programing language, PowerShell (b2006), adopted much of unix shell's syntax and the pipe paradigm, but with a much consistent and formal design. (see: Xah's PowerShell Tutorial) Xah ? http://xahlee.org/ ? From showell30 at yahoo.com Mon Jan 25 13:00:03 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 10:00:03 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> Message-ID: <18177f2a-ca15-4d3c-9f6c-09beb21a4947@y12g2000yqh.googlegroups.com> On Jan 24, 10:07?pm, Steven D'Aprano wrote: > On Sun, 24 Jan 2010 20:12:11 -0800, Steve Howell wrote: > >> > The most ambitious proposal is to fix the memory manager itself to > >> > allow the release of memory from the start of the chunk. > > >> That's inappropriate given the memory fragmentation it would cause. > > > Bullshit. ?Memory managers consolidate free memory chunks all the time. > > That's their job. > > So let me get this straight... > > You've complained that Python's list.pop(0) is lame because it moves > memory around. And your solution to that is to have the memory manager > move the memory around instead? > > Perhaps I'm missing something, but I don't see the advantage here. At > best, you consolidate all those moves you wanted to avoid and do them all > at once instead of a few at a time. At worst, you get a situation where > the application periodically, and unpredictably, grinds to a halt while > the memory manager tries to defrag all those lists. > You are misunderstanding what I meant, because I did not explain it very well. When you release memory from the front of the list, if the memory before it was also free, the memory manager could consolidate the two chunks as one free chunk. There is no rational scenario where the memory manager grinds to a halt tries to "defrag all those lists." Of course, once the list gets fully garbage collected, then entire chunk of memory is freed up. > >> Your approach of snarling against list is not persuading anyone that > >> list needs to be changed, because most everyone is satisfied with the > >> existing solution. > > > Please provide evidence of that. ?I am pretty sure that everybody who > > chooses alternatives to Python would disagree. > > Do you honestly believe that "everybody" who prefers another language > over Python does so because they dislike the performance of list.pop(0)? > No I don't believe any statement that makes gross generalizations, so I also don't believe "most everyone is satisfied with the existing solution." > >> You might change approaches and discuss deque, what's wrong with it, > >> and whether it can be fixed. ?Getting a change approved for deque is > >> probably much easier than getting one approved for list, just because > >> nowhere near as many things depend on deque's performance. > > > Again...I am not looking to improve deque, which is a perfectly valid > > data structure for a limited set of problems. > > >> And when discussing performance in this contextc additive constants do > >> matter. > > > Wrong again. ?Operations that mutate lists are already expensive, and a > > few checks to see if unreleased memory can be reclaimed are totally > > NEGLIGIBLE. > > Popping from the end of the list isn't expensive. Reversing lists is > relatively cheap. In-place modifications are very cheap. > I am talking in relative terms here. I am saying that checking a single flag in C code isn't gonna significantly slow down any operation that calls list_resize(). Delete operations would already be doing a memmove operation, and insert operations already have to decide whether to optimistically allocate memory and create the new list element. Regarding the extra use of memory, I addressed this in my prior posting. Here is code for list_resize: static int list_resize(PyListObject *self, Py_ssize_t newsize) { PyObject **items; size_t new_allocated; Py_ssize_t allocated = self->allocated; /* Bypass realloc() when a previous overallocation is large enough to accommodate the newsize. If the newsize falls lower than half the allocated size, then proceed with the realloc() to shrink the list. */ if (allocated >= newsize && newsize >= (allocated >> 1)) { assert(self->ob_item != NULL || newsize == 0); Py_SIZE(self) = newsize; return 0; } /* This over-allocates proportional to the list size, making room * for additional growth. The over-allocation is mild, but is * enough to give linear-time amortized behavior over a long * sequence of appends() in the presence of a poorly-performing * system realloc(). * The growth pattern is: 0, 4, 8, 16, 25, 35, 46, 58, 72, 88, ... */ new_allocated = (newsize >> 3) + (newsize < 9 ? 3 : 6); /* check for integer overflow */ if (new_allocated > PY_SIZE_MAX - newsize) { PyErr_NoMemory(); return -1; } else { new_allocated += newsize; } if (newsize == 0) new_allocated = 0; items = self->ob_item; if (new_allocated <= ((~(size_t)0) / sizeof(PyObject *))) PyMem_RESIZE(items, PyObject *, new_allocated); else items = NULL; if (items == NULL) { PyErr_NoMemory(); return -1; } self->ob_item = items; Py_SIZE(self) = newsize; self->allocated = new_allocated; return 0; } Here is the code for list_ass_slice: static int list_ass_slice(PyListObject *a, Py_ssize_t ilow, Py_ssize_t ihigh, PyObject *v) { /* Because [X]DECREF can recursively invoke list operations on this list, we must postpone all [X]DECREF activity until after the list is back in its canonical shape. Therefore we must allocate an additional array, 'recycle', into which we temporarily copy the items that are deleted from the list. :-( */ PyObject *recycle_on_stack[8]; PyObject **recycle = recycle_on_stack; /* will allocate more if needed */ PyObject **item; PyObject **vitem = NULL; PyObject *v_as_SF = NULL; /* PySequence_Fast(v) */ Py_ssize_t n; /* # of elements in replacement list */ Py_ssize_t norig; /* # of elements in list getting replaced */ Py_ssize_t d; /* Change in size */ Py_ssize_t k; size_t s; int result = -1; /* guilty until proved innocent */ #define b ((PyListObject *)v) if (v == NULL) n = 0; else { if (a == b) { /* Special case "a[i:j] = a" -- copy b first */ v = list_slice(b, 0, Py_SIZE(b)); if (v == NULL) return result; result = list_ass_slice(a, ilow, ihigh, v); Py_DECREF(v); return result; } v_as_SF = PySequence_Fast(v, "can only assign an iterable"); if(v_as_SF == NULL) goto Error; n = PySequence_Fast_GET_SIZE(v_as_SF); vitem = PySequence_Fast_ITEMS(v_as_SF); } if (ilow < 0) ilow = 0; else if (ilow > Py_SIZE(a)) ilow = Py_SIZE(a); if (ihigh < ilow) ihigh = ilow; else if (ihigh > Py_SIZE(a)) ihigh = Py_SIZE(a); norig = ihigh - ilow; assert(norig >= 0); d = n - norig; if (Py_SIZE(a) + d == 0) { Py_XDECREF(v_as_SF); return list_clear(a); } item = a->ob_item; /* recycle the items that we are about to remove */ s = norig * sizeof(PyObject *); if (s > sizeof(recycle_on_stack)) { recycle = (PyObject **)PyMem_MALLOC(s); if (recycle == NULL) { PyErr_NoMemory(); goto Error; } } memcpy(recycle, &item[ilow], s); if (d < 0) { /* Delete -d items */ memmove(&item[ihigh+d], &item[ihigh], (Py_SIZE(a) - ihigh)*sizeof(PyObject *)); list_resize(a, Py_SIZE(a) + d); item = a->ob_item; } else if (d > 0) { /* Insert d items */ k = Py_SIZE(a); if (list_resize(a, k+d) < 0) goto Error; item = a->ob_item; memmove(&item[ihigh+d], &item[ihigh], (k - ihigh)*sizeof(PyObject *)); } for (k = 0; k < n; k++, ilow++) { PyObject *w = vitem[k]; Py_XINCREF(w); item[ilow] = w; } for (k = norig - 1; k >= 0; --k) Py_XDECREF(recycle[k]); result = 0; Error: if (recycle != recycle_on_stack) PyMem_FREE(recycle); Py_XDECREF(v_as_SF); return result; #undef b } From showell30 at yahoo.com Mon Jan 25 13:13:57 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 10:13:57 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> <7xpr4ylk9d.fsf@ruckus.brouhaha.com> Message-ID: <1e6c08e2-3d37-484c-a137-eeb1cd3fed59@c4g2000yqa.googlegroups.com> On Jan 25, 9:31?am, Steve Howell wrote: > On Jan 24, 11:24?pm, Paul Rubin wrote: > > > Steve Howell writes: > > > There is nothing wrong with deque, at least as far as I know, if the > > > data strucure actually applies to your use case. ?It does not apply to > > > my use case. > > > You haven't explained why deque doesn't apply to your use case. ?Until a > > convincing explanation emerges, the sentiment you're creating seems to > > be "what's wrong with that guy and why doesn't he just use deque?". ?So, > > why aren't you using deque? ?If deque somehow isn't adequate for your > > use case, maybe it can be improved. > > These are the reasons I am not using deque: > > ? 1) I want to use native lists, so that downstream methods can use > them as lists. > ? 2) Lists are faster for accessing elements. > ? 3) I want to be able to insert elements into the middle of the list. > ? 4) I have no need for rotating elements. > > I also have reasons for not using the other workarounds, such as > reversing the list. > > > >> And when discussing performance in this context, additive constants > > >> do matter. > > > Wrong again. ?Operations that mutate lists are already expensive > > > I'm talking about memory consumption, which is part of Python's concept > > of performance. ?You're proposing adding a word or two to every list, > > with insufficient justification presented so far. ?Any such > > justification would have to include a clear and detailed explanation of > > why using deque is insufficient, so that would be a good place to start. > > Adding a word or two to a list is an O(1) addition to a data structure > that takes O(N) memory to begin with. > > That extra pointer should really be taken not just in context of the > list itself taking O(N) memory, but also the fact that all the > elements in the list are also consuming memory (until they get popped > off). ?So adding the pointer has neglible cost. > > Another way of looking at it is that you would need to have 250 or so > lists in memory at the same time before the extra pointer was even > costing you kilobytes of memory. ?My consumer laptop has 3027908k of > memory. I should also point out that my telephone has gigabytes of memory. It's a fairly expensive device, but I regularly carry multiple gigabytes of memory around in my front pants pocket. There are some valid reasons to reject a proposal to make deleting elements off the top of the list be O(1). Memory consumption is not one of them. Even the most naive patch to make pop(0) and del lst[0] advance the pointer would eventually reclaim memory once the list is garbage collected. Also, by allowing users to pop elements off the list without a memmove, you encourage users to discard elements earlier in the process, which means you can amortize the garbage collection for the list elements themselves (i.e. less spiky), and do it earlier. From showell30 at yahoo.com Mon Jan 25 13:24:04 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 10:24:04 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> Message-ID: On Jan 24, 1:51?pm, Daniel Stutzbach wrote: > On Sun, Jan 24, 2010 at 1:53 PM, Steve Howell wrote: > > I don't think anybody provided an actual link, but please correct me > > if I overlooked it. > > I have to wonder if my messages are all ending up in your spam folder > for some reason. :-) > > PEP 3128 (which solves your problem, but not using the implementation > you suggest)http://www.python.org/dev/peps/pep-3128/ > > Implementation as an extension module:http://pypi.python.org/pypi/blist/ > > Related discussion:http://mail.python.org/pipermail/python-3000/2007-April/006757.htmlhttp://mail.python.org/pipermail/python-3000/2007-May/007491.html > be > Detailed performance comparison:http://stutzbachenterprises.com/performance-blist > > I maintain a private fork of Python 3 with the blist replacing the > regular list, as a way of rigorously testing the blist implementation. > > Although I originally proposed a PEP, I am content to have the blist > exist as a third-party module. > Hi Daniel, I agree with what Raymond Hettinger says toward the top of the PEP. Blist, while extremely useful, does seem to have to trade off performance of common operations, notably "get item", in order to get better performance for other operations (notably insert/delete). My algorithm does exactly N pops and roughly N list accesses, so I would be going from N*N + N to N + N log N if switched to blist. That would be at least a theoretical gain over the current performance, but if pop() were O(1), I could get the whole thing down to N time. From wegwerp at gmail.com Mon Jan 25 13:31:04 2010 From: wegwerp at gmail.com (Bas) Date: Mon, 25 Jan 2010 10:31:04 -0800 (PST) Subject: medians for degree measurements References: <5b9beaa6-f707-46f6-b439-c9adb1fd7794@r19g2000yqb.googlegroups.com> <5f4a7bb8-5e5e-408c-98a1-0d2077590ce5@v25g2000yqk.googlegroups.com> Message-ID: <6da539f0-0b63-4467-ae3a-4e7b9d86dc98@l19g2000yqb.googlegroups.com> > >> On 2010-01-25 10:16 AM, Bas wrote: > >>> P.S. > >>> Slightly off-topic rant against both numpy and matlab implementation > >>> of unwrap: They always assume data is in radians. There is some option > >>> to specify the maximum jump size in radians, but to me it would be > >>> more useful to specify the interval of a complete cycle, so that you > >>> can do [snip] > > I never saw a use case for the discontinuity argument, so in my > > preferred version it would be removed. Of course this breaks old code > > (by who uses this option anyhow??) and breaks compatibility between > > matlab and numpy. On Jan 25, 6:39 pm, Robert Kern wrote: > Sometimes legitimate features have phase discontinuities greater than pi. We are dwelling more and more off-topic here, but anyhow: According to me, the use of unwrap is inherently related to measurement instruments that wrap around, like rotation encoders, interferometers or up/down counters. Say you have a real phase step of +1.5 pi, how could you possibly discern if from a real phase step of -pi/2? This is like an aliasing problem, so the only real solution would be to increase the sampling speed of your system. To me, the discontinuity parameter is serving some hard to explain corner case (see matlab manual), which is better left to be solved by hand in cases it appears. I regret matlab ever added the feature. > If you want your feature to be accepted, please submit a patch that does not break > backwards compatibility and which updates the docstring and tests appropriately. > I look forward to seeing the complete patch! Thank you. I think my 'cycle' argument does have real uses, like the degrees in this thread and the digital-counter example (which comes from own experience and required me to write my own unwrap). I'll try to submit a non-breaking patch if I ever have time. Bas From showell30 at yahoo.com Mon Jan 25 13:36:19 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 10:36:19 -0800 (PST) Subject: medians for degree measurements References: <044268c0-c308-406e-8d5b-a38ef5b5e381@p24g2000yqm.googlegroups.com> <036acf91$0$1357$c3e8da3@news.astraweb.com> Message-ID: <4131428a-8497-4b4d-84ab-fdfce1dee270@j14g2000yqm.googlegroups.com> On Jan 24, 5:26?pm, Robert Kern wrote: > On 2010-01-23 05:52 , Steven D'Aprano wrote: > > On Fri, 22 Jan 2010 22:09:54 -0800, Steve Howell wrote: > > >> On Jan 22, 5:12 pm, MRAB ?wrote: > >>> Steve Howell wrote: > >>>> I just saw the thread for medians, and it reminded me of a problem > >>>> that I need to solve. ?We are writing some Python software for > >>>> sailing, and we need to detect when we've departed from the median > >>>> heading on the leg. ?Calculating arithmetic medians is > >>>> straightforward, but compass bearings add a twist. > > [...] > >> I like this implementation, and it would probably work 99.9999% of the > >> time for my particular use case. ?The only (very contrived) edge case > >> that I can think of is when you have 10 bearings to SSW, 10 bearings to > >> SSE, and the two outliers are unfortunately in the NE and NW quadrants. > >> It seems like the algorithm above would pick one of the outliers. > > > The trouble is that median of angular measurements is not a meaningful > > concept. The median depends on the values being ordered, but angles can't > > be sensibly ordered. Which is larger, 1 degree north or 359 degrees? Is > > the midpoint between them 0 degree or 180 degree? > > Then don't define the median that way. Instead, define the median as the point > that minimizes the sum of the absolute deviations of the data from that point > (the L1 norm of the deviations, for those familiar with that terminology). For > 1-D data on the real number line, that corresponds to sorting the data and > taking the middle element (or the artithmetic mean of the middle two in the case > of even-numbered data). My definition applies to other spaces, too, that don't > have a total order attached to them including the space of angles. > > The "circular median" is a real, well-defined statistic that is used for exactly > what the OP intends to use it for. > I admitted pretty early in the thread that I did not define the statistic with much rigor, although most people got the gist of the problem, and as Robert points out, you can more clearly the define the problem, although I think under any definition, some inputs will have multiple solutions, such as (0, 90, 180, 270) and (0, 120, 240). If you've ever done lake sailing, you probably have encountered days where the wind seems to be coming from those exact angles. This is the code that I'll be using (posted by "Nobody"). I'll report back it if it has any issues. def mean(bearings): x = sum(sin(radians(a)) for a in bearings) y = sum(cos(radians(a)) for a in bearings) return degrees(atan2(x, y)) def median(bearings): m = mean(bearings) bearings = [(a - m + 180) % 360 - 180 for a in bearings] bearings.sort() median = bearings[len(bearings) / 2] median += m median %= 360 return median From moibenko at fnal.gov Mon Jan 25 14:05:28 2010 From: moibenko at fnal.gov (Alexander Moibenko) Date: Mon, 25 Jan 2010 13:05:28 -0600 Subject: Total maximal size of data Message-ID: <4B5DEB78.10406@fnal.gov> I have a simple question to which I could not find an answer. What is the total maximal size of list including size of its elements? I do not like to look into python source. Here is a code example: import struct KB=1024 MB=KB*KB GB=MB*KB buf=[] bs=32*KB n=4*GB/bs print "N",n i=0 size=0L while i < n: data = struct.pack("%ss" % (bs,), "") buf.append(data) size = size+bs if size % (100*MB) == 0: print "SIZE", size/MB, "MB" i=i+1 while 1: # to keep script running while I am looking at the machine status pass Here is what I get on 32-bit architecture: cat /proc/meminfo MemTotal: 8309860 kB MemFree: 5964888 kB Buffers: 84396 kB Cached: 865644 kB SwapCached: 0 kB ...... The program output: N 131072 SIZE 100 MB SIZE 200 MB SIZE 300 MB SIZE 400 MB SIZE 500 MB SIZE 600 MB SIZE 700 MB SIZE 800 MB SIZE 900 MB SIZE 1000 MB SIZE 1100 MB SIZE 1200 MB SIZE 1300 MB SIZE 1400 MB SIZE 1500 MB SIZE 1600 MB SIZE 1700 MB SIZE 1800 MB SIZE 1900 MB SIZE 2000 MB SIZE 2100 MB SIZE 2200 MB SIZE 2300 MB SIZE 2400 MB SIZE 2500 MB SIZE 2600 MB SIZE 2700 MB SIZE 2800 MB SIZE 2900 MB SIZE 3000 MB Traceback (most recent call last): File "bs.py", line 14, in ? data = struct.pack("%ss" % (bs,), "") MemoryError ++++++++++++++++++++ The number of list elements for a given block size is 131072. If I change block size the script traces back at the same total size 3000MB. Somewhere I read that list could have 2147483647 items, on most platforms. Somewhere else that it is *536,870,912 (http://stackoverflow.com/questions/855191/how-big-can-a-python-array-get)** *But what is the maximal size of the whole list including the size of its elements? Thanks. From deets at nospam.web.de Mon Jan 25 14:23:50 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Mon, 25 Jan 2010 20:23:50 +0100 Subject: Total maximal size of data In-Reply-To: References: Message-ID: <4B5DEFC6.9080907@nospam.web.de> Am 25.01.10 20:05, schrieb Alexander Moibenko: > I have a simple question to which I could not find an answer. > What is the total maximal size of list including size of its elements? > I do not like to look into python source. But it would answer that question pretty fast. Because then you'd see that all list-object-methods are defined in terms of Py_ssize_t, which is an alias for ssize_t of your platform. 64bit that should be a 64bit long. Diez From deets at nospam.web.de Mon Jan 25 14:23:57 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Mon, 25 Jan 2010 20:23:57 +0100 Subject: Total maximal size of data In-Reply-To: References: Message-ID: <7s69efF2u0U1@mid.uni-berlin.de> Am 25.01.10 20:05, schrieb Alexander Moibenko: > I have a simple question to which I could not find an answer. > What is the total maximal size of list including size of its elements? > I do not like to look into python source. But it would answer that question pretty fast. Because then you'd see that all list-object-methods are defined in terms of Py_ssize_t, which is an alias for ssize_t of your platform. 64bit that should be a 64bit long. Diez From tjreedy at udel.edu Mon Jan 25 14:27:17 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 25 Jan 2010 14:27:17 -0500 Subject: Sikuli: the coolest Python project I have yet seen... In-Reply-To: References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> <4B5D9F09.7050607@it.uu.se> Message-ID: On 1/25/2010 9:14 AM, Javier Collado wrote: > I think the site is under maintenance. I tried a couple of hours ago > and it worked fine. > > As an alternative, I found that this link also worked: > http://www.sikuli.org/ This just redirects to the link below >>> http://sikuli.csail.mit.edu/ I also did this >> This link is broken! Worked for me both yesterday and now. From daniel at stutzbachenterprises.com Mon Jan 25 14:28:10 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 25 Jan 2010 13:28:10 -0600 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> Message-ID: On Mon, Jan 25, 2010 at 12:24 PM, Steve Howell wrote: > Hi Daniel, I agree with what Raymond Hettinger says toward the top of > the PEP. Blist, while extremely useful, does seem to have to trade > off performance of common operations, notably "get item", in order to > get better performance for other operations (notably insert/delete). > Actually, the latest version of blist is competitive for "get item" and similar operations. See: http://stutzbachenterprises.com/performance-blist/item http://stutzbachenterprises.com/performance-blist/set-item http://stutzbachenterprises.com/performance-blist/lifo http://stutzbachenterprises.com/performance-blist/shuffle I added a flat cache of the leaf nodes, yielding O(1) get/set item operations whenever those operations dominate over insert/delete operations. The cache adds around 1.5% memory overhead. > My algorithm does exactly N pops and roughly N list accesses, so I > would be going from N*N + N to N + N log N if switched to blist. That > would be at least a theoretical gain over the current performance, but > if pop() were O(1), I could get the whole thing down to N time. > If I understand correctly, you feel strongly that a list.pop(0) that runs in O(n) time is "broken", but you're comfortable with a list.pop(1) that runs in O(n) time. Is that correct? How do you feel about a bisect.insort(list, item) that takes O(n) time? Different people are bound to have different opinions about which operations are most important and where lies the best tradeoff between different operations (as well as code complexity). I am not sure why you feel so strongly that particular spot is best. Obviously, I prefer a slightly different spot, but I also respect the core developers' choice. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From anm12345 at gmail.com Mon Jan 25 14:39:09 2010 From: anm12345 at gmail.com (AlexM) Date: Mon, 25 Jan 2010 11:39:09 -0800 (PST) Subject: Total maximal size of data References: <7s69efF2u0U1@mid.uni-berlin.de> Message-ID: <96b3f223-2f5f-4af1-b896-2d926986441f@g29g2000yqe.googlegroups.com> On Jan 25, 1:23?pm, "Diez B. Roggisch" wrote: > Am 25.01.10 20:05, schrieb Alexander Moibenko: > > > I have a simple question to which I could not find an answer. > > What is the total maximal size of list including size of its elements? > > I do not like to look into python source. > > But it would answer that question pretty fast. Because then you'd see > that all list-object-methods are defined in terms of Py_ssize_t, which > is an alias for ssize_t of your platform. 64bit that should be a 64bit long. > > Diez Then how do explain the program output? Alex. From gagsl-py2 at yahoo.com.ar Mon Jan 25 14:44:14 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 25 Jan 2010 16:44:14 -0300 Subject: start .pyo files with doubleclick on windows References: <4b5cb96a$0$23933$426a74cc@news.free.fr> <4b5ccb85$0$22008$426a74cc@news.free.fr> Message-ID: En Sun, 24 Jan 2010 19:36:53 -0300, News123 escribi?: > Hi Alf, > > > > > > Alf P. Steinbach wrote: >> * News123: >>> Hi, >>> >>> >>> I'd like to start .pyo files under windows with a double click. >> >> C:\> assoc .pyo >> .pyo=Python.CompiledFile >> >> C:\> ftype python.compiledfile >> python.compiledfile="C:\Program Files\cpython\python31\python.exe" "%1" >> %* >> >> C:\> _ >> >> >> Use ftype to change the association. > > Thanks a lot, I leared something new about Windows > > > What I did now is this: > > assoc .pyo=Python.CompiledOptimizedFile > ftype Python.CompiledOptimizedFile="C:\Python26\python.exe" -OO "%1" %* This looks like a bug (or two) - "python foo.pyo" should fail; a .pyo has a different magic number than a .pyc file. Importing a .pyo file fails in this case. A .pyo file should be run with "python -O foo.pyo" (or -OO) - on Windows, .pyo files should be associated with python -O by default, you should not need to do that by yourself. -- Gabriel Genellina From gagsl-py2 at yahoo.com.ar Mon Jan 25 14:47:57 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Mon, 25 Jan 2010 16:47:57 -0300 Subject: Default path for files References: <0neu27-csn.ln1@spamfence.net> Message-ID: En Sun, 24 Jan 2010 15:04:48 -0300, G?nther Dietrich escribi?: > Rotwang wrote: > >>> Check out http://docs.python.org/library/os.html and the function >>> chdir it is what you are looking for. >> >> Thank you. So would adding >> >> import os >> os.chdir() >> >> to site.py (or any other module which is automatically imported during >> initialisation) change the default location to every time I used >> Python? > > Don't change the library modules. It would catch you anytime when you > expect it least. > > See for the environment variable PYTHONSTARTUP and the associated > startup file. sitecustomize.py would be a better place. PYTHONSTARTUP is only used when running in interactive mode. Anyway, I'd do that explicitely on each script that requires it; after upgrading the Python version, or moving to another PC, those scripts would start failing... -- Gabriel Genellina From deets at nospam.web.de Mon Jan 25 15:03:51 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Mon, 25 Jan 2010 21:03:51 +0100 Subject: Total maximal size of data In-Reply-To: <96b3f223-2f5f-4af1-b896-2d926986441f@g29g2000yqe.googlegroups.com> References: <7s69efF2u0U1@mid.uni-berlin.de> <96b3f223-2f5f-4af1-b896-2d926986441f@g29g2000yqe.googlegroups.com> Message-ID: <7s6bp7Fgl6U1@mid.uni-berlin.de> Am 25.01.10 20:39, schrieb AlexM: > On Jan 25, 1:23 pm, "Diez B. Roggisch" wrote: >> Am 25.01.10 20:05, schrieb Alexander Moibenko: >> >>> I have a simple question to which I could not find an answer. >>> What is the total maximal size of list including size of its elements? >>> I do not like to look into python source. >> >> But it would answer that question pretty fast. Because then you'd see >> that all list-object-methods are defined in terms of Py_ssize_t, which >> is an alias for ssize_t of your platform. 64bit that should be a 64bit long. >> >> Diez > > Then how do explain the program output? What exactly? That after 3GB it ran out of memory? Because you don't have 4GB memory available for processes. Diez From tjreedy at udel.edu Mon Jan 25 15:07:09 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 25 Jan 2010 15:07:09 -0500 Subject: Total maximal size of data In-Reply-To: <4B5DEB78.10406@fnal.gov> References: <4B5DEB78.10406@fnal.gov> Message-ID: On 1/25/2010 2:05 PM, Alexander Moibenko wrote: > I have a simple question to which I could not find an answer. Because it has no finite answer > What is the total maximal size of list including size of its elements? In theory, unbounded. In practice, limited by the memory of the interpreter. The maximum # of elements depends on the interpreter. Each element can be a list whose maximum # of elements ..... and recursively so on... Terry Jan Reedy From anm12345 at gmail.com Mon Jan 25 15:15:09 2010 From: anm12345 at gmail.com (AlexM) Date: Mon, 25 Jan 2010 12:15:09 -0800 (PST) Subject: Total maximal size of data References: <7s69efF2u0U1@mid.uni-berlin.de> <96b3f223-2f5f-4af1-b896-2d926986441f@g29g2000yqe.googlegroups.com> <7s6bp7Fgl6U1@mid.uni-berlin.de> Message-ID: On Jan 25, 2:03?pm, "Diez B. Roggisch" wrote: > Am 25.01.10 20:39, schrieb AlexM: > > > On Jan 25, 1:23 pm, "Diez B. Roggisch" ?wrote: > >> Am 25.01.10 20:05, schrieb Alexander Moibenko: > > >>> I have a simple question to which I could not find an answer. > >>> What is the total maximal size of list including size of its elements? > >>> I do not like to look into python source. > > >> But it would answer that question pretty fast. Because then you'd see > >> that all list-object-methods are defined in terms of Py_ssize_t, which > >> is an alias for ssize_t of your platform. 64bit that should be a 64bit long. > > >> Diez > > > Then how do explain the program output? > > What exactly? That after 3GB it ran out of memory? Because you don't > have 4GB memory available for processes. > > Diez Did you see my posting? .... Here is what I get on 32-bit architecture: cat /proc/meminfo MemTotal: 8309860 kB MemFree: 5964888 kB Buffers: 84396 kB Cached: 865644 kB SwapCached: 0 kB ..... I have more than 5G in memory not speaking of swap space. From anm12345 at gmail.com Mon Jan 25 15:21:48 2010 From: anm12345 at gmail.com (AlexM) Date: Mon, 25 Jan 2010 12:21:48 -0800 (PST) Subject: Total maximal size of data References: <4B5DEB78.10406@fnal.gov> Message-ID: <48f81b87-661e-4b3b-9dd6-3511bc84f607@36g2000yqu.googlegroups.com> On Jan 25, 2:07?pm, Terry Reedy wrote: > On 1/25/2010 2:05 PM, Alexander Moibenko wrote: > > > I have a simple question to which I could not find an answer. > > Because it has no finite answer > > > What is the total maximal size of list including size of its elements? > > In theory, unbounded. In practice, limited by the memory of the interpreter. > > The maximum # of elements depends on the interpreter. Each element can > be a list whose maximum # of elements ..... and recursively so on... > > Terry Jan Reedy I am not asking about maximum numbers of elements I am asking about total maximal size of list including size of its elements. In other words: if size of each list element is ELEMENT_SIZE and all elements have the same size what would be the maximal number of these elements in 32 - bit architecture? I see 3 GB, and wonder why? Why not 2 GB or not 4 GB? AlexM AlexM From alfps at start.no Mon Jan 25 15:37:51 2010 From: alfps at start.no (Alf P. Steinbach) Date: Mon, 25 Jan 2010 21:37:51 +0100 Subject: Total maximal size of data In-Reply-To: <48f81b87-661e-4b3b-9dd6-3511bc84f607@36g2000yqu.googlegroups.com> References: <4B5DEB78.10406@fnal.gov> <48f81b87-661e-4b3b-9dd6-3511bc84f607@36g2000yqu.googlegroups.com> Message-ID: * AlexM: > On Jan 25, 2:07 pm, Terry Reedy wrote: >> On 1/25/2010 2:05 PM, Alexander Moibenko wrote: >> >>> I have a simple question to which I could not find an answer. >> Because it has no finite answer >> >>> What is the total maximal size of list including size of its elements? >> In theory, unbounded. In practice, limited by the memory of the interpreter. >> >> The maximum # of elements depends on the interpreter. Each element can >> be a list whose maximum # of elements ..... and recursively so on... >> >> Terry Jan Reedy > > I am not asking about maximum numbers of elements I am asking about > total maximal size of list including size of its elements. In other > words: > if size of each list element is ELEMENT_SIZE and all elements have the > same size what would be the maximal number of these elements in 32 - > bit architecture? > I see 3 GB, and wonder why? Why not 2 GB or not 4 GB? At a guess you were running this in 32-bit Windows. By default it reserves the upper two gig of address space for mapping system DLLs. It can be configured to use just 1 gig for that, and it seems like your system is, or you're using some other system with that kind of behavior, or, it's just arbitrary... Cheers & hth., - Alf (by what mechanism do socks disappear from the washer?) From deets at nospam.web.de Mon Jan 25 15:42:35 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Mon, 25 Jan 2010 21:42:35 +0100 Subject: Total maximal size of data In-Reply-To: References: <7s69efF2u0U1@mid.uni-berlin.de> <96b3f223-2f5f-4af1-b896-2d926986441f@g29g2000yqe.googlegroups.com> <7s6bp7Fgl6U1@mid.uni-berlin.de> Message-ID: <7s6e1sFu8nU1@mid.uni-berlin.de> Am 25.01.10 21:15, schrieb AlexM: > On Jan 25, 2:03 pm, "Diez B. Roggisch" wrote: >> Am 25.01.10 20:39, schrieb AlexM: >> >>> On Jan 25, 1:23 pm, "Diez B. Roggisch" wrote: >>>> Am 25.01.10 20:05, schrieb Alexander Moibenko: >> >>>>> I have a simple question to which I could not find an answer. >>>>> What is the total maximal size of list including size of its elements? >>>>> I do not like to look into python source. >> >>>> But it would answer that question pretty fast. Because then you'd see >>>> that all list-object-methods are defined in terms of Py_ssize_t, which >>>> is an alias for ssize_t of your platform. 64bit that should be a 64bit long. >> >>>> Diez >> >>> Then how do explain the program output? >> >> What exactly? That after 3GB it ran out of memory? Because you don't >> have 4GB memory available for processes. >> >> Diez > > Did you see my posting? > .... > Here is what I get on 32-bit architecture: > cat /proc/meminfo > MemTotal: 8309860 kB > MemFree: 5964888 kB > Buffers: 84396 kB > Cached: 865644 kB > SwapCached: 0 kB > ..... > > I have more than 5G in memory not speaking of swap space. Yes, I saw your posting. 32Bit is 32Bit. Do you know about PAE? http://de.wikipedia.org/wiki/Physical_Address_Extension Just because the system can deal with more overall memory - one process can't get more than 4 GB (or even less, through re-mapped memory). Except it uses specific APIs like the old hi-mem-stuff under DOS. Diez From anm12345 at gmail.com Mon Jan 25 15:49:05 2010 From: anm12345 at gmail.com (AlexM) Date: Mon, 25 Jan 2010 12:49:05 -0800 (PST) Subject: Total maximal size of data References: <4B5DEB78.10406@fnal.gov> <48f81b87-661e-4b3b-9dd6-3511bc84f607@36g2000yqu.googlegroups.com> Message-ID: <662d9b37-ac93-4360-81d9-1562b0235836@o28g2000yqh.googlegroups.com> On Jan 25, 2:37?pm, "Alf P. Steinbach" wrote: > * AlexM: > > > > > On Jan 25, 2:07 pm, Terry Reedy wrote: > >> On 1/25/2010 2:05 PM, Alexander Moibenko wrote: > > >>> I have a simple question to which I could not find an answer. > >> Because it has no finite answer > > >>> What is the total maximal size of list including size of its elements? > >> In theory, unbounded. In practice, limited by the memory of the interpreter. > > >> The maximum # of elements depends on the interpreter. Each element can > >> be a list whose maximum # of elements ..... and recursively so on... > > >> Terry Jan Reedy > > > I am not asking about maximum numbers of elements I am asking about > > total maximal size of list including size of its elements. In other > > words: > > if size of each list element is ELEMENT_SIZE and all elements have the > > same size what would be the maximal number of these elements in 32 - > > bit architecture? > > I see 3 GB, and wonder why? Why not 2 GB or not 4 GB? > > At a guess you were running this in 32-bit Windows. By default it reserves the > upper two gig of address space for mapping system DLLs. It can be configured to > use just 1 gig for that, and it seems like your system is, or you're using some > other system with that kind of behavior, or, it's just arbitrary... > > Cheers & hth., > > - Alf (by what mechanism do socks disappear from the washer?) No, it is 32-bit Linux. Alex From hansmu at xs4all.nl Mon Jan 25 15:55:03 2010 From: hansmu at xs4all.nl (Hans Mulder) Date: Mon, 25 Jan 2010 21:55:03 +0100 Subject: Terminal application with non-standard print In-Reply-To: References: Message-ID: <4b5e059f$0$22944$e4fe514c@news.xs4all.nl> Grant Edwards wrote: > On 2010-01-24, R?mi wrote: > >> I would like to do a Python application that prints data to stdout, but >> not the common way. I do not want the lines to be printed after each >> other, but the old lines to be replaced with the new ones, like wget >> does it for example (when downloading a file you can see the percentage >> increasing on a same line). > > sys.stdout.write("Here's the first line") > time.sleep(1) > sys.stdout.write("\rAnd this line replaces it.") That does not work on my system, because sys.stdout is line buffered. This causes both strings to be written when sys.stdout is closed because Python is shutting down. This works better: import sys, time sys.stdout.write("Here's the first line") sys.stdout.flush() time.sleep(1) sys.stdout.write("\rAnd this line replaces it.") sys.stdout.flush() Hope this helps, -- HansM From deets at nospam.web.de Mon Jan 25 15:56:58 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Mon, 25 Jan 2010 21:56:58 +0100 Subject: Total maximal size of data In-Reply-To: <662d9b37-ac93-4360-81d9-1562b0235836@o28g2000yqh.googlegroups.com> References: <4B5DEB78.10406@fnal.gov> <48f81b87-661e-4b3b-9dd6-3511bc84f607@36g2000yqu.googlegroups.com> <662d9b37-ac93-4360-81d9-1562b0235836@o28g2000yqh.googlegroups.com> Message-ID: <7s6esqF3q7U1@mid.uni-berlin.de> Am 25.01.10 21:49, schrieb AlexM: > On Jan 25, 2:37 pm, "Alf P. Steinbach" wrote: >> * AlexM: >> >> >> >>> On Jan 25, 2:07 pm, Terry Reedy wrote: >>>> On 1/25/2010 2:05 PM, Alexander Moibenko wrote: >> >>>>> I have a simple question to which I could not find an answer. >>>> Because it has no finite answer >> >>>>> What is the total maximal size of list including size of its elements? >>>> In theory, unbounded. In practice, limited by the memory of the interpreter. >> >>>> The maximum # of elements depends on the interpreter. Each element can >>>> be a list whose maximum # of elements ..... and recursively so on... >> >>>> Terry Jan Reedy >> >>> I am not asking about maximum numbers of elements I am asking about >>> total maximal size of list including size of its elements. In other >>> words: >>> if size of each list element is ELEMENT_SIZE and all elements have the >>> same size what would be the maximal number of these elements in 32 - >>> bit architecture? >>> I see 3 GB, and wonder why? Why not 2 GB or not 4 GB? >> >> At a guess you were running this in 32-bit Windows. By default it reserves the >> upper two gig of address space for mapping system DLLs. It can be configured to >> use just 1 gig for that, and it seems like your system is, or you're using some >> other system with that kind of behavior, or, it's just arbitrary... >> >> Cheers& hth., >> >> - Alf (by what mechanism do socks disappear from the washer?) > > No, it is 32-bit Linux. > Alex I already answered that (as did Alf, the principle applies for both OSs) - kernel memory space is mapped into the address-space, reducing it by 1 or 2 GB. Diez From zuo at chopin.edu.pl Mon Jan 25 15:57:48 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Mon, 25 Jan 2010 21:57:48 +0100 Subject: how can i know if a python object have a attribute such as 'attr1'? In-Reply-To: References: Message-ID: 24-01-2010, 00:38:29 Terry Reedy wrote: > On 1/23/2010 10:56 AM, Arnaud Delobelle wrote: >> thinke365 writes: >> >>> for example, i may define a python class: >>> class A: >>> def sayHello(): >>> print 'hello' >>> >>> a = A() >>> a.attr1 = 'hello' >>> a.attr2 = 'bb' >>> >>> b = A() >>> a.attr2 = 'aa' >>> >>> how can i know whether an object have an attribute named attr1? >> >> hasattr(a, 'attr1') > > or > try: a.attr1 > except AttributeError: pass or -- if you are interested only in attributes contained by attributes dict of this particular object (and no in attributes of its type, base types nor attributes calculated on-demand by __getattr__/__getattribute__ methods) -- you can check its __dict__ -- * using vars(), e.g.: if 'attr1' in vars(a)... * or directly (less elegant?), e.g.: if 'attr1' in a.__dict__... But please remember that it doesn't work for instances of types with __slots__ defined (see: http://docs.python.org/reference/datamodel.html#slots). Regards, *j -- Jan Kaliszewski (zuo) From no.email at nospam.invalid Mon Jan 25 16:00:02 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 25 Jan 2010 13:00:02 -0800 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> <7xpr4ylk9d.fsf@ruckus.brouhaha.com> Message-ID: <7xy6jlnbn1.fsf@ruckus.brouhaha.com> Steve Howell writes: > These are the reasons I am not using deque: Thanks for these. Now we are getting somewhere. > 1) I want to use native lists, so that downstream methods can use > them as lists. It sounds like that could be fixed by making the deque API a proper superset of the list API. > 2) Lists are faster for accessing elements. It sounds like that could be fixed by optimizing deque somewhat. Also, have you profiled your application to show that accessing list elements is actually using a significant fraction of its runtime and that it would be slowed down noticably by deque? If not, it's a red herring. > 3) I want to be able to insert elements into the middle of the list. I just checked, and was surprised to find that deque doesn't support this. I'd say go ahead and file a feature request to add it to deque. > 4) I have no need for rotating elements. That's unpersuasive since you're advocating adding a feature to list that many others have no need for. > Adding a word or two to a list is an O(1) addition to a data structure > that takes O(N) memory to begin with. Yes, as mentioned, additive constants matter. > Another way of looking at it is that you would need to have 250 or so > lists in memory at the same time before the extra pointer was even > costing you kilobytes of memory. I've often run applications with millions of lists, maybe tens of millions. Of course it would be 100's of millions if the machines were big enough. > My consumer laptop has 3027908k of memory. I thought the idea of buying bigger machines was to solve bigger problems, not to solve the same problems more wastefully. From anm12345 at gmail.com Mon Jan 25 16:22:17 2010 From: anm12345 at gmail.com (AlexM) Date: Mon, 25 Jan 2010 13:22:17 -0800 (PST) Subject: Total maximal size of data References: <7s69efF2u0U1@mid.uni-berlin.de> <96b3f223-2f5f-4af1-b896-2d926986441f@g29g2000yqe.googlegroups.com> <7s6bp7Fgl6U1@mid.uni-berlin.de> <7s6e1sFu8nU1@mid.uni-berlin.de> Message-ID: <8255c8da-4028-42a7-a48b-03ea8fccf971@v25g2000yqk.googlegroups.com> On Jan 25, 2:42?pm, "Diez B. Roggisch" wrote: > Am 25.01.10 21:15, schrieb AlexM: > > > > > On Jan 25, 2:03 pm, "Diez B. Roggisch" ?wrote: > >> Am 25.01.10 20:39, schrieb AlexM: > > >>> On Jan 25, 1:23 pm, "Diez B. Roggisch" ? ?wrote: > >>>> Am 25.01.10 20:05, schrieb Alexander Moibenko: > > >>>>> I have a simple question to which I could not find an answer. > >>>>> What is the total maximal size of list including size of its elements? > >>>>> I do not like to look into python source. > > >>>> But it would answer that question pretty fast. Because then you'd see > >>>> that all list-object-methods are defined in terms of Py_ssize_t, which > >>>> is an alias for ssize_t of your platform. 64bit that should be a 64bit long. > > >>>> Diez > > >>> Then how do explain the program output? > > >> What exactly? That after 3GB it ran out of memory? Because you don't > >> have 4GB memory available for processes. > > >> Diez > > > Did you see my posting? > > .... > > Here is what I get on 32-bit architecture: > > cat /proc/meminfo > > MemTotal: ? ? ?8309860 kB > > MemFree: ? ? ? 5964888 kB > > Buffers: ? ? ? ? 84396 kB > > Cached: ? ? ? ? 865644 kB > > SwapCached: ? ? ? ? ?0 kB > > ..... > > > I have more than 5G in memory not speaking of swap space. > > Yes, I saw your posting. 32Bit is 32Bit. Do you know about PAE? > > ? ?http://de.wikipedia.org/wiki/Physical_Address_Extension > > Just because the system can deal with more overall memory - one process > can't get more than 4 GB (or even less, through re-mapped memory). > Except it uses specific APIs like the old hi-mem-stuff under DOS. > > Diez Yes, I do. Good catch! I have PAE enabled, but I guess I have compiled python without extended memory. So I was looking in the wrong place. Thanks! AlexM From deets at nospam.web.de Mon Jan 25 16:31:49 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Mon, 25 Jan 2010 22:31:49 +0100 Subject: Total maximal size of data In-Reply-To: <8255c8da-4028-42a7-a48b-03ea8fccf971@v25g2000yqk.googlegroups.com> References: <7s69efF2u0U1@mid.uni-berlin.de> <96b3f223-2f5f-4af1-b896-2d926986441f@g29g2000yqe.googlegroups.com> <7s6bp7Fgl6U1@mid.uni-berlin.de> <7s6e1sFu8nU1@mid.uni-berlin.de> <8255c8da-4028-42a7-a48b-03ea8fccf971@v25g2000yqk.googlegroups.com> Message-ID: <7s6gu5FgdfU1@mid.uni-berlin.de> Am 25.01.10 22:22, schrieb AlexM: > On Jan 25, 2:42 pm, "Diez B. Roggisch" wrote: >> Am 25.01.10 21:15, schrieb AlexM: >> >> >> >>> On Jan 25, 2:03 pm, "Diez B. Roggisch" wrote: >>>> Am 25.01.10 20:39, schrieb AlexM: >> >>>>> On Jan 25, 1:23 pm, "Diez B. Roggisch" wrote: >>>>>> Am 25.01.10 20:05, schrieb Alexander Moibenko: >> >>>>>>> I have a simple question to which I could not find an answer. >>>>>>> What is the total maximal size of list including size of its elements? >>>>>>> I do not like to look into python source. >> >>>>>> But it would answer that question pretty fast. Because then you'd see >>>>>> that all list-object-methods are defined in terms of Py_ssize_t, which >>>>>> is an alias for ssize_t of your platform. 64bit that should be a 64bit long. >> >>>>>> Diez >> >>>>> Then how do explain the program output? >> >>>> What exactly? That after 3GB it ran out of memory? Because you don't >>>> have 4GB memory available for processes. >> >>>> Diez >> >>> Did you see my posting? >>> .... >>> Here is what I get on 32-bit architecture: >>> cat /proc/meminfo >>> MemTotal: 8309860 kB >>> MemFree: 5964888 kB >>> Buffers: 84396 kB >>> Cached: 865644 kB >>> SwapCached: 0 kB >>> ..... >> >>> I have more than 5G in memory not speaking of swap space. >> >> Yes, I saw your posting. 32Bit is 32Bit. Do you know about PAE? >> >> http://de.wikipedia.org/wiki/Physical_Address_Extension >> >> Just because the system can deal with more overall memory - one process >> can't get more than 4 GB (or even less, through re-mapped memory). >> Except it uses specific APIs like the old hi-mem-stuff under DOS. >> >> Diez > > Yes, I do. Good catch! I have PAE enabled, but I guess I have compiled > python without extended memory. So I was looking in the wrong place. You can't compile it with PAE. It's an extension that doesn't make sense in a general purpose language. It is used by Databases or some such, that can hold large structures in memory that don't need random access, but can cope with windowing. Diez From arnodel at googlemail.com Mon Jan 25 16:32:10 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 25 Jan 2010 21:32:10 +0000 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> Message-ID: Steve Howell writes: [...] > My algorithm does exactly N pops and roughly N list accesses, so I > would be going from N*N + N to N + N log N if switched to blist. Can you post your algorithm? It would be interesting to have a concrete use case to base this discussion on. -- Arnaud From jeremy+complangpythonannounce at jeremysanders.net Mon Jan 25 16:42:52 2010 From: jeremy+complangpythonannounce at jeremysanders.net (Jeremy Sanders) Date: Mon, 25 Jan 2010 21:42:52 +0000 Subject: ANN: Veusz 1.6 Message-ID: Veusz 1.6 --------- Velvet Ember Under Sky Zenith ----------------------------- http://home.gna.org/veusz/ Veusz is Copyright (C) 2003-2010 Jeremy Sanders Licenced under the GPL (version 2 or greater). Veusz is a Qt4 based scientific plotting package. It is written in Python, using PyQt4 for display and user-interfaces, and numpy for handling the numeric data. Veusz is designed to produce publication-ready Postscript/PDF/SVG output. The user interface aims to be simple, consistent and powerful. Veusz provides a GUI, command line, embedding and scripting interface (based on Python) to its plotting facilities. It also allows for manipulation and editing of datasets. Data can be captured from external sources such as internet sockets or other programs. Changes in 1.6: * User defined constants, functions or external Python imports can be defined for use when evaluating expressions. * Import descriptor is much more tolerant of syntax, e.g. "x,+- y,+,-" can now be specified as "x +- y + -". * New SVG export (PyQt >= 4.6). Supports clipping and exports text as paths for full WYSIWYG. * Dataset names can now contain any character except "`". Names containing non-alphanumeric characters can be quoted in expressions `like so`*1.23 * Widget names can contain any character except "/" * A transparency dataset can be provided to specify the per-pixel transparency of the image widget. * A polygon widget has been added. * There is a new option to place axis ticks outside the plot (outer ticks setting on axis widget) * Several new line styles have been added. * Several new plotting markers have been added. * The capture dialog can optionally retain the last N values captured. Minor changes: * Use of flat cap line style for plotting error bars for exactness. * Add fixes for saving imported unicode text. * Fix image colors for big endian systems (e.g. Mac PPC). * Add boxfill error bar style, plotting errors as filled boxes. * Positive and negative error bars are forced to have the correct sign. Features of package: * X-Y plots (with errorbars) * Line and function plots * Contour plots * Images (with colour mappings and colorbars) * Stepped plots (for histograms) * Bar graphs * Plotting dates * Fitting functions to data * Stacked plots and arrays of plots * Plot keys * Plot labels * Shapes and arrows on plots * LaTeX-like formatting for text * EPS/PDF/PNG/SVG/EMF export * Scripting interface * Dataset creation/manipulation * Embed Veusz within other programs * Text, CSV and FITS importing * Data can be captured from external sources Requirements for source install: Python (2.4 or greater required) http://www.python.org/ Qt >= 4.3 (free edition) http://www.trolltech.com/products/qt/ PyQt >= 4.3 (SIP is required to be installed first) http://www.riverbankcomputing.co.uk/pyqt/ http://www.riverbankcomputing.co.uk/sip/ numpy >= 1.0 http://numpy.scipy.org/ Optional: Microsoft Core Fonts (recommended for nice output) http://corefonts.sourceforge.net/ PyFITS >= 1.1 (optional for FITS import) http://www.stsci.edu/resources/software_hardware/pyfits pyemf >= 2.0.0 (optional for EMF export) http://pyemf.sourceforge.net/ For EMF and better SVG export, PyQt >= 4.6 or better is required, to fix a bug in the C++ wrapping For documentation on using Veusz, see the "Documents" directory. The manual is in PDF, HTML and text format (generated from docbook). The examples are also useful documentation. Issues with the current version: * Due to Qt, hatched regions sometimes look rather poor when exported to PostScript, PDF or SVG. * Due to a bug in Qt, some long lines, or using log scales, can lead to very slow plot times under X11. It is fixed by upgrading to Qt-4.5.1 (or using a binary). Switching off antialiasing in the options may help. If you enjoy using Veusz, I would love to hear from you. Please join the mailing lists at https://gna.org/mail/?group=veusz to discuss new features or if you'd like to contribute code. The latest code can always be found in the SVN repository. Jeremy Sanders From apt.shansen at gmail.com Mon Jan 25 16:43:30 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Mon, 25 Jan 2010 13:43:30 -0800 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> <7xpr4ylk9d.fsf@ruckus.brouhaha.com> Message-ID: <7a9c25c21001251343q7d1ee864oafcb0e58e1c3b71e@mail.gmail.com> On Mon, Jan 25, 2010 at 9:31 AM, Steve Howell wrote: > Another way of looking at it is that you would need to have 250 or so > lists in memory at the same time before the extra pointer was even > costing you kilobytes of memory. My consumer laptop has 3027908k of > memory. > Umm, I think the issue here is that some people have use-cases which are talking of number of lists whole orders of magnitude higher then you're talking about here. In your program, maybe you only count the number of lists in the hundreds, and so a few extra words wouldn't matter. I have applications that have hundreds of thousands to millions of lists in memory-- and which have to be managed somewhat carefully to avoid the 32-bit memory allocation limit not smacking them (64-bit python isn't an option for me presently). I've never had an algorithm which needed to pop off the top of a list that I couldn't with utter triviality simply operate in the reverse. If Python's gonna get more memory hungry, I'd like to see how it benefits me in some way. I mean, Unladen Swallow is talking about boosting Python's memory need for the JIT, but I'm getting distinct performance improvements out of that. That sounds like a fair trade. You want Python to eat up a few more of my megs that I'd rather put to use elsewhere, because... you don't want to just reverse your algorithm to treat the FIFO as a LILO? Sure, I can break my program up to run in separate processes and double how much data I can have at once, with some IPC overhead. And if I got something out of it, I'd be happy to! Or you can alter your algorithm. Why must I be the one to change? :) --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From anm12345 at gmail.com Mon Jan 25 16:46:17 2010 From: anm12345 at gmail.com (AlexM) Date: Mon, 25 Jan 2010 13:46:17 -0800 (PST) Subject: Total maximal size of data References: <7s69efF2u0U1@mid.uni-berlin.de> <96b3f223-2f5f-4af1-b896-2d926986441f@g29g2000yqe.googlegroups.com> <7s6bp7Fgl6U1@mid.uni-berlin.de> <7s6e1sFu8nU1@mid.uni-berlin.de> <8255c8da-4028-42a7-a48b-03ea8fccf971@v25g2000yqk.googlegroups.com> <7s6gu5FgdfU1@mid.uni-berlin.de> Message-ID: <9fb9ce60-7d0e-4990-a7ca-b23751bdaabe@f12g2000yqn.googlegroups.com> On Jan 25, 3:31?pm, "Diez B. Roggisch" wrote: > Am 25.01.10 22:22, schrieb AlexM: > > > > > On Jan 25, 2:42 pm, "Diez B. Roggisch" ?wrote: > >> Am 25.01.10 21:15, schrieb AlexM: > > >>> On Jan 25, 2:03 pm, "Diez B. Roggisch" ? ?wrote: > >>>> Am 25.01.10 20:39, schrieb AlexM: > > >>>>> On Jan 25, 1:23 pm, "Diez B. Roggisch" ? ? ?wrote: > >>>>>> Am 25.01.10 20:05, schrieb Alexander Moibenko: > > >>>>>>> I have a simple question to which I could not find an answer. > >>>>>>> What is the total maximal size of list including size of its elements? > >>>>>>> I do not like to look into python source. > > >>>>>> But it would answer that question pretty fast. Because then you'd see > >>>>>> that all list-object-methods are defined in terms of Py_ssize_t, which > >>>>>> is an alias for ssize_t of your platform. 64bit that should be a 64bit long. > > >>>>>> Diez > > >>>>> Then how do explain the program output? > > >>>> What exactly? That after 3GB it ran out of memory? Because you don't > >>>> have 4GB memory available for processes. > > >>>> Diez > > >>> Did you see my posting? > >>> .... > >>> Here is what I get on 32-bit architecture: > >>> cat /proc/meminfo > >>> MemTotal: ? ? ?8309860 kB > >>> MemFree: ? ? ? 5964888 kB > >>> Buffers: ? ? ? ? 84396 kB > >>> Cached: ? ? ? ? 865644 kB > >>> SwapCached: ? ? ? ? ?0 kB > >>> ..... > > >>> I have more than 5G in memory not speaking of swap space. > > >> Yes, I saw your posting. 32Bit is 32Bit. Do you know about PAE? > > >> ? ?http://de.wikipedia.org/wiki/Physical_Address_Extension > > >> Just because the system can deal with more overall memory - one process > >> can't get more than 4 GB (or even less, through re-mapped memory). > >> Except it uses specific APIs like the old hi-mem-stuff under DOS. > > >> Diez > > > Yes, I do. Good catch! I have PAE enabled, but I guess I have compiled > > python without extended memory. So I was looking in the wrong place. > > You can't compile it with PAE. It's an extension that doesn't make sense > in a general purpose language. It is used by Databases or some such, > that can hold large structures in memory that don't need random access, > but can cope with windowing. > > Diez Well, there actually is a way of building programs that may use more than 4GB of memory on 32 machines for Linux with higmem kernels, but I guess this would not work for python. I'll just switch to 64-bit architecture. Thanks again. AlexM From peteRE at MpeteOzilla.Vco.ukE Mon Jan 25 17:04:18 2010 From: peteRE at MpeteOzilla.Vco.ukE (Peter Chant) Date: Mon, 25 Jan 2010 22:04:18 +0000 Subject: Python, PIL and 16 bit per channel images Message-ID: <24h137-e99.ln1@phoenix.fire> Does anyone know whether PIL can handle 16 bit per channel RGB images? PyPNG site (http://packages.python.org/pypng/ca.html) states PIL uses 8 bits per channel internally. Thanks, Pete -- http://www.petezilla.co.uk From showell30 at yahoo.com Mon Jan 25 17:05:59 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 14:05:59 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> Message-ID: <7655fa8f-bdc1-4bfd-8bd0-f3bba2babc91@a6g2000yqm.googlegroups.com> On Jan 25, 1:32?pm, Arnaud Delobelle wrote: > Steve Howell writes: > > [...] > > > My algorithm does exactly N pops and roughly N list accesses, so I > > would be going from N*N + N to N + N log N if switched to blist. > > Can you post your algorithm? ?It would be interesting to have a concrete > use case to base this discussion on. > It is essentially this, in list_ass_slice: if (d < 0) { /* Delete -d items */ if (ilow == 0) { a->popped -= d; a->ob_item -= d * sizeof(PyObject *); list_resize(a, Py_SIZE(a)); } else { memmove(&item[ihigh+d], &item[ihigh], (Py_SIZE(a) - ihigh)*sizeof(PyObject *)); list_resize(a, Py_SIZE(a) + d); } item = a->ob_item; } I am still working through the memory management issues, but when I have a complete working patch, I will give more detail. From showell30 at yahoo.com Mon Jan 25 17:09:00 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 14:09:00 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> Message-ID: <7ea02185-0bc1-4f49-9931-0974d8a0db5c@a5g2000yqi.googlegroups.com> On Jan 25, 1:32?pm, Arnaud Delobelle wrote: > Steve Howell writes: > > [...] > > > My algorithm does exactly N pops and roughly N list accesses, so I > > would be going from N*N + N to N + N log N if switched to blist. > > Can you post your algorithm? ?It would be interesting to have a concrete > use case to base this discussion on. > I just realized you meant the Python code itself. It is here: https://bitbucket.org/showell/shpaml_website/src/tip/shpaml.py From deets at nospam.web.de Mon Jan 25 17:12:29 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Mon, 25 Jan 2010 23:12:29 +0100 Subject: Total maximal size of data In-Reply-To: <9fb9ce60-7d0e-4990-a7ca-b23751bdaabe@f12g2000yqn.googlegroups.com> References: <7s69efF2u0U1@mid.uni-berlin.de> <96b3f223-2f5f-4af1-b896-2d926986441f@g29g2000yqe.googlegroups.com> <7s6bp7Fgl6U1@mid.uni-berlin.de> <7s6e1sFu8nU1@mid.uni-berlin.de> <8255c8da-4028-42a7-a48b-03ea8fccf971@v25g2000yqk.googlegroups.com> <7s6gu5FgdfU1@mid.uni-berlin.de> <9fb9ce60-7d0e-4990-a7ca-b23751bdaabe@f12g2000yqn.googlegroups.com> Message-ID: <7s6jadFmfdU1@mid.uni-berlin.de> > > Well, there actually is a way of building programs that may use more > than 4GB of memory on 32 machines for Linux with higmem kernels, but I > guess this would not work for python. As I said, it's essentially paging: http://kerneltrap.org/node/2450 And it's not something you can just compile in, you need explicit code-support for it. Which python hasn't. And most other programs. So there is not a magic compile option. > I'll just switch to 64-bit architecture. That's the solution, yes :) Diez From invalid at invalid.invalid Mon Jan 25 17:20:54 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Mon, 25 Jan 2010 22:20:54 +0000 (UTC) Subject: Terminal application with non-standard print References: <4b5e059f$0$22944$e4fe514c@news.xs4all.nl> Message-ID: On 2010-01-25, Hans Mulder wrote: > Grant Edwards wrote: >> On 2010-01-24, R?mi wrote: >> >>> I would like to do a Python application that prints data to stdout, but >>> not the common way. I do not want the lines to be printed after each >>> other, but the old lines to be replaced with the new ones, like wget >>> does it for example (when downloading a file you can see the percentage >>> increasing on a same line). >> >> sys.stdout.write("Here's the first line") >> time.sleep(1) >> sys.stdout.write("\rAnd this line replaces it.") > > That does not work on my system, because sys.stdout is line buffered. That's correct of course. > This causes both strings to be written when sys.stdout is closed because > Python is shutting down. > > This works better: > > import sys, time > > sys.stdout.write("Here's the first line") > sys.stdout.flush() > time.sleep(1) > sys.stdout.write("\rAnd this line replaces it.") > sys.stdout.flush() Or you can tell Python to do unbuffered output: #!/usr/bin/python -u -- Grant Edwards grante Yow! I'm using my X-RAY at VISION to obtain a rare visi.com glimpse of the INNER WORKINGS of this POTATO!! From george.trojan at noaa.gov Mon Jan 25 17:27:19 2010 From: george.trojan at noaa.gov (George Trojan) Date: Mon, 25 Jan 2010 22:27:19 +0000 Subject: site.py confusion Message-ID: Inspired by the 'Default path for files' thread I tried to use sitecustomize in my code. What puzzles me is that the site.py's main() is not executed. My sitecustomize.py is def main(): print 'In Main()' main() and the test program is import site #site.main() print 'Hi' The output is $ python try.py Hi When I uncomment the site.main() line the output is $ python try.py In Main() Hi If I change import site to import sitecustomize the output is as above. What gives? Adding to the confusion, I found http://code.activestate.com/recipes/552729/ which contradicts http://docs.python.org/library/site.html George From half.italian at gmail.com Mon Jan 25 17:30:58 2010 From: half.italian at gmail.com (Sean DiZazzo) Date: Mon, 25 Jan 2010 14:30:58 -0800 (PST) Subject: Terminal application with non-standard print References: Message-ID: On Jan 24, 11:27?am, R?mi wrote: > Hello everyone, > > I would like to do a Python application that prints data to stdout, but > not the common way. I do not want the lines to be printed after each > other, but the old lines to be replaced with the new ones, like wget > does it for example (when downloading a file you can see the percentage > increasing on a same line). > > I looked into the curses module, but this seems adapted only to do a > whole application, and the terminal history is not visible anymore when > the application starts. > > Any ideas? > > Thanks, > > Remi You might want to take a look at the readline module. ~Sean From sccolbert at gmail.com Mon Jan 25 17:42:38 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Mon, 25 Jan 2010 17:42:38 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <7ea02185-0bc1-4f49-9931-0974d8a0db5c@a5g2000yqi.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> <7ea02185-0bc1-4f49-9931-0974d8a0db5c@a5g2000yqi.googlegroups.com> Message-ID: <7f014ea61001251442y430019d8qdee626cff1f01a7f@mail.gmail.com> On Mon, Jan 25, 2010 at 5:09 PM, Steve Howell wrote: > On Jan 25, 1:32 pm, Arnaud Delobelle wrote: > > Steve Howell writes: > > > > [...] > > > > > My algorithm does exactly N pops and roughly N list accesses, so I > > > would be going from N*N + N to N + N log N if switched to blist. > > > > Can you post your algorithm? It would be interesting to have a concrete > > use case to base this discussion on. > > > > I just realized you meant the Python code itself. It is here: > > https://bitbucket.org/showell/shpaml_website/src/tip/shpaml.py > > -- > http://mail.python.org/mailman/listinfo/python-list > looking at that code, i think you could solve your whole problem with a single called to reversed() (which is NOT the same as list.reverse()) -------------- next part -------------- An HTML attachment was scrubbed... URL: From sccolbert at gmail.com Mon Jan 25 17:57:07 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Mon, 25 Jan 2010 17:57:07 -0500 Subject: Python, PIL and 16 bit per channel images In-Reply-To: <24h137-e99.ln1@phoenix.fire> References: <24h137-e99.ln1@phoenix.fire> Message-ID: <7f014ea61001251457u33db2a6at981f55503065ef55@mail.gmail.com> On Mon, Jan 25, 2010 at 5:04 PM, Peter Chant wrote: > Does anyone know whether PIL can handle 16 bit per channel RGB images? > PyPNG site (http://packages.python.org/pypng/ca.html) states PIL uses 8 > bits > per channel internally. > > Thanks, > > Pete > > > -- > http://www.petezilla.co.uk > > -- > http://mail.python.org/mailman/listinfo/python-list > Mode The mode of an image defines the type and depth of a pixel in the image. The current release supports the following standard modes: - *1* (1-bit pixels, black and white, stored with one pixel per byte) - *L* (8-bit pixels, black and white) - *P* (8-bit pixels, mapped to any other mode using a colour palette) - *RGB* (3x8-bit pixels, true colour) - *RGBA* (4x8-bit pixels, true colour with transparency mask) - *CMYK* (4x8-bit pixels, colour separation) - *YCbCr* (3x8-bit pixels, colour video format) - *I* (32-bit signed integer pixels) - *F* (32-bit floating point pixels) http://www.pythonware.com/library/pil/handbook/concepts.htm -------------- next part -------------- An HTML attachment was scrubbed... URL: From showell30 at yahoo.com Mon Jan 25 18:00:48 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 15:00:48 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> <7xpr4ylk9d.fsf@ruckus.brouhaha.com> <7xy6jlnbn1.fsf@ruckus.brouhaha.com> Message-ID: <81319a79-cf25-4444-ab71-bfe982ae7fee@m16g2000yqc.googlegroups.com> On Jan 25, 1:00?pm, Paul Rubin wrote: > Steve Howell writes: > > These are the reasons I am not using deque: > > Thanks for these. ?Now we are getting somewhere. > > > ? 1) I want to use native lists, so that downstream methods can use > > them as lists. > > It sounds like that could be fixed by making the deque API a proper > superset of the list API. That is probably a good idea. > > ? 2) Lists are faster for accessing elements. > > It sounds like that could be fixed by optimizing deque somewhat. ?Also, > have you profiled your application to show that accessing list elements > is actually using a significant fraction of its runtime and that it > would be slowed down noticably by deque? ?If not, it's a red herring. I haven't profiled deque vs. list, but I think you are correct about pop() possibly being a red herring. It appears that the main bottleneck might still be the processing I do on each line of text, which in my cases is regexes. For really large lists, I suppose memmove() would eventually start to become a bottleneck, but it's brutally fast when it just moves a couple kilobytes of data around. > > ? 3) I want to be able to insert elements into the middle of the list. > > I just checked, and was surprised to find that deque doesn't support > this. ?I'd say go ahead and file a feature request to add it to deque. > It might be a good thing to add just for consistency sake. If somebody first implements an algorithm with lists, then discovers it has overhead relating to inserting/appending at the end of the list, then the more deque behaves like a list, the more easily they could switch over their code to deque. Not knowing much about deque's internals, I assume its performance for insert() would O(N) just like list, although maybe a tiny bit slower. > > ? 4) I have no need for rotating elements. > > That's unpersuasive since you're advocating adding a feature to list > that many others have no need for. ? > To be precise, I wasn't really advocating for a new feature but an internal optimization of a feature that already exists. > > Adding a word or two to a list is an O(1) addition to a data structure > > that takes O(N) memory to begin with. > > Yes, as mentioned, additive constants matter. > > > Another way of looking at it is that you would need to have 250 or so > > lists in memory at the same time before the extra pointer was even > > costing you kilobytes of memory. > > I've often run applications with millions of lists, maybe tens of > millions. ?Of course it would be 100's of millions if the machines were > big enough. > I bet even in your application, the amount of memory consumed by the PyListObjects themselves is greatly dwarfed by other objects, notably the list elements themselves, not to mention any dictionaries that your app uses. > > My consumer laptop has 3027908k of memory. > > I thought the idea of buying bigger machines was to solve bigger > problems, not to solve the same problems more wastefully. Well, I am not trying to solve problems wastefully here. CPU cycles are also scarce, so it seems wasteful to do an O(N) memmove that could be avoided by storing an extra pointer per list. I also think that encouraging the use of pop(0) would actually make many programs more memory efficient, in the sense that you can garbage collect list elements earlier. Thanks for your patience in responding to me, despite the needlessly abrasive tone of my earlier postings. I am coming around to this thinking: 1) Summarize all this discussion and my lessons learned in some kind of document. It does not have to be a PEP per se, but I could provide a useful service to the community by listing pros/cons/etc. 2) I would still advocate for removing the warning against list.pop (0) from the tutorial. I agree with Steven D'Aprano that docs really should avoid describing implementation details in many instances (although I do not know what he thinks about this particular case). I also think that the performance penalty for pop(0) is negligible for most medium-sized programs. For large-sized programs where you really want to swap in deque, I think most authors are beyond reading the tutorial and are looking elsewhere for insight on Python data structures. 3) I am gonna try to implement the patch anyway for my own edification. 4) I do think that there are ways that deque could be improved, but it is not high on my priority list. I will try to mention it in the PEP, though. From showell30 at yahoo.com Mon Jan 25 18:30:16 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 15:30:16 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> Message-ID: <55d2d213-7779-4344-ad79-d9142c8d2a3e@a5g2000yqi.googlegroups.com> On Jan 25, 1:32?pm, Arnaud Delobelle wrote: > Steve Howell writes: > > [...] > > > My algorithm does exactly N pops and roughly N list accesses, so I > > would be going from N*N + N to N + N log N if switched to blist. > > Can you post your algorithm? ?It would be interesting to have a concrete > use case to base this discussion on. > These are the profile results for an admittedly very large file (430,000 lines), which shows that pop() consumes more time than any other low level method. So pop() is not a total red herring. But I have to be honest and admit that I grossly overestimated the penalty for smaller files. Typical files are a couple hundred lines, and for that use case, pop()'s expense gets totally drowned out by regex handling. In other words, it's a lot cheaper to move a couple hundred pointers per list element pop than it is to apply a series of regexes to them, which shouldn't be surprising. ncalls tottime percall cumtime percall filename:lineno (function) 230001/1 149.508 0.001 222.432 222.432 /home/showell/workspace/ shpaml_website/shpaml.py:192(recurse) 429999 17.667 0.000 17.667 0.000 {method 'pop' of 'list' objects} 530000 8.428 0.000 14.125 0.000 /home/showell/workspace/ shpaml_website/shpaml.py:143(get_indented_block) 3700008 7.877 0.000 7.877 0.000 {built-in method match} 5410125/5410121 5.697 0.000 5.697 0.000 {len} 300000 3.938 0.000 22.286 0.000 /home/showell/workspace/ shpaml_website/shpaml.py:96(convert_line) 959999 3.847 0.000 6.759 0.000 /home/showell/workspace/ shpaml_website/shpaml.py:29(INDENT) 959999 3.717 0.000 12.547 0.000 /home/showell/workspace/ shpaml_website/shpaml.py:138(find_indentation) 370000 3.495 0.000 20.204 0.000 /home/showell/workspace/ shpaml_website/shpaml.py:109(apply_jquery) 370000 3.322 0.000 6.528 0.000 {built-in method sub} 1469999 2.575 0.000 2.575 0.000 {built-in method groups} As an aside, I am a little surprised by how often I call len() and that it also takes a large chunk of time, but that's my problem to fix. From jim.waddle at boeing.com Mon Jan 25 18:36:32 2010 From: jim.waddle at boeing.com (Waddle, Jim) Date: Mon, 25 Jan 2010 15:36:32 -0800 Subject: ctypes for AIX In-Reply-To: <50697b2c1001240630r33845a1bl7f94b40a4f0c4436@mail.gmail.com> References: <9C240725D2ABD949A80B46DD93AB107042E8D44101@XCH-NW-08V.nw.nos.boeing.com> <50697b2c1001240630r33845a1bl7f94b40a4f0c4436@mail.gmail.com> Message-ID: <9C240725D2ABD949A80B46DD93AB107042E8D44574@XCH-NW-08V.nw.nos.boeing.com> Chris, Thanks for responding to my email. I apologize for the remark about python only being developed for windows. I got the impression when I was looking at the ActivePython web site and saw that the version of python that they had available was not supported on very many unix systems. I should not make general statement based on only one web site. After reading your email I decided to see for myself what the issue was about compiling python on AIX 5.3. This is the error I saw the first time I tried to use ctypes. Python 2.4.3 (#1, Jul 17 2006, 20:00:23) [C] on aix5 Type "help", "copyright", "credits" or "license" for more information. >>> import ctypes Traceback (most recent call last): File "", line 1, in ? ImportError: No module named ctypes This version of python was downloaded and installed from ActivePython and when I checked their webpage it states that ctypes is not available on AIX. I then figured I would get a new copy of python and install it on AIX. I downloaded python.2.5.5c2 from http://www.python.org. I did the configure and make which posted many errors in the ctypes function which I guess is the reason that is does not get include in the final make. an example of the build error I get when doing the make is: xlc_r -q64 -DNDEBUG -O -I. -I/s/users/cz030a/xferjunk/python/Python-2.5.5c2/./Include -Ibuild/temp.aix-5.3-2.5/libffi/inclu de -Ibuild/temp.aix-5.3-2.5/libffi -I/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/libffi/src -I/s/users/c z030a/xferjunk/ots/python2.5/include -I. -IInclude -I./Include -I/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Include -I/ s/users/cz030a/xferjunk/python/Python-2.5.5c2 -c /s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.c - o build/temp.aix-5.3-2.5/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.o "/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.c", line 2820.31: 1506-068 (W) Operation between ty pes "void*" and "int(*)(void)" is not allowed. "/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.c", line 3363.28: 1506-280 (W) Function argument as signment between types "int(*)(void)" and "void*" is not allowed. "/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.c", line 4768.67: 1506-280 (W) Function argument as signment between types "void*" and "void*(*)(void*,const void*,unsigned long)" is not allowed. "/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.c", line 4769.66: 1506-280 (W) Function argument as signment between types "void*" and "void*(*)(void*,int,unsigned long)" is not allowed. I do not have sufficient knowledge to know how to fix this. I would think that this error somehow is related to compiling on aix. If you have any suggestions on how to correct this problem , I would appreciate it Jim Waddle KIT-D 425-785-5194 -----Original Message----- From: chris at rebertia.com [mailto:chris at rebertia.com] On Behalf Of Chris Rebert Sent: Sunday, January 24, 2010 7:31 AM To: Waddle, Jim Cc: python-list at python.org Subject: Re: ctypes for AIX On Sun, Jan 24, 2010 at 5:54 AM, Waddle, Jim wrote: > I need to use ctypes with python running on AIX. According to the ctypes readme, ctypes is based on libffi, which according to its website, supports AIX for PowerPC64. So, perhaps you could state what the actual error or problem you're encountering is? It is theoretically possible the ctypes-bundled libffi is either outdated or had the AIX-specific bits removed; I don't know, I'm not a CPython dev. > It appears that python is being developed mostly for windows. No, not really; your statement is especially ironic considering one of Python's primary areas of use is for web applications as part of a LAMP stack. > Is there a policy concerning getting functions like ctypes working on AIX. No idea. Someone will probably chime in though. Cheers, Chris -- http://blog.rebertia.com From showell30 at yahoo.com Mon Jan 25 18:40:29 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 15:40:29 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <7f014ea61001251442y430019d8qdee626cff1f01a7f@mail.gmail.com> Message-ID: <7965.79187.qm@web33505.mail.mud.yahoo.com> --- On Mon, 1/25/10, Chris Colbert wrote: > > looking at that code, i think you could solve > your whole problem with a single called to reversed() (which > is NOT the same as list.reverse())? > I do not think that's actually true. It does no good to pop elements off a copy of the list if there is still code that refers to the original list. So I think you really do want list.reverse(). The problem with reversing the lists is that it gets sliced and diced and passed around to other methods, one of which, html_block_tag, recursively calls back to the main method. So you could say that everybody just has to work with a reversed list, but in my mind, that would be just backward and overly complicated. I am not completely ruling out the approach, though. The idea of modelling the program essentially as a stack has some validity, and it probably would run faster. https://bitbucket.org/showell/shpaml_website/src/tip/shpaml.py From debatem1 at gmail.com Mon Jan 25 18:45:00 2010 From: debatem1 at gmail.com (geremy condra) Date: Mon, 25 Jan 2010 18:45:00 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> <2ffb4ab8-ae8f-4b69-b449-56d0232a404b@y23g2000yqm.googlegroups.com> Message-ID: On Sat, Jan 23, 2010 at 4:38 AM, Alf P. Steinbach wrote: > Hm, it would be nice if the Python docs offered complexity (time) > guarantees in general... > > Cheers, > > - Alf This would be a very welcome improvement IMHO- especially in collections. Geremy Condra From ethan at stoneleaf.us Mon Jan 25 19:25:31 2010 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 25 Jan 2010 16:25:31 -0800 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <44ccd731-4a98-4eec-a270-08f66b192d93@l19g2000yqb.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> <036b0836$0$1357$c3e8da3@news.astraweb.com> <44ccd731-4a98-4eec-a270-08f66b192d93@l19g2000yqb.googlegroups.com> Message-ID: <4B5E367B.9060705@stoneleaf.us> Steve Howell wrote: >> On Sat, 23 Jan 2010 09:57:04 -0500, Roy Smith wrote: >>> So, we're right back to my statement earlier in this thread that the >>> docs are deficient in that they describe behavior with no hint about >>> cost. Given that, it should be no surprise that users make incorrect >>> assumptions about cost. No hint? Looking at the below snippet of docs -- "not efficient" and "slow" sound like pretty good hints to me. > Bringing this thread full circle, does it make sense to strike this > passage from the tutorial?: > > ''' > It is also possible to use a list as a queue, where the first element > added is the first element retrieved (?first-in, first-out?); however, > lists are not efficient for this purpose. While appends and pops from > the end of list are fast, doing inserts or pops from the beginning of > a list is slow (because all of the other elements have to be shifted > by one). > ''' > > I think points #3 and #6 possibly apply. Regarding points #2 and #4, > the tutorial is at least not overly technical or specific; it just > explains the requirement to shift other elements one by one in simple > layman's terms. > I think the paragraph is fine. Instead of waiting for the (hundreds of?) posts wondering why making a FIFO queue from a list is so slow, and what's wrong with Python, etc, etc, it points out up front that yes you can, and here's why you don't want to. This does not strike me as too much knowledge. ~Ethan~ From alfps at start.no Mon Jan 25 20:13:18 2010 From: alfps at start.no (Alf P. Steinbach) Date: Tue, 26 Jan 2010 02:13:18 +0100 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> <036b0836$0$1357$c3e8da3@news.astraweb.com> <44ccd731-4a98-4eec-a270-08f66b192d93@l19g2000yqb.googlegroups.com> Message-ID: * Ethan Furman: > Steve Howell wrote: >>> On Sat, 23 Jan 2010 09:57:04 -0500, Roy Smith wrote: >>>> So, we're right back to my statement earlier in this thread that the >>>> docs are deficient in that they describe behavior with no hint about >>>> cost. Given that, it should be no surprise that users make incorrect >>>> assumptions about cost. > > No hint? Looking at the below snippet of docs -- "not efficient" and > "slow" sound like pretty good hints to me. > >> Bringing this thread full circle, does it make sense to strike this >> passage from the tutorial?: >> >> ''' >> It is also possible to use a list as a queue, where the first element >> added is the first element retrieved (?first-in, first-out?); however, >> lists are not efficient for this purpose. While appends and pops from >> the end of list are fast, doing inserts or pops from the beginning of >> a list is slow (because all of the other elements have to be shifted >> by one). >> ''' >> >> I think points #3 and #6 possibly apply. Regarding points #2 and #4, >> the tutorial is at least not overly technical or specific; it just >> explains the requirement to shift other elements one by one in simple >> layman's terms. >> > > I think the paragraph is fine. Instead of waiting for the (hundreds > of?) posts wondering why making a FIFO queue from a list is so slow, and > what's wrong with Python, etc, etc, it points out up front that yes you > can, and here's why you don't want to. This does not strike me as too > much knowledge. Is the tutorial regarded as part of the language specification? I understand that the standard library docs are part (e.g. 'object' is only described there), and that at least some PEPs are. Cheers, - Alf From malaclypse2 at gmail.com Mon Jan 25 20:58:00 2010 From: malaclypse2 at gmail.com (Jerry Hill) Date: Mon, 25 Jan 2010 20:58:00 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <83082e19-9130-45a8-91f2-8601c1fdaac3@22g2000yqr.googlegroups.com> <2ffb4ab8-ae8f-4b69-b449-56d0232a404b@y23g2000yqm.googlegroups.com> Message-ID: <16651e81001251758j6fc2c2edj550edc25bbee96f@mail.gmail.com> On Sat, Jan 23, 2010 at 4:38 AM, Alf P. Steinbach wrote: > Hm, it would be nice if > the Python docs offered complexity (time) guarantees in general... Last time it came up, I don't think there was any core developer interest in putting complexity guarantees in the Python Language Reference. Some folks did document the behavior of most of the common CPython containers though: http://wiki.python.org/moin/TimeComplexity -- Jerry From siddhartha.veedaluru at gmail.com Mon Jan 25 23:17:51 2010 From: siddhartha.veedaluru at gmail.com (siddhartha veedaluru) Date: Tue, 26 Jan 2010 09:47:51 +0530 Subject: TypeError not caught by except statement Message-ID: <424b71ec1001252017i64d28375oc3644433c66ae40b@mail.gmail.com> Hi, except not able to caught the TypeError exception occured in the below code log.info("refer",ret) in the try block throws a TypeError which is not caught . Also sometimes process is getting hanged. ------------------------------------------------------------------------------------------------------------------------------------------------ import logging log = logging.getLogger() fileName = strftime("%d-%b-%Y-", gmtime()) + str(int(time.time())) + "- Log.log" log = logging.getLogger() log.setLevel(logging.NOTSET) fh = logging.FileHandler(logFile) logFileLevel = logging.DEBUG fh.setLevel(logFileLevel) format_string = '%(process)d %(thread)d %(asctime)-15s %(levelname)-5s at %(filename)-15s in %(funcName)-10s at line %(lineno)-3d "%(message) s"' fh.setFormatter(logging.Formatter(format_string)) log.addHandler(fh) try: log.info("start") log.info("refer",ret) log.info("end") except TypeError: log.exception("Exception raised") ---------------------------------------------------------------------------------------------------------------------------------------------- OUTPUT message: Traceback (most recent call last): File "C:\Python26\lib\logging\__init__.py", line 768, in emit msg = self.format(record) File "C:\Python26\lib\logging\__init__.py", line 648, in format return fmt.format(record) File "C:\Python26\lib\logging\__init__.py", line 436, in format record.message = record.getMessage() File "C:\Python26\lib\logging\__init__.py", line 306, in getMessage msg = msg % self.args TypeError: not all arguments converted during string formatting -------------- next part -------------- An HTML attachment was scrubbed... URL: From ursusmaximus at gmail.com Mon Jan 25 23:25:47 2010 From: ursusmaximus at gmail.com (Ron) Date: Mon, 25 Jan 2010 20:25:47 -0800 (PST) Subject: Sikuli: the coolest Python project I have yet seen... References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> Message-ID: <5acc067c-f650-4e2f-90f0-ee73ed7382cc@a5g2000yqi.googlegroups.com> OK, here's an idea. I used to do screen scraping scripts and run them as CGI scripts with an HTMl user interface. Why not run Sikuli on Jython on a JVM running on my server, so that I can do my screen scraping with Sikuli? I can take user inputs by using CGI forms from a web client, process the requests using a Sikuli script on the server, and send the results back to the web client. This sounds like fun to me, and easier to highlight and capture the appropriate screen information on targeted web sites using Sikuli than to hand code location information or even using Beautiful Soup. From no.email at nospam.invalid Mon Jan 25 23:31:07 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 25 Jan 2010 20:31:07 -0800 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <4869b7c1-ce91-4405-aa98-9cff6ae3c833@r19g2000yqb.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> <7xpr4ylk9d.fsf@ruckus.brouhaha.com> <7xy6jlnbn1.fsf@ruckus.brouhaha.com> <81319a79-cf25-4444-ab71-bfe982ae7fee@m16g2000yqc.googlegroups.com> Message-ID: <7xeildij1w.fsf@ruckus.brouhaha.com> Steve Howell writes: > I haven't profiled deque vs. list, but I think you are correct about > pop() possibly being a red herring.... > For really large lists, I suppose memmove() would eventually start to > become a bottleneck, but it's brutally fast when it just moves a > couple kilobytes of data around. One way to think of Python is as a scripting wrapper around a bunch of C functions, rather than as a full-fledged programming language. Viewed that way, list operations like pop(0) are essentially constant time unless the list is quite large. By that I mean you can implement classic structures like doubly-linked lists using Python tuples, but even though inserting into the middle of them is theoretically O(1), the memmove's of the native list operations will be much faster in practice. Programs dealing with large lists (more than a few thousand elements) are obviously different and if your program is using such large lists, you have to plan a little differently when writing the code. >> I've often run applications with millions of lists > I bet even in your application, the amount of memory consumed by the > PyListObjects themselves is greatly dwarfed by other objects, notably > the list elements themselves Such lists often would just one element or even be empty. For example, you might have a dictionary mapping names to addresses. Most people have just one address, but some might have no address, and a few might have more than one address, so you would have a list of addresses for each name. Of course the dictionary slots in that example would also use space. > Well, I am not trying to solve problems wastefully here. CPU cycles > are also scarce, so it seems wasteful to do an O(N) memmove that could > be avoided by storing an extra pointer per list. Realistically the CPython interpreter is so slow that the memmove is unnoticable, and Python (at least CPython) just isn't all that conductive to writing fast code. It makes up for this in programmer productivity for the many sorts of problems in which moderate speed is acceptable. > Thanks for your patience in responding to me, despite the needlessly > abrasive tone of my earlier postings. I wondered whether you might have come over from the Lisp newsgroups, which are pretty brutal. We try to be friendlier here (not that we're always successful). Anyway, welcome. > 1) Summarize all this discussion and my lessons learned in some kind > of document. It does not have to be a PEP per se, but I could provide > a useful service to the community by listing pros/cons/etc. I suppose that can't hurt, but there are probably other areas (multicore parallelism is a perennial one) of much higher community interest. http://wiki.python.org/moin/ is probably a good place to put such a document. > 2) I would still advocate for removing the warning against list.pop > (0) from the tutorial. I agree with Steven D'Aprano that docs really > should avoid describing implementation details in many instances On general principles I agree with Alex Stepanov that the running time of a function should be part of its interface (nobody wants to use a stack of popping an element takes quadratic time) and therefore should be stated in the docs. Python just has a weird incongruence between the interpreter layer and the C layer, combined with a library well-evolved for everyday problem sizes, so the traditional asymptotic approach to algorithm selection often doesn't give the best practical choice. I don't feel like looking up what the tutorial says about pop(0), but if it just warns against it without qualification, it should probably be updated. From tekion at gmail.com Mon Jan 25 23:31:38 2010 From: tekion at gmail.com (tekion) Date: Mon, 25 Jan 2010 20:31:38 -0800 (PST) Subject: easy_install error ... References: <5e9c94df-531f-4725-bcd5-87c603d8df1c@o26g2000vbd.googlegroups.com> <2d3c4b85-c0fe-4f65-94ea-d246acdd0345@d30g2000vbl.googlegroups.com> Message-ID: FYI, I figured out what I was doing wrong. After reading the setuptools docs, I noticed took out the quotes around the package name and it works, see details below: python setup.py easy_install -m docutils==0.4 running easy_install Searching for docutils==0.4 Best match: docutils 0.4 Processing docutils-0.4-py2.6.egg Removing docutils 0.4 from easy-install.pth file Installing rst2html.py script to C:\Python26\Scripts Installing rst2latex.py script to C:\Python26\Scripts Installing rst2newlatex.py script to C:\Python26\Scripts Installing rst2pseudoxml.py script to C:\Python26\Scripts Installing rst2s5.py script to C:\Python26\Scripts Installing rst2xml.py script to C:\Python26\Scripts Using c:\python26\lib\site-packages\docutils-0.4-py2.6.egg Because this distribution was installed --multi-version, before you can import modules from this package in an application, you will need to 'import pkg_resources' and then use a 'require()' call similar to one of these examples, in order to select the desired version: pkg_resources.require("docutils") # latest installed version pkg_resources.require("docutils==0.4") # this exact version pkg_resources.require("docutils>=0.4") # this version or higher Processing dependencies for docutils==0.4 Finished processing dependencies for docutils==0.4 I hope this helps other having the similar issue as me. From showell30 at yahoo.com Tue Jan 26 00:00:30 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 21:00:30 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> Message-ID: <0b8ed84f-0a42-42c5-a1ba-e70ed048f39d@k19g2000yqc.googlegroups.com> On Jan 24, 11:28?am, a... at pythoncraft.com (Aahz) wrote: > In article , > Steve Howell ? wrote: > > > > >Even with realloc()'s brokenness, you could improve pop(0) in a way > >that does not impact list access at all, and the patch would not change > >the time complexity of any operation; it would just add negligible > >extract bookkeeping within list_resize() and a few other places. > > Again, your responsibility is to provide a patch and a spectrum of > benchmarking tests to prove it. ?Then you would still have to deal with > the objection that extensions use the list internals -- that might be an > okay sell given the effort otherwise required to port extensions to > Python 3, but that's not the way to bet. > Ok, I just submitted a patch to python-dev that illustrates a 100x speedup on an admittedly artificial program. It still has a long way to go, but it demonstrates proof of concept. I'm done for the day, but tomorrow I will try to polish it up and improve it, even if its doomed for rejection. Apologies to all I have offended in this thread. I frankly found some of the pushback to be a bit hasty and disrespectful, but I certainly overreacted to some of the criticism. And now I'm in the awkward position of asking the people I offended to help me with the patch. If anybody can offer me a hand in understanding some of CPython's internals, particularly with regard to memory management, it would be greatly appreciated. (Sorry I don't have a link to the python-dev posting; it is not showing up in the archives yet for some reason.) From showell30 at yahoo.com Tue Jan 26 00:13:00 2010 From: showell30 at yahoo.com (Steve Howell) Date: Mon, 25 Jan 2010 21:13:00 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <3ac173bd-4124-434d-b726-0b9baaeec752@36g2000yqu.googlegroups.com> <036a4966$0$1357$c3e8da3@news.astraweb.com> <036ad433$0$1357$c3e8da3@news.astraweb.com> <740e2c3a-ab20-49f3-8305-fc4ff498174e@v25g2000yqk.googlegroups.com> <7xaaw3b5cm.fsf@ruckus.brouhaha.com> <393d8bd9-e84a-4b68-afef-41ce9dabd6c2@k19g2000yqc.googlegroups.com> <7xpr4ylk9d.fsf@ruckus.brouhaha.com> <7xy6jlnbn1.fsf@ruckus.brouhaha.com> <81319a79-cf25-4444-ab71-bfe982ae7fee@m16g2000yqc.googlegroups.com> <7xeildij1w.fsf@ruckus.brouhaha.com> Message-ID: On Jan 25, 8:31?pm, Paul Rubin wrote: > Steve Howell writes: > > I haven't profiled deque vs. list, but I think you are correct about > > pop() possibly being a red herring.... > > For really large lists, I suppose memmove() would eventually start to > > become a bottleneck, but it's brutally fast when it just moves a > > couple kilobytes of data around. > > One way to think of Python is as a scripting wrapper around a bunch of C > functions, rather than as a full-fledged programming language. ?Viewed > that way, list operations like pop(0) are essentially constant time > unless the list is quite large. ?By that I mean you can implement > classic structures like doubly-linked lists using Python tuples, but > even though inserting into the middle of them is theoretically O(1), the > memmove's of the native list operations will be much faster in practice. > Programs dealing with large lists (more than a few thousand elements) > are obviously different and if your program is using such large lists, > you have to plan a little differently when writing the code. Thanks. That is a good way of looking at. > > Realistically the CPython interpreter is so slow that the memmove is > unnoticable, and Python (at least CPython) just isn't all that > conductive to writing fast code. ?It makes up for this in programmer > productivity for the many sorts of problems in which moderate speed > is acceptable. > Definitely, and moderate speed is enough in a surprisingly large number of applications. > > Thanks for your patience in responding to me, despite the needlessly > > abrasive tone of my earlier postings. ? > > I wondered whether you might have come over from the Lisp newsgroups, > which are pretty brutal. ?We try to be friendlier here (not that we're > always successful). ?Anyway, welcome. > :) > > ? 1) Summarize all this discussion and my lessons learned in some kind > > of document. ?It does not have to be a PEP per se, but I could provide > > a useful service to the community by listing pros/cons/etc. > > I suppose that can't hurt, but there are probably other areas (multicore > parallelism is a perennial one) of much higher community interest. > > http://wiki.python.org/moin/is probably a good place to put such > a document. > Ok, that's where I'll start. > > ? 2) I would still advocate for removing the warning against list.pop > > (0) from the tutorial. ?I agree with Steven D'Aprano that docs really > > should avoid describing implementation details in many instances > > On general principles I agree with Alex Stepanov that the running time > of a function should be part of its interface (nobody wants to use a > stack of popping an element takes quadratic time) and therefore should > be stated in the docs. ?Python just has a weird incongruence between the > interpreter layer and the C layer, combined with a library well-evolved > for everyday problem sizes, so the traditional asymptotic approach to > algorithm selection often doesn't give the best practical choice. > > I don't feel like looking up what the tutorial says about pop(0), but if > it just warns against it without qualification, it should probably > be updated. Here it is: http://docs.python.org/tutorial/datastructures.html#using-lists-as-queues My opinion is that the warning should be either removed or qualified, but it is probably fine as written. ''' It is also possible to use a list as a queue, where the first element added is the first element retrieved (?first-in, first-out?); however, lists are not efficient for this purpose. While appends and pops from the end of list are fast, doing inserts or pops from the beginning of a list is slow (because all of the other elements have to be shifted by one). ''' The qualifications would be that deque lacks some features that list has, and that the shift-by-one operation is actually a call to memmove () and may not apply to all implementations. From millerdev at gmail.com Tue Jan 26 00:26:49 2010 From: millerdev at gmail.com (Daniel) Date: Mon, 25 Jan 2010 21:26:49 -0800 (PST) Subject: Authenticated encryption with PyCrypto Message-ID: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> Just got done reading this thread: http://groups.google.com/group/comp.lang.python/browse_thread/thread/b31a5b5f58084f12/0e09f5f5542812c3 and I'd appreciate feedback on this recipe: http://code.activestate.com/recipes/576980/ Of course, it does not meet all of the requirements set forth by the OP in the referenced thread (the pycrypto dependency is a problem), but it is an attempt to provide a simple interface for performing strong, password-based encryption. Are there already modules out there that provide such a simple interface? If there are, they seem to be hiding somewhere out of Google's view. I looked at ezPyCrypto, but it seemed to require public and private keys, which was not convenient in my situation... maybe password-based encryption is trivial to do with ezPyCrypto as well? In addition to ezPyCrypto, I looked at Google's keyczar, but despite the claims of the documentation, the API seemed overly complicated. Is it possible to have a simple API for an industry-strength encryption module? If not, is it possible to document that complicated API such that a non- cryptographer could use it and feel confident that he hadn't made a critical mistake? Also, slightly related, is there an easy way to get the sha/md5 deprecation warnings emitted by PyCrypto in Python 2.6 to go away? ~ Daniel From kirby.urner at gmail.com Tue Jan 26 01:10:32 2010 From: kirby.urner at gmail.com (kirby urner) Date: Mon, 25 Jan 2010 22:10:32 -0800 Subject: [Edu-sig] some turtle questions In-Reply-To: <8F7BCECA-CD55-425D-A173-94D7F30DFB53@bryant.edu> References: <8F7BCECA-CD55-425D-A173-94D7F30DFB53@bryant.edu> Message-ID: Hi Brian -- If you wanna go to a lot of work, but not a huge amount, write wrapper class for the Standard Library turtle that intercepts its commands and updates an on-board data structure, representing pixels x pixels, specifying self position, keep color info stashed per each one. That's a lot of data, depending on screen resolution. Consider a thick line option if you have one, make your turtle "wide body". Or stay with thin. So then if you go turtle.forward(10) you will send it to your self-made forward method. Stop and smell the pixels, see what color was stashed there, either by another turtle (! -- shared data structure) or by this turtle, or maybe it's still the default untrammeled color. You can add new methods, like "glide" or "explode" that translate to the underlying turtle somehow -- use your imagination. Kirby On Sun, Jan 24, 2010 at 7:29 AM, Brian Blais wrote: > Hello, > > I am trying to think of things to do with the turtle module with my > students, and I have some ideas where I am not sure whether the turtle > module can do it. > > 1) is there a way to determine the current screen pixel color? I am > thinking about having the turtle go forward until it reaches an object, say > a red circle. I can probably do this by making circle objects (drawn with > turtles themselves) which know their own position, and check against this > info. But I thought it might be useful also for the turtle to know. > > 2) is there a way to put a limit on the extent the turtle can travel? it > seems I can keep moving off of the screen. Is there a way to make it so > that a forward(50) command, at the edge, either raises an exception (at the > wall) or simply doesn't move the turtle because of the limit? > > > thanks! > > > bb > > -- > Brian Blais > bblais at bryant.edu > http://web.bryant.edu/~bblais > http://bblais.blogspot.com/ > > > > > _______________________________________________ > Edu-sig mailing list > Edu-sig at python.org > http://mail.python.org/mailman/listinfo/edu-sig > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gabriel.rossetti at arimaz.com Tue Jan 26 03:14:48 2010 From: gabriel.rossetti at arimaz.com (Gabriel Rossetti) Date: Tue, 26 Jan 2010 09:14:48 +0100 Subject: Any Swisses here? In-Reply-To: <145e6acd-4260-495b-b562-da634fccfa1a@j19g2000yqk.googlegroups.com> References: <145e6acd-4260-495b-b562-da634fccfa1a@j19g2000yqk.googlegroups.com> Message-ID: <4B5EA478.607@arimaz.com> n00m wrote: > Congrats! > Your choice -- to ban building of muslim mosques -- is the only choice > to save our civililazation. > This yellow plague (incl. Chinese etc) must be eliminated from our > Planet, > They are very cunning critters, they can play on strings of > compassion, > but its riffraffs. What do you know about Russia? Who are yuo against > Russia? > Nothing. Dust. We are simply tired people. > Actually, I am Swiss and I have to say that the vote does not represent Switzerland. In fact, a lot of people didn't vote because they were sure it would never pass. They just came out with an official analysis/statistics of the vote results and it shows that people with higher education voted against the ban, that Christians voted for the ban and atheists against it. Also, if you look at the outcome for each Canton (State), the rural ones voted for the ban and the metropolitan/cosmopolitan ones against it. What happened is basically what happened in Nazi Germany, a strong campaign for the masses and sadly it worked. I have several Russian friends and I have no idea what their view is on the subject. Russia has several Muslim ethnic groups and with most there seems to be no problem (I'm excluding Chechnya, even thought the conflict is more about power than religion). Gabriel Disclaimer: the views expressed in this email are solely mine and not in any way representative of my company's views or ideologies. From showell30 at yahoo.com Tue Jan 26 04:27:54 2010 From: showell30 at yahoo.com (Steve Howell) Date: Tue, 26 Jan 2010 01:27:54 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <0b8ed84f-0a42-42c5-a1ba-e70ed048f39d@k19g2000yqc.googlegroups.com> Message-ID: <2ba54ccf-b0b5-4a30-a31a-fa82571af20e@a6g2000yqm.googlegroups.com> On Jan 25, 9:00?pm, Steve Howell wrote: > On Jan 24, 11:28?am, a... at pythoncraft.com (Aahz) wrote: > > > > > In article , > > Steve Howell ? wrote: > > > >Even with realloc()'s brokenness, you could improve pop(0) in a way > > >that does not impact list access at all, and the patch would not change > > >the time complexity of any operation; it would just add negligible > > >extract bookkeeping within list_resize() and a few other places. > > > Again, your responsibility is to provide a patch and a spectrum of > > benchmarking tests to prove it. ?Then you would still have to deal with > > the objection that extensions use the list internals -- that might be an > > okay sell given the effort otherwise required to port extensions to > > Python 3, but that's not the way to bet. > > Ok, I just submitted a patch to python-dev that illustrates a 100x > speedup on an admittedly artificial program. ?It still has a long way > to go, but it demonstrates proof of concept. ?I'm done for the day, > but tomorrow I will try to polish it up and improve it, even if its > doomed for rejection. ?Apologies to all I have offended in this > thread. ?I frankly found some of the pushback to be a bit hasty and > disrespectful, but I certainly overreacted to some of the criticism. > And now I'm in the awkward position of asking the people I offended to > help me with the patch. ?If anybody can offer me a hand in > understanding some of CPython's internals, particularly with regard to > memory management, it would be greatly appreciated. > > (Sorry I don't have a link to the python-dev posting; it is not > showing up in the archives yet for some reason.) Here is the latest version of the patch, which passes all the tests on my debug build. Not exactly trivial, but not super complicated either. Index: Include/listobject.h =================================================================== --- Include/listobject.h (revision 77751) +++ Include/listobject.h (working copy) @@ -36,6 +36,7 @@ * the list is not yet visible outside the function that builds it. */ Py_ssize_t allocated; + Py_ssize_t orphans; } PyListObject; PyAPI_DATA(PyTypeObject) PyList_Type; Index: Objects/listobject.c =================================================================== --- Objects/listobject.c (revision 77751) +++ Objects/listobject.c (working copy) @@ -27,13 +27,25 @@ PyObject **items; size_t new_allocated; Py_ssize_t allocated = self->allocated; + Py_ssize_t needed; + if (self->orphans >= newsize) { + items = self->ob_item - self->orphans; + memmove(items, &items[self->orphans], + (newsize)*sizeof(PyObject *)); + self->ob_item = items; + self->orphans = 0; + } + + needed = newsize + self->orphans; + items = self->ob_item - self->orphans; + /* Bypass realloc() when a previous overallocation is large enough to accommodate the newsize. If the newsize falls lower than half the allocated size, then proceed with the realloc() to shrink the list. */ - if (allocated >= newsize && newsize >= (allocated >> 1)) { - assert(self->ob_item != NULL || newsize == 0); + if (allocated >= needed && needed >= (allocated >> 1)) { + assert(items != NULL || newsize == 0); Py_SIZE(self) = newsize; return 0; } @@ -45,28 +57,32 @@ * system realloc(). * The growth pattern is: 0, 4, 8, 16, 25, 35, 46, 58, 72, 88, ... */ - new_allocated = (newsize >> 3) + (newsize < 9 ? 3 : 6); + new_allocated = (needed >> 3) + (needed < 9 ? 3 : 6); /* check for integer overflow */ - if (new_allocated > PY_SIZE_MAX - newsize) { + if (new_allocated > PY_SIZE_MAX - needed) { PyErr_NoMemory(); return -1; } else { - new_allocated += newsize; + new_allocated += needed; } - if (newsize == 0) + if (needed == 0) new_allocated = 0; - items = self->ob_item; - if (new_allocated <= ((~(size_t)0) / sizeof(PyObject *))) + /* + fprintf(stderr, "ob_item: %p", self->ob_item); + fprintf(stderr, "items: %p", items); + */ + if (new_allocated <= ((~(size_t)0) / sizeof(PyObject *))) { PyMem_RESIZE(items, PyObject *, new_allocated); + } else items = NULL; if (items == NULL) { PyErr_NoMemory(); return -1; } - self->ob_item = items; + self->ob_item = items + self->orphans; Py_SIZE(self) = newsize; self->allocated = new_allocated; return 0; @@ -158,6 +174,7 @@ } Py_SIZE(op) = size; op->allocated = size; + op->orphans = 0; _PyObject_GC_TRACK(op); return (PyObject *) op; } @@ -304,9 +321,10 @@ There's a simple test case where somehow this reduces thrashing when a *very* large list is created and immediately deleted. */ + op->ob_item -= op->orphans; i = Py_SIZE(op); while (--i >= 0) { - Py_XDECREF(op->ob_item[i]); + Py_XDECREF(op->ob_item[i+op->orphans]); } PyMem_FREE(op->ob_item); } @@ -548,12 +566,13 @@ if (item != NULL) { /* Because XDECREF can recursively invoke operations on this list, we make it empty first. */ + item -= a->orphans; i = Py_SIZE(a); Py_SIZE(a) = 0; a->ob_item = NULL; a->allocated = 0; while (--i >= 0) { - Py_XDECREF(item[i]); + Py_XDECREF(item[i + a->orphans]); } PyMem_FREE(item); } @@ -638,18 +657,32 @@ memcpy(recycle, &item[ilow], s); if (d < 0) { /* Delete -d items */ - memmove(&item[ihigh+d], &item[ihigh], - (Py_SIZE(a) - ihigh)*sizeof(PyObject *)); + if (ilow == 0) { + a->orphans += (-1 * d); + a->ob_item += (-1 * d); + } + else { + memmove(&item[ihigh+d], &item[ihigh], + (Py_SIZE(a) - ihigh)*sizeof(PyObject *)); + } list_resize(a, Py_SIZE(a) + d); item = a->ob_item; } else if (d > 0) { /* Insert d items */ - k = Py_SIZE(a); - if (list_resize(a, k+d) < 0) - goto Error; - item = a->ob_item; - memmove(&item[ihigh+d], &item[ihigh], - (k - ihigh)*sizeof(PyObject *)); + if (ilow == 0 && d <= a->orphans) { + a->orphans += (-1 * d); + a->ob_item += (-1 * d); + item = a->ob_item; + list_resize(a, Py_SIZE(a) + d); + } + else { + k = Py_SIZE(a); + if (list_resize(a, k+d) < 0) + goto Error; + item = a->ob_item; + memmove(&item[ihigh+d], &item[ihigh], + (k - ihigh)*sizeof(PyObject *)); + } } for (k = 0; k < n; k++, ilow++) { PyObject *w = vitem[k]; @@ -838,7 +871,7 @@ } break; } - if (Py_SIZE(self) < self->allocated) { + if (Py_SIZE(self) + self->orphans < self->allocated) { /* steals ref */ PyList_SET_ITEM(self, Py_SIZE(self), item); ++Py_SIZE(self); @@ -852,7 +885,7 @@ } /* Cut back result list if initial guess was too large. */ - if (Py_SIZE(self) < self->allocated) + if (Py_SIZE(self) + self->orphans < self->allocated) list_resize(self, Py_SIZE(self)); /* shrinking can't fail */ Py_DECREF(it); @@ -2290,7 +2323,7 @@ { Py_ssize_t res; - res = sizeof(PyListObject) + self->allocated * sizeof(void*); + res = sizeof(PyListObject) + (self->allocated + self->orphans) * sizeof(void*); return PyLong_FromSsize_t(res); } Index: Lib/test/test_list.py =================================================================== --- Lib/test/test_list.py (revision 77751) +++ Lib/test/test_list.py (working copy) @@ -51,6 +51,15 @@ self.assertEqual(len([0]), 1) self.assertEqual(len([0, 1, 2]), 3) + def test_pop_and_prepend(self): + # This guards against faulty optimizations on list that + # attempt to makes pops and prepends at the beginning of the + # list work faster. + lst = [5] * 100 + del lst[0] + lst.insert(0, 4) + self.assertEqual(lst, [4] + [5] * 99) + def test_overflow(self): lst = [4, 5, 6, 7] n = int((sys.maxsize*2+2) // len(lst)) Index: Lib/test/test_iter.py =================================================================== --- Lib/test/test_iter.py (revision 77751) +++ Lib/test/test_iter.py (working copy) @@ -875,7 +875,18 @@ except TypeError: pass + def test_extends(self): + # This test would break on an incomplete patch to listobject.c + def gen(): + for i in range(500): + yield i + lst = [0] * 500 + for i in range(240): + lst.pop(0) + lst.extend(gen()) + + def test_main(): run_unittest(TestCase) Index: Lib/test/test_sys.py =================================================================== --- Lib/test/test_sys.py (revision 77751) +++ Lib/test/test_sys.py (working copy) @@ -515,7 +515,7 @@ # bool objects are not gc tracked self.assertEqual(sys.getsizeof(True), size(vh) + self.longdigit) # but lists are - self.assertEqual(sys.getsizeof([]), size(vh + 'PP') + gc_header_size) + self.assertEqual(sys.getsizeof([]), size(vh + 'PPi') + gc_header_size) def test_default(self): h = self.header @@ -638,7 +638,7 @@ # list samples = [[], [1,2,3], ['1', '2', '3']] for sample in samples: - check(sample, size(vh + 'PP') + len(sample)*self.P) + check(sample, size(vh + 'PPi') + len(sample)*self.P) # sortwrapper (list) # XXX # cmpwrapper (list) From jeanmichel at sequans.com Tue Jan 26 04:49:54 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Tue, 26 Jan 2010 10:49:54 +0100 Subject: Sikuli: the coolest Python project I have yet seen... In-Reply-To: References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> <4B5D9F09.7050607@it.uu.se> Message-ID: <4B5EBAC2.4080209@sequans.com> Terry Reedy wrote: > On 1/25/2010 9:14 AM, Javier Collado wrote: > >> I think the site is under maintenance. I tried a couple of hours ago >> and it worked fine. >> >> As an alternative, I found that this link also worked: >> http://www.sikuli.org/ > > This just redirects to the link below > >>>> http://sikuli.csail.mit.edu/ I also did this > >>> This link is broken! > > Worked for me both yesterday and now. > I confirmed it now works. The demo is pretty impressive. However one should not be the kind of guy changing its desktop theme on a regular basis. If I got it well, all is based on bitmap recognition (with some tolerance though). I'll still give it a try. JM From lanyjie at yahoo.com Tue Jan 26 06:17:23 2010 From: lanyjie at yahoo.com (Yingjie Lan) Date: Tue, 26 Jan 2010 03:17:23 -0800 (PST) Subject: announcement: expy release 0.5 Message-ID: <290176.43047.qm@web54204.mail.re2.yahoo.com> Hi all, This is to announce expy release 0.5. expy is an express way to extend Python. For more information and tutorials on expy, see: http://expy.sf.net/ Cheers, Yingjie From mail at hellmutweber.de Tue Jan 26 06:48:32 2010 From: mail at hellmutweber.de (Hellmut Weber) Date: Tue, 26 Jan 2010 12:48:32 +0100 Subject: Accessing the name of an actual parameter Message-ID: <4B5ED690.1020706@hellmutweber.de> Hi, consider the following piece of code, please ----- ----- def f(param): nameOfParam = ??? # here I want to access the name of the variable # which was given as parameter to the function print nameOfParam, param return if __name__ == __main__: a = 1 f(a) b = 'abcd' f(a) ----- ----- The output should be: 'a' 1 'b' 'abcd' ----- ----- I tried to look at globals() and locals(), gave a look to the frames (sys._getframe(0) and sys._getframe(1), but did not see a possibility to access the information a want How can this be done? TIA Hellmut -- Dr. Hellmut Weber mail at hellmutweber.de Degenfeldstra?e 2 tel +49-89-3081172 D-80803 M?nchen-Schwabing mobil +49-172-8450321 please: No DOCs, no PPTs. why: tinyurl.com/cbgq From clp2 at rebertia.com Tue Jan 26 06:58:42 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Tue, 26 Jan 2010 03:58:42 -0800 Subject: Accessing the name of an actual parameter In-Reply-To: <4B5ED690.1020706@hellmutweber.de> References: <4B5ED690.1020706@hellmutweber.de> Message-ID: <50697b2c1001260358h4cc93b8cva11b9e524ed77eae@mail.gmail.com> On Tue, Jan 26, 2010 at 3:48 AM, Hellmut Weber wrote: > Hi, > > consider the following piece of code, please > > ----- ----- > > def f(param): > ?nameOfParam = ??? > ?# here I want to access the name of the variable > ?# which was given as parameter to the function > ?print nameOfParam, param > ?return > > if __name__ == __main__: > > ?a = 1 > ?f(a) > > ?b = 'abcd' > ?f(a) > > ----- ----- > > The output should be: > > 'a' 1 > 'b' 'abcd' > > ----- ----- > > I tried to look at globals() and locals(), gave a look to the frames > (sys._getframe(0) and sys._getframe(1), > but did not see a possibility to access the information a want > > How can this be done? It's an FAQ that's not in the FAQ. The popular answer is "Don't try to do that! There's usually a less magic way to accomplish your goal.". Alternatives can usually be proffered if the exact use case is explained specifically. Cheers, Chris -- http://blog.rebertia.com From alfps at start.no Tue Jan 26 07:01:13 2010 From: alfps at start.no (Alf P. Steinbach) Date: Tue, 26 Jan 2010 13:01:13 +0100 Subject: Accessing the name of an actual parameter In-Reply-To: References: Message-ID: * Hellmut Weber: > > consider the following piece of code, please > > ----- ----- > > def f(param): > nameOfParam = ??? > # here I want to access the name of the variable > # which was given as parameter to the function > print nameOfParam, param > return > > if __name__ == __main__: > > a = 1 > f(a) > > b = 'abcd' > f(a) > > ----- ----- > > The output should be: > > 'a' 1 > 'b' 'abcd' > > ----- ----- > > I tried to look at globals() and locals(), gave a look to the frames > (sys._getframe(0) and sys._getframe(1), > but did not see a possibility to access the information a want > > How can this be done? Not in any efficient way. A debugger can do it, and you can do it in the same way as a debugger, checking stack frames and the source code. But if it's debugging that you're after then use a debugger -- that's what they're for. Otherwise, just change the way that you invoke the routine. For example, >>> def f( **kwa ): ... print( kwa ) ... >>> a = 1234 >>> b = "huh" >>> f( a = a, b = b ) {'a': 1234, 'b': 'huh'} >>> _ Cheers & hth., - Alf From iurisilvio at gmail.com Tue Jan 26 07:06:18 2010 From: iurisilvio at gmail.com (Iuri) Date: Tue, 26 Jan 2010 10:06:18 -0200 Subject: Sikuli: the coolest Python project I have yet seen... In-Reply-To: <4B5EBAC2.4080209@sequans.com> References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> <4B5D9F09.7050607@it.uu.se> <4B5EBAC2.4080209@sequans.com> Message-ID: <789aac5a1001260406s12005ffcye8be6f2bfd2d59ed@mail.gmail.com> I liked this tool! I'm thinking about how I can write acceptance tests with Sikuli. []s iurisilvio On Tue, Jan 26, 2010 at 7:49 AM, Jean-Michel Pichavant < jeanmichel at sequans.com> wrote: > Terry Reedy wrote: > >> On 1/25/2010 9:14 AM, Javier Collado wrote: >> >> I think the site is under maintenance. I tried a couple of hours ago >>> and it worked fine. >>> >>> As an alternative, I found that this link also worked: >>> http://www.sikuli.org/ >>> >> >> This just redirects to the link below >> >> http://sikuli.csail.mit.edu/ I also did this >>>>> >>>> >> This link is broken! >>>> >>> >> Worked for me both yesterday and now. >> >> I confirmed it now works. > The demo is pretty impressive. > > However one should not be the kind of guy changing its desktop theme on a > regular basis. If I got it well, all is based on bitmap recognition (with > some tolerance though). I'll still give it a try. > > JM > > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeanmichel at sequans.com Tue Jan 26 07:06:44 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Tue, 26 Jan 2010 13:06:44 +0100 Subject: Accessing the name of an actual parameter In-Reply-To: <4B5ED690.1020706@hellmutweber.de> References: <4B5ED690.1020706@hellmutweber.de> Message-ID: <4B5EDAD4.4030902@sequans.com> Hellmut Weber wrote: > Hi, > > consider the following piece of code, please > > ----- ----- > > def f(param): > nameOfParam = ??? > # here I want to access the name of the variable > # which was given as parameter to the function > print nameOfParam, param > return > > if __name__ == __main__: > > a = 1 > f(a) > > b = 'abcd' > f(a) > > ----- ----- > > The output should be: > > 'a' 1 > 'b' 'abcd' > > ----- ----- > > I tried to look at globals() and locals(), gave a look to the frames > (sys._getframe(0) and sys._getframe(1), > but did not see a possibility to access the information a want > > How can this be done? > > TIA > > Hellmut > > I don't know if it's possible (maybe using smelling hackish code). What I know for sure, is that your function should ask the required informations as parameters. That what parameters are for. Tell us whant you want to achieve precisely with the parameter names, maybe we'll find something suitable. JM From roy at panix.com Tue Jan 26 07:40:15 2010 From: roy at panix.com (Roy Smith) Date: Tue, 26 Jan 2010 07:40:15 -0500 Subject: Splitting text at whitespace but keeping the whitespace in thereturned list References: <1264352736.30280.1356298585@webmail.messagingengine.com> Message-ID: In article , "Tim Arnold" wrote: > also, partition works though it returns a tuple instead of a list. > >>> s = 'hello world' > >>> s.partition(' ') > ('hello', ' ', 'world') I've never used partition() before; my first thought on reading the above was, "That's weird, it should be returning a list". Then I went and looked at the docs. Given the description (returns specifically a 3-tuple), I guess a tuple makes sense, but now I'm wondering what the use case was for this method when it was invented? Having a variant of split() which either leaves the delimiter on the end of each word, or returns a list of alternating [word, delimiter, word, delimiter, word] seems logical and orthogonal. In fact, partition() is really just the hypothetical whitespace-preserving variant of split(), with maxsplit=1, except that it returns a tuple instead of a list. So, what was the original problem partition() was trying to solve? From jeanmichel at sequans.com Tue Jan 26 08:43:07 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Tue, 26 Jan 2010 14:43:07 +0100 Subject: pylint and the 'missing docstring' warning Message-ID: <4B5EF16B.6080505@sequans.com> Hello, Does anyone using pylint knows a way to make pylint ignore these 'missing docstring' warnings when the base class version of the method actually defines the docstring ? 'Cause my doc builder (epydoc) handle it properly and propagate docstrings if asked to. Too bad pylint is complaining about it. I don't want to deactivate this warning. Any hint would be apreciated JM From mlee3891 at gmail.com Tue Jan 26 08:51:12 2010 From: mlee3891 at gmail.com (Cascade3891) Date: Tue, 26 Jan 2010 05:51:12 -0800 (PST) Subject: My experiences building a small app on Python Message-ID: <62c367db-49e3-485b-80d6-ed2f99065f48@22g2000yqr.googlegroups.com> Hello Python community, I built a small app with Python for the first time. It has served a very useful and productive purpose for me and my colleagues at work. I have shared my experiences on the following link due to the better web presentation tools and coding presentation tools: http://www.daniweb.com/forums/thread255458.html It's a bit of a read. But insightful. If you have any feedback or tips and advice, please let me know. I've fallen in love with programming because of Python. =) Thank you in advance. Cheers. From frankborell at gmail.com Tue Jan 26 08:52:56 2010 From: frankborell at gmail.com (frankborell at gmail.com) Date: Tue, 26 Jan 2010 05:52:56 -0800 (PST) Subject: PIL: Wiping out metadata on save only on progressive JPG's Message-ID: Hello, I've tested on PIL 1.16 and 1.17. Metadata(IPTC and EXIF) are saved fine when the jpg image is NOT progressive. But when the jpg is progressive, the metadata gets wiped out. Does anyone have any ideas on how to preserve? im = Image.open('progressive jpeg path') im.save('jpeg path') or im.save('jpeg path',option={'progression':False,'quality': 60,'optimize':True}) Thanks, Frank From jjposner at optimum.net Tue Jan 26 08:56:50 2010 From: jjposner at optimum.net (John Posner) Date: Tue, 26 Jan 2010 08:56:50 -0500 Subject: pylint and the 'missing docstring' warning In-Reply-To: References: Message-ID: <4b5ef47f$0$31262$607ed4bc@cv.net> On 1/26/2010 8:43 AM, Jean-Michel Pichavant wrote: > Hello, > > Does anyone using pylint knows a way to make pylint ignore these > 'missing docstring' warnings when the base class version of the method > actually defines the docstring ? > 'Cause my doc builder (epydoc) handle it properly and propagate > docstrings if asked to. Too bad pylint is complaining about it. > I don't want to deactivate this warning. > Look for this setting in the pylint configuration file: # not require a docstring no-docstring-rgx=__.*__ -John From jeanmichel at sequans.com Tue Jan 26 09:22:33 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Tue, 26 Jan 2010 15:22:33 +0100 Subject: pylint and the 'missing docstring' warning In-Reply-To: <4b5ef47f$0$31262$607ed4bc@cv.net> References: <4b5ef47f$0$31262$607ed4bc@cv.net> Message-ID: <4B5EFAA9.8090608@sequans.com> John Posner wrote: > On 1/26/2010 8:43 AM, Jean-Michel Pichavant wrote: >> Hello, >> >> Does anyone using pylint knows a way to make pylint ignore these >> 'missing docstring' warnings when the base class version of the method >> actually defines the docstring ? >> 'Cause my doc builder (epydoc) handle it properly and propagate >> docstrings if asked to. Too bad pylint is complaining about it. >> I don't want to deactivate this warning. >> > > Look for this setting in the pylint configuration file: > > # not require a docstring > no-docstring-rgx=__.*__ > > -John > > If I'm no wrong this just disables docstring checks for methods matching __.*__ I'd like to disable it for any method which base class version already defines the docstring. JM From simon at brunningonline.net Tue Jan 26 10:02:55 2010 From: simon at brunningonline.net (Simon Brunning) Date: Tue, 26 Jan 2010 15:02:55 +0000 Subject: My experiences building a small app on Python In-Reply-To: <62c367db-49e3-485b-80d6-ed2f99065f48@22g2000yqr.googlegroups.com> References: <62c367db-49e3-485b-80d6-ed2f99065f48@22g2000yqr.googlegroups.com> Message-ID: <8c7f10c61001260702xf4f71c2t538661f73f66399f@mail.gmail.com> 2010/1/26 Cascade3891 : > It's a bit of a read. But insightful. We'll be the judge of that, surely? ;-) -- Cheers, Simon B. From gherron at islandtraining.com Tue Jan 26 10:50:31 2010 From: gherron at islandtraining.com (Gary Herron) Date: Tue, 26 Jan 2010 07:50:31 -0800 Subject: Accessing the name of an actual parameter In-Reply-To: <4B5ED690.1020706@hellmutweber.de> References: <4B5ED690.1020706@hellmutweber.de> Message-ID: <4B5F0F47.4080007@islandtraining.com> Hellmut Weber wrote: > Hi, > > consider the following piece of code, please > > ----- ----- > > def f(param): > nameOfParam = ??? > # here I want to access the name of the variable > # which was given as parameter to the function > print nameOfParam, param > return > > if __name__ == __main__: > > a = 1 > f(a) > > b = 'abcd' > f(a) > > ----- ----- > > The output should be: > > 'a' 1 > 'b' 'abcd' > > ----- ----- > > I tried to look at globals() and locals(), gave a look to the frames > (sys._getframe(0) and sys._getframe(1), > but did not see a possibility to access the information a want > > How can this be done? > > TIA > > Hellmut > > It's naive to think this question even makes sense. There are many ways f can be called which don't involve a parameter: f(42) f(time()) f(a+123) f(sin(a)) f(f(1)) and so on. Cheers, Gary Herron From gherron at islandtraining.com Tue Jan 26 10:54:06 2010 From: gherron at islandtraining.com (Gary Herron) Date: Tue, 26 Jan 2010 07:54:06 -0800 Subject: Accessing the name of an actual parameter In-Reply-To: <4B5F0F47.4080007@islandtraining.com> References: <4B5ED690.1020706@hellmutweber.de> <4B5F0F47.4080007@islandtraining.com> Message-ID: <4B5F101E.2080501@islandtraining.com> Gary Herron wrote: > Hellmut Weber wrote: >> Hi, >> >> consider the following piece of code, please >> >> ----- ----- >> >> def f(param): >> nameOfParam = ??? >> # here I want to access the name of the variable >> # which was given as parameter to the function >> print nameOfParam, param >> return >> >> if __name__ == __main__: >> >> a = 1 >> f(a) >> >> b = 'abcd' >> f(a) >> >> ----- ----- >> >> The output should be: >> >> 'a' 1 >> 'b' 'abcd' >> >> ----- ----- >> >> I tried to look at globals() and locals(), gave a look to the frames >> (sys._getframe(0) and sys._getframe(1), >> but did not see a possibility to access the information a want >> >> How can this be done? >> >> TIA >> >> Hellmut >> >> > > It's naive to think this question even makes sense. There are many > ways f can be called which don't involve a parameter: That's worded poorly, Rather I should have said: "don't involve a parameter that comes from a single variable." > > f(42) > f(time()) > f(a+123) > f(sin(a)) > f(f(1)) > > and so on. > Cheers, > Gary Herron > From pruebauno at latinmail.com Tue Jan 26 11:07:51 2010 From: pruebauno at latinmail.com (nn) Date: Tue, 26 Jan 2010 08:07:51 -0800 (PST) Subject: ctypes for AIX References: <9C240725D2ABD949A80B46DD93AB107042E8D44101@XCH-NW-08V.nw.nos.boeing.com> <50697b2c1001240630r33845a1bl7f94b40a4f0c4436@mail.gmail.com> Message-ID: On Jan 25, 6:36?pm, "Waddle, Jim" wrote: > Chris, > Thanks for responding to my email. > I apologize for the remark about python only being developed for windows. I got the impression when I was looking at the ActivePython web site and saw that the version of python that they had available was not supported on very many unix systems. I should not make general statement based on only one web site. After reading your email I decided to see for myself what the issue was about compiling python on AIX 5.3. > > This is the error I saw the first time I tried to use ctypes. > > Python 2.4.3 (#1, Jul 17 2006, 20:00:23) [C] on aix5 > Type "help", "copyright", "credits" or "license" for more information.>>> import ctypes > > Traceback (most recent call last): > ? File "", line 1, in ? > ImportError: No module named ctypes > > This version of python was downloaded and installed from ActivePython and when I checked their webpage it states that ctypes is not available on AIX. > I then figured I would get a new copy of python and install it on AIX. I downloaded python.2.5.5c2 fromhttp://www.python.org. I did the configure and make which posted many errors in the ctypes function which I guess is the reason that is does not get include in the final make. > > an example of the build error I get when doing the make is: > xlc_r -q64 -DNDEBUG -O -I. -I/s/users/cz030a/xferjunk/python/Python-2.5.5c2/./Include -Ibuild/temp.aix-5.3-2.5/libffi/inclu > de -Ibuild/temp.aix-5.3-2.5/libffi -I/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/libffi/src -I/s/users/c > z030a/xferjunk/ots/python2.5/include -I. -IInclude -I./Include -I/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Include -I/ > s/users/cz030a/xferjunk/python/Python-2.5.5c2 -c /s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.c - > o build/temp.aix-5.3-2.5/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.o > "/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.c", line 2820.31: 1506-068 (W) Operation between ty > pes "void*" and "int(*)(void)" is not allowed. > "/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.c", line 3363.28: 1506-280 (W) Function argument as > signment between types "int(*)(void)" and "void*" is not allowed. > "/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.c", line 4768.67: 1506-280 (W) Function argument as > signment between types "void*" and "void*(*)(void*,const void*,unsigned long)" is not allowed. > "/s/users/cz030a/xferjunk/python/Python-2.5.5c2/Modules/_ctypes/_ctypes.c", line 4769.66: 1506-280 (W) Function argument as > signment between types "void*" and "void*(*)(void*,int,unsigned long)" is not allowed. > > I do not have sufficient knowledge to know how to fix this. I would think that this error somehow is related to compiling on aix. If you have any suggestions on how to correct this problem , I would appreciate it > > Jim Waddle > KIT-D > 425-785-5194 > > -----Original Message----- > From: ch... at rebertia.com [mailto:ch... at rebertia.com] On Behalf Of Chris Rebert > Sent: Sunday, January 24, 2010 7:31 AM > To: Waddle, Jim > > Cc: python-l... at python.org > Subject: Re: ctypes for AIX > > On Sun, Jan 24, 2010 at 5:54 AM, Waddle, Jim wrote: > > I need to use ctypes with python running on AIX. > > According to the ctypes readme, ctypes is based on libffi, which > according to its website, supports AIX for PowerPC64. > So, perhaps you could state what the actual error or problem you're > encountering is? > It is theoretically possible the ctypes-bundled libffi is either > outdated or had the AIX-specific bits removed; I don't know, I'm not a > CPython dev. > > > It appears that python is being developed mostly for windows. > > No, not really; your statement is especially ironic considering one of > Python's primary areas of use is for web applications as part of a > LAMP stack. > > > Is there a policy concerning getting functions like ctypes working on AIX. > > No idea. Someone will probably chime in though. > > Cheers, > Chris > --http://blog.rebertia.com > > Look at this website: http://pware.hvcc.edu/news.html "8/1/2009 Python with ctypes!" Good luck. From duncan.booth at invalid.invalid Tue Jan 26 11:13:16 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 26 Jan 2010 16:13:16 GMT Subject: Accessing the name of an actual parameter References: <4B5ED690.1020706@hellmutweber.de> Message-ID: Gary Herron wrote: > It's naive to think this question even makes sense. There are many ways > f can be called which don't involve a parameter: > > f(42) > f(time()) > f(a+123) > f(sin(a)) > f(f(1)) > > and so on. So long as the OP doesn't care if they get no names or several name that might not matter. However, explicitly passing in the name you want to use is likely to be more useful. >>> def getcallersnames(obj): f = inspect.currentframe().f_back.f_back return [name for name in f.f_locals if f.f_locals[name] is obj] >>> def bip(x): print getcallersnames(x) >>> def foo(bar): baz = bar bip(baz) bip(baz+1) >>> foo(3) ['bar', 'baz'] [] >>> bip(bip) ['bip'] -- Duncan Booth http://kupuguy.blogspot.com From jjposner at optimum.net Tue Jan 26 11:19:04 2010 From: jjposner at optimum.net (John Posner) Date: Tue, 26 Jan 2010 11:19:04 -0500 Subject: pylint and the 'missing docstring' warning In-Reply-To: References: <4b5ef47f$0$31262$607ed4bc@cv.net> Message-ID: <4B5F15F8.3010004@optimum.net> On 1/26/2010 9:22 AM, Jean-Michel Pichavant wrote: > John Posner wrote: >> On 1/26/2010 8:43 AM, Jean-Michel Pichavant wrote: >>> Hello, >>> >>> Does anyone using pylint knows a way to make pylint ignore these >>> 'missing docstring' warnings when the base class version of the method >>> actually defines the docstring ? >>> 'Cause my doc builder (epydoc) handle it properly and propagate >>> docstrings if asked to. Too bad pylint is complaining about it. >>> I don't want to deactivate this warning. >>> >> >> Look for this setting in the pylint configuration file: >> >> # not require a docstring >> no-docstring-rgx=__.*__ >> >> -John >> >> > If I'm no wrong this just disables docstring checks for methods matching > __.*__ > I'd like to disable it for any method which base class version already > defines the docstring. > I'm a Pylint newbie myself. The only (half-baked) idea I can come up with is to use a naming convention for such derived classes. For example, if all such class names ended with "_d" (ugh!), you could modify the configuration setting like this: no-docstring-rgx=(__.*__|.*_d) -John From steve at holdenweb.com Tue Jan 26 11:54:23 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 26 Jan 2010 11:54:23 -0500 Subject: TypeError not caught by except statement In-Reply-To: <424b71ec1001252017i64d28375oc3644433c66ae40b@mail.gmail.com> References: <424b71ec1001252017i64d28375oc3644433c66ae40b@mail.gmail.com> Message-ID: <4B5F1E3F.3010800@holdenweb.com> siddhartha veedaluru wrote: > Hi, > > except not able to caught the TypeError exception occured in the below > code > > log.info ("refer",ret) in the try block > > throws a TypeError which is not caught . The traceback you provide does not show the line in question, so I am confused as to why you would think it was causing the problem. The purpose of the traceback is to show you where the exception occurred, and I question whether that is happening here. Anyway, I suspect your error might go away if you turned the first argument of hte log.info() call into a format string such as log.info("refer: %s", ret) regards Steve > Also sometimes process is getting hanged. > > ------------------------------------------------------------------------------------------------------------------------------------------------ > import logging > log = logging.getLogger() > fileName = strftime("%d-%b-%Y-", gmtime()) + str(int(time.time())) + "- > Log.log" > log = logging.getLogger() > log.setLevel(logging.NOTSET) > fh = logging.FileHandler(logFile) > logFileLevel = logging.DEBUG > fh.setLevel(logFileLevel) > format_string = '%(process)d %(thread)d %(asctime)-15s %(levelname)-5s > at %(filename)-15s in %(funcName)-10s at line %(lineno)-3d "%(message) > s"' > fh.setFormatter(logging.Formatter(format_string)) > log.addHandler(fh) > > try: > log.info ("start") > log.info ("refer",ret) > log.info ("end") > except TypeError: > log.exception("Exception raised") > > ---------------------------------------------------------------------------------------------------------------------------------------------- > OUTPUT message: > > Traceback (most recent call last): > File "C:\Python26\lib\logging\__init__.py", line 768, in emit > msg = self.format(record) > File "C:\Python26\lib\logging\__init__.py", line 648, in format > return fmt.format(record) > File "C:\Python26\lib\logging\__init__.py", line 436, in format > record.message = record.getMessage() > File "C:\Python26\lib\logging\__init__.py", line 306, in getMessage > msg = msg % self.args > TypeError: not all arguments converted during string formatting > -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Tue Jan 26 11:54:23 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 26 Jan 2010 11:54:23 -0500 Subject: TypeError not caught by except statement In-Reply-To: <424b71ec1001252017i64d28375oc3644433c66ae40b@mail.gmail.com> References: <424b71ec1001252017i64d28375oc3644433c66ae40b@mail.gmail.com> Message-ID: <4B5F1E3F.3010800@holdenweb.com> siddhartha veedaluru wrote: > Hi, > > except not able to caught the TypeError exception occured in the below > code > > log.info ("refer",ret) in the try block > > throws a TypeError which is not caught . The traceback you provide does not show the line in question, so I am confused as to why you would think it was causing the problem. The purpose of the traceback is to show you where the exception occurred, and I question whether that is happening here. Anyway, I suspect your error might go away if you turned the first argument of hte log.info() call into a format string such as log.info("refer: %s", ret) regards Steve > Also sometimes process is getting hanged. > > ------------------------------------------------------------------------------------------------------------------------------------------------ > import logging > log = logging.getLogger() > fileName = strftime("%d-%b-%Y-", gmtime()) + str(int(time.time())) + "- > Log.log" > log = logging.getLogger() > log.setLevel(logging.NOTSET) > fh = logging.FileHandler(logFile) > logFileLevel = logging.DEBUG > fh.setLevel(logFileLevel) > format_string = '%(process)d %(thread)d %(asctime)-15s %(levelname)-5s > at %(filename)-15s in %(funcName)-10s at line %(lineno)-3d "%(message) > s"' > fh.setFormatter(logging.Formatter(format_string)) > log.addHandler(fh) > > try: > log.info ("start") > log.info ("refer",ret) > log.info ("end") > except TypeError: > log.exception("Exception raised") > > ---------------------------------------------------------------------------------------------------------------------------------------------- > OUTPUT message: > > Traceback (most recent call last): > File "C:\Python26\lib\logging\__init__.py", line 768, in emit > msg = self.format(record) > File "C:\Python26\lib\logging\__init__.py", line 648, in format > return fmt.format(record) > File "C:\Python26\lib\logging\__init__.py", line 436, in format > record.message = record.getMessage() > File "C:\Python26\lib\logging\__init__.py", line 306, in getMessage > msg = msg % self.args > TypeError: not all arguments converted during string formatting > -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Tue Jan 26 11:57:17 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 26 Jan 2010 11:57:17 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <0b8ed84f-0a42-42c5-a1ba-e70ed048f39d@k19g2000yqc.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <0b8ed84f-0a42-42c5-a1ba-e70ed048f39d@k19g2000yqc.googlegroups.com> Message-ID: <4B5F1EED.5000503@holdenweb.com> Steve Howell wrote: > On Jan 24, 11:28 am, a... at pythoncraft.com (Aahz) wrote: >> In article , >> Steve Howell wrote: >> >> >> >>> Even with realloc()'s brokenness, you could improve pop(0) in a way >>> that does not impact list access at all, and the patch would not change >>> the time complexity of any operation; it would just add negligible >>> extract bookkeeping within list_resize() and a few other places. >> Again, your responsibility is to provide a patch and a spectrum of >> benchmarking tests to prove it. Then you would still have to deal with >> the objection that extensions use the list internals -- that might be an >> okay sell given the effort otherwise required to port extensions to >> Python 3, but that's not the way to bet. >> > > Ok, I just submitted a patch to python-dev that illustrates a 100x > speedup on an admittedly artificial program. It still has a long way > to go, but it demonstrates proof of concept. I'm done for the day, > but tomorrow I will try to polish it up and improve it, even if its > doomed for rejection. Apologies to all I have offended in this > thread. I frankly found some of the pushback to be a bit hasty and > disrespectful, but I certainly overreacted to some of the criticism. > And now I'm in the awkward position of asking the people I offended to > help me with the patch. If anybody can offer me a hand in > understanding some of CPython's internals, particularly with regard to > memory management, it would be greatly appreciated. > > (Sorry I don't have a link to the python-dev posting; it is not > showing up in the archives yet for some reason.) > > Fortunately for you this is a very welcoming group, and particularly responsive to individuals who have seen the error of their ways ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Tue Jan 26 11:57:17 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 26 Jan 2010 11:57:17 -0500 Subject: list.pop(0) vs. collections.dequeue In-Reply-To: <0b8ed84f-0a42-42c5-a1ba-e70ed048f39d@k19g2000yqc.googlegroups.com> References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <0b8ed84f-0a42-42c5-a1ba-e70ed048f39d@k19g2000yqc.googlegroups.com> Message-ID: <4B5F1EED.5000503@holdenweb.com> Steve Howell wrote: > On Jan 24, 11:28 am, a... at pythoncraft.com (Aahz) wrote: >> In article , >> Steve Howell wrote: >> >> >> >>> Even with realloc()'s brokenness, you could improve pop(0) in a way >>> that does not impact list access at all, and the patch would not change >>> the time complexity of any operation; it would just add negligible >>> extract bookkeeping within list_resize() and a few other places. >> Again, your responsibility is to provide a patch and a spectrum of >> benchmarking tests to prove it. Then you would still have to deal with >> the objection that extensions use the list internals -- that might be an >> okay sell given the effort otherwise required to port extensions to >> Python 3, but that's not the way to bet. >> > > Ok, I just submitted a patch to python-dev that illustrates a 100x > speedup on an admittedly artificial program. It still has a long way > to go, but it demonstrates proof of concept. I'm done for the day, > but tomorrow I will try to polish it up and improve it, even if its > doomed for rejection. Apologies to all I have offended in this > thread. I frankly found some of the pushback to be a bit hasty and > disrespectful, but I certainly overreacted to some of the criticism. > And now I'm in the awkward position of asking the people I offended to > help me with the patch. If anybody can offer me a hand in > understanding some of CPython's internals, particularly with regard to > memory management, it would be greatly appreciated. > > (Sorry I don't have a link to the python-dev posting; it is not > showing up in the archives yet for some reason.) > > Fortunately for you this is a very welcoming group, and particularly responsive to individuals who have seen the error of their ways ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From jeanmichel at sequans.com Tue Jan 26 12:12:48 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Tue, 26 Jan 2010 18:12:48 +0100 Subject: Accessing the name of an actual parameter In-Reply-To: References: <4B5ED690.1020706@hellmutweber.de> Message-ID: <4B5F2290.2040806@sequans.com> Duncan Booth wrote: > Gary Herron wrote: > > >> It's naive to think this question even makes sense. There are many ways >> f can be called which don't involve a parameter: >> >> f(42) >> f(time()) >> f(a+123) >> f(sin(a)) >> f(f(1)) >> >> and so on. >> > > So long as the OP doesn't care if they get no names or several name that > might not matter. However, explicitly passing in the name you want to use > is likely to be more useful. > > >>>> def getcallersnames(obj): >>>> > f = inspect.currentframe().f_back.f_back > return [name for name in f.f_locals if f.f_locals[name] is obj] > > >>>> def bip(x): >>>> > print getcallersnames(x) > > > >>>> def foo(bar): >>>> > baz = bar > bip(baz) > bip(baz+1) > > > >>>> foo(3) >>>> > ['bar', 'baz'] > [] > How do you know which between baz and bar has been used ? Here you get all names (in locals()) referencing the object passed in argument, you don't get the one actually used for the call. Another issue: class Foo: test = 'test' def foo(bar): bip(bar.test) f = Foo() foo(f) [] That restricts pretty much the use of getcallersnames. As said before, there is hardly any legit design which would require to know the actual refrence used to pass a parameter. JM From jeanmichel at sequans.com Tue Jan 26 12:20:42 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Tue, 26 Jan 2010 18:20:42 +0100 Subject: pylint and the 'missing docstring' warning In-Reply-To: <4B5F15F8.3010004@optimum.net> References: <4b5ef47f$0$31262$607ed4bc@cv.net> <4B5F15F8.3010004@optimum.net> Message-ID: <4B5F246A.6050009@sequans.com> John Posner wrote: > On 1/26/2010 9:22 AM, Jean-Michel Pichavant wrote: >> John Posner wrote: >>> On 1/26/2010 8:43 AM, Jean-Michel Pichavant wrote: >>>> Hello, >>>> >>>> Does anyone using pylint knows a way to make pylint ignore these >>>> 'missing docstring' warnings when the base class version of the method >>>> actually defines the docstring ? >>>> 'Cause my doc builder (epydoc) handle it properly and propagate >>>> docstrings if asked to. Too bad pylint is complaining about it. >>>> I don't want to deactivate this warning. >>>> >>> >>> Look for this setting in the pylint configuration file: >>> >>> # not require a docstring >>> no-docstring-rgx=__.*__ >>> >>> -John >>> >>> >> If I'm no wrong this just disables docstring checks for methods matching >> __.*__ >> I'd like to disable it for any method which base class version already >> defines the docstring. >> > > I'm a Pylint newbie myself. The only (half-baked) idea I can come up > with is to use a naming convention for such derived classes. For > example, if all such class names ended with "_d" (ugh!), you could > modify the configuration setting like this: > > no-docstring-rgx=(__.*__|.*_d) > > -John That would work but I definitely don't want to do that. JM From mal at egenix.com Tue Jan 26 12:21:09 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 26 Jan 2010 18:21:09 +0100 Subject: ctypes for AIX In-Reply-To: <9C240725D2ABD949A80B46DD93AB107042E8D44101@XCH-NW-08V.nw.nos.boeing.com> References: <9C240725D2ABD949A80B46DD93AB107042E8D44101@XCH-NW-08V.nw.nos.boeing.com> Message-ID: <4B5F2485.8070909@egenix.com> Waddle, Jim wrote: > I need to use ctypes with python running on AIX. It appears that python is being developed mostly for windows. Is there a policy concerning getting functions like ctypes working on AIX. If you can provide patches to get ctypes working on AIX, we'd consider adding those to the core. Please file patch or bug describing the problems you are having on AIX with the tracker: http://bugs.python.org/ Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 26 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From mal at egenix.com Tue Jan 26 12:37:53 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 26 Jan 2010 18:37:53 +0100 Subject: Authenticated encryption with PyCrypto In-Reply-To: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> Message-ID: <4B5F2871.6010102@egenix.com> Daniel wrote: > Just got done reading this thread: > > http://groups.google.com/group/comp.lang.python/browse_thread/thread/b31a5b5f58084f12/0e09f5f5542812c3 > > and I'd appreciate feedback on this recipe: > > http://code.activestate.com/recipes/576980/ > > Of course, it does not meet all of the requirements set forth by the > OP in the referenced thread (the pycrypto dependency is a problem), > but it is an attempt to provide a simple interface for performing > strong, password-based encryption. Are there already modules out there > that provide such a simple interface? If there are, they seem to be > hiding somewhere out of Google's view. > > I looked at ezPyCrypto, but it seemed to require public and private > keys, which was not convenient in my situation... maybe password-based > encryption is trivial to do with ezPyCrypto as well? In addition to > ezPyCrypto, I looked at Google's keyczar, but despite the claims of > the documentation, the API seemed overly complicated. Is it possible > to have a simple API for an industry-strength encryption module? If > not, is it possible to document that complicated API such that a non- > cryptographer could use it and feel confident that he hadn't made a > critical mistake? Yes, it is possible, but whatever you come up with will usually be bound to just one (or a few) different use cases, e.g. just look at the different cipher modes there are, the different key sizes, block sizes (for block ciphers), IV strings, padding, etc. etc. Note that your code has a padding bug: the decoder doesn't undo the padding. You're lucky though, since pickle will only read as much data as it needs and not complain about the extra data it finds. You are also using CBC mode, even though you are really after ECB mode (your code doesn't use chaining). With ECB mode, you don't need the IV string. > Also, slightly related, is there an easy way to get the sha/md5 > deprecation warnings emitted by PyCrypto in Python 2.6 to go away? Yes: you silence them via the warnings module. I suppose that the latest version of PyCrypto fixes these warnings. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 26 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From tmohr at s.netic.de Tue Jan 26 12:41:44 2010 From: tmohr at s.netic.de (Torsten Mohr) Date: Tue, 26 Jan 2010 18:41:44 +0100 Subject: deriving from array.array Message-ID: Hello, i try to derive a class from array.array: import array class Abc(array.array): def __init__(self, a, b): array.array.__init__(self, 'B') self.a = a self.b = b a = Abc(4, 5) print a print a.a I get an error for "a = Abc(4, 5)", seems the parameters are forwarded to array's __init__ as they are. Though i explicitly call __init__() for array. I'd like to use array and make sure it's type is always 'B'. I'd like to derive because i don't want to rewrite all the methods like __getiem__ for my class and then call array's __getitem__. How do i best derive from array.array? Thanks for any hints, Torsten. From alfps at start.no Tue Jan 26 13:24:27 2010 From: alfps at start.no (Alf P. Steinbach) Date: Tue, 26 Jan 2010 19:24:27 +0100 Subject: deriving from array.array In-Reply-To: References: Message-ID: * Torsten Mohr: > Hello, > > i try to derive a class from array.array: > > > import array > > class Abc(array.array): > def __init__(self, a, b): > array.array.__init__(self, 'B') > self.a = a > self.b = b > > > a = Abc(4, 5) > print a > print a.a > > > I get an error for "a = Abc(4, 5)", seems the parameters are > forwarded to array's __init__ as they are. No, with CPython they're forwarded to __new__. > Though i explicitly > call __init__() for array. That's the constructor inherited from 'object', it takes no args (except the self arg). > I'd like to use array and make sure it's type is always 'B'. > I'd like to derive because i don't want to rewrite all the methods like > __getiem__ for my class and then call array's __getitem__. > > How do i best derive from array.array? import array class ByteArray( array.array ): def __new__( self, *args ): return array.array.__new__( self, "B" ) def __init__( self, a, b ): array.array.__init__( self ) self.a = a self.b = b a = ByteArray( 4, 5 ) print( a ) print( a.a ) Disclaimer: I'm not a Python programmer. :-) Cheers & hth., - Alf From gslindstrom at gmail.com Tue Jan 26 13:58:47 2010 From: gslindstrom at gmail.com (gslindstrom at gmail.com) Date: Tue, 26 Jan 2010 18:58:47 +0000 Subject: Python or Ant Message-ID: <000e0cd70e4e3246b9047e15e0fb@google.com> My company is looking at creating a tool to allow us to define and manage a process for each job we run (a typical job may be look on a customers ftp site for a file, download it, decrypt it and load it into our database). We would like something which would allow us to glue together various existing processes we currently use into a single unit with multiple steps. Along the way, this new routine would need to log its progress and be able to report and even handle errors. A coworker has suggested we look at Ant ("Another Neat Tool") and, though it looks promising, I have reservations. If I recall correctly, it was intended as a replacement for "Make" and I worry that we may be trying to force Ant to be something it is not. Also, most of our code base is in Python and I'd really like to stay that way, of possible. Are there any systems out there that will allow me to run multiple programs as one process? We could write our own, of course, and the Twisted package looks like it would be fun to use. Or, is Ant a viable solution to our problem? Your constructive comments would be appreciated Greg Lindstrom Novasys Health Little Rock, Arkansas -------------- next part -------------- An HTML attachment was scrubbed... URL: From cmpython at gmail.com Tue Jan 26 13:59:10 2010 From: cmpython at gmail.com (CM) Date: Tue, 26 Jan 2010 10:59:10 -0800 (PST) Subject: Sikuli: the coolest Python project I have yet seen... References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> Message-ID: <89c2550d-057d-4cc2-a855-d4f3f11cb272@b10g2000vbh.googlegroups.com> On Jan 24, 10:18?pm, Ron wrote: > Sikuli is the coolest Python project I have ever seen in my ten year > hobbyist career. An MIT oepn source project, Sikuli uses Python to > automate GUI tasks (in any GUI or GUI baed app that runs the JVM) by > simply drag and dropping GUI elements into Python scripts as function > arguments. Download athttp://sikuli.csail.mit.edu/I also did this > podcast about Sikulihttp://media.libsyn.com/media/awaretek/Python411_20100124_Sikuli.mp3 How is this preferable to a macro recorder? From alfps at start.no Tue Jan 26 14:23:57 2010 From: alfps at start.no (Alf P. Steinbach) Date: Tue, 26 Jan 2010 20:23:57 +0100 Subject: deriving from array.array In-Reply-To: References: Message-ID: * Alf P. Steinbach: > * Torsten Mohr: >> Hello, >> >> i try to derive a class from array.array: >> >> >> import array >> >> class Abc(array.array): >> def __init__(self, a, b): >> array.array.__init__(self, 'B') >> self.a = a >> self.b = b >> >> >> a = Abc(4, 5) >> print a >> print a.a >> >> >> I get an error for "a = Abc(4, 5)", seems the parameters are >> forwarded to array's __init__ as they are. > > No, with CPython they're forwarded to __new__. > > >> Though i explicitly >> call __init__() for array. > > That's the constructor inherited from 'object', it takes no args (except > the self arg). > > >> I'd like to use array and make sure it's type is always 'B'. >> I'd like to derive because i don't want to rewrite all the methods >> like __getiem__ for my class and then call array's __getitem__. >> >> How do i best derive from array.array? > > > import array > > class ByteArray( array.array ): > def __new__( self, *args ): > return array.array.__new__( self, "B" ) > > def __init__( self, a, b ): > array.array.__init__( self ) > self.a = a > self.b = b > > a = ByteArray( 4, 5 ) > print( a ) > print( a.a ) > > > > Disclaimer: I'm not a Python programmer. :-) Hm, good that I included a disclaimer. The above code is technically OK but it is misleading. The first argument to '__new__' is not a self argument but a type argument, better called 'cls' or some such. From the docs, "__new__() is a static method (special-cased so you need not declare it as such) that takes the class of which an instance was requested as its first argument" Cheers, - Alf (self-correcting) From tmohr at s.netic.de Tue Jan 26 14:28:26 2010 From: tmohr at s.netic.de (Torsten Mohr) Date: Tue, 26 Jan 2010 20:28:26 +0100 Subject: deriving from array.array References: Message-ID: Hello, thanks a lot for your hint, it works fine. >> I get an error for "a = Abc(4, 5)", seems the parameters are >> forwarded to array's __init__ as they are. > > No, with CPython they're forwarded to __new__. Not sure if i understand this correctly, if i derive from other classes (like wxpython widgets) i always call the bases __init__ . >> Though i explicitly >> call __init__() for array. > > That's the constructor inherited from 'object', it takes no args (except > the self arg). Is there a way to find out what i need to call? I haven't found much in the documentation. From writing C extensions i knew about the "new" entry in the PyTypeObject struct but it seems there's more behind it. In docs.python.org i did not find much, is there an URL where i can read more? Best regards, Torsten. From aahz at pythoncraft.com Tue Jan 26 14:35:06 2010 From: aahz at pythoncraft.com (Aahz) Date: 26 Jan 2010 11:35:06 -0800 Subject: Py 3: Terminal script can't find relative path References: <58e9b1e4-049a-4a7f-bea9-8d7e7e9f191a@e27g2000yqd.googlegroups.com> <9541dcf0-8060-4851-bdbd-0aed6de13bf9@v25g2000yqk.googlegroups.com> <317ebb29-8c20-4fe5-82c4-b2fe05efef38@h9g2000yqa.googlegroups.com> <98c70d6f-5872-436b-b160-01b39d17d270@a32g2000yqm.googlegroups.com> Message-ID: In article <98c70d6f-5872-436b-b160-01b39d17d270 at a32g2000yqm.googlegroups.com>, Gnarlodious wrote: > >OK I guess that is normal, I fixed it with this: >path=os.path.dirname(__file__)+"/Data/" Use os.path.join() instead -- just in case you ever port to Windows -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From debatem1 at gmail.com Tue Jan 26 14:40:08 2010 From: debatem1 at gmail.com (geremy condra) Date: Tue, 26 Jan 2010 14:40:08 -0500 Subject: Authenticated encryption with PyCrypto In-Reply-To: <4B5F2871.6010102@egenix.com> References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> <4B5F2871.6010102@egenix.com> Message-ID: On Tue, Jan 26, 2010 at 12:37 PM, M.-A. Lemburg wrote: > You are also using CBC mode, even though you are really after > ECB mode (your code doesn't use chaining). With ECB mode, you > don't need the IV string. However, ECB mode is not as secure- the IV is the right way to go here. I'd also note that you aren't supposed to use RandomPool anymore, and that AES-192 is frequently recommended over AES-256 for new applications due to a number of recent developments in the cryptanalysis of its key schedule. Geremy Condra From no.email at nospam.invalid Tue Jan 26 14:45:14 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Tue, 26 Jan 2010 11:45:14 -0800 Subject: Authenticated encryption with PyCrypto References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> Message-ID: <7xwrz4abw5.fsf@ruckus.brouhaha.com> Daniel writes: > Of course, it does not meet all of the requirements set forth by the > OP in the referenced thread (the pycrypto dependency is a problem), > but it is an attempt to provide a simple interface for performing > strong, password-based encryption. Are there already modules out there > that provide such a simple interface? If there are, they seem to be > hiding somewhere out of Google's view. http://www.nightsong.com/phr/crypto/p3.py I need to update it to handle 64-bit OS's and use Python 3.x-style print statements, but that is pretty trivial. From steve at holdenweb.com Tue Jan 26 14:46:55 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 26 Jan 2010 14:46:55 -0500 Subject: Python or Ant In-Reply-To: <000e0cd70e4e3246b9047e15e0fb@google.com> References: <000e0cd70e4e3246b9047e15e0fb@google.com> Message-ID: <4B5F46AF.7020407@holdenweb.com> gslindstrom at gmail.com wrote: > My company is looking at creating a tool to allow us to define and > manage a process for each job we run (a typical job may be look on a > customers ftp site for a file, download it, decrypt it and load it into > our database). We would like something which would allow us to glue > together various existing processes we currently use into a single unit > with multiple steps. Along the way, this new routine would need to log > its progress and be able to report and even handle errors. A coworker > has suggested we look at Ant ("Another Neat Tool") and, though it looks > promising, I have reservations. If I recall correctly, it was intended > as a replacement for "Make" and I worry that we may be trying to force > Ant to be something it is not. Also, most of our code base is in Python > and I'd really like to stay that way, of possible. > > Are there any systems out there that will allow me to run multiple > programs as one process? We could write our own, of course, and the > Twisted package looks like it would be fun to use. Or, is Ant a viable > solution to our problem? > > Your constructive comments would be appreciated > > Greg Lindstrom > Novasys Health > Little Rock, Arkansas > Take a look at fabric. It was designed for installation, but it might do what you want. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Tue Jan 26 14:46:55 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 26 Jan 2010 14:46:55 -0500 Subject: Python or Ant In-Reply-To: <000e0cd70e4e3246b9047e15e0fb@google.com> References: <000e0cd70e4e3246b9047e15e0fb@google.com> Message-ID: <4B5F46AF.7020407@holdenweb.com> gslindstrom at gmail.com wrote: > My company is looking at creating a tool to allow us to define and > manage a process for each job we run (a typical job may be look on a > customers ftp site for a file, download it, decrypt it and load it into > our database). We would like something which would allow us to glue > together various existing processes we currently use into a single unit > with multiple steps. Along the way, this new routine would need to log > its progress and be able to report and even handle errors. A coworker > has suggested we look at Ant ("Another Neat Tool") and, though it looks > promising, I have reservations. If I recall correctly, it was intended > as a replacement for "Make" and I worry that we may be trying to force > Ant to be something it is not. Also, most of our code base is in Python > and I'd really like to stay that way, of possible. > > Are there any systems out there that will allow me to run multiple > programs as one process? We could write our own, of course, and the > Twisted package looks like it would be fun to use. Or, is Ant a viable > solution to our problem? > > Your constructive comments would be appreciated > > Greg Lindstrom > Novasys Health > Little Rock, Arkansas > Take a look at fabric. It was designed for installation, but it might do what you want. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From alfps at start.no Tue Jan 26 14:51:38 2010 From: alfps at start.no (Alf P. Steinbach) Date: Tue, 26 Jan 2010 20:51:38 +0100 Subject: deriving from array.array In-Reply-To: References: Message-ID: * Torsten Mohr: > > thanks a lot for your hint, it works fine. > >>> I get an error for "a = Abc(4, 5)", seems the parameters are >>> forwarded to array's __init__ as they are. >> No, with CPython they're forwarded to __new__. > > Not sure if i understand this correctly, if i derive from other > classes (like wxpython widgets) i always call the bases __init__ . Well, creation of an object happens in two phases: allocation and initialization. Allocation reserves memory for the object and creates a sort of default object in that memory. This is the job of __new__. The object doesn't exist at all before __new__ is called. Initialization then outfits the object with all that's required for a proper object of class T. This is the job of __init__. __init__ as passed as argument the default object created by __new__. CPython's array.array does both in __new__. It does not override the __init__ it inherits from 'object'. To wit: >>> import array >>> id( array.array.__init__ ) 10965560 >>> id( object.__init__ ) 10965560 >>> array.array.__init__ is object.__init__ True >>> _ So overriding __init__ doesn't override any of array.array's functionality. But you'd have to override __new__ anyway, because with array.array the first argument has to be a type code specifying the element type of the array object the __new__ should create. An alternative design instead of those type codes could have had one subclass for each element type, essentially what you're doing, except you're only doing one of those subclasses. Then there would be no issue with __new__. >>> Though i explicitly >>> call __init__() for array. >> That's the constructor inherited from 'object', it takes no args (except >> the self arg). > > Is there a way to find out what i need to call? Testing. :-) > I haven't found much in > the documentation. From writing C extensions i knew about the "new" entry > in the PyTypeObject struct but it seems there's more behind it. > In docs.python.org i did not find much, is there an URL where i can read > more? Don't know, sorry. Google? Cheers, - Alf From malaclypse2 at gmail.com Tue Jan 26 14:53:02 2010 From: malaclypse2 at gmail.com (Jerry Hill) Date: Tue, 26 Jan 2010 14:53:02 -0500 Subject: deriving from array.array In-Reply-To: References: Message-ID: <16651e81001261153k7556ad50l79fe7b6afa12c4ff@mail.gmail.com> On Tue, Jan 26, 2010 at 2:28 PM, Torsten Mohr wrote: > Is there a way to find out what i need to call? I haven't found much in > the documentation. From writing C extensions i knew about the "new" entry > in the PyTypeObject struct but it seems there's more behind it. > In docs.python.org i did not find much, is there an URL where i can read > more? > http://docs.python.org/reference/datamodel.html#basic-customization talks about __new__() and __init__(). The basic rule of thumb is that when you inherit from an immutable type, you need to override __new__, and when you inherit from a mutable type you need to override __init__. I don't think that rule of thumb applies here, though, since an array.array is mutable. The actual rule is that you need to override __new__ if the class you're inheriting from overrides __new__. I don't think there's a straightforward way to know that ahead of time unless you look at the source code for the class. Either that, or you just see that you get a TypeError when initializing your object and realize it means you need to override the constructor. It would be nice if the TypeError was a little more explicit about what was going on, though. Here's what I get in IDLE for a class similar to yours: >>> import array >>> class my_a(array.array): def __init__(self, a, b): array.array.__init__(self, 'B') >>> a = my_a(1, 2) Traceback (most recent call last): File "", line 1, in a = my_a(1, 2) TypeError: array() argument 1 must be char, not int It would be nice if the traceback mentioned something about array.__new__() instead of just array(). -- Jerry -------------- next part -------------- An HTML attachment was scrubbed... URL: From john at castleamber.com Tue Jan 26 15:21:15 2010 From: john at castleamber.com (John Bokma) Date: Tue, 26 Jan 2010 14:21:15 -0600 Subject: Python or Ant References: <000e0cd70e4e3246b9047e15e0fb@google.com> Message-ID: <87ljfk1utg.fsf@castleamber.com> Steve Holden writes: Apologies for the piggy back, my Usenet provider can't get the OP's message. > gslindstrom at gmail.com wrote: >> My company is looking at creating a tool to allow us to define and >> manage a process for each job we run (a typical job may be look on a >> customers ftp site for a file, download it, decrypt it and load it into >> our database). We would like something which would allow us to glue >> together various existing processes we currently use into a single unit >> with multiple steps. Along the way, this new routine would need to log >> its progress and be able to report and even handle errors. A coworker >> has suggested we look at Ant ("Another Neat Tool") and, though it looks >> promising, I have reservations. If I recall correctly, it was intended >> as a replacement for "Make" and I worry that we may be trying to force >> Ant to be something it is not. It was intended as a replacement. But you can use it for many other things, without making it feel out of place in my opinion. I do use it exactly the way you describe for various tasks: calling external programs, uploading a file, etc. My personal site, see sig, is generated by a Perl program, and uploaded by another, and some other programs have to run as well, and all is glued together using ant. If I modify an XML file (my site is defined using XML), I do: ant local to update a local version. If I am happy with how it looks, I do ant upload Ant makes several things really easy. The things I do with it would take me more lines in either Perl or Python. So I use Perl (or Python) when it makes things easier, and ant to glue everything together if that's easier. -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From ursusmaximus at gmail.com Tue Jan 26 15:24:48 2010 From: ursusmaximus at gmail.com (Ron) Date: Tue, 26 Jan 2010 12:24:48 -0800 (PST) Subject: Sikuli: the coolest Python project I have yet seen... References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> <89c2550d-057d-4cc2-a855-d4f3f11cb272@b10g2000vbh.googlegroups.com> Message-ID: <73d864ff-bd11-46af-ba80-c5081189c92c@e25g2000yqh.googlegroups.com> On Jan 26, 10:59?am, CM wrote: > On Jan 24, 10:18?pm, Ron wrote: > > > Sikuli is the coolest Python project I have ever seen in my ten year > > hobbyist career. An MIT open source project, Sikuli uses Python to > > automate GUI tasks (in any GUI or GUI based app that runs the JVM) by > > simply drag and dropping GUI elements into Python scripts as function > > Arguments. Download athttp://sikuli.csail.mit.edu/Ialso did this > > podcast about Sikulihttp://media.libsyn.com/media/awaretek/Python411_20100124_Sikuli.mp3 > > How is this preferable to a macro recorder? Well, the pattern recognition engine allows you to recognize partial matches to any image, to any desired degree of accuracy. In other words, you can specify to take action only when an exact match is found, or else when a 50% match is found. This allows applications like the baby monitor (to tell you when your sleeping baby wakes up, and the imminent bus arrival monitor (to tell you when your bus is within one mile or any distance you want), and the route mapper from one city to another on a digital map. Another thing, Sikuli works with web pages. In other words, you can automate interaction with web sites as well as with desktop applications. You can also automate the entry of text. And it works (theoretically) on any graphical platform (Mac, Linux, Window, smartphones, etc). Probably other advantages. Those are just the ones I see off the top of my head. Good question. Ron From clp2 at rebertia.com Tue Jan 26 15:31:24 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Tue, 26 Jan 2010 12:31:24 -0800 Subject: Python or Ant In-Reply-To: <000e0cd70e4e3246b9047e15e0fb@google.com> References: <000e0cd70e4e3246b9047e15e0fb@google.com> Message-ID: <50697b2c1001261231g4a7dd9a4w388c5ac766fe6d84@mail.gmail.com> On Tue, Jan 26, 2010 at 10:58 AM, wrote: > My company is looking at creating a tool to allow us to define and manage a > process for each job we run (a typical job may be look on a customers ftp > site for a file, download it, decrypt it and load it into our database). We > would like something which would allow us to glue together various existing > processes we currently use into a single unit with multiple steps. Along the > way, this new routine would need to log its progress and be able to report > and even handle errors. A coworker has suggested we look at Ant ("Another > Neat Tool") and, though it looks promising, I have reservations. If I recall > correctly, it was intended as a replacement for "Make" and I worry that we > may be trying to force Ant to be something it is not. Also, most of our code > base is in Python and I'd really like to stay that way, of possible. > > Are there any systems out there that will allow me to run multiple programs > as one process? We could write our own, of course, and the Twisted package > looks like it would be fun to use. Or, is Ant a viable solution to our > problem? > > Your constructive comments would be appreciated There are several make-replacements written in Python. They could be an option. Here's a list of some of them (courtesy some googling): - Paver (http://www.blueskyonmars.com/projects/paver/) - SCons (http://www.scons.org/) - Vellum (https://launchpad.net/vellum) - Aap (http://www.a-a-p.org/) (List is in no particular order and likely incomplete; I have not tried any of these myself.) Cheers, Chris -- http://blog.rebertia.com From javier.collado at gmail.com Tue Jan 26 15:42:31 2010 From: javier.collado at gmail.com (Javier Collado) Date: Tue, 26 Jan 2010 21:42:31 +0100 Subject: Python or Ant In-Reply-To: <50697b2c1001261231g4a7dd9a4w388c5ac766fe6d84@mail.gmail.com> References: <000e0cd70e4e3246b9047e15e0fb@google.com> <50697b2c1001261231g4a7dd9a4w388c5ac766fe6d84@mail.gmail.com> Message-ID: Hello, One tool that I really like is doit: http://python-doit.sourceforge.net/ If you need to execute jobs remotely, you may like to take a look at STAF: http://staf.sourceforge.net/index.php Best regards, Javier 2010/1/26 Chris Rebert : > On Tue, Jan 26, 2010 at 10:58 AM, ? wrote: >> My company is looking at creating a tool to allow us to define and manage a >> process for each job we run (a typical job may be look on a customers ftp >> site for a file, download it, decrypt it and load it into our database). We >> would like something which would allow us to glue together various existing >> processes we currently use into a single unit with multiple steps. Along the >> way, this new routine would need to log its progress and be able to report >> and even handle errors. A coworker has suggested we look at Ant ("Another >> Neat Tool") and, though it looks promising, I have reservations. If I recall >> correctly, it was intended as a replacement for "Make" and I worry that we >> may be trying to force Ant to be something it is not. Also, most of our code >> base is in Python and I'd really like to stay that way, of possible. >> >> Are there any systems out there that will allow me to run multiple programs >> as one process? We could write our own, of course, and the Twisted package >> looks like it would be fun to use. Or, is Ant a viable solution to our >> problem? >> >> Your constructive comments would be appreciated > > There are several make-replacements written in Python. They could be an option. > > Here's a list of some of them (courtesy some googling): > - Paver (http://www.blueskyonmars.com/projects/paver/) > - SCons (http://www.scons.org/) > - Vellum (https://launchpad.net/vellum) > - Aap (http://www.a-a-p.org/) > > (List is in no particular order and likely incomplete; I have not > tried any of these myself.) > > Cheers, > Chris > -- > http://blog.rebertia.com > -- > http://mail.python.org/mailman/listinfo/python-list > From mal at egenix.com Tue Jan 26 15:52:31 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 26 Jan 2010 21:52:31 +0100 Subject: Authenticated encryption with PyCrypto In-Reply-To: <4B5F2871.6010102@egenix.com> References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> <4B5F2871.6010102@egenix.com> Message-ID: <4B5F560F.2010201@egenix.com> M.-A. Lemburg wrote: > Daniel wrote: >> Just got done reading this thread: >> >> http://groups.google.com/group/comp.lang.python/browse_thread/thread/b31a5b5f58084f12/0e09f5f5542812c3 >> >> and I'd appreciate feedback on this recipe: >> >> http://code.activestate.com/recipes/576980/ >> > [...] > You are also using CBC mode, even though you are really after > ECB mode (your code doesn't use chaining). With ECB mode, you > don't need the IV string. Sorry. Forget that last comment - your code does use chaining. It's hidden away in the PyCrypto code for block cipher encoding: the .encode() method will automatically apply the cipher to all blocks of 32 byte input data and apply chaining to all of them (initializing the data area with the IV string). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 26 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From mal at egenix.com Tue Jan 26 16:00:22 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 26 Jan 2010 22:00:22 +0100 Subject: Authenticated encryption with PyCrypto In-Reply-To: References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> <4B5F2871.6010102@egenix.com> Message-ID: <4B5F57E6.9050200@egenix.com> geremy condra wrote: > On Tue, Jan 26, 2010 at 12:37 PM, M.-A. Lemburg wrote: > > > >> You are also using CBC mode, even though you are really after >> ECB mode (your code doesn't use chaining). With ECB mode, you >> don't need the IV string. > > However, ECB mode is not as secure- the IV is the right way to go > here. Right - I forgot that PyCrypto applies the chaining internally when being passed data of more than 32 bytes. > I'd also note that you aren't supposed to use RandomPool anymore, > and that AES-192 is frequently recommended over AES-256 for > new applications due to a number of recent developments in > the cryptanalysis of its key schedule. Do you have pointers for this ? I could only find http://en.wikipedia.org/wiki/Advanced_Encryption_Standard#Security and https://cryptolux.org/Block My reading of their FAQ (https://cryptolux.org/FAQ_on_the_attacks) is that using AES-128 is the way to go (and it's faster too) - at least for the time being. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 26 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From tjreedy at udel.edu Tue Jan 26 16:09:25 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 26 Jan 2010 16:09:25 -0500 Subject: Sikuli: the coolest Python project I have yet seen... In-Reply-To: <89c2550d-057d-4cc2-a855-d4f3f11cb272@b10g2000vbh.googlegroups.com> References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> <89c2550d-057d-4cc2-a855-d4f3f11cb272@b10g2000vbh.googlegroups.com> Message-ID: On 1/26/2010 1:59 PM, CM wrote: > On Jan 24, 10:18 pm, Ron wrote: >> Sikuli is the coolest Python project I have ever seen in my ten year >> hobbyist career. An MIT oepn source project, Sikuli uses Python to >> automate GUI tasks (in any GUI or GUI baed app that runs the JVM) by >> simply drag and dropping GUI elements into Python scripts as function >> arguments. Download athttp://sikuli.csail.mit.edu/I also did this >> podcast about Sikulihttp://media.libsyn.com/media/awaretek/Python411_20100124_Sikuli.mp3 > > How is this preferable to a macro recorder? One can add program logic to the interaction. For instance, one of the demos on YouTube uses sikuli to read a Bejeweled (game) board. Program logic then calculates a move. From tjreedy at udel.edu Tue Jan 26 16:22:53 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 26 Jan 2010 16:22:53 -0500 Subject: My experiences building a small app on Python In-Reply-To: <62c367db-49e3-485b-80d6-ed2f99065f48@22g2000yqr.googlegroups.com> References: <62c367db-49e3-485b-80d6-ed2f99065f48@22g2000yqr.googlegroups.com> Message-ID: On 1/26/2010 8:51 AM, Cascade3891 wrote: > Hello Python community, > > I built a small app with Python for the first time. It has served a > very useful and productive purpose for me and my colleagues at work. > > I have shared my experiences on the following link due to the better > web presentation tools and coding presentation tools: > http://www.daniweb.com/forums/thread255458.html > > It's a bit of a read. But insightful. > > If you have any feedback or tips and advice, please let me know. When one has multiple small images that are all the same size, a standard trick is to have *one* image (and one file -- a lossless .bmp or .png) that is a tiled composite of the images. Read in the image and 'snip' apart the tiles. I have also seen bitmaps encoded in code as a hex string (for use with tk), but I do not know how well that would work with wx. > I've fallen in love with programming because of Python. =) You are not the only one. Welcome to the club. Terry Jan Reedy From millerdev at gmail.com Tue Jan 26 17:51:00 2010 From: millerdev at gmail.com (Daniel) Date: Tue, 26 Jan 2010 14:51:00 -0800 (PST) Subject: Authenticated encryption with PyCrypto References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> Message-ID: <837d2986-29b4-4cf7-b6a7-347008281d11@p6g2000vbl.googlegroups.com> On Jan 26, 12:37?pm, "M.-A. Lemburg" wrote: > Note that your code has a padding bug: the decoder doesn't > undo the padding. You're lucky though, since pickle will only > read as much data as it needs and not complain about the extra > data it finds. Doesn't the last line in decrypt() do it? return data[:-ord(data[-1])] Given, it's a bit cryptic... no pun intended :) > > Also, slightly related, is there an easy way to get the sha/md5 > > deprecation warnings emitted by PyCrypto in Python 2.6 to go away? > > Yes: you silence them via the warnings module. I suppose that the > latest version of PyCrypto fixes these warnings. The version that gets installed by easy_install or pip (2.0.1) emits those warnings. Is there a more recent version? Thanks for the feedback. ~ Daniel From mal at egenix.com Tue Jan 26 18:22:15 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 27 Jan 2010 00:22:15 +0100 Subject: Authenticated encryption with PyCrypto In-Reply-To: <837d2986-29b4-4cf7-b6a7-347008281d11@p6g2000vbl.googlegroups.com> References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> <837d2986-29b4-4cf7-b6a7-347008281d11@p6g2000vbl.googlegroups.com> Message-ID: <4B5F7927.1010900@egenix.com> Daniel wrote: > On Jan 26, 12:37 pm, "M.-A. Lemburg" wrote: >> Note that your code has a padding bug: the decoder doesn't >> undo the padding. You're lucky though, since pickle will only >> read as much data as it needs and not complain about the extra >> data it finds. > > Doesn't the last line in decrypt() do it? > > return data[:-ord(data[-1])] > > Given, it's a bit cryptic... no pun intended :) That's cryptic indeed... I just found that you're not padding with zero bytes, but instead with char(pad) where pad is the number of bytes you add: pad = AES_BLOCK_SIZE - len(data) % AES_BLOCK_SIZE data = data + pad * chr(pad) This code will pad with 16 bytes of chr(16) in case len(data) is in fact on a block size boundary. When using pickle, you don't need this, since pickle includes all necessary length information in the serialized data stream. I'd just pad with \0 and not worry about the extra bytes at the end when using pickle to serialize the objects. It's more important to worry about whether you really want to unpickle the data or not, since pickle opens up lots of possibilities of executing code on the decoding side of the communication channel. >>> Also, slightly related, is there an easy way to get the sha/md5 >>> deprecation warnings emitted by PyCrypto in Python 2.6 to go away? >> >> Yes: you silence them via the warnings module. I suppose that the >> latest version of PyCrypto fixes these warnings. > > The version that gets installed by easy_install or pip (2.0.1) emits > those warnings. Is there a more recent version? This is the most recent version: http://www.dlitz.net/software/pycrypto/ -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 27 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From aahz at pythoncraft.com Tue Jan 26 18:36:26 2010 From: aahz at pythoncraft.com (Aahz) Date: 26 Jan 2010 15:36:26 -0800 Subject: Splitting text at whitespace but keeping the whitespace in thereturned list References: <1264352736.30280.1356298585@webmail.messagingengine.com> Message-ID: In article , Roy Smith wrote: > >I've never used partition() before; my first thought on reading the above >was, "That's weird, it should be returning a list". Then I went and looked >at the docs. Given the description (returns specifically a 3-tuple), I >guess a tuple makes sense, but now I'm wondering what the use case was for >this method when it was invented? http://docs.python.org/whatsnew/2.5.html -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From xahlee at gmail.com Tue Jan 26 18:47:35 2010 From: xahlee at gmail.com (Xah Lee) Date: Tue, 26 Jan 2010 15:47:35 -0800 (PST) Subject: python 3's adoption Message-ID: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Some thoughts about Python 3 Adoption. Xah Lee, 2010-01-26 Some notes of Wikipedia readings related to Python. Unladen Swallow, a new project from Google. It is a new python compiler with the goal of 5 times faster than the de facto standand implementation CPython. Also note Stackless Python, which is already been used in some major commercial projects. Was looking into what's new in Python 3. See: http://docs.python.org/dev/3.0/whatsnew/3.0.html. >From a quick reading, i don't really like it. Here's some highlights: * Print is now a function. Great, much improvement. * Many functions that return lists now returns ?Views? or ?Iterators? Instead. A fucking fuck all fucked up shit. A extraneous ?oop engineering? complication. (See: Lambda in Python 3000) * The cmp() function used in sort is basically gone, users are now supposed to use the ?key? parameter instead. This is a flying-face- fuck to computer science. This would be the most serious fuckup in python 3. (See: Sorting in Python and Perl) * Integers by default is long. Great! * Much more integrated unicode support, rewrite of most its text or string semantics. Fantastic. Finally. Am looking because i wonder if i should switch to python 3 for my own few scripts, and rewrite my Python Tutorial for version 3. Am also interested to know how python 3 is received by the computing industry. Apparantly, a little search on the web indicates that vast majority of python base have not switched, as expected, for many good reasons. Vast majority of major python modules and tools have not switched. Most linux distro have not switched, i don't find any large corporation having adopted Python 3 (Google, Yahoo, Facebook, NASA,... ). (sources: Source, Source) Basically, such a incompatible change with trivial, ideological improvements, is too costy to switch. I wonder, if by 2015, will most large corporate users have switched to python 3. I give it a maybe. In today's Proliferation of Computing Languages, such a major antic by Guido can just hurt itself. What is he thinking? He of course thought himself as a god of lang designer, who sincerely wants to push towards perfection, all future-looking. Unfortunately, the tens of other major language designers all think similarly. perm archive of this post with possible updates here: http://xahlee.org/comp/python3.html Any comment on this? Xah ? http://xahlee.org/ ? From millerdev at gmail.com Tue Jan 26 19:07:23 2010 From: millerdev at gmail.com (Daniel) Date: Tue, 26 Jan 2010 16:07:23 -0800 (PST) Subject: Authenticated encryption with PyCrypto References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> <4B5F2871.6010102@egenix.com> Message-ID: <0eb13c9a-29d0-4019-b221-794fdb500213@22g2000yqr.googlegroups.com> geremy condra wrote: > I'd also note that you aren't supposed to use RandomPool anymore, OK, I updated the recipe to use os.urandom() > and that AES-192 is frequently recommended over AES-256 for > new applications due to a number of recent developments in > the cryptanalysis of its key schedule. Changed easily enough. The updated recipe defaults to AES-192. I also made one other minor tweak: the global constants were moved to class-level so they can be overridden more easily if needed. Daniel From millerdev at gmail.com Tue Jan 26 19:23:02 2010 From: millerdev at gmail.com (Daniel) Date: Tue, 26 Jan 2010 16:23:02 -0800 (PST) Subject: Authenticated encryption with PyCrypto References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> <837d2986-29b4-4cf7-b6a7-347008281d11@p6g2000vbl.googlegroups.com> Message-ID: <7169bb89-c22b-4826-83bc-b72b77150c30@e25g2000yqh.googlegroups.com> M.-A. Lemburg wrote: > Daniel wrote: > > On Jan 26, 12:37 pm, "M.-A. Lemburg" wrote: > >> Note that your code has a padding bug: the decoder doesn't > >> undo the padding. You're lucky though, since pickle will only > >> read as much data as it needs and not complain about the extra > >> data it finds. > > > Doesn't the last line in decrypt() do it? > > > ? ? return data[:-ord(data[-1])] > > > Given, it's a bit cryptic... no pun intended :) > > That's cryptic indeed... I just found that you're not padding > with zero bytes, but instead with char(pad) where pad is the > number of bytes you add: > > ? ? ? ? pad = AES_BLOCK_SIZE - len(data) % AES_BLOCK_SIZE > ? ? ? ? data = data + pad * chr(pad) > > This code will pad with 16 bytes of chr(16) in case len(data) > is in fact on a block size boundary. > > When using pickle, you don't need this, since pickle includes > all necessary length information in the serialized data stream. > > I'd just pad with \0 and not worry about the extra bytes > at the end when using pickle to serialize the objects. I think I'll leave the padding in there since it keeps the encrypt/ decrypt methods usable for non-pickle data. > It's more important to worry about whether you really > want to unpickle the data or not, since pickle opens > up lots of possibilities of executing code on the decoding > side of the communication channel. I understand the risks of unpickle. With strong, authenticated encryption I think it is reasonably safe to send an encrypted pickle through an untrusted medium (the Internet) and know that it has not been modified enroute. That is, unless someone has obtained the key, in which case I have a bigger problem to worry about. > >>> Also, slightly related, is there an easy way to get the sha/md5 > >>> deprecation warnings emitted by PyCrypto in Python 2.6 to go away? > > >> Yes: you silence them via the warnings module. I suppose that the > >> latest version of PyCrypto fixes these warnings. > > > The version that gets installed by easy_install or pip (2.0.1) emits > > those warnings. Is there a more recent version? > > This is the most recent version: > > ? ? ? ?http://www.dlitz.net/software/pycrypto/ Thanks. Daniel From fordhaivat at gmail.com Tue Jan 26 19:36:24 2010 From: fordhaivat at gmail.com (Someone Something) Date: Tue, 26 Jan 2010 19:36:24 -0500 Subject: Just drawing lines and plotting points? Message-ID: Hello, I need a python library that makes drawing lines and plotting points (these two things are the only things I need to do) easy. Or, how can I do something like this with pygame? Basically, what I want to do is make graphs. In pygame, since the coordinate system switches the x's and the y's I would have to switch them again when I plot my points so my graphs look okay. I hope this was enough info for me to get a good answer. -------------- next part -------------- An HTML attachment was scrubbed... URL: From clp2 at rebertia.com Tue Jan 26 19:54:56 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Tue, 26 Jan 2010 16:54:56 -0800 Subject: Just drawing lines and plotting points? In-Reply-To: References: Message-ID: <50697b2c1001261654i7a64adaai3df49eac8a859eae@mail.gmail.com> On Tue, Jan 26, 2010 at 4:36 PM, Someone Something wrote: > Hello, > > I need a python library that makes drawing lines and plotting points (these > two things are the only things I need to do) easy. Or, how can I do > something like this with pygame? Basically, what I want to do is make > graphs. In pygame, since the coordinate system switches the x's and the y's > I would have to switch them again when I plot my points so my graphs look > okay. I hope this was enough info for me to get a good answer. matplotlib, perhaps?: http://matplotlib.sourceforge.net/ Cheers, Chris -- http://blog.rebertia.com From aharrisreid at googlemail.com Tue Jan 26 19:55:41 2010 From: aharrisreid at googlemail.com (Alan Harris-Reid) Date: Wed, 27 Jan 2010 00:55:41 +0000 Subject: python 3's adoption In-Reply-To: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: <4B5F8F0D.3060205@googlemail.com> Xah Lee wrote: > Some thoughts about Python 3 Adoption. > > Xah Lee, 2010-01-26 > > Some notes of Wikipedia readings related to Python. > > Unladen Swallow, a new project from Google. It is a new python > compiler with the goal of 5 times faster than the de facto standand > implementation CPython. Also note Stackless Python, which is already > been used in some major commercial projects. > > Was looking into what's new in Python 3. See: http://docs.python.org/dev/3.0/whatsnew/3.0.html. > >From a quick reading, i don't really like it. Here's some highlights: > > * Print is now a function. Great, much improvement. > * Many functions that return lists now returns ?Views? or > ?Iterators? Instead. A fucking fuck all fucked up shit. A extraneous > ?oop engineering? complication. (See: Lambda in Python 3000) > * The cmp() function used in sort is basically gone, users are now > supposed to use the ?key? parameter instead. This is a flying-face- > fuck to computer science. This would be the most serious fuckup in > python 3. (See: Sorting in Python and Perl) > * Integers by default is long. Great! > * Much more integrated unicode support, rewrite of most its text > or string semantics. Fantastic. Finally. > > Am looking because i wonder if i should switch to python 3 for my own > few scripts, and rewrite my Python Tutorial for version 3. Am also > interested to know how python 3 is received by the computing industry. > Apparantly, a little search on the web indicates that vast majority of > python base have not switched, as expected, for many good reasons. > Vast majority of major python modules and tools have not switched. > Most linux distro have not switched, i don't find any large > corporation having adopted Python 3 (Google, Yahoo, Facebook, > NASA,... ). (sources: Source, Source) Basically, such a incompatible > change with trivial, ideological improvements, is too costy to switch. > > I wonder, if by 2015, will most large corporate users have switched to > python 3. I give it a maybe. In today's Proliferation of Computing > Languages, such a major antic by Guido can just hurt itself. What is > he thinking? He of course thought himself as a god of lang designer, > who sincerely wants to push towards perfection, all future-looking. > Unfortunately, the tens of other major language designers all think > similarly. > > perm archive of this post with possible updates here: > http://xahlee.org/comp/python3.html > > Any comment on this? > > Xah > ? http://xahlee.org/ Hello Xah, I have no figures to base this on (just what I have read on the web), but although the vast majority of comanies with big Python investments are probably waiting for the 'critical mass' to use Python3 regularly (oil-tanker effect), I would like to think that smaller operations are experimenting with it more-and-more. I think that for beginners who have dived into Python in the last 6-12 months (like me), it doesn't make sense to start with an older version. I do not want to learn 'old' syntax and functions of a language, only to have to learn the new versions when most developers upgrade to 3 - I might as well learn the new syntax now and be ahead-of-the-game ;-) . I know my choice of related packages (web-framework, ORM, templating-engine) is very limited at present, but there are enough branches and beta-versions around to help me with my learning-curve, and I figure there will be some full-production-releases around by the time I am 'fluent' with Python (12-18 months?). Currently I am using Python 3.1 and CherryPy (3.20 rc1) every day, and have had no serious problems (yet). I would be interested to hear how other people are using Python 3, and with what compatible packages. Regards, Alan Harris-Reid From alfps at start.no Tue Jan 26 20:28:00 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 02:28:00 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: I'm responding to the original message by Xah Lee, which is not carried by my Usenet provider. * Alan Harris-Reid: > Xah Lee wrote: >> Some thoughts about Python 3 Adoption. >> >> Xah Lee, 2010-01-26 >> >> Some notes of Wikipedia readings related to Python. >> >> Unladen Swallow, a new project from Google. It is a new python >> compiler with the goal of 5 times faster than the de facto standand >> implementation CPython. Also note Stackless Python, which is already >> been used in some major commercial projects. >> >> Was looking into what's new in Python 3. See: >> http://docs.python.org/dev/3.0/whatsnew/3.0.html. >> >From a quick reading, i don't really like it. Here's some highlights: >> >> * Print is now a function. Great, much improvement. Actually not, IMHO. All it does is is to provide incompatibility. They forgot Ronald Reagan's old maxim: if it don't need fixin', don't fix it. >> * Many functions that return lists now returns ?Views? or >> ?Iterators? Instead. A fucking fuck all fucked up shit. A extraneous >> ?oop engineering? complication. (See: Lambda in Python 3000) On the contrary, this is a great improvement. It makes no sense to have to write more code and less readable code for the common efficient case. And the default, the way that takes 0.5 seconds less to do, does have a great influence on what people choose, even in elections (Obama's latest "opt out" proposal is an example that this principle is recognized even by the President of the United States). When you want an explicit collection such as a 'list', introducing some overhead for presumably a good reason, it's as easy as e.g. 'list(range(42))'. >> * The cmp() function used in sort is basically gone, users are now >> supposed to use the ?key? parameter instead. This is a flying-face- >> fuck to computer science. This would be the most serious fuckup in >> python 3. (See: Sorting in Python and Perl) I agree. :-) Probably there must have been some rationale, but to put it bluntly removing the comparator is more like moronic than pythonic. If the rationale was efficiency, then a rational solution could be to provide two sort methods, like 'sort' using direct comparisions and 'custom_sort' using a specified comparator. Come to think of it, that door is still open, except for the language spec freeze. It's possible to work around the removal of 'cmp' in various kludgy ways, but you shouldn't have to "work around" a library sort routine. >> * Integers by default is long. Great! Yes, totally agree. Nothing confuses the newbie or the now-and-then programmer so much as limited range integers. Well, perhaps floating point, but... >> * Much more integrated unicode support, rewrite of most its text >> or string semantics. Fantastic. Finally. I'm not sure if the Unicode support is fantastic. I have the impression that there are still some edges to iron out and corners to smooth, + bugs to kill. But it's Very Good. [snip] Cheers, - Alf From invalid at invalid.invalid Tue Jan 26 20:38:37 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Wed, 27 Jan 2010 01:38:37 +0000 (UTC) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: On 2010-01-27, Alf P. Steinbach wrote: > I'm responding to the original message by Xah Lee, which is > not carried by my Usenet provider. A Usenet provider that doesn't carry messages from Xah Lee. So... many... jokes..... -- Grant From wuwei23 at gmail.com Tue Jan 26 20:54:15 2010 From: wuwei23 at gmail.com (alex23) Date: Tue, 26 Jan 2010 17:54:15 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> "Alf P. Steinbach" wrote: > Actually not, IMHO. All it does is is to provide incompatibility. They forgot > Ronald Reagan's old maxim: if it don't need fixin', don't fix it. [...] > Probably there must have been some rationale, but to put it bluntly removing the > comparator is more like moronic than pythonic. So in both cases, you haven't put any effort into researching why these decisions were made, but you're happy to jump to conclusions about the intelligence of the implementer(s) while regularly trotting out complaints of ad hominem attacks against yourself. The saddest part of all is you seem to consider yourself some kind of educator. From python at mrabarnett.plus.com Tue Jan 26 21:03:32 2010 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 27 Jan 2010 02:03:32 +0000 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: <4B5F9EF4.7070304@mrabarnett.plus.com> Alf P. Steinbach wrote: > I'm responding to the original message by Xah Lee, which is not carried > by my Usenet provider. > > * Alan Harris-Reid: >> Xah Lee wrote: >>> Some thoughts about Python 3 Adoption. >>> >>> Xah Lee, 2010-01-26 >>> >>> Some notes of Wikipedia readings related to Python. >>> >>> Unladen Swallow, a new project from Google. It is a new python >>> compiler with the goal of 5 times faster than the de facto standand >>> implementation CPython. Also note Stackless Python, which is already >>> been used in some major commercial projects. >>> >>> Was looking into what's new in Python 3. See: >>> http://docs.python.org/dev/3.0/whatsnew/3.0.html. >>> >From a quick reading, i don't really like it. Here's some highlights: >>> >>> * Print is now a function. Great, much improvement. > > Actually not, IMHO. All it does is is to provide incompatibility. They > forgot Ronald Reagan's old maxim: if it don't need fixin', don't fix it. > It was an inconsistency in the language. BTW, the usual form of the maxim is: if it ain't broke, don't fix it. > >>> * Many functions that return lists now returns ?Views? or >>> ?Iterators? Instead. A fucking fuck all fucked up shit. A extraneous >>> ?oop engineering? complication. (See: Lambda in Python 3000) > > On the contrary, this is a great improvement. It makes no sense to have > to write more code and less readable code for the common efficient case. > And the default, the way that takes 0.5 seconds less to do, does have a > great influence on what people choose, even in elections (Obama's latest > "opt out" proposal is an example that this principle is recognized even > by the President of the United States). When you want an explicit > collection such as a 'list', introducing some overhead for presumably a > good reason, it's as easy as e.g. 'list(range(42))'. > > >>> * The cmp() function used in sort is basically gone, users are now >>> supposed to use the ?key? parameter instead. This is a flying-face- >>> fuck to computer science. This would be the most serious fuckup in >>> python 3. (See: Sorting in Python and Perl) > > I agree. :-) > > Probably there must have been some rationale, but to put it bluntly > removing the comparator is more like moronic than pythonic. If the > rationale was efficiency, then a rational solution could be to provide > two sort methods, like 'sort' using direct comparisions and > 'custom_sort' using a specified comparator. Come to think of it, that > door is still open, except for the language spec freeze. > > It's possible to work around the removal of 'cmp' in various kludgy > ways, but you shouldn't have to "work around" a library sort routine. > Why two sort methods? It would be simpler just to retain the 'cmp' argument, like it is in Python 2.6, but deprecated. > >>> * Integers by default is long. Great! > > Yes, totally agree. > > Nothing confuses the newbie or the now-and-then programmer so much as > limited range integers. > Limited range? That hasn't been the case with 'int' for long time! (It's automatically promoted to long.) [snip] From alfps at start.no Tue Jan 26 21:08:29 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 03:08:29 +0100 Subject: python 3's adoption In-Reply-To: <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> Message-ID: * alex23: > "Alf P. Steinbach" wrote: >> Actually not, IMHO. All it does is is to provide incompatibility. They forgot >> Ronald Reagan's old maxim: if it don't need fixin', don't fix it. > [...] >> Probably there must have been some rationale, but to put it bluntly removing the >> comparator is more like moronic than pythonic. > > So in both cases, you haven't put any effort into researching why > these decisions were made, but you're happy to jump to conclusions > about the intelligence of the implementer(s) while regularly trotting > out complaints of ad hominem attacks against yourself. > > The saddest part of all is you seem to consider yourself some kind of > educator. Hi "alex23". Ad hominem attacks like you try above, just reflect on the one doing it. Adding in some obviously false assertions just makes you look even worse. And posting anonymously, well, that article's got it all. Such attacks constitute noise and, except for how they reflect on you, only noise. Cheers & hth., - Alf From steve at holdenweb.com Tue Jan 26 21:09:53 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 26 Jan 2010 21:09:53 -0500 Subject: python 3's adoption In-Reply-To: <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> Message-ID: [Off-list] alex23 wrote: > "Alf P. Steinbach" wrote: >> Actually not, IMHO. All it does is is to provide incompatibility. They forgot >> Ronald Reagan's old maxim: if it don't need fixin', don't fix it. > [...] >> Probably there must have been some rationale, but to put it bluntly removing the >> comparator is more like moronic than pythonic. > > So in both cases, you haven't put any effort into researching why > these decisions were made, but you're happy to jump to conclusions > about the intelligence of the implementer(s) while regularly trotting > out complaints of ad hominem attacks against yourself. > > The saddest part of all is you seem to consider yourself some kind of > educator. ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From alfps at start.no Tue Jan 26 21:12:37 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 03:12:37 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> Message-ID: * Steve Holden: > [Off-list] > alex23 wrote: >> "Alf P. Steinbach" wrote: >>> Actually not, IMHO. All it does is is to provide incompatibility. They forgot >>> Ronald Reagan's old maxim: if it don't need fixin', don't fix it. >> [...] >>> Probably there must have been some rationale, but to put it bluntly removing the >>> comparator is more like moronic than pythonic. >> So in both cases, you haven't put any effort into researching why >> these decisions were made, but you're happy to jump to conclusions >> about the intelligence of the implementer(s) while regularly trotting >> out complaints of ad hominem attacks against yourself. >> >> The saddest part of all is you seem to consider yourself some kind of >> educator. > > ;-) Please don't post more noise and ad hominem attacks to the group, Steve. - Alf From debatem1 at gmail.com Tue Jan 26 21:43:17 2010 From: debatem1 at gmail.com (geremy condra) Date: Tue, 26 Jan 2010 21:43:17 -0500 Subject: Authenticated encryption with PyCrypto In-Reply-To: <7169bb89-c22b-4826-83bc-b72b77150c30@e25g2000yqh.googlegroups.com> References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> <837d2986-29b4-4cf7-b6a7-347008281d11@p6g2000vbl.googlegroups.com> <7169bb89-c22b-4826-83bc-b72b77150c30@e25g2000yqh.googlegroups.com> Message-ID: On Tue, Jan 26, 2010 at 7:23 PM, Daniel wrote: > I understand the risks of unpickle. With strong, authenticated > encryption I think it is reasonably safe to send an encrypted pickle > through an untrusted medium (the Internet) and know that it has not > been modified enroute. That is, unless someone has obtained the key, > in which case I have a bigger problem to worry about. > Not to sound pedantic, but the fact that somebody is executing exactly the malware they sent you is kind of cold comfort. Might I suggest that a warning to that effect would be prudent? Geremy Condra From john at castleamber.com Tue Jan 26 22:04:53 2010 From: john at castleamber.com (John Bokma) Date: Tue, 26 Jan 2010 21:04:53 -0600 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> Message-ID: <878wbk1c4q.fsf@castleamber.com> "Alf P. Steinbach" writes: > Please don't post more noise and ad hominem attacks to the group, > Steve. Funny that you talk about noise while replying yourself to noise. Xah Lee is just a pathetic spammer. He's not going to reply in this thread. He just shits out his stuff in as many groups as possible to promote his website. -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From pydota at gmail.com Tue Jan 26 22:11:10 2010 From: pydota at gmail.com (moon sky) Date: Wed, 27 Jan 2010 11:11:10 +0800 Subject: python for net manager Message-ID: <6e5636721001261911i6d5ca5fma2dff5b8a89ed277@mail.gmail.com> hi there, i just want to write a small tool to manage the network, including detect the ip addr,netmask, and send the arp request to find out locale's ip-mac turtple, is there any way to succeed this exlude 'subprocess call'? thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From alfps at start.no Tue Jan 26 22:12:07 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 04:12:07 +0100 Subject: python 3's adoption In-Reply-To: <878wbk1c4q.fsf@castleamber.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> <878wbk1c4q.fsf@castleamber.com> Message-ID: * John Bokma: > "Alf P. Steinbach" writes: > >> Please don't post more noise and ad hominem attacks to the group, >> Steve. > > Funny that you talk about noise while replying yourself to noise. Xah > Lee is just a pathetic spammer. He's not going to reply in this > thread. He just shits out his stuff in as many groups as possible to > promote his website. Sorry, I didn't know. Thanks for the heads-up. Cheers, - Alf From john at castleamber.com Tue Jan 26 22:18:04 2010 From: john at castleamber.com (John Bokma) Date: Tue, 26 Jan 2010 21:18:04 -0600 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> <878wbk1c4q.fsf@castleamber.com> Message-ID: <874om81bir.fsf@castleamber.com> "Alf P. Steinbach" writes: > * John Bokma: >> "Alf P. Steinbach" writes: >> >>> Please don't post more noise and ad hominem attacks to the group, >>> Steve. >> >> Funny that you talk about noise while replying yourself to noise. Xah >> Lee is just a pathetic spammer. He's not going to reply in this >> thread. He just shits out his stuff in as many groups as possible to >> promote his website. > > Sorry, I didn't know. > > Thanks for the heads-up. My apologies for the harsh tone of my reply. I am more than tired of how Xah Lee spams weekly, and shouldn't have assumed you were aware of that. -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From tjreedy at udel.edu Tue Jan 26 22:37:45 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 26 Jan 2010 22:37:45 -0500 Subject: python 3's adoption In-Reply-To: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: On 1/26/2010 6:47 PM, Xah Lee wrote: > Some thoughts about Python 3 Adoption. > > Xah Lee, 2010-01-26 > > Some notes of Wikipedia readings related to Python. > > Unladen Swallow, a new project from Google. It is a new python > compiler with the goal of 5 times faster than the de facto standand > implementation CPython. Also note Stackless Python, which is already > been used in some major commercial projects. > > Was looking into what's new in Python 3. See: http://docs.python.org/dev/3.0/whatsnew/3.0.html. > From a quick reading, i don't really like it. Here's some highlights: > > * Print is now a function. Great, much improvement. Something we agree on. For simple scripts, it is a bit of a nuisance, but it eliminates kludgy special-case syntax for options, which are now normal keyword arguments. > * Many functions that return lists now returns ?Views? or > ?Iterators? Instead. A fucking fuck all fucked up shit. A extraneous > ?oop engineering? complication. (See: Lambda in Python 3000) Something we disagree on. I consider the shift from list or iterator orientation a major advance. If you ever want to iterate a 10 million item dict or map or filter a 100 million item sequence, you might then see the point. > * The cmp() function used in sort is basically gone, users are now > supposed to use the ?key? parameter instead. This is a flying-face- > fuck to computer science. This would be the most serious fuckup in > python 3. (See: Sorting in Python and Perl) It is completely gone. Cmp was kludgy. Sort paid no attention to the tri-state return but only paid attention to the less-than branch. Which means a) that it was redundant with the less than method and that the effort to split == from > was completely wasted. It is difficult to find use cases where cmp= is superior to key= > * Integers by default is long. Great! > * Much more integrated unicode support, rewrite of most its text > or string semantics. Fantastic. Finally. What is missing now on Windows is better support from the OS, at least with XP. I have not tried 3.1 on Win7 yet (and never touched Vista). > > Am looking because i wonder if i should switch to python 3 for my own > few scripts, and rewrite my Python Tutorial for version 3. I am writing my book on algorithms using 3.1 because of the above and other improvements (consistent / and elimination of old classes, for instance). > Am also > interested to know how python 3 is received by the computing industry. > Apparantly, a little search on the web indicates that vast majority of > python base have not switched, Some people have panicked prematurely. Guido never expected that working old apps would all be switched. There is probably code still being run with 1.5 from 15 years ago. And there will be 2.7 code running at least that far in the future. Even before 3.0 was out, he also never expected major uptake until 3.2, which is a year away. 3.0 was, well, a .0 release with a couple of .0 problems. 3.1 is pretty solid and modules are appearing for it. > as expected, for many good reasons. > Vast majority of major python modules and tools have not switched. A few have, more will in the next year or after 3.2 is released, some never will. The most important one, in my opinion, is numpy, since some module authors who *want* to do a 3.x version are waiting on that. > Most linux distro have not switched, Have any switched to 2.6 yet, as the default? Some do have 3.1 version available. > i don't find any large > corporation having adopted Python 3 (Google, Yahoo, Facebook, > NASA,... ). (sources: Source, Source) Basically, such a incompatible > change with trivial, ideological improvements, is too costy to switch. The major reduction of redundancy by the overdue removal of obsolete features makes it a much nicer language to learn than, say, 2.6. It is much closer to the simplicity of 1.4, which is what I started with. Terry Jan Reedy From tjreedy at udel.edu Tue Jan 26 22:38:45 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 26 Jan 2010 22:38:45 -0500 Subject: Just drawing lines and plotting points? In-Reply-To: <50697b2c1001261654i7a64adaai3df49eac8a859eae@mail.gmail.com> References: <50697b2c1001261654i7a64adaai3df49eac8a859eae@mail.gmail.com> Message-ID: On 1/26/2010 7:54 PM, Chris Rebert wrote: > On Tue, Jan 26, 2010 at 4:36 PM, Someone Something wrote: >> Hello, >> >> I need a python library that makes drawing lines and plotting points (these >> two things are the only things I need to do) easy. Or, how can I do >> something like this with pygame? Basically, what I want to do is make >> graphs. In pygame, since the coordinate system switches the x's and the y's >> I would have to switch them again when I plot my points so my graphs look >> okay. I hope this was enough info for me to get a good answer. > > matplotlib, perhaps?: http://matplotlib.sourceforge.net/ or dislin? From debatem1 at gmail.com Tue Jan 26 22:55:16 2010 From: debatem1 at gmail.com (geremy condra) Date: Tue, 26 Jan 2010 22:55:16 -0500 Subject: Authenticated encryption with PyCrypto In-Reply-To: <4B5F57E6.9050200@egenix.com> References: <15a63b14-e688-4dfb-8ef8-582b45185c94@a22g2000yqc.googlegroups.com> <4B5F2871.6010102@egenix.com> <4B5F57E6.9050200@egenix.com> Message-ID: On Tue, Jan 26, 2010 at 4:00 PM, M.-A. Lemburg wrote: > > > geremy condra wrote: >> On Tue, Jan 26, 2010 at 12:37 PM, M.-A. Lemburg wrote: >> >> >> >>> You are also using CBC mode, even though you are really after >>> ECB mode (your code doesn't use chaining). With ECB mode, you >>> don't need the IV string. >> >> However, ECB mode is not as secure- the IV is the right way to go >> here. > > Right - I forgot that PyCrypto applies the chaining internally > when being passed data of more than 32 bytes. > >> I'd also note that you aren't supposed to use RandomPool anymore, >> and that AES-192 is frequently recommended over AES-256 for >> new applications due to a number of recent developments in >> the cryptanalysis of its key schedule. > > Do you have pointers for this ? > > I could only find > http://en.wikipedia.org/wiki/Advanced_Encryption_Standard#Security > and > https://cryptolux.org/Block > Sure. The best source is the papers themselves: http://eprint.iacr.org/2009/374.pdf http://eprint.iacr.org/2009/317.pdf http://eprint.iacr.org/2009/242.pdf If you're interested I can provide more. The other thing to keep in mind is that several of the new attacks abuse algebraic properties of the key schedule, and that older attacks like XSL (eXtended Sparse Linearisation) and models (like the BES) are useful tools. There's also been some recent research into using bent functions for cryptanalysis, including cryptanalysis over GF(p), which falls into the lap of older models like the BES, which views AES in terms of GF(2^8), although I don't seem to be able to find free links to those papers. > My reading of their FAQ (https://cryptolux.org/FAQ_on_the_attacks) > is that using AES-128 is the way to go (and it's faster too) - at > least for the time being. In applications where speed is not of the essence, there doesn't appear to be a good reason to prefer AES-128 to AES-192, and while the margin is small, recent attacks have consistently led to results that are 2^10 or 2^12 larger for 192 than for 128. I also doubt highly that the wells opened by these newer results are the end of this question- so my advice would be to use 192 where possible, and keep an eye out just in case. Geremy Condra From rantingrick at gmail.com Tue Jan 26 23:50:08 2010 From: rantingrick at gmail.com (rantingrick) Date: Tue, 26 Jan 2010 20:50:08 -0800 (PST) Subject: Just drawing lines and plotting points? References: <50697b2c1001261654i7a64adaai3df49eac8a859eae@mail.gmail.com> Message-ID: On Jan 26, 9:38?pm, Terry Reedy wrote: > On 1/26/2010 7:54 PM, Chris Rebert wrote: > > > On Tue, Jan 26, 2010 at 4:36 PM, Someone Something ?wrote: > >> Hello, > > >> I need a python library that makes drawing lines and plotting points (these > >> two things are the only things I need to do) easy. Or, how can I do > >> something like this with pygame? Basically, what I want to do is make > >> graphs. In pygame, since the coordinate system switches the x's and the y's > >> I would have to switch them again when I plot my points so my graphs look > >> okay. I hope this was enough info for me to get a good answer. > > > matplotlib, perhaps?:http://matplotlib.sourceforge.net/ > > or dislin? it doesn't get any easier than the tkCanvas widget... import Tkinter as tk #import tkinter as tk #python3.0+ app = tk.Tk() canvas = tk.Canvas(app) canvas.pack() canvas.create_line(sx,sy, ex,ey) #... #... app.mainloop() From alfps at start.no Tue Jan 26 23:52:13 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 05:52:13 +0100 Subject: Just drawing lines and plotting points? In-Reply-To: References: <50697b2c1001261654i7a64adaai3df49eac8a859eae@mail.gmail.com> Message-ID: * rantingrick: > On Jan 26, 9:38 pm, Terry Reedy wrote: >> On 1/26/2010 7:54 PM, Chris Rebert wrote: >> >>> On Tue, Jan 26, 2010 at 4:36 PM, Someone Something wrote: >>>> Hello, >>>> I need a python library that makes drawing lines and plotting points (these >>>> two things are the only things I need to do) easy. Or, how can I do >>>> something like this with pygame? Basically, what I want to do is make >>>> graphs. In pygame, since the coordinate system switches the x's and the y's >>>> I would have to switch them again when I plot my points so my graphs look >>>> okay. I hope this was enough info for me to get a good answer. >>> matplotlib, perhaps?:http://matplotlib.sourceforge.net/ >> or dislin? > > it doesn't get any easier than the tkCanvas widget... > > import Tkinter as tk > #import tkinter as tk #python3.0+ > app = tk.Tk() > canvas = tk.Canvas(app) > canvas.pack() > canvas.create_line(sx,sy, ex,ey) > #... > #... > app.mainloop() He he, it's even easier to draw a graph using the turtle module. Cheers, - Alf From rantingrick at gmail.com Wed Jan 27 00:05:46 2010 From: rantingrick at gmail.com (rantingrick) Date: Tue, 26 Jan 2010 21:05:46 -0800 (PST) Subject: Just drawing lines and plotting points? References: <50697b2c1001261654i7a64adaai3df49eac8a859eae@mail.gmail.com> Message-ID: <67a2a20b-b17d-4677-b8cc-4793adf077f4@m16g2000yqc.googlegroups.com> On Jan 26, 10:52?pm, "Alf P. Steinbach" wrote: > * rantingrick: > > > > > > > On Jan 26, 9:38 pm, Terry Reedy wrote: > >> On 1/26/2010 7:54 PM, Chris Rebert wrote: > > >>> On Tue, Jan 26, 2010 at 4:36 PM, Someone Something ?wrote: > >>>> Hello, > >>>> I need a python library that makes drawing lines and plotting points (these > >>>> two things are the only things I need to do) easy. Or, how can I do > >>>> something like this with pygame? Basically, what I want to do is make > >>>> graphs. In pygame, since the coordinate system switches the x's and the y's > >>>> I would have to switch them again when I plot my points so my graphs look > >>>> okay. I hope this was enough info for me to get a good answer. > >>> matplotlib, perhaps?:http://matplotlib.sourceforge.net/ > >> or dislin? > > > it doesn't get any easier than the tkCanvas widget... > > > import Tkinter as tk > > #import tkinter as tk #python3.0+ > > app = tk.Tk() > > canvas = tk.Canvas(app) > > canvas.pack() > > canvas.create_line(sx,sy, ex,ey) > > #... > > #... > > app.mainloop() > > He he, it's even easier to draw a graph using the turtle module. > > Cheers, > > - Alf Yes Alf please forgive me, i had completely forgotten about the "visual" learners amongst us. turtle, turtle, watch him go... turtle, turtle, why he so slow... turtle, turtle, not even he don't know...? ;-) From whyadd9 at gmail.com Wed Jan 27 00:26:14 2010 From: whyadd9 at gmail.com (Patrick) Date: Tue, 26 Jan 2010 21:26:14 -0800 (PST) Subject: scraping with urllib2 Message-ID: I'm trying to scrape the attached link for the price listed $99.99: http://bananarepublic.gap.com/browse/product.do?cid=41559&vid=1&pid=692392 I can see the price if I view the source(I even turned off java and javascript), but when I use urllib2, the price doesn't show up. Is there another library other than urllib2 that would work? From mallik.v.arjun at gmail.com Wed Jan 27 00:48:10 2010 From: mallik.v.arjun at gmail.com (=?UTF-8?B?TWFsbGlrYXJqdW4o4LKu4LKy4LON4LKy4LK/4LKV4LK+4LKw4LON4LKc4LOB4LKo4LONKQ==?=) Date: Wed, 27 Jan 2010 11:18:10 +0530 Subject: Simple Password Strength Checker Review Help needed Message-ID: <158b9ef31001262148g1926d067ja8ba840ba074eca4@mail.gmail.com> Dear friends, I am newbie to Python + pygtk + Glade and recently wrote a simple password strength checker app. Since this is my first app/program, can someone review my code (just over 150 lines) and help me improve my programming capabilities Thank you Mallikarjun -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PyPassChecker.glade Type: application/x-glade Size: 6363 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: somaSoftware.py Type: text/x-python Size: 5956 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PyPassChecker.xml Type: text/xml Size: 5445 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PyPassChecker.zip Type: application/zip Size: 3844 bytes Desc: not available URL: From mallik.v.arjun at gmail.com Wed Jan 27 00:56:21 2010 From: mallik.v.arjun at gmail.com (=?UTF-8?B?TWFsbGlrYXJqdW4o4LKu4LKy4LON4LKy4LK/4LKV4LK+4LKw4LON4LKc4LOB4LKo4LONKQ==?=) Date: Wed, 27 Jan 2010 11:26:21 +0530 Subject: Simple Password Strength Checker Review Help needed In-Reply-To: <158b9ef31001262148g1926d067ja8ba840ba074eca4@mail.gmail.com> References: <158b9ef31001262148g1926d067ja8ba840ba074eca4@mail.gmail.com> Message-ID: <158b9ef31001262156j18af3367sd05938338696a880@mail.gmail.com> I am really sorry, I uploaded a wrong python file. Here is the correct file 2010/1/27 Mallikarjun > Dear friends, > I am newbie to Python + pygtk + Glade and recently wrote a simple password > strength checker app. > Since this is my first app/program, can someone review my code (just over > 150 lines) and help me improve my programming capabilities > > Thank you > Mallikarjun > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PyPassChecker.py Type: text/x-python Size: 5578 bytes Desc: not available URL: From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 01:12:50 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 27 Jan 2010 06:12:50 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: On Wed, 27 Jan 2010 02:28:00 +0100, Alf P. Steinbach wrote: >>> * Print is now a function. Great, much improvement. > > Actually not, IMHO. All it does is is to provide incompatibility. They > forgot Ronald Reagan's old maxim: if it don't need fixin', don't fix it. print as a function is more consistent and more convenient than print as a statement. Or to put it another way, print as a statement is less consistent and convenient, and therefore in need of fixing. By Ronald Reagan's aphorism, it should be fixed, and now does. Of course there's a bit of a learning curve for those used to the old ways, but in two years from now, I expect no-one will remember what the fuss was about. Python 2.6 users will start their script with "from __future__ import print_function", Python 2.7 users may not even need to do that, and we'll all pity those stuck with 2.5 or 2.4. >>> * Many functions that return lists now returns ?Views? or >>> ?Iterators? Instead. A fucking fuck all fucked up shit. A extraneous >>> ?oop engineering? complication. (See: Lambda in Python 3000) > > On the contrary, this is a great improvement. Agreed. >>> * The cmp() function used in sort is basically gone, users are now >>> supposed to use the ?key? parameter instead. This is a flying-face- >>> fuck to computer science. This would be the most serious fuckup in >>> python 3. (See: Sorting in Python and Perl) > > I agree. :-) I did too, when I first heard cmp was to be dumped. But I changed my mind and now agree with the decision to drop cmp. Custom sorts are nearly always much better written with key rather than cmp: key adds an O(N) overheard to the sorting, while cmp makes sorting O(N**2). In other words, key is *much* faster, and generally easier to write as well. There may be a handful of rare applications where there is no obvious way to convert a cmp function to a key, but they are vanishingly rare. I think, if you search the archives, Paul Rubin may have come up with one example. There are a number of recipes for easily converting cmp sorts to key sorts. I think, in my perfect world, list.sort() and sorted() should continue being key based, while the standard library contained (perhaps in the functools module?) a sort function that contains all the bells and whistles. You want cmp, it's there, and you can pay the extra cost if you need it. You want to sort multiple lists by the contents of one? One import away. But keep the general sort() function lean and super-fast. -- Steven From no.email at nospam.invalid Wed Jan 27 01:23:11 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Tue, 26 Jan 2010 22:23:11 -0800 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: <7xr5pcyskw.fsf@ruckus.brouhaha.com> Steven D'Aprano writes: > print as a function is more consistent and more convenient than print as > a statement. Convenience is subjective, but the 3.x 'print' behavior is definitely inconsistent (i.e. different from 2.x). The change makes a lot of my code silently produce wrong results, too. I often print tuples to show what a program is doing: print (timestamp, 'transmogrified', blob) which in 2.x prints a parenthesized tuple that I can later read back in with eval. That line of code still prints a message, but in a different format, instead of throwing an error. From andreengels at gmail.com Wed Jan 27 01:45:48 2010 From: andreengels at gmail.com (Andre Engels) Date: Wed, 27 Jan 2010 07:45:48 +0100 Subject: scraping with urllib2 In-Reply-To: References: Message-ID: <6faf39c91001262245t7c99120fy1e3c990c079b7f61@mail.gmail.com> On Wed, Jan 27, 2010 at 6:26 AM, Patrick wrote: > I'm trying to scrape the attached link for the price listed $99.99: > http://bananarepublic.gap.com/browse/product.do?cid=41559&vid=1&pid=692392 > > I can see the price if I view the source(I even turned off java and > javascript), but when I use urllib2, the price doesn't show up. > > Is there another library other than urllib2 that would work? To see that page you need to accept cookies from the site and send them back. -- Andr? Engels, andreengels at gmail.com From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 01:57:06 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 27 Jan 2010 06:57:06 GMT Subject: Simple Password Strength Checker Review Help needed References: <158b9ef31001262148g1926d067ja8ba840ba074eca4@mail.gmail.com> Message-ID: On Wed, 27 Jan 2010 11:26:21 +0530, Mallikarjun wrote: I don't know much about pygtk and Glade, but I can make a couple of comments: > class PassChecker: > > def on_check_button_toggled(self, button): > if self.pass_entry.get_visibility(): > self.pass_entry.set_visibility(False) > else: > self.pass_entry.set_visibility(True) This can be written as: self.pass_entry.set_visibility(not self.pass_entry.get_visibility()) Better still, get rid of the "set_visiblity" and "get_visibility" methods. This isn't Java, setters and getters are rarely needed. Just make an attribute "visible" and toggle it: self.pass_entry.visible = not self.pass_entry.visible > def on_guidelines_button_clicked(self, button): > message = "Here are some guidelines to be followed while > choosing password.\n\ [...] It is best to write such long strings as a triple-quoted string, to avoid needing all those backslashes. Instead of: message = "Here are some guidelines... \n\ blah blah blah \n\ blah blah" it is better to write: message = """Here are some guidelines... blah blah blah blah blah""" > def on_pass_entry_changed(self, text_entry): > password = self.pass_entry.get_text() strength = > self.password_strength_checker(password) > > if strength >= 0.0 and strength <= 0.20: > strength_in_str = "Very Week" Python allows you to write multiple comparisons like this: if 0.0 <= strength <= 0.20: strength_in_str = "Very Weak" elif 0.20 < strength <= 0.40: strength_in_str = "Weak" Note also the correct spelling of weak. ("Week" is seven days.) > def password_strength_checker(self, password): > > if not password: > return 0.0 > > # Check for having same Characters > has_same_characters = True > first_letter = password[0] > for i in range(1, password.__len__()): You nearly never need to explicitly call double-underscore methods like __len__. The above is better written as: for i in range(1, len(password)): > if not first_letter == password[i]: > has_same_characters = False > if not has_same_characters: > break > if has_same_characters: > return 0.0 I'm not sure what you're trying to do here. By the names, I think you're looking for any repeated characters, but judging by the code, you're only checking to see if the *first* character is repeated. If you want to check only the first character, this is best written as: # get the first character, and see it if is in the rest of the string first_repeated = password[0] in password[1:] If you want to check for any duplicate: any_duplicate = len(set(password)) != len(password) If that's a bit too mysterious for you, try this: for c in password: # check each character for duplicates if password.count(c) != 1: print "character %s is duplicated" % c I'm sure you can adapt that to do what you want. > lower_alphabets_count = 0 > upper_alphabets_count = 0 > digits_count = 0 > special_chars_count = 0 > for i in range(0, password.__len__()): Again, use len(password). > char = password[i] Since you *only* use i as an index to grab the ith character, this is better written by iterating over the string itself: for char in password: ... > if char.isalpha(): > if char.isupper(): > upper_alphabets_count += 1 > else: > lower_alphabets_count += 1 > elif char.isdigit(): > digits_count += 1 > else: > special_chars_count += 1 if char.isupper(): upper_alphabets_count += 1 elif char.isupper(): lower_alphabets_count += 1 elif char.isdigit(): digits_count += 1 else: special_chars_count += 1 > chars_count = [upper_alphabets_count, lower_alphabets_count, > digits_count, special_chars_count] > > if password.__len__() < 8: len(password) > if not chars_count.__contains__(0): > strength = 0.5 if not 0 in chars_count: strength = 0.5 Hope this helps. -- Steven From no.email at nospam.invalid Wed Jan 27 02:14:31 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Tue, 26 Jan 2010 23:14:31 -0800 Subject: Simple Password Strength Checker Review Help needed References: <158b9ef31001262148g1926d067ja8ba840ba074eca4@mail.gmail.com> Message-ID: <7xfx5sxbmw.fsf@ruckus.brouhaha.com> "Mallikarjun(?????????????)" writes: > Since this is my first app/program, can someone review my code (just > over 150 lines) and help me improve my programming capabilities I didn't look at the code. From a security point of view, the concept of "password strength checking" is pretty dubious. If you want secure passwords, generate them from a random number source and assign them to the users. Don't have the users make up their own passwords. It's relatively (compared to using a computer file exposed to remote internet attacks) for users to write down the the random passwords on paper, as long as they're a little bit careful. As Bruce Schneier put it: "My wallet is already a secure container; it has valuable things in it, and I have a lifetime of experience keeping it safe. Adding a piece of paper with my passwords seems like a natural thing to do." From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 02:26:42 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 27 Jan 2010 07:26:42 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> Message-ID: On Tue, 26 Jan 2010 22:23:11 -0800, Paul Rubin wrote: > Steven D'Aprano writes: >> print as a function is more consistent and more convenient than print >> as a statement. > > Convenience is subjective, but the 3.x 'print' behavior is definitely > inconsistent (i.e. different from 2.x). Sorry, I meant consistent with the rest of Python, which mostly uses functions/methods and only rarely statements (e.g. del and import). > The change makes a lot of my > code silently produce wrong results, too. I often print tuples to show > what a program is doing: > > print (timestamp, 'transmogrified', blob) > > which in 2.x prints a parenthesized tuple that I can later read back in > with eval. That line of code still prints a message, but in a different > format, instead of throwing an error. I don't pretend that the transition between statement and function syntax will be anything but inconvenient, but I believe the end result will be worth it. -- Steven From arnodel at googlemail.com Wed Jan 27 02:34:56 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Wed, 27 Jan 2010 07:34:56 +0000 Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> <7ea02185-0bc1-4f49-9931-0974d8a0db5c@a5g2000yqi.googlegroups.com> Message-ID: Steve Howell writes: > On Jan 25, 1:32?pm, Arnaud Delobelle wrote: >> Steve Howell writes: >> >> [...] >> >> > My algorithm does exactly N pops and roughly N list accesses, so I >> > would be going from N*N + N to N + N log N if switched to blist. >> >> Can you post your algorithm? ?It would be interesting to have a concrete >> use case to base this discussion on. >> > > I just realized you meant the Python code itself. It is here: > > https://bitbucket.org/showell/shpaml_website/src/tip/shpaml.py Hi - I didn't have the time to look at it before today. You scan through a list called prefix_lines, and you use pop(0) as a means to keep track of your position in this list. It seems to me that it would be more effective to explicitely keep track of it - and it would remove the need to the numerous copies of sublists of indent_lines that you have to create. I have rewritten part of the three relevant functions (html_block_tag, get_indented_block and recurse, within indent_lines) to show you what I mean (see below). I have tried to keep the changes to a minimum - you can see that the code is no more complex like this. The advantage is that only one list is created, prefix_lines, and there is no need to mutate it or copy parts of it during the algorithm. I have not had the time to test it though (only on one of the examples on the examples webpage). Code follows: [...] def html_block_tag(output, block, start, end, recurse): append = output.append prefix, tag = block[start] if RAW_HTML.regex.match(tag): append(prefix + tag) recurse(block, start + 1, end) else: start_tag, end_tag = apply_jquery_sugar(tag) append(prefix + start_tag) recurse(block, start + 1, end) append(prefix + end_tag) [...] def get_indented_block(prefix_lines, start, end): prefix, line = prefix_lines[start] len_prefix = len(prefix) i = start + 1 while i < end: new_prefix, line = prefix_lines[i] if line and len(new_prefix) <= len_prefix: break i += 1 while i-1 > start and prefix_lines[i-1][1] == '': i -= 1 return i [...] def indent_lines(lines, output, branch_method, leaf_method, pass_syntax, flush_left_syntax, flush_left_empty_line, indentation_method, get_block, ): append = output.append def recurse(prefix_lines, start, end): while start < end: prefix, line = prefix_lines[start] if line == '': start += 1 append('') else: block_end = get_block(prefix_lines, start, end) if block_end == start + 1: start += 1 if line == pass_syntax: pass elif line.startswith(flush_left_syntax): append(line[len(flush_left_syntax):]) elif line.startswith(flush_left_empty_line): append('') else: append(prefix + leaf_method(line)) else: branch_method(output, prefix_lines, start, block_end, recurse) start = block_end return prefix_lines = map(indentation_method, lines) recurse(prefix_lines, 0, len(prefix_lines)) From no.email at nospam.invalid Wed Jan 27 02:37:00 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Tue, 26 Jan 2010 23:37:00 -0800 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> Message-ID: <7x636oc82r.fsf@ruckus.brouhaha.com> Steven D'Aprano writes: > Sorry, I meant consistent with the rest of Python, which mostly uses > functions/methods and only rarely statements (e.g. del and import). yield, assert, if/else, return, etc. If we're after that kind of consistency, why not get rid of all those statements too? They have already partly done it with yield, and they ended up introducing a new separate if/else expression syntax as an alternative to the statement. > I don't pretend that the transition between statement and function syntax > will be anything but inconvenient, but I believe the end result will be > worth it. This just seems like a gratuitous change with very little benefit. Changing a bunch of list functions to instead return iterators is a much deeper change that will cause larger amounts of breakage, but it has quite a lot of benefits, so there's a stronger case for it. From apt.shansen at gmail.com Wed Jan 27 02:41:32 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Tue, 26 Jan 2010 23:41:32 -0800 Subject: python 3's adoption In-Reply-To: <7xr5pcyskw.fsf@ruckus.brouhaha.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> Message-ID: <7a9c25c21001262341o47742922n801a802ef9708a4e@mail.gmail.com> On Tue, Jan 26, 2010 at 10:23 PM, Paul Rubin wrote: > Steven D'Aprano writes: > > print as a function is more consistent and more convenient than print as > > a statement. > > Convenience is subjective, but the 3.x 'print' behavior is definitely > inconsistent (i.e. different from 2.x). This is a strange definition of "consistent" in my mind. To me, consistent means that it matches the patterns and idioms of the rest of the language at that point in time, not that it implies anything about backwards-compatibility. "print" as a statement was always inconsistent with the rest of the Python language; the rules of how it behaved had to be defined in a unique way (e.g., the behavior of trailing commas, the >> syntax to redirect what you print to). Everything about the statement was inconsistent. Yes, 3.0 changing the print statement to a function made it inconsistent with previous versions, but at the same time it made it MORE consistent with the rest of the language and Python code. If we are defining "consistent" as "backwards-compatible", we have a problem. Python3 is about fixing -language- inconsistencies, and allowing for backwards incompatibility when that meant the language itself was more consistent. > The change makes a lot of my > code silently produce wrong results, too. I often print tuples to show > what a program is doing: > > print (timestamp, 'transmogrified', blob) > > which in 2.x prints a parenthesized tuple that I can later read back in > with eval. That line of code still prints a message, but in a different > format, instead of throwing an error. > That seems like bad code. If one needs to print out what a program is doing, use the logging module. Maybe its just me, but in my experience-- real professional code never uses print unless it was a small script with a very narrowly focused purpose. Changing that focus to use Python3's print function vs a statement isn't really all that hard. And the language is more consistent. --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From timr at probo.com Wed Jan 27 02:47:25 2010 From: timr at probo.com (Tim Roberts) Date: Tue, 26 Jan 2010 23:47:25 -0800 Subject: Sikuli: the coolest Python project I have yet seen... References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> <89c2550d-057d-4cc2-a855-d4f3f11cb272@b10g2000vbh.googlegroups.com> Message-ID: CM wrote: > >How is this preferable to a macro recorder? Macro recorders work by measuring mouse motion and capturing click locations, or by recording the control IDs of the clicked windows. The former is sensitive to changing window locations, the latter to application updates. Sikuli works by using image analysis to locate the regions on the screen to be tickled. It's a novel idea, although others have correctly pointed out that it's not the most efficient way to automate applications. -- Tim Roberts, timr at probo.com Providenza & Boekelheide, Inc. From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 02:47:47 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 27 Jan 2010 07:47:47 GMT Subject: Simple Password Strength Checker Review Help needed References: <158b9ef31001262148g1926d067ja8ba840ba074eca4@mail.gmail.com> <7xfx5sxbmw.fsf@ruckus.brouhaha.com> Message-ID: On Tue, 26 Jan 2010 23:14:31 -0800, Paul Rubin wrote: > "Mallikarjun(?????????????)" writes: >> Since this is my first app/program, can someone review my code (just >> over 150 lines) and help me improve my programming capabilities > > I didn't look at the code. From a security point of view, the concept > of "password strength checking" is pretty dubious. If you want secure > passwords, generate them from a random number source But why do they need to be random, if password strength is dubious? *wink* > and assign them to > the users. Don't have the users make up their own passwords. It's > relatively (compared to using a computer file exposed to remote internet > attacks) for users to write down the the random passwords on paper, as > long as they're a little bit careful. I think you're missing a word there. Relatively secure perhaps? The problem is that most users will not be a little bit careful. They will stick the password on a Post-it note on the side of the monitor, or write it down and lose it, or leave the paper sitting on their desk while they go to lunch. > As Bruce Schneier put it: > > "My wallet is already a secure container; it has valuable things in > it, and I have a lifetime of experience keeping it safe. Adding a > piece of paper with my passwords seems like a natural thing to do." And people frequently lose their wallets. Besides, with the number of on-line identities and passwords many people need, you'll need a separate wallet just for the passwords. I have something of the order of 80 or 90 passwords written down, and another dozen in my head. Because there are so many, I need to keep account information with them: there's no way I'd be able to remember what the passwords were for otherwise. That's way too much to put in my wallet, and even if I did, if I lost it, I'd lose *everything*. Not only would some stranger likely have access to all my accounts, but I wouldn't even be able to identify what those accounts were, let alone authenticate into them. So I need at least one (and likely more) password I can keep in my head, so I can encrypt my list of rarely-used passwords. Because it needs to be something I can remember, it can't be a random string of digits, but it needs to *look* random. In other words, it needs to be a good password that is meaningful to me, but not to anyone else, and since I'm really bad at judging randomness (like nearly all humans), I'll take all the help I can get. -- Steven From no.email at nospam.invalid Wed Jan 27 03:03:36 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 27 Jan 2010 00:03:36 -0800 Subject: Simple Password Strength Checker Review Help needed References: <158b9ef31001262148g1926d067ja8ba840ba074eca4@mail.gmail.com> <7xfx5sxbmw.fsf@ruckus.brouhaha.com> Message-ID: <7xd40wrn3b.fsf@ruckus.brouhaha.com> Steven D'Aprano writes: > I think you're missing a word there. Relatively secure perhaps? Yes, something like that, oops. > The problem is that most users will not be a little bit careful. They > will stick the password on a Post-it note on the side of the monitor,... Right, that's what I mean about careful. But, people do generally develop skills at keeping important bits of paper (like their drivers' license) safe. > Besides, with the number of on-line identities and passwords many people > need, you'll need a separate wallet just for the passwords. I have > something of the order of 80 or 90 passwords written down.. I don't have anywhere that many important ones. For the less important ones I just use my browser's auto-fill feature (i.e. the passwords are stored in the browser). I generate important ones with a program (2-6 words selected at random from a dictionary). When I generate a new one, I write it on a slip of paper that I keep in my pocket and refer to as necessary. After referring to the paper a few times I usually remember the password and don't need the paper any more, so I dispose of it. > So I need at least one (and likely more) password I can keep in my > head, so I can encrypt my list of rarely-used passwords. If there's just one, and it's a phrase, you can remember it. > since I'm really bad at judging randomness (like nearly all humans), Right, humans are no good at generating or judging randomness. It's best to use an entropy source. www.diceware.com explains how to do it with dice and a wordlist they supply. I use a program something like the following instead: from math import log dictfile = '/usr/share/dict/words' def genrandom(nbytes): with open('/dev/urandom') as f: return int(f.read(nbytes).encode('hex'), 16) def main(): wordlist = list(x.strip() for x in open(dictfile) if len(x) < 7) nwords = len(wordlist) print "%d words, entropy=%.3f bits/word"% ( nwords, log(nwords, 2)) print '-'.join(wordlist[genrandom(10)%nwords] for i in xrange(5)) main() You can also use the diceware word list instead of the unix wordlist, of course. From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 03:20:57 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 27 Jan 2010 08:20:57 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> Message-ID: On Tue, 26 Jan 2010 23:37:00 -0800, Paul Rubin wrote: > Steven D'Aprano writes: >> Sorry, I meant consistent with the rest of Python, which mostly uses >> functions/methods and only rarely statements (e.g. del and import). > > yield, assert, if/else, return, etc. If we're after that kind of > consistency, why not get rid of all those statements too? They have > already partly done it with yield, and they ended up introducing a new > separate if/else expression syntax as an alternative to the statement. Without becoming a purely functional language, you won't get rid of all statements. In my opinion, outside of such purely functional languages and unconventional languages like Forth, statements play a valuable role in that they are controlling syntax. For example: for, while, if, try, break, yield, return are all used for flow control, and should remain as statements. But print is not -- it's a function disguised as a statement. print, after all, is just sugar for stdout.write, which is fine as far as it goes, but when you want to do something slightly different from what the sugar does, it becomes terribly inconvenient. A consequence of print being a statement is that my modules are often filled with functions and methods called "print_". Suppose I want to print the type of each argument as well as it's value. I would like to do this: def decorator(pr): def inner(*args, **kwargs): args = [(type(a), a) for a in args] pr(*args, **kwargs) return inner print = decorator(print) and then use print as normal, but of course I can't. So instead I do something like this: def print_(*args, **kwargs): args = [(type(a), a) for a in args] kw = {'file': sys.stdout, 'sep': ' ', 'end': '\n'} kw.update(kwargs) if len(kw) != 3: raise TypeError('invalid keyword argument') file = kw['file'] sep = kw['sep'] end = kw['end'] for arg in args: print >>file, (str(arg) + sep), # Don't forget the comma. print >>file, end, # Another comma. And then, I have to remember to call print_ instead of print. -- Steven From no.email at nospam.invalid Wed Jan 27 03:36:52 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 27 Jan 2010 00:36:52 -0800 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> Message-ID: <7x636ovt97.fsf@ruckus.brouhaha.com> Steven D'Aprano writes: > Without becoming a purely functional language, you won't get rid of all > statements. Why not? GCC lets you use any statement in an expression: #include main() { int i, x, p=0; x = ({ for (i=1; i<=10; i++) p += i; p;}); printf ("x=%d\n", x); } and C is certainly not a purely functional language. > for, while, if, try, break, yield, return > are all used for flow control, and should remain as statements. What about assert, import, and pass? From wuwei23 at gmail.com Wed Jan 27 03:45:22 2010 From: wuwei23 at gmail.com (alex23) Date: Wed, 27 Jan 2010 00:45:22 -0800 (PST) Subject: Sikuli: the coolest Python project I have yet seen... References: <4c30c5a3-d11a-4fb5-b95d-827a6162a6f6@p24g2000yqm.googlegroups.com> <89c2550d-057d-4cc2-a855-d4f3f11cb272@b10g2000vbh.googlegroups.com> Message-ID: <9546c710-0a24-4dc2-aa07-e92d4bb72eea@m16g2000yqc.googlegroups.com> Tim Roberts wrote: > it's not the most efficient way to automate applications Sikuli doesn't seem that much different from Python in this way: it may not be the most efficient use of the computer's time, but I dare say it's significantly less demanding on the end user's. I can see Sikuli easily progressing to a full visual programming interface, replacing the 'click' keyword et al with iconic representations. Simple "script-less" GUI macro-ing for the masses? Fantastic. From fetchinson at googlemail.com Wed Jan 27 04:35:00 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Wed, 27 Jan 2010 10:35:00 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: >>> * Print is now a function. Great, much improvement. > > Actually not, IMHO. All it does is is to provide incompatibility. What incompatibility are you exactly talking about? Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> print( 'hello' ) hello >>> print 'hello' hello >>> Yes, this is with python 2.6.2 which is in the 2.x line of releases. So? Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From mmm286 at gmail.com Wed Jan 27 04:43:05 2010 From: mmm286 at gmail.com (mierdatutis mi) Date: Wed, 27 Jan 2010 10:43:05 +0100 Subject: Help parsing a page with python Message-ID: Hi, I would like to parse a webpage to can get the url of the video download. I use pyhton and firebug but I cant get the url link. Example: The url where I have to get the video link is: http://www.rtve.es/mediateca/videos/20100125/saber-comer---salsa-verde-judiones-25-01-10/676590.shtml " The video is http://www.rtve.es/resources/TE_SSAC011/flv/8/2/1264426362028.flv Could you help me please? Many thanks and sorry for my english! -------------- next part -------------- An HTML attachment was scrubbed... URL: From simon at brunningonline.net Wed Jan 27 05:13:32 2010 From: simon at brunningonline.net (Simon Brunning) Date: Wed, 27 Jan 2010 10:13:32 +0000 Subject: Help parsing a page with python In-Reply-To: References: Message-ID: <8c7f10c61001270213p10ad7d12ra39c75bea1f5e0f4@mail.gmail.com> 2010/1/27 mierdatutis mi : > Hi, > > I would like to parse a webpage to can get the url of the video download. I > use pyhton and firebug but I cant get the url link. > > Example: > > The url where I have to get the video link is: > http://www.rtve.es/mediateca/videos/20100125/saber-comer---salsa-verde-judiones-25-01-10/676590.shtml" > > The video is > http://www.rtve.es/resources/TE_SSAC011/flv/8/2/1264426362028.flv > Could you help me please? That URL doesn't appear to be in the HTML - it must be being brought in by the JavaScript somehow. -- Cheers, Simon B. From steve at holdenweb.com Wed Jan 27 05:19:35 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 27 Jan 2010 05:19:35 -0500 Subject: python 3's adoption In-Reply-To: <7x636ovt97.fsf@ruckus.brouhaha.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> Message-ID: Paul Rubin wrote: > Steven D'Aprano writes: >> Without becoming a purely functional language, you won't get rid of all >> statements. > > Why not? GCC lets you use any statement in an expression: > > #include > > main() > { > int i, x, p=0; > x = ({ for (i=1; i<=10; i++) p += i; p;}); > printf ("x=%d\n", x); > } > > and C is certainly not a purely functional language. > >> for, while, if, try, break, yield, return >> are all used for flow control, and should remain as statements. > > What about assert, import, and pass? Remember that Guido stated (I think in the original FAQ entry for why assignments don't produce results, but I could be wrong) that he explicitly wanted Python to be a statement-based language. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From floris.bruynooghe at gmail.com Wed Jan 27 05:20:53 2010 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Wed, 27 Jan 2010 02:20:53 -0800 (PST) Subject: Ad hoc lists vs ad hoc tuples Message-ID: <8a57c64f-0c67-4ec1-b4be-34d76c7e7571@q4g2000yqm.googlegroups.com> One thing I ofter wonder is which is better when you just need a throwaway sequence: a list or a tuple? E.g.: if foo in ['some', 'random', 'strings']: ... if [bool1, bool2, boo3].count(True) != 1: ... (The last one only works with tuples since python 2.6) Is a list or tuple better or more efficient in these situations? Regards Floris PS: This is inspired by some of the space-efficiency comments from the list.pop(0) discussion. From fetchinson at googlemail.com Wed Jan 27 05:32:08 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Wed, 27 Jan 2010 11:32:08 +0100 Subject: myths about python 3 Message-ID: Hi folks, I was going to write this post for a while because all sorts of myths periodically come up on this list about python 3. I don't think the posters mean to spread false information on purpose, they simply are not aware of the facts. My list is surely incomplete, please feel free to post your favorite misconception about python 3 that people periodically state, claim or ask about. 1. Print statement/function creates incompatibility between 2.x and 3.x! Certainly false or misleading, if one uses 2.6 and 3.x the incompatibility is not there. Print as a function works in 2.6: Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> print( 'hello' ) hello >>> print 'hello' hello >>> 2. Integer division creates incompatibility between 2.x and 3.x! Again false or misleading, because one can get the 3.x behavior with 2.6: Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 6/5 1 >>> from __future__ import division >>> 6/5 1.2 Please feel free to post your favorite false or misleading claim about python 3! Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From stefan_ml at behnel.de Wed Jan 27 05:40:27 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 27 Jan 2010 11:40:27 +0100 Subject: myths about python 3 In-Reply-To: References: Message-ID: <4b60181c$0$6732$9b4e6d93@newsspool2.arcor-online.net> Daniel Fetchinson, 27.01.2010 11:32: > 1. Print statement/function creates incompatibility between 2.x and 3.x! > > Certainly false or misleading, if one uses 2.6 and 3.x the > incompatibility is not there. Print as a function works in 2.6: > > Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) > [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> print( 'hello' ) > hello > >>> print 'hello' > hello This is actually misleading by itself, as the first statement is not a function call in Py2: Python 2.6.4 (r264:75706, Dec 7 2009, 18:43:55) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> print(1,2) (1, 2) It can, however, be made a function call through a future import in 2.6: >>> from __future__ import print_function >>> print(1,2) 1 2 Stefan From andreengels at gmail.com Wed Jan 27 05:45:48 2010 From: andreengels at gmail.com (Andre Engels) Date: Wed, 27 Jan 2010 11:45:48 +0100 Subject: myths about python 3 In-Reply-To: References: Message-ID: <6faf39c91001270245q155a932aj3e0bb98d377ddad1@mail.gmail.com> On Wed, Jan 27, 2010 at 11:32 AM, Daniel Fetchinson wrote: > Hi folks, > > I was going to write this post for a while because all sorts of myths > periodically come up on this list about python 3. I don't think the > posters mean to spread false information on purpose, they simply are > not aware of the facts. > > My list is surely incomplete, please feel free to post your favorite > misconception about python 3 that people periodically state, claim or > ask about. > > 1. Print statement/function creates incompatibility between 2.x and 3.x! > > Certainly false or misleading, if one uses 2.6 and 3.x the > incompatibility is not there. Print as a function works in 2.6: > > Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) > [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> print( 'hello' ) > hello >>>> print 'hello' > hello >>>> > > > 2. Integer division creates incompatibility between 2.x and 3.x! > > Again false or misleading, because one can get the 3.x behavior with 2.6: > > Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) > [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> 6/5 > 1 >>>> from __future__ import division >>>> 6/5 > 1.2 > > > Please feel free to post your favorite false or misleading claim about python 3! Well, I see two false or misleading claims just above - namely that the two claims above are false or misleading. They tell just half of the story, and that half is indeed easy. A Python 3 program can be unchanged (in the case of print) or with only trivial modifications (in the case of integer division) be made to run on Python 2.6. The other way around this is _not_ the case. To say that two things are compatible if one can be used for the other, but the other not for the first, is false or misleading. -- Andr? Engels, andreengels at gmail.com From babedoudi at yahoo.fr Wed Jan 27 06:23:34 2010 From: babedoudi at yahoo.fr (=?ISO-8859-1?Q?R=E9mi?=) Date: Wed, 27 Jan 2010 03:23:34 -0800 (PST) Subject: Terminal application with non-standard print References: Message-ID: On 25 jan, 23:30, Sean DiZazzo wrote: > On Jan 24, 11:27?am, R?mi wrote: > > > > > Hello everyone, > > > I would like to do a Python application that prints data to stdout, but > > not the common way. I do not want the lines to be printed after each > > other, but the old lines to be replaced with the new ones, like wget > > does it for example (when downloading a file you can see the percentage > > increasing on a same line). > > > I looked into the curses module, but this seems adapted only to do a > > whole application, and the terminal history is not visible anymore when > > the application starts. > > > Any ideas? > > > Thanks, > > > Remi > > You might want to take a look at the readline module. > > ~Sean Thanks everyone for your answers, that helped a lot. From mmm286 at gmail.com Wed Jan 27 06:33:17 2010 From: mmm286 at gmail.com (mierdatutis mi) Date: Wed, 27 Jan 2010 12:33:17 +0100 Subject: Help parsing a page with python In-Reply-To: <8c7f10c61001270213p10ad7d12ra39c75bea1f5e0f4@mail.gmail.com> References: <8c7f10c61001270213p10ad7d12ra39c75bea1f5e0f4@mail.gmail.com> Message-ID: Those videos are generated by javascript. There is some parser with python for javascript??? Thanks a lot! 2010/1/27 Simon Brunning > 2010/1/27 mierdatutis mi : > > Hi, > > > > I would like to parse a webpage to can get the url of the video download. > I > > use pyhton and firebug but I cant get the url link. > > > > Example: > > > > The url where I have to get the video link is: > > > http://www.rtve.es/mediateca/videos/20100125/saber-comer---salsa-verde-judiones-25-01-10/676590.shtml > " > > > > The video is > > http://www.rtve.es/resources/TE_SSAC011/flv/8/2/1264426362028.flv > > Could you help me please? > > That URL doesn't appear to be in the HTML - it must be being brought > in by the JavaScript somehow. > > -- > Cheers, > Simon B. > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From xahlee at gmail.com Wed Jan 27 06:40:38 2010 From: xahlee at gmail.com (Xah Lee) Date: Wed, 27 Jan 2010 03:40:38 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> <878wbk1c4q.fsf@castleamber.com> Message-ID: <0c142d80-5f54-4b63-a8ce-f826ec9ad82d@r19g2000yqb.googlegroups.com> Someone is badmouthing me, and it has been doing so over the years. I feel obliged to post a off topic relpy. See: ? DreamHost.com and A Incidence of Harassment http://xahlee.org/Periodic_dosage_dir/t2/harassment.html ? Why Can't You Be Normal? http://xahlee.org/Netiquette_dir/why_cant_you_be_normal.html Xah On Jan 26, 7:12 pm, "Alf P. Steinbach" wrote: > * John Bokma: > > > "Alf P. Steinbach" writes: > > >> Please don't post more noise and ad hominem attacks to the group, > >> Steve. > > > Funny that you talk about noise while replying yourself to noise. Xah > > Lee is just a pathetic spammer. He's not going to reply in this > > thread. He just shits out his stuff in as many groups as possible to > > promote his website. > > Sorry, I didn't know. > > Thanks for the heads-up. > > Cheers, > > - Alf From simon at brunningonline.net Wed Jan 27 06:51:05 2010 From: simon at brunningonline.net (Simon Brunning) Date: Wed, 27 Jan 2010 11:51:05 +0000 Subject: Help parsing a page with python In-Reply-To: References: <8c7f10c61001270213p10ad7d12ra39c75bea1f5e0f4@mail.gmail.com> Message-ID: <8c7f10c61001270351q5ff78b6bv30a2aa03c5cd98a3@mail.gmail.com> 2010/1/27 mierdatutis mi : > Those videos are generated by javascript. > There is some parser with python for javascript??? There is , but simulating the whole context of a browser is going to be a horror. You are probably far better off automating a real browser. WebDriver () has Python bindings these days. It's primarily intended for functional testing, but it might be a good fit here too. -- Cheers, Simon B. From javier.collado at gmail.com Wed Jan 27 06:58:34 2010 From: javier.collado at gmail.com (Javier Collado) Date: Wed, 27 Jan 2010 12:58:34 +0100 Subject: scraping with urllib2 In-Reply-To: <6faf39c91001262245t7c99120fy1e3c990c079b7f61@mail.gmail.com> References: <6faf39c91001262245t7c99120fy1e3c990c079b7f61@mail.gmail.com> Message-ID: Hello, To accept cookies, use the HTTPCookieProcessor as explained here: http://www.nomadjourney.com/2009/03/automatic-site-login-using-python-urllib2/ Best regards, Javier 2010/1/27 Andre Engels : > On Wed, Jan 27, 2010 at 6:26 AM, Patrick wrote: >> I'm trying to scrape the attached link for the price listed $99.99: >> http://bananarepublic.gap.com/browse/product.do?cid=41559&vid=1&pid=692392 >> >> I can see the price if I view the source(I even turned off java and >> javascript), but when I use urllib2, the price doesn't show up. >> >> Is there another library other than urllib2 that would work? > > To see that page you need to accept cookies from the site and send them back. > > > > -- > Andr? Engels, andreengels at gmail.com > -- > http://mail.python.org/mailman/listinfo/python-list > From javier.collado at gmail.com Wed Jan 27 07:00:53 2010 From: javier.collado at gmail.com (Javier Collado) Date: Wed, 27 Jan 2010 13:00:53 +0100 Subject: Help parsing a page with python In-Reply-To: References: <8c7f10c61001270213p10ad7d12ra39c75bea1f5e0f4@mail.gmail.com> Message-ID: Hello, A test case for Windmill might also be used to extract the information that you're looking for. Best regards, Javier 2010/1/27 mierdatutis mi : > Those videos are generated by javascript. > There is some parser with python for javascript??? > > Thanks a lot! > > > 2010/1/27 Simon Brunning >> >> 2010/1/27 mierdatutis mi : >> > Hi, >> > >> > I would like to parse a webpage to can get the url of the video >> > download. I >> > use pyhton and firebug but I cant get the url link. >> > >> > Example: >> > >> > The url where I have to get the video link is: >> > >> > http://www.rtve.es/mediateca/videos/20100125/saber-comer---salsa-verde-judiones-25-01-10/676590.shtml" >> > >> > The video is >> > http://www.rtve.es/resources/TE_SSAC011/flv/8/2/1264426362028.flv >> > Could you help me please? >> >> That URL doesn't appear to be in the HTML - it must be being brought >> in by the JavaScript somehow. >> >> -- >> Cheers, >> Simon B. >> -- >> http://mail.python.org/mailman/listinfo/python-list > > > -- > http://mail.python.org/mailman/listinfo/python-list > > From mmm286 at gmail.com Wed Jan 27 07:08:14 2010 From: mmm286 at gmail.com (mierdatutis mi) Date: Wed, 27 Jan 2010 13:08:14 +0100 Subject: Help parsing a page with python In-Reply-To: References: <8c7f10c61001270213p10ad7d12ra39c75bea1f5e0f4@mail.gmail.com> Message-ID: Hello again, What test case for Windmill? Can you say me the link, please? Many thanks 2010/1/27 Javier Collado > Hello, > > A test case for Windmill might also be used to extract the information > that you're looking for. > > Best regards, > Javier > > 2010/1/27 mierdatutis mi : > > Those videos are generated by javascript. > > There is some parser with python for javascript??? > > > > Thanks a lot! > > > > > > 2010/1/27 Simon Brunning > >> > >> 2010/1/27 mierdatutis mi : > >> > Hi, > >> > > >> > I would like to parse a webpage to can get the url of the video > >> > download. I > >> > use pyhton and firebug but I cant get the url link. > >> > > >> > Example: > >> > > >> > The url where I have to get the video link is: > >> > > >> > > http://www.rtve.es/mediateca/videos/20100125/saber-comer---salsa-verde-judiones-25-01-10/676590.shtml > " > >> > > >> > The video is > >> > http://www.rtve.es/resources/TE_SSAC011/flv/8/2/1264426362028.flv > >> > Could you help me please? > >> > >> That URL doesn't appear to be in the HTML - it must be being brought > >> in by the JavaScript somehow. > >> > >> -- > >> Cheers, > >> Simon B. > >> -- > >> http://mail.python.org/mailman/listinfo/python-list > > > > > > -- > > http://mail.python.org/mailman/listinfo/python-list > > > > > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From simon at brunningonline.net Wed Jan 27 07:14:23 2010 From: simon at brunningonline.net (Simon Brunning) Date: Wed, 27 Jan 2010 12:14:23 +0000 Subject: Help parsing a page with python In-Reply-To: References: <8c7f10c61001270213p10ad7d12ra39c75bea1f5e0f4@mail.gmail.com> Message-ID: <8c7f10c61001270414x37c7c1dci93a8a5667c923e20@mail.gmail.com> 2010/1/27 mierdatutis mi : > Hello again, > > What test case for Windmill? Can you say me the link, please? http://lmgtfy.com/?q=windmill+test -- Cheers, Simon B. From iainking at gmail.com Wed Jan 27 07:22:30 2010 From: iainking at gmail.com (Iain King) Date: Wed, 27 Jan 2010 04:22:30 -0800 (PST) Subject: Ad hoc lists vs ad hoc tuples References: <8a57c64f-0c67-4ec1-b4be-34d76c7e7571@q4g2000yqm.googlegroups.com> Message-ID: On Jan 27, 10:20?am, Floris Bruynooghe wrote: > One thing I ofter wonder is which is better when you just need a > throwaway sequence: a list or a tuple? ?E.g.: > > if foo in ['some', 'random', 'strings']: > ? ? ... > if [bool1, bool2, boo3].count(True) != 1: > ? ?... > > (The last one only works with tuples since python 2.6) > > Is a list or tuple better or more efficient in these situations? > > Regards > Floris > > PS: This is inspired by some of the space-efficiency comments from the > list.pop(0) discussion. I tend to use tuples unless using a list makes it easier to read. For example: if foo in ('some', 'random', 'strings'): draw.text((10,30), "WHICH IS WHITE", font=font) draw.line([(70,25), (85,25), (105,45)]) I've no idea what the performance difference is; I've always assumed it's negligible. Iain From xahlee at gmail.com Wed Jan 27 07:26:00 2010 From: xahlee at gmail.com (Xah Lee) Date: Wed, 27 Jan 2010 04:26:00 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> On Jan 26, 3:47 pm, Xah Lee wrote: > * Many functions that return lists now returns ?Views? or > ?Iterators? Instead. A fucking fuck all fucked up shit. A extraneous > ?oop engineering? complication. (See: Lambda in Python 3000) See also: ?Iterators: Signs of Weakness in Object-Oriented Languages? (1992) By Henry G Baker. http://home.pipeline.com/~hbaker1/Iterator.html Xah wrote: > * The cmp() function used in sort is basically gone, users are now > supposed to use the ?key? parameter instead. This is a flying-face- > fuck to computer science. This would be the most serious fuckup in > python 3. (See: Sorting in Python and Perl) Steven D'Aprano wrote: > I did too, when I first heard cmp was to be dumped. But I changed my mind > and now agree with the decision to drop cmp. Custom sorts are nearly > always much better written with key rather than cmp: key adds an O(N) > overheard to the sorting, while cmp makes sorting O(N**2). In other > words, key is *much* faster, and generally easier to write as well. The reason cmp is a order slower is due to Python lang's limitation and compiler implementation limitation. ? Sorting in Python and Perl http://xahlee.org/perl-python/sort_list.html the python lang, by the very nature of being a dynamic lang, makes it hard for compiler analize and optimize a sort function call with cmp argument that is equivalent to a key. So, for practical reasons, i think a ?key? parameter is fine. But chopping off ?cmp? is damaging. When your data structure is complex, its order is not embedded in some ?key?. Taking out ?cmp? makes it impossible to sort your data structure. in the essay, i also gave a few examples of data structures where you need a general means to specify the ordering function in order to sort. Without a cmp, you'll need to munge your data by some decorate- sort-dedecorate technique. >From another perspective, often it is considered a good principle of design that the lang be a bit flexible to let programers do what they want, instead of force them into one way. Python, being a loosely typed lang, with so-called ?duck typing?, in contrast to Java or OCaml in different ways, certainly is more on the lose extreme of the lang spectrum with respect to code construction. So, i don't see how python 3 people made the decision to removed cmp. (am pretty sure, at least part of the reason, is a ill attitude towards functional programing and lambda, lead by Guido.) > There may be a handful of rare applications where there is no obvious way > to convert a cmp function to a key, but they are vanishingly rare. I > think, if you search the archives, Paul Rubin may have come up with one > example. There are a number of recipes for easily converting cmp sorts to > key sorts. You say that it is rare. You are probably right. Though, that may just be caused by the language itself and consequently the type of people who uses it. If Python's lambda is not limited, and the python community doesn't look down on lambda, it is likely cmp will used more. The more your data structure becomes complex, the more cmp will be needed. > I think, in my perfect world, list.sort() and sorted() should continue > being key based, while the standard library contained (perhaps in the > functools module?) a sort function that contains all the bells and > whistles. You want cmp, it's there, and you can pay the extra cost if you > need it. You want to sort multiple lists by the contents of one? One > import away. But keep the general sort() function lean and super-fast. btw, is something like cmp still available in some module for sort? Xah ? http://xahlee.org/ ? From wissme at hotmail.com Wed Jan 27 08:01:40 2010 From: wissme at hotmail.com (Jean Guillaume Pyraksos) Date: Wed, 27 Jan 2010 14:01:40 +0100 Subject: Python and Ruby Message-ID: What are the arguments for choosing Python against Ruby for introductory programming ? Python has no provisions for tail recursion, Ruby is going to... So what ? Thanks, JG From stefan_ml at behnel.de Wed Jan 27 08:04:41 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 27 Jan 2010 14:04:41 +0100 Subject: Python and Ruby In-Reply-To: References: Message-ID: <4b6039ea$0$6725$9b4e6d93@newsspool2.arcor-online.net> Jean Guillaume Pyraksos, 27.01.2010 14:01: > What are the arguments for choosing Python against Ruby > for introductory programming ? PEP 20: http://www.python.org/dev/peps/pep-0020/ Stefan From richard.lamboj at bilcom.at Wed Jan 27 08:06:23 2010 From: richard.lamboj at bilcom.at (Richard Lamboj) Date: Wed, 27 Jan 2010 14:06:23 +0100 Subject: starting a thread in a nother thread Message-ID: <201001271406.23584.richard.lamboj@bilcom.at> hello, just for _curiosity_. What would be if i start a thread in a nother thread and acquire a lock in the "child" thread. Is there anything that could go wrong if someone try to start threads in threads? Kind Regards, Richi From stefan_ml at behnel.de Wed Jan 27 08:10:13 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 27 Jan 2010 14:10:13 +0100 Subject: starting a thread in a nother thread In-Reply-To: References: Message-ID: <4b603b35$0$6725$9b4e6d93@newsspool2.arcor-online.net> Richard Lamboj, 27.01.2010 14:06: > just for _curiosity_. What would be if i start a thread in a nother thread and > acquire a lock in the "child" thread. Is there anything that could go wrong > if someone try to start threads in threads? There's usually tons of things that can go wrong w.r.t. threads: http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf However, there's nothing special to a thread that was started from another thread, so the problems don't change. Stefan From simon at brunningonline.net Wed Jan 27 08:47:25 2010 From: simon at brunningonline.net (Simon Brunning) Date: Wed, 27 Jan 2010 13:47:25 +0000 Subject: Python and Ruby In-Reply-To: References: Message-ID: <8c7f10c61001270547t10c15144wbc372407621b738f@mail.gmail.com> 2010/1/27 Jean Guillaume Pyraksos : > What are the arguments for choosing Python against Ruby > for introductory programming ? Frankly, either would be a good choice. I think Python is a little cleaner, but I'm sure you'd find Ruby fans who'd argue the complete opposite. Both have good ecosystems, (i.e. good communities, and plenty of good libraries and frameworks) - but Python is probably a bit ahead here having been around a bit longer. > Python has no provisions > for tail recursion, Ruby is going to... So what ? This would be a very strange reason to pick one language over the other - it's a very minor point. -- Cheers, Simon B. From alfps at start.no Wed Jan 27 09:05:40 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 15:05:40 +0100 Subject: Just drawing lines and plotting points? In-Reply-To: <67a2a20b-b17d-4677-b8cc-4793adf077f4@m16g2000yqc.googlegroups.com> References: <50697b2c1001261654i7a64adaai3df49eac8a859eae@mail.gmail.com> <67a2a20b-b17d-4677-b8cc-4793adf077f4@m16g2000yqc.googlegroups.com> Message-ID: * rantingrick: > On Jan 26, 10:52 pm, "Alf P. Steinbach" wrote: >> * rantingrick: >> >> >> >> >> >>> On Jan 26, 9:38 pm, Terry Reedy wrote: >>>> On 1/26/2010 7:54 PM, Chris Rebert wrote: >>>>> On Tue, Jan 26, 2010 at 4:36 PM, Someone Something wrote: >>>>>> Hello, >>>>>> I need a python library that makes drawing lines and plotting points (these >>>>>> two things are the only things I need to do) easy. Or, how can I do >>>>>> something like this with pygame? Basically, what I want to do is make >>>>>> graphs. In pygame, since the coordinate system switches the x's and the y's >>>>>> I would have to switch them again when I plot my points so my graphs look >>>>>> okay. I hope this was enough info for me to get a good answer. >>>>> matplotlib, perhaps?:http://matplotlib.sourceforge.net/ >>>> or dislin? >>> it doesn't get any easier than the tkCanvas widget... >>> import Tkinter as tk >>> #import tkinter as tk #python3.0+ >>> app = tk.Tk() >>> canvas = tk.Canvas(app) >>> canvas.pack() >>> canvas.create_line(sx,sy, ex,ey) >>> #... >>> #... >>> app.mainloop() >> He he, it's even easier to draw a graph using the turtle module. >> >> Cheers, >> >> - Alf > > Yes Alf please forgive me, i had completely forgotten about the > "visual" learners amongst us. > > turtle, turtle, watch him go... > turtle, turtle, why he so slow... > turtle, turtle, not even he don't know...? > > ;-) Uhm, that is a misconception. Just do turtle.hideturtle() turtle.tracer( 0 ) to turn off the animation stuff. That said, the turtle module bundled with Python is somewhat limited, but for plotting simple graphs it's certainly simpler than using tkinter directly. Cheers & hth., - Alf From javier.collado at gmail.com Wed Jan 27 09:07:15 2010 From: javier.collado at gmail.com (Javier Collado) Date: Wed, 27 Jan 2010 15:07:15 +0100 Subject: Help parsing a page with python In-Reply-To: References: <8c7f10c61001270213p10ad7d12ra39c75bea1f5e0f4@mail.gmail.com> Message-ID: Hello, You can find some advice here: http://www.packtpub.com/article/web-scraping-with-python-part-2 Best regards, Javier 2010/1/27 mierdatutis mi : > Hello again, > > What test case for Windmill? Can you say me the link, please? > > Many thanks > > 2010/1/27 Javier Collado >> >> Hello, >> >> A test case for Windmill might also be used to extract the information >> that you're looking for. >> >> Best regards, >> ? ?Javier >> >> 2010/1/27 mierdatutis mi : >> > Those videos are generated by javascript. >> > There is some parser with python for javascript??? >> > >> > Thanks a lot! >> > >> > >> > 2010/1/27 Simon Brunning >> >> >> >> 2010/1/27 mierdatutis mi : >> >> > Hi, >> >> > >> >> > I would like to parse a webpage to can get the url of the video >> >> > download. I >> >> > use pyhton and firebug but I cant get the url link. >> >> > >> >> > Example: >> >> > >> >> > The url where I have to get the video link is: >> >> > >> >> > >> >> > http://www.rtve.es/mediateca/videos/20100125/saber-comer---salsa-verde-judiones-25-01-10/676590.shtml" >> >> > >> >> > The video is >> >> > http://www.rtve.es/resources/TE_SSAC011/flv/8/2/1264426362028.flv >> >> > Could you help me please? >> >> >> >> That URL doesn't appear to be in the HTML - it must be being brought >> >> in by the JavaScript somehow. >> >> >> >> -- >> >> Cheers, >> >> Simon B. >> >> -- >> >> http://mail.python.org/mailman/listinfo/python-list >> > >> > >> > -- >> > http://mail.python.org/mailman/listinfo/python-list >> > >> > >> -- >> http://mail.python.org/mailman/listinfo/python-list > > > -- > http://mail.python.org/mailman/listinfo/python-list > > From fetchinson at googlemail.com Wed Jan 27 09:22:39 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Wed, 27 Jan 2010 15:22:39 +0100 Subject: myths about python 3 In-Reply-To: <6faf39c91001270245q155a932aj3e0bb98d377ddad1@mail.gmail.com> References: <6faf39c91001270245q155a932aj3e0bb98d377ddad1@mail.gmail.com> Message-ID: >> Hi folks, >> >> I was going to write this post for a while because all sorts of myths >> periodically come up on this list about python 3. I don't think the >> posters mean to spread false information on purpose, they simply are >> not aware of the facts. >> >> My list is surely incomplete, please feel free to post your favorite >> misconception about python 3 that people periodically state, claim or >> ask about. >> >> 1. Print statement/function creates incompatibility between 2.x and 3.x! >> >> Certainly false or misleading, if one uses 2.6 and 3.x the >> incompatibility is not there. Print as a function works in 2.6: >> >> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >>>>> print( 'hello' ) >> hello >>>>> print 'hello' >> hello >>>>> >> >> >> 2. Integer division creates incompatibility between 2.x and 3.x! >> >> Again false or misleading, because one can get the 3.x behavior with 2.6: >> >> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >>>>> 6/5 >> 1 >>>>> from __future__ import division >>>>> 6/5 >> 1.2 >> >> >> Please feel free to post your favorite false or misleading claim about >> python 3! > > Well, I see two false or misleading claims just above - namely that > the two claims above are false or misleading. They tell just half of > the story, and that half is indeed easy. A Python 3 program can be > unchanged (in the case of print) or with only trivial modifications > (in the case of integer division) be made to run on Python 2.6. Okay, so we agree that as long as print and integer division is concerned, a program can easily be written that runs on both 2.6 and 3.x. My statements are exactly this, so I don't understand why you disagree. > The other way around this is _not_ the case. What do you mean? > To say that two things are > compatible if one can be used for the other, but the other not for the > first, is false or misleading. I'm not sure what you mean here. Maybe I didn't make myself clear enough, but what I mean is this: as long as print and integer division is concerned, it is trivial to write code that runs on both 2.6 and 3.x. Hence if someone wants to highlight incompatibility (which surely exists) between 2.6 and 3.x he/she has to look elsewhere. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From richard.lamboj at bilcom.at Wed Jan 27 09:23:35 2010 From: richard.lamboj at bilcom.at (Richard Lamboj) Date: Wed, 27 Jan 2010 15:23:35 +0100 Subject: starting a thread in a nother thread In-Reply-To: <4b603b35$0$6725$9b4e6d93@newsspool2.arcor-online.net> References: <4b603b35$0$6725$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <201001271523.35060.richard.lamboj@bilcom.at> Am Wednesday 27 January 2010 14:10:13 schrieb Stefan Behnel: > Richard Lamboj, 27.01.2010 14:06: > > just for _curiosity_. What would be if i start a thread in a nother > > thread and acquire a lock in the "child" thread. Is there anything that > > could go wrong if someone try to start threads in threads? > > There's usually tons of things that can go wrong w.r.t. threads: > > http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf > > However, there's nothing special to a thread that was started from another > thread, so the problems don't change. > > Stefan Hello, i have tried a little bit around with psycopg2 and threads, I'am sharing one connection for all threads. When i'am starting the threads "normal" everything works without any Problem. When i'am starting the threads from another thread than i got a "segmentation fault" - i'am using locks. I'am sending 2048 Reqeuest at once. The "normal" Methode with one thread for every Request needs 10 seconds. The other Methode starting a thread and starting two other threads from this thread crashes after 10 to 40 requests with the segmentation fault error. Any Idea why? Its a 64 bit maschine. Maybe i'am making something wrong? Kind Regards From fetchinson at googlemail.com Wed Jan 27 09:30:00 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Wed, 27 Jan 2010 15:30:00 +0100 Subject: myths about python 3 In-Reply-To: <4b60181c$0$6732$9b4e6d93@newsspool2.arcor-online.net> References: <4b60181c$0$6732$9b4e6d93@newsspool2.arcor-online.net> Message-ID: >> 1. Print statement/function creates incompatibility between 2.x and 3.x! >> >> Certainly false or misleading, if one uses 2.6 and 3.x the >> incompatibility is not there. Print as a function works in 2.6: >> >> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >> >>> print( 'hello' ) >> hello >> >>> print 'hello' >> hello > > This is actually misleading by itself, as the first statement is not a > function call in Py2: > > Python 2.6.4 (r264:75706, Dec 7 2009, 18:43:55) > [GCC 4.4.1] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> print(1,2) > (1, 2) > > It can, however, be made a function call through a future import in 2.6: > > >>> from __future__ import print_function > >>> print(1,2) > 1 2 Thanks! This is true, luckily you provided a better solution and the conclusion is not changed, as long as print is concerned, 2.6 and 3.x can trivially be made compatible. Surely there are incompatibilities, but first of all there are many tools that help the transition such as 2to3 and there is a clear and officially documented migration guide too (quoted by Steve Holden in another thread not so long ago), second of all the most vocal arguments that one hears mostly from ill-informed people are related to print and similar non-issues. These then get quoted over and over again, which led me to write this post :) Cheers, Daniel Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From stefan_ml at behnel.de Wed Jan 27 09:30:17 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 27 Jan 2010 15:30:17 +0100 Subject: starting a thread in a nother thread In-Reply-To: References: <4b603b35$0$6725$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <4b604df9$0$6723$9b4e6d93@newsspool2.arcor-online.net> Richard Lamboj, 27.01.2010 15:23: > Am Wednesday 27 January 2010 14:10:13 schrieb Stefan Behnel: >> Richard Lamboj, 27.01.2010 14:06: >>> just for _curiosity_. What would be if i start a thread in a nother >>> thread and acquire a lock in the "child" thread. Is there anything that >>> could go wrong if someone try to start threads in threads? >> There's usually tons of things that can go wrong w.r.t. threads: >> >> http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf >> >> However, there's nothing special to a thread that was started from another >> thread, so the problems don't change. > > i have tried a little bit around with psycopg2 and threads, > > I'am sharing one connection for all threads. When i'am starting the > threads "normal" everything works without any Problem. When i'am starting the > threads from another thread than i got a "segmentation fault" Sounds like a bug that you might want to report to the maintainers of psycopg2. Stefan From jeanmichel at sequans.com Wed Jan 27 09:32:17 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Wed, 27 Jan 2010 15:32:17 +0100 Subject: myths about python 3 In-Reply-To: References: <6faf39c91001270245q155a932aj3e0bb98d377ddad1@mail.gmail.com> Message-ID: <4B604E71.5010706@sequans.com> Daniel Fetchinson wrote: >>> Hi folks, >>> >>> I was going to write this post for a while because all sorts of myths >>> periodically come up on this list about python 3. I don't think the >>> posters mean to spread false information on purpose, they simply are >>> not aware of the facts. >>> >>> My list is surely incomplete, please feel free to post your favorite >>> misconception about python 3 that people periodically state, claim or >>> ask about. >>> >>> 1. Print statement/function creates incompatibility between 2.x and 3.x! >>> >>> Certainly false or misleading, if one uses 2.6 and 3.x the >>> incompatibility is not there. Print as a function works in 2.6: >>> >>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>> >>>>>> print( 'hello' ) >>>>>> >>> hello >>> >>>>>> print 'hello' >>>>>> >>> hello >>> >>> 2. Integer division creates incompatibility between 2.x and 3.x! >>> >>> Again false or misleading, because one can get the 3.x behavior with 2.6: >>> >>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>> >>>>>> 6/5 >>>>>> >>> 1 >>> >>>>>> from __future__ import division >>>>>> 6/5 >>>>>> >>> 1.2 >>> >>> >>> Please feel free to post your favorite false or misleading claim about >>> python 3! >>> >> Well, I see two false or misleading claims just above - namely that >> the two claims above are false or misleading. They tell just half of >> the story, and that half is indeed easy. A Python 3 program can be >> unchanged (in the case of print) or with only trivial modifications >> (in the case of integer division) be made to run on Python 2.6. >> > > Okay, so we agree that as long as print and integer division is > concerned, a program can easily be written that runs on both 2.6 and > 3.x. > > My statements are exactly this, so I don't understand why you disagree. > > >> The other way around this is _not_ the case. >> > > What do you mean? > > >> To say that two things are >> compatible if one can be used for the other, but the other not for the >> first, is false or misleading. >> > > I'm not sure what you mean here. Maybe I didn't make myself clear > enough, but what I mean is this: as long as print and integer division > is concerned, it is trivial to write code that runs on both 2.6 and > 3.x. Hence if someone wants to highlight incompatibility (which surely > exists) between 2.6 and 3.x he/she has to look elsewhere. > > Cheers, > Daniel > > How would you write in python 2.6 if print: print('Hello') --- def myPrint(*args): for arg in args: sys.stdout.write(str(arg)) print = myPrint JM From spam2008 at nney.com Wed Jan 27 09:41:40 2010 From: spam2008 at nney.com (D HANNEY) Date: Wed, 27 Jan 2010 06:41:40 -0800 (PST) Subject: Generic proxy (that proxies methods like __iter__) Message-ID: <635f48e9-b8de-402b-8215-1e5c26a33ed1@k35g2000yqb.googlegroups.com> >>> def show(opened): ... with opened as file: ... for line in file: ... print line.strip() ... >>> show(open("test.txt")) blah1 blah2 blah3 # # Good! I wonder if I can do that with StringIO ... # >>> import StringIO >>> show(StringIO.StringIO("blah1\nblah2\nblah3\n")) Traceback (most recent call last): File "", line 1, in File "", line 2, in show AttributeError: StringIO instance has no attribute '__exit__' # # Oh dear ... I'll need to proxy it ... # >>> class IterNoGuardProxy(object): ... def __init__(self, t): ... self.t = t ... def __enter__(self): ... return self ... def __exit__(self, type, value, traceback): ... return False ... def __iter__(self): ... return self.t.__iter__() ... >>> show(IterNoGuardProxy(StringIO.StringIO("blah1\nblah2\nblah3\n"))) blah1 blah2 blah3 >>> # # Great! That works well. Now I wonder if I can do it generically # in a way that can proxy for ALL methods (including, but not limited to, __iter__) # >>> class NoGuardProxy(object): ... def __init__(self, t): ... self.t = t ... def __enter__(self): ... return self ... def __exit__(self, type, value, traceback): ... return False ... def __getattr__(self): ... return self.t.__getattr__(self) ... >>> show(NoGuardProxy(StringIO.StringIO("blah1\nblah2\nblah3\n"))) Traceback (most recent call last): File "", line 1, in File "", line 3, in show TypeError: 'NoGuardProxy' object is not iterable # # Now I'm stuck. # Can anyone help? # From lie.1296 at gmail.com Wed Jan 27 09:42:21 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Thu, 28 Jan 2010 01:42:21 +1100 Subject: myths about python 3 In-Reply-To: References: <6faf39c91001270245q155a932aj3e0bb98d377ddad1@mail.gmail.com> Message-ID: <4b6050ef$1@dnews.tpgi.com.au> On 01/28/10 01:32, Jean-Michel Pichavant wrote: > Daniel Fetchinson wrote: >>>> Hi folks, >>>> >>>> I was going to write this post for a while because all sorts of myths >>>> periodically come up on this list about python 3. I don't think the >>>> posters mean to spread false information on purpose, they simply are >>>> not aware of the facts. >>>> >>>> My list is surely incomplete, please feel free to post your favorite >>>> misconception about python 3 that people periodically state, claim or >>>> ask about. >>>> >>>> 1. Print statement/function creates incompatibility between 2.x and >>>> 3.x! >>>> >>>> Certainly false or misleading, if one uses 2.6 and 3.x the >>>> incompatibility is not there. Print as a function works in 2.6: >>>> >>>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >>>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >>>> Type "help", "copyright", "credits" or "license" for more information. >>>> >>>>>>> print( 'hello' ) >>>>>>> >>>> hello >>>> >>>>>>> print 'hello' >>>>>>> >>>> hello >>>> 2. Integer division creates incompatibility between 2.x and 3.x! >>>> >>>> Again false or misleading, because one can get the 3.x behavior with >>>> 2.6: >>>> >>>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >>>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >>>> Type "help", "copyright", "credits" or "license" for more information. >>>> >>>>>>> 6/5 >>>>>>> >>>> 1 >>>> >>>>>>> from __future__ import division >>>>>>> 6/5 >>>>>>> >>>> 1.2 >>>> >>>> >>>> Please feel free to post your favorite false or misleading claim about >>>> python 3! >>>> >>> Well, I see two false or misleading claims just above - namely that >>> the two claims above are false or misleading. They tell just half of >>> the story, and that half is indeed easy. A Python 3 program can be >>> unchanged (in the case of print) or with only trivial modifications >>> (in the case of integer division) be made to run on Python 2.6. >>> >> >> Okay, so we agree that as long as print and integer division is >> concerned, a program can easily be written that runs on both 2.6 and >> 3.x. >> >> My statements are exactly this, so I don't understand why you disagree. >> >> >>> The other way around this is _not_ the case. >>> >> >> What do you mean? >> >> >>> To say that two things are >>> compatible if one can be used for the other, but the other not for the >>> first, is false or misleading. >>> >> >> I'm not sure what you mean here. Maybe I didn't make myself clear >> enough, but what I mean is this: as long as print and integer division >> is concerned, it is trivial to write code that runs on both 2.6 and >> 3.x. Hence if someone wants to highlight incompatibility (which surely >> exists) between 2.6 and 3.x he/she has to look elsewhere. >> >> Cheers, >> Daniel >> >> > How would you write in python 2.6 > > if print: > print('Hello') > > --- > > def myPrint(*args): > for arg in args: > sys.stdout.write(str(arg)) > > print = myPrint > > JM from __future__ import print_function if print: print('Hello') def myPrint(*args): for arg in args: sys.stdout.write(str(arg)) print = myPrint From mal at egenix.com Wed Jan 27 09:47:06 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 27 Jan 2010 15:47:06 +0100 Subject: Ad hoc lists vs ad hoc tuples In-Reply-To: References: <8a57c64f-0c67-4ec1-b4be-34d76c7e7571@q4g2000yqm.googlegroups.com> Message-ID: <4B6051EA.50508@egenix.com> Iain King wrote: > On Jan 27, 10:20 am, Floris Bruynooghe > wrote: >> One thing I ofter wonder is which is better when you just need a >> throwaway sequence: a list or a tuple? E.g.: >> >> if foo in ['some', 'random', 'strings']: >> ... >> if [bool1, bool2, boo3].count(True) != 1: >> ... >> >> (The last one only works with tuples since python 2.6) >> >> Is a list or tuple better or more efficient in these situations? >> >> Regards >> Floris >> >> PS: This is inspired by some of the space-efficiency comments from the >> list.pop(0) discussion. > > I tend to use tuples unless using a list makes it easier to read. For > example: > > if foo in ('some', 'random', 'strings'): > > draw.text((10,30), "WHICH IS WHITE", font=font) > draw.line([(70,25), (85,25), (105,45)]) > > I've no idea what the performance difference is; I've always assumed > it's negligible. Tuples are a faster to create than lists, so if you create lots of them and don't need the list features, use tuples. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 27 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From showell30 at yahoo.com Wed Jan 27 09:53:24 2010 From: showell30 at yahoo.com (Steve Howell) Date: Wed, 27 Jan 2010 06:53:24 -0800 (PST) Subject: list.pop(0) vs. collections.dequeue References: <0acfecbe-a6aa-4438-bcd1-4bca08e5a2bd@k17g2000yqh.googlegroups.com> <8223b821-8600-4e83-af75-c5c2c97e5bab@h2g2000yqj.googlegroups.com> <970bb852-bbd4-4936-b123-73ef8e3552cf@b10g2000yqa.googlegroups.com> <7ea02185-0bc1-4f49-9931-0974d8a0db5c@a5g2000yqi.googlegroups.com> Message-ID: On Jan 26, 11:34?pm, Arnaud Delobelle wrote: > Steve Howell writes: > > On Jan 25, 1:32?pm, Arnaud Delobelle wrote: > >> Steve Howell writes: > > >> [...] > > >> > My algorithm does exactly N pops and roughly N list accesses, so I > >> > would be going from N*N + N to N + N log N if switched to blist. > > >> Can you post your algorithm? ?It would be interesting to have a concrete > >> use case to base this discussion on. > > > I just realized you meant the Python code itself. ?It is here: > > >https://bitbucket.org/showell/shpaml_website/src/tip/shpaml.py > > Hi - I didn't have the time to look at it before today. ?You scan > through a list called prefix_lines, and you use pop(0) as a means to > keep track of your position in this list. ?It seems to me that it would > be more effective to explicitely keep track of it - and it would remove > the need to the numerous copies of sublists of indent_lines that you > have to create. > > I have rewritten part of the three relevant functions (html_block_tag, > get_indented_block and recurse, within indent_lines) to show you what I > mean (see below). ?I have tried to keep the changes to a minimum - you > can see that the code is no more complex like this. ?The advantage is > that only one list is created, prefix_lines, and there is no need to > mutate it or copy parts of it during the algorithm. ?I have not had the > time to test it though (only on one of the examples on the examples > webpage). > > Code follows: > > [...] > > def html_block_tag(output, block, start, end, recurse): > ? ? append = output.append > ? ? prefix, tag = block[start] > ? ? if RAW_HTML.regex.match(tag): > ? ? ? ? append(prefix + tag) > ? ? ? ? recurse(block, start + 1, end) > ? ? else: > ? ? ? ? start_tag, end_tag = apply_jquery_sugar(tag) > ? ? ? ? append(prefix + start_tag) > ? ? ? ? recurse(block, start + 1, end) > ? ? ? ? append(prefix + end_tag) > > [...] > > def get_indented_block(prefix_lines, start, end): > ? ? prefix, line = prefix_lines[start] > ? ? len_prefix = len(prefix) > ? ? i = start + 1 > ? ? while i < end: > ? ? ? ? new_prefix, line = prefix_lines[i] > ? ? ? ? if line and len(new_prefix) <= len_prefix: > ? ? ? ? ? ? break > ? ? ? ? i += 1 > ? ? while i-1 > start and prefix_lines[i-1][1] == '': > ? ? ? ? i -= 1 > ? ? return i > > [...] > > def indent_lines(lines, > ? ? ? ? ? ? output, > ? ? ? ? ? ? branch_method, > ? ? ? ? ? ? leaf_method, > ? ? ? ? ? ? pass_syntax, > ? ? ? ? ? ? flush_left_syntax, > ? ? ? ? ? ? flush_left_empty_line, > ? ? ? ? ? ? indentation_method, > ? ? ? ? ? ? get_block, > ? ? ? ? ? ? ): > ? ? append = output.append > ? ? def recurse(prefix_lines, start, end): > ? ? ? ? while start < end: > ? ? ? ? ? ? prefix, line = prefix_lines[start] > ? ? ? ? ? ? if line == '': > ? ? ? ? ? ? ? ? start += 1 > ? ? ? ? ? ? ? ? append('') > ? ? ? ? ? ? else: > ? ? ? ? ? ? ? ? block_end = get_block(prefix_lines, start, end) > ? ? ? ? ? ? ? ? if block_end == start + 1: > ? ? ? ? ? ? ? ? ? ? start += 1 > ? ? ? ? ? ? ? ? ? ? if line == pass_syntax: > ? ? ? ? ? ? ? ? ? ? ? ? pass > ? ? ? ? ? ? ? ? ? ? elif line.startswith(flush_left_syntax): > ? ? ? ? ? ? ? ? ? ? ? ? append(line[len(flush_left_syntax):]) > ? ? ? ? ? ? ? ? ? ? elif line.startswith(flush_left_empty_line): > ? ? ? ? ? ? ? ? ? ? ? ? append('') > ? ? ? ? ? ? ? ? ? ? else: > ? ? ? ? ? ? ? ? ? ? ? ? append(prefix + leaf_method(line)) > ? ? ? ? ? ? ? ? else: > ? ? ? ? ? ? ? ? ? ? branch_method(output, prefix_lines, start, block_end, recurse) > ? ? ? ? ? ? ? ? ? ? start = block_end > ? ? ? ? return > ? ? prefix_lines = map(indentation_method, lines) > ? ? recurse(prefix_lines, 0, len(prefix_lines)) I think your rewrite makes sense for the way that Python is implemented today. Instead of mutating the list as you consume elements, you propose to advance the "start" variable, which is essentially a pointer. There are only two disadvantage of the "start" approach that I can think of: 1) You have to pass around two parameters where before there was only one. (Aside: for stylistic concerns, would you bundle them in a tuple, since they kind of go hand in hand?) 2) You hold on to memory from the elements longer. Pushing this complexity down into CPython of course would have similar disadvantages: 1) you have to store two pointers where before there was only one 2) you hold on to memory from pointers to orphaned elements longer Disadvantage #1 is more than offset by the fact that you would have had to waste memory with the "start" in the Python program. Disadvantage #2 is offset by the fact that you would have been holding on to the elements themselves in the Python program. Admittedly this a pretty narrow context where pushing the logic down into CPython gains overall simplicity and performance. Disadvantage #1 is the sticky point when you consider the patch in the broader context of all programs. Thanks for looking at the code, by the way. From rdv at roalddevries.nl Wed Jan 27 09:56:24 2010 From: rdv at roalddevries.nl (Roald de Vries) Date: Wed, 27 Jan 2010 15:56:24 +0100 Subject: Python and Ruby In-Reply-To: References: Message-ID: <589DBDBD-5847-442A-B124-806D742B4189@roalddevries.nl> On Jan 27, 2010, at 2:01 PM, Jean Guillaume Pyraksos wrote: > What are the arguments for choosing Python against Ruby > for introductory programming ? Python has no provisions > for tail recursion, Ruby is going to... So what ? > Thanks, I think the main difference is in culture, especially for *introductory* programming. For example, I have the impression that Rubyists like to write the same thing in as many ways as possible (example: rubyquiz.com), Python prefers one obvious way; this (arguably) makes Ruby more writable and Python more readable. From alfps at start.no Wed Jan 27 09:57:14 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 15:57:14 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: * Daniel Fetchinson: >>>> * Print is now a function. Great, much improvement. >> Actually not, IMHO. All it does is is to provide incompatibility. > > > What incompatibility are you exactly talking about? > > Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) > [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> print( 'hello' ) > hello >>>> print 'hello' > hello > > Yes, this is with python 2.6.2 which is in the 2.x line of releases. So? I gather that your example is about code that technically executes fine with both versions and produces the same result, i.e. that there is a subset with the same syntax and semantics. But 'print' calls that technically execute fine with both versions may and will in general produce different results. I.e. not just the syntax but also the semantics have changed: >>> import sys >>> sys.version '2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]' >>> >>> print( "2+2 =", 2+2 ) ('2+2 =', 4) >>> _ >>> import sys >>> sys.version '3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)]' >>> >>> print( "2+2 =", 2+2 ) 2+2 = 4 >>> _ Cheers & hth., - Alf From arnodel at googlemail.com Wed Jan 27 10:07:23 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Wed, 27 Jan 2010 07:07:23 -0800 (PST) Subject: Generic proxy (that proxies methods like __iter__) References: <635f48e9-b8de-402b-8215-1e5c26a33ed1@k35g2000yqb.googlegroups.com> Message-ID: On 27 Jan, 14:41, D HANNEY wrote: [...] > >>> class NoGuardProxy(object): > > ... def __init__(self, t): > ... self.t = t > ... def __enter__(self): > ... return self > ... def __exit__(self, type, value, traceback): > ... return False > ... ? ? ? ? def __getattr__(self): > ... ? ? ? ? ? ? ? ? return self.t.__getattr__(self) > ...>>> show(NoGuardProxy(StringIO.StringIO("blah1\nblah2\nblah3\n"))) > > Traceback (most recent call last): > ? File "", line 1, in > ? File "", line 3, in show > TypeError: 'NoGuardProxy' object is not iterable > > # > # Now I'm stuck. > # Can anyone help? > # See [1] for an explanation. Here is an idea: you could get round that by generating a class on the fly, if you don't mind changing the class of the object (untested): def noguardproxy(obj): class NoGuardProxy(type(obj)): def __enter__(self): return self def __exit__(self, type, value, traceback): return False obj.__class__ = NoGuardProxy return obj If you want to keep obj untouched, you could add the line obj = copy(obj) before changing the class. -- Arnaud [1] http://docs.python.org/reference/datamodel.html#new-style-special-lookup From steve at holdenweb.com Wed Jan 27 10:18:20 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 27 Jan 2010 10:18:20 -0500 Subject: Ad hoc lists vs ad hoc tuples In-Reply-To: References: <8a57c64f-0c67-4ec1-b4be-34d76c7e7571@q4g2000yqm.googlegroups.com> Message-ID: <4B60593C.8050202@holdenweb.com> Iain King wrote: > On Jan 27, 10:20 am, Floris Bruynooghe > wrote: >> One thing I ofter wonder is which is better when you just need a >> throwaway sequence: a list or a tuple? E.g.: >> >> if foo in ['some', 'random', 'strings']: >> ... >> if [bool1, bool2, boo3].count(True) != 1: >> ... >> >> (The last one only works with tuples since python 2.6) >> >> Is a list or tuple better or more efficient in these situations? >> >> Regards >> Floris >> >> PS: This is inspired by some of the space-efficiency comments from the >> list.pop(0) discussion. > > I tend to use tuples unless using a list makes it easier to read. For > example: > > if foo in ('some', 'random', 'strings'): > > draw.text((10,30), "WHICH IS WHITE", font=font) > draw.line([(70,25), (85,25), (105,45)]) > > I've no idea what the performance difference is; I've always assumed > it's negligible. > The fact that you have felt able to neglect the performance difference makes it quite literally negligible. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Wed Jan 27 10:18:20 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 27 Jan 2010 10:18:20 -0500 Subject: Ad hoc lists vs ad hoc tuples In-Reply-To: References: <8a57c64f-0c67-4ec1-b4be-34d76c7e7571@q4g2000yqm.googlegroups.com> Message-ID: <4B60593C.8050202@holdenweb.com> Iain King wrote: > On Jan 27, 10:20 am, Floris Bruynooghe > wrote: >> One thing I ofter wonder is which is better when you just need a >> throwaway sequence: a list or a tuple? E.g.: >> >> if foo in ['some', 'random', 'strings']: >> ... >> if [bool1, bool2, boo3].count(True) != 1: >> ... >> >> (The last one only works with tuples since python 2.6) >> >> Is a list or tuple better or more efficient in these situations? >> >> Regards >> Floris >> >> PS: This is inspired by some of the space-efficiency comments from the >> list.pop(0) discussion. > > I tend to use tuples unless using a list makes it easier to read. For > example: > > if foo in ('some', 'random', 'strings'): > > draw.text((10,30), "WHICH IS WHITE", font=font) > draw.line([(70,25), (85,25), (105,45)]) > > I've no idea what the performance difference is; I've always assumed > it's negligible. > The fact that you have felt able to neglect the performance difference makes it quite literally negligible. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From kw at codebykevin.com Wed Jan 27 10:23:17 2010 From: kw at codebykevin.com (Kevin Walzer) Date: Wed, 27 Jan 2010 10:23:17 -0500 Subject: Library support for Python 3.x Message-ID: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> I'm going to be starting some new Python projects in Python 2.6, but am concerned that at least three of the libraries I will be using--pycrypto, paramiko and feedparser--are not currently supported in Python 3.x. The authors of these programs have not given any indication that work is underway to support Python 3.x. Eventually I plan to migrate to Python 3.x. I don't want to be stuck using libraries that may not be updated for the next generation of Python. How are others handling this issue? --Kevin -- Kevin Walzer Code by Kevin http://www.codebykevin.com From fetchinson at googlemail.com Wed Jan 27 10:26:37 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Wed, 27 Jan 2010 16:26:37 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: >>>>> * Print is now a function. Great, much improvement. >>> Actually not, IMHO. All it does is is to provide incompatibility. >> >> >> What incompatibility are you exactly talking about? >> >> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >>>>> print( 'hello' ) >> hello >>>>> print 'hello' >> hello >> >> Yes, this is with python 2.6.2 which is in the 2.x line of releases. So? > > I gather that your example is about code that technically executes fine with > both versions and produces the same result, i.e. that there is a subset with > the > same syntax and semantics. > > But 'print' calls that technically execute fine with both versions may and > will > in general produce different results. > > I.e. not just the syntax but also the semantics have changed: > > > >>> import sys > >>> sys.version > '2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]' > >>> > >>> print( "2+2 =", 2+2 ) > ('2+2 =', 4) > >>> _ > > > >>> import sys > >>> sys.version > '3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)]' > >>> > >>> print( "2+2 =", 2+2 ) > 2+2 = 4 > >>> _ True. However, as someone else pointed out in a neighbouring thread you can do Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from __future__ import print_function >>> print( "2+2 =", 2+2 ) 2+2 = 4 >>> which gives 100% compatibility as far as print is concerned between 2.6 and 3.x. So again, what sort of an incompatibility are you talking about concerning 'print' between 2.6 and 3.x? Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From alfps at start.no Wed Jan 27 10:47:04 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 16:47:04 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: * Daniel Fetchinson: >>>>>> * Print is now a function. Great, much improvement. >>>> Actually not, IMHO. All it does is is to provide incompatibility. >>> >>> What incompatibility are you exactly talking about? >>> >>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>>>>> print( 'hello' ) >>> hello >>>>>> print 'hello' >>> hello >>> >>> Yes, this is with python 2.6.2 which is in the 2.x line of releases. So? >> I gather that your example is about code that technically executes fine with >> both versions and produces the same result, i.e. that there is a subset with >> the >> same syntax and semantics. >> >> But 'print' calls that technically execute fine with both versions may and >> will >> in general produce different results. >> >> I.e. not just the syntax but also the semantics have changed: >> >> >> >>> import sys >> >>> sys.version >> '2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]' >> >>> >> >>> print( "2+2 =", 2+2 ) >> ('2+2 =', 4) >> >>> _ >> >> >> >>> import sys >> >>> sys.version >> '3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)]' >> >>> >> >>> print( "2+2 =", 2+2 ) >> 2+2 = 4 >> >>> _ > > True. However, as someone else pointed out in a neighbouring thread you can do > > Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) > [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> from __future__ import print_function >>>> print( "2+2 =", 2+2 ) > 2+2 = 4 > > which gives 100% compatibility as far as print is concerned between 2.6 and 3.x. That makes the code behave with 3.x syntax and semantics regarding print. I.e. it chooses one language. It doesn't make them compatible: if they were, then you wouldn't have to choose. For example, it doesn't fix up 2.x code that prints tuples: incompatible, it will still execute but no longer do what it was supposed to do. But lest you focus down on that rare case, consider 2.x code that contains print "2+2", 2 All such statements need to be fixed up in a 2->3 transition. Adding __future__ just makes the code invalid, it doesn't give you any shred of compatibility for this. A code rewrite can be partially automated like 2to3 but the incompatibility needlessly introduces an additional set of things that Can Go Wrong(TM), and with Murphy present, as He always is, well. > So again, what sort of an incompatibility are you talking about > concerning 'print' between 2.6 and 3.x? Syntax and semantics. They're different. Cheers & hth., - Alf From solipsis at pitrou.net Wed Jan 27 10:56:10 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 27 Jan 2010 15:56:10 +0000 (UTC) Subject: ctypes for AIX References: <9C240725D2ABD949A80B46DD93AB107042E8D44101@XCH-NW-08V.nw.nos.boeing.com> <50697b2c1001240630r33845a1bl7f94b40a4f0c4436@mail.gmail.com> <9C240725D2ABD949A80B46DD93AB107042E8D44574@XCH-NW-08V.nw.nos.boeing.com> Message-ID: Hello, > I then figured I would get a new copy of python and install it > on AIX. I downloaded python.2.5.5c2 from http://www.python.org. I did > the configure and make which posted many errors in the ctypes function > which I guess is the reason that is does not get include in the final > make. I would suggest two things: 1) try with the latest Python 2.6.x, because 2.5 only receives security fixes now. 2.6 is the stable branch which receives regular bug fixes (which includes build problems) 2) if Python 2.6.x also fails compiling, please file a bug report on http://bugs.python.org Regards Antoine. From Eric_Dexter at msn.com Wed Jan 27 11:11:29 2010 From: Eric_Dexter at msn.com (Eric_Dexter at msn.com) Date: Wed, 27 Jan 2010 08:11:29 -0800 (PST) Subject: myths about python 3 References: <6faf39c91001270245q155a932aj3e0bb98d377ddad1@mail.gmail.com> <4b6050ef$1@dnews.tpgi.com.au> Message-ID: <2cd4f02e-ffef-4c50-b4d5-78d92111c7fa@a32g2000yqm.googlegroups.com> On Jan 27, 8:42?am, Lie Ryan wrote: > On 01/28/10 01:32, Jean-Michel Pichavant wrote: > > > > > Daniel Fetchinson wrote: > >>>> Hi folks, > > >>>> I was going to write this post for a while because all sorts of myths > >>>> periodically come up on this list about python 3. I don't think the > >>>> posters mean to spread false information on purpose, they simply are > >>>> not aware of the facts. > > >>>> My list is surely incomplete, please feel free to post your favorite > >>>> misconception about python 3 that people periodically state, claim or > >>>> ask about. > > >>>> 1. Print statement/function creates incompatibility between 2.x and > >>>> 3.x! > > >>>> Certainly false or misleading, if one uses 2.6 and 3.x the > >>>> incompatibility is not there. Print as a function works in 2.6: > > >>>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) > >>>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 > >>>> Type "help", "copyright", "credits" or "license" for more information. > > >>>>>>> print( 'hello' ) > > >>>> hello > > >>>>>>> print 'hello' > > >>>> hello > >>>> ? ? ? 2. Integer division creates incompatibility between 2.x and 3.x! > > >>>> Again false or misleading, because one can get the 3.x behavior with > >>>> 2.6: > > >>>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) > >>>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 > >>>> Type "help", "copyright", "credits" or "license" for more information. > > >>>>>>> 6/5 > > >>>> 1 > > >>>>>>> from __future__ import division > >>>>>>> 6/5 > > >>>> 1.2 > > >>>> Please feel free to post your favorite false or misleading claim about > >>>> python 3! > > >>> Well, I see two false or misleading claims just above - namely that > >>> the two claims above are false or misleading. They tell just half of > >>> the story, and that half is indeed easy. A Python 3 program can be > >>> unchanged (in the case of print) or with only trivial modifications > >>> (in the case of integer division) be made to run on Python 2.6. > > >> Okay, so we agree that as long as print and integer division is > >> concerned, a program can easily be written that runs on both 2.6 and > >> 3.x. > > >> My statements are exactly this, so I don't understand why you disagree. > > >>> The other way around this is _not_ the case. > > >> What do you mean? > > >>> To say that two things are > >>> compatible if one can be used for the other, but the other not for the > >>> first, is false or misleading. > > >> I'm not sure what you mean here. Maybe I didn't make myself clear > >> enough, but what I mean is this: as long as print and integer division > >> is concerned, it is trivial to write code that runs on both 2.6 and > >> 3.x. Hence if someone wants to highlight incompatibility (which surely > >> exists) between 2.6 and 3.x he/she has to look elsewhere. > > >> Cheers, > >> Daniel > > > How would you write in python 2.6 > > > if print: > > ? ?print('Hello') > > > --- > > > def myPrint(*args): > > ? ?for arg in args: > > ? ? ? ?sys.stdout.write(str(arg)) > > > print = myPrint > > > JM > > from __future__ import print_function > > if print: > ? ? print('Hello') > > def myPrint(*args): > ? ?for arg in args: > ? ? ? ?sys.stdout.write(str(arg)) > > print = myPrint I can't say that I am that keen on 2.6 all my favorite graphics libraries are in 2.5. If there was money involved I would probably think y'all were doing it to stay employed so I am thinking I should wait till 3.4 and 3.5 to get involved with this but much sooner than python 4.0. I did notice that I had trouble compiling a library because some version of microsoft c is no longer available...sort of forced migration. ______________________________________ http://dextracker.blogspot.com/ From stefan_ml at behnel.de Wed Jan 27 11:24:37 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 27 Jan 2010 17:24:37 +0100 Subject: python 3's adoption In-Reply-To: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: <4b6068c6$0$6725$9b4e6d93@newsspool2.arcor-online.net> Xah Lee, 27.01.2010 00:47: > Any comment on this? No, sorry. Not worth bothering. Stefan From mmm286 at gmail.com Wed Jan 27 11:49:39 2010 From: mmm286 at gmail.com (mierdatutis mi) Date: Wed, 27 Jan 2010 17:49:39 +0100 Subject: Help parsing a page with python In-Reply-To: References: <8c7f10c61001270213p10ad7d12ra39c75bea1f5e0f4@mail.gmail.com> Message-ID: Hello, Thanks Javier, But I think that the page embeds a viewer. Only the viewer knows the URL to the FLV file itself. I can't see any direct correspondence between the elements of the two URLs, I cant see a way to construct the FLV's URL from the contents of that page :-( I have to do manually, I dont see other way :-( Many thanks for your answer Muchas gracias 2010/1/27 Javier Collado > Hello, > > You can find some advice here: > http://www.packtpub.com/article/web-scraping-with-python-part-2 > > Best regards, > Javier > > 2010/1/27 mierdatutis mi : > > Hello again, > > > > What test case for Windmill? Can you say me the link, please? > > > > Many thanks > > > > 2010/1/27 Javier Collado > >> > >> Hello, > >> > >> A test case for Windmill might also be used to extract the information > >> that you're looking for. > >> > >> Best regards, > >> Javier > >> > >> 2010/1/27 mierdatutis mi : > >> > Those videos are generated by javascript. > >> > There is some parser with python for javascript??? > >> > > >> > Thanks a lot! > >> > > >> > > >> > 2010/1/27 Simon Brunning > >> >> > >> >> 2010/1/27 mierdatutis mi : > >> >> > Hi, > >> >> > > >> >> > I would like to parse a webpage to can get the url of the video > >> >> > download. I > >> >> > use pyhton and firebug but I cant get the url link. > >> >> > > >> >> > Example: > >> >> > > >> >> > The url where I have to get the video link is: > >> >> > > >> >> > > >> >> > > http://www.rtve.es/mediateca/videos/20100125/saber-comer---salsa-verde-judiones-25-01-10/676590.shtml > " > >> >> > > >> >> > The video is > >> >> > http://www.rtve.es/resources/TE_SSAC011/flv/8/2/1264426362028.flv > >> >> > Could you help me please? > >> >> > >> >> That URL doesn't appear to be in the HTML - it must be being brought > >> >> in by the JavaScript somehow. > >> >> > >> >> -- > >> >> Cheers, > >> >> Simon B. > >> >> -- > >> >> http://mail.python.org/mailman/listinfo/python-list > >> > > >> > > >> > -- > >> > http://mail.python.org/mailman/listinfo/python-list > >> > > >> > > >> -- > >> http://mail.python.org/mailman/listinfo/python-list > > > > > > -- > > http://mail.python.org/mailman/listinfo/python-list > > > > > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fetchinson at googlemail.com Wed Jan 27 11:58:10 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Wed, 27 Jan 2010 17:58:10 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: >>>>>>> * Print is now a function. Great, much improvement. >>>>> Actually not, IMHO. All it does is is to provide incompatibility. >>>> >>>> What incompatibility are you exactly talking about? >>>> >>>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >>>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >>>> Type "help", "copyright", "credits" or "license" for more information. >>>>>>> print( 'hello' ) >>>> hello >>>>>>> print 'hello' >>>> hello >>>> >>>> Yes, this is with python 2.6.2 which is in the 2.x line of releases. So? >>> I gather that your example is about code that technically executes fine >>> with >>> both versions and produces the same result, i.e. that there is a subset >>> with >>> the >>> same syntax and semantics. >>> >>> But 'print' calls that technically execute fine with both versions may >>> and >>> will >>> in general produce different results. >>> >>> I.e. not just the syntax but also the semantics have changed: >>> >>> >>> >>> import sys >>> >>> sys.version >>> '2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]' >>> >>> >>> >>> print( "2+2 =", 2+2 ) >>> ('2+2 =', 4) >>> >>> _ >>> >>> >>> >>> import sys >>> >>> sys.version >>> '3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)]' >>> >>> >>> >>> print( "2+2 =", 2+2 ) >>> 2+2 = 4 >>> >>> _ >> >> True. However, as someone else pointed out in a neighbouring thread you >> can do >> >> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >>>>> from __future__ import print_function >>>>> print( "2+2 =", 2+2 ) >> 2+2 = 4 >> >> which gives 100% compatibility as far as print is concerned between 2.6 >> and 3.x. > > That makes the code behave with 3.x syntax and semantics regarding print. > I.e. it chooses one language. > > It doesn't make them compatible: Of course it makes them compatible. I'm not saying any print-related code in python 2.6 is valid python 3 code, but that it is *possible* to write print-related code in python 2.6 that is also valid in python 3. > if they were, then you wouldn't have to choose. It seems to me you are arguing with the statement "Any print-related python 2.6 code is valid python 3 code". Nobody is making this statement. Let me repeat, what you should be arguing with is "It is possible to write print-related python 2.6 code that is also valid python 3 code". Perhaps I didn't make myself clear before, but at least now I hope it's clear what I mean. Cheers, Daniel > For example, it doesn't fix up 2.x code that prints tuples: incompatible, it > will still execute but no longer do what it was supposed to do. > > But lest you focus down on that rare case, consider 2.x code that contains > > print "2+2", 2 > > All such statements need to be fixed up in a 2->3 transition. Adding > __future__ > just makes the code invalid, it doesn't give you any shred of compatibility > for > this. A code rewrite can be partially automated like 2to3 but the > incompatibility needlessly introduces an additional set of things that Can > Go > Wrong(TM), and with Murphy present, as He always is, well. > > >> So again, what sort of an incompatibility are you talking about >> concerning 'print' between 2.6 and 3.x? > > Syntax and semantics. They're different. -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From aahz at pythoncraft.com Wed Jan 27 12:25:09 2010 From: aahz at pythoncraft.com (Aahz) Date: 27 Jan 2010 09:25:09 -0800 Subject: Library support for Python 3.x References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> Message-ID: In article <9f3c3$4b605a65$4275d90a$30451 at FUSE.NET>, Kevin Walzer wrote: > >I'm going to be starting some new Python projects in Python 2.6, but am >concerned that at least three of the libraries I will be >using--pycrypto, paramiko and feedparser--are not currently supported in >Python 3.x. The authors of these programs have not given any indication >that work is underway to support Python 3.x. Eventually I plan to >migrate to Python 3.x. > >I don't want to be stuck using libraries that may not be updated for the >next generation of Python. How are others handling this issue? >From my POV, your question would be precisely identical if you had started your project when Python 2.3 was just released and wanted to know if the libraries you selected would be available for Python 2.6. Choose the best libraries for your circumstances now and worry about the future later. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From chiragdhyani at gmail.com Wed Jan 27 12:27:43 2010 From: chiragdhyani at gmail.com (Chirag Dhyani) Date: Wed, 27 Jan 2010 22:57:43 +0530 Subject: GUI-pyqt solution needed Message-ID: <83ab036c1001270927p4cc07a3dna3c5a08b2c5432d1@mail.gmail.com> Hi, My sincere apologies if I am mailing in wrong list. My mission: I have to develop GUI based tool on python and will be using PyQT with qtdesigner. I am automating creation of test script for call flows just by drawing them from tool. Challenges: In this tool there will be a drawing area where I should have to utility to create call flow by drag and droping the elements. I will have a small database from which relevent piece of codes will be retrieved during the execution time and the script will be retrieved. I know that I will be quering the database based on cordinates so think that cordinate programming will be the correct solution. However, can anybody have idea for the ideal solution that can be? Thank you in advance. Chi -------------- next part -------------- An HTML attachment was scrubbed... URL: From aahz at pythoncraft.com Wed Jan 27 12:27:47 2010 From: aahz at pythoncraft.com (Aahz) Date: 27 Jan 2010 09:27:47 -0800 Subject: Simple Password Strength Checker Review Help needed References: <158b9ef31001262148g1926d067ja8ba840ba074eca4@mail.gmail.com> <7xfx5sxbmw.fsf@ruckus.brouhaha.com> Message-ID: In article <7xfx5sxbmw.fsf at ruckus.brouhaha.com>, Paul Rubin wrote: > >From a security point of view, the concept >of "password strength checking" is pretty dubious. If you want secure >passwords, generate them from a random number source and assign them to >the users. Don't have the users make up their own passwords. It's >relatively (compared to using a computer file exposed to remote internet >attacks) for users to write down the the random passwords on paper, as >long as they're a little bit careful. As Bruce Schneier put it: > > "My wallet is already a secure container; it has valuable things in > it, and I have a lifetime of experience keeping it safe. Adding a > piece of paper with my passwords seems like a natural thing to do." Actually, I treat my wallet as a source of trouble and only keep replaceable items in it. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From alfps at start.no Wed Jan 27 12:29:25 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 18:29:25 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: * Daniel Fetchinson: >>>>>>>> * Print is now a function. Great, much improvement. >>>>>> Actually not, IMHO. All it does is is to provide incompatibility. >>>>> What incompatibility are you exactly talking about? >>>>> >>>>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >>>>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >>>>> Type "help", "copyright", "credits" or "license" for more information. >>>>>>>> print( 'hello' ) >>>>> hello >>>>>>>> print 'hello' >>>>> hello >>>>> >>>>> Yes, this is with python 2.6.2 which is in the 2.x line of releases. So? >>>> I gather that your example is about code that technically executes fine >>>> with >>>> both versions and produces the same result, i.e. that there is a subset >>>> with >>>> the >>>> same syntax and semantics. >>>> >>>> But 'print' calls that technically execute fine with both versions may >>>> and >>>> will >>>> in general produce different results. >>>> >>>> I.e. not just the syntax but also the semantics have changed: >>>> >>>> >>>> >>> import sys >>>> >>> sys.version >>>> '2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]' >>>> >>> >>>> >>> print( "2+2 =", 2+2 ) >>>> ('2+2 =', 4) >>>> >>> _ >>>> >>>> >>>> >>> import sys >>>> >>> sys.version >>>> '3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)]' >>>> >>> >>>> >>> print( "2+2 =", 2+2 ) >>>> 2+2 = 4 >>>> >>> _ >>> True. However, as someone else pointed out in a neighbouring thread you >>> can do >>> >>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>>>>> from __future__ import print_function >>>>>> print( "2+2 =", 2+2 ) >>> 2+2 = 4 >>> >>> which gives 100% compatibility as far as print is concerned between 2.6 >>> and 3.x. >> That makes the code behave with 3.x syntax and semantics regarding print. >> I.e. it chooses one language. >> >> It doesn't make them compatible: > > Of course it makes them compatible. I'm not saying any print-related > code in python 2.6 is valid python 3 code, but that it is *possible* > to write print-related code in python 2.6 that is also valid in python > 3. > >> if they were, then you wouldn't have to choose. > > It seems to me you are arguing with the statement "Any print-related > python 2.6 code is valid python 3 code". Nobody is making this > statement. Let me repeat, what you should be arguing with is "It is > possible to write print-related python 2.6 code that is also valid > python 3 code". Perhaps I didn't make myself clear before, but at > least now I hope it's clear what I mean. Perhaps we're in violent agreement. :-) However, the root node of this subthread was my statement about the needless incompatibility introduced by changing print in 3.x, whether that statement was correct or reasonable / whatever. The main problem with the incompatibility is for porting code, not for writing code from scratch. It's also a problem wrt. learning the language. And I see no good reason for it: print can't really do more, or less, or more conveniently (rather, one has to write a bit more now for same effect). Cheers, - Alf From solipsis at pitrou.net Wed Jan 27 12:32:27 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 27 Jan 2010 17:32:27 +0000 (UTC) Subject: Ad hoc lists vs ad hoc tuples References: <8a57c64f-0c67-4ec1-b4be-34d76c7e7571@q4g2000yqm.googlegroups.com> Message-ID: Le Wed, 27 Jan 2010 02:20:53 -0800, Floris Bruynooghe a ?crit?: > > Is a list or tuple better or more efficient in these situations? Tuples are faster to allocate (they are allocated in one single step) and quite a bit smaller too. In some situations, in Python 2.7 and 3.1, they can also be ignored by the garbage collector, yielding faster collections. (not to mention that they are hashable, which can be useful) From adityashukla1983 at gmail.com Wed Jan 27 12:37:33 2010 From: adityashukla1983 at gmail.com (aditya shukla) Date: Wed, 27 Jan 2010 11:37:33 -0600 Subject: Extract half screen of a video Message-ID: <73045cca1001270937k5894ae2dg92f472fe89b7b8e2@mail.gmail.com> Hello Guys, I have a video and what I want is to extract only half of the screen of it.By half screen i mean when i run the video i can say the left half(from the monitor's screen) of the video and dump the right half. Thanks Aditya -------------- next part -------------- An HTML attachment was scrubbed... URL: From luismgz at gmail.com Wed Jan 27 12:38:36 2010 From: luismgz at gmail.com (=?ISO-8859-1?Q?Luis_M=2E_Gonz=E1lez?=) Date: Wed, 27 Jan 2010 09:38:36 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> Message-ID: > Please don't post more noise and ad hominem attacks to the group, Steve. "Ad hominem"? Please, operor non utor lingua non notus per vulgaris populus. Gratias ago vos... From steve at holdenweb.com Wed Jan 27 12:50:10 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 27 Jan 2010 12:50:10 -0500 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: <4B607CD2.1060408@holdenweb.com> Alf P. Steinbach wrote: [...] > The main problem with the incompatibility is for porting code, not for > writing code from scratch. It's also a problem wrt. learning the > language. And I see no good reason for it: print can't really do more, > or less, or more conveniently (rather, one has to write a bit more now > for same effect). Of course it can do more: it allows you to layer your own print functionality into the system much more easily than you could with the print statement. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From alfps at start.no Wed Jan 27 12:52:57 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 18:52:57 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: * Steve Holden: > Alf P. Steinbach wrote: > [...] >> The main problem with the incompatibility is for porting code, not for >> writing code from scratch. It's also a problem wrt. learning the >> language. And I see no good reason for it: print can't really do more, >> or less, or more conveniently (rather, one has to write a bit more now >> for same effect). > > Of course it can do more: it allows you to layer your own print > functionality into the system much more easily than you could with the > print statement. Yeah, point. Very minor though. :-) Cheers, - Alf From arnodel at googlemail.com Wed Jan 27 12:57:59 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Wed, 27 Jan 2010 17:57:59 +0000 Subject: site.py confusion References: Message-ID: George Trojan writes: > Inspired by the 'Default path for files' thread I tried to use > sitecustomize in my code. What puzzles me is that the site.py's main() > is not executed. My sitecustomize.py is > def main(): > print 'In Main()' > main() > and the test program is > import site > #site.main() > print 'Hi' > The output is > $ python try.py > Hi That's normal as site.py is automatically imported on initialisation. So when you do: import site the module is not re-executed as it already has been imported. Try this: ---- file: foo.py print 'Foo' ---- end --- Interactive session >>> import foo # First time, print statement executed Foo >>> import foo # Second time, print statement not executed >>> > When I uncomment the site.main() line the output is > $ python try.py > In Main() > Hi Now you explicitely call site.main(), so it executes it! > If I change import site to import sitecustomize the output is as > above. What gives? It's normal, this time it's the first time you import it so its content is executed. HTH -- Arnaud From MLMDev at Comcast.net Wed Jan 27 13:29:41 2010 From: MLMDev at Comcast.net (Mitchell L Model) Date: Wed, 27 Jan 2010 13:29:41 -0500 Subject: I really need webbrowser.open('file://') to open a web browser In-Reply-To: References: Message-ID: <5A08CF3C-63E4-4103-B4B2-6C17E1D6B6D0@Comcast.net> On Jan 15, 2010, at 3:59 PM, Timur Tabi > After reading several web pages and mailing list threads, I've learned > that the webbrowser module does not really support opening local > files, even if I use a file:// URL designator. In most cases, > webbrowser.open() will indeed open the default web browser, but with > Python 2.6 on my Fedora 10 system, it opens a text editor instead. On > Python 2.5, it opens the default web browser. > > This is a problem because my Python script creates a local HTML file > and I want it displayed on the web browser. > > So is there any way to force webbrowser.open() to always use an actual > web browser? I had some discussions with the Python documentation writers that led to the following note being included in the Python 3.1 library documentation for webbrowser.open: "Note that on some platforms, trying to open a filename using this function, may work and start the operating system?s associated program. However, this is neither supported nor portable." The discussions suggested that this lack of support and portability was actually always the case and that the webbrowser module is simply not meant to handle file URLs. I had taken advantage of the accidental functionality to generate HTML reports and open them, as well as to open specific documentation pages from within a program. You can control which browser opens the URL by using webbrowser.get to obtain a controller for a particular browser, specified by its argument, then call the open method on the controller instead of the module. For opening files reliability and the ability to pick a particular program (browser or otherwise) to open it with you might have to resort to invoking a command line via subprocess.Popen. From george.trojan at noaa.gov Wed Jan 27 13:48:23 2010 From: george.trojan at noaa.gov (George Trojan) Date: Wed, 27 Jan 2010 18:48:23 +0000 Subject: site.py confusion In-Reply-To: References: Message-ID: Arnaud Delobelle wrote: > George Trojan writes: > >> Inspired by the 'Default path for files' thread I tried to use >> sitecustomize in my code. What puzzles me is that the site.py's main() >> is not executed. My sitecustomize.py is >> def main(): >> print 'In Main()' >> main() >> and the test program is >> import site >> #site.main() >> print 'Hi' >> The output is >> $ python try.py >> Hi > > That's normal as site.py is automatically imported on initialisation. > So when you do: > > import site > > the module is not re-executed as it already has been imported. > Try this: > > ---- file: foo.py > print 'Foo' > ---- end > > --- Interactive session > >>> import foo # First time, print statement executed > Foo > >>> import foo # Second time, print statement not executed > >>> > >> When I uncomment the site.main() line the output is >> $ python try.py >> In Main() >> Hi > > Now you explicitely call site.main(), so it executes it! > >> If I change import site to import sitecustomize the output is as >> above. What gives? > > It's normal, this time it's the first time you import it so its content > is executed. > > HTH > I understand that importing a module repeatedly does nothing. Also, I made a typo in my previous posting - I meant sitecustomize.main(), not site.main(). My understanding of the code in site.py is that when the module is imported, main() is executed. main() calls execsitecustomize() that attempts to import sitecustomize. That action should trigger execution of code in sitecustomize.py, which is located in the current directory. But that does not work. I changed execsitecustomize() to def execsitecustomize(): """Run custom site specific code, if available.""" try: import sitecustomize except ImportError: pass import sys print sys.path raise That gave me the explanation why the above happens: when site is imported, the current directory is not yet prepended to sys.path. $ python2.6 -v try.py ... ['/usr/local/Python-2.6.3/lib/python26.zip', '/usr/local/Python-2.6.3/lib/python2.6', '/usr/local/Python-2.6.3/lib/python2.6/plat-linux2', '/usr/local/Python-2.6.3/lib/python2.6/lib-tk', '/usr/local/Python-2.6.3/lib/python2.6/lib-old', '/usr/local/Python-2.6.3/lib/python2.6/lib-dynload', '/usr/local/Python-2.6.3/lib/python2.6/site-packages'] 'import site' failed; traceback: Traceback (most recent call last): File "/usr/local/Python-2.6.3/lib/python2.6/site.py", line 516, in main() File "/usr/local/Python-2.6.3/lib/python2.6/site.py", line 507, in main execsitecustomize() File "/usr/local/Python-2.6.3/lib/python2.6/site.py", line 472, in execsitecustomize import sitecustomize ... This also explains the recipe http://code.activestate.com/recipes/552729/ I wanted to have library location specific to application without having to explicitly change sys.path in all App-Top-Dir/bin/*.py. I thought creating bin/sitecustomize.py would do the trick. I guess the documentation should mention this fact. The comment in recipe 552729 is: Since Python 2.5 the automatic import of the module "sitecustomize.py" in the directory of the main program is not supported any more (even if the documentation says that it is). George From no.email at nospam.invalid Wed Jan 27 14:03:44 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 27 Jan 2010 11:03:44 -0800 Subject: Library support for Python 3.x References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> Message-ID: <7xk4v3mktr.fsf@ruckus.brouhaha.com> aahz at pythoncraft.com (Aahz) writes: > From my POV, your question would be precisely identical if you had > started your project when Python 2.3 was just released and wanted to > know if the libraries you selected would be available for Python 2.6. I didn't realize 2.6 broke libraries that had worked in 2.3, at least on any scale. Did I miss something? From exarkun at twistedmatrix.com Wed Jan 27 14:23:36 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Wed, 27 Jan 2010 19:23:36 -0000 Subject: Library support for Python 3.x In-Reply-To: <7xk4v3mktr.fsf@ruckus.brouhaha.com> References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> Message-ID: <20100127192336.1898.1244562725.divmod.xquotient.811@localhost.localdomain> On 07:03 pm, no.email at nospam.invalid wrote: >aahz at pythoncraft.com (Aahz) writes: >> From my POV, your question would be precisely identical if you had >>started your project when Python 2.3 was just released and wanted to >>know if the libraries you selected would be available for Python 2.6. > >I didn't realize 2.6 broke libraries that had worked in 2.3, at least >on >any scale. Did I miss something? Lots of tiny things change. Any of these can break a library. With the 3 releases between 2.3 and 2.6, there are lots of opportunities for these changes. I don't know what you mean by "any scale", but I know that I've seen lots of things break on 2.6 that worked on 2.3, 2.4, or 2.5. Jean-Paul From sjdevnull at yahoo.com Wed Jan 27 14:26:25 2010 From: sjdevnull at yahoo.com (sjdevnull at yahoo.com) Date: Wed, 27 Jan 2010 11:26:25 -0800 (PST) Subject: Library support for Python 3.x References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> Message-ID: <29a1c902-0fba-4736-8c24-649e768a5ef2@c29g2000yqd.googlegroups.com> On Jan 27, 2:03?pm, Paul Rubin wrote: > a... at pythoncraft.com (Aahz) writes: > > From my POV, your question would be precisely identical if you had > > started your project when Python 2.3 was just released and wanted to > > know if the libraries you selected would be available for Python 2.6. > > I didn't realize 2.6 broke libraries that had worked in 2.3, at least on > any scale. ?Did I miss something? I certainly had to update several modules I use (C extensions) to work with the new memory management in a recent release (changing PyMem_Del to Py_DECREF being a pretty common alteration); I can't remember whether that was for 2.6 or 2.5. From kyosohma at gmail.com Wed Jan 27 14:29:04 2010 From: kyosohma at gmail.com (Mike Driscoll) Date: Wed, 27 Jan 2010 11:29:04 -0800 (PST) Subject: Pyowa Meeting Next Monday Message-ID: <0834b458-e941-45d4-9be3-280daf97db46@21g2000yqj.googlegroups.com> Hi, Next Monday, February 1st, we will be having our first Pyowa meeting of 2010! It will be held at the Marshall County Sheriff's Office and start at 7 p.m. (barring inclement weather). I will be demoing some software I created for our Sheriff's office and will talk about the various modules used to put it all together. We'll probably have plenty of free time to chat and you can show off anything neat that you've been doing too. I hope to see you there! ----------------- Mike Driscoll Website: www.pyowa.org Twitter: www.twitter.com/pyowa From invalid at invalid.invalid Wed Jan 27 15:05:35 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Wed, 27 Jan 2010 20:05:35 +0000 (UTC) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: On 2010-01-27, Alf P. Steinbach wrote: > * Steve Holden: >> Alf P. Steinbach wrote: >> [...] >>> The main problem with the incompatibility is for porting code, not for >>> writing code from scratch. It's also a problem wrt. learning the >>> language. And I see no good reason for it: print can't really do more, >>> or less, or more conveniently (rather, one has to write a bit more now >>> for same effect). >> >> Of course it can do more: it allows you to layer your own print >> functionality into the system much more easily than you could with the >> print statement. > > Yeah, point. Very minor though. :-) I don't think it's minor at all. Being able to globally redefine the behavior of "print" for all modules is a big win when you want to import a module that's too chatty about what it's doing. The "right" way for modules to chatter about internals is using the logging module, but not everybody follows that convention. -- Grant Edwards grante Yow! Four thousand at different MAGNATES, MOGULS visi.com & NABOBS are romping in my gothic solarium!! From mcfletch at vrplumber.com Wed Jan 27 15:23:25 2010 From: mcfletch at vrplumber.com (Mike C. Fletcher) Date: Wed, 27 Jan 2010 15:23:25 -0500 Subject: Library support for Python 3.x In-Reply-To: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> Message-ID: <4B60A0BD.5050301@vrplumber.com> Kevin Walzer wrote: > I'm going to be starting some new Python projects in Python 2.6, but > am concerned that at least three of the libraries I will be > using--pycrypto, paramiko and feedparser--are not currently supported > in Python 3.x. The authors of these programs have not given any > indication that work is underway to support Python 3.x. Eventually I > plan to migrate to Python 3.x. > > I don't want to be stuck using libraries that may not be updated for > the next generation of Python. How are others handling this issue? > > --Kevin Some of us are saying: * When Python 3.x has something compelling to offer: o Users will start asking for Python 3.x with a *reason* to justify the cost o Libraries will begin to see the pain of porting, and most importantly, *maintaining*, as a reasonable trade-off o Libraries will port (and maintain) o Applications should *then* port * or When everyone is already on a (basically) compatible 2.x version (i.e. 2.6+); say in 3 years: o Conversion and maintenance costs will lower to the point where we can justify them for libraries Python 3.x has very little that is a compelling use-case (versus 2.x) for most people I've talked to. My paying customers aren't screaming "we want to spend a week of your time to rewrite, re-test and re-deploy our working code into Python 3.x so that it can do exactly the same thing with no visible benefit and lots of likely subtle failures". Unicode-as-default doesn't really make customers say "wow" when all their Unicode-needing code is already Unicode-using. A few syntax changes here and there... well, no, they certainly don't care (can't say I *really* do either). The initial marked slowdown for 3.0 (which I gather should be mostly fixed in 3.1) didn't do much of a sales-job either. Possible compelling arguments that would get people to convert (with the projects working on them): * 5x (or more) speedup (PyPy, Unladen Swallow) * adaptive parallelization/execution-kernel mechanism as a first-class primitive (Apple's C extensions for OpenCL) * continuation-like mechanisms, anonymous code blocks a-la Ruby (PyPy, Stackless) * (free) threading, GIL removal (IronPython, Jython) * compilation-to-native (PyPy) * compilation to mobile-platform native (PyPy?) None of those in in Python 3.x, though there's talk of merging Unladen Swallow into CPython to provide a carrot for conversions (though AFAIK it's not yet a 5x improvement across the board). To compound the problem, Python 3.x doesn't include any of the syntax changes you'd want to see to support e.g. anonymous blocks, continuations, OpenCL integration, etceteras... so if we want those, we're going to have to either introduce new syntax (which is blocked) or hack it in... which we could do on Python 2.x too. I don't know about other maintainers, but I've started running PyOpenGL tests with -3 on Python 2.6 to get a feel for how many issues are going to come up. Most have been minimal. But when I sit down and actually consider *maintaining* a 3.x release when I'm already *barely* able to keep up with the 2.x maintenance in my tiny amounts of free time... well... I do *that* stuff for *fun* after all, and double the headaches for no noticeable benefit doesn't really sound like fun... oh, and numpy isn't ported, so screw it ;) ... Interestingly, at PyGTA Myles mentioned that he'd found his first-ever Python 3.x-only library! Which he then converted to Python 2.x, because he actually wanted to use it in real code :) . Projects which haven't ported to Python 3.x aren't necessarily dead, they're just not nailed to that particular perch (yet), Mike -- ________________________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://www.vrplumber.com http://blog.vrplumber.com From awilliam at opengroupware.us Wed Jan 27 15:28:34 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Wed, 27 Jan 2010 15:28:34 -0500 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: <1264624114.3103.3.camel@linux-m3mt> On Wed, 2010-01-27 at 18:52 +0100, Alf P. Steinbach wrote: > * Steve Holden: > > Alf P. Steinbach wrote: > > [...] > >> The main problem with the incompatibility is for porting code, not for > >> writing code from scratch. It's also a problem wrt. learning the > >> language. And I see no good reason for it: print can't really do more, > >> or less, or more conveniently (rather, one has to write a bit more now > >> for same effect). > > Of course it can do more: it allows you to layer your own print > > functionality into the system much more easily than you could with the > > print statement. > Yeah, point. Very minor though. :-) So you get to determine that? I'd call the whole print thing (a) a legitimate change to increase consistency and (b) a fairly minor porting nuisance since application code [as in big-chunks-o-code] almost never contains print statements. I know at least two shops that have scripts they run on all Python code, prior to it entering production, to ensure there are no print statements. From timur at freescale.com Wed Jan 27 15:31:37 2010 From: timur at freescale.com (Timur Tabi) Date: Wed, 27 Jan 2010 14:31:37 -0600 Subject: I really need webbrowser.open('file://') to open a web browser In-Reply-To: <5A08CF3C-63E4-4103-B4B2-6C17E1D6B6D0@Comcast.net> References: <5A08CF3C-63E4-4103-B4B2-6C17E1D6B6D0@Comcast.net> Message-ID: On Wed, Jan 27, 2010 at 12:29 PM, Mitchell L Model wrote: > I had some discussions with the Python documentation writers that led to the > following note being included in the Python 3.1 library documentation for > webbrowser.open: "Note that on some platforms, trying to open a filename > using this function, may work and start the operating system?s associated > program. However, this is neither supported nor portable." Then they should have renamed the API. I appreciate that they're finally documenting this, but I still think it's a bunch of baloney. > You can control which browser opens the URL by using webbrowser.get to > obtain a controller for a particular browser, specified by its argument, > then call the open method on the controller instead of the module. How can I know which controller (application) the system will use when it opens an http URL? I depend on webbrowser.open('http') to choose the best web browser on the installed system. Does webbrowser.get() tell me which application that will be? > For opening files reliability and the ability to pick a particular program > (browser or otherwise) to open it with you might have to resort to invoking > a command line via subprocess.Popen. But that only works if I know which application to open. -- Timur Tabi Linux kernel developer at Freescale From nagle at animats.com Wed Jan 27 15:41:57 2010 From: nagle at animats.com (John Nagle) Date: Wed, 27 Jan 2010 12:41:57 -0800 Subject: Simple Password Strength Checker Review Help needed In-Reply-To: References: Message-ID: <4b60a16e$0$1670$742ec2ed@news.sonic.net> Mallikarjun(?????????????) wrote: > Dear friends, > I am newbie to Python + pygtk + Glade and recently wrote a simple password > strength checker app. > Since this is my first app/program, can someone review my code (just over > 150 lines) and help me improve my programming capabilities Here's my classic "Obvious password detector": http://www.animats.com/source/obvious/obvious.c This prevents dictionary attacks using an English dictionary, but needs only a small bit table and does no I/O. John Nagle From invalid at invalid.invalid Wed Jan 27 15:45:21 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Wed, 27 Jan 2010 20:45:21 +0000 (UTC) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On 2010-01-27, John Nagle wrote: > Arguably, Python 3 has been rejected by the market. Let's just say that it hasn't yet been accepted by the market. ;) > Instead, there's now Python 2.6, Python 2.7, and Python 2.8. > Python 3 has turned into a debacle like Perl 6, now 10 years > old. I think I'd have to wait a couple more years before making that sort of pronouncement. That said, I don't expect to start using Python 3 until library availability or my Linux distro forces me to. -- Grant Edwards grante Yow! Inside, I'm already at SOBBING! visi.com From nagle at animats.com Wed Jan 27 15:56:10 2010 From: nagle at animats.com (John Nagle) Date: Wed, 27 Jan 2010 12:56:10 -0800 Subject: myths about python 3 In-Reply-To: References: Message-ID: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Daniel Fetchinson wrote: > Hi folks, > > I was going to write this post for a while because all sorts of myths > periodically come up on this list about python 3. I don't think the > posters mean to spread false information on purpose, they simply are > not aware of the facts. > > My list is surely incomplete, please feel free to post your favorite > misconception about python 3 that people periodically state, claim or > ask about. Myths about Python 3: 1. Python 3 is supported by major Linux distributions. FALSE - most distros are shipping with Python 2.4, or 2.5 at best. 2. Python 3 is supported by multiple Python implementations. FALSE - Only CPython supports 3.x. Iron Python, Unladen Swallow, PyPy, and Jython have all stayed with 2.x versions of Python. 3. Python 3 is supported by most 3rd party Python packages. FALSE - it's not supported by MySQLdb, OpenSSL, feedparser, etc. Arguably, Python 3 has been rejected by the market. Instead, there's now Python 2.6, Python 2.7, and Python 2.8. Python 3 has turned into a debacle like Perl 6, now 10 years old. That's the reality, Python 3 fanboys. John Nagle From nagle at animats.com Wed Jan 27 16:03:04 2010 From: nagle at animats.com (John Nagle) Date: Wed, 27 Jan 2010 13:03:04 -0800 Subject: starting a thread in a nother thread In-Reply-To: <4b604df9$0$6723$9b4e6d93@newsspool2.arcor-online.net> References: <4b603b35$0$6725$9b4e6d93@newsspool2.arcor-online.net> <4b604df9$0$6723$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <4b60a661$0$1598$742ec2ed@news.sonic.net> Stefan Behnel wrote: > Richard Lamboj, 27.01.2010 15:23: >> Am Wednesday 27 January 2010 14:10:13 schrieb Stefan Behnel: >>> Richard Lamboj, 27.01.2010 14:06: >>>> just for _curiosity_. What would be if i start a thread in a nother >>>> thread and acquire a lock in the "child" thread. Is there anything that >>>> could go wrong if someone try to start threads in threads? >>> There's usually tons of things that can go wrong w.r.t. threads: >>> >>> http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf >>> >>> However, there's nothing special to a thread that was started from another >>> thread, so the problems don't change. >> i have tried a little bit around with psycopg2 and threads, >> >> I'am sharing one connection for all threads. When i'am starting the >> threads "normal" everything works without any Problem. When i'am starting the >> threads from another thread than i got a "segmentation fault" > > Sounds like a bug that you might want to report to the maintainers of psycopg2. > > Stefan If a C package called from Python crashes, the package is defective. Nothing you can do from Python should be able to cause a segmentation fault. Google search: "Results 1 - 10 of about 29,400 for psycopg2 crash". John Nagle From nagle at animats.com Wed Jan 27 16:04:48 2010 From: nagle at animats.com (John Nagle) Date: Wed, 27 Jan 2010 13:04:48 -0800 Subject: Library support for Python 3.x In-Reply-To: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> Message-ID: <4b60a6c9$0$1598$742ec2ed@news.sonic.net> Kevin Walzer wrote: > I'm going to be starting some new Python projects in Python 2.6, but am > concerned that at least three of the libraries I will be > using--pycrypto, paramiko and feedparser--are not currently supported in > Python 3.x. The authors of these programs have not given any indication > that work is underway to support Python 3.x. Eventually I plan to > migrate to Python 3.x. Maybe by 2015 or so, that might be feasible. Wait until Python 3.x ships as standard with major Linux distros. Right now, 2.4 or 2.5 is the production version of Python. John Nagle From awilliam at opengroupware.us Wed Jan 27 16:06:24 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Wed, 27 Jan 2010 16:06:24 -0500 Subject: myths about python 3 In-Reply-To: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <1264626384.3103.8.camel@linux-m3mt> On Wed, 2010-01-27 at 12:56 -0800, John Nagle wrote: > Daniel Fetchinson wrote: > > Hi folks, > > I was going to write this post for a while because all sorts of myths > > periodically come up on this list about python 3. I don't think the > > posters mean to spread false information on purpose, they simply are > > not aware of the facts. > > My list is surely incomplete, please feel free to post your favorite > > misconception about python 3 that people periodically state, claim or > > ask about. > Myths about Python 3: > 1. Python 3 is supported by major Linux distributions. > FALSE - most distros are shipping with Python 2.4, or 2.5 at best. CentOS: python26-2.6.4-1.ius.parallel.el5 openSUSE: python-2.6.2-6.3.i586, python3-3.1-3.3.i586 Darn, those pesky facts. > 2. Python 3 is supported by multiple Python implementations. > FALSE - Only CPython supports 3.x. Iron Python, Unladen Swallow, > PyPy, and Jython have all stayed with 2.x versions of Python. And of all Python development what percentage takes place on all those combined? 2%? Maybe. From sjdevnull at yahoo.com Wed Jan 27 16:14:23 2010 From: sjdevnull at yahoo.com (sjdevnull at yahoo.com) Date: Wed, 27 Jan 2010 13:14:23 -0800 (PST) Subject: myths about python 3 References: <6faf39c91001270245q155a932aj3e0bb98d377ddad1@mail.gmail.com> Message-ID: On Jan 27, 9:22?am, Daniel Fetchinson wrote: > >> Hi folks, > > >> I was going to write this post for a while because all sorts of myths > >> periodically come up on this list about python 3. I don't think the > >> posters mean to spread false information on purpose, they simply are > >> not aware of the facts. > > >> My list is surely incomplete, please feel free to post your favorite > >> misconception about python 3 that people periodically state, claim or > >> ask about. > > >> 1. Print statement/function creates incompatibility between 2.x and 3.x! > > >> Certainly false or misleading, if one uses 2.6 and 3.x the > >> incompatibility is not there. Print as a function works in 2.6: > > >> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) > >> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 > >> Type "help", "copyright", "credits" or "license" for more information. > >>>>> print( 'hello' ) > >> hello > >>>>> print 'hello' > >> hello > > >> 2. Integer division creates incompatibility between 2.x and 3.x! > > >> Again false or misleading, because one can get the 3.x behavior with 2.6: > > >> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) > >> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 > >> Type "help", "copyright", "credits" or "license" for more information. > >>>>> 6/5 > >> 1 > >>>>> from __future__ import division > >>>>> 6/5 > >> 1.2 > > >> Please feel free to post your favorite false or misleading claim about > >> python 3! > > > Well, I see two false or misleading claims just above - namely that > > the two claims above are false or misleading. They tell just half of > > the story, and that half is indeed easy. A Python 3 program can be > > unchanged (in the case of print) or with only trivial modifications > > (in the case of integer division) be made to run on Python 2.6. > > Okay, so we agree that as long as print and integer division is > concerned, a program can easily be written that runs on both 2.6 and > 3.x. > > My statements are exactly this, so I don't understand why you disagree. > > > The other way around this is _not_ the case. > > What do you mean? > > > To say that two things are > > compatible if one can be used for the other, but the other not for the > > first, is false or misleading. > > I'm not sure what you mean here. Maybe I didn't make myself clear > enough, but what I mean is this: as long as print and integer division > is concerned, it is trivial to write code that runs on both 2.6 and > 3.x. Hence if someone wants to highlight incompatibility (which surely > exists) between 2.6 and 3.x he/she has to look elsewhere. I think you're misunderstanding what "compatibility" means in a programming language context. Python 3 and Python 2 are not mutually compatible, as arbitrary programs written in one will not run in the other. The most important fallout of that is that Python 3 is not backwards compatible, in that existing Python 2 programs won't run unaltered in Python 3--while it's easy to write a new program in a subset of the languages that runs on both Python 2 and 3, the huge existing codebase of Python 2 code won't run under Python 3. That there exists an intersection of the two languages that is compatible with both doesn't make the two languages compatible with each other--although it being a fairly large subset does help mitigate a sizable chunk of the problems caused by incompatibility (as do tools like 2to3). In your example, a python2 program that uses print and division will fail in python3. The print problem is less significant, since the failure will probably be a syntax error or a rapid exception raised. The division problem is more problematic, since a program may appear to run fine but silently misbehave; such errors are much more likely to result in significant damage to data or other long-term badness. From nobody at nowhere.org Wed Jan 27 16:17:08 2010 From: nobody at nowhere.org (Georg) Date: Wed, 27 Jan 2010 22:17:08 +0100 Subject: C Structure rebuild with ctypes References: <7p6ksnFkg1U1@mid.individual.net><7qo9avFivmU1@mid.individual.net> Message-ID: <7sboqmFe9sU1@mid.individual.net> Hi Mark, many thanks for your hints. > --------------- func.py ------------------ > import ctypes as c > > # int func (int numVars, char **varNames, int *varTypes) > > INT = c.c_int > PINT = c.POINTER(INT) > PCHAR = c.c_char_p > PPCHAR = c.POINTER(PCHAR) > > func = c.CDLL('func').func > func.restype = None > func.argtypes = [INT,PPCHAR,PINT] > > numVars = 3 > varNames = (PCHAR * numVars)('abc','def','ghi') > varTypes = (INT * numVars)(1,2,3) > > func(numVars,varNames,varTypes) The function to call has the following signiture: int SetVarLabel(int handle, const char *varName, const char *varLabel) I did as you suggested by doing the following: INT = c_int PCHAR = c_char_p libc.argtypes = [INT,PCHAR,PCHAR] vn = create_string_buffer("var1") vl = create_string_buffer("Label") print "vn: ", vn.value print "vl: ", vl.value returnCode = libc.SetVarLabel(h,byref(vn), byref(vl)) print "Return Code: ", returnCode The return code is always "variable not found". Thus my call to the C library is not working. What do I do wrong? Best regards Georg From ben+python at benfinney.id.au Wed Jan 27 16:20:30 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 28 Jan 2010 08:20:30 +1100 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <878wbj4541.fsf@benfinney.id.au> John Nagle writes: > Myths about Python 3: > > 1. Python 3 is supported by major Linux distributions. > > FALSE - most distros are shipping with Python 2.4, or 2.5 at > best. There's a big difference between ?What list of versions of Python does ship with?? versus ?Which one version of Python does use for its implementation of ?python???. I think the latter is the more pertinent question. Do you have evidence to back up a claim (not that I'm saying you have necessarily made the claim; it's not clear from what you wrote) that most operating systems use Python 2.4 as their implementation of ?python?? -- \ ?Anyone who believes exponential growth can go on forever in a | `\ finite world is either a madman or an economist.? ?Kenneth | _o__) Boulding | Ben Finney From benjamin.kaplan at case.edu Wed Jan 27 16:25:46 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Wed, 27 Jan 2010 16:25:46 -0500 Subject: myths about python 3 In-Reply-To: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On Wed, Jan 27, 2010 at 3:56 PM, John Nagle wrote: > Daniel Fetchinson wrote: >> >> Hi folks, >> >> I was going to write this post for a while because all sorts of myths >> periodically come up on this list about python 3. I don't think the >> posters mean to spread false information on purpose, they simply are >> not aware of the facts. >> >> My list is surely incomplete, please feel free to post your favorite >> misconception about python 3 that people periodically state, claim or >> ask about. > > Myths about Python 3: > > 1. ?Python 3 is supported by major Linux distributions. > > ? ? ? ?FALSE - most distros are shipping with Python 2.4, or 2.5 at best. > The latest versions of Ubuntu Jaunty and Karmic, Fedora 11 and 12, and OpenSUSE 11.2 all use Python 2.6. Ubuntu has been shipping python 3 since Jaunty came out last April. According to Fedora's package index, Python 3 is in the devel version which probably means it will be in upcoming versions of Fedora as well. > 2. ?Python 3 is supported by multiple Python implementations. > > ? ? ? ?FALSE - Only CPython supports 3.x. ?Iron Python, Unladen Swallow, > ? ? ? ?PyPy, and Jython have all stayed with 2.x versions of Python. > When Python 2.6 came out, Jython was still on 2.2. The difference between 2.2 and 2.6 is almost as big of a difference as between 2.6 and 3.0. In that time, you had the introduction of the boolean type, generators, list comprehensions, the addition of the "yield" and "with" keywords, universal newline support, and decorators in addition to the large number of changes to the standard library such as the introduction of the subprocess module. > 3. ?Python 3 is supported by most 3rd party Python packages. > > ? ? ? ?FALSE - it's not supported by MySQLdb, OpenSSL, feedparser, etc. > > Arguably, Python 3 has been rejected by the market. ?Instead, there's > now Python 2.6, Python 2.7, and Python 2.8. ?Python 3 has turned into > a debacle like Perl 6, now 10 years old. > Give the package maintainers time to update. There were some pretty big changes to the C API. Most of the major 3rd party packages like numpy and MySQLdb have already commited to having a Python 3 version. They just haven't gotten them out yet. > That's the reality, Python 3 fanboys. > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?John Nagle > -- > http://mail.python.org/mailman/listinfo/python-list > From lists at cheimes.de Wed Jan 27 16:26:04 2010 From: lists at cheimes.de (Christian Heimes) Date: Wed, 27 Jan 2010 22:26:04 +0100 Subject: myths about python 3 In-Reply-To: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <4B60AF6C.3090304@cheimes.de> John Nagle wrote: > 1. Python 3 is supported by major Linux distributions. > > FALSE - most distros are shipping with Python 2.4, or 2.5 at best. You are wrong. Modern versions of Debian / Ubuntu are using Python 2.6. My Ubuntu box has python3.0, too. > 2. Python 3 is supported by multiple Python implementations. > > FALSE - Only CPython supports 3.x. Iron Python, Unladen Swallow, > PyPy, and Jython have all stayed with 2.x versions of Python. You are wrong again. We have an active discussion on the Python developer mailinglist to integrate Unladen Swallow into py3k, see http://jessenoller.com/2010/01/06/unladen-swallow-python-3s-best-feature/ . The other implementations have been planing Python 3.x support for more than a year. > > 3. Python 3 is supported by most 3rd party Python packages. > > FALSE - it's not supported by MySQLdb, OpenSSL, feedparser, etc. Python 3.x has builtin support for several aspects of OpenSSL. Other important projects like Cython, lxml, some Postgres DBAs, Django, mod_wsgi and many more have been ported to Python 3.x for more than a year. In fact Graham has helped us to identify several issues in 3.0 beta while he was porting mod_wsgi to Python 3.0. (Personally I neither see MySQL as mission critical nor as a powerful RDBMS. Just try to combine foreign keys with triggers and you'll see what I'm talking about. *scnr*) Christian From ben+python at benfinney.id.au Wed Jan 27 16:30:58 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 28 Jan 2010 08:30:58 +1100 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <874om744ml.fsf@benfinney.id.au> Adam Tauno Williams writes: > On Wed, 2010-01-27 at 12:56 -0800, John Nagle wrote: > > 2. Python 3 is supported by multiple Python implementations. > > FALSE - Only CPython supports 3.x. Iron Python, Unladen Swallow, > > PyPy, and Jython have all stayed with 2.x versions of Python. > > And of all Python development what percentage takes place on all those > combined? That's irrelevant to the point, though. Regardless of how small the minority of developers on the platform, it clearly matters to those developers which versions of Python they can use. -- \ ?When I get new information, I change my position. What, sir, | `\ do you do with new information?? ?John Maynard Keynes | _o__) | Ben Finney From ben+python at benfinney.id.au Wed Jan 27 16:50:12 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 28 Jan 2010 08:50:12 +1100 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <87zl3z2p63.fsf@benfinney.id.au> Christian Heimes writes: > John Nagle wrote: > > 1. Python 3 is supported by major Linux distributions. > > > > FALSE - most distros are shipping with Python 2.4, or 2.5 at best. > > You are wrong. Modern versions of Debian / Ubuntu are using Python > 2.6. Only if by ?modern? you mean ?not released yet?. The latest stable Debian (Debian 5.0, Lenny) has only Python 2.4 and Python 2.5. It does not have Python 2.6 at all, and until this month Python 2.6 was not even in the in-development suite of Debian. Fortunately, Debian Squeeze now finally has added Python 2.6 (though currently ?python? still uses Python 2.5). But Squeeze is currently a long way from being released. Python 3 is in the play-pen of Debian ?unstable?, but only arrived this week. It's even further from being released; it has to pass the filter from ?unstable? to ?testing? before it gets consideration for that. So I think your statement above is at least a mischaracterisation of the truth. -- \ ?As scarce as truth is, the supply has always been in excess of | `\ the demand.? ?Josh Billings | _o__) | Ben Finney From alfps at start.no Wed Jan 27 16:50:28 2010 From: alfps at start.no (Alf P. Steinbach) Date: Wed, 27 Jan 2010 22:50:28 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: * Adam Tauno Williams: > On Wed, 2010-01-27 at 18:52 +0100, Alf P. Steinbach wrote: >> * Steve Holden: >>> Alf P. Steinbach wrote: >>> [...] >>>> The main problem with the incompatibility is for porting code, not for >>>> writing code from scratch. It's also a problem wrt. learning the >>>> language. And I see no good reason for it: print can't really do more, >>>> or less, or more conveniently (rather, one has to write a bit more now >>>> for same effect). >>> Of course it can do more: it allows you to layer your own print >>> functionality into the system much more easily than you could with the >>> print statement. >> Yeah, point. Very minor though. :-) > > So you get to determine that? I'm sorry, I don't find that question meaningful. > I'd call the whole print thing (a) a legitimate change to increase > consistency and (b) a fairly minor porting nuisance since application > code [as in big-chunks-o-code] almost never contains print statements. > I know at least two shops that have scripts they run on all Python code, > prior to it entering production, to ensure there are no print > statements. Considering that in your opinion "application code [as in big-chunks-o-code] almost never contains print statements", is the point about being able to replace print, as you see it, more than a minor point? Cheers & sorry for not grokking your question, - Alf From MLMDev at comcast.net Wed Jan 27 17:00:45 2010 From: MLMDev at comcast.net (Mitchell L Model) Date: Wed, 27 Jan 2010 17:00:45 -0500 Subject: I really need webbrowser.open('file://') to open a web browser In-Reply-To: References: <5A08CF3C-63E4-4103-B4B2-6C17E1D6B6D0@Comcast.net> Message-ID: <813FC2BA-3F3D-4C08-B6F4-F44837CBB275@comcast.net> On Jan 27, 2010, at 3:31 PM, Timur Tabi wrote: > On Wed, Jan 27, 2010 at 12:29 PM, Mitchell L Model > wrote: > >> I had some discussions with the Python documentation writers that >> led to the >> following note being included in the Python 3.1 library >> documentation for >> webbrowser.open: "Note that on some platforms, trying to open a >> filename >> using this function, may work and start the operating system?s >> associated >> program. However, this is neither supported nor portable." > > Then they should have renamed the API. I appreciate that they're > finally documenting this, but I still think it's a bunch of baloney. I agree, but I am pretty sure that, based on the discussions I had with the Python documenters and developers, that there's no hope of winning this argument. I suppose that since a file: URL is not, strictly speaking, on the web, that it shouldn't be opened with a "web" browser. It's just that the "web" part of "web browser" became more or less obsolete a long time ago since there are so many more ways of using browsers and so many more things they can do then just browse the web. So if you interpret the name "webbrowser" to mean that it browses the web, as opposed to files, which means going through some kind of server-based protocol, the module does what it says. But I still like the idea of using it to open files, especially when I want the file to be opened by its associated application and not a browser. > >> You can control which browser opens the URL by using webbrowser.get >> to >> obtain a controller for a particular browser, specified by its >> argument, >> then call the open method on the controller instead of the module. > > How can I know which controller (application) the system will use when > it opens an http URL? I depend on webbrowser.open('http') to choose > the best web browser on the installed system. Does webbrowser.get() > tell me which application that will be? webbrowser.get() with no arguments gives you the default kind of browser controller, just as if you had used webbrowser.open() directly. > >> For opening files reliability and the ability to pick a particular >> program >> (browser or otherwise) to open it with you might have to resort to >> invoking >> a command line via subprocess.Popen. > > But that only works if I know which application to open. Aha. You could use subprocess to specify the application from within your Python code, but not to indicate "the user's default browser", unless the platform has a command for that. On OS X, for instance, the command line: open file.html opens file.html with the application the user has associated with html files, whereas open -a safari file.html will open it with Safari even if the user has chosen Firefox for html files. There's stuff like this for Windows, I suppose, but hardly as convenient. And I think that Linux environments are all over the place on this, but I'm not sure. webbrowser.get() returns a control object of the default class for the user's environment -- the one that means "use the default browser" so it won't help. From awilliam at opengroupware.us Wed Jan 27 17:00:56 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Wed, 27 Jan 2010 17:00:56 -0500 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <1264629656.3103.10.camel@linux-m3mt> On Wed, 2010-01-27 at 16:25 -0500, Benjamin Kaplan wrote: > On Wed, Jan 27, 2010 at 3:56 PM, John Nagle wrote: > Give the package maintainers time to update. There were some pretty > big changes to the C API. Most of the major 3rd party packages like > numpy and MySQLdb have already commited to having a Python 3 version. > They just haven't gotten them out yet. PostgreSQL support is available for Python3. Just switch to using a real database and one of you problems is solved. :) From pavlovevidence at gmail.com Wed Jan 27 17:07:15 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Wed, 27 Jan 2010 14:07:15 -0800 (PST) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> On Jan 27, 12:56?pm, John Nagle wrote: > Arguably, Python 3 has been rejected by the market. No it's not fathomably arguable, because there's no reasonable way that Python 3 could have fully replaced Python 2 so quickly. At best, you could reasonably argue there hasn't been enough time to tell. >?Instead, there's > now Python 2.6, Python 2.7, and Python 2.8. It was always the plan to continue developing Python 2.x alongside Python 3.x during the transition period. Last I heard, don't remember where, the plan was for Python 2.7 to be the last version in the Python 2 line. If that's true, Python 3 acceptance is further along at this point than anticipated, since they originally thought they might have to go up to 2.9. >?Python 3 has turned into > a debacle like Perl 6, now 10 years old. Perl 6 has never been released. The situations aren't even comparable. > That's the reality, Python 3 fanboys. You're the fanboy, fanboi. You are so hellbent on badmouthing Python 3 that you throw three ridiculous, straw-grasping arguments at us. Here's the real reality. Python 3 is going to replace Python 2, and it has nothing to do with technical merit. The developers are planning to stop development on 2.x line, and only continue with 3.x, so anyone who wants to stay current--which is most people--with Python will have to use 3.x. There is no hope that developers will be pressured by the market to change their plans; we would have seen enough of a backlash by now. There is also no hope someone will fork Python 2.x and continue it in perpetuity. Well, someone might try to fork it, but they won't be able to call it Python. No, don't be silly, a fork of Python not called Python won't gain market share. So rail if it makes you feel better but you've already lost. Carl Banks From tjreedy at udel.edu Wed Jan 27 17:15:10 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 27 Jan 2010 17:15:10 -0500 Subject: Ad hoc lists vs ad hoc tuples In-Reply-To: References: <8a57c64f-0c67-4ec1-b4be-34d76c7e7571@q4g2000yqm.googlegroups.com> Message-ID: On 1/27/2010 12:32 PM, Antoine Pitrou wrote: > Le Wed, 27 Jan 2010 02:20:53 -0800, Floris Bruynooghe a ?crit : >> >> Is a list or tuple better or more efficient in these situations? > > Tuples are faster to allocate (they are allocated in one single step) and > quite a bit smaller too. > In some situations, in Python 2.7 and 3.1, they can also be ignored by > the garbage collector, yielding faster collections. > > (not to mention that they are hashable, which can be useful) Constant tuples (a tuple whose members are all seen as constants by the compiler) are now pre-compiled and constructed once and put into the code object as such rather than re-constructed with each run of the code. >>> from dis import dis >>> def l(): return [1,2,3] >>> def t(): return 1,2,3 >>> dis(l) 1 0 LOAD_CONST 1 (1) 3 LOAD_CONST 2 (2) 6 LOAD_CONST 3 (3) 9 BUILD_LIST 3 12 RETURN_VALUE >>> dis(t) 1 0 LOAD_CONST 4 ((1, 2, 3)) 3 RETURN_VALUE >>> # 3.1 Terry Jan Reedy From exarkun at twistedmatrix.com Wed Jan 27 17:19:00 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Wed, 27 Jan 2010 22:19:00 -0000 Subject: myths about python 3 In-Reply-To: <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> Message-ID: <20100127221900.1898.256361359.divmod.xquotient.848@localhost.localdomain> On 10:07 pm, pavlovevidence at gmail.com wrote: >On Jan 27, 12:56?pm, John Nagle wrote: >>Arguably, Python 3 has been rejected by the market. > >No it's not fathomably arguable, because there's no reasonable way >that Python 3 could have fully replaced Python 2 so quickly. > >At best, you could reasonably argue there hasn't been enough time to >tell. >>?Instead, there's >>now Python 2.6, Python 2.7, and Python 2.8. > >It was always the plan to continue developing Python 2.x alongside >Python 3.x during the transition period. > >Last I heard, don't remember where, the plan was for Python 2.7 to be >the last version in the Python 2 line. If that's true, Python 3 >acceptance is further along at this point than anticipated, since they >originally thought they might have to go up to 2.9. This assumes that the decision to stop making new 2.x releases is based on Python 3 adoption, rather than on something else. As far as I can tell, it's based on the personal desire of many of the core developers to stop bothering with 2.x. In other words, it's more a gauge of adoption of Python 3 amongst Python core developers. Jean-Paul From paul at boddie.org.uk Wed Jan 27 17:19:32 2010 From: paul at boddie.org.uk (Paul Boddie) Date: Wed, 27 Jan 2010 14:19:32 -0800 (PST) Subject: I really need webbrowser.open('file://') to open a web browser References: <5A08CF3C-63E4-4103-B4B2-6C17E1D6B6D0@Comcast.net> Message-ID: <38cbe4b0-3a23-4c68-a2ef-d7d7acec3270@c29g2000yqd.googlegroups.com> On 27 Jan, 23:00, Mitchell L Model wrote: > > I suppose that since a file: URL is not, strictly speaking, on the ? > web, that it shouldn't be opened with a "web" browser. But anything with a URL is (or should be regarded as being) on the Web. It may not be anything more than a local resource and thus have no universal or common meaning - someone else may not be able to resolve the URL to a file at all, or it may resolve to a different file - but it's part of the Web as observed by one party. Paul From missive at hotmail.com Wed Jan 27 17:24:01 2010 From: missive at hotmail.com (Lee Harr) Date: Thu, 28 Jan 2010 02:54:01 +0430 Subject: some turtle questions Message-ID: > I am trying to think of things to do with the turtle module > 1) is there a way to determine the current screen pixel color? This would not use the included turtle module, but you could use the turtle module from the pygsear collection: http://www.nongnu.org/pygsear/ It requires pygame, but if you have that installed already it is pretty easy to get it working. Code to get a drawn color would look like: >>> from pygsear.Drawable import Turtle >>> t=Turtle() >>> t.bg.get_at((0,0)) (0, 0, 0, 255) >>> t.forward(100) >>> t.position [400.0, 200.0] >>> t.bg.get_at((400,300)) (255, 255, 255, 255) >>> t.left(90) >>> t.set_color((255,0,0)) >>> t.forward(100) >>> t.position [300.0, 200.0] >>> t.bg.get_at((350,200)) (255, 0, 0, 255) >>> > 2) is there a way to put a limit on the extent the turtle can > travel? it seems I can keep moving off of the screen. You could write a subclass for interactive use. Maybe like this: from pygsear.Drawable import Turtle class T2(Turtle): ??? def forward(self, dist): ??????? Turtle.forward(self, dist) ??????? if not self.onscreen(slack=10): ??????????? self.backward(dist) ... but I'm not sure that's the best approach. Often when doing something interesting the turtle will go offscreen for a bit and come back later to finish up the visible portion of his drawing. _________________________________________________________________ Hotmail: Free, trusted and rich email service. https://signup.live.com/signup.aspx?id=60969 From mensanator at aol.com Wed Jan 27 17:24:38 2010 From: mensanator at aol.com (Mensanator) Date: Wed, 27 Jan 2010 14:24:38 -0800 (PST) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On Jan 27, 2:56?pm, John Nagle wrote: > Daniel Fetchinson wrote: > > Hi folks, > > > I was going to write this post for a while because all sorts of myths > > periodically come up on this list about python 3. I don't think the > > posters mean to spread false information on purpose, they simply are > > not aware of the facts. > > > My list is surely incomplete, please feel free to post your favorite > > misconception about python 3 that people periodically state, claim or > > ask about. > > Myths about Python 3: > > 1. ?Python 3 is supported by major Linux distributions. > > ? ? ? ? FALSE - most distros are shipping with Python 2.4, or 2.5 at best. So? I use Mac OSX 10.6, not Linux. And that comes with 2.6. Nothing stopped me from adding 3.1. > > 2. ?Python 3 is supported by multiple Python implementations. > > ? ? ? ? FALSE - Only CPython supports 3.x. ?Iron Python, Unladen Swallow, > ? ? ? ? PyPy, and Jython have all stayed with 2.x versions of Python. So? I only use CPython. > > 3. ?Python 3 is supported by most 3rd party Python packages. > > ? ? ? ? FALSE - it's not supported by MySQLdb, OpenSSL, feedparser, etc. So? The only 3rd party module I use is gmpy, and that's been updated to 3.x. > > Arguably, Python 3 has been rejected by the market. ?Instead, there's > now Python 2.6, Python 2.7, and Python 2.8. ?Python 3 has turned into > a debacle like Perl 6, now 10 years old. > > That's the reality, Python 3 fanboys. Maybe in *your* world. I'm perfectly happy in my world using 3.1. > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? John Nagle From tjreedy at udel.edu Wed Jan 27 17:34:23 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 27 Jan 2010 17:34:23 -0500 Subject: myths about python 3 In-Reply-To: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On 1/27/2010 3:56 PM, John Nagle wrote: > 2. Python 3 is supported by multiple Python implementations. > > FALSE - Only CPython supports 3.x. Iron Python, Unladen Swallow, > PyPy, and Jython have all stayed with 2.x versions of Python. Actually, Unladen Swallow is now targeted at 3.1; its developers have conservatively proposed its integration in CPython 3.3. I would not be completely shocked if it happens in 3.2. > Arguably, Python 3 has been rejected by the market. Almost everything is 'arguable'. Based on experience, Guido never expected major uptake until 3.2 (a year away). > Instead, there's now Python 2.6, Just who produced that? and why? > Python 2.7, Does not exist yet, but again, coming from the same devs for the purpose of helping transition to 3.x. > and Python 2.8. Unlikely to ever exist. > Python 3 has turned into a debacle like Perl 6, now 10 years old. You have to wait another 9 years to really say that. However, my impression is that Python 3 alreays surpasses Perl 6. > That's the reality, Python 3 fanboys. Why are you such a Python 3 hateboy? Terry Jan Reedy From tjreedy at udel.edu Wed Jan 27 17:38:31 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 27 Jan 2010 17:38:31 -0500 Subject: Library support for Python 3.x In-Reply-To: <7xk4v3mktr.fsf@ruckus.brouhaha.com> References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> Message-ID: On 1/27/2010 2:03 PM, Paul Rubin wrote: > aahz at pythoncraft.com (Aahz) writes: >> From my POV, your question would be precisely identical if you had >> started your project when Python 2.3 was just released and wanted to >> know if the libraries you selected would be available for Python 2.6. > > I didn't realize 2.6 broke libraries that had worked in 2.3, at least on > any scale. Did I miss something? For a windows user who depends on pre-built binaries, every new release breaks *every* library that is not pure Python and needs to be compiled. From dmalcolm at redhat.com Wed Jan 27 17:58:54 2010 From: dmalcolm at redhat.com (David Malcolm) Date: Wed, 27 Jan 2010 17:58:54 -0500 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <1264633134.5245.198.camel@brick> On Wed, 2010-01-27 at 16:25 -0500, Benjamin Kaplan wrote: > On Wed, Jan 27, 2010 at 3:56 PM, John Nagle wrote: > > Daniel Fetchinson wrote: > >> > >> Hi folks, > >> > >> I was going to write this post for a while because all sorts of myths > >> periodically come up on this list about python 3. I don't think the > >> posters mean to spread false information on purpose, they simply are > >> not aware of the facts. > >> > >> My list is surely incomplete, please feel free to post your favorite > >> misconception about python 3 that people periodically state, claim or > >> ask about. > > > > Myths about Python 3: > > > > 1. Python 3 is supported by major Linux distributions. > > > > FALSE - most distros are shipping with Python 2.4, or 2.5 at best. > > > The latest versions of Ubuntu Jaunty and Karmic, Fedora 11 and 12, and > OpenSUSE 11.2 all use Python 2.6. Ubuntu has been shipping python 3 > since Jaunty came out last April. According to Fedora's package index, > Python 3 is in the devel version which probably means it will be in > upcoming versions of Fedora as well. FWIW, more information on Fedora python 3 status here: https://fedoraproject.org/wiki/Features/Python3F13 [snip] > Give the package maintainers time to update. There were some pretty > big changes to the C API. Most of the major 3rd party packages like > numpy and MySQLdb have already commited to having a Python 3 version. > They just haven't gotten them out yet. I'll take this opportunity to make a shameless plug for my 2to3c tool: http://dmalcolm.livejournal.com/3935.html which takes some of the grunt work out of taking C code and making it compilable against both 2 and 3. (it will still require a human to do some of the work, alas). Hope this is helpful Dave From john at castleamber.com Wed Jan 27 18:03:12 2010 From: john at castleamber.com (John Bokma) Date: Wed, 27 Jan 2010 17:03:12 -0600 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> <878wbk1c4q.fsf@castleamber.com> <0c142d80-5f54-4b63-a8ce-f826ec9ad82d@r19g2000yqb.googlegroups.com> Message-ID: <87zl3z40cv.fsf@castleamber.com> Xah Lee writes: > Someone is badmouthing me, and it has been doing so over the years. I > feel obliged to post a off topic relpy. See: > > ? DreamHost.com and A Incidence of Harassment > http://xahlee.org/Periodic_dosage_dir/t2/harassment.html Dreamhost is not the only ISP that has agreed with me in the past regarding your spamming. And yes, it's spamming if you copy paste an article from your web site with the link to that site and cross post it to as many groups as your Usenet provider allows for. Thank goodness that GG has a limit on how many groups you can spam at a time. Post to on-topic groups only, really on-topic groups, not as many as you can select. And participate like you know do, and nobody will complain about /how/ you post. And if you cross-post /set a follow-up-to/. That's netiquette. -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From jgardner at jonathangardner.net Wed Jan 27 18:29:05 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Wed, 27 Jan 2010 15:29:05 -0800 (PST) Subject: Python and Ruby References: Message-ID: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> On Jan 27, 5:47?am, Simon Brunning wrote: > > I think Python is a little cleaner, but I'm sure you'd find Ruby fans > who'd argue the complete opposite. > Are you sure about that? There's a lot of line noise in Ruby. How are you supposed to pronounce "@@"? What about "{|..| ... }"? There's a lot of "magic" in Ruby as well. For instance, function calls are made without parentheses. Blocks can only appear as the first argument. There's a lot more, if you put your mind to it. Indentation is also optional in Ruby. You can quickly fool a newbie by not indenting your code properly, which is impossible in Python. Python is much, much cleaner. I don't know how anyone can honestly say Ruby is cleaner than Python. From jgardner at jonathangardner.net Wed Jan 27 18:31:18 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Wed, 27 Jan 2010 15:31:18 -0800 (PST) Subject: Python and Ruby References: Message-ID: <43cad4e7-6b54-4bd9-9ca2-217cb109560f@g29g2000yqe.googlegroups.com> On Jan 27, 6:56?am, Roald de Vries wrote: > On Jan 27, 2010, at 2:01 PM, Jean Guillaume Pyraksos wrote: > > > What are the arguments for choosing Python against Ruby > > for introductory programming? > > I think the main difference is in culture, especially for ? > *introductory* programming. To add to that, Python is the type of language where experienced programmers can pick it up by reading code, and newbies won't get hopelessly lost. I've taught less-than-formal introductory programming classes to people who are new to programming. Python gets out of the way, and allows you to focus on the programming concepts, such as variable, functions, parameters, classes, and algorithms. From fetchinson at googlemail.com Wed Jan 27 18:38:14 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Thu, 28 Jan 2010 00:38:14 +0100 Subject: myths about python 3 In-Reply-To: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: >> Hi folks, >> >> I was going to write this post for a while because all sorts of myths >> periodically come up on this list about python 3. I don't think the >> posters mean to spread false information on purpose, they simply are >> not aware of the facts. >> >> My list is surely incomplete, please feel free to post your favorite >> misconception about python 3 that people periodically state, claim or >> ask about. > > Myths about Python 3: I said myths, not facts! :) s/Myths/Facts/ > 1. Python 3 is supported by major Linux distributions. > > FALSE - most distros are shipping with Python 2.4, or 2.5 at best. This latter statement is false, Fedora 11 and 12 come with python 2.6. > 2. Python 3 is supported by multiple Python implementations. > > FALSE - Only CPython supports 3.x. Iron Python, Unladen Swallow, > PyPy, and Jython have all stayed with 2.x versions of Python. This latter statement is false, unladen swallow is currently targeting 3.x and may even be merged into cpython 3.x. > 3. Python 3 is supported by most 3rd party Python packages. > > FALSE - it's not supported by MySQLdb, OpenSSL, feedparser, etc. s/most/my favorite/ Actually, tons of 3rd party packages are already ported, postgres and django are just 2 examples, the next release of PIL will be another. > Arguably, Python 3 has been rejected by the market. Instead, there's > now Python 2.6, Python 2.7, and Python 2.8. Python 3 has turned into > a debacle like Perl 6, now 10 years old. These are the kinds of myths I had in mind when starting the thread. All sorts of BS is kept circulating without any facts to back up the claims. Actually, in this thread tons of rebuttals can be found to your statements but I doubt you will change your mind :) Hopefully others reading all of this will at least see what is BS and what is the actual situation about python 3. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From falk at mauve.rahul.net Wed Jan 27 18:50:19 2010 From: falk at mauve.rahul.net (Edward A. Falk) Date: Wed, 27 Jan 2010 23:50:19 +0000 (UTC) Subject: myths about python 3 References: Message-ID: In article , Daniel Fetchinson wrote: >Hi folks, > >1. Print statement/function creates incompatibility between 2.x and 3.x! > >Certainly false or misleading, if one uses 2.6 and 3.x the >incompatibility is not there. Print as a function works in 2.6: Yes, but does print as a statement work? Your argument is that python 2.x code can be ported to 3.x and still run under 2.6 if you're careful about how you do the port. That's not the same as saying they're compatible. >2. Integer division creates incompatibility between 2.x and 3.x! Same as above. Saying you can make it work by rewriting your code is not the same as saying that it works. If they'd wanted wide adoption of python 3, they should have made it as compatible as possible with python 2 code. If they dropped the print statement, then they did not do so. -- -Ed Falk, falk at despams.r.us.com http://thespamdiaries.blogspot.com/ From jgardner at jonathangardner.net Wed Jan 27 18:51:15 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Wed, 27 Jan 2010 15:51:15 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> Message-ID: <642d67e6-781b-4157-b917-449139854bad@b9g2000yqd.googlegroups.com> On Jan 27, 9:38?am, Luis M. Gonz?lez wrote: > > Please don't post more noise and ad hominem attacks to the group, Steve. > > "Ad hominem"? > Please, operor non utor lingua non notus per vulgaris populus. > Gratias ago vos... My rough, machine-assisted translation: "Don't try to use language that the people don't know. Thanks." From falk at mauve.rahul.net Wed Jan 27 18:52:07 2010 From: falk at mauve.rahul.net (Edward A. Falk) Date: Wed, 27 Jan 2010 23:52:07 +0000 (UTC) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: In article , Grant Edwards wrote: > >That said, I don't expect to start using Python 3 until library >availability or my Linux distro forces me to. If python 3 is much more efficient than python 2, or it has features I really need for some application I'll write in the future, I might be tempted to switch. Maybe some future version of python 3 will be compatible with python 2 source code. That would help. -- -Ed Falk, falk at despams.r.us.com http://thespamdiaries.blogspot.com/ From jgardner at jonathangardner.net Wed Jan 27 18:54:23 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Wed, 27 Jan 2010 15:54:23 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: On Jan 26, 10:12?pm, Steven D'Aprano wrote: > > I did too, when I first heard cmp was to be dumped. But I changed my mind > and now agree with the decision to drop cmp. Custom sorts are nearly > always much better written with key rather than cmp: key adds an O(N) > overheard to the sorting, while cmp makes sorting O(N**2). In other > words, key is *much* faster, and generally easier to write as well. > Imagine what sort() could do if all the keys were platform-native types. In other words, if you could bypass the Python compare because all the keys were native int, char, char*, or float. You can't do that with cmp either. From no.email at nospam.invalid Wed Jan 27 18:54:55 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 27 Jan 2010 15:54:55 -0800 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: <7xvdenje7k.fsf@ruckus.brouhaha.com> Steven D'Aprano writes: > always much better written with key rather than cmp: key adds an O(N) > overheard to the sorting, while cmp makes sorting O(N**2). Whaaaaaaaaaa ...... ???? No I don't think so. From cookiecaper at gmail.com Wed Jan 27 19:00:57 2010 From: cookiecaper at gmail.com (cookiecaper) Date: Wed, 27 Jan 2010 16:00:57 -0800 (PST) Subject: Updating form action attribute with mechanize.Browser Message-ID: <662d367b-267e-4c9a-b53a-201b64a036c4@r19g2000yqb.googlegroups.com> Hi all. I am using mechanize.Browser to download web pages and such. One of these has the wrong URL in the action attribute of the form I need to submit. Anyone know how I can update this quickly? ClientForm.__init__ seems to take the action attribute but that is passed by b.select_form(), I think. From jgardner at jonathangardner.net Wed Jan 27 19:12:04 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Wed, 27 Jan 2010 16:12:04 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> Message-ID: On Jan 27, 12:36?am, Paul Rubin wrote: > Steven D'Aprano writes: > > Without becoming a purely functional language, you won't get rid of all > > statements. > > Why not? ?GCC lets you use any statement in an expression: > > ? ? #include > > ? ? main() > ? ? { > ? ? ? int i, x, p=0; > ? ? ? x = ({ for (i=1; i<=10; i++) p += i; p;}); > ? ? ? printf ("x=%d\n", x); > ? ? } > > and C is certainly not a purely functional language. > > > for, while, if, try, break, yield, return > > are all used for flow control, and should remain as statements. > > What about assert, import, and pass? I think you missed the point about what a functional language is and is not. Let me see if I can help clear up that confusion a bit. In a functional language, you don't use statements. For instance, when you declare a new function, you call the function-declaring-function with parameters describing the parameters and body of that function. In such a language, flow control is very, very strange for those not familiar with it. For instance, how can you call an "if" function that only evaluates one or the other branches? In such a case, you're not making a function call, at least not as you know it. What you're doing is calling a special kind of function that doesn't evaluate its parameters until later, if at all. With the above paradigm, you have to start thinking about the programming task in a completely different light. Rather than organizing your code into a sequence of evaluated statements, you organize it into one giant function that will evaluate its parameters and return some result. This is a difficult paradigm for programmers to grasp, and is one reason why Lisp and other functional languages aren't know for being newbie-friendly. Whether or not a statement can be used as an expression is really orthogonal to this conversation. If "yield", "break", and "continue" were functions, then you could "call" them from within another function and have the call-ee yield, break, or continue. For instance: def do_something(): break() while 1: do_something() # The next line is never reached do_something_else() This could be useful for some situations, but it is really, really confusing and so it's not allowed. Such a function would have to rely on some type of magic so that it is correctly identified as a yield, break, or continue, and not a typical function call. "import" is another case. It does something very special. It assigns to values in the namespace of the code from which it was called. No function can do that, at least not without some weird magic. It's better to leave it as a statement and teach new users that it is very, very special, than to redefine how functions work with the call-ees namespace, or make it a common task to muck with such things. If "while", "for", "try", "if", "def", and "class" were functions, they would have to be written like so: if(expr, if-true-expr, if-false-expr) Note, however, that you can't execute if-true of if-false until AFTER the if() function is called. That means either we re-define what it means to call a function, or we have some notation that means "Don't execute this yet". Either way, it's really, really confusing. Also, note that statements allow you to combine many statements into one. Python's rule is that you can't pass statements to a function, only expression, so you'd have to re-define how that works to allow some function that would combine multiple statements together. In the end, print is a great candidate for a function, while the others are most definitely not. Adding any of them as functions would mean other, fundamental aspects of Python would have to be re-defined as well. I hope this clears up Steve's point about functional languages. From ethan at stoneleaf.us Wed Jan 27 19:14:53 2010 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 27 Jan 2010 16:14:53 -0800 Subject: myths about python 3 In-Reply-To: References: <6faf39c91001270245q155a932aj3e0bb98d377ddad1@mail.gmail.com> Message-ID: <4B60D6FD.4070708@stoneleaf.us> Daniel Fetchinson wrote: >>> Hi folks, >>> >>> I was going to write this post for a while because all sorts of myths >>> periodically come up on this list about python 3. I don't think the >>> posters mean to spread false information on purpose, they simply are >>> not aware of the facts. >>> >>> My list is surely incomplete, please feel free to post your favorite >>> misconception about python 3 that people periodically state, claim or >>> ask about. >>> >>> 1. Print statement/function creates incompatibility between 2.x and 3.x! >>> >>> Certainly false or misleading, if one uses 2.6 and 3.x the >>> incompatibility is not there. Print as a function works in 2.6: >>> >>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>>>>> print( 'hello' ) >>> hello >>>>>> print 'hello' >>> hello >>> >>> 2. Integer division creates incompatibility between 2.x and 3.x! >>> >>> Again false or misleading, because one can get the 3.x behavior with 2.6: >>> >>> Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) >>> [GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>>>>> 6/5 >>> 1 >>>>>> from __future__ import division >>>>>> 6/5 >>> 1.2 >>> >>> >>> Please feel free to post your favorite false or misleading claim about >>> python 3! >> Well, I see two false or misleading claims just above - namely that >> the two claims above are false or misleading. They tell just half of >> the story, and that half is indeed easy. A Python 3 program can be >> unchanged (in the case of print) or with only trivial modifications >> (in the case of integer division) be made to run on Python 2.6. > > Okay, so we agree that as long as print and integer division is > concerned, a program can easily be written that runs on both 2.6 and > 3.x. > > My statements are exactly this, so I don't understand why you disagree. > >> The other way around this is _not_ the case. > > What do you mean? > >> To say that two things are >> compatible if one can be used for the other, but the other not for the >> first, is false or misleading. > > I'm not sure what you mean here. Maybe I didn't make myself clear > enough, but what I mean is this: as long as print and integer division > is concerned, it is trivial to write code that runs on both 2.6 and > 3.x. Hence if someone wants to highlight incompatibility (which surely > exists) between 2.6 and 3.x he/she has to look elsewhere. > > Cheers, > Daniel > I think what Andre is saying is that you can't get 2.x behavior in 3.x, only the other way 'round. In the integer division instance, the 2.x behavior of 6/5 = 1 is not going to happen in 3.x. ~Ethan~ From jgardner at jonathangardner.net Wed Jan 27 19:16:59 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Wed, 27 Jan 2010 16:16:59 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xvdenje7k.fsf@ruckus.brouhaha.com> Message-ID: <5b9e69bc-b261-4e81-84bc-045936011e82@o9g2000yqa.googlegroups.com> On Jan 27, 3:54?pm, Paul Rubin wrote: > Steven D'Aprano writes: > > always much better written with key rather than cmp: key adds an O(N) > > overheard to the sorting, while cmp makes sorting O(N**2). > > Whaaaaaaaaaa ...... ???? ?No I don't think so. You're referring to the O(N**2) bit, right? I am sure he knew it was O (N*log(N)), which is still worse than O(N) for key. If he didn't, well, Python has some fundamental flaws in its basic sort algorithm. From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 19:19:24 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 28 Jan 2010 00:19:24 GMT Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On Wed, 27 Jan 2010 12:56:10 -0800, John Nagle wrote: > Daniel Fetchinson wrote: >> Hi folks, >> >> I was going to write this post for a while because all sorts of myths >> periodically come up on this list about python 3. I don't think the >> posters mean to spread false information on purpose, they simply are >> not aware of the facts. >> >> My list is surely incomplete, please feel free to post your favorite >> misconception about python 3 that people periodically state, claim or >> ask about. > > Myths about Python 3: > > 1. Python 3 is supported by major Linux distributions. > 2. Python 3 is supported by multiple Python implementations. > 3. Python 3 is supported by most 3rd party Python packages. A myth actually needs to be believed by some sector of the population, even if a small one. (Isolated nut cases don't count.) Star Wars is not a myth, because nobody -- not even those wacky people who put down "Jedi" as their religion on census forms -- actually believes it is a documentary. I've never heard anyone claiming any of those three "myths". I conclude that you just made them up, that they are fictional claims rather than genuine myths mistakenly believed by some people. So, in that spirit, here are three more for your collection. Perhaps you could start a list on a website somewhere. 4. Python 3 will make you irresistible to women. FALSE - Python 3 coders are no more likely to get a date than any other programmer. 5. Python 3 programs cannot be buggy, no matter how poorly you code. FALSE - programs written in Python 3 can contain bugs. 6. The code for Python 3 was handed down to Guido from the Heavens, carved into stone tablets by the Gods themselves. FALSE - Python 3 was designed and written by fallible human beings, and consequently there is no guarantee that it will be perfect in all ways for all purposes. > That's the reality, Python 3 fanboys. Speaks for itself. -- Steven From no.email at nospam.invalid Wed Jan 27 19:25:20 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 27 Jan 2010 16:25:20 -0800 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> Message-ID: <7xockfjcsv.fsf@ruckus.brouhaha.com> Jonathan Gardner writes: >> What about assert, import, and pass? >... > For instance, how can you call an "if" function ... > If "yield", "break", and "continue" were functions, ... > "import" ... does something very special. It assigns > to values in the namespace of the code from which it was called. > In the end, print is a great candidate for a function, Reasonable point about import. "If", "yield", "break", and "continue" are irrelevant if one accepts the flow control argument, although there is already expression syntax for "if" and "yield". What about assert and pass? > Also, note that statements allow you to combine many statements into > one. Python's rule is that you can't pass statements to a function, But if we're going around introducing changes that break things, why not get rid of that silly restriction? I don't mind that 3.x is breaking stuff for the sake of improving things. That's the whole idea of 3.x, after all. What bugs me is that the improvements are mostly quite superficial, and the breakage seems often gratuitous. I'd rather see much more drastic changes and correspondingly much larger improvements. Right now I rent an apartment in a city where real estate values have dropped a couple percent in the past few years. So in principle, I could move to a 2% bigger apartment while paying the same rent, or move to a similarly sized apartment while paying 2% less. Either of those is objectively an improvement, but am I going to do it? Hell no--moving even down the same street is way too big a hassle to even consider doing it for such a tiny gain. But if the "2%" were replaced by 3x or 5x, the proposition would be a heck of a lot more attractive, even if I had to move a larger distance. That's sort of the way I feel about switching from Python 2.x to 3.x. From no.email at nospam.invalid Wed Jan 27 19:28:08 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 27 Jan 2010 16:28:08 -0800 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <7xk4v3jco7.fsf@ruckus.brouhaha.com> Steven D'Aprano writes: > 6. The code for Python 3 was handed down to Guido from the Heavens, > carved into stone tablets by the Gods themselves. That is heresy. The direction was up, not down. From briandenzer at gmail.com Wed Jan 27 19:28:35 2010 From: briandenzer at gmail.com (Brian D) Date: Wed, 27 Jan 2010 16:28:35 -0800 (PST) Subject: Stuck on a three word street name regex Message-ID: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> I've tackled this kind of problem before by looping through a patterns dictionary, but there must be a smarter approach. Two addresses. Note that the first has incorrectly transposed the direction and street name. The second has an extra space in it before the street type. Clearly done by someone who didn't know how to concatenate properly -- or didn't care. 1000 RAMPART S ST 100 JOHN CHURCHILL CHASE ST I want to parse the elements into an array of values that can be inserted into new database fields. Anyone who loves solving these kinds of puzzles care to relieve my frazzled brain? The pattern I'm using doesn't keep the "CHASE" with the "JOHN CHURCHILL": >>> p = re.compile(r'(?P\d+)\s(?P[A-Z\s]*)\s(?P\w*)\s(?P\w{2})$') >>> s = '1405 RAMPART S ST' >>> m = re.search(p, s) >>> m <_sre.SRE_Match object at 0x011A4440> >>> print m.groups() ('1405', 'RAMPART', 'S', 'ST') >>> s = '45 JOHN CHURCHILL CHASE ST' >>> m = re.search(p, s) >>> m <_sre.SRE_Match object at 0x011A43E8> >>> print m.groups() ('45', 'JOHN CHURCHILL', 'CHASE', 'ST') From no.email at nospam.invalid Wed Jan 27 19:30:40 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 27 Jan 2010 16:30:40 -0800 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xvdenje7k.fsf@ruckus.brouhaha.com> <5b9e69bc-b261-4e81-84bc-045936011e82@o9g2000yqa.googlegroups.com> Message-ID: <7xfx5rjcjz.fsf@ruckus.brouhaha.com> Jonathan Gardner writes: > You're referring to the O(N**2) bit, right? I am sure he knew it was O > (N*log(N)), which is still worse than O(N) for key. It's O(n log n) for both key and cmp. key is usually more convenient and usually gives a constant-factor speedup (DSU pattern), but it uses more memory and there are some types of orderings that it can't handle without a bunch of contortion. From pavlovevidence at gmail.com Wed Jan 27 19:32:39 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Wed, 27 Jan 2010 16:32:39 -0800 (PST) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> Message-ID: On Jan 27, 2:19?pm, exar... at twistedmatrix.com wrote: > On 10:07 pm, pavlovevide... at gmail.com wrote: > >Last I heard, don't remember where, the plan was for Python 2.7 to be > >the last version in the Python 2 line. ?If that's true, Python 3 > >acceptance is further along at this point than anticipated, since they > >originally thought they might have to go up to 2.9. > > This assumes that the decision to stop making new 2.x releases is based > on Python 3 adoption, rather than on something else. I should have said, "If anything...". Carl Banks From no.email at nospam.invalid Wed Jan 27 19:35:27 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 27 Jan 2010 16:35:27 -0800 Subject: Stuck on a three word street name regex References: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> Message-ID: <7xbpgfjcc0.fsf@ruckus.brouhaha.com> Brian D writes: > I've tackled this kind of problem before by looping through a patterns > dictionary, but there must be a smarter approach.> > Two addresses. Note that the first has incorrectly transposed the > direction and street name. .... If you're really serious about it (e.g. you are the post office trying to program automatic mail sorting machines) there is no simple regex trick anything like what you want. A lot of addresses will be ambiguous. You have use all the info you have about your entire address corpus (e.g. you need a complete street directory of the whole US) and do a bunch of Bayesian inference. As a very simple example, for an address like "1000 RAMPART S ST" you'd use the zip code to identify the address's geographic neighborhood, and then use your street directory to find candidate correct addresses within that zip code. The USPS does an amazing job of delivering mail to completely mangled addresses based on methods like that. From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 19:35:42 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 28 Jan 2010 00:35:42 GMT Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On Wed, 27 Jan 2010 16:25:46 -0500, Benjamin Kaplan wrote: > When Python 2.6 came out, Jython was still on 2.2. The difference > between 2.2 and 2.6 is almost as big of a difference as between 2.6 and > 3.0. In that time, you had the introduction of the boolean type, > generators, list comprehensions, the addition of the "yield" and "with" > keywords, universal newline support, and decorators in addition to the > large number of changes to the standard library such as the introduction > of the subprocess module. THANK YOU Benjamin for injecting this note of sanity into the discussion. I believe that, with the possible exception of the change from byte strings to unicode strings, virtually *all* the hoo-har over Python 3 is simply due to the tactical mistake of Guido and the Python Dev team of *calling* Python 3 a backward incompatible release. Python has had previous major changes in the past (e.g. 1.5 to 2.0 and 2.1 to 2.2) and hardly anyone made a complaint. Certainly the move from 2.x to 3.x is a big move. If you have to support both series simultaneously, I don't envy your job, but if CherryPy can do it, so can others. But it's not qualitatively different from supporting (say) 2.4 through 2.6. Targeting multiple versions is always a PITA. I also find it telling that perhaps the biggest change of all, the one from byte strings to unicode, hardly rates a mention from the skeptics and haters. Instead we get rants about how division behaves differently (forgetting that "from __future__ import division" has worked since at least 2.4 and possibly older) and complaints that print is different. -- Steven From pavlovevidence at gmail.com Wed Jan 27 19:44:18 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Wed, 27 Jan 2010 16:44:18 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xvdenje7k.fsf@ruckus.brouhaha.com> <5b9e69bc-b261-4e81-84bc-045936011e82@o9g2000yqa.googlegroups.com> Message-ID: On Jan 27, 4:16?pm, Jonathan Gardner wrote: > On Jan 27, 3:54?pm, Paul Rubin wrote: > > > Steven D'Aprano writes: > > > always much better written with key rather than cmp: key adds an O(N) > > > overheard to the sorting, while cmp makes sorting O(N**2). > > > Whaaaaaaaaaa ...... ???? ?No I don't think so. > > You're referring to the O(N**2) bit, right? I am sure he knew it was O > (N*log(N)), which is still worse than O(N) for key. > > If he didn't, well, Python has some fundamental flaws in its basic > sort algorithm. Quicksort is O(N**2) worst case, perhaps that's what he meant. I'm not sure about timsort but since it's based on quicksort I'd guess it is O(N**2) or worse, worst case, as well. Typical case of a sort is still O(N*log(N)) whether using key or cmp. People recommended against cmp not because of overall algorithmic time considerations, but because of the number of function calls. cmp function was called a minimum of N-1 times and typically around N*log2 (N) times, whereas key was called exactly N times. Since the overhead of a Python function call was very large compared to the built-in cmp operation, it usually outweighed the algorithmic time considerations. Carl Banks From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 19:47:41 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 28 Jan 2010 00:47:41 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: On Wed, 27 Jan 2010 18:29:25 +0100, Alf P. Steinbach wrote: > The main problem with the incompatibility is for porting code, not for > writing code from scratch. Correct. It's a trivial problem, but still a problem. > It's also a problem wrt. learning the language. This makes no sense. Why is it harder to learn print(x, y, z) than this? print x, y, z The first case is like the other functions you have to learn, like len(). In fact, many newbies to Python put unneeded parentheses around arguments to print simply because they assume it is a function. I would argue that the new print function is *simpler* to learn. It is more consistent with other built-ins, and has fewer magic idioms to learn. Instead of: print >>fileObj, x, y, z you use regular function syntax with a meaningful keyword: print(x, y, z, file=fileObj) If you want suppress the newline at the end of each print: print x, y, z, # note the final comma compared to: print(x, y, z, end='') If you want to change the space between elements, instead of: sys.stdout.write(str(x) + "*" + str(y) + "*" + str(z) + '\n') you use: print(x, y, z, sep='*') If you want to override the behaviour of print in a module, instead of having to edit the source code of the module (which might not even be available), all you need to do is monkey-patch it: import module module.print = myprint -- Steven From drsalists at gmail.com Wed Jan 27 20:02:33 2010 From: drsalists at gmail.com (Dan Stromberg) Date: Wed, 27 Jan 2010 17:02:33 -0800 Subject: Orange County, California Python User Group Message-ID: <5def086a1001271702t2d371eeficb0d2cda7e830d4c@mail.gmail.com> http://www.meetup.com/Orange-County-CA-Python-User-Group/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From drsalists at gmail.com Wed Jan 27 20:03:39 2010 From: drsalists at gmail.com (Dan Stromberg) Date: Wed, 27 Jan 2010 17:03:39 -0800 Subject: Orange County, California Python User Group In-Reply-To: <5def086a1001271702t2d371eeficb0d2cda7e830d4c@mail.gmail.com> References: <5def086a1001271702t2d371eeficb0d2cda7e830d4c@mail.gmail.com> Message-ID: <5def086a1001271703v3ea5f932ra0d4b986df4ea33b@mail.gmail.com> On Wed, Jan 27, 2010 at 5:02 PM, Dan Stromberg wrote: > http://www.meetup.com/Orange-County-CA-Python-User-Group/ > Sigh. 'little quick on the trigger there. Anyway, as you might have inferred, I'm attempting to start a Python User Group for Orange County California. If you're interested, please visit the URL above. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 20:09:58 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 28 Jan 2010 01:09:58 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> Message-ID: On Wed, 27 Jan 2010 00:36:52 -0800, Paul Rubin wrote: > Steven D'Aprano writes: >> Without becoming a purely functional language, you won't get rid of all >> statements. > > Why not? GCC lets you use any statement in an expression: I stand corrected. Whether it is a good idea or not is another story. >> for, while, if, try, break, yield, return are all used for flow >> control, and should remain as statements. > > What about assert, import, and pass? Both assert and pass are special and need to be treated specially by the compiler. Arguably we could replace pass with None, but pass makes the intent more obvious. Although assert could be a function, it is treated specially by the compiler and so making it a statement emphases that it is special. The regular form of import could easily be a function returning the module object. After all, that's exactly what __import__ is! But using __import__ for anything but the simplest cases is quite awkward, hence we have syntactic sugar in the form of the import statement. You also missed del as another statement-based command. All the current statements have good strong reasons for being statements. print on the other hand lacks a strong reason for being a statement, and the only good argument against changing is to avoid an incompatible change. But since it is not a gratuitous incompatible change, the long term benefit out-weighs the short-term pain, in my opinion. -- Steven From python at mrabarnett.plus.com Wed Jan 27 20:27:42 2010 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 28 Jan 2010 01:27:42 +0000 Subject: Stuck on a three word street name regex In-Reply-To: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> References: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> Message-ID: <4B60E80E.4010301@mrabarnett.plus.com> Brian D wrote: > I've tackled this kind of problem before by looping through a patterns > dictionary, but there must be a smarter approach. > > Two addresses. Note that the first has incorrectly transposed the > direction and street name. The second has an extra space in it before > the street type. Clearly done by someone who didn't know how to > concatenate properly -- or didn't care. > > 1000 RAMPART S ST > > 100 JOHN CHURCHILL CHASE ST > > I want to parse the elements into an array of values that can be > inserted into new database fields. > > Anyone who loves solving these kinds of puzzles care to relieve my > frazzled brain? > > The pattern I'm using doesn't keep the "CHASE" with the "JOHN > CHURCHILL": > [snip] Regex doesn't gain you much. I'd split the string and then fix the parts as necessary: >>> def parse_address(address): ... parts = address.split() ... if parts[-2] == "S": ... parts[1 : -1] = [parts[-2]] + parts[1 : -2] ... parts[1 : -1] = [" ".join(parts[1 : -1])] ... return parts ... >>> print parse_address("1000 RAMPART S ST") ['1000', 'S RAMPART', 'ST'] >>> print parse_address("100 JOHN CHURCHILL CHASE ST") ['100', 'JOHN CHURCHILL CHASE', 'ST'] From wuwei23 at gmail.com Wed Jan 27 20:36:29 2010 From: wuwei23 at gmail.com (alex23) Date: Wed, 27 Jan 2010 17:36:29 -0800 (PST) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: Terry Reedy wrote: > Actually, Unladen Swallow is now targeted at 3.1; its developers have > conservatively proposed its integration in CPython 3.3. I would not be > completely shocked if it happens in 3.2. Why do I feel like there's less of an onus on Unladen Swallow to _actually prove itself in substantial real world usage_ before integration into CPython than there is on even the smallest of modules for inclusion in the standard library? Are we really expected to just ditch everything we know about CPython's performance characteristics just for some questionable and possibly uneven gains? I've been a big supporter of Py3 from the beginning, but this repeated claim of US becoming the mainline interpreter for 3.x pretty much kills dead a lot of my interest. What am I not seeing amidst the high memory usage and variable performance results of US's _custom-made_ benchmarks? Doesn't that fact alone worry anyone else? Or that LLVM is listed as only having "partial support" with non-Cygwin x86 Windows? Yes, I'd _love_ Python to be faster, who wouldn't? But I'm not seeing the evidence here to support the almost unbridled eagerness that's being demonstrated. From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 20:44:48 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 28 Jan 2010 01:44:48 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xvdenje7k.fsf@ruckus.brouhaha.com> <5b9e69bc-b261-4e81-84bc-045936011e82@o9g2000yqa.googlegroups.com> Message-ID: On Wed, 27 Jan 2010 16:16:59 -0800, Jonathan Gardner wrote: > On Jan 27, 3:54?pm, Paul Rubin wrote: >> Steven D'Aprano writes: >> > always much better written with key rather than cmp: key adds an O(N) >> > overheard to the sorting, while cmp makes sorting O(N**2). >> >> Whaaaaaaaaaa ...... ???? ?No I don't think so. > > You're referring to the O(N**2) bit, right? I am sure he knew it was O > (N*log(N)), which is still worse than O(N) for key. > > If he didn't, well, Python has some fundamental flaws in its basic sort > algorithm. I may have the details wrong, but the performance of sort with cmp is very poor. sort with key calls the key function once for each element in the list, which is O(N), and then sorts O(N*log N), giving a total of O(N*log N). sort with cmp has to compare the comparison function multiple times for each element. I *thought* it ended up calling it N times each, giving O(N**2), but I may be delusional. Whatever it is though, it's quite slow. Here's my test code (for Python 2.5): from timeit import Timer from string import letters from random import shuffle def mycmp(s, t): return cmp(s.upper(), t.upper()) def randstr(): L = list(letters) shuffle(L) return ''.join(L) setup = "from __main__ import mycmp, data" N = 300 data = [randstr() for _ in xrange(N)] t1 = Timer("d = data[:]; d.sort()", setup) # raw sort t2 = Timer("d = data[:]; d.sort(cmp=mycmp)", setup) t3 = Timer("d = data[:]; d.sort(key=str.upper)", setup) for t in (t1, t2, t3): print min(t.repeat(number=500)) N *= 10 # Do it again with bigger N. data = [randstr() for _ in xrange(N)] for t in (t1, t2, t3): print min(t.repeat(number=500)) And here are the results on my PC: # N = 300 0.0631489753723 2.36756515503 0.226485967636 # N = 3000 1.03457903862 35.3092911243 2.77242517471 This doesn't support my claim of O(N**2), but it does demonstrate that sort with cmp should be avoided if at all possible. -- Steven From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 20:46:12 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 28 Jan 2010 01:46:12 GMT Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <7xk4v3jco7.fsf@ruckus.brouhaha.com> Message-ID: On Wed, 27 Jan 2010 16:28:08 -0800, Paul Rubin wrote: > Steven D'Aprano writes: >> 6. The code for Python 3 was handed down to Guido from the Heavens, >> carved into stone tablets by the Gods themselves. > > That is heresy. The direction was up, not down. SPLITTER!!! -- Steven From steven at REMOVE.THIS.cybersource.com.au Wed Jan 27 20:56:00 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 28 Jan 2010 01:56:00 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xvdenje7k.fsf@ruckus.brouhaha.com> <5b9e69bc-b261-4e81-84bc-045936011e82@o9g2000yqa.googlegroups.com> Message-ID: On Wed, 27 Jan 2010 16:44:18 -0800, Carl Banks wrote: >> You're referring to the O(N**2) bit, right? I am sure he knew it was O >> (N*log(N)), which is still worse than O(N) for key. >> >> If he didn't, well, Python has some fundamental flaws in its basic sort >> algorithm. > > Quicksort is O(N**2) worst case, perhaps that's what he meant. No, apparently I was delusional. > I'm not > sure about timsort but since it's based on quicksort I'd guess it is > O(N**2) or worse, worst case, as well. timsort is actually a mergesort, not a quicksort. You may like to read this: http://svn.python.org/projects/python/trunk/Objects/listsort.txt > Typical case of a sort is still O(N*log(N)) whether using key or cmp. > People recommended against cmp not because of overall algorithmic time > considerations, but because of the number of function calls. cmp > function was called a minimum of N-1 times and typically around N*log2 > (N) times, whereas key was called exactly N times. Since the overhead > of a Python function call was very large compared to the built-in cmp > operation, it usually outweighed the algorithmic time considerations. Yes, that sounds more like it. Thanks for the corrections. -- Steven From pavlovevidence at gmail.com Wed Jan 27 20:59:39 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Wed, 27 Jan 2010 17:59:39 -0800 (PST) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <384da2bb-1130-4bfe-8fd8-a343ca4a000c@x10g2000prk.googlegroups.com> On Jan 27, 5:36?pm, alex23 wrote: > Terry Reedy wrote: > > Actually, Unladen Swallow is now targeted at 3.1; its developers have > > conservatively proposed its integration in CPython 3.3. I would not be > > completely shocked if it happens in 3.2. > > Why do I feel like there's less of an onus on Unladen Swallow to > _actually prove itself in substantial real world usage_ before > integration into CPython than there is on even the smallest of modules > for inclusion in the standard library? I don't sense that. I get a sense that there's a lot of people licking their chops because it's sponsored by Google and everything Google touches turns to gold, but that's just nameless plebians. I trust the developers not to be easily convinced. If GvR allows this into CPython without something like a typical 4x speed increase I'll eat my hat. Carl Banks From martien.friedeman at gmail.com Wed Jan 27 21:00:33 2010 From: martien.friedeman at gmail.com (hans moleman) Date: Wed, 27 Jan 2010 18:00:33 -0800 (PST) Subject: CodeInvestigator testers Message-ID: I need Python programmers to test my Python tracing tool CodeInvestigator please. It records all debug information in a recording phase. The program runs as per usual, albeit a lot slower, while this recording takes place. You can then move anywhere in your code and click words in the code to obtain debug information. Useful if you have no idea where to start to debug or you want to see how a program works. Your Python installation isn't changed by this tool. It can easily be uninstalled or forgotten. It requires Python 2.6 and a Firefox browser. It comes with an installer for Windows, Linux and Mac OSX. Any bugs, ideas and remarks please. http://sourceforge.net/projects/cde-investigatr/ Thank you. Martien Friedeman From sg552 at hotmail.co.uk Wed Jan 27 21:06:28 2010 From: sg552 at hotmail.co.uk (Rotwang) Date: Thu, 28 Jan 2010 02:06:28 +0000 Subject: List weirdness - what the heck is going on here? Message-ID: Hi all, I've been trying to make a class with which to manipulate sound data, and have run into some behaviour I don't understand which I hope somebody here can explain. The class has an attribute called data, which is a list with two elements, one for each audio channel, each of which is a list containing the audio data for that channel. It also has various methods to write data such as sine waves and so on, and a method to insert data from one sound at the start of data from another. Schematically, the relevant bits look like this: class sound: def f(self): self.data = [[0]]*2 def insert(self, other): for c in xrange(2): self.data[c][0:0] = other.data[c] However, the insert method doesn't work properly; x.insert(y) adds two copies of y's data to the start of x's data, instead of one. From a session in IDLE: >>> x = sound() >>> y = sound() >>> x.f() >>> y.f() >>> x.data [[0], [0]] >>> x.insert(y) >>> x.data [[0, 0, 0], [0, 0, 0]] But suppose I replace the line self.data = [[0]]*2 with self.data = [[0] for c in xrange(2)] Then it works fine: >>> x = sound() >>> y = sound() >>> x.f() >>> y.f() >>> x.data [[0], [0]] >>> x.insert(y) >>> x.data [[0, 0], [0, 0]] Can anybody tell me what's going on? From tekion at gmail.com Wed Jan 27 21:18:21 2010 From: tekion at gmail.com (tekion) Date: Wed, 27 Jan 2010 18:18:21 -0800 (PST) Subject: deriving MySQLdb class References: <9b9755ad-c3ca-42eb-b95c-0e41c5052cd7@p24g2000yqm.googlegroups.com> <10c80803-4934-4857-80e9-ba63648d9be4@14g2000yqp.googlegroups.com> <9f413105-5572-4bc7-93f4-ac7f252e3174@y23g2000yqm.googlegroups.com> Message-ID: On Jan 21, 11:48?pm, Sean DiZazzo wrote: > On Jan 21, 8:17?pm, tekion wrote: > > > Sean, > > I did a little investigation, there are other classes besides > > Connection. So, could I only set up a derived class from Connection > > and still be able to use the connection to query database and retrieve > > data? > > Im not sure I understand you completely... > > In theory, you could use the C API directly and subclass > _mysql.connection to get at the database. ?But I think the point of > MySQLdb is that they've done all the hard work. ?Why not use it? Regarding above statement, Yeah; what you said is true. I am just playing with inheritance to get a better understanding of it and since I am working with database connection, then I might as learn it by doing it. > I think the other stuff in the module is in support of the Connection > () class. ?ie. ?You cant get a cursor unless you already have a > connection. Yes, I have confirmed it just by inheriting the connection class, I am able to connect and perform normal database functions. Sweet!!! This is why I like Python. Thanks for your help. From nyamatongwe+thunder at gmail.com Wed Jan 27 21:18:33 2010 From: nyamatongwe+thunder at gmail.com (Neil Hodgson) Date: Thu, 28 Jan 2010 02:18:33 GMT Subject: myths about python 3 In-Reply-To: <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> Message-ID: Carl Banks: > There is also no hope someone will fork Python 2.x and continue it in > perpetuity. Well, someone might try to fork it, but they won't be > able to call it Python. Over time there may be more desire from those unable or unwilling to upgrade to 3.x to work on improvements to 2.x, perhaps leading to a version 2.8. One of the benefits of open source is that you are not trapped into following vendor decisions like Microsoft abandoning classic VB in favour of VB.NET. It would be unreasonable for the core developers to try to block this. Refusing use of the Python trademark for a version that was reasonably compatible in both directions would be particularly petty. Neil From angrybaldguy at gmail.com Wed Jan 27 21:18:34 2010 From: angrybaldguy at gmail.com (Owen Jacobson) Date: Wed, 27 Jan 2010 21:18:34 -0500 Subject: List weirdness - what the heck is going on here? References: Message-ID: <2010012721183416807-angrybaldguy@gmailcom> On 2010-01-27 21:06:28 -0500, Rotwang said: > Hi all, I've been trying to make a class with which to manipulate sound > data, and have run into some behaviour I don't understand which I hope > somebody here can explain. The class has an attribute called data, > which is a list with two elements, one for each audio channel, each of > which is a list containing the audio data for that channel. It also has > various methods to write data such as sine waves and so on, and a > method to insert data from one sound at the start of data from another. > Schematically, the relevant bits look like this: > > class sound: > def f(self): > self.data = [[0]]*2 Consider that this is equivalent to def f(self): x = [0] self.data = [x, x] self.data is now a list containing two references to the list referenced by x -- so changes via either of the elements of self.data will affect both elements. Your comprehension version creates a list containing two distinct list objects, so this doesn't happen. > Can anybody tell me what's going on? -o From apt.shansen at gmail.com Wed Jan 27 21:22:25 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Wed, 27 Jan 2010 18:22:25 -0800 Subject: List weirdness - what the heck is going on here? In-Reply-To: References: Message-ID: <7a9c25c21001271822g5a5df06dg2e8b5471951562b4@mail.gmail.com> On Wed, Jan 27, 2010 at 6:06 PM, Rotwang wrote: > But suppose I replace the line > self.data = [[0]]*2 > > with > > self.data = [[0] for c in xrange(2)] > The first line does not do what you think it does: it doesn't make a copy of that internal [0]. Python almost never implicitly copies anything, things that look like they may make a copy of a mutable object-- usually don't, unless there's something very explicit in the action that says 'making a new object out of that other object' What it is doing is creating a list which contains a reference to the same [0] twice, which you can see as: print self.data[0] is self.data[1] The "is" operator doesn't test equality, but precise identity-- those two objects are exactly the same. So when you loop over later to insert data, what you're doing is inserting the same data into the same exact list... that just happens to be known by two different names right now. :) The list comprehension, on the other hand, is creating a new [0] each time it loops over your xrange. --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From groups_ads12 at yahoo.com Wed Jan 27 21:35:45 2010 From: groups_ads12 at yahoo.com (groups_ads12) Date: Thu, 28 Jan 2010 02:35:45 -0000 Subject: www.visualstudio.me Message-ID: www.visualstudio.me buy visual studio buy visual studio 2008 download microsoft visual studio 2005 download visual studio 2003 download visual studio 2005 download visual studio 2008 download visual studio 6.0 download visual studio net free visual studio learn visual studio microsoft visual studio microsoft visual studio .net microsoft visual studio .net 2005 microsoft visual studio 2003 microsoft visual studio 2005 microsoft visual studio 2005 express edition microsoft visual studio 2005 free download microsoft visual studio 2005 professional microsoft visual studio 2005 professional edition microsoft visual studio 2005 standard edition microsoft visual studio 2008 microsoft visual studio 2008 express microsoft visual studio 2008 professional microsoft visual studio 2008 professional edition microsoft visual studio 2008 professional with msdn professional microsoft visual studio 2008 programming microsoft visual studio 2008 standard microsoft visual studio 2008 standard edition microsoft visual studio 2008 standard upgrade microsoft visual studio 6 microsoft visual studio 6.0 microsoft visual studio 8 microsoft visual studio download microsoft visual studio net microsoft visual studio net 2003 microsoft visual studio net 2005 microsoft visual studio pro 2008 microsoft visual studio tips ms visual studio ms visual studio 2005 ms visual studio 2008 visual studio visual studio 10 visual studio 2..8 visual studio 2002 visual studio 2003 visual studio 2003 service pack visual studio 2005 visual studio 2005 c# visual studio 2005 download visual studio 2005 pro visual studio 2005 professional visual studio 2005 professional edition visual studio 2005 rapidshare visual studio 2005 service pack visual studio 2005 service pack 1 visual studio 2005 service pack 2 visual studio 2005 sp visual studio 2005 sp1 visual studio 2005 sp2 visual studio 2005 standard edition visual studio 2005 team suite visual studio 2005 trial visual studio 2005 tutorial visual studio 2005 vista visual studio 2006 visual studio 2007 visual studio 2008 visual studio 2008 book visual studio 2008 c# visual studio 2008 free download visual studio 2008 price visual studio 2008 professional visual studio 2008 professional edition visual studio 2008 sdk visual studio 2008 service pack visual studio 2008 standard visual studio 2008 standard edition visual studio 2008 team system visual studio 2008 trial visual studio 2008 tutorial visual studio 2008 upgrade visual studio 2009 visual studio 2010 visual studio 2010 download visual studio 6 visual studio 6 download visual studio 6 service pack visual studio 6.0 visual studio 7 visual studio 8 visual studio c visual studio download visual studio enterprise visual studio gallery visual studio ide visual studio magazine visual studio msdn visual studio net visual studio net 2003 visual studio net 2003 download visual studio net 2005 visual studio net 2008 visual studio pro visual studio pro 2008 visual studio professional visual studio professional 2008 visual studio sdk visual studio setup project visual studio software visual studio standard visual studio team edition visual studio team suite visual studio team system visual studio tools visual studio tools for office visual studio training visual studio trial visual studio tutorial visual studio tutorials visual studio upgrade visual studio wiki visual studios -------------- next part -------------- An HTML attachment was scrubbed... URL: From tanix at mongo.net Wed Jan 27 21:36:41 2010 From: tanix at mongo.net (tanix) Date: Thu, 28 Jan 2010 02:36:41 GMT Subject: Python Goldmine has been updated: http://preciseinfo.org/Convert/index_Convert_Python.html References: Message-ID: In article , tanix at mongo.net (tanix) wrote: Good news: Site search has been fully implemented. You can search the entire Python collection or a single chapter related to specific context to find what you are looking for. Bad news: We are currently out of sync with google. So, if you do a google search you might see a different article than what google shows. >Python Goldmine collection contains the extensive collection of articles >going back several years. It includes thousands of code >examples and expert discussions on all major topics. > >The information is organized by relevant topics, covered >by the corresponding chapters. > >The information was filtered with sophisticated filters and vast >majority of artilces with little relevance have been filtered out. > >If you have any specific requests for some new chapters to be added >and it is of interest to others, please post your requests on this >thread. > >If anyone feels he has above average level of competence, or can >reccommend someone who posts on this group, you may request to be >included in the expert chapters. > >The Python Goldmine is at: > >http://preciseinfo.org/Convert/index_Convert_Python.html > -- Programmer's Goldmine collections: http://preciseinfo.org Tens of thousands of code examples and expert discussions on C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, organized by major topics of language, tools, methods, techniques. All collections are fully searchable down to specific chapter. From steve at holdenweb.com Wed Jan 27 21:37:21 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 27 Jan 2010 21:37:21 -0500 Subject: myths about python 3 In-Reply-To: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <4B60F861.1060408@holdenweb.com> John Nagle wrote: > Daniel Fetchinson wrote: >> Hi folks, >> >> I was going to write this post for a while because all sorts of myths >> periodically come up on this list about python 3. I don't think the >> posters mean to spread false information on purpose, they simply are >> not aware of the facts. >> >> My list is surely incomplete, please feel free to post your favorite >> misconception about python 3 that people periodically state, claim or >> ask about. > > Myths about Python 3: > > 1. Python 3 is supported by major Linux distributions. > > FALSE - most distros are shipping with Python 2.4, or 2.5 at best. > Why would a "major Linux distribution" want to saddle themselves with such a new technology so erly in its lifetime? > 2. Python 3 is supported by multiple Python implementations. > > FALSE - Only CPython supports 3.x. Iron Python, Unladen Swallow, > PyPy, and Jython have all stayed with 2.x versions of Python. > Your selective information here is particularly partial to your case. I have spoken with developers from IronPython and Jython, and both teams are committed to eventual support of 3.x. > 3. Python 3 is supported by most 3rd party Python packages. > > FALSE - it's not supported by MySQLdb, OpenSSL, feedparser, etc. > I would argue it's up to Python to support those facilities rather than the other way round. > Arguably, Python 3 has been rejected by the market. Instead, there's > now Python 2.6, Python 2.7, and Python 2.8. Python 3 has turned into > a debacle like Perl 6, now 10 years old. > > That's the reality, Python 3 fanboys. > Kindly confine your debate to the facts and keep the snide remarks to yourself. Like it or not Python 3 is the future, and unladen swallow's recent announcement that they would target only Python 3 represented a ground-breaking advance for the language. Happily my Python 2.x interpreters all continue to work just as they have since they were installed. If you have to stretch as far as Perl 6 for an analogy then you have clearly stretched a little too far. The situations are not even closely comparable, and I defy you to argue otherwise. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Wed Jan 27 21:37:21 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 27 Jan 2010 21:37:21 -0500 Subject: myths about python 3 In-Reply-To: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <4B60F861.1060408@holdenweb.com> John Nagle wrote: > Daniel Fetchinson wrote: >> Hi folks, >> >> I was going to write this post for a while because all sorts of myths >> periodically come up on this list about python 3. I don't think the >> posters mean to spread false information on purpose, they simply are >> not aware of the facts. >> >> My list is surely incomplete, please feel free to post your favorite >> misconception about python 3 that people periodically state, claim or >> ask about. > > Myths about Python 3: > > 1. Python 3 is supported by major Linux distributions. > > FALSE - most distros are shipping with Python 2.4, or 2.5 at best. > Why would a "major Linux distribution" want to saddle themselves with such a new technology so erly in its lifetime? > 2. Python 3 is supported by multiple Python implementations. > > FALSE - Only CPython supports 3.x. Iron Python, Unladen Swallow, > PyPy, and Jython have all stayed with 2.x versions of Python. > Your selective information here is particularly partial to your case. I have spoken with developers from IronPython and Jython, and both teams are committed to eventual support of 3.x. > 3. Python 3 is supported by most 3rd party Python packages. > > FALSE - it's not supported by MySQLdb, OpenSSL, feedparser, etc. > I would argue it's up to Python to support those facilities rather than the other way round. > Arguably, Python 3 has been rejected by the market. Instead, there's > now Python 2.6, Python 2.7, and Python 2.8. Python 3 has turned into > a debacle like Perl 6, now 10 years old. > > That's the reality, Python 3 fanboys. > Kindly confine your debate to the facts and keep the snide remarks to yourself. Like it or not Python 3 is the future, and unladen swallow's recent announcement that they would target only Python 3 represented a ground-breaking advance for the language. Happily my Python 2.x interpreters all continue to work just as they have since they were installed. If you have to stretch as far as Perl 6 for an analogy then you have clearly stretched a little too far. The situations are not even closely comparable, and I defy you to argue otherwise. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From rantingrick at gmail.com Wed Jan 27 21:41:37 2010 From: rantingrick at gmail.com (rantingrick) Date: Wed, 27 Jan 2010 18:41:37 -0800 (PST) Subject: Python and Ruby References: <43cad4e7-6b54-4bd9-9ca2-217cb109560f@g29g2000yqe.googlegroups.com> Message-ID: <1d501ad6-39f8-4e5d-aabc-222b71553940@e25g2000yqh.googlegroups.com> On Jan 27, 5:31?pm, Jonathan Gardner wrote: > To add to that, Python is the type of language where experienced > programmers can pick it up by reading code, and newbies won't get > hopelessly lost. I've taught less-than-formal introductory programming > classes to people who are new to programming. Python gets out of the > way, and allows you to focus on the programming concepts, such as > variable, functions, parameters, classes, and algorithms. Well said Jonathan!! Well said!! (both posts) From no.email at nospam.invalid Wed Jan 27 21:44:05 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 27 Jan 2010 18:44:05 -0800 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <7xzl3zq77u.fsf@ruckus.brouhaha.com> Steve Holden writes: > Kindly confine your debate to the facts and keep the snide remarks to > yourself. Like it or not Python 3 is the future, and unladen swallow's > recent announcement that they would target only Python 3 represented a > ground-breaking advance for the language. My take on things is that doing unladen swallow really "right" will require yet more incompatible changes; i.e., the result will either still leave quite a bit of performance on the table, or else it won't be compatible with the current specification of Python 3 and they'll presumably have to call it Python 4. And if Python 4 is as good as I believe it could possibly be, then it might get wide acceptance before Python 3 really has all that much uptake. If I have to accept incompatibility anyway, and Python 4 gives huge improvements while Python 3's improvements are tiny or moderate, why not skip over Python 3? From wuwei23 at gmail.com Wed Jan 27 21:59:26 2010 From: wuwei23 at gmail.com (alex23) Date: Wed, 27 Jan 2010 18:59:26 -0800 (PST) Subject: List weirdness - what the heck is going on here? References: Message-ID: <4702e667-c089-4349-a349-76ec844b6828@a17g2000pre.googlegroups.com> Rotwang wrote: > Can anybody tell me what's going on? Your problem is basically this: >>> a = [1] >>> b = [a] * 2 >>> b [[1], [1]] >>> a.append(2) >>> b [[1, 2], [1, 2]] The expression '[a] * 2' doesn't make two copies of list of a list of a, it makes two nested _references_ to it. When you modify the 'first' list, you're seeing that changed reflected in the second reference. The list comprehension, however, is returning a _new_ list each time. Changing its contents doesn't affect any of the other independently created lists. From cournape at gmail.com Wed Jan 27 22:27:41 2010 From: cournape at gmail.com (David Cournapeau) Date: Thu, 28 Jan 2010 12:27:41 +0900 Subject: python 3's adoption In-Reply-To: <7xockfjcsv.fsf@ruckus.brouhaha.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> <7xockfjcsv.fsf@ruckus.brouhaha.com> Message-ID: <5b8d13221001271927l11dfd032s3b8159c831444691@mail.gmail.com> On Thu, Jan 28, 2010 at 9:25 AM, Paul Rubin wrote: > I don't mind that 3.x is breaking stuff for the sake of improving > things. ?That's the whole idea of 3.x, after all. ?What bugs me is that > the improvements are mostly quite superficial, and the breakage seems > often gratuitous. ?I'd rather see much more drastic changes and > correspondingly much larger improvements. At the risk of saying something trivial, that depends on what you consider superficial or not. It totally depends on your usage. I know I would have taken something to improve python packaging (dropping distutils, stable ABI across the whole 3.x range) over any feature in py3k. That's why your real estate comparison is not adequate IMHO - what is 2 % for you is maybe 30 % for someone else, and my own 30 % for packaging may be 1 % for most people who don't care about C extensions at all. David. From briandenzer at gmail.com Wed Jan 27 22:39:44 2010 From: briandenzer at gmail.com (Brian D) Date: Wed, 27 Jan 2010 19:39:44 -0800 (PST) Subject: Stuck on a three word street name regex References: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> <7xbpgfjcc0.fsf@ruckus.brouhaha.com> Message-ID: On Jan 27, 6:35?pm, Paul Rubin wrote: > Brian D writes: > > I've tackled this kind of problem before by looping through a patterns > > dictionary, but there must be a smarter approach.> > > Two addresses. Note that the first has incorrectly transposed the > > direction and street name. .... > > If you're really serious about it (e.g. you are the post office trying > to program automatic mail sorting machines) there is no simple regex > trick anything like what you want. ?A lot of addresses will be > ambiguous. ?You have use all the info you have about your entire address > corpus (e.g. you need a complete street directory of the whole US) and > do a bunch of Bayesian inference. ?As a very simple example, for an > address like "1000 RAMPART S ST" you'd use the zip code to identify the > address's geographic neighborhood, and then use your street directory to > find candidate correct addresses within that zip code. ?The USPS does > an amazing job of delivering mail to completely mangled addresses > based on methods like that. Paul, That's a sound methodology. I actually have a routine that will compare an address to a list of all streets in the city using a Short Distance function. I have used that in circumstances when there are a lot of problems with addresses. In this case, however, the streets are actually structured very well -- except for the transposed street directions. I was really hoping to see if there's a solution that handles one, two, and three word strings, followed by an occasional single character, and then a two character suffix. I'm still hoping for that kind of a solution if it exists. The reason? It's actually a very small number of addresses that aren't being captured with the current regex. I don't see the need for overkill, and I'm always stretching to learn something I haven't already succeeded at accomplishing. I may just make a second pass at the data with a different regex. From briandenzer at gmail.com Wed Jan 27 22:57:25 2010 From: briandenzer at gmail.com (Brian D) Date: Wed, 27 Jan 2010 19:57:25 -0800 (PST) Subject: Stuck on a three word street name regex References: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> Message-ID: <0a712e20-facf-4a9b-8852-6136a8bad7dc@n7g2000yqb.googlegroups.com> On Jan 27, 7:27?pm, MRAB wrote: > Brian D wrote: > > I've tackled this kind of problem before by looping through a patterns > > dictionary, but there must be a smarter approach. > > > Two addresses. Note that the first has incorrectly transposed the > > direction and street name. The second has an extra space in it before > > the street type. Clearly done by someone who didn't know how to > > concatenate properly -- or didn't care. > > > 1000 RAMPART S ST > > > 100 JOHN CHURCHILL CHASE ?ST > > > I want to parse the elements into an array of values that can be > > inserted into new database fields. > > > Anyone who loves solving these kinds of puzzles care to relieve my > > frazzled brain? > > > The pattern I'm using doesn't keep the "CHASE" with the "JOHN > > CHURCHILL": > > [snip] > Regex doesn't gain you much. I'd split the string and then fix the parts > as necessary: > > ?>>> def parse_address(address): > ... ? ? parts = address.split() > ... ? ? if parts[-2] == "S": > ... ? ? ? ? parts[1 : -1] = [parts[-2]] + parts[1 : -2] > ... ? ? parts[1 : -1] = [" ".join(parts[1 : -1])] > ... ? ? return parts > ... > ?>>> print parse_address("1000 RAMPART S ST") > ['1000', 'S RAMPART', 'ST'] > ?>>> print parse_address("100 JOHN CHURCHILL CHASE ?ST") > ['100', 'JOHN CHURCHILL CHASE', 'ST'] This is a nice approach I wouldn't have thought to pursue. I've never seen this referencing of list elements in reverse order with negative values, so that certainly expands my knowledge of Python. Of course, I'd want to check for other directionals -- probably with a list check, e.g., if parts[-2] in ('E', 'W', 'N', 'S'): Thanks for sharing your approach. From g.bogle at auckland.no.spam.ac.nz Wed Jan 27 23:03:14 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Thu, 28 Jan 2010 17:03:14 +1300 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <460a2134-b59b-4bed-918c-fd1bbb09a3b3@o28g2000yqh.googlegroups.com> Message-ID: Luis M. Gonz?lez wrote: >> Please don't post more noise and ad hominem attacks to the group, Steve. > > "Ad hominem"? > Please, operor non utor lingua non notus per vulgaris populus. > Gratias ago vos... > "ad hominem" is "lingua notus per vulgaris populus", the vulgar pop of these parts, anyway. From g.bogle at auckland.no.spam.ac.nz Wed Jan 27 23:04:27 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Thu, 28 Jan 2010 17:04:27 +1300 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: Steven D'Aprano wrote: > On Wed, 27 Jan 2010 02:28:00 +0100, Alf P. Steinbach wrote: > >>>> * Print is now a function. Great, much improvement. >> Actually not, IMHO. All it does is is to provide incompatibility. They >> forgot Ronald Reagan's old maxim: if it don't need fixin', don't fix it. The aphorism needs fixing: If it ain't broke, don't fix it. From hackingkk at gmail.com Wed Jan 27 23:12:22 2010 From: hackingkk at gmail.com (hackingKK) Date: Thu, 28 Jan 2010 09:42:22 +0530 Subject: Python and Ruby In-Reply-To: <1d501ad6-39f8-4e5d-aabc-222b71553940@e25g2000yqh.googlegroups.com> References: <43cad4e7-6b54-4bd9-9ca2-217cb109560f@g29g2000yqe.googlegroups.com> <1d501ad6-39f8-4e5d-aabc-222b71553940@e25g2000yqh.googlegroups.com> Message-ID: <4B610EA6.2050607@gmail.com> On Thursday 28 January 2010 08:11 AM, rantingrick wrote: > On Jan 27, 5:31 pm, Jonathan Gardner > wrote: > > >> To add to that, Python is the type of language where experienced >> programmers can pick it up by reading code, and newbies won't get >> hopelessly lost. I've taught less-than-formal introductory programming >> classes to people who are new to programming. Python gets out of the >> way, and allows you to focus on the programming concepts, such as >> variable, functions, parameters, classes, and algorithms. >> > Well said Jonathan!! Well said!! (both posts) > I share the same experience. As pointed out in the "how to think like a computer scientist in Python" I found that when I was teaching first time programmers, Python was the best choice. Not just the clean syntax but preciseness of the language itself was very helpful. as I callit "get in get your work done and get out!" methodology works very well for absolute beginner students for programming. new comers to this list might be happy to note that python reads very closely to spoken English. But I want to add more. while it is great tool for beginners, it is a power box for advanced users who have the attitude of "cut the crap off and come to the point" For such programmers python is extremly good, I teach all types of students and now have convinced a few univercities here in India to include python in engineering curriculam. Happy hacking. Krishnakant. From ppearson at nowhere.invalid Wed Jan 27 23:33:13 2010 From: ppearson at nowhere.invalid (Peter Pearson) Date: 28 Jan 2010 04:33:13 GMT Subject: ISO module for binomial coefficients, etc. References: Message-ID: <7scic9F5s8U1@mid.individual.net> On Sun, 24 Jan 2010 16:11:03 -0500, Dave Angel wrote: > I didn't think of simply summing the logs. A couple terms of Stirling's approximation work pretty well: def log_fact_half( N ): """log_fact_half( n ) returns the natural logarithm of the factorial of n/2. n need not be an integer. Domain: n from 0 to infinity Range: from something around -0.12 to infinity """ SmallFacts = ( 0.0000000000000000, -0.1207822376352453, 0.0000000000000000, 0.2846828704729192, 0.6931471805599453, 1.2009736023470743, 1.7917594692280550, 2.4537365708424423, 3.1780538303479458, 3.9578139676187165, 4.7874917427820458, 5.6625620598571418, 6.5792512120101012, 7.5343642367587327, 8.5251613610654147, 9.5492672573009969, 10.6046029027452509, 11.6893334207972686, 12.8018274800814691, 13.9406252194037634 ) if N < 0: raise RuntimeError, \ "Negative argument in LogHalfFact!" if N < len( SmallFacts ): RetVal = SmallFacts[ N ] else: n = 0.5 * N RetVal = (n+0.5)*math.log(n) - n + HALFLOG2PI + 1.0/(12*n) - \ 1.0/(360*n*n*n) return RetVal -- To email me, substitute nowhere->spamcop, invalid->net. From daniel at stutzbachenterprises.com Thu Jan 28 00:11:25 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Wed, 27 Jan 2010 23:11:25 -0600 Subject: ISO module for binomial coefficients, etc. In-Reply-To: References: Message-ID: On Sat, Jan 23, 2010 at 4:55 PM, kj wrote: > Before I go off to re-invent a thoroughly invented wheel, I thought > I'd ask around for some existing module for computing binomial > coefficient, hypergeometric coefficients, and other factorial-based > combinatorial indices. I'm looking for something that can handle > fairly large factorials (on the order of 10000!), using floating-point > approximations as needed, and is smart about optimizations, > memoizations, etc. > I don't have code for any of the other stuff, but I have some efficient code (see below) for computing exact integer factorials. It's a divide-and-conquer algorithm. I'm not sure of the exact time complexity, but it appears to somewhere between n*log n and n**2. python2.6 -m timeit -s 'from utils import factorial' 'x = factorial(10000)' 10 loops, best of 3: 27.7 msec per loop If that's not fast enough, I suggest using the gamma function to compute floating-point approximations, as gamma(n) == (n-1)!. Gamma isn't yet included in the Python standard library (http://bugs.python.org/issue3366), but you can use the ctypes module to get it from the system C library on many platforms. For computing binomial coefficients, you can use the lgamma function (log of gamma), also found in my system C libraries. Since choose(n, k) = exp(lgamma(n) - lgamma(k) - lgamma(n-k)). If you go with floating point, watch out for floating point rounding error. ;-) def factorial(n): "http://www.luschny.de/math/factorial/binarysplitfact.html" p = 1 r = 1 p, r = factorial_loop(n, p, r) return r << n_minus_num_of_bits(n) def factorial_loop(n, p, r): if n <= 2: return p, r p, r = factorial_loop(n // 2, p, r) p *= part_product(n // 2 + 1 + ((n // 2) & 1), n - 1 + (n & 1)) r *= p return p, r def part_product(n, m): if (m <= (n+1)): return n if (m == (n+2)): return n*m k = (n+m) // 2 if ((k & 1) != 1): k -= 1 return part_product(n, k) * part_product(k+2, m) def n_minus_num_of_bits(v): w = v if w >= 2**32-1: raise OverflowError w -= (0xaaaaaaaa & w) >> 1 w = (w & 0x33333333) + ((w >> 2) & 0x33333333) w = w + (w >> 4) & 0x0f0f0f0f w += w >> 8 w += w >> 16 return v - (w & 0xff) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From cournape at gmail.com Thu Jan 28 00:58:15 2010 From: cournape at gmail.com (David Cournapeau) Date: Thu, 28 Jan 2010 14:58:15 +0900 Subject: Library support for Python 3.x In-Reply-To: References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> Message-ID: <5b8d13221001272158l96082d8sf96521ad00adee43@mail.gmail.com> On Thu, Jan 28, 2010 at 7:38 AM, Terry Reedy wrote: > > For a windows user who depends on pre-built binaries, every new release > breaks *every* library that is not pure Python and needs to be compiled. That's not windows specific - most packages which distribute binary packages need to package binaries for every minor version (2.4, 2.5, etc...). That's certainly the case for numpy and scipy. Python does not have a stable ABI across minor releases, only micro releases. I doubt that's what Paul was referring to, though - he seemed more concern with API/language changes than ABI issues. cheers, David From no.email at nospam.invalid Thu Jan 28 01:37:52 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 27 Jan 2010 22:37:52 -0800 Subject: Library support for Python 3.x References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> Message-ID: <7xk4v2g2f3.fsf@ruckus.brouhaha.com> David Cournapeau writes: > That's not windows specific - most packages which distribute binary > packages need to package binaries for every minor version (2.4, 2.5, > etc...).... > I doubt that's what Paul was referring to, though - he seemed more > concern with API/language changes than ABI issues. I didn't realize the ABI situation was that unstable. I thought you could just package up a .so or .dll and people could keep using it. I tend to not want to use extension modules that are not in the stdlib, and I guess this is another reason to keep staying away from them. From tjreedy at udel.edu Thu Jan 28 01:39:14 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 28 Jan 2010 01:39:14 -0500 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On 1/27/2010 8:36 PM, alex23 wrote: > Terry Reedy wrote: >> Actually, Unladen Swallow is now targeted at 3.1; its developers have >> conservatively proposed its integration in CPython 3.3. This statement was to counter the 'myth' that US was only targeted at 2.x when the current situation is quite the opposite. >> I would not be completely shocked if it happens in 3.2. I was initially rather dubious about the idea. I based the above on the team's acceptance of and response to reasonable requirements. In particular, several people said that the speed/space traceoff should be optional, and that compilation 'without llvm' should really be without, not just with llvm present but disabled. Instead of arguing, Colin went ahead and patched the build process to make it be this way. > Why do I feel like there's less of an onus on Unladen Swallow to > _actually prove itself in substantial real world usage_ before > integration into CPython than there is on even the smallest of modules > for inclusion in the standard library? I have no idea. It will have to improve its speedup more before adoption. I will not be surprised if that happens. > Are we really expected to just ditch everything we know about > CPython's performance characteristics just for some questionable and > possibly uneven gains? > > I've been a big supporter of Py3 from the beginning, but this repeated > claim of US becoming the mainline interpreter for 3.x US is not a new or separate interpreter. It will be an optional jit replacement for one component of CPython, the eval loop. All the code for builting functions, types, and modules will be untouched, as will their big O performance characteristics. > pretty much kills dead a lot of my interest. If you can still have a binary free of the traceoff, why would you care? > What am I not seeing amidst the high > memory usage and variable performance results of US's _custom-made_ > benchmarks? Doesn't that fact alone worry anyone else? Or that LLVM is > listed as only having "partial support" with non-Cygwin x86 Windows? They claim they have pretty well fixed that. They know that complete Windows support, including 64 bit versions, is a necessity. Terry Jan Reedy From python at bdurham.com Thu Jan 28 01:43:57 2010 From: python at bdurham.com (python at bdurham.com) Date: Thu, 28 Jan 2010 01:43:57 -0500 Subject: Python interface to Google translate? Message-ID: <1264661037.19646.1356985801@webmail.messagingengine.com> Looking for any recommendations on a Python module/package that would allow me to submit small HTML Unicode documents to Google and retrieve the translated results. Or is this the type of task that one should use a urllib-like module and do manually? Note: By small HTML Unicode documents I mean files with about 50 paragraphs of text marked up with basic HTML tags. Any suggestions appreciated. Thanks, Malcolm -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnodel at googlemail.com Thu Jan 28 02:21:28 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Thu, 28 Jan 2010 07:21:28 +0000 Subject: List weirdness - what the heck is going on here? References: Message-ID: Rotwang writes: > Hi all, I've been trying to make a class with which to manipulate > sound data, and have run into some behaviour I don't understand which > I hope somebody here can explain. The class has an attribute called > data, which is a list with two elements, one for each audio channel, > each of which is a list containing the audio data for that channel. It > also has various methods to write data such as sine waves and so on, > and a method to insert data from one sound at the start of data from > another. Schematically, the relevant bits look like this: > > class sound: > def f(self): > self.data = [[0]]*2 > > def insert(self, other): > for c in xrange(2): > self.data[c][0:0] = other.data[c] > > However, the insert method doesn't work properly; x.insert(y) adds two > copies of y's data to the start of x's data, instead of one. From a > session in IDLE: > >>>> x = sound() >>>> y = sound() >>>> x.f() >>>> y.f() >>>> x.data > [[0], [0]] >>>> x.insert(y) >>>> x.data > [[0, 0, 0], [0, 0, 0]] > > But suppose I replace the line > > self.data = [[0]]*2 > > with > > self.data = [[0] for c in xrange(2)] > > Then it works fine: > >>>> x = sound() >>>> y = sound() >>>> x.f() >>>> y.f() >>>> x.data > [[0], [0]] >>>> x.insert(y) >>>> x.data > [[0, 0], [0, 0]] > > Can anybody tell me what's going on? It's a FAQ! http://www.python.org/doc/faq/programming/#how-do-i-create-a-multidimensional-list -- Arnaud From cournape at gmail.com Thu Jan 28 02:28:53 2010 From: cournape at gmail.com (David Cournapeau) Date: Thu, 28 Jan 2010 16:28:53 +0900 Subject: Library support for Python 3.x In-Reply-To: <7xk4v2g2f3.fsf@ruckus.brouhaha.com> References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> <7xk4v2g2f3.fsf@ruckus.brouhaha.com> Message-ID: <5b8d13221001272328u55b26275j68277b555c1058e8@mail.gmail.com> On Thu, Jan 28, 2010 at 3:37 PM, Paul Rubin wrote: > David Cournapeau writes: >> That's not windows specific - most packages which distribute binary >> packages need to package binaries for every minor version (2.4, 2.5, >> etc...).... >> I doubt that's what Paul was referring to, though - he seemed more >> concern with API/language changes than ABI issues. > > I didn't realize the ABI situation was that unstable. Unstable may be strong - every minor version of python has a lifespan of several years. But yes, that's an hindrance for packagers: you need to package binaries for every minor version of python, although I guess for trivial extensions, you may get away with it on some platforms. That's why as far as I am concerned, something like PEP 384 worths more than any feature in py3k I am aware of. I think it will have more impact than py3k's features for the scientific python, if the stable API is rich enough. It would certainly make more incentive for me to work on porting packages to py3k than just doing it because we will have to at some point. cheers, David From floris.bruynooghe at gmail.com Thu Jan 28 02:32:23 2010 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Wed, 27 Jan 2010 23:32:23 -0800 (PST) Subject: Ad hoc lists vs ad hoc tuples References: <8a57c64f-0c67-4ec1-b4be-34d76c7e7571@q4g2000yqm.googlegroups.com> Message-ID: <8e2f9215-ee5e-4114-a955-82a88d2be9fe@q4g2000yqm.googlegroups.com> On Jan 27, 10:15?pm, Terry Reedy wrote: > On 1/27/2010 12:32 PM, Antoine Pitrou wrote: > > > Le Wed, 27 Jan 2010 02:20:53 -0800, Floris Bruynooghe a ?crit : > > >> Is a list or tuple better or more efficient in these situations? > > > Tuples are faster to allocate (they are allocated in one single step) and > > quite a bit smaller too. Thanks for all the answers! This is what I was expecting but it's nice to see it confirmed. Regards Floris From richard.lamboj at bilcom.at Thu Jan 28 02:42:26 2010 From: richard.lamboj at bilcom.at (Richard Lamboj) Date: Thu, 28 Jan 2010 08:42:26 +0100 Subject: starting a thread in a nother thread In-Reply-To: <4b604df9$0$6723$9b4e6d93@newsspool2.arcor-online.net> References: <4b604df9$0$6723$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <201001280842.26177.richard.lamboj@bilcom.at> Am Wednesday 27 January 2010 15:30:17 schrieb Stefan Behnel: > Richard Lamboj, 27.01.2010 15:23: > > Am Wednesday 27 January 2010 14:10:13 schrieb Stefan Behnel: > >> Richard Lamboj, 27.01.2010 14:06: > >>> just for _curiosity_. What would be if i start a thread in a nother > >>> thread and acquire a lock in the "child" thread. Is there anything that > >>> could go wrong if someone try to start threads in threads? > >> > >> There's usually tons of things that can go wrong w.r.t. threads: > >> > >> http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf > >> > >> However, there's nothing special to a thread that was started from > >> another thread, so the problems don't change. > > > > i have tried a little bit around with psycopg2 and threads, > > > > I'am sharing one connection for all threads. When i'am starting the > > threads "normal" everything works without any Problem. When i'am starting > > the threads from another thread than i got a "segmentation fault" > > Sounds like a bug that you might want to report to the maintainers of > psycopg2. > > Stefan Yes i should make a bug report and thanks for your help. Just some Debugginginfo: [New Thread 0x429b5950 (LWP 27486)] python-dbg: ../Objects/stringobject.c:116: PyString_FromString: Assertion `str != ((void *)0)' failed. Program received signal SIGABRT, Aborted. [Switching to Thread 0x431b6950 (LWP 27484)] 0x00007f6d932feed5 in raise () from /lib/libc.so.6 From jgardner at jonathangardner.net Thu Jan 28 02:50:55 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Wed, 27 Jan 2010 23:50:55 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> <7xockfjcsv.fsf@ruckus.brouhaha.com> Message-ID: <6b89dc67-ceee-4b83-a11f-c7376185dcd1@b9g2000pri.googlegroups.com> On Jan 27, 4:25?pm, Paul Rubin wrote: > What about assert and pass? > If you're going to have statements, you're going to need the null statement. That's "pass". It could be renamed "null_statement" but "pass" is a better description. "None" and "pass" are cousins of sorts, since "None" is the null expression and "pass" the null statement. I agree on "assert". I don't like running a program in test mode and then running it in production mode with different code. I would rather test what I am going to actually run. "assert" should be a function, and support for removing assert statements should be eliminated. I simply don't use assert statements at all. From stefan_ml at behnel.de Thu Jan 28 03:00:23 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 28 Jan 2010 09:00:23 +0100 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <4b614417$0$6715$9b4e6d93@newsspool2.arcor-online.net> Benjamin Kaplan, 27.01.2010 22:25: > On Wed, Jan 27, 2010 at 3:56 PM, John Nagle wrote: >> 2. Python 3 is supported by multiple Python implementations. >> >> FALSE - Only CPython supports 3.x. Iron Python, Unladen Swallow, >> PyPy, and Jython have all stayed with 2.x versions of Python. >> > When Python 2.6 came out, Jython was still on 2.2. The difference > between 2.2 and 2.6 is almost as big of a difference as between 2.6 > and 3.0. >From an implementors point of view, it's actually quite the opposite. Most syntax features of Python 3 can be easily implemented on top of an existing Py2 Implementation (we have most of them in Cython already, and I really found them fun to write), and the shifting-around in the standard library can hardly be called non-trivial. All the hard work that went into the design of CPython 3.x (and into its test suite) now makes it easy to just steal from what's there already. The amount of work that the Jython project put into catching up from 2.1 to 2.5/6 (new style classes! generators!) is really humongous compared to the adaptations that an implementation needs to do to support Python 3 code. I have great respect for the Jython project for what they achieved in the last couple of years. (I also have great respect for the IronPython project for fighting the One Microsoft Way into opening up, but that's a different kind of business.) If there was enough interest from the respective core developers, I wouldn't be surprised if we had more than one 'mostly compatible' alternative Python 3 implementation in a couple of months. But it's the obvious vicious circle business. As long as there aren't enough important users of Py3, alternative implementations won't have enough incentives to refocus their scarce developer time. Going for 2.6/7 first means that most of the Py3 work gets done anyway, so it'll be even easier then. That makes 2.6->2.7->3.2/3 the most natural implementation path. (And that, again, makes it a *really* good decision that 2.7 will be the last 2.x release line.) >> 3. Python 3 is supported by most 3rd party Python packages. >> >> FALSE - it's not supported by MySQLdb, OpenSSL, feedparser, etc. >> >> Arguably, Python 3 has been rejected by the market. Instead, there's >> now Python 2.6, Python 2.7, and Python 2.8. Python 3 has turned into >> a debacle like Perl 6, now 10 years old. > > Give the package maintainers time to update. There were some pretty > big changes to the C API. Most of the major 3rd party packages like > numpy and MySQLdb have already commited to having a Python 3 version. > They just haven't gotten them out yet. I second that. NumPy has already announced that they'll rewrite some of their code in Cython to make it more maintainable and portable (and to simplify the port to Py3, I guess). Other projects will equally well find their ways to get their code ready. It's just a matter of time. I think there's enough pressure on the maintainers by now (both from users and from personal pride) to consider their packages tainted if they aren't portable enough to run on all major (C)Python versions (and Py3.1 certainly is one of them), even if they don't use them themselves. That appears to be an impressively good incentive. Stefan From durumdara at gmail.com Thu Jan 28 03:07:00 2010 From: durumdara at gmail.com (Durumdara) Date: Thu, 28 Jan 2010 09:07:00 +0100 Subject: Data exchange between Delphi and Python (Win) Message-ID: <9e384ef61001280007j38f231e4we085e9af6d1b725d@mail.gmail.com> Hi! I have an exotic db, with exotic drivers, and it have buggy ODBC driver. But I have native driver - under Delphi. I need to access this DB under Pylons (or mod_python). I wrote one solution that working with XML. But I search for easier way to transform and move data between apps. I saw Python for Delphi, but the installer is showing only Python 2.3 as selectable engine. I think to COM/OLE, because it is accessable from all program, and I think to DLL (but DLL have problematic parameterisation). The input data (what Delphi got) are SQL commands, and the output are the rows (if got). What do you thinking about it? Have anyone experience in this theme? Thanks for it! dd -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben+python at benfinney.id.au Thu Jan 28 03:08:41 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 28 Jan 2010 19:08:41 +1100 Subject: Library support for Python 3.x References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> <7xk4v2g2f3.fsf@ruckus.brouhaha.com> Message-ID: <873a1q3b3q.fsf@benfinney.id.au> David Cournapeau writes: > Unstable may be strong - every minor version of python has a lifespan > of several years. But yes, that's an hindrance for packagers: you need > to package binaries for every minor version of python It's important to note that this is mitigated, ironically enough, by intentionally targeting a minimum Python minor version because the code base makes use of Python features not available in older versions. That is, any minor version of Python that doesn't have the features your code base uses can be ignored (given the set of supported versions is explicitly declared) ? and hence one doesn't need to package binaries for every minor version. -- \ ?Our products just aren't engineered for security.? ?Brian | `\ Valentine, senior vice-president of Microsoft Windows | _o__) development, 2002 | Ben Finney From stefan_ml at behnel.de Thu Jan 28 03:11:31 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 28 Jan 2010 09:11:31 +0100 Subject: myths about python 3 In-Reply-To: <87zl3z2p63.fsf@benfinney.id.au> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <87zl3z2p63.fsf@benfinney.id.au> Message-ID: <4b6146b3$0$6715$9b4e6d93@newsspool2.arcor-online.net> Ben Finney, 27.01.2010 22:50: > Christian Heimes writes: > >> John Nagle wrote: >>> 1. Python 3 is supported by major Linux distributions. >>> >>> FALSE - most distros are shipping with Python 2.4, or 2.5 at best. >> You are wrong. Modern versions of Debian / Ubuntu are using Python >> 2.6. > > Only if by ?modern? you mean ?not released yet?. > > The latest stable Debian (Debian 5.0, Lenny) has only Python 2.4 and > Python 2.5. It does not have Python 2.6 at all, and until this month > Python 2.6 was not even in the in-development suite of Debian. 'Stable Debian' has a long tradition of being late and outdated on arrival. That doesn't mean you can't use existing Debian packages on it. And there certainly are .deb packages available for Py3.1.1 - Ubuntu uses them, and other Debian based distributions do, too. And Ubuntu Karmic definitely uses Py2.6 as 'python'. Stefan From no.email at nospam.invalid Thu Jan 28 03:37:54 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 28 Jan 2010 00:37:54 -0800 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> <7xockfjcsv.fsf@ruckus.brouhaha.com> <6b89dc67-ceee-4b83-a11f-c7376185dcd1@b9g2000pri.googlegroups.com> Message-ID: <7xy6ji62vx.fsf@ruckus.brouhaha.com> Jonathan Gardner writes: > If you're going to have statements, you're going to need the null > statement. That's "pass". Why? Expressions are statements, so you could just say "pass" (in quotes, denoting a string literal), or 0, or None, os anything else like that, instead of having a special statement. From no.email at nospam.invalid Thu Jan 28 03:40:09 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 28 Jan 2010 00:40:09 -0800 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <4b614417$0$6715$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <7xtyu662s6.fsf@ruckus.brouhaha.com> Stefan Behnel writes: > The amount of work that the Jython project put into catching up from 2.1 to > 2.5/6 (new style classes! generators!) is really humongous compared to the > adaptations that an implementation needs to do to support Python 3 code. I wonder whether Jython could borrow code from Clojure for some of this stuff, to save a little work. Cross-fertilization between free software projects is usually a good thing. From stefan_ml at behnel.de Thu Jan 28 03:40:42 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 28 Jan 2010 09:40:42 +0100 Subject: Library support for Python 3.x In-Reply-To: References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> Message-ID: <4b614d8a$0$6718$9b4e6d93@newsspool2.arcor-online.net> David Cournapeau, 28.01.2010 06:58: > On Thu, Jan 28, 2010 at 7:38 AM, Terry Reedy wrote: > >> For a windows user who depends on pre-built binaries, every new release >> breaks *every* library that is not pure Python and needs to be compiled. > > That's not windows specific - most packages which distribute binary > packages need to package binaries for every minor version (2.4, 2.5, > etc...). That's certainly the case for numpy and scipy. Python does > not have a stable ABI across minor releases, only micro releases. That doesn't completely match my experience. It's true that there is no guarantee that the ABI will stay compatible, but when you compile lxml against Py2.4 on a 32bit machine, it will continue to import in Py2.5 and (IIRC) Py2.6. It won't be as fast and it won't use some newer features, but it will work. Don't remember my experience with 2.3, though. It obviously can't work the other way round, i.e. when compiling against 2.6, it will never work in 2.5 or earlier. But there is definitely a certain degree of ABI compatibility available. Stefan From cournape at gmail.com Thu Jan 28 03:48:47 2010 From: cournape at gmail.com (David Cournapeau) Date: Thu, 28 Jan 2010 17:48:47 +0900 Subject: Library support for Python 3.x In-Reply-To: <873a1q3b3q.fsf@benfinney.id.au> References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> <7xk4v2g2f3.fsf@ruckus.brouhaha.com> <873a1q3b3q.fsf@benfinney.id.au> Message-ID: <5b8d13221001280048u10596800xfe68108902b0ecb3@mail.gmail.com> On Thu, Jan 28, 2010 at 5:08 PM, Ben Finney wrote: > > It's important to note that this is mitigated, ironically enough, by > intentionally targeting a minimum Python minor version because the code > base makes use of Python features not available in older versions. > > That is, any minor version of Python that doesn't have the features your > code base uses can be ignored (given the set of supported versions is > explicitly declared) ? and hence one doesn't need to package binaries > for every minor version. This has nothing to do whatsoever with feature, since we are talking about ABI issues. David From cournape at gmail.com Thu Jan 28 03:54:02 2010 From: cournape at gmail.com (David Cournapeau) Date: Thu, 28 Jan 2010 17:54:02 +0900 Subject: Library support for Python 3.x In-Reply-To: <4b614d8a$0$6718$9b4e6d93@newsspool2.arcor-online.net> References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> <4b614d8a$0$6718$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <5b8d13221001280054w54d07eb3s6be5384644c01b38@mail.gmail.com> On Thu, Jan 28, 2010 at 5:40 PM, Stefan Behnel wrote: > > That doesn't completely match my experience. It's true that there is no > guarantee that the ABI will stay compatible, but when you compile lxml > against Py2.4 on a 32bit machine, it will continue to import in Py2.5 and > (IIRC) Py2.6. It won't be as fast and it won't use some newer features, but > it will work. Don't remember my experience with 2.3, though. Importing fine is a very low expectation for ABI compatibility :) Since python does not make ABI guarantees between minor releases, you don't know whether some structures layouts are changed between versions, and in general, tracking crashes due to those is no fun. It really depends on how much you depend on the C API, but for something extensive like NumPy, I don't think it would ever work. So yes, you could say "just try and if it crashes, check that it is not ABI-related". In practice, this is very poor engineering in my book... David From alfps at start.no Thu Jan 28 04:12:18 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 28 Jan 2010 10:12:18 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: * Steven D'Aprano: > On Wed, 27 Jan 2010 18:29:25 +0100, Alf P. Steinbach wrote: > >> The main problem with the incompatibility is for porting code, not for >> writing code from scratch. > > Correct. It's a trivial problem, but still a problem. > >> It's also a problem wrt. learning the language. > > This makes no sense. Why is it harder to learn > > print(x, y, z) > > than this? > > print x, y, z I think it's actually easier to learn just the 3.x form. But it's more difficult to learn that there are /two different/ syntactic forms, depending on which version of Python you're using and in addition depending on __future__ declarations! E.g., picking up some example code from the web, and it does not work... > The first case is like the other functions you have to learn, like len(). > In fact, many newbies to Python put unneeded parentheses around arguments > to print simply because they assume it is a function. > > I would argue that the new print function is *simpler* to learn. It is > more consistent with other built-ins, and has fewer magic idioms to > learn. Yes, yes, yes, I agree. > Instead of: > > print >>fileObj, x, y, z > > you use regular function syntax with a meaningful keyword: > > print(x, y, z, file=fileObj) > > If you want suppress the newline at the end of each print: > > print x, y, z, # note the final comma > > compared to: > > print(x, y, z, end='') Actually I thought the final comma thing was nice. It was like Basic. I think the 2.x 'print' must have been modeled on Basic's 'print'. > If you want to change the space between elements, instead of: > > sys.stdout.write(str(x) + "*" + str(y) + "*" + str(z) + '\n') > > you use: > > print(x, y, z, sep='*') > > > If you want to override the behaviour of print in a module, instead of > having to edit the source code of the module (which might not even be > available), all you need to do is monkey-patch it: > > import module > module.print = myprint >>> import builtins >>> >>> org_print = print >>> builtins.print = 666 >>> >>> print( "trallala" ) Traceback (most recent call last): File "", line 1, in TypeError: 'int' object is not callable >>> org_print( "but is that really so smart?" ) but is that really so smart? >>> _ Cheers, - Alf From no.email at nospam.invalid Thu Jan 28 04:21:23 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 28 Jan 2010 01:21:23 -0800 Subject: Library support for Python 3.x References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> <4b614d8a$0$6718$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <7xiqamziss.fsf@ruckus.brouhaha.com> David Cournapeau writes: > So yes, you could say "just try and if it crashes, check that it is > not ABI-related". In practice, this is very poor engineering in my > book... I just looked at PEP 384 and I don't see anything in it about version numbers in the interfaces. I certainly think something like that should be added if it's not too late. Basically any extension module should check that the CPython loading it is new enough, and CPython should (when feasible) continue to support old interfaces when changes are made. This is pretty standard stuff as done in COM, Java, and presumably .NET, along with many communications protocols. From stefan_ml at behnel.de Thu Jan 28 04:25:26 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 28 Jan 2010 10:25:26 +0100 Subject: Library support for Python 3.x In-Reply-To: References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> <4b614d8a$0$6718$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <4b615807$0$6724$9b4e6d93@newsspool2.arcor-online.net> David Cournapeau, 28.01.2010 09:54: > On Thu, Jan 28, 2010 at 5:40 PM, Stefan Behnel wrote: > >> That doesn't completely match my experience. It's true that there is no >> guarantee that the ABI will stay compatible, but when you compile lxml >> against Py2.4 on a 32bit machine, it will continue to import in Py2.5 and >> (IIRC) Py2.6. It won't be as fast and it won't use some newer features, but >> it will work. Don't remember my experience with 2.3, though. > > Importing fine is a very low expectation for ABI compatibility :) Ok, I should have said "imports and runs its test suite successfully". I just wanted to add a "works for me" to counter your rather pessimistic comments. > Since python does not make ABI guarantees between minor releases, you > don't know whether some structures layouts are changed between > versions, and in general, tracking crashes due to those is no fun. It > really depends on how much you depend on the C API, but for something > extensive like NumPy, I don't think it would ever work. I wouldn't be so sure that NumPy is so much more "extensive" in C-API usage than lxml. > So yes, you could say "just try and if it crashes, check that it is > not ABI-related". In practice, this is very poor engineering in my > book... Well, I don't know if there is any 'official' core developer statement regarding ABI compatibility, but I know that Guido doesn't take it easy to break it for a release. He tried pretty hard to keep it up for 2.5->2.6, at least, even if he was aware that it would be futile for 2.x->3.x. Stefan From duncan.booth at invalid.invalid Thu Jan 28 04:45:21 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 28 Jan 2010 09:45:21 GMT Subject: Ad hoc lists vs ad hoc tuples References: <8a57c64f-0c67-4ec1-b4be-34d76c7e7571@q4g2000yqm.googlegroups.com> Message-ID: Terry Reedy wrote: > Constant tuples (a tuple whose members are all seen as constants by the > compiler) are now pre-compiled and constructed once and put into the > code object as such rather than re-constructed with each run of the code. Sadly that's not entirely accurate. Tuples whose members are all simple constants are pre-compiled, but here's an example of a tuple whose members are all constants but doesn't get optimised: >>> def foo(): data = ( ('hello', 42), ('world', 24), ) return data >>> import dis >>> dis.dis(foo) 3 0 LOAD_CONST 5 (('hello', 42)) 4 3 LOAD_CONST 6 (('world', 24)) 6 BUILD_TUPLE 2 9 STORE_FAST 0 (data) 6 12 LOAD_FAST 0 (data) 15 RETURN_VALUE -- Duncan Booth http://kupuguy.blogspot.com From gandalf at shopzeus.com Thu Jan 28 05:50:16 2010 From: gandalf at shopzeus.com (Laszlo Nagy) Date: Thu, 28 Jan 2010 15:20:16 +0430 Subject: Portable way to tell if a process is still alive In-Reply-To: References: Message-ID: <4B616BE8.7080202@shopzeus.com> >> Suppose we have a program that writes its process id into a pid file. >> Usually the program deletes the pid file when it exists... But in some >> cases (for example, killed with kill -9 or TerminateProcess) pid file is >> left there. I would like to know if a process (given with its process >> id) is still running or not. I know that this can be done with OS >> specific calls. But that is not portable. It can also be done by >> executing "ps -p 23423" with subprocess module, but that is not portable >> either. Is there a portable way to do this? >> >> If not, would it be a good idea to implement this (I think very >> primitive) function in the os module? >> > > Not only is there no way to do it portably, there is no way to do it > reliably for the general case. The problem is that processes do not have > unique identifiers. A PID only uniquely identifies a running process; once > the process terminates, its PID becomes available for re-use. > Non-general case: the process is a service and only one instance should be running. There could be a pid file left on the disk. It is possible to e.g. mount procfs, and check if the given PID belongs to a command line / executed program that is in question. It cannot be guaranteed that a service will always delete its pid file when it exists. It happens for example, somebody kills it with "kill -9" or exits on signal 11 etc. It actually did happened to me, and then the service could not be restarted because the PID file was there. (It is an error to run two instances of the same service, but it is also an error to not run it....) Whan I would like to do upon startup is to check if the process is already running. This way I could create a "guardian" that checks other services, and (re)starts them if they stopped working. And no, it is not a solution to write "good" a service that will never stop, because: 1. It is particulary not possible in my case - there is a software bug in a third party library that causes my service exit on various wreid signals. 2. It is not possible anyway. There are users killing processes accidentally, and other unforeseen bugs. 3. In a mission critical environment, I would use a guardian even if guarded services are not likely to stop I understand that this is a whole different question now, and possibly there is no portable way to do it. Just I wonder if there are others facing a similar problem here. Any thoughs or comments - is it bad that I would like to achieve? Is there a better approach? Thanks, Laszlo From denis-bz-gg at t-online.de Thu Jan 28 06:54:43 2010 From: denis-bz-gg at t-online.de (denis) Date: Thu, 28 Jan 2010 03:54:43 -0800 (PST) Subject: Total maximal size of data References: Message-ID: <7647d906-f9dd-43cc-9732-11f0d4e46669@n7g2000yqb.googlegroups.com> On Jan 25, 8:05?pm, Alexander Moibenko wrote: > I have a simple question to which I could not find an answer. > What is the [total maximal] size of list including size of its elements? Beware, sys.getsizeof(alist) is 4*len(alist) + a bit, regardless of alists's contents ?! See http://stackoverflow.com/questions/2117255/python-deep-getsizeof-list-with-contents cheers -- denis From sg552 at hotmail.co.uk Thu Jan 28 07:11:05 2010 From: sg552 at hotmail.co.uk (Rotwang) Date: Thu, 28 Jan 2010 12:11:05 +0000 Subject: List weirdness - what the heck is going on here? In-Reply-To: <2010012721183416807-angrybaldguy@gmailcom> References: <2010012721183416807-angrybaldguy@gmailcom> Message-ID: Owen Jacobson wrote: > On 2010-01-27 21:06:28 -0500, Rotwang said: > >> Hi all, I've been trying to make a class with which to manipulate >> sound data, and have run into some behaviour I don't understand which >> I hope somebody here can explain. The class has an attribute called >> data, which is a list with two elements, one for each audio channel, >> each of which is a list containing the audio data for that channel. It >> also has various methods to write data such as sine waves and so on, >> and a method to insert data from one sound at the start of data from >> another. Schematically, the relevant bits look like this: >> >> class sound: >> def f(self): >> self.data = [[0]]*2 > > Consider that this is equivalent to > > def f(self): > x = [0] > self.data = [x, x] > > self.data is now a list containing two references to the list referenced > by x -- so changes via either of the elements of self.data will affect > both elements. Your comprehension version creates a list containing two > distinct list objects, so this doesn't happen. Thanks, and likewise to everyone else who replied. From lie.1296 at gmail.com Thu Jan 28 07:12:22 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Thu, 28 Jan 2010 23:12:22 +1100 Subject: interaction of mode 'r+', file.write(), and file.tell(): a bug or undefined behavior? Message-ID: <4b617f4a$1@dnews.tpgi.com.au> In the code: """ f = open('input.txt', 'r+') for line in f: s = line.replace('python', 'PYTHON') # f.tell() f.write(s) """ When f.tell() is commented, 'input.txt' does not change; but when uncommented, the f.write() succeeded writing into the 'input.txt' (surprisingly, but not entirely unexpected, at the end of the file). $ ##################################### $ $ cp orig.txt input.txt $ cat input.txt abcde abc python abc python abc python $ python Python 2.6.4 (r264:75706, Jan 12 2010, 05:24:27) [GCC 4.3.4] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> f = open('input.txt', 'r+') >>> for line in f: ... s = line.replace('python', 'PYTHON') ... f.write(s) ... >>> $ cat input.txt abcde abc python abc python abc python $ $ ##################################### $ $ cp orig.txt input.txt $ cat input.txt abcde abc python abc python abc python $ python Python 2.6.4 (r264:75706, Jan 12 2010, 05:24:27) [GCC 4.3.4] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> f = open('input.txt', 'r+') >>> for line in f: ... s = line.replace('python', 'PYTHON') ... f.tell() ... f.write(s) ... 39 45 60 >>> $ cat input.txt abcde abc python abc python abc python abcde abc PYTHON abc $ $ ##################################### Do you think this should be a bug or undefined behavior governed by the underlying OS and C library? Shouldn't file.tell() be purely informational, and not have side effect? The machine is Gentoo (amd64, gcc-4.3.4, glibc-2.10.1-r1), Linux (2.6.31-gentoo-r6), and Python 2.6.4 From robert.kiwanuka at gmail.com Thu Jan 28 07:29:01 2010 From: robert.kiwanuka at gmail.com (kiwanuka) Date: Thu, 28 Jan 2010 04:29:01 -0800 (PST) Subject: is there any alternative to savefig? Message-ID: Hi all, I wonder if anyone knows any alternative function in pylab (or otherwise) that could be used to save an image. My problem is as follows: --------------- from pylab import * ... figure(1) fig1 = gca() figure(2) fig2 = gca() figure(3) fig3 = gca() for i,data_file in enumerate(data_file_list): time,x, y,x2, y2 = read_csv_file_4(open (data_file),elements=num_of_elements) fig1.plot(-x,-y,color=colours[i],label=labellist[i]) fig2.plot(time,-y,color=colours[i],label=labellist[i]) fig3.plot(time,-x,color=colours[i],label=labellist[i]) fig1.legend(loc='best') fig1.set_title("y1 - x1") fig1.set_ylabel("y1") fig1.set_xlabel("x1") #savefig("y1-x1.png") fig2.legend(loc='best') fig2.set_title("y1 - time") fig2.set_ylabel("y1") fig2.set_xlabel("time[s]") #savefig("y1-time.png") fig3.legend(loc='best') fig3.set_title("x1 - time") fig3.set_ylabel("x1") fig3.set_xlabel("time[s]") #savefig("x1-time.png") show() --------------------------- In the above code, I read multiple data files and plot three separate figures. Now I would like to save each of the figures to a file as the commented savefig satements suggest. The trouble is that if I uncomment all those savefig statements, I get three saved images all containing the plot belonging to figure(3), which was the last figure declared. I understand this to be happening because savefig will save the "current" figure, which in this case happens to be the last one declared. If I could do something like fig1.savefig("y1-x1.png") or savefig("y1- x1.png").fig1, this would solve the problem but I'm not aware of any such methods or modules to enable this. This is thus a flaw in the general design/implementation of the savefig function, but is there an alternative function to enable me achieve what I need? Is there perhaps a possible tweak to savefig to make it do the same? Thanks in advance, Robert From fetchinson at googlemail.com Thu Jan 28 07:41:47 2010 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Thu, 28 Jan 2010 13:41:47 +0100 Subject: Library support for Python 3.x In-Reply-To: <4b60a6c9$0$1598$742ec2ed@news.sonic.net> References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <4b60a6c9$0$1598$742ec2ed@news.sonic.net> Message-ID: >> I'm going to be starting some new Python projects in Python 2.6, but am >> concerned that at least three of the libraries I will be >> using--pycrypto, paramiko and feedparser--are not currently supported in >> Python 3.x. The authors of these programs have not given any indication >> that work is underway to support Python 3.x. Eventually I plan to >> migrate to Python 3.x. > > Maybe by 2015 or so, that might be feasible. Wait until Python 3.x > ships as standard with major Linux distros. Right now, 2.4 or 2.5 is > the production version of Python. You keep repeating this nonsense even though it has been pointed out to you in a neighbouring thread that many (most?) of the main linux distros ship python 2.6 and not 2.5 or 2.4. For example Fedora 11 and 12 both ship python 2.6, others mentioned lots of other examples in said other thread, anyone can look them up. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From richard.lamboj at bilcom.at Thu Jan 28 08:25:30 2010 From: richard.lamboj at bilcom.at (Richard Lamboj) Date: Thu, 28 Jan 2010 14:25:30 +0100 Subject: Killing a Thread Message-ID: <201001281425.30434.richard.lamboj@bilcom.at> Hello, which Method is better to kill a Thread? Using Thread Events, or a raising a Exception? Maybe someone has a small example for me? Kind Regards, Richi From spam2008 at nney.com Thu Jan 28 08:31:36 2010 From: spam2008 at nney.com (D HANNEY) Date: Thu, 28 Jan 2010 05:31:36 -0800 (PST) Subject: Generic proxy (that proxies methods like __iter__) References: <635f48e9-b8de-402b-8215-1e5c26a33ed1@k35g2000yqb.googlegroups.com> Message-ID: <40f612b7-8b9c-4cf8-81df-65d4386e52de@r24g2000yqd.googlegroups.com> On Jan 27, 3:07?pm, Arnaud Delobelle wrote: > On 27 Jan, 14:41, D HANNEY wrote: > [...] > > See [1] for an explanation. ?Here is an idea: you could get round that > by generating a class on the fly, if you don't mind changing the class > of the object (untested): > > def noguardproxy(obj): > ? ? class NoGuardProxy(type(obj)): > ? ? ? ? def __enter__(self): > ? ? ? ? ? ? ? ?return self > ? ? ? ? def __exit__(self, type, value, traceback): > ? ? ? ? ? ? ? ?return False > ? ? obj.__class__ = NoGuardProxy > ? ? return obj > > If you want to keep obj untouched, you could add the line > > ? ? obj = copy(obj) > > before changing the class. > > -- > Arnaud > > [1]http://docs.python.org/reference/datamodel.html#new-style-special-lookup Thank you! Your solution works if I change "type(obj)" to say "obj.__class__". If I don't make this change Python complains "TypeError: Error when calling the metaclass bases type 'instance' is not an acceptable base type". So, I've got something that works but I don't understand why it works. I've read your docs.python reference but that hasn't helped yet. It might come to me but in the meantime, can you (or anyone) offer an explanation for this? Thanks in advance, David From briandenzer at gmail.com Thu Jan 28 08:40:09 2010 From: briandenzer at gmail.com (Brian D) Date: Thu, 28 Jan 2010 05:40:09 -0800 (PST) Subject: Stuck on a three word street name regex References: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> <0a712e20-facf-4a9b-8852-6136a8bad7dc@n7g2000yqb.googlegroups.com> Message-ID: <4e222e90-e088-4a82-a7d2-c62f509e2517@g29g2000yqe.googlegroups.com> > > [snip] > > Regex doesn't gain you much. I'd split the string and then fix the parts > > as necessary: > > > ?>>> def parse_address(address): > > ... ? ? parts = address.split() > > ... ? ? if parts[-2] == "S": > > ... ? ? ? ? parts[1 : -1] = [parts[-2]] + parts[1 : -2] > > ... ? ? parts[1 : -1] = [" ".join(parts[1 : -1])] > > ... ? ? return parts > > ... > > ?>>> print parse_address("1000 RAMPART S ST") > > ['1000', 'S RAMPART', 'ST'] > > ?>>> print parse_address("100 JOHN CHURCHILL CHASE ?ST") > > ['100', 'JOHN CHURCHILL CHASE', 'ST'] > > This is a nice approach I wouldn't have thought to pursue. I've never > seen this referencing of list elements in reverse order with negative > values, so that certainly expands my knowledge of Python. Of course, > I'd want to check for other directionals -- probably with a list > check, e.g., > > if parts[-2] in ('E', 'W', 'N', 'S'): > > Thanks for sharing your approach. After studying this again today, I realized the ingeniousness of reverse slicing the list (or perhaps right slicing) -- that one doesn't have to worry about the number of words in the string. To translate for those who may follow, the expression "parts[1 : -1]" means gather list items from position one in the list (index position 2) to one index position before the end of the list. The value in this is that we already know the first list element after a split() will be the street number. The last element will be the street type. Everything in between, no matter how many words, will be the street name -- excepting, of course, the instances where there's a street direction added in, as captured in example above. A very nice solution. Thanks again! From briandenzer at gmail.com Thu Jan 28 08:48:48 2010 From: briandenzer at gmail.com (Brian D) Date: Thu, 28 Jan 2010 05:48:48 -0800 (PST) Subject: Stuck on a three word street name regex References: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> <0a712e20-facf-4a9b-8852-6136a8bad7dc@n7g2000yqb.googlegroups.com> <4e222e90-e088-4a82-a7d2-c62f509e2517@g29g2000yqe.googlegroups.com> Message-ID: <1b7dba4d-3f12-4d31-8e9a-b36cd4f40217@u41g2000yqe.googlegroups.com> On Jan 28, 7:40?am, Brian D wrote: > > > [snip] > > > Regex doesn't gain you much. I'd split the string and then fix the parts > > > as necessary: > > > > ?>>> def parse_address(address): > > > ... ? ? parts = address.split() > > > ... ? ? if parts[-2] == "S": > > > ... ? ? ? ? parts[1 : -1] = [parts[-2]] + parts[1 : -2] > > > ... ? ? parts[1 : -1] = [" ".join(parts[1 : -1])] > > > ... ? ? return parts > > > ... > > > ?>>> print parse_address("1000 RAMPART S ST") > > > ['1000', 'S RAMPART', 'ST'] > > > ?>>> print parse_address("100 JOHN CHURCHILL CHASE ?ST") > > > ['100', 'JOHN CHURCHILL CHASE', 'ST'] > > > This is a nice approach I wouldn't have thought to pursue. I've never > > seen this referencing of list elements in reverse order with negative > > values, so that certainly expands my knowledge of Python. Of course, > > I'd want to check for other directionals -- probably with a list > > check, e.g., > > > if parts[-2] in ('E', 'W', 'N', 'S'): > > > Thanks for sharing your approach. > > After studying this again today, I realized the ingeniousness of > reverse slicing the list (or perhaps right slicing) -- that one > doesn't have to worry about the number of words in the string. > > To translate for those who may follow, the expression "parts[1 : -1]" > means gather list items from position one in the list (index position > 2) to one index position before the end of the list. The value in this > is that we already know the first list element after a split() will be > the street number. The last element will be the street type. > Everything in between, no matter how many words, will be the street > name -- excepting, of course, the instances where there's a street > direction added in, as captured in example above. > > A very nice solution. Thanks again! Correction: [snip] the expression "parts[1 : -1]" means gather list items from the second element in the list (index value 1) to one index position before the end of the list. [snip] From spam2008 at nney.com Thu Jan 28 08:54:16 2010 From: spam2008 at nney.com (D HANNEY) Date: Thu, 28 Jan 2010 05:54:16 -0800 (PST) Subject: Generic proxy (that proxies methods like __iter__) References: <635f48e9-b8de-402b-8215-1e5c26a33ed1@k35g2000yqb.googlegroups.com> <40f612b7-8b9c-4cf8-81df-65d4386e52de@r24g2000yqd.googlegroups.com> Message-ID: <5c153055-d88b-49d2-8938-2efac70f0548@k19g2000yqc.googlegroups.com> On Jan 28, 1:31 pm, D HANNEY wrote: > > Your solution works if I change "type(obj)" to say "obj.__class__". > If I don't make this change Python complains "TypeError: Error when > calling the metaclass bases type 'instance' is not an acceptable base > type". > So, I've got something that works but I don't understand why it works. > I've read your docs.python reference but that hasn't helped yet. It > might come to me but in the meantime, can you (or anyone) offer an > explanation for this? > > Thanks in advance, > > David Oh it looks like this is py3K thing. Python 3.0.1+ (r301:69556, Apr 15 2009, 15:59:22) >>> from io import StringIO >>> type(StringIO("x")) >>> StringIO("x").__class__ >>> Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41) >>> from StringIO import StringIO >>> type(StringIO("x")) >>> StringIO("x").__class__ >>> I was using 2.6.2. Your suggestion would have been fine in py3k. Even if I stuck with 2.6.2 and StringIO was a "new style class" there then it would have been fine ... but it isn't. As it stands __class__ seems the only way that works on both 3k and 2k. Bit weird, but I get it now. Thanks again! David From lie.1296 at gmail.com Thu Jan 28 09:27:59 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Fri, 29 Jan 2010 01:27:59 +1100 Subject: Stuck on a three word street name regex In-Reply-To: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> References: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> Message-ID: <4b619f12$1@dnews.tpgi.com.au> On 01/28/10 11:28, Brian D wrote: > I've tackled this kind of problem before by looping through a patterns > dictionary, but there must be a smarter approach. > > Two addresses. Note that the first has incorrectly transposed the > direction and street name. The second has an extra space in it before > the street type. Clearly done by someone who didn't know how to > concatenate properly -- or didn't care. > > 1000 RAMPART S ST > > 100 JOHN CHURCHILL CHASE ST > > I want to parse the elements into an array of values that can be > inserted into new database fields. > > Anyone who loves solving these kinds of puzzles care to relieve my > frazzled brain? > > The pattern I'm using doesn't keep the "CHASE" with the "JOHN > CHURCHILL": How does the following perform? pat = re.compile(r'(?P\d+)\s+(?P[A-Z\s]+)\s+(?PN|S|W|E|)\s+(?PST|RD|AVE?|)$') or more legibly: pat = re.compile( r''' (?P \d+ ) #M series of digits \s+ (?P [A-Z\s]+ ) #M one-or-more word \s+ (?P S?E|SW?|N?W|NE?| ) #O direction or nothing \s+ (?P ST|RD|AVE? ) #M street type $ #M END ''', re.VERBOSE) From exarkun at twistedmatrix.com Thu Jan 28 09:33:23 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Thu, 28 Jan 2010 14:33:23 -0000 Subject: Portable way to tell if a process is still alive In-Reply-To: <4B616BE8.7080202@shopzeus.com> References: <4B616BE8.7080202@shopzeus.com> Message-ID: <20100128143323.1898.2126720850.divmod.xquotient.889@localhost.localdomain> On 10:50 am, gandalf at shopzeus.com wrote: > >>>Suppose we have a program that writes its process id into a pid file. >>>Usually the program deletes the pid file when it exists... But in >>>some cases (for example, killed with kill -9 or TerminateProcess) pid >>>file is left there. I would like to know if a process (given with its >>>process id) is still running or not. I know that this can be done >>>with OS specific calls. But that is not portable. It can also be done >>>by executing "ps -p 23423" with subprocess module, but that is not >>>portable either. Is there a portable way to do this? >>> >>>If not, would it be a good idea to implement this (I think very >>>primitive) function in the os module? >> >>Not only is there no way to do it portably, there is no way to do it >>reliably for the general case. The problem is that processes do not >>have >>unique identifiers. A PID only uniquely identifies a running process; >>once >>the process terminates, its PID becomes available for re-use. > >Non-general case: the process is a service and only one instance should >be running. There could be a pid file left on the disk. It is possible >to e.g. mount procfs, and check if the given PID belongs to a command >line / executed program that is in question. It cannot be guaranteed >that a service will always delete its pid file when it exists. It >happens for example, somebody kills it with "kill -9" or exits on >signal 11 etc. It actually did happened to me, and then the service >could not be restarted because the PID file was there. (It is an error >to run two instances of the same service, but it is also an error to >not run it....) Whan I would like to do upon startup is to check if the >process is already running. This way I could create a "guardian" that >checks other services, and (re)starts them if they stopped working. > >And no, it is not a solution to write "good" a service that will never >stop, because: > >1. It is particulary not possible in my case - there is a software bug >in a third party library that causes my service exit on various wreid >signals. >2. It is not possible anyway. There are users killing processes >accidentally, and other unforeseen bugs. >3. In a mission critical environment, I would use a guardian even if >guarded services are not likely to stop > >I understand that this is a whole different question now, and possibly >there is no portable way to do it. Just I wonder if there are others >facing a similar problem here. Any thoughs or comments - is it bad that >I would like to achieve? Is there a better approach? I've been pondering using a listening unix socket for this. As long as the process is running, a client can connect to the unix socket. As soon as the process isn't, no matter the cause, clients can no longer connect to it. A drawback of this approach in some cases is probably that the process should be accepting these connections (and then dropping them). This may not always be easy to add to an existing app. Jean-Paul From anthony.tolle at gmail.com Thu Jan 28 09:38:54 2010 From: anthony.tolle at gmail.com (Anthony Tolle) Date: Thu, 28 Jan 2010 06:38:54 -0800 (PST) Subject: interaction of mode 'r+', file.write(), and file.tell(): a bug or undefined behavior? References: <4b617f4a$1@dnews.tpgi.com.au> Message-ID: <35ee0c46-f4e5-4dcb-ac32-7fe9d0b68104@15g2000vbg.googlegroups.com> On Jan 28, 7:12?am, Lie Ryan wrote: > In the code: > > """ > f = open('input.txt', 'r+') > for line in f: > ? ? s = line.replace('python', 'PYTHON') > ? ? # f.tell() > ? ? f.write(s) > """ > [snip] My guess is that there are a few possible problems: 1) In this case, writing to file opened with 'r+' without an explicit f.seek is probably not a good idea. The file iterator (for line in f) uses a readahead buffer, which means you can't guarantee what the current file position will be. 2) It may be necessary to do an explicit f.flush or f.close when writing to an 'r+' file. In your case, the close should automatically happen when the f object falls out of scope, which tells me that were still looking at some other problem, like not using f.seek 3) It is possible that f.tell implicitly flushes buffers used by the file object. That would explain why uncommenting the f.tell causes the writes to show up. What are you trying to accomplish? Overwrite the original file, or append to it? If you want to overwrite the file, it may be better to generate a new file, delete the old one, then rename the new one. If you want to append, then it would be better to open the file with append mode ('a') From alfps at start.no Thu Jan 28 10:00:37 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 28 Jan 2010 16:00:37 +0100 Subject: interaction of mode 'r+', file.write(), and file.tell(): a bug or undefined behavior? In-Reply-To: <35ee0c46-f4e5-4dcb-ac32-7fe9d0b68104@15g2000vbg.googlegroups.com> References: <4b617f4a$1@dnews.tpgi.com.au> <35ee0c46-f4e5-4dcb-ac32-7fe9d0b68104@15g2000vbg.googlegroups.com> Message-ID: * Anthony Tolle: > On Jan 28, 7:12 am, Lie Ryan wrote: >> In the code: >> >> """ >> f = open('input.txt', 'r+') >> for line in f: >> s = line.replace('python', 'PYTHON') >> # f.tell() >> f.write(s) >> """ >> [snip] > > My guess is that there are a few possible problems: > > 1) In this case, writing to file opened with 'r+' without an explicit > f.seek is probably not a good idea. The file iterator (for line in f) > uses a readahead buffer, which means you can't guarantee what the > current file position will be. > > 2) It may be necessary to do an explicit f.flush or f.close when > writing to an 'r+' file. In your case, the close should automatically > happen when the f object falls out of scope, which tells me that were > still looking at some other problem, like not using f.seek > > 3) It is possible that f.tell implicitly flushes buffers used by the > file object. That would explain why uncommenting the f.tell causes > the writes to show up. As far as I understand it the behavior stems from CPython file operations being implemented fairly directly as forwarding to C library FILE* operations, and the C standard prescribes Undefined Behavior to the case above. I think the Python language/library specification should specify the effect (perhaps just as UB, but anyway, specified). For as it is, it may/will be different with different Python implementations, meaning that code that works OK with one implementation may fail with another implementation. > What are you trying to accomplish? Overwrite the original file, or > append to it? If you want to overwrite the file, it may be better to > generate a new file, delete the old one, then rename the new one. If > you want to append, then it would be better to open the file with > append mode ('a') Cheers, - Alf From karthikramaswamy88 at gmail.com Thu Jan 28 10:07:04 2010 From: karthikramaswamy88 at gmail.com (evilweasel) Date: Thu, 28 Jan 2010 07:07:04 -0800 (PST) Subject: Need help with a program Message-ID: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Hi folks, I am a newbie to python, and I would be grateful if someone could point out the mistake in my program. Basically, I have a huge text file similar to the format below: AAAAAGACTCGAGTGCGCGGA 0 AAAAAGATAAGCTAATTAAGCTACTGG 0 AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 AAAAAGGTCGCCTGACGGCTGC 0 The text is nothing but DNA sequences, and there is a number next to it. What I will have to do is, ignore those lines that have 0 in it, and print all other lines (excluding the number) in a new text file (in a particular format called as FASTA format). This is the program I wrote for that: seq1 = [] list1 = [] lister = [] listers = [] listers1 = [] a = [] d = [] i = 0 j = 0 num = 0 file1 = open(sys.argv[1], 'r') for line in file1: if not line.startswith('\n'): seq1 = line.split() if len(seq1) == 0: continue a = seq1[0] list1.append(a) d = seq1[1] lister.append(d) b = len(lister) for j in range(0, b): if lister[j] == 0: listers.append(j) else: listers1.append(j) print listers1 resultsfile = open("sequences1.txt", 'w') for i in listers1: resultsfile.write('\n>seq' + str(i) + '\n' + list1[i] + '\n') But this isn't working. I am not able to find the bug in this. I would be thankful if someone could point it out. Thanks in advance! Cheers! From alfps at start.no Thu Jan 28 10:17:41 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 28 Jan 2010 16:17:41 +0100 Subject: Need help with a program In-Reply-To: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: * evilweasel: > Hi folks, > > I am a newbie to python, and I would be grateful if someone could > point out the mistake in my program. Basically, I have a huge text > file similar to the format below: > > AAAAAGACTCGAGTGCGCGGA 0 > AAAAAGATAAGCTAATTAAGCTACTGG 0 > AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 > AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 > AAAAAGGTCGCCTGACGGCTGC 0 > > The text is nothing but DNA sequences, and there is a number next to > it. What I will have to do is, ignore those lines that have 0 in it, > and print all other lines (excluding the number) in a new text file > (in a particular format called as FASTA format). This is the program I > wrote for that: > > seq1 = [] > list1 = [] > lister = [] > listers = [] > listers1 = [] > a = [] > d = [] > i = 0 > j = 0 > num = 0 > > file1 = open(sys.argv[1], 'r') > for line in file1: > if not line.startswith('\n'): > seq1 = line.split() > if len(seq1) == 0: > continue > > a = seq1[0] > list1.append(a) > > d = seq1[1] > lister.append(d) > > > b = len(lister) > for j in range(0, b): > if lister[j] == 0: > listers.append(j) > else: > listers1.append(j) > > > print listers1 > resultsfile = open("sequences1.txt", 'w') > for i in listers1: > resultsfile.write('\n>seq' + str(i) + '\n' + list1[i] + '\n') > > But this isn't working. What do you mean by "isn't working"? > I am not able to find the bug in this. I would > be thankful if someone could point it out. Thanks in advance! What do you expect as output, and what do you actually get as output? Cheers, - Alf From dickinsm at gmail.com Thu Jan 28 10:22:00 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Thu, 28 Jan 2010 07:22:00 -0800 (PST) Subject: Need help with a program References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: <2181c046-f9fe-48d2-b6cf-95b9c5f9c946@l19g2000yqb.googlegroups.com> On Jan 28, 3:07?pm, evilweasel wrote: > Hi folks, > > I am a newbie to python, and I would be grateful if someone could > point out the mistake in my program. > for j in range(0, b): > ? ? if lister[j] == 0: At a guess, this line should be: if lister[j] == '0': ... -- Mark From krister.svanlund at gmail.com Thu Jan 28 10:28:11 2010 From: krister.svanlund at gmail.com (Krister Svanlund) Date: Thu, 28 Jan 2010 16:28:11 +0100 Subject: Need help with a program In-Reply-To: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: <2cf430a61001280728h1a2f4977xe1fee5c269c009c7@mail.gmail.com> On Thu, Jan 28, 2010 at 4:07 PM, evilweasel wrote: > Hi folks, > > I am a newbie to python, and I would be grateful if someone could > point out the mistake in my program. Basically, I have a huge text > file similar to the format below: > > AAAAAGACTCGAGTGCGCGGA ? 0 > AAAAAGATAAGCTAATTAAGCTACTGG ? ? 0 > AAAAAGATAAGCTAATTAAGCTACTGGGTT ? 1 > AAAAAGGGGGCTCACAGGGGAGGGGTAT ? ? 1 > AAAAAGGTCGCCTGACGGCTGC ?0 > > The text is nothing but DNA sequences, and there is a number next to > it. What I will have to do is, ignore those lines that have 0 in it, > and print all other lines (excluding the number) in a new text file > (in a particular format called as FASTA format). This is the program I > wrote for that: > > seq1 = [] > list1 = [] > lister = [] > listers = [] > listers1 = [] > a = [] > d = [] > i = 0 > j = 0 > num = 0 > > file1 = open(sys.argv[1], 'r') > for line in file1: > ? ?if not line.startswith('\n'): > ? ? ? ?seq1 = line.split() > ? ? ? ?if len(seq1) == 0: > ? ? ? ? ? ?continue > > ? ? ? ?a = seq1[0] > ? ? ? ?list1.append(a) > > ? ? ? ?d = seq1[1] > ? ? ? ?lister.append(d) > > > b = len(lister) > for j in range(0, b): > ? ?if lister[j] == 0: > ? ? ? ?listers.append(j) > ? ?else: > ? ? ? ?listers1.append(j) > > > print listers1 > resultsfile = open("sequences1.txt", 'w') > for i in listers1: > ? ?resultsfile.write('\n>seq' + str(i) + '\n' + list1[i] + '\n') > > But this isn't working. I am not able to find the bug in this. I would > be thankful if someone could point it out. Thanks in advance! > > Cheers! I'm not totaly sure what you want to do but try this (python2.6+): newlines = [] with open(sys.argv[1], 'r') as f: text = f.read(); for line in text.splitlines(): if not line.strip() and line.strip().endswith('1'): newlines.append('seq'+line) with open(sys.argv[2], 'w') as f: f.write('\n'.join(newlines)) From krister.svanlund at gmail.com Thu Jan 28 10:31:49 2010 From: krister.svanlund at gmail.com (Krister Svanlund) Date: Thu, 28 Jan 2010 16:31:49 +0100 Subject: Need help with a program In-Reply-To: <2cf430a61001280728h1a2f4977xe1fee5c269c009c7@mail.gmail.com> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> <2cf430a61001280728h1a2f4977xe1fee5c269c009c7@mail.gmail.com> Message-ID: <2cf430a61001280731w1ea33884r5d09d48b9fa3eb6c@mail.gmail.com> On Thu, Jan 28, 2010 at 4:28 PM, Krister Svanlund wrote: > On Thu, Jan 28, 2010 at 4:07 PM, evilweasel > wrote: >> Hi folks, >> >> I am a newbie to python, and I would be grateful if someone could >> point out the mistake in my program. Basically, I have a huge text >> file similar to the format below: >> >> AAAAAGACTCGAGTGCGCGGA ? 0 >> AAAAAGATAAGCTAATTAAGCTACTGG ? ? 0 >> AAAAAGATAAGCTAATTAAGCTACTGGGTT ? 1 >> AAAAAGGGGGCTCACAGGGGAGGGGTAT ? ? 1 >> AAAAAGGTCGCCTGACGGCTGC ?0 >> >> The text is nothing but DNA sequences, and there is a number next to >> it. What I will have to do is, ignore those lines that have 0 in it, >> and print all other lines (excluding the number) in a new text file >> (in a particular format called as FASTA format). This is the program I >> wrote for that: >> >> seq1 = [] >> list1 = [] >> lister = [] >> listers = [] >> listers1 = [] >> a = [] >> d = [] >> i = 0 >> j = 0 >> num = 0 >> >> file1 = open(sys.argv[1], 'r') >> for line in file1: >> ? ?if not line.startswith('\n'): >> ? ? ? ?seq1 = line.split() >> ? ? ? ?if len(seq1) == 0: >> ? ? ? ? ? ?continue >> >> ? ? ? ?a = seq1[0] >> ? ? ? ?list1.append(a) >> >> ? ? ? ?d = seq1[1] >> ? ? ? ?lister.append(d) >> >> >> b = len(lister) >> for j in range(0, b): >> ? ?if lister[j] == 0: >> ? ? ? ?listers.append(j) >> ? ?else: >> ? ? ? ?listers1.append(j) >> >> >> print listers1 >> resultsfile = open("sequences1.txt", 'w') >> for i in listers1: >> ? ?resultsfile.write('\n>seq' + str(i) + '\n' + list1[i] + '\n') >> >> But this isn't working. I am not able to find the bug in this. I would >> be thankful if someone could point it out. Thanks in advance! >> >> Cheers! > > I'm not totaly sure what you want to do but try this (python2.6+): > > newlines = [] > > with open(sys.argv[1], 'r') as f: > ? ?text = f.read(); > ? ?for line in text.splitlines(): > ? ? ? ?if not line.strip() and line.strip().endswith('1'): ? ? ? ? ? ?newlines.append('seq'+line.strip()[:-1].strip()) > > with open(sys.argv[2], 'w') as f: > ? ?f.write('\n'.join(newlines)) > Gah, made some errors From krister.svanlund at gmail.com Thu Jan 28 10:39:16 2010 From: krister.svanlund at gmail.com (Krister Svanlund) Date: Thu, 28 Jan 2010 16:39:16 +0100 Subject: Need help with a program In-Reply-To: <2cf430a61001280737p58fcfe2dj606a6c08f78a121f@mail.gmail.com> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> <2cf430a61001280728h1a2f4977xe1fee5c269c009c7@mail.gmail.com> <2cf430a61001280731w1ea33884r5d09d48b9fa3eb6c@mail.gmail.com> <2cf430a61001280737x73202974w358804b4580a3a7f@mail.gmail.com> <2cf430a61001280737p58fcfe2dj606a6c08f78a121f@mail.gmail.com> Message-ID: <2cf430a61001280739n44c11f67gb642d0a47c7509cc@mail.gmail.com> On Thu, Jan 28, 2010 at 4:31 PM, Krister Svanlund wrote: > On Thu, Jan 28, 2010 at 4:28 PM, Krister Svanlund > wrote: >> On Thu, Jan 28, 2010 at 4:07 PM, evilweasel >> wrote: >>> Hi folks, >>> >>> I am a newbie to python, and I would be grateful if someone could >>> point out the mistake in my program. Basically, I have a huge text >>> file similar to the format below: >>> >>> AAAAAGACTCGAGTGCGCGGA ? 0 >>> AAAAAGATAAGCTAATTAAGCTACTGG ? ? 0 >>> AAAAAGATAAGCTAATTAAGCTACTGGGTT ? 1 >>> AAAAAGGGGGCTCACAGGGGAGGGGTAT ? ? 1 >>> AAAAAGGTCGCCTGACGGCTGC ?0 >>> >>> The text is nothing but DNA sequences, and there is a number next to >>> it. What I will have to do is, ignore those lines that have 0 in it, >>> and print all other lines (excluding the number) in a new text file >>> (in a particular format called as FASTA format). This is the program I >>> wrote for that: >>> >>> seq1 = [] >>> list1 = [] >>> lister = [] >>> listers = [] >>> listers1 = [] >>> a = [] >>> d = [] >>> i = 0 >>> j = 0 >>> num = 0 >>> >>> file1 = open(sys.argv[1], 'r') >>> for line in file1: >>> ? ?if not line.startswith('\n'): >>> ? ? ? ?seq1 = line.split() >>> ? ? ? ?if len(seq1) == 0: >>> ? ? ? ? ? ?continue >>> >>> ? ? ? ?a = seq1[0] >>> ? ? ? ?list1.append(a) >>> >>> ? ? ? ?d = seq1[1] >>> ? ? ? ?lister.append(d) >>> >>> >>> b = len(lister) >>> for j in range(0, b): >>> ? ?if lister[j] == 0: >>> ? ? ? ?listers.append(j) >>> ? ?else: >>> ? ? ? ?listers1.append(j) >>> >>> >>> print listers1 >>> resultsfile = open("sequences1.txt", 'w') >>> for i in listers1: >>> ? ?resultsfile.write('\n>seq' + str(i) + '\n' + list1[i] + '\n') >>> >>> But this isn't working. I am not able to find the bug in this. I would >>> be thankful if someone could point it out. Thanks in advance! >>> >>> Cheers! I'm trying this again: newlines = [] with open(sys.argv[1], 'r') as f: ? ?text = f.read(); ? ?for line in (l.strip() for l in text.splitlines()): ? ? ? ?if line: ? ? ? ? ? ?line_elem = line.split() ? ? ? ? ? ?if len(line_elem) == 2 and line_elem[1] == '1': ? ? ? ? ? ? ? ?newlines.append('seq'+line_elem[0]) with open(sys.argv[2], 'w') as f: ? ?f.write('\n'.join(newlines)) From darcy at druid.net Thu Jan 28 10:44:37 2010 From: darcy at druid.net (D'Arcy J.M. Cain) Date: Thu, 28 Jan 2010 10:44:37 -0500 Subject: Need help with a program In-Reply-To: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: <20100128104437.3c48ec68.darcy@druid.net> On Thu, 28 Jan 2010 07:07:04 -0800 (PST) evilweasel wrote: > I am a newbie to python, and I would be grateful if someone could Welcome. > point out the mistake in my program. Basically, I have a huge text > file similar to the format below: You don't say how it isn't working. As a first step you should read http://catb.org/~esr/faqs/smart-questions.html. > The text is nothing but DNA sequences, and there is a number next to > it. What I will have to do is, ignore those lines that have 0 in it, Your code doesn't completely ignore them. See below. > and print all other lines (excluding the number) in a new text file > (in a particular format called as FASTA format). This is the program I > wrote for that: > > seq1 = [] > list1 = [] > lister = [] > listers = [] > listers1 = [] > a = [] > d = [] > i = 0 > j = 0 > num = 0 This seems like an awful lot of variables for such a simple task. > > file1 = open(sys.argv[1], 'r') > for line in file1: This is good. You aren't trying to load the whole file into memory at once. If the file is huge as you say then that would have been bad. I would have made one small optimization that saves one assignment and one extra variable. for line in open(sys.argv[1], 'r'): > if not line.startswith('\n'): > seq1 = line.split() > if len(seq1) == 0: > continue This is redundant and perhaps not even correct at the end of the file. It assumes that the last line ends with a newline. Look at what '\n'.split() gives you and see if you can't improve the above code. Another small optimization - "if seq1" is better than "if len(seq1)". > > a = seq1[0] > list1.append(a) Aha! I may have found your bug. Are you mixing tabs and spaces? Don't do that. Either always use spaces or always use tabs. My suggestion is to use spaces and choose a short indent such as three or even two but that's a religious issue. > > d = seq1[1] > lister.append(d) You can also do "a, d = seq1". Of course you must be sure that you have two fields. Perhaps that's guaranteed for your input but a quick sanity test wouldn't hurt here. However, I don't understand all of the above. It may also be a source of problems. You say the files are huge. Are you filling up memory here? You did the smart thing reading the file but you lose it here. In any case, see below. > b = len(lister) > for j in range(0, b): Go lookup zip() > if lister[j] == 0: I think that you will find that lister[j] is "0", not 0. > listers.append(j) > else: > listers1.append(j) Why are you collecting the input? Just toss the '0' ones and write the others lines directly to the output. Hope this helps with this script and in further understanding the power and simplicity of Python. Good luck. -- D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner. From robert.kiwanuka at gmail.com Thu Jan 28 10:45:28 2010 From: robert.kiwanuka at gmail.com (kiwanuka) Date: Thu, 28 Jan 2010 07:45:28 -0800 (PST) Subject: is there any alternative to savefig? References: Message-ID: <887d573b-a32b-4cf7-a262-1675c971a316@m25g2000yqc.googlegroups.com> On Jan 28, 12:29?pm, kiwanuka wrote: > Hi all, > > I wonder if anyone knows any alternative function in pylab (or > otherwise) that could be used to save an image. My problem is as > follows: > > --------------- > from pylab import * > ... > > figure(1) > fig1 = gca() > figure(2) > fig2 = gca() > figure(3) > fig3 = gca() > > for i,data_file in enumerate(data_file_list): > ? ?time,x, y,x2, y2 = read_csv_file_4(open > (data_file),elements=num_of_elements) > ? ?fig1.plot(-x,-y,color=colours[i],label=labellist[i]) > ? ?fig2.plot(time,-y,color=colours[i],label=labellist[i]) > ? ?fig3.plot(time,-x,color=colours[i],label=labellist[i]) > > fig1.legend(loc='best') > fig1.set_title("y1 - x1") > fig1.set_ylabel("y1") > fig1.set_xlabel("x1") > #savefig("y1-x1.png") > > fig2.legend(loc='best') > fig2.set_title("y1 - time") > fig2.set_ylabel("y1") > fig2.set_xlabel("time[s]") > #savefig("y1-time.png") > > fig3.legend(loc='best') > fig3.set_title("x1 - time") > fig3.set_ylabel("x1") > fig3.set_xlabel("time[s]") > #savefig("x1-time.png") > show() > --------------------------- > > In the above code, I read multiple data files and plot three separate > figures. Now I would like to save each of the figures to a file as the > commented savefig satements suggest. The trouble is that if I > uncomment all those savefig statements, I get three saved images all > containing the plot belonging to figure(3), which was the last figure > declared. > > I understand this to be happening because savefig will save the > "current" figure, which in this case happens to be the last one > declared. > > If I could do something like fig1.savefig("y1-x1.png") or savefig("y1- > x1.png").fig1, this would solve the problem but I'm not aware of any > such methods ?or modules to enable this. This is thus a flaw in the > general design/implementation of the savefig function, but is there an > alternative function to enable me achieve what I need? Is there > perhaps a possible tweak to savefig to make it do the same? > > Thanks in advance, > > Robert Problem solved. From alfps at start.no Thu Jan 28 10:47:33 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 28 Jan 2010 16:47:33 +0100 Subject: is there any alternative to savefig? In-Reply-To: <887d573b-a32b-4cf7-a262-1675c971a316@m25g2000yqc.googlegroups.com> References: <887d573b-a32b-4cf7-a262-1675c971a316@m25g2000yqc.googlegroups.com> Message-ID: * kiwanuka: > On Jan 28, 12:29 pm, kiwanuka wrote: >> Hi all, >> >> I wonder if anyone knows any alternative function in pylab (or >> otherwise) that could be used to save an image. My problem is as >> follows: >> >> --------------- >> from pylab import * >> ... >> >> figure(1) >> fig1 = gca() >> figure(2) >> fig2 = gca() >> figure(3) >> fig3 = gca() >> >> for i,data_file in enumerate(data_file_list): >> time,x, y,x2, y2 = read_csv_file_4(open >> (data_file),elements=num_of_elements) >> fig1.plot(-x,-y,color=colours[i],label=labellist[i]) >> fig2.plot(time,-y,color=colours[i],label=labellist[i]) >> fig3.plot(time,-x,color=colours[i],label=labellist[i]) >> >> fig1.legend(loc='best') >> fig1.set_title("y1 - x1") >> fig1.set_ylabel("y1") >> fig1.set_xlabel("x1") >> #savefig("y1-x1.png") >> >> fig2.legend(loc='best') >> fig2.set_title("y1 - time") >> fig2.set_ylabel("y1") >> fig2.set_xlabel("time[s]") >> #savefig("y1-time.png") >> >> fig3.legend(loc='best') >> fig3.set_title("x1 - time") >> fig3.set_ylabel("x1") >> fig3.set_xlabel("time[s]") >> #savefig("x1-time.png") >> show() >> --------------------------- >> >> In the above code, I read multiple data files and plot three separate >> figures. Now I would like to save each of the figures to a file as the >> commented savefig satements suggest. The trouble is that if I >> uncomment all those savefig statements, I get three saved images all >> containing the plot belonging to figure(3), which was the last figure >> declared. >> >> I understand this to be happening because savefig will save the >> "current" figure, which in this case happens to be the last one >> declared. >> >> If I could do something like fig1.savefig("y1-x1.png") or savefig("y1- >> x1.png").fig1, this would solve the problem but I'm not aware of any >> such methods or modules to enable this. This is thus a flaw in the >> general design/implementation of the savefig function, but is there an >> alternative function to enable me achieve what I need? Is there >> perhaps a possible tweak to savefig to make it do the same? >> >> Thanks in advance, >> >> Robert > > Problem solved. That's nice, but would you mind telling the group the solution? Assuming that you haven't (I don't see all messages posted to the e-mail list, only what's propagated to Usenet or originating there). Cheers, - Alf From karthikramaswamy88 at gmail.com Thu Jan 28 10:50:31 2010 From: karthikramaswamy88 at gmail.com (evilweasel) Date: Thu, 28 Jan 2010 07:50:31 -0800 (PST) Subject: Need help with a program References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: I will make my question a little more clearer. I have close to 60,000 lines of the data similar to the one I posted. There are various numbers next to the sequence (this is basically the number of times the sequence has been found in a particular sample). So, I would need to ignore the ones containing '0' and write all other sequences (excluding the number, since it is trivial) in a new text file, in the following format: >seq59902 TTTTTTTATAAAATATATAGT >seq59903 TTTTTTTATTTCTTGGCGTTGT >seq59904 TTTTTTTGGTTGCCCTGCGTGG >seq59905 TTTTTTTGTTTATTTTTGGG The number next to 'seq' is the line number of the sequence. When I run the above program, what I expect is an output file that is similar to the above output but with the ones containing '0' ignored. But, I am getting all the sequences printed in the file. Kindly excuse the 'newbieness' of the program. :) I am hoping to improve in the next few months. Thanks to all those who replied. I really appreciate it. :) From rdv at roalddevries.nl Thu Jan 28 11:01:38 2010 From: rdv at roalddevries.nl (Roald de Vries) Date: Thu, 28 Jan 2010 17:01:38 +0100 Subject: Symbols as parameters? In-Reply-To: <4CD97E94-B40E-481D-B367-5FCE2820F48D@roalddevries.nl> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <4CD97E94-B40E-481D-B367-5FCE2820F48D@roalddevries.nl> Message-ID: On Jan 22, 2010, at 11:56 AM, Roald de Vries wrote: > Hi Martin, > > On Jan 21, 2010, at 8:43 AM, Martin Drautzburg wrote: >> Hello all, >> >> When passing parameters to a function, you sometimes need a paramter >> which can only assume certain values, e.g. >> >> def move (direction): >> ... >> If direction can only be "up", "down", "left" or "right", you can >> solve >> this by passing strings, but this is not quite to the point: >> >> - you could pass invalid strings easily >> - you need to quote thigs, which is a nuisance >> - the parameter IS REALLY NOT A STRING, but a direction >> >> Alternatively you could export such symbols, so when you "import *" >> you >> have them available in the caller's namespace. But that forces you >> to "import *" which pollutes your namespace. >> >> What I am really looking for is a way >> >> - to be able to call move(up) >> - having the "up" symbol only in the context of the function >> call >> >> So it should look something like this >> >> ... magic, magic ... >> move(up) >> ... unmagic, unmagic ... >> print up >> >> This should complain that "up" is not defined during the "print" >> call, >> but not when move() is called. And of course there should be as >> little >> magic as possible. >> >> Any way to achieve this? > > You could do something like this: > > class Move(object): > def __call__(self, direction): > print(direction) > return 0 > > def up(self): > return self('up') > > move = Move() > > Now move.up() means move('up'), and you can obviously do similar > things for other directions. Question out of general interest in the language: If I would want to generate such functions in a for-loop, what would I have to do? This doesn't work: class Move(object): def __call__(self, direction): return direction move = Move() for f in ['up', 'down', 'right', 'left']: move.__dict__[f] = lambda: move(f) ... because now 'move.up()' returns 'left' because thats the current value of f. Is there a way to 'expand' f in the loop? Or a reason that you never should use this? From MLMDev at Comcast.net Thu Jan 28 11:03:20 2010 From: MLMDev at Comcast.net (Mitchell L Model) Date: Thu, 28 Jan 2010 11:03:20 -0500 Subject: python 3's adoption In-Reply-To: References: Message-ID: <2021A225-2BEE-4DE9-AE88-B9199E5D4491@Comcast.net> I have been working with Python 3 for over a year. I used it in writing my book "Bioinformatics Programming Using Python" (http://oreilly.com/catalog/9780596154509 ). I didn't see any point in teaching an incompatible earlier version of a language in transition. In preparing the book and its examples I explored a large number of Python modules in some depth and encountered most of the differences between the language and libraries of Python 2 and Python 3. The change was a bit awkward for a while, and there were some surprises, but in the end I have found nothing in Python 3 for which I would prefer Python 2's version. Removal of old-style classes is a big win. Having print as a function provides a tremendous amount of flexibility. I use the sep and end keywords all the time. There is no reason for print to be a statement, and it was an awkward inconsistency in a language that leans towards functional styles. Likewise the elimination of cmp, while shocking, leads to much simpler comparison arguments to sort, since all the function does is return a key; then, sort uses __lt__ (I think) so it automatically uses each class's definition of that. The weird objects returned from things like sorted, dict.keys/values/items, and so on are values that in practice are used primarily in iterations; you can always turn the result into a list, though I have to admit that while developing and debugging I trip trying to pick out a specific element from one of these using indexing (typically [0]); I've learned to think of them as generators, even though they aren't. The rearrangements and name changes in the libraries are quite helpful. I could go on, but basically the language and library changes are on the whole large improvements with little, if any, downside. Conversion of old code is greatly facilitied by the 2to3 tool that comes with Python 3. The big issue in moving from 2 to 3 is the external libraries and development tools you use. Different IDEs have released versions that support Python 3 at different times. (I believe Wing was the first.) If you use numpy, for example, or one of the many libraries that require it, you are stuck. Possibly some important facilities will never be ported to Python 3, but probably most active projects will eventually produce a Python 3 version -- for example, according to its web page, a Python 3 version of PIL is on the way. I was able to cover all the topics in my book using only Python library modules, something I felt would be best for readers -- I used libraries such as elementree, sqlite3, and tkinter. The only disappointment was that I couldn't include a chapter on BioPython, since there is no Python 3 version. By now, many large facilities support both Python 2 and Python 3. I am currently building a complex GUI/Visualization application based on the Python 3 version of PyQt4 and Wing IDE and am delighted with all of it. It may well be that some very important large From aahz at pythoncraft.com Thu Jan 28 11:06:33 2010 From: aahz at pythoncraft.com (Aahz) Date: 28 Jan 2010 08:06:33 -0800 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: In article , Steven D'Aprano wrote: >On Wed, 27 Jan 2010 16:25:46 -0500, Benjamin Kaplan wrote: >> >> When Python 2.6 came out, Jython was still on 2.2. The difference >> between 2.2 and 2.6 is almost as big of a difference as between 2.6 and >> 3.0. In that time, you had the introduction of the boolean type, >> generators, list comprehensions, the addition of the "yield" and "with" >> keywords, universal newline support, and decorators in addition to the >> large number of changes to the standard library such as the introduction >> of the subprocess module. > >I believe that, with the possible exception of the change from byte >strings to unicode strings, virtually *all* the hoo-har over Python 3 is >simply due to the tactical mistake of Guido and the Python Dev team of >*calling* Python 3 a backward incompatible release. Python has had >previous major changes in the past (e.g. 1.5 to 2.0 and 2.1 to 2.2) and >hardly anyone made a complaint. But as Steven points out, the difference from 2.2 to 2.6 is roughly the same as 2.6 to 3.1. Python has never before had such a large difference from one release to the next, and I think few people try to support serious apps on the full range from 2.2 to 2.6. Moreover, the task of using a single codebase without 2to3 is much easier in 1.4 through 2.6. Admittedly, it wouldn't be much fun to write 1.4 code these days without all the neat features that have been added, but you can't argue that it's hard. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From allah.27 at gmail.com Thu Jan 28 11:09:50 2010 From: allah.27 at gmail.com (talal awadh) Date: Thu, 28 Jan 2010 08:09:50 -0800 (PST) Subject: Good Book Message-ID: <7dbf7dff-cf08-463c-b7ff-ef83ff65a304@36g2000yqu.googlegroups.com> Many people who want to learn Islam or are new converts find it hard to have a simplified guide that explains to them the basics of Islam in a nutshell; so I decided to collect the basic guidelines and gather them in an e-book I named it "Basic Islam" for Introducing Islam http://www.saaid.net/book/9/2012.doc From roy at panix.com Thu Jan 28 11:09:58 2010 From: roy at panix.com (Roy Smith) Date: Thu, 28 Jan 2010 11:09:58 -0500 Subject: python 3's adoption References: Message-ID: In article , Mitchell L Model wrote: > I use the sep and end keywords all the time. What are 'sep' and 'end'? I'm looking in http://docs.python.org/3.1/genindex-all.html and don't see those mentioned at all. Am I just looking in the wrong place? From aahz at pythoncraft.com Thu Jan 28 11:10:15 2010 From: aahz at pythoncraft.com (Aahz) Date: 28 Jan 2010 08:10:15 -0800 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> Message-ID: In article , Neil Hodgson wrote: >Carl Banks: >> >> There is also no hope someone will fork Python 2.x and continue it in >> perpetuity. Well, someone might try to fork it, but they won't be >> able to call it Python. > > Over time there may be more desire from those unable or unwilling to >upgrade to 3.x to work on improvements to 2.x, perhaps leading to a >version 2.8. One of the benefits of open source is that you are not >trapped into following vendor decisions like Microsoft abandoning >classic VB in favour of VB.NET. > > It would be unreasonable for the core developers to try to block >this. Refusing use of the Python trademark for a version that was >reasonably compatible in both directions would be particularly petty. Agreed, and as a PSF member, I'd certainly be opposed to anyone trying to prevent the release of Python 2.8, and I would actively favor providing PSF and python.org resources to them. OTOH, I would also be likely to push anyone working on Python 2.8 to come up with a solid release plan first. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From pruebauno at latinmail.com Thu Jan 28 11:13:09 2010 From: pruebauno at latinmail.com (nn) Date: Thu, 28 Jan 2010 08:13:09 -0800 (PST) Subject: Need help with a program References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: On Jan 28, 10:50?am, evilweasel wrote: > I will make my question a little more clearer. I have close to 60,000 > lines of the data similar to the one I posted. There are various > numbers next to the sequence (this is basically the number of times > the sequence has been found in a particular sample). So, I would need > to ignore the ones containing '0' and write all other sequences > (excluding the number, since it is trivial) in a new text file, in the > following format: > > >seq59902 > > TTTTTTTATAAAATATATAGT > > >seq59903 > > TTTTTTTATTTCTTGGCGTTGT > > >seq59904 > > TTTTTTTGGTTGCCCTGCGTGG > > >seq59905 > > TTTTTTTGTTTATTTTTGGG > > The number next to 'seq' is the line number of the sequence. When I > run the above program, what I expect is an output file that is similar > to the above output but with the ones containing '0' ignored. But, I > am getting all the sequences printed in the file. > > Kindly excuse the 'newbieness' of the program. :) I am hoping to > improve in the next few months. Thanks to all those who replied. I > really appreciate it. :) People have already given you some pointers to your problem. In the end you will have to "tweak the details" because only you have access to the data not us. Just as example here is another way to do what you are doing: with open('dnain.dat') as infile, open('dnaout.dat','w') as outfile: partgen=(line.split() for line in infile) dnagen=(str(i+1)+'\n'+part[0]+'\n' for i,part in enumerate(partgen) if len(part)>1 and part[1]!='0') outfile.writelines(dnagen) From alfps at start.no Thu Jan 28 11:13:22 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 28 Jan 2010 17:13:22 +0100 Subject: python 3's adoption In-Reply-To: References: Message-ID: * Roy Smith: > In article , > Mitchell L Model wrote: > >> I use the sep and end keywords all the time. > > What are 'sep' and 'end'? I'm looking in > http://docs.python.org/3.1/genindex-all.html and don't see those mentioned > at all. Am I just looking in the wrong place? >>> print( print.__doc__ ) print(value, ..., sep=' ', end='\n', file=sys.stdout) Prints the values to a stream, or to sys.stdout by default. Optional keyword arguments: file: a file-like object (stream); defaults to the current sys.stdout. sep: string inserted between values, default a space. end: string appended after the last value, default a newline. >>> _ Cheers & hth., - Alf From dreadpiratejeff at gmail.com Thu Jan 28 11:34:15 2010 From: dreadpiratejeff at gmail.com (J) Date: Thu, 28 Jan 2010 11:34:15 -0500 Subject: Good Book In-Reply-To: <7dbf7dff-cf08-463c-b7ff-ef83ff65a304@36g2000yqu.googlegroups.com> References: <7dbf7dff-cf08-463c-b7ff-ef83ff65a304@36g2000yqu.googlegroups.com> Message-ID: <36dec4ff1001280834n41aa5268o13d8bfe8e0acffc8@mail.gmail.com> On Thu, Jan 28, 2010 at 11:09, talal awadh wrote: > Many people who want to learn Islam or are new converts find it hard I just wanted to thank you for reminding me that I needed to write a mail filter to delete this kind of drivel. I appreciate the reminder! Cheers, Jeff From lie.1296 at gmail.com Thu Jan 28 11:35:36 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Fri, 29 Jan 2010 03:35:36 +1100 Subject: python 3's adoption In-Reply-To: <7xy6ji62vx.fsf@ruckus.brouhaha.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> <7xockfjcsv.fsf@ruckus.brouhaha.com> <6b89dc67-ceee-4b83-a11f-c7376185dcd1@b9g2000pri.googlegroups.com> <7xy6ji62vx.fsf@ruckus.brouhaha.com> Message-ID: <4b61bd02@dnews.tpgi.com.au> On 01/28/10 19:37, Paul Rubin wrote: > Jonathan Gardner writes: >> If you're going to have statements, you're going to need the null >> statement. That's "pass". > > Why? Expressions are statements, so you could just say "pass" (in > quotes, denoting a string literal), or 0, or None, os anything else like > that, instead of having a special statement. or, if the null statement "pass" is removed, you could define: pass = None or pass = object() # sentinel and have essentially the same thing. hmm... on second thought, not special-casing pass means doing a LOAD_GLOBAL or LOAD_CONST for operation that is supposed to be doing nothing. From dinov at microsoft.com Thu Jan 28 11:37:24 2010 From: dinov at microsoft.com (Dino Viehland) Date: Thu, 28 Jan 2010 16:37:24 +0000 Subject: myths about python 3 In-Reply-To: <4b614417$0$6715$9b4e6d93@newsspool2.arcor-online.net> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <4b614417$0$6715$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <1A472770E042064698CB5ADC83A12ACD34CAEAB2@TK5EX14MBXC116.redmond.corp.microsoft.com> Stefan wrote: > >From an implementors point of view, it's actually quite the opposite. Most > syntax features of Python 3 can be easily implemented on top of an existing > Py2 Implementation (we have most of them in Cython already, and I really > found them fun to write), and the shifting-around in the standard library > can hardly be called non-trivial. All the hard work that went into the > design of CPython 3.x (and into its test suite) now makes it easy to just > steal from what's there already. > > The amount of work that the Jython project put into catching up from 2.1 to > 2.5/6 (new style classes! generators!) is really humongous compared to the > adaptations that an implementation needs to do to support Python 3 code. I > have great respect for the Jython project for what they achieved in the > last couple of years. (I also have great respect for the IronPython project > for fighting the One Microsoft Way into opening up, but that's a different > kind of business.) > > If there was enough interest from the respective core developers, I > wouldn't be surprised if we had more than one 'mostly compatible' > alternative Python 3 implementation in a couple of months. But it's the > obvious vicious circle business. As long as there aren't enough important > users of Py3, alternative implementations won't have enough incentives to > refocus their scarce developer time. Going for 2.6/7 first means that most > of the Py3 work gets done anyway, so it'll be even easier then. That makes > 2.6->2.7->3.2/3 the most natural implementation path. (And that, again, > makes it a *really* good decision that 2.7 will be the last 2.x release line.) I just want to echo this as I completely agree. Last time I went through the list it looked like there were around 10 major new features (some of them even not so major) that we needed to implement to bring IronPython up to the 3.0 level. It shouldn't be too time consuming, and it greatly improves our compatibility by finally having the same string types, but our users don't yet want us to stop supporting 2.x. From solipsis at pitrou.net Thu Jan 28 11:52:19 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 28 Jan 2010 16:52:19 +0000 (UTC) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: Le Thu, 28 Jan 2010 00:19:24 +0000, Steven D'Aprano a ?crit?: > 4. Python 3 will make you irresistible to women. > > FALSE - Python 3 coders are no more likely to get a date than any > other programmer. They spend less time coding, so they /can/ get more "dates" (what a strange English word) :-) Those dates don't have to be with women of course. From lie.1296 at gmail.com Thu Jan 28 11:54:26 2010 From: lie.1296 at gmail.com (Lie Ryan) Date: Fri, 29 Jan 2010 03:54:26 +1100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> Message-ID: <4b61c166$1@dnews.tpgi.com.au> On 01/28/10 20:12, Alf P. Steinbach wrote: > * Steven D'Aprano: >> On Wed, 27 Jan 2010 18:29:25 +0100, Alf P. Steinbach wrote:> >> Instead of: >> >> print >>fileObj, x, y, z >> >> you use regular function syntax with a meaningful keyword: >> >> print(x, y, z, file=fileObj) >> >> If you want suppress the newline at the end of each print: >> >> print x, y, z, # note the final comma >> >> compared to: >> >> print(x, y, z, end='') > > Actually I thought the final comma thing was nice. It was like Basic. I > think the 2.x 'print' must have been modeled on Basic's 'print'. if that was true, then python missed the final semicolon >> If you want to change the space between elements, instead of: >> >> sys.stdout.write(str(x) + "*" + str(y) + "*" + str(z) + '\n') >> >> you use: >> >> print(x, y, z, sep='*') >> >> >> If you want to override the behaviour of print in a module, instead of >> having to edit the source code of the module (which might not even be >> available), all you need to do is monkey-patch it: >> >> import module >> module.print = myprint > > >>> import builtins > >>> > >>> org_print = print > >>> builtins.print = 666 > >>> > >>> print( "trallala" ) > Traceback (most recent call last): > File "", line 1, in > TypeError: 'int' object is not callable > >>> org_print( "but is that really so smart?" ) > but is that really so smart? > >>> _ Monkey patching follows (or should follow) the same rule as class inheritance overriding: the overrider's input domain must be a superset of the overriden's input domain and the overrider's output range must be a subset of the overriden's output range. 666 object (int) is not even remotely compatible with function object. From solipsis at pitrou.net Thu Jan 28 11:55:29 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 28 Jan 2010 16:55:29 +0000 (UTC) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: Le Wed, 27 Jan 2010 17:36:29 -0800, alex23 a ?crit?: > > I've been a big supporter of Py3 from the beginning, but this repeated > claim of US becoming the mainline interpreter for 3.x pretty much kills > dead a lot of my interest. As long as the U-S JIT can be disabled at compile-time (and also at runtime), I don't think there's much of a contention actually. The other changes probably aren't controversial, although I haven't looked at them. Antoine. From arnodel at googlemail.com Thu Jan 28 12:00:54 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Thu, 28 Jan 2010 17:00:54 +0000 Subject: Need help with a program References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: nn writes: > On Jan 28, 10:50?am, evilweasel wrote: >> I will make my question a little more clearer. I have close to 60,000 >> lines of the data similar to the one I posted. There are various >> numbers next to the sequence (this is basically the number of times >> the sequence has been found in a particular sample). So, I would need >> to ignore the ones containing '0' and write all other sequences >> (excluding the number, since it is trivial) in a new text file, in the >> following format: >> >> >seq59902 >> >> TTTTTTTATAAAATATATAGT >> >> >seq59903 >> >> TTTTTTTATTTCTTGGCGTTGT >> >> >seq59904 >> >> TTTTTTTGGTTGCCCTGCGTGG >> >> >seq59905 >> >> TTTTTTTGTTTATTTTTGGG >> >> The number next to 'seq' is the line number of the sequence. When I >> run the above program, what I expect is an output file that is similar >> to the above output but with the ones containing '0' ignored. But, I >> am getting all the sequences printed in the file. >> >> Kindly excuse the 'newbieness' of the program. :) I am hoping to >> improve in the next few months. Thanks to all those who replied. I >> really appreciate it. :) > > People have already given you some pointers to your problem. In the > end you will have to "tweak the details" because only you have access > to the data not us. > > Just as example here is another way to do what you are doing: > > with open('dnain.dat') as infile, open('dnaout.dat','w') as outfile: > partgen=(line.split() for line in infile) > dnagen=(str(i+1)+'\n'+part[0]+'\n' > for i,part in enumerate(partgen) > if len(part)>1 and part[1]!='0') > outfile.writelines(dnagen) I think that generator expressions are overrated :) What's wrong with: with open('dnain.dat') as infile, open('dnaout.dat','w') as outfile: for i, line in enumerate(infile): parts = line.split() if len(parts) > 1 and parts[1] != '0': outfile.write(">seq%s\n%s\n" % (i+1, parts[0])) (untested) -- Arnaud From jjposner at optimum.net Thu Jan 28 12:23:36 2010 From: jjposner at optimum.net (John Posner) Date: Thu, 28 Jan 2010 12:23:36 -0500 Subject: Need help with a program In-Reply-To: References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: <4B61C818.3040302@optimum.net> On 1/28/2010 10:50 AM, evilweasel wrote: > I will make my question a little more clearer. I have close to 60,000 > lines of the data similar to the one I posted. There are various > numbers next to the sequence (this is basically the number of times > the sequence has been found in a particular sample). So, I would need > to ignore the ones containing '0' and write all other sequences > (excluding the number, since it is trivial) in a new text file, in the > following format: > >> seq59902 > TTTTTTTATAAAATATATAGT > >> seq59903 > TTTTTTTATTTCTTGGCGTTGT > >> seq59904 > TTTTTTTGGTTGCCCTGCGTGG > >> seq59905 > TTTTTTTGTTTATTTTTGGG > > The number next to 'seq' is the line number of the sequence. When I > run the above program, what I expect is an output file that is similar > to the above output but with the ones containing '0' ignored. But, I > am getting all the sequences printed in the file. > > Kindly excuse the 'newbieness' of the program. :) I am hoping to > improve in the next few months. Thanks to all those who replied. I > really appreciate it. :) Your program is a good first try. It contains a newbie error (looking for the number 0 instead of the string "0"). But more importantly, you're doing too much work yourself, rather than letting Python do the heavy lifting for you. These practices and tools make life a lot easier: * As others have noted, don't accumulate output in a list. Just write data to the output file line-by-line. * You don't need to initialize every variable at the beginning of the program. But there's no harm in it. * Use the enumerate() function to provide a line counter: for counter, line in enumerate(file1): This eliminates the need to accumulate output data in a list, then use the index variable "j" as the line counter. * Use string formatting. Each chunk of output is a two-line string, with the line-counter and the DNA sequence as variables: outformat = """seq%05d %s """ ... later, inside your loop ... resultsfile.write(outformat % (counter, sequence)) HTH, John From ethan at stoneleaf.us Thu Jan 28 12:35:53 2010 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 28 Jan 2010 09:35:53 -0800 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <4B61CAF9.6090701@stoneleaf.us> Steven D'Aprano wrote: > 4. Python 3 will make you irresistible to women. > > FALSE What?!? Drat!!! Guess I'll have to learn Lisp... ;) ~Ethan~ From briandenzer at gmail.com Thu Jan 28 12:36:52 2010 From: briandenzer at gmail.com (Brian D) Date: Thu, 28 Jan 2010 09:36:52 -0800 (PST) Subject: Stuck on a three word street name regex References: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> <0a712e20-facf-4a9b-8852-6136a8bad7dc@n7g2000yqb.googlegroups.com> <4e222e90-e088-4a82-a7d2-c62f509e2517@g29g2000yqe.googlegroups.com> <1b7dba4d-3f12-4d31-8e9a-b36cd4f40217@u41g2000yqe.googlegroups.com> Message-ID: <82c1cf49-eeff-42a1-96fd-012ee50922f2@c29g2000yqd.googlegroups.com> > Correction: > > [snip] the expression "parts[1 : -1]" means gather list items from the > second element in the list (index value 1) to one index position > before the end of the list. [snip] MRAB's solution was deserving of a more complete solution: >>> def parse_address(address): # Handles poorly-formatted addresses: # 100 RAMPART S ST -- direction in wrong position # 45 JOHN CHURCHILL CHASE ST -- two spaces before type #addresslist = ['num', 'dir', 'name', 'type'] addresslist = ['', '', '', ''] parts = address.split() if parts[-2] in ('E', 'W', 'N', 'S'): addresslist[1] = parts[-2] addresslist[2] = ' '.join(parts[1 : -2]) else: addresslist[2] = ' '.join(parts[1 : -1]) addresslist[0] = parts[0] addresslist[3] = parts[-1] return addresslist >>> parse_address('45 John Churchill Chase N St') ['45', 'N', 'John Churchill Chase', 'St'] >>> parse_address('45 John Churchill Chase St') ['45', '', 'John Churchill Chase', 'St'] From alfps at start.no Thu Jan 28 12:37:56 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 28 Jan 2010 18:37:56 +0100 Subject: python 3's adoption In-Reply-To: <4b61c166$1@dnews.tpgi.com.au> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <4b61c166$1@dnews.tpgi.com.au> Message-ID: * Lie Ryan: > On 01/28/10 20:12, Alf P. Steinbach wrote: >> >>> import builtins >> >>> >> >>> org_print = print >> >>> builtins.print = 666 >> >>> >> >>> print( "trallala" ) >> Traceback (most recent call last): >> File "", line 1, in >> TypeError: 'int' object is not callable >> >>> org_print( "but is that really so smart?" ) >> but is that really so smart? >> >>> _ > > Monkey patching follows (or should follow) the same rule as class > inheritance overriding: the overrider's input domain must be a superset > of the overriden's input domain and the overrider's output range must be > a subset of the overriden's output range. 666 object (int) is not even > remotely compatible with function object. Yes, that's my point. A 'print' replacement should ideally provide all the guarantees on behavior that standard 'print' does. And with that it's not so easy to put in a /correct/ replacement of 'print'; in particular, it has to honor the 'file' keyword argument. Thus the 3.x design makes it easy to replace 'print' incorrectly. I'd rather still had 'print' as keyword... ;-) Cheers, - Alf From jeanmichel at sequans.com Thu Jan 28 12:49:02 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Thu, 28 Jan 2010 18:49:02 +0100 Subject: Need help with a program In-Reply-To: References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: <4B61CE0E.5000801@sequans.com> evilweasel wrote: > I will make my question a little more clearer. I have close to 60,000 > lines of the data similar to the one I posted. There are various > numbers next to the sequence (this is basically the number of times > the sequence has been found in a particular sample). So, I would need > to ignore the ones containing '0' and write all other sequences > (excluding the number, since it is trivial) in a new text file, in the > following format: > > >> seq59902 >> > TTTTTTTATAAAATATATAGT > > >> seq59903 >> > TTTTTTTATTTCTTGGCGTTGT > > >> seq59904 >> > TTTTTTTGGTTGCCCTGCGTGG > > >> seq59905 >> > TTTTTTTGTTTATTTTTGGG > > The number next to 'seq' is the line number of the sequence. When I > run the above program, what I expect is an output file that is similar > to the above output but with the ones containing '0' ignored. But, I > am getting all the sequences printed in the file. > > Kindly excuse the 'newbieness' of the program. :) I am hoping to > improve in the next few months. Thanks to all those who replied. I > really appreciate it. :) > Using regexp may increase readability (if you are familiar with it). What about import re output = open("sequences1.txt", 'w') for index, line in enumerate(open(sys.argv[1], 'r')): match = re.match('(?P[GATC]+)\s+1') if match: output.write('seq%s\n%s\n' % (index, match.group('sequence'))) Jean-Michel From briandenzer at gmail.com Thu Jan 28 12:50:37 2010 From: briandenzer at gmail.com (Brian D) Date: Thu, 28 Jan 2010 09:50:37 -0800 (PST) Subject: Stuck on a three word street name regex References: <4583db8f-886a-4520-9ccd-02c76e8d1723@o9g2000yqa.googlegroups.com> <4b619f12$1@dnews.tpgi.com.au> Message-ID: <3b3b1880-70f1-4a18-994c-40c3d6ba7f85@f12g2000yqn.googlegroups.com> On Jan 28, 8:27?am, Lie Ryan wrote: > On 01/28/10 11:28, Brian D wrote: > > > > > I've tackled this kind of problem before by looping through a patterns > > dictionary, but there must be a smarter approach. > > > Two addresses. Note that the first has incorrectly transposed the > > direction and street name. The second has an extra space in it before > > the street type. Clearly done by someone who didn't know how to > > concatenate properly -- or didn't care. > > > 1000 RAMPART S ST > > > 100 JOHN CHURCHILL CHASE ?ST > > > I want to parse the elements into an array of values that can be > > inserted into new database fields. > > > Anyone who loves solving these kinds of puzzles care to relieve my > > frazzled brain? > > > The pattern I'm using doesn't keep the "CHASE" with the "JOHN > > CHURCHILL": > > How does the following perform? > > pat = > re.compile(r'(?P\d+)\s+(?P[A-Z\s]+)\s+(?PN|S|W|E|)\s+(?PST|RD|AVE?|)$') > > or more legibly: > > pat = re.compile( > ? ? r''' > ? ? ? (?P ?\d+ ? ? ? ? ? ? ?) ?#M series of digits > ? ? ? \s+ > ? ? ? (?P [A-Z\s]+ ? ? ? ? ) ?#M one-or-more word > ? ? ? \s+ > ? ? ? (?P ?S?E|SW?|N?W|NE?| ) ?#O direction or nothing > ? ? ? \s+ > ? ? ? (?P ST|RD|AVE? ? ? ? ) ?#M street type > ? ? ? $ ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? #M END > ? ? ''', re.VERBOSE) Is that all? That little empty space after the "|" OR metacharacter? Wow. As a test, to create a failure, if I remove that last "|" metacharacter from the "N|S|W|E|" string (i.e., "N|S|W|E"), the match fails on addresses that do not have that malformed direction after the street name (e.g., '45 JOHN CHURCHILL CHASE ST') Very clever. I don't think I've ever seen documentation showing that little trick. Thanks for enlightening me! From darcy at druid.net Thu Jan 28 13:03:44 2010 From: darcy at druid.net (D'Arcy J.M. Cain) Date: Thu, 28 Jan 2010 13:03:44 -0500 Subject: Need help with a program In-Reply-To: <4B61CE0E.5000801@sequans.com> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> <4B61CE0E.5000801@sequans.com> Message-ID: <20100128130344.af14ef8e.darcy@druid.net> On Thu, 28 Jan 2010 18:49:02 +0100 Jean-Michel Pichavant wrote: > Using regexp may increase readability (if you are familiar with it). If you have a problem and you think that regular expressions are the solution then now you have two problems. Regex is really overkill for the OP's problem and it certainly doesn't improve readability. -- D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner. From tjreedy at udel.edu Thu Jan 28 13:06:52 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 28 Jan 2010 13:06:52 -0500 Subject: Library support for Python 3.x In-Reply-To: <7xk4v2g2f3.fsf@ruckus.brouhaha.com> References: <9f3c3$4b605a65$4275d90a$30451@FUSE.NET> <7xk4v3mktr.fsf@ruckus.brouhaha.com> <7xk4v2g2f3.fsf@ruckus.brouhaha.com> Message-ID: On 1/28/2010 1:37 AM, Paul Rubin wrote: > David Cournapeau writes: >> That's not windows specific - most packages which distribute binary >> packages need to package binaries for every minor version (2.4, 2.5, >> etc...).... >> I doubt that's what Paul was referring to, though - he seemed more >> concern with API/language changes than ABI issues. > > I didn't realize the ABI situation was that unstable. I thought you > could just package up a .so or .dll and people could keep using it. I > tend to not want to use extension modules that are not in the stdlib, > and I guess this is another reason to keep staying away from them. My impression is that there is something 'special' about Windows (msvc) such that binaries compiled against x.y automatically do not work for x.y+1, even is the ABI is unchanged from Python's viewpoint. The point of my post that David responded to is that most Windows users have always been effectively dependent on 3rd party module/package developers to produce a new binary for each new version, whereas many *nix users could download the source and compile, or at least give it a go. tjr From tjreedy at udel.edu Thu Jan 28 13:20:02 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 28 Jan 2010 13:20:02 -0500 Subject: python 3's adoption In-Reply-To: <7xy6ji62vx.fsf@ruckus.brouhaha.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> <7xockfjcsv.fsf@ruckus.brouhaha.com> <6b89dc67-ceee-4b83-a11f-c7376185dcd1@b9g2000pri.googlegroups.com> <7xy6ji62vx.fsf@ruckus.brouhaha.com> Message-ID: On 1/28/2010 3:37 AM, Paul Rubin wrote: > Jonathan Gardner writes: >> If you're going to have statements, you're going to need the null >> statement. That's "pass". > > Why? Expressions are statements, so you could just say "pass" (in > quotes, denoting a string literal), or 0, or None, os anything else like > that, instead of having a special statement. As Python is currently compiled, you are right, pass is not needed. A string becomes the doc attribute, and becomes local var 0, but 0 is just ignored. I actually expected a load_const but that is now optimized away. I am not sure this was always true. Perhaps 'pass' is easier than '0' for mewcomers reading the tutorial, but I have no data. >>> def f(): '' >>> def g(): pass >>> def h(): 0 >>> from dis import dis >>> dis(f) 1 0 LOAD_CONST 1 (None) 3 RETURN_VALUE >>> dis(g) 1 0 LOAD_CONST 0 (None) 3 RETURN_VALUE >>> dis(h) 1 0 LOAD_CONST 0 (None) 3 RETURN_VALUE >>> f.__doc__ '' >>> g.__doc__ >>> Terry Jan Reedy From peloko45 at gmail.com Thu Jan 28 13:20:46 2010 From: peloko45 at gmail.com (Joan Miller) Date: Thu, 28 Jan 2010 10:20:46 -0800 (PST) Subject: Wrap a function Message-ID: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> I've to call to many functions with the format: >>> run("cmd") were "cmd" is a command with its arguments to pass them to the shell and run it, i.e. >>> run("pwd") or >>> run("ls /home") Does anybody knows any library to help me to avoid the use of the main quotes, and brackets? I would to use anything as: $ ls /home => run("ls /home") or, at least run pwd => run("pwd") From jeanmichel at sequans.com Thu Jan 28 13:21:42 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Thu, 28 Jan 2010 19:21:42 +0100 Subject: Need help with a program In-Reply-To: <20100128130344.af14ef8e.darcy@druid.net> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> <4B61CE0E.5000801@sequans.com> <20100128130344.af14ef8e.darcy@druid.net> Message-ID: <4B61D5B6.9010004@sequans.com> D'Arcy J.M. Cain wrote: > On Thu, 28 Jan 2010 18:49:02 +0100 > Jean-Michel Pichavant wrote: > >> Using regexp may increase readability (if you are familiar with it). >> > > If you have a problem and you think that regular expressions are the > solution then now you have two problems. Regex is really overkill for > the OP's problem and it certainly doesn't improve readability. > > It depends on the reader ability to understand a *simple* regexp. It is also strange to get such answer after taking so much precautions, so let me quote myself: "Using regexp *may* increase readability (*if* you are *familiar* with it)." I honestly find it quite readable in the sample code I provided and spares all the if-len-startwith-strip logic, but If the OP does not agree, fine with me. But there's no need to get certain that I'm completly wrong. JM From pavlovevidence at gmail.com Thu Jan 28 13:21:52 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Thu, 28 Jan 2010 10:21:52 -0800 (PST) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> Message-ID: On Jan 28, 8:10?am, a... at pythoncraft.com (Aahz) wrote: > In article , > Neil Hodgson ? wrote: > > >Carl Banks: > > >> There is also no hope someone will fork Python 2.x and continue it in > >> perpetuity. ?Well, someone might try to fork it, but they won't be > >> able to call it Python. > > > ? Over time there may be more desire from those unable or unwilling to > >upgrade to 3.x to work on improvements to 2.x, perhaps leading to a > >version 2.8. One of the benefits of open source is that you are not > >trapped into following vendor decisions like Microsoft abandoning > >classic VB in favour of VB.NET. > > > ? It would be unreasonable for the core developers to try to block > >this. Refusing use of the Python trademark for a version that was > >reasonably compatible in both directions would be particularly petty. > > Agreed, and as a PSF member, I'd certainly be opposed to anyone trying to > prevent the release of Python 2.8, and I would actively favor providing > PSF and python.org resources to them. ?OTOH, I would also be likely to > push anyone working on Python 2.8 to come up with a solid release plan > first. Well, I'd consider that an official release. Note that I didn't claim there was no hope PSF wouldn't change it's mind on 2.8. All I saying is that if PSF decides to shut down 2.x there's no hope of a rogue Python 2.x series replacing Python 3.x. Regardless of how magnaminous the people of PSF are, the unfortunate reality is that trademark owners are forced by the law to be "particularly petty". PSF's IP lawyer will advise not to allow unsanctioned fork of Python 2.7 to call itself Python 2.8. Carl Banks From howe.steven at gmail.com Thu Jan 28 13:28:54 2010 From: howe.steven at gmail.com (Steven Howe) Date: Thu, 28 Jan 2010 10:28:54 -0800 Subject: Need help with a program In-Reply-To: <4B61CE0E.5000801@sequans.com> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> <4B61CE0E.5000801@sequans.com> Message-ID: <4B61D766.70300@gmail.com> On 01/28/2010 09:49 AM, Jean-Michel Pichavant wrote: > evilweasel wrote: >> I will make my question a little more clearer. I have close to 60,000 >> lines of the data similar to the one I posted. There are various >> numbers next to the sequence (this is basically the number of times >> the sequence has been found in a particular sample). So, I would need >> to ignore the ones containing '0' and write all other sequences >> (excluding the number, since it is trivial) in a new text file, in the >> following format: >> >>> seq59902 >> TTTTTTTATAAAATATATAGT >> >>> seq59903 >> TTTTTTTATTTCTTGGCGTTGT >> >>> seq59904 >> TTTTTTTGGTTGCCCTGCGTGG >> >>> seq59905 >> TTTTTTTGTTTATTTTTGGG >> >> The number next to 'seq' is the line number of the sequence. When I >> run the above program, what I expect is an output file that is similar >> to the above output but with the ones containing '0' ignored. But, I >> am getting all the sequences printed in the file. >> >> Kindly excuse the 'newbieness' of the program. :) I am hoping to >> improve in the next few months. Thanks to all those who replied. I >> really appreciate it. :) > Using regexp may increase readability (if you are familiar with it). > What about > > import re > > output = open("sequences1.txt", 'w') > > for index, line in enumerate(open(sys.argv[1], 'r')): > match = re.match('(?P[GATC]+)\s+1') > if match: > output.write('seq%s\n%s\n' % (index, match.group('sequence'))) > > > Jean-Michel Finally! After ready 8 or 9 messages about find a line ending with '1', someone suggests Regex. It was my first thought. Steven From tjreedy at udel.edu Thu Jan 28 13:32:11 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 28 Jan 2010 13:32:11 -0500 Subject: python 3's adoption In-Reply-To: <2021A225-2BEE-4DE9-AE88-B9199E5D4491@Comcast.net> References: <2021A225-2BEE-4DE9-AE88-B9199E5D4491@Comcast.net> Message-ID: On 1/28/2010 11:03 AM, Mitchell L Model wrote: > I have been working with Python 3 for over a year. I used it in writing > my book "Bioinformatics Programming Using Python" > (http://oreilly.com/catalog/9780596154509). I didn't see any point in > teaching an incompatible earlier version of a language in transition. In > preparing the book and its examples I explored a large number of Python > modules in some depth and encountered most of the differences between > the language and libraries of Python 2 and Python 3. The change was a > bit awkward for a while, and there were some surprises, but in the end I > have found nothing in Python 3 for which I would prefer Python 2's version. > > Removal of old-style classes is a big win. Having print as a function > provides a tremendous amount of flexibility. I use the sep and end > keywords all the time. There is no reason for print to be a statement, > and it was an awkward inconsistency in a language that leans towards > functional styles. Likewise the elimination of cmp, while shocking, > leads to much simpler comparison arguments to sort, since all the > function does is return a key; then, sort uses __lt__ (I think) so it > automatically uses each class's definition of that. The weird objects > returned from things like sorted, dict.keys/values/items, and so on are > values that in practice are used primarily in iterations; you can always > turn the result into a list, though I have to admit that while > developing and debugging I trip trying to pick out a specific element > from one of these using indexing (typically [0]); I've learned to think > of them as generators, even though they aren't. The rearrangements and > name changes in the libraries are quite helpful. I could go on, but > basically the language and library changes are on the whole large > improvements with little, if any, downside. I agree completely. > Conversion of old code is greatly facilitied by the 2to3 tool that comes > with Python 3. The big issue in moving from 2 to 3 is the external > libraries and development tools you use. Different IDEs have released > versions that support Python 3 at different times. (I believe Wing was > the first.) If you use numpy, for example, or one of the many libraries > that require it, you are stuck. Possibly some important facilities will > never be ported to Python 3, but probably most active projects will > eventually produce a Python 3 version -- for example, according to its > web page, a Python 3 version of PIL is on the way. I was able to cover > all the topics in my book using only Python library modules, something I > felt would be best for readers -- I used libraries such as elementree, > sqlite3, and tkinter. The only disappointment was that I couldn't > include a chapter on BioPython, since there is no Python 3 version. > > By now, many large facilities support both Python 2 and Python 3. I am > currently building a complex GUI/Visualization application based on the > Python 3 version of PyQt4 and Wing IDE and am delighted with all of it. > It may well be that some very important large Something got clipped ;-) Anyway, thank you for the report. Terry Jan Reedy From mensanator at aol.com Thu Jan 28 13:38:17 2010 From: mensanator at aol.com (Mensanator) Date: Thu, 28 Jan 2010 10:38:17 -0800 (PST) Subject: Need help with a program References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> <4B61CE0E.5000801@sequans.com> Message-ID: <2e0ae97b-3aef-4ebf-afdf-0ed53c022b02@k19g2000yqc.googlegroups.com> On Jan 28, 12:28?pm, Steven Howe wrote: > On 01/28/2010 09:49 AM, Jean-Michel Pichavant wrote: > > > > > > > evilweasel wrote: > >> I will make my question a little more clearer. I have close to 60,000 > >> lines of the data similar to the one I posted. There are various > >> numbers next to the sequence (this is basically the number of times > >> the sequence has been found in a particular sample). So, I would need > >> to ignore the ones containing '0' and write all other sequences > >> (excluding the number, since it is trivial) in a new text file, in the > >> following format: > > >>> seq59902 > >> TTTTTTTATAAAATATATAGT > > >>> seq59903 > >> TTTTTTTATTTCTTGGCGTTGT > > >>> seq59904 > >> TTTTTTTGGTTGCCCTGCGTGG > > >>> seq59905 > >> TTTTTTTGTTTATTTTTGGG > > >> The number next to 'seq' is the line number of the sequence. When I > >> run the above program, what I expect is an output file that is similar > >> to the above output but with the ones containing '0' ignored. But, I > >> am getting all the sequences printed in the file. > > >> Kindly excuse the 'newbieness' of the program. :) I am hoping to > >> improve in the next few months. Thanks to all those who replied. I > >> really appreciate it. :) > > Using regexp may increase readability (if you are familiar with it). > > What about > > > import re > > > output = open("sequences1.txt", 'w') > > > for index, line in enumerate(open(sys.argv[1], 'r')): > > ? ?match = re.match('(?P[GATC]+)\s+1') > > ? ?if match: > > ? ? ? ?output.write('seq%s\n%s\n' % (index, match.group('sequence'))) > > > Jean-Michel > > Finally! > > After ready 8 or 9 messages about find a line ending with '1', someone > suggests Regex. > It was my first thought. And as a first thought, it is, of course, wrong. You don't want lines ending in '1', you want ANY non-'0' amount. Likewise, you don't want to exclude lines ending in '0' because you'll end up excluding counts of 10, 20, 30, etc. You need a regex that extracts ALL the numeric characters at the end of the line and exclude those that evaluate to 0. > > Steven From nagle at animats.com Thu Jan 28 13:38:25 2010 From: nagle at animats.com (John Nagle) Date: Thu, 28 Jan 2010 10:38:25 -0800 Subject: Why doesn't "filterwarnings" suppress this message? Message-ID: <4b61d5fb$0$1658$742ec2ed@news.sonic.net> Feedparser isn't supported for Python 3.0, so in Python 2.6, many warning messages appear. I'm trying, in Python 2.6, to suppress the warning message: ./feedparser\feedparser.py:69: DeprecationWarning: the sgmllib module has been removed in Python 3.0 import sgmllib, re, sys, copy, urlparse, time, rfc822, types, cgi, urllib, urllib2 with this warning filter: warnings.filterwarnings(action='ignore', category=DeprecationWarning, module='feedparser') The filter suppresses other deprecation warnings associated with the feedparser module, but not that one. How do I get rid of that one? And no, warnings.filterwarnings(action='ignore', category=DeprecationWarning, module='sgmllib') doesn't do it. John Nagle From __peter__ at web.de Thu Jan 28 14:01:51 2010 From: __peter__ at web.de (Peter Otten) Date: Thu, 28 Jan 2010 20:01:51 +0100 Subject: Why doesn't "filterwarnings" suppress this message? References: <4b61d5fb$0$1658$742ec2ed@news.sonic.net> Message-ID: John Nagle wrote: > Feedparser isn't supported for Python 3.0, so in Python 2.6, many > warning > messages appear. I'm trying, in Python 2.6, to suppress the warning > message: > > ./feedparser\feedparser.py:69: DeprecationWarning: > the sgmllib module has been removed in Python 3.0 > import sgmllib, re, sys, copy, urlparse, time, rfc822, types, > cgi, urllib, urllib2 > > with this warning filter: > > warnings.filterwarnings(action='ignore', > category=DeprecationWarning, module='feedparser') > > The filter suppresses other deprecation warnings associated with > the feedparser module, but not that one. How do I get rid of that > one? > > And no, > > warnings.filterwarnings(action='ignore', > category=DeprecationWarning, module='sgmllib') > > doesn't do it. > > John Nagle I can't confirm that: $ cat nowarning.py import sys if "--off" in sys.argv: import warnings warnings.filterwarnings(action="ignore", category=DeprecationWarning, module="feedparser") import feedparser $ python nowarning.py $ python -3 nowarning.py /usr/lib/python2.6/site.py:1: DeprecationWarning: The 'new' module has been removed in Python 3.0; use the 'types' module instead. """Append module search paths for third-party packages to sys.path. /usr/lib/pymodules/python2.6/feedparser.py:69: DeprecationWarning: the sgmllib module has been removed in Python 3.0 import sgmllib, re, sys, copy, urlparse, time, rfc822, types, cgi, urllib, urllib2 /usr/lib/pymodules/python2.6/feedparser.py:69: DeprecationWarning: in 3.x, rfc822 has been removed in favor of the email package import sgmllib, re, sys, copy, urlparse, time, rfc822, types, cgi, urllib, urllib2 $ python -3 nowarning.py --off /usr/lib/python2.6/site.py:1: DeprecationWarning: The 'new' module has been removed in Python 3.0; use the 'types' module instead. """Append module search paths for third-party packages to sys.path. $ python -V Python 2.6.4 Peter From jrh at joshh.co.uk Thu Jan 28 14:16:11 2010 From: jrh at joshh.co.uk (Josh Holland) Date: Thu, 28 Jan 2010 19:16:11 +0000 (UTC) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> Message-ID: On 2010-01-28, Joan Miller wrote: > I've to call to many functions with the format: > >>>> run("cmd") Check the docs on os.system(). -- Josh "dutchie" Holland http://www.joshh.co.uk/ http://twitter.com/jshholland http://identi.ca/jshholland From stu.doherty at gmail.com Thu Jan 28 14:17:49 2010 From: stu.doherty at gmail.com (Big Stu) Date: Thu, 28 Jan 2010 11:17:49 -0800 (PST) Subject: Great example of a python module/package following up to date conventions. Message-ID: <0876c5e7-e8c5-4334-a000-df310b056723@h2g2000yqj.googlegroups.com> I'm hoping someone on here can point me to an example of a python package that is a great example of how to put it all together. I'm hoping for example code that demonstrates: -Strict adherence to PEP 8 -thorough use of Docstrings -Conventional directory structure/package layout -Appropriate use of the latest accepted coding guidelines in the python community (e.g., new classes versus old classes, Python 3000 compatibility, newer language features, etc. etc.) -Some amount of object oriented design Bonus: -Unit tests -Logging mechanism I can't imagine a package that's been around longer than a few years will hit upon all these things well unless the maintainer went back and did some serious refactoring and re-tooling. Is this question possible to answer? From peloko45 at gmail.com Thu Jan 28 14:24:28 2010 From: peloko45 at gmail.com (Joan Miller) Date: Thu, 28 Jan 2010 11:24:28 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> Message-ID: On 28 ene, 19:16, Josh Holland wrote: > On 2010-01-28, Joan Miller wrote: > > > I've to call to many functions with the format: > > >>>> run("cmd") > > Check the docs on os.system(). No. I've a function that uses subprocess to run commands on the same shell and so substitute to bash scrips. But a script full of run ("shell_command --with --arguments") is too verbose. From jrh at joshh.co.uk Thu Jan 28 14:28:54 2010 From: jrh at joshh.co.uk (Josh Holland) Date: Thu, 28 Jan 2010 19:28:54 +0000 (UTC) Subject: Great example of a python module/package following up to date conventions. References: <0876c5e7-e8c5-4334-a000-df310b056723@h2g2000yqj.googlegroups.com> Message-ID: On 2010-01-28, Big Stu wrote: > I'm hoping someone on here can point me to an example of a python > package that is a great example of how to put it all together. I'm > hoping for example code that demonstrates: Surely most of the Standard Library should satisfy all your requirements? -- Josh "dutchie" Holland http://www.joshh.co.uk/ http://twitter.com/jshholland http://identi.ca/jshholland From no.i.dont at want.mail.from.spammers.com Thu Jan 28 14:31:27 2010 From: no.i.dont at want.mail.from.spammers.com (Fencer) Date: Thu, 28 Jan 2010 20:31:27 +0100 Subject: python 3's adoption In-Reply-To: References: Message-ID: <7se70fFd0mU1@mid.individual.net> On 2010-01-28 17:03, Mitchell L Model wrote: > I have been working with Python 3 for over a year. I used it in writing > my book "Bioinformatics Programming Using Python" > (http://oreilly.com/catalog/9780596154509). That book sounds very interesting, even though I am more interested in the bioinformatic parts, I already know some python. I'll show my boss the link, thanks! - Fencer From stu.doherty at gmail.com Thu Jan 28 14:35:43 2010 From: stu.doherty at gmail.com (Big Stu) Date: Thu, 28 Jan 2010 11:35:43 -0800 (PST) Subject: Great example of a python module/package following up to date conventions. References: <0876c5e7-e8c5-4334-a000-df310b056723@h2g2000yqj.googlegroups.com> Message-ID: <40409dfe-05df-4f4a-b2d0-d7527d650ecf@n7g2000yqb.googlegroups.com> On Jan 28, 2:28?pm, Josh Holland wrote: > On 2010-01-28, Big Stu wrote: > > > I'm hoping someone on here can point me to an example of a python > > package that is a great example of how to put it all together. ?I'm > > hoping for example code that demonstrates: > > Surely most of the Standard Library should satisfy all your > requirements? > > -- > Josh "dutchie" Holland http://www.joshh.co.uk/http://twitter.com/jshhollandhttp://identi.ca/jshholland That's definitely a place I've started to poke around, but the standard library stuff always comes to me by way of my standard python installation. I was hoping to have a template of a 3rd party package to follow. Complete with conventions to follow for easily packaging and distributing via the usual python channels (pypi, easy_install, egg, etc.). From exarkun at twistedmatrix.com Thu Jan 28 14:38:58 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Thu, 28 Jan 2010 19:38:58 -0000 Subject: Great example of a python module/package following up to date conventions. In-Reply-To: References: <0876c5e7-e8c5-4334-a000-df310b056723@h2g2000yqj.googlegroups.com> Message-ID: <20100128193858.1898.1910918008.divmod.xquotient.906@localhost.localdomain> On 07:28 pm, jrh at joshh.co.uk wrote: >On 2010-01-28, Big Stu wrote: >>I'm hoping someone on here can point me to an example of a python >>package that is a great example of how to put it all together. I'm >>hoping for example code that demonstrates: > >Surely most of the Standard Library should satisfy all your >requirements? Have you actually looked at any of the standard library? Jean-Paul From peloko45 at gmail.com Thu Jan 28 14:41:06 2010 From: peloko45 at gmail.com (Joan Miller) Date: Thu, 28 Jan 2010 11:41:06 -0800 (PST) Subject: Great example of a python module/package following up to date conventions. References: <0876c5e7-e8c5-4334-a000-df310b056723@h2g2000yqj.googlegroups.com> Message-ID: On 28 ene, 19:17, Big Stu wrote: > I'm hoping someone on here can point me to an example of a python > package that is a great example of how to put it all together. ?I'm > hoping for example code that demonstrates: > > -Strict adherence to PEP 8 > -thorough use of Docstrings > -Conventional directory structure/package layout > -Appropriate use of the latest accepted coding guidelines in the > python community (e.g., new classes versus old classes, Python 3000 > compatibility, newer language features, etc. etc.) > -Some amount of object oriented design > > Bonus: > -Unit tests > -Logging mechanism > > I can't imagine a package that's been around longer than a few years > will hit upon all these things well unless the maintainer went back > and did some serious refactoring and re-tooling. > > Is this question possible to answer? Look here: http://bitbucket.org/ares/scripy/src/ From jrh at joshh.co.uk Thu Jan 28 14:47:26 2010 From: jrh at joshh.co.uk (Josh Holland) Date: Thu, 28 Jan 2010 19:47:26 +0000 (UTC) Subject: Great example of a python module/package following up to date conventions. References: <0876c5e7-e8c5-4334-a000-df310b056723@h2g2000yqj.googlegroups.com> Message-ID: On 2010-01-28, exarkun at twistedmatrix.com wrote: > Have you actually looked at any of the standard library? Not recently or in depth, no. I would have thought that it would be of high quality. I must have been mistaken. -- Josh "dutchie" Holland http://www.joshh.co.uk/ http://twitter.com/jshholland http://identi.ca/jshholland From stu.doherty at gmail.com Thu Jan 28 14:49:13 2010 From: stu.doherty at gmail.com (Big Stu) Date: Thu, 28 Jan 2010 11:49:13 -0800 (PST) Subject: Great example of a python module/package following up to date conventions. References: <0876c5e7-e8c5-4334-a000-df310b056723@h2g2000yqj.googlegroups.com> Message-ID: > Have you actually looked at any of the standard library? > > Jean-Paul I'm looking at urllib2 right now and it is covering a bunch of the bases I'm looking for. And grepping in the /usr/lib/python2.5/ folder for import statements on various things I'm interested in is bringing up some good examples to check out as well. Given that I'm still fairly novice to this I'm not yet in the position to make a good judgment on what is and isn't a good python practice so I was hoping someone on here might be able to point at a module or 2 that has really done a good job of following best practices. Seems like a reasonable question with an answer that others in a similar position to me might find useful. From steve at holdenweb.com Thu Jan 28 14:51:48 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 28 Jan 2010 14:51:48 -0500 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> Message-ID: <4B61EAD4.3040102@holdenweb.com> Carl Banks wrote: > On Jan 28, 8:10 am, a... at pythoncraft.com (Aahz) wrote: >> In article , >> Neil Hodgson wrote: >> >>> Carl Banks: >>>> There is also no hope someone will fork Python 2.x and continue it in >>>> perpetuity. Well, someone might try to fork it, but they won't be >>>> able to call it Python. >>> Over time there may be more desire from those unable or unwilling to >>> upgrade to 3.x to work on improvements to 2.x, perhaps leading to a >>> version 2.8. One of the benefits of open source is that you are not >>> trapped into following vendor decisions like Microsoft abandoning >>> classic VB in favour of VB.NET. >>> It would be unreasonable for the core developers to try to block >>> this. Refusing use of the Python trademark for a version that was >>> reasonably compatible in both directions would be particularly petty. >> Agreed, and as a PSF member, I'd certainly be opposed to anyone trying to >> prevent the release of Python 2.8, and I would actively favor providing >> PSF and python.org resources to them. OTOH, I would also be likely to >> push anyone working on Python 2.8 to come up with a solid release plan >> first. > > Well, I'd consider that an official release. Note that I didn't claim > there was no hope PSF wouldn't change it's mind on 2.8. All I saying > is that if PSF decides to shut down 2.x there's no hope of a rogue > Python 2.x series replacing Python 3.x. > > Regardless of how magnaminous the people of PSF are, the unfortunate > reality is that trademark owners are forced by the law to be > "particularly petty". PSF's IP lawyer will advise not to allow > unsanctioned fork of Python 2.7 to call itself Python 2.8. > But if it were sanctioned ... ? We *are* pretty magnanimous ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Thu Jan 28 14:51:48 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 28 Jan 2010 14:51:48 -0500 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> Message-ID: <4B61EAD4.3040102@holdenweb.com> Carl Banks wrote: > On Jan 28, 8:10 am, a... at pythoncraft.com (Aahz) wrote: >> In article , >> Neil Hodgson wrote: >> >>> Carl Banks: >>>> There is also no hope someone will fork Python 2.x and continue it in >>>> perpetuity. Well, someone might try to fork it, but they won't be >>>> able to call it Python. >>> Over time there may be more desire from those unable or unwilling to >>> upgrade to 3.x to work on improvements to 2.x, perhaps leading to a >>> version 2.8. One of the benefits of open source is that you are not >>> trapped into following vendor decisions like Microsoft abandoning >>> classic VB in favour of VB.NET. >>> It would be unreasonable for the core developers to try to block >>> this. Refusing use of the Python trademark for a version that was >>> reasonably compatible in both directions would be particularly petty. >> Agreed, and as a PSF member, I'd certainly be opposed to anyone trying to >> prevent the release of Python 2.8, and I would actively favor providing >> PSF and python.org resources to them. OTOH, I would also be likely to >> push anyone working on Python 2.8 to come up with a solid release plan >> first. > > Well, I'd consider that an official release. Note that I didn't claim > there was no hope PSF wouldn't change it's mind on 2.8. All I saying > is that if PSF decides to shut down 2.x there's no hope of a rogue > Python 2.x series replacing Python 3.x. > > Regardless of how magnaminous the people of PSF are, the unfortunate > reality is that trademark owners are forced by the law to be > "particularly petty". PSF's IP lawyer will advise not to allow > unsanctioned fork of Python 2.7 to call itself Python 2.8. > But if it were sanctioned ... ? We *are* pretty magnanimous ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Thu Jan 28 14:54:09 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 28 Jan 2010 14:54:09 -0500 Subject: Wrap a function In-Reply-To: References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> Message-ID: Joan Miller wrote: > On 28 ene, 19:16, Josh Holland wrote: >> On 2010-01-28, Joan Miller wrote: >> >>> I've to call to many functions with the format: >>>>>> run("cmd") >> Check the docs on os.system(). > No. I've a function that uses subprocess to run commands on the same > shell and so substitute to bash scrips. But a script full of run > ("shell_command --with --arguments") is too verbose. So rewrite the script to read the commands from a file and execute them one by one? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From mrholtsr at sbcglobal.net Thu Jan 28 14:57:05 2010 From: mrholtsr at sbcglobal.net (Ray Holt) Date: Thu, 28 Jan 2010 14:57:05 -0500 Subject: Why am I getting this Error message Message-ID: <61E1CBCB55F4401AAEAA0976543E5C0E@ray> Why am I getting the following error message. Area has been declared as an attribute of Circle. Thanks, Ray class Circle: def __init__(self): self.radius = 1 def area(self): return self.radius * self.radius * 3.14159 c = Circle() c.radius = 3 print c.area() Traceback (most recent call last): File "", line 1, in class Circle: File "", line 8, in Circle print c.area() AttributeError: Circle instance has no attribute 'area' -------------- next part -------------- An HTML attachment was scrubbed... URL: From jjposner at optimum.net Thu Jan 28 14:58:50 2010 From: jjposner at optimum.net (John Posner) Date: Thu, 28 Jan 2010 14:58:50 -0500 Subject: Wrap a function In-Reply-To: References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> Message-ID: <4B61EC7A.7020406@optimum.net> On 1/28/2010 2:24 PM, Joan Miller wrote: > On 28 ene, 19:16, Josh Holland wrote: >> On 2010-01-28, Joan Miller wrote: >> >>> I've to call to many functions with the format: >> >>>>>> run("cmd") >> >> Check the docs on os.system(). > No. I've a function that uses subprocess to run commands on the same > shell and so substitute to bash scrips. But a script full of run > ("shell_command --with --arguments") is too verbose. I'm suspicious of your original intent. Essentially, you want to write code in which a literal string, such as ... ls -l ... is *not* enclosed in quotes. Why run the risk of creating confusion (in other people who look at your code, in syntax-checking tools, etc.) between variables and literals? But I'm in sympathy with your desire to make the code as clean as possible and to minimize the number of times you have to type a quote character. My suggestions: 1. Create a function (say, "Run") that encapsulates as much of the syntax as possible: os.system(), subprocess.call(), string-splitting, whatever. So an invocation would look like this: Run("ls -l *.txt") (I think you've already done this step.) 2. Find a text editor that supports keyboard macros, so that a single keystroke turns this text line: ls -l *.txt ... into this one: Run("ls -l *.txt") HTH, John From python at mrabarnett.plus.com Thu Jan 28 14:59:44 2010 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 28 Jan 2010 19:59:44 +0000 Subject: Need help with a program In-Reply-To: <4B61D766.70300@gmail.com> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> <4B61CE0E.5000801@sequans.com> <4B61D766.70300@gmail.com> Message-ID: <4B61ECB0.4030707@mrabarnett.plus.com> Steven Howe wrote: > On 01/28/2010 09:49 AM, Jean-Michel Pichavant wrote: >> evilweasel wrote: >>> I will make my question a little more clearer. I have close to 60,000 >>> lines of the data similar to the one I posted. There are various >>> numbers next to the sequence (this is basically the number of times >>> the sequence has been found in a particular sample). So, I would need >>> to ignore the ones containing '0' and write all other sequences >>> (excluding the number, since it is trivial) in a new text file, in the >>> following format: >>> >>>> seq59902 >>> TTTTTTTATAAAATATATAGT >>> >>>> seq59903 >>> TTTTTTTATTTCTTGGCGTTGT >>> >>>> seq59904 >>> TTTTTTTGGTTGCCCTGCGTGG >>> >>>> seq59905 >>> TTTTTTTGTTTATTTTTGGG >>> >>> The number next to 'seq' is the line number of the sequence. When I >>> run the above program, what I expect is an output file that is similar >>> to the above output but with the ones containing '0' ignored. But, I >>> am getting all the sequences printed in the file. >>> >>> Kindly excuse the 'newbieness' of the program. :) I am hoping to >>> improve in the next few months. Thanks to all those who replied. I >>> really appreciate it. :) >> Using regexp may increase readability (if you are familiar with it). >> What about >> >> import re >> >> output = open("sequences1.txt", 'w') >> >> for index, line in enumerate(open(sys.argv[1], 'r')): >> match = re.match('(?P[GATC]+)\s+1') >> if match: >> output.write('seq%s\n%s\n' % (index, match.group('sequence'))) >> >> >> Jean-Michel > > Finally! > > After ready 8 or 9 messages about find a line ending with '1', someone > suggests Regex. > It was my first thought. > I'm a great fan of regexes, but I never though of using them for this because it doesn't look like a regex type of problem to me. From peloko45 at gmail.com Thu Jan 28 15:06:04 2010 From: peloko45 at gmail.com (Joan Miller) Date: Thu, 28 Jan 2010 12:06:04 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> Message-ID: <012da98d-959e-4de4-a4a1-17ec9fdcc209@a5g2000yqi.googlegroups.com> On 28 ene, 19:54, Steve Holden wrote: > Joan Miller wrote: > > On 28 ene, 19:16, Josh Holland wrote: > >> On 2010-01-28, Joan Miller wrote: > > >>> I've to call to many functions with the format: > >>>>>> run("cmd") > >> Check the docs on os.system(). > > No. I've a function that uses subprocess to run commands on the same > > shell and so substitute to bash scrips. But a script full of run > > ("shell_command --with --arguments") is too verbose. > > So rewrite the script to read the commands from a file and execute them > one by one? > I had thinked about that but the problem is that I would that were mixed with python code, so can be get the output from a system command and manipulate it from python From exarkun at twistedmatrix.com Thu Jan 28 15:11:14 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Thu, 28 Jan 2010 20:11:14 -0000 Subject: Great example of a python module/package following up to date conventions. In-Reply-To: References: <0876c5e7-e8c5-4334-a000-df310b056723@h2g2000yqj.googlegroups.com> Message-ID: <20100128201114.1898.1971415063.divmod.xquotient.918@localhost.localdomain> On 07:49 pm, stu.doherty at gmail.com wrote: > >>Have you actually looked at any of the standard library? >> >>Jean-Paul > >I'm looking at urllib2 right now and it is covering a bunch of the >bases I'm looking for. And grepping in the /usr/lib/python2.5/ folder >for import statements on various things I'm interested in is bringing >up some good examples to check out as well. Given that I'm still >fairly novice to this I'm not yet in the position to make a good >judgment on what is and isn't a good python practice so I was hoping >someone on here might be able to point at a module or 2 that has >really done a good job of following best practices. > >Seems like a reasonable question with an answer that others in a >similar position to me might find useful. You're right. I was actually wondering if Josh had looked before suggesting it. :) The stdlib varies wildly in quality, with much of it not serving as a particular good example of most of the points you mentioned. urllib2 is probably better than a lot, but, for example, even it only manages about 75% line coverage by its test suite. Jean-Paul From peloko45 at gmail.com Thu Jan 28 15:13:14 2010 From: peloko45 at gmail.com (Joan Miller) Date: Thu, 28 Jan 2010 12:13:14 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> <4B61EC7A.7020406@optimum.net> Message-ID: On 28 ene, 19:58, John Posner wrote: > On 1/28/2010 2:24 PM, Joan Miller wrote: > > > On 28 ene, 19:16, Josh Holland ?wrote: > >> On 2010-01-28, Joan Miller ?wrote: > > >>> I've to call to many functions with the format: > > >>>>>> run("cmd") > > >> Check the docs on os.system(). > > No. I've a function that uses subprocess to run commands on the same > > shell and so substitute to bash scrips. But a script full of run > > ("shell_command --with --arguments") is too verbose. > > I'm suspicious of your original intent. Essentially, you want to write > code in which a literal string, such as ... > > ? ?ls -l > > ... is *not* enclosed in quotes. Why run the risk of creating confusion > (in other people who look at your code, in syntax-checking tools, etc.) > between variables and literals? Yes but to that code could be prepend a sign as '$' to be identified and so be parsed. > > But I'm in sympathy with your desire to make the code as clean as > possible and to minimize the number of times you have to type a quote > character. My suggestions: > > 1. Create a function (say, "Run") that encapsulates as much of the > syntax as possible: os.system(), subprocess.call(), string-splitting, > whatever. So an invocation would look like this: > > ? ?Run("ls -l *.txt") > > (I think you've already done this step.) Yes, I made a funtion very cool to call to system commands, that works well with pipes and passes the variables (i.e. "LANG=C grep -e 'foo' / home") > 2. Find a text editor that supports keyboard macros, so that a single > keystroke turns this text line: > > ? ?ls -l *.txt > > ... into this one: > > ? ?Run("ls -l *.txt") This is not what I'm looking for. I'm supposing that could be solved with a DSL or a macro library, any? From clp2 at rebertia.com Thu Jan 28 15:16:14 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Thu, 28 Jan 2010 12:16:14 -0800 Subject: Why am I getting this Error message In-Reply-To: <61E1CBCB55F4401AAEAA0976543E5C0E@ray> References: <61E1CBCB55F4401AAEAA0976543E5C0E@ray> Message-ID: <50697b2c1001281216y5f89145csd38f2a0e5fff0c70@mail.gmail.com> On Thu, Jan 28, 2010 at 11:57 AM, Ray Holt wrote: > Why am I getting the following error message. Area has been declared as an > attribute of Circle. Thanks, Ray > > class Circle: > ?def __init__(self): > ???? self.radius = 1 > ?def area(self): > ???? return self.radius * self.radius * 3.14159 > ?c = Circle() > ?c.radius = 3 > ?print c.area() > > Traceback (most recent call last): > ? File "", line 1, in > ??? class Circle: > ? File "", line 8, in Circle > ??? print c.area() > AttributeError: Circle instance has no attribute 'area' Unable to reproduce: Python 2.6.4 (r264:75706, Dec 20 2009, 15:52:35) [GCC 4.2.1 (Apple Inc. build 5646) (dot 1)] on darwin >>> class Circle: ... def __init__(self): ... self.radius = 1 ... def area(self): ... return self.radius * self.radius * 3.14159 ... >>> c = Circle() >>> c.radius = 3 >>> print c.area() 28.27431 Cheers, Chris -- http://blog.rebertia.com From half.italian at gmail.com Thu Jan 28 15:18:26 2010 From: half.italian at gmail.com (Sean DiZazzo) Date: Thu, 28 Jan 2010 12:18:26 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> <4B61EC7A.7020406@optimum.net> Message-ID: On Jan 28, 12:13?pm, Joan Miller wrote: > On 28 ene, 19:58, John Posner wrote: > > > > > On 1/28/2010 2:24 PM, Joan Miller wrote: > > > > On 28 ene, 19:16, Josh Holland ?wrote: > > >> On 2010-01-28, Joan Miller ?wrote: > > > >>> I've to call to many functions with the format: > > > >>>>>> run("cmd") > > > >> Check the docs on os.system(). > > > No. I've a function that uses subprocess to run commands on the same > > > shell and so substitute to bash scrips. But a script full of run > > > ("shell_command --with --arguments") is too verbose. > > > I'm suspicious of your original intent. Essentially, you want to write > > code in which a literal string, such as ... > > > ? ?ls -l > > > ... is *not* enclosed in quotes. Why run the risk of creating confusion > > (in other people who look at your code, in syntax-checking tools, etc.) > > between variables and literals? > > Yes but to that code could be prepend a sign as '$' to be identified > and so be parsed. > > > > > But I'm in sympathy with your desire to make the code as clean as > > possible and to minimize the number of times you have to type a quote > > character. My suggestions: > > > 1. Create a function (say, "Run") that encapsulates as much of the > > syntax as possible: os.system(), subprocess.call(), string-splitting, > > whatever. So an invocation would look like this: > > > ? ?Run("ls -l *.txt") > > > (I think you've already done this step.) > > Yes, I made a funtion very cool to call to system commands, that works > well with pipes and passes the variables (i.e. "LANG=C grep -e 'foo' / > home") > > > 2. Find a text editor that supports keyboard macros, so that a single > > keystroke turns this text line: > > > ? ?ls -l *.txt > > > ... into this one: > > > ? ?Run("ls -l *.txt") > > This is not what I'm looking for. I'm supposing that could be solved > with a DSL or a macro library, any? Python is not perl. Thank God/Guido. From peter.milliken at gmail.com Thu Jan 28 15:20:48 2010 From: peter.milliken at gmail.com (Peter) Date: Thu, 28 Jan 2010 12:20:48 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> <4B61EC7A.7020406@optimum.net> Message-ID: On Jan 29, 6:58?am, John Posner wrote: > On 1/28/2010 2:24 PM, Joan Miller wrote: > > > On 28 ene, 19:16, Josh Holland ?wrote: > >> On 2010-01-28, Joan Miller ?wrote: > > >>> I've to call to many functions with the format: > > >>>>>> run("cmd") > > >> Check the docs on os.system(). > > No. I've a function that uses subprocess to run commands on the same > > shell and so substitute to bash scrips. But a script full of run > > ("shell_command --with --arguments") is too verbose. > > I'm suspicious of your original intent. Essentially, you want to write > code in which a literal string, such as ... > > ? ?ls -l > > ... is *not* enclosed in quotes. Why run the risk of creating confusion > (in other people who look at your code, in syntax-checking tools, etc.) > between variables and literals? > > But I'm in sympathy with your desire to make the code as clean as > possible and to minimize the number of times you have to type a quote > character. My suggestions: > > 1. Create a function (say, "Run") that encapsulates as much of the > syntax as possible: os.system(), subprocess.call(), string-splitting, > whatever. So an invocation would look like this: > > ? ?Run("ls -l *.txt") > > (I think you've already done this step.) > > 2. Find a text editor that supports keyboard macros, so that a single > keystroke turns this text line: > > ? ?ls -l *.txt > > ... into this one: > > ? ?Run("ls -l *.txt") > > HTH, > John I can't see you avoiding quotes etc, but extending on John's comment, the obvious next step would be to run everything in a loop i.e. place all the commands into a list and create a loop that ran each command in the list. Almost all editors support macros - most editors support some form of language sensitive editing (NOT the prompt call parameters style but rather help with the syntax via a 'form' style of fill-in) that will allow you to reduce typing effort. But macros would be the first and easiest choice for this activity. Peter From steve at holdenweb.com Thu Jan 28 15:25:19 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 28 Jan 2010 15:25:19 -0500 Subject: Why am I getting this Error message In-Reply-To: <61E1CBCB55F4401AAEAA0976543E5C0E@ray> References: <61E1CBCB55F4401AAEAA0976543E5C0E@ray> Message-ID: Ray Holt wrote: > Why am I getting the following error message. Area has been declared as > an attribute of Circle. Thanks, Ray > > > class Circle: > def __init__(self): > self.radius = 1 > def area(self): > return self.radius * self.radius * 3.14159 > c = Circle() > c.radius = 3 > print c.area() > > > > Traceback (most recent call last): > File "", line 1, in > class Circle: > File "", line 8, in Circle > print c.area() > AttributeError: Circle instance has no attribute 'area' > Because you have indented the last three lines to make them a part of the Circle definition. Generally four spaces makes a better indent, and that kind of problem is then much more obvious. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From python at mrabarnett.plus.com Thu Jan 28 15:33:50 2010 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 28 Jan 2010 20:33:50 +0000 Subject: Why am I getting this Error message In-Reply-To: <61E1CBCB55F4401AAEAA0976543E5C0E@ray> References: <61E1CBCB55F4401AAEAA0976543E5C0E@ray> Message-ID: <4B61F4AE.8080803@mrabarnett.plus.com> Ray Holt wrote: > Why am I getting the following error message. Area has been declared as > an attribute of Circle. Thanks, Ray > > > class Circle: > def __init__(self): > self.radius = 1 > def area(self): > return self.radius * self.radius * 3.14159 > c = Circle() > c.radius = 3 > print c.area() > > > > Traceback (most recent call last): > File "", line 1, in > class Circle: > File "", line 8, in Circle > print c.area() > AttributeError: Circle instance has no attribute 'area' > Probably incorrect indentation. This works: class Circle: def __init__(self): self.radius = 1 def area(self): return self.radius * self.radius * 3.14159 c = Circle() c.radius = 3 print c.area() From peloko45 at gmail.com Thu Jan 28 15:34:50 2010 From: peloko45 at gmail.com (Joan Miller) Date: Thu, 28 Jan 2010 12:34:50 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> <4B61EC7A.7020406@optimum.net> Message-ID: <64dc1630-6aa5-4983-b061-261457bf0153@g29g2000yqe.googlegroups.com> On 28 ene, 20:20, Peter wrote: > On Jan 29, 6:58?am, John Posner wrote: > > > > > On 1/28/2010 2:24 PM, Joan Miller wrote: > > > > On 28 ene, 19:16, Josh Holland ?wrote: > > >> On 2010-01-28, Joan Miller ?wrote: > > > >>> I've to call to many functions with the format: > > > >>>>>> run("cmd") > > > >> Check the docs on os.system(). > > > No. I've a function that uses subprocess to run commands on the same > > > shell and so substitute to bash scrips. But a script full of run > > > ("shell_command --with --arguments") is too verbose. > > > I'm suspicious of your original intent. Essentially, you want to write > > code in which a literal string, such as ... > > > ? ?ls -l > > > ... is *not* enclosed in quotes. Why run the risk of creating confusion > > (in other people who look at your code, in syntax-checking tools, etc.) > > between variables and literals? > > > But I'm in sympathy with your desire to make the code as clean as > > possible and to minimize the number of times you have to type a quote > > character. My suggestions: > > > 1. Create a function (say, "Run") that encapsulates as much of the > > syntax as possible: os.system(), subprocess.call(), string-splitting, > > whatever. So an invocation would look like this: > > > ? ?Run("ls -l *.txt") > > > (I think you've already done this step.) > > > 2. Find a text editor that supports keyboard macros, so that a single > > keystroke turns this text line: > > > ? ?ls -l *.txt > > > ... into this one: > > > ? ?Run("ls -l *.txt") > > > HTH, > > John > > I can't see you avoiding quotes etc, but extending on John's comment, > the obvious next step would be to run everything in a loop i.e. place > all the commands into a list and create a loop that ran each command > in the list. Yes, but could be necessary that were mixed with python code. > Almost all editors support macros - most editors support some form of > language sensitive editing (NOT the prompt call parameters style but > rather help with the syntax via a 'form' style of fill-in) that will > allow you to reduce typing effort. But macros would be the first and > easiest choice for this activity. The goal of my program is substitute to bash scripts, so the macros in editors are irrelevant fo this one. From peloko45 at gmail.com Thu Jan 28 15:45:24 2010 From: peloko45 at gmail.com (Joan Miller) Date: Thu, 28 Jan 2010 12:45:24 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> <4B61EC7A.7020406@optimum.net> <64dc1630-6aa5-4983-b061-261457bf0153@g29g2000yqe.googlegroups.com> Message-ID: <8508278f-0cf2-46c2-8313-2c640a71cdbd@q4g2000yqm.googlegroups.com> On 28 ene, 20:34, Joan Miller wrote: > On 28 ene, 20:20, Peter wrote: > > > On Jan 29, 6:58?am, John Posner wrote: > > > > On 1/28/2010 2:24 PM, Joan Miller wrote: > > > > > On 28 ene, 19:16, Josh Holland ?wrote: > > > >> On 2010-01-28, Joan Miller ?wrote: > > > > >>> I've to call to many functions with the format: > > > > >>>>>> run("cmd") > > > > >> Check the docs on os.system(). > > > > No. I've a function that uses subprocess to run commands on the same > > > > shell and so substitute to bash scrips. But a script full of run > > > > ("shell_command --with --arguments") is too verbose. > > > > I'm suspicious of your original intent. Essentially, you want to write > > > code in which a literal string, such as ... > > > > ? ?ls -l > > > > ... is *not* enclosed in quotes. Why run the risk of creating confusion > > > (in other people who look at your code, in syntax-checking tools, etc.) > > > between variables and literals? > > > > But I'm in sympathy with your desire to make the code as clean as > > > possible and to minimize the number of times you have to type a quote > > > character. My suggestions: > > > > 1. Create a function (say, "Run") that encapsulates as much of the > > > syntax as possible: os.system(), subprocess.call(), string-splitting, > > > whatever. So an invocation would look like this: > > > > ? ?Run("ls -l *.txt") > > > > (I think you've already done this step.) > > > > 2. Find a text editor that supports keyboard macros, so that a single > > > keystroke turns this text line: > > > > ? ?ls -l *.txt > > > > ... into this one: > > > > ? ?Run("ls -l *.txt") > > > > HTH, > > > John > > > I can't see you avoiding quotes etc, but extending on John's comment, > > the obvious next step would be to run everything in a loop i.e. place > > all the commands into a list and create a loop that ran each command > > in the list. > > Yes, but could be necessary that were mixed with python code. > > > Almost all editors support macros - most editors support some form of > > language sensitive editing (NOT the prompt call parameters style but > > rather help with the syntax via a 'form' style of fill-in) that will > > allow you to reduce typing effort. But macros would be the first and > > easiest choice for this activity. > > The goal of my program is substitute to bash scripts, so the macros in > editors are irrelevant fo this one. I think that the best solution that I've is to build a program that parses the script to convert *$ command* to run("command") before of be called by python. From ps.ohms at gmail.com Thu Jan 28 15:53:30 2010 From: ps.ohms at gmail.com (PS.OHM) Date: Thu, 28 Jan 2010 12:53:30 -0800 (PST) Subject: get error install MySQLdb on Mac OS X Message-ID: <30c0946e-fa6a-4a54-930c-1962d167fffe@x10g2000prk.googlegroups.com> Hello Guys I have get some error when i install MySQLdb on Mac OS X after i key command $python setup.py build rusult is : : error: command 'gcc-4.0' failed with exit status 1 How to config this poblem? From lanyjie at yahoo.com Thu Jan 28 16:12:55 2010 From: lanyjie at yahoo.com (Yingjie Lan) Date: Thu, 28 Jan 2010 13:12:55 -0800 (PST) Subject: mix statically typed with dynamically typed In-Reply-To: <64dc1630-6aa5-4983-b061-261457bf0153@g29g2000yqe.googlegroups.com> Message-ID: <708505.12453.qm@web54201.mail.re2.yahoo.com> We all know that Python is dynamically typed, and dynamically typed languages are generally slower than statically typed ones. I wonder if it is possible at all for Python to mix statically-typed-ness with dynamically-typed-ness to boost up its speed a little bit, especially when speed is needed. For example, you define a function like this: def speed(float dist, float time): return dist/time then the compiler would generate code to first check parameter types (or even do some casts if appropriate, say cast an int into float) in the beginning of this function. and the rest of the function would then be compiled with the assumption that 'dist' and 'time' are of the type float. Of course, dynamically-typed-ness is still the same as before. Python is well known for providing multiple programming paradigms, I wonder if we could also sneak this in nicely. Any thoughts? From tjreedy at udel.edu Thu Jan 28 16:21:59 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 28 Jan 2010 16:21:59 -0500 Subject: myths about python 3 In-Reply-To: <4B61EAD4.3040102@holdenweb.com> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> <4B61EAD4.3040102@holdenweb.com> Message-ID: On 1/28/2010 2:51 PM, Steve Holden wrote: > Carl Banks wrote: >> Regardless of how magnaminous the people of PSF are, the unfortunate >> reality is that trademark owners are forced by the law to be >> "particularly petty". PSF's IP lawyer will advise not to allow >> unsanctioned fork of Python 2.7 to call itself Python 2.8. >> > But if it were sanctioned ... ? We *are* pretty magnanimous ;-) I think it foolish to speculate in the absence of specifics. If some people wanted to coninue bug-fix maintainance of 2.7 after the main group of developers is done with it, in 5 years, then no new name is needed. If some people wanted to backport additional 3.x features, while still keeping the old, obsolete stuff around, then '2.8' would be appropriate. If some people wanted to add a collection of incompatible new features, perhaps some that Guido has rejected for 'Python', so that they were producing a real fork, then a new name should be used. I consider the first option possible, assuming that significant bugs still remain in 5 years. The second seems more dubious, as the developers have already backported most of what they thought sensible. The third has always been possible, and has been done, and there would be nothing really special about using 2.7 as a base. Terry Jan Reedy From pruebauno at latinmail.com Thu Jan 28 16:22:10 2010 From: pruebauno at latinmail.com (nn) Date: Thu, 28 Jan 2010 13:22:10 -0800 (PST) Subject: Need help with a program References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: Arnaud Delobelle wrote: > nn writes: > > > On Jan 28, 10:50?am, evilweasel wrote: > >> I will make my question a little more clearer. I have close to 60,000 > >> lines of the data similar to the one I posted. There are various > >> numbers next to the sequence (this is basically the number of times > >> the sequence has been found in a particular sample). So, I would need > >> to ignore the ones containing '0' and write all other sequences > >> (excluding the number, since it is trivial) in a new text file, in the > >> following format: > >> > >> >seq59902 > >> > >> TTTTTTTATAAAATATATAGT > >> > >> >seq59903 > >> > >> TTTTTTTATTTCTTGGCGTTGT > >> > >> >seq59904 > >> > >> TTTTTTTGGTTGCCCTGCGTGG > >> > >> >seq59905 > >> > >> TTTTTTTGTTTATTTTTGGG > >> > >> The number next to 'seq' is the line number of the sequence. When I > >> run the above program, what I expect is an output file that is similar > >> to the above output but with the ones containing '0' ignored. But, I > >> am getting all the sequences printed in the file. > >> > >> Kindly excuse the 'newbieness' of the program. :) I am hoping to > >> improve in the next few months. Thanks to all those who replied. I > >> really appreciate it. :) > > > > People have already given you some pointers to your problem. In the > > end you will have to "tweak the details" because only you have access > > to the data not us. > > > > Just as example here is another way to do what you are doing: > > > > with open('dnain.dat') as infile, open('dnaout.dat','w') as outfile: > > partgen=(line.split() for line in infile) > > dnagen=(str(i+1)+'\n'+part[0]+'\n' > > for i,part in enumerate(partgen) > > if len(part)>1 and part[1]!='0') > > outfile.writelines(dnagen) > > I think that generator expressions are overrated :) What's wrong with: > > with open('dnain.dat') as infile, open('dnaout.dat','w') as outfile: > for i, line in enumerate(infile): > parts = line.split() > if len(parts) > 1 and parts[1] != '0': > outfile.write(">seq%s\n%s\n" % (i+1, parts[0])) > > (untested) > > -- > Arnaud Nothing really, After posting I was thinking I should have posted a more straightforward version like the one you wrote. Now there is! It probably is more efficient too. I just have a tendency to think in terms of pipes: "pipe this junk in here, then in here, get output". Probably damage from too much Unix scripting.Since I can't resist the urge to post crazy code here goes the bonus round (don't do this at work): open('dnaout.dat','w').writelines( 'seq%s\n%s\n'%(i+1,part[0]) for i,part in enumerate(line.split() for line in open('dnain.dat')) if len(part)>1 and part[1]!='0') From deets at nospam.web.de Thu Jan 28 16:27:21 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Thu, 28 Jan 2010 22:27:21 +0100 Subject: mix statically typed with dynamically typed In-Reply-To: References: Message-ID: <4B620139.2080004@nospam.web.de> Am 28.01.10 22:12, schrieb Yingjie Lan: > We all know that Python is dynamically typed, and dynamically typed languages are generally slower than statically typed ones. I wonder if it is possible at all for Python to mix statically-typed-ness with dynamically-typed-ness to boost up its speed a little bit, especially when speed is needed. For example, you define a function like this: > > def speed(float dist, float time): > return dist/time > > then the compiler would generate code to first check parameter types (or even do some casts if appropriate, say cast an int into float) in the beginning of this function. and the rest of the function would then be compiled with the assumption that 'dist' and 'time' are of the type float. > > Of course, dynamically-typed-ness is still the same as before. Python is well known for providing multiple programming paradigms, I wonder if we could also sneak this in nicely. There are various attempts to achieve this. The most generic one, which is most promising in the long run is PyPy, the implementation of Python in itself, with the added benefit of making code-generators that emit e.g. C based on Python-code. Then there is Cython, which blends Python with C & integrates very nicely. Last but not least, for you actual example, psyco is the easiest thing to use, it's a JIT aimed to especially optimize numeric operations as the one you present. Diez From ben+python at benfinney.id.au Thu Jan 28 16:33:58 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 29 Jan 2010 08:33:58 +1100 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <87tyu529tl.fsf@benfinney.id.au> Antoine Pitrou writes: > Le Thu, 28 Jan 2010 00:19:24 +0000, Steven D'Aprano a ?crit?: > > 4. Python 3 will make you irresistible to women. > > > > FALSE - Python 3 coders are no more likely to get a date than > > any other programmer. > > They spend less time coding, so they /can/ get more "dates" (what a > strange English word) :-) Perhaps Steven could tell you about a lovely Australian meaning for the word ?date?. -- \ ?Even if the voices in my head are not real, they have pretty | `\ good ideas.? ?anonymous | _o__) | Ben Finney From steve at holdenweb.com Thu Jan 28 16:37:31 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 28 Jan 2010 16:37:31 -0500 Subject: mix statically typed with dynamically typed In-Reply-To: <708505.12453.qm@web54201.mail.re2.yahoo.com> References: <64dc1630-6aa5-4983-b061-261457bf0153@g29g2000yqe.googlegroups.com> <708505.12453.qm@web54201.mail.re2.yahoo.com> Message-ID: Yingjie Lan wrote: > We all know that Python is dynamically typed, and dynamically typed languages are generally slower than statically typed ones. I wonder if it is possible at all for Python to mix statically-typed-ness with dynamically-typed-ness to boost up its speed a little bit, especially when speed is needed. For example, you define a function like this: > > def speed(float dist, float time): > return dist/time > > then the compiler would generate code to first check parameter types (or even do some casts if appropriate, say cast an int into float) in the beginning of this function. and the rest of the function would then be compiled with the assumption that 'dist' and 'time' are of the type float. > > Of course, dynamically-typed-ness is still the same as before. Python is well known for providing multiple programming paradigms, I wonder if we could also sneak this in nicely. > > Any thoughts? > > > Google for "Python function annotations": the features you want are already there in the language specification. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From jgardner at jonathangardner.net Thu Jan 28 16:40:20 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Thu, 28 Jan 2010 13:40:20 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> Message-ID: <9a28cf01-20af-4ce2-9b78-d8529800e4b9@y7g2000prc.googlegroups.com> On Jan 28, 10:20?am, Joan Miller wrote: > I've to call to many functions with the format: > > >>> run("cmd") > > were "cmd" is a command with its arguments to pass them to the shell > and run it, i.e. > > > > >>> ?run("pwd") > or > >>> run("ls /home") > > Does anybody knows any library to help me to avoid the use of the main > quotes, and brackets? > > I would to use anything as: > > $ ls /home => run("ls /home") > > or, at least > > run pwd => run("pwd") How about this? def pwd(): return run("pwd") pwd() def ls(l=False, files=()): args = [] if l: args.insert(0, '-l') args.append(files) return run("ls", args) ls(l=True, "/foo") From arnodel at googlemail.com Thu Jan 28 16:42:17 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Thu, 28 Jan 2010 21:42:17 +0000 Subject: Need help with a program References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: nn writes: > After posting I was thinking I should have posted a more > straightforward version like the one you wrote. Now there is! It > probably is more efficient too. I just have a tendency to think in > terms of pipes: "pipe this junk in here, then in here, get output". > Probably damage from too much Unix scripting. This is funny, I did think *exactly* this when I saw your code :) -- Arnaud From ben+python at benfinney.id.au Thu Jan 28 16:42:38 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 29 Jan 2010 08:42:38 +1100 Subject: Great example of a python module/package following up to date conventions. References: <0876c5e7-e8c5-4334-a000-df310b056723@h2g2000yqj.googlegroups.com> Message-ID: <87pr4t29f5.fsf@benfinney.id.au> Josh Holland writes: > On 2010-01-28, exarkun at twistedmatrix.com wrote: > > Have you actually looked at any of the standard library? > Not recently or in depth, no. I would have thought that it would be of > high quality. I must have been mistaken. Style conventions were introduced relatively late in the history of Python (PEP 7, PEP 8, and PEP 257 were created in 2001). The existing standard library code works as-is, so no particular effort has gone into cleaning it up to retroactively conform. Also, existing APIs in the standard library tend to be preserved as non-conformant (I'm looking at you, ?logging? and ?unittest?) rather than breaking existing code by changing the API. The end result is that there are huge swaths of the standard library that do not adhere to the style conventions. It could even be argued that part of the reason for introducing the official conventions was to limit the scale of the damage already done. -- \ ?Facts do not cease to exist because they are ignored.? ?Aldous | `\ Huxley | _o__) | Ben Finney From alfps at start.no Thu Jan 28 16:43:37 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 28 Jan 2010 22:43:37 +0100 Subject: mix statically typed with dynamically typed In-Reply-To: References: Message-ID: * Yingjie Lan: [snip] > > def speed(float dist, float time): > return dist/time > > then the compiler would generate code to first check parameter types > (or even do some casts if appropriate, say cast an int into float) in > the beginning of this function. and the rest of the function would then > be compiled with the assumption that 'dist' and 'time' are of the type > float. > > Of course, dynamically-typed-ness is still the same as before. Python > is well known for providing multiple programming paradigms, I wonder if > we could also sneak this in nicely. > > Any thoughts? Python already has the /syntax/, e.g. >>> def speed( dist: float, time: float ) -> float: ... return dist/time ... >>> print( speed.__annotations__ ) {'dist': , 'return': , 'time': } >>> _ However, this syntax, while exploitable, is by default nothing but an annotation device, like doc strings. I'm not sure I like your idea of introducing static typing to increase speed, but it could be done without introducing new syntax simply by defining a special meaning to such annotation expressions that are 'type' invocations, say, then like def speed( dist: type( float ), time: type( float ) ) -> type( float ) Since there are umpteen projects to increase speed of Python this idea may already have been explored... Cheers & hth., - Alf (who has some other ideas) From ben+python at benfinney.id.au Thu Jan 28 16:52:52 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 29 Jan 2010 08:52:52 +1100 Subject: Great example of a python module/package following up to date conventions. References: <0876c5e7-e8c5-4334-a000-df310b056723@h2g2000yqj.googlegroups.com> Message-ID: <87ljfh28y3.fsf@benfinney.id.au> Big Stu writes: > I'm hoping someone on here can point me to an example of a python > package that is a great example of how to put it all together. I'm > hoping for example code that demonstrates [good adherence to style and > practice conventions] I find the code base for Bazaar to be particularly good quality : $ sudo aptitude install bzr $ bzr branch lp:bzr $ cd bzr/ $ # browse browse browse -- \ ?Probably the earliest flyswatters were nothing more than some | `\ sort of striking surface attached to the end of a long stick.? | _o__) ?Jack Handey | Ben Finney From duncan.booth at invalid.invalid Thu Jan 28 16:53:45 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 28 Jan 2010 21:53:45 GMT Subject: mix statically typed with dynamically typed References: Message-ID: "Alf P. Steinbach" wrote: > I'm not sure I like your idea of introducing static typing to increase > speed, but it could be done without introducing new syntax simply by > defining a special meaning to such annotation expressions that are > 'type' invocations, say, then like > > def speed( dist: type( float ), time: type( float ) ) -> type( > float ) > That would be particularly useless: >>> type(float) is type True So your declaration is identical to: def speed(dist: type, time: type) -> type: Much better just to stick to something like: def speed( dist: float, time: float ) -> float: where at least you can tell from the annotations what types were actually used. From MLMDev at Comcast.net Thu Jan 28 16:54:38 2010 From: MLMDev at Comcast.net (Mitchell L Model) Date: Thu, 28 Jan 2010 16:54:38 -0500 Subject: python 3's adoption In-Reply-To: References: Message-ID: On Jan 28, 2010, at 12:00 PM, python-list-request at python.org wrote: > From: Roy Smith > Date: January 28, 2010 11:09:58 AM EST > To: python-list at python.org > Subject: Re: python 3's adoption > > > In article , > Mitchell L Model wrote: > >> I use the sep and end keywords all the time. > > What are 'sep' and 'end'? I'm looking in > http://docs.python.org/3.1/genindex-all.html and don't see those > mentioned > at all. Am I just looking in the wrong place? > Sorry -- I wasn't clear. They are keyword arguments to the print function. From MLMDev at Comcast.net Thu Jan 28 17:00:10 2010 From: MLMDev at Comcast.net (Mitchell L Model) Date: Thu, 28 Jan 2010 17:00:10 -0500 Subject: python 3's adoption In-Reply-To: References: Message-ID: On Jan 28, 2010, at 1:40 PM, Terry Reedy wrote > ... > On 1/28/2010 11:03 AM, Mitchell L Model wrote: >> I have been working with Python 3 for over a year. ... > > I agree completely. Such sweet words to read! > >> Conversion of old code is greatly facilitied by the 2to3 tool that >> comes >> with Python 3. The big issue in moving from 2 to 3 is the external >> libraries and development tools you use. Different IDEs have released >> versions that support Python 3 at different times. (I believe Wing >> was >> the first.) If you use numpy, for example, or one of the many >> libraries >> that require it, you are stuck. Possibly some important facilities >> will >> never be ported to Python 3, but probably most active projects will >> eventually produce a Python 3 version -- for example, according to >> its >> web page, a Python 3 version of PIL is on the way. I was able to >> cover >> all the topics in my book using only Python library modules, >> something I >> felt would be best for readers -- I used libraries such as >> elementree, >> sqlite3, and tkinter. The only disappointment was that I couldn't >> include a chapter on BioPython, since there is no Python 3 version. >> >> By now, many large facilities support both Python 2 and Python 3. I >> am >> currently building a complex GUI/Visualization application based on >> the >> Python 3 version of PyQt4 and Wing IDE and am delighted with all of >> it. >> It may well be that some very important large > > Something got clipped ;-) Thanks for noticing. Actually, I had abandoned that sentence and went back and added more to the prior paragraph. Just never went back and deleted the false start. > > Anyway, thank you for the report. > Glad to contribute; gladder to be appreciated. From half.italian at gmail.com Thu Jan 28 17:02:39 2010 From: half.italian at gmail.com (Sean DiZazzo) Date: Thu, 28 Jan 2010 14:02:39 -0800 (PST) Subject: get error install MySQLdb on Mac OS X References: <30c0946e-fa6a-4a54-930c-1962d167fffe@x10g2000prk.googlegroups.com> Message-ID: On Jan 28, 12:53?pm, "PS.OHM" wrote: > Hello Guys > > I have get some error when i install MySQLdb on Mac OS X > > after i key command $python setup.py build > > rusult is > : > : > error: command 'gcc-4.0' failed with exit status 1 > > How to config this poblem? Please show a little bit more of the error From alfps at start.no Thu Jan 28 17:07:24 2010 From: alfps at start.no (Alf P. Steinbach) Date: Thu, 28 Jan 2010 23:07:24 +0100 Subject: mix statically typed with dynamically typed In-Reply-To: References: Message-ID: * Duncan Booth: > "Alf P. Steinbach" wrote: > >> I'm not sure I like your idea of introducing static typing to increase >> speed, but it could be done without introducing new syntax simply by >> defining a special meaning to such annotation expressions that are >> 'type' invocations, say, then like >> >> def speed( dist: type( float ), time: type( float ) ) -> type( >> float ) >> > > That would be particularly useless: > >>>> type(float) is type > True > > So your declaration is identical to: > > def speed(dist: type, time: type) -> type: That's the point. > Much better just to stick to something like: > > def speed( dist: float, time: float ) -> float: > > where at least you can tell from the annotations what types were actually > used. No, you do not want to redefine the meaning of existing (actually used) constructs. That would be particularly useless, to use your own words. :-) Cheers & hth., - Alf From peloko45 at gmail.com Thu Jan 28 17:16:52 2010 From: peloko45 at gmail.com (Joan Miller) Date: Thu, 28 Jan 2010 14:16:52 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> <9a28cf01-20af-4ce2-9b78-d8529800e4b9@y7g2000prc.googlegroups.com> Message-ID: On 28 ene, 21:40, Jonathan Gardner wrote: > On Jan 28, 10:20?am, Joan Miller wrote: > > > > > I've to call to many functions with the format: > > > >>> run("cmd") > > > were "cmd" is a command with its arguments to pass them to the shell > > and run it, i.e. > > > >>> ?run("pwd") > > or > > >>> run("ls /home") > > > Does anybody knows any library to help me to avoid the use of the main > > quotes, and brackets? > > > I would to use anything as: > > > $ ls /home => run("ls /home") > > > or, at least > > > run pwd => run("pwd") > > How about this? > > def pwd(): return run("pwd") > > pwd() > > def ls(l=False, files=()): > ? ? args = [] > ? ? if l: args.insert(0, '-l') > ? ? args.append(files) > ? ? return run("ls", args) > > ls(l=True, "/foo") There would be to make a function for each system command to use so it would be too inefficient, and follow the problem with the quotes. The best is make a parser into a compiled language From jjposner at optimum.net Thu Jan 28 17:57:09 2010 From: jjposner at optimum.net (John Posner) Date: Thu, 28 Jan 2010 17:57:09 -0500 Subject: Wrap a function In-Reply-To: <8508278f-0cf2-46c2-8313-2c640a71cdbd@q4g2000yqm.googlegroups.com> References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> <4B61EC7A.7020406@optimum.net> <64dc1630-6aa5-4983-b061-261457bf0153@g29g2000yqe.googlegroups.com> <8508278f-0cf2-46c2-8313-2c640a71cdbd@q4g2000yqm.googlegroups.com> Message-ID: <4B621645.1000906@optimum.net> On 1/28/2010 3:45 PM, Joan Miller wrote: > On 28 ene, 20:34, Joan Miller wrote: >> On 28 ene, 20:20, Peter wrote: >> >>> On Jan 29, 6:58 am, John Posner wrote: >> >>>> On 1/28/2010 2:24 PM, Joan Miller wrote: >> >>>>> On 28 ene, 19:16, Josh Holland wrote: >>>>>> On 2010-01-28, Joan Miller wrote: >> >>>>>>> I've to call to many functions with the format: >> >>>>>>>>>> run("cmd") >> >>>>>> Check the docs on os.system(). >>>>> No. I've a function that uses subprocess to run commands on the same >>>>> shell and so substitute to bash scrips. But a script full of run >>>>> ("shell_command --with --arguments") is too verbose. >> >>>> I'm suspicious of your original intent. Essentially, you want to write >>>> code in which a literal string, such as ... >> >>>> ls -l >> >>>> ... is *not* enclosed in quotes. Why run the risk of creating confusion >>>> (in other people who look at your code, in syntax-checking tools, etc.) >>>> between variables and literals? >> >>>> But I'm in sympathy with your desire to make the code as clean as >>>> possible and to minimize the number of times you have to type a quote >>>> character. My suggestions: >> >>>> 1. Create a function (say, "Run") that encapsulates as much of the >>>> syntax as possible: os.system(), subprocess.call(), string-splitting, >>>> whatever. So an invocation would look like this: >> >>>> Run("ls -l *.txt") >> >>>> (I think you've already done this step.) >> >>>> 2. Find a text editor that supports keyboard macros, so that a single >>>> keystroke turns this text line: >> >>>> ls -l *.txt >> >>>> ... into this one: >> >>>> Run("ls -l *.txt") >> >>>> HTH, >>>> John >> >>> I can't see you avoiding quotes etc, but extending on John's comment, >>> the obvious next step would be to run everything in a loop i.e. place >>> all the commands into a list and create a loop that ran each command >>> in the list. >> >> Yes, but could be necessary that were mixed with python code. >> >>> Almost all editors support macros - most editors support some form of >>> language sensitive editing (NOT the prompt call parameters style but >>> rather help with the syntax via a 'form' style of fill-in) that will >>> allow you to reduce typing effort. But macros would be the first and >>> easiest choice for this activity. >> >> The goal of my program is substitute to bash scripts, so the macros in >> editors are irrelevant fo this one. > > I think that the best solution that I've is to build a program that > parses the script to convert *$ command* to run("command") before of > be called by python. I believe you're working on Linux, so how about using "sed"? Here's a (prettified) BASH transcript of a sed script (edit.sed) transforming a 6-line text file (myprog.py). The text file has both Python statements and "special commands", which have "$ " at the beginning of the line. >>> cat myprog.py print "hello" $ ls -l r = range(10) $ grep foo bar.data pass print "bye" >>> cat edit.sed s/^\$ \(.*\)/Run("\1")/ >>> sed -f edit.sed data.txt print "hello" Run("ls -l") r = range(10) Run("grep foo bar.data") pass print "bye" -John From steve at REMOVE-THIS-cybersource.com.au Thu Jan 28 18:33:19 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 28 Jan 2010 23:33:19 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> <7xockfjcsv.fsf@ruckus.brouhaha.com> <6b89dc67-ceee-4b83-a11f-c7376185dcd1@b9g2000pri.googlegroups.com> <7xy6ji62vx.fsf@ruckus.brouhaha.com> Message-ID: <03720b25$0$1309$c3e8da3@news.astraweb.com> On Thu, 28 Jan 2010 13:20:02 -0500, Terry Reedy wrote: > On 1/28/2010 3:37 AM, Paul Rubin wrote: >> Jonathan Gardner writes: >>> If you're going to have statements, you're going to need the null >>> statement. That's "pass". >> >> Why? Expressions are statements, so you could just say "pass" (in >> quotes, denoting a string literal), or 0, or None, os anything else >> like that, instead of having a special statement. > > As Python is currently compiled, you are right, pass is not needed. A > string becomes the doc attribute, and becomes local var 0, but 0 is just > ignored. I actually expected a load_const but that is now optimized > away. I am not sure this was always true. Perhaps 'pass' is easier than > '0' for mewcomers reading the tutorial, but I have no data. As I said earlier, a dedicated statement `pass` indicates the intent of the author better than some arbitrary constant. if flag: 0 else: do_something() could be a mistake, perhaps the programmer intended to write "x = 0", but there is no ambiguity with the pass statement. I would hope that PyChecker and the equivalent would flag the use of bare constants in code and give a warning. In any case, while such a idiom works in code, it plays havoc with the interactive interpreter: >>> while 1: ... "pass" ... 'pass' 'pass' 'pass' 'pass' 'pass' Traceback (most recent call last): File "", line 2, in KeyboardInterrupt I have trimmed the output for convenience -- in reality, it fills my terminal's output buffer faster than I can hit ctrl-C. Using a constant as a proxy for the pass statement would get obnoxious really fast. -- Steven From morganhein at gmail.com Thu Jan 28 18:34:22 2010 From: morganhein at gmail.com (JohnnyFive) Date: Thu, 28 Jan 2010 15:34:22 -0800 (PST) Subject: Sleep timer but still responsive? Message-ID: I need help with something that is probably fairly simple, but i'm having a heck of a time getting it work. Basically, I need my program to sleep for a certain amount of time, but I don't want the console to become unresponsive while sleeping. As soon as the time is up, I want the main program to run it's course again. I tried using a Timer, threads, etc, but I really can't figure it out. What am I missing? I can post what I have, but I don't want to get caught up on how i'm doing it wrong (as none of it works), but rather the correct way to do it. Thanks in advance! From jgardner at jonathangardner.net Thu Jan 28 18:36:37 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Thu, 28 Jan 2010 15:36:37 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> <9a28cf01-20af-4ce2-9b78-d8529800e4b9@y7g2000prc.googlegroups.com> Message-ID: On Jan 28, 2:16?pm, Joan Miller wrote: > > There would be to make a function for each system command to use so it > would be too inefficient, and follow the problem with the quotes. > > The best is make a parser into a compiled language > Yeah, you could do that. Or you can simply rely on /bin/sh to do the parsing and everything else for you. No need to re-invent the wheel. I don't think Python will ever beat sh as a shell replacement. When people say that Python is great for some situations, but not so much for others, I think they thought of running commands like this as "other", From paul at boddie.org.uk Thu Jan 28 18:47:53 2010 From: paul at boddie.org.uk (Paul Boddie) Date: Thu, 28 Jan 2010 15:47:53 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> Message-ID: On 27 Jan, 13:26, Xah Lee wrote: > > So, for practical reasons, i think a ?key? parameter is fine. But > chopping off ?cmp? is damaging. When your data structure is complex, > its order is not embedded in some ?key?. Taking out ?cmp? makes it > impossible to sort your data structure. What would annoy me if I used Python 3.x would be the apparent lack of the __cmp__ method for conveniently defining comparisons between instances of my own classes. Having to define all the rich comparison methods frequently isn't even as much fun as it sounds. Paul From mensanator at aol.com Thu Jan 28 18:49:55 2010 From: mensanator at aol.com (Mensanator) Date: Thu, 28 Jan 2010 15:49:55 -0800 (PST) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On Jan 28, 11:35?am, Ethan Furman wrote: > Steven D'Aprano wrote: > > 4. Python 3 will make you irresistible to women. > > > ? ? FALSE > > What?!? ?Drat!!! ?Guess I'll have to learn Lisp... ?;) Irresisible? Ha! The chicks will think you have a harelip. > > ~Ethan~ From kerensaelise at hotmail.com Thu Jan 28 18:52:14 2010 From: kerensaelise at hotmail.com (elsa) Date: Thu, 28 Jan 2010 15:52:14 -0800 (PST) Subject: For loop searching takes too long! Message-ID: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> Hi guys, I've got a problem with my program, in that the code just takes too long to run. Here's what I'm doing. If anyone has any tips, they'd be much appreciated! So, say I have a list of lists that looks something like this (I'm using a list of lists, rather than a list of tuples, as I need it to be mutable): myList = [[786,0],[45, 1],[673,1],...................[23,46]] there are enough entries in the outer list, that the sum of myList[i] [0] across all i could be as high as 10^7. Now, what I need to do is randomly choose one myList[i], however the distribution of my random choice needs to be proportional to the values of myList[i][0]. So, for this list above, I'd have a much higher chance of choosing myList[0] than myList[1]. Here is how I'm doing it at the moment: def chooseI(myList): mySum=0 choice = random.choice(range(1,sum([i[0] for i in myList])+1)) for i in range(len(myList)): mySum+=myList[i][0] if mySum>=choice: return i break This works just fine if sum([i[0] for i in myList]) is less than 10,000, say. However if its up around 10^7, the whole thing crashes. Is there a faster way of doing this, that doesn't involve as many computational steps? Thanks! elsa From steve at REMOVE-THIS-cybersource.com.au Thu Jan 28 19:00:57 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 00:00:57 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <4b61c166$1@dnews.tpgi.com.au> Message-ID: <0372119f$0$1309$c3e8da3@news.astraweb.com> On Thu, 28 Jan 2010 18:37:56 +0100, Alf P. Steinbach wrote: > * Lie Ryan: >> On 01/28/10 20:12, Alf P. Steinbach wrote: >>> >>> import builtins >>> >>> >>> >>> org_print = print >>> >>> builtins.print = 666 >>> >>> >>> >>> print( "trallala" ) >>> Traceback (most recent call last): >>> File "", line 1, in >>> TypeError: 'int' object is not callable >>> >>> org_print( "but is that really so smart?" ) >>> but is that really so smart? >>> >>> _ >> >> Monkey patching follows (or should follow) the same rule as class >> inheritance overriding: the overrider's input domain must be a superset >> of the overriden's input domain and the overrider's output range must >> be a subset of the overriden's output range. 666 object (int) is not >> even remotely compatible with function object. > > Yes, that's my point. > > A 'print' replacement should ideally provide all the guarantees on > behavior that standard 'print' does. > > And with that it's not so easy to put in a /correct/ replacement of > 'print'; in particular, it has to honor the 'file' keyword argument. Oh come on, that's easy. This is Python we're talking about. Any object with a write() method is valid. >>> class MyIO: ... def __init__(self): ... self.buffer = [] ... def write(self, *args): ... self.buffer.extend(args) ... >>> io = MyIO() >>> print("Say goodnight Gracie", "Goodnight Gracie!", file=io) >>> io.buffer ['Say goodnight Gracie', ' ', 'Goodnight Gracie!', '\n'] In many cases, probably most, you won't be writing your own full-blown print replacement, but merely decorating the existing function. How easy is that? Trivial. Here's how to shut down a noisy module that prints too much: >>> def make_quieter(func): ... def inner(*args, **kwargs): ... out = kwargs.get('file') ... if out in (None, sys.stdout): ... pass ... else: ... func(*args, **kwargs) ... return inner ... >>> print = make_quieter(print) >>> print("spam spam spam") >>> print("spam spam spam", file=io) >>> io.buffer ['Say goodnight Gracie', ' ', 'Goodnight Gracie!', '\n', 'spam spam spam', '\n'] (except of course you would monkey-patch the module). Because print is a built-in, getting the original version back is even easier: >>> del print >>> print("spam spam spam") 'spam spam spam' > Thus the 3.x design makes it easy to replace 'print' incorrectly. This is Python -- the compiler won't stop you from shooting yourself in the foot. >>> len = 666 >>> len("hello world") Traceback (most recent call last): File "", line 1, in TypeError: 'int' object is not callable > I'd rather still had 'print' as keyword... ;-) We've shown many benefits of print as a function. Can you give any benefits of it being a statement, other than backward compatibility for those who learned it from Python 2.x? If you were designing your own language from scratch, so that backwards compatibility wasn't an issue, why would you make print a statement? -- Steven From ben+python at benfinney.id.au Thu Jan 28 19:01:21 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 29 Jan 2010 11:01:21 +1100 Subject: Sorting a list with entries of unequal types Message-ID: <87sk9pzsmm.fsf@benfinney.id.au> Howdy all, Python 2.6 tells me that, in Python 3, sorting a list with entries of incompatible types is not allowed: ===== $ python2.5 -c "foo = [1, True, 'green', 4, -27, 15.3]; foo.sort(); print foo;" [-27, 1, True, 4, 15.300000000000001, 'green'] $ python2.6 -c "foo = [1, True, 'green', 4, -27, 15.3]; foo.sort(); print foo;" [-27, 1, True, 4, 15.300000000000001, 'green'] $ python2.6 -3 -c "foo = [1, True, 'green', 4, -27, 15.3]; foo.sort(); print foo;" -c:1: DeprecationWarning: comparing unequal types not supported in 3.x [-27, 1, True, 4, 15.300000000000001, 'green'] ===== So how should I be sorting a list with entries of ?unequal types? such that it will work in Python 3? -- \ ?The idea that He would take his attention away from the | `\ universe in order to give me a bicycle with three speeds is | _o__) just so unlikely that I can't go along with it.? ?Quentin Crisp | Ben Finney From aahz at pythoncraft.com Thu Jan 28 19:18:43 2010 From: aahz at pythoncraft.com (Aahz) Date: 28 Jan 2010 16:18:43 -0800 Subject: py2exe deal with python command line inside a program References: Message-ID: In article , wrote: > >I need to create a python subprogress, like this: >myProcess = subprocess.Popen([sys.executable, 'C:\myscript.py'], > env=env, stdin=subprocess.PIPE, > stdout=subprocess.PIPE) You need to change your setup.py so that myscript.py is also an exe. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From steve at REMOVE-THIS-cybersource.com.au Thu Jan 28 19:27:10 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 00:27:10 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> <7xockfjcsv.fsf@ruckus.brouhaha.com> <6b89dc67-ceee-4b83-a11f-c7376185dcd1@b9g2000pri.googlegroups.com> Message-ID: <037217c3$0$1309$c3e8da3@news.astraweb.com> On Wed, 27 Jan 2010 23:50:55 -0800, Jonathan Gardner wrote: > I agree on "assert". I don't like running a program in test mode and > then running it in production mode with different code. I would rather > test what I am going to actually run. "assert" should be a function, and > support for removing assert statements should be eliminated. But then it wouldn't be an assert, it would be a test, and tests are best written explicitly. The whole point of assertions (outside of test code) is that they should ALWAYS pass. Since they should always pass, they're safe to optimize away, if the user explicitly runs Python with the "optimize" runtime switch. This is under the end-user's control, not the application writer. If you don't trust optimized code, don't use it. Assertions should be treated as "this will never happen" code, and are only there to catch coding errors and logic mistakes. If you expect that a test could fail, it shouldn't be written as an assert but as an explicit test. > I simply don't use assert statements at all. Outside of explicit test code (e.g. unit tests or equivalent), asserts should be rare. If you never use them at all, it probably means you're either not programming defensively enough, or you're needlessly writing explicit tests to cover situations that can't happen. assert exists to cover the middle ground between those two extremes. -- Steven From no.email at nospam.invalid Thu Jan 28 19:28:43 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 28 Jan 2010 16:28:43 -0800 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <4b61c166$1@dnews.tpgi.com.au> <0372119f$0$1309$c3e8da3@news.astraweb.com> Message-ID: <7xockdycsk.fsf@ruckus.brouhaha.com> Steven D'Aprano writes: > If you were designing your own language from scratch, so that backwards > compatibility wasn't an issue, why would you make print a statement? As another real estate analogy, my apartment has some problems with its plumbing, plus an ugly spot on the kitchen wall that could use a coat of paint to fix. The plumbing problem is fairly serious (they keep shutting off the water in the building when it acts up) but the kitchen spot is a minor annoyance that is mostly hidden by the refrigerator anyway. Fixing either of those things would be a hassle: I'd have to cover all my stuff with plastic and stay out of the apartment for a day or so while the maintenance guys hacked on things, and I'd expect some of my stuff to get damaged inadvertently no matter how careful everyone was. It's worth dealing with the repair hassles to get the plumbing fixed, and if I'm going to have to deal with that disruption anyway, then sure, I'd like them to paint the kitchen spot at the same time. But if they say they want me to cover up all my stuff and leave so they can JUST fix the kitchen spot, and then do the same thing a second time so they can fix the plumbing at some unspecified date in the future, then I'd rather just live with the kitchen spot the way it is. Yes it's a cosmetic blemish, but it's not causing any real problems, and I'd rather not deal with the hassle and risk of fixing it if there's no other benefit. In Python terms, the print statement is the spot on the wall, while the plumbing is something like the GIL and the legacy codebase that would break in a hundred ways if Python had real parallelism and a tracing garbage collector and a native-code compiler and the various language changes it would take to make all that stuff really fly rather than just limp along. If they are going to make everyone deal with the disruption of migrating to an incompatible version, they should do it once rather than twice. In short, there's a mythical Python 4 that only exists in my imagination, but it already interests me a lot more than Python 3. From no.email at nospam.invalid Thu Jan 28 19:32:49 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 28 Jan 2010 16:32:49 -0800 Subject: Sorting a list with entries of unequal types References: <87sk9pzsmm.fsf@benfinney.id.au> Message-ID: <7xk4v1yclq.fsf@ruckus.brouhaha.com> Ben Finney writes: > So how should I be sorting a list with entries of ?unequal types? such > that it will work in Python 3? Um, what ordering do you want? Basically you have to write a custom key function (they removed the option of writing a comparison function). Maybe something like foo.sort(key=lambda x: (type(x), x)) would be enough to fool the sorting function? I don't have python 3 so I haven't tried it. From __peter__ at web.de Thu Jan 28 19:35:52 2010 From: __peter__ at web.de (Peter Otten) Date: Fri, 29 Jan 2010 01:35:52 +0100 Subject: Sorting a list with entries of unequal types References: <87sk9pzsmm.fsf@benfinney.id.au> Message-ID: Ben Finney wrote: > Python 2.6 tells me that, in Python 3, sorting a list with entries of > incompatible types is not allowed: > $ python2.6 -3 -c "foo = [1, True, 'green', 4, -27, 15.3]; foo.sort(); > print foo;" -c:1: DeprecationWarning: comparing unequal types not > supported in 3.x > [-27, 1, True, 4, 15.300000000000001, 'green'] > ===== > > So how should I be sorting a list with entries of ?unequal types? such > that it will work in Python 3? I can't find the relevant part of the 2.6 documentation, but something like >>> def key(x): ... t = type(x) ... t = compat.get(t, t) ... return t.__name__, id(t), x ... >>> compat = {bool: float, int: float} >>> sorted([1, True, 'green', 4, -27, 15.3], key=key) [-27, 1, True, 4, 15.300000000000001, 'green'] should work. Peter From __peter__ at web.de Thu Jan 28 19:41:58 2010 From: __peter__ at web.de (Peter Otten) Date: Fri, 29 Jan 2010 01:41:58 +0100 Subject: For loop searching takes too long! References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> Message-ID: elsa wrote: > Hi guys, > > I've got a problem with my program, in that the code just takes too > long to run. Here's what I'm doing. If anyone has any tips, they'd be > much appreciated! > > So, say I have a list of lists that looks something like this (I'm > using a list of lists, rather than a list of tuples, as I need it to > be mutable): > > myList = [[786,0],[45, 1],[673,1],...................[23,46]] > > there are enough entries in the outer list, that the sum of myList[i] > [0] across all i could be as high as 10^7. > > Now, what I need to do is randomly choose one myList[i], however the > distribution of my random choice needs to be proportional to the > values of myList[i][0]. So, for this list above, I'd have a much > higher chance of choosing myList[0] than myList[1]. > > Here is how I'm doing it at the moment: > > def chooseI(myList): > mySum=0 > choice = random.choice(range(1,sum([i[0] for i in myList])+1)) > for i in range(len(myList)): > mySum+=myList[i][0] > if mySum>=choice: > return i > break > > This works just fine if sum([i[0] for i in myList]) is less than > 10,000, say. However if its up around 10^7, the whole thing crashes. > Is there a faster way of doing this, that doesn't involve as many > computational steps? Do the first items of the inner lists change often? If not you can put the running sum into them, i. e. myList = [[768, ...], [786+45, ...], [786+45+673, ...], ...] and use bisect.bisect() to choose the item. Peter From jjposner at optimum.net Thu Jan 28 19:50:32 2010 From: jjposner at optimum.net (John Posner) Date: Thu, 28 Jan 2010 19:50:32 -0500 Subject: For loop searching takes too long! In-Reply-To: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> Message-ID: <4B6230D8.30704@optimum.net> On 1/28/2010 6:52 PM, elsa wrote: > Hi guys, > > I've got a problem with my program, in that the code just takes too > long to run. Here's what I'm doing. If anyone has any tips, they'd be > much appreciated! > > So, say I have a list of lists that looks something like this (I'm > using a list of lists, rather than a list of tuples, as I need it to > be mutable): > > myList = [[786,0],[45, 1],[673,1],...................[23,46]] > > there are enough entries in the outer list, that the sum of myList[i] > [0] across all i could be as high as 10^7. > > Now, what I need to do is randomly choose one myList[i], however the > distribution of my random choice needs to be proportional to the > values of myList[i][0]. So, for this list above, I'd have a much > higher chance of choosing myList[0] than myList[1]. > > Here is how I'm doing it at the moment: > > def chooseI(myList): > mySum=0 > choice = random.choice(range(1,sum([i[0] for i in myList])+1)) > for i in range(len(myList)): > mySum+=myList[i][0] > if mySum>=choice: > return i > break > > This works just fine if sum([i[0] for i in myList]) is less than > 10,000, say. However if its up around 10^7, the whole thing crashes. > Is there a faster way of doing this, that doesn't involve as many > computational steps? > This way of generating *choice* might be less resource-intensive: list_total = 0 for x in myList: list_total += x[0] choice = random.randint(1, list_total) This avoids creating another list (the list comprehension) as big as myList. BTW, I don't think you need to *break* after you've already *return*ed. -John From gagsl-py2 at yahoo.com.ar Thu Jan 28 19:55:53 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Thu, 28 Jan 2010 21:55:53 -0300 Subject: Sleep timer but still responsive? References: Message-ID: En Thu, 28 Jan 2010 20:34:22 -0300, JohnnyFive escribi?: > I need help with something that is probably fairly simple, but i'm > having a heck of a time getting it work. > > Basically, I need my program to sleep for a certain amount of time, > but I don't want the console to become unresponsive while sleeping. Please provide more details. What do you want your program to do while sleeping? What kind of actions do you want a response to? Do you have a GUI? A curses-based interfase? -- Gabriel Genellina From python at mrabarnett.plus.com Thu Jan 28 19:56:55 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 29 Jan 2010 00:56:55 +0000 Subject: Sorting a list with entries of unequal types In-Reply-To: <87sk9pzsmm.fsf@benfinney.id.au> References: <87sk9pzsmm.fsf@benfinney.id.au> Message-ID: <4B623257.6030106@mrabarnett.plus.com> Ben Finney wrote: > Howdy all, > > Python 2.6 tells me that, in Python 3, sorting a list with entries of > incompatible types is not allowed: > > ===== > $ python2.5 -c "foo = [1, True, 'green', 4, -27, 15.3]; foo.sort(); print foo;" > [-27, 1, True, 4, 15.300000000000001, 'green'] > > $ python2.6 -c "foo = [1, True, 'green', 4, -27, 15.3]; foo.sort(); print foo;" > [-27, 1, True, 4, 15.300000000000001, 'green'] > > $ python2.6 -3 -c "foo = [1, True, 'green', 4, -27, 15.3]; foo.sort(); print foo;" > -c:1: DeprecationWarning: comparing unequal types not supported in 3.x > [-27, 1, True, 4, 15.300000000000001, 'green'] > ===== > > So how should I be sorting a list with entries of ?unequal types? such > that it will work in Python 3? > First you need to decide how they should be ordered, for example by name of class then value. Then make tuple keys for the values. Python compares tuple members one by one, only looking at the next members if the current members are equal, so if the first members are the class names (str) and the second members are the values (possibly unequal types), then Python will compare the values only if the types are the same. >>> sorted([1, True, 'green', 4, -27, 15.3], key=lambda value: (type(value).__name__, value)) [True, 15.3, -27, 1, 4, 'green'] This does mean that Boolean < float < int, but that's easy to fix: >>> def make_key(value): ... if isinstance(value, (int, float)): ... return "float", float(value) ... return type(value).__name__, value ... >>> sorted([1, True, 'green', 4, -27, 15.3], key=make_key) [-27, 1, True, 4, 15.3, 'green'] From g.bogle at auckland.no.spam.ac.nz Thu Jan 28 19:57:18 2010 From: g.bogle at auckland.no.spam.ac.nz (Gib Bogle) Date: Fri, 29 Jan 2010 13:57:18 +1300 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: Ethan Furman wrote: > Steven D'Aprano wrote: >> 4. Python 3 will make you irresistible to women. >> >> FALSE > > What?!? Drat!!! Guess I'll have to learn Lisp... ;) > > ~Ethan~ Learn to say this fast, you'll impress the hell out of them: Chaps with chapped lips lisp. From python at mrabarnett.plus.com Thu Jan 28 19:58:34 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 29 Jan 2010 00:58:34 +0000 Subject: Sorting a list with entries of unequal types In-Reply-To: <7xk4v1yclq.fsf@ruckus.brouhaha.com> References: <87sk9pzsmm.fsf@benfinney.id.au> <7xk4v1yclq.fsf@ruckus.brouhaha.com> Message-ID: <4B6232BA.1070608@mrabarnett.plus.com> Paul Rubin wrote: > Ben Finney writes: >> So how should I be sorting a list with entries of ?unequal types? such >> that it will work in Python 3? > > Um, what ordering do you want? Basically you have to write a custom key > function (they removed the option of writing a comparison function). > > Maybe something like > > foo.sort(key=lambda x: (type(x), x)) > > would be enough to fool the sorting function? I don't have python 3 > so I haven't tried it. You can't compare types: TypeError: unorderable types: type() < type() but you can compare their names. From steve at REMOVE-THIS-cybersource.com.au Thu Jan 28 20:00:52 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 01:00:52 GMT Subject: For loop searching takes too long! References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> Message-ID: <03721fab$0$1309$c3e8da3@news.astraweb.com> On Thu, 28 Jan 2010 15:52:14 -0800, elsa wrote: > Now, what I need to do is randomly choose one myList[i], however the > distribution of my random choice needs to be proportional to the values > of myList[i][0]. So, for this list above, I'd have a much higher chance > of choosing myList[0] than myList[1]. > > Here is how I'm doing it at the moment: > > def chooseI(myList): > mySum=0 > choice = random.choice(range(1,sum([i[0] for i in myList])+1)) > for i in range(len(myList)): > mySum+=myList[i][0] > if mySum>=choice: > return i > break This isn't related to your problem, but you don't need the break after the return -- the return will leave the loop and the function, and the break will never be reached. You could probably speed the code up a little by changing all the calls to range into xrange. range actually generates a list of integers, which is time consuming, while xrange is a lazy generator which just produces each integer one at a time. (You can ignore this if you are using Python 3.0 or 3.1.) Another small optimization you could use is to use a generator expression instead of a list comprehension inside the call to sum. That should generate the sum lazily, instead of calculating a giant list first and then adding it up. But I'd try something like this: # Untested. def chooseI(myList): total = sum(i[0] for i in myList) mySum = 0 choice = random.randrange(total) for i in myList: mySum += i[0] if mySum >= choice: return i -- Steven From steve at REMOVE-THIS-cybersource.com.au Thu Jan 28 20:11:05 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 01:11:05 GMT Subject: Sorting a list with entries of unequal types References: <87sk9pzsmm.fsf@benfinney.id.au> Message-ID: <03722210$0$1309$c3e8da3@news.astraweb.com> On Fri, 29 Jan 2010 11:01:21 +1100, Ben Finney wrote: > So how should I be sorting a list with entries of ?unequal types? such > that it will work in Python 3? That depends on how you want the items to be sorted. Python 2.x sorted unequal types in some arbitrary but consistent order. If that's all you want, then a key function like this will work in Python 3.1: >>> foo = [1, True, 'green', 4, -27, 15.3] >>> foo.sort() Traceback (most recent call last): File "", line 1, in TypeError: unorderable types: str() < bool() >>> foo.sort(key=lambda x: (str(type(x)), x)) >>> foo [True, 15.3, -27, 1, 4, 'green'] If you want to match the sort order from a specific version of Python, or if you want a specific ordering, you'll need to write your own key function. -- Steven From ben+python at benfinney.id.au Thu Jan 28 20:13:50 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 29 Jan 2010 12:13:50 +1100 Subject: Sorting a list with entries of unequal types References: <87sk9pzsmm.fsf@benfinney.id.au> Message-ID: <87fx5pzp9t.fsf@benfinney.id.au> Paul Rubin writes: > Ben Finney writes: > > So how should I be sorting a list with entries of ?unequal types? > > such that it will work in Python 3? > > Um, what ordering do you want? The same ordering I'd get in Python 2; that is, determined by the types of the elements. Peter Otten <__peter__ at web.de> writes: > I can't find the relevant part of the 2.6 documentation, but something like > > >>> def key(x): > ... t = type(x) > ... t = compat.get(t, t) > ... return t.__name__, id(t), x > ... > >>> compat = {bool: float, int: float} The problem with this approach is that it assumes I know all the types of elements that will be sorted; I don't. I want to sort a list coming from some other part of the code, and I don't want to arbitrarily limit the types of the list elements. Essentially, I want the same behaviour I'd get from Python 2's sort: the order of the elements is determined by the ordering built into the types of the elements themselves, with sensible defaults. -- \ ?I distrust those people who know so well what God wants them | `\ to do to their fellows, because it always coincides with their | _o__) own desires.? ?Susan Brownell Anthony, 1896 | Ben Finney From davea at ieee.org Thu Jan 28 20:15:29 2010 From: davea at ieee.org (Dave Angel) Date: Thu, 28 Jan 2010 20:15:29 -0500 Subject: For loop searching takes too long! In-Reply-To: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> Message-ID: <4B6236B1.1040208@ieee.org> elsa wrote: > Hi guys, > > I've got a problem with my program, in that the code just takes too > long to run. Here's what I'm doing. If anyone has any tips, they'd be > much appreciated! > > So, say I have a list of lists that looks something like this (I'm > using a list of lists, rather than a list of tuples, as I need it to > be mutable): > > myList = [[786,0],[45, 1],[673,1],...................[23,46]] > > there are enough entries in the outer list, that the sum of myList[i] > [0] across all i could be as high as 10^7. > > Now, what I need to do is randomly choose one myList[i], however the > distribution of my random choice needs to be proportional to the > values of myList[i][0]. So, for this list above, I'd have a much > higher chance of choosing myList[0] than myList[1]. > > Here is how I'm doing it at the moment: > > def chooseI(myList): > mySum=0 > choice = random.choice(range(1,sum([i[0] for i in myList])+1)) > for i in range(len(myList)): > mySum+=myList[i][0] > if mySum>=choice: > return i > break > > This works just fine if sum([i[0] for i in myList]) is less than > 10,000, say. However if its up around 10^7, the whole thing crashes. > Is there a faster way of doing this, that doesn't involve as many > computational steps? > > Thanks! > > elsa > > "the whole thing crashes" -- give us the specific crash you see, including the stack trace. "Crash" is far too vague. At first glance you could be running out of memory. But not with the values you're quoting. If a typical average value for i[0] is 50, that's only 200k elements. Anyway, you could print out the len(myList) to see how big the list is. There are quicker ways to do the sum, but the first thing to change is the random.choice() stuff. Once you have a sum, you can use random.randint() on it directly. No need to make a range list. Another approach might be to build a new list with a running sum in it. Then after doing the randint() on the last (highest) item, you can do a bisect.bisect on that list. The index that returns will be your return value. DaveA From steve at REMOVE-THIS-cybersource.com.au Thu Jan 28 20:20:03 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 01:20:03 GMT Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <87tyu529tl.fsf@benfinney.id.au> Message-ID: <03722429$0$1309$c3e8da3@news.astraweb.com> On Fri, 29 Jan 2010 08:33:58 +1100, Ben Finney wrote: > Antoine Pitrou writes: > >> Le Thu, 28 Jan 2010 00:19:24 +0000, Steven D'Aprano a ?crit?: >> > 4. Python 3 will make you irresistible to women. >> > >> > FALSE - Python 3 coders are no more likely to get a date than any >> > other programmer. >> >> They spend less time coding, so they /can/ get more "dates" (what a >> strange English word) :-) > > Perhaps Steven could tell you about a lovely Australian meaning for the > word ?date?. This is a family list, so perhaps I shouldn't. :) In Australia slang, "date" is short for "date hole", which is the part of the anatomy which is also known as "the [one] brown eye". In parts of the US, it is also known as the "corn hole", and in Cockney rhyming slang it is a jam role. I trust I don't need to be any more explicit... -- Steven From sccolbert at gmail.com Thu Jan 28 20:20:22 2010 From: sccolbert at gmail.com (Chris Colbert) Date: Thu, 28 Jan 2010 20:20:22 -0500 Subject: For loop searching takes too long! In-Reply-To: <03721fab$0$1309$c3e8da3@news.astraweb.com> References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> <03721fab$0$1309$c3e8da3@news.astraweb.com> Message-ID: <7f014ea61001281720l3d06273cwc0b1f7f9c500a445@mail.gmail.com> if you're open to other libraries, this could be done extremely fast in numpy. On my machine summing that whole array takes 75ms. On Thu, Jan 28, 2010 at 8:00 PM, Steven D'Aprano < steve at remove-this-cybersource.com.au> wrote: > On Thu, 28 Jan 2010 15:52:14 -0800, elsa wrote: > > > Now, what I need to do is randomly choose one myList[i], however the > > distribution of my random choice needs to be proportional to the values > > of myList[i][0]. So, for this list above, I'd have a much higher chance > > of choosing myList[0] than myList[1]. > > > > Here is how I'm doing it at the moment: > > > > def chooseI(myList): > > mySum=0 > > choice = random.choice(range(1,sum([i[0] for i in myList])+1)) > > for i in range(len(myList)): > > mySum+=myList[i][0] > > if mySum>=choice: > > return i > > break > > This isn't related to your problem, but you don't need the break after > the return -- the return will leave the loop and the function, and the > break will never be reached. > > You could probably speed the code up a little by changing all the calls > to range into xrange. range actually generates a list of integers, which > is time consuming, while xrange is a lazy generator which just produces > each integer one at a time. (You can ignore this if you are using Python > 3.0 or 3.1.) > > Another small optimization you could use is to use a generator expression > instead of a list comprehension inside the call to sum. That should > generate the sum lazily, instead of calculating a giant list first and > then adding it up. > > But I'd try something like this: > > # Untested. > def chooseI(myList): > total = sum(i[0] for i in myList) > mySum = 0 > choice = random.randrange(total) > for i in myList: > mySum += i[0] > if mySum >= choice: > return i > > > > -- > Steven > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at REMOVE-THIS-cybersource.com.au Thu Jan 28 20:24:33 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 01:24:33 GMT Subject: Sorting a list with entries of unequal types References: <87sk9pzsmm.fsf@benfinney.id.au> <87fx5pzp9t.fsf@benfinney.id.au> Message-ID: <03722537$0$1309$c3e8da3@news.astraweb.com> On Fri, 29 Jan 2010 12:13:50 +1100, Ben Finney wrote: > Paul Rubin writes: > >> Ben Finney writes: >> > So how should I be sorting a list with entries of ?unequal types? >> > such that it will work in Python 3? >> >> Um, what ordering do you want? > > The same ordering I'd get in Python 2; that is, determined by the types > of the elements. The ordering has not been consistent across minor versions of Python 2: $ python2.0 Python 2.0.1 (#1, Jan 14 2010, 15:43:17) [GCC 4.1.2 20070925 (Red Hat 4.1.2-27)] on linux2 Type "copyright", "credits" or "license" for more information. >>> L = [None, {}, [], 1, "x"] >>> L.sort() >>> L [1, None, {}, [], 'x'] $ python2.6 Python 2.6.1 (r261:67515, Dec 24 2008, 00:33:13) [GCC 4.1.2 20070502 (Red Hat 4.1.2-12)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> L = [None, {}, [], 1, "x"] >>> L.sort() >>> L [None, 1, {}, [], 'x'] Python has never guaranteed a particular ordering for unequal types, only that it would be consistent between repeated calls to sort. -- Steven From steve at REMOVE-THIS-cybersource.com.au Thu Jan 28 20:30:05 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 01:30:05 GMT Subject: Symbols as parameters? References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <4CD97E94-B40E-481D-B367-5FCE2820F48D@roalddevries.nl> Message-ID: <03722683$0$1309$c3e8da3@news.astraweb.com> On Thu, 28 Jan 2010 17:01:38 +0100, Roald de Vries wrote: > Question out of general interest in the language: If I would want to > generate such functions in a for-loop, what would I have to do? This > doesn't work: > > class Move(object): > def __call__(self, direction): > return direction > > move = Move() > > for f in ['up', 'down', 'right', 'left']: > move.__dict__[f] = lambda: move(f) > > ... because now 'move.up()' returns 'left' because thats the current > value of f. Is there a way to 'expand' f in the loop? Or a reason that > you never should use this? Possibly the simplest way is to use Python's handling of default values to get the result you want: for f in ['up', 'down', 'right', 'left']: move.__dict__[f] = lambda f=f: move(f) BTW, there's no need to explicitly reference move.__dict__: for f in ['up', 'down', 'right', 'left']: setattr(move, f, lambda f=f: move(f)) -- Steven From mdj.mdj at gmail.com Thu Jan 28 20:38:23 2010 From: mdj.mdj at gmail.com (mdj) Date: Thu, 28 Jan 2010 17:38:23 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> Message-ID: <184f0182-acbc-4a85-a8d9-7f813ebd2eb0@k2g2000pro.googlegroups.com> On Jan 29, 9:47?am, Paul Boddie wrote: > On 27 Jan, 13:26, Xah Lee wrote: > > > > > So, for practical reasons, i think a ?key? parameter is fine. But > > chopping off ?cmp? is damaging. When your data structure is complex, > > its order is not embedded in some ?key?. Taking out ?cmp? makes it > > impossible to sort your data structure. > > What would annoy me if I used Python 3.x would be the apparent lack of > the __cmp__ method for conveniently defining comparisons between > instances of my own classes. Having to define all the rich comparison > methods frequently isn't even as much fun as it sounds. OT, but you can always define the other operators in terms of a cmp and mix it in, restoring the original behaviour. Unfortunately it won't restore the original performance until someone comes to their senses and restores __cmp__ Matt From ben+python at benfinney.id.au Thu Jan 28 21:16:11 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 29 Jan 2010 13:16:11 +1100 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <87tyu529tl.fsf@benfinney.id.au> <03722429$0$1309$c3e8da3@news.astraweb.com> Message-ID: <87ockdy7tg.fsf@benfinney.id.au> Steven D'Aprano writes: > On Fri, 29 Jan 2010 08:33:58 +1100, Ben Finney wrote: > > Perhaps Steven could tell you about a lovely Australian meaning for > > the word ?date?. > > This is a family list, so perhaps I shouldn't. :) > > In Australia slang, "date" is short for "date hole", which is the part > of the anatomy which is also known as "the [one] brown eye". In parts > of the US, it is also known as the "corn hole", and in Cockney rhyming > slang it is a jam role. > > I trust I don't need to be any more explicit... I think the reason ?date? was initially used is because dates are most familiar to us as fleshy, dark brown, wrinkled, compressed points. My interests in etymology and scatology unite here. -- \ ?In the long run, the utility of all non-Free software | `\ approaches zero. All non-Free software is a dead end.? ?Mark | _o__) Pilgrim, 2006 | Ben Finney From steve at REMOVE-THIS-cybersource.com.au Thu Jan 28 21:21:28 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 02:21:28 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> <184f0182-acbc-4a85-a8d9-7f813ebd2eb0@k2g2000pro.googlegroups.com> Message-ID: <0372328c$0$1309$c3e8da3@news.astraweb.com> On Thu, 28 Jan 2010 17:38:23 -0800, mdj wrote: > On Jan 29, 9:47?am, Paul Boddie wrote: >> On 27 Jan, 13:26, Xah Lee wrote: >> >> >> >> > So, for practical reasons, i think a ?key? parameter is fine. But >> > chopping off ?cmp? is damaging. When your data structure is complex, >> > its order is not embedded in some ?key?. Taking out ?cmp? makes it >> > impossible to sort your data structure. >> >> What would annoy me if I used Python 3.x would be the apparent lack of >> the __cmp__ method for conveniently defining comparisons between >> instances of my own classes. Having to define all the rich comparison >> methods frequently isn't even as much fun as it sounds. > > OT, but you can always define the other operators in terms of a cmp and > mix it in, restoring the original behaviour. Unfortunately it won't > restore the original performance until someone comes to their senses and > restores __cmp__ "Comes to their senses"? There's nothing you can do with __cmp__ that you can't do better with rich comparisons, and plenty that rich comparisons can do that __cmp__ is utterly incapable of dealing with. __cmp__ is crippled since it can only be used for defining classes where the operators < etc return flags. It can't be used if you want to implement some other behaviour for the operators. E.g. here's a silly example: >>> class Silly(object): ... def __init__(self): ... self.link = None ... def __gt__(self, other): ... self.link = other ... >>> x = Silly() >>> x > Silly() >>> x.link <__main__.X object at 0xb7cda74c> More importantly, __cmp__ is only suitable for classes that implement total ordering. If you have a data type that does not have total ordering, for example sets, you can't implement it using __cmp__. E.g.: >>> s = set([1, 2, 3, 4]) >>> t = set([3, 4, 5, 6]) >>> s < t False >>> s > t False >>> s == t False >>> cmp(s, t) Traceback (most recent call last): File "", line 1, in TypeError: cannot compare sets using cmp() Sets have partial ordering, and __cmp__ is simply not up to the job of dealing with it. Having two mechanisms for implementing comparisons is unnecessary. It adds complications to the language that we are better off without. The only advantage of the obsolete __cmp__ is that lazy programmers only need to write one method instead of six. This is an advantage, I accept that (hey, I'm a lazy programmer too, that's why I use Python!) but it's not a big advantage. If you really care about it you can create a mixin class, a decorator, or a metaclass to simplify creation of the methods. For example, a quick and dirty decorator: >>> def make_comparisons(cls): ... cls.__gt__ = lambda self, other: self.__cmp__(other) == 1 ... cls.__ge__ = lambda self, other: self.__cmp__(other) >= 0 ... cls.__eq__ = lambda self, other: self.__cmp__(other) == 0 ... cls.__ne__ = lambda self, other: self.__cmp__(other) != 0 ... cls.__le__ = lambda self, other: self.__cmp__(other) <= 0 ... cls.__lt__ = lambda self, other: self.__cmp__(other) == -1 ... return cls ... >>> @make_comparisons ... class BiggerThanEverything(object): ... def __cmp__(self, other): ... return 1 ... >>> x = BiggerThanEverything() >>> x > 1000 True >>> x < 0 False -- Steven From steve at holdenweb.com Thu Jan 28 21:22:38 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 28 Jan 2010 21:22:38 -0500 Subject: For loop searching takes too long! In-Reply-To: <4B6236B1.1040208@ieee.org> References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> <4B6236B1.1040208@ieee.org> Message-ID: Dave Angel wrote: > elsa wrote: >> Hi guys, >> >> I've got a problem with my program, in that the code just takes too >> long to run. Here's what I'm doing. If anyone has any tips, they'd be >> much appreciated! >> >> So, say I have a list of lists that looks something like this (I'm >> using a list of lists, rather than a list of tuples, as I need it to >> be mutable): >> >> myList = [[786,0],[45, 1],[673,1],...................[23,46]] >> >> there are enough entries in the outer list, that the sum of myList[i] >> [0] across all i could be as high as 10^7. >> >> Now, what I need to do is randomly choose one myList[i], however the >> distribution of my random choice needs to be proportional to the >> values of myList[i][0]. So, for this list above, I'd have a much >> higher chance of choosing myList[0] than myList[1]. >> >> Here is how I'm doing it at the moment: >> >> def chooseI(myList): >> mySum=0 >> choice = random.choice(range(1,sum([i[0] for i in myList])+1)) >> for i in range(len(myList)): >> mySum+=myList[i][0] >> if mySum>=choice: >> return i >> break >> >> This works just fine if sum([i[0] for i in myList]) is less than >> 10,000, say. However if its up around 10^7, the whole thing crashes. >> Is there a faster way of doing this, that doesn't involve as many >> computational steps? >> >> Thanks! >> >> elsa >> >> > "the whole thing crashes" -- give us the specific crash you see, > including the stack trace. "Crash" is far too vague. > > At first glance you could be running out of memory. But not with the > values you're quoting. If a typical average value for i[0] is 50, > that's only 200k elements. Anyway, you could print out the len(myList) > to see how big the list is. > > There are quicker ways to do the sum, but the first thing to change is > the random.choice() stuff. Once you have a sum, you can use > random.randint() on it directly. No need to make a range list. > > Another approach might be to build a new list with a running sum in it. > Then after doing the randint() on the last (highest) item, you can do a > bisect.bisect on that list. The index that returns will be your return > value. > Your approach seems to validate an informal impression I had that with N choices one ought to be able to build a binary tree where each decision went to left or right according to whether a number drawn from a linear [0,1) distribution was greater that some arbitrary margin, or not. There are then arithmetical questions of exactly how to arrive at the correct threshold values for each bifurcation. If the most probable paths are deliberately placed at the top of the tree then the most frequent cases will be fastest to generate, also. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Thu Jan 28 21:26:04 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 28 Jan 2010 21:26:04 -0500 Subject: myths about python 3 In-Reply-To: <03722429$0$1309$c3e8da3@news.astraweb.com> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <87tyu529tl.fsf@benfinney.id.au> <03722429$0$1309$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > On Fri, 29 Jan 2010 08:33:58 +1100, Ben Finney wrote: > >> Antoine Pitrou writes: >> >>> Le Thu, 28 Jan 2010 00:19:24 +0000, Steven D'Aprano a ?crit : >>>> 4. Python 3 will make you irresistible to women. >>>> >>>> FALSE - Python 3 coders are no more likely to get a date than any >>>> other programmer. >>> They spend less time coding, so they /can/ get more "dates" (what a >>> strange English word) :-) >> Perhaps Steven could tell you about a lovely Australian meaning for the >> word ?date?. > > This is a family list, so perhaps I shouldn't. :) > > In Australia slang, "date" is short for "date hole", which is the part of > the anatomy which is also known as "the [one] brown eye". In parts of the > US, it is also known as the "corn hole", and in Cockney rhyming slang it > is a jam role. > > I trust I don't need to be any more explicit... > > > My God, and I just nominated you for membership of the PSF. Trust an Australian to descend to normally unplumbed depths (pun intended) the very second you stake your reputation on them. I guess that means I know my place ... there-goes-my-american-citizenship-ly y'rs - steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From roy at panix.com Thu Jan 28 21:26:43 2010 From: roy at panix.com (Roy Smith) Date: Thu, 28 Jan 2010 21:26:43 -0500 Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> <7xockfjcsv.fsf@ruckus.brouhaha.com> <6b89dc67-ceee-4b83-a11f-c7376185dcd1@b9g2000pri.googlegroups.com> <7xy6ji62vx.fsf@ruckus.brouhaha.com> <03720b25$0$1309$c3e8da3@news.astraweb.com> Message-ID: In article <03720b25$0$1309$c3e8da3 at news.astraweb.com>, Steven D'Aprano wrote: > In any case, while such a idiom works in code, it plays havoc with the > interactive interpreter: > > >>> while 1: > ... "pass" > ... > 'pass' > 'pass' > 'pass' > 'pass' > 'pass' It's not particularly useful without the quotes either :-) From roy at panix.com Thu Jan 28 21:30:45 2010 From: roy at panix.com (Roy Smith) Date: Thu, 28 Jan 2010 21:30:45 -0500 Subject: python 3's adoption References: Message-ID: In article , "Alf P. Steinbach" wrote: > * Roy Smith: > > In article , > > Mitchell L Model wrote: > > > >> I use the sep and end keywords all the time. > > > > What are 'sep' and 'end'? I'm looking in > > http://docs.python.org/3.1/genindex-all.html and don't see those mentioned > > at all. Am I just looking in the wrong place? > > >>> print( print.__doc__ ) > print(value, ..., sep=' ', end='\n', file=sys.stdout) > > Prints the values to a stream, or to sys.stdout by default. > Optional keyword arguments: > file: a file-like object (stream); defaults to the current sys.stdout. > sep: string inserted between values, default a space. > end: string appended after the last value, default a newline. > >>> _ I'm inclined to call it a docs bug that these keywords are not in the HTML index. Yes? From wuwei23 at gmail.com Thu Jan 28 21:38:20 2010 From: wuwei23 at gmail.com (alex23) Date: Thu, 28 Jan 2010 18:38:20 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> Message-ID: <8a7053f7-43ad-4766-a106-ef1b94826028@k2g2000pro.googlegroups.com> Joan Miller wrote: > Does anybody knows any library to help me to avoid the use of the main > quotes, and brackets? > I would to use anything as: > $ ls /home => run("ls /home") It's not a library, but IPython[1] provides a lot of what you're after: IPython 0.9.1 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object'. ?object also works, ?? prints more. In [1]: ls /home bb/ dcallan/ ehornsby/ ftp/ uqamartl/ uqckorte/ uqmtrev2/ In [2]: path = '/home' In [3]: ls $path bb/ dcallan/ ehornsby/ ftp/ uqamartl/ uqckorte/ uqmtrev2/ In [4]: output = !ls $path In [5]: output Out[5]: SList (.p, .n, .l, .s, .grep(), .fields(), sort() available): 0: bb 1: dcallan 2: ehornsby 3: ftp 4: uqamartl 5: uqckorte 6: uqmtrev2 In [6]: output[6] Out[6]: 'uqmtrev2' You'll need to run your scripts with IPython, so this may not be a solution if you plan on distributing them. [1] http://ipython.scipy.org/ From wuwei23 at gmail.com Thu Jan 28 21:56:11 2010 From: wuwei23 at gmail.com (alex23) Date: Thu, 28 Jan 2010 18:56:11 -0800 (PST) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <284dc9bc-6e7f-47d2-9982-a09dc3cb9dc6@b9g2000pri.googlegroups.com> Terry Reedy wrote: > This statement was to counter the 'myth' that US was only targeted at > 2.x when the current situation is quite the opposite. Not so much 'myth' as 'outdated information', they were very clear that 2.x was the initial target. > In particular, several people said that the speed/space traceoff > should be optional, and that compilation 'without llvm' should really > be without, not just with llvm present but disabled. Instead of arguing, > Colin went ahead and patched the build process to make it be this way. Ah, that's excellent. The impression being given off is that it's a total replacement. > I have no idea. It will have to improve its speedup more before > adoption. I will not be surprised if that happens. It's not so much about being surprised or not, it's wanting actual evidence and not just claims, and moreso _extensive real world usage_ before it's integrated. This seems far more intimate a change than adding a module to the stdlib, I expect it to have at _least_ the evaluation time & vague consensus of approval expected of those. > US is not a new or separate interpreter. It will be an optional jit > replacement for one component of CPython, the eval loop. All the code > for builting functions, types, and modules will be untouched, as will > their big O performance characteristics. As long as there aren't any related decreases in performance in other areas, I'll be happy. > If you can still have a binary free of the traceoff, why would you care? Well, I didn't know I could, so now I don't quite as much :) > They claim they have pretty well fixed that. They know that complete > Windows support, including 64 bit versions, is a necessity. Maybe I'll be a lot more convinced after the Q4 report. The 'incomplete' Windows support may not be as big an issue as I thought, it seems to refer to a lack of support for older Windows versions rather than an incomplete implementation on the supported ones. Cheers, Terry, this addressed a lot of my concerns, although I'm still keen to see more facts & real-world usage over custom-crafted benchmarks and enthusiastic claims. From steve at holdenweb.com Thu Jan 28 22:02:01 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 28 Jan 2010 22:02:01 -0500 Subject: For loop searching takes too long! In-Reply-To: References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> <4B6236B1.1040208@ieee.org> Message-ID: Steve Holden wrote: > Dave Angel wrote: >> elsa wrote: >>> Hi guys, >>> >>> I've got a problem with my program, in that the code just takes too >>> long to run. Here's what I'm doing. If anyone has any tips, they'd be >>> much appreciated! >>> >>> So, say I have a list of lists that looks something like this (I'm >>> using a list of lists, rather than a list of tuples, as I need it to >>> be mutable): >>> >>> myList = [[786,0],[45, 1],[673,1],...................[23,46]] >>> >>> there are enough entries in the outer list, that the sum of myList[i] >>> [0] across all i could be as high as 10^7. >>> >>> Now, what I need to do is randomly choose one myList[i], however the >>> distribution of my random choice needs to be proportional to the >>> values of myList[i][0]. So, for this list above, I'd have a much >>> higher chance of choosing myList[0] than myList[1]. >>> >>> Here is how I'm doing it at the moment: >>> >>> def chooseI(myList): >>> mySum=0 >>> choice = random.choice(range(1,sum([i[0] for i in myList])+1)) >>> for i in range(len(myList)): >>> mySum+=myList[i][0] >>> if mySum>=choice: >>> return i >>> break >>> >>> This works just fine if sum([i[0] for i in myList]) is less than >>> 10,000, say. However if its up around 10^7, the whole thing crashes. >>> Is there a faster way of doing this, that doesn't involve as many >>> computational steps? >>> >>> Thanks! >>> >>> elsa >>> >>> >> "the whole thing crashes" -- give us the specific crash you see, >> including the stack trace. "Crash" is far too vague. >> >> At first glance you could be running out of memory. But not with the >> values you're quoting. If a typical average value for i[0] is 50, >> that's only 200k elements. Anyway, you could print out the len(myList) >> to see how big the list is. >> >> There are quicker ways to do the sum, but the first thing to change is >> the random.choice() stuff. Once you have a sum, you can use >> random.randint() on it directly. No need to make a range list. >> >> Another approach might be to build a new list with a running sum in it. >> Then after doing the randint() on the last (highest) item, you can do a >> bisect.bisect on that list. The index that returns will be your return >> value. >> > Your approach seems to validate an informal impression I had that with > N choices one ought to be able to build a binary tree where each > decision went to left or right according to whether a number drawn from > a linear [0,1) distribution was greater that some arbitrary margin, or not. > > There are then arithmetical questions of exactly how to arrive at the > correct threshold values for each bifurcation. If the most probable > paths are deliberately placed at the top of the tree then the most > frequent cases will be fastest to generate, also. > Bad form, I know, to reply to oneself, but since I was egotistical enough to read what I wrote again when it was published I must also record the conjecture that the resulting alphabet, expressed as the binary history of the bifurcations chosen for any given symbol, must surely be a Hamming code to be most efficient. No proof is adduced. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Thu Jan 28 22:03:00 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 28 Jan 2010 22:03:00 -0500 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> <7xockfjcsv.fsf@ruckus.brouhaha.com> <6b89dc67-ceee-4b83-a11f-c7376185dcd1@b9g2000pri.googlegroups.com> <7xy6ji62vx.fsf@ruckus.brouhaha.com> <03720b25$0$1309$c3e8da3@news.astraweb.com> Message-ID: Roy Smith wrote: > In article <03720b25$0$1309$c3e8da3 at news.astraweb.com>, > Steven D'Aprano wrote: > >> In any case, while such a idiom works in code, it plays havoc with the >> interactive interpreter: >> >>>>> while 1: >> ... "pass" >> ... >> 'pass' >> 'pass' >> 'pass' >> 'pass' >> 'pass' > > It's not particularly useful without the quotes either :-) Perhaps not. but at least it keeps its misery to itself. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From davea at ieee.org Thu Jan 28 22:09:06 2010 From: davea at ieee.org (Dave Angel) Date: Thu, 28 Jan 2010 22:09:06 -0500 Subject: For loop searching takes too long! In-Reply-To: References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> <4B6236B1.1040208@ieee.org> Message-ID: <4B625152.7050403@ieee.org> Steve Holden wrote: > Dave Angel wrote: > >> elsa wrote: >> >>> Hi guys, >>> >>> I've got a problem with my program, in that the code just takes too >>> long to run. Here's what I'm doing. If anyone has any tips, they'd be >>> much appreciated! >>> >>> So, say I have a list of lists that looks something like this (I'm >>> using a list of lists, rather than a list of tuples, as I need it to >>> be mutable): >>> >>> myList = [[786,0],[45, 1],[673,1],...................[23,46]] >>> >>> there are enough entries in the outer list, that the sum of myList[i] >>> [0] across all i could be as high as 10^7. >>> >>> Now, what I need to do is randomly choose one myList[i], however the >>> distribution of my random choice needs to be proportional to the >>> values of myList[i][0]. So, for this list above, I'd have a much >>> higher chance of choosing myList[0] than myList[1]. >>> >>> Here is how I'm doing it at the moment: >>> >>> def chooseI(myList): >>> mySum=0 >>> choice = random.choice(range(1,sum([i[0] for i in myList])+1)) >>> for i in range(len(myList)): >>> mySum+=myList[i][0] >>> if mySum>=choice: >>> return i >>> break >>> >>> This works just fine if sum([i[0] for i in myList]) is less than >>> 10,000, say. However if its up around 10^7, the whole thing crashes. >>> Is there a faster way of doing this, that doesn't involve as many >>> computational steps? >>> >>> Thanks! >>> >>> elsa >>> >>> >>> >> "the whole thing crashes" -- give us the specific crash you see, >> including the stack trace. "Crash" is far too vague. >> >> At first glance you could be running out of memory. But not with the >> values you're quoting. If a typical average value for i[0] is 50, >> that's only 200k elements. Anyway, you could print out the len(myList) >> to see how big the list is. >> >> There are quicker ways to do the sum, but the first thing to change is >> the random.choice() stuff. Once you have a sum, you can use >> random.randint() on it directly. No need to make a range list. >> >> Another approach might be to build a new list with a running sum in it. >> Then after doing the randint() on the last (highest) item, you can do a >> bisect.bisect on that list. The index that returns will be your return >> value. >> >> > Your approach seems to validate an informal impression I had that with > N choices one ought to be able to build a binary tree where each > decision went to left or right according to whether a number drawn from > a linear [0,1) distribution was greater that some arbitrary margin, or not. > > There are then arithmetical questions of exactly how to arrive at the > correct threshold values for each bifurcation. If the most probable > paths are deliberately placed at the top of the tree then the most > frequent cases will be fastest to generate, also. > > regards > If the list will remain the same, it may be worth pre-computing the running sum into it, and then the function to get a random value just has to do the bisect. But as you say, if the values are not evenly distributed, a binary split (which is what bisect does) might not give the best average time. I doubt if it would help for small lists (Python code would be slower than the presumably C code in bisect), but perhaps for larger lists, an index something like this would be better: Assuming the list had 40 items, and the sum was 1000 for the sake of example. you'd build a tree with 500 as the top node. that would "point" not to 20, but to whatever index had a value that was closest to 500. Then you'd build child nodes that similarly split each portion. And you'd repeat it till the remaining size was fairly small, at which point you'd use standard bisect logic. On a different tack, someone could probably improve on the bisect() function, for cases where the list is a sorted list of numbers. Typically a binary algorithm is used because a comparison function gives either less than, equal, or greater. But if you know they are all well-behaved numbers, you could do a formula to narrow down the choices faster. Let's say that at some point you know the value is between items[5] and items[105], and the values respectively are 40 and 140. You're searching for a value of 50. Instead of trying items[55], you do a scale factor, and do 5 + ( 105-5 ) * (50-40) / (140-40). So you'd try element 15 next. All these values are plus/minus 1, but I think I've come close to an algorithm. Kind of an analogy to Newton's method. DaveA From steve at holdenweb.com Thu Jan 28 22:15:53 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 28 Jan 2010 22:15:53 -0500 Subject: python 3's adoption In-Reply-To: <0372328c$0$1309$c3e8da3@news.astraweb.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> <184f0182-acbc-4a85-a8d9-7f813ebd2eb0@k2g2000pro.googlegroups.com> <0372328c$0$1309$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > On Thu, 28 Jan 2010 17:38:23 -0800, mdj wrote: > >> On Jan 29, 9:47 am, Paul Boddie wrote: >>> On 27 Jan, 13:26, Xah Lee wrote: >>> >>> >>> >>>> So, for practical reasons, i think a ?key? parameter is fine. But >>>> chopping off ?cmp? is damaging. When your data structure is complex, >>>> its order is not embedded in some ?key?. Taking out ?cmp? makes it >>>> impossible to sort your data structure. >>> What would annoy me if I used Python 3.x would be the apparent lack of >>> the __cmp__ method for conveniently defining comparisons between >>> instances of my own classes. Having to define all the rich comparison >>> methods frequently isn't even as much fun as it sounds. >> OT, but you can always define the other operators in terms of a cmp and >> mix it in, restoring the original behaviour. Unfortunately it won't >> restore the original performance until someone comes to their senses and >> restores __cmp__ > > "Comes to their senses"? > > There's nothing you can do with __cmp__ that you can't do better with > rich comparisons, and plenty that rich comparisons can do that __cmp__ is > utterly incapable of dealing with. __cmp__ is crippled since it can only > be used for defining classes where the operators < etc return flags. It > can't be used if you want to implement some other behaviour for the > operators. E.g. here's a silly example: > >>>> class Silly(object): > ... def __init__(self): > ... self.link = None > ... def __gt__(self, other): > ... self.link = other > ... >>>> x = Silly() >>>> x > Silly() >>>> x.link > <__main__.X object at 0xb7cda74c> > > > More importantly, __cmp__ is only suitable for classes that implement > total ordering. If you have a data type that does not have total > ordering, for example sets, you can't implement it using __cmp__. > > E.g.: > >>>> s = set([1, 2, 3, 4]) >>>> t = set([3, 4, 5, 6]) >>>> s < t > False >>>> s > t > False >>>> s == t > False >>>> cmp(s, t) > Traceback (most recent call last): > File "", line 1, in > TypeError: cannot compare sets using cmp() > > > Sets have partial ordering, and __cmp__ is simply not up to the job of > dealing with it. > > Having two mechanisms for implementing comparisons is unnecessary. It > adds complications to the language that we are better off without. The > only advantage of the obsolete __cmp__ is that lazy programmers only need > to write one method instead of six. This is an advantage, I accept that > (hey, I'm a lazy programmer too, that's why I use Python!) but it's not a > big advantage. If you really care about it you can create a mixin class, > a decorator, or a metaclass to simplify creation of the methods. For > example, a quick and dirty decorator: > > >>>> def make_comparisons(cls): > ... cls.__gt__ = lambda self, other: self.__cmp__(other) == 1 > ... cls.__ge__ = lambda self, other: self.__cmp__(other) >= 0 > ... cls.__eq__ = lambda self, other: self.__cmp__(other) == 0 > ... cls.__ne__ = lambda self, other: self.__cmp__(other) != 0 > ... cls.__le__ = lambda self, other: self.__cmp__(other) <= 0 > ... cls.__lt__ = lambda self, other: self.__cmp__(other) == -1 > ... return cls > ... >>>> @make_comparisons > ... class BiggerThanEverything(object): > ... def __cmp__(self, other): > ... return 1 > ... >>>> x = BiggerThanEverything() >>>> x > 1000 > True >>>> x < 0 > False > > While I am fully aware that premature optimization, etc., but I cannot resist an appeal to efficiency if it finally kills off this idea that "they took 'cmp()' away" is a bad thing. Passing a cmp= argument to sort provides the interpreter with a function that will be called each time any pair of items have to be compared. The key= argument, however, specifies a transformation from [x0, x1, ..., xN] to [(key(x0), x0), (key(x1), x1), ..., (key(xN), xN)] (which calls the key function precisely once per sortable item). >From a C routine like sort() [in CPython, anyway] calling out from C to a Python function to make a low-level decision like "is A less than B?" turns out to be disastrous for execution efficiency (unlike the built-in default comparison, which can be called directly from C in CPython). If your data structures have a few hundred items in them it isn't going to make a huge difference. If they have a few million thenit is already starting to affect performance ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From mdj.mdj at gmail.com Thu Jan 28 23:38:22 2010 From: mdj.mdj at gmail.com (mdj) Date: Thu, 28 Jan 2010 20:38:22 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> <184f0182-acbc-4a85-a8d9-7f813ebd2eb0@k2g2000pro.googlegroups.com> <0372328c$0$1309$c3e8da3@news.astraweb.com> Message-ID: On Jan 29, 12:21?pm, Steven D'Aprano wrote: > "Comes to their senses"? > > There's nothing you can do with __cmp__ that you can't do better with > rich comparisons, and plenty that rich comparisons can do that __cmp__ is > utterly incapable of dealing with. __cmp__ is crippled since it can only > be used for defining classes where the operators < etc return flags. It > can't be used if you want to implement some other behaviour for the > operators. E.g. here's a silly example: [example snipped] Okay, but I think that overloading to change the semantic meaning of operators is another issue, and one that there's a great deal of difference of opinion on. Personally, I find arbitrarily redefining operators with well understood semantics to mean something totally different to be poor style, and those who disagree would find my style poor :-) > Having two mechanisms for implementing comparisons is unnecessary. It > adds complications to the language that we are better off without. The > only advantage of the obsolete __cmp__ is that lazy programmers only need > to write one method instead of six. This is an advantage, I accept that > (hey, I'm a lazy programmer too, that's why I use Python!) but it's not a > big advantage. If you really care about it you can create a mixin class, > a decorator, or a metaclass to simplify creation of the methods. For > example, a quick and dirty decorator: I agree it's unnecessary, but deleting a time tested idiom in the name of consistency seems excessive to me. Is there not a compromise? For those of us who prefer limiting operator overloads to semantically compatible cases the extra flexibility is seldom necessary, and the price extracted in terms of inconvenience is therefore quite high. You can (as I mentioned and you demonstrated) work around the problem, but I'd hate to see 50 different implementations of a common idiom wasting brain space in future projects. Matt (Note I remove the crosspost to comp.lang.lisp where I saw this - one of the reasons I love lisp is that discussions like this become totally unnecessary) From daniel at stutzbachenterprises.com Thu Jan 28 23:51:16 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Thu, 28 Jan 2010 22:51:16 -0600 Subject: For loop searching takes too long! In-Reply-To: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> Message-ID: On Thu, Jan 28, 2010 at 5:52 PM, elsa wrote: > choice = random.choice(range(1,sum([i[0] for i in myList])+1)) > That's the line causing the problem. It's generating all of the numbers between 1 and your sum, then picking one. Try the following instead, which will pick a number between 1 and your sum without generating each and every one. choice = random.randrange(1, sum(i[0] for i in myList)+1) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From timr at probo.com Fri Jan 29 00:21:05 2010 From: timr at probo.com (Tim Roberts) Date: Thu, 28 Jan 2010 21:21:05 -0800 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: John Nagle wrote: > >Arguably, Python 3 has been rejected by the market. Instead, there's >now Python 2.6, Python 2.7, and Python 2.8. Python 3 has turned into >a debacle like Perl 6, now 10 years old. Although I happen to be one of the folks who are reluctant to switch to Python 3, I have to say that this comparison is entirely unfair. Python 3 exists in the wild. It has been released, and has even had a couple of updates. Eventually, it will prevail. Resistance is futile, you WILL be assimilated. Perl 6, on the other hand, is still fantasyware a decade after its announcement. It is, for the most part, THE canonical example of the wrong way to conduct a development effort. -- Tim Roberts, timr at probo.com Providenza & Boekelheide, Inc. From wuwei23 at gmail.com Fri Jan 29 00:33:01 2010 From: wuwei23 at gmail.com (alex23) Date: Thu, 28 Jan 2010 21:33:01 -0800 (PST) Subject: python 3's adoption References: Message-ID: Roy Smith wrote: > I'm inclined to call it a docs bug that these keywords are not in the HTML > index. ?Yes? No. The indices don't list the named parameters for any other built- in, so why would they for print()? From steve at REMOVE-THIS-cybersource.com.au Fri Jan 29 00:34:01 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 05:34:01 GMT Subject: [OT] Perl 6 [was Re: myths about python 3] References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <03725fae$0$1309$c3e8da3@news.astraweb.com> On Thu, 28 Jan 2010 21:21:05 -0800, Tim Roberts wrote: > Perl 6, on the other hand, is still fantasyware a decade after its > announcement. It is, for the most part, THE canonical example of the > wrong way to conduct a development effort. Out of curiosity, and completely off-topic, why has Perl 6 gone so badly? -- Steven From steve at REMOVE-THIS-cybersource.com.au Fri Jan 29 00:34:13 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 05:34:13 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <7xr5pcyskw.fsf@ruckus.brouhaha.com> <7x636oc82r.fsf@ruckus.brouhaha.com> <7x636ovt97.fsf@ruckus.brouhaha.com> <7xockfjcsv.fsf@ruckus.brouhaha.com> <6b89dc67-ceee-4b83-a11f-c7376185dcd1@b9g2000pri.googlegroups.com> <7xy6ji62vx.fsf@ruckus.brouhaha.com> <03720b25$0$1309$c3e8da3@news.astraweb.com> Message-ID: <03725fba$0$1309$c3e8da3@news.astraweb.com> On Thu, 28 Jan 2010 21:26:43 -0500, Roy Smith wrote: > In article <03720b25$0$1309$c3e8da3 at news.astraweb.com>, > Steven D'Aprano wrote: > >> In any case, while such a idiom works in code, it plays havoc with the >> interactive interpreter: >> >> >>> while 1: >> ... "pass" >> ... >> 'pass' >> 'pass' >> 'pass' >> 'pass' >> 'pass' > > It's not particularly useful without the quotes either :-) Of course "while 1: pass" is useless, but it doesn't fill the terminal with thousands of lines of "pass". The point I was making is that the statement pass does nothing, while in the interactive interpreter a string or other object prints to the screen. -- Steven From raghavendra.gv.vanam at gmail.com Fri Jan 29 00:36:27 2010 From: raghavendra.gv.vanam at gmail.com (vanam) Date: Thu, 28 Jan 2010 21:36:27 -0800 (PST) Subject: [File Input Module]Replacing string in a file Message-ID: <9d24353e-b48f-4869-9e3b-f5e7555de47f@u15g2000prd.googlegroups.com> Hi all, For replacing string in the file, i have used the module File input for replacing the string in the file. For understanding and execution purpose, i have just included Python as a string in the file and want it to be replaced to PYTHON. Below are my queries and code: (Correct me if my understanding is wrong???????) 1)) import fileinput x = fileinput.input('data.txt',inplace=0) for line in x: line = line.replace('Python','PYTHON) print line, x.close() The above piece of code will not create any backup file but it will replace PYTHON (Print on the console) but not physically write to the file. 2))) import fileinput x = fileinput.input('data.txt',inplace=1) for line in x: line = line.replace('Python','PYTHON') print line, x.close() The above piece of code will create backup file but hidden (in the form of bak file) and it will physically write to the file -- I have verified the contents of data.txt after the file operation and it had written successfully.But why it is not printing line i.e. string in the file on the console. 3))) import fileinput x = fileinput.input('data.txt',inplace=1) for line in x: line = line.replace('Python','PYTHON') x.close() The above piece of code after execution is wiping out the full contents. But addition of print line, is exactly replacing the string, what exactly addition of print is making difference??? 4))) import fileinput x = fileinput.input('data.txt',inplace=1,backup='content.txt') for line in x: line = line.replace('Python','PYTHON') print line, x.close() The above piece is creating a backup file by name data.txtcontent.txt (I am not sure whether created file name is correct or not?) and to the back up file it had added previous content i.e., Python and it had replaced the contents in data.txt to PYTHON From tjreedy at udel.edu Fri Jan 29 00:56:26 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 29 Jan 2010 00:56:26 -0500 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> Message-ID: On 1/28/2010 6:47 PM, Paul Boddie wrote: > On 27 Jan, 13:26, Xah Lee wrote: >> >> So, for practical reasons, i think a ?key? parameter is fine. But >> chopping off ?cmp? is damaging. When your data structure is complex, >> its order is not embedded in some ?key?. Taking out ?cmp? makes it >> impossible to sort your data structure. > > What would annoy me if I used Python 3.x would be the apparent lack of > the __cmp__ method for conveniently defining comparisons between > instances of my own classes. Having to define all the rich comparison > methods frequently isn't even as much fun as it sounds. You define __eq__, which automatically generates __ne__. You define __lt__, which is all sort and heap need. This should be about as easier than __eq__, which is really needed, and __cmp__. If you need the other 3, either copy the recipe in the Cookbook, or, even faster def __ge__(s,o): return o.__lt__(s) def __le__(s,o): return s < o or s == o def __ge__(s,o): return s > o or s == o All done. Terry Jan Reedy From ben+python at benfinney.id.au Fri Jan 29 01:01:36 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 29 Jan 2010 17:01:36 +1100 Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> Message-ID: <87vdelwitb.fsf@benfinney.id.au> Dennis Lee Bieber writes: > On Thu, 28 Jan 2010 11:24:28 -0800 (PST), Joan Miller: > > On 28 ene, 19:16, Josh Holland wrote: > > > Check the docs on os.system(). > > No. I've a function that uses subprocess to run commands on the same > > shell and so substitute to bash scrips. But a script full of run > > ("shell_command --with --arguments") is too verbose. > > I shall blaspheme, and suggest that maybe the language you want > to use is REXX (ooREXX or Regina). Heh. That isn't blasphemy, because no true Pythonista [0] would claim Python to be the god of that domain. It's no sin to say that Python isn't a good choice for specific things; and ?I want to write programs by indistinguishably mixing statements with external system calls? is one of them, IMO. [0] fully aware of , thanks in advance. -- \ ?Natural catastrophes are rare, but they come often enough. We | `\ need not force the hand of nature.? ?Carl Sagan, _Cosmos_, 1980 | _o__) | Ben Finney From steve at REMOVE-THIS-cybersource.com.au Fri Jan 29 01:01:40 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 06:01:40 GMT Subject: [File Input Module]Replacing string in a file References: <9d24353e-b48f-4869-9e3b-f5e7555de47f@u15g2000prd.googlegroups.com> Message-ID: <03726626$0$1309$c3e8da3@news.astraweb.com> On Thu, 28 Jan 2010 21:36:27 -0800, vanam wrote: > Hi all, > > For replacing string in the file, i have used the module File input for > replacing the string in the file. If you're processing a single file, using fileinput is overkill, and slow as well. Here is a quick-and-dirty way of processing one file, with no error checking: text = open("some file", "r").read() # modify the text however you like text = text.replace("ham", "spam") # replace in place open("some file", "w").write(text) In a real application, you would need to do significant error checking. But it shows the basic technique: you read a file, modify it in memory, then write it back to disk. > For understanding and execution purpose, i have just included Python as > a string in the file and want it to be replaced to PYTHON. > > Below are my queries and code: (Correct me if my understanding is > wrong???????) > > 1)) > > import fileinput > x = fileinput.input('data.txt',inplace=0) > for line in x: > line = line.replace('Python','PYTHON) > print line, > x.close() > > The above piece of code will not create any backup file but it will > replace PYTHON (Print on the console) but not physically write to the > file. Correct. > 2))) > > import fileinput > x = fileinput.input('data.txt',inplace=1) > for line in x: > line = line.replace('Python','PYTHON') > print line, > x.close() > > The above piece of code will create backup file but hidden (in the form > of bak file) and it will physically write to the file -- I have verified > the contents of data.txt after the file operation and it had written > successfully.But why it is not printing line i.e. string in the file on > the console. You should read the documentation to fileinput. The inplace=1 argument tells fileinput to redirect stdout to the current file, so the output of print will go to the file instead of the console. Personally, I find that an incredibly unintuitive and dangerous API, and would recommend against fileinput just for that reason alone. > 3))) > > import fileinput > x = fileinput.input('data.txt',inplace=1) > for line in x: > line = line.replace('Python','PYTHON') > x.close() > > The above piece of code after execution is wiping out the full contents. > But addition of print line, is exactly replacing the string, what > exactly addition of print is making difference??? That's how the module works: printing will go to the file instead of the console. Since you don't print anything, the file gets written with the empty string. > 4))) > > import fileinput > x = fileinput.input('data.txt',inplace=1,backup='content.txt') > for line in x: > line = line.replace('Python','PYTHON') print line, > x.close() > > The above piece is creating a backup file by name data.txtcontent.txt Yes, that's write, you told it to use "content.txt" as the file extension on the backup file. So it starts with the existing name, "data.txt" and adds "content.txt" as the extension, giving "data.txtcontent.txt". You really should read the documentation. In the interactive interpreter: import fileinput help(fileinput) > (I am not sure whether created file name is correct or not?) and to the > back up file it had added previous content i.e., Python and it had > replaced the contents in data.txt to PYTHON Is there a question there? -- Steven From debatem1 at gmail.com Fri Jan 29 01:02:46 2010 From: debatem1 at gmail.com (geremy condra) Date: Fri, 29 Jan 2010 01:02:46 -0500 Subject: [OT] Perl 6 [was Re: myths about python 3] In-Reply-To: <03725fae$0$1309$c3e8da3@news.astraweb.com> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <03725fae$0$1309$c3e8da3@news.astraweb.com> Message-ID: On Fri, Jan 29, 2010 at 12:34 AM, Steven D'Aprano wrote: > On Thu, 28 Jan 2010 21:21:05 -0800, Tim Roberts wrote: > >> Perl 6, on the other hand, is still fantasyware a decade after its >> announcement. ?It is, for the most part, THE canonical example of the >> wrong way to conduct a development effort. > > Out of curiosity, and completely off-topic, why has Perl 6 gone so badly? > > > -- > Steven > -- Too much like Perl. Geremy Condra From clp2 at rebertia.com Fri Jan 29 01:07:35 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Thu, 28 Jan 2010 22:07:35 -0800 Subject: [OT] Perl 6 [was Re: myths about python 3] In-Reply-To: <03725fae$0$1309$c3e8da3@news.astraweb.com> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <03725fae$0$1309$c3e8da3@news.astraweb.com> Message-ID: <50697b2c1001282207h7c671f16j1f6ed6064731f970@mail.gmail.com> On Thu, Jan 28, 2010 at 9:34 PM, Steven D'Aprano < steve at remove-this-cybersource.com.au> wrote: > On Thu, 28 Jan 2010 21:21:05 -0800, Tim Roberts wrote: > >> Perl 6, on the other hand, is still fantasyware a decade after its >> announcement. It is, for the most part, THE canonical example of the >> wrong way to conduct a development effort. > > Out of curiosity, and completely off-topic, why has Perl 6 gone so badly? - Super-dramatic changes to the language (some people think Python 3 is/was radical, but it's got nothing on Perl 6) - Complete from-scratch rewrite (c.f. Joel Spolsky's "Things You Should Never Do") - Ambitious implementation strategy, i.e. Parrot. It will be powerful and useful beyond just Perl when complete, but that extra complexity comes at a price. => Although why Pugs and the other implementations are likewise held up, I have no idea. But then I'm not a Perlite (Is that the right word?). For that matter, it remains to be seen whether the Python 3 changeover will fare better. But I'm still pretty hopeful. At least we have a complete reference implementation. Cheers, Chris -- What we need is some sort of coordinated library porting effort. Or a 3.x-specific killer app. http://blog.rebertia.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From alfps at start.no Fri Jan 29 01:10:01 2010 From: alfps at start.no (Alf P. Steinbach) Date: Fri, 29 Jan 2010 07:10:01 +0100 Subject: python 3's adoption In-Reply-To: References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> <184f0182-acbc-4a85-a8d9-7f813ebd2eb0@k2g2000pro.googlegroups.com> <0372328c$0$1309$c3e8da3@news.astraweb.com> Message-ID: * Steve Holden: >> > While I am fully aware that premature optimization, etc., but I cannot > resist an appeal to efficiency if it finally kills off this idea that > "they took 'cmp()' away" is a bad thing. > > Passing a cmp= argument to sort provides the interpreter with a function > that will be called each time any pair of items have to be compared. The > key= argument, however, specifies a transformation from [x0, x1, ..., > xN] to [(key(x0), x0), (key(x1), x1), ..., (key(xN), xN)] (which calls > the key function precisely once per sortable item). > >>From a C routine like sort() [in CPython, anyway] calling out from C to > a Python function to make a low-level decision like "is A less than B?" > turns out to be disastrous for execution efficiency (unlike the built-in > default comparison, which can be called directly from C in CPython). > > If your data structures have a few hundred items in them it isn't going > to make a huge difference. If they have a few million thenit is already > starting to affect performance ;-) It's not either/or, it's do programmers still need the cmp functionality? Consider that *correctness* is a bit more important than efficiency, and that sorting strings is quite common... Possibly you can show me the way forward towards sorting these strings (shown below) correctly for a Norwegian locale. Possibly you can't. But one thing is for sure, if there was a cmp functionality it would not be a problem. Python 3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> L = ["?", "?", "?"] # This is in SORTED ORDER in Norwegian >>> L ['?', '?', '?'] >>> L.sort() >>> L ['?', '?', '?'] >>> >>> import locale >>> locale.getdefaultlocale() ('nb_NO', 'cp1252') >>> locale.setlocale( locale.LC_ALL ) # Just checking... 'C' >>> locale.setlocale( locale.LC_ALL, "" ) # Setting default locale, Norwgian. 'Norwegian (Bokm?l)_Norway.1252' >>> locale.strxfrm( "???" ) '???' >>> L.sort( key = locale.strxfrm ) >>> L ['?', '?', '?'] >>> locale.strcoll( "?", "?" ) 1 >>> locale.strcoll( "?", "?" ) -1 >>> Note that strcoll correctly orders the strings as ["?", "?", "?"], that is, it would have if it could have been used as cmp function to sort (or better, to a separate routine named e.g. custom_sort). And strcoll can be so used in 2.x: C:\Documents and Settings\Alf\test> py2 Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> def show( list ): ... print "[" + ", ".join( list ) + "]" ... >>> L = [u"?", u"?", u"?"] >>> show( L ) [?, ?, ?] >>> L.sort() >>> show( L ) [?, ?, ?] >>> import locale >>> locale.setlocale( locale.LC_ALL, "" ) 'Norwegian (Bokm\xe5l)_Norway.1252' >>> L.sort( cmp = locale.strcoll ) >>> show( L ) [?, ?, ?] >>> L [u'\xe6', u'\xf8', u'\xe5'] >>> _ The above may just be a bug in the 3.x stxfrm. But it illustrates that sometimes you have your sort order defined by a comparision function. Transforming that into a key can be practically impossible (it can also be quite inefficient). Cheers & hth., - Alf From clp2 at rebertia.com Fri Jan 29 01:14:09 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Thu, 28 Jan 2010 22:14:09 -0800 Subject: Wrap a function In-Reply-To: References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> Message-ID: <50697b2c1001282214h7100ec75v374bfdc81aa985c1@mail.gmail.com> On Thu, Jan 28, 2010 at 9:44 PM, Dennis Lee Bieber wrote: > On Thu, 28 Jan 2010 11:24:28 -0800 (PST), Joan Miller > declaimed the following in > gmane.comp.python.general: > > > On 28 ene, 19:16, Josh Holland wrote: > > > On 2010-01-28, Joan Miller wrote: > > > > > > > I've to call to many functions with the format: > > > > > > >>>> run("cmd") > > > > > > Check the docs on os.system(). > > No. I've a function that uses subprocess to run commands on the same > > shell and so substitute to bash scrips. But a script full of run > > ("shell_command --with --arguments") is too verbose. > > ? ? ? ?I shall blaspheme, and suggest that maybe the language you want to > use is REXX (ooREXX or Regina). Sounds like the REXX designers already got the blaspheming covered when they came up with such an inelegant-sounding feature... > ? ? ? ?By default, ANY statement that can not be confused for a REXX > language statement is sent to the currently defined command handler > (Which on most OSs is equivalent to Python's os.system() call; the late > Amiga, and IBM's mainframe OS had features that support defining other > applications as command handlers). > > ? ? ? ?A common practice is to put quotes about the first word of the > command to ensure it gets treated as external command. Cheers, Chris -- http://blog.rebertia.com From wuwei23 at gmail.com Fri Jan 29 01:16:08 2010 From: wuwei23 at gmail.com (alex23) Date: Thu, 28 Jan 2010 22:16:08 -0800 (PST) Subject: Perl 6 [was Re: myths about python 3] References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <03725fae$0$1309$c3e8da3@news.astraweb.com> Message-ID: geremy condra wrote: > Steven D'Aprano wrote: > > Out of curiosity, and completely off-topic, why has Perl 6 gone so badly? > > Too much like Perl. I was going to suggest that it's probably due to the multitude of ways to it could be done :) From tjreedy at udel.edu Fri Jan 29 01:18:22 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 29 Jan 2010 01:18:22 -0500 Subject: Sorting a list with entries of unequal types In-Reply-To: <03722537$0$1309$c3e8da3@news.astraweb.com> References: <87sk9pzsmm.fsf@benfinney.id.au> <87fx5pzp9t.fsf@benfinney.id.au> <03722537$0$1309$c3e8da3@news.astraweb.com> Message-ID: On 1/28/2010 8:24 PM, Steven D'Aprano wrote: >>> Um, what ordering do you want? >> >> The same ordering I'd get in Python 2; that is, determined by the types >> of the elements. > > The ordering has not been consistent across minor versions of Python 2: And complex numbers, and probably some other things, cannot be sorted even in Python2. Universal sorting is a broken idea, so Python3 leaves it to you to say what *you* mean. From 54wutong at gmail.com Fri Jan 29 01:29:53 2010 From: 54wutong at gmail.com (Stephen.Wu) Date: Thu, 28 Jan 2010 22:29:53 -0800 (PST) Subject: which one is faster? Message-ID: str.find(targetStr) str.index(targetStr) with exception str.count(targetStr) targetStr in str which is the fastest way to check whether targetStr is in str? thanks all From wuwei23 at gmail.com Fri Jan 29 01:39:32 2010 From: wuwei23 at gmail.com (alex23) Date: Thu, 28 Jan 2010 22:39:32 -0800 (PST) Subject: which one is faster? References: Message-ID: <140a7cf6-b4fa-4bab-a31d-aacf2c7678c5@k2g2000pro.googlegroups.com> On Jan 29, 4:29?pm, "Stephen.Wu" <54wut... at gmail.com> wrote: > str.find(targetStr) > str.index(targetStr) with exception > str.count(targetStr) > targetStr in str > > which is the fastest way to check whether targetStr is in str? It's generally a lot quicker to investigate this kind of question yourself using the interpreter & the timeit module. You'll need skills like these to be able to profile your code to look for actual performance bottlenecks, generic advice on the fastest of a set of functions will only get you so far. IPython is pretty handy for simple timing tests as it provides convenience wrappers around timeit: In [1]: t = 'foo' In [2]: s = 'djoemdmsllsodmedmsoskemozpleaoleodspsfooosoapxooeplaapakekoda' In [3]: timeit s.find(t) 1000000 loops, best of 3: 374 ns per loop In [4]: timeit s.index(t) 1000000 loops, best of 3: 381 ns per loop In [7]: timeit s.count(t) 1000000 loops, best of 3: 397 ns per loop In [8]: timeit t in s 1000000 loops, best of 3: 219 ns per loop >From the looks of those results, using 'in' seems to be the fastest. From ben+python at benfinney.id.au Fri Jan 29 02:08:35 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 29 Jan 2010 18:08:35 +1100 Subject: Sorting a list with entries of unequal types References: <87sk9pzsmm.fsf@benfinney.id.au> <87fx5pzp9t.fsf@benfinney.id.au> <03722537$0$1309$c3e8da3@news.astraweb.com> Message-ID: <87r5p9wfpo.fsf@benfinney.id.au> Terry Reedy writes: > And complex numbers, and probably some other things, cannot be sorted > even in Python2. Universal sorting is a broken idea, so Python3 leaves > it to you to say what *you* mean. Okay. I guess I'd better figure out what that is, for this code base, before porting it to Python 3 :-) Thanks for the helpful responses. -- \ ?Most people, I think, don't even know what a rootkit is, so | `\ why should they care about it?? ?Thomas Hesse, Sony BMG, 2006 | _o__) | Ben Finney From nyamatongwe+thunder at gmail.com Fri Jan 29 02:22:03 2010 From: nyamatongwe+thunder at gmail.com (Neil Hodgson) Date: Fri, 29 Jan 2010 07:22:03 GMT Subject: [OT] Perl 6 [was Re: myths about python 3] In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <03725fae$0$1309$c3e8da3@news.astraweb.com> Message-ID: Looks to me like the problem with Perl 6 was that it was too ambitious, wanting to fix all perceived problems with the language. Python 3 is much more limited in scope: at its core its Python with Unicode fixed and old junk removed. Neil From rdv at roalddevries.nl Fri Jan 29 03:08:56 2010 From: rdv at roalddevries.nl (Roald de Vries) Date: Fri, 29 Jan 2010 09:08:56 +0100 Subject: Symbols as parameters? In-Reply-To: <03722683$0$1309$c3e8da3@news.astraweb.com> References: <1495297.2JfmrJjJTh@beaureve.gmx.net> <4CD97E94-B40E-481D-B367-5FCE2820F48D@roalddevries.nl> <03722683$0$1309$c3e8da3@news.astraweb.com> Message-ID: <21315400-1778-4F2F-A644-06BDA9FCBDA8@roalddevries.nl> On Jan 29, 2010, at 2:30 AM, Steven D'Aprano wrote: > On Thu, 28 Jan 2010 17:01:38 +0100, Roald de Vries wrote: >> Question out of general interest in the language: If I would want to >> generate such functions in a for-loop, what would I have to do? This >> doesn't work: >> >> class Move(object): >> def __call__(self, direction): >> return direction >> >> move = Move() >> >> for f in ['up', 'down', 'right', 'left']: >> move.__dict__[f] = lambda: move(f) >> >> ... because now 'move.up()' returns 'left' because thats the current >> value of f. Is there a way to 'expand' f in the loop? Or a reason >> that >> you never should use this? > > > Possibly the simplest way is to use Python's handling of default > values > to get the result you want: > > for f in ['up', 'down', 'right', 'left']: > move.__dict__[f] = lambda f=f: move(f) This still leaves open the possibility of move.up('down') resulting in move('down'), but for the rest I like it. > BTW, there's no need to explicitly reference move.__dict__: > > for f in ['up', 'down', 'right', 'left']: > setattr(move, f, lambda f=f: move(f)) I expected something like that. Thanks. Roald From martin at v.loewis.de Fri Jan 29 03:16:17 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Fri, 29 Jan 2010 09:16:17 +0100 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <4B629951.6020700@v.loewis.de> > Python has had > previous major changes in the past (e.g. 1.5 to 2.0 and 2.1 to 2.2) and > hardly anyone made a complaint. I think this is actually false for the switch from 1.5 to 2.0. People complained a lot, and announced that they won't switch to Python 2 in any foreseeable future, and indeed, they stayed with Python 1.5.2 for several years after Python 2 was released. Of course, the Python user base was much smaller then than it is now, so the absolute number of complainers surely was never as high is it is now. Only when Python 4 gets released, even more people will complain and announce that Python 4 has failed and that they stay with Python 3.8 forever. Regards, Martin From martin at v.loewis.de Fri Jan 29 03:25:15 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Fri, 29 Jan 2010 09:25:15 +0100 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> Message-ID: <4B629B6B.3000100@v.loewis.de> > Well, I'd consider that an official release. Note that I didn't claim > there was no hope PSF wouldn't change it's mind on 2.8. I'd like to point out that the PSF formally doesn't have any say in this. Instead, releases are created by the release manager, who gets appointed by Guido van Rossum. Those two listen primarily to the opinions of the fellow committers (which may or may not happen to be PSF members as well). > Regardless of how magnaminous the people of PSF are, the unfortunate > reality is that trademark owners are forced by the law to be > "particularly petty". PSF's IP lawyer will advise not to allow > unsanctioned fork of Python 2.7 to call itself Python 2.8. However, calling it "Passive Python", or "John Smith Python", for example, would certainly be fine. Calling something "Python 2.8" may not be unreasonable, also "Python 2.9", but surely "Python 2.10" will hit strong objections (GvR always announced that there will never be a two-digit minor release number). Regards, Martin From martin at v.loewis.de Fri Jan 29 03:33:45 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Fri, 29 Jan 2010 09:33:45 +0100 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <4B629D69.3070905@v.loewis.de> > Why do I feel like there's less of an onus on Unladen Swallow to > _actually prove itself in substantial real world usage_ before > integration into CPython than there is on even the smallest of modules > for inclusion in the standard library? Because it's a VM change, not an end-user change. VM changes can be reviewed and evaluated by core developers, without requiring necessarily feedback from end users (although end users can and do certainly evaluate VM changes themselves also and provide feedback). For library changes, it's more difficult to evaluate them, because you not only need to find out whether the change is correct, but also whether it is useful. > Are we really expected to just ditch everything we know about > CPython's performance characteristics just for some questionable and > possibly uneven gains? That's the point of writing a PEP. Provide feedback to the PEP authors, and ask them to incorporate your objections into the PEP in case they forget. Then, when the PEP is about to be approved, that feedback gets taken into consideration. Posting in the 665th message on a Usenet thread is unlikely to have any effect on the PEP process, though. > I've been a big supporter of Py3 from the beginning, but this repeated > claim of US becoming the mainline interpreter for 3.x pretty much > kills dead a lot of my interest. It's not a claim, it's a PEP. Not being interested in the PEP process is your choice, of course, but you shouldn't complain afterwards that your opinion wasn't considered if you didn't actually voice it appropriately. Regards, Martin From peloko45 at gmail.com Fri Jan 29 03:41:33 2010 From: peloko45 at gmail.com (Joan Miller) Date: Fri, 29 Jan 2010 00:41:33 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> <9a28cf01-20af-4ce2-9b78-d8529800e4b9@y7g2000prc.googlegroups.com> Message-ID: On 28 ene, 23:36, Jonathan Gardner wrote: > On Jan 28, 2:16?pm, Joan Miller wrote: > > > > > There would be to make a function for each system command to use so it > > would be too inefficient, and follow the problem with the quotes. > > > The best is make a parser into a compiled language > > Yeah, you could do that. Or you can simply rely on /bin/sh to do the > parsing and everything else for you. No need to re-invent the wheel. I > don't think Python will ever beat sh as a shell replacement. > > When people say that Python is great for some situations, but not so > much for others, I think they thought of running commands like this as > "other", I started to working on this project (Scripy [1]) because I wanted to hacking cryptsetup in my ubuntu. The funcionts to manage its initialization are in bash and it goes to be non-maintainable code, cryptic and very hard to debug (as whatever bash script of medium size). Here you have the beast: http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/karmic/cryptsetup/karmic/annotate/head%3A/debian/cryptdisks.functions Using Scripy I can debug easily the commands run from the shell, and log all if I would. Now, thanks to Scripy I've created a script for easily disks partitioning [2] using a simple cofiguration in YAML [3]. The great thing is that I can add volumes and encrypted partitions :) The only problem is that it's too verbose but I could rename *run()* by a simple function as *_()* or *r()* [1] http://bitbucket.org/ares/scripy/src/ [2] http://bitbucket.org/ares/scripypartition/src/tip/lib/scripy/part/disk.py#cl-22 [3] http://bitbucket.org/ares/scripypartition/src/tip/bin/init_crypto.py#cl-46 From nagle at animats.com Fri Jan 29 03:41:54 2010 From: nagle at animats.com (John Nagle) Date: Fri, 29 Jan 2010 00:41:54 -0800 Subject: Why doesn't "filterwarnings" suppress this message? In-Reply-To: References: <4b61d5fb$0$1658$742ec2ed@news.sonic.net> Message-ID: <4b629bac$0$1620$742ec2ed@news.sonic.net> It turned out to be a loading order issue. That message is generated at import time, and "filterwarnings" hadn't been called yet. John Nagle Peter Otten wrote: > John Nagle wrote: > >> Feedparser isn't supported for Python 3.0, so in Python 2.6, many >> warning >> messages appear. I'm trying, in Python 2.6, to suppress the warning >> message: >> >> ./feedparser\feedparser.py:69: DeprecationWarning: >> the sgmllib module has been removed in Python 3.0 >> import sgmllib, re, sys, copy, urlparse, time, rfc822, types, >> cgi, urllib, urllib2 >> >> with this warning filter: >> >> warnings.filterwarnings(action='ignore', >> category=DeprecationWarning, module='feedparser') >> >> The filter suppresses other deprecation warnings associated with >> the feedparser module, but not that one. How do I get rid of that >> one? From peloko45 at gmail.com Fri Jan 29 03:43:19 2010 From: peloko45 at gmail.com (Joan Miller) Date: Fri, 29 Jan 2010 00:43:19 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> <4B61EC7A.7020406@optimum.net> <64dc1630-6aa5-4983-b061-261457bf0153@g29g2000yqe.googlegroups.com> <8508278f-0cf2-46c2-8313-2c640a71cdbd@q4g2000yqm.googlegroups.com> <4B621645.1000906@optimum.net> Message-ID: <8595827f-f698-46d2-9178-8138f9ceac24@f12g2000yqn.googlegroups.com> On 28 ene, 22:57, John Posner wrote: > On 1/28/2010 3:45 PM, Joan Miller wrote: > > > > > On 28 ene, 20:34, Joan Miller ?wrote: > >> On 28 ene, 20:20, Peter ?wrote: > > >>> On Jan 29, 6:58 am, John Posner ?wrote: > > >>>> On 1/28/2010 2:24 PM, Joan Miller wrote: > > >>>>> On 28 ene, 19:16, Josh Holland ? ?wrote: > >>>>>> On 2010-01-28, Joan Miller ? ?wrote: > > >>>>>>> I've to call to many functions with the format: > > >>>>>>>>>> run("cmd") > > >>>>>> Check the docs on os.system(). > >>>>> No. I've a function that uses subprocess to run commands on the same > >>>>> shell and so substitute to bash scrips. But a script full of run > >>>>> ("shell_command --with --arguments") is too verbose. > > >>>> I'm suspicious of your original intent. Essentially, you want to write > >>>> code in which a literal string, such as ... > > >>>> ? ? ls -l > > >>>> ... is *not* enclosed in quotes. Why run the risk of creating confusion > >>>> (in other people who look at your code, in syntax-checking tools, etc.) > >>>> between variables and literals? > > >>>> But I'm in sympathy with your desire to make the code as clean as > >>>> possible and to minimize the number of times you have to type a quote > >>>> character. My suggestions: > > >>>> 1. Create a function (say, "Run") that encapsulates as much of the > >>>> syntax as possible: os.system(), subprocess.call(), string-splitting, > >>>> whatever. So an invocation would look like this: > > >>>> ? ? Run("ls -l *.txt") > > >>>> (I think you've already done this step.) > > >>>> 2. Find a text editor that supports keyboard macros, so that a single > >>>> keystroke turns this text line: > > >>>> ? ? ls -l *.txt > > >>>> ... into this one: > > >>>> ? ? Run("ls -l *.txt") > > >>>> HTH, > >>>> John > > >>> I can't see you avoiding quotes etc, but extending on John's comment, > >>> the obvious next step would be to run everything in a loop i.e. place > >>> all the commands into a list and create a loop that ran each command > >>> in the list. > > >> Yes, but could be necessary that were mixed with python code. > > >>> Almost all editors support macros - most editors support some form of > >>> language sensitive editing (NOT the prompt call parameters style but > >>> rather help with the syntax via a 'form' style of fill-in) that will > >>> allow you to reduce typing effort. But macros would be the first and > >>> easiest choice for this activity. > > >> The goal of my program is substitute to bash scripts, so the macros in > >> editors are irrelevant fo this one. > > > I think that the best solution that I've is to build a program that > > parses the script to convert *$ command* to run("command") before of > > be called by python. > > I believe you're working on Linux, so how about using "sed"? Here's a > (prettified) BASH transcript of a sed script (edit.sed) transforming a > 6-line text file (myprog.py). The text file has both Python statements > and "special commands", which have "$ " at the beginning of the line. > > ?>>> cat myprog.py > print "hello" > $ ls -l > r = range(10) > $ grep foo bar.data > pass > print "bye" > > ?>>> cat edit.sed > s/^\$ \(.*\)/Run("\1")/ > > ?>>> sed -f edit.sed data.txt > print "hello" > Run("ls -l") > r = range(10) > Run("grep foo bar.data") > pass > print "bye" > > -John Yes, this would a well solution. Simple and fast to build. From ben+python at benfinney.id.au Fri Jan 29 03:48:26 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 29 Jan 2010 19:48:26 +1100 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <4B629D69.3070905@v.loewis.de> Message-ID: <87k4v1wb39.fsf@benfinney.id.au> "Martin v. Loewis" writes: > Not being interested in the PEP process is your choice, of course, but > you shouldn't complain afterwards that your opinion wasn't considered > if you didn't actually voice it appropriately. +1 QOTW -- \ ?I installed a skylight in my apartment. The people who live | `\ above me are furious!? ?Steven Wright | _o__) | Ben Finney From nagle at animats.com Fri Jan 29 03:50:18 2010 From: nagle at animats.com (John Nagle) Date: Fri, 29 Jan 2010 00:50:18 -0800 Subject: mix statically typed with dynamically typed In-Reply-To: References: Message-ID: <4b629da4$0$1593$742ec2ed@news.sonic.net> Yingjie Lan wrote: > We all know that Python is dynamically typed, and dynamically typed languages > are generally slower than statically typed ones. That's mostly a problem with the CPython interpreter, which is a naive interpreter. Many dynamically typed languages have implementations which optimize out much of the run-time type handling. Shed Skin does this for Python. LISP has been doing this better for decades. The JIT systems for Javascript also do this. Psyco has some explicit typing capability, but it doesn't do much about eliminating redundant attribute lookups. The two big wins that Python needs for performance are 1) at least recognize when a variable can be represented as "long" "double", or "char", and 2) recognize when an object or module doesn't need dynamic attribute lookup and the attribute slots can be nailed down at compile time. (Plus, of course, something so that multithreaded programs don't suck so bad on multicore CPUs.) John Nagle From __peter__ at web.de Fri Jan 29 03:59:03 2010 From: __peter__ at web.de (Peter Otten) Date: Fri, 29 Jan 2010 09:59:03 +0100 Subject: Sorting a list with entries of unequal types References: <87sk9pzsmm.fsf@benfinney.id.au> <87fx5pzp9t.fsf@benfinney.id.au> Message-ID: Ben Finney wrote: > Peter Otten <__peter__ at web.de> writes: > >> I can't find the relevant part of the 2.6 documentation, but something >> like >> >> >>> def key(x): >> ... t = type(x) >> ... t = compat.get(t, t) >> ... return t.__name__, id(t), x >> ... >> >>> compat = {bool: float, int: float} > > The problem with this approach is that it assumes I know all the types > of elements that will be sorted; I don't. I want to sort a list coming > from some other part of the code, and I don't want to arbitrarily limit > the types of the list elements. I don't see the limitation here. The above key functions sorts unknown types by name, then id(type) then instance. Peter From peloko45 at gmail.com Fri Jan 29 04:06:56 2010 From: peloko45 at gmail.com (Joan Miller) Date: Fri, 29 Jan 2010 01:06:56 -0800 (PST) Subject: Wrap a function References: <38336c6d-c82d-4b83-96c9-5f1210132027@l19g2000yqb.googlegroups.com> Message-ID: On 29 ene, 05:44, Dennis Lee Bieber wrote: > On Thu, 28 Jan 2010 11:24:28 -0800 (PST), Joan Miller > declaimed the following in > gmane.comp.python.general: > > > On 28 ene, 19:16, Josh Holland wrote: > > > On 2010-01-28, Joan Miller wrote: > > > > > I've to call to many functions with the format: > > > > >>>> run("cmd") > > > > Check the docs on os.system(). > > No. I've a function that uses subprocess to run commands on the same > > shell and so substitute to bash scrips. But a script full of run > > ("shell_command --with --arguments") is too verbose. > > ? ? ? ? I shall blaspheme, and suggest that maybe the language you want to > use is REXX (ooREXX or Regina). > > ? ? ? ? By default, ANY statement that can not be confused for a REXX > language statement is sent to the currently defined command handler > (Which on most OSs is equivalent to Python's os.system() call; the late > Amiga, and IBM's mainframe OS had features that support defining other > applications as command handlers). > > ? ? ? ? A common practice is to put quotes about the first word of the > command to ensure it gets treated as external command. I prefer Python since that has a great standard library where I get a well logging system between another things. In addition, the main intended audience will be for system administrators who many of them already have used it. From siddhartha.veedaluru at gmail.com Fri Jan 29 04:17:56 2010 From: siddhartha.veedaluru at gmail.com (siddhartha veedaluru) Date: Fri, 29 Jan 2010 14:47:56 +0530 Subject: Python-list Digest, Vol 76, Issue 303 In-Reply-To: References: Message-ID: <424b71ec1001290117y259a134dkeb31abf906cd32bf@mail.gmail.com> Hi, Thats the output i got in output window.it is not listing the statement which caused it. copy paste and try it on you setup. it behaves the same way. I know that error is in log.info() function call.but it is not caught by "except". which is not getting logged using log.exception Thanks Siddhartha ---------- Forwarded message ---------- From: Steve Holden To: Date: Tue, 26 Jan 2010 11:54:23 -0500 Subject: Re: TypeError not caught by except statement siddhartha veedaluru wrote: > Hi, > > except not able to caught the TypeError exception occured in the below > code > > log.info ("refer",ret) in the try block > > throws a TypeError which is not caught . The traceback you provide does not show the line in question, so I am confused as to why you would think it was causing the problem. The purpose of the traceback is to show you where the exception occurred, and I question whether that is happening here. Anyway, I suspect your error might go away if you turned the first argument of hte log.info() call into a format string such as log.info("refer: %s", ret) regards Steve > Also sometimes process is getting hanged. > > ------------------------------------------------------------------------------------------------------------------------------------------------ > import logging > log = logging.getLogger() > fileName = strftime("%d-%b-%Y-", gmtime()) + str(int(time.time())) + "- > Log.log" > log = logging.getLogger() > log.setLevel(logging.NOTSET) > fh = logging.FileHandler(logFile) > logFileLevel = logging.DEBUG > fh.setLevel(logFileLevel) > format_string = '%(process)d %(thread)d %(asctime)-15s %(levelname)-5s > at %(filename)-15s in %(funcName)-10s at line %(lineno)-3d "%(message) > s"' > fh.setFormatter(logging.Formatter(format_string)) > log.addHandler(fh) > > try: > log.info ("start ") > log.info ("refer",ret ) > log.info ("end ") > except TypeError: > log.exception("Exception raised") > > ---------------------------------------------------------------------------------------------------------------------------------------------- > OUTPUT message: > > Traceback (most recent call last): > File "C:\Python26\lib\logging\__init__.py", line 768, in emit > msg = self.format(record) > File "C:\Python26\lib\logging\__init__.py", line 648, in format > return fmt.format(record) > File "C:\Python26\lib\logging\__init__.py", line 436, in format > record.message = record.getMessage() > File "C:\Python26\lib\logging\__init__.py", line 306, in getMessage > msg = msg % self.args > TypeError: not all arguments converted during string formatting > -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From siddhartha.veedaluru at gmail.com Fri Jan 29 04:18:42 2010 From: siddhartha.veedaluru at gmail.com (siddhartha veedaluru) Date: Fri, 29 Jan 2010 14:48:42 +0530 Subject: TypeError not caught by except statement Message-ID: <424b71ec1001290118n5492939x1f740471b3c17bc0@mail.gmail.com> Hi, Thats the output i got in output window.it is not listing the statement which caused it. copy paste and try it on you setup. it behaves the same way. I know that error is in log.info() function call.but it is not caught by "except". which is not getting logged using log.exception Thanks Siddhartha ---------- Forwarded message ---------- From: Steve Holden To: Date: Tue, 26 Jan 2010 11:54:23 -0500 Subject: Re: TypeError not caught by except statement siddhartha veedaluru wrote: > Hi, > > except not able to caught the TypeError exception occured in the below > code > > log.info ("refer",ret) in the try block > > throws a TypeError which is not caught . The traceback you provide does not show the line in question, so I am confused as to why you would think it was causing the problem. The purpose of the traceback is to show you where the exception occurred, and I question whether that is happening here. Anyway, I suspect your error might go away if you turned the first argument of hte log.info() call into a format string such as log.info("refer: %s", ret) regards Steve > Also sometimes process is getting hanged. > > ------------------------------------------------------------------------------------------------------------------------------------------------ > import logging > log = logging.getLogger() > fileName = strftime("%d-%b-%Y-", gmtime()) + str(int(time.time())) + "- > Log.log" > log = logging.getLogger() > log.setLevel(logging.NOTSET) > fh = logging.FileHandler(logFile) > logFileLevel = logging.DEBUG > fh.setLevel(logFileLevel) > format_string = '%(process)d %(thread)d %(asctime)-15s %(levelname)-5s > at %(filename)-15s in %(funcName)-10s at line %(lineno)-3d "%(message) > s"' > fh.setFormatter(logging.Formatter(format_string)) > log.addHandler(fh) > > try: > log.info ("start ") > log.info ("refer",ret ) > log.info ("end ") > except TypeError: > log.exception("Exception raised") > > ---------------------------------------------------------------------------------------------------------------------------------------------- > OUTPUT message: > > Traceback (most recent call last): > File "C:\Python26\lib\logging\__init__.py", line 768, in emit > msg = self.format(record) > File "C:\Python26\lib\logging\__init__.py", line 648, in format > return fmt.format(record) > File "C:\Python26\lib\logging\__init__.py", line 436, in format > record.message = record.getMessage() > File "C:\Python26\lib\logging\__init__.py", line 306, in getMessage > msg = msg % self.args > TypeError: not all arguments converted during string formatting > -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jspies at sun.ac.za Fri Jan 29 04:23:54 2010 From: jspies at sun.ac.za (Johann Spies) Date: Fri, 29 Jan 2010 11:23:54 +0200 Subject: Need help with a program In-Reply-To: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: <20100129092354.GG1718@sun.ac.za> On Thu, Jan 28, 2010 at 07:07:04AM -0800, evilweasel wrote: > Hi folks, > > I am a newbie to python, and I would be grateful if someone could > point out the mistake in my program. Basically, I have a huge text > file similar to the format below: > > AAAAAGACTCGAGTGCGCGGA 0 > AAAAAGATAAGCTAATTAAGCTACTGG 0 > AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 > AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 > AAAAAGGTCGCCTGACGGCTGC 0 I know this is a python list but if you really want to get the job done quickly this is one method without writing python code: $ cat /tmp/y AAAAAGACTCGAGTGCGCGGA 0 AAAAAGATAAGCTAATTAAGCTACTGG 0 AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 AAAAAGGTCGCCTGACGGCTGC 0 $ grep -v 0 /tmp/y > tmp/z $ cat /tmp/z AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 Regards Johann -- Johann Spies Telefoon: 021-808 4599 Informasietegnologie, Universiteit van Stellenbosch "My son, if sinners entice thee, consent thou not." Proverbs 1:10 From steve at REMOVE-THIS-cybersource.com.au Fri Jan 29 05:04:33 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 10:04:33 GMT Subject: Need help with a program References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: <03729f14$0$1309$c3e8da3@news.astraweb.com> On Fri, 29 Jan 2010 11:23:54 +0200, Johann Spies wrote: > On Thu, Jan 28, 2010 at 07:07:04AM -0800, evilweasel wrote: >> Hi folks, >> >> I am a newbie to python, and I would be grateful if someone could point >> out the mistake in my program. Basically, I have a huge text file >> similar to the format below: >> >> AAAAAGACTCGAGTGCGCGGA 0 >> AAAAAGATAAGCTAATTAAGCTACTGG 0 >> AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 >> AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 >> AAAAAGGTCGCCTGACGGCTGC 0 > > I know this is a python list but if you really want to get the job done > quickly this is one method without writing python code: > > $ cat /tmp/y > AAAAAGACTCGAGTGCGCGGA 0 > AAAAAGATAAGCTAATTAAGCTACTGG 0 > AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 > AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 > AAAAAGGTCGCCTGACGGCTGC 0 > $ grep -v 0 /tmp/y > tmp/z > $ cat /tmp/z > AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 > AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 That will do the wrong thing for lines like: AAAAAGATAAGCTAATTAAGCTACTGGGTT 10 -- Steven From bruno.42.desthuilliers at websiteburo.invalid Fri Jan 29 06:00:35 2010 From: bruno.42.desthuilliers at websiteburo.invalid (Bruno Desthuilliers) Date: Fri, 29 Jan 2010 12:00:35 +0100 Subject: which one is faster? In-Reply-To: <140a7cf6-b4fa-4bab-a31d-aacf2c7678c5@k2g2000pro.googlegroups.com> References: <140a7cf6-b4fa-4bab-a31d-aacf2c7678c5@k2g2000pro.googlegroups.com> Message-ID: <4b62bfb8$0$23526$426a74cc@news.free.fr> alex23 a ?crit : > On Jan 29, 4:29 pm, "Stephen.Wu" <54wut... at gmail.com> wrote: >> str.find(targetStr) >> str.index(targetStr) with exception >> str.count(targetStr) >> targetStr in str >> >> which is the fastest way to check whether targetStr is in str? > > It's generally a lot quicker to investigate this kind of question > yourself using the interpreter & the timeit module. You'll need skills > like these to be able to profile your code to look for actual > performance bottlenecks, generic advice on the fastest of a set of > functions will only get you so far. Indeed. Another point to take into consideration is the _real_ dataset on which the functions under test will have to work. And possibly the available resources at runtime. As an example, an iterator-based solution is likely to be slower than a more straightforward "load everything in ram" one, but will scale better for large datasets and/or heavy concurrent access to the system. From rajat.dudeja at gmail.com Fri Jan 29 06:30:18 2010 From: rajat.dudeja at gmail.com (Rajat) Date: Fri, 29 Jan 2010 03:30:18 -0800 (PST) Subject: Thread get off from CPU occasionally. I suspect. Message-ID: <964a0563-cd3f-407e-8c32-9ebd07ef8829@a16g2000pre.googlegroups.com> Hi, I'm using Python 2.6 and using the threading module. In a python module (parent module) I'm importing main() function from another python module. I'm runnin this main() function on a Python thread so that I can periodically check in the parent thread/module if the new thread is up and running. This information that the python script running on the new thread is still up and running is passed to a 3rd party application (in a manner known to 3rd party). As 3rd party is waiting for this new thread to finish. It waits for 3 mins and on told to it by the parent thread that new thread is still running, the 3rd party then starts to wait again for 3 mins. I've successfully implemented this functionality and it runs well except for a rare chance when the new thread does not completely runs. However, most of the cases the new thread runs completely. I cannot understand why and when the new thread gets off the CPU or crashes or any other thing? I'm not able to figure out what is happening.? The code in the parent thread that starts the new thread is as follows? exec("from " + passed_script_file + " import main") #imported main function. It works. import threading import time p = RunTestScript(main) p.start() t_nought = time.time() logSystemInfo(t_nought) seconds_passed = 0 keep_alive_set = 0 while p.isAlive() == True: p.join(60.0) if p.isAlive() == True: seconds_passed = time.time() - t_nought if seconds_passed >= 120: t_nought = time.time() if keep_alive_set != 1: # 3rd party way of informing that the new thread is alive. request = 'SET SHARED VAR KEEP_ALIVE=%d' %(1) res = handle.submit('local', 'VAR', request) if (res.rc != 0): logSystemError('[PyInt] Error: STAF local VAR %s failed, RC: %s, Result: %s\n' % (request, res.rc, res.result)) return res.rc keep_alive_set = 1 Earlier, when I did not run the main function on a new thread, it always used to run completely. I used to run it as: exec("from " + passed_script_file + " import main") retCode = main() Thanks and regards, Rajat From steve at REMOVE-THIS-cybersource.com.au Fri Jan 29 06:47:26 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 29 Jan 2010 11:47:26 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> <184f0182-acbc-4a85-a8d9-7f813ebd2eb0@k2g2000pro.googlegroups.com> <0372328c$0$1309$c3e8da3@news.astraweb.com> Message-ID: <0372b731$0$1309$c3e8da3@news.astraweb.com> On Fri, 29 Jan 2010 07:10:01 +0100, Alf P. Steinbach wrote: > >>> L = ["?", "?", "?"] # This is in SORTED ORDER in Norwegian L [...] > >>> L.sort( key = locale.strxfrm ) > >>> L > ['?', '?', '?'] > >>> locale.strcoll( "?", "?" ) > 1 > >>> locale.strcoll( "?", "?" ) > -1 > > Note that strcoll correctly orders the strings as ["?", "?", "?"], that > is, it would have if it could have been used as cmp function to sort (or > better, to a separate routine named e.g. custom_sort). This is in Python2.5, so I'm explicitly specifying unicode strings: >>> L = [u"?", u"?", u"?"] >>> assert sorted(L) == [u'?', u'?', u'?'] The default C-locale sorting of L does not equal to L. Now let's change to Norwegian locale: >>> import locale >>> locale.setlocale(locale.LC_ALL, 'nb_NO') 'nb_NO' >>> print u''.join(sorted(L, cmp=locale.strcoll)) ??? So far so good, we've confirmed that in Python 2.5 we can sort in a locale-aware form. Now, can we do this with a key function? Thanks to Raymond Hettinger's recipe here: http://code.activestate.com/recipes/576653/ >>> print u''.join(sorted(L, key=CmpToKey(locale.strcoll))) ??? Success! Let's try it in Python 3.1: >>> L = ["?", "?", "?"] >>> assert sorted(L) == ['?', '?', '?'] >>> >>> import locale >>> locale.setlocale(locale.LC_ALL, 'nb_NO') 'nb_NO' >>> ''.join(sorted(L, key=CmpToKey(locale.strcoll))) '???' The definition of CmpToKey can be found at the URL above. It's not very exciting, but here it is: def CmpToKey(mycmp): 'Convert a cmp= function into a key= function' class K(object): def __init__(self, obj, *args): self.obj = obj def __lt__(self, other): return mycmp(self.obj, other.obj) == -1 def __gt__(self, other): return mycmp(self.obj, other.obj) == 1 def __eq__(self, other): return mycmp(self.obj, other.obj) == 0 def __le__(self, other): return mycmp(self.obj, other.obj) != 1 def __ge__(self, other): return mycmp(self.obj, other.obj) != -1 def __ne__(self, other): return mycmp(self.obj, other.obj) != 0 return K If that's too verbose for you, stick this as a helper function in your application: def CmpToKey(mycmp): 'Convert a cmp= function into a key= function' class K(object): def __init__(self, obj, *args): self.obj = obj __lt__ = lambda s, o: mycmp(s.obj, o.obj) == -1 __gt__ = lambda s, o: mycmp(s.obj, o.obj) == 1 __eq__ = lambda s, o: mycmp(s.obj, o.obj) == 0 __le__ = lambda s, o: mycmp(s.obj, o.obj) != 1 __ge__ = lambda s, o: mycmp(s.obj, o.obj) != -1 __ne__ = lambda s, o: mycmp(s.obj, o.obj) != 0 return K [...] > The above may just be a bug in the 3.x stxfrm. But it illustrates that > sometimes you have your sort order defined by a comparision function. > Transforming that into a key can be practically impossible (it can also > be quite inefficient). This might be true, but you haven't demonstrated it. With one little helper function, you should be able to convert any comparison function into a key function, with relatively little overhead. -- Steven From solipsis at pitrou.net Fri Jan 29 06:48:26 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Jan 2010 11:48:26 +0000 (UTC) Subject: which one is faster? References: <140a7cf6-b4fa-4bab-a31d-aacf2c7678c5@k2g2000pro.googlegroups.com> Message-ID: Le Thu, 28 Jan 2010 22:39:32 -0800, alex23 a ?crit?: > On Jan 29, 4:29?pm, "Stephen.Wu" <54wut... at gmail.com> wrote: >> str.find(targetStr) >> str.index(targetStr) with exception >> str.count(targetStr) >> targetStr in str >> >> which is the fastest way to check whether targetStr is in str? > [...] > >>From the looks of those results, using 'in' seems to be the fastest. To answer the question more precisely: * "in" is the fastest because it doesn't have the method call overhead. This is only a fixed cost, though, and doesn't depend on the inputs. * all four alternatives use the same underlying algorithm, *but* count() has to go to the end of the input string in order to count all occurrences. The other expressions can stop as soon as the first occurence is found, which can of course be a big win if the occurrence is near the start of the string and the string is very long So, to sum it up: * "in" is faster by a small fixed cost advantage * "find" and "index" are almost exactly equivalent * "count" will often be slower because it can't early exit Regards Antoine. From invalid at invalid.invalid Fri Jan 29 06:52:53 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Fri, 29 Jan 2010 11:52:53 +0000 (UTC) Subject: [OT] Perl 6 [was Re: myths about python 3] References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <03725fae$0$1309$c3e8da3@news.astraweb.com> Message-ID: On 2010-01-29, Neil Hodgson wrote: > Looks to me like the problem with Perl 6 was that it was too > ambitious, wanting to fix all perceived problems with the > language. I thought Python was Perl with all the perceived problems fixed. -- Grant From solipsis at pitrou.net Fri Jan 29 06:55:02 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Jan 2010 11:55:02 +0000 (UTC) Subject: [OT] myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <87tyu529tl.fsf@benfinney.id.au> <03722429$0$1309$c3e8da3@news.astraweb.com> <87ockdy7tg.fsf@benfinney.id.au> Message-ID: Le Fri, 29 Jan 2010 13:16:11 +1100, Ben Finney a ?crit?: > > I think the reason ?date? was initially used is because dates are most > familiar to us as fleshy, dark brown, wrinkled, compressed points. > > My interests in etymology and scatology unite here. Ah, I suppose it explains the strange ASCII art drawing in your signature then? (Here) | | v > -- > \ ?In the long run, the utility of all non-Free software | > `\ approaches zero. All non-Free software is a dead end.? ?Mark | > _o__) Pilgrim, 2006 | > Ben Finney From as at sci.fi Fri Jan 29 08:00:08 2010 From: as at sci.fi (Anssi Saari) Date: Fri, 29 Jan 2010 15:00:08 +0200 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <87zl3z2p63.fsf@benfinney.id.au> <4b6146b3$0$6715$9b4e6d93@newsspool2.arcor-online.net> Message-ID: Stefan Behnel writes: > 'Stable Debian' has a long tradition of being late and outdated on arrival. > That doesn't mean you can't use existing Debian packages on it. Yes, but that's beside the point. No released version of Debian ships with Python3 or even 2.6. Oh, and RHEL5 and CentOS5 ship with 2.4. From as at sci.fi Fri Jan 29 08:13:01 2010 From: as at sci.fi (Anssi Saari) Date: Fri, 29 Jan 2010 15:13:01 +0200 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: Daniel Fetchinson writes: >> 1. Python 3 is supported by major Linux distributions. >> >> FALSE - most distros are shipping with Python 2.4, or 2.5 at best. > > This latter statement is false, Fedora 11 and 12 come with python 2.6. How does your mention of one distro counter that claim? Personally, I'd like to see a study of what version of Python ships with what Linux distribution. Say, include the top 100 distros from distrowatch.com? I think there may a surprising number of distros that ship with no version of Python what so ever. From jspies at sun.ac.za Fri Jan 29 08:21:40 2010 From: jspies at sun.ac.za (Johann Spies) Date: Fri, 29 Jan 2010 15:21:40 +0200 Subject: Need help with a program In-Reply-To: <03729f14$0$1309$c3e8da3@news.astraweb.com> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> <03729f14$0$1309$c3e8da3@news.astraweb.com> Message-ID: <20100129132140.GH10729@sun.ac.za> On Fri, Jan 29, 2010 at 10:04:33AM +0000, Steven D'Aprano wrote: > > I know this is a python list but if you really want to get the job done > > quickly this is one method without writing python code: > > > > $ cat /tmp/y > > AAAAAGACTCGAGTGCGCGGA 0 > > AAAAAGATAAGCTAATTAAGCTACTGG 0 > > AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 > > AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 > > AAAAAGGTCGCCTGACGGCTGC 0 > > $ grep -v 0 /tmp/y > tmp/z > > $ cat /tmp/z > > AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 > > AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 > > That will do the wrong thing for lines like: > > AAAAAGATAAGCTAATTAAGCTACTGGGTT 10 In that case change the grep to ' 0$' then only the lines with a singel digit '0' at the end of the line will be excluded. One can do the same using regulare expressions in Python and it will probably a lot slower on large files. Regards Johann -- Johann Spies Telefoon: 021-808 4599 Informasietegnologie, Universiteit van Stellenbosch "My son, if sinners entice thee, consent thou not." Proverbs 1:10 From andrew at acooke.org Fri Jan 29 08:30:27 2010 From: andrew at acooke.org (andrew cooke) Date: Fri, 29 Jan 2010 05:30:27 -0800 (PST) Subject: Function name unchanged in error message Message-ID: Is there any way to change the name of the function in an error message? In the example below I'd like the error to refer to bar(), for example (the motivation is related function decorators - I'd like the wrapper function to give the same name) >>> def foo(): ... return 7 ... >>> foo.__name__ = 'bar' >>> foo(123) Traceback (most recent call last): File "", line 1, in TypeError: foo() takes no arguments (1 given) Thanks, Andrew From duncan.booth at invalid.invalid Fri Jan 29 08:41:38 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 29 Jan 2010 13:41:38 GMT Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: Anssi Saari wrote: > Daniel Fetchinson writes: > >>> 1. Python 3 is supported by major Linux distributions. >>> >>> FALSE - most distros are shipping with Python 2.4, or 2.5 at best. >> >> This latter statement is false, Fedora 11 and 12 come with python 2.6. > > How does your mention of one distro counter that claim? Personally, > I'd like to see a study of what version of Python ships with what > Linux distribution. Say, include the top 100 distros from > distrowatch.com? I think there may a surprising number of distros that > ship with no version of Python what so ever. > Is there any easier way to build that list than working through all the distributions by hand? Maybe someone should create a wiki page to record this. Here's what I see in the Ubuntu packages. Python 3 seems only to be in the universe repositories so far. Dapper: Python 2.4.2 Hardy: Python 2.5.2 Intrepid: Python 2.5.2, 3.0~b3 (universe) Jaunty: Python 2.6.2, 3.0.1 (universe) Karmic: Python 2.6.4rc1, 3.1 (universe) Lucid: Python 2.6.4, 3.1 (universe) WTF is rc anything doing in the main repository? -- Duncan Booth http://kupuguy.blogspot.com From jeanmichel at sequans.com Fri Jan 29 08:44:49 2010 From: jeanmichel at sequans.com (Jean-Michel Pichavant) Date: Fri, 29 Jan 2010 14:44:49 +0100 Subject: Function name unchanged in error message In-Reply-To: References: Message-ID: <4B62E651.80007@sequans.com> andrew cooke wrote: > Is there any way to change the name of the function in an error > message? In the example below I'd like the error to refer to bar(), > for example (the motivation is related function decorators - I'd like > the wrapper function to give the same name) > > >>>> def foo(): >>>> > ... return 7 > ... > >>>> foo.__name__ = 'bar' >>>> foo(123) >>>> > Traceback (most recent call last): > File "", line 1, in > TypeError: foo() takes no arguments (1 given) > > Thanks, > Andrew > In [9]: def foo(): ...: return 7 ...: In [10]: bar = foo In [11]: bar(54) TypeError: foo() takes no arguments (1 given) JM From benjamin.kaplan at case.edu Fri Jan 29 08:54:25 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Fri, 29 Jan 2010 08:54:25 -0500 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On Fri, Jan 29, 2010 at 8:13 AM, Anssi Saari wrote: > Daniel Fetchinson writes: > >>> 1. ?Python 3 is supported by major Linux distributions. >>> >>> ? ? ?FALSE - most distros are shipping with Python 2.4, or 2.5 at best. >> >> This latter statement is false, Fedora 11 and 12 come with python 2.6. > > How does your mention of one distro counter that claim? Personally, > I'd like to see a study of what version of Python ships with what > Linux distribution. Say, include the top 100 distros from > distrowatch.com? I think there may a surprising number of distros that > ship with no version of Python what so ever. > -- Just from a quick look, 6 of the top 10 distros on distrowatch have Python 2.6 including all of the top 4. > http://mail.python.org/mailman/listinfo/python-list > From duncan.booth at invalid.invalid Fri Jan 29 08:59:54 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 29 Jan 2010 13:59:54 GMT Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> <184f0182-acbc-4a85-a8d9-7f813ebd2eb0@k2g2000pro.googlegroups.com> <0372328c$0$1309$c3e8da3@news.astraweb.com> <0372b731$0$1309$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > If that's too verbose for you, stick this as a helper function in your > application: > > > def CmpToKey(mycmp): > 'Convert a cmp= function into a key= function' > class K(object): > def __init__(self, obj, *args): > self.obj = obj > __lt__ = lambda s, o: mycmp(s.obj, o.obj) == -1 > __gt__ = lambda s, o: mycmp(s.obj, o.obj) == 1 > __eq__ = lambda s, o: mycmp(s.obj, o.obj) == 0 > __le__ = lambda s, o: mycmp(s.obj, o.obj) != 1 > __ge__ = lambda s, o: mycmp(s.obj, o.obj) != -1 > __ne__ = lambda s, o: mycmp(s.obj, o.obj) != 0 > return K > > Shouldn't that be: def CmpToKey(mycmp): 'Convert a cmp= function into a key= function' class K(object): def __init__(self, obj, *args): self.obj = obj __lt__ = lambda s, o: mycmp(s.obj, o.obj) < 0 __gt__ = lambda s, o: mycmp(s.obj, o.obj) > 0 __eq__ = lambda s, o: mycmp(s.obj, o.obj) == 0 __le__ = lambda s, o: mycmp(s.obj, o.obj) <= 0 __ge__ = lambda s, o: mycmp(s.obj, o.obj) >= 0 __ne__ = lambda s, o: mycmp(s.obj, o.obj) != 0 return K After all the cmp function needn't return -1,0,1 only negative, 0, positive. And if that's still too verbose for you try: def CmpToKey(mycmp): 'Convert a cmp= function into a key= function' class K(object): def __init__(self, obj, *args): self.obj = obj __lt__ = lambda s, o: mycmp(s.obj, o.obj) < 0 return K That works on 3.1 and 2.6 though of course there's no guarantee that a future Python version won't use a different comparison operator so only use it if you like living dangerously and have problems typing. :^) -- Duncan Booth http://kupuguy.blogspot.com From clp2 at rebertia.com Fri Jan 29 09:10:59 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Fri, 29 Jan 2010 06:10:59 -0800 Subject: Function name unchanged in error message In-Reply-To: References: Message-ID: <50697b2c1001290610t1a2ac5c8hcd5f9856aa85392b@mail.gmail.com> On Fri, Jan 29, 2010 at 5:30 AM, andrew cooke wrote: > Is there any way to change the name of the function in an error > message? ?In the example below I'd like the error to refer to bar(), > for example (the motivation is related function decorators - I'd like > the wrapper function to give the same name) > >>>> def foo(): > ... ? ? return 7 > ... >>>> foo.__name__ = 'bar' >>>> foo(123) > Traceback (most recent call last): > ?File "", line 1, in > TypeError: foo() takes no arguments (1 given) It gets weirder: >>> print(foo) Cheers, Chris -- http://blog.rebertia.com From dirknbr at googlemail.com Fri Jan 29 09:22:01 2010 From: dirknbr at googlemail.com (dirknbr) Date: Fri, 29 Jan 2010 06:22:01 -0800 (PST) Subject: Python 3.1 simplejson install Message-ID: <2be8271a-f97d-4ff8-a806-c6e1d1d4adfd@n7g2000yqb.googlegroups.com> I am trying to install simplejson on Python 3.1 on Windows. When I do 'python setup.py install' I get 'except DisutilsPlatformError, x: SyntaxError' with a dash under the comma. Any ideas? Dirk From __peter__ at web.de Fri Jan 29 09:22:56 2010 From: __peter__ at web.de (Peter Otten) Date: Fri, 29 Jan 2010 15:22:56 +0100 Subject: Function name unchanged in error message References: Message-ID: andrew cooke wrote: > Is there any way to change the name of the function in an error > message? In the example below I'd like the error to refer to bar(), > for example (the motivation is related function decorators - I'd like > the wrapper function to give the same name) > >>>> def foo(): > ... return 7 > ... >>>> foo.__name__ = 'bar' >>>> foo(123) > Traceback (most recent call last): > File "", line 1, in > TypeError: foo() takes no arguments (1 given) The name is looked up in the code object. As that is immutable you have to make a new one: argnames = 'argcount nlocals stacksize flags code consts names varnames filename name firstlineno lnotab'.split() def f(): return 42 code = type(f.func_code) function = type(f) def make_code(proto, **kw): for name in argnames: if name not in kw: kw[name] = getattr(proto, "co_" + name) values = [kw[name] for name in argnames] return code(*values) if __name__ == "__main__": def foo(): print "foo" c = make_code(foo.func_code, name="bar") foo.func_code = c foo(42) Peter From darcy at druid.net Fri Jan 29 09:32:52 2010 From: darcy at druid.net (D'Arcy J.M. Cain) Date: Fri, 29 Jan 2010 09:32:52 -0500 Subject: Need help with a program In-Reply-To: <20100129092354.GG1718@sun.ac.za> References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> <20100129092354.GG1718@sun.ac.za> Message-ID: <20100129093252.204146d0.darcy@druid.net> On Fri, 29 Jan 2010 11:23:54 +0200 Johann Spies wrote: > I know this is a python list but if you really want to get the job > done quickly this is one method without writing python code: > [...] > $ grep -v 0 /tmp/y > tmp/z There's plenty of ways to do it without writing Python. C, C++, Perl, Forth, Awk, BASIC, Intercal, etc. So what? Besides, your solution doesn't work. You want "grep -vw 0 /tmp/y > tmp/z" and even then it doesn't meet the requirements. It extracts the lines the OP wants but doesn't reformat them. It also assumes a Unix system or at least something with grep installed so it isn't portable. If you want to see how the same task can be done in many different languages see http://www.roesler-ac.de/wolfram/hello.htm. -- D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner. From __peter__ at web.de Fri Jan 29 09:33:11 2010 From: __peter__ at web.de (Peter Otten) Date: Fri, 29 Jan 2010 15:33:11 +0100 Subject: Python 3.1 simplejson install References: <2be8271a-f97d-4ff8-a806-c6e1d1d4adfd@n7g2000yqb.googlegroups.com> Message-ID: dirknbr wrote: > I am trying to install simplejson on Python 3.1 on Windows. When I do > 'python setup.py install' I get 'except DisutilsPlatformError, x: > SyntaxError' with a dash under the comma. You are trying to install a package written for Python 2.x on 3.x, and some of the 2.x Syntax is illegal in Python 3. I recommend that you use the json module instead that is already included in the 3.1 distribution, see http://docs.python.org/3.1/library/json.html Peter From austin.bingham at gmail.com Fri Jan 29 09:37:09 2010 From: austin.bingham at gmail.com (Austin Bingham) Date: Fri, 29 Jan 2010 15:37:09 +0100 Subject: Some C-API functions clear the error indicator? Message-ID: I've noticed that several (many?) python functions seem to clear the error/exception indicators when they're called from a C/C++ program. For example, both PyImport_ImportModule and traceback.extract_tb() (called via the function call methods) do this: if error indicators are set prior to their call (as indicated by PyErr_Fetch, and including a call to PyErr_Restore), I see that they are unset (using the same method) after the call. This happens even when the functions succeed. The functions that do this don't seem to indicate in their documentation that this will happen. So first, does anyone know why this is happening? Is it because of the context in which I'm making the calls? Is there any pattern or reason behind which functions will do this? Or am I just doing something wrong? If the problem is context-dependent (e.g. I haven't properly established a call stack, or something of that flavor), any pointers on doing things properly would be great. Here's some example code demonstrating the problem: --- #include int main(int argc, char** argv) { Py_Initialize(); // Cause an IndexError PyObject* list = PyList_New(0); PyObject* obj = PyList_GetItem(list, 100); PyObject *t = NULL, *v = NULL, *tb = NULL; // Verify that we see the error PyErr_Fetch(&t, &v, &tb); assert(t); PyErr_Restore(t, v, tb); // Import a module, which seems to be clearing the error indicator PyObject* mod = PyImport_ImportModule("sys"); assert(PyObject_HasAttrString(mod, "path")); // Verify that the error indicator has been cleared PyErr_Fetch(&t, &v, &tb); assert(!t); // <=== The error is gone! PyErr_Restore(t, v, tb); Py_Finalize(); return 0; } --- Thanks in advance. Austin From lists at cheimes.de Fri Jan 29 09:39:06 2010 From: lists at cheimes.de (Christian Heimes) Date: Fri, 29 Jan 2010 15:39:06 +0100 Subject: Python 3.1 simplejson install In-Reply-To: <2be8271a-f97d-4ff8-a806-c6e1d1d4adfd@n7g2000yqb.googlegroups.com> References: <2be8271a-f97d-4ff8-a806-c6e1d1d4adfd@n7g2000yqb.googlegroups.com> Message-ID: dirknbr wrote: > I am trying to install simplejson on Python 3.1 on Windows. When I do > 'python setup.py install' I get 'except DisutilsPlatformError, x: > SyntaxError' with a dash under the comma. There is no need to install simplejson yourself. Python 3 and 2.6 already have a JSON package called "json". Christian From pruebauno at latinmail.com Fri Jan 29 09:41:27 2010 From: pruebauno at latinmail.com (nn) Date: Fri, 29 Jan 2010 06:41:27 -0800 (PST) Subject: Need help with a program References: <96db4cd6-bda3-42db-a592-83d72e517e2d@28g2000vbf.googlegroups.com> Message-ID: <89a02db5-4e9f-4c0e-937c-b37ddea90283@l30g2000yqb.googlegroups.com> Johann Spies wrote: > On Thu, Jan 28, 2010 at 07:07:04AM -0800, evilweasel wrote: > > Hi folks, > > > > I am a newbie to python, and I would be grateful if someone could > > point out the mistake in my program. Basically, I have a huge text > > file similar to the format below: > > > > AAAAAGACTCGAGTGCGCGGA 0 > > AAAAAGATAAGCTAATTAAGCTACTGG 0 > > AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 > > AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 > > AAAAAGGTCGCCTGACGGCTGC 0 > > I know this is a python list but if you really want to get the job > done quickly this is one method without writing python code: > > $ cat /tmp/y > AAAAAGACTCGAGTGCGCGGA 0 > AAAAAGATAAGCTAATTAAGCTACTGG 0 > AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 > AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 > AAAAAGGTCGCCTGACGGCTGC 0 > $ grep -v 0 /tmp/y > tmp/z > $ cat /tmp/z > AAAAAGATAAGCTAATTAAGCTACTGGGTT 1 > AAAAAGGGGGCTCACAGGGGAGGGGTAT 1 > > Regards > Johann > -- > Johann Spies Telefoon: 021-808 4599 > Informasietegnologie, Universiteit van Stellenbosch > > "My son, if sinners entice thee, consent thou not." > Proverbs 1:10 I would rather use awk for this: awk 'NF==2 && $2!~/^0$/ {printf("seq%s\n%s\n",NR,$1)}' dnain.dat but I think that is getting a bit off topic... From clp2 at rebertia.com Fri Jan 29 09:41:28 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Fri, 29 Jan 2010 06:41:28 -0800 Subject: Python 3.1 simplejson install In-Reply-To: References: <2be8271a-f97d-4ff8-a806-c6e1d1d4adfd@n7g2000yqb.googlegroups.com> Message-ID: <50697b2c1001290641p646d23d3r412179d4c28ec5a7@mail.gmail.com> On Fri, Jan 29, 2010 at 6:33 AM, Peter Otten <__peter__ at web.de> wrote: > dirknbr wrote: >> I am trying to install simplejson on Python 3.1 on Windows. When I do >> 'python setup.py install' I get 'except DisutilsPlatformError, x: >> SyntaxError' with a dash under the comma. > > You are trying to install a package written for Python 2.x on 3.x, and some > of the 2.x Syntax is illegal in Python 3. To be specific (http://www.python.org/dev/peps/pep-3110/): "the ambiguous except A, B: which would mean different things in Python 2.x and 3.x -- leading to hard-to-catch bugs -- cannot legally occur in 3.x code." The except statement in your error would be properly written in 3.x as: except DisutilsPlatformError as x: Cheers, Chris -- http://blog.rebertia.com From exarkun at twistedmatrix.com Fri Jan 29 09:50:34 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Fri, 29 Jan 2010 14:50:34 -0000 Subject: Function name unchanged in error message In-Reply-To: <50697b2c1001290610t1a2ac5c8hcd5f9856aa85392b@mail.gmail.com> References: <50697b2c1001290610t1a2ac5c8hcd5f9856aa85392b@mail.gmail.com> Message-ID: <20100129145034.1898.1276929215.divmod.xquotient.937@localhost.localdomain> On 02:10 pm, clp2 at rebertia.com wrote: >On Fri, Jan 29, 2010 at 5:30 AM, andrew cooke >wrote: >>Is there any way to change the name of the function in an error >>message? ?In the example below I'd like the error to refer to bar(), >>for example (the motivation is related function decorators - I'd like >>the wrapper function to give the same name) >>>>>def foo(): >>... ? ? return 7 >>... >>>>>foo.__name__ = 'bar' >>>>>foo(123) >>Traceback (most recent call last): >>?File "", line 1, in >>TypeError: foo() takes no arguments (1 given) > >It gets weirder: >>>>print(foo) > The name is represented in (at least) two places, on the function object and on the code object: >>> def foo(): pass ... >>> foo.func_name 'foo' >>> foo.func_code.co_name 'foo' >>> foo.func_name = 'bar' >>> foo >>> foo.func_code.co_name = 'baz' Traceback (most recent call last): File "", line 1, in TypeError: readonly attribute >>> new.function and new.code will let you construct new objects with different values (and copying over whichever existing attributes you want to preserve). Jean-Paul From duncan.booth at invalid.invalid Fri Jan 29 09:58:42 2010 From: duncan.booth at invalid.invalid (Duncan Booth) Date: 29 Jan 2010 14:58:42 GMT Subject: Some C-API functions clear the error indicator? References: Message-ID: Austin Bingham wrote: > The functions that do this don't seem to indicate in their > documentation that this will happen. So first, does anyone know why > this is happening? Is it because of the context in which I'm making > the calls? Is there any pattern or reason behind which functions will > do this? Or am I just doing something wrong? > (Just guessing here) I would expect that any function that executes Python code will clear the error. I think that has to happen otherwise the Python code will throw an exception whenever it gets round to checking for errors. In the past I've found that if you fail to check for an error in C code before returning to the interpreter you get the exception thrown a few instructions later, so something similar would probably happen if you call other Python code from C. If it is anything that executes Python then that would include any function that creates or destroys an object with Python constructor or destructor code. or that compares or otherwise operates on instances defined in Python. In particular it might mean that any function that doesn't appear to clear the error could do so in a slightly different situation. -- Duncan Booth http://kupuguy.blogspot.com From mattsteel at hotmail.it Fri Jan 29 10:32:58 2010 From: mattsteel at hotmail.it (Mattsteel) Date: Fri, 29 Jan 2010 07:32:58 -0800 (PST) Subject: Funny behaviour with __future__ and doctest between 2.6 and 3.1 Message-ID: Hello all. I'm using Python 2.6.4 and Python 3.1.1. My wish is to code in a 3.1-compliant way using 2.6, so I'm importing the __future__ module. I've found a funny thing comparing the two folliwing snippets that differ for one line only, that is the position of __future__ import (before or after the doc string). Well, I understand the subtle difference but still I wander what really happen behind the scenes. Comments are welcome. --------------------------------------- #!/usr/bin/env python ''' >>> concat('hello','world') 'hello world' ''' from __future__ import unicode_literals def concat( first, second ): return first + ' ' + second if __name__ == "__main__": import doctest doctest.testmod() --------------------------------------- #!/usr/bin/env python from __future__ import unicode_literals ''' >>> concat('hello','world') 'hello world' ''' def concat( first, second ): return first + ' ' + second if __name__ == "__main__": import doctest doctest.testmod() --------------------------------------- The first way shows the following failure: --------------------------------------- Failed example: concat('hello','world') Expected: 'hello world' Got: u'hello world' --------------------------------------- Regards. Matt. From paul at boddie.org.uk Fri Jan 29 10:34:31 2010 From: paul at boddie.org.uk (Paul Boddie) Date: Fri, 29 Jan 2010 07:34:31 -0800 (PST) Subject: python 3's adoption References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> Message-ID: <35eac267-620a-410d-9a01-ae2b3c5790d7@e37g2000yqn.googlegroups.com> On 29 Jan, 06:56, Terry Reedy wrote: > On 1/28/2010 6:47 PM, Paul Boddie wrote: > > > What would annoy me if I used Python 3.x would be the apparent lack of > > the __cmp__ method for conveniently defining comparisons between > > instances of my own classes. Having to define all the rich comparison > > methods frequently isn't even as much fun as it sounds. > > You define __eq__, which automatically generates __ne__. >From the Python 3.1 language reference: "There are no implied relationships among the comparison operators. The truth of x==y does not imply that x!=y is false. Accordingly, when defining __eq__(), one should also define __ne__() so that the operators will behave as expected." Maybe Python 3.1 plugs a default method in, anyway. > You define __lt__, which is all sort and heap need. > This should be about as easier than __eq__, which is really needed, and > __cmp__. If you need the other 3, either copy the recipe in the > Cookbook, or, even faster > > def __ge__(s,o): return o.__lt__(s) > def __le__(s,o): return s < o or s == o > def __ge__(s,o): return s > o or s == o Spot the error in the above. ;-) Of course, this could be defined in a base class and be inherited everywhere, but the case that made me raise this issue actually involved instances of a class delegating comparison/sorting to another object. With __cmp__ this can be done concisely and only involve the delegation of one method - not so with the rich comparison "protocol". Paul From alfps at start.no Fri Jan 29 10:35:50 2010 From: alfps at start.no (Alf P. Steinbach) Date: Fri, 29 Jan 2010 16:35:50 +0100 Subject: python 3's adoption In-Reply-To: <0372b731$0$1309$c3e8da3@news.astraweb.com> References: <2ca5d639-73bc-4e67-80ef-0a8f44574536@f12g2000yqn.googlegroups.com> <35212615-0aeb-492b-88ab-1ee62b6677bd@b10g2000yqa.googlegroups.com> <184f0182-acbc-4a85-a8d9-7f813ebd2eb0@k2g2000pro.googlegroups.com> <0372328c$0$1309$c3e8da3@news.astraweb.com> <0372b731$0$1309$c3e8da3@news.astraweb.com> Message-ID: * Steven D'Aprano: > On Fri, 29 Jan 2010 07:10:01 +0100, Alf P. Steinbach wrote: > >> >>> L = ["?", "?", "?"] # This is in SORTED ORDER in Norwegian L > > [...] > >> >>> L.sort( key = locale.strxfrm ) >> >>> L >> ['?', '?', '?'] >> >>> locale.strcoll( "?", "?" ) >> 1 >> >>> locale.strcoll( "?", "?" ) >> -1 >> >> Note that strcoll correctly orders the strings as ["?", "?", "?"], that >> is, it would have if it could have been used as cmp function to sort (or >> better, to a separate routine named e.g. custom_sort). > > This is in Python2.5, so I'm explicitly specifying unicode strings: > >>>> L = [u"?", u"?", u"?"] >>>> assert sorted(L) == [u'?', u'?', u'?'] > > The default C-locale sorting of L does not equal to L. Now let's change > to Norwegian locale: > >>>> import locale >>>> locale.setlocale(locale.LC_ALL, 'nb_NO') > 'nb_NO' >>>> print u''.join(sorted(L, cmp=locale.strcoll)) > ??? > > So far so good, we've confirmed that in Python 2.5 we can sort in a > locale-aware form. Now, can we do this with a key function? Thanks to > Raymond Hettinger's recipe here: > > http://code.activestate.com/recipes/576653/ > > >>>> print u''.join(sorted(L, key=CmpToKey(locale.strcoll))) > ??? > > > Success! > > Let's try it in Python 3.1: > >>>> L = ["?", "?", "?"] >>>> assert sorted(L) == ['?', '?', '?'] >>>> >>>> import locale >>>> locale.setlocale(locale.LC_ALL, 'nb_NO') > 'nb_NO' Hm. A bit off-topic, but... >>> import locale >>> locale.setlocale(locale.LC_ALL, 'nb_NO') Traceback (most recent call last): File "", line 1, in File "C:\Program Files\cpython\python31\lib\locale.py", line 527, in return _setlocale(category, locale) locale.Error: unsupported locale setting >>> _ This on a machine where the Python default locale is Norwegian. >>>> ''.join(sorted(L, key=CmpToKey(locale.strcoll))) > '???' > > > The definition of CmpToKey can be found at the URL above. It's not very > exciting, but here it is: > > > def CmpToKey(mycmp): > 'Convert a cmp= function into a key= function' > class K(object): > def __init__(self, obj, *args): > self.obj = obj > def __lt__(self, other): > return mycmp(self.obj, other.obj) == -1 > def __gt__(self, other): > return mycmp(self.obj, other.obj) == 1 > def __eq__(self, other): > return mycmp(self.obj, other.obj) == 0 > def __le__(self, other): > return mycmp(self.obj, other.obj) != 1 > def __ge__(self, other): > return mycmp(self.obj, other.obj) != -1 > def __ne__(self, other): > return mycmp(self.obj, other.obj) != 0 > return K This is pretty smart as a generic solution. Thanks! :-) I was thinking more of sticking those comparisions in some custom string class or such, which would be rather ugly... The above does have quite high overhead though! That is, it's /inefficient/ compared to using a 'cmp' function directly. > If that's too verbose for you, stick this as a helper function in your > application: > > > def CmpToKey(mycmp): > 'Convert a cmp= function into a key= function' > class K(object): > def __init__(self, obj, *args): > self.obj = obj > __lt__ = lambda s, o: mycmp(s.obj, o.obj) == -1 > __gt__ = lambda s, o: mycmp(s.obj, o.obj) == 1 > __eq__ = lambda s, o: mycmp(s.obj, o.obj) == 0 > __le__ = lambda s, o: mycmp(s.obj, o.obj) != 1 > __ge__ = lambda s, o: mycmp(s.obj, o.obj) != -1 > __ne__ = lambda s, o: mycmp(s.obj, o.obj) != 0 > return K > > > [...] >> The above may just be a bug in the 3.x stxfrm. But it illustrates that >> sometimes you have your sort order defined by a comparision function. >> Transforming that into a key can be practically impossible (it can also >> be quite inefficient). > > This might be true, but you haven't demonstrated it. The "can be ... practically impossible" was hogwash. I posted late, sorry. The "quite inefficient" holds. > With one little > helper function, you should be able to convert any comparison function > into a key function, with relatively little overhead. I wouldn't call an extra Python method call per comparision "relatively little overhead". Note that the same number of comparisions as with a 'cmp' based sort is performed. But with the wrapper every comparision is indirected through a Python method call (I think, but not sure, that the creation of those comparision objects does not introduce significant overhead, but the calls must). # -*- coding: utf-8 -*- from __future__ import print_function from __future__ import unicode_literals try: range = xrange except: pass import locale import sys import random import timeit def CmpToKey( mycmp ): "Convert a cmp= function into a key= function" class K( object ): def __init__( self, obj, *args ): self.obj = obj def __lt__( self, other ): return mycmp( self.obj, other.obj ) == -1 def __gt__( self, other ): return mycmp( self.obj, other.obj ) == 1 def __eq__( self, other ): return mycmp( self.obj, other.obj ) == 0 def __le__( self, other ): return mycmp( self.obj, other.obj ) != 1 def __ge__( self, other ): return mycmp( self.obj, other.obj ) != -1 def __ne__( self, other ): return mycmp( self.obj, other.obj ) != 0 return K def elapsed_time_for( f ): n_calls = 1 return timeit.timeit( f, number = n_calls ) def major_pyversion(): return sys.version_info[0] def sorting_of( string_list ): def sort2x(): string_list.sort( cmp = locale.strcoll ) def sort3x(): string_list.sort( key = CmpToKey( locale.strcoll ) ) return sort2x if major_pyversion() <= 2 else sort3x alphabet = "abcdefghijklmnopqrstuvwxyz???" n_strings = int( 1e5 ) nbno_locale = "" # "nb_NO" raises exception on my machine. def rnd_string(): len = random.randint( 1, 80 ) s = "" for i in range( len ): s = s + random.choice( alphabet ) return s locale.setlocale( locale.LC_ALL, nbno_locale ) random.seed( 666 ) assert( rnd_string() == "?moxqudxlsz?a?cte?rmuoc?jrd?pbvyabwhn" ) print( "Creating array of pseudo-random strings..." ) L = [rnd_string() for i in range( n_strings )] print( "Sorting..." ) time = elapsed_time_for( sorting_of( L ) ) print( "Last string is '{0}'".format( L[-1] ) ) print( "{0}".format( time ) ) assert( L[-1][0] == "?" ) C:\Documents and Settings\Alf\test> py3 sort.py Creating array of pseudo-random strings... Sorting... Last string is '???ywxuybrxgvnstoyvcmd?jdlnxwwfsbnzsinxncmtrxuxgtqtduzv?ixqovtjxiflfesuvfa' 8.45987772189 C:\Documents and Settings\Alf\test> py2 sort.py Creating array of pseudo-random strings... Sorting... Last string is '???ywxuybrxgvnstoyvcmd?jdlnxwwfsbnzsinxncmtrxuxgtqtduzv?ixqovtjxiflfesuvfa' 3.41336790011 C:\Documents and Settings\Alf\test> _ In the run above Python 3.x was only roughly 2.5 times slower. On my machine it ranges from 2.5 times slower to 3 times slower. Most often around 2.5 x slower. Of course this does not measure how much faster a Python 3.x cmp/"<" based sort /could be/, only how slow, sluggish and snail-like it is compared to 2.x. :-) In other words, that's the speed price, 2.5 to 3 times slower sorting, for ditching cmp instead of e.g. replacing it with something better. Cheers & hth., - Alf From Eric_Dexter at msn.com Fri Jan 29 10:48:43 2010 From: Eric_Dexter at msn.com (Eric_Dexter at msn.com) Date: Fri, 29 Jan 2010 07:48:43 -0800 (PST) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <8a830b83-d949-4af1-b0d7-25f68ec5f47a@e25g2000yqh.googlegroups.com> On Jan 27, 2:56?pm, John Nagle wrote: > Daniel Fetchinson wrote: > > Hi folks, > > > I was going to write this post for a while because all sorts of myths > > periodically come up on this list about python 3. I don't think the > > posters mean to spread false information on purpose, they simply are > > not aware of the facts. > > > My list is surely incomplete, please feel free to post your favorite > > misconception about python 3 that people periodically state, claim or > > ask about. > > Myths about Python 3: > > 1. ?Python 3 is supported by major Linux distributions. > > ? ? ? ? FALSE - most distros are shipping with Python 2.4, or 2.5 at best. > > 2. ?Python 3 is supported by multiple Python implementations. > > ? ? ? ? FALSE - Only CPython supports 3.x. ?Iron Python, Unladen Swallow, > ? ? ? ? PyPy, and Jython have all stayed with 2.x versions of Python. > > 3. ?Python 3 is supported by most 3rd party Python packages. > > ? ? ? ? FALSE - it's not supported by MySQLdb, OpenSSL, feedparser, etc. > > Arguably, Python 3 has been rejected by the market. ?Instead, there's > now Python 2.6, Python 2.7, and Python 2.8. ?Python 3 has turned into > a debacle like Perl 6, now 10 years old. > > That's the reality, Python 3 fanboys. > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? John Nagle I am seeing alot of bleeding edge linux distro's with 2.5 and 2.6 I would perfer they stayed with 2.4 or 2.5 .. or if there is a version that I can come up with a convincing cross- platform os.startfile I would want to use a vmware version of that linux. _______________________________________ http://dextracker.blogspot.com/ From __peter__ at web.de Fri Jan 29 10:50:12 2010 From: __peter__ at web.de (Peter Otten) Date: Fri, 29 Jan 2010 16:50:12 +0100 Subject: Funny behaviour with __future__ and doctest between 2.6 and 3.1 References: Message-ID: Mattsteel wrote: > Hello all. > I'm using Python 2.6.4 and Python 3.1.1. > My wish is to code in a 3.1-compliant way using 2.6, so I'm importing > the __future__ module. > I've found a funny thing comparing the two folliwing snippets that > differ for one line only, that is the position of __future__ import > (before or after the doc string). > > Well, I understand the subtle difference but still I wander what > really happen behind the scenes. Are you sure? The second script has no module docstring, just a string literal somewhere in the module, and therefore no tests. You can see that by running it with the verbose option -v. Also, from __future__ import unicode_literals doesn't affect the repr() of a unicode instance. But the interactive interpreter invokes repr() on the result before it is printed: >>> from __future__ import unicode_literals >>> "yadda" u'yadda' > Comments are welcome. > > --------------------------------------- > #!/usr/bin/env python > ''' > >>> concat('hello','world') > 'hello world' > ''' > from __future__ import unicode_literals > def concat( first, second ): > return first + ' ' + second > if __name__ == "__main__": > import doctest > doctest.testmod() > --------------------------------------- > #!/usr/bin/env python > from __future__ import unicode_literals > ''' > >>> concat('hello','world') > 'hello world' > ''' > def concat( first, second ): > return first + ' ' + second > if __name__ == "__main__": > import doctest > doctest.testmod() > --------------------------------------- > > > The first way shows the following failure: > > --------------------------------------- > Failed example: > concat('hello','world') > Expected: > 'hello world' > Got: > u'hello world' > > --------------------------------------- > > Regards. > > Matt. From thomasmallen at gmail.com Fri Jan 29 10:54:23 2010 From: thomasmallen at gmail.com (Thomas Allen) Date: Fri, 29 Jan 2010 07:54:23 -0800 (PST) Subject: SimpleXMLRPCServer daemon Message-ID: <1e689e5c-8e2a-4896-b867-717316fa783e@o26g2000vbd.googlegroups.com> I have a script that runs an instance of SimpleXMLRPCServer and in general it works as expected. In its __del__, it is supposed to clean up its PID file (written on boot). I have two problems with this server instance: The first is that tt doesn't always clean up its PID file; is there a more reliable way to do this than how I am currently? The second is that when it does crash, I don't know about it...what would be sufficient as a "keep-alive" script to restart it? I suppose I could use something like EventMachine (already installed on my server) to watch the PID file if it were deleted reliably. Thomas Allen From awilliam at opengroupware.us Fri Jan 29 11:07:07 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Fri, 29 Jan 2010 11:07:07 -0500 Subject: SimpleXMLRPCServer daemon In-Reply-To: <1e689e5c-8e2a-4896-b867-717316fa783e@o26g2000vbd.googlegroups.com> References: <1e689e5c-8e2a-4896-b867-717316fa783e@o26g2000vbd.googlegroups.com> Message-ID: <1264781227.6660.17.camel@linux-m3mt> On Fri, 2010-01-29 at 07:54 -0800, Thomas Allen wrote: > I have a script that runs an instance of SimpleXMLRPCServer and in > general it works as expected. In its __del__, it is supposed to clean > up its PID file (written on boot). I have two problems with this > server instance: The first is that tt doesn't always clean up its PID > file; is there a more reliable way to do this than how I am currently? > The second is that when it does crash, I don't know about it...what > would be sufficient as a "keep-alive" script to restart it? I suppose > I could use something like EventMachine (already installed on my > server) to watch the PID file if it were deleted reliably. Why don't you wrap the server in a try/except block? Then if an exception occurs killing the server you can automatically restart and/or send an e-mail to an admin address. From michele.simionato at gmail.com Fri Jan 29 11:09:40 2010 From: michele.simionato at gmail.com (Michele Simionato) Date: Fri, 29 Jan 2010 08:09:40 -0800 (PST) Subject: Function name unchanged in error message References: Message-ID: On Jan 29, 2:30?pm, andrew cooke wrote: > Is there any way to change the name of the function in an error > message? ?In the example below I'd like the error to refer to bar(), > for example (the motivation is related function decorators - I'd like > the wrapper function to give the same name) Use the decorator module which does the right thing: http://pypi.python.org/pypi/decorator From mattsteel at hotmail.it Fri Jan 29 11:21:55 2010 From: mattsteel at hotmail.it (Mattsteel) Date: Fri, 29 Jan 2010 08:21:55 -0800 (PST) Subject: Funny behaviour with __future__ and doctest between 2.6 and 3.1 References: Message-ID: <3a02ac42-337b-4778-99e0-1309e012163d@g29g2000yqe.googlegroups.com> Hi Peter. Sadly (for me), you're right... then the only way to use doctest to work both in 2.6 and 3.1 (without modifications between them) is something like this: #!/usr/bin/env python ''' >>> str(concat('hello','world')) 'hello world' ''' from __future__ import unicode_literals def concat( first, second ): return first + ' ' + second if __name__ == "__main__": import doctest doctest.testmod() Is there any way to avoid using str(...) to protect the string? M. --------------------------------------- #!/usr/bin/env python ''' >>> concat('hello','world') 'hello world' ''' from __future__ import unicode_literals def concat( first, second ): return first + ' ' + second if __name__ == "__main__": import doctest doctest.testmod() On 29 Gen, 16:50, Peter Otten <__pete... at web.de> wrote: > Mattsteel wrote: > > Hello all. > > I'm using Python 2.6.4 and Python 3.1.1. > > My wish is to code in a 3.1-compliant way using 2.6, so I'm importing > > the __future__ module. > > I've found a funny thing comparing the two folliwing snippets that > > differ for one line only, that is the position of __future__ import > > (before or after the doc string). > > > Well, I understand the subtle difference but still I wander what > > really happen behind the scenes. > > Are you sure? The second script has no module docstring, just a string > literal somewhere in the module, and therefore no tests. You can see that by > running it with the verbose option -v. Also, > > from __future__ import unicode_literals > > doesn't affect the repr() of a unicode instance. But the interactive > interpreter invokes repr() on the result before it is printed: > > >>> from __future__ import unicode_literals > >>> "yadda" > > u'yadda' > > > Comments are welcome. > > > --------------------------------------- > > ? ? #!/usr/bin/env python > > ? ? ''' > > ? ? >>> concat('hello','world') > > ? ? 'hello world' > > ? ? ''' > > ? ? from __future__ ?import unicode_literals > > ? ? def concat( first, second ): > > ? ? ? ? return first + ' ' + second > > ? ? if __name__ == "__main__": > > ? ? ? ? import doctest > > ? ? ? ? doctest.testmod() > > --------------------------------------- > > ? ? #!/usr/bin/env python > > ? ? from __future__ ?import unicode_literals > > ? ? ''' > > ? ? >>> concat('hello','world') > > ? ? 'hello world' > > ? ? ''' > > ? ? def concat( first, second ): > > ? ? ? ? return first + ' ' + second > > ? ? if __name__ == "__main__": > > ? ? ? ? import doctest > > ? ? ? ? doctest.testmod() > > --------------------------------------- > > > The first way shows the following failure: > > > --------------------------------------- > > ? Failed example: > > ? ? ? concat('hello','world') > > ? Expected: > > ? ? ? 'hello world' > > ? Got: > > ? ? ? u'hello world' > > > --------------------------------------- > > > Regards. > > > Matt. > > From __peter__ at web.de Fri Jan 29 11:30:30 2010 From: __peter__ at web.de (Peter Otten) Date: Fri, 29 Jan 2010 17:30:30 +0100 Subject: Funny behaviour with __future__ and doctest between 2.6 and 3.1 References: <3a02ac42-337b-4778-99e0-1309e012163d@g29g2000yqe.googlegroups.com> Message-ID: Mattsteel wrote: > Sadly (for me), you're right... then the only way to use doctest to > work both in 2.6 and 3.1 (without modifications between them) is > something like this: > > #!/usr/bin/env python > ''' >>>> str(concat('hello','world')) > 'hello world' > ''' > from __future__ import unicode_literals > def concat( first, second ): > return first + ' ' + second > if __name__ == "__main__": > import doctest > doctest.testmod() > > Is there any way to avoid using str(...) to protect the string? > M. I think you can work around the problem. The following should pass in Python 2.6 and 3.1: ''' >>> concat('hello','world') == 'hello world' True ''' from __future__ import unicode_literals def concat( first, second ): return first + ' ' + second if __name__ == "__main__": import doctest doctest.testmod() Peter From tim.wintle at teamrubber.com Fri Jan 29 11:35:18 2010 From: tim.wintle at teamrubber.com (Tim Wintle) Date: Fri, 29 Jan 2010 16:35:18 +0000 Subject: SimpleXMLRPCServer daemon In-Reply-To: <1e689e5c-8e2a-4896-b867-717316fa783e@o26g2000vbd.googlegroups.com> References: <1e689e5c-8e2a-4896-b867-717316fa783e@o26g2000vbd.googlegroups.com> Message-ID: <1264782918.28992.3.camel@localhost> On Fri, 2010-01-29 at 07:54 -0800, Thomas Allen wrote: > The second is that when it does crash, I don't know about it...what > would be sufficient as a "keep-alive" script to restart it? I suppose > I could use something like EventMachine (already installed on my > server) to watch the PID file if it were deleted reliably. If the server crashes then it clearly won't get around to deleting it's pid file. The way I do it is to use os.kill (with signal 0 IIRC) to check if the process is still alive when the script starts. If it's not then I delete the pid file and carry on starting up. Tim From aharrisreid at googlemail.com Fri Jan 29 11:37:39 2010 From: aharrisreid at googlemail.com (Alan Harris-Reid) Date: Fri, 29 Jan 2010 16:37:39 +0000 Subject: Threading issue with SQLite Message-ID: <4B630ED3.5000706@googlemail.com> Hi, I am creating a web application (using Python 3.1 and CherryPy 3.2) where a SQLite connection and cursor object are created using the following code (simplified from the original): class MainSite: con = sqlite.connect('MyDatabase.db') cursor = con.cursor() def index_page(): some HTML code cursor.execute(some SQL statement) more HTML code def another_page(): some HTML code cursor.execute(anotherSQL statement) more HTML code When I call a URL which launches the another_page() method I get the error message "sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread." Questions... 1. Is there a large overhead in opening a new SQLite connection for each thread (ie. within each method)? 2. Is there any way to use the same connection for the whole class (or should I forget that idea completely?) 3. When a method returns to the calling method, is the connection automatically closed (assuming the object is local, of course) or does it have to be done explicitly using connection.close()? TIA, Alan Harris-Reid From mr.spoon21 at gmail.com Fri Jan 29 11:53:43 2010 From: mr.spoon21 at gmail.com (Mr.SpOOn) Date: Fri, 29 Jan 2010 16:53:43 +0000 Subject: Keyboard input Message-ID: <8f67b6f81001290853i39bab1acs49d02ac5be948d13@mail.gmail.com> Hi, I need to get keyboard input in a python program. I need it to let the user choose some options, for example: 1) option 1 2) option 2 3) option 3 and then to input some data to the program. I'm using the raw_input method and it works fine, but I noted that I can't use backspace and when I accidentally press shift space (because I need to input uppercase letters separated by a space) it writes some strange characters. So, is there another way to get keyboard input? Thanks, Carlo From mrm at unknown.nospam Fri Jan 29 12:00:34 2010 From: mrm at unknown.nospam (Mr.M) Date: Fri, 29 Jan 2010 17:00:34 GMT Subject: get PyObject* knowing its string name Message-ID: I think this sounds like a stupid question, but I searched the C/Api doc and google but I wasn't able to find any hint: how can I retrive PyObject pointer if I only know it's name? What I'd like to do is something like this: [code] PyObject* obj = PyFindWhatImLookingFor("w.z.y.x"); [/code] Of course if I know that w is a module, z is a class and so on I can search w dictionary than z members than... But I wonder if there is a way to retrive without knowing what w.z.y.x means. Thank you, Luca. From morganhein at gmail.com Fri Jan 29 12:13:28 2010 From: morganhein at gmail.com (JohnnyFive) Date: Fri, 29 Jan 2010 09:13:28 -0800 (PST) Subject: Sleep timer but still responsive? References: Message-ID: <8c76dbf2-8853-48cc-9ca9-37929936f73a@b35g2000vbc.googlegroups.com> On Jan 28, 4:55?pm, "Gabriel Genellina" wrote: > Please provide more details. What do you want your program to do while ? > sleeping? What kind of actions do you want a response to? > Do you have a GUI? A curses-based interfase? > > -- > Gabriel Genellina My app is purely console based. I just don't want the console to lock up (on Windows using time.sleep(x) causes the console to become unresponsive until the timer is done), and I want people to be able to CTRL+C to stop the script if need be (which can't be done if it's unresponsive!). Thanks. From jakecjacobson at gmail.com Fri Jan 29 12:25:19 2010 From: jakecjacobson at gmail.com (jakecjacobson) Date: Fri, 29 Jan 2010 09:25:19 -0800 (PST) Subject: Processing XML File Message-ID: <6bac2d7b-509d-4f92-912f-5e8dc68c3f30@36g2000yqu.googlegroups.com> I need to take a XML web resource and split it up into smaller XML files. I am able to retrieve the web resource but I can't find any good XML examples. I am just learning Python so forgive me if this question has been answered many times in the past. My resource is like: ... ... ... ... So in this example, I would need to output 2 files with the contents of each file what is between the open and close document tag. From andrej.mitrovich at gmail.com Fri Jan 29 12:27:06 2010 From: andrej.mitrovich at gmail.com (Andrej Mitrovic) Date: Fri, 29 Jan 2010 09:27:06 -0800 (PST) Subject: Changing tab behavior in Python Interactive mode Message-ID: <93ae66b7-7fe2-4542-b8d0-039f9423cdd8@n7g2000yqb.googlegroups.com> I've noticed that when running Python in interactive mode (via cmd on windows), the first time I hit tab it will move 4 spaces to the right, however each new tab will move 8 spaces instead of 4. Why this inconsistent behavior? And how could I change this to be consistent and always move only 4 spaces? From robert.kern at gmail.com Fri Jan 29 12:29:01 2010 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 29 Jan 2010 11:29:01 -0600 Subject: get PyObject* knowing its string name In-Reply-To: References: Message-ID: On 2010-01-29 11:00 AM, Mr.M wrote: > I think this sounds like a stupid question, but I searched the C/Api doc > and google but I wasn't able to find any hint: > > how can I retrive PyObject pointer if I only know it's name? > > What I'd like to do is something like this: > > [code] > PyObject* obj = PyFindWhatImLookingFor("w.z.y.x"); > [/code] > > Of course if I know that w is a module, z is a class and so on I can > search w dictionary than z members than... > > But I wonder if there is a way to retrive without knowing what w.z.y.x > means. There is probably a clever/careful way to do it, but most systems that do this just separate the package/module parts from the object parts: "w:z.y.x" Import the module part, then getattr() your way down the object part. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From andreas.tawn at ubisoft.com Fri Jan 29 12:33:19 2010 From: andreas.tawn at ubisoft.com (Andreas Tawn) Date: Fri, 29 Jan 2010 18:33:19 +0100 Subject: Sleep timer but still responsive? In-Reply-To: <8c76dbf2-8853-48cc-9ca9-37929936f73a@b35g2000vbc.googlegroups.com> References: <8c76dbf2-8853-48cc-9ca9-37929936f73a@b35g2000vbc.googlegroups.com> Message-ID: <654D9D97DA51AD479973BC2D5578603C06AAD6AC51@PDC-MAIL-CMS01.ubisoft.org> > On Jan 28, 4:55?pm, "Gabriel Genellina" > wrote: > > Please provide more details. What do you want your program to do > while > > sleeping? What kind of actions do you want a response to? > > Do you have a GUI? A curses-based interfase? > > > > -- > > Gabriel Genellina > > My app is purely console based. I just don't want the console to lock > up (on Windows using time.sleep(x) causes the console to become > unresponsive until the timer is done), and I want people to be able to > CTRL+C to stop the script if need be (which can't be done if it's > unresponsive!). > > Thanks. How about this? Responds to ctrl+c, but still sleeps. import time def responsiveSleep(n): while n > 0: time.sleep(1) n -= 1 Cheers, Drea From python at mrabarnett.plus.com Fri Jan 29 12:39:31 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 29 Jan 2010 17:39:31 +0000 Subject: Sleep timer but still responsive? In-Reply-To: <8c76dbf2-8853-48cc-9ca9-37929936f73a@b35g2000vbc.googlegroups.com> References: <8c76dbf2-8853-48cc-9ca9-37929936f73a@b35g2000vbc.googlegroups.com> Message-ID: <4B631D53.6030602@mrabarnett.plus.com> JohnnyFive wrote: > On Jan 28, 4:55 pm, "Gabriel Genellina" > wrote: >> Please provide more details. What do you want your program to do while >> sleeping? What kind of actions do you want a response to? >> Do you have a GUI? A curses-based interfase? >> >> -- >> Gabriel Genellina > > My app is purely console based. I just don't want the console to lock > up (on Windows using time.sleep(x) causes the console to become > unresponsive until the timer is done), and I want people to be able to > CTRL+C to stop the script if need be (which can't be done if it's > unresponsive!). > > Thanks. Which version of Python are you using? time.sleep(x) can be interrupted in Python 2.6. If the version you're using can't be interrupted then you could use multiple sleeps: # Wait for a total of 10 secs. for i in range(10): time.sleep(1) From steve at holdenweb.com Fri Jan 29 12:47:46 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 29 Jan 2010 12:47:46 -0500 Subject: Changing tab behavior in Python Interactive mode In-Reply-To: <93ae66b7-7fe2-4542-b8d0-039f9423cdd8@n7g2000yqb.googlegroups.com> References: <93ae66b7-7fe2-4542-b8d0-039f9423cdd8@n7g2000yqb.googlegroups.com> Message-ID: Andrej Mitrovic wrote: > I've noticed that when running Python in interactive mode (via cmd on > windows), the first time I hit tab it will move 4 spaces to the right, > however each new tab will move 8 spaces instead of 4. Why this > inconsistent behavior? And how could I change this to be consistent > and always move only 4 spaces? The first tab you are starting four characters in due to the interpreter already having printed ">>> ". regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From awilliam at opengroupware.us Fri Jan 29 13:04:44 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Fri, 29 Jan 2010 13:04:44 -0500 Subject: Processing XML File In-Reply-To: <6bac2d7b-509d-4f92-912f-5e8dc68c3f30@36g2000yqu.googlegroups.com> References: <6bac2d7b-509d-4f92-912f-5e8dc68c3f30@36g2000yqu.googlegroups.com> Message-ID: <1264788284.3027.0.camel@linux-m3mt> On Fri, 2010-01-29 at 09:25 -0800, jakecjacobson wrote: > I need to take a XML web resource and split it up into smaller XML > files. I am able to retrieve the web resource but I can't find any > good XML examples. I am just learning Python so forgive me if this > question has been answered many times in the past. > My resource is like: > > ... > ... > > > ... > ... > > So in this example, I would need to output 2 files with the contents > of each file what is between the open and close document tag. Do you want to parse the document or SaX? I have a SaX example at From gagsl-py2 at yahoo.com.ar Fri Jan 29 13:19:19 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 15:19:19 -0300 Subject: Sleep timer but still responsive? References: <8c76dbf2-8853-48cc-9ca9-37929936f73a@b35g2000vbc.googlegroups.com> <4B631D53.6030602@mrabarnett.plus.com> Message-ID: En Fri, 29 Jan 2010 14:39:31 -0300, MRAB escribi?: > JohnnyFive wrote: >> My app is purely console based. I just don't want the console to lock >> up (on Windows using time.sleep(x) causes the console to become >> unresponsive until the timer is done), and I want people to be able to >> CTRL+C to stop the script if need be (which can't be done if it's >> unresponsive!). >> Thanks. > > Which version of Python are you using? time.sleep(x) can be interrupted > in Python 2.6. I'm able to be more precise: time.sleep() can be interrupted in any Python version since 2.3. To the OP: beware of any unqualified 'except' clauses; a block like this: try: ... except: do_something_or_pass may "swallow" the KeyboardInterrupt exception (generated by a Ctrl-C press). From Python 2.5 and up, the most generic exception clause should read `except Exception: ...` In previous versions, you had to explicitely re-raise KeyboardInterrupt and SystemExit in any catch-all block: try: ... except (KeyboardInterrupt, SystemExit): raise except: ... -- Gabriel Genellina From morganhein at gmail.com Fri Jan 29 13:24:48 2010 From: morganhein at gmail.com (JohnnyFive) Date: Fri, 29 Jan 2010 10:24:48 -0800 (PST) Subject: Sleep timer but still responsive? References: <8c76dbf2-8853-48cc-9ca9-37929936f73a@b35g2000vbc.googlegroups.com> Message-ID: On Jan 29, 9:33?am, Andreas Tawn wrote: > > On Jan 28, 4:55?pm, "Gabriel Genellina" > > wrote: > > > Please provide more details. What do you want your program to do > > while > > > sleeping? What kind of actions do you want a response to? > > > Do you have a GUI? A curses-based interfase? > > > > -- > > > Gabriel Genellina > > > My app is purely console based. I just don't want the console to lock > > up (on Windows using time.sleep(x) causes the console to become > > unresponsive until the timer is done), and I want people to be able to > > CTRL+C to stop the script if need be (which can't be done if it's > > unresponsive!). > > > Thanks. > > How about this? Responds to ctrl+c, but still sleeps. > > import time > > def responsiveSleep(n): > ? ? while n > 0: > ? ? ? ? time.sleep(1) > ? ? ? ? n -= 1 > > Cheers, > > Drea Thanks for the ideas! Maybe it's just my computer, but using your solution still causes the prompt to become unresponsive during the sleeps. I am using 2.6.4 btw. It's not a major deal though, I just thought there had to be a way to do this fairly easily. From jakecjacobson at gmail.com Fri Jan 29 13:34:17 2010 From: jakecjacobson at gmail.com (jakecjacobson) Date: Fri, 29 Jan 2010 10:34:17 -0800 (PST) Subject: Processing XML File References: <6bac2d7b-509d-4f92-912f-5e8dc68c3f30@36g2000yqu.googlegroups.com> Message-ID: On Jan 29, 1:04?pm, Adam Tauno Williams wrote: > On Fri, 2010-01-29 at 09:25 -0800, jakecjacobson wrote: > > I need to take a XML web resource and split it up into smaller XML > > files. ?I am able to retrieve the web resource but I can't find any > > good XML examples. ?I am just learning Python so forgive me if this > > question has been answered many times in the past. > > My resource is like: > > > > ? ? ?... > > ? ? ?... > > > > > > ? ? ?... > > ? ? ?... > > > > So in this example, I would need to output 2 files with the contents > > of each file what is between the open and close document tag. > > Do you want to parse the document or SaX? > > I have a SaX example at > Thanks but I am way over my head with XML, Python. I am working with DDMS and need to output the individual resource nodes to their own file. I hope that this helps and I need a good example and how to use it. Here is what a resource node looks like: Sample Taxonomy This is a sample taxonomy created for the Help page. Sample Developer FGM, Inc. 703-885-1000 sampleDeveloper at fgm.com You can see the DDMS site at https://metadata.dod.mil/. From awilliam at opengroupware.us Fri Jan 29 13:54:26 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Fri, 29 Jan 2010 13:54:26 -0500 Subject: Processing XML File In-Reply-To: References: <6bac2d7b-509d-4f92-912f-5e8dc68c3f30@36g2000yqu.googlegroups.com> Message-ID: <1264791266.3492.3.camel@linux-m3mt> On Fri, 2010-01-29 at 10:34 -0800, jakecjacobson wrote: > On Jan 29, 1:04 pm, Adam Tauno Williams > wrote: > > On Fri, 2010-01-29 at 09:25 -0800, jakecjacobson wrote: > > > I need to take a XML web resource and split it up into smaller XML > > > files. I am able to retrieve the web resource but I can't find any > > > good XML examples. I am just learning Python so forgive me if this > > > question has been answered many times in the past. > > > My resource is like: > > > > > > ... > > > ... > > > > > > > > > > > > So in this example, I would need to output 2 files with the contents > > > of each file what is between the open and close document tag. > > Do you want to parse the document or SaX? > > I have a SaX example at > > > Thanks but I am way over my head with XML, Python. I am working with > DDMS and need to output the individual resource nodes to their own > file. I hope that this helps and I need a good example and how to use > it. If that is all you need XPath will spit it apart for you like doc = etree.parse(self._rfile) results = doc.xpath(xpath) for result in results: print str(result) For example if your XML has an outermost element of ResultSet with inner row elements just do: for record in doc.xpath(u'/ResultSet/row') Implied import for these examples is "from lxml import etree" > Here is what a resource node looks like: > xsi:schemaLocation="https://metadata.dod.mil/mdr/ns/DDMS/1.4/ > https://metadata.dod.mil/mdr/ns/DDMS/1.4/" > xmlns:ddms="https://metadata.dod.mil/mdr/ns/DDMS/1.4/" > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" > xmlns:ICISM="urn:us:gov:ic:ism:v2"> > > > > ICISM:classification="U">Sample Taxonomy > ICISM:classification="U"> > This is a sample taxonomy created for the Help page. > > > ICISM:classification="U"> > > Sample > Developer > FGM, Inc. > 703-885-1000 > sampleDeveloper at fgm.com > > > ICISM:classification="U" ICISM:nonICmarkings="DIST_STMT_A" /> > > > > You can see the DDMS site at https://metadata.dod.mil/. -- OpenGroupware developer: awilliam at whitemice.org OpenGroupare & Cyrus IMAPd documenation @ From stefan_ml at behnel.de Fri Jan 29 14:24:35 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 29 Jan 2010 20:24:35 +0100 Subject: Processing XML File In-Reply-To: <6bac2d7b-509d-4f92-912f-5e8dc68c3f30@36g2000yqu.googlegroups.com> References: <6bac2d7b-509d-4f92-912f-5e8dc68c3f30@36g2000yqu.googlegroups.com> Message-ID: <4b6335f3$0$6573$9b4e6d93@newsspool3.arcor-online.net> jakecjacobson, 29.01.2010 18:25: > I need to take a XML web resource and split it up into smaller XML > files. I am able to retrieve the web resource but I can't find any > good XML examples. I am just learning Python so forgive me if this > question has been answered many times in the past. > > My resource is like: > > > ... > ... > > > ... > ... > Is this what you get as a document or is this just /contained/ in the document? Note that XML does not allow more than one root element, so the above is not XML. Each of the two ... parts form an XML document by themselves, though. > So in this example, I would need to output 2 files with the contents > of each file what is between the open and close document tag. Are the two files formatted as you show above? In that case, you can simply iterate over the lines and cut the document when you see "". Or, if you are sure that "" only appears as top-most elements and not inside of the documents, you can search for "" in the content (a string, I guess) and split it there. As was pointed out before, once you have these two documents, use the xml.etree package to work with them. Something like this might work: import xml.etree.ElementTree as ET data = urllib2.urlopen(url).read() for part in data.split(''): document = ET.fromstring(''+part) print(document.tag) # ... do other stuff Stefan From fred.sells at adventistcare.org Fri Jan 29 14:31:56 2010 From: fred.sells at adventistcare.org (Sells, Fred) Date: Fri, 29 Jan 2010 14:31:56 -0500 Subject: Processing XML File In-Reply-To: <4b6335f3$0$6573$9b4e6d93@newsspool3.arcor-online.net> References: <6bac2d7b-509d-4f92-912f-5e8dc68c3f30@36g2000yqu.googlegroups.com> <4b6335f3$0$6573$9b4e6d93@newsspool3.arcor-online.net> Message-ID: Google is your friend. Elementtree is one of the better documented IMHO, but there are many modules to do this. > -----Original Message----- > From: python-list-bounces+frsells=adventistcare.org at python.org > [mailto:python-list-bounces+frsells=adventistcare.org at python.org] On > Behalf Of Stefan Behnel > Sent: Friday, January 29, 2010 2:25 PM > To: python-list at python.org > Subject: Re: Processing XML File > > jakecjacobson, 29.01.2010 18:25: > > I need to take a XML web resource and split it up into smaller XML > > files. I am able to retrieve the web resource but I can't find any > > good XML examples. I am just learning Python so forgive me if this > > question has been answered many times in the past. > > > > My resource is like: > > > > > > ... > > ... > > > > > > ... > > ... > > > > Is this what you get as a document or is this just /contained/ in the > document? > > Note that XML does not allow more than one root element, so the above is > not XML. Each of the two ... parts form an XML > document by themselves, though. > > > > So in this example, I would need to output 2 files with the contents > > of each file what is between the open and close document tag. > > Are the two files formatted as you show above? In that case, you can > simply > iterate over the lines and cut the document when you see "". Or, > if you are sure that "" only appears as top-most elements and > not > inside of the documents, you can search for "" in the content (a > string, I guess) and split it there. > > As was pointed out before, once you have these two documents, use the > xml.etree package to work with them. > > Something like this might work: > > import xml.etree.ElementTree as ET > > data = urllib2.urlopen(url).read() > > for part in data.split(''): > document = ET.fromstring(''+part) > print(document.tag) > # ... do other stuff > > Stefan > -- > http://mail.python.org/mailman/listinfo/python-list From stefan_ml at behnel.de Fri Jan 29 14:41:58 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 29 Jan 2010 20:41:58 +0100 Subject: Processing XML File In-Reply-To: References: <6bac2d7b-509d-4f92-912f-5e8dc68c3f30@36g2000yqu.googlegroups.com> <4b6335f3$0$6573$9b4e6d93@newsspool3.arcor-online.net> Message-ID: <4b633a07$0$6582$9b4e6d93@newsspool3.arcor-online.net> Sells, Fred, 29.01.2010 20:31: > Google is your friend. Elementtree is one of the better documented > IMHO, but there are many modules to do this. Unless the OP provides some more information, "do this" is rather underdefined. And sending someone off to Google who is just learning the basics of Python and XML and trying to solve a very specific problem with them is not exactly the spirit I'm used to in this newsgroup. Stefan From half.italian at gmail.com Fri Jan 29 15:04:08 2010 From: half.italian at gmail.com (Sean DiZazzo) Date: Fri, 29 Jan 2010 12:04:08 -0800 (PST) Subject: SimpleXMLRPCServer daemon References: <1e689e5c-8e2a-4896-b867-717316fa783e@o26g2000vbd.googlegroups.com> Message-ID: <9f5d00f3-8485-43d0-bf55-4e71fb389065@a5g2000prg.googlegroups.com> On Jan 29, 7:54?am, Thomas Allen wrote: > I have a script that runs an instance of SimpleXMLRPCServer and in > general it works as expected. In its __del__, it is supposed to clean > up its PID file (written on boot). I have two problems with this > server instance: The first is that tt doesn't always clean up its PID > file; is there a more reliable way to do this than how I am currently? > The second is that when it does crash, I don't know about it...what > would be sufficient as a "keep-alive" script to restart it? I suppose > I could use something like EventMachine (already installed on my > server) to watch the PID file if it were deleted reliably. > > Thomas Allen You should check out python-daemon. I use home grown daemons all the time, and have only played a little with the python-daemon library, but it is surely written well, and handles lockfiles and pidfiles. I believe you can map a function to any signal that the daemon receives to do any kind of cleanup etc. Ssometimes those PID files might be left around, but the runner included with the module (i think) checks for stale pid files. There's all kinds of useful stuff. In my home grown daemons I use Adam's technique, and wrap the "while 1:" block in a try except clause. ~Sean From gagsl-py2 at yahoo.com.ar Fri Jan 29 15:04:49 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 17:04:49 -0300 Subject: Some C-API functions clear the error indicator? References: Message-ID: En Fri, 29 Jan 2010 11:37:09 -0300, Austin Bingham escribi?: > I've noticed that several (many?) python functions seem to clear the > error/exception indicators when they're called from a C/C++ program. > For example, both PyImport_ImportModule and traceback.extract_tb() > (called via the function call methods) do this: if error indicators > are set prior to their call (as indicated by PyErr_Fetch, and > including a call to PyErr_Restore), I see that they are unset (using > the same method) after the call. This happens even when the functions > succeed. It's simple: you have to check *every* function call for failure. Many functions return new object references and you have to properly decrement them in case of failure, so in most cases this means that you have to check each and every call. -- Gabriel Genellina From gagsl-py2 at yahoo.com.ar Fri Jan 29 15:18:42 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 17:18:42 -0300 Subject: site.py confusion References: Message-ID: En Wed, 27 Jan 2010 15:48:23 -0300, George Trojan escribi?: > Arnaud Delobelle wrote: >> George Trojan writes: >> >>> Inspired by the 'Default path for files' thread I tried to use >>> sitecustomize in my code. What puzzles me is that the site.py's main() >>> is not executed. My sitecustomize.py is > > That gave me the explanation why the above happens: when site is > imported, the current directory is not yet prepended to sys.path. > > I wanted to have library location specific to application without having > to explicitly change sys.path in all App-Top-Dir/bin/*.py. I thought > creating bin/sitecustomize.py would do the trick. Put sitecustomize.py in any of those directories already in sys.path; on Python 2.6, you may put sitecustomize.py on your "user site directory", ~/.local/lib/python2.6/site-packages (see PEP370; it won't show up in sys.path unless the directory actually exists) -- Gabriel Genellina From gagsl-py2 at yahoo.com.ar Fri Jan 29 15:28:24 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 17:28:24 -0300 Subject: SimpleXMLRPCServer daemon References: <1e689e5c-8e2a-4896-b867-717316fa783e@o26g2000vbd.googlegroups.com> Message-ID: En Fri, 29 Jan 2010 12:54:23 -0300, Thomas Allen escribi?: > I have a script that runs an instance of SimpleXMLRPCServer and in > general it works as expected. In its __del__, it is supposed to clean > up its PID file (written on boot). I have two problems with this > server instance: The first is that tt doesn't always clean up its PID > file; is there a more reliable way to do this than how I am currently? > The second is that when it does crash, I don't know about it...what > would be sufficient as a "keep-alive" script to restart it? I suppose > I could use something like EventMachine (already installed on my > server) to watch the PID file if it were deleted reliably. I agree with the recommendation of using some daemon library; doing it right is better left to the experts :) But if you can't or don't want to alter your code so much, I suggest: - use atexit.register, instead of __del__, to delete the PID file - keep a lock on the pid file; if a second instance is able to write to it, it means it's an orphan from a previous, aborted run. (Checking whether a process with such pid exists is not enough - pids are recycled rather fast) -- Gabriel Genellina From andrej.mitrovich at gmail.com Fri Jan 29 15:35:58 2010 From: andrej.mitrovich at gmail.com (Andrej Mitrovic) Date: Fri, 29 Jan 2010 12:35:58 -0800 (PST) Subject: Changing tab behavior in Python Interactive mode References: <93ae66b7-7fe2-4542-b8d0-039f9423cdd8@n7g2000yqb.googlegroups.com> Message-ID: <073949c2-787d-44a4-a471-c77e605d32de@y12g2000yqh.googlegroups.com> On Jan 29, 6:47?pm, Steve Holden wrote: > Andrej Mitrovic wrote: > > I've noticed that when running Python in interactive mode (via cmd on > > windows), the first time I hit tab it will move 4 spaces to the right, > > however each new tab will move 8 spaces instead of 4. Why this > > inconsistent behavior? And how could I change this to be consistent > > and always move only 4 spaces? > > The first tab you are starting four characters in due to the interpreter > already having printed ">>> ". > > regards > ?Steve > -- > Steve Holden ? ? ? ? ? +1 571 484 6266 ? +1 800 494 3119 > PyCon is coming! Atlanta, Feb 2010 ?http://us.pycon.org/ > Holden Web LLC ? ? ? ? ? ? ? ?http://www.holdenweb.com/ > UPCOMING EVENTS: ? ? ? ?http://holdenweb.eventbrite.com/ I see. I wonder if I could modify this behavior.. I guess I'll have to take a look at the source. Thanks anyway! From gagsl-py2 at yahoo.com.ar Fri Jan 29 15:37:18 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 17:37:18 -0300 Subject: Function name unchanged in error message References: Message-ID: En Fri, 29 Jan 2010 13:09:40 -0300, Michele Simionato escribi?: > On Jan 29, 2:30 pm, andrew cooke wrote: >> Is there any way to change the name of the function in an error >> message? In the example below I'd like the error to refer to bar(), >> for example (the motivation is related function decorators - I'd like >> the wrapper function to give the same name) > > Use the decorator module which does the right thing: > http://pypi.python.org/pypi/decorator The decorator module is a very fine addition to anyone's tool set -- but in this case it is enough to use the wraps() function from the functools standard module. -- Gabriel Genellina From gagsl-py2 at yahoo.com.ar Fri Jan 29 15:43:43 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 17:43:43 -0300 Subject: python for net manager References: <6e5636721001261911i6d5ca5fma2dff5b8a89ed277@mail.gmail.com> Message-ID: En Wed, 27 Jan 2010 00:11:10 -0300, moon sky escribi?: > i just want to write a small tool to manage the network, including detect > the ip addr,netmask, and send the arp request to find out locale's > ip-mac > turtple, > is there any way to succeed this exlude 'subprocess call'? Try scapy: http://www.secdev.org/projects/scapy/ -- Gabriel Genellina From steve at holdenweb.com Fri Jan 29 16:07:09 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 29 Jan 2010 16:07:09 -0500 Subject: Changing tab behavior in Python Interactive mode In-Reply-To: <073949c2-787d-44a4-a471-c77e605d32de@y12g2000yqh.googlegroups.com> References: <93ae66b7-7fe2-4542-b8d0-039f9423cdd8@n7g2000yqb.googlegroups.com> <073949c2-787d-44a4-a471-c77e605d32de@y12g2000yqh.googlegroups.com> Message-ID: <4B634DFD.20309@holdenweb.com> Andrej Mitrovic wrote: > On Jan 29, 6:47 pm, Steve Holden wrote: >> Andrej Mitrovic wrote: >>> I've noticed that when running Python in interactive mode (via cmd on >>> windows), the first time I hit tab it will move 4 spaces to the right, >>> however each new tab will move 8 spaces instead of 4. Why this >>> inconsistent behavior? And how could I change this to be consistent >>> and always move only 4 spaces? >> The first tab you are starting four characters in due to the interpreter >> already having printed ">>> ". >> >> regards >> Steve >> -- >> Steve Holden +1 571 484 6266 +1 800 494 3119 >> PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ >> Holden Web LLC http://www.holdenweb.com/ >> UPCOMING EVENTS: http://holdenweb.eventbrite.com/ > > I see. I wonder if I could modify this behavior.. I guess I'll have to > take a look at the source. Thanks anyway! I's nothing to do with the source: it isn't the interpreter that's deciding where the "tab stops" are, it's the console window the interpreter is running in. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Fri Jan 29 16:07:09 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 29 Jan 2010 16:07:09 -0500 Subject: Changing tab behavior in Python Interactive mode In-Reply-To: <073949c2-787d-44a4-a471-c77e605d32de@y12g2000yqh.googlegroups.com> References: <93ae66b7-7fe2-4542-b8d0-039f9423cdd8@n7g2000yqb.googlegroups.com> <073949c2-787d-44a4-a471-c77e605d32de@y12g2000yqh.googlegroups.com> Message-ID: <4B634DFD.20309@holdenweb.com> Andrej Mitrovic wrote: > On Jan 29, 6:47 pm, Steve Holden wrote: >> Andrej Mitrovic wrote: >>> I've noticed that when running Python in interactive mode (via cmd on >>> windows), the first time I hit tab it will move 4 spaces to the right, >>> however each new tab will move 8 spaces instead of 4. Why this >>> inconsistent behavior? And how could I change this to be consistent >>> and always move only 4 spaces? >> The first tab you are starting four characters in due to the interpreter >> already having printed ">>> ". >> >> regards >> Steve >> -- >> Steve Holden +1 571 484 6266 +1 800 494 3119 >> PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ >> Holden Web LLC http://www.holdenweb.com/ >> UPCOMING EVENTS: http://holdenweb.eventbrite.com/ > > I see. I wonder if I could modify this behavior.. I guess I'll have to > take a look at the source. Thanks anyway! I's nothing to do with the source: it isn't the interpreter that's deciding where the "tab stops" are, it's the console window the interpreter is running in. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From gagsl-py2 at yahoo.com.ar Fri Jan 29 16:17:52 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 18:17:52 -0300 Subject: Killing a Thread References: <201001281425.30434.richard.lamboj@bilcom.at> Message-ID: En Thu, 28 Jan 2010 10:25:30 -0300, Richard Lamboj escribi?: > which Method is better to kill a Thread? Using Thread Events, or a > raising a > Exception? Maybe someone has a small example for me? The best way is simply NOT to do that. You don't kill a thread, you ask it to commit suicide. There is no reliable way to forcefully shut down another thread. The thread must periodically check some value (perhaps an Event object; in simple cases any variable will do) and (cleanly) exit when asked to. Threads that run pure Python code may be interrupted using the API call PyThreadState_SetAsyncExc; look for a recipe using ctypes to call it from Python. -- Gabriel Genellina From austin.bingham at gmail.com Fri Jan 29 16:25:14 2010 From: austin.bingham at gmail.com (Austin Bingham) Date: Fri, 29 Jan 2010 22:25:14 +0100 Subject: Some C-API functions clear the error indicator? In-Reply-To: References: Message-ID: Maybe I'm not following what you're saying. In my case, I already know that an exception has been thrown. In the course of processing that exception, I call another function which, for whatever reason and even when it succeeds, clears the exception indicators. How can I address this issue by checking function calls for failure? Austin On Fri, Jan 29, 2010 at 9:04 PM, Gabriel Genellina wrote: > En Fri, 29 Jan 2010 11:37:09 -0300, Austin Bingham > escribi?: > >> I've noticed that several (many?) python functions seem to clear the >> error/exception indicators when they're called from a C/C++ program. >> For example, both PyImport_ImportModule and traceback.extract_tb() >> (called via the function call methods) do this: if error indicators >> are set prior to their call (as indicated by PyErr_Fetch, and >> including a call to PyErr_Restore), I see that they are unset (using >> the same method) after the call. This happens even when the functions >> succeed. > > It's simple: you have to check *every* function call for failure. Many > functions return new object references and you have to properly decrement > them in case of failure, so in most cases this means that you have to check > each and every call. > > -- > Gabriel Genellina > > -- > http://mail.python.org/mailman/listinfo/python-list > From python at bdurham.com Fri Jan 29 16:33:11 2010 From: python at bdurham.com (python at bdurham.com) Date: Fri, 29 Jan 2010 16:33:11 -0500 Subject: Thread safe locale techniques? Message-ID: <1264800791.976.1357306091@webmail.messagingengine.com> We're currently writing a web application based on a threaded python web server framework (cherrypy) and would like to simultaneously support users from multiple locales. The locale module doesn't appear to be thread safe. Are there 3rd party libraries or modules that provide locale parsing and formatting functionality in a thread-safe way? Suggestions appreciated! Regards, Malcolm -------------- next part -------------- An HTML attachment was scrubbed... URL: From gagsl-py2 at yahoo.com.ar Fri Jan 29 16:38:47 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 18:38:47 -0300 Subject: Keyboard input References: <8f67b6f81001290853i39bab1acs49d02ac5be948d13@mail.gmail.com> Message-ID: En Fri, 29 Jan 2010 13:53:43 -0300, Mr.SpOOn escribi?: > I need to get keyboard input in a python program. I need it to let the > user choose some options, for example: > > 1) option 1 > 2) option 2 > 3) option 3 > > and then to input some data to the program. > > I'm using the raw_input method and it works fine, but I noted that I > can't use backspace and when I accidentally press shift space (because > I need to input uppercase letters separated by a space) it writes some > strange characters. That's strange. If you're using Linux, make sure you have the readline package installed. > So, is there another way to get keyboard input? If readline capabilities are not enough, you could try Urwid, a library for writing console programs: http://excess.org/urwid/ - but it's a big move. wxPython would let you write a graphical interface (and it's a bigger move!) -- Gabriel Genellina From andrej.mitrovich at gmail.com Fri Jan 29 16:43:05 2010 From: andrej.mitrovich at gmail.com (Andrej Mitrovic) Date: Fri, 29 Jan 2010 13:43:05 -0800 (PST) Subject: Changing tab behavior in Python Interactive mode References: <93ae66b7-7fe2-4542-b8d0-039f9423cdd8@n7g2000yqb.googlegroups.com> <073949c2-787d-44a4-a471-c77e605d32de@y12g2000yqh.googlegroups.com> Message-ID: <2c8c062f-0622-49ea-950b-80afde5371de@u41g2000yqe.googlegroups.com> On Jan 29, 10:07?pm, Steve Holden wrote: > Andrej Mitrovic wrote: > > On Jan 29, 6:47 pm, Steve Holden wrote: > >> Andrej Mitrovic wrote: > >>> I've noticed that when running Python in interactive mode (via cmd on > >>> windows), the first time I hit tab it will move 4 spaces to the right, > >>> however each new tab will move 8 spaces instead of 4. Why this > >>> inconsistent behavior? And how could I change this to be consistent > >>> and always move only 4 spaces? > >> The first tab you are starting four characters in due to the interpreter > >> already having printed ">>> ". > > >> regards > >> ?Steve > >> -- > >> Steve Holden ? ? ? ? ? +1 571 484 6266 ? +1 800 494 3119 > >> PyCon is coming! Atlanta, Feb 2010 ?http://us.pycon.org/ > >> Holden Web LLC ? ? ? ? ? ? ? ?http://www.holdenweb.com/ > >> UPCOMING EVENTS: ? ? ? ?http://holdenweb.eventbrite.com/ > > > I see. I wonder if I could modify this behavior.. I guess I'll have to > > take a look at the source. Thanks anyway! > > I's nothing to do with the source: it isn't the interpreter that's > deciding where the "tab stops" are, it's the console window the > interpreter is running in. > > regards > ?Steve > -- > Steve Holden ? ? ? ? ? +1 571 484 6266 ? +1 800 494 3119 > PyCon is coming! Atlanta, Feb 2010 ?http://us.pycon.org/ > Holden Web LLC ? ? ? ? ? ? ? ?http://www.holdenweb.com/ > UPCOMING EVENTS: ? ? ? ?http://holdenweb.eventbrite.com/ I see. Well I'm nitpicking here, it's not all that important (I just use cmd when I'm trying out small pieces of code). And I doubt there's a tabstop setting for cmd that I could use. Thanks for all your help! From gagsl-py2 at yahoo.com.ar Fri Jan 29 17:08:45 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 19:08:45 -0300 Subject: Data exchange between Delphi and Python (Win) References: <9e384ef61001280007j38f231e4we085e9af6d1b725d@mail.gmail.com> Message-ID: En Thu, 28 Jan 2010 05:07:00 -0300, Durumdara escribi?: > I have an exotic db, with exotic drivers, and it have buggy ODBC driver. > > But I have native driver - under Delphi. > > I need to access this DB under Pylons (or mod_python). > [...] > I think to COM/OLE, because it is accessable from all program, and I > think > to DLL (but DLL have problematic parameterisation). So, you can query the database in Delphi, and you want to be able to do that from Python. Your two ideas are feasible: you may write a DLL (or a COM object) in Delphi, that takes a query string, executes it, and returns the resulting data. All in Delphi code, no Python involved. Keep the memory allocation in Delphi - allocate and return a plain PChar buffer, and include a free() function to deallocate it when the other side is done using it. From Python, you may use ctypes [1] to call the DLL functions, or pywin32 [2] to invoke the COM object. Both ways are easy enough. [1] http://docs.python.org/library/ctypes.html [2] http://sourceforge.net/projects/pywin32/ -- Gabriel Genellina From rantingrick at gmail.com Fri Jan 29 17:12:45 2010 From: rantingrick at gmail.com (rantingrick) Date: Fri, 29 Jan 2010 14:12:45 -0800 (PST) Subject: Keyboard input References: <8f67b6f81001290853i39bab1acs49d02ac5be948d13@mail.gmail.com> Message-ID: On Jan 29, 3:38?pm, "Gabriel Genellina" wrote: > but it's a big ? > move. wxPython would let you write a graphical interface (and it's a ? > bigger move!) > > -- > Gabriel Genellina *ahem*!, tkinter is just a slightly medium move, and my god it's in the stdlib for crying out loud!! From jgardner at jonathangardner.net Fri Jan 29 17:15:30 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Fri, 29 Jan 2010 14:15:30 -0800 (PST) Subject: Keyboard input References: Message-ID: <24cdaadb-a8bd-47c1-8347-8229ccd3caba@b36g2000pri.googlegroups.com> On Jan 29, 8:53?am, "Mr.SpOOn" wrote: > Hi, > I need to get keyboard input in a python program. I need it to let the > user choose some options, for example: > > 1) option 1 > 2) option 2 > 3) option 3 > > and then to input some data to the program. > > I'm using the raw_input method and it works fine, but I noted that I > can't use backspace and when I accidentally press shift space (because > I need to input uppercase letters separated by a space) it writes some > strange characters. > > So, is there another way to get keyboard input? > > Maybe curses is overkill, but it will do what you need. Most interactive console apps use curses. You've run into some of the problems of simply reading STDIN. From jgardner at jonathangardner.net Fri Jan 29 17:18:04 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Fri, 29 Jan 2010 14:18:04 -0800 (PST) Subject: Threading issue with SQLite References: Message-ID: <09ca16b8-4e10-40f5-a69d-29fcdf6c2593@b9g2000pri.googlegroups.com> On Jan 29, 8:37?am, Alan Harris-Reid wrote: > > Questions... > 1. ?Is there a large overhead in opening a new SQLite connection for > each thread (ie. within each method)? Yes, but not as bad as some other DBs. > 2. ?Is there any way to use the same connection for the whole class (or > should I forget that idea completely?) Forget it. The thread stuff is pretty important. > 3. ?When a method returns to the calling method, is the connection > automatically closed (assuming the object is local, of course) or does > it have to be done explicitly using connection.close()? > If the object falls out of scope, that is, no code anywhere can reference it, it will be garbage collected and thus closed. Suggestion: Use something like SQLAlchemy to manage you DB interactions. One day, you'll move away from SQLite, and you'll be glad you were programming at a higher level. SQLAlchemy also does things like connection pooling and such for you. From pavlovevidence at gmail.com Fri Jan 29 17:23:48 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Fri, 29 Jan 2010 14:23:48 -0800 (PST) Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <85197b8b-b319-4da4-b84d-fdb060d0f127@b10g2000yqa.googlegroups.com> <4B629B6B.3000100@v.loewis.de> Message-ID: <5cd5a9f7-a42d-4f5a-a3f5-814a4992e700@b7g2000pro.googlegroups.com> On Jan 29, 12:25?am, "Martin v. Loewis" wrote: > > Well, I'd consider that an official release. ?Note that I didn't claim > > there was no hope PSF wouldn't change it's mind on 2.8. > > I'd like to point out that the PSF formally doesn't have any say in > this. Doesn't PSF own the Python trademark? Then it has to have a say, not over whether someone can fork the project or make another official release, but over whether they can do so and still call it Python. Carl Banks From jgardner at jonathangardner.net Fri Jan 29 17:27:19 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Fri, 29 Jan 2010 14:27:19 -0800 (PST) Subject: which one is faster? References: Message-ID: <002c2bc5-ed0a-4351-937a-358b987cadda@h9g2000prn.googlegroups.com> On Jan 28, 10:29?pm, "Stephen.Wu" <54wut... at gmail.com> wrote: > str.find(targetStr) > str.index(targetStr) with exception > str.count(targetStr) > targetStr in str > > which is the fastest way to check whether targetStr is in str? > The fastest way of all is to forget about this and finish the rest of your program. Developer time is much, much more valuable than processor time. When you are all done, and have solved all the other problems in the world, you can come back and pontificate on how many nanoseconds you can save by using "find" or "in". From ben+python at benfinney.id.au Fri Jan 29 17:29:31 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 30 Jan 2010 09:29:31 +1100 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <873a1ownn8.fsf@benfinney.id.au> Duncan Booth writes: > Here's what I see in the Ubuntu packages. Python 3 seems only to be in the > universe repositories so far. > > Dapper: Python 2.4.2 > Hardy: Python 2.5.2 > Intrepid: Python 2.5.2, 3.0~b3 (universe) > Jaunty: Python 2.6.2, 3.0.1 (universe) > Karmic: Python 2.6.4rc1, 3.1 (universe) > Lucid: Python 2.6.4, 3.1 (universe) > > WTF is rc anything doing in the main repository? It's to be expected if the release team process specifies announcement messages of the form ?RELEASED: Python X.Y.Zrc2?. I have long argued, without much traction, that the process should reserve the term ?RELEASED? for use only in reference to releases, not other things. -- \ ?First they came for the verbs, and I said nothing, for verbing | `\ weirds language. Then, they arrival for the nouns and I speech | _o__) nothing, for I no verbs.? ?Peter Ellis | Ben Finney From jgardner at jonathangardner.net Fri Jan 29 17:49:06 2010 From: jgardner at jonathangardner.net (Jonathan Gardner) Date: Fri, 29 Jan 2010 14:49:06 -0800 (PST) Subject: For loop searching takes too long! References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> Message-ID: On Jan 28, 3:52?pm, elsa wrote: > > I've got a problem with my program, in that the code just takes too > long to run. Here's what I'm doing. If anyone has any tips, they'd be > much appreciated! > First of all, don't play with large lists. Large lists have a tendency to grow larger over time, until they get absurdly large. If you're dealing with a long list, work with it like you'd work with a file. If you need random access, stick it in some form of database, such as BDB or PostgreSQL. If you need an index, stick it in some form of DB. Eventually, large lists end up like that anyway. Don't fool yourself early on---prepare for the worst and solve the problems of tomorrow today. > So, say I have a list of lists that looks something like this (I'm > using a list of lists, rather than a list of tuples, as I need it to > be mutable): > > myList = [[786,0],[45, 1],[673,1],...................[23,46]] > > there are enough entries in the outer list, that the sum of myList[i] > [0] across all i could be as high as 10^7. > > Now, what I need to do is randomly choose one myList[i], however the > distribution of my random choice needs to be proportional to the > values of myList[i][0]. So, for this list above, I'd have a much > higher chance of choosing myList[0] than myList[1]. > Let's do some thinking here. Let's say you have a list of N items. You need to choose one, but you don't know how many there are. One algorithm that works is you start with the first item. If that's the only item, you've chosen it. If there's another item, you roll the dice. There's a 1/2 chance you drop the first item and take the sedon. If there's a third item, you roll the dice, There's a 1/3 chance you drop the first or second item, and take the third. You see the pattern? Keep the nth item 1/n of the time. In your case, you're actually pulling off N items each time, all with the same value. Your chance of keeping the next item is (number of items in the next one)/(total items seen so far, including the next one). Do the math for this. It's really simple. Let's walk through this: [[786,0],[45, 1],[673,2],...................[23,46]] Step 1: See [786,0]. Remember 786. Keep 0. Step 2: See [45,1]. Remember 786+45=831. Keep 1 45/831 of the time. Step 3: See [673,2]. Remember 831+673=1504. Keep 2 673/1504 of the time. Etc... Now, the algorithm I've just described is much less memory intensive and can deal with very long lists of numbers. However, you are calling rand() a lot. rand() is not a cheap operation. One optimization is to roll rand() once, and keep using that number over and over again. Granted, for very large numbers of items, you're going to need a very precise random number of some very small probabilities will never be chosen. This is left as an exercise for the reader. From mrm at unknown.nospam Fri Jan 29 17:50:52 2010 From: mrm at unknown.nospam (Mr.M) Date: Fri, 29 Jan 2010 22:50:52 GMT Subject: C API: module cleanup function Message-ID: Hi, I can't figure out if there is a way to run a specialized cleanup function when a module needs to be "unloaded" (i.e. just before a reload() or when i quit the interpreter). I'm thinking of something like tp_dealloc. If I call Py_InitModule3 and look at module->ob_type->tp_dealloc, I find that Python provides a default tp_dealloc for me. Now, suppose my module needs to allocate some resources at startup, I'm not sure, but I think I'd have to do it in my PyMODINIT_FUNC, right? But, if I reload() my module or if I quit the Python interpreter, I'd like to free those resources (before allocate them again, in case of a reload). Is there a way to make this work? Thank you, Luca. From gagsl-py2 at yahoo.com.ar Fri Jan 29 18:45:47 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 20:45:47 -0300 Subject: C API: module cleanup function References: Message-ID: En Fri, 29 Jan 2010 19:50:52 -0300, Mr.M escribi?: > I can't figure out if there is a way to run a specialized cleanup > function when a module needs to be "unloaded" (i.e. just before a > reload() or when i quit the interpreter). I think what you want to do isn't possible with Python 2, and it's one of the reasons the module handling was redesigned in Python 3.x; see PEP 3121. > I'm thinking of something like tp_dealloc. m_free (PyModuleDef member, in Python 3) might work for this. -- Gabriel Genellina From mrm at unknown.nospam Fri Jan 29 18:52:49 2010 From: mrm at unknown.nospam (Mr.M) Date: Fri, 29 Jan 2010 23:52:49 GMT Subject: C API: module cleanup function In-Reply-To: References: Message-ID: Gabriel Genellina ha scritto: > I think what you want to do isn't possible with Python 2, and it's one > of the reasons the module handling was redesigned in Python 3.x; see PEP > 3121. Thank you Gabriel for your help. Unlucky I can't use Python 3.x in my project, sob! Luca. From gagsl-py2 at yahoo.com.ar Fri Jan 29 19:02:14 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 21:02:14 -0300 Subject: Python-list Digest, Vol 76, Issue 303 References: <424b71ec1001290117y259a134dkeb31abf906cd32bf@mail.gmail.com> Message-ID: [top posting corrected] > From: Steve Holden > Date: Tue, 26 Jan 2010 11:54:23 -0500 > Anyway, I suspect your error might go away if you turned the first > argument of hte log.info() call into a format string such as > > log.info("refer: %s", ret) > >> try: >> log.info("start") >> log.info("refer",ret) >> log.info("end") >> except TypeError: >> log.exception("Exception raised") En Fri, 29 Jan 2010 06:17:56 -0300, siddhartha veedaluru escribi?: > Thats the output i got in output window.it is not listing the statement > which caused it. copy paste and try it on you setup. it behaves the same > way. > > I know that error is in log.info() function call.but it is not caught by > "except". which is not getting logged using log.exception Errors that occur inside the logging system are not logged themselves (users aren't usually interested on those kind of errors, but on application errors; also, if logging errors were logged too, this could create an infinite loop). To handle those errors in the logging system yourself, override Handler.handleError() -- Gabriel Genellina From gagsl-py2 at yahoo.com.ar Fri Jan 29 19:11:08 2010 From: gagsl-py2 at yahoo.com.ar (Gabriel Genellina) Date: Fri, 29 Jan 2010 21:11:08 -0300 Subject: Some C-API functions clear the error indicator? References: Message-ID: En Fri, 29 Jan 2010 18:25:14 -0300, Austin Bingham escribi?: > Maybe I'm not following what you're saying. In my case, I already know > that an exception has been thrown. In the course of processing that > exception, I call another function which, for whatever reason and even > when it succeeds, clears the exception indicators. How can I address > this issue by checking function calls for failure? Maybe if you provide an actual use case we can suggest how to handle it. The code in your original post does not make any sense to me (except by showing that PyImport_ImportModule does clear the error indicator). If you already know there was an error, and you even have retrieved the error details, why do you care if the error indicator gets reset? -- Gabriel Genellina From tjreedy at udel.edu Fri Jan 29 19:24:15 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 29 Jan 2010 19:24:15 -0500 Subject: Changing tab behavior in Python Interactive mode In-Reply-To: <2c8c062f-0622-49ea-950b-80afde5371de@u41g2000yqe.googlegroups.com> References: <93ae66b7-7fe2-4542-b8d0-039f9423cdd8@n7g2000yqb.googlegroups.com> <073949c2-787d-44a4-a471-c77e605d32de@y12g2000yqh.googlegroups.com> <2c8c062f-0622-49ea-950b-80afde5371de@u41g2000yqe.googlegroups.com> Message-ID: On 1/29/2010 4:43 PM, Andrej Mitrovic wrote: > On Jan 29, 10:07 pm, Steve Holden wrote: >> Andrej Mitrovic wrote: >>> On Jan 29, 6:47 pm, Steve Holden wrote: >>>> Andrej Mitrovic wrote: >>>>> I've noticed that when running Python in interactive mode (via cmd on >>>>> windows), the first time I hit tab it will move 4 spaces to the right, >> I's nothing to do with the source: it isn't the interpreter that's >> deciding where the "tab stops" are, it's the console window the >> interpreter is running in. > I see. Well I'm nitpicking here, it's not all that important (I just > use cmd when I'm trying out small pieces of code). And I doubt there's > a tabstop setting for cmd that I could use. Nope. I checked the properties dialog (rt click upper left icon). You might want to turn on quickeditmode (as admin). Terry Jan Reedy From aahz at pythoncraft.com Fri Jan 29 21:04:31 2010 From: aahz at pythoncraft.com (Aahz) Date: 29 Jan 2010 18:04:31 -0800 Subject: image processing - inverse filtering References: <5bfefbb6-89a8-49f6-9f02-7d36dfbc0f16@c29g2000yqd.googlegroups.com> Message-ID: In article <5bfefbb6-89a8-49f6-9f02-7d36dfbc0f16 at c29g2000yqd.googlegroups.com>, suresh.amritapuri wrote: > >If I am using scipy.ndimage.gaussian_filter() for filtering an image, >how to do the inverse filtering? In general how to do this using >scipy.ndimage? http://projects.scipy.org/mailman/listinfo/scipy-user -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From benjamin.kaplan at case.edu Fri Jan 29 21:28:03 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Fri, 29 Jan 2010 21:28:03 -0500 Subject: myths about python 3 In-Reply-To: <873a1ownn8.fsf@benfinney.id.au> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <873a1ownn8.fsf@benfinney.id.au> Message-ID: On Fri, Jan 29, 2010 at 5:29 PM, Ben Finney wrote: > Duncan Booth writes: > >> Here's what I see in the Ubuntu packages. Python 3 seems only to be in the >> universe repositories so far. >> >> Dapper: Python 2.4.2 >> Hardy: Python 2.5.2 >> Intrepid: Python 2.5.2, 3.0~b3 (universe) >> Jaunty: Python 2.6.2, 3.0.1 (universe) >> Karmic: Python 2.6.4rc1, 3.1 (universe) >> Lucid: Python 2.6.4, 3.1 (universe) >> >> WTF is rc anything doing in the main repository? > > It's to be expected if the release team process specifies announcement > messages of the form ?RELEASED: Python X.Y.Zrc2?. I have long argued, > without much traction, that the process should reserve the term > ?RELEASED? for use only in reference to releases, not other things. > Don't worry, it's not just Python. Ubuntu Hardy (IIRC) had a Firefox 3 beta in the repositories until a little while after the final release. I don't know what's taking them so long to upgrade to the final 2.6.4 though. > -- > ?\ ? ? ?First they came for the verbs, and I said nothing, for verbing | > ?`\ ? ?weirds language. Then, they arrival for the nouns and I speech | > _o__) ? ? ? ? ? ? ? ? ? ? ? ? ? nothing, for I no verbs.? ?Peter Ellis | > Ben Finney > -- > http://mail.python.org/mailman/listinfo/python-list > From steve at REMOVE-THIS-cybersource.com.au Fri Jan 29 22:03:04 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 30 Jan 2010 03:03:04 GMT Subject: Keyboard input References: <8f67b6f81001290853i39bab1acs49d02ac5be948d13@mail.gmail.com> Message-ID: <03738dc7$0$1309$c3e8da3@news.astraweb.com> On Fri, 29 Jan 2010 18:38:47 -0300, Gabriel Genellina wrote: >> I'm using the raw_input method and it works fine, but I noted that I >> can't use backspace and when I accidentally press shift space (because >> I need to input uppercase letters separated by a space) it writes some >> strange characters. > > That's strange. If you're using Linux, make sure you have the readline > package installed. Yes, I have this same issue too, I've just recently installed Python 2.0 (as well as 1.5, 2.4, 2.5, 2.6, 3.0 and 3.1), and while all the other versions recognise the readline package, this doesn't: >>> x = 1 >>> # hit up arrow to get previous line ... ^[[A I'm pretty sure I built 2.0 the same way I built all the others, anyone have any hints as to what I missed? -- Steven From pavlovevidence at gmail.com Fri Jan 29 22:03:28 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Fri, 29 Jan 2010 19:03:28 -0800 (PST) Subject: Perl 6 [was Re: myths about python 3] References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <03725fae$0$1309$c3e8da3@news.astraweb.com> Message-ID: <9e3c036a-b1ba-4d12-b47b-64e8aca8e69b@33g2000vbe.googlegroups.com> On Jan 28, 9:34?pm, Steven D'Aprano wrote: > On Thu, 28 Jan 2010 21:21:05 -0800, Tim Roberts wrote: > > Perl 6, on the other hand, is still fantasyware a decade after its > > announcement. ?It is, for the most part, THE canonical example of the > > wrong way to conduct a development effort. > > Out of curiosity, and completely off-topic, why has Perl 6 gone so badly? Because Larry Wall saw the writing on the wall that his "nice little scripting language" was way out of its league in terms of what it was trying to be (i.e., a real programming language suitable for building applications), and so he bailed out and left the Perl community with no leadership. This is just my impression, but some of the efforts to create Perl 6 seem to be clinging to a notion that Perl has always been a good language and are trying to ensure that it doesn't stray very far from Perl's core principles. Problem is, Perl wasn't ever a good language so they won't succeed. IOW, as Geremy Condra said, "it's too much like Perl". Carl Banks From steve at REMOVE-THIS-cybersource.com.au Fri Jan 29 22:07:52 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 30 Jan 2010 03:07:52 GMT Subject: For loop searching takes too long! References: <6ea6ebda-43af-4a77-a17e-0930cdd13258@b9g2000pri.googlegroups.com> Message-ID: <03738ee7$0$1309$c3e8da3@news.astraweb.com> On Fri, 29 Jan 2010 14:49:06 -0800, Jonathan Gardner wrote: > On Jan 28, 3:52?pm, elsa wrote: >> >> I've got a problem with my program, in that the code just takes too >> long to run. Here's what I'm doing. If anyone has any tips, they'd be >> much appreciated! >> >> > First of all, don't play with large lists. Large lists have a tendency > to grow larger over time, until they get absurdly large. > > If you're dealing with a long list, work with it like you'd work with a > file. If you need random access, stick it in some form of database, such > as BDB or PostgreSQL. If you need an index, stick it in some form of DB. > Eventually, large lists end up like that anyway. Don't fool yourself > early on---prepare for the worst and solve the problems of tomorrow > today. Talk about premature optimization. The OP probably has a few hundreds of thousands of items, millions at most, which is trivial on a modern PC. Your way of thinking is what has led to Firefox using a database to manage a few thousand bookmarks and cookies, which in turn leads to consistent data corruption problems. (I've been keeping note, and so far my installation of Firefox 3 has had database corruption at startup one time out of five.) -- Steven From apt.shansen at gmail.com Fri Jan 29 22:23:29 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Fri, 29 Jan 2010 19:23:29 -0800 Subject: Threading issue with SQLite In-Reply-To: <4B630ED3.5000706@googlemail.com> References: <4B630ED3.5000706@googlemail.com> Message-ID: <7a9c25c21001291923u339f7363oda95b783f4030135@mail.gmail.com> On Fri, Jan 29, 2010 at 8:37 AM, Alan Harris-Reid < aharrisreid at googlemail.com> wrote: > Hi, > > I am creating a web application (using Python 3.1 and CherryPy 3.2) where a > SQLite connection and cursor object are created using the following code > (simplified from the original): > > class MainSite: > con = sqlite.connect('MyDatabase.db') > cursor = con.cursor() > This is bad. For one thing, every thread should have its own connection at least-- you can store it in thread-local storage (see the threading module) for re-use if you want, if cherrypy does thread pooling techniques or some such (I'd be sorta surprised if it didn't). For another thing, cursors are NOT meant to be long-running interfaces. You should not leave one open beyond one concrete action (e.g., one user request, with however many SQL statements it takes to accomplish that work) even if you are re-using the same connection across multiple methods/invocations. Questions... > 1. Is there a large overhead in opening a new SQLite connection for each > thread (ie. within each method)? > You're making a web-app in Python, man. :) The overhead of a SQLite connection is not exactly significant at that point :) Though you could use thread pooling techniques with each thread having its own dedicated connection to mitigate what overhead there is. > 2. Is there any way to use the same connection for the whole class (or > should I forget that idea completely?) > You shouldn't think of "connection" and "class" as having any kind of meaningful link, it just doesn't make sense. It won't work and what you're -trying- to do doesn't actually even make sense. Besides the fact that you're worrying prematurely about overhead and trying to optimize it away, connections (and from them, cursors) to databases are things that are used for binding together a bunch of statements into transactions and maintaining 'current state' of your interaction with the database: its a a logical abstraction of "what I'm doing Now to the database", whereas the controller class is, "what can be done." 3. When a method returns to the calling method, is the connection > automatically closed (assuming the object is local, of course) or does it > have to be done explicitly using connection.close()? > If it goes out of scope, it will be closed if its a local variable (be sure you commit if you need to first!). Assuming you aren't caching/saving the connection in the thread for future calls it handles, if you have a thread pool doing your work (again, I'd be -really- surprised if CherryPy with threads didn't involve thread pools as opposed to starting up and stopping threads continually as requests come in). HTH, --S > > TIA, > Alan Harris-Reid > > > -- > http://mail.python.org/mailman/listinfo/python-list > --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From Blogtest77 at gmail.com Fri Jan 29 22:33:12 2010 From: Blogtest77 at gmail.com (Blog) Date: Sat, 30 Jan 2010 09:33:12 +0600 Subject: myths about python 3 In-Reply-To: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On 1/28/2010 2:56 AM, John Nagle wrote: > Daniel Fetchinson wrote: > 1. Python 3 is supported by major Linux distributions. > > FALSE - most distros are shipping with Python 2.4, or 2.5 at best. Where did you come up with that information? Almost all of the major distros ship with 2.6.x - CentOS, OpenSuSe, Ubuntu, Fedora. (Debian does ship with 2.5, but the next major release "sid' is due out in Q2) > Arguably, Python 3 has been rejected by the market. Instead, there's > now Python 2.6, Python 2.7, and Python 2.8. Python 3 has turned into > a debacle like Perl 6, now 10 years old. WTF? Where'd you hear about version 2.8? FRI, 2.7 is and will be THE LAST version of the 2.x series - "the" End-Of-Life for Python 2. At least py3k is a real product - unlike the perl 6 vaporware. That said, I think python 2 will be the dominant player in the market for the next couple of years. However, there's been some momentum behind Py3k recently. Unladen swallow is going to be merged with Py3k (not 2.x for which it was originally developed). I hear the django guys have made huge progress in porting the framework to Py3k (although it will be a good few months before the code is released in the wild) From Blogtest77 at gmail.com Fri Jan 29 22:50:39 2010 From: Blogtest77 at gmail.com (Blog) Date: Sat, 30 Jan 2010 09:50:39 +0600 Subject: myths about python 3 In-Reply-To: <7xzl3zq77u.fsf@ruckus.brouhaha.com> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <7xzl3zq77u.fsf@ruckus.brouhaha.com> Message-ID: On 1/28/2010 8:44 AM, Paul Rubin wrote: > Steve Holden writes: >> Kindly confine your debate to the facts and keep the snide remarks to >> yourself. Like it or not Python 3 is the future, and unladen swallow's >> recent announcement that they would target only Python 3 represented a >> ground-breaking advance for the language. > > My take on things is that doing unladen swallow really "right" will > require yet more incompatible changes; i.e., the result will either > still leave quite a bit of performance on the table, or else it won't be > compatible with the current specification of Python 3 and they'll > presumably have to call it Python 4. And if Python 4 is as good as I > believe it could possibly be, then it might get wide acceptance before > Python 3 really has all that much uptake. If I have to accept > incompatibility anyway, and Python 4 gives huge improvements while > Python 3's improvements are tiny or moderate, why not skip over Python 3? There's a prime example - it's called Windows Vista! ;) From ben+python at benfinney.id.au Fri Jan 29 23:06:29 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 30 Jan 2010 15:06:29 +1100 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <87zl3wtewq.fsf@benfinney.id.au> Blog writes: > (Debian does ship with 2.5, but the next major release "sid' is due > out in Q2) Sid is the perpetual development playground (?unstable?), never released as a suite, but a proving ground for packages to determine their fitness for going to the next level of testing. The next-to-be-released suite is Squeeze (currently ?testing?), which has Python 2.5 (the default ?python?) and Python 2.6. -- \ ?If you define cowardice as running away at the first sign of | `\ danger, screaming and tripping and begging for mercy, then yes, | _o__) Mr. Brave man, I guess I'm a coward.? ?Jack Handey | Ben Finney From steve at REMOVE-THIS-cybersource.com.au Sat Jan 30 01:06:18 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 30 Jan 2010 06:06:18 GMT Subject: Code snippet: dualmethod descriptor Message-ID: <0373b8b9$0$1309$c3e8da3@news.astraweb.com> I came up with this descriptor a few years ago, but never used it for anything. I've just found an actual use for it in my own code, so I thought I'd make it public for anyone else who might have a use for it. Use-case: if you have a class without an __init__ or __new__ method, then this may be of use to you. It enables the caller to call methods on either the class, or an instance, and the first argument passed the method will be the class or the instance respectively. This differs from classmethods, which always passes the class, and staticmethods, which don't pass either. The name "dualmethod" is my own. Here's the implementation: class dualmethod(object): """Descriptor implementing dualmethods (combination class/instance method). Returns a method which takes either an instance or a class as the first argument. When called on an instance, the instance is passed as the first argument. When called as a class, the class itself is passed instead. >>> class Example(object): ... @dualmethod ... def method(this): ... if type(this) is type: ... print("I am the class '%s'." % this.__name__) ... else: ... print("I am an instance of the class '%s'." % ... this.__class__.__name__) ... >>> Example.method() I am the class 'Example'. >>> Example().method() I am an instance of the class 'Example'. """ def __init__(self, func): self.func = func def __get__(self, obj, cls=None): if obj is None: obj = cls or type(obj) def newfunc(*args, **kwargs): return self.func(obj, *args, **kwargs) return newfunc (Tested with Python 2.4, 2.5, 2.6, 3.0 and 3.1.) My use-case is a class that has constants in class attributes, and methods which refer to those constants. Because they are constants, there is no reason to give each instance a separate reference to the attribute, hence the attribute is on the class: class Example: ATTR = "something" @dualmethod def method(this): return this.ATTR + " else" Normally I have no reason to instantiate the class: there is no __init__ or __new__ method. Normally I would use classmethod, but every now and again I want to change the behaviour by modifying ATTR, and using class- methods would force me to subclass in order to make that change. dualmethod lets me get the best of both worlds. If I'm using the default value of ATTR, I can just call the method on the class without bothering to instantiate it. But if I want to customise the behaviour of the method, rather than subclassing, I can instantiate and add an instance attribute: >>> Example.method() 'something else' >>> x = Example() >>> x.ATTR = "nothing" >>> x.method() 'nothing else' >>> Example.method() # default still works as expected 'something else' without effecting the default behaviour. I hope this is useful to you. You can use this without condition, although I would be grateful for credit. -- Steven From stefan_ml at behnel.de Sat Jan 30 01:09:26 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 30 Jan 2010 07:09:26 +0100 Subject: C API: module cleanup function In-Reply-To: References: Message-ID: <4b63cd1b$0$7618$9b4e6d93@newsspool1.arcor-online.net> Mr.M, 29.01.2010 23:50: > I can't figure out if there is a way to run a specialized cleanup > function when a module needs to be "unloaded" (i.e. just before a > reload() or when i quit the interpreter). > > I'm thinking of something like tp_dealloc. > > If I call Py_InitModule3 and look at module->ob_type->tp_dealloc, I find > that Python provides a default tp_dealloc for me. > > Now, suppose my module needs to allocate some resources at startup, I'm > not sure, but I think I'd have to do it in my PyMODINIT_FUNC, right? > > But, if I reload() my module or if I quit the Python interpreter, I'd > like to free those resources (before allocate them again, in case of a > reload). > > Is there a way to make this work? Gabriel already pointed you to the module cleanup support in Py3, which can be used to provide reload capabilities to your module. In Py2, there are at least some ways to free resources when terminating the interpreter. See the "atexit" module and the Py_AtExit() function: http://docs.python.org/library/atexit.html http://docs.python.org/c-api/sys.html Note that both have their specific limitations, though, as you can see from the docs. Also note that it might help you to take a look at Cython, a Python-to-C compiler for writing fast C extensions. It has an option for generating module-level cleanup code automatically, and generally simplifies writing binary extension modules quite a bit. Stefan From austin.bingham at gmail.com Sat Jan 30 02:10:39 2010 From: austin.bingham at gmail.com (Austin Bingham) Date: Sat, 30 Jan 2010 08:10:39 +0100 Subject: Some C-API functions clear the error indicator? In-Reply-To: References: Message-ID: The original post was, in a nutshell, the "use case"; it's very scaled down the from the real example, and not intended to "make sense". The notion on which I was (apparently incorrectly) basing my exception translation was that I could 1) get and reset references to the error indicators, 2) call other python methods that don't themselves throw, and 3) later retrieve the same "active" exceptions. I was relying on this ability to "re-fetch" exceptions insofar as the functions in my code which dealt with python exceptions all looked up the exception separately. The predicate that "a successful function won't modify the error indicators" appears to be wrong, however, and I've modified my code accordingly. Austin On Sat, Jan 30, 2010 at 1:11 AM, Gabriel Genellina wrote: > En Fri, 29 Jan 2010 18:25:14 -0300, Austin Bingham > escribi?: > >> Maybe I'm not following what you're saying. In my case, I already know >> that an exception has been thrown. In the course of processing that >> exception, I call another function which, for whatever reason and even >> when it succeeds, clears the exception indicators. How can I address >> this issue by checking function calls for failure? > > Maybe if you provide an actual use case we can suggest how to handle it. The > code in your original post does not make any sense to me (except by showing > that PyImport_ImportModule does clear the error indicator). If you already > know there was an error, and you even have retrieved the error details, why > do you care if the error indicator gets reset? > > -- > Gabriel Genellina > > -- > http://mail.python.org/mailman/listinfo/python-list > From austin.bingham at gmail.com Sat Jan 30 02:17:30 2010 From: austin.bingham at gmail.com (Austin Bingham) Date: Sat, 30 Jan 2010 08:17:30 +0100 Subject: Some C-API functions clear the error indicator? In-Reply-To: References: Message-ID: That makes a lot of sense. And if I take the approach that any Py* function might do this, it actually looks like I can simplify my code (rather than managing some list of ill-behaved functions or something.) Thanks! On Fri, Jan 29, 2010 at 3:58 PM, Duncan Booth wrote: > Austin Bingham wrote: > >> The functions that do this don't seem to indicate in their >> documentation that this will happen. So first, does anyone know why >> this is happening? Is it because of the context in which I'm making >> the calls? Is there any pattern or reason behind which functions will >> do this? Or am I just doing something wrong? >> > (Just guessing here) > I would expect that any function that executes Python code will clear the > error. > > I think that has to happen otherwise the Python code will throw an > exception whenever it gets round to checking for errors. In the past I've > found that if you fail to check for an error in C code before returning to > the interpreter you get the exception thrown a few instructions later, so > something similar would probably happen if you call other Python code from > C. > > If it is anything that executes Python then that would include any function > that creates or destroys an object with Python constructor or destructor > code. or that compares or otherwise operates on instances defined in > Python. In particular it might mean that any function that doesn't appear > to clear the error could do so in a slightly different situation. > > -- > Duncan Booth http://kupuguy.blogspot.com > -- > http://mail.python.org/mailman/listinfo/python-list > From __peter__ at web.de Sat Jan 30 04:28:19 2010 From: __peter__ at web.de (Peter Otten) Date: Sat, 30 Jan 2010 10:28:19 +0100 Subject: Function name unchanged in error message References: Message-ID: Gabriel Genellina wrote: > En Fri, 29 Jan 2010 13:09:40 -0300, Michele Simionato > escribi?: > >> On Jan 29, 2:30 pm, andrew cooke wrote: >>> Is there any way to change the name of the function in an error >>> message? In the example below I'd like the error to refer to bar(), >>> for example (the motivation is related function decorators - I'd like >>> the wrapper function to give the same name) >> >> Use the decorator module which does the right thing: >> http://pypi.python.org/pypi/decorator > > The decorator module is a very fine addition to anyone's tool set -- but > in this case it is enough to use the wraps() function from the functools > standard module. I don't know about the decorator module, but functools.wraps() doesn't affect the error message: >>> from functools import wraps >>> def f(): pass ... >>> @wraps(f) ... def g(): pass ... >>> g.__name__ 'f' >>> try: g(42) ... except TypeError as e: ... print e ... g() takes no arguments (1 given) Peter From davea at ieee.org Sat Jan 30 05:07:17 2010 From: davea at ieee.org (Dave Angel) Date: Sat, 30 Jan 2010 05:07:17 -0500 Subject: Sleep timer but still responsive? In-Reply-To: References: <8c76dbf2-8853-48cc-9ca9-37929936f73a@b35g2000vbc.googlegroups.com> Message-ID: <4B6404D5.4070809@ieee.org> JohnnyFive wrote: > On Jan 29, 9:33 am, Andreas Tawn wrote: > >>> On Jan 28, 4:55 pm, "Gabriel Genellina" >>> wrote: >>> >>>> Please provide more details. What do you want your program to do >>>> >>> while >>> >>>> sleeping? What kind of actions do you want a response to? >>>> Do you have a GUI? A curses-based interfase? >>>> >>>> -- >>>> Gabriel Genellina >>>> >>> My app is purely console based. I just don't want the console to lock >>> up (on Windows using time.sleep(x) causes the console to become >>> unresponsive until the timer is done), and I want people to be able to >>> CTRL+C to stop the script if need be (which can't be done if it's >>> unresponsive!). >>> >>> Thanks. >>> >> How about this? Responds to ctrl+c, but still sleeps. >> >> import time >> >> def responsiveSleep(n): >> while n > 0: >> time.sleep(1) >> n -= >> >> Cheers, >> >> Drea >> > > Thanks for the ideas! Maybe it's just my computer, but using your > solution still causes the prompt to become unresponsive during the > sleeps. > > I am using 2.6.4 btw. It's not a major deal though, I just thought > there had to be a way to do this fairly easily. > > You responded to my message off-list, so I have to paste it here. """ Dave, Thanks for the response. Here's my scenario. I have a program that checks an ftp site every hour for updated files, downloads them, and then sleeps again for another hour when done. It's 100% console. When I run the app, no matter how I try, I can't get the window to act normal during the sleep cycle. I've tried putting the sleep in another thread and have the main thread.join(), but it all has the same behavior: the console window behaves like it is crashing. Using the small sleep() increments also causes the same thing, the becomes undraggable for periods of time, and again acts like the program has crashed (even though it's just asleep). I am running a version of it right now, and i've managed to minimize it, but it won't maximize so I can ctrl+c it. All it's doing is exactly what Andreas recommended. This is not user-friendly behavior! I've also tried having the main thread sit at a raw_input, and have another thread have the timer, but there was odd behavior when I wanted the program to continue what it was doing, even if the user hadn't pressed "enter" to get passed the command prompt. """ You probably need to tell us your complete environment. I know you're running 2.6.4, but don't know which version of Windows, so I'll guess XP. You're running in a cmd.exe console. There must be something else going on in your system, since a console does not become unresponsive during a sleep. The python program is sleeping, but the console is very much alive; it's a separate process. So dragging, minimizing, restoring and maximizing is unaffected. Try the following simple script from a cmd console: import time print print print "going to sleep" print "Use Ctrl-C to get my attention, and end the program", time.sleep(30) print "done" On XP SP3, running Python 2.6.4, this ignores regular keystrokes, but responds nicely to control C. And it can be dragged around, resized, minimized, etc. with no problem. If you get different behavior, tell us more precisely how your environment differs from my guesses. Once we've solved Ctrl-C, drag the console, minimize the console, then maybe you're going to request "respond to other keystrokes". It can all be done, but only with more careful wording than "behaves like its crashing." Regards, DaveA From mattsteel at hotmail.it Sat Jan 30 05:25:01 2010 From: mattsteel at hotmail.it (Mattsteel) Date: Sat, 30 Jan 2010 02:25:01 -0800 (PST) Subject: Funny behaviour with __future__ and doctest between 2.6 and 3.1 References: <3a02ac42-337b-4778-99e0-1309e012163d@g29g2000yqe.googlegroups.com> Message-ID: <20136170-8eda-41c2-a0ff-7a701e6c006b@z26g2000yqm.googlegroups.com> On 29 Gen, 17:30, Peter Otten <__pete... at web.de> wrote: > I think you can work around the problem. The following should pass in Python > 2.6 and 3.1: > > '''>>> concat('hello','world') == 'hello world' > True > ''' I see. Thank for the concern. M. From solipsis at pitrou.net Sat Jan 30 08:11:24 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 30 Jan 2010 13:11:24 +0000 (UTC) Subject: Some C-API functions clear the error indicator? References: Message-ID: Le Fri, 29 Jan 2010 22:25:14 +0100, Austin Bingham a ?crit?: > Maybe I'm not following what you're saying. In my case, I already know > that an exception has been thrown. In the course of processing that > exception, I call another function which, for whatever reason and even > when it succeeds, clears the exception indicators. If you know an error occurred and need to retain it somewhere, just use PyErr_Fetch() and PyErr_Restore(). Regards Antoine. From mrm at unknown.nospam Sat Jan 30 08:24:51 2010 From: mrm at unknown.nospam (Mr.M) Date: Sat, 30 Jan 2010 13:24:51 GMT Subject: C API: module cleanup function In-Reply-To: <4b63cd1b$0$7618$9b4e6d93@newsspool1.arcor-online.net> References: <4b63cd1b$0$7618$9b4e6d93@newsspool1.arcor-online.net> Message-ID: Stefan Behnel ha scritto: > Gabriel already pointed you to the module cleanup support in Py3, which can > be used to provide reload capabilities to your module. > > In Py2, there are at least some ways to free resources when terminating the > interpreter. See the "atexit" module and the Py_AtExit() function: > > http://docs.python.org/library/atexit.html > http://docs.python.org/c-api/sys.html > > Note that both have their specific limitations, though, as you can see from > the docs. > > Also note that it might help you to take a look at Cython, a Python-to-C > compiler for writing fast C extensions. It has an option for generating > module-level cleanup code automatically, and generally simplifies writing > binary extension modules quite a bit. > > Stefan Thank you very much Stefan for your reply, I'll study the sources you have pointed me to. Could I allocate my resources in a "static" object (without publishing the type of that object so that I can't instantiate another one) linked to my module? This way, when I stop the interpreter, the object will be destroyed calling its destructor. There's something I'm missing? Thank you, Luca. From keith at nekotaku.com Sat Jan 30 09:21:01 2010 From: keith at nekotaku.com (KB) Date: Sat, 30 Jan 2010 06:21:01 -0800 (PST) Subject: Utility to screenscrape sites using javascript ? Message-ID: <4c4298c0-b93d-4691-bceb-e982373f87e3@f17g2000prh.googlegroups.com> Hi there, I have a service I subscribe to that uses javascript to stream news. Ideally I would like to use python to parse the information for me. Note there is an option to take a static snapshot of the current stream but that is still done via Javascript. (I can reference the snapshot with a unique URL though, so I can pass that to a parser as long as it can "resolve" the javascript and get at the content) I had a quick look at Windmill but it doesn't appear to be what I am looking for. Does anyone else have any experience in screenscraping sites that utilise javascript? Can you share how you did it and perhaps some sample code if possible? Thanks a bunch! From 3lvss0809 at gmail.com Sat Jan 30 09:37:48 2010 From: 3lvss0809 at gmail.com (peskar.miha@hotmail.com) Date: Sat, 30 Jan 2010 06:37:48 -0800 (PST) Subject: Microsoft Office Word and Python (Win XP) References: Message-ID: <85ee4831-3b9c-4096-b32a-387e59d6e7c1@r6g2000yqn.googlegroups.com> 21 days has passed and still noone is willing to help :-( From stefan_ml at behnel.de Sat Jan 30 09:44:56 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 30 Jan 2010 15:44:56 +0100 Subject: C API: module cleanup function In-Reply-To: References: <4b63cd1b$0$7618$9b4e6d93@newsspool1.arcor-online.net> Message-ID: <4b6445e9$0$7621$9b4e6d93@newsspool1.arcor-online.net> Mr.M, 30.01.2010 14:24: > Could I allocate my resources in a "static" object linked to my module? Sure, but you will have to let CPython know about it so that it can see that the reference is only held by the module that it is cleaning up. Otherwise it can't collect the reference. This works easily in Py3 but not in Py2. In Cython, you would just write cdef ResourceKeeper resource_keeper = ResourceKeeper() at the module level and let Cython generate the cleanup code for it. In C, you have to implement an atexit function and decref the resource object either there or in the module cleanup function of Py3. > (without publishing > the type of that object so that I can't instantiate another one) Note that there is the type() builtin function which returns the type given an instance. So you can't hide the type. Stefan From alfps at start.no Sat Jan 30 09:54:21 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sat, 30 Jan 2010 15:54:21 +0100 Subject: Microsoft Office Word and Python (Win XP) In-Reply-To: <85ee4831-3b9c-4096-b32a-387e59d6e7c1@r6g2000yqn.googlegroups.com> References: <85ee4831-3b9c-4096-b32a-387e59d6e7c1@r6g2000yqn.googlegroups.com> Message-ID: * peskar.miha at hotmail.com: > 21 days has passed and still noone is willing to help :-( Did you see the reply from Marco Nawin? If you don't see that reply, dated (a bit less than) 2 hours after your original posting on the 9th, I can repost it here. If you have any follow-up questions just post them. Cheers & hth., - Alf From steve at holdenweb.com Sat Jan 30 10:01:33 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 30 Jan 2010 10:01:33 -0500 Subject: Microsoft Office Word and Python (Win XP) In-Reply-To: <85ee4831-3b9c-4096-b32a-387e59d6e7c1@r6g2000yqn.googlegroups.com> References: <85ee4831-3b9c-4096-b32a-387e59d6e7c1@r6g2000yqn.googlegroups.com> Message-ID: peskar.miha at hotmail.com wrote: > 21 days has passed and still noone is willing to help :-( Y'know, the Internet isn't a magic lantern. Perhaps you need to ask your question differently, or provide more information. Maybe there's something *you* could do to help you get closer to the answer to *your* question. Read the question at the time, don't remember having much to contribute right then. But you lucked out today: it's Saturday morning and I am at home in my family room, and it's snowing outside. Hey, at least you do know already that people will respond to your questions. Look for a (second-hand?) copy of Andy Robinson's and Mark Hammond's "Python Programming on Win32", which taught me a lot about COM I have thankfully since been able to forget. Go to Microsoft channels for details of the Office object models, and good luck with that (the quality and quantity of that used to be something of a movable feast, though I imagine things have improved since). You might also ask yourself whether IronPython and the .NET interfaces aren't a better and more modern way of controlling Microsoft products. The .NET documentation is pretty well organized and complete, I believe. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From mrm at unknown.nospam Sat Jan 30 10:02:25 2010 From: mrm at unknown.nospam (Mr.M) Date: Sat, 30 Jan 2010 15:02:25 GMT Subject: C API: module cleanup function In-Reply-To: <4b6445e9$0$7621$9b4e6d93@newsspool1.arcor-online.net> References: <4b63cd1b$0$7618$9b4e6d93@newsspool1.arcor-online.net> <4b6445e9$0$7621$9b4e6d93@newsspool1.arcor-online.net> Message-ID: <5SX8n.18688$Ua.307@twister2.libero.it> Stefan Behnel ha scritto: > Note that there is the type() builtin function which returns the type given > an instance. So you can't hide the type. Argh! Yes, you are right. So I'd better have a look to Cython, right? L- From fsenzeru at gmail.com Sat Jan 30 10:32:41 2010 From: fsenzeru at gmail.com (Dan Brown) Date: Sat, 30 Jan 2010 07:32:41 -0800 (PST) Subject: list.extend([]) Question Message-ID: <088e7a24-b0d0-4d43-bee7-193e5eaefe57@b7g2000pro.googlegroups.com> Why does extending a list with the empty list result in None? It seems very counterintuitive to me, at least --- I expected ['a'].extend ([]) to result in ['a'], not None. From alfps at start.no Sat Jan 30 10:38:07 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sat, 30 Jan 2010 16:38:07 +0100 Subject: list.extend([]) Question In-Reply-To: <088e7a24-b0d0-4d43-bee7-193e5eaefe57@b7g2000pro.googlegroups.com> References: <088e7a24-b0d0-4d43-bee7-193e5eaefe57@b7g2000pro.googlegroups.com> Message-ID: * Dan Brown: > Why does extending a list with the empty list result in None? It > seems very counterintuitive to me, at least --- I expected ['a'].extend > ([]) to result in ['a'], not None. It does. 'extend' is an operation that /modifies/ the array. It just returns None as its expression result, in the same way as e.g. the Python 3.x 'print' (another pure "doer" operation). >>> L = ['a'] >>> L ['a'] >>> L2 = L.extend( [] ) >>> L2 >>> L2 is None True >>> L ['a'] >>> _ Cheers & hth., - Alf From andreengels at gmail.com Sat Jan 30 10:41:45 2010 From: andreengels at gmail.com (Andre Engels) Date: Sat, 30 Jan 2010 16:41:45 +0100 Subject: list.extend([]) Question In-Reply-To: <088e7a24-b0d0-4d43-bee7-193e5eaefe57@b7g2000pro.googlegroups.com> References: <088e7a24-b0d0-4d43-bee7-193e5eaefe57@b7g2000pro.googlegroups.com> Message-ID: <6faf39c91001300741u7f3973at30ac32e8f421cbf@mail.gmail.com> On Sat, Jan 30, 2010 at 4:32 PM, Dan Brown wrote: > Why does extending a list with the empty list result in None? ?It > seems very counterintuitive to me, at least --- I expected ['a'].extend > ([]) to result in ['a'], not None. Extend is a method of the list. The list itself is changed, it does not return itself: >>> A = [1,2] >>> B = [3,4] >>> C = A.extend(B) >>> C >>> C is None True >>> A [1, 2, 3, 4] Thus, nothing special about extend([]), this is the general behaviour of extend -- Andr? Engels, andreengels at gmail.com From fsenzeru at gmail.com Sat Jan 30 10:43:37 2010 From: fsenzeru at gmail.com (Dan Brown) Date: Sat, 30 Jan 2010 07:43:37 -0800 (PST) Subject: list.extend([]) Question References: <088e7a24-b0d0-4d43-bee7-193e5eaefe57@b7g2000pro.googlegroups.com> Message-ID: <1dbe4fc8-c712-4524-87b0-41ec19c9117e@v37g2000prh.googlegroups.com> On Jan 30, 8:38?am, "Alf P. Steinbach" wrote: > It does. > > 'extend' is an operation that /modifies/ the array. > > It just returns None as its expression result, in the same way as e.g. the > Python 3.x 'print' (another pure "doer" operation). > > ? ?>>> L = ['a'] > ? ?>>> L > ? ?['a'] > ? ?>>> L2 = L.extend( [] ) > ? ?>>> L2 > ? ?>>> L2 is None > ? ?True > ? ?>>> L > ? ?['a'] > ? ?>>> _ > > Cheers & hth., > > - Alf Aha. Well, I feel a bit silly for not thinking to try it that way. Thanks! From steve at holdenweb.com Sat Jan 30 10:45:45 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 30 Jan 2010 10:45:45 -0500 Subject: list.extend([]) Question In-Reply-To: <088e7a24-b0d0-4d43-bee7-193e5eaefe57@b7g2000pro.googlegroups.com> References: <088e7a24-b0d0-4d43-bee7-193e5eaefe57@b7g2000pro.googlegroups.com> Message-ID: Dan Brown wrote: > Why does extending a list with the empty list result in None? It > seems very counterintuitive to me, at least --- I expected ['a'].extend > ([]) to result in ['a'], not None. How very inconvenient of Python! What it actually does is create an anonymous list containing only the element 'a', and leave it unchanged by extending it with an empty list. Since there is no longer any reference to the list it has become garbage. Contrast that with: >>> lst = ['a'] >>> lst.extend([]) >>> lst ['a'] >>> lst.append([]) >>> lst ['a', []] >>> lst.extend(['1']) >>> lst ['a', [], '1'] >>> As you can see by the absence of output, both the .extend() and .append() list methods return None. They mutate the list instance upon which they are called. In your example you were expecting the methods to return the mutated list. They don't. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From nobody at nowhere.com Sat Jan 30 11:01:19 2010 From: nobody at nowhere.com (Nobody) Date: Sat, 30 Jan 2010 16:01:19 +0000 Subject: Default path for files References: Message-ID: On Sun, 24 Jan 2010 15:08:15 +0000, Rotwang wrote: > Hi all, can anybody tell me whether there's a way to change the default > location for files to be opened by open()? I'd like to be able to create > files somewhere other than my Python folder without having to write the > full path in the filename every time. Sorry if this is a stupid question, > I don't know much about programming. If you pass a relative pathname to open() (or any other function which expects a filename), it will be interpreted relative to the current directory. Given that the current directory always seems to be the Python directory, and you refer to it as a "folder", I'm guessing that you're running Python on Windows via a shortcut in the Start Menu or on the desktop. In which case, the ideal solution is probably to open the Properties dialog for the shortcut and change the "Start in" field to your "My Documents" directory (or some subdirectory of it). Python itself won't care which directory it starts in. From nobody at nowhere.com Sat Jan 30 11:29:02 2010 From: nobody at nowhere.com (Nobody) Date: Sat, 30 Jan 2010 16:29:02 +0000 Subject: myths about python 3 References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On Wed, 27 Jan 2010 12:56:10 -0800, John Nagle wrote: > Arguably, Python 3 has been rejected by the market. Arguably, Python 3 has not yet been accepted by the market. Part of it is down to a catch-22: applications won't use Python 3 if the libraries on which they depend don't support it, and support for Python 3 by libraries will be influenced by the perceived demand. OTOH, it's safe to assume that there will remain areas where Python 2 is preferred. Primarily Unix scripting, where most data is byte strings with the encoding either unknown or irrelevant. That alone will ensure that Python 2 is alive and well even as Python 4 is released. Even if python.org doesn't support Python 2, it's a safe bet that e.g. ActiveState will. From nobody at nowhere.com Sat Jan 30 11:43:18 2010 From: nobody at nowhere.com (Nobody) Date: Sat, 30 Jan 2010 16:43:18 +0000 Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> Message-ID: On Wed, 27 Jan 2010 15:29:05 -0800, Jonathan Gardner wrote: > There's a lot of "magic" in Ruby as well. For instance, function calls are > made without parentheses. That's also true for most functional languages, e.g. Haskell and ML, as well as e.g. Tcl and most shells. Why require "f(x)" or "(f x)" if "f x" will suffice? > Python is much, much cleaner. I don't know how anyone can honestly say > Ruby is cleaner than Python. I'm not familiar with Ruby, but most languages are cleaner than Python once you get beyond the "10-minute introduction" stage. From kw at codebykevin.com Sat Jan 30 11:45:17 2010 From: kw at codebykevin.com (Kevin Walzer) Date: Sat, 30 Jan 2010 11:45:17 -0500 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: On 1/30/10 11:29 AM, Nobody wrote: > > Arguably, Python 3 has not yet been accepted by the market. > > Part of it is down to a catch-22: applications won't use Python 3 if the > libraries on which they depend don't support it, and support for Python 3 > by libraries will be influenced by the perceived demand. This is part of my reason for not yet moving to Python 3--several libraries that I will need do not currently support Python 3. -- Kevin Walzer Code by Kevin http://www.codebykevin.com From nobody at nowhere.com Sat Jan 30 11:54:52 2010 From: nobody at nowhere.com (Nobody) Date: Sat, 30 Jan 2010 16:54:52 +0000 Subject: Keyboard input References: Message-ID: On Fri, 29 Jan 2010 16:53:43 +0000, Mr.SpOOn wrote: > I'm using the raw_input method and it works fine, but I noted that I can't > use backspace and when I accidentally press shift space (because I need to > input uppercase letters separated by a space) it writes some strange > characters. > > So, is there another way to get keyboard input? sys.stdin.readline() From tanix at mongo.net Sat Jan 30 11:58:34 2010 From: tanix at mongo.net (tanix) Date: Sat, 30 Jan 2010 16:58:34 GMT Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> Message-ID: In article , Nobody wrote: >On Wed, 27 Jan 2010 15:29:05 -0800, Jonathan Gardner wrote: > >> There's a lot of "magic" in Ruby as well. For instance, function calls are >> made without parentheses. > >That's also true for most functional languages, e.g. Haskell and ML, as >well as e.g. Tcl and most shells. Why require "f(x)" or "(f x)" if "f x" >will suffice? > >> Python is much, much cleaner. I don't know how anyone can honestly say >> Ruby is cleaner than Python. > >I'm not familiar with Ruby, but most languages are cleaner than Python >once you get beyond the "10-minute introduction" stage. I'd have to agree. The only ones that beat Python in that department are Javascript and PHP. Plus CSS and HTML if you can call those languages. The very idea of using a number of blanks to identify your block level is as insane as it gets. First of all, combinations of blanks and tabs, depending on how your ide is setup to expand tabs, may get you bugs, you'd never imagine in your wild dreams. Braces is the most reliable way to identify blocks. Sane compilers ignore blanks altogether. -- Programmer's Goldmine collections: http://preciseinfo.org Tens of thousands of code examples and expert discussions on C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript, PHP, organized by major topics of language, tools, methods, techniques. All collections are fully searchable down to specific chapter. From nobody at nowhere.com Sat Jan 30 12:00:43 2010 From: nobody at nowhere.com (Nobody) Date: Sat, 30 Jan 2010 17:00:43 +0000 Subject: Utility to screenscrape sites using javascript ? References: <4c4298c0-b93d-4691-bceb-e982373f87e3@f17g2000prh.googlegroups.com> Message-ID: On Sat, 30 Jan 2010 06:21:01 -0800, KB wrote: > I have a service I subscribe to that uses javascript to stream news. > Ideally I would like to use python to parse the information for me. Note > there is an option to take a static snapshot of the current stream but > that is still done via Javascript. (I can reference the snapshot with a > unique URL though, so I can pass that to a parser as long as it can > "resolve" the javascript and get at the content) > > I had a quick look at Windmill but it doesn't appear to be what I am > looking for. Does anyone else have any experience in screenscraping sites > that utilise javascript? Can you share how you did it and perhaps some > sample code if possible? There's a Python interface to SpiderMonkey (Mozilla's JavaScript interpreter): http://pypi.python.org/pypi/python-spidermonkey From e_d_k at yahoo.com Sat Jan 30 12:06:26 2010 From: e_d_k at yahoo.com (Ed Keith) Date: Sat, 30 Jan 2010 09:06:26 -0800 (PST) Subject: Python and Ruby In-Reply-To: Message-ID: <162882.48205.qm@web58906.mail.re1.yahoo.com> --- On Sat, 1/30/10, Nobody wrote: > From: Nobody > > Python is much, much cleaner. I don't know how anyone > can honestly say > > Ruby is cleaner than Python. > > I'm not familiar with Ruby, but most languages are cleaner > than Python > once you get beyond the "10-minute introduction" stage. You need to be clear about what you mean by "clean". Is Python scoping "clean"? I suspect lots of people would argue either side. -EdK Ed Keith e_d_k at yahoo.com Blog: edkeith.blogspot.com From lists at cheimes.de Sat Jan 30 12:47:17 2010 From: lists at cheimes.de (Christian Heimes) Date: Sat, 30 Jan 2010 18:47:17 +0100 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <4B6470A5.5080308@cheimes.de> Blog wrote: > WTF? Where'd you hear about version 2.8? FRI, 2.7 is and will be THE > LAST version of the 2.x series - "the" End-Of-Life for Python 2 Where do you get your information from? Your answer is the first that clearly marks the end of lifetime for the 2.x series. I didn't know that and I'm a Python core dev as well as a PSF member ... *scnr* Christian From Blogtest77 at gmail.com Sat Jan 30 13:08:49 2010 From: Blogtest77 at gmail.com (Blog) Date: Sun, 31 Jan 2010 00:08:49 +0600 Subject: myths about python 3 In-Reply-To: <4B6470A5.5080308@cheimes.de> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <4B6470A5.5080308@cheimes.de> Message-ID: On 1/30/2010 11:47 PM, Christian Heimes wrote: > Blog wrote: >> WTF? Where'd you hear about version 2.8? FRI, 2.7 is and will be THE >> LAST version of the 2.x series - "the" End-Of-Life for Python 2 > > Where do you get your information from? Your answer is the first that > clearly marks the end of lifetime for the 2.x series. I didn't know that > and I'm a Python core dev as well as a PSF member ... *scnr* > > Christian Well, here's what the Python 2.7 alpha 2 (January 9th, 2010) release note says: "Python 2.7 is scheduled to be the last major version in the 2.x series before it moves into 5 years of bugfix-only mode. This release contains many of the features that were first released in Python 3.1..." Ref: http://www.python.org/download/releases/2.7/ Here are some more resources: "Python 2 nears end of life" "Python 2.7, expected to be the last major version of the 2.x series of the dynamic language, was released as a second alpha earlier month by the Python Software Foundation, with the final release set for June. When 2.7 is released, the 2.x line will move into five years of a bug fix-only mode." http://www.computerworlduk.com/technology/development/software/news/index.cfm?newsid=18331 "2.7's the end of the line. There was some discussion around the release of 2.6 as to how far the 2.x series should go, and the conclusion which came out of it was that 2.7 is it. Beyond this, the world is 3.x (which, given the time it takes OS distributors to catch up -- most are still on 2.5 -- is about right with the projected time frame for most projects to port)." http://www.reddit.com/r/Python/comments/aoloc/python_27_alpha_2_has_been_released/ "Let?s also account for the fact that, as of this writing, Python 2.7 (scheduled for next year) is *intended* to be the End of Life release of the Python 2.x syntax ? Python 3 being the next evolutionary step." http://jessenoller.com/2009/12/04/pythons-moratorium-lets-think-about-this/ Of course, this isn't written in stone - there may very well be another version after 2.7. However, at this instant, it does look like 2.7 will be the veritable last version. Best regards. From Blogtest77 at gmail.com Sat Jan 30 13:09:52 2010 From: Blogtest77 at gmail.com (Blog) Date: Sun, 31 Jan 2010 00:09:52 +0600 Subject: myths about python 3 In-Reply-To: <87zl3wtewq.fsf@benfinney.id.au> References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> <87zl3wtewq.fsf@benfinney.id.au> Message-ID: On 1/30/2010 10:06 AM, Ben Finney wrote: > Blog writes: > >> (Debian does ship with 2.5, but the next major release "sid' is due >> out in Q2) > > Sid is the perpetual development playground (?unstable?), never released > as a suite, but a proving ground for packages to determine their fitness > for going to the next level of testing. > > The next-to-be-released suite is Squeeze (currently ?testing?), which > has Python 2.5 (the default ?python?) and Python 2.6. > Oops! My bad! I actually meant Squeeze. Thanks for catching the "typo". From nagle at animats.com Sat Jan 30 13:23:39 2010 From: nagle at animats.com (John Nagle) Date: Sat, 30 Jan 2010 10:23:39 -0800 Subject: Threading issue with SQLite In-Reply-To: <09ca16b8-4e10-40f5-a69d-29fcdf6c2593@b9g2000pri.googlegroups.com> References: <09ca16b8-4e10-40f5-a69d-29fcdf6c2593@b9g2000pri.googlegroups.com> Message-ID: <4b647586$0$1636$742ec2ed@news.sonic.net> Jonathan Gardner wrote: > On Jan 29, 8:37 am, Alan Harris-Reid > wrote: >> Questions... >> 1. Is there a large overhead in opening a new SQLite connection for >> each thread (ie. within each method)? > Suggestion: Use something like SQLAlchemy to manage you DB > interactions. One day, you'll move away from SQLite, and you'll be > glad you were programming at a higher level. SQLAlchemy also does > things like connection pooling and such for you. Generally, if you get to the point where you're concerned about concurrency performance issues with SQLite, it's time to upgrade to a more serious database. Especially if you're doing much updating. SQLite can do multiple SELECT operations in parallel, but the entire database is locked for all operations that write. John Nagle From uraniumore238 at gmail.com Sat Jan 30 13:33:42 2010 From: uraniumore238 at gmail.com (uche) Date: Sat, 30 Jan 2010 10:33:42 -0800 (PST) Subject: unencountered error in FFT python Message-ID: Hi, I have the following FFT python code and it doesn't seem to compile correctly. To run it, please create a file called output.csv with 1,2,3,4,5,6,7,8. simply run the main function. I get an error such as the following: x[a], x[b] = x[(a)] + W[(n % N)] * x[(b)], x[(a)] - W[(n % N)] * x [(b)] TypeError: list indices must be integers, not float How can I fixe this problem ? I have tried puttin int on all of the variables, but I don't think that is the intension of the person who wore the original code. #!/usr/bin/python """ FFT using Cooley-Tukey algorithm where N = 2^n. The choice of e^{-j2\pi/N} or e^{j2\pi/N} is made by 'sign=-1' or 'sign=1' respectively. Since I prefer Engineering convention, I chose 'sign=-1' as the default. FFT is performed as follows: 1. bit-reverse the array. 2. partition the data into group of m = 2, 4, 8, ..., N data points. 3. for each group with m data points, 1. divide into upper half (section A) and lower half (section B), each containing m/2 data points. 2. divide unit circle by m. 3. apply "butterfly" operation |a| = |1 w||a| or a, b = a+w*b, a-w*b |b| |1 -w||b| where a and b are data points of section A and B starting from the top of each section, and w is data points along the unit circle starting from z = 1+0j. FFT ends after applying "butterfly" operation on the entire data array as whole, when m = N. """ def main(): array = [] array2 = [] import os import time os.path.exists("input.csv") fin=open('input.csv', 'r') for line in fin: #read the line from the file array=line.split(',') for a in range(len(array)): #convert into integers array2.append((array[a])) Ti=time.time() print (fft(array2)) Tf=time.time() print (("It took"),Tf-Ti,("seconds to calculate an FFT of size"),len(array)) #end timer """ Find 2^n that is equal to or greater than. """ def nextpow2(i): n = 2 while n < i: n = n * 2 return n """ Return bit-reversed list, whose length is assumed to be 2^n: eg. 0111 <--> 1110 for N=2^4. """ def bitrev(x): N, x = len(x), x[:] if N != nextpow2(N): raise ValueError ('N is not power of 2') for i in range(N): k, b, a = 0, N>>1, 1 while b >= a: if b & i: k = k | a if a & i: k = k | b b, a = b>>1, a<<1 if i < k: x[i], x[k] = (x[k],x[i]) return x def fft(x, sign=-1): from cmath import pi, exp N, W = len(x), [] for i in range(N): # exp(-j...) is default W.append(exp(sign * 2j * pi * i / N)) x = bitrev(x) m = 2 while m <= N: for s in range(0,N,m): for i in range (int(m/2)): n = i * N / m a, b = s + i, s + i + m/2 x[a], x[b] = x[(a)] + W[(n % N)] * x[(b)], x[(a)] - W [(n % N)] * x[(b)] m = m * 2 return x From martin at v.loewis.de Sat Jan 30 13:39:24 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Sat, 30 Jan 2010 19:39:24 +0100 Subject: myths about python 3 In-Reply-To: References: <4b60a4c3$0$1655$742ec2ed@news.sonic.net> Message-ID: <4B647CDC.2070409@v.loewis.de> Christian Heimes wrote: > Blog wrote: >> WTF? Where'd you hear about version 2.8? FRI, 2.7 is and will be THE >> LAST version of the 2.x series - "the" End-Of-Life for Python 2 > > Where do you get your information from? It was discussed repeatedly on python-dev, last time when the release announcements for the 2.7 alpha releases were discussed. See http://www.python.org/download/releases/2.7/ Regards, Martin From stefan_ml at behnel.de Sat Jan 30 13:52:57 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 30 Jan 2010 19:52:57 +0100 Subject: unencountered error in FFT python In-Reply-To: References: Message-ID: <4b64800a$0$6735$9b4e6d93@newsspool2.arcor-online.net> uche, 30.01.2010 19:33: > I have the following FFT python code You didn't seriously implement an FFT in plain Python code, did you? FFTs are amongst the first thing that come to my mind when I try to imagine what I'd use NumPy for. (and I *never* used it!) Stefan From aahz at pythoncraft.com Sat Jan 30 13:59:47 2010 From: aahz at pythoncraft.com (Aahz) Date: 30 Jan 2010 10:59:47 -0800 Subject: Default path for files References: Message-ID: In article , Rotwang wrote: > >Hi all, can anybody tell me whether there's a way to change the default >location for files to be opened by open()? I'd like to be able to create >files somewhere other than my Python folder without having to write the >full path in the filename every time. Sorry if this is a stupid >question, I don't know much about programming. from os.path import join BASE = '/path/to/root' f = open(join(BASE, filename)) Trust me, you'll be much happier with this. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ import antigravity From groups_ads12 at yahoo.com Sat Jan 30 14:05:25 2010 From: groups_ads12 at yahoo.com (groups_ads12) Date: Sat, 30 Jan 2010 19:05:25 -0000 Subject: www.assembly.learn.net.in Message-ID: http://www.assembly.learn.net.in/ assembly assembly of god assemble assembly jobs assembly language assembly work assembler electronic assembly pcb assembly assemblies crib assembly cable assembly christian assembly contract assembly furniture assembly hub assembly the assembly assembly code assembly instructions assembly language programming assembly programming document assembly smt assembly x86 assembly axle assembly computer assembly electronics assembly urban assembly wheel bearing assembly assembly magazine assembly services bearing assembly mechanical assembly arm assembly assembly required assembly table assembly tools car assembly medical assembly mips assembly wheel assembly ikea assembly assembly program assembly service assembly worker product assembly assembly instruction assembly language step by step assembly systems assembly testing c# assembly central assembly distributor assembly learn assembly language pc assembly american assembly assembly solutions component assembly hand assembly windows assembly assembly area assembly language program assembly test assembly to c parts assembly 8051 assembler free assembly grand assembly industrial assembly office furniture assembly rear assembly system assembly assembly technologies front assembly 8051 assembly assembly example flat pack furniture assembly hardware assembly full assembly x86 assembly programming 8086 assembly a happy assembly -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Sat Jan 30 14:08:03 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 30 Jan 2010 19:08:03 +0000 Subject: unencountered error in FFT python In-Reply-To: References: Message-ID: <4B648393.4000204@mrabarnett.plus.com> uche wrote: > Hi, > I have the following FFT python code and it doesn't seem to compile > correctly. To run it, please create a file called output.csv with > 1,2,3,4,5,6,7,8. simply run the main function. I get an error such as > the following: > > x[a], x[b] = x[(a)] + W[(n % N)] * x[(b)], x[(a)] - W[(n % N)] * x > [(b)] > TypeError: list indices must be integers, not float > > How can I fixe this problem ? I have tried puttin int on all of the > variables, but I don't think that is the intension of the person who > wore the original code. > Which version of Python are you using? In Python 3 the division operator '/' returns a float, whereas in Python 2 it returns an int if both operands are int. In Python 3 the int division operator is '//', which is also accepted in recent versions of Python 2. [snip] > > os.path.exists("input.csv") > > fin=open('input.csv', 'r') > > for line in fin: #read the line from the file > > array=line.split(',') > These lines should be indented more: > for a in range(len(array)): #convert into integers > > array2.append((array[a])) > array2.append(int(array[a])) [snip] From steve at REMOVE-THIS-cybersource.com.au Sat Jan 30 14:16:04 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 30 Jan 2010 19:16:04 GMT Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> Message-ID: <037471d0$0$1309$c3e8da3@news.astraweb.com> On Sat, 30 Jan 2010 16:58:34 +0000, tanix wrote: > In article , Nobody > wrote: >>On Wed, 27 Jan 2010 15:29:05 -0800, Jonathan Gardner wrote: >> >>> There's a lot of "magic" in Ruby as well. For instance, function calls >>> are made without parentheses. >> >>That's also true for most functional languages, e.g. Haskell and ML, as >>well as e.g. Tcl and most shells. Why require "f(x)" or "(f x)" if "f x" >>will suffice? >> >>> Python is much, much cleaner. I don't know how anyone can honestly say >>> Ruby is cleaner than Python. >> >>I'm not familiar with Ruby, but most languages are cleaner than Python >>once you get beyond the "10-minute introduction" stage. > > I'd have to agree. The only ones that beat Python in that department are > Javascript and PHP. Plus CSS and HTML if you can call those languages. Your sentence makes no sense. You agree that "most" languages are cleaner than Python, and then in the very next sentence, out of the hundreds if not thousands of languages invented, you can only list TWO that are better than Python -- and those are Javascript and PHP!!! That's like saying "most things are harder than diamond -- the only things that beat diamond are jelly and talc". > The very idea of using a number of blanks to identify your block level > is as insane as it gets. Not at all. People do it all the time. The very idea of expecting people to count nested braces to identify block level is what is crazy, which is why in languages with braces people still indent the blocks. > First of all, combinations of blanks and tabs, > depending on how your ide is setup to expand tabs, may get you bugs, > you'd never imagine in your wild dreams. Not really. The bugs are quite simple, and generally easy to fix. To describe them as unimaginable is stupid. >>> for x in [1, 2, 3]: ... print x ... print x+1 File "", line 3 print x+1 ^ IndentationError: unexpected indent If you can't imagine getting an IndentationError from making an indentation error, there's something wrong with you. In any case, if your IDE mixes tabs and spaces, your IDE is broken and you should fix your tools rather than blame the language. > Braces is the most reliable way to identify blocks. Nonsense. For the compiler, both are equally reliable, and for the human reader, indents beat braces easily. > Sane compilers ignore blanks altogether. Really? So a "sane compiler" sees no difference between: for x in mylist: and forxinmylist: I'm glad I don't have to program using a compiler you consider "sane". -- Steven From uraniumore238 at gmail.com Sat Jan 30 14:18:51 2010 From: uraniumore238 at gmail.com (uche) Date: Sat, 30 Jan 2010 11:18:51 -0800 (PST) Subject: unencountered error in FFT python References: Message-ID: On Jan 30, 2:08?pm, MRAB wrote: > uche wrote: > > Hi, > > I have the following FFT python code and it doesn't seem to compile > > correctly. To run it, please create a file called output.csv with > > 1,2,3,4,5,6,7,8. simply run the main function. I get an error such as > > the following: > > > ? ? x[a], x[b] = x[(a)] + W[(n % N)] * x[(b)], x[(a)] - W[(n % N)] * x > > [(b)] > > TypeError: list indices must be integers, not float > > > How can I fixe this problem ? I have tried puttin int on all of the > > variables, but I don't think that is the intension of the person who > > wore the original code. > > Which version of Python are you using? In Python 3 the division operator > '/' returns a float, whereas in Python 2 it returns an int if both > operands are int. In Python 3 the int division operator is '//', which > is also accepted in recent versions of Python 2. > > [snip] > > > ? ? os.path.exists("input.csv") > > > ? ? fin=open('input.csv', 'r') > > > ? ? for line in fin: ? ?#read the line from the file > > > ? ? ? ? array=line.split(',') > > These lines should be indented more: > > > ? ? for a in range(len(array)): #convert into integers > > > ? ? ? ? array2.append((array[a])) > > ? ? ? ? ? array2.append(int(array[a])) > > [snip] Thanks, I just got this code from a site and trying to compile it. it's just a nightmare! I got another problem after changing / to // . Yes, I am using 3.1. W.append(exp(sign * 2j * pi * i // N)) TypeError: can't take floor of complex number. From stefan_ml at behnel.de Sat Jan 30 14:21:58 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 30 Jan 2010 20:21:58 +0100 Subject: unencountered error in FFT python In-Reply-To: <4b64800a$0$6735$9b4e6d93@newsspool2.arcor-online.net> References: <4b64800a$0$6735$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <4b6486d6$0$6724$9b4e6d93@newsspool2.arcor-online.net> Stefan Behnel, 30.01.2010 19:52: > uche, 30.01.2010 19:33: >> I have the following FFT python code > > You didn't seriously implement an FFT in plain Python code, did you? Sorry, no, you didn't. Should have read your post a little closer. > FFTs > are amongst the first thing that come to my mind when I try to imagine what > I'd use NumPy for. (and I *never* used it!) On second thought, I'd probably use FFTW for that, and there seem to be Python bindings for it: http://developer.berlios.de/projects/pyfftw/ Stefan From keith at nekotaku.com Sat Jan 30 14:28:47 2010 From: keith at nekotaku.com (KB) Date: Sat, 30 Jan 2010 11:28:47 -0800 (PST) Subject: Utility to screenscrape sites using javascript ? References: <4c4298c0-b93d-4691-bceb-e982373f87e3@f17g2000prh.googlegroups.com> Message-ID: <3df16722-7726-4b05-b8c3-d1b6a9f7e98b@k6g2000prg.googlegroups.com> > On Sat, 30 Jan 2010 06:21:01 -0800, KB wrote: > > I have a service I subscribe to that uses javascript to stream news. > There's a Python interface to SpiderMonkey (Mozilla's JavaScript > interpreter): > > http://pypi.python.org/pypi/python-spidermonkey Thanks! I don't see a documentation page, but how would one use this? Would you download the HTML using urllib2/mechanize, then parse for the .js script and use spider-monkey to "execute" the script and the output is passed back to python? TIA. From stefan_ml at behnel.de Sat Jan 30 14:28:47 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 30 Jan 2010 20:28:47 +0100 Subject: unencountered error in FFT python In-Reply-To: References: Message-ID: <4b64886f$0$6724$9b4e6d93@newsspool2.arcor-online.net> uche, 30.01.2010 20:18: > I got another problem after changing / to // . Yes, I am using 3.1. > > W.append(exp(sign * 2j * pi * i // N)) > TypeError: can't take floor of complex number. Don't change it everywhere, just where it deals with integers. In the above, "/" is perfectly right. Stefan From uraniumore238 at gmail.com Sat Jan 30 14:30:15 2010 From: uraniumore238 at gmail.com (uche) Date: Sat, 30 Jan 2010 11:30:15 -0800 (PST) Subject: unencountered error in FFT python References: <4b64800a$0$6735$9b4e6d93@newsspool2.arcor-online.net> <4b6486d6$0$6724$9b4e6d93@newsspool2.arcor-online.net> Message-ID: On Jan 30, 2:21?pm, Stefan Behnel wrote: > Stefan Behnel, 30.01.2010 19:52: > > > uche, 30.01.2010 19:33: > >> I have the following FFT python code > > > You didn't seriously implement an FFT in plain Python code, did you? > > Sorry, no, you didn't. Should have read your post a little closer. > > > FFTs > > are amongst the first thing that come to my mind when I try to imagine what > > I'd use NumPy for. (and I *never* used it!) > > On second thought, I'd probably use FFTW for that, and there seem to be > Python bindings for it: > > http://developer.berlios.de/projects/pyfftw/ > > Stefan Thanks for the suggestions and site. I just wanted to know what the heck is going wrong with this code. Please take a look at the code , thanks From apt.shansen at gmail.com Sat Jan 30 14:36:08 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Sat, 30 Jan 2010 11:36:08 -0800 Subject: Python and Ruby In-Reply-To: References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> Message-ID: <7a9c25c21001301136r25747681oefd7dc0a380db526@mail.gmail.com> On Sat, Jan 30, 2010 at 8:58 AM, tanix wrote: > The very idea of using a number of blanks to identify your block level > is as insane as it gets. First of all, combinations of blanks and tabs, > depending on how your ide is setup to expand tabs, may get you bugs, > you'd never imagine in your wild dreams. > > Braces is the most reliable way to identify blocks. > > Sane compilers ignore blanks altogether. > God forbid the compiler use the same syntactical structure as the human eye in reading code and assigning meaning. I mean, everyone knows that in C, no one ever has any issues with indentation levels causing human confusion and misunderstanding between what they think the structure of the code is and what the compiler interprets it as. Also, we all know very well that no modern IDE's or text editors let you manage tabs and spaces in a consistent fashion, and Python doesn't error out when you use inconsistent or nonsensical indentation. It just lets you create anonymous blocks of code when you indent too far in for no reason at all. --S P.S. Yes, my response is juvenile and snarky. However, I am weary of the imbecilic OMG WITESPACE IS RONG AND STUPID response. If you don't like whitespace being structurally significant, so be it-- aesthetics are subjective. If its not for you, its not for you. Python isn't for everyone. "Its insane", "Its stupid", "Its causes all kinds of confusion", "Its causes all kinds of hard to find bugs" is just complete, utter, unsupportable nonsense. -------------- next part -------------- An HTML attachment was scrubbed... URL: From uraniumore238 at gmail.com Sat Jan 30 15:20:48 2010 From: uraniumore238 at gmail.com (uche) Date: Sat, 30 Jan 2010 12:20:48 -0800 (PST) Subject: unencountered error in FFT python References: <4b64800a$0$6735$9b4e6d93@newsspool2.arcor-online.net> <4b6486d6$0$6724$9b4e6d93@newsspool2.arcor-online.net> Message-ID: On Jan 30, 2:30?pm, uche wrote: > On Jan 30, 2:21?pm, Stefan Behnel wrote: > > > > > > > Stefan Behnel, 30.01.2010 19:52: > > > > uche, 30.01.2010 19:33: > > >> I have the following FFT python code > > > > You didn't seriously implement an FFT in plain Python code, did you? > > > Sorry, no, you didn't. Should have read your post a little closer. > > > > FFTs > > > are amongst the first thing that come to my mind when I try to imagine what > > > I'd use NumPy for. (and I *never* used it!) > > > On second thought, I'd probably use FFTW for that, and there seem to be > > Python bindings for it: > > >http://developer.berlios.de/projects/pyfftw/ > > > Stefan > > Thanks for the suggestions and site. I just wanted to know what the > heck is going wrong with this code. Please take a look at the code , > thanks- Hide quoted text - > > - Show quoted text - Thanks Stephan. You are the best! Another issue: x[a], x[b] = x[(a)] + W[(n % N)] * x[(b)], x[(a)] - W[(n % N)] * x [(b)] TypeError: can't multiply sequence by non-int of type 'complex' From marcmagransdeabril at gmail.com Sat Jan 30 15:36:59 2010 From: marcmagransdeabril at gmail.com (marc magrans de abril) Date: Sat, 30 Jan 2010 12:36:59 -0800 (PST) Subject: Slow down while creating a big list and iterating over it Message-ID: <2fdd4e1b-db58-45d7-9b7d-73dcdca6384a@c29g2000yqd.googlegroups.com> Dear colleagues, I was doing a small program to classify log files for a cluster of PCs, I just wanted to simplify a quite repetitive task in order to find errors and so. My first naive implementation was something like: patterns = [] while(logs): pattern = logs[0] new_logs = [l for l in logs if dist(pattern,l)>THERESHOLD] entry = (len(logs)-len(new_logs),pattern) patterns.append(entry) logs = new_logs Where dist(...) is the levenshtein distance (i.e. edit distance) and logs is something like 1.5M logs (700 MB file). I thought that python will be an easy choice although not really fast.. I was not surprised when the first iteration of the while loop was taking ~10min. I thought "not bad, let's how much it takes". However, it seemed that the second iteration never finished. My surprise was big when I added a print instead of the list comprehension: new_logs=[] for count,l in enumerate(logs): print count if dist(pattern,l)>THERESHOLD: new_logs.append(l) The surprise was that the displayed counter was running ~10 times slower on the second iteration of the while loop. I am a little lost. Anyone knows the reson of this behavior? How should I write a program that deals with large data sets in python? Thanks a lot! marc magrans de abril From alfps at start.no Sat Jan 30 17:08:56 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sat, 30 Jan 2010 23:08:56 +0100 Subject: Slow down while creating a big list and iterating over it In-Reply-To: <2fdd4e1b-db58-45d7-9b7d-73dcdca6384a@c29g2000yqd.googlegroups.com> References: <2fdd4e1b-db58-45d7-9b7d-73dcdca6384a@c29g2000yqd.googlegroups.com> Message-ID: * marc magrans de abril: > Dear colleagues, > > I was doing a small program to classify log files for a cluster of > PCs, I just wanted to simplify a quite repetitive task in order to > find errors and so. > > My first naive implementation was something like: > patterns = [] > while(logs): > pattern = logs[0] > new_logs = [l for l in logs if dist(pattern,l)>THERESHOLD] > entry = (len(logs)-len(new_logs),pattern) > patterns.append(entry) > logs = new_logs > > Where dist(...) is the levenshtein distance (i.e. edit distance) and > logs is something like 1.5M logs (700 MB file). I thought that python > will be an easy choice although not really fast.. > > I was not surprised when the first iteration of the while loop was > taking ~10min. I thought "not bad, let's how much it takes". However, > it seemed that the second iteration never finished. > > My surprise was big when I added a print instead of the list > comprehension: > new_logs=[] > for count,l in enumerate(logs): > print count > if dist(pattern,l)>THERESHOLD: > new_logs.append(l) > > The surprise was that the displayed counter was running ~10 times > slower on the second iteration of the while loop. > > I am a little lost. Anyone knows the reson of this behavior? It's on line 42 of your program. :-) That is, it's in the dist function. Evidently it doesn't like a more complex 'pattern'. > How should I write a program that deals with large data sets in python? As in any other language. Try to avoid repeating the same computations. Try to make the data fit the computational task. Cheers & hth., - Alf From johnroth1 at gmail.com Sat Jan 30 17:14:54 2010 From: johnroth1 at gmail.com (John Roth) Date: Sat, 30 Jan 2010 14:14:54 -0800 (PST) Subject: PEP 3147 - new .pyc format Message-ID: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> PEP 3147 has just been posted, proposing that, beginning in release 3.2 (and possibly 2.7) compiled .pyc and .pyo files be placed in a directory with a .pyr extension. The reason is so that compiled versions of a program can coexist, which isn't possible now. Frankly, I think this is a really good idea, although I've got a few comments. 1. Apple's MAC OS X should be mentioned, since 10.5 (and presumably 10.6) ship with both Python release 2.3 and 2.5 installed. 2. I think the proposed logic is too complex. If this is installed in 3.2, then that release should simply store its .pyc file in the .pyr directory, without the need for either a command line switch or an environment variable (both are mentioned in the PEP.) 3. Tool support. There are tools that look for the .pyc files; these need to be upgraded somehow. The ones that ship with Python should, of course, be fixed with the PEP, but there are others. 4. I'm in favor of putting the source in the .pyr directory as well, but that's got a couple more issues. One is tool support, which is likely to be worse for source, and the other is some kind of algorithm for identifying which source goes with which object. Summary: I like it, but I think it needs a bit more work. John Roth From andrew at acooke.org Sat Jan 30 17:16:19 2010 From: andrew at acooke.org (andrew cooke) Date: Sat, 30 Jan 2010 14:16:19 -0800 (PST) Subject: Function name unchanged in error message References: Message-ID: <29d9b330-a57f-48a8-9d1e-bdad7c038ad9@k19g2000yqc.googlegroups.com> On Jan 29, 11:22?am, Peter Otten <__pete... at web.de> wrote: > The name is looked up in the code object. As that is immutable you have to > make a new one: [details snipped] thanks very much! sorry i didn't reply earlier - been travelling. (also, thanks to any other replies - i'm just reading through at the moment and this is the first one i've got to that will help me solve it, but i don't mean to exclude anything later...!) andrew From andrew at acooke.org Sat Jan 30 17:17:39 2010 From: andrew at acooke.org (andrew cooke) Date: Sat, 30 Jan 2010 14:17:39 -0800 (PST) Subject: Function name unchanged in error message References: Message-ID: <8e7d79dc-5f91-4e8c-921b-399fe0da77bf@m16g2000yqc.googlegroups.com> On Jan 29, 5:37?pm, "Gabriel Genellina" wrote: > The decorator module is a very fine addition to anyone's tool set -- but ? > in this case it is enough to use the wraps() function from the functools ? > standard module. ah, thanks! i thought something like this existed in the standard lib, but couldn't find it. andrew From andrew at acooke.org Sat Jan 30 17:26:43 2010 From: andrew at acooke.org (andrew cooke) Date: Sat, 30 Jan 2010 14:26:43 -0800 (PST) Subject: Function name unchanged in error message References: <50697b2c1001290610t1a2ac5c8hcd5f9856aa85392b@mail.gmail.com> Message-ID: <5303e556-e9df-4ef3-bd4d-fa4928399c6e@r19g2000yqb.googlegroups.com> On Jan 29, 11:50?am, exar... at twistedmatrix.com wrote: > new.function and new.code will let you construct new objects with > different values (and copying over whichever existing attributes you > want to preserve). unfortunately new is deprecated and dropped from 3. i can't see how the same functionality is available in the types module for 3 - am i missing something obvious? http://docs.python.org/library/new.html http://docs.python.org/3.1/library/types.html#module-types thanks, andrew From andrew at acooke.org Sat Jan 30 17:31:51 2010 From: andrew at acooke.org (andrew cooke) Date: Sat, 30 Jan 2010 14:31:51 -0800 (PST) Subject: Function name unchanged in error message References: <8e7d79dc-5f91-4e8c-921b-399fe0da77bf@m16g2000yqc.googlegroups.com> Message-ID: On Jan 30, 7:17?pm, andrew cooke wrote: > On Jan 29, 5:37?pm, "Gabriel Genellina" > wrote: > > > The decorator module is a very fine addition to anyone's tool set -- but ? > > in this case it is enough to use the wraps() function from the functools ? > > standard module. > > ah, thanks! ?i thought something like this existed in the standard > lib, but couldn't find it. > > andrew ah, sorry, peter's code uses types, so i assume that's the way to go (i was hoping that there was something a bit simpler - i don't like the fact that the code in peter's code has a fixed list of special names). andrew From mensanator at aol.com Sat Jan 30 17:36:52 2010 From: mensanator at aol.com (Mensanator) Date: Sat, 30 Jan 2010 14:36:52 -0800 (PST) Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> Message-ID: On Jan 30, 4:14?pm, John Roth wrote: > PEP 3147 has just been posted, proposing that, beginning in release > 3.2 (and possibly 2.7) compiled .pyc and .pyo files be placed in a > directory with a .pyr extension. The reason is so that compiled > versions of a program can coexist, which isn't possible now. > > Frankly, I think this is a really good idea, although I've got a few > comments. > > 1. Apple's MAC OS X should be mentioned, since 10.5 (and presumably > 10.6) ship with both Python release 2.3 and 2.5 installed. Mac OSX 10.6 has 2.6 installed. > > 2. I think the proposed logic is too complex. If this is installed in > 3.2, then that release should simply store its .pyc file in the .pyr > directory, without the need for either a command line switch or an > environment variable (both are mentioned in the PEP.) > > 3. Tool support. There are tools that look for the .pyc files; these > need to be upgraded somehow. The ones that ship with Python should, of > course, be fixed with the PEP, but there are others. > > 4. I'm in favor of putting the source in the .pyr directory as well, > but that's got a couple more issues. One is tool support, which is > likely to be worse for source, and the other is some kind of algorithm > for identifying which source goes with which object. > > Summary: I like it, but I think it needs a bit more work. > > John Roth From python at mrabarnett.plus.com Sat Jan 30 17:57:57 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 30 Jan 2010 22:57:57 +0000 Subject: PEP 3147 - new .pyc format In-Reply-To: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> Message-ID: <4B64B975.7060404@mrabarnett.plus.com> John Roth wrote: > PEP 3147 has just been posted, proposing that, beginning in release > 3.2 (and possibly 2.7) compiled .pyc and .pyo files be placed in a > directory with a .pyr extension. The reason is so that compiled > versions of a program can coexist, which isn't possible now. > > Frankly, I think this is a really good idea, although I've got a few > comments. > > 1. Apple's MAC OS X should be mentioned, since 10.5 (and presumably > 10.6) ship with both Python release 2.3 and 2.5 installed. > > 2. I think the proposed logic is too complex. If this is installed in > 3.2, then that release should simply store its .pyc file in the .pyr > directory, without the need for either a command line switch or an > environment variable (both are mentioned in the PEP.) > > 3. Tool support. There are tools that look for the .pyc files; these > need to be upgraded somehow. The ones that ship with Python should, of > course, be fixed with the PEP, but there are others. > > 4. I'm in favor of putting the source in the .pyr directory as well, > but that's got a couple more issues. One is tool support, which is > likely to be worse for source, and the other is some kind of algorithm > for identifying which source goes with which object. > > Summary: I like it, but I think it needs a bit more work. > The PEP has a .pyr directory for each .py file: foo.py foo.pyr/ f2b30a0d.pyc # Python 2.5 f2d10a0d.pyc # Python 2.6 f2d10a0d.pyo # Python 2.6 -O f2d20a0d.pyc # Python 2.6 -U 0c4f0a0d.pyc # Python 3.1 Other possibilities are: 1. A single pyr directory: foo.py pyr/ foo.f2b30a0d.pyc # Python 2.5 foo.f2d10a0d.pyc # Python 2.6 foo.f2d10a0d.pyo # Python 2.6 -O foo.f2d20a0d.pyc # Python 2.6 -U foo.0c4f0a0d.pyc # Python 3.1 2. A .pyr directory for each version of Python: foo.py f2b30a0d.pyr/ # Python 2.5 foo.pyc f2d10a0d.pyr/ # Python 2.6/Python 2.6 -O foo.pyc foo.pyo f2d20a0d.pyr/ # Python 2.6 -U foo.pyc 0c4f0a0d.pyr/ # Python 3.1 foo.pyc From kerensaelise at hotmail.com Sat Jan 30 18:08:20 2010 From: kerensaelise at hotmail.com (elsa) Date: Sat, 30 Jan 2010 15:08:20 -0800 (PST) Subject: Still too slow Message-ID: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> Hello again, Thanks for the tips r.e random.ranint(). This improved matters somewhat, however my program is still too slow. If anyone has any further tips on how to speed it up, they would be much appreciated! So, I'm calling evolve(L,limit) from the interactive prompt. L is initally [[100],['NA']]. Ideally limit would be 10^7. Here is my program: import random n=100 def evolve(L,limit): global n while n References: <2fdd4e1b-db58-45d7-9b7d-73dcdca6384a@c29g2000yqd.googlegroups.com> Message-ID: <4B64BCBA.5040905@mrabarnett.plus.com> Alf P. Steinbach wrote: > * marc magrans de abril: >> Dear colleagues, >> >> I was doing a small program to classify log files for a cluster of >> PCs, I just wanted to simplify a quite repetitive task in order to >> find errors and so. >> >> My first naive implementation was something like: >> patterns = [] >> while(logs): >> pattern = logs[0] >> new_logs = [l for l in logs if dist(pattern,l)>THERESHOLD] >> entry = (len(logs)-len(new_logs),pattern) >> patterns.append(entry) >> logs = new_logs >> >> Where dist(...) is the levenshtein distance (i.e. edit distance) and >> logs is something like 1.5M logs (700 MB file). I thought that python >> will be an easy choice although not really fast.. >> >> I was not surprised when the first iteration of the while loop was >> taking ~10min. I thought "not bad, let's how much it takes". However, >> it seemed that the second iteration never finished. >> >> My surprise was big when I added a print instead of the list >> comprehension: >> new_logs=[] >> for count,l in enumerate(logs): >> print count >> if dist(pattern,l)>THERESHOLD: >> new_logs.append(l) >> >> The surprise was that the displayed counter was running ~10 times >> slower on the second iteration of the while loop. >> >> I am a little lost. Anyone knows the reson of this behavior? > > It's on line 42 of your program. :-) That is, it's in the dist function. > Evidently it doesn't like a more complex 'pattern'. > Find out which pattern is being used on the second iteration and then try it on the first iteration. Is it just as slow? If so, then it's down to the length/complexity of that pattern - a much longer/more complex pattern might take much longer when computing the distance. >> How should I write a program that deals with large data sets in python? > > As in any other language. Try to avoid repeating the same computations. > Try to make the data fit the computational task. > True. Basically, you're computing the distance between every pair of logs! From andrew at acooke.org Sat Jan 30 18:13:06 2010 From: andrew at acooke.org (andrew cooke) Date: Sat, 30 Jan 2010 15:13:06 -0800 (PST) Subject: Function name unchanged in error message References: Message-ID: <33a3e0e9-2051-41a9-9ddd-f1c8db713bd3@z26g2000yqm.googlegroups.com> On Jan 29, 1:09?pm, Michele Simionato wrote: > On Jan 29, 2:30?pm, andrew cooke wrote: > > > Is there any way to change the name of the function in an error > > message? ?In the example below I'd like the error to refer to bar(), > > for example (the motivation is related function decorators - I'd like > > the wrapper function to give the same name) > > Use the decorator module which does the right thing:http://pypi.python.org/pypi/decorator curiously, decorator doesn't have this issue, because the way it defines decorators uses *args. so the error i gave cannot occur at the level of the decorator - the extra arg is passed to the wrapped function, and so the error message is correct because it is generated by the inner function. i need to look at my code; this might be the simplest solution of all. thanks, andrew From philip at semanchuk.com Sat Jan 30 18:31:41 2010 From: philip at semanchuk.com (Philip Semanchuk) Date: Sat, 30 Jan 2010 18:31:41 -0500 Subject: Still too slow In-Reply-To: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> References: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> Message-ID: On Jan 30, 2010, at 6:08 PM, elsa wrote: > Hello again, > > Thanks for the tips r.e random.ranint(). This improved matters > somewhat, however my program is still too slow. If anyone has any > further tips on how to speed it up, they would be much appreciated! > > So, I'm calling evolve(L,limit) from the interactive prompt. L is > initally [[100],['NA']]. Ideally limit would be 10^7. > > Here is my program: > > import random > n=100 > > def evolve(L,limit): > global n > while n evnt = event() > if evnt!="None": > ind = chooseInd(L,n) > action(evnt,L,ind) > > def chooseInd(L,n): > choiceSum=0 > index=0 > choice = random.randint(1,n) > while choiceSum < choice: > choiceSum+=L[index][0] > index +=1 > return (index-1) > > def event(): > choice = random.random() > if choice <= .3: > event='b' > elif choice <= .4: > event ='d' > elif choice<= .5: > event = 'm' > else: > event = 'None' > return event > > def action(event, L, index): > global n > if event == 'b': > L[index][0]+=1 > n +=1 > elif event == 'd': > L[index][0]-=1 > n -=1 > elif event == 'm': > L.append([1,index]) > n +=1 Hi Elsa, I didn't follow the earlier discussion on this, so pardon me if I'm repeating what others have said. First, have you profiled the code? That will help you find bottlenecks. I haven't profiled it myself so I am shooting from the hip, but a couple of obvious speedups in event() jump out at me. I'd rewrite it like this: def event(): choice = random.random() if choice <= .3: return 'b' elif choice <= .4: return 'd' elif choice <= .5: return 'm' return None Returning immediately when you've found your choice will save evaluating a couple of elifs -- not a big help, but it is a little. It comes at the cost having multiple exits to the function which is something I prefer to avoid, but this function is small enough that having multiple exits won't get confusing. Second, returning the Python object None rather than the string "None" allows you to change evolve() from this string comparison: if evnt!="None": to this comparison to the Python singleton None: if evnt != None: or this, which is even simpler (although it may not be faster): if evnt: Hope this helps Philip From john at castleamber.com Sat Jan 30 18:38:52 2010 From: john at castleamber.com (John Bokma) Date: Sat, 30 Jan 2010 17:38:52 -0600 Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> Message-ID: <87iqaj9n8z.fsf@castleamber.com> MRAB writes: > The PEP has a .pyr directory for each .py file: > > foo.py > foo.pyr/ > f2b30a0d.pyc # Python 2.5 > f2d10a0d.pyc # Python 2.6 > f2d10a0d.pyo # Python 2.6 -O > f2d20a0d.pyc # Python 2.6 -U > 0c4f0a0d.pyc # Python 3.1 wow: so much for human readable file names :-( -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From alfps at start.no Sat Jan 30 18:46:03 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 31 Jan 2010 00:46:03 +0100 Subject: PEP 3147 - new .pyc format In-Reply-To: <87iqaj9n8z.fsf@castleamber.com> References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <87iqaj9n8z.fsf@castleamber.com> Message-ID: * John Bokma: > MRAB writes: > >> The PEP has a .pyr directory for each .py file: >> >> foo.py >> foo.pyr/ >> f2b30a0d.pyc # Python 2.5 >> f2d10a0d.pyc # Python 2.6 >> f2d10a0d.pyo # Python 2.6 -O >> f2d20a0d.pyc # Python 2.6 -U >> 0c4f0a0d.pyc # Python 3.1 > > wow: so much for human readable file names :-( I agree. Human readable filenames would be much better. Cheers, - Alf From pavlovevidence at gmail.com Sat Jan 30 19:14:53 2010 From: pavlovevidence at gmail.com (Carl Banks) Date: Sat, 30 Jan 2010 16:14:53 -0800 (PST) Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> Message-ID: On Jan 30, 2:14?pm, John Roth wrote: > PEP 3147 has just been posted, proposing that, beginning in release > 3.2 (and possibly 2.7) compiled .pyc and .pyo files be placed in a > directory with a .pyr extension. The reason is so that compiled > versions of a program can coexist, which isn't possible now. > > Frankly, I think this is a really good idea, although I've got a few > comments. -1 I think it's a terrible, drastic approach to a minor problem. I'm not sure why the simple approach of just appending a number (perhaps the major-minor version, or a serial number) to the filename wouldn't work, like this: foo.pyc25 All I can think of is they are concerned with the typically minor expense of listing the directory (to see if there's already .pyc?? file present). This operation can be reasonably cached; when scanning a directory listing it need only record all occurrencs of .pyc?? and mark those modules as subject to version-specific .pyc files. Anyway, I'd expect the proposed -R switch would only be used in special cases (like installation time) when a minor inefficiency would be tolerable. > 1. Apple's MAC OS X should be mentioned, since 10.5 (and presumably > 10.6) ship with both Python release 2.3 and 2.5 installed. > > 2. I think the proposed logic is too complex. If this is installed in > 3.2, then that release should simply store its .pyc file in the .pyr > directory, without the need for either a command line switch or an > environment variable (both are mentioned in the PEP.) This is utterly unacceptable. Versioned *.pyc files should only be optionally requested by people who have to deal multiple versions, such as distro package maintainers. For my projects I don't give a flying F about versioned *.pyc and I don't want my project directory cluttered with a million subdirectories. (It would be a bit more tolerable if my directory was merely cluttered with *.pyc?? files, but I'd still rather Python didn't do that unless asked.) > 3. Tool support. There are tools that look for the .pyc files; these > need to be upgraded somehow. The ones that ship with Python should, of > course, be fixed with the PEP, but there are others. How will this affect tools like Py2exe? Now you have a bunch of identically-named *.pyc files. > 4. I'm in favor of putting the source in the .pyr directory as well, > but that's got a couple more issues. One is tool support, which is > likely to be worse for source, and the other is some kind of algorithm > for identifying which source goes with which object. Now this just too much. I didn't like the suggestion that I should be forced to put up with dozens of subdirectories, now you want me to force me to put the source files into the subdirectories as well? That would be a deal-breaker. Thankfully it is too ridiculous to ever happen. > Summary: I like it, but I think it needs a bit more work. I hope it's replaced with something less drastic. Carl Banks From python at mrabarnett.plus.com Sat Jan 30 19:19:50 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 31 Jan 2010 00:19:50 +0000 Subject: PEP 3147 - new .pyc format In-Reply-To: References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <87iqaj9n8z.fsf@castleamber.com> Message-ID: <4B64CCA6.9050607@mrabarnett.plus.com> Alf P. Steinbach wrote: > * John Bokma: >> MRAB writes: >> >>> The PEP has a .pyr directory for each .py file: >>> >>> foo.py >>> foo.pyr/ >>> f2b30a0d.pyc # Python 2.5 >>> f2d10a0d.pyc # Python 2.6 >>> f2d10a0d.pyo # Python 2.6 -O >>> f2d20a0d.pyc # Python 2.6 -U >>> 0c4f0a0d.pyc # Python 3.1 >> >> wow: so much for human readable file names :-( > > I agree. > > Human readable filenames would be much better. > The names are the magic numbers. It's all in the PEP. From jjposner at optimum.net Sat Jan 30 19:27:46 2010 From: jjposner at optimum.net (John Posner) Date: Sat, 30 Jan 2010 19:27:46 -0500 Subject: Still too slow In-Reply-To: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> References: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> Message-ID: <4B64CE82.1040109@optimum.net> On 1/30/2010 6:08 PM, elsa wrote: > Hello again, > > Thanks for the tips r.e random.ranint(). This improved matters > somewhat, however my program is still too slow. If anyone has any > further tips on how to speed it up, they would be much appreciated! > > So, I'm calling evolve(L,limit) from the interactive prompt. L is > initally [[100],['NA']]. Ideally limit would be 10^7. > > Here is my program: > > import random > n=100 > > def evolve(L,limit): > global n > while n evnt = event() > if evnt!="None": > ind = chooseInd(L,n) > action(evnt,L,ind) > > def chooseInd(L,n): > choiceSum=0 > index=0 > choice = random.randint(1,n) > while choiceSum< choice: > choiceSum+=L[index][0] > index +=1 > return (index-1) > > def event(): > choice = random.random() > if choice<= .3: > event='b' > elif choice<= .4: > event ='d' > elif choice<= .5: > event = 'm' > else: > event = 'None' > return event > > def action(event, L, index): > global n > if event == 'b': > L[index][0]+=1 > n +=1 > elif event == 'd': > L[index][0]-=1 > n -=1 > elif event == 'm': > L.append([1,index]) > n +=1 > > > thanks in advance, > > Elsa. Elsa, 1. You changed the subject line from "For loop searching takes too long!" to "Still too slow". This causes newsreader programs to start a new discussion thread, which makes life difficult for people who need to refer back to previous messages. Please don't change the subject line any more. 2. You provided a very clear description of your original question: Now, what I need to do is randomly choose one myList[i], however the distribution of my random choice needs to be proportional to the values of myList[i][0]. This description should be the doc string for the chooseInd() function -- for example: def chooseInd(L,n): """ randomly choose one L[i], so that the distribution of choices is proportional to the values of L[i][0] """ It is not clear (to me, anyway) what the other functions are trying to accomplish. So please add a doc string to each of these functions, with descriptions of similar clarity. This will help us a lot. And it will help you, too, if you return to the code after a few days/weeks/months of directing your attention elsewhere. 3. Please provide a *complete* transcript of an interactive session that exercises this code. Tx, John From john at castleamber.com Sat Jan 30 19:51:31 2010 From: john at castleamber.com (John Bokma) Date: Sat, 30 Jan 2010 18:51:31 -0600 Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <87iqaj9n8z.fsf@castleamber.com> Message-ID: <877hqz2j1o.fsf@castleamber.com> MRAB writes: > Alf P. Steinbach wrote: >> * John Bokma: >>> MRAB writes: >>> >>>> The PEP has a .pyr directory for each .py file: >>>> >>>> foo.py >>>> foo.pyr/ >>>> f2b30a0d.pyc # Python 2.5 >>>> f2d10a0d.pyc # Python 2.6 >>>> f2d10a0d.pyo # Python 2.6 -O >>>> f2d20a0d.pyc # Python 2.6 -U >>>> 0c4f0a0d.pyc # Python 3.1 >>> >>> wow: so much for human readable file names :-( >> >> I agree. >> >> Human readable filenames would be much better. >> > The names are the magic numbers. It's all in the PEP. Naming files using magic numbers is really beyond me. The fact that the above needs comments to explain what's what already shows to me that there's a problem with this naming scheme. What if for one reason or another I want to delete all pyc files for Python 2.5? Where do I look up the magic number? -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From nyamatongwe+thunder at gmail.com Sat Jan 30 20:09:07 2010 From: nyamatongwe+thunder at gmail.com (Neil Hodgson) Date: Sun, 31 Jan 2010 01:09:07 GMT Subject: PEP 3147 - new .pyc format In-Reply-To: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> Message-ID: John Roth: > 4. I'm in favor of putting the source in the .pyr directory as well, > but that's got a couple more issues. One is tool support, which is > likely to be worse for source, and the other is some kind of algorithm > for identifying which source goes with which object. Many tools work recursively except for hidden directories so would return both the source in the repository as well as the original source. If you want to do this then the repository directory should be hidden by starting with ".". Neil From python at mrabarnett.plus.com Sat Jan 30 20:16:54 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 31 Jan 2010 01:16:54 +0000 Subject: PEP 3147 - new .pyc format In-Reply-To: <877hqz2j1o.fsf@castleamber.com> References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <87iqaj9n8z.fsf@castleamber.com> <877hqz2j1o.fsf@castleamber.com> Message-ID: <4B64DA06.7020200@mrabarnett.plus.com> John Bokma wrote: > MRAB writes: > >> Alf P. Steinbach wrote: >>> * John Bokma: >>>> MRAB writes: >>>> >>>>> The PEP has a .pyr directory for each .py file: >>>>> >>>>> foo.py >>>>> foo.pyr/ >>>>> f2b30a0d.pyc # Python 2.5 >>>>> f2d10a0d.pyc # Python 2.6 >>>>> f2d10a0d.pyo # Python 2.6 -O >>>>> f2d20a0d.pyc # Python 2.6 -U >>>>> 0c4f0a0d.pyc # Python 3.1 >>>> wow: so much for human readable file names :-( >>> I agree. >>> >>> Human readable filenames would be much better. >>> >> The names are the magic numbers. It's all in the PEP. > > Naming files using magic numbers is really beyond me. The fact that the > above needs comments to explain what's what already shows to me that > there's a problem with this naming scheme. What if for one reason or > another I want to delete all pyc files for Python 2.5? Where do I look > up the magic number? > True. You might also want to note that "Python 2.6 -U" appears to have a different magic number from "Python 2.6" and "Python 2.6 -O". I don't know whether they always change for each new version. From steve at REMOVE-THIS-cybersource.com.au Sat Jan 30 20:18:52 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 31 Jan 2010 01:18:52 GMT Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> Message-ID: <0374c6d6$0$1309$c3e8da3@news.astraweb.com> On Sat, 30 Jan 2010 14:14:54 -0800, John Roth wrote: > PEP 3147 has just been posted, proposing that, beginning in release 3.2 > (and possibly 2.7) compiled .pyc and .pyo files be placed in a directory > with a .pyr extension. The reason is so that compiled versions of a > program can coexist, which isn't possible now. http://www.python.org/dev/peps/pep-3147/ Reading through the PEP, I went from an instinctive "oh no, that sounds horrible" reaction to "hmmm, well, that doesn't sound too bad". I don't think I need this, but I could live with it. Firstly, it does sound like there is a genuine need for a solution to the problem of multiple Python versions. This is not the first PEP trying to solve it, so even if you personally don't see the need, we can assume that others do. Secondly, the current behaviour will remain unchanged. Python will compile spam.py to spam.pyc (or spam.pyo with the -O switch) by default. If you don't need to support multiple versions, you don't need to care about this PEP. I like this aspect of the PEP very much. I would argue that any solution MUST support the status quo for those who don't care about multiple versions. To get the new behaviour, you have to explicitly ask for it. You ask for it by calling python with the -R switch, by setting an environment variable, or explicitly providing the extra spam/.pyc files. Thirdly, the magic file names aren't quite as magic as they appear at first glance. They represent the hexified magic number of the version of Python. More about the magic number here: http://nedbatchelder.com/blog/200804/the_structure_of_pyc_files.html Unfortunately the magic number doesn't seem to be documented anywhere I can find other than in the source code (import.c). The PEP gives some examples: f2b30a0d.pyc # Python 2.5 f2d10a0d.pyc # Python 2.6 f2d10a0d.pyo # Python 2.6 -O f2d20a0d.pyc # Python 2.6 -U 0c4f0a0d.pyc # Python 3.1 but how can one map magic numbers to versions, short of reading import.c? I propose that sys grow an object sys.magic which is the hexlified magic number. > 2. I think the proposed logic is too complex. If this is installed in > 3.2, then that release should simply store its .pyc file in the .pyr > directory, without the need for either a command line switch or an > environment variable (both are mentioned in the PEP.) I disagree. Making the new behaviour optional is an advantage, even if it leads to extra complexity. It is pointless forcing .pyc files to be in a subdirectory if you don't need multiple versions. > 3. Tool support. There are tools that look for the .pyc files; these > need to be upgraded somehow. The ones that ship with Python should, of > course, be fixed with the PEP, but there are others. Third party tools will be the responsibility of the third parties. > 4. I'm in favor of putting the source in the .pyr directory as well, but > that's got a couple more issues. One is tool support, which is likely to > be worse for source, and the other is some kind of algorithm for > identifying which source goes with which object. It certain does. What's the advantage of forcing .py files to live inside a directory with the same name? Modules: mymodule.py => mymodule/mymodule.py Packages: mypackage/__init__.py => mypackage/__init__/__init__.py mypackage/spam.py => mypackage/spam/spam.py Seems like a pointless and annoying extra layer to me. -- Steven From alfps at start.no Sat Jan 30 20:34:24 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 31 Jan 2010 02:34:24 +0100 Subject: Still too slow In-Reply-To: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> References: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> Message-ID: * elsa: > > Thanks for the tips r.e random.ranint(). This improved matters > somewhat, however my program is still too slow. If anyone has any > further tips on how to speed it up, they would be much appreciated! > > So, I'm calling evolve(L,limit) from the interactive prompt. L is > initally [[100],['NA']]. This would produce an unhandled exception since your code tries to compute the sum of the values in the list. You can't add strings and numbers. > Ideally limit would be 10^7. > > Here is my program: > > import random > n=100 > > def evolve(L,limit): > global n > while n evnt = event() > if evnt!="None": > ind = chooseInd(L,n) > action(evnt,L,ind) > > def chooseInd(L,n): > choiceSum=0 > index=0 > choice = random.randint(1,n) > while choiceSum < choice: > choiceSum+=L[index][0] > index +=1 > return (index-1) This is the problematic part of your program, because it uses time linear in the length of L. Consider if you know the sum s_low of the lower half of the array, and the sum s_high of the higher half of the array. Then you can choose the lower half with probability s_low/(s_low+s_high), otherwise the higher half. Then repeat this recursively for the half chosen, until you get down to a single array element. This requires keeping track of the sums of halfs of the array, e.g. in a tree like structure or a set of arrays of diminishing lengths. Generally it will use some constant factor times twice the storage that you're now using. But it reduces the time for choosing an index from O(n) to O(log n). For example, if you have a logical structure like * / \ * 1 / \ 98 1 then at the top level * you choose the left branch with probability 99/(99+1) = 99/100 = 0.99. At the second level * you choose the right branch with probability 1/(98+1) = 1/99. The combined probability of choosing that lower 1 is then 0.99*(1/99) = 0.01, as it should be. > def event(): > choice = random.random() > if choice <= .3: > event='b' > elif choice <= .4: > event ='d' > elif choice<= .5: > event = 'm' > else: > event = 'None' > return event Note that you can double the speed of your program by removing the 'None' value. Adjust the limits you're checking against so as to retain the same probabilities of the choices. Further constant factor speedup is possible by simply returning a function to Do Stuff instead of returning a character saying what to do. > def action(event, L, index): > global n > if event == 'b': > L[index][0]+=1 > n +=1 > elif event == 'd': > L[index][0]-=1 > n -=1 > elif event == 'm': > L.append([1,index]) > n +=1 > > > thanks in advance, An observation about design: you have a global n (the current total sum) and an array L that you're passing around everywhere, so that it's effectively also a global. This indicates that they should ideally instead be attributes of an object, with your routines above as methods. However in Python this may cause some overhead, so, perhaps first make it Work (and then make a Copy). :-) Cheers & hth., - Alf From alfps at start.no Sat Jan 30 20:48:10 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 31 Jan 2010 02:48:10 +0100 Subject: Still too slow In-Reply-To: References: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> Message-ID: * Alf P. Steinbach: > * elsa: >> >> Thanks for the tips r.e random.ranint(). This improved matters >> somewhat, however my program is still too slow. If anyone has any >> further tips on how to speed it up, they would be much appreciated! >> >> So, I'm calling evolve(L,limit) from the interactive prompt. L is >> initally [[100],['NA']]. > > This would produce an unhandled exception since your code tries to > compute the sum of the values in the list. You can't add strings and > numbers. > > >> Ideally limit would be 10^7. >> >> Here is my program: >> >> import random >> n=100 >> >> def evolve(L,limit): >> global n >> while n> evnt = event() >> if evnt!="None": >> ind = chooseInd(L,n) >> action(evnt,L,ind) >> >> def chooseInd(L,n): >> choiceSum=0 >> index=0 >> choice = random.randint(1,n) >> while choiceSum < choice: >> choiceSum+=L[index][0] >> index +=1 >> return (index-1) > > This is the problematic part of your program, because it uses time > linear in the length of L. > > Consider if you know the sum s_low of the lower half of the array, and > the sum s_high of the higher half of the array. Then you can choose the > lower half with probability s_low/(s_low+s_high), otherwise the higher > half. Then repeat this recursively for the half chosen, until you get > down to a single array element. > > This requires keeping track of the sums of halfs of the array, e.g. in a > tree like structure or a set of arrays of diminishing lengths. Generally > it will use some constant factor times twice the storage that you're now > using. But it reduces the time for choosing an index from O(n) to O(log n). > > For example, if you have a logical structure like > > * > / \ > * 1 > / \ > 98 1 > > then at the top level * you choose the left branch with probability > 99/(99+1) = 99/100 = 0.99. At the second level * you choose the right > branch with probability 1/(98+1) = 1/99. The combined probability of > choosing that lower 1 is then 0.99*(1/99) = 0.01, as it should be. Oh, forgot to add: since your array's first item will be the one that's far most often chosen a "blind" sort of literal implementation of the above will perhaps slow down the program instead of speeding it up, because it only speeds up those searches that progress beyond the first item. But anyway, I'm guessing that checking the first item first, as a special case, will really improve things. Only when it turns out that it's not the first item, apply binary search. >> def event(): >> choice = random.random() >> if choice <= .3: >> event='b' >> elif choice <= .4: >> event ='d' >> elif choice<= .5: >> event = 'm' >> else: >> event = 'None' >> return event > > Note that you can double the speed of your program by removing the /s/double/increase/g > 'None' value. Adjust the limits you're checking against so as to retain > the same probabilities of the choices. Further constant factor speedup > is possible by simply returning a function to Do Stuff instead of > returning a character saying what to do. > > > >> def action(event, L, index): >> global n >> if event == 'b': >> L[index][0]+=1 >> n +=1 >> elif event == 'd': >> L[index][0]-=1 >> n -=1 >> elif event == 'm': >> L.append([1,index]) >> n +=1 >> >> >> thanks in advance, > > An observation about design: you have a global n (the current total sum) > and an array L that you're passing around everywhere, so that it's > effectively also a global. This indicates that they should ideally > instead be attributes of an object, with your routines above as methods. > However in Python this may cause some overhead, so, perhaps first make > it Work (and then make a Copy). :-) Cheers & hth., - Alf From kerensaelise at hotmail.com Sat Jan 30 20:58:57 2010 From: kerensaelise at hotmail.com (elsa) Date: Sat, 30 Jan 2010 17:58:57 -0800 (PST) Subject: Still too slow References: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> <4B64CE82.1040109@optimum.net> Message-ID: Hi John and others, sorry about my etiquette errors. As you can tell I'm a newbie, and appreciate all the help I can get. I'm trying to master this thing with only the net and a couple of books as tutors. Here is what I'm running at the interactive prompt: >>> import myBDM >>> L=[[100,'NA']] >>> myBDM.evolve(L,100000) Ideally, I'd like to bump it up to myBDM.evolve(L,10000000). L keeps track of the number of subpopulations, how many individuals are in each subpopulation, and the parent subpopulation each subpopulation arose from. Here is my code again: import random n=100 def evolve(L,limit): """ evolves the population until the population size reaches limit, by choosing an individual of a particular subpopulation type, then randomly performing a birth, death, or mutation on this individual """ global n while n References: <85ee4831-3b9c-4096-b32a-387e59d6e7c1@r6g2000yqn.googlegroups.com> Message-ID: <4B64E9DD.7050609@ieee.org> peskar.miha at hotmail.com wrote: > 21 days has passed and still noone is willing to help :-( > > ch /willing/able/ I wouldn't say no-one, even then, since there were at least 10 messages in the thread on the 19th and 20th. Presumably they weren't all from you. If you were doing this to text files, I would have been happy to help. But interacting with MS Word isn't easy, and I don't know how. If you're a beginner as you stated, you need to pick an easier first task. If you're determined to do it, best advice I could give is to head over to the python-Win32 mailing list. At least they're used to dealing with COM, and other MS-specific stuff. DaveA From ben+python at benfinney.id.au Sat Jan 30 21:39:49 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 31 Jan 2010 13:39:49 +1100 Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <0374c6d6$0$1309$c3e8da3@news.astraweb.com> Message-ID: <871vh7t2tm.fsf@benfinney.id.au> Steven D'Aprano writes: > Unfortunately the magic number doesn't seem to be documented anywhere > I can find other than in the source code (import.c). The PEP gives > some examples: > > f2b30a0d.pyc # Python 2.5 > f2d10a0d.pyc # Python 2.6 > f2d10a0d.pyo # Python 2.6 -O > f2d20a0d.pyc # Python 2.6 -U > 0c4f0a0d.pyc # Python 3.1 > > but how can one map magic numbers to versions, short of reading > import.c? Mapping magic numbers to versions is infeasible and will be incomplete: Any mapping that exists in (say) Python 3.1 can't know in advance what the magic number will be for Python 4.5. The more important mapping is from version to magic number. > I propose that sys grow an object sys.magic which is the hexlified magic > number. The ?imp? module already has this:: >>> import sys >>> import imp >>> sys.version '2.5.4 (r254:67916, Jan 24 2010, 16:09:54) \n[GCC 4.4.2]' >>> imp.get_magic().encode('hex') 'b3f20d0a' Unfortunately, I think the examples in the PEP have mangled the magic number into little-endian byte ordering. -- \ ?I got up the other day, and everything in my apartment has | `\ been stolen and replaced with an exact replica.? ?Steven Wright | _o__) | Ben Finney From no.email at nospam.invalid Sat Jan 30 21:40:42 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 30 Jan 2010 18:40:42 -0800 Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <0374c6d6$0$1309$c3e8da3@news.astraweb.com> <871vh7t2tm.fsf@benfinney.id.au> Message-ID: <7xljffngid.fsf@ruckus.brouhaha.com> Ben Finney writes: >> 0c4f0a0d.pyc # Python 3.1 > Mapping magic numbers to versions is infeasible and will be incomplete: > Any mapping that exists in (say) Python 3.1 can't know in advance what > the magic number will be for Python 4.5. But why do the filenames have magic numbers instead of version numbers? From python at mrabarnett.plus.com Sat Jan 30 21:45:24 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 31 Jan 2010 02:45:24 +0000 Subject: Still too slow In-Reply-To: References: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> <4B64CE82.1040109@optimum.net> Message-ID: <4B64EEC4.7010004@mrabarnett.plus.com> elsa wrote: > Hi John and others, > > sorry about my etiquette errors. As you can tell I'm a newbie, and > appreciate all the help I can get. I'm trying to master this thing > with only the net and a couple of books as tutors. > > Here is what I'm running at the interactive prompt: > [snip] > > def event(): > """ > randomly chooses a birth (prob .3), death (prob .1) or mutation > (prob .1) event > """ > choice = random.random() > if choice <= .3: > event='b' > elif choice <= .4: > event ='d' > elif choice<= .5: > event = 'm' > else: > event = 'None' > return event > [snip] Here's my line of thought: Currently, 0 <= choice < 1.0. If you multiplied the random number by 10 then the probability thresholds could all be ints, and if you're using ints then 'choice' could be an index into a list: event_choices = ['b', 'b', 'b', 'd', 'm'] + ['None'] * 5 which then suggests using random.choice() instead: def event(): """ randomly chooses a birth (prob .3), death (prob .1) or mutation (prob .1) event """ return random.choice(event_choices) and you could put the event code inline instead of in a function, which saves the cost of a function call. Incidentally, for a probability of 0.3 the test should be "choice < 0.3", etc, not "choice <= 0.3", etc, although I suspect that you won't notice any difference in the results! :-) From steve at REMOVE-THIS-cybersource.com.au Sat Jan 30 22:09:25 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 31 Jan 2010 03:09:25 GMT Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <0374c6d6$0$1309$c3e8da3@news.astraweb.com> <871vh7t2tm.fsf@benfinney.id.au> <7xljffngid.fsf@ruckus.brouhaha.com> Message-ID: <0374e0be$0$1309$c3e8da3@news.astraweb.com> On Sat, 30 Jan 2010 18:40:42 -0800, Paul Rubin wrote: > Ben Finney writes: >>> 0c4f0a0d.pyc # Python 3.1 >> Mapping magic numbers to versions is infeasible and will be incomplete: >> Any mapping that exists in (say) Python 3.1 can't know in advance what >> the magic number will be for Python 4.5. > > But why do the filenames have magic numbers instead of version numbers? The magic number changes with each incompatible change in the byte code format, which is not the same as each release. Selected values taken from import.c: Python 2.5a0: 62071 Python 2.5a0: 62081 (ast-branch) Python 2.5a0: 62091 (with) Python 2.5a0: 62092 (changed WITH_CLEANUP opcode) Python 2.5b3: 62101 (fix wrong code: for x, in ...) Python 2.5b3: 62111 (fix wrong code: x += yield) Python 2.5c1: 62121 (fix wrong lnotab with for loops and storing constants that should have been removed) Python 2.5c2: 62131 (fix wrong code: for x, in ... in listcomp/genexp) http://svn.python.org/view/python/trunk/Python/import.c?view=markup The relationship between byte code magic number and release version number is not one-to-one. We could have, for the sake of the argument, releases 3.2.3 through 3.5.0 (say) all having the same byte codes. What version number should the .pyc file show? -- Steven From davea at ieee.org Sat Jan 30 22:42:21 2010 From: davea at ieee.org (Dave Angel) Date: Sat, 30 Jan 2010 22:42:21 -0500 Subject: Still too slow In-Reply-To: <4B64CE82.1040109@optimum.net> References: <6f4dc6f5-7af6-4fe0-ba8c-6da7e366af3b@z10g2000prh.googlegroups.com> <4B64CE82.1040109@optimum.net> Message-ID: <4B64FC1D.9040200@ieee.org> John Posner wrote: >
On > 1/30/2010 6:08 PM, elsa wrote: >> Hello again, >> >> Thanks for the tips r.e random.ranint(). This improved matters >> somewhat, however my program is still too slow. If anyone has any >> further tips on how to speed it up, they would be much appreciated! >> >> So, I'm calling evolve(L,limit) from the interactive prompt. L is >> initally [[100],['NA']]. Ideally limit would be 10^7. >> >> Here is my program: >> >> import random >> n=100 >> >> def evolve(L,limit): >> global n >> while n> evnt = event() >> if evnt!="None": >> ind = chooseInd(L,n) >> action(evnt,L,ind) >> >> def chooseInd(L,n): >> choiceSum=0 >> index=0 >> choice = random.randint(1,n) >> while choiceSum< choice: >> choiceSum+=L[index][0] >> index +=1 >> return (index-1) >> >> def event(): >> choice = random.random() >> if choice<= .3: >> event='b' >> elif choice<= .4: >> event ='d' >> elif choice<= .5: >> event = 'm' >> else: >> event = 'None' >> return event >> >> def action(event, L, index): >> global n >> if event == 'b': >> L[index][0]+=1 >> n +=1 >> elif event == 'd': >> L[index][0]-=1 >> n -=1 >> elif event == 'm': >> L.append([1,index]) >> n +=1 >> >> >> thanks in advance, >> >> Elsa. > > Elsa, > > 1. You changed the subject line from "For loop searching takes too > long!" to "Still too slow". This causes newsreader programs to start a > new discussion thread, which makes life difficult for people who need > to refer back to previous messages. Please don't change the subject > line any more. > > 2. You provided a very clear description of your original question: > > Now, what I need to do is randomly choose one myList[i], however the > distribution of my random choice needs to be proportional to the > values of myList[i][0]. > > This description should be the doc string for the chooseInd() function > -- for example: > > def chooseInd(L,n): > """ > randomly choose one L[i], so that the distribution > of choices is proportional to the values of L[i][0] > """ > > It is not clear (to me, anyway) what the other functions are trying to > accomplish. So please add a doc string to each of these functions, > with descriptions of similar clarity. This will help us a lot. And it > will help you, too, if you return to the code after a few > days/weeks/months of directing your attention elsewhere. > > 3. Please provide a *complete* transcript of an interactive session > that exercises this code. > > Tx, > John > > Remarks aimed at elsa, not John. You used tabs in your source code, which don't survive mail very well. So indentation is different in my quoted copy than presumably in your original. No bug there, but I much prefer having my editor expand all tabs into spaces (4 colums) You have a typo in the initialization of L. Try pasting it from a working session, instead of retyping it. Presumably you wanted L= [[100,'NA']]. rather than [[100],['NA']]. The latter value throws an exception in the code. You could start simply by changing the values in the event() function. No point in returning None half the time when that'll accomplish nothing. So just double the fractions you're checking against, or scale the random value. The pattern produced is interesting. The only count that gets very big is L[0][0]. ("Them what has, gets") Lots of the other counts reach zero, and once they do, they won't ever change. The algorithm is O(N*N), and it already takes a long time for 10**4. So 10**7 would be enormous. If the improvement needed were minor, you should profile it, figure where all the time is spent, and optimize that. But I think that's hopeless with this data structure. At a guess, I'd think you should build a single list that grows to size 'limit', where you start with 100 items of value "NA". No counts needed, because they're all implicitly count of 1. Then write your code to manipulate that list. When you finish, construct the list you really want, by using something like the groupby() function. DaveA From steve at REMOVE-THIS-cybersource.com.au Sat Jan 30 22:43:56 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 31 Jan 2010 03:43:56 GMT Subject: Function name unchanged in error message References: <50697b2c1001290610t1a2ac5c8hcd5f9856aa85392b@mail.gmail.com> <5303e556-e9df-4ef3-bd4d-fa4928399c6e@r19g2000yqb.googlegroups.com> Message-ID: <0374e8d5$0$1309$c3e8da3@news.astraweb.com> On Sat, 30 Jan 2010 14:26:43 -0800, andrew cooke wrote: > On Jan 29, 11:50?am, exar... at twistedmatrix.com wrote: >> new.function and new.code will let you construct new objects with >> different values (and copying over whichever existing attributes you >> want to preserve). > > unfortunately new is deprecated and dropped from 3. i can't see how the > same functionality is available in the types module for 3 - am i missing > something obvious? You have to get the constructor from an existing object. type(obj) will always return the type object, which you can use as a constructor. >>> newfunction = type(lambda: None) >>> newcode = type((lambda: None).__code__) Possibly even easier: >>> import types >>> types.FunctionType is newfunction True >>> types.CodeType is newcode True So that's two ways to get a constructor. Now all we need is to learn how they work: >>> help(newfunction) Help on class function in module builtins: class function(object) | function(code, globals[, name[, argdefs[, closure]]]) | | Create a function object from a code object and a dictionary. [...] >>> help(newcode) Help on class code in module builtins: class code(object) | code(argcount, kwonlyargcount, nlocals, stacksize, flags, codestring, | constants, names, varnames, filename, name, firstlineno, | lnotab[, freevars[, cellvars]]) | | Create a code object. Not for the faint of heart. [...] Not for the faint of heart indeed! The best way of using this is to copy the parameters from an existing code object, one created by using def. Look for attributes of f.__code__ starting with "co_". Don't forget compile as well, which may help. -- Steven From alfps at start.no Sat Jan 30 22:44:18 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 31 Jan 2010 04:44:18 +0100 Subject: PEP 3147 - new .pyc format In-Reply-To: <0374e0be$0$1309$c3e8da3@news.astraweb.com> References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <0374c6d6$0$1309$c3e8da3@news.astraweb.com> <871vh7t2tm.fsf@benfinney.id.au> <7xljffngid.fsf@ruckus.brouhaha.com> <0374e0be$0$1309$c3e8da3@news.astraweb.com> Message-ID: * Steven D'Aprano: > On Sat, 30 Jan 2010 18:40:42 -0800, Paul Rubin wrote: > >> Ben Finney writes: >>>> 0c4f0a0d.pyc # Python 3.1 >>> Mapping magic numbers to versions is infeasible and will be incomplete: >>> Any mapping that exists in (say) Python 3.1 can't know in advance what >>> the magic number will be for Python 4.5. >> But why do the filenames have magic numbers instead of version numbers? > > The magic number changes with each incompatible change in the byte code > format, which is not the same as each release. Selected values taken from > import.c: > > Python 2.5a0: 62071 > Python 2.5a0: 62081 (ast-branch) > Python 2.5a0: 62091 (with) > Python 2.5a0: 62092 (changed WITH_CLEANUP opcode) > Python 2.5b3: 62101 (fix wrong code: for x, in ...) > Python 2.5b3: 62111 (fix wrong code: x += yield) > Python 2.5c1: 62121 (fix wrong lnotab with for loops and > storing constants that should have been > removed) > Python 2.5c2: 62131 (fix wrong code: for x, in ... in > listcomp/genexp) > > > http://svn.python.org/view/python/trunk/Python/import.c?view=markup > > The relationship between byte code magic number and release version > number is not one-to-one. We could have, for the sake of the argument, > releases 3.2.3 through 3.5.0 (say) all having the same byte codes. What > version number should the .pyc file show? I don't know enough about Python yet to comment on your question, but, just an idea: how about a human readable filename /with/ some bytecode version id (that added id could be the magic number)? I think that combo could serve both the human and machine needs, so to speak. :-) Cheers, - Alf From steve at REMOVE-THIS-cybersource.com.au Sun Jan 31 00:03:35 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 31 Jan 2010 05:03:35 GMT Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <0374c6d6$0$1309$c3e8da3@news.astraweb.com> <871vh7t2tm.fsf@benfinney.id.au> <7xljffngid.fsf@ruckus.brouhaha.com> <0374e0be$0$1309$c3e8da3@news.astraweb.com> Message-ID: <0374fb80$0$1309$c3e8da3@news.astraweb.com> On Sun, 31 Jan 2010 04:44:18 +0100, Alf P. Steinbach wrote: >> The relationship between byte code magic number and release version >> number is not one-to-one. We could have, for the sake of the argument, >> releases 3.2.3 through 3.5.0 (say) all having the same byte codes. What >> version number should the .pyc file show? > > I don't know enough about Python yet to comment on your question, but, > just an idea: how about a human readable filename /with/ some bytecode > version id (that added id could be the magic number)? Sorry, that still doesn't work. Consider the hypothetical given above. For simplicity, I'm going to drop the micro point versions, so let's say that releases 3.2 through 3.5 all use the same byte-code. (In reality, you'll be likely looking at version numbers like 3.2.1 rather than just 3.2.) Now suppose you have 3.2 and 3.5 both installed, and you look inside your $PYTHONPATH and see this: spam.py spam.pyr/ 3.2-f2e70a0d.pyc It would be fairly easy to have the import machinery clever enough to ignore the version number prefix, so that Python 3.5 correctly uses 3.2-f2e70a0d.pyc. That part is easy. (I trust nobody is going to suggest that Python should create multiple redundant, identical, copies of the .pyc files. That would be just lame.) But now consider the human reader. You want human-readable file names for the benefit of the human reader. How is the human reader supposed to know that Python 3.5 is going to use 3.2-f2e70a0d.pyc? Suppose I'm running Python 3.5, and have a troubling bug, and I think "I know, maybe there's some sort of problem with the compiled byte code, I should delete it". I go looking for spam.pyr/3.5-*.pyc and don't find anything. Now I have two obvious choices: (1) Start worrying about why Python 3.5 isn't writing .pyc files. Is my installation broken? Have I set the PYTHONDONTWRITEBYTECODE environment variable? Do I not have write permission to the folder? WTF is going on? Confusion will reign. (2) Learn that the 3.2- prefix is meaningless, and ignore it. Since you have to ignore the version number prefix anyway, let's not lay a trap for people by putting it there. You need to know the magic number to do anything sensible with the .pyc files other than delete them, so let's not pretend otherwise. If you don't wish to spend time looking up the magic number, the solution is simple: hit it with a sledgehammer. Why do you care *which* specific .pyc file is being used anyway? rm -r spam.pyr/ or for the paranoid: rm spam.pyr/*.py[co] (Obviously there will be people who care about the specific .pyc file being deleted. Those people can't use a sledgehammer, they need one of those little tack-hammers with the narrow head. But 99% of the time, a sledgehammer is fine.) Frankly, unless you're a core developer or you're hacking byte-code, you nearly always won't care about the .pyc files. You want the compiler to manage them. And for the few times you do care, it isn't hard to find out: >>> import binascii, imp >>> binascii.hexlify(imp.get_magic()) b'4f0c0d0a' Stick that in your "snippets" folder (you do have a snippets folder, don't you?) and, as they say Down Under, "She'll be right mate". -- Steven From a.abushkevich at gmail.com Sun Jan 31 03:17:21 2010 From: a.abushkevich at gmail.com (Aliaksandr Abushkevich) Date: Sun, 31 Jan 2010 10:17:21 +0200 Subject: Slow down while creating a big list and iterating over it In-Reply-To: <2fdd4e1b-db58-45d7-9b7d-73dcdca6384a@c29g2000yqd.googlegroups.com> References: <2fdd4e1b-db58-45d7-9b7d-73dcdca6384a@c29g2000yqd.googlegroups.com> Message-ID: Maybe it is a good idea to use Disco (http://discoproject.org/) to process your data. Yours faithfully, Alexander Abushkevich On Sat, Jan 30, 2010 at 10:36 PM, marc magrans de abril wrote: > Dear colleagues, > > I was doing a small program to classify log files for a cluster of > PCs, I just wanted to simplify a quite repetitive task in order to > find errors and so. > > My first naive implementation was something like: > ? ?patterns = [] > ? ?while(logs): > ? ? ? ?pattern = logs[0] > ? ? ? ?new_logs = [l for l in logs if dist(pattern,l)>THERESHOLD] > ? ? ? ?entry = (len(logs)-len(new_logs),pattern) > ? ? ? ?patterns.append(entry) > ? ? ? ?logs = new_logs > > Where dist(...) is the levenshtein distance (i.e. edit distance) and > logs is something like 1.5M logs (700 MB file). I thought that python > will be an easy choice although not really fast.. > > I was not surprised when the first iteration of the while loop was > taking ~10min. I thought "not bad, let's how much it takes". However, > it seemed that the second iteration never finished. > > My surprise was big when I added a print instead of the list > comprehension: > new_logs=[] > for count,l in enumerate(logs): > ? print count > ? if dist(pattern,l)>THERESHOLD: > ? ? ?new_logs.append(l) > > The surprise was that the displayed counter was running ~10 times > slower on the second iteration of the while loop. > > I am a little lost. Anyone knows the reson of this behavior? ?How > should I write a program that deals with large data sets in python? > > Thanks a lot! > marc magrans de abril > -- > http://mail.python.org/mailman/listinfo/python-list > From buzzilo at gmail.com Sun Jan 31 04:47:37 2010 From: buzzilo at gmail.com (buzz) Date: Sun, 31 Jan 2010 01:47:37 -0800 (PST) Subject: pyjon: pythonic javascript interpreter Message-ID: <2758e72f-c818-4827-8dda-34cd19db0ef6@c4g2000yqa.googlegroups.com> hi, Here is: http://code.google.com/p/pyjon/ Although the code is far from stable and completed, it's a good place to start play with. -- Evgen From dickinsm at gmail.com Sun Jan 31 05:02:01 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Sun, 31 Jan 2010 02:02:01 -0800 (PST) Subject: unencountered error in FFT python References: <4b64800a$0$6735$9b4e6d93@newsspool2.arcor-online.net> <4b6486d6$0$6724$9b4e6d93@newsspool2.arcor-online.net> Message-ID: On Jan 30, 8:20?pm, uche wrote: > Another issue: > ? ? x[a], x[b] = x[(a)] + W[(n % N)] * x[(b)], x[(a)] - W[(n % N)] * x > [(b)] > TypeError: can't multiply sequence by non-int of type 'complex' With your original code, the elements of array2 are strings, and here Python is refusing to multiply the string x[a] by the complex number W [n % N]. Presumably you want the elements of array2 to be integers or floats instead? (Your comments suggest integers, though I think floats would be more natural here.) MRAB's earlier reply suggests how to fix this: array2 = [] for a in range(len(array)): array2.append(int(array[a])) # note the extra int! That works, but the code is cleaner if you iterate over the elements of the array directly, instead of over their indices: array2 = [] for x in array: array2.append(int(x)) There are other ways, too: a seasoned Python user would probably write this either as a list comprehension: array2 = [int(x) for x in array] ... or using the built-in map function: array2 = map(int, array) -- Mark From dickinsm at gmail.com Sun Jan 31 05:07:11 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Sun, 31 Jan 2010 02:07:11 -0800 (PST) Subject: unencountered error in FFT python References: <4b64800a$0$6735$9b4e6d93@newsspool2.arcor-online.net> <4b6486d6$0$6724$9b4e6d93@newsspool2.arcor-online.net> Message-ID: <1546ea7e-6e22-4185-8aaf-0ce52caf4ed7@a5g2000yqi.googlegroups.com> On Jan 31, 10:02?am, Mark Dickinson wrote: > Python is refusing to multiply the string x[a] by the complex number W > [n % N]. Whoops, that should have been "x[b]", not "x[a]". Why is it that a post-submission proofread turns up errors so much more often than a pre-submission proofread? -- Mark From martin at v.loewis.de Sun Jan 31 05:13:06 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Sun, 31 Jan 2010 11:13:06 +0100 Subject: PEP 3147 - new .pyc format In-Reply-To: <877hqz2j1o.fsf@castleamber.com> References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <87iqaj9n8z.fsf@castleamber.com> <877hqz2j1o.fsf@castleamber.com> Message-ID: <4B6557B2.5040805@v.loewis.de> > Naming files using magic numbers is really beyond me. The fact that the > above needs comments to explain what's what already shows to me that > there's a problem with this naming scheme. What if for one reason or > another I want to delete all pyc files for Python 2.5? Where do I look > up the magic number? py> import imp, binascii py> binascii.hexlify(imp.get_magic()) 'b3f20d0a' (note: this is on a little-endian system) Regards, Martin From martin at v.loewis.de Sun Jan 31 05:17:19 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Sun, 31 Jan 2010 11:17:19 +0100 Subject: PEP 3147 - new .pyc format In-Reply-To: References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <87iqaj9n8z.fsf@castleamber.com> <877hqz2j1o.fsf@castleamber.com> Message-ID: <4B6558AF.1090009@v.loewis.de> > True. You might also want to note that "Python 2.6 -U" appears to have a > different magic number from "Python 2.6" and "Python 2.6 -O". > > I don't know whether they always change for each new version. Here is a recent list of magic numbers: Python 2.6a0: 62151 (peephole optimizations and STORE_MAP opcode) Python 2.6a1: 62161 (WITH_CLEANUP optimization) Python 2.7a0: 62171 (optimize list comprehensions/change LIST_APPEND) Python 2.7a0: 62181 (optimize conditional branches: introduce POP_JUMP_IF_FALSE and POP_JUMP_IF_TRUE) Python 2.7a0 62191 (introduce SETUP_WITH) Python 2.7a0 62201 (introduce BUILD_SET) Python 2.7a0 62211 (introduce MAP_ADD and SET_ADD) #define MAGIC (62211 | ((long)'\r'<<16) | ((long)'\n'<<24)) Regards, Martin From stefan_ml at behnel.de Sun Jan 31 05:35:43 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 31 Jan 2010 11:35:43 +0100 Subject: unencountered error in FFT python In-Reply-To: <1546ea7e-6e22-4185-8aaf-0ce52caf4ed7@a5g2000yqi.googlegroups.com> References: <4b64800a$0$6735$9b4e6d93@newsspool2.arcor-online.net> <4b6486d6$0$6724$9b4e6d93@newsspool2.arcor-online.net> <1546ea7e-6e22-4185-8aaf-0ce52caf4ed7@a5g2000yqi.googlegroups.com> Message-ID: <4b655cff$0$7617$9b4e6d93@newsspool1.arcor-online.net> Mark Dickinson, 31.01.2010 11:07: > On Jan 31, 10:02 am, Mark Dickinson wrote: >> Python is refusing to multiply the string x[a] by the complex number W >> [n % N]. > > Whoops, that should have been "x[b]", not "x[a]". Why is it that a > post-submission proofread turns up errors so much more often than a > pre-submission proofread? Because it's easier to find mistakes in other people's text than in your own ones. Once the message is sent off to the Aether, when it comes back, you read it like someone else has written it, so you immediately find all those obvious bugs in it. Fits into the same corner as rubber-duck debugging. Stefan From rantingrick at gmail.com Sun Jan 31 06:01:51 2010 From: rantingrick at gmail.com (rantingrick) Date: Sun, 31 Jan 2010 03:01:51 -0800 (PST) Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> Message-ID: <7536f9c0-a36a-479d-ba95-9405b13793fd@e25g2000yqh.googlegroups.com> On Jan 30, 10:43?am, Nobody wrote: > That's also true for most functional languages, e.g. Haskell and ML, as > well as e.g. Tcl and most shells. Why require "f(x)" or "(f x)" if "f x" > will suffice? yuck! wrapping the arg list with parenthesis (python way) makes the most sense. Its to easy to misread somthing like this onetwothree four five six onetwothree(four, five, six) #ahhh... plain english. From steve at REMOVE-THIS-cybersource.com.au Sun Jan 31 06:35:10 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 31 Jan 2010 11:35:10 GMT Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> <7536f9c0-a36a-479d-ba95-9405b13793fd@e25g2000yqh.googlegroups.com> Message-ID: <03755745$0$1309$c3e8da3@news.astraweb.com> On Sun, 31 Jan 2010 03:01:51 -0800, rantingrick wrote: > On Jan 30, 10:43?am, Nobody wrote: > >> That's also true for most functional languages, e.g. Haskell and ML, as >> well as e.g. Tcl and most shells. Why require "f(x)" or "(f x)" if "f >> x" will suffice? > > yuck! wrapping the arg list with parenthesis (python way) makes the most > sense. Its to easy to misread somthing like this > > onetwothree four five six > > onetwothree(four, five, six) #ahhh... plain english. I think the readability factor is mostly down to what you're familiar with. But consistency is also important: in Python, you always refer to an object the same way. Given an object called x, you ALWAYS refer to the object itself as x. In languages that don't use parentheses, x refers to the object, unless the object is a function, in which case x refers to the result of calling the object with no arguments. Other languages need special syntax to get access to the function object itself. Because it's hard to do, people don't do it often. But in Python, getting the function object is easy, and so treating functions as first- class objects is easy. -- Steven From mr.spoon21 at gmail.com Sun Jan 31 06:51:46 2010 From: mr.spoon21 at gmail.com (Mr.SpOOn) Date: Sun, 31 Jan 2010 11:51:46 +0000 Subject: Keyboard input In-Reply-To: References: <8f67b6f81001290853i39bab1acs49d02ac5be948d13@mail.gmail.com> Message-ID: <8f67b6f81001310351m3485fd40o8944c11af8a4161f@mail.gmail.com> 2010/1/29 Gabriel Genellina : > > That's strange. If you're using Linux, make sure you have the readline > package installed. I'm using Linux. Ubuntu. I checked on synaptic and I have readline-common. Do you mean something else? Anyway I tried running the program in the iPython shell and it works great. I mean, I can do backspace and other stuff. > If readline capabilities are not enough, you could try Urwid, a library for > writing console programs: http://excess.org/urwid/ - but it's a big move. > wxPython would let you write a graphical interface (and it's a bigger move!) Yes, I know about Urwid and wxPython and they're just too much. I was thinking about curses too, as suggested by others here, but that's too much too. From affdfsdfdsfsd at b.com Sun Jan 31 07:07:55 2010 From: affdfsdfdsfsd at b.com (Tracubik) Date: 31 Jan 2010 12:07:55 GMT Subject: create a string of variable lenght Message-ID: <4b65729b$0$1126$4fafbaef@reader3.news.tin.it> Hi all, i want to print on linux console (terminal) a message like this one: ******************************** error message of variable lenght ******************************** to print the asterisks line i do this: def StringOfAsterisks(myString): asterisksString = "*" for i in range(1,len(myString): asterisksString += "*" print asterisksString so i create the asterisksString of the lenght i need There is a better way of creating asterisksString? the code seem to work for me, but it doesn't work properly if in errorMsg there is some "esotic" char like euro char (?): >>> euro = "?" >>> len(euro) 3 how i can solve this? Thank you all in advance and sorry for my bad english Nico From e_d_k at yahoo.com Sun Jan 31 07:28:41 2010 From: e_d_k at yahoo.com (Ed Keith) Date: Sun, 31 Jan 2010 04:28:41 -0800 (PST) Subject: Python and Ruby In-Reply-To: <03755745$0$1309$c3e8da3@news.astraweb.com> Message-ID: <681892.91862.qm@web58903.mail.re1.yahoo.com> --- On Sun, 1/31/10, Steven D'Aprano wrote: > From: Steven D'Aprano > Subject: Re: Python and Ruby > To: python-list at python.org > Date: Sunday, January 31, 2010, 6:35 AM > On Sun, 31 Jan 2010 03:01:51 -0800, > rantingrick wrote: > > > On Jan 30, 10:43?am, Nobody > wrote: > > > >> That's also true for most functional languages, > e.g. Haskell and ML, as > >> well as e.g. Tcl and most shells. Why require > "f(x)" or "(f x)" if "f > >> x" will suffice? > > > > yuck! wrapping the arg list with parenthesis (python > way) makes the most > > sense. Its to easy to misread somthing like this > > > > onetwothree four five six > > > > onetwothree(four, five, six) #ahhh... plain english. > > I think the readability factor is mostly down to what > you're familiar > with. But consistency is also important: in Python, you > always refer to > an object the same way. Given an object called x, you > ALWAYS refer to the > object itself as x. In languages that don't use > parentheses, x refers to > the object, unless the object is a function, in which case > x refers to > the result of calling the object with no arguments. > > Other languages need special syntax to get access to the > function object > itself. Because it's hard to do, people don't do it often. > But in Python, > getting the function object is easy, and so treating > functions as first- > class objects is easy. > In most functional languages you just name a function to access it and you do it ALL the time. for example, in if you have a function 'f' which takes two parameters to call the function and get the result you use: f 2 3 If you want the function itself you use: f The reason no parentheses are used is to support Currying (http://en.wikipedia.org/wiki/Currying). To get a new function which is equivalent to f with the first parameter set to a constant 2 you use: f 2 this give you a function which take only one parameter. Using parenthesis make currying more complicated, so most functional languages do not use them. It did take me a LONG time to get used to this, but it is only syntax, I do not let syntax bother me. Semantics on the other hand, are a big deal. -EdK Ed Keith e_d_k at yahoo.com Blog: edkeith.blogspot.com > From python.list at tim.thechases.com Sun Jan 31 07:35:31 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Sun, 31 Jan 2010 06:35:31 -0600 Subject: create a string of variable lenght In-Reply-To: <4b65729b$0$1126$4fafbaef@reader3.news.tin.it> References: <4b65729b$0$1126$4fafbaef@reader3.news.tin.it> Message-ID: <4B657913.5010303@tim.thechases.com> Tracubik wrote: > ******************************** > error message of variable lenght > ******************************** > > to print the asterisks line i do this: > > def StringOfAsterisks(myString): > asterisksString = "*" > for i in range(1,len(myString): > asterisksString += "*" > print asterisksString > > so i create the asterisksString of the lenght i need > > There is a better way of creating asterisksString? well, to make it more pythonic (ignoring camel-case variable-names for now), I'd just use print '*' * len(myString) print myString print '*' * len(myString) possibly stashing the resulting asterisksString once: asterisks_string = '*' * len(my_string) print asterisks_string print my_string print asterisks_string If I used it in multiple places, I might wrap it in a function and/or define a "DIVIDER_CHARACTER" constant, something like DIVIDER_CHARACTER = '*' def surround(s, divider=DIVIDER_CHARACTER): d = divider[0] print d * len(s) print s print d * len(s) surround('hello') surround('world', '-') surround('foo', 'xo') depending on the sort of behavior you want > the code seem to work for me, but it doesn't work properly if in errorMsg > there is some "esotic" char like euro char (?): > >>>> euro = "?" >>>> len(euro) > 3 I suspect you're seeing the byte-representation of your string with a particular-yet-undisclosed encoding. If it was a unicode string (a good idea to specify the encoding at the top of your file), the length should be accurate, so what happens if you >>> euro = u"?" >>> len(euro) ? (I don't have ready access to a terminal where I can enter unicode characters, and you don't display the representation of the string with print repr(euro) so I can steal the byte values to recreate it; but I suspect the result will correctly be 1). -tkc From __peter__ at web.de Sun Jan 31 07:41:04 2010 From: __peter__ at web.de (Peter Otten) Date: Sun, 31 Jan 2010 13:41:04 +0100 Subject: create a string of variable lenght References: <4b65729b$0$1126$4fafbaef@reader3.news.tin.it> Message-ID: Tracubik wrote: > Hi all, > > i want to print on linux console (terminal) a message like this one: > > ******************************** > error message of variable lenght > ******************************** > > to print the asterisks line i do this: > > def StringOfAsterisks(myString): > asterisksString = "*" > for i in range(1,len(myString): > asterisksString += "*" > print asterisksString > > so i create the asterisksString of the lenght i need > > There is a better way of creating asterisksString? > >>> "*" * 10 '**********' > the code seem to work for me, but it doesn't work properly if in errorMsg > there is some "esotic" char like euro char (?): > >>>> euro = "?" >>>> len(euro) > 3 > > how i can solve this? This is less likely to fail when you use unicode strings: >>> def print_message(s): ... print "*" * len(s) ... print s ... print "*" * len(s) ... >>> print_message(u"You are leaving the ? zone") ************************** You are leaving the ? zone ************************** Peter From gd_usenet at spamfence.net Sun Jan 31 07:46:16 2010 From: gd_usenet at spamfence.net (=?UTF-8?Q?G=C3=BCnther?= Dietrich) Date: Sun, 31 Jan 2010 13:46:16 +0100 Subject: create a string of variable lenght References: <4b65729b$0$1126$4fafbaef@reader3.news.tin.it> Message-ID: Tracubik wrote: >i want to print on linux console (terminal) a message like this one: > >******************************** >error message of variable lenght >******************************** > >to print the asterisks line i do this: > >def StringOfAsterisks(myString): > asterisksString = "*" > for i in range(1,len(myString): > asterisksString += "*" > print asterisksString def StringOfAsterisks(myString): print(''.ljust(len(myString), '*')) You might use string methods .rjust() or .center() instead of method .ljust(). >the code seem to work for me, but it doesn't work properly if in errorMsg >there is some "esotic" char like euro char (?): > >>>> euro = "?" >>>> len(euro) >3 Maybe you might solve this if you decode your string to unicode. Example: |>>> euro = "?" |>>> len(euro) |3 |>>> u_euro = euro.decode('utf_8') |>>> len(u_euro) |1 Adapt the encoding ('utf_8' in my example) to whatever you use. Or create the unicode string directly: |>>> u_euro = u'?' |>>> len(u_euro) |1 Best regards, G?nther From vicente.soler at gmail.com Sun Jan 31 09:03:14 2010 From: vicente.soler at gmail.com (vsoler) Date: Sun, 31 Jan 2010 06:03:14 -0800 (PST) Subject: Unable to install numpy References: <3a772f49-aee6-4450-b60e-8f05daec136a@21g2000yqj.googlegroups.com> Message-ID: On Jan 18, 9:08?pm, Robert Kern wrote: > On 2010-01-18 14:02 PM, vsoler wrote: > > > Hi all, > > > I just download Numpy, and tried to install it using "numpy-1.4.0- > > win32-superpack-python2.6.exe" > > > I get an error: ?"Python version 2.6 required, which was not found in > > the Registry" > > > However, I am using Python 2.6 every day. I'm running Windows 7. > > > What can I do? > > Please ask numpy installation questions on the numpy mailing list. > > ? ?http://www.scipy.org/Mailing_Lists > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless enigma > ? that is made terrible by our own mad attempt to interpret it as though it had > ? an underlying truth." > ? ?-- Umberto Eco Thank you Robert. I'm going to direct my questions towards scipy.org. However, since I do not find my questions already answered, could you tell me where to post it? I beg your pardon for still asking this question outside of scipy.org, but I've been unable to move on. Regards From andrej.mitrovich at gmail.com Sun Jan 31 09:13:44 2010 From: andrej.mitrovich at gmail.com (Andrej Mitrovic) Date: Sun, 31 Jan 2010 06:13:44 -0800 (PST) Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <87iqaj9n8z.fsf@castleamber.com> <877hqz2j1o.fsf@castleamber.com> <4B6558AF.1090009@v.loewis.de> Message-ID: Leave magic to the witches of Perl. :) From marcmagransdeabril at gmail.com Sun Jan 31 09:53:15 2010 From: marcmagransdeabril at gmail.com (marc magrans de abril) Date: Sun, 31 Jan 2010 06:53:15 -0800 (PST) Subject: Slow down while creating a big list and iterating over it References: <2fdd4e1b-db58-45d7-9b7d-73dcdca6384a@c29g2000yqd.googlegroups.com> Message-ID: <29a69a7c-a8a2-4c53-9521-760e13665cd3@f12g2000yqn.googlegroups.com> >Find out which pattern is being used on the second iteration and then try it on the first iteration. Is it just as slow? You were right, the second pattern was 1891 bytes but the first was just 142 :P I will need to put more thought than I expect in the "small script". From john at castleamber.com Sun Jan 31 10:06:18 2010 From: john at castleamber.com (John Bokma) Date: Sun, 31 Jan 2010 09:06:18 -0600 Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <0374c6d6$0$1309$c3e8da3@news.astraweb.com> <871vh7t2tm.fsf@benfinney.id.au> <7xljffngid.fsf@ruckus.brouhaha.com> <0374e0be$0$1309$c3e8da3@news.astraweb.com> <0374fb80$0$1309$c3e8da3@news.astraweb.com> Message-ID: <878wbenwk5.fsf@castleamber.com> Steven D'Aprano writes: > On Sun, 31 Jan 2010 04:44:18 +0100, Alf P. Steinbach wrote: > >>> The relationship between byte code magic number and release version >>> number is not one-to-one. We could have, for the sake of the argument, >>> releases 3.2.3 through 3.5.0 (say) all having the same byte codes. What >>> version number should the .pyc file show? >> >> I don't know enough about Python yet to comment on your question, but, >> just an idea: how about a human readable filename /with/ some bytecode >> version id (that added id could be the magic number)? > > Sorry, that still doesn't work. Consider the hypothetical given above. > For simplicity, I'm going to drop the micro point versions, so let's say > that releases 3.2 through 3.5 all use the same byte-code. (In reality, Based on the magic numbers I've seen so far it looks like that not an option. They increment with every minor change. So to me, at this moment (and maybe it's my ignorance) it looks like a made up example to justify what to me still looks like a bad decision. -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From skip at pobox.com Sun Jan 31 10:13:19 2010 From: skip at pobox.com (skip at pobox.com) Date: Sun, 31 Jan 2010 09:13:19 -0600 Subject: How to get UTC offset for non-standard time zone names? Message-ID: <19301.40463.29743.197241@montanaro.dyndns.org> I have the pytz package but it doesn't know about non-standard timezone names like "CDT5" or "CST6". I can obviously infer that they are either five or six hours behind UTC. Are they constructed in some standard way so that I can assume that if a timezone name is not known to pytz I can assume the trailing number represents the number of hours behind UTC? Is there some standard definition of the format of such semi-numeric timezone names? Thx, -- Skip Montanaro - skip at pobox.com - http://www.smontanaro.net/ From contact at xavierho.com Sun Jan 31 10:23:29 2010 From: contact at xavierho.com (Xavier Ho) Date: Mon, 1 Feb 2010 01:23:29 +1000 Subject: How to get UTC offset for non-standard time zone names? In-Reply-To: <19301.40463.29743.197241@montanaro.dyndns.org> References: <19301.40463.29743.197241@montanaro.dyndns.org> Message-ID: <2d56febf1001310723n1010a4f6t58d760c9e47ad7ce@mail.gmail.com> This may not answer your question directly, but have you thought about ingoring the number at the end of these non-standard timezones? CDT is Central Daylight-saving Timezone, while CST is Central Standard Timezone. And you are correct they are -5 and -6 hours respectively. Does pytz know about CDT and CST? Cheers, -Xav On Mon, Feb 1, 2010 at 1:13 AM, wrote: > > I have the pytz package but it doesn't know about non-standard timezone > names like "CDT5" or "CST6". I can obviously infer that they are either > five or six hours behind UTC. Are they constructed in some standard way so > that I can assume that if a timezone name is not known to pytz I can assume > the trailing number represents the number of hours behind UTC? Is there > some standard definition of the format of such semi-numeric timezone names? > > Thx, > > -- > Skip Montanaro - skip at pobox.com - http://www.smontanaro.net/ > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From no.email at please.post Sun Jan 31 10:38:04 2010 From: no.email at please.post (kj) Date: Sun, 31 Jan 2010 15:38:04 +0000 (UTC) Subject: How to set default encoding for print? Message-ID: It gets tedious to have to append .encode('utf-8') to all my unicode strings when I print them, as in: print foobar.encode('utf-8') I want to tell python to apply this encoding automatically to anything argument passed to print. How can I do this? TIA! K PS: BTW, sys.setdefaultencoding is *not* the answer, since this method gets removed by site.py... From no.email at please.post Sun Jan 31 10:52:28 2010 From: no.email at please.post (kj) Date: Sun, 31 Jan 2010 15:52:28 +0000 (UTC) Subject: How to pass Chinese characters as command-line arguments? Message-ID: I want to pass Chinese characters as command-line arguments to a Python script. My terminal has no problem displaying these characters, and passing them to the script, but I can't get Python to understand them properly. E.g. if I pass one such character to the simple script import sys print sys.argv[1] print type(sys.argv[1]) the first line of the output looks fine (identical to the input), but the second line says "". If I add the line arg = unicode(sys.argv[1]) I get the error Traceback (most recent call last): File "kgrep.py", line 4, in arg = unicode(sys.argv[1]) UnicodeDecodeError: 'ascii' codec can't decode byte 0xe8 in position 0: ordinal not in range(128) What must I do to get Python to recognize command-line arguments as utf-8 Unicode? FWIW, my os is Darwin, my shell (zsh) runs on Terminal, and none of my locale (LC_*) variables is set. TIA! kynn From deets at nospam.web.de Sun Jan 31 10:54:23 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Sun, 31 Jan 2010 16:54:23 +0100 Subject: How to set default encoding for print? In-Reply-To: References: Message-ID: <7slndhFnomU1@mid.uni-berlin.de> Am 31.01.10 16:38, schrieb kj: > It gets tedious to have to append .encode('utf-8') to all my unicode > strings when I print them, as in: > > print foobar.encode('utf-8') > > I want to tell python to apply this encoding automatically to > anything argument passed to print. > > How can I do this? > > TIA! > > K > > PS: BTW, sys.setdefaultencoding is *not* the answer, since this > method gets removed by site.py... Just wrap sys.stdout in a codecs-module based file-wrapper. import codecs import sys sys.stdout = codecs.getwriter("utf-8")(sys.stdout) print u"?" Diez From skip at pobox.com Sun Jan 31 10:57:31 2010 From: skip at pobox.com (Skip Montanaro) Date: Sun, 31 Jan 2010 15:57:31 +0000 (UTC) Subject: How to get UTC offset for non-standard time zone names? References: <19301.40463.29743.197241@montanaro.dyndns.org> <2d56febf1001310723n1010a4f6t58d760c9e47ad7ce@mail.gmail.com> Message-ID: > Does pytz know about CDT and CST? Nope... Skip From kyp at stsci.edu Sun Jan 31 11:22:42 2010 From: kyp at stsci.edu (Kyp) Date: Sun, 31 Jan 2010 08:22:42 -0800 (PST) Subject: iglob performance no better than glob Message-ID: <5ebd4965-32fc-4a03-94dd-c61ccdc9d630@f5g2000vbm.googlegroups.com> I have a dir with a large # of files that I need to perform operations on, but only needing to access a subset of the files, i.e. the first 100 files. Using glob is very slow, so I ran across iglob, which returns an iterator, which seemed just like what I wanted. I could iterate over the files that I wanted, not having to read the entire dir. So the iglob was faster, but accessing the first file took about the same time as glob.glob. Here's some code to compare glob vs. iglob performance, it outputs the time before/after a glob.iglob('*.*') files.next() sequence and a glob.glob('*.*') sequence. #!/usr/bin/env python import glob,time print '\nTest of glob.iglob' print 'before iglob:', time.asctime() files = glob.iglob('*.*') print 'after iglob:',time.asctime() print files.next() print 'after files.next():', time.asctime() print '\nTest of glob.glob' print 'before glob:', time.asctime() files = glob.glob('*.*') print 'after glob:',time.asctime() Here are the results: Test of glob.iglob before iglob: Sun Jan 31 11:09:08 2010 after iglob: Sun Jan 31 11:09:08 2010 foo.bar after files.next(): Sun Jan 31 11:09:59 2010 Test of glob.glob before glob: Sun Jan 31 11:09:59 2010 after glob: Sun Jan 31 11:10:51 2010 The results are about the same for the 2 approaches, both took about 51 seconds. Am I doing something wrong with iglob? Is there a way to get the first X # of files from a dir with lots of files, that does not take a long time to run? thanx, mark From no.email at please.post Sun Jan 31 11:28:55 2010 From: no.email at please.post (kj) Date: Sun, 31 Jan 2010 16:28:55 +0000 (UTC) Subject: How to set default encoding for print? References: <7slndhFnomU1@mid.uni-berlin.de> Message-ID: In <7slndhFnomU1 at mid.uni-berlin.de> "Diez B. Roggisch" writes: >Am 31.01.10 16:38, schrieb kj: >> It gets tedious to have to append .encode('utf-8') to all my unicode >> strings when I print them, as in: >> >> print foobar.encode('utf-8') >> >> I want to tell python to apply this encoding automatically to >> anything argument passed to print. >> >> How can I do this? >> >> TIA! >> >> K >> >> PS: BTW, sys.setdefaultencoding is *not* the answer, since this >> method gets removed by site.py... >Just wrap sys.stdout in a codecs-module based file-wrapper. >import codecs >import sys >sys.stdout = codecs.getwriter("utf-8")(sys.stdout) >print u"??" That's exactly what I was looking for. Thanks! kynn From contact at xavierho.com Sun Jan 31 11:38:16 2010 From: contact at xavierho.com (Xavier Ho) Date: Mon, 1 Feb 2010 02:38:16 +1000 Subject: How to get UTC offset for non-standard time zone names? In-Reply-To: References: <19301.40463.29743.197241@montanaro.dyndns.org> <2d56febf1001310723n1010a4f6t58d760c9e47ad7ce@mail.gmail.com> Message-ID: <2d56febf1001310838ub9bc0e5n4d290b34626d2740@mail.gmail.com> Would it hurt if you put in some extra information? http://www.timeanddate.com/library/abbreviations/timezones/ HTH, -Xav P.S: You, sir, have an awesome first name. On Mon, Feb 1, 2010 at 1:57 AM, Skip Montanaro wrote: > > Does pytz know about CDT and CST? > > Nope... > > Skip > > > -- > http://mail.python.org/mailman/listinfo/python-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From deets at nospam.web.de Sun Jan 31 11:58:25 2010 From: deets at nospam.web.de (Diez B. Roggisch) Date: Sun, 31 Jan 2010 17:58:25 +0100 Subject: How to pass Chinese characters as command-line arguments? In-Reply-To: References: Message-ID: <7slr5iFe66U1@mid.uni-berlin.de> Am 31.01.10 16:52, schrieb kj: > I want to pass Chinese characters as command-line arguments to a > Python script. My terminal has no problem displaying these > characters, and passing them to the script, but I can't get Python > to understand them properly. > > E.g. if I pass one such character to the simple script > > import sys > print sys.argv[1] > print type(sys.argv[1]) > > the first line of the output looks fine (identical to the input), > but the second line says "". If I add the line > > arg = unicode(sys.argv[1]) > > I get the error > > Traceback (most recent call last): > File "kgrep.py", line 4, in > arg = unicode(sys.argv[1]) > UnicodeDecodeError: 'ascii' codec can't decode byte 0xe8 in position 0: ordinal not in range(128) > > What must I do to get Python to recognize command-line arguments > as utf-8 Unicode? The last sentence reveals your problem: utf-8 is *not* unicode. It's an encoding of unicode, which is a crucial difference. From the outside you get byte-streams, and if these happen to be encoded in utf-8, you can simply decode them: arg = unicode(sys.argv[1], "utf-8") Diez From skip at pobox.com Sun Jan 31 12:29:57 2010 From: skip at pobox.com (Skip Montanaro) Date: Sun, 31 Jan 2010 17:29:57 +0000 (UTC) Subject: How to get UTC offset for non-standard time zone names? References: <19301.40463.29743.197241@montanaro.dyndns.org> <2d56febf1001310723n1010a4f6t58d760c9e47ad7ce@mail.gmail.com> <2d56febf1001310838ub9bc0e5n4d290b34626d2740@mail.gmail.com> Message-ID: > Would it hurt if you put in some extra information? > http://www.timeanddate.com/library/abbreviations/timezones/ In theory, no. At work we still use the ancient Rogue Wave C++ libraries in a number of applications. It has hard-coded timezone info so when the US changed the start and end of daylight savings a few years ago we adapted by switching from US/Central to either CDT5 or CST6 for a week around each entry/exit. For those couple weeks our TZ environment variable is set to one of those two values. I could hard-code a similar exception in my Python code, but it also has to run in London and New York (at least). In the past we've had offices in Hawaii and Asia, so I'd really like it if my code was location-independent. Skip P.S. Thanks for the compliment on my nickname. My first name is actually George. Still impressed? ;-) From skip at pobox.com Sun Jan 31 12:59:49 2010 From: skip at pobox.com (Skip Montanaro) Date: Sun, 31 Jan 2010 17:59:49 +0000 (UTC) Subject: iglob performance no better than glob References: <5ebd4965-32fc-4a03-94dd-c61ccdc9d630@f5g2000vbm.googlegroups.com> Message-ID: > So the iglob was faster, but accessing the first file took about the > same time as glob.glob. I'll wager most of the time required to access the first file is due to filesystem overhead, not any inherent limitation in Python. Skip Montanaro From john at castleamber.com Sun Jan 31 13:06:29 2010 From: john at castleamber.com (John Bokma) Date: Sun, 31 Jan 2010 12:06:29 -0600 Subject: iglob performance no better than glob References: <5ebd4965-32fc-4a03-94dd-c61ccdc9d630@f5g2000vbm.googlegroups.com> Message-ID: <87k4uyw3mi.fsf@castleamber.com> Kyp writes: > Is there a way to get the first X # of files from a dir with lots of > files, that does not take a long time to run? Assuming Linux: what does time ls thedir | head give? with thedir the name of the actual dir Also how many is many files? -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From wolftracks at invalid.com Sun Jan 31 13:10:53 2010 From: wolftracks at invalid.com (W. eWatson) Date: Sun, 31 Jan 2010 10:10:53 -0800 Subject: What's the Scoop on \\ for Paths? (Win) Message-ID: I'm sure that \\ is used in some way for paths in Win Python, but I have not found anything after quite a search. I even have a six page pdf on a file tutorial. Nothing. Two books. Nothing. When I try to open a file along do I need, for example, "Events\\record\\year\\today"? Are paths like, ".\\Events" allowed, or am I mixing up my Linux memory on this? From tinnews at isbd.co.uk Sun Jan 31 13:15:32 2010 From: tinnews at isbd.co.uk (tinnews at isbd.co.uk) Date: Sun, 31 Jan 2010 18:15:32 +0000 Subject: Can't get sys.stdin.readlines() to work Message-ID: <4vtg37-n89.ln1@chris.isbd.net> I'm trying to read some data from standard input, what I'm actually trying to do is process some date pasted in using the mouse cut and paste on a Linux box (xubuntu 9.10) in a terminal window. First attempts failed so I'm now trying the trivial:- import sys data = sys.stdin.readlines() print "Counted", len(data), "lines." When I run this and try to paste something into the terminal window I get the following errors:- /home/chris/bin/m2r.py: line 2: syntax error near unexpected token `(' /home/chris/bin/m2r.py: line 2: `data = sys.stdin.readlines()' It does exactly the same if I try:- cat | m2r.py and then paste something into the window. So - what on earth am I doing wrong? -- Chris Green From alfps at start.no Sun Jan 31 13:23:35 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 31 Jan 2010 19:23:35 +0100 Subject: What's the Scoop on \\ for Paths? (Win) In-Reply-To: References: Message-ID: * W. eWatson: > I'm sure that \\ is used in some way for paths in Win Python, but I have > not found anything after quite a search. I even have a six page pdf on a > file tutorial. Nothing. Two books. Nothing. When I try to open a file > along do I need, for example, "Events\\record\\year\\today"? Are paths > like, ".\\Events" allowed, or am I mixing up my Linux memory on this? The Python issue with \\ is that in a literal string \\ denotes a single \ character, like >>> print( "back\\slash" ) back\slash >>> _ This is just like in other languages with syntax inherited from C. Look up "escape sequences". It has nothing to do with files and paths per se, but means that you cannot write e.g. "c:\windows\system32", but must write something like "c:\\windows\\system32" (try to print that string), or, since Windows handles forward slashes as well, you can write "c:/windows/system32" :-). The Window issue with \\ is that \\ as a path prefix denotes an UNC (Universal Naming Convention) path. Usually that would be a LAN or WAN network path, but it can also denote a printer or a pipe or a mailslot or just about anything. Using UNC paths opens the door to creating files and directories that other programs won't be able to handle, so Just Say No(TM), if you can. Cheers & hth., - Alf From steve at holdenweb.com Sun Jan 31 13:26:16 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 31 Jan 2010 13:26:16 -0500 Subject: What's the Scoop on \\ for Paths? (Win) In-Reply-To: References: Message-ID: <4B65CB48.1040409@holdenweb.com> W. eWatson wrote: > I'm sure that \\ is used in some way for paths in Win Python, but I have > not found anything after quite a search. I even have a six page pdf on a > file tutorial. Nothing. Two books. Nothing. When I try to open a file > along do I need, for example, "Events\\record\\year\\today"? Are paths > like, ".\\Events" allowed, or am I mixing up my Linux memory on this? You need to read up on string literals is all. "\\" is simply the literal representation of a string containing a single backslash. This comes about because string literals are allowed to contain special "escape sequences" which are introduced by a backslash; since this gives the backslash a special meaning in string literals we also have to use an escape sequence ("\\") to represent a backslash. In practice you will find that a) Many Windows APIs (but not the command line) are just as happy with a forward slash as a backslash to separate file path components; and b) The best practice is to build filenames using the routines provided in the os.path module, which guarantees to give results correct for the current platform. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Sun Jan 31 13:26:16 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 31 Jan 2010 13:26:16 -0500 Subject: What's the Scoop on \\ for Paths? (Win) In-Reply-To: References: Message-ID: <4B65CB48.1040409@holdenweb.com> W. eWatson wrote: > I'm sure that \\ is used in some way for paths in Win Python, but I have > not found anything after quite a search. I even have a six page pdf on a > file tutorial. Nothing. Two books. Nothing. When I try to open a file > along do I need, for example, "Events\\record\\year\\today"? Are paths > like, ".\\Events" allowed, or am I mixing up my Linux memory on this? You need to read up on string literals is all. "\\" is simply the literal representation of a string containing a single backslash. This comes about because string literals are allowed to contain special "escape sequences" which are introduced by a backslash; since this gives the backslash a special meaning in string literals we also have to use an escape sequence ("\\") to represent a backslash. In practice you will find that a) Many Windows APIs (but not the command line) are just as happy with a forward slash as a backslash to separate file path components; and b) The best practice is to build filenames using the routines provided in the os.path module, which guarantees to give results correct for the current platform. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From chardster at gmail.com Sun Jan 31 13:26:26 2010 From: chardster at gmail.com (Richard Thomas) Date: Sun, 31 Jan 2010 10:26:26 -0800 (PST) Subject: Can't get sys.stdin.readlines() to work References: <4vtg37-n89.ln1@chris.isbd.net> Message-ID: <3a3c2ba0-7221-4402-9ed6-0b74e101f95b@m16g2000yqc.googlegroups.com> On Jan 31, 6:15?pm, tinn... at isbd.co.uk wrote: > I'm trying to read some data from standard input, what I'm actually > trying to do is process some date pasted in using the mouse cut and > paste on a Linux box (xubuntu 9.10) in a terminal window. > > First attempts failed so I'm now trying the trivial:- > > ? ? import sys > ? ? data = sys.stdin.readlines() > ? ? print "Counted", len(data), "lines." > > When I run this and try to paste something into the terminal window I > get the following errors:- > > ? ? /home/chris/bin/m2r.py: line 2: syntax error near unexpected token `(' > ? ? /home/chris/bin/m2r.py: line 2: `data = sys.stdin.readlines()' > > It does exactly the same if I try:- > > ? ? cat | m2r.py > > and then paste something into the window. > > So - what on earth am I doing wrong? > > -- > Chris Green You haven't put a shebang line at the top. #!/usr/bin/env python import sys data = sys.stdin.readlines() ... From steve at holdenweb.com Sun Jan 31 13:36:55 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 31 Jan 2010 13:36:55 -0500 Subject: Can't get sys.stdin.readlines() to work In-Reply-To: <4vtg37-n89.ln1@chris.isbd.net> References: <4vtg37-n89.ln1@chris.isbd.net> Message-ID: <4B65CDC7.5080305@holdenweb.com> tinnews at isbd.co.uk wrote: > I'm trying to read some data from standard input, what I'm actually > trying to do is process some date pasted in using the mouse cut and > paste on a Linux box (xubuntu 9.10) in a terminal window. > > First attempts failed so I'm now trying the trivial:- > > import sys > data = sys.stdin.readlines() > print "Counted", len(data), "lines." > > > When I run this and try to paste something into the terminal window I > get the following errors:- > > /home/chris/bin/m2r.py: line 2: syntax error near unexpected token `(' > /home/chris/bin/m2r.py: line 2: `data = sys.stdin.readlines()' > > > It does exactly the same if I try:- > > cat | m2r.py > > and then paste something into the window. > > > So - what on earth am I doing wrong? > > Trying to run Python scripts with /bin/sh, probably. First of all, does you PATH include the current directory (.)? It looks like it does, because your computer seems to be trying to execute the m2r.py script. However, because that script does not start with a "shebang" line - the one you need is probably either #!/usr/bin/python or #!/usr/bin/env python the shell is trying to run the script using the wrong interpreter. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From steve at holdenweb.com Sun Jan 31 13:36:55 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 31 Jan 2010 13:36:55 -0500 Subject: Can't get sys.stdin.readlines() to work In-Reply-To: <4vtg37-n89.ln1@chris.isbd.net> References: <4vtg37-n89.ln1@chris.isbd.net> Message-ID: <4B65CDC7.5080305@holdenweb.com> tinnews at isbd.co.uk wrote: > I'm trying to read some data from standard input, what I'm actually > trying to do is process some date pasted in using the mouse cut and > paste on a Linux box (xubuntu 9.10) in a terminal window. > > First attempts failed so I'm now trying the trivial:- > > import sys > data = sys.stdin.readlines() > print "Counted", len(data), "lines." > > > When I run this and try to paste something into the terminal window I > get the following errors:- > > /home/chris/bin/m2r.py: line 2: syntax error near unexpected token `(' > /home/chris/bin/m2r.py: line 2: `data = sys.stdin.readlines()' > > > It does exactly the same if I try:- > > cat | m2r.py > > and then paste something into the window. > > > So - what on earth am I doing wrong? > > Trying to run Python scripts with /bin/sh, probably. First of all, does you PATH include the current directory (.)? It looks like it does, because your computer seems to be trying to execute the m2r.py script. However, because that script does not start with a "shebang" line - the one you need is probably either #!/usr/bin/python or #!/usr/bin/env python the shell is trying to run the script using the wrong interpreter. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From wolftracks at invalid.com Sun Jan 31 13:42:25 2010 From: wolftracks at invalid.com (W. eWatson) Date: Sun, 31 Jan 2010 10:42:25 -0800 Subject: What's the Scoop on \\ for Paths? (Win) In-Reply-To: References: Message-ID: Alf P. Steinbach wrote: > * W. eWatson: >> I'm sure that \\ is used in some way for paths in Win Python, but I >> have not found anything after quite a search. I even have a six page >> pdf on a file tutorial. Nothing. Two books. Nothing. When I try to >> open a file along do I need, for example, >> "Events\\record\\year\\today"? Are paths like, ".\\Events" allowed, or >> am I mixing up my Linux memory on this? > > The Python issue with \\ is that in a literal string \\ denotes a single > \ character, like > > >>> print( "back\\slash" ) > back\slash > >>> _ > > This is just like in other languages with syntax inherited from C. Look > up "escape sequences". It has nothing to do with files and paths per se, > but means that you cannot write e.g. "c:\windows\system32", but must > write something like "c:\\windows\\system32" (try to print that string), > or, since Windows handles forward slashes as well, you can write > "c:/windows/system32" :-). > > The Window issue with \\ is that \\ as a path prefix denotes an UNC > (Universal Naming Convention) path. Usually that would be a LAN or WAN > network path, but it can also denote a printer or a pipe or a mailslot > or just about anything. Using UNC paths opens the door to creating files > and directories that other programs won't be able to handle, so Just Say > No(TM), if you can. > > > Cheers & hth., > > - Alf Ah, yes. Thanks for the memory jog. From invalid at invalid.invalid Sun Jan 31 13:50:56 2010 From: invalid at invalid.invalid (Grant Edwards) Date: Sun, 31 Jan 2010 18:50:56 +0000 (UTC) Subject: Can't get sys.stdin.readlines() to work References: <4vtg37-n89.ln1@chris.isbd.net> Message-ID: On 2010-01-31, Steve Holden wrote: > tinnews at isbd.co.uk wrote: >> I'm trying to read some data from standard input, what I'm actually >> trying to do is process some date pasted in using the mouse cut and >> paste on a Linux box (xubuntu 9.10) in a terminal window. >> >> First attempts failed so I'm now trying the trivial:- >> >> import sys >> data = sys.stdin.readlines() >> print "Counted", len(data), "lines." >> >> >> When I run this and try to paste something into the terminal window I >> get the following errors:- >> >> /home/chris/bin/m2r.py: line 2: syntax error near unexpected token `(' >> /home/chris/bin/m2r.py: line 2: `data = sys.stdin.readlines()' >> >> >> It does exactly the same if I try:- >> >> cat | m2r.py >> >> and then paste something into the window. >> >> >> So - what on earth am I doing wrong? >> >> > Trying to run Python scripts with /bin/sh, probably. Yup. I can't count the number of times I've run ImageMagick's "import" program unintentionally... -- Grant Edwards grante Yow! The SAME WAVE keeps at coming in and COLLAPSING visi.com like a rayon MUU-MUU ... From no.email at please.post Sun Jan 31 14:35:51 2010 From: no.email at please.post (kj) Date: Sun, 31 Jan 2010 19:35:51 +0000 (UTC) Subject: How to pass Chinese characters as command-line arguments? References: <7slr5iFe66U1@mid.uni-berlin.de> Message-ID: In <7slr5iFe66U1 at mid.uni-berlin.de> "Diez B. Roggisch" writes: >Am 31.01.10 16:52, schrieb kj: >> I want to pass Chinese characters as command-line arguments to a >> Python script. My terminal has no problem displaying these >> characters, and passing them to the script, but I can't get Python >> to understand them properly. >> >> E.g. if I pass one such character to the simple script >> >> import sys >> print sys.argv[1] >> print type(sys.argv[1]) >> >> the first line of the output looks fine (identical to the input), >> but the second line says "". If I add the line >> >> arg = unicode(sys.argv[1]) >> >> I get the error >> >> Traceback (most recent call last): >> File "kgrep.py", line 4, in >> arg = unicode(sys.argv[1]) >> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe8 in position 0: ordinal not in range(128) >> >> What must I do to get Python to recognize command-line arguments >> as utf-8 Unicode? >The last sentence reveals your problem: utf-8 is *not* unicode. It's an >encoding of unicode, which is a crucial difference. > From the outside you get byte-streams, and if these happen to be >encoded in utf-8, you can simply decode them: >arg = unicode(sys.argv[1], "utf-8") Thanks! kynn From python.list at tim.thechases.com Sun Jan 31 14:41:55 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Sun, 31 Jan 2010 13:41:55 -0600 Subject: What's the Scoop on \\ for Paths? (Win) In-Reply-To: References: Message-ID: <4B65DD03.50706@tim.thechases.com> Alf P. Steinbach wrote: > that you cannot write e.g. "c:\windows\system32", but must > write something like "c:\\windows\\system32" (try to print > that string), or, since Windows handles forward slashes as > well, you can write "c:/windows/system32" :-). Forward slashes work for some relative paths for some commands but not for others like absolute non-drive-specified paths: Microsoft Windows XP [Version 5.1.2600] C:\>md abc C:\>md abc\123 C:\>md abc\234 C:\>cd abc C:\abc>tree /f /a Folder PATH listing Volume serial number is 940C-3F80 C:. +---123 | \---234 \---234 C:\abc>cd 123 C:\abc\123>cd ../234 C:\abc\234>type ../123/234/hello.txt The syntax of the command is incorrect. C:\abc\234>cd ../123 C:\abc\123>cd /abc/123 The system cannot find the path specified. C:\abc>x: X:\>type c:/abc/123/234/hello.txt The syntax of the command is incorrect. ##### The previous absolute-path fails in cmd.exe for a variety of apps because the "/" is treated as a parameter/switch to the various programs. Fortunately, the Python path-handling sub-system is smart enough to do the right thing, even when Win32's internal handling is too dumb to behave: C:\abc\123>echo hello > 234/hello.txt C:\abc\123>cd .. C:\abc>tree /f /a Folder PATH listing Volume serial number is 940C-3F80 C:. +---123 | \---234 | hello.txt | \---234 C:\abc>python Python 2.4.3 (#69, Mar 29 2006, 17:35:34) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> file('/abc/123/234/hello.txt').read() 'hello \n' So as long as you stick within Python's insulation, forward slashes are a nice solution. But if you have to interact with external programs, use the \\ notation or raw strings: pth = r"c:\windows\system32" [mutters under breath about Win32 headaches] -tkc From wolftracks at invalid.com Sun Jan 31 14:43:15 2010 From: wolftracks at invalid.com (W. eWatson) Date: Sun, 31 Jan 2010 11:43:15 -0800 Subject: What's the Scoop on \\ for Paths? (Win) In-Reply-To: References: Message-ID: Steve Holden wrote: > You need to read up on string literals is all. "\\" is simply the > literal representation of a string containing a single backslash. This > comes about because string literals are allowed to contain special > "escape sequences" which are introduced by a backslash; since this gives > the backslash a special meaning in string literals we also have to use > an escape sequence ("\\") to represent a backslash. > > In practice you will find that > > a) Many Windows APIs (but not the command line) are just as happy with a > forward slash as a backslash to separate file path components; and > > b) The best practice is to build filenames using the routines provided > in the os.path module, which guarantees to give results correct for the > current platform. > > regards > Steve Basic sys functions brought out the \ separator for paths. What am I missing here? Looks OK to me. >>> abc=r'xyz\\' >>> abc 'xyz\\\\' >>> print abc xyz\\ >>> abc.replace(r'\',r'z') SyntaxError: invalid syntax >>> abc 'xyz\\\\' From __peter__ at web.de Sun Jan 31 14:44:22 2010 From: __peter__ at web.de (Peter Otten) Date: Sun, 31 Jan 2010 20:44:22 +0100 Subject: iglob performance no better than glob References: <5ebd4965-32fc-4a03-94dd-c61ccdc9d630@f5g2000vbm.googlegroups.com> Message-ID: Kyp wrote: > I have a dir with a large # of files that I need to perform operations > on, but only needing to access a subset of the files, i.e. the first > 100 files. > > Using glob is very slow, so I ran across iglob, which returns an > iterator, which seemed just like what I wanted. I could iterate over > the files that I wanted, not having to read the entire dir. > > So the iglob was faster, but accessing the first file took about the > same time as glob.glob. > > Here's some code to compare glob vs. iglob performance, it outputs > the time before/after a glob.iglob('*.*') files.next() sequence and a > glob.glob('*.*') sequence. > > #!/usr/bin/env python > > import glob,time > print '\nTest of glob.iglob' > print 'before iglob:', time.asctime() > files = glob.iglob('*.*') > print 'after iglob:',time.asctime() > print files.next() > print 'after files.next():', time.asctime() > > print '\nTest of glob.glob' > print 'before glob:', time.asctime() > files = glob.glob('*.*') > print 'after glob:',time.asctime() > > > Here are the results: > > Test of glob.iglob > before iglob: Sun Jan 31 11:09:08 2010 > after iglob: Sun Jan 31 11:09:08 2010 > foo.bar > after files.next(): Sun Jan 31 11:09:59 2010 > > Test of glob.glob > before glob: Sun Jan 31 11:09:59 2010 > after glob: Sun Jan 31 11:10:51 2010 > > The results are about the same for the 2 approaches, both took about > 51 seconds. Am I doing something wrong with iglob? No, but iglob() being lazy is pointless in your case because it uses os.listdir() and fnmatch.filter() underneath which both read the whole directory before returning anything. > Is there a way to get the first X # of files from a dir with lots of > files, that does not take a long time to run? Here's my attempt. It turned out to be more work than expected, so I cut a few corners. It's Linux-only "works on my machine" code, but may give you some hints on how to proceed. from ctypes import * import fnmatch import glob import os import re from itertools import ifilter, imap class dirent(Structure): "works on my machine ;)" _fields_ = [ ("d_ino", c_long), ("d_off", c_long), ("d_reclen", c_ushort), ("d_type", c_ubyte), ("d_name", c_char*256)] direntp = POINTER(dirent) LIBC = "libc.so.6" cdll.LoadLibrary(LIBC) libc = CDLL(LIBC) libc.readdir.restype = direntp def diriter(dir): "lazy partial replacement for os.listdir()" # errors? what errors? dirp = libc.opendir(dir) if not dirp: return try: while True: ep = libc.readdir(dirp) if not ep: break yield ep.contents.d_name finally: libc.closedir(dirp) def filter(names, pattern): "lazy partial replacement for fnmatch.filter()" import posixpath pattern = os.path.normcase(pattern) r = fnmatch.translate(pattern) r = re.compile(r) if os.path is not posixpath: names = imap(os.path.normcase, names) return ifilter(r.match, names) def globiter(path): "lazy partial replacement for glob.glob()" dir, filename = os.path.split(path) if glob.has_magic(dir): raise ValueError("wildcards in directory not supported") return filter(diriter(dir), filename) if __name__ == "__main__": import sys [pattern] = sys.argv[1:] for name in globiter(pattern): print name Peter From steve at holdenweb.com Sun Jan 31 14:54:45 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 31 Jan 2010 14:54:45 -0500 Subject: What's the Scoop on \\ for Paths? (Win) In-Reply-To: References: Message-ID: W. eWatson wrote: > Steve Holden wrote: > >> You need to read up on string literals is all. "\\" is simply the >> literal representation of a string containing a single backslash. This >> comes about because string literals are allowed to contain special >> "escape sequences" which are introduced by a backslash; since this gives >> the backslash a special meaning in string literals we also have to use >> an escape sequence ("\\") to represent a backslash. >> >> In practice you will find that >> >> a) Many Windows APIs (but not the command line) are just as happy with a >> forward slash as a backslash to separate file path components; and >> >> b) The best practice is to build filenames using the routines provided >> in the os.path module, which guarantees to give results correct for the >> current platform. >> >> regards >> Steve > Basic sys functions brought out the \ separator for paths. > > What am I missing here? Looks OK to me. > > >>>> abc=r'xyz\\' >>>> abc > 'xyz\\\\' >>>> print abc > xyz\\ >>>> abc.replace(r'\',r'z') > > SyntaxError: invalid syntax >>>> abc > 'xyz\\\\' The SyntaxError is thrown because although backslashes lose their special "escape sequence" meaning, for some reason I have never really understood they retain the requirement that they be followed by another character. abc.replace("\\", 'z') would have worked fine. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ From no-spam at non-existing.invalid Sun Jan 31 14:57:31 2010 From: no-spam at non-existing.invalid (Robert) Date: Sun, 31 Jan 2010 20:57:31 +0100 Subject: HTML Parser which allows low-keyed local changes? Message-ID: I tried lxml, but after walking and making changes in the element tree, I'm forced to do a full serialization of the whole document (etree.tostring(tree)) - which destroys the "human edited" format of the original HTML code. makes it rather unreadable. is there an existing HTML parser which supports tracking/writing back particular changes in a cautious way by just making local changes? or a least tracks the tag start/end positions in the file? Robert From python.list at tim.thechases.com Sun Jan 31 14:59:43 2010 From: python.list at tim.thechases.com (Tim Chase) Date: Sun, 31 Jan 2010 13:59:43 -0600 Subject: What's the Scoop on \\ for Paths? (Win) In-Reply-To: References: Message-ID: <4B65E12F.3070606@tim.thechases.com> W. eWatson wrote: > What am I missing here? Looks OK to me. > >>> abc.replace(r'\',r'z') > > SyntaxError: invalid syntax A raw string can't end in a single backslash (something that occasionally annoys me, but I've learned to deal with it). >>> s=r'\' File "", line 1 s=r'\' ^ SyntaxError: EOL while scanning single-quoted string My guess is that the parser sees the first call as abc.replace( r"r\'r" <-- how your single-quoted is interpreted z' <-- the leftovers that cause the syntax error ) -tkc From python at mrabarnett.plus.com Sun Jan 31 15:02:00 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 31 Jan 2010 20:02:00 +0000 Subject: What's the Scoop on \\ for Paths? (Win) In-Reply-To: References: Message-ID: <4B65E1B8.8070905@mrabarnett.plus.com> W. eWatson wrote: > Steve Holden wrote: > >> You need to read up on string literals is all. "\\" is simply the >> literal representation of a string containing a single backslash. This >> comes about because string literals are allowed to contain special >> "escape sequences" which are introduced by a backslash; since this gives >> the backslash a special meaning in string literals we also have to use >> an escape sequence ("\\") to represent a backslash. >> >> In practice you will find that >> >> a) Many Windows APIs (but not the command line) are just as happy with a >> forward slash as a backslash to separate file path components; and >> >> b) The best practice is to build filenames using the routines provided >> in the os.path module, which guarantees to give results correct for the >> current platform. >> >> regards >> Steve > Basic sys functions brought out the \ separator for paths. > > What am I missing here? Looks OK to me. > > > >>> abc=r'xyz\\' > >>> abc > 'xyz\\\\' > >>> print abc > xyz\\ > >>> abc.replace(r'\',r'z') > > SyntaxError: invalid syntax > >>> abc > 'xyz\\\\' It's not possible to end a raw string with a single backslash, and if you end it with a double backslash then you'll get a double backslash. Very annoying. From half.italian at gmail.com Sun Jan 31 15:04:36 2010 From: half.italian at gmail.com (Sean DiZazzo) Date: Sun, 31 Jan 2010 12:04:36 -0800 (PST) Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <87iqaj9n8z.fsf@castleamber.com> <877hqz2j1o.fsf@castleamber.com> <4B6558AF.1090009@v.loewis.de> Message-ID: > Here is a recent list of magic numbers: > > ? ? ? ?Python 2.6a0: 62151 (peephole optimizations and STORE_MAP opcode) > ? ? ? ?Python 2.6a1: 62161 (WITH_CLEANUP optimization) > ? ? ? ?Python 2.7a0: 62171 (optimize list comprehensions/change LIST_APPEND) > ? ? ? ?Python 2.7a0: 62181 (optimize conditional branches: > ? ? ? ? ? ? ? ? ? ? ? ? ? ? introduce POP_JUMP_IF_FALSE and POP_JUMP_IF_TRUE) > ? ? ? ?Python 2.7a0 ?62191 (introduce SETUP_WITH) > ? ? ? ?Python 2.7a0 ?62201 (introduce BUILD_SET) > ? ? ? ?Python 2.7a0 ?62211 (introduce MAP_ADD and SET_ADD) > > #define MAGIC (62211 | ((long)'\r'<<16) | ((long)'\n'<<24)) > > Regards, > Martin Does "magic" really need to be used? Why not just use the revision number? From marcmagransdeabril at gmail.com Sun Jan 31 15:08:19 2010 From: marcmagransdeabril at gmail.com (marc magrans de abril) Date: Sun, 31 Jan 2010 12:08:19 -0800 (PST) Subject: Slow down while creating a big list and iterating over it References: <2fdd4e1b-db58-45d7-9b7d-73dcdca6384a@c29g2000yqd.googlegroups.com> <29a69a7c-a8a2-4c53-9521-760e13665cd3@f12g2000yqn.googlegroups.com> Message-ID: <4b484bc6-f903-40a0-a8dc-9a00c8fc002a@k5g2000yqf.googlegroups.com> Hi! ...I have found a good enough solution, although it only works if the number of patterns (clusters) is not very big: def classify(f): THERESHOLD=0.1 patterns={} for l in enumerate(f): found = False for p,c in patterns.items(): if dist(l,p) < THERESHOLD: found=True patterns[p] = c +1 if not found: patterns[l] = 1 return patterns This algorithm is O(n*np*m^2). Where n is the number of logs, np the number of patterns, and m is the log length (i.e. m^2 is the distance cost). So it's way better O(n^2*m^2) and I can run it for some hours to get back the results. I wonder if there is a single threaded/process clustering algorithm than runs in O(n)? Cheers, marc From nobody at nowhere.com Sun Jan 31 15:14:47 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 31 Jan 2010 20:14:47 +0000 Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> Message-ID: On Sat, 30 Jan 2010 16:58:34 +0000, tanix wrote: >>I'm not familiar with Ruby, but most languages are cleaner than Python >>once you get beyond the "10-minute introduction" stage. > > I'd have to agree. The only ones that beat Python in that department are > Javascript and PHP. Plus CSS and HTML if you can call those languages. > > The very idea of using a number of blanks to identify your block level is > as insane as it gets. I don't have a problem with layout (indentation level as a syntax element). > First of all, combinations of blanks and tabs, > depending on how your ide is setup to expand tabs, may get you bugs, you'd > never imagine in your wild dreams. If you IDE places tab stops other than every 8 columns, it's broken. Configurable tab stops in a text editor is one of those "features" that differentiates a "coder" from a software engineer. A coder implements it because it's easy to implement, without giving a moment's thought to the wider context (such as: how to communicate the non-standard tab stops to any other program which needs to read the file). Even so, it's quite simple to implement layout in a way which doesn't care about tab widths: 1. If the current line begins with exactly the same sequence of whitespace characters as the last non-blank line, it's part of the same block. 2. If the sequence of whitespace characters at the beginning of the current line is an extension of that for the last non-blank line (i.e. it starts with the previous sequence, then adds some more whitespace), then it's the first line of a inner block. 3. If the current line begins with a left substring of the sequence for the last non-blank line, then it belongs to an outer block (whichever one has that sequence). 4. If none of the above are true, it's a syntax error. > Braces is the most reliable way to identify blocks. No they aren't. Indentation is much easier for humans to process than scanning the file for braces (and not missing any). If you have any indentation in the code, humans will interpret it as indicating the nesting, even if the compiler doesn't. E.g.: if(x) { if(y) foo(); else bar(); } See the problem? The compiler will match the else with if(y), but a human will tend to match it with if(x). Layout ensures that a human sees what the compiler "sees". Given that any sane program uses indentation to reflect the program's structure, braces (or "begin", or "end" (or endif/endwhile/etc)) are redundant. From mrholtsr at sbcglobal.net Sun Jan 31 15:25:57 2010 From: mrholtsr at sbcglobal.net (Ray Holt) Date: Sun, 31 Jan 2010 15:25:57 -0500 Subject: Why this error message Message-ID: <6B218CF5831243C08A438194D7F4E43E@ray> Why am I getting the error that test is not defined. Thanks, Ray class SpecialFile: def __init__(self, fileName): self.__file = open(fileName, 'W') self.__file.write('***** Start Special File *****\n\n') def write(self, str): self.__file.write(str) def writelines(self, strList): self.__file.writelines(strList) def __del__(self): # Destructor method, __del__. print "entered __del__" self.close() def close(self): # Clean up method if self.__file: self.__file.write('\\n***** End Special File *****') self.__file.close() self.__file = None def test(): f = SpecialFile('testfile') f.write('111111\n') f.close() test() Traceback (most recent call last): File "C:\Python26\Classes_and_OOP_Programing.py", line 145, in test() NameError: name 'test' is not defined -------------- next part -------------- An HTML attachment was scrubbed... URL: From alfps at start.no Sun Jan 31 15:36:57 2010 From: alfps at start.no (Alf P. Steinbach) Date: Sun, 31 Jan 2010 21:36:57 +0100 Subject: What's the Scoop on \\ for Paths? (Win) In-Reply-To: References: Message-ID: * Tim Chase: > Alf P. Steinbach wrote: >> that you cannot write e.g. "c:\windows\system32", but must >> write something like "c:\\windows\\system32" (try to print >> that string), or, since Windows handles forward slashes as >> well, you can write "c:/windows/system32" :-). > > Forward slashes work for some relative paths for some commands but not > for others like absolute non-drive-specified paths: > > Microsoft Windows XP [Version 5.1.2600] > C:\>md abc > C:\>md abc\123 > C:\>md abc\234 > C:\>cd abc > C:\abc>tree /f /a > Folder PATH listing > Volume serial number is 940C-3F80 > C:. > +---123 > | \---234 > \---234 > > C:\abc>cd 123 > C:\abc\123>cd ../234 > C:\abc\234>type ../123/234/hello.txt > The syntax of the command is incorrect. > C:\abc\234>cd ../123 > C:\abc\123>cd /abc/123 > The system cannot find the path specified. > C:\abc>x: > X:\>type c:/abc/123/234/hello.txt > The syntax of the command is incorrect. > > ##### > The previous absolute-path fails in cmd.exe for a variety of apps > because the "/" is treated as a parameter/switch to the various > programs. Yes, that's a valid concern when invoking external programs. > Fortunately, the Python path-handling sub-system is smart > enough to do the right thing, even when Win32's internal handling is too > dumb to behave: > > C:\abc\123>echo hello > 234/hello.txt > C:\abc\123>cd .. > C:\abc>tree /f /a > Folder PATH listing > Volume serial number is 940C-3F80 > C:. > +---123 > | \---234 > | hello.txt > | > \---234 > > C:\abc>python > Python 2.4.3 (#69, Mar 29 2006, 17:35:34) [MSC v.1310 32 bit (Intel)] on > win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> file('/abc/123/234/hello.txt').read() > 'hello \n' Oops, this is a dangerous misconception, because it can lead to Wrong Solutions to the perceived problems. It's not the "Win32's internal handling" that you're up against above, but the generally unpredictable quirks of syntax and semantics of commands in the Windows command interpreter: C:\test> echo bah >foo.txt C:\test> type ../test/foo.txt The syntax of the command is incorrect. C:\test> type "../test/foo.txt" bah C:\test> _ Generally the details of that command interpreter are undocumented. > So as long as you stick within Python's insulation, forward slashes are > a nice solution. No, it's not Python's insulation that makes things work (although perhaps it redundantly tries to help), it's simply the Windows API, which generally accepts forward or backward slashes. It's the same in other programming languages. In particular, it's not a good idea to carry the idea of using backslashes over to #include directives in C or C++, thinking that there's no Python insulation there (at least one compiler accepts and Windows compilers used to accept it at one time, but it's invalid, and non-portable, whereas forward slashes work). > But if you have to interact with external programs, > use the \\ notation or raw strings: > > pth = r"c:\windows\system32" > > [mutters under breath about Win32 headaches] He he. Yes. But anyways, the thing to remember is that when paths are passed to /programs/ in Windows, they generally need to be /quoted/. That's particularly important for paths containing spaces. For internal file handling in a program, including using libraries, forward slashes work fine -- unless a library is particularly ill-behaved and requires backslashes -- and provide some (but not complete) measure of portability. For example, >>> f = open( "c:/foo.txt", "w" ) >>> f.write( "works\n" ) 6 >>> f.close() >>> f = open( "c:/foo.txt", "r" ) >>> f.readlines() ['works\n'] >>> f.close() >>> _ In retrospect, I should have been more clear about that distinction between invoking programs and invoking routines such as Python 'open', yes. And as Steve Holden remarked else-thread, best practice for Python is to build paths using the os.path functionality. E.g., >>> import os.path >>> os.path.normpath( "c:/foo.txt" ) 'c:\\foo.txt' >>> _ Cheers & hth., - Alf From nobody at nowhere.com Sun Jan 31 15:43:03 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 31 Jan 2010 20:43:03 +0000 Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> <7536f9c0-a36a-479d-ba95-9405b13793fd@e25g2000yqh.googlegroups.com> Message-ID: On Sun, 31 Jan 2010 03:01:51 -0800, rantingrick wrote: >> That's also true for most functional languages, e.g. Haskell and ML, as >> well as e.g. Tcl and most shells. Why require "f(x)" or "(f x)" if "f x" >> will suffice? > > yuck! wrapping the arg list with parenthesis (python way) makes the most > sense. Its to easy to misread somthing like this > > onetwothree four five six > > onetwothree(four, five, six) #ahhh... plain english. Note: Functional languages allow: f (a,b,c) but that means something else, i.e. calling a function with a tuple as its argument (in functional languages, a function always has exactly one argument). The syntax: f a b c is equivalent to the Python expression: f(a)(b)(c) i.e. each argument is applied in turn, with all applications except the last yielding a function. Defining f as: f a b c = is shorthand for: f = \a -> (\b -> (\c -> )) or, in Python syntax: f = lambda a: (lambda b: (lambda c: )) This style (known as Currying, or a Curried function, after the mathematician Haskell Curry) is common in functional languages, as it allows you to partially apply functions, e.g.: map (f a b) someList whereas an uncurried function would require a lambda expression: map (\c -> f (a,b,c)) someList IOW, while the uncurried form is allowed, it has no advantages and one disadvantage, so it's seldom used (and where it is used, it's normally a case of a function whose sole argument is a naturally-occurring tuple, rather than one which is constructed simply to satisfy the mechanics of a function call). Partial application is common enough that Haskell supports it for infix operators as well, e.g. map (/ 2) someList -- (/ 2) => \x -> x / 2, i.e. halve map (1 /) someList -- (1 /) => \x -> 1 / x, i.e. reciprocal If it was common-place to use Curried functions and partial application in Python, you'd probably prefer "f a b c" to "f(a)(b)(c)" as well. From daniel at stutzbachenterprises.com Sun Jan 31 15:45:31 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Sun, 31 Jan 2010 14:45:31 -0600 Subject: ANN: blist 1.1.1 - now with sortedlist, sortedset, and sorteddict Message-ID: blist 1.1.1 is now available: http://pypi.python.org/pypi/blist/ What is blist? -------------- The blist is a drop-in replacement for the Python list the provides better performance when modifying large lists. Python's built-in list is a dynamically-sized array; to insert or removal an item from the beginning or middle of the list, it has to move most of the list in memory, i.e., O(n) operations. The blist uses a flexible, hybrid array/tree structure and only needs to move a small portion of items in memory, specifically using O(log n) operations. For small lists, the blist and the built-in list have virtually identical performance. What's new? ----------- blist 1.1 introduces other data structures based on the blist: - sortedlist - sortedset - weaksortedlist - weaksorteset - sorteddict - btuple These additional data structures are only available in Python 2.6 or higher, as they make use of Abstract Base Classes. The sortedlist is a list that's always sorted. It's iterable and indexable like a Python list, but to modify a sortedlist the same methods you would use on a Python set (add, discard, or remove). >>> from blist import sortedlist >>> my_list = sortedlist([3,7,2,1]) >>> my_list sortedlist([1, 2, 3, 7]) >>> my_list.add(5) >>> my_list[3] 5 >>> The sortedlist constructor takes an optional "key" argument, which may be used to change the sort order just like the sorted() function. >>> from blist import sortedlist >>> my_list = sortedlist([3,7,2,1], key=lambda i: -i) sortedlist([7, 3, 2, 1] >>> The sortedset is a set that's always sorted. It's iterable and indexable like a Python list, but modified like a set. Essentially, it's just like a sortedlist except that duplicates are ignored. >>> from blist import sortedset >>> my_set = sortedset([3,7,2,2]) sortedset([2, 3, 7] >>> The weaksortedlist and weaksortedset are weakref variations of the sortedlist and sortedset. The sorteddict works just like a regular dict, except the keys are always sorted. The sorteddict should not be confused with Python 2.7's OrderedDict type, which remembers the insertion order of the keys. >>> from blist import sorteddict >>> my_dict = sorteddict({1: 5, 6: 8, -5: 9}) >>> my_dict.keys() [-5, 1, 6] >>> The btuple is a drop-in replacement for the built-in tuple. Compared to the built-in tuple, the btuple offers the following advantages: - Constructing a btuple from a blist takes O(1) time. - Taking a slice of a btuple takes O(n) time, where n is the size of the original tuple. The size of the slice does not matter. >>> from blist import blist, btuple >>> x = blist([0]) # x is a blist with one element >>> x *= 2**29 # x is a blist with > 500 million elements >>> y = btuple(x) # y is a btuple with > 500 million elements Feedback -------- We're eager to hear about your experiences with the blist. You can email me at daniel at stutzbachenterprises.com. Alternately, bug reports and feature requests may be reported on our bug tracker at: http://github.com/DanielStutzbach/blist/issues -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrholtsr at sbcglobal.net Sun Jan 31 15:46:38 2010 From: mrholtsr at sbcglobal.net (Ray Holt) Date: Sun, 31 Jan 2010 15:46:38 -0500 Subject: Why this error message Message-ID: Guys, I apologize for the last email asking for help I am going to have to remember to check my indentation. Ray -------------- next part -------------- An HTML attachment was scrubbed... URL: From john at castleamber.com Sun Jan 31 15:47:08 2010 From: john at castleamber.com (John Bokma) Date: Sun, 31 Jan 2010 14:47:08 -0600 Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> Message-ID: <87y6jef1df.fsf@castleamber.com> Nobody writes: > Configurable tab stops in a text editor is one of those "features" that > differentiates a "coder" from a software engineer. A coder implements it > because it's easy to implement, without giving a moment's thought to the > wider context (such as: how to communicate the non-standard tab stops to > any other program which needs to read the file). [..] > if(x) { > if(y) > foo(); > else > bar(); > } > > See the problem? Nope, because a good editor will format this correctly. One written by software engineers ;-) > Given that any sane program uses indentation to reflect the program's > structure, braces (or "begin", or "end" (or endif/endwhile/etc)) are > redundant. An editor can correct the indenting of the braces example but can't with this one. if x: if y: foo() else: bar() While braces might be considered redundant they are not when for one reason or another formatting is lost or done incorrectly. FWIW: I have no problem with how Python doesn't use braces nor on how other languages do insist on braces or other structure markers. -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From ryan at rfk.id.au Sun Jan 31 15:53:00 2010 From: ryan at rfk.id.au (Ryan Kelly) Date: Mon, 01 Feb 2010 07:53:00 +1100 Subject: Why this error message In-Reply-To: <6B218CF5831243C08A438194D7F4E43E@ray> References: <6B218CF5831243C08A438194D7F4E43E@ray> Message-ID: <1264971180.2900.0.camel@durian> On Sun, 2010-01-31 at 15:25 -0500, Ray Holt wrote: > Why am I getting the error that test is not defined. Thanks, Ray > class SpecialFile: > def __init__(self, fileName): > self.__file = open(fileName, 'W') > self.__file.write('***** Start Special File *****\n\n') > def write(self, str): > self.__file.write(str) > def writelines(self, strList): > self.__file.writelines(strList) > > def __del__(self): # Destructor method, __del__. > print "entered __del__" > self.close() > > def close(self): # Clean up method > if self.__file: > self.__file.write('\\n***** End Special File *****') > self.__file.close() > self.__file = None > > def test(): > f = SpecialFile('testfile') > f.write('111111\n') > f.close() > test() > Traceback (most recent call last): > File "C:\Python26\Classes_and_OOP_Programing.py", line 145, in > > test() > NameError: name 'test' is not defined Based on the indentation, it looks like you've defined "test" as a method on the SpecialFile class. Try dedenting it to begin in the first column. Ryan -- Ryan Kelly http://www.rfk.id.au | This message is digitally signed. Please visit ryan at rfk.id.au | http://www.rfk.id.au/ramblings/gpg/ for details -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: From nobody at nowhere.com Sun Jan 31 15:57:44 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 31 Jan 2010 20:57:44 +0000 Subject: Utility to screenscrape sites using javascript ? References: <4c4298c0-b93d-4691-bceb-e982373f87e3@f17g2000prh.googlegroups.com> <3df16722-7726-4b05-b8c3-d1b6a9f7e98b@k6g2000prg.googlegroups.com> Message-ID: On Sat, 30 Jan 2010 11:28:47 -0800, KB wrote: >> > I have a service I subscribe to that uses javascript to stream news. > >> There's a Python interface to SpiderMonkey (Mozilla's JavaScript >> interpreter): >> >> http://pypi.python.org/pypi/python-spidermonkey > > Thanks! I don't see a documentation page, but how would one use this? > Would you download the HTML using urllib2/mechanize, then parse for the > .js script and use spider-monkey to "execute" the script and the output is > passed back to python? Something like that. The "homepage" link: http://github.com/davisp/python-spidermonkey has some examples further down. The first one starts with: >>> import spidermonkey >>> rt = spidermonkey.Runtime() >>> cx = rt.new_context() >>> cx.execute("var x = 3; x *= 4; x;") 12 It goes on to mention the .add_global(name, object) method, using it to name a Python object such that its variables and methods can be accessed from JS. For scraping web pages, I suspect that you'll at least need to create an object and name it "document", so that "document.location" etc works. How much you'll need to implement will depend upon what the web page uses, although you can probably use .__getattr__() to serve up dummy handlers for calls which can be ignored. Futher documentation is probably a case of "read the spidermonkey documentation, then read the python-spidermonkey source if it isn't clear how the wrapper relates to spidermonkey itself". From victorsubervi at gmail.com Sun Jan 31 16:15:34 2010 From: victorsubervi at gmail.com (Victor Subervi) Date: Sun, 31 Jan 2010 13:15:34 -0800 Subject: OT: Instant Messenger Clients Message-ID: <4dc0cfea1001311315u3494ad6bk792327d05447c96d@mail.gmail.com> Hi; I need to record my IM conversations. I'm using Gmal's IM client and I can't figure out how to do it, nor do I find any help googling it. Is it possible with Gmail? If so, how? If not, is there a good IM client that will allow me to do this? TIA, beno -- The Logos has come to bear http://logos.13gems.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From wolfgang.lipp at gmail.com Sun Jan 31 16:17:04 2010 From: wolfgang.lipp at gmail.com (_wolf) Date: Sun, 31 Jan 2010 13:17:04 -0800 (PST) Subject: whassup? builtins? python3000? Naah can't be right? Message-ID: <799ea994-d00f-485c-9164-ed4b66e750a5@k41g2000yqm.googlegroups.com> dear pythoneers, i would be very gladly accept any commentaries about what this sentence, gleaned from http://celabs.com/python-3.1/reference/executionmodel.html, is meant to mean, or why gods have decided this is the way to go. i anticipate this guy named Kay Schluehr will have a say on that, or maybe even the BDFL will care to pronounce ``__builtins__`` the correct way to his fallovers, followers, and fellownerds:: The built-in namespace associated with the execution of a code block is actually found by looking up the name __builtins__ in its global namespace; this should be a dictionary or a module (in the latter case the module?s dictionary is used). By default, when in the __main__ module, __builtins__ is the built-in module builtins; when in any other module, __builtins__ is an alias for the dictionary of the builtins module itself. __builtins__ can be set to a user-created dictionary to create a weak form of restricted execution. it used to be the case that there were at least two distinct terms, ?builtin? (in the singular) and ?builtins? (in the plural), some of which existed both in module and in dict form (?just guessing?). now there is only ?builtins?, so fortunately the ambivalence between singular and plural has gone?good riddance. but why does ``__builtins__`` change its meaning depending on whether this is the scope of the ?script? (i.e. the module whose name was present, when calling ``python foobar.py``) or whether this is the scope of a secondary module (imported or executed, directly or indirectly, by ``foobar.py``)? i cannot understand the reasoning behind this and find it highly confusing. rationale: why do i care??i want to be able to ?export names to the global namespace that were not marked private (by an underscore prefix) in a python module that i execute via ``exec( compile( get ( locator ), locator, 'exec' ), R )`` where ``R`` is supposed to going to hold the private names of said module?. it *is* a little arcane but the basic exercise is to by-pass python?s import system and get similr results... it is all about injecting names into the all-global and the module-global namespaces. still i get trapped by the above wordings in tzhe docs, and i have a weird case of a vanishing names, so maybe some people will care to share their thoughts. love & ~flow thank From burslem2001 at yahoo.com Sun Jan 31 16:27:46 2010 From: burslem2001 at yahoo.com (gazza) Date: Sun, 31 Jan 2010 13:27:46 -0800 (PST) Subject: gmtime Message-ID: <62938b5b-7eda-426c-b91d-3c7061840b2d@l26g2000yqd.googlegroups.com> Hi, I am trying to discover how to obtain the correct time of say CST/ America and EST/America in python? Any help on this would be appreciated. Thanks, Garyc From awilliam at opengroupware.us Sun Jan 31 16:29:07 2010 From: awilliam at opengroupware.us (Adam Tauno Williams) Date: Sun, 31 Jan 2010 16:29:07 -0500 Subject: OT: Instant Messenger Clients In-Reply-To: <4dc0cfea1001311315u3494ad6bk792327d05447c96d@mail.gmail.com> References: <4dc0cfea1001311315u3494ad6bk792327d05447c96d@mail.gmail.com> Message-ID: <1264973347.19333.0.camel@linux-m3mt> On Sun, 2010-01-31 at 13:15 -0800, Victor Subervi wrote: > Hi; > I need to record my IM conversations. I'm using Gmal's IM client and I > can't figure out how to do it, nor do I find any help googling it. Is > it possible with Gmail? If so, how? If not, is there a good IM client > that will allow me to do this? "OT" is an understatement. Why not ask somewhere appropriate? From benjamin at python.org Sun Jan 31 16:30:34 2010 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 31 Jan 2010 21:30:34 +0000 (UTC) Subject: iglob performance no better than glob References: <5ebd4965-32fc-4a03-94dd-c61ccdc9d630@f5g2000vbm.googlegroups.com> Message-ID: Kyp stsci.edu> writes: > So the iglob was faster, but accessing the first file took about the > same time as glob.glob. That would be because glob is implemented in terms of iglob. From benjamin at python.org Sun Jan 31 16:32:42 2010 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 31 Jan 2010 21:32:42 +0000 (UTC) Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <87iqaj9n8z.fsf@castleamber.com> <877hqz2j1o.fsf@castleamber.com> <4B6558AF.1090009@v.loewis.de> Message-ID: Sean DiZazzo gmail.com> writes: > Does "magic" really need to be used? Why not just use the revision > number? Because magic is easier and otherwise CPython developers would have to rebuild their pycs everytime their working copy was updated. From nobody at nowhere.com Sun Jan 31 16:52:26 2010 From: nobody at nowhere.com (Nobody) Date: Sun, 31 Jan 2010 21:52:26 +0000 Subject: What's the Scoop on \\ for Paths? (Win) References: Message-ID: On Sun, 31 Jan 2010 13:41:55 -0600, Tim Chase wrote: > The previous absolute-path fails in cmd.exe for a variety of apps because > the "/" is treated as a parameter/switch to the various programs. > Fortunately, the Python path-handling sub-system is smart enough to do the > right thing, even when Win32's internal handling is too dumb to behave: Let's not conflate multiple issues: 1. Windows API functions almost invariably accept either \ or /. 2. Python functions just pass filename strings verbatim; they won't try to "fix" them in cases where a specific separator is required (e.g. system(), which calls cmd.exe or %COMSPEC%, which has its own requirements). 3. Command-line programs may treat / as indicating a switch, even in what would normally be considered the middle of an argument. 4. Windows doesn't enforce the "char **argv" convention. Programs get passed the literal command line as a single string, and are free to parse it however they wish. 4a. Programs compiled with MSVC and using a main(argc, argv) interface will have the command-line parsed as documented in: http://msdn.microsoft.com/en-us/library/17w5ykft.aspx Programs built with other compilers, programs using WinMain(), and programs which use the "raw" command line may behave differently, and certain Windows command-line programs (most notably those inherited from DOS) *do* behave differently. 5. Python functions which accept command arguments as a list of strings (e.g. subprocess.call()) will assemble a command string according to the above specification. If you're invoking a program which doesn't follow that specification, you'll need to pass the command-line as a string rather than as a list of arguments. From newssw at gmail.com Sun Jan 31 17:07:44 2010 From: newssw at gmail.com (Mik0b0) Date: Sun, 31 Jan 2010 14:07:44 -0800 (PST) Subject: ftp.storlines error Message-ID: <24ffaab0-f319-4019-97cf-56a7531e3eef@n7g2000yqb.googlegroups.com> Good day/night/etc. I am rather a newb in Python (learning Python 3). I am trying to create a small script for FTP file uploads on my home network. The script looks like this: from ftplib import FTP ftp=FTP('10.0.0.1') ftp.login('mike','*****') directory='/var/www/blabla/' ftp.cwd(directory) ftp.retrlines('LIST') print('<- - - - - - - - - >') file_to_change='test' file=1 file=open(file_to_change,'w') text='test' file.write(text) ftp.storlines('STOR ' + file_to_change,file) ftp.retrlines('LIST') file.close() The output is like this: Traceback (most recent call last): File "ftp.py", line 13, in ftp.storlines('STOR ' + file_to_change,i) File "/usr/lib/python3.1/ftplib.py", line 474, in storlines buf = fp.readline() IOError: not readable What is wrong? From affdfsdfdsfsd at b.com Sun Jan 31 17:12:38 2010 From: affdfsdfdsfsd at b.com (Tracubik) Date: 31 Jan 2010 22:12:38 GMT Subject: create a string of variable lenght References: <4b65729b$0$1126$4fafbaef@reader3.news.tin.it> Message-ID: <4b660056$0$1131$4fafbaef@reader3.news.tin.it> Il Sun, 31 Jan 2010 13:46:16 +0100, G?nther Dietrich ha scritto: > Maybe you might solve this if you decode your string to unicode. > Example: > > |>>> euro = "?" > |>>> len(euro) > |3 > |>>> u_euro = euro.decode('utf_8') > |>>> len(u_euro) > |1 > > Adapt the encoding ('utf_8' in my example) to whatever you use. > > Or create the unicode string directly: > > |>>> u_euro = u'?' > |>>> len(u_euro) > |1 > > > > Best regards, > > G?nther thank you, your two solution is really interesting. is there a possible to set unicode encoding by default for my python scripts? i've tried inserting # -*- coding: utf-8 -*- at the beginning of my script but doesn't solve the problem From clp2 at rebertia.com Sun Jan 31 17:19:49 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 31 Jan 2010 14:19:49 -0800 Subject: ftp.storlines error In-Reply-To: <24ffaab0-f319-4019-97cf-56a7531e3eef@n7g2000yqb.googlegroups.com> References: <24ffaab0-f319-4019-97cf-56a7531e3eef@n7g2000yqb.googlegroups.com> Message-ID: <50697b2c1001311419p68b42c0dmf0ab855e452b326d@mail.gmail.com> On Sun, Jan 31, 2010 at 2:07 PM, Mik0b0 wrote: > Good day/night/etc. > I am rather a newb in Python (learning Python 3). I am trying to > create a small script for FTP file uploads ?on my home network. The > script looks like this: > > from ftplib import FTP > ftp=FTP('10.0.0.1') > ftp.login('mike','*****') > directory='/var/www/blabla/' > ftp.cwd(directory) > ftp.retrlines('LIST') > print('<- - - - - - - - - >') > file_to_change='test' > file=1 > file=open(file_to_change,'w') Here you open the file in (w)rite mode. Also, don't call it `file` as that shadows the name of the built-in type. > text='test' > file.write(text) And indeed, here you've written something to it. > ftp.storlines('STOR ' + file_to_change,file) storlines() needs a file opened in (r)ead mode however, hence the error. Obviously, it needs to read the contents of the file in order to send it over the network. Either do the writing separately, close the file, and then open it again in read mode, or open it in one of the modes that allows for both reading and writing [see help(open) for details]. Cheers, Chris -- http://blog.rebertia.com > ftp.retrlines('LIST') > file.close() > > The output is like this: > Traceback (most recent call last): > ?File "ftp.py", line 13, in > ? ?ftp.storlines('STOR ' + file_to_change,i) > ?File "/usr/lib/python3.1/ftplib.py", line 474, in storlines > ? ?buf = fp.readline() > IOError: not readable > > What is wrong? From tinnews at isbd.co.uk Sun Jan 31 17:24:13 2010 From: tinnews at isbd.co.uk (tinnews at isbd.co.uk) Date: Sun, 31 Jan 2010 22:24:13 +0000 Subject: Can't get sys.stdin.readlines() to work References: <4vtg37-n89.ln1@chris.isbd.net> <3a3c2ba0-7221-4402-9ed6-0b74e101f95b@m16g2000yqc.googlegroups.com> Message-ID: Richard Thomas wrote: > On Jan 31, 6:15?pm, tinn... at isbd.co.uk wrote: > > I'm trying to read some data from standard input, what I'm actually > > trying to do is process some date pasted in using the mouse cut and > > paste on a Linux box (xubuntu 9.10) in a terminal window. > > > > First attempts failed so I'm now trying the trivial:- > > > > ? ? import sys > > ? ? data = sys.stdin.readlines() > > ? ? print "Counted", len(data), "lines." > > > > When I run this and try to paste something into the terminal window I > > get the following errors:- > > > > ? ? /home/chris/bin/m2r.py: line 2: syntax error near unexpected token `(' > > ? ? /home/chris/bin/m2r.py: line 2: `data = sys.stdin.readlines()' > > > > It does exactly the same if I try:- > > > > ? ? cat | m2r.py > > > > and then paste something into the window. > > > > So - what on earth am I doing wrong? > > > > -- > > Chris Green > > You haven't put a shebang line at the top. > > #!/usr/bin/env python > import sys > data = sys.stdin.readlines() > ... Urk!!! Silly me. That catches me out every few months! :-) -- Chris Green From andrewdalke at gmail.com Sun Jan 31 17:29:09 2010 From: andrewdalke at gmail.com (Andrew Dalke) Date: Sun, 31 Jan 2010 14:29:09 -0800 (PST) Subject: recv_into(bytearray) complains about a "pinned buffer" Message-ID: <8dff1c70-c141-4f0e-aebb-3a92733b272b@o28g2000yqh.googlegroups.com> In Python 2.6 I can't socket.recv_into(a byte array instance). I get a TypeError which complains about a "pinned buffer". I have only an inkling of what that means. Since an array.array("b") works there, and since it works in Python 3.1.1, and since I thought the point of a bytearray was to make things like recv_into easier, I think this exception is a bug in Python 2.6. I want to double check before posting it to the tracker. Here's my reproducibles: Python 2.6.1 (r261:67515, Jul 7 2009, 23:51:51) [GCC 4.2.1 (Apple Inc. build 5646)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import socket >>> sock = socket.socket() >>> sock.connect( ("python.org", 80) ) >>> sock.send(b"GET / HTTP/1.0\r\n\r\n") 18 >>> buf = bytearray(b" " * 10) >>> sock.recv_into(buf) Traceback (most recent call last): File "", line 1, in TypeError: recv_into() argument 1 must be pinned buffer, not bytearray >>> I expected a bytearray to work there. In fact, I thought the point of bytearray was to allow this to work. By comparison, an array of bytes does work: >>> import array >>> arr = array.array("b") >>> arr.extend(map(ord, "This is a test")) >>> len(arr) 14 >>> sock.recv_into(arr) 14 >>> arr array('b', [72, 84, 84, 80, 47, 49, 46, 49, 32, 51, 48, 50, 32, 70]) >>> "".join(map(chr, arr)) 'HTTP/1.1 302 F' I don't even know what a "pinned buffer" means, and searching python.org isn't helpful. Using a bytearray in Python 3.1.1 *does* work: Python 3.1.1 (r311:74480, Jan 31 2010, 23:07:16) [GCC 4.2.1 (Apple Inc. build 5646) (dot 1)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import socket >>> sock = socket.socket() >>> sock.connect( ("python.org", 80) ) >>> sock.send(b"GET / HTTP/1.0\r\n\r\n") 18 >>> buf = bytearray(b" " * 10) >>> sock.recv_into(buf) 10 >>> buf bytearray(b'HTTP/1.1 3') >>> Is this a bug in Python 2.6 or a deliberate choice regarding implementation concerns I don't know about? If it's a bug, I'll add it to the tracker. Andrew Dalke dalke at dalkescientific.com From steve at REMOVE-THIS-cybersource.com.au Sun Jan 31 17:36:32 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 31 Jan 2010 22:36:32 GMT Subject: Python and Ruby References: Message-ID: <0375f245$0$1309$c3e8da3@news.astraweb.com> On Sun, 31 Jan 2010 04:28:41 -0800, Ed Keith wrote: > In most functional languages you just name a function to access it and > you do it ALL the time. > > for example, in if you have a function 'f' which takes two parameters to > call the function and get the result you use: > > f 2 3 > > If you want the function itself you use: > > f How do you call a function of no arguments? -- Steven From steve at REMOVE-THIS-cybersource.com.au Sun Jan 31 17:37:04 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 31 Jan 2010 22:37:04 GMT Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> <0374c6d6$0$1309$c3e8da3@news.astraweb.com> <871vh7t2tm.fsf@benfinney.id.au> <7xljffngid.fsf@ruckus.brouhaha.com> <0374e0be$0$1309$c3e8da3@news.astraweb.com> <0374fb80$0$1309$c3e8da3@news.astraweb.com> <878wbenwk5.fsf@castleamber.com> Message-ID: <0375f265$0$1309$c3e8da3@news.astraweb.com> On Sun, 31 Jan 2010 09:06:18 -0600, John Bokma wrote: > Based on the magic numbers I've seen so far it looks like that not an > option. They increment with every minor change. They increment with every *incompatible* change to the marshal format, not every change to the compiler. > So to me, at this moment > (and maybe it's my ignorance) it looks like a made up example to justify > what to me still looks like a bad decision. Of course it's a made-up example. But with Python now entering a period where there is a moratorium on changes to the language, is it really so difficult to imagine that the marshal format will settle down for a while even as the standard library goes through upgrades? -- Steven From steve at REMOVE-THIS-cybersource.com.au Sun Jan 31 17:37:25 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 31 Jan 2010 22:37:25 GMT Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> <87y6jef1df.fsf@castleamber.com> Message-ID: <0375f27a$0$1309$c3e8da3@news.astraweb.com> On Sun, 31 Jan 2010 14:47:08 -0600, John Bokma wrote: > An editor can correct the indenting of the braces example but can't with > this one. > > if x: > if y: > foo() > else: > bar() > > While braces might be considered redundant they are not when for one > reason or another formatting is lost or done incorrectly. I've heard this argument before, and I don't buy it. Why should we expect the editor to correct malformed code? Would you expect your editor to correct this malformed code? result = sin(x+)y Why should broken indentation be held to a higher standard than any other breakage in code? -- Steven From steve at REMOVE-THIS-cybersource.com.au Sun Jan 31 17:40:03 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 31 Jan 2010 22:40:03 GMT Subject: PEP 3147 - new .pyc format References: <5e9e2096-3bd0-4873-8b99-7ce7f1c5fc97@b7g2000pro.googlegroups.com> Message-ID: <0375f318$0$1309$c3e8da3@news.astraweb.com> On Sun, 31 Jan 2010 14:10:34 -0800, Dennis Lee Bieber wrote: > Ugh... That would mean that for an application using, say 20 > files, > one now has 20 subdirectories for what, in a lot of cases, will contain > just one file each (and since I doubt older Python's will be modified to > support this scheme, it will only be applicable to 3.x, and maybe a > 2.7?) If you only use one version of Python, then don't run it with the -R switch. Have you read the PEP? It is quite explicit that the default behaviour of .pyc files will remain unchanged, that to get the proposed behaviour you have to specifically ask for it. -- Steven From steve at REMOVE-THIS-cybersource.com.au Sun Jan 31 17:40:55 2010 From: steve at REMOVE-THIS-cybersource.com.au (Steven D'Aprano) Date: 31 Jan 2010 22:40:55 GMT Subject: Keyboard input References: <8f67b6f81001290853i39bab1acs49d02ac5be948d13@mail.gmail.com> <8f67b6f81001310351m3485fd40o8944c11af8a4161f@mail.gmail.com> Message-ID: <0375f34c$0$1309$c3e8da3@news.astraweb.com> On Sun, 31 Jan 2010 14:10:34 -0800, Dennis Lee Bieber wrote: > On Sun, 31 Jan 2010 11:51:46 +0000, "Mr.SpOOn" > declaimed the following in gmane.comp.python.general: > >> 2010/1/29 Gabriel Genellina : >> > >> > That's strange. If you're using Linux, make sure you have the >> > readline package installed. >> >> I'm using Linux. Ubuntu. I checked on synaptic and I have >> readline-common. Do you mean something else? >> > But was your Python /built/ using readline? How do you ensure that it is? -- Steven From python at mrabarnett.plus.com Sun Jan 31 17:42:52 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 31 Jan 2010 22:42:52 +0000 Subject: Slow down while creating a big list and iterating over it In-Reply-To: <4b484bc6-f903-40a0-a8dc-9a00c8fc002a@k5g2000yqf.googlegroups.com> References: <2fdd4e1b-db58-45d7-9b7d-73dcdca6384a@c29g2000yqd.googlegroups.com> <29a69a7c-a8a2-4c53-9521-760e13665cd3@f12g2000yqn.googlegroups.com> <4b484bc6-f903-40a0-a8dc-9a00c8fc002a@k5g2000yqf.googlegroups.com> Message-ID: <4B66076C.4050102@mrabarnett.plus.com> marc magrans de abril wrote: > Hi! > > ...I have found a good enough solution, although it only works if the > number of patterns (clusters) is not very big: > def classify(f): > THERESHOLD=0.1 > > patterns={} > for l in enumerate(f): > found = False > for p,c in patterns.items(): > if dist(l,p) < THERESHOLD: > found=True > patterns[p] = c +1 > > if not found: > patterns[l] = 1 > > return patterns > > This algorithm is O(n*np*m^2). Where n is the number of logs, np the > number of patterns, and m is the log length (i.e. m^2 is the distance > cost). So it's way better O(n^2*m^2) and I can run it for some hours > to get back the results. > > I wonder if there is a single threaded/process clustering algorithm > than runs in O(n)? > Your original code used the first entry in the remaining logs for each pattern, but your new code stores the patterns in a dict, which is unordered, so you might get different results. But that doesn't matter, because your new code increments the count when it has found a match, and then continues looking, so it might match and increment more than once. Finally, your original code treated it as a match if distance <= threshold but your new code treats it as a match if distance < threshold. patterns = [] for x in logs: for index, (pat, count) in enumerate(patterns): if dist(pat, x) <= THRESHOLD: patterns[index] = pat, count + 1 break else: # Didn't break out of the loop, therefore no match. patterns.append((x, 1)) From python at mrabarnett.plus.com Sun Jan 31 17:49:56 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 31 Jan 2010 22:49:56 +0000 Subject: create a string of variable lenght In-Reply-To: <4b660056$0$1131$4fafbaef@reader3.news.tin.it> References: <4b65729b$0$1126$4fafbaef@reader3.news.tin.it> <4b660056$0$1131$4fafbaef@reader3.news.tin.it> Message-ID: <4B660914.1040409@mrabarnett.plus.com> Tracubik wrote: > Il Sun, 31 Jan 2010 13:46:16 +0100, G?nther Dietrich ha > scritto: > >> Maybe you might solve this if you decode your string to unicode. >> Example: >> >> |>>> euro = "?" >> |>>> len(euro) >> |3 >> |>>> u_euro = euro.decode('utf_8') >> |>>> len(u_euro) >> |1 >> >> Adapt the encoding ('utf_8' in my example) to whatever you use. >> >> Or create the unicode string directly: >> >> |>>> u_euro = u'?' >> |>>> len(u_euro) >> |1 >> >> >> >> Best regards, >> >> G?nther > > thank you, your two solution is really interesting. > is there a possible to set unicode encoding by default for my python > scripts? > i've tried inserting > # -*- coding: utf-8 -*- > > at the beginning of my script but doesn't solve the problem That tells Python which encoding the file is using, but you still need to save the file in that encoding. From newssw at gmail.com Sun Jan 31 18:06:25 2010 From: newssw at gmail.com (Mik0b0) Date: Sun, 31 Jan 2010 15:06:25 -0800 (PST) Subject: ftp.storlines error References: <24ffaab0-f319-4019-97cf-56a7531e3eef@n7g2000yqb.googlegroups.com> Message-ID: On Feb 1, 12:19?am, Chris Rebert wrote: > On Sun, Jan 31, 2010 at 2:07 PM, Mik0b0 wrote: > > Good day/night/etc. > > I am rather a newb in Python (learning Python 3). I am trying to > > create a small script for FTP file uploads ?on my home network. The > > script looks like this: > > > from ftplib import FTP > > ftp=FTP('10.0.0.1') > > ftp.login('mike','*****') > > directory='/var/www/blabla/' > > ftp.cwd(directory) > > ftp.retrlines('LIST') > > print('<- - - - - - - - - >') > > file_to_change='test' > > file=1 > > file=open(file_to_change,'w') > > Here you open the file in (w)rite mode. Also, don't call it `file` as > that shadows the name of the built-in type. > > > text='test' > > file.write(text) > > And indeed, here you've written something to it. > > > ftp.storlines('STOR ' + file_to_change,file) > > storlines() needs a file opened in (r)ead mode however, hence the > error. Obviously, it needs to read the contents of the file in order > to send it over the network. > Either do the writing separately, close the file, and then open it > again in read mode, or open it in one of the modes that allows for > both reading and writing [see help(open) for details]. > > Cheers, > Chris > --http://blog.rebertia.com > > > ftp.retrlines('LIST') > > file.close() > > > The output is like this: > > Traceback (most recent call last): > > ?File "ftp.py", line 13, in > > ? ?ftp.storlines('STOR ' + file_to_change,i) > > ?File "/usr/lib/python3.1/ftplib.py", line 474, in storlines > > ? ?buf = fp.readline() > > IOError: not readable > > > What is wrong? Thanks Chris, the final version looks like this and it works: from ftplib import FTP ftp=FTP('10.0.0.1') ftp.login('mike','******') directory='/var/www/blabla/' ftp.cwd(directory) ftp.retrlines('LIST') print('<- - - - - - - - - >') file_to_change='test.php' i=1 i=open(file_to_change,'w') text='test' i.write(text) i.close() i=open(file_to_change,'rb') ftp.storbinary('STOR ' + file_to_change,i) ftp.retrlines('LIST') i.close() From clp2 at rebertia.com Sun Jan 31 18:40:36 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 31 Jan 2010 15:40:36 -0800 Subject: Python and Ruby In-Reply-To: <0375f245$0$1309$c3e8da3@news.astraweb.com> References: <0375f245$0$1309$c3e8da3@news.astraweb.com> Message-ID: <50697b2c1001311540m1810839ic985f3071f283558@mail.gmail.com> On Sun, Jan 31, 2010 at 2:36 PM, Steven D'Aprano wrote: > On Sun, 31 Jan 2010 04:28:41 -0800, Ed Keith wrote: >> In most functional languages you just name a function to access it and >> you do it ALL the time. >> >> for example, in if you have a function 'f' which takes two parameters to >> call the function and get the result you use: >> >> ?f 2 3 >> >> If you want the function itself you use: >> >> ? ?f > > How do you call a function of no arguments? It's not really a function in that case, it's just a named constant. (Recall that functions don't/can't have side-effects.) Cheers, Chris -- http://blog.rebertia.com From burslem2001 at yahoo.com Sun Jan 31 19:01:43 2010 From: burslem2001 at yahoo.com (gazza) Date: Sun, 31 Jan 2010 16:01:43 -0800 (PST) Subject: gmtime References: <62938b5b-7eda-426c-b91d-3c7061840b2d@l26g2000yqd.googlegroups.com> Message-ID: <5c1b6f40-6620-4c0e-8e93-e04a9c65d922@l19g2000yqb.googlegroups.com> On Jan 31, 3:27?pm, gazza wrote: > Hi, > > I am trying to discover how to obtain the correct time of say CST/ > America and EST/America in python? > > Any help on this would be appreciated. > > Thanks, > Garyc I found some information. Someone suggested I use the pytz library? Cheers, Garyc From solipsis at pitrou.net Sun Jan 31 19:04:57 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 1 Feb 2010 00:04:57 +0000 (UTC) Subject: recv_into(bytearray) complains about a "pinned buffer" References: <8dff1c70-c141-4f0e-aebb-3a92733b272b@o28g2000yqh.googlegroups.com> Message-ID: Hello Andrew, > I don't even know what a "pinned buffer" means, and searching python.org > isn't helpful. > > Using a bytearray in Python 3.1.1 *does* work: > [...] Agreed, the error message is cryptic. The problem is that socket.recv_into() in 2.6 doesn't recognize the new buffer API which is needed to accept bytearray objects. (it does in 3.1, because the old buffer API doesn't exist anymore there) You could open an issue on the bug tracker for this. Thank you Antoine. From arnodel at googlemail.com Sun Jan 31 19:14:52 2010 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 01 Feb 2010 00:14:52 +0000 Subject: Python and Ruby References: <0375f245$0$1309$c3e8da3@news.astraweb.com> Message-ID: Steven D'Aprano writes: > On Sun, 31 Jan 2010 04:28:41 -0800, Ed Keith wrote: > >> In most functional languages you just name a function to access it and >> you do it ALL the time. >> >> for example, in if you have a function 'f' which takes two parameters to >> call the function and get the result you use: >> >> f 2 3 >> >> If you want the function itself you use: >> >> f > > How do you call a function of no arguments? In a functional language, a function of no arguments will always return the same value. So, from a non-functional point of vue, f is both the function and its value. -- Arnaud From kyp at stsci.edu Sun Jan 31 19:22:32 2010 From: kyp at stsci.edu (Kyp) Date: Sun, 31 Jan 2010 16:22:32 -0800 (PST) Subject: iglob performance no better than glob References: <5ebd4965-32fc-4a03-94dd-c61ccdc9d630@f5g2000vbm.googlegroups.com> <87k4uyw3mi.fsf@castleamber.com> Message-ID: On Jan 31, 1:06?pm, John Bokma wrote: > Kyp writes: > > Is there a way to get the first X # of files from a dir with lots of > > files, that does not take a long time to run? > > Assuming Linux: what does time > > ?ls thedir | head > > give? > > with thedir the name of the actual dir about 3 seconds. 3.086u 0.201s 0:03.32 98.7% 0+0k 0+0io 0pf+0w > > Also how many is many files? over 100K (I know I should not do that, but it's a temp dir holding files to be transferred) thanx, mark From kyp at stsci.edu Sun Jan 31 19:23:05 2010 From: kyp at stsci.edu (Kyp) Date: Sun, 31 Jan 2010 16:23:05 -0800 (PST) Subject: iglob performance no better than glob References: <5ebd4965-32fc-4a03-94dd-c61ccdc9d630@f5g2000vbm.googlegroups.com> Message-ID: <449a19de-42f7-4ae3-8099-99c27572adc8@m16g2000yqc.googlegroups.com> On Jan 31, 2:44?pm, Peter Otten <__pete... at web.de> wrote: > Kyp wrote: > > I have a dir with a large # of files that I need to perform operations > > on, but only needing to access a subset of the files, i.e. the first > > 100 files. > > > Using glob is very slow, so I ran across iglob, which returns an > > iterator, which seemed just like what I wanted. I could iterate over > > the files that I wanted, not having to read the entire dir. > > > So the iglob was faster, but accessing the first file took about the > > same time as glob.glob. > > > Here's some code to compare glob vs. iglob performance, ?it outputs > > the time before/after a glob.iglob('*.*') files.next() sequence and a > > glob.glob('*.*') sequence. > > > #!/usr/bin/env python > > > import glob,time > > print '\nTest of glob.iglob' > > print 'before ? ? ? iglob:', time.asctime() > > files = glob.iglob('*.*') > > print 'after ? ? ? ?iglob:',time.asctime() > > print files.next() > > print 'after files.next():', time.asctime() > > > print '\nTest of glob.glob' > > print 'before ? ? ? ?glob:', time.asctime() > > files = glob.glob('*.*') > > print 'after ? ? ? ? glob:',time.asctime() > > > Here are the results: > > > Test of glob.iglob > > before ? ? ? iglob: Sun Jan 31 11:09:08 2010 > > after ? ? ? ?iglob: Sun Jan 31 11:09:08 2010 > > foo.bar > > after files.next(): Sun Jan 31 11:09:59 2010 > > > Test of glob.glob > > before ? ? ? ?glob: Sun Jan 31 11:09:59 2010 > > after ? ? ? ? glob: Sun Jan 31 11:10:51 2010 > > > The results are about the same for the 2 approaches, both took about > > 51 seconds. Am I doing something wrong with iglob? > > No, but iglob() being lazy is pointless in your case because it uses > os.listdir() and fnmatch.filter() underneath which both read the whole > directory before returning anything. > > > Is there a way to get the first X # of files from a dir with lots of > > files, that does not take a long time to run? > > Here's my attempt. It turned out to be more work than expected, so I cut a > few corners. It's Linux-only "works on my machine" code, but may give you > some hints on how to proceed. > > from ctypes import * > import fnmatch > import glob > import os > import re > from itertools import ifilter, imap > > class dirent(Structure): > ? ? "works on my machine ;)" > ? ? _fields_ = [ > ? ? ? ? ("d_ino", c_long), > ? ? ? ? ("d_off", c_long), > ? ? ? ? ("d_reclen", c_ushort), > ? ? ? ? ("d_type", c_ubyte), > ? ? ? ? ("d_name", c_char*256)] > > direntp = POINTER(dirent) > > LIBC = "libc.so.6" > cdll.LoadLibrary(LIBC) > libc = CDLL(LIBC) > libc.readdir.restype = direntp > > def diriter(dir): > ? ? "lazy partial replacement for os.listdir()" > ? ? # errors? what errors? > ? ? dirp = libc.opendir(dir) > ? ? if not dirp: > ? ? ? ? return > ? ? try: > ? ? ? ? while True: > ? ? ? ? ? ? ep = libc.readdir(dirp) > ? ? ? ? ? ? if not ep: > ? ? ? ? ? ? ? ? break > ? ? ? ? ? ? yield ep.contents.d_name > ? ? finally: > ? ? ? ? libc.closedir(dirp) > > def filter(names, pattern): > ? ? "lazy partial replacement for fnmatch.filter()" > ? ? import posixpath > > ? ? pattern = os.path.normcase(pattern) > ? ? r = fnmatch.translate(pattern) > ? ? r = re.compile(r) > > ? ? if os.path is not posixpath: > ? ? ? ? names = imap(os.path.normcase, names) > > ? ? return ifilter(r.match, names) > > def globiter(path): > ? ? "lazy partial replacement for glob.glob()" > ? ? dir, filename = os.path.split(path) > ? ? if glob.has_magic(dir): > ? ? ? ? raise ValueError("wildcards in directory not supported") > ? ? return filter(diriter(dir), filename) > > if __name__ == "__main__": > ? ? import sys > ? ? [pattern] = sys.argv[1:] > ? ? for name in globiter(pattern): > ? ? ? ? print name > > Peter I'll give it a try, thanx for the reply. mark From steven at REMOVE.THIS.cybersource.com.au Sun Jan 31 19:25:44 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 01 Feb 2010 00:25:44 GMT Subject: Python and Ruby References: <0375f245$0$1309$c3e8da3@news.astraweb.com> Message-ID: On Sun, 31 Jan 2010 15:40:36 -0800, Chris Rebert wrote: > On Sun, Jan 31, 2010 at 2:36 PM, Steven D'Aprano > wrote: >> On Sun, 31 Jan 2010 04:28:41 -0800, Ed Keith wrote: >>> In most functional languages you just name a function to access it and >>> you do it ALL the time. >>> >>> for example, in if you have a function 'f' which takes two parameters >>> to call the function and get the result you use: >>> >>> ?f 2 3 >>> >>> If you want the function itself you use: >>> >>> ? ?f >> >> How do you call a function of no arguments? > > It's not really a function in that case, it's just a named constant. > (Recall that functions don't/can't have side-effects.) >>> time.time(), random.random() (1264983502.7505889, 0.29974255140479633) >>> time.time(), random.random() (1264983505.9283719, 0.74207867411026329) They don't look terribly constant to me. There is a difference between a function that does "give me whatever value is specified by a fixed description" and a function that does "give me a fixed value". -- Steven From e_d_k at yahoo.com Sun Jan 31 19:27:09 2010 From: e_d_k at yahoo.com (Ed Keith) Date: Sun, 31 Jan 2010 16:27:09 -0800 (PST) Subject: Python and Ruby In-Reply-To: <0375f245$0$1309$c3e8da3@news.astraweb.com> Message-ID: <553505.49272.qm@web58905.mail.re1.yahoo.com> --- On Sun, 1/31/10, Steven D'Aprano wrote: > From: Steven D'Aprano > Subject: Re: Python and Ruby > To: python-list at python.org > Date: Sunday, January 31, 2010, 5:36 PM > On Sun, 31 Jan 2010 04:28:41 -0800, > Ed Keith wrote: > > > In most functional languages you just name a function > to access it and > > you do it ALL the time. > > > > for example, in if you have a function 'f' which takes > two parameters to > > call the function and get the result you use: > > > >? f 2 3 > > > > If you want the function itself you use: > > > >? ? f > > How do you call a function of no arguments? > > In a 'pure' functional language a function with no arguments is, by definition, a constant. This is because a 'pure' function will always return the same result whenever given the same arguments. so if it has no argument it always returns a constant value. -EdK Ed Keith e_d_k at yahoo.com Blog: edkeith.blogspot.com From bblais at bryant.edu Sun Jan 31 19:27:39 2010 From: bblais at bryant.edu (Brian Blais) Date: Sun, 31 Jan 2010 19:27:39 -0500 Subject: odd drawing problem with turtle.py Message-ID: I'm on Python 2.5, but using the updated turtle.py Version 1.0.1 - 24. 9. 2009. The following script draws 5 circles, which it is supposed to, but then doesn't draw the second turtle which is supposed to simply move forward. Any ideas? from turtle import * from numpy.random import randint resetscreen() class Circle(object): def __init__(self,x,y,r,color): self.x=x self.y=y self.r=r self.color=color self.turtle=Turtle(visible=False) self.turtle.tracer(False) self.draw() def draw(self): self.turtle.penup() self.turtle.setposition(self.x,self.y) self.turtle.setheading(0) self.turtle.backward(self.r) self.turtle.pendown() self.turtle.fill(True) self.turtle.pencolor("black") self.turtle.fillcolor(self.color) self.turtle.circle(self.r) self.turtle.fill(False) self.turtle.penup() for i in range(5): c=Circle(randint(-350,350),randint(-250,250),10,"red") T=Turtle() T.forward(100) T.forward(100) thanks, bb -- Brian Blais bblais at bryant.edu http://web.bryant.edu/~bblais http://bblais.blogspot.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From john at castleamber.com Sun Jan 31 19:47:42 2010 From: john at castleamber.com (John Bokma) Date: Sun, 31 Jan 2010 18:47:42 -0600 Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> <87y6jef1df.fsf@castleamber.com> <0375f27a$0$1309$c3e8da3@news.astraweb.com> Message-ID: <878wbdye6p.fsf@castleamber.com> Steven D'Aprano writes: > On Sun, 31 Jan 2010 14:47:08 -0600, John Bokma wrote: > >> An editor can correct the indenting of the braces example but can't with >> this one. >> >> if x: >> if y: >> foo() >> else: >> bar() >> >> While braces might be considered redundant they are not when for one >> reason or another formatting is lost or done incorrectly. > > I've heard this argument before, and I don't buy it. Why should we expect > the editor to correct malformed code? Or a prettyfier. It doesn't matter. The point is that with braces there *is* redundancy that be used to fix the code. > Would you expect your editor to correct this malformed code? > > result = sin(x+)y Nice straw man. Let me repeat again: I am ok with how Python works. To be honest I think it's cleaner compared to using {}. But in there are real life examples in which Python code will break where code with braces will survive. -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From clp2 at rebertia.com Sun Jan 31 19:50:50 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 31 Jan 2010 16:50:50 -0800 Subject: Python and Ruby In-Reply-To: References: <0375f245$0$1309$c3e8da3@news.astraweb.com> Message-ID: <50697b2c1001311650qb7b1025p1e5b5b2d0ef21c8c@mail.gmail.com> On Sun, Jan 31, 2010 at 4:25 PM, Steven D'Aprano wrote: > On Sun, 31 Jan 2010 15:40:36 -0800, Chris Rebert wrote: >> On Sun, Jan 31, 2010 at 2:36 PM, Steven D'Aprano >> wrote: >>> On Sun, 31 Jan 2010 04:28:41 -0800, Ed Keith wrote: >>>> In most functional languages you just name a function to access it and >>>> you do it ALL the time. >>>> >>>> for example, in if you have a function 'f' which takes two parameters >>>> to call the function and get the result you use: >>>> >>>> ?f 2 3 >>>> >>>> If you want the function itself you use: >>>> >>>> ? ?f >>> >>> How do you call a function of no arguments? >> >> It's not really a function in that case, it's just a named constant. >> (Recall that functions don't/can't have side-effects.) > > >>>> time.time(), random.random() > (1264983502.7505889, 0.29974255140479633) >>>> time.time(), random.random() > (1264983505.9283719, 0.74207867411026329) > > > They don't look terribly constant to me. Those aren't functions in the pure functional programming sense; which is unsurprising since Python isn't a [pure] functional language. They both involve side-effects. time() does I/O to the clock chip to see what time it is, and random() uses and changes a global seed value variable (which, in a double-whammy, takes its initial value from time()). Pure functions must be referentially transparent [http://en.wikipedia.org/wiki/Referential_transparency_(computer_science)], and as you've demonstrated, neither of those Python functions qualify. Cheers, Chris -- http://blog.rebertia.com From john at castleamber.com Sun Jan 31 19:53:16 2010 From: john at castleamber.com (John Bokma) Date: Sun, 31 Jan 2010 18:53:16 -0600 Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> <87y6jef1df.fsf@castleamber.com> <0375f27a$0$1309$c3e8da3@news.astraweb.com> Message-ID: <874om1ydxf.fsf@castleamber.com> Steven D'Aprano writes: > On Sun, 31 Jan 2010 14:47:08 -0600, John Bokma wrote: > >> An editor can correct the indenting of the braces example but can't with >> this one. >> >> if x: >> if y: >> foo() >> else: >> bar() >> >> While braces might be considered redundant they are not when for one >> reason or another formatting is lost or done incorrectly. > > I've heard this argument before, and I don't buy it. Why should we expect > the editor to correct malformed code? I do expect my editor to assist me in coding. In Emacs I have to do some effort to enter the broken C code in the earlier post, and when I reformat the code, it will be lined out correctly. I can't do that with the above example, because it's correctly formatted. You don't have to buy my argument, I am not selling it. While in the past I wrote that an editor can't make you that more productive I want to take that back, on the record. Since I've switched to Emacs the editor has saved me several times from minor issues. Either because it refused to indent correctly thanks to a missing closing }, ), ] or other error. With the correct mode in Emacs one gets, in my experience, immediate feedback when making mistakes one otherwise find during the run/compiling phase. Note that I am also not selling Emacs. It's free after all. -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From benjamin.kaplan at case.edu Sun Jan 31 19:54:17 2010 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Sun, 31 Jan 2010 19:54:17 -0500 Subject: create a string of variable lenght In-Reply-To: <4b660056$0$1131$4fafbaef@reader3.news.tin.it> References: <4b65729b$0$1126$4fafbaef@reader3.news.tin.it> <4b660056$0$1131$4fafbaef@reader3.news.tin.it> Message-ID: On Sun, Jan 31, 2010 at 5:12 PM, Tracubik wrote: > Il Sun, 31 Jan 2010 13:46:16 +0100, G?nther Dietrich ha > scritto: > >> Maybe you might solve this if you decode your string to unicode. >> Example: >> >> |>>> euro = "?" >> |>>> len(euro) >> |3 >> |>>> u_euro = euro.decode('utf_8') >> |>>> len(u_euro) >> |1 >> >> Adapt the encoding ('utf_8' in my example) to whatever you use. >> >> Or create the unicode string directly: >> >> |>>> u_euro = u'?' >> |>>> len(u_euro) >> |1 >> >> >> >> Best regards, >> >> G?nther > > thank you, your two solution is really interesting. > is there a possible to set unicode encoding by default for my python > scripts? > i've tried inserting > # -*- coding: utf-8 -*- > > at the beginning of my script but doesn't solve the problem First of all, if you haven't read this before, please do. It will make this much clearer. http://www.joelonsoftware.com/articles/Unicode.html To reiterate: UTF-8 IS NOT UNICODE!!!! In Python 2, '*' signifies a byte string. It is read as a sequence of bytes and interpreted as a sequence of bytes When Python encounters the sequence 0x27 0xe2 0x82 0xac 0x27 in the code (the UTF-8 bytes for '?') it interprets it as 3 bytes between the two quotes. It doesn't care about characters or anything like that. u'*' signifies a Unicode string. Python will attempt to convert the sequence of bytes into a sequence of characters. It can use any encoding for that: cp1252, utf-8, MacRoman, ISO-8859-15. UTF-8 isn't special, it's just one of the few encodings capable of storing all of the possible Unicode characters. What the line at the top says is that the file should be read using UTF-8. Byte strings are still just sequences of bytes- this doesn't affect them. But any Unicode string will be decoded using UTF-8. IF python looks at the above sequence of bytes as a Unicode string, it views the 3 bytes as a single character. When you ask for it's length, it returns the number of characters. Solution to your problem: in addition to keeping the #-*- coding ... line, go with G?nther's advice and use Unicode strings. > -- > http://mail.python.org/mailman/listinfo/python-list > From purui.wang at gmail.com Sun Jan 31 20:19:41 2010 From: purui.wang at gmail.com (pograph) Date: Sun, 31 Jan 2010 17:19:41 -0800 (PST) Subject: gmtime References: <62938b5b-7eda-426c-b91d-3c7061840b2d@l26g2000yqd.googlegroups.com> <5c1b6f40-6620-4c0e-8e93-e04a9c65d922@l19g2000yqb.googlegroups.com> Message-ID: <04b2f529-a66f-431f-8e76-3c773a183653@b7g2000pro.googlegroups.com> On Jan 31, 4:01?pm, gazza wrote: > On Jan 31, 3:27?pm, gazza wrote: > > > Hi, > > > I am trying to discover how to obtain the correct time of say CST/ > > America and EST/America in python? > > > Any help on this would be appreciated. > > > Thanks, > > Garyc > > I found some information. Someone suggested I use the pytz library? > > Cheers, > Garyc tz = pytz.timezone('US/Pacific') t = datetime.datetime.now(tz) From steven at REMOVE.THIS.cybersource.com.au Sun Jan 31 20:22:23 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 01 Feb 2010 01:22:23 GMT Subject: Python and Ruby References: <0375f245$0$1309$c3e8da3@news.astraweb.com> Message-ID: On Sun, 31 Jan 2010 16:50:50 -0800, Chris Rebert wrote: >>>> How do you call a function of no arguments? >>> >>> It's not really a function in that case, it's just a named constant. >>> (Recall that functions don't/can't have side-effects.) >> >> >>>>> time.time(), random.random() >> (1264983502.7505889, 0.29974255140479633) >>>>> time.time(), random.random() >> (1264983505.9283719, 0.74207867411026329) >> >> >> They don't look terribly constant to me. > > Those aren't functions in the pure functional programming sense; which > is unsurprising since Python isn't a [pure] functional language. They > both involve side-effects. time() does I/O to the clock chip to see what > time it is, and random() uses and changes a global seed value variable > (which, in a double-whammy, takes its initial value from time()). Yes, but these tasks -- get the time, get a (pseudo) random number -- are not unique to Python. Surely even Lisp and Haskell code will sometimes need to know the time. Whether they are "pure functions" (functions in the mathematical sense) or impure, they're still functions in some sense. How do you deal with such impure functions? -- Steven From e_d_k at yahoo.com Sun Jan 31 20:31:05 2010 From: e_d_k at yahoo.com (Ed Keith) Date: Sun, 31 Jan 2010 17:31:05 -0800 (PST) Subject: Python and Ruby In-Reply-To: Message-ID: <934786.78871.qm@web58905.mail.re1.yahoo.com> --- On Sun, 1/31/10, Steven D'Aprano wrote: > From: Steven D'Aprano > Subject: Re: Python and Ruby > To: python-list at python.org > Date: Sunday, January 31, 2010, 8:22 PM > On Sun, 31 Jan 2010 16:50:50 -0800, > Chris Rebert wrote: > > >>>> How do you call a function of no > arguments? > >>> > >>> It's not really a function in that case, it's > just a named constant. > >>> (Recall that functions don't/can't have > side-effects.) > >> > >> > >>>>> time.time(), random.random() > >> (1264983502.7505889, 0.29974255140479633) > >>>>> time.time(), random.random() > >> (1264983505.9283719, 0.74207867411026329) > >> > >> > >> They don't look terribly constant to me. > > > > Those aren't functions in the pure functional > programming sense; which > > is unsurprising since Python isn't a [pure] functional > language. They > > both involve side-effects. time() does I/O to the > clock chip to see what > > time it is, and random() uses and changes a global > seed value variable > > (which, in a double-whammy, takes its initial value > from time()). > > Yes, but these tasks -- get the time, get a (pseudo) random > number -- are > not unique to Python. Surely even Lisp and Haskell code > will sometimes > need to know the time. Whether they are "pure functions" > (functions in > the mathematical sense) or impure, they're still functions > in some sense. > How do you deal with such impure functions? > > > You pass it a monad (http://en.wikipedia.org/wiki/Monad_(functional_programming)). -EdK Ed Keith e_d_k at yahoo.com Blog: edkeith.blogspot.com From steven at REMOVE.THIS.cybersource.com.au Sun Jan 31 20:33:26 2010 From: steven at REMOVE.THIS.cybersource.com.au (Steven D'Aprano) Date: 01 Feb 2010 01:33:26 GMT Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> <87y6jef1df.fsf@castleamber.com> <0375f27a$0$1309$c3e8da3@news.astraweb.com> <878wbdye6p.fsf@castleamber.com> Message-ID: On Sun, 31 Jan 2010 18:47:42 -0600, John Bokma wrote: > Steven D'Aprano writes: > >> On Sun, 31 Jan 2010 14:47:08 -0600, John Bokma wrote: >> >>> An editor can correct the indenting of the braces example but can't >>> with this one. >>> >>> if x: >>> if y: >>> foo() >>> else: >>> bar() >>> >>> While braces might be considered redundant they are not when for one >>> reason or another formatting is lost or done incorrectly. >> >> I've heard this argument before, and I don't buy it. Why should we >> expect the editor to correct malformed code? > > Or a prettyfier. It doesn't matter. The point is that with braces there > *is* redundancy that be used to fix the code. Prettyfiers are significant in languages that allow braces (or begin/end tokens) and indentation to go out of sync. Since that can't happen with Python, it's not a problem that needs solving. Prettyfiers exist to work around a limitation of languages where indentation is not significant. Arguing that an advantage of braces is that prettyfiers can work with them easily is a silly argument. That's like saying an advantage of horse- drawn buggies over cars is that it will go faster when you hit the horse with a whip. That's true, but you only need the whip because of the lack of accelerator pedal. A much better argument would be that an advantage of significant indentation is that you no longer need a prettyfier, and a second advantage is a major reduction in flame wars over coding styles. >> Would you expect your editor to correct this malformed code? >> >> result = sin(x+)y > > Nice straw man. It's not a straw man. It's a serious argument. There is an infinite number of problems with malformed code that your editor can't fix, and your prettifier can't deal with. Why should we care if indentation is one more? There are tools out there (such as some web forum software) that corrupt whitespace. Those tools are broken, and if you (generic you), as a developer, are relying on those tools to code with, then shame on you. And if you're the creator of such broken tools, then shame on you more. You wouldn't arbitrarily decide to remove leading "E"s from the user's text, or trailing full stops, so why do you arbitrarily remove whitespace? > Let me repeat again: I am ok with how Python works. To be honest I think > it's cleaner compared to using {}. But in there are real life examples > in which Python code will break where code with braces will survive. Yes. So what? They are rare and insignificant in practice. For every line of code that you get via a webforum that mangles indentation, you get ten thousand lines of code from some place that doesn't. So long as code is written by and for human beings, the benefit of significant indentation is 100% practical, and the practical benefit of braces will remain insignificant. -- Steven From clp2 at rebertia.com Sun Jan 31 20:33:51 2010 From: clp2 at rebertia.com (Chris Rebert) Date: Sun, 31 Jan 2010 17:33:51 -0800 Subject: Python and Ruby In-Reply-To: References: <0375f245$0$1309$c3e8da3@news.astraweb.com> Message-ID: <50697b2c1001311733r5cc6181fy1c71c88e035a5efd@mail.gmail.com> On Sun, Jan 31, 2010 at 5:22 PM, Steven D'Aprano wrote: > On Sun, 31 Jan 2010 16:50:50 -0800, Chris Rebert wrote: >>>>> How do you call a function of no arguments? >>>> >>>> It's not really a function in that case, it's just a named constant. >>>> (Recall that functions don't/can't have side-effects.) >>> >>> >>>>>> time.time(), random.random() >>> (1264983502.7505889, 0.29974255140479633) >>>>>> time.time(), random.random() >>> (1264983505.9283719, 0.74207867411026329) >>> >>> >>> They don't look terribly constant to me. >> >> Those aren't functions in the pure functional programming sense; which >> is unsurprising since Python isn't a [pure] functional language. They >> both involve side-effects. time() does I/O to the clock chip to see what >> time it is, and random() uses and changes a global seed value variable >> (which, in a double-whammy, takes its initial value from time()). > > Yes, but these tasks -- get the time, get a (pseudo) random number -- are > not unique to Python. Surely even Lisp and Haskell code will sometimes > need to know the time. Whether they are "pure functions" (functions in > the mathematical sense) or impure, they're still functions in some sense. > How do you deal with such impure functions? Make the state explicit, i.e. monads or uniqueness typing. In the case of random(), you have it take the seed explicitly as an argument and have it return a new seed along with the "randomly" generated number. In the case of time(), you pass around an argument to represent the state of the outside world. I don't quite grok monads, but http://en.wikipedia.org/wiki/Uniqueness_type has a decent example of the general uniqueness type approach. Cheers, Chris -- http://blog.rebertia.com From andrewdalke at gmail.com Sun Jan 31 20:56:32 2010 From: andrewdalke at gmail.com (Andrew Dalke) Date: Sun, 31 Jan 2010 17:56:32 -0800 (PST) Subject: recv_into(bytearray) complains about a "pinned buffer" References: <8dff1c70-c141-4f0e-aebb-3a92733b272b@o28g2000yqh.googlegroups.com> Message-ID: <58ef4ced-bd16-44c3-8c44-678b671c4f9a@z26g2000yqm.googlegroups.com> On Feb 1, 1:04?am, Antoine Pitrou wrote: > The problem is that socket.recv_into() in 2.6 doesn't recognize the new > buffer API which is needed to accept bytearray objects. > (it does in 3.1, because the old buffer API doesn't exist anymore there) That's about what I thought it was, but I don't know if this was a deliberate choice or accidental. BTW, 2.7 (freshly built from version control) also has the same exception. > You could open an issue on the bug tracker for this. I've done that. It's http://bugs.python.org/issue7827 . Cheers! Andrew dalke at dalkescientific.com From andrej.mitrovich at gmail.com Sun Jan 31 20:59:41 2010 From: andrej.mitrovich at gmail.com (Andrej Mitrovic) Date: Sun, 31 Jan 2010 17:59:41 -0800 (PST) Subject: Python distutils build problems with MinGW Message-ID: <67168350-0f64-454c-a89b-767cba82e439@f12g2000yqn.googlegroups.com> Hi, I've made a similar post on the Cython mailing list, however I think this is more python-specific. I'm having trouble setting up distutils to use MinGW instead of Visual Studio when building a module. Even tho I've just uninstalled VS, and cleared out any leftover VS environment variables, distutils keeps wanting to use it. The steps I took: Fresh installation of Python 3.1.1 Successfully installed MinGW, added to the path variable (gcc in command prompt works) Successfully installed Cython, imports from Cython in Python work. Added a distutils.cfg file in \Python31\Lib\distutils\ directory with: [build] compiler=mingw32 (also tried adding [build_ext] compiler=mingw32) There's a demo setup.py module that came with Cython, I tried the following commands: ---------------------------------------------------- > python setup.py build_ext --inplace error: Unable to find vcvarsall.bat > python setup.py build error: Unable to find vcvarsall.bat ---------------------------------------------------- I'm having the exact same issue with trying to build the Polygon library via MinGW. In fact, the reason I had installed Visual Studio in the first place was to be able to build the Polygon library, since I was having these errors. What do I need to do to make distutils/python use MinGW? From kirby.urner at gmail.com Sun Jan 31 21:11:34 2010 From: kirby.urner at gmail.com (kirby urner) Date: Sun, 31 Jan 2010 18:11:34 -0800 Subject: [Edu-sig] odd drawing problem with turtle.py In-Reply-To: References: Message-ID: I don't see where you've defined a Turtle class to instantiate sir. Perhaps rename Circle to Turtle and rewrite the circle-drawing expression as: > c=Turtle(randint(-350,350),randint(-250,250),10,"red") You are making progress with a wrapper class for the Standard Library turtle. That's a well-known design pattern and a good way to get extra features sometimes. Kirby On Sun, Jan 31, 2010 at 4:27 PM, Brian Blais wrote: > I'm on Python 2.5, but using the updated turtle.py?Version 1.0.1 - 24. 9. > 2009. ?The following script draws 5 circles, which it is supposed to, but > then doesn't draw the second turtle which is supposed to simply move > forward. ?Any ideas? > from turtle import * > from numpy.random import randint > resetscreen() > class Circle(object): > ?? ?def __init__(self,x,y,r,color): > ?? ? ? ?self.x=x > ?? ? ? ?self.y=y > ?? ? ? ?self.r=r > ?? ? ? ?self.color=color > > ?? ? ? ?self.turtle=Turtle(visible=False) > ?? ? ? ?self.turtle.tracer(False) > ?? ? ? ?self.draw() > > ?? ?def draw(self): > ?? ? ? ?self.turtle.penup() > ?? ? ? ?self.turtle.setposition(self.x,self.y) > ?? ? ? ?self.turtle.setheading(0) > ?? ? ? ?self.turtle.backward(self.r) > ?? ? ? ?self.turtle.pendown() > ?? ? ? ?self.turtle.fill(True) > ?? ? ? ?self.turtle.pencolor("black") > ?? ? ? ?self.turtle.fillcolor(self.color) > ?? ? ? ?self.turtle.circle(self.r) > ?? ? ? ?self.turtle.fill(False) > ?? ? ? ?self.turtle.penup() > > for i in range(5): > ?? ?c=Circle(randint(-350,350),randint(-250,250),10,"red") > > > T=Turtle() > T.forward(100) > T.forward(100) > > > > > > > > thanks, > > bb > -- > Brian Blais > bblais at bryant.edu > http://web.bryant.edu/~bblais > http://bblais.blogspot.com/ > > > > _______________________________________________ > Edu-sig mailing list > Edu-sig at python.org > http://mail.python.org/mailman/listinfo/edu-sig > > From martin at v.loewis.de Sun Jan 31 21:30:56 2010 From: martin at v.loewis.de (Martin v. Loewis) Date: Mon, 01 Feb 2010 03:30:56 +0100 Subject: recv_into(bytearray) complains about a "pinned buffer" In-Reply-To: <8dff1c70-c141-4f0e-aebb-3a92733b272b@o28g2000yqh.googlegroups.com> References: <8dff1c70-c141-4f0e-aebb-3a92733b272b@o28g2000yqh.googlegroups.com> Message-ID: <4B663CE0.4030302@v.loewis.de> > In Python 2.6 I can't socket.recv_into(a byte array instance). I get a > TypeError which complains about a "pinned buffer". I have only an > inkling of what that means. A pinned buffer is one that cannot move in memory, even if another thread tries to behind your back. Typically, resizable containers are not inherently pinned, and "a user" (i.e. the API function) must explicitly pin it, which recv_into fails to do. > Is this a bug in Python 2.6 or a deliberate choice regarding > implementation concerns I don't know about? It's actually a bug also that you pass an array; doing so *should* give the very same error. It may be that the bugs surrounding buffers will never get fully resolved in the lifetime of Python 2.x, so I would probably just ignore recv_into. Regards, Martin From andrej.mitrovich at gmail.com Sun Jan 31 22:03:27 2010 From: andrej.mitrovich at gmail.com (Andrej Mitrovic) Date: Sun, 31 Jan 2010 19:03:27 -0800 (PST) Subject: Python distutils build problems with MinGW References: <67168350-0f64-454c-a89b-767cba82e439@f12g2000yqn.googlegroups.com> Message-ID: On Feb 1, 2:59?am, Andrej Mitrovic wrote: > Hi, > > I've made a similar post on the Cython mailing list, however I think > this is more python-specific. I'm having trouble setting up distutils > to use MinGW instead of Visual Studio when building a module. Even tho > I've just uninstalled VS, and cleared out any leftover VS environment > variables, distutils keeps wanting to use it. > > The steps I took: > > Fresh installation of Python 3.1.1 > Successfully installed MinGW, added to the path variable (gcc in > command prompt works) > Successfully installed Cython, imports from Cython in Python work. > Added a distutils.cfg file in \Python31\Lib\distutils\ directory with: > > [build] > compiler=mingw32 > > (also tried adding [build_ext] compiler=mingw32) > > There's a demo setup.py module that came with Cython, I tried the > following commands: > > ---------------------------------------------------- > > > python setup.py build_ext --inplace > > error: Unable to find vcvarsall.bat > > > python setup.py build > > error: Unable to find vcvarsall.bat > ---------------------------------------------------- > > I'm having the exact same issue with trying to build the Polygon > library via MinGW. In fact, the reason I had installed Visual Studio > in the first place was to be able to build the Polygon library, since > I was having these errors. > > What do I need to do to make distutils/python use MinGW? Update: I installed and tried building with Python 2.6, it calls MinGW when I have the distutils.cfg file configured properly (same configuration as the Python 3.1.1 one) But why doesn't it work on a fresh Python 3.1.1 installation as well? Is this a bug? From tjreedy at udel.edu Sun Jan 31 22:20:52 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 31 Jan 2010 22:20:52 -0500 Subject: whassup? builtins? python3000? Naah can't be right? In-Reply-To: <799ea994-d00f-485c-9164-ed4b66e750a5@k41g2000yqm.googlegroups.com> References: <799ea994-d00f-485c-9164-ed4b66e750a5@k41g2000yqm.googlegroups.com> Message-ID: On 1/31/2010 4:17 PM, _wolf wrote: > but why does ``__builtins__`` change its meaning depending on whether > this is the scope of the ?script? (i.e. the module whose name was > present, when calling ``python foobar.py``) or whether this is the > scope of a secondary module (imported or executed, directly or > indirectly, by ``foobar.py``)? There is a technical reason, which I believe Guido once gave in a post to the now-closed py3k list as part of the thread on the change to a single name. I do not remember, but I presume it has something to do with a difference between executing the main module and imported modules. Rest assured that it is intentional and has a reason. You can, of course, check if __name__ == '__main__': to know how to access. Terry Jan Reedy From john at castleamber.com Sun Jan 31 22:30:15 2010 From: john at castleamber.com (John Bokma) Date: Sun, 31 Jan 2010 21:30:15 -0600 Subject: Python and Ruby References: <1519c5ed-e883-470b-bb82-06078215fd5b@r19g2000yqb.googlegroups.com> <87y6jef1df.fsf@castleamber.com> <0375f27a$0$1309$c3e8da3@news.astraweb.com> <878wbdye6p.fsf@castleamber.com> Message-ID: <87tyu1ws3c.fsf@castleamber.com> Steven D'Aprano writes: > On Sun, 31 Jan 2010 18:47:42 -0600, John Bokma wrote: > >> Steven D'Aprano writes: >> >>> On Sun, 31 Jan 2010 14:47:08 -0600, John Bokma wrote: >>> >>>> An editor can correct the indenting of the braces example but can't >>>> with this one. >>>> >>>> if x: >>>> if y: >>>> foo() >>>> else: >>>> bar() >>>> >>>> While braces might be considered redundant they are not when for one >>>> reason or another formatting is lost or done incorrectly. >>> >>> I've heard this argument before, and I don't buy it. Why should we >>> expect the editor to correct malformed code? >> >> Or a prettyfier. It doesn't matter. The point is that with braces there >> *is* redundancy that be used to fix the code. > > Prettyfiers are significant in languages that allow braces (or begin/end > tokens) and indentation to go out of sync. Since that can't happen with > Python, Yes it can. I *have* seen Python with broken indentation on web pages, and good luck sorting it out. Blaming it on "broken tools" is just another straw man. It happens, and if you're learning Python and interested in that code you have a problem. Snipped the rest, because you start to sound like a zealot. I should've know better. -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development From tjreedy at udel.edu Sun Jan 31 22:48:33 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 31 Jan 2010 22:48:33 -0500 Subject: Python and Ruby In-Reply-To: References: <0375f245$0$1309$c3e8da3@news.astraweb.com> Message-ID: On 1/31/2010 7:25 PM, Steven D'Aprano wrote: > On Sun, 31 Jan 2010 15:40:36 -0800, Chris Rebert wrote: > >> On Sun, Jan 31, 2010 at 2:36 PM, Steven D'Aprano >> wrote: >>> On Sun, 31 Jan 2010 04:28:41 -0800, Ed Keith wrote: >>>> In most functional languages you just name a function to access it and >>>> you do it ALL the time. >>>> >>>> for example, in if you have a function 'f' which takes two parameters >>>> to call the function and get the result you use: >>>> >>>> f 2 3 >>>> >>>> If you want the function itself you use: >>>> >>>> f >>> >>> How do you call a function of no arguments? >> >> It's not really a function in that case, it's just a named constant. >> (Recall that functions don't/can't have side-effects.) Three of you gave essentially identical answers, but I still do not see how given something like def f(): return 1 I differentiate between 'function object at address xxx' and 'int 1' objects. >>>> time.time(), random.random() > (1264983502.7505889, 0.29974255140479633) >>>> time.time(), random.random() > (1264983505.9283719, 0.74207867411026329) > > They don't look terribly constant to me. I believe these are not functions in a functional language sense. Neither are input and output 'functions'. So all of these are either missing or some sort of special function-like something. I personally take a broader view of functions and include the relevant environment in their input and output, so that there are no 'side effects'. The concept of 'side-effect' is somewhat arbitrary. In fields other than computing, like pharmacology and politics, its meaning is somewhat different and a bit corrupt. > There is a difference between a function that does "give me whatever > value is specified by a fixed description" and a function that does "give > me a fixed value". Terry Jan Reedy From andrej.mitrovich at gmail.com Sun Jan 31 23:01:57 2010 From: andrej.mitrovich at gmail.com (Andrej Mitrovic) Date: Sun, 31 Jan 2010 20:01:57 -0800 (PST) Subject: whassup? builtins? python3000? Naah can't be right? References: <799ea994-d00f-485c-9164-ed4b66e750a5@k41g2000yqm.googlegroups.com> Message-ID: Hey, it's really simple! Just like the excerpt from the Learning Python book says: "Really, the built-in scope is just a built-in module called builtins, but you have to import builtins to query built-ins because the name builtins is not itself built-in...." :) From jjposner at optimum.net Sun Jan 31 23:05:24 2010 From: jjposner at optimum.net (John Posner) Date: Sun, 31 Jan 2010 23:05:24 -0500 Subject: odd drawing problem with turtle.py In-Reply-To: References: Message-ID: <4B665304.8040703@optimum.net> > I'm on Python 2.5, but using the updated turtle.py Version 1.0.1 - 24. 9. 2009. > The following script draws 5 circles, which it is supposed to, but then > doesn't draw the second turtle which is supposed to simply move forward. > Any ideas? Try commenting out this statement: self.turtle.tracer(False) That helps on Python 2.6.4. -John From no.email at nospam.invalid Sun Jan 31 23:22:36 2010 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 31 Jan 2010 20:22:36 -0800 Subject: Python and Ruby References: <0375f245$0$1309$c3e8da3@news.astraweb.com> Message-ID: <7x1vh5shyr.fsf@ruckus.brouhaha.com> Terry Reedy writes: > Three of you gave essentially identical answers, but I still do not > see how given something like > > def f(): return 1 > > I differentiate between 'function object at address xxx' and 'int 1' > objects. In the languages they are talking about, there is no such thing as a function with no args. A function is closer to a mathematical function, i.e. a mapping from one type to another, so every function has an arg. Haskell and ML do have a "Unit" type, written "()", which is something like Python's "None". So in Haskell you could write f () = 1 which means the function f, applied to the Unit value (), results in the value 1. The parens are not function-calling syntax. () is simply a value. For example, you could say y = () z = f y which would mean that z = 1. From alfps at start.no Sun Jan 31 23:33:36 2010 From: alfps at start.no (Alf P. Steinbach) Date: Mon, 01 Feb 2010 05:33:36 +0100 Subject: odd drawing problem with turtle.py In-Reply-To: References: Message-ID: * John Posner: > > I'm on Python 2.5, but using the updated turtle.py Version 1.0.1 - > 24. 9. 2009. > > The following script draws 5 circles, which it is supposed to, but then > > doesn't draw the second turtle which is supposed to simply move forward. > > Any ideas? > > Try commenting out this statement: > > self.turtle.tracer(False) > > That helps on Python 2.6.4. I can't see the original message, but when turning off animation via .tracer there should be a final .update call to flush things, so to speak. Cheers & hth., - Alf From kirby.urner at gmail.com Sun Jan 31 23:44:35 2010 From: kirby.urner at gmail.com (kirby urner) Date: Sun, 31 Jan 2010 20:44:35 -0800 Subject: [Edu-sig] odd drawing problem with turtle.py In-Reply-To: <4B665390.4080602@canterburyschool.org> References: <4B665390.4080602@canterburyschool.org> Message-ID: On Sun, Jan 31, 2010 at 8:07 PM, Vern Ceder wrote: > kirby urner wrote: >> >> I don't see where you've defined a Turtle class to instantiate sir. > > The Turtle class is part of the turtle library, so that's not an issue. > Hey, good point Vern, not firing on all cylinders over here. So I just commented out the Circle loop, went straight to the Turtle part at the end. It works OK for me, even without the visible=True. I'm on Python 2.6 on Win7 and I took out the numpy randint in favor of random randint. turtle.py in lib_tk: # Version 1.0.1 - 24. 9. 2009 I'm maybe not the right beta tester though, as after I uncomment the circle loop, I just get an hour-glass on my Tk canvas and no drawing, no error message either, even with visible=True in the init. Not sure what that's about yet... Hey, I just found out about piratepad.net, another shared white board solution. Was helping a dude in Indonesia debug a VPython script just a moment ago, with variable names in Indonesian (Latin-1 spellings at least). I've appended what we're working on. [ on second thought, I'll spare ya ] Kirby >>> ? ? ? ?self.turtle=Turtle(visible=False) Does visibility *and* pen need to be controlled? Sorry for the basic question, but seems if pen is up... From keakon at gmail.com Sun Jan 31 23:58:50 2010 From: keakon at gmail.com (keakon) Date: Sun, 31 Jan 2010 20:58:50 -0800 (PST) Subject: A performance issue when using default value Message-ID: <4754e6b7-2d79-4995-8f3f-193b646b524e@z10g2000prh.googlegroups.com> I've found strange performance issue when using default value, the test code is list below: from timeit import Timer def f(x): y = x y.append(1) return y def g(x=[]): y = [] y.append(1) return y def h(x=[]): y = x y.append(1) return y def f2(x): y = x y.append(1) return y + [] def g2(x=[]): y = [] y.append(1) return y + [] def h2(x=[]): y = x y.append(1) return y + [] TIMES = 10000 print Timer('f([])','from __main__ import f, g, h').timeit(TIMES) print Timer('g()','from __main__ import f, g, h').timeit(TIMES) print Timer('h([])','from __main__ import f, g, h').timeit(TIMES) print Timer('h()','from __main__ import f, g, h').timeit(TIMES) print Timer('f2([])','from __main__ import f2, g2, h2').timeit(TIMES) print Timer('g2()','from __main__ import f2, g2, h2').timeit(TIMES) print Timer('h2([])','from __main__ import f2, g2, h2').timeit(TIMES) print Timer('h2()','from __main__ import f2, g2, h2').timeit(TIMES) I tested it with Python 2.5.4, 2.6.4 and 3.1.1 on Windows XP, and get almost the same result: 0.00449247041174 0.00439608944712 0.00455867994396 0.00327471787615 0.00791581052899 0.00684919452053 0.00734311204357 0.30974942346 h2() is about 42 times slower than h2([]), but h() is a litter faster than h([]). If change TIMES to 20000, other results are 2 times than before, but h2 () is 4 times(about 1.2 sec) than before. Is there any tricks in it?